* WIP * WIP - make test_model_definition tests pass * WIP - make test_model_methods pass * WIP - make whole test suit at least run - failing 49/443 tests * WIP fix part of the getting pydantic tests as types of fields are now kept in core schema and not on fieldsinfo * WIP fix validation in update by creating individual fields validators, failing 36/443 * WIP fix __pydantic_extra__ in intializing model, fix test related to pydantic config checks, failing 32/442 * WIP - fix enum schema in model_json_schema, failing 31/442 * WIP - fix copying through model, fix setting pydantic fields on through, fix default config and inheriting from it, failing 26/442 * WIP fix tests checking pydantic schema, fix excluding parent fields, failing 21/442 * WIP some missed files * WIP - fix validators inheritance and fix validators in generated pydantic, failing 17/442 * WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442 * WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442 * WIP - working on proper populating __dict__ for relations for new schema dumping, some work on openapi docs, failing 13/442 * WIP - remove property fields as pydantic has now computed_field on its own, failing 9/442 * WIP - fixes in docs, failing 8/442 * WIP - fix tests for largebinary schema, wrapped bytes fields fail in pydantic, will be fixed in pydantic-core, remaining is circural schema for related models, failing 6/442 * WIP - fix to pk only models in schemas * Getting test suites to pass (#1249) * wip, fixing tests * iteration, fixing some more tests * iteration, fixing some more tests * adhere to comments * adhere to comments * remove unnecessary dict call, re-add getattribute for testing * todo for reverse relationship * adhere to comments, remove prints * solve circular refs * all tests pass 🎉 * remove 3.7 from tests * add lint and type check jobs * reforat with ruff, fix jobs * rename jobs * fix imports * fix evaluate in py3.8 * partially fix coverage * fix coverage, add more tests * fix test ids * fix test ids * fix lint, fix docs, make docs fully working scripts, add test docs job * fix pyproject * pin py ver in test docs * change dir in test docs * fix pydantic warning hack * rm poetry call in test_docs * switch to pathlib in test docs * remove coverage req test docs * fix type check tests, fix part of types * fix/skip next part of types * fix next part of types * fix next part of types * fix coverage * fix coverage * fix type (bit dirty 🤷) * fix some code smells * change pre-commit * tweak workflows * remove no root from tests * switch to full python path by passing sys.executable * some small refactor in new base model, one sample test, change makefile * small refactors to reduce complexity of methods * temp add tests for prs against pydantic_v2 * remove all references to __fields__ * remove all references to construct, deprecate the method and update model_construct to be in line with pydantic * deprecate dict and add model_dump, todo switch to model_dict in calls * fix tests * change to union * change to union * change to model_dump and model_dump_json from dict and json deprecated methods, deprecate them in ormar too * finish switching dict() -> model_dump() * finish switching json() -> model_dump_json() * remove fully pydantic_only * switch to extra for payment card, change missed json calls * fix coverage - no more warnings internal * fix coverage - no more warnings internal - part 2 * split model_construct into own and pydantic parts * split determine pydantic field type * change to new field validators * fix benchmarks, add codspeed instead of pytest-benchmark, add action and gh workflow * restore pytest-benchmark * remove codspeed * pin pydantic version, restore codspeed * change on push to pydantic_v2 to trigger first one * Use lifespan function instead of event (#1259) * check return types * fix imports order, set warnings=False on json that passes the dict, fix unnecessary loop in one of the test * remove references to model's meta as it's now ormar config, rename related methods too * filter out pydantic serializer warnings * remove choices leftovers * remove leftovers after property_fields, keep only enough to exclude them in initialization * add migration guide * fix meta references * downgrade databases for now * Change line numbers in documentation (#1265) * proofread and fix the docs, part 1 * proofread and fix the docs for models * proofread and fix the docs for fields * proofread and fix the docs for relations * proofread and fix rest of the docs, add release notes for 0.20 * create tables in new docs src * cleanup old deps, uncomment docs publish on tag * fix import reorder --------- Co-authored-by: TouwaStar <30479449+TouwaStar@users.noreply.github.com> Co-authored-by: Goran Mekić <meka@tilda.center>
166 lines
5.6 KiB
Markdown
166 lines
5.6 KiB
Markdown
# Encryption
|
|
|
|
`ormar` provides you with a way to encrypt a field in the database only.
|
|
Provided encryption backends allow for both one-way encryption (`HASH` backend) as
|
|
well as both-way encryption/decryption (`FERNET` backend).
|
|
|
|
!!!warning
|
|
Note that in order for encryption to work you need to install optional `cryptography` package.
|
|
|
|
You can do it manually `pip install cryptography` or with ormar by `pip install ormar[crypto]`
|
|
|
|
!!!warning
|
|
Note that adding `encrypt_backend` changes the database column type to `TEXT`,
|
|
which needs to be reflected in db either by migration (`alembic`) or manual change
|
|
|
|
## Defining a field encryption
|
|
|
|
To encrypt a field you need to pass at minimum `encrypt_secret` and `encrypt_backend` parameters.
|
|
|
|
```python hl_lines="10-12"
|
|
base_ormar_config = ormar.OrmarConfig(
|
|
metadata=metadata
|
|
database=database
|
|
)
|
|
|
|
class Filter(ormar.Model):
|
|
ormar_config = base_ormar_config.copy()
|
|
|
|
id: int = ormar.Integer(primary_key=True)
|
|
name: str = ormar.String(max_length=100,
|
|
encrypt_secret="secret123",
|
|
encrypt_backend=ormar.EncryptBackends.FERNET)
|
|
```
|
|
|
|
!!!warning
|
|
You can encrypt all `Field` types apart from `primary_key` column and relation
|
|
columns (`ForeignKey` and `ManyToMany`). Check backends details for more information.
|
|
|
|
## Available backends
|
|
|
|
### HASH
|
|
|
|
HASH is a one-way hash (like for password), never decrypted on retrieval
|
|
|
|
To set it up pass appropriate backend value.
|
|
|
|
```python
|
|
... # rest of model definition
|
|
password: str = ormar.String(max_length=128,
|
|
encrypt_secret="secret123",
|
|
encrypt_backend=ormar.EncryptBackends.HASH)
|
|
```
|
|
|
|
Note that since this backend never decrypt the stored value it's only applicable for
|
|
`String` fields. Used hash is a `sha512` hash, so the field length has to be >=128.
|
|
|
|
!!!warning
|
|
Note that in `HASH` backend you can filter by full value but filters like `contain` will not work as comparison is make on encrypted values
|
|
|
|
!!!note
|
|
Note that provided `encrypt_secret` is first hashed itself and used as salt, so in order to
|
|
compare to stored string you need to recreate this steps. The `order_by` will not work as encrypted strings are compared so you cannot reliably order by.
|
|
|
|
```python
|
|
class Hash(ormar.Model):
|
|
ormar_config = base_ormar_config.copy(tablename="hashes")
|
|
|
|
id: int = ormar.Integer(primary_key=True)
|
|
name: str = ormar.String(max_length=128,
|
|
encrypt_secret="udxc32",
|
|
encrypt_backend=ormar.EncryptBackends.HASH)
|
|
|
|
|
|
await Hash(name='test1').save()
|
|
|
|
# note the steps to recreate the stored value
|
|
# you can use also cryptography package instead of hashlib
|
|
secret = hashlib.sha256("udxc32".encode()).digest()
|
|
secret = base64.urlsafe_b64encode(secret)
|
|
hashed_test1 = hashlib.sha512(secret + 'test1'.encode()).hexdigest()
|
|
|
|
# full value comparison works
|
|
hash1 = await Hash.objects.get(name='test1')
|
|
assert hash1.name == hashed_test1
|
|
|
|
# but partial comparison does not (hashed strings are compared)
|
|
with pytest.raises(NoMatch):
|
|
await Filter.objects.get(name__icontains='test')
|
|
```
|
|
|
|
### FERNET
|
|
|
|
FERNET is a two-way encrypt/decrypt backend
|
|
|
|
To set it up pass appropriate backend value.
|
|
|
|
```python
|
|
... # rest of model definition
|
|
year: int = ormar.Integer(encrypt_secret="secret123",
|
|
encrypt_backend=ormar.EncryptBackends.FERNET)
|
|
```
|
|
|
|
Value is encrypted on way to database end decrypted on way out. Can be used on all types,
|
|
as the returned value is parsed to corresponding python type.
|
|
|
|
!!!warning
|
|
Note that in `FERNET` backend you loose `filter`ing possibility altogether as part of the encrypted value is a timestamp.
|
|
The same goes for `order_by` as encrypted strings are compared so you cannot reliably order by.
|
|
|
|
```python
|
|
class Filter(ormar.Model):
|
|
ormar_config = base_ormar_config.copy()
|
|
|
|
id: int = ormar.Integer(primary_key=True)
|
|
name: str = ormar.String(max_length=100,
|
|
encrypt_secret="asd123",
|
|
encrypt_backend=ormar.EncryptBackends.FERNET)
|
|
|
|
await Filter(name='test1').save()
|
|
await Filter(name='test1').save()
|
|
|
|
# values are properly encrypted and later decrypted
|
|
filters = await Filter.objects.all()
|
|
assert filters[0].name == filters[1].name == 'test1'
|
|
|
|
# but you cannot filter at all since part of the fernet hash is a timestamp
|
|
# which means that even if you encrypt the same string 2 times it will be different
|
|
with pytest.raises(NoMatch):
|
|
await Filter.objects.get(name='test1')
|
|
```
|
|
|
|
## Custom Backends
|
|
|
|
If you wish to support other type of encryption (i.e. AES) you can provide your own `EncryptionBackend`.
|
|
|
|
To setup a backend all you need to do is subclass `ormar.fields.EncryptBackend` class and provide required backend.
|
|
|
|
Sample dummy backend (that does nothing) can look like following:
|
|
|
|
```python
|
|
class DummyBackend(ormar.fields.EncryptBackend):
|
|
def _initialize_backend(self, secret_key: bytes) -> None:
|
|
pass
|
|
|
|
def encrypt(self, value: Any) -> str:
|
|
return value
|
|
|
|
def decrypt(self, value: Any) -> str:
|
|
return value
|
|
```
|
|
|
|
To use this backend set `encrypt_backend` to `CUSTOM` and provide your backend as
|
|
argument by `encrypt_custom_backend`.
|
|
|
|
```python
|
|
class Filter(ormar.Model):
|
|
ormar_config = base_ormar_config.copy()
|
|
|
|
id: int = ormar.Integer(primary_key=True)
|
|
name: str = ormar.String(max_length=100,
|
|
encrypt_secret="secret123",
|
|
encrypt_backend=ormar.EncryptBackends.CUSTOM,
|
|
encrypt_custom_backend=DummyBackend
|
|
)
|
|
```
|