Prodream/ormar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WIP - Pydantic v2 support (#1238)

Browse Source 
* WIP

* WIP - make test_model_definition tests pass

* WIP - make test_model_methods pass

* WIP - make whole test suit at least run - failing 49/443 tests

* WIP fix part of the getting pydantic tests as types of fields are now kept in core schema and not on fieldsinfo

* WIP fix validation in update by creating individual fields validators, failing 36/443

* WIP fix __pydantic_extra__ in intializing model, fix test related to pydantic config checks, failing 32/442

* WIP - fix enum schema in model_json_schema, failing 31/442

* WIP - fix copying through model, fix setting pydantic fields on through, fix default config and inheriting from it, failing 26/442

* WIP fix tests checking pydantic schema, fix excluding parent fields, failing 21/442

* WIP some missed files

* WIP - fix validators inheritance and fix validators in generated pydantic, failing 17/442

* WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442

* WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442

* WIP - working on proper populating __dict__ for relations for new schema dumping, some work on openapi docs, failing 13/442

* WIP - remove property fields as pydantic has now computed_field on its own, failing 9/442

* WIP - fixes in docs, failing 8/442

* WIP - fix tests for largebinary schema, wrapped bytes fields fail in pydantic, will be fixed in pydantic-core, remaining is circural schema for related models, failing 6/442

* WIP - fix to pk only models in schemas

* Getting test suites to pass (#1249)

* wip, fixing tests

* iteration, fixing some more tests

* iteration, fixing some more tests

* adhere to comments

* adhere to comments

* remove unnecessary dict call, re-add getattribute for testing

* todo for reverse relationship

* adhere to comments, remove prints

* solve circular refs

* all tests pass 🎉

* remove 3.7 from tests

* add lint and type check jobs

* reforat with ruff, fix jobs

* rename jobs

* fix imports

* fix evaluate in py3.8

* partially fix coverage

* fix coverage, add more tests

* fix test ids

* fix test ids

* fix lint, fix docs, make docs fully working scripts, add test docs job

* fix pyproject

* pin py ver in test docs

* change dir in test docs

* fix pydantic warning hack

* rm poetry call in test_docs

* switch to pathlib in test docs

* remove coverage req test docs

* fix type check tests, fix part of types

* fix/skip next part of types

* fix next part of types

* fix next part of types

* fix coverage

* fix coverage

* fix type (bit dirty 🤷)

* fix some code smells

* change pre-commit

* tweak workflows

* remove no root from tests

* switch to full python path by passing sys.executable

* some small refactor in new base model, one sample test, change makefile

* small refactors to reduce complexity of methods

* temp add tests for prs against pydantic_v2

* remove all references to __fields__

* remove all references to construct, deprecate the method and update model_construct to be in line with pydantic

* deprecate dict and add model_dump, todo switch to model_dict in calls

* fix tests

* change to union

* change to union

* change to model_dump and model_dump_json from dict and json deprecated methods, deprecate them in ormar too

* finish switching dict() -> model_dump()

* finish switching json() -> model_dump_json()

* remove fully pydantic_only

* switch to extra for payment card, change missed json calls

* fix coverage - no more warnings internal

* fix coverage - no more warnings internal - part 2

* split model_construct into own and pydantic parts

* split determine pydantic field type

* change to new field validators

* fix benchmarks, add codspeed instead of pytest-benchmark, add action and gh workflow

* restore pytest-benchmark

* remove codspeed

* pin pydantic version, restore codspeed

* change on push to pydantic_v2 to trigger first one

* Use lifespan function instead of event (#1259)

* check return types

* fix imports order, set warnings=False on json that passes the dict, fix unnecessary loop in one of the test

* remove references to model's meta as it's now ormar config, rename related methods too

* filter out pydantic serializer warnings

* remove choices leftovers

* remove leftovers after property_fields, keep only enough to exclude them in initialization

* add migration guide

* fix meta references

* downgrade databases for now

* Change line numbers in documentation (#1265)

* proofread and fix the docs, part 1

* proofread and fix the docs for models

* proofread and fix the docs for fields

* proofread and fix the docs for relations

* proofread and fix rest of the docs, add release notes for 0.20

* create tables in new docs src

* cleanup old deps, uncomment docs publish on tag

* fix import reorder

---------

Co-authored-by: TouwaStar <30479449+TouwaStar@users.noreply.github.com>
Co-authored-by: Goran Mekić <meka@tilda.center>
This commit is contained in:
collerek
2024-03-23 19:28:28 +01:00
committed by GitHub
parent 3a206dd8dc
commit 500625f0ec
294 changed files with 8132 additions and 9311 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
docs/fields/common-parameters.md
View File

@ -29,7 +29,6 @@ Automatically changed to True if user provide one of the following:
* `default` value or function is provided
* `server_default` value or function is provided
* `autoincrement` is set on `Integer` `primary_key` field
* **[DEPRECATED]**`pydantic_only=True` is set
Specifies if field is optional or required, used both with sql and pydantic.
@ -109,7 +108,7 @@ Used in sql only.
Sample usage:
```Python hl_lines="21-23"
```Python hl_lines="20-22"
--8<-- "../docs_src/fields/docs004.py"
```
@ -167,20 +166,6 @@ Sets the unique constraint on a table's column.
Used in sql only.
## pydantic_only (**DEPRECATED**)
**This parameter is deprecated and will be removed in one of next releases!**
**To check how to declare pydantic only fields that are not saved into database see [pydantic fields section](pydantic-fields.md)**
`pydantic_only`: `bool` = `False`
Prevents creation of a sql column for given field.
Used for data related to given model but not to be stored in the database.
Used in pydantic only.
## overwrite_pydantic_type
By default, ormar uses predefined pydantic field types that it applies on model creation (hence the type hints are optional).
@ -189,22 +174,25 @@ If you want to, you can apply your own type, that will be **completely** replaci
So it's on you as a user to provide a type that is valid in the context of given ormar field type.
!!!warning
Note that by default you should use build in arguments that are passed to underlying pydantic field.
You can check what arguments are supported in field types section or in [pydantic](https://pydantic-docs.helpmanual.io/usage/schema/#field-customisation) docs.
Note that by default you should use build in arguments that are passed to underlying pydantic field.
You can check what arguments are supported in field types section or in [pydantic](https://pydantic-docs.helpmanual.io/usage/schema/#field-customisation) docs.
!!!danger
Setting a wrong type of pydantic field can break your model, so overwrite it only when you know what you are doing.
As it's easy to break functionality of ormar the `overwrite_pydantic_type` argument is not available on relation fields!
Setting a wrong type of pydantic field can break your model, so overwrite it only when you know what you are doing.
As it's easy to break functionality of ormar the `overwrite_pydantic_type` argument is not available on relation fields!
```python
base_ormar_config = ormar.OrmarConfig(
metadata=metadata
database=database
)
# sample overwrites
class OverwriteTest(ormar.Model):
class Meta:
tablename = "overwrites"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="overwrites")
id: int = ormar.Integer(primary_key=True)
my_int: str = ormar.Integer(overwrite_pydantic_type=PositiveInt)
@ -212,18 +200,6 @@ class OverwriteTest(ormar.Model):
overwrite_pydantic_type=Optional[Json[Dict[str, int]]])
```
## choices
`choices`: `Sequence` = `[]`
A set of choices allowed to be used for given field.
Used for data validation on pydantic side.
Prevents insertion of value not present in the choices list.
Used in pydantic only.
[relations]: ../relations/index.md
[queries]: ../queries/index.md
[pydantic]: https://pydantic-docs.helpmanual.io/usage/types/#constrained-types

21
docs/fields/encryption.md
View File

@ -17,10 +17,14 @@ well as both-way encryption/decryption (`FERNET` backend).
To encrypt a field you need to pass at minimum `encrypt_secret` and `encrypt_backend` parameters.
```python hl_lines="7-8"
```python hl_lines="10-12"
base_ormar_config = ormar.OrmarConfig(
metadata=metadata
database=database
)
class Filter(ormar.Model):
class Meta(BaseMeta):
tablename = "filters"
ormar_config = base_ormar_config.copy()
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100,
@ -59,8 +63,7 @@ Note that since this backend never decrypt the stored value it's only applicable
```python
class Hash(ormar.Model):
class Meta(BaseMeta):
tablename = "hashes"
ormar_config = base_ormar_config.copy(tablename="hashes")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=128,
@ -106,8 +109,7 @@ as the returned value is parsed to corresponding python type.
```python
class Filter(ormar.Model):
class Meta(BaseMeta):
tablename = "filters"
ormar_config = base_ormar_config.copy()
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100,
@ -152,8 +154,7 @@ argument by `encrypt_custom_backend`.
```python
class Filter(ormar.Model):
class Meta(BaseMeta):
tablename = "filters"
ormar_config = base_ormar_config.copy()
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100,
@ -161,4 +162,4 @@ class Filter(ormar.Model):
encrypt_backend=ormar.EncryptBackends.CUSTOM,
encrypt_custom_backend=DummyBackend
)
```
```

57
docs/fields/field-types.md
View File

@ -1,10 +1,10 @@
# Fields
There are 12 basic model field types and a special `ForeignKey` and `Many2Many` fields to establish relationships between models.
There are 12 basic model field types and a special `ForeignKey` and `ManyToMany` fields to establish relationships between models.
!!!tip
For explanation of `ForeignKey` and `Many2Many` fields check [relations][relations].
For explanation of `ForeignKey` and `ManyToMany` fields check [relations][relations].
Each of the `Fields` has assigned both `sqlalchemy` column class and python type that is used to create `pydantic` model.
@ -160,11 +160,16 @@ That way you can i.e. set the value by API, even if value is not `utf-8` compati
```python
import base64
... # other imports skipped for brevity
base_ormar_config = ormar.OrmarConfig(
metadata=metadata
database=database
)
class LargeBinaryStr(ormar.Model):
class Meta:
tablename = "my_str_blobs"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="my_str_blobs")
id: int = ormar.Integer(primary_key=True)
test_binary: str = ormar.LargeBinary(
@ -215,46 +220,6 @@ So which one to use depends on the backend you use and on the column/ data type
* Sqlalchemy column: `sqlalchemy.Enum`
* Type (used for pydantic): `Type[Enum]`
#### Choices
You can change any field into `Enum` like field by passing a `choices` list that is accepted by all Field types.
It will add both: validation in `pydantic` model and will display available options in schema,
therefore it will be available in docs of `fastapi`.
If you still want to use `Enum` in your application you can do this by passing a `Enum` into choices
and later pass value of given option to a given field (note that Enum is not JsonSerializable).
```python
# note that imports and endpoints declaration
# is skipped here for brevity
from enum import Enum
class TestEnum(Enum):
val1 = 'Val1'
val2 = 'Val2'
class TestModel(ormar.Model):
class Meta:
tablename = "org"
metadata = metadata
database = database
id: int = ormar.Integer(primary_key=True)
# pass list(Enum) to choices
enum_string: str = ormar.String(max_length=100, choices=list(TestEnum))
# sample payload coming to fastapi
response = client.post(
"/test_models/",
json={
"id": 1,
# you need to refer to the value of the `Enum` option
# if called like this, alternatively just use value
# string "Val1" in this case
"enum_string": TestEnum.val1.value
},
)
```
[relations]: ../relations/index.md
[queries]: ../queries.md

52
docs/fields/pydantic-fields.md
View File

@ -22,17 +22,14 @@ If you set a field as `Optional`, it defaults to `None` if not provided and that
exactly what's going to happen during loading from database.
```python
database = databases.Database(DATABASE_URL)
metadata = sqlalchemy.MetaData()
base_ormar_config = ormar.OrmarConfig(
metadata=sqlalchemy.MetaData(),
database=databases.Database(DATABASE_URL),
)
class BaseMeta(ormar.ModelMeta):
metadata = metadata
database = database
class ModelTest(ormar.Model):
class Meta(BaseMeta):
pass
ormar_config = base_ormar_config.copy()
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=200)
@ -57,17 +54,14 @@ By setting a default value, this value will be set on initialization and databas
Note that setting a default to `None` is the same as setting the field to `Optional`.
```python
database = databases.Database(DATABASE_URL)
metadata = sqlalchemy.MetaData()
base_ormar_config = ormar.OrmarConfig(
metadata=sqlalchemy.MetaData(),
database=databases.Database(DATABASE_URL),
)
class BaseMeta(ormar.ModelMeta):
metadata = metadata
database = database
class ModelTest(ormar.Model):
class Meta(BaseMeta):
pass
ormar_config = base_ormar_config.copy()
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=200)
@ -97,13 +91,12 @@ on initialization and each database load.
from pydantic import Field, PaymentCardNumber
# ...
database = databases.Database(DATABASE_URL)
metadata = sqlalchemy.MetaData()
base_ormar_config = ormar.OrmarConfig(
metadata=sqlalchemy.MetaData(),
database=databases.Database(DATABASE_URL),
)
class BaseMeta(ormar.ModelMeta):
metadata = metadata
database = database
CARD_NUMBERS = [
"123456789007",
@ -119,8 +112,7 @@ def get_number():
class ModelTest2(ormar.Model):
class Meta(BaseMeta):
pass
ormar_config = base_ormar_config.copy()
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=200)
@ -149,13 +141,12 @@ You can provide a value for the field in your `__init__()` method before calling
from pydantic import BaseModel
# ...
database = databases.Database(DATABASE_URL)
metadata = sqlalchemy.MetaData()
base_ormar_config = ormar.OrmarConfig(
metadata=sqlalchemy.MetaData(),
database=databases.Database(DATABASE_URL),
)
class BaseMeta(ormar.ModelMeta):
metadata = metadata
database = database
class PydanticTest(BaseModel):
aa: str
@ -163,8 +154,7 @@ class PydanticTest(BaseModel):
class ModelTest3(ormar.Model):
class Meta(BaseMeta):
pass
ormar_config = base_ormar_config.copy()
# provide your custom init function
def __init__(self, **kwargs):
@ -192,4 +182,4 @@ assert test_check.pydantic_test.aa == "random"
```
!!!warning
If you do not provide a value in one of the above ways `ValidationError` will be raised on load from database.
If you do not provide a value in one of the above ways `ValidationError` will be raised on load from database.