Prodream/ormar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WIP - Pydantic v2 support (#1238)

Browse Source 
* WIP

* WIP - make test_model_definition tests pass

* WIP - make test_model_methods pass

* WIP - make whole test suit at least run - failing 49/443 tests

* WIP fix part of the getting pydantic tests as types of fields are now kept in core schema and not on fieldsinfo

* WIP fix validation in update by creating individual fields validators, failing 36/443

* WIP fix __pydantic_extra__ in intializing model, fix test related to pydantic config checks, failing 32/442

* WIP - fix enum schema in model_json_schema, failing 31/442

* WIP - fix copying through model, fix setting pydantic fields on through, fix default config and inheriting from it, failing 26/442

* WIP fix tests checking pydantic schema, fix excluding parent fields, failing 21/442

* WIP some missed files

* WIP - fix validators inheritance and fix validators in generated pydantic, failing 17/442

* WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442

* WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442

* WIP - working on proper populating __dict__ for relations for new schema dumping, some work on openapi docs, failing 13/442

* WIP - remove property fields as pydantic has now computed_field on its own, failing 9/442

* WIP - fixes in docs, failing 8/442

* WIP - fix tests for largebinary schema, wrapped bytes fields fail in pydantic, will be fixed in pydantic-core, remaining is circural schema for related models, failing 6/442

* WIP - fix to pk only models in schemas

* Getting test suites to pass (#1249)

* wip, fixing tests

* iteration, fixing some more tests

* iteration, fixing some more tests

* adhere to comments

* adhere to comments

* remove unnecessary dict call, re-add getattribute for testing

* todo for reverse relationship

* adhere to comments, remove prints

* solve circular refs

* all tests pass 🎉

* remove 3.7 from tests

* add lint and type check jobs

* reforat with ruff, fix jobs

* rename jobs

* fix imports

* fix evaluate in py3.8

* partially fix coverage

* fix coverage, add more tests

* fix test ids

* fix test ids

* fix lint, fix docs, make docs fully working scripts, add test docs job

* fix pyproject

* pin py ver in test docs

* change dir in test docs

* fix pydantic warning hack

* rm poetry call in test_docs

* switch to pathlib in test docs

* remove coverage req test docs

* fix type check tests, fix part of types

* fix/skip next part of types

* fix next part of types

* fix next part of types

* fix coverage

* fix coverage

* fix type (bit dirty 🤷)

* fix some code smells

* change pre-commit

* tweak workflows

* remove no root from tests

* switch to full python path by passing sys.executable

* some small refactor in new base model, one sample test, change makefile

* small refactors to reduce complexity of methods

* temp add tests for prs against pydantic_v2

* remove all references to __fields__

* remove all references to construct, deprecate the method and update model_construct to be in line with pydantic

* deprecate dict and add model_dump, todo switch to model_dict in calls

* fix tests

* change to union

* change to union

* change to model_dump and model_dump_json from dict and json deprecated methods, deprecate them in ormar too

* finish switching dict() -> model_dump()

* finish switching json() -> model_dump_json()

* remove fully pydantic_only

* switch to extra for payment card, change missed json calls

* fix coverage - no more warnings internal

* fix coverage - no more warnings internal - part 2

* split model_construct into own and pydantic parts

* split determine pydantic field type

* change to new field validators

* fix benchmarks, add codspeed instead of pytest-benchmark, add action and gh workflow

* restore pytest-benchmark

* remove codspeed

* pin pydantic version, restore codspeed

* change on push to pydantic_v2 to trigger first one

* Use lifespan function instead of event (#1259)

* check return types

* fix imports order, set warnings=False on json that passes the dict, fix unnecessary loop in one of the test

* remove references to model's meta as it's now ormar config, rename related methods too

* filter out pydantic serializer warnings

* remove choices leftovers

* remove leftovers after property_fields, keep only enough to exclude them in initialization

* add migration guide

* fix meta references

* downgrade databases for now

* Change line numbers in documentation (#1265)

* proofread and fix the docs, part 1

* proofread and fix the docs for models

* proofread and fix the docs for fields

* proofread and fix the docs for relations

* proofread and fix rest of the docs, add release notes for 0.20

* create tables in new docs src

* cleanup old deps, uncomment docs publish on tag

* fix import reorder

---------

Co-authored-by: TouwaStar <30479449+TouwaStar@users.noreply.github.com>
Co-authored-by: Goran Mekić <meka@tilda.center>
This commit is contained in:
collerek
2024-03-23 19:28:28 +01:00
committed by GitHub
parent 3a206dd8dc
commit 500625f0ec
294 changed files with 8132 additions and 9311 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs_src/models/docs001.py
View File

@ -1,16 +1,16 @@
import databases
import sqlalchemy
import ormar
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)

13
docs_src/models/docs002.py
View File

@ -1,19 +1,20 @@
import databases
import sqlalchemy
import ormar
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
# if you omit this parameter it will be created automatically
# as class.__name__.lower()+'s' -> "courses" in this example
tablename = "my_courses"
database = database
metadata = metadata
tablename="my_courses",
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)

18
docs_src/models/docs003.py
View File

@ -1,34 +1,34 @@
import databases
import sqlalchemy
import ormar
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
completed: bool = ormar.Boolean(default=False)
print(Course.__fields__)
print(Course.model_fields)
"""
Will produce:
{'id': ModelField(name='id',
{'id': Field(name='id',
type=Optional[int],
required=False,
default=None),
'name': ModelField(name='name',
'name': Field(name='name',
type=Optional[str],
required=False,
default=None),
'completed': ModelField(name='completed',
'completed': Field(name='completed',
type=bool,
required=False,
default=False)}

20
docs_src/models/docs004.py
View File

@ -1,24 +1,28 @@
import databases
import ormar
import sqlalchemy
import ormar
DATABASE_URL = "sqlite:///test.db"
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
ormar_base_config = ormar.OrmarConfig(
database=databases.Database(DATABASE_URL), metadata=sqlalchemy.MetaData()
)
class Course(ormar.Model):
class Meta(ormar.ModelMeta): # note you don't have to subclass - but it's recommended for ide completion and mypy
database = database
metadata = metadata
ormar_config = ormar.OrmarConfig(
tablename="courses",
database=databases.Database(DATABASE_URL),
metadata=sqlalchemy.MetaData(),
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
completed: bool = ormar.Boolean(default=False)
print(Course.Meta.table.columns)
print(Course.ormar_config.table.columns)
"""
Will produce:
['courses.id', 'courses.name', 'courses.completed']
ImmutableColumnCollection(courses.id, courses.name, courses.completed)
"""

177
docs_src/models/docs005.py
View File

@ -1,67 +1,144 @@
import databases
import sqlalchemy
import pprint
import databases
import ormar
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta(ormar.ModelMeta):
database = database
metadata = metadata
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
completed: bool = ormar.Boolean(default=False)
print({x: v.__dict__ for x, v in Course.Meta.model_fields.items()})
pprint.pp({x: v.__dict__ for x, v in Course.ormar_config.model_fields.items()})
"""
Will produce:
{'completed': mappingproxy({'autoincrement': False,
'choices': set(),
'column_type': Boolean(),
'default': False,
'index': False,
'name': 'completed',
'nullable': True,
'primary_key': False,
'pydantic_only': False,
'server_default': None,
'unique': False}),
'id': mappingproxy({'autoincrement': True,
'choices': set(),
'column_type': Integer(),
'default': None,
'ge': None,
'index': False,
'le': None,
'maximum': None,
'minimum': None,
'multiple_of': None,
'name': 'id',
'nullable': False,
'primary_key': True,
'pydantic_only': False,
'server_default': None,
'unique': False}),
'name': mappingproxy({'allow_blank': False,
'autoincrement': False,
'choices': set(),
'column_type': String(max_length=100),
'curtail_length': None,
'default': None,
'index': False,
'max_length': 100,
'min_length': None,
'name': 'name',
'nullable': False,
'primary_key': False,
'pydantic_only': False,
'regex': None,
'server_default': None,
'strip_whitespace': False,
'unique': False})}
{'id': {'__type__': <class 'int'>,
'__pydantic_type__': <class 'int'>,
'__sample__': 0,
'related_name': None,
'column_type': Integer(),
'constraints': [],
'name': 'id',
'db_alias': None,
'primary_key': True,
'autoincrement': True,
'nullable': True,
'sql_nullable': False,
'index': False,
'unique': False,
'virtual': None,
'is_multi': None,
'is_relation': None,
'is_through': False,
'through_relation_name': None,
'through_reverse_relation_name': None,
'skip_reverse': False,
'skip_field': False,
'owner': <class '__main__.Course'>,
'to': None,
'to_pk_only': None,
'through': None,
'self_reference': False,
'self_reference_primary': None,
'orders_by': None,
'related_orders_by': None,
'encrypt_secret': None,
'encrypt_backend': <EncryptBackends.NONE: 0>,
'encrypt_custom_backend': None,
'ormar_default': None,
'server_default': None,
'comment': None,
'represent_as_base64_str': False,
'minimum': None,
'maximum': None,
'multiple_of': None,
'ge': None,
'le': None},
'name': {'__type__': <class 'str'>,
'__pydantic_type__': <class 'str'>,
'__sample__': 'string',
'related_name': None,
'column_type': String(length=100),
'constraints': [],
'name': 'name',
'db_alias': None,
'primary_key': False,
'autoincrement': False,
'nullable': False,
'sql_nullable': False,
'index': False,
'unique': False,
'virtual': None,
'is_multi': None,
'is_relation': None,
'is_through': False,
'through_relation_name': None,
'through_reverse_relation_name': None,
'skip_reverse': False,
'skip_field': False,
'owner': <class '__main__.Course'>,
'to': None,
'to_pk_only': None,
'through': None,
'self_reference': False,
'self_reference_primary': None,
'orders_by': None,
'related_orders_by': None,
'encrypt_secret': None,
'encrypt_backend': <EncryptBackends.NONE: 0>,
'encrypt_custom_backend': None,
'ormar_default': None,
'server_default': None,
'comment': None,
'represent_as_base64_str': False,
'max_length': 100,
'min_length': None,
'regex': None},
'completed': {'__type__': <class 'bool'>,
'__pydantic_type__': <class 'bool'>,
'__sample__': True,
'related_name': None,
'column_type': Boolean(),
'constraints': [],
'name': 'completed',
'db_alias': None,
'primary_key': False,
'autoincrement': False,
'nullable': True,
'sql_nullable': True,
'index': False,
'unique': False,
'virtual': None,
'is_multi': None,
'is_relation': None,
'is_through': False,
'through_relation_name': None,
'through_reverse_relation_name': None,
'skip_reverse': False,
'skip_field': False,
'owner': <class '__main__.Course'>,
'to': None,
'to_pk_only': None,
'through': None,
'self_reference': False,
'self_reference_primary': None,
'orders_by': None,
'related_orders_by': None,
'encrypt_secret': None,
'encrypt_backend': <EncryptBackends.NONE: 0>,
'encrypt_custom_backend': None,
'ormar_default': False,
'server_default': None,
'comment': None,
'represent_as_base64_str': False}}
"""

14
docs_src/models/docs006.py
View File

@ -1,20 +1,20 @@
import databases
import sqlalchemy
import ormar
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
# define your constraints in Meta class of the model
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
# define your constraints in OrmarConfig of the model
# it's a list that can contain multiple constraints
# hera a combination of name and column will have to be unique in db
constraints = [ormar.UniqueColumns("name", "completed")]
constraints=[ormar.UniqueColumns("name", "completed")],
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)

28
docs_src/models/docs007.py
View File

@ -1,23 +1,31 @@
import asyncio
import databases
import sqlalchemy
import ormar
import sqlalchemy
from examples import create_drop_database
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
DATABASE_URL = "sqlite:///test.db"
ormar_base_config = ormar.OrmarConfig(
database=databases.Database(DATABASE_URL), metadata=sqlalchemy.MetaData()
)
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
ormar_config = ormar_base_config.copy(tablename="courses")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
completed: bool = ormar.Boolean(default=False)
course = Course(name="Painting for dummies", completed=False)
await course.save()
@create_drop_database(base_config=ormar_base_config)
async def run_query():
course = Course(name="Painting for dummies", completed=False)
await course.save()
await Course.objects.create(name="Painting for dummies", completed=False)
await Course.objects.create(name="Painting for dummies", completed=False)
asyncio.run(run_query())

14
docs_src/models/docs008.py
View File

@ -1,17 +1,19 @@
import databases
import ormar
import sqlalchemy
import ormar
DATABASE_URl = "sqlite:///test.db"
database = databases.Database("sqlite:///test.db", force_rollback=True)
database = databases.Database(DATABASE_URl, force_rollback=True)
metadata = sqlalchemy.MetaData()
class Child(ormar.Model):
class Meta:
tablename = "children"
metadata = metadata
database = database
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
tablename="children",
)
id: int = ormar.Integer(name="child_id", primary_key=True)
first_name: str = ormar.String(name="fname", max_length=100)

27
docs_src/models/docs009.py
View File

@ -1,20 +1,33 @@
from typing import Optional
import databases
import sqlalchemy
import ormar
from .docs010 import Artist # previous example
import sqlalchemy
database = databases.Database("sqlite:///test.db", force_rollback=True)
metadata = sqlalchemy.MetaData()
class Artist(ormar.Model):
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
tablename="artists",
)
id: int = ormar.Integer(name="artist_id", primary_key=True)
first_name: str = ormar.String(name="fname", max_length=100)
last_name: str = ormar.String(name="lname", max_length=100)
born_year: int = ormar.Integer(name="year")
class Album(ormar.Model):
class Meta:
tablename = "music_albums"
metadata = metadata
database = database
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
tablename="music_albums",
)
id: int = ormar.Integer(name="album_id", primary_key=True)
name: str = ormar.String(name="album_name", max_length=100)

37
docs_src/models/docs010.py
View File

@ -1,25 +1,40 @@
import databases
import ormar
import sqlalchemy
import ormar
from .docs008 import Child
DATABASE_URl = "sqlite:///test.db"
database = databases.Database("sqlite:///test.db", force_rollback=True)
database = databases.Database(DATABASE_URl, force_rollback=True)
metadata = sqlalchemy.MetaData()
class Child(ormar.Model):
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
tablename="children",
)
id: int = ormar.Integer(name="child_id", primary_key=True)
first_name: str = ormar.String(name="fname", max_length=100)
last_name: str = ormar.String(name="lname", max_length=100)
born_year: int = ormar.Integer(name="year_born", nullable=True)
class ArtistChildren(ormar.Model):
class Meta:
tablename = "children_x_artists"
metadata = metadata
database = database
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
tablename="children_x_artists",
)
class Artist(ormar.Model):
class Meta:
tablename = "artists"
metadata = metadata
database = database
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
tablename="artists",
)
id: int = ormar.Integer(name="artist_id", primary_key=True)
first_name: str = ormar.String(name="fname", max_length=100)

19
docs_src/models/docs011.py
View File

@ -1,19 +0,0 @@
import databases
import sqlalchemy
import ormar
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
id: ormar.Integer(primary_key=True)
name: ormar.String(max_length=100)
completed: ormar.Boolean(default=False)
c1 = Course()

10
docs_src/models/docs012.py
View File

@ -1,16 +1,16 @@
import databases
import sqlalchemy
import ormar
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
)
id = ormar.Integer(primary_key=True)
name = ormar.String(max_length=100)

28
docs_src/models/docs013.py
View File

@ -1,27 +1,22 @@
from typing import Optional
import databases
import ormar
import sqlalchemy
import ormar
DATABASE_URL = "sqlite:///test.db"
database = databases.Database("sqlite:///test.db", force_rollback=True)
metadata = sqlalchemy.MetaData()
# note that you do not have to subclass ModelMeta,
# it's useful for type hints and code completion
class MainMeta(ormar.ModelMeta):
metadata = metadata
database = database
ormar_base_config = ormar.OrmarConfig(
database=databases.Database(DATABASE_URL),
metadata=sqlalchemy.MetaData(),
)
class Artist(ormar.Model):
class Meta(MainMeta):
# note that tablename is optional
# if not provided ormar will user class.__name__.lower()+'s'
# -> artists in this example
pass
# note that tablename is optional
# if not provided ormar will user class.__name__.lower()+'s'
# -> artists in this example
ormar_config = ormar_base_config.copy()
id: int = ormar.Integer(primary_key=True)
first_name: str = ormar.String(max_length=100)
@ -30,8 +25,7 @@ class Artist(ormar.Model):
class Album(ormar.Model):
class Meta(MainMeta):
pass
ormar_config = ormar_base_config.copy()
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)

13
docs_src/models/docs014.py
View File

@ -1,18 +1,19 @@
import databases
import sqlalchemy
import ormar
import pydantic
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
completed: bool = ormar.Boolean(default=False)
non_db_field: str = ormar.String(max_length=100, pydantic_only=True)
non_db_field: str = pydantic.Field(max_length=100)

13
docs_src/models/docs015.py
View File

@ -1,22 +1,21 @@
import databases
import sqlalchemy
import ormar
from ormar import property_field
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
completed: bool = ormar.Boolean(default=False)
@property_field
@property
def prefixed_name(self):
return "custom_prefix__" + self.name

14
docs_src/models/docs016.py
View File

@ -1,19 +1,19 @@
import databases
import sqlalchemy
import ormar
import pydantic
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
)
class Config:
allow_mutation = False
model_config = pydantic.ConfigDict(frozen=True)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)

14
docs_src/models/docs017.py
View File

@ -1,20 +1,20 @@
import databases
import sqlalchemy
import ormar
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
# define your constraints in Meta class of the model
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
# define your constraints in OrmarConfig of the model
# it's a list that can contain multiple constraints
# hera a combination of name and column will have a compound index in the db
constraints = [ormar.IndexColumns("name", "completed")]
constraints=[ormar.IndexColumns("name", "completed")],
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)

17
docs_src/models/docs018.py
View File

@ -1,23 +1,24 @@
import datetime
import databases
import sqlalchemy
import databases
import ormar
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
# define your constraints in Meta class of the model
ormar_config = ormar.OrmarConfig(
database=database,
metadata=metadata,
# define your constraints in OrmarConfig of the model
# it's a list that can contain multiple constraints
# hera a combination of name and column will have a level check in the db
constraints = [
constraints=[
ormar.CheckColumns("start_time < end_time", name="date_check"),
]
],
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)