WIP - Pydantic v2 support (#1238)
* WIP * WIP - make test_model_definition tests pass * WIP - make test_model_methods pass * WIP - make whole test suit at least run - failing 49/443 tests * WIP fix part of the getting pydantic tests as types of fields are now kept in core schema and not on fieldsinfo * WIP fix validation in update by creating individual fields validators, failing 36/443 * WIP fix __pydantic_extra__ in intializing model, fix test related to pydantic config checks, failing 32/442 * WIP - fix enum schema in model_json_schema, failing 31/442 * WIP - fix copying through model, fix setting pydantic fields on through, fix default config and inheriting from it, failing 26/442 * WIP fix tests checking pydantic schema, fix excluding parent fields, failing 21/442 * WIP some missed files * WIP - fix validators inheritance and fix validators in generated pydantic, failing 17/442 * WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442 * WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442 * WIP - working on proper populating __dict__ for relations for new schema dumping, some work on openapi docs, failing 13/442 * WIP - remove property fields as pydantic has now computed_field on its own, failing 9/442 * WIP - fixes in docs, failing 8/442 * WIP - fix tests for largebinary schema, wrapped bytes fields fail in pydantic, will be fixed in pydantic-core, remaining is circural schema for related models, failing 6/442 * WIP - fix to pk only models in schemas * Getting test suites to pass (#1249) * wip, fixing tests * iteration, fixing some more tests * iteration, fixing some more tests * adhere to comments * adhere to comments * remove unnecessary dict call, re-add getattribute for testing * todo for reverse relationship * adhere to comments, remove prints * solve circular refs * all tests pass 🎉 * remove 3.7 from tests * add lint and type check jobs * reforat with ruff, fix jobs * rename jobs * fix imports * fix evaluate in py3.8 * partially fix coverage * fix coverage, add more tests * fix test ids * fix test ids * fix lint, fix docs, make docs fully working scripts, add test docs job * fix pyproject * pin py ver in test docs * change dir in test docs * fix pydantic warning hack * rm poetry call in test_docs * switch to pathlib in test docs * remove coverage req test docs * fix type check tests, fix part of types * fix/skip next part of types * fix next part of types * fix next part of types * fix coverage * fix coverage * fix type (bit dirty 🤷) * fix some code smells * change pre-commit * tweak workflows * remove no root from tests * switch to full python path by passing sys.executable * some small refactor in new base model, one sample test, change makefile * small refactors to reduce complexity of methods * temp add tests for prs against pydantic_v2 * remove all references to __fields__ * remove all references to construct, deprecate the method and update model_construct to be in line with pydantic * deprecate dict and add model_dump, todo switch to model_dict in calls * fix tests * change to union * change to union * change to model_dump and model_dump_json from dict and json deprecated methods, deprecate them in ormar too * finish switching dict() -> model_dump() * finish switching json() -> model_dump_json() * remove fully pydantic_only * switch to extra for payment card, change missed json calls * fix coverage - no more warnings internal * fix coverage - no more warnings internal - part 2 * split model_construct into own and pydantic parts * split determine pydantic field type * change to new field validators * fix benchmarks, add codspeed instead of pytest-benchmark, add action and gh workflow * restore pytest-benchmark * remove codspeed * pin pydantic version, restore codspeed * change on push to pydantic_v2 to trigger first one * Use lifespan function instead of event (#1259) * check return types * fix imports order, set warnings=False on json that passes the dict, fix unnecessary loop in one of the test * remove references to model's meta as it's now ormar config, rename related methods too * filter out pydantic serializer warnings * remove choices leftovers * remove leftovers after property_fields, keep only enough to exclude them in initialization * add migration guide * fix meta references * downgrade databases for now * Change line numbers in documentation (#1265) * proofread and fix the docs, part 1 * proofread and fix the docs for models * proofread and fix the docs for fields * proofread and fix the docs for relations * proofread and fix rest of the docs, add release notes for 0.20 * create tables in new docs src * cleanup old deps, uncomment docs publish on tag * fix import reorder --------- Co-authored-by: TouwaStar <30479449+TouwaStar@users.noreply.github.com> Co-authored-by: Goran Mekić <meka@tilda.center>
This commit is contained in:
collerek
@ -1,29 +1,20 @@
|
||||
# type: ignore
|
||||
import base64
|
||||
import datetime
|
||||
import decimal
|
||||
import hashlib
|
||||
import uuid
|
||||
import datetime
|
||||
from typing import Any
|
||||
|
||||
import databases
|
||||
import pytest
|
||||
import sqlalchemy
|
||||
|
||||
import ormar
|
||||
import pytest
|
||||
from ormar import ModelDefinitionError, NoMatch
|
||||
from ormar.fields.sqlalchemy_encrypted import EncryptedString
|
||||
from tests.settings import DATABASE_URL
|
||||
|
||||
database = databases.Database(DATABASE_URL)
|
||||
metadata = sqlalchemy.MetaData()
|
||||
|
||||
|
||||
class BaseMeta(ormar.ModelMeta):
|
||||
metadata = metadata
|
||||
database = database
|
||||
|
||||
from tests.lifespan import init_tests
|
||||
from tests.settings import create_config
|
||||
|
||||
base_ormar_config = create_config()
|
||||
default_fernet = dict(
|
||||
encrypt_secret="asd123", encrypt_backend=ormar.EncryptBackends.FERNET
|
||||
)
|
||||
@ -41,8 +32,7 @@ class DummyBackend(ormar.fields.EncryptBackend):
|
||||
|
||||
|
||||
class Author(ormar.Model):
|
||||
class Meta(BaseMeta):
|
||||
tablename = "authors"
|
||||
ormar_config = base_ormar_config.copy(tablename="authors")
|
||||
|
||||
id: int = ormar.Integer(primary_key=True)
|
||||
name: str = ormar.String(max_length=100, **default_fernet)
|
||||
@ -70,6 +60,10 @@ class Author(ormar.Model):
|
||||
test_smallint: int = ormar.SmallInteger(default=0, **default_fernet)
|
||||
test_decimal = ormar.Decimal(scale=2, precision=10, **default_fernet)
|
||||
test_decimal2 = ormar.Decimal(max_digits=10, decimal_places=2, **default_fernet)
|
||||
test_bytes = ormar.LargeBinary(max_length=100, **default_fernet)
|
||||
test_b64bytes = ormar.LargeBinary(
|
||||
max_length=100, represent_as_base64_str=True, **default_fernet
|
||||
)
|
||||
custom_backend: str = ormar.String(
|
||||
max_length=200,
|
||||
encrypt_secret="asda8",
|
||||
@ -79,8 +73,7 @@ class Author(ormar.Model):
|
||||
|
||||
|
||||
class Hash(ormar.Model):
|
||||
class Meta(BaseMeta):
|
||||
tablename = "hashes"
|
||||
ormar_config = base_ormar_config.copy(tablename="hashes")
|
||||
|
||||
id: int = ormar.Integer(primary_key=True)
|
||||
name: str = ormar.String(
|
||||
@ -91,8 +84,7 @@ class Hash(ormar.Model):
|
||||
|
||||
|
||||
class Filter(ormar.Model):
|
||||
class Meta(BaseMeta):
|
||||
tablename = "filters"
|
||||
ormar_config = base_ormar_config.copy(tablename="filters")
|
||||
|
||||
id: int = ormar.Integer(primary_key=True)
|
||||
name: str = ormar.String(max_length=100, **default_fernet)
|
||||
@ -100,29 +92,21 @@ class Filter(ormar.Model):
|
||||
|
||||
|
||||
class Report(ormar.Model):
|
||||
class Meta(BaseMeta):
|
||||
tablename = "reports"
|
||||
ormar_config = base_ormar_config.copy(tablename="reports")
|
||||
|
||||
id: int = ormar.Integer(primary_key=True)
|
||||
name: str = ormar.String(max_length=100)
|
||||
filters = ormar.ManyToMany(Filter)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True, scope="module")
|
||||
def create_test_database():
|
||||
engine = sqlalchemy.create_engine(DATABASE_URL)
|
||||
metadata.drop_all(engine)
|
||||
metadata.create_all(engine)
|
||||
yield
|
||||
metadata.drop_all(engine)
|
||||
create_test_database = init_tests(base_ormar_config)
|
||||
|
||||
|
||||
def test_error_on_encrypted_pk():
|
||||
with pytest.raises(ModelDefinitionError):
|
||||
|
||||
class Wrong(ormar.Model):
|
||||
class Meta(BaseMeta):
|
||||
tablename = "wrongs"
|
||||
ormar_config = base_ormar_config.copy(tablename="wrongs")
|
||||
|
||||
id: int = ormar.Integer(
|
||||
primary_key=True,
|
||||
@ -135,8 +119,7 @@ def test_error_on_encrypted_relation():
|
||||
with pytest.raises(ModelDefinitionError):
|
||||
|
||||
class Wrong2(ormar.Model):
|
||||
class Meta(BaseMeta):
|
||||
tablename = "wrongs2"
|
||||
ormar_config = base_ormar_config.copy(tablename="wrongs2")
|
||||
|
||||
id: int = ormar.Integer(primary_key=True)
|
||||
author = ormar.ForeignKey(
|
||||
@ -150,8 +133,7 @@ def test_error_on_encrypted_m2m_relation():
|
||||
with pytest.raises(ModelDefinitionError):
|
||||
|
||||
class Wrong3(ormar.Model):
|
||||
class Meta(BaseMeta):
|
||||
tablename = "wrongs3"
|
||||
ormar_config = base_ormar_config.copy(tablename="wrongs3")
|
||||
|
||||
id: int = ormar.Integer(primary_key=True)
|
||||
author = ormar.ManyToMany(
|
||||
@ -165,8 +147,7 @@ def test_wrong_backend():
|
||||
with pytest.raises(ModelDefinitionError):
|
||||
|
||||
class Wrong3(ormar.Model):
|
||||
class Meta(BaseMeta):
|
||||
tablename = "wrongs3"
|
||||
ormar_config = base_ormar_config.copy(tablename="wrongs3")
|
||||
|
||||
id: int = ormar.Integer(primary_key=True)
|
||||
author = ormar.Integer(
|
||||
@ -177,12 +158,12 @@ def test_wrong_backend():
|
||||
|
||||
|
||||
def test_db_structure():
|
||||
assert Author.Meta.table.c.get("name").type.__class__ == EncryptedString
|
||||
assert Author.ormar_config.table.c.get("name").type.__class__ == EncryptedString
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_save_and_retrieve():
|
||||
async with database:
|
||||
async with base_ormar_config.database:
|
||||
test_uuid = uuid.uuid4()
|
||||
await Author(
|
||||
name="Test",
|
||||
@ -191,10 +172,12 @@ async def test_save_and_retrieve():
|
||||
uuid_test=test_uuid,
|
||||
test_float=1.2,
|
||||
test_bool=True,
|
||||
test_decimal=decimal.Decimal(3.5),
|
||||
test_decimal=3.57,
|
||||
test_decimal2=decimal.Decimal(5.5),
|
||||
test_json=dict(aa=12),
|
||||
custom_backend="test12",
|
||||
test_bytes=b"test",
|
||||
test_b64bytes=b"test2",
|
||||
).save()
|
||||
author = await Author.objects.get()
|
||||
|
||||
@ -214,14 +197,17 @@ async def test_save_and_retrieve():
|
||||
assert author.test_float2 is None
|
||||
assert author.test_bigint == 0
|
||||
assert author.test_json == {"aa": 12}
|
||||
assert author.test_decimal == 3.5
|
||||
assert float(author.test_decimal) == 3.57
|
||||
assert author.test_decimal2 == 5.5
|
||||
assert author.custom_backend == "test12"
|
||||
assert author.test_bytes == "test".encode("utf-8")
|
||||
assert author.test_b64bytes == "dGVzdDI="
|
||||
assert base64.b64decode(author.test_b64bytes) == b"test2"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fernet_filters_nomatch():
|
||||
async with database:
|
||||
async with base_ormar_config.database:
|
||||
await Filter(name="test1").save()
|
||||
await Filter(name="test1").save()
|
||||
|
||||
@ -236,7 +222,7 @@ async def test_fernet_filters_nomatch():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_hash_filters_works():
|
||||
async with database:
|
||||
async with base_ormar_config.database:
|
||||
await Hash(name="test1").save()
|
||||
await Hash(name="test2").save()
|
||||
|
||||
@ -253,7 +239,7 @@ async def test_hash_filters_works():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_related_model_fields_properly_decrypted():
|
||||
async with database:
|
||||
async with base_ormar_config.database:
|
||||
hash1 = await Hash(name="test1").save()
|
||||
report = await Report.objects.create(name="Report1")
|
||||
await report.filters.create(name="test1", hash=hash1)
|
||||
|
||||
Reference in New Issue
Block a user