Prodream/ormar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WIP - Pydantic v2 support (#1238)

Browse Source 
* WIP

* WIP - make test_model_definition tests pass

* WIP - make test_model_methods pass

* WIP - make whole test suit at least run - failing 49/443 tests

* WIP fix part of the getting pydantic tests as types of fields are now kept in core schema and not on fieldsinfo

* WIP fix validation in update by creating individual fields validators, failing 36/443

* WIP fix __pydantic_extra__ in intializing model, fix test related to pydantic config checks, failing 32/442

* WIP - fix enum schema in model_json_schema, failing 31/442

* WIP - fix copying through model, fix setting pydantic fields on through, fix default config and inheriting from it, failing 26/442

* WIP fix tests checking pydantic schema, fix excluding parent fields, failing 21/442

* WIP some missed files

* WIP - fix validators inheritance and fix validators in generated pydantic, failing 17/442

* WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442

* WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442

* WIP - working on proper populating __dict__ for relations for new schema dumping, some work on openapi docs, failing 13/442

* WIP - remove property fields as pydantic has now computed_field on its own, failing 9/442

* WIP - fixes in docs, failing 8/442

* WIP - fix tests for largebinary schema, wrapped bytes fields fail in pydantic, will be fixed in pydantic-core, remaining is circural schema for related models, failing 6/442

* WIP - fix to pk only models in schemas

* Getting test suites to pass (#1249)

* wip, fixing tests

* iteration, fixing some more tests

* iteration, fixing some more tests

* adhere to comments

* adhere to comments

* remove unnecessary dict call, re-add getattribute for testing

* todo for reverse relationship

* adhere to comments, remove prints

* solve circular refs

* all tests pass 🎉

* remove 3.7 from tests

* add lint and type check jobs

* reforat with ruff, fix jobs

* rename jobs

* fix imports

* fix evaluate in py3.8

* partially fix coverage

* fix coverage, add more tests

* fix test ids

* fix test ids

* fix lint, fix docs, make docs fully working scripts, add test docs job

* fix pyproject

* pin py ver in test docs

* change dir in test docs

* fix pydantic warning hack

* rm poetry call in test_docs

* switch to pathlib in test docs

* remove coverage req test docs

* fix type check tests, fix part of types

* fix/skip next part of types

* fix next part of types

* fix next part of types

* fix coverage

* fix coverage

* fix type (bit dirty 🤷)

* fix some code smells

* change pre-commit

* tweak workflows

* remove no root from tests

* switch to full python path by passing sys.executable

* some small refactor in new base model, one sample test, change makefile

* small refactors to reduce complexity of methods

* temp add tests for prs against pydantic_v2

* remove all references to __fields__

* remove all references to construct, deprecate the method and update model_construct to be in line with pydantic

* deprecate dict and add model_dump, todo switch to model_dict in calls

* fix tests

* change to union

* change to union

* change to model_dump and model_dump_json from dict and json deprecated methods, deprecate them in ormar too

* finish switching dict() -> model_dump()

* finish switching json() -> model_dump_json()

* remove fully pydantic_only

* switch to extra for payment card, change missed json calls

* fix coverage - no more warnings internal

* fix coverage - no more warnings internal - part 2

* split model_construct into own and pydantic parts

* split determine pydantic field type

* change to new field validators

* fix benchmarks, add codspeed instead of pytest-benchmark, add action and gh workflow

* restore pytest-benchmark

* remove codspeed

* pin pydantic version, restore codspeed

* change on push to pydantic_v2 to trigger first one

* Use lifespan function instead of event (#1259)

* check return types

* fix imports order, set warnings=False on json that passes the dict, fix unnecessary loop in one of the test

* remove references to model's meta as it's now ormar config, rename related methods too

* filter out pydantic serializer warnings

* remove choices leftovers

* remove leftovers after property_fields, keep only enough to exclude them in initialization

* add migration guide

* fix meta references

* downgrade databases for now

* Change line numbers in documentation (#1265)

* proofread and fix the docs, part 1

* proofread and fix the docs for models

* proofread and fix the docs for fields

* proofread and fix the docs for relations

* proofread and fix rest of the docs, add release notes for 0.20

* create tables in new docs src

* cleanup old deps, uncomment docs publish on tag

* fix import reorder

---------

Co-authored-by: TouwaStar <30479449+TouwaStar@users.noreply.github.com>
Co-authored-by: Goran Mekić <meka@tilda.center>
This commit is contained in:
collerek
2024-03-23 19:28:28 +01:00
committed by GitHub
parent 3a206dd8dc
commit 500625f0ec
294 changed files with 8132 additions and 9311 deletions
Download Patch File Download Diff File Expand all files Collapse all files

281
tests/test_model_definition/test_models.py
View File

@ -3,26 +3,22 @@ import base64
import datetime
import os
import uuid
from typing import List
from enum import Enum
import databases
import ormar
import pydantic
import pytest
import sqlalchemy
import ormar
from ormar.exceptions import ModelError, NoMatch, QueryDefinitionError
from tests.settings import DATABASE_URL
database = databases.Database(DATABASE_URL, force_rollback=True)
metadata = sqlalchemy.MetaData()
from tests.lifespan import init_tests
from tests.settings import create_config
base_ormar_config = create_config()
class JsonSample(ormar.Model):
class Meta:
tablename = "jsons"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="jsons")
id: int = ormar.Integer(primary_key=True)
test_json = ormar.JSON(nullable=True)
@ -33,13 +29,10 @@ blob2 = b"test2icac89uc98"
class LargeBinarySample(ormar.Model):
class Meta:
tablename = "my_bolbs"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="my_bolbs")
id: int = ormar.Integer(primary_key=True)
test_binary: bytes = ormar.LargeBinary(max_length=100000, choices=[blob, blob2])
test_binary: bytes = ormar.LargeBinary(max_length=100000)
blob3 = os.urandom(64)
@ -47,47 +40,34 @@ blob4 = os.urandom(100)
class LargeBinaryStr(ormar.Model):
class Meta:
tablename = "my_str_blobs"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="my_str_bolbs")
id: int = ormar.Integer(primary_key=True)
test_binary: str = ormar.LargeBinary(
max_length=100000, choices=[blob3, blob4], represent_as_base64_str=True
max_length=100000, represent_as_base64_str=True
)
class LargeBinaryNullableStr(ormar.Model):
class Meta:
tablename = "my_str_blobs2"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="my_str_bolbs2")
id: int = ormar.Integer(primary_key=True)
test_binary: str = ormar.LargeBinary(
max_length=100000,
choices=[blob3, blob4],
represent_as_base64_str=True,
nullable=True,
)
class UUIDSample(ormar.Model):
class Meta:
tablename = "uuids"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="uuids")
id: uuid.UUID = ormar.UUID(primary_key=True, default=uuid.uuid4)
test_text: str = ormar.Text()
class UUIDSample2(ormar.Model):
class Meta:
tablename = "uuids2"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="uuids2")
id: uuid.UUID = ormar.UUID(
primary_key=True, default=uuid.uuid4, uuid_format="string"
@ -96,95 +76,78 @@ class UUIDSample2(ormar.Model):
class User(ormar.Model):
class Meta:
tablename = "users"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="users")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100, default="")
class User2(ormar.Model):
class Meta:
tablename = "users2"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="users2")
id: str = ormar.String(primary_key=True, max_length=100)
name: str = ormar.String(max_length=100, default="")
class Product(ormar.Model):
class Meta:
tablename = "product"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="product")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
rating: int = ormar.Integer(minimum=1, maximum=5)
in_stock: bool = ormar.Boolean(default=False)
last_delivery: datetime.date = ormar.Date(default=datetime.datetime.now)
last_delivery: datetime.date = ormar.Date(default=datetime.date.today)
country_name_choices = ("Canada", "Algeria", "United States", "Belize")
country_taxed_choices = (True,)
country_country_code_choices = (-10, 1, 213, 1200)
class CountryNameEnum(Enum):
CANADA = "Canada"
ALGERIA = "Algeria"
USA = "United States"
BELIZE = "Belize"
class CountryCodeEnum(int, Enum):
MINUS_TEN = -10
ONE = 1
TWO_HUNDRED_THIRTEEN = 213
THOUSAND_TWO_HUNDRED = 1200
class Country(ormar.Model):
class Meta:
tablename = "country"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="country")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(
max_length=9, choices=country_name_choices, default="Canada"
)
taxed: bool = ormar.Boolean(choices=country_taxed_choices, default=True)
country_code: int = ormar.Integer(
minimum=0, maximum=1000, choices=country_country_code_choices, default=1
)
name: CountryNameEnum = ormar.Enum(enum_class=CountryNameEnum, default="Canada")
taxed: bool = ormar.Boolean(default=True)
country_code: int = ormar.Enum(enum_class=CountryCodeEnum, default=1)
class NullableCountry(ormar.Model):
class Meta:
tablename = "country2"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="country2")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=9, choices=country_name_choices, nullable=True)
name: CountryNameEnum = ormar.Enum(enum_class=CountryNameEnum, nullable=True)
class NotNullableCountry(ormar.Model):
class Meta:
tablename = "country3"
metadata = metadata
database = database
ormar_config = base_ormar_config.copy(tablename="country3")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=9, choices=country_name_choices, nullable=False)
name: CountryNameEnum = ormar.Enum(enum_class=CountryNameEnum, nullable=False)
@pytest.fixture(autouse=True, scope="module")
def create_test_database():
engine = sqlalchemy.create_engine(DATABASE_URL)
metadata.drop_all(engine)
metadata.create_all(engine)
yield
metadata.drop_all(engine)
create_test_database = init_tests(base_ormar_config)
def test_model_class():
assert list(User.Meta.model_fields.keys()) == ["id", "name"]
assert issubclass(User.Meta.model_fields["id"].__class__, pydantic.fields.FieldInfo)
assert User.Meta.model_fields["id"].primary_key is True
assert isinstance(User.Meta.model_fields["name"], pydantic.fields.FieldInfo)
assert User.Meta.model_fields["name"].max_length == 100
assert isinstance(User.Meta.table, sqlalchemy.Table)
assert list(User.ormar_config.model_fields.keys()) == ["id", "name"]
assert issubclass(
User.ormar_config.model_fields["id"].__class__, pydantic.fields.FieldInfo
)
assert User.ormar_config.model_fields["id"].primary_key is True
assert isinstance(User.ormar_config.model_fields["name"], pydantic.fields.FieldInfo)
assert User.ormar_config.model_fields["name"].max_length == 100
assert isinstance(User.ormar_config.table, sqlalchemy.Table)
def test_wrong_field_name():
@ -200,8 +163,8 @@ def test_model_pk():
@pytest.mark.asyncio
async def test_json_column():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await JsonSample.objects.create(test_json=dict(aa=12))
await JsonSample.objects.create(test_json='{"aa": 12}')
@ -216,8 +179,8 @@ async def test_json_column():
@pytest.mark.asyncio
async def test_binary_column():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await LargeBinarySample.objects.create(test_binary=blob)
await LargeBinarySample.objects.create(test_binary=blob2)
@ -232,8 +195,8 @@ async def test_binary_column():
@pytest.mark.asyncio
async def test_binary_str_column():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await LargeBinaryStr(test_binary=blob3).save()
await LargeBinaryStr.objects.create(test_binary=blob4)
@ -248,8 +211,8 @@ async def test_binary_str_column():
@pytest.mark.asyncio
async def test_binary_nullable_str_column():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await LargeBinaryNullableStr().save()
await LargeBinaryNullableStr.objects.create()
items = await LargeBinaryNullableStr.objects.all()
@ -279,8 +242,8 @@ async def test_binary_nullable_str_column():
@pytest.mark.asyncio
async def test_uuid_column():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
u1 = await UUIDSample.objects.create(test_text="aa")
u2 = await UUIDSample.objects.create(test_text="bb")
@ -303,7 +266,7 @@ async def test_uuid_column():
assert item2.id == item3.id
assert isinstance(item3.id, uuid.UUID)
u3 = await UUIDSample2(**u1.dict()).save()
u3 = await UUIDSample2(**u1.model_dump()).save()
u1_2 = await UUIDSample.objects.get(pk=u3.id)
assert u1_2 == u1
@ -314,8 +277,8 @@ async def test_uuid_column():
@pytest.mark.asyncio
async def test_model_crud():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
users = await User.objects.all()
assert users == []
@ -341,8 +304,8 @@ async def test_model_crud():
@pytest.mark.asyncio
async def test_model_get():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
with pytest.raises(ormar.NoMatch):
await User.objects.get()
@ -366,8 +329,8 @@ async def test_model_get():
@pytest.mark.asyncio
async def test_model_filter():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await User.objects.create(name="Tom")
await User.objects.create(name="Jane")
await User.objects.create(name="Lucy")
@ -422,7 +385,7 @@ async def test_model_filter():
@pytest.mark.asyncio
async def test_wrong_query_contains_model():
async with database:
async with base_ormar_config.database:
with pytest.raises(QueryDefinitionError):
product = Product(name="90%-Cotton", rating=2)
await Product.objects.filter(name__contains=product).count()
@ -430,8 +393,8 @@ async def test_wrong_query_contains_model():
@pytest.mark.asyncio
async def test_model_exists():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await User.objects.create(name="Tom")
assert await User.objects.filter(name="Tom").exists() is True
assert await User.objects.filter(name="Jane").exists() is False
@ -439,8 +402,8 @@ async def test_model_exists():
@pytest.mark.asyncio
async def test_model_count():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await User.objects.create(name="Tom")
await User.objects.create(name="Jane")
await User.objects.create(name="Lucy")
@ -451,8 +414,8 @@ async def test_model_count():
@pytest.mark.asyncio
async def test_model_limit():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await User.objects.create(name="Tom")
await User.objects.create(name="Jane")
await User.objects.create(name="Lucy")
@ -462,8 +425,8 @@ async def test_model_limit():
@pytest.mark.asyncio
async def test_model_limit_with_filter():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await User.objects.create(name="Tom")
await User.objects.create(name="Tom")
await User.objects.create(name="Tom")
@ -475,8 +438,8 @@ async def test_model_limit_with_filter():
@pytest.mark.asyncio
async def test_offset():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await User.objects.create(name="Tom")
await User.objects.create(name="Jane")
@ -486,8 +449,8 @@ async def test_offset():
@pytest.mark.asyncio
async def test_model_first():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
tom = await User.objects.create(name="Tom")
jane = await User.objects.create(name="Jane")
@ -500,27 +463,11 @@ async def test_model_first():
assert await User.objects.order_by("name").first() == jane
def not_contains(a, b):
return a not in b
def contains(a, b):
return a in b
def check_choices(values: tuple, ops: List):
ops_dict = {"in": contains, "out": not_contains}
checks = (country_name_choices, country_taxed_choices, country_country_code_choices)
assert all(
[ops_dict[op](value, check) for value, op, check in zip(values, ops, checks)]
)
@pytest.mark.asyncio
async def test_model_choices():
"""Test that choices work properly for various types of fields."""
async with database:
# Test valid choices.
"""Test that enum work properly for various types of fields."""
async with base_ormar_config.database:
# Test valid enums values.
await asyncio.gather(
Country.objects.create(name="Canada", taxed=True, country_code=1),
Country.objects.create(name="Algeria", taxed=True, country_code=213),
@ -529,54 +476,12 @@ async def test_model_choices():
with pytest.raises(ValueError):
name, taxed, country_code = "Saudi Arabia", True, 1
check_choices((name, taxed, country_code), ["out", "in", "in"])
await Country.objects.create(
name=name, taxed=taxed, country_code=country_code
)
with pytest.raises(ValueError):
name, taxed, country_code = "Algeria", False, 1
check_choices((name, taxed, country_code), ["in", "out", "in"])
await Country.objects.create(
name=name, taxed=taxed, country_code=country_code
)
with pytest.raises(ValueError):
name, taxed, country_code = "Algeria", True, 967
check_choices((name, taxed, country_code), ["in", "in", "out"])
await Country.objects.create(
name=name, taxed=taxed, country_code=country_code
)
with pytest.raises(ValueError):
name, taxed, country_code = (
"United States",
True,
1,
) # name is too long but is a valid choice
check_choices((name, taxed, country_code), ["in", "in", "in"])
await Country.objects.create(
name=name, taxed=taxed, country_code=country_code
)
with pytest.raises(ValueError):
name, taxed, country_code = (
"Algeria",
True,
-10,
) # country code is too small but is a valid choice
check_choices((name, taxed, country_code), ["in", "in", "in"])
await Country.objects.create(
name=name, taxed=taxed, country_code=country_code
)
with pytest.raises(ValueError):
name, taxed, country_code = (
"Algeria",
True,
1200,
) # country code is too large but is a valid choice
check_choices((name, taxed, country_code), ["in", "in", "in"])
await Country.objects.create(
name=name, taxed=taxed, country_code=country_code
)
@ -584,34 +489,24 @@ async def test_model_choices():
# test setting after init also triggers validation
with pytest.raises(ValueError):
name, taxed, country_code = "Algeria", True, 967
check_choices((name, taxed, country_code), ["in", "in", "out"])
country = Country()
country.country_code = country_code
with pytest.raises(ValueError):
name, taxed, country_code = "Saudi Arabia", True, 1
check_choices((name, taxed, country_code), ["out", "in", "in"])
country = Country()
country.name = name
with pytest.raises(ValueError):
name, taxed, country_code = "Algeria", False, 1
check_choices((name, taxed, country_code), ["in", "out", "in"])
country = Country()
country.taxed = taxed
# check also update from queryset
with pytest.raises(ValueError):
name, taxed, country_code = "Algeria", False, 1
check_choices((name, taxed, country_code), ["in", "out", "in"])
await Country(name="Belize").save()
await Country.objects.filter(name="Belize").update(name="Vietnam")
@pytest.mark.asyncio
async def test_nullable_field_model_choices():
"""Test that choices work properly for according to nullable setting"""
async with database:
async def test_nullable_field_model_enum():
"""Test that enum work properly for according to nullable setting"""
async with base_ormar_config.database:
c1 = await NullableCountry(name=None).save()
assert c1.name is None
@ -621,8 +516,8 @@ async def test_nullable_field_model_choices():
@pytest.mark.asyncio
async def test_start_and_end_filters():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await User.objects.create(name="Markos Uj")
await User.objects.create(name="Maqua Bigo")
await User.objects.create(name="maqo quidid")
@ -651,8 +546,8 @@ async def test_start_and_end_filters():
@pytest.mark.asyncio
async def test_get_and_first():
async with database:
async with database.transaction(force_rollback=True):
async with base_ormar_config.database:
async with base_ormar_config.database.transaction(force_rollback=True):
await User.objects.create(name="Tom")
await User.objects.create(name="Jane")
await User.objects.create(name="Lucy")
@ -681,4 +576,4 @@ async def test_get_and_first():
def test_constraints():
with pytest.raises(pydantic.ValidationError) as e:
Product(name="T-Shirt", rating=50, in_stock=True)
assert "ensure this value is less than or equal to 5" in str(e.value)
assert "Input should be less than or equal to 5 " in str(e.value)