* WIP * WIP - make test_model_definition tests pass * WIP - make test_model_methods pass * WIP - make whole test suit at least run - failing 49/443 tests * WIP fix part of the getting pydantic tests as types of fields are now kept in core schema and not on fieldsinfo * WIP fix validation in update by creating individual fields validators, failing 36/443 * WIP fix __pydantic_extra__ in intializing model, fix test related to pydantic config checks, failing 32/442 * WIP - fix enum schema in model_json_schema, failing 31/442 * WIP - fix copying through model, fix setting pydantic fields on through, fix default config and inheriting from it, failing 26/442 * WIP fix tests checking pydantic schema, fix excluding parent fields, failing 21/442 * WIP some missed files * WIP - fix validators inheritance and fix validators in generated pydantic, failing 17/442 * WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442 * WIP - fix through models setting - only on reverse side of relation, but always on reverse side, failing 15/442 * WIP - working on proper populating __dict__ for relations for new schema dumping, some work on openapi docs, failing 13/442 * WIP - remove property fields as pydantic has now computed_field on its own, failing 9/442 * WIP - fixes in docs, failing 8/442 * WIP - fix tests for largebinary schema, wrapped bytes fields fail in pydantic, will be fixed in pydantic-core, remaining is circural schema for related models, failing 6/442 * WIP - fix to pk only models in schemas * Getting test suites to pass (#1249) * wip, fixing tests * iteration, fixing some more tests * iteration, fixing some more tests * adhere to comments * adhere to comments * remove unnecessary dict call, re-add getattribute for testing * todo for reverse relationship * adhere to comments, remove prints * solve circular refs * all tests pass 🎉 * remove 3.7 from tests * add lint and type check jobs * reforat with ruff, fix jobs * rename jobs * fix imports * fix evaluate in py3.8 * partially fix coverage * fix coverage, add more tests * fix test ids * fix test ids * fix lint, fix docs, make docs fully working scripts, add test docs job * fix pyproject * pin py ver in test docs * change dir in test docs * fix pydantic warning hack * rm poetry call in test_docs * switch to pathlib in test docs * remove coverage req test docs * fix type check tests, fix part of types * fix/skip next part of types * fix next part of types * fix next part of types * fix coverage * fix coverage * fix type (bit dirty 🤷) * fix some code smells * change pre-commit * tweak workflows * remove no root from tests * switch to full python path by passing sys.executable * some small refactor in new base model, one sample test, change makefile * small refactors to reduce complexity of methods * temp add tests for prs against pydantic_v2 * remove all references to __fields__ * remove all references to construct, deprecate the method and update model_construct to be in line with pydantic * deprecate dict and add model_dump, todo switch to model_dict in calls * fix tests * change to union * change to union * change to model_dump and model_dump_json from dict and json deprecated methods, deprecate them in ormar too * finish switching dict() -> model_dump() * finish switching json() -> model_dump_json() * remove fully pydantic_only * switch to extra for payment card, change missed json calls * fix coverage - no more warnings internal * fix coverage - no more warnings internal - part 2 * split model_construct into own and pydantic parts * split determine pydantic field type * change to new field validators * fix benchmarks, add codspeed instead of pytest-benchmark, add action and gh workflow * restore pytest-benchmark * remove codspeed * pin pydantic version, restore codspeed * change on push to pydantic_v2 to trigger first one * Use lifespan function instead of event (#1259) * check return types * fix imports order, set warnings=False on json that passes the dict, fix unnecessary loop in one of the test * remove references to model's meta as it's now ormar config, rename related methods too * filter out pydantic serializer warnings * remove choices leftovers * remove leftovers after property_fields, keep only enough to exclude them in initialization * add migration guide * fix meta references * downgrade databases for now * Change line numbers in documentation (#1265) * proofread and fix the docs, part 1 * proofread and fix the docs for models * proofread and fix the docs for fields * proofread and fix the docs for relations * proofread and fix rest of the docs, add release notes for 0.20 * create tables in new docs src * cleanup old deps, uncomment docs publish on tag * fix import reorder --------- Co-authored-by: TouwaStar <30479449+TouwaStar@users.noreply.github.com> Co-authored-by: Goran Mekić <meka@tilda.center>
20 KiB
Inheritance
Out of various types of ORM models inheritance ormar currently supports two of them:
- Mixins
- Concrete table inheritance (with parents set to
abstract=True)
Types of inheritance
The short summary of different types of inheritance:
- Mixins [SUPPORTED] - don't subclass
ormar.Model, just define fields that are later used on different models (likecreated_dateandupdated_dateon each model), only actual models create tables, but those fields from mixins are added - Concrete table inheritance [SUPPORTED] - means that parent is marked as abstract and each child has its own table with columns from a parent and own child columns, kind of similar to Mixins but parent also is a Model
- Single table inheritance [NOT SUPPORTED] - means that only one table is created with fields that are combination/sum of the parent and all children models but child models use only subset of column in db (all parent and own ones, skipping the other children ones)
- Multi/ Joined table inheritance [NOT SUPPORTED] - means that part of the columns is saved on parent model and part is saved on child model that are connected to each other by kind of one to one relation and under the hood you operate on two models at once
- Proxy models [NOT SUPPORTED] - means that only parent has an actual table, children just add methods, modify settings etc.
Mixins
To use Mixins just define a class that is not inheriting from an ormar.Model but is
defining ormar.Fields as class variables.
base_ormar_config = ormar.OrmarConfig(
database=databases.Database(DATABASE_URL),
metadata=sqlalchemy.MetaData(),
engine=sqlalchemy.create_engine(DATABASE_URL),
)
# a mixin defines the fields but is a normal python class
class AuditMixin:
created_by: str = ormar.String(max_length=100)
updated_by: str = ormar.String(max_length=100, default="Sam")
class DateFieldsMixins:
created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
updated_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
# a models can inherit from one or more mixins
class Category(ormar.Model, DateFieldsMixins, AuditMixin):
ormar_config = base_ormar_config.copy(tablename="categories")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50, unique=True, index=True)
code: int = ormar.Integer()
!!!tip
Note that Mixins are not models, so you still need to inherit
from ormar.Model as well as define ormar_config field in the final model.
A Category class above will have four additional fields: created_date, updated_date,
created_by and updated_by.
There will be only one table created for model Category (categories), with Category class fields
combined with all Mixins fields.
Note that Mixin in class name is optional but is a good python practice.
Concrete table inheritance
In concept concrete table inheritance is very similar to Mixins, but uses
actual ormar.Models as base classes.
!!!warning
Note that base classes have abstract=True set in ormar_config object, if you try
to inherit from non abstract marked class ModelDefinitionError will be raised.
Since this abstract Model will never be initialized you can skip metadata
and database in it's ormar_config definition.
But if you provide it - it will be inherited, that way you do not have to
provide metadata and databases in the final/concrete class
Note that you can always overwrite it in child/concrete class if you need to.
More over at least one of the classes in inheritance chain have to provide both database and metadata -
otherwise an error will be raised.
# note that base classes have abstract=True
# since this model will never be initialized you can skip metadata and database
class AuditModel(ormar.Model):
ormar_config = base_ormar_config.copy(abstract=True)
created_by: str = ormar.String(max_length=100)
updated_by: str = ormar.String(max_length=100, default="Sam")
# but if you provide it it will be inherited - DRY (Don't Repeat Yourself) in action
class DateFieldsModel(ormar.Model):
ormar_config = base_ormar_config.copy(
abstract=True,
metadata=metadata,
database=db,
)
created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
updated_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
# that way you do not have to provide metadata and databases in concrete class
class Category(DateFieldsModel, AuditModel):
ormar_config = base_ormar_config.copy(tablename="categories")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50, unique=True, index=True)
code: int = ormar.Integer()
The list of inherited options/settings is as follows: metadata, database
and constraints.
Of course apart from that all fields from base classes are combined and created in the concrete table of the final Model.
!!!tip
Note how you don't have to provide abstarct=False in the final class - it's the default setting
that is not inherited.
Redefining fields in subclasses
Note that you can redefine previously created fields like in normal python class inheritance.
Whenever you define a field with same name and new definition it will completely replace the previously defined one.
# base class
class DateFieldsModel(ormar.Model):
ormar_config = OrmarConfig(
abstract=True,
metadata=metadata,
database=db,
# note that UniqueColumns need sqlalchemy db columns names not the ormar ones
constraints=[ormar.UniqueColumns("creation_date", "modification_date")]
)
created_date: datetime.datetime = ormar.DateTime(
default=datetime.datetime.now, name="creation_date"
)
updated_date: datetime.datetime = ormar.DateTime(
default=datetime.datetime.now, name="modification_date"
)
class RedefinedField(DateFieldsModel):
ormar_config = OrmarConfig(
tablename="redefines",
metadata=metadata,
database=db,
)
id: int = ormar.Integer(primary_key=True)
# here the created_date is replaced by the String field
created_date: str = ormar.String(max_length=200, name="creation_date")
# you can verify that the final field is correctly declared and created
changed_field = RedefinedField.ormar_config.model_fields["created_date"]
assert changed_field.default is None
assert changed_field.alias == "creation_date"
assert any(x.name == "creation_date" for x in RedefinedField.ormar_config.table.columns)
assert isinstance(
RedefinedField.ormar_config.table.columns["creation_date"].type,
sqlalchemy.sql.sqltypes.String,
)
!!!warning
If you declare UniqueColumns constraint with column names, the final model has to have
a column with the same name declared. Otherwise, the ModelDefinitionError will be raised.
So in example above if you do not provide `name` for `created_date` in `RedefinedField` model
ormar will complain.
`created_date: str = ormar.String(max_length=200) # exception`
`created_date: str = ormar.String(max_length=200, name="creation_date2") # exception`
Relations in inheritance
You can declare relations in every step of inheritance, so both in parent and child classes.
When you define a relation on a child model level it's either overwriting the relation defined in parent model (if the same field name is used), or is accessible only to this child if you define a new relation.
When inheriting relations, you always need to be aware of related_name parameter, that
has to be unique across a related model, when you define multiple child classes that inherit the
same relation.
If you do not provide related_name parameter ormar calculates it for you. This works
with inheritance as all child models have to have different class names, which are used
to calculate the default related_name (class.name.lower()+'s').
But, if you provide a related_name this name cannot be reused in all child models as
they would overwrite each other on the related model side.
Therefore, you have two options:
- redefine relation field in child models and manually provide different
related_nameparameters - let this for
ormarto handle -> auto adjusted related_name are: original related_name + "_" + child model table name
That might sound complicated but let's look at the following example:
ForeignKey relations
# normal model used in relation
class Person(ormar.Model):
ormar_config = base_ormar_config.copy()
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
# parent model - needs to be abstract
class Car(ormar.Model):
ormar_config = base_ormar_config.copy(abstract=True)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50)
owner: Person = ormar.ForeignKey(Person)
# note that we refer to the Person model again so we **have to** provide related_name
co_owner: Person = ormar.ForeignKey(Person, related_name="coowned")
created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
class Truck(Car):
ormar_config = base_ormar_config.copy()
max_capacity: int = ormar.Integer()
class Bus(Car):
ormar_config = base_ormar_config.copy(tablename="buses")
max_persons: int = ormar.Integer()
Now when you will inspect the fields on Person model you will get:
Person.ormar_config.model_fields
"""
{'id': <class 'ormar.fields.model_fields.Integer'>,
'name': <class 'ormar.fields.model_fields.String'>,
'trucks': <class 'ormar.fields.foreign_key.ForeignKey'>,
'coowned_trucks': <class 'ormar.fields.foreign_key.ForeignKey'>,
'buss': <class 'ormar.fields.foreign_key.ForeignKey'>,
'coowned_buses': <class 'ormar.fields.foreign_key.ForeignKey'>}
"""
Note how you have trucks and buss fields that leads to Truck and Bus class that
this Person owns. There were no related_name parameter so default names were used.
At the same time the co-owned cars need to be referenced by coowned_trucks
and coowned_buses. Ormar appended _trucks and _buses suffixes taken from child
model table names.
Seems fine, but the default name for owned trucks is ok (trucks) but the buss is
ugly, so how can we change it?
The solution is pretty simple - just redefine the field in Bus class and provide
different related_name parameter.
# rest of the above example remains the same
class Bus(Car):
ormar_config = base_ormar_config.copy(tablename="buses")
# new field that changes the related_name
owner: Person = ormar.ForeignKey(Person, related_name="buses")
max_persons: int = ormar.Integer()
Now the columns looks much better.
Person.ormar_config.model_fields
"""
{'id': <class 'ormar.fields.model_fields.Integer'>,
'name': <class 'ormar.fields.model_fields.String'>,
'trucks': <class 'ormar.fields.foreign_key.ForeignKey'>,
'coowned_trucks': <class 'ormar.fields.foreign_key.ForeignKey'>,
'buses': <class 'ormar.fields.foreign_key.ForeignKey'>,
'coowned_buses': <class 'ormar.fields.foreign_key.ForeignKey'>}
"""
!!!note
You could also provide related_name for the owner field, that way the proper suffixes
would be added.
`owner: Person = ormar.ForeignKey(Person, related_name="owned")`
and model fields for Person owned cars would become `owned_trucks` and `owned_buses`.
ManyToMany relations
Similarly, you can inherit from Models that have ManyToMany relations declared but there is one, but substantial difference - the Through model.
Since the Through model will be able to hold additional fields, and now it links only two Tables
(from and to ones), each child that inherits the m2m relation field has to have separate
Through model.
Of course, you can overwrite the relation in each Child model, but that requires additional
code and undermines the point of the whole inheritance. Ormar will handle this for you if
you agree with default naming convention, which you can always manually overwrite in
children if needed.
Again, let's look at the example to easier grasp the concepts.
We will modify the previous example described above to use m2m relation for co_owners.
# person remain the same as above
class Person(ormar.Model):
ormar_config = base_ormar_config.copy()
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
# new through model between Person and Car2
class PersonsCar(ormar.Model):
ormar_config = base_ormar_config.copy(tablename="cars_x_persons")
# note how co_owners is now ManyToMany relation
class Car2(ormar.Model):
ormar_config = base_ormar_config.copy(abstract=True)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50)
# note the related_name - needs to be unique across Person
# model, regardless of how many different models leads to Person
owner: Person = ormar.ForeignKey(Person, related_name="owned")
co_owners: List[Person] = ormar.ManyToMany(
Person, through=PersonsCar, related_name="coowned"
)
created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
# child models define only additional Fields
class Truck2(Car2):
ormar_config = base_ormar_config.copy(tablename="trucks2")
max_capacity: int = ormar.Integer()
class Bus2(Car2):
ormar_config = base_ormar_config.copy(tablename="buses2")
max_persons: int = ormar.Integer()
Ormar automatically modifies related_name of the fields to include the table name
of the children models. The default name is original related_name + '_' + child table name.
That way for class Truck2 the relation defined in
owner: Person = ormar.ForeignKey(Person, related_name="owned") becomes owned_trucks2
You can verify the names by inspecting the list of fields present on Person model.
Person.ormar_config.model_fields
{
# note how all relation fields need to be unique on Person
# regardless if autogenerated or manually overwritten
'id': <class 'ormar.fields.model_fields.Integer'>,
'name': <class 'ormar.fields.model_fields.String'>,
# note that we expanded on previous example so all 'old' fields are here
'trucks': <class 'ormar.fields.foreign_key.ForeignKey'>,
'coowned_trucks': <class 'ormar.fields.foreign_key.ForeignKey'>,
'buses': <class 'ormar.fields.foreign_key.ForeignKey'>,
'coowned_buses': <class 'ormar.fields.foreign_key.ForeignKey'>,
# newly defined related fields
'owned_trucks2': <class 'ormar.fields.foreign_key.ForeignKey'>,
'coowned_trucks2': <class 'abc.ManyToMany'>,
'owned_buses2': <class 'ormar.fields.foreign_key.ForeignKey'>,
'coowned_buses2': <class 'abc.ManyToMany'>
}
But that's not all. It's kind of internal to ormar but affects the data structure in the database,
so let's examine the through models for both Bus2 and Truck2 models.
Bus2.ormar_config.model_fields['co_owners'].through
<class 'abc.PersonsCarBus2'>
Bus2.ormar_config.model_fields['co_owners'].through.ormar_config.tablename
'cars_x_persons_buses2'
Truck2.ormar_config.model_fields['co_owners'].through
<class 'abc.PersonsCarTruck2'>
Truck2.ormar_config.model_fields['co_owners'].through.ormar_config.tablename
'cars_x_persons_trucks2'
As you can see above ormar cloned the Through model for each of the Child classes and added
Child class name at the end, while changing the table names of the cloned fields
the name of the table from the child is used.
Note that original model is not only not used, the table for this model is removed from metadata:
Bus2.ormar_config.metadata.tables.keys()
dict_keys(['test_date_models', 'categories', 'subjects', 'persons', 'trucks', 'buses',
'cars_x_persons_trucks2', 'trucks2', 'cars_x_persons_buses2', 'buses2'])
So be aware that if you introduce inheritance along the way and convert a model into abstract parent model you may lose your data on through table if not careful.
!!!note Note that original table name and model name of the Through model is never used. Only the cloned models tables are created and used.
!!!warning
Note that each subclass of the Model that has ManyToMany relation defined generates
a new Through model, meaning also new database table.
That means that each time you define a Child model you need to either manually create
the table in the database, or run a migration (with alembic).
exclude_parent_fields
Ormar allows you to skip certain fields in inherited model that are coming from a parent model.
!!!Note Note that the same behaviour can be achieved by splitting the model into more abstract models and mixins - which is a preferred way in normal circumstances.
To skip certain fields from a child model, list all fields that you want to skip in model.ormar_config.exclude_parent_fields parameter like follows:
base_ormar_config = OrmarConfig(
metadata=sa.MetaData(),
database=databases.Database(DATABASE_URL),
)
class AuditModel(ormar.Model):
ormar_config = base_ormar_config.copy(abstract=True)
created_by: str = ormar.String(max_length=100)
updated_by: str = ormar.String(max_length=100, default="Sam")
class DateFieldsModel(ormar.Model):
ormar_config = base_ormar_config.copy(abstract=True)
created_date: datetime.datetime = ormar.DateTime(
default=datetime.datetime.now, name="creation_date"
)
updated_date: datetime.datetime = ormar.DateTime(
default=datetime.datetime.now, name="modification_date"
)
class Category(DateFieldsModel, AuditModel):
ormar_config = base_ormar_config.copy(
tablename="categories",
# set fields that should be skipped
exclude_parent_fields=["updated_by", "updated_date"],
)
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50, unique=True, index=True)
code: int = ormar.Integer()
# Note that now the update fields in Category are gone in all places -> ormar fields, pydantic fields and sqlachemy table columns
# so full list of available fields in Category is: ["created_by", "created_date", "id", "name", "code"]
Note how you simply need to provide field names and it will exclude the parent field regardless of from which parent model the field is coming from.
!!!Note Note that if you want to overwrite a field in child model you do not have to exclude it, simply overwrite the field declaration in child model with same field name.
!!!Warning Note that this kind of behavior can confuse mypy and static type checkers, yet accessing the non existing fields will fail at runtime. That's why splitting the base classes is preferred.
The same effect can be achieved by splitting base classes like:
base_ormar_config = OrmarConfig(
metadata=sa.MetaData(),
database=databases.Database(DATABASE_URL),
)
class AuditCreateModel(ormar.Model):
ormar_config = base_ormar_config.copy(abstract=True)
created_by: str = ormar.String(max_length=100)
class AuditUpdateModel(ormar.Model):
ormar_config = base_ormar_config.copy(abstract=True)
updated_by: str = ormar.String(max_length=100, default="Sam")
class CreateDateFieldsModel(ormar.Model):
ormar_config = base_ormar_config.copy(abstract=True)
created_date: datetime.datetime = ormar.DateTime(
default=datetime.datetime.now, name="creation_date"
)
class UpdateDateFieldsModel(ormar.Model):
ormar_config = base_ormar_config.copy(abstract=True)
updated_date: datetime.datetime = ormar.DateTime(
default=datetime.datetime.now, name="modification_date"
)
class Category(CreateDateFieldsModel, AuditCreateModel):
ormar_config = base_ormar_config.copy(tablename="categories")
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50, unique=True, index=True)
code: int = ormar.Integer()
That way you can inherit from both create and update classes if needed, and only one of them otherwise.