Prodream/ormar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add cloning through model and working inheritance with many to many fields - for further tests

Browse Source
This commit is contained in:
collerek
2020-12-29 16:40:46 +01:00
parent 63f7b0d572
commit 27c377ec5c
6 changed files with 229 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/models/inheritance.md
View File

@ -13,7 +13,7 @@ The short summary of different types of inheritance is:
later used on different models (like `created_date` and `updated_date` on each model),
only actual models create tables, but those fields from mixins are added
* **Concrete table inheritance [SUPPORTED]** - means that parent is marked as abstract
and each child has it's own table with columns from parent and own child columns, kind
and each child has its own table with columns from a parent and own child columns, kind
of similar to Mixins but parent also is a Model
* **Single table inheritance [NOT SUPPORTED]** - means that only one table is created
with fields that are combination/sum of the parent and all children models but child
@ -196,8 +196,12 @@ assert isinstance(
You can declare relations in every step of inheritance, so both in parent and child
classes.
When you define a relation on a child model level it's either overwriting the relation
defined in parent model (if the same field name is used), or is accessible only to this
child if you define a new relation.
When inheriting relations, you always need to be aware of `related_name` parameter, that
has to be unique across a model, when you define multiple child classes that inherit the
has to be unique across a related model, when you define multiple child classes that inherit the
same relation.
If you do not provide `related_name` parameter ormar calculates it for you. This works

6
docs/releases.md
View File

@ -4,12 +4,13 @@
* **Breaking:** `remove()` parent from child side in reverse ForeignKey relation now requires passing a relation `name`,
as the same model can be registered multiple times and `ormar` needs to know from which relation on the parent you want to remove the child.
* **Breaking:** applying `limit` and `offset` with `select_related` is by default applied only on the main table before the join -> meaning that not the total
number of rows is limited but just main models (first one in the query, the one to used to construct it). Yu can still limit all rows from db response with `limit_raw_sql=True` flag on either `limit` or `offset` (or both)
number of rows is limited but just number of main models (first one in the query, the one used to construct it). You can still limit all rows from db response with `limit_raw_sql=True` flag on either `limit` or `offset` (or both)
* **Breaking:** issuing `first()` now fetches the first row ordered by the primary key asc (so first one inserted (can be different for non number primary keys - i.e. alphabetical order of string))
* **Breaking:** issuing `get()` **without any filters** now fetches the first row ordered by the primary key desc (so should be last one inserted (can be different for non number primary keys - i.e. alphabetical order of string))
* **Breaking (internal):** sqlalchemy columns kept at `Meta.columns` are no longer bind to table, so you cannot get the column straight from there
## Features
* Introduce **inheritance**, for now two types of inheritance are possible:
* Introduce **inheritance**. For now two types of inheritance are possible:
* **Mixins** - don't subclass `ormar.Model`, just define fields that are later used on different models (like `created_date` and `updated_date` on each child model), only actual models create tables, but those fields from mixins are added
* **Concrete table inheritance** - means that parent is marked as `abstract=True` in Meta class and each child has its own table with columns from the parent and own child columns, kind of similar to Mixins but parent also is a (an abstract) Model
* To read more check the docs on models -> inheritance section.
@ -18,6 +19,7 @@ as the same model can be registered multiple times and `ormar` needs to know fro
## Fixes
* Fix minor bug in `order_by` for primary model order bys
* Fix in `prefetch_query` for multiple related_names for the same model.
* Fix using same `related_name` on different models leading to the same related `Model` overwriting each other, now `ModelDefinitionError` is raised and you need to change the name.
## Docs
* Split and cleanup in docs:

55
ormar/models/helpers/pydantic.py
View File

@ -12,12 +12,50 @@ if TYPE_CHECKING: # pragma no cover
from ormar import Model
def verify_related_name_dont_duplicate(
child: Type["Model"], parent_model: Type["Model"], related_name: str,
) -> None:
"""
Verifies whether the used related_name (regardless of the fact if user defined or
auto generated) is already used on related model, but is connected with other model
than the one that we connect right now.
:raises: ModelDefinitionError if name is already used but lead to different related
model
:param child: related Model class
:type child: ormar.models.metaclass.ModelMetaclass
:param parent_model: parent Model class
:type parent_model: ormar.models.metaclass.ModelMetaclass
:param related_name:
:type related_name:
:return: None
:rtype: None
"""
if parent_model.Meta.model_fields.get(related_name):
fk_field = parent_model.Meta.model_fields.get(related_name)
if not fk_field:
return
if fk_field.to != child and fk_field.to.Meta != child.Meta:
raise ormar.ModelDefinitionError(
f"Relation with related_name "
f"'{related_name}' "
f"leading to model "
f"{parent_model.get_name(lower=False)} "
f"cannot be used on model "
f"{child.get_name(lower=False)} "
f"because it's already used by model "
f"{fk_field.to.get_name(lower=False)}"
)
def reverse_field_not_already_registered(
child: Type["Model"], child_model_name: str, parent_model: Type["Model"]
) -> bool:
"""
Checks if child is already registered in parents pydantic fields.
:raises: ModelDefinitionError if related name is already used but lead to different
related model
:param child: related Model class
:type child: ormar.models.metaclass.ModelMetaclass
:param child_model_name: related_name of the child if provided
@ -27,10 +65,19 @@ def reverse_field_not_already_registered(
:return: result of the check
:rtype: bool
"""
return (
child_model_name not in parent_model.__fields__
and child.get_name() not in parent_model.__fields__
)
check_result = child_model_name not in parent_model.Meta.model_fields
check_result2 = child.get_name() not in parent_model.Meta.model_fields
if not check_result:
verify_related_name_dont_duplicate(
child=child, parent_model=parent_model, related_name=child_model_name
)
if not check_result2:
verify_related_name_dont_duplicate(
child=child, parent_model=parent_model, related_name=child.get_name()
)
return check_result and check_result2
def create_pydantic_field(

43
ormar/models/helpers/sqlalchemy.py
View File

@ -1,3 +1,4 @@
import copy
import logging
from typing import Dict, List, Optional, TYPE_CHECKING, Tuple, Type
@ -9,7 +10,7 @@ from ormar.models.helpers.models import validate_related_names_in_relations
from ormar.models.helpers.pydantic import create_pydantic_field
if TYPE_CHECKING: # pragma no cover
from ormar import Model
from ormar import Model, ModelMeta
def adjust_through_many_to_many_model(
@ -55,17 +56,24 @@ def create_and_append_m2m_fk(
:param model_field: field with ManyToMany relation
:type model_field: ManyToManyField field
"""
pk_alias = model.get_column_alias(model.Meta.pkname)
pk_column = next((col for col in model.Meta.columns if col.name == pk_alias), None)
if not pk_column: # pragma: no cover
raise ModelDefinitionError(
"ManyToMany relation cannot lead to field without pk"
)
column = sqlalchemy.Column(
model.get_name(),
model.Meta.table.columns.get(model.get_column_alias(model.Meta.pkname)).type,
pk_column.type,
sqlalchemy.schema.ForeignKey(
model.Meta.tablename + "." + model.get_column_alias(model.Meta.pkname),
model.Meta.tablename + "." + pk_alias,
ondelete="CASCADE",
onupdate="CASCADE",
),
)
model_field.through.Meta.columns.append(column)
model_field.through.Meta.table.append_column(column)
# breakpoint()
model_field.through.Meta.table.append_column(copy.deepcopy(column))
def check_pk_column_validity(
@ -122,8 +130,6 @@ def sqlalchemy_columns_from_model_fields(
:return: pkname, list of sqlalchemy columns
:rtype: Tuple[Optional[str], List[sqlalchemy.Column]]
"""
columns = []
pkname = None
if len(model_fields.keys()) == 0:
model_fields["id"] = Integer(name="id", primary_key=True)
logging.warning(
@ -131,6 +137,8 @@ def sqlalchemy_columns_from_model_fields(
"Integer primary key named `id` created."
)
validate_related_names_in_relations(model_fields, new_model)
columns = []
pkname = None
for field_name, field in model_fields.items():
if field.primary_key:
pkname = check_pk_column_validity(field_name, field, pkname)
@ -171,7 +179,7 @@ def populate_meta_tablename_columns_and_pk(
pkname: Optional[str]
if hasattr(new_model.Meta, "columns"):
columns = new_model.Meta.table.columns
columns = new_model.Meta.columns
pkname = new_model.Meta.pkname
else:
pkname, columns = sqlalchemy_columns_from_model_fields(
@ -186,23 +194,20 @@ def populate_meta_tablename_columns_and_pk(
return new_model
def populate_meta_sqlalchemy_table_if_required(
new_model: Type["Model"],
) -> Type["Model"]:
def populate_meta_sqlalchemy_table_if_required(meta: "ModelMeta") -> None:
"""
Constructs sqlalchemy table out of columns and parameters set on Meta class.
It populates name, metadata, columns and constraints.
:param new_model: class without sqlalchemy table constructed
:type new_model: Model class
:param meta: Meta class of the Model without sqlalchemy table constructed
:type meta: Model class Meta
:return: class with populated Meta.table
:rtype: Model class
"""
if not hasattr(new_model.Meta, "table"):
new_model.Meta.table = sqlalchemy.Table(
new_model.Meta.tablename,
new_model.Meta.metadata,
*new_model.Meta.columns,
*new_model.Meta.constraints,
if not hasattr(meta, "table"):
meta.table = sqlalchemy.Table(
meta.tablename,
meta.metadata,
*[copy.deepcopy(col) for col in meta.columns],
*meta.constraints,
)
return new_model

43
ormar/models/metaclass.py
View File

@ -354,10 +354,45 @@ def copy_data_from_parent_model( # noqa: CCR001
else attrs.get("__name__", "").lower() + "s"
)
for field_name, field in base_class.Meta.model_fields.items():
if issubclass(field, ForeignKeyField) and field.related_name:
copy_field = type(field.__name__, (field,), dict(field.__dict__))
if issubclass(field, ManyToManyField):
copy_field: Type[BaseField] = type( # type: ignore
field.__name__, (ManyToManyField, BaseField), dict(field.__dict__)
)
related_name = field.related_name + "_" + table_name
copy_field.related_name = related_name # type: ignore
copy_field.related_name = related_name
through_class = field.through
new_meta: ormar.ModelMeta = type(
"Meta",
(), # type: ignore
dict(through_class.Meta.__dict__),
)
new_meta.tablename += "_" + meta.tablename
# create new table with copied columns but remove foreign keys
# they will be populated later in expanding reverse relation
del new_meta.table
new_meta.columns = [
col for col in new_meta.columns if not col.foreign_keys
]
new_meta.model_fields = {
name: field
for name, field in new_meta.model_fields.items()
if not issubclass(field, ForeignKeyField)
}
populate_meta_sqlalchemy_table_if_required(new_meta)
copy_name = through_class.__name__ + attrs.get("__name__", "")
# TODO: when adding additional fields they need to be copied here
copy_through = type(copy_name, (ormar.Model,), {"Meta": new_meta})
copy_field.through = copy_through
parent_fields[field_name] = copy_field
elif issubclass(field, ForeignKeyField) and field.related_name:
copy_field = type( # type: ignore
field.__name__, (ForeignKeyField, BaseField), dict(field.__dict__)
)
related_name = field.related_name + "_" + table_name
copy_field.related_name = related_name
parent_fields[field_name] = copy_field
else:
parent_fields[field_name] = field
@ -500,7 +535,7 @@ class ModelMetaclass(pydantic.main.ModelMetaclass):
if not new_model.Meta.abstract:
new_model = populate_meta_tablename_columns_and_pk(name, new_model)
new_model = populate_meta_sqlalchemy_table_if_required(new_model)
populate_meta_sqlalchemy_table_if_required(new_model.Meta)
expand_reverse_relationships(new_model)
for field_name, field in new_model.Meta.model_fields.items():
register_relation_in_alias_manager(new_model, field, field_name)

105
tests/test_inheritance_concrete.py
View File

@ -1,6 +1,6 @@
# type: ignore
import datetime
from typing import Optional
from typing import List, Optional
import databases
import pytest
@ -122,6 +122,42 @@ class Bus(Car):
max_persons: int = ormar.Integer()
class PersonsCar(ormar.Model):
class Meta:
tablename = "cars_x_persons"
metadata = metadata
database = db
class Car2(ormar.Model):
class Meta:
abstract = True
metadata = metadata
database = db
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50)
owner: Person = ormar.ForeignKey(Person, related_name="owned")
co_owners: List[Person] = ormar.ManyToMany(
Person, through=PersonsCar, related_name="coowned"
)
created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
class Truck2(Car2):
class Meta:
tablename = "trucks2"
max_capacity: int = ormar.Integer()
class Bus2(Car2):
class Meta:
tablename = "buses2"
max_persons: int = ormar.Integer()
@pytest.fixture(autouse=True, scope="module")
def create_test_database():
metadata.create_all(engine)
@ -134,6 +170,17 @@ def test_init_of_abstract_model():
DateFieldsModel()
def test_duplicated_related_name_on_different_model():
with pytest.raises(ModelDefinitionError):
class Bus3(Car2): # pragma: no cover
class Meta:
tablename = "buses3"
owner: Person = ormar.ForeignKey(Person, related_name="buses")
max_persons: int = ormar.Integer()
def test_field_redefining_in_concrete_models():
class RedefinedField(DateFieldsModel):
class Meta(ormar.ModelMeta):
@ -310,3 +357,59 @@ async def test_inheritance_with_relation():
assert joe_check.coowned_trucks[0].created_date is None
assert joe_check.coowned_buses[0] == unicorn
assert joe_check.coowned_buses[0].created_date is None
@pytest.mark.asyncio
async def test_inheritance_with_multi_relation():
async with db:
async with db.transaction(force_rollback=True):
sam = await Person(name="Sam").save()
joe = await Person(name="Joe").save()
alex = await Person(name="Alex").save()
truck = await Truck2(
name="Shelby wanna be 2", max_capacity=1400, owner=sam
).save()
await truck.co_owners.add(joe)
await truck.co_owners.add(alex)
bus = await Bus2(name="Unicorn 2", max_persons=50, owner=sam).save()
await bus.co_owners.add(joe)
await bus.co_owners.add(alex)
shelby = await Truck2.objects.select_related(["owner", "co_owners"]).get()
assert shelby.name == "Shelby wanna be 2"
assert shelby.owner.name == "Sam"
assert shelby.co_owners[0].name == "Joe"
assert len(shelby.co_owners) == 2
assert shelby.max_capacity == 1400
unicorn = await Bus2.objects.select_related(["owner", "co_owners"]).get()
assert unicorn.name == "Unicorn 2"
assert unicorn.owner.name == "Sam"
assert unicorn.co_owners[0].name == "Joe"
assert len(unicorn.co_owners) == 2
assert unicorn.max_persons == 50
joe_check = await Person.objects.select_related(
["coowned_trucks2", "coowned_buses2"]
).get(name="Joe")
assert joe_check.pk == joe.pk
assert joe_check.coowned_trucks2[0] == shelby
assert joe_check.coowned_trucks2[0].created_date is not None
assert joe_check.coowned_buses2[0] == unicorn
assert joe_check.coowned_buses2[0].created_date is not None
joe_check = (
await Person.objects.exclude_fields(
{
"coowned_trucks2": {"created_date"},
"coowned_buses2": {"created_date"},
}
)
.prefetch_related(["coowned_trucks2", "coowned_buses2"])
.get(name="Joe")
)
assert joe_check.pk == joe.pk
assert joe_check.coowned_trucks2[0] == shelby
assert joe_check.coowned_trucks2[0].created_date is None
assert joe_check.coowned_buses2[0] == unicorn
assert joe_check.coowned_buses2[0].created_date is None