Prodream/ormar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

divide docs in sections, provide Meta attributes inheritance, add tests for fastapi use wih mixins and concrete

Browse Source
This commit is contained in:
collerek
2020-12-11 15:51:29 +01:00
parent bae2d8e1c8 ef4b687957
commit 4cca8fe63f
10 changed files with 328 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

119
docs/models/inheritance.md Normal file
View File

@ -0,0 +1,119 @@
# Inheritance
Out of various types of ORM models inheritance `ormar` currently supports two of them:
* **Mixins**
* **Concrete table inheritance** (with parents set to `abstract=True`)
## Types of inheritance
The short summary of different types of inheritance is:
* **Mixins [SUPPORTED]** - don't even subclass `ormar.Model`, just define fields that are later used on several different models (like `created_date` and `updated_date` on each model), only actual models create tables but those fields from mixins are added
* **Concrete table inheritance [SUPPORTED]** - means that parent is marked as abstract and each child has it's own table with columns from parent and own child columns, kind of similar to Mixins but parent also is a Model
* **Single table inheritance [NOT SUPPORTED]** - means that only one table is created with fields that are combination/sum of the parent and all children models but child models use only subset of column in db (all parent and own ones, skipping the other children ones)
* **Multi/ Joined table inheritance [NOT SUPPORTED]** - means that part of the columns is saved on parent model and part is saved on child model that are connected to each other by kind of one to one relation and under the hood you operate on two models at once
* **Proxy models [NOT SUPPORTED]** - means that only parent has an actual table, children just add methods, modify settings etc.
## Mixins
To use Mixins just define a class that is not inheriting from an `ormar.Model` but is defining `ormar.Fields` as class variables.
```python
# a mixin defines the fields but is a normal python class
class AuditMixin:
created_by: str = ormar.String(max_length=100)
updated_by: str = ormar.String(max_length=100, default="Sam")
class DateFieldsMixins:
created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
updated_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
# a models can inherit from one or more mixins
class Category(ormar.Model, DateFieldsMixins, AuditMixin):
class Meta(ormar.ModelMeta):
tablename = "categories"
metadata = metadata
database = db
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50, unique=True, index=True)
code: int = ormar.Integer()
```
!!!note
Note that Mixins are **not** models, so you still need to inherit from `ormar.Model` as well as define `Meta` class in the final model.
A Category class above will have four additional fields: `created_date`, `updated_date`, `created_by` and `updated_by`.
There will be only one table created for model Category, with `Category` class fields combined with all `Mixins` fields.
Note that Mixin in class name is optional but is a good python practice.
!!!warning
You cannot declare a field in a `Model` that is already defined in one of the `Mixins` you inherit from.
So in example above `Category` cannot declare it's own `created_date` as this filed will be inherited from `DateFieldsMixins`.
If you try to the `ModelDefinitionError` will be raised.
## Concrete table inheritance
In concept concrete table inheritance is very similar to Mixins, but uses actual `ormar.Models` as base classes.
!!!warning
Note that base classes have `abstract=True` set in `Meta` class, if you try to inherit from non abstract marked class `ModelDefinitionError` will be raised.
Since this abstract Model will never be initialized you can skip `metadata` and `database` in it's `Meta` definition.
But if you provide it - it will be inherited, that way you do not have to provide `metadata` and `databases` in concrete class
Note that you can always overwrite it in child/concrete class if you need to.
More over at least one of the classes in inheritance chain have to provide it - otherwise an error will be raised.
```python
# note that base classes have abstract=True
# since this model will never be initialized you can skip metadata and database
class AuditModel(ormar.Model):
class Meta:
abstract = True
created_by: str = ormar.String(max_length=100)
updated_by: str = ormar.String(max_length=100, default="Sam")
# but if you provide it it will be inherited
class DateFieldsModel(ormar.Model):
class Meta:
abstract = True
metadata = metadata
database = db
created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
updated_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
# that way you do not have to provide metadata and databases in concrete class
class Category(DateFieldsModel, AuditModel):
class Meta(ormar.ModelMeta):
tablename = "categories"
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50, unique=True, index=True)
code: int = ormar.Integer()
```
The list of inherited options/settings is as follows: `metadata`, `database` and `constraints`.
Also methods decorated with `@property_field` decorator will be inherited/recognized.
Of course apart from that all fields from base classes are combined and created in the concrete table of the final Model.
!!!warning
You cannot declare a field in a `Model` that is already defined in one of the `Mixins` you inherit from.
So in example above `Category` cannot declare it's own `created_date` as this filed will be inherited from `DateFieldsMixins`.
If you try to the `ModelDefinitionError` will be raised.

34
docs/relations/index.md
View File

@ -45,11 +45,43 @@ class Department(ormar.Model):
!!!tip
Reverse ForeignKey allows you to query the related models with [queryset-proxy][queryset-proxy].
It allows you to use `await department.courses.all()` to fetch data related only to specific department etc.
##ManyToMany
To define many-to-many relation use `ManyToMany` field.
```python hl_lines="25-26"
class Category(ormar.Model):
class Meta:
tablename = "categories"
database = database
metadata = metadata
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=40)
# note: you need to specify through model
class PostCategory(ormar.Model):
class Meta:
tablename = "posts_categories"
database = database
metadata = metadata
class Post(ormar.Model):
class Meta:
tablename = "posts"
database = database
metadata = metadata
id: int = ormar.Integer(primary_key=True)
title: str = ormar.String(max_length=200)
categories: Optional[Union[Category, List[Category]]] = ormar.ManyToMany(
Category, through=PostCategory
)
```
!!!tip
To read more about many-to-many relations visit [many-to-many][many-to-many] section
@ -58,6 +90,8 @@ To define many-to-many relation use `ManyToMany` field.
!!!tip
ManyToMany allows you to query the related models with [queryset-proxy][queryset-proxy].
It allows you to use `await post.categories.all()` but also `await category.posts.all()` to fetch data related only to specific post, category etc.
[foreign-keys]: ./foreign-key.md
[many-to-many]: ./many-to-many.md

7
docs/releases.md
View File

@ -1,3 +1,7 @@
# 0.7.3
* Fix for setting fetching related model with UUDI pk, which is a string in raw (fix [#71][#71])
# 0.7.2
* Fix for overwriting related models with pk only in `Model.update() with fields passed as parameters` (fix [#70][#70])
@ -188,4 +192,5 @@ Add queryset level methods
[#19]: https://github.com/collerek/ormar/issues/19
[#60]: https://github.com/collerek/ormar/issues/60
[#68]: https://github.com/collerek/ormar/issues/68
[#70]: https://github.com/collerek/ormar/issues/70
[#70]: https://github.com/collerek/ormar/issues/70
[#71]: https://github.com/collerek/ormar/issues/71

7
mkdocs.yml
View File

@ -5,6 +5,7 @@ nav:
- Installation: install.md
- Models:
- Definition: models/index.md
- Inheritance: models/inheritance.md
- Methods: models/methods.md
- Migrations: models/migrations.md
- Internals: models/internals.md
@ -25,9 +26,9 @@ nav:
- Release Notes: releases.md
repo_name: collerek/ormar
repo_url: https://github.com/collerek/ormar
#google_analytics:
# - UA-72514911-3
# - auto
google_analytics:
- UA-72514911-3
- auto
theme:
name: material
highlightjs: true

2
ormar/__init__.py
View File

@ -44,7 +44,7 @@ class UndefinedType: # pragma no cover
Undefined = UndefinedType()
__version__ = "0.7.2"
__version__ = "0.7.4"
__all__ = [
"Integer",
"BigInteger",

3
ormar/fields/foreign_key.py
View File

@ -1,3 +1,4 @@
import uuid
from typing import Any, List, Optional, TYPE_CHECKING, Type, Union
import sqlalchemy
@ -252,6 +253,8 @@ class ForeignKeyField(BaseField):
:return: (if needed) registered Model
:rtype: Model
"""
if cls.to.pk_type() == uuid.UUID and isinstance(value, str):
value = uuid.UUID(value)
if not isinstance(value, cls.to.pk_type()):
raise RelationshipInstanceError(
f"Relationship error - ForeignKey {cls.to.__name__} "

146
ormar/models/metaclass.py
View File

@ -1,6 +1,6 @@
import logging
import warnings
from typing import Any, Dict, List, Optional, Set, TYPE_CHECKING, Tuple, Type, Union
from typing import Any, Dict, List, Optional, Set, TYPE_CHECKING, Tuple, Type, Union, cast
import databases
import pydantic
@ -50,7 +50,7 @@ def register_relation_on_build(table_name: str, field: Type[ForeignKeyField]) ->
def register_many_to_many_relation_on_build(
table_name: str, field: Type[ManyToManyField]
table_name: str, field: Type[ManyToManyField]
) -> None:
alias_manager.add_relation_type(field.through.Meta.tablename, table_name)
alias_manager.add_relation_type(
@ -59,11 +59,11 @@ def register_many_to_many_relation_on_build(
def reverse_field_not_already_registered(
child: Type["Model"], child_model_name: str, parent_model: Type["Model"]
child: Type["Model"], child_model_name: str, parent_model: Type["Model"]
) -> bool:
return (
child_model_name not in parent_model.__fields__
and child.get_name() not in parent_model.__fields__
child_model_name not in parent_model.__fields__
and child.get_name() not in parent_model.__fields__
)
@ -74,7 +74,7 @@ def expand_reverse_relationships(model: Type["Model"]) -> None:
parent_model = model_field.to
child = model
if reverse_field_not_already_registered(
child, child_model_name, parent_model
child, child_model_name, parent_model
):
register_reverse_model_fields(
parent_model, child, child_model_name, model_field
@ -82,10 +82,10 @@ def expand_reverse_relationships(model: Type["Model"]) -> None:
def register_reverse_model_fields(
model: Type["Model"],
child: Type["Model"],
child_model_name: str,
model_field: Type["ForeignKeyField"],
model: Type["Model"],
child: Type["Model"],
child_model_name: str,
model_field: Type["ForeignKeyField"],
) -> None:
if issubclass(model_field, ManyToManyField):
model.Meta.model_fields[child_model_name] = ManyToMany(
@ -100,7 +100,7 @@ def register_reverse_model_fields(
def adjust_through_many_to_many_model(
model: Type["Model"], child: Type["Model"], model_field: Type[ManyToManyField]
model: Type["Model"], child: Type["Model"], model_field: Type[ManyToManyField]
) -> None:
model_field.through.Meta.model_fields[model.get_name()] = ForeignKey(
model, real_name=model.get_name(), ondelete="CASCADE"
@ -117,7 +117,7 @@ def adjust_through_many_to_many_model(
def create_pydantic_field(
field_name: str, model: Type["Model"], model_field: Type[ManyToManyField]
field_name: str, model: Type["Model"], model_field: Type[ManyToManyField]
) -> None:
model_field.through.__fields__[field_name] = ModelField(
name=field_name,
@ -139,7 +139,7 @@ def get_pydantic_field(field_name: str, model: Type["Model"]) -> "ModelField":
def create_and_append_m2m_fk(
model: Type["Model"], model_field: Type[ManyToManyField]
model: Type["Model"], model_field: Type[ManyToManyField]
) -> None:
column = sqlalchemy.Column(
model.get_name(),
@ -155,7 +155,7 @@ def create_and_append_m2m_fk(
def check_pk_column_validity(
field_name: str, field: BaseField, pkname: Optional[str]
field_name: str, field: BaseField, pkname: Optional[str]
) -> Optional[str]:
if pkname is not None:
raise ModelDefinitionError("Only one primary key column is allowed.")
@ -165,7 +165,7 @@ def check_pk_column_validity(
def sqlalchemy_columns_from_model_fields(
model_fields: Dict, table_name: str
model_fields: Dict, table_name: str
) -> Tuple[Optional[str], List[sqlalchemy.Column]]:
columns = []
pkname = None
@ -179,9 +179,9 @@ def sqlalchemy_columns_from_model_fields(
if field.primary_key:
pkname = check_pk_column_validity(field_name, field, pkname)
if (
not field.pydantic_only
and not field.virtual
and not issubclass(field, ManyToManyField)
not field.pydantic_only
and not field.virtual
and not issubclass(field, ManyToManyField)
):
columns.append(field.get_column(field.get_alias()))
register_relation_in_alias_manager(table_name, field)
@ -189,7 +189,7 @@ def sqlalchemy_columns_from_model_fields(
def register_relation_in_alias_manager(
table_name: str, field: Type[ForeignKeyField]
table_name: str, field: Type[ForeignKeyField]
) -> None:
if issubclass(field, ManyToManyField):
register_many_to_many_relation_on_build(table_name, field)
@ -198,7 +198,7 @@ def register_relation_in_alias_manager(
def populate_default_pydantic_field_value(
ormar_field: Type[BaseField], field_name: str, attrs: dict
ormar_field: Type[BaseField], field_name: str, attrs: dict
) -> dict:
curr_def_value = attrs.get(field_name, ormar.Undefined)
if lenient_issubclass(curr_def_value, ormar.fields.BaseField):
@ -243,7 +243,7 @@ def extract_annotations_and_default_vals(attrs: dict) -> Tuple[Dict, Dict]:
def populate_meta_tablename_columns_and_pk(
name: str, new_model: Type["Model"]
name: str, new_model: Type["Model"]
) -> Type["Model"]:
tablename = name.lower() + "s"
new_model.Meta.tablename = (
@ -269,7 +269,7 @@ def populate_meta_tablename_columns_and_pk(
def populate_meta_sqlalchemy_table_if_required(
new_model: Type["Model"],
new_model: Type["Model"],
) -> Type["Model"]:
"""
Constructs sqlalchemy table out of columns and parameters set on Meta class.
@ -360,7 +360,7 @@ def populate_choices_validators(model: Type["Model"]) -> None: # noqa CCR001
def populate_default_options_values(
new_model: Type["Model"], model_fields: Dict
new_model: Type["Model"], model_fields: Dict
) -> None:
"""
Sets all optional Meta values to it's defaults
@ -479,11 +479,11 @@ def get_potential_fields(attrs: Dict) -> Dict:
def check_conflicting_fields(
new_fields: Set,
attrs: Dict,
base_class: type,
curr_class: type,
previous_fields: Set = None,
new_fields: Set,
attrs: Dict,
base_class: type,
curr_class: type,
previous_fields: Set = None,
) -> None:
"""
You cannot redefine fields with same names in inherited classes.
@ -513,11 +513,11 @@ def check_conflicting_fields(
def update_attrs_and_fields(
attrs: Dict,
new_attrs: Dict,
model_fields: Dict,
new_model_fields: Dict,
new_fields: Set,
attrs: Dict,
new_attrs: Dict,
model_fields: Dict,
new_model_fields: Dict,
new_fields: Set,
) -> None:
"""
Updates __annotations__, values of model fields (so pydantic FieldInfos)
@ -540,13 +540,42 @@ def update_attrs_and_fields(
model_fields.update(new_model_fields)
def update_attrs_from_base_meta(
base_class: "Model",
attrs: Dict, ) -> None:
"""
Updates Meta parameters in child from parent if needed.
:param base_class: one of the parent classes
:type base_class: Model or model parent class
:param attrs: new namespace for class being constructed
:type attrs: Dict
"""
params_to_update = ["metadata", "database", "constraints", "property_fields"]
for param in params_to_update:
if hasattr(base_class.Meta, param):
if hasattr(attrs["Meta"], param):
curr_value = getattr(attrs["Meta"], param)
if isinstance(curr_value, list):
curr_value.extend(getattr(base_class.Meta, param))
elif isinstance(curr_value, dict): # pragma: no cover
curr_value.update(getattr(base_class.Meta, param))
elif isinstance(curr_value, Set):
curr_value.union(getattr(base_class.Meta, param))
else:
# overwrite with child value if both set and its param / object
setattr(attrs["Meta"], param, getattr(base_class.Meta, param)) # pragma: no cover
else:
setattr(attrs["Meta"], param, getattr(base_class.Meta, param))
def extract_mixin_fields_from_dict(
base_class: type,
curr_class: type,
attrs: Dict,
model_fields: Dict[
str, Union[Type[BaseField], Type[ForeignKeyField], Type[ManyToManyField]]
],
base_class: type,
curr_class: type,
attrs: Dict,
model_fields: Dict[
str, Union[Type[BaseField], Type[ForeignKeyField], Type[ManyToManyField]]
],
) -> Tuple[Dict, Dict]:
"""
Extracts fields from base classes if they have valid oramr fields.
@ -572,21 +601,23 @@ def extract_mixin_fields_from_dict(
:rtype: Tuple[Dict, Dict]
"""
if hasattr(base_class, "Meta"):
new_fields = set(base_class.Meta.model_fields.keys()) # type: ignore
previous_fields = set({k for k, v in attrs.items() if isinstance(v, FieldInfo)})
check_conflicting_fields(
new_fields=new_fields,
attrs=attrs,
base_class=base_class,
curr_class=curr_class,
previous_fields=previous_fields,
)
if previous_fields and not base_class.Meta.abstract: # type: ignore
raise ModelDefinitionError(
f"{curr_class.__name__} cannot inherit "
f"from non abstract class {base_class.__name__}"
if attrs.get("Meta"):
new_fields = set(base_class.Meta.model_fields.keys()) # type: ignore
previous_fields = set({k for k, v in attrs.items() if isinstance(v, FieldInfo)})
check_conflicting_fields(
new_fields=new_fields,
attrs=attrs,
base_class=base_class,
curr_class=curr_class,
previous_fields=previous_fields,
)
model_fields.update(base_class.Meta.model_fields) # type: ignore
if previous_fields and not base_class.Meta.abstract: # type: ignore
raise ModelDefinitionError(
f"{curr_class.__name__} cannot inherit "
f"from non abstract class {base_class.__name__}"
)
update_attrs_from_base_meta(base_class=base_class, attrs=attrs) # type: ignore
model_fields.update(base_class.Meta.model_fields)
return attrs, model_fields
key = "__annotations__"
@ -644,17 +675,14 @@ def extract_mixin_fields_from_dict(
class ModelMetaclass(pydantic.main.ModelMetaclass):
def __new__( # type: ignore
mcs: "ModelMetaclass", name: str, bases: Any, attrs: dict
mcs: "ModelMetaclass", name: str, bases: Any, attrs: dict
) -> "ModelMetaclass":
attrs["Config"] = get_pydantic_base_orm_config()
attrs["__name__"] = name
attrs, model_fields = extract_annotations_and_default_vals(attrs)
for base in reversed(bases):
for ind, base in enumerate(reversed(bases)):
attrs, model_fields = extract_mixin_fields_from_dict(
base_class=base,
curr_class=mcs,
attrs=attrs,
model_fields=model_fields
base_class=base, curr_class=mcs, attrs=attrs, model_fields=model_fields
)
new_model = super().__new__( # type: ignore
mcs, name, bases, attrs

2
ormar/models/newbasemodel.py
View File

@ -72,6 +72,8 @@ class NewBaseModel(
# noinspection PyMissingConstructor
def __init__(self, *args: Any, **kwargs: Any) -> None: # type: ignore
if self.Meta.abstract:
raise ModelError(f"You cannot initialize abstract model {self.get_name()}")
object.__setattr__(self, "_orm_id", uuid.uuid4().hex)
object.__setattr__(self, "_orm_saved", False)
object.__setattr__(self, "_pk_column", None)

14
tests/test_inheritance_concrete.py
View File

@ -9,6 +9,7 @@ from sqlalchemy import create_engine
import ormar
from ormar import ModelDefinitionError
from ormar.exceptions import ModelError
from tests.settings import DATABASE_URL
metadata = sa.MetaData()
@ -38,6 +39,8 @@ class DateFieldsModelNoSubclass(ormar.Model):
class DateFieldsModel(ormar.Model):
class Meta:
abstract = True
metadata = metadata
database = db
created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
updated_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
@ -46,8 +49,6 @@ class DateFieldsModel(ormar.Model):
class Category(DateFieldsModel, AuditModel):
class Meta(ormar.ModelMeta):
tablename = "categories"
metadata = metadata
database = db
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50, unique=True, index=True)
@ -56,9 +57,7 @@ class Category(DateFieldsModel, AuditModel):
class Subject(DateFieldsModel):
class Meta(ormar.ModelMeta):
tablename = "subjects"
metadata = metadata
database = db
pass
id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=50, unique=True, index=True)
@ -72,6 +71,11 @@ def create_test_database():
metadata.drop_all(engine)
def test_init_of_abstract_model():
with pytest.raises(ModelError):
DateFieldsModel()
def test_field_redefining_raises_error():
with pytest.raises(ModelDefinitionError):
class WrongField(DateFieldsModel): # pragma: no cover

63
tests/test_uuid_fks.py Normal file
View File

@ -0,0 +1,63 @@
import uuid
import databases
import pytest
import sqlalchemy
from sqlalchemy import create_engine
import ormar
from tests.settings import DATABASE_URL
metadata = sqlalchemy.MetaData()
db = databases.Database(DATABASE_URL)
class User(ormar.Model):
class Meta:
tablename = "user"
metadata = metadata
database = db
id: uuid.UUID = ormar.UUID(
primary_key=True, default=uuid.uuid4, uuid_format="string"
)
username = ormar.String(index=True, unique=True, null=False, max_length=255)
email = ormar.String(index=True, unique=True, nullable=False, max_length=255)
hashed_password = ormar.String(null=False, max_length=255)
is_active = ormar.Boolean(default=True, nullable=False)
is_superuser = ormar.Boolean(default=False, nullable=False)
class Token(ormar.Model):
class Meta:
tablename = "token"
metadata = metadata
database = db
id = ormar.Integer(primary_key=True)
text = ormar.String(max_length=4, unique=True)
user = ormar.ForeignKey(User, related_name="tokens")
created_at = ormar.DateTime(server_default=sqlalchemy.func.now())
@pytest.fixture(autouse=True, scope="module")
def create_test_database():
engine = create_engine(DATABASE_URL)
metadata.create_all(engine)
yield
metadata.drop_all(engine)
@pytest.mark.asyncio
async def test_uuid_fk():
async with db:
async with db.transaction(force_rollback=True):
user = await User.objects.create(
username="User1",
email="email@example.com",
hashed_password="^$EDACVS(&A&Y@2131aa",
is_active=True,
is_superuser=False,
)
await Token.objects.create(text="AAAA", user=user)
await Token.objects.order_by("-created_at").all()