introduce docs -> models section mostly finished

This commit is contained in:
collerek
2020-08-12 16:24:45 +02:00
parent dd20fd9f01
commit 24eb0b30e7
23 changed files with 475 additions and 85 deletions

BIN
.coverage

Binary file not shown.

0
docs/fastapi.md Normal file
View File

0
docs/fields.md Normal file
View File

17
docs/index.md Normal file
View File

@ -0,0 +1,17 @@
# Welcome to MkDocs
For full documentation visit [mkdocs.org](https://www.mkdocs.org).
## Commands
* `mkdocs new [dir-name]` - Create a new project.
* `mkdocs serve` - Start the live-reloading docs server.
* `mkdocs build` - Build the documentation site.
* `mkdocs -h` - Print help message and exit.
## Project layout
mkdocs.yml # The configuration file.
docs/
index.md # The documentation homepage.
... # Other markdown pages, images and other files.

171
docs/models.md Normal file
View File

@ -0,0 +1,171 @@
# Models
## Defining models
By defining an orm Model you get corresponding **Pydantic model** as well as **Sqlalchemy table** for free.
They are being managed in the background and you do not have to create them on your own.
### Model Class
To build an ORM model you simply need to inherit a `orm.Model` class.
```Python hl_lines="10"
--8<-- "../docs_src/models/docs001.py"
```
### Defining Fields
Next assign one or more of the [Fields][fields] as a class level variables.
Each table **has to** have a primary key column, which you specify by setting `primary_key=True` on selected field.
Only one primary key column is allowed.
```Python hl_lines="14 15 16"
--8<-- "../docs_src/models/docs001.py"
```
!!! warning
Not assigning `primary_key` column or assigning more than one column per `Model` will raise `ModelDefinitionError`
exception.
By default if you assign primary key to `Integer` field, the `autoincrement` option is set to true.
You can disable by passing `autoincremant=False`.
```Python
id = orm.Integer(primary_key=True, autoincrement=False)
```
Names of the fields will be used for both the underlying `pydantic` model and `sqlalchemy` table.
### Dependencies
Since orm depends on [`databases`][databases] and [`sqlalchemy-core`][sqlalchemy-core] for database connection
and table creation you need to assign each `Model` with two special parameters.
#### Databases
One is `Database` instance created with your database url in [sqlalchemy connection string][sqlalchemy connection string] format.
Created instance needs to be passed to every `Model` with `__database__` parameter.
```Python hl_lines="1 6 11"
--8<-- "../docs_src/models/docs001.py"
```
!!! tip
You need to create the `Database` instance **only once** and use it for all models.
You can create several ones if you want to use multiple databases.
#### Sqlalchemy
Second dependency is sqlalchemy `MetaData` instance.
Created instance needs to be passed to every `Model` with `__metadata__` parameter.
```Python hl_lines="2 7 12"
--8<-- "../docs_src/models/docs001.py"
```
!!! tip
You need to create the `MetaData` instance **only once** and use it for all models.
You can create several ones if you want to use multiple databases.
### Table Names
By default table name is created from Model class name as lowercase name plus 's'.
You can overwrite this parameter by providing `__tablename__` argument.
```Python hl_lines="11 12 13"
--8<-- "../docs_src/models/docs002.py"
```
## Initialization
There are two ways to create and persist the `Model` instance in the database.
!!!tip
Use `ipython` to try this from the console, since it supports `await`.
If you plan to modify the instance in the later execution of your program you can initiate your `Model` as a normal class and later await a `save()` call.
```Python hl_lines="19 20"
--8<-- "../docs_src/models/docs007.py"
```
If you want to initiate your `Model` and at the same time save in in the database use a QuerySet's method `create()`.
Each model has a `QuerySet` initialised as `objects` parameter
```Python hl_lines="22"
--8<-- "../docs_src/models/docs007.py"
```
!!!info
To read more about `QuerySets` and available methods visit [queries][queries]
## Attributes Delegation
Each call to `Model` fields parameter under the hood is delegated to either the `pydantic` model
or other related `Model` in case of relations.
The fields and relations are not stored on the `Model` itself
```Python hl_lines="31 32 33 34 35 36 37 38 39 40 41"
--8<-- "../docs_src/models/docs006.py"
```
!!! warning
In example above model instances are created but not persisted that's why `id` of `department` is None!
!!!info
To read more about `ForeignKeys` and `Model` relations visit [relations][relations]
## Internals
Apart from special parameters defined in the `Model` during definition (tablename, metadata etc.) the `Model` provides you with useful internals.
### Pydantic Model
To access auto created pydantic model you can use `Model.__pydantic_model__` parameter
For example to list model fields you can:
```Python hl_lines="18"
--8<-- "../docs_src/models/docs003.py"
```
!!!tip
Note how the primary key `id` field is optional as `Integer` primary key by default has `autoincrement` set to `True`.
!!!info
For more options visit official [pydantic][pydantic] documentation.
### Sqlalchemy Table
To access auto created sqlalchemy table you can use `Model.__table__` parameter
For example to list table columns you can:
```Python hl_lines="18"
--8<-- "../docs_src/models/docs004.py"
```
!!!tip
You can access table primary key name by `Course.__pkname__`
!!!info
For more options visit official [sqlalchemy-metadata][sqlalchemy-metadata] documentation.
### Fields Definition
To access orm `Fields` you can use `Model.__model_fields__` parameter
For example to list table model fields you can:
```Python hl_lines="18"
--8<-- "../docs_src/models/docs005.py"
```
[fields]: ./fields.md
[relations]: ./relations.md
[queries]: ./queries.md
[pydantic]: https://pydantic-docs.helpmanual.io/
[sqlalchemy-core]: https://docs.sqlalchemy.org/en/latest/core/
[sqlalchemy-metadata]: https://docs.sqlalchemy.org/en/13/core/metadata.html
[databases]: https://github.com/encode/databases
[sqlalchemy connection string]: https://docs.sqlalchemy.org/en/13/core/engines.html#database-urls

0
docs/pydantic.md Normal file
View File

0
docs/queries.md Normal file
View File

0
docs/relations.md Normal file
View File

View File

@ -0,0 +1,16 @@
import databases
import sqlalchemy
import orm
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
completed = orm.Boolean(default=False)

View File

@ -0,0 +1,19 @@
import databases
import sqlalchemy
import orm
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(orm.Model):
# if you omit this parameter it will be created automatically
# as class.__name__.lower()+'s' -> "courses" in this example
__tablename__ = "my_courses"
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
completed = orm.Boolean(default=False)

View File

@ -0,0 +1,33 @@
import databases
import sqlalchemy
import orm
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
completed = orm.Boolean(default=False)
print(Course.__pydantic_model__.__fields__)
"""
Will produce:
{'completed': ModelField(name='completed',
type=bool,
required=False,
default=False),
'id': ModelField(name='id',
type=Optional[int],
required=False,
default=None),
'name': ModelField(name='name',
type=Optional[str],
required=False,
default=None)}
"""

View File

@ -0,0 +1,22 @@
import databases
import sqlalchemy
import orm
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
completed = orm.Boolean(default=False)
print(Course.__table__.columns)
"""
Will produce:
['courses.id', 'courses.name', 'courses.completed']
"""

View File

@ -0,0 +1,51 @@
import databases
import sqlalchemy
import orm
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
completed = orm.Boolean(default=False)
print(Course.__model_fields__)
"""
Will produce:
{
'id': {'name': 'id',
'primary_key': True,
'autoincrement': True,
'nullable': False,
'default': None,
'server_default': None,
'index': None,
'unique': None,
'pydantic_only': False},
'name': {'name': 'name',
'primary_key': False,
'autoincrement': False,
'nullable': True,
'default': None,
'server_default': None,
'index': None,
'unique': None,
'pydantic_only': False,
'length': 100},
'completed': {'name': 'completed',
'primary_key': False,
'autoincrement': False,
'nullable': True,
'default': False,
'server_default': None,
'index': None,
'unique': None,
'pydantic_only': False}
}
"""

View File

@ -0,0 +1,41 @@
import databases
import sqlalchemy
import orm
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Department(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
class Course(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
completed = orm.Boolean(default=False)
department = orm.ForeignKey(Department)
department = Department(name="Science")
course = Course(name="Math", completed=False, department=department)
print('name' in course.__dict__)
# False <- property name is not stored on Course instance
print(course.name)
# Math <- value returned from underlying pydantic model
print('department' in course.__dict__)
# False <- related model is not stored on Course instance
print(course.department)
# Department(id=None, name='Science') <- Department model
# returned from RelationshipManager
print(course.department.name)
# Science

View File

@ -0,0 +1,22 @@
import databases
import sqlalchemy
import orm
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Course(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
completed = orm.Boolean(default=False)
course = Course(name="Painting for dummies", completed=False)
await course.save()
await Course.objects.create(name="Painting for dummies", completed=False)

29
mkdocs.yml Normal file
View File

@ -0,0 +1,29 @@
site_name: Async ORM
nav:
- Home: index.md
- Models: models.md
- Fields: fields.md
- Relations: relations.md
- Queries: queries.md
- Pydantic models: pydantic.md
- Use with Fastapi: fastapi.md
theme:
name: material
highlightjs: true
hljs_languages:
- python
palette:
primary: indigo
markdown_extensions:
- admonition
- pymdownx.superfences
- pymdownx.snippets:
base_path: docs
- pymdownx.inlinehilite
- pymdownx.highlight:
linenums: true
extra_javascript:
- https://cdnjs.cloudflare.com/ajax/libs/highlight.js/10.1.1/highlight.min.js
- javascripts/config.js
extra_css:
- https://cdnjs.cloudflare.com/ajax/libs/highlight.js/10.1.1/styles/default.min.css

View File

@ -11,18 +11,8 @@ if TYPE_CHECKING: # pragma no cover
class BaseField:
__type__ = None
def __init__(self, *args: Any, **kwargs: Any) -> None:
name = kwargs.pop("name", None)
args = list(args)
if args:
if isinstance(args[0], str):
if name is not None:
raise ModelDefinitionError(
"Column name cannot be passed positionally and as a keyword."
)
name = args.pop(0)
self.name = name
def __init__(self, **kwargs: Any) -> None:
self.name = None
self._populate_from_kwargs(kwargs)
def _populate_from_kwargs(self, kwargs: Dict) -> None:
@ -64,7 +54,7 @@ class BaseField:
return False
def get_column(self, name: str = None) -> sqlalchemy.Column:
self.name = self.name or name
self.name = name
constraints = self.get_constraints()
return sqlalchemy.Column(
self.name,
@ -87,3 +77,6 @@ class BaseField:
def expand_relationship(self, value: Any, child: "Model") -> Any:
return value
def __repr__(self): # pragma no cover
return str(self.__dict__)

View File

@ -14,8 +14,8 @@ class RequiredParams:
old_init = model_field_class.__init__
model_field_class._old_init = old_init
def __init__(instance: "BaseField", *args: Any, **kwargs: Any) -> None:
super(instance.__class__, instance).__init__(*args, **kwargs)
def __init__(instance: "BaseField", **kwargs: Any) -> None:
super(instance.__class__, instance).__init__(**kwargs)
for arg in self._required:
if arg not in kwargs:
raise ModelDefinitionError(

View File

@ -75,6 +75,8 @@ def sqlalchemy_columns_from_model_fields(
}
for field_name, field in model_fields.items():
if field.primary_key:
if pkname is not None:
raise ModelDefinitionError("Only one primary key column is allowed.")
pkname = field_name
if not field.pydantic_only:
columns.append(field.get_column(field_name))
@ -100,7 +102,8 @@ class ModelMetaclass(type):
if attrs.get("__abstract__"):
return new_model
tablename = attrs["__tablename__"]
tablename = attrs.get("__tablename__", name.lower() + "s")
attrs["__tablename__"] = tablename
metadata = attrs["__metadata__"]
# sqlalchemy table creation

View File

@ -144,7 +144,9 @@ class QueryClause:
) -> Tuple[str, bool]:
has_escaped_character = False
if op in ["contains", "icontains"]:
if op not in ["contains", "icontains"]:
return value, has_escaped_character
if isinstance(value, orm.Model):
raise QueryDefinitionError(
"You cannot use contains and icontains with instance of the Model"

View File

@ -52,8 +52,7 @@ class Query:
if (
not self.model_cls.__model_fields__[key].nullable
and isinstance(
self.model_cls.__model_fields__[key],
orm.fields.foreign_key.ForeignKey,
self.model_cls.__model_fields__[key], orm.fields.ForeignKey,
)
and key not in self._select_related
):

View File

@ -2,7 +2,7 @@ import pprint
import string
import uuid
from random import choices
from typing import Dict, List, TYPE_CHECKING, Union
from typing import List, TYPE_CHECKING, Union
from weakref import proxy
from orm import ForeignKey
@ -15,38 +15,20 @@ def get_table_alias() -> str:
return "".join(choices(string.ascii_uppercase, k=2)) + uuid.uuid4().hex[:4]
def get_relation_config(
relation_type: str, table_name: str, field: ForeignKey
) -> Dict[str, str]:
alias = get_table_alias()
config = {
"type": relation_type,
"table_alias": alias,
"source_table": table_name
if relation_type == "primary"
else field.to.__tablename__,
"target_table": field.to.__tablename__
if relation_type == "primary"
else table_name,
}
return config
class RelationshipManager:
def __init__(self) -> None:
self._relations = dict()
self._aliases = dict()
def add_relation_type(
self, relations_key: str, reverse_key: str, field: ForeignKey, table_name: str
) -> None:
if relations_key not in self._relations:
self._relations[relations_key] = get_relation_config(
"primary", table_name, field
)
self._relations[relations_key] = {"type": "primary"}
self._aliases[f"{table_name}_{field.to.__tablename__}"] = get_table_alias()
if reverse_key not in self._relations:
self._relations[reverse_key] = get_relation_config(
"reverse", table_name, field
)
self._relations[reverse_key] = {"type": "reverse"}
self._aliases[f"{field.to.__tablename__}_{table_name}"] = get_table_alias()
def deregister(self, model: "FakePydantic") -> None:
for rel_type in self._relations.keys():
@ -57,10 +39,11 @@ class RelationshipManager:
def add_relation(
self, parent: "FakePydantic", child: "FakePydantic", virtual: bool = False,
) -> None:
parent_id = parent._orm_id
child_id = child._orm_id
parent_name = parent.get_name()
child_name = child.get_name()
parent_id, child_id = parent._orm_id, child._orm_id
parent_name, child_name = (
parent.get_name(title=True),
child.get_name(title=True),
)
if virtual:
child_name, parent_name = parent_name, child_name
child_id, parent_id = parent_id, child_id
@ -68,11 +51,11 @@ class RelationshipManager:
else:
child = proxy(child)
parent_relation_name = parent_name.lower().title() + "_" + child_name + "s"
parent_relation_name = parent_name + "_" + child_name.lower() + "s"
parents_list = self._relations[parent_relation_name].setdefault(parent_id, [])
self.append_related_model(parents_list, child)
child_relation_name = child_name.lower().title() + "_" + parent_name
child_relation_name = child_name + "_" + parent_name.lower()
children_list = self._relations[child_relation_name].setdefault(child_id, [])
self.append_related_model(children_list, parent)
@ -102,13 +85,7 @@ class RelationshipManager:
return self._relations[relations_key][instance._orm_id]
def resolve_relation_join(self, from_table: str, to_table: str) -> str:
for relation_name, relation in self._relations.items():
if (
relation["source_table"] == from_table
and relation["target_table"] == to_table
):
return self._relations[relation_name]["table_alias"]
return ""
return self._aliases.get(f"{from_table}_{to_table}", "")
def __str__(self) -> str: # pragma no cover
return pprint.pformat(self._relations, indent=4, width=1)

View File

@ -43,8 +43,8 @@ fields_to_check = [
class ExampleModel2(Model):
__tablename__ = "example2"
__metadata__ = metadata
test = fields.Integer(name="test12", primary_key=True)
test_string = fields.String("test_string2", length=250)
test = fields.Integer(primary_key=True)
test_string = fields.String(length=250)
@pytest.fixture()
@ -93,49 +93,44 @@ def test_sqlalchemy_table_is_created(example):
assert all([field in example.__table__.columns for field in fields_to_check])
def test_double_column_name_in_model_definition():
with pytest.raises(ModelDefinitionError):
class ExampleModel2(Model):
__tablename__ = "example3"
__metadata__ = metadata
test_string = fields.String("test_string2", name="test_string2", length=250)
def test_no_pk_in_model_definition():
with pytest.raises(ModelDefinitionError):
class ExampleModel2(Model):
__tablename__ = "example3"
__metadata__ = metadata
test_string = fields.String(name="test_string2", length=250)
test_string = fields.String(length=250)
def test_two_pks_in_model_definition():
with pytest.raises(ModelDefinitionError):
class ExampleModel2(Model):
__tablename__ = "example3"
__metadata__ = metadata
id = fields.Integer(primary_key=True)
test_string = fields.String(length=250, primary_key=True)
def test_setting_pk_column_as_pydantic_only_in_model_definition():
with pytest.raises(ModelDefinitionError):
class ExampleModel2(Model):
__tablename__ = "example4"
__metadata__ = metadata
test = fields.Integer(name="test12", primary_key=True, pydantic_only=True)
test = fields.Integer(primary_key=True, pydantic_only=True)
def test_decimal_error_in_model_definition():
with pytest.raises(ModelDefinitionError):
class ExampleModel2(Model):
__tablename__ = "example4"
__metadata__ = metadata
test = fields.Decimal(name="test12", primary_key=True)
test = fields.Decimal(primary_key=True)
def test_string_error_in_model_definition():
with pytest.raises(ModelDefinitionError):
class ExampleModel2(Model):
__tablename__ = "example4"
__metadata__ = metadata
test = fields.String(name="test12", primary_key=True)
test = fields.String(primary_key=True)
def test_json_conversion_in_model():