Prodream/ormar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

finish fields docs intial ver,add test for related name, fix child_name(s) in reverse relations

Browse Source
This commit is contained in:
collerek
2020-08-13 12:54:42 +02:00
parent 24eb0b30e7
commit 8c7051b07e
11 changed files with 572 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
.coverage
View File

Binary file not shown.

206
docs/fields.md
View File

@ -0,0 +1,206 @@
# Fields
There are 11 basic model field types and a special `ForeignKey` field to establish relationships between models.
Each of the `Fields` has assigned both `sqlalchemy` column class and python type that is used to create `pydantic` model.
## Common Parameters
All `Field` types have a set of common parameters.
### primary_key
`primary_key`: `bool` = `False` -> by default False.
Sets the primary key column on a table, foreign keys always refer to the pk of the `Model`.
Used in sql only.
### autoincrement
`autoincrement`: `bool` = `primary_key and type == int` -> defaults to True if column is a primary key and of type Integer, otherwise False.
Can be only used with int fields.
If a field has autoincrement it becomes optional.
Used only in sql.
### nullable
`nullable`: `bool` = `not primary_key` -> defaults to False for primary key column, and True for all other.
Specifies if field is optional or required, used both with sql and pydantic.
!!!note
By default all `ForeignKeys` are also nullable, meaning the related `Model` is not required.
If you change the `ForeignKey` column to `nullable`, it not only becomes required, it changes also the way in which data is loaded in queries.
If you select `Model` without explicitly adding related `Model` assigned by not nullable `ForeignKey`, the `Model` is still gona be appended automatically, see example below.
```Python hl_lines="24 32 33 34 35 37 38 39 40 41"
--8<-- "../docs_src/fields/docs003.py"
```
!!!info
If you want to know more about how you can preload related models during queries and how the relations work read the [queries][queries] and [relations][relations] sections.
### default
`default`: `Any` = `None` -> defaults to None.
A default value used if no other value is passed.
In sql invoked on an insert, used during pydantic model definition.
If the field has a default value it becomes optional.
You can pass a static value or a Callable (function etc.)
Used both in sql and pydantic.
### server default
`server_default`: `Any` = `None` -> defaults to None.
A default value used if no other value is passed.
In sql invoked on the server side so you can pass i.e. sql function (like now() wrapped in sqlalchemy text() clause).
If the field has a server_default value it becomes optional.
You can pass a static value or a Callable (function etc.)
Used in sql only.
### index
`index`: `bool` = `False` -> by default False,
Sets the index on a table's column.
Used in sql only.
### unique
`unique`: `bool` = `False`
Sets the unique constraint on a table's column.
Used in sql only.
## Fields Types
### String
`String(length)` has a required `length` parameter.
* Sqlalchemy column: `sqlalchemy.String`
* Type (used for pydantic): `str`
### Text
`Text()` has no required parameters.
* Sqlalchemy column: `sqlalchemy.Text`
* Type (used for pydantic): `str`
### Boolean
`Boolean()` has no required parameters.
* Sqlalchemy column: `sqlalchemy.Boolean`
* Type (used for pydantic): `bool`
### Integer
`Integer()` has no required parameters.
* Sqlalchemy column: `sqlalchemy.Integer`
* Type (used for pydantic): `int`
### BigInteger
`BigInteger()` has no required parameters.
* Sqlalchemy column: `sqlalchemy.BigInteger`
* Type (used for pydantic): `int`
### Float
`Float()` has no required parameters.
* Sqlalchemy column: `sqlalchemy.Float`
* Type (used for pydantic): `float`
### Decimal
`Decimal(lenght, precision)` has required `length` and `precision` parameters.
* Sqlalchemy column: `sqlalchemy.DECIMAL`
* Type (used for pydantic): `decimal.Decimal`
### Date
`Date()` has no required parameters.
* Sqlalchemy column: `sqlalchemy.Date`
* Type (used for pydantic): `datetime.date`
### Time
`Time()` has no required parameters.
* Sqlalchemy column: `sqlalchemy.Time`
* Type (used for pydantic): `datetime.time`
### DateTime
`DateTime()` has no required parameters.
* Sqlalchemy column: `sqlalchemy.DateTime`
* Type (used for pydantic): `datetime.datetime`
### JSON
`JSON()` has no required parameters.
* Sqlalchemy column: `sqlalchemy.JSON`
* Type (used for pydantic): `pydantic.Json`
### ForeignKey
`ForeignKey(to, related_name=None)` has required parameters `to` that takes target `Model` class.
Sqlalchemy column and Type are automatically taken from target `Model`.
* Sqlalchemy column: class of a target `Model` primary key column
* Type (used for pydantic): type of a target `Model` primary key column
`ForeignKey` fields are automatically registering reverse side of the relation.
By default it's child (source) `Model` name + s, like courses in snippet below:
```Python hl_lines="25 31"
--8<-- "../docs_src/fields/docs001.py"
```
But you can overwrite this name by providing `related_name` parameter like below:
```Python hl_lines="25 30"
--8<-- "../docs_src/fields/docs002.py"
```
!!!tip
Since related models are coming from Relationship Manager the reverse relation on access returns list of `wekref.proxy` to avoid circular references.
!!!info
All relations are stored in lists, but when you access parent `Model` the ORM is unpacking the value for you.
Read more in [relations][relations].
[relations]: ./relations.md
[queries]: ./queries.md

214
docs/index.md
View File

@ -1,17 +1,207 @@
# Welcome to MkDocs
# Async-ORM
For full documentation visit [mkdocs.org](https://www.mkdocs.org).
<p>
<a href="https://travis-ci.com/collerek/async-orm">
<img src="https://travis-ci.com/collerek/async-orm.svg?branch=master" alt="Build Status">
</a>
<a href="https://codecov.io/gh/collerek/async-orm">
<img src="https://codecov.io/gh/collerek/async-orm/branch/master/graph/badge.svg" alt="Coverage">
</a>
<a href="https://www.codefactor.io/repository/github/collerek/async-orm">
<img src="https://www.codefactor.io/repository/github/collerek/async-orm/badge" alt="CodeFactor" />
</a>
<a href="https://app.codacy.com/manual/collerek/async-orm?utm_source=github.com&utm_medium=referral&utm_content=collerek/async-orm&utm_campaign=Badge_Grade_Dashboard">
<img src="https://api.codacy.com/project/badge/Grade/62568734f70f49cd8ea7a1a0b2d0c107" alt="Codacy" />
</a>
</p>
## Commands
The `async-orm` package is an async ORM for Python, with support for Postgres,
MySQL, and SQLite. ORM is built with:
* `mkdocs new [dir-name]` - Create a new project.
* `mkdocs serve` - Start the live-reloading docs server.
* `mkdocs build` - Build the documentation site.
* `mkdocs -h` - Print help message and exit.
* [`SQLAlchemy core`][sqlalchemy-core] for query building.
* [`databases`][databases] for cross-database async support.
* [`pydantic`][pydantic] for data validation.
## Project layout
Because ORM is built on SQLAlchemy core, you can use [`alembic`][alembic] to provide
database migrations.
mkdocs.yml # The configuration file.
docs/
index.md # The documentation homepage.
... # Other markdown pages, images and other files.
The goal was to create a simple orm that can be used directly with [`fastapi`][fastapi] that bases it's data validation on pydantic.
Initial work was inspired by [`encode/orm`][encode/orm].
The encode package was too simple (i.e. no ability to join two times to the same table) and used typesystem for data checks.
**async-orm is still under development:** We recommend pinning any dependencies with `aorm~=0.0.1`
**Note**: Use `ipython` to try this from the console, since it supports `await`.
```python
import databases
import orm
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Note(orm.Model):
__tablename__ = "notes"
__database__ = database
__metadata__ = metadata
# primary keys of type int by dafault are set to autoincrement
id = orm.Integer(primary_key=True)
text = orm.String(length=100)
completed = orm.Boolean(default=False)
# Create the database
engine = sqlalchemy.create_engine(str(database.url))
metadata.create_all(engine)
# .create()
await Note.objects.create(text="Buy the groceries.", completed=False)
await Note.objects.create(text="Call Mum.", completed=True)
await Note.objects.create(text="Send invoices.", completed=True)
# .all()
notes = await Note.objects.all()
# .filter()
notes = await Note.objects.filter(completed=True).all()
# exact, iexact, contains, icontains, lt, lte, gt, gte, in
notes = await Note.objects.filter(text__icontains="mum").all()
# .get()
note = await Note.objects.get(id=1)
# .update()
await note.update(completed=True)
# .delete()
await note.delete()
# 'pk' always refers to the primary key
note = await Note.objects.get(pk=2)
note.pk # 2
```
ORM supports loading and filtering across foreign keys...
```python
import databases
import orm
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Album(orm.Model):
__tablename__ = "album"
__metadata__ = metadata
__database__ = database
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
class Track(orm.Model):
__tablename__ = "track"
__metadata__ = metadata
__database__ = database
id = orm.Integer(primary_key=True)
album = orm.ForeignKey(Album)
title = orm.String(length=100)
position = orm.Integer()
# Create some records to work with.
malibu = await Album.objects.create(name="Malibu")
await Track.objects.create(album=malibu, title="The Bird", position=1)
await Track.objects.create(album=malibu, title="Heart don't stand a chance", position=2)
await Track.objects.create(album=malibu, title="The Waters", position=3)
fantasies = await Album.objects.create(name="Fantasies")
await Track.objects.create(album=fantasies, title="Help I'm Alive", position=1)
await Track.objects.create(album=fantasies, title="Sick Muse", position=2)
# Fetch an instance, without loading a foreign key relationship on it.
track = await Track.objects.get(title="The Bird")
# We have an album instance, but it only has the primary key populated
print(track.album) # Album(id=1) [sparse]
print(track.album.pk) # 1
print(track.album.name) # Raises AttributeError
# Load the relationship from the database
await track.album.load()
assert track.album.name == "Malibu"
# This time, fetch an instance, loading the foreign key relationship.
track = await Track.objects.select_related("album").get(title="The Bird")
assert track.album.name == "Malibu"
# By default you also get a second side of the relation
# constructed as lowercase source model name +'s' (tracks in this case)
# you can also provide custom name with parameter related_name
album = await Album.objects.select_related("tracks").all()
assert len(album.tracks) == 3
# Fetch instances, with a filter across an FK relationship.
tracks = Track.objects.filter(album__name="Fantasies")
assert len(tracks) == 2
# Fetch instances, with a filter and operator across an FK relationship.
tracks = Track.objects.filter(album__name__iexact="fantasies")
assert len(tracks) == 2
# Limit a query
tracks = await Track.objects.limit(1).all()
assert len(tracks) == 1
```
## Data types
The following keyword arguments are supported on all field types.
* `primary_key`
* `nullable`
* `default`
* `server_default`
* `index`
* `unique`
## Model Fields
### Common parameters
All fields are required unless one of the following is set:
* `nullable` - Creates a nullable column. Sets the default to `None`.
* `default` - Set a default value for the field.
* `server_default` - Set a default value for the field on server side (like sqlalchemy's `func.now()`).
* `primary key` - Set a primary key on a column.
* `autoincrement` - When a column is set to primary key and autoincrement is set on this column.
Autoincrement is set by default on int primary keys.
### Fields Types
* `orm.String(length)`
* `orm.Text()`
* `orm.Boolean()`
* `orm.Integer()`
* `orm.Float()`
* `orm.Date()`
* `orm.Time()`
* `orm.DateTime()`
* `orm.JSON()`
* `orm.BigInteger()`
* `orm.Decimal(lenght, precision)`
[sqlalchemy-core]: https://docs.sqlalchemy.org/en/latest/core/
[databases]: https://github.com/encode/databases
[pydantic]: https://pydantic-docs.helpmanual.io/
[encode/orm]: https://github.com/encode/orm/
[alembic]: https://alembic.sqlalchemy.org/en/latest/
[fastapi]: https://fastapi.tiangolo.com/

36
docs_src/fields/docs001.py Normal file
View File

@ -0,0 +1,36 @@
import databases
import sqlalchemy
import orm
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Department(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
class Course(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
completed = orm.Boolean(default=False)
department = orm.ForeignKey(Department)
department = Department(name='Science')
course = Course(name='Math', completed=False, department=department)
print(department.courses[0])
# Will produce:
# Course(id=None,
# name='Math',
# completed=False,
# department=Department(id=None, name='Science'))

36
docs_src/fields/docs002.py Normal file
View File

@ -0,0 +1,36 @@
import databases
import sqlalchemy
import orm
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Department(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
class Course(orm.Model):
__database__ = database
__metadata__ = metadata
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
completed = orm.Boolean(default=False)
department = orm.ForeignKey(Department, related_name="my_courses")
department = Department(name='Science')
course = Course(name='Math', completed=False, department=department)
print(department.my_courses[0])
# Will produce:
# Course(id=None,
# name='Math',
# completed=False,
# department=Department(id=None, name='Science'))

41
docs_src/fields/docs003.py Normal file
View File

@ -0,0 +1,41 @@
import orm
import databases
import sqlalchemy
database = databases.Database("sqlite:///db.sqlite")
metadata = sqlalchemy.MetaData()
class Album(orm.Model):
__tablename__ = "album"
__metadata__ = metadata
__database__ = database
id = orm.Integer(primary_key=True)
name = orm.String(length=100)
class Track(orm.Model):
__tablename__ = "track"
__metadata__ = metadata
__database__ = database
id = orm.Integer(primary_key=True)
album = orm.ForeignKey(Album, nullable=False)
title = orm.String(length=100)
position = orm.Integer()
album = await Album.objects.create(name="Brooklyn")
await Track.objects.create(album=album, title="The Bird", position=1)
# explicit preload of related Album Model
track = await Track.objects.select_related("album").get(title="The Bird")
assert track.album.name == 'Brooklyn'
# Will produce: True
# even without explicit select_related if ForeignKey is not nullable,
# the Album Model is still preloaded.
track2 = await Track.objects.get(title="The Bird")
assert track2.album.name == 'Brooklyn'
# Will produce: True

4
orm/fields/base.py
View File

@ -35,12 +35,12 @@ class BaseField:
@property
def is_required(self) -> bool:
return (
not self.nullable and not self.has_default and not self.is_auto_primary_key
not self.nullable and not self.has_default and not self.is_auto_primary_key
)
@property
def default_value(self) -> Any:
default = self.default if self.default is not None else self.server_default
default = self.default
return default() if callable(default) else default
@property

19
orm/fields/foreign_key.py
View File

@ -25,12 +25,12 @@ def create_dummy_instance(fk: Type["Model"], pk: Any = None) -> "Model":
class ForeignKey(BaseField):
def __init__(
self,
to: Type["Model"],
name: str = None,
related_name: str = None,
nullable: bool = True,
virtual: bool = False,
self,
to: Type["Model"],
name: str = None,
related_name: str = None,
nullable: bool = True,
virtual: bool = False,
) -> None:
super().__init__(nullable=nullable, name=name)
self.virtual = virtual
@ -50,7 +50,7 @@ class ForeignKey(BaseField):
return to_column.get_column_type()
def _extract_model_from_sequence(
self, value: List, child: "Model"
self, value: List, child: "Model"
) -> Union["Model", List["Model"]]:
return [self.expand_relationship(val, child) for val in value]
@ -75,10 +75,11 @@ class ForeignKey(BaseField):
return model
def register_relation(self, model: "Model", child: "Model") -> None:
model._orm_relationship_manager.add_relation(model, child, virtual=self.virtual)
child_model_name = self.related_name or child.get_name()
model._orm_relationship_manager.add_relation(model, child, child_model_name, virtual=self.virtual)
def expand_relationship(
self, value: Any, child: "Model"
self, value: Any, child: "Model"
) -> Optional[Union["Model", List["Model"]]]:
if value is None:

12
orm/models/metaclass.py
View File

@ -28,8 +28,8 @@ def parse_pydantic_field_from_model_fields(object_dict: dict) -> Dict[str, Tuple
def register_relation_on_build(table_name: str, field: ForeignKey, name: str) -> None:
child_relation_name = field.to.get_name(title=True) + "_" + name.lower() + "s"
reverse_name = field.related_name or child_relation_name
child_relation_name = field.to.get_name(title=True) + "_" + (field.related_name or (name.lower() + "s"))
reverse_name = child_relation_name
relation_name = name.lower().title() + "_" + field.to.get_name()
relationship_manager.add_relation_type(
relation_name, reverse_name, field, table_name
@ -43,14 +43,14 @@ def expand_reverse_relationships(model: Type["Model"]) -> None:
parent_model = model_field.to
child = model
if (
child_model_name not in parent_model.__fields__
and child.get_name() not in parent_model.__fields__
child_model_name not in parent_model.__fields__
and child.get_name() not in parent_model.__fields__
):
register_reverse_model_fields(parent_model, child, child_model_name)
def register_reverse_model_fields(
model: Type["Model"], child: Type["Model"], child_model_name: str
model: Type["Model"], child: Type["Model"], child_model_name: str
) -> None:
model.__fields__[child_model_name] = ModelField(
name=child_model_name,
@ -64,7 +64,7 @@ def register_reverse_model_fields(
def sqlalchemy_columns_from_model_fields(
name: str, object_dict: Dict, table_name: str
name: str, object_dict: Dict, table_name: str
) -> Tuple[Optional[str], List[sqlalchemy.Column], Dict[str, BaseField]]:
columns = []
pkname = None

19
orm/relations.py
View File

@ -37,25 +37,28 @@ class RelationshipManager:
del self._relations[rel_type][model._orm_id]
def add_relation(
self, parent: "FakePydantic", child: "FakePydantic", virtual: bool = False,
self,
parent: "FakePydantic",
child: "FakePydantic",
child_model_name: str,
virtual: bool = False,
) -> None:
parent_id, child_id = parent._orm_id, child._orm_id
parent_name, child_name = (
parent.get_name(title=True),
child.get_name(title=True),
)
parent_name =parent.get_name(title=True)
child_name = child_model_name if child.get_name() != child_model_name else child.get_name()+'s'
if virtual:
child_name, parent_name = parent_name, child_name
child_name, parent_name = parent_name, child.get_name()
child_id, parent_id = parent_id, child_id
child, parent = parent, proxy(child)
child_name = child_name.lower()+'s'
else:
child = proxy(child)
parent_relation_name = parent_name + "_" + child_name.lower() + "s"
parent_relation_name = parent_name.title() + "_" + child_name
parents_list = self._relations[parent_relation_name].setdefault(parent_id, [])
self.append_related_model(parents_list, child)
child_relation_name = child_name + "_" + parent_name.lower()
child_relation_name = child.get_name(title=True) + "_" + parent_name.lower()
children_list = self._relations[child_relation_name].setdefault(child_id, [])
self.append_related_model(children_list, parent)

26
tests/test_foreign_keys.py
View File

@ -3,7 +3,6 @@ import pytest
import sqlalchemy
import orm
import orm.fields.foreign_key
from orm.exceptions import NoMatch, MultipleMatches, RelationshipInstanceError
from tests.settings import DATABASE_URL
@ -26,11 +25,21 @@ class Track(orm.Model):
__database__ = database
id = orm.Integer(primary_key=True)
album = orm.fields.foreign_key.ForeignKey(Album)
album = orm.ForeignKey(Album)
title = orm.String(length=100)
position = orm.Integer()
class Cover(orm.Model):
__tablename__ = "covers"
__metadata__ = metadata
__database__ = database
id = orm.Integer(primary_key=True)
album = orm.ForeignKey(Album, related_name='cover_pictures')
title = orm.String(length=100)
class Organisation(orm.Model):
__tablename__ = "org"
__metadata__ = metadata
@ -46,7 +55,7 @@ class Team(orm.Model):
__database__ = database
id = orm.Integer(primary_key=True)
org = orm.fields.foreign_key.ForeignKey(Organisation)
org = orm.ForeignKey(Organisation)
name = orm.String(length=100)
@ -56,7 +65,7 @@ class Member(orm.Model):
__database__ = database
id = orm.Integer(primary_key=True)
team = orm.fields.foreign_key.ForeignKey(Team)
team = orm.ForeignKey(Team)
email = orm.String(length=100)
@ -81,6 +90,15 @@ async def test_setting_explicitly_empty_relation():
assert track.album is None
@pytest.mark.asyncio
async def test_related_name():
async with database:
album = await Album.objects.create(name="Vanilla")
await Cover.objects.create(album=album, title="The cover file")
assert len(album.cover_pictures) == 1
@pytest.mark.asyncio
async def test_model_crud():
async with database: