finish fields docs intial ver,add test for related name, fix child_name(s) in reverse relations

docs/fields.md

@@ -0,0 +1,206 @@
+# Fields
+
+
+There are 11 basic model field types and a special `ForeignKey` field to establish relationships between models.
+
+Each of the `Fields` has assigned both `sqlalchemy` column class and python type that is used to create `pydantic` model.
+
+
+## Common Parameters
+
+All `Field` types have a set of common parameters.
+
+### primary_key
+
+`primary_key`: `bool` = `False` -> by default False.
+
+Sets the primary key column on a table, foreign keys always refer to the pk of the `Model`.
+
+Used in sql only.
+
+### autoincrement
+
+`autoincrement`: `bool` = `primary_key and type == int` -> defaults to True if column is a primary key and of type Integer, otherwise False.
+
+Can be only used with int fields.
+
+If a field has autoincrement it becomes optional.
+
+Used only in sql.
+
+### nullable
+
+`nullable`: `bool` = `not primary_key` -> defaults to False for primary key column, and True for all other. 
+
+Specifies if field is optional or required, used both with sql and pydantic.
+
+!!!note
+    By default all `ForeignKeys` are also nullable, meaning the related `Model` is not required.
+    
+    If you change the `ForeignKey` column to `nullable`, it not only becomes required, it changes also the way in which data is loaded in queries.
+    
+    If you select `Model` without explicitly adding related `Model` assigned by not nullable `ForeignKey`, the `Model` is still gona be appended automatically, see example below.
+
+```Python hl_lines="24 32 33 34 35 37 38 39 40 41"
+--8<-- "../docs_src/fields/docs003.py"
+```
+
+!!!info
+    If you want to know more about how you can preload related models during queries and how the relations work read the [queries][queries] and [relations][relations] sections. 
+
+
+### default
+
+`default`: `Any` = `None` -> defaults to None. 
+
+A default value used if no other value is passed.
+
+In sql invoked on an insert, used during pydantic model definition.
+
+If the field has a default value it becomes optional.
+
+You can pass a static value or a Callable (function etc.)
+
+Used both in sql and pydantic.
+
+### server default
+
+`server_default`: `Any` = `None`  -> defaults to None. 
+
+A default value used if no other value is passed.
+
+In sql invoked on the server side so you can pass i.e. sql function (like now() wrapped in sqlalchemy text() clause).
+
+If the field has a server_default value it becomes optional.
+
+You can pass a static value or a Callable (function etc.)
+
+Used in sql only.
+ 
+### index
+
+`index`: `bool` = `False` -> by default False, 
+
+Sets the index on a table's column.
+
+Used in sql only.
+
+### unique
+
+`unique`: `bool` = `False` 
+
+Sets the unique constraint on a table's column.
+
+Used in sql only.
+
+## Fields Types
+
+### String
+
+`String(length)` has a required `length` parameter.  
+
+* Sqlalchemy column: `sqlalchemy.String`  
+* Type (used for pydantic): `str` 
+
+### Text
+
+`Text()` has no required parameters.  
+
+* Sqlalchemy column: `sqlalchemy.Text`  
+* Type (used for pydantic): `str` 
+
+### Boolean
+
+`Boolean()` has no required parameters.  
+
+* Sqlalchemy column: `sqlalchemy.Boolean`  
+* Type (used for pydantic): `bool` 
+
+### Integer
+
+`Integer()` has no required parameters.  
+
+* Sqlalchemy column: `sqlalchemy.Integer`  
+* Type (used for pydantic): `int` 
+
+### BigInteger
+
+`BigInteger()` has no required parameters.  
+
+* Sqlalchemy column: `sqlalchemy.BigInteger`  
+* Type (used for pydantic): `int` 
+
+### Float
+
+`Float()` has no required parameters.  
+
+* Sqlalchemy column: `sqlalchemy.Float`  
+* Type (used for pydantic): `float` 
+
+### Decimal
+
+`Decimal(lenght, precision)` has required `length` and `precision` parameters.  
+
+* Sqlalchemy column: `sqlalchemy.DECIMAL`  
+* Type (used for pydantic): `decimal.Decimal` 
+
+### Date
+
+`Date()` has no required parameters.  
+
+* Sqlalchemy column: `sqlalchemy.Date`  
+* Type (used for pydantic): `datetime.date` 
+
+### Time
+
+`Time()` has no required parameters.  
+
+* Sqlalchemy column: `sqlalchemy.Time`  
+* Type (used for pydantic): `datetime.time` 
+
+### DateTime
+
+`DateTime()` has no required parameters.  
+
+* Sqlalchemy column: `sqlalchemy.DateTime`  
+* Type (used for pydantic): `datetime.datetime` 
+
+### JSON
+
+`JSON()` has no required parameters.  
+
+* Sqlalchemy column: `sqlalchemy.JSON`  
+* Type (used for pydantic): `pydantic.Json` 
+
+### ForeignKey
+
+`ForeignKey(to, related_name=None)` has required parameters `to` that takes target `Model` class.  
+
+Sqlalchemy column and Type are automatically taken from target `Model`.
+
+* Sqlalchemy column: class of a target `Model` primary key column  
+* Type (used for pydantic): type of a target `Model` primary key column 
+
+`ForeignKey` fields are automatically registering reverse side of the relation.
+
+By default it's child (source) `Model` name + s, like courses in snippet below: 
+
+```Python hl_lines="25 31"
+--8<-- "../docs_src/fields/docs001.py"
+```
+
+But you can overwrite this name by providing `related_name` parameter like below:
+
+```Python hl_lines="25 30"
+--8<-- "../docs_src/fields/docs002.py"
+```
+
+!!!tip
+    Since related models are coming from Relationship Manager the reverse relation on access returns list of `wekref.proxy` to avoid circular references.
+
+!!!info
+    All relations are stored in lists, but when you access parent `Model` the ORM is unpacking the value for you. 
+    Read more in [relations][relations].
+
+[relations]: ./relations.md
+[queries]: ./queries.md

docs/index.md

@@ -1,17 +1,207 @@
-# Welcome to MkDocs
+# Async-ORM
 
-For full documentation visit [mkdocs.org](https://www.mkdocs.org).
+<p>
+<a href="https://travis-ci.com/collerek/async-orm">
+    <img src="https://travis-ci.com/collerek/async-orm.svg?branch=master" alt="Build Status">
+</a>
+<a href="https://codecov.io/gh/collerek/async-orm">
+    <img src="https://codecov.io/gh/collerek/async-orm/branch/master/graph/badge.svg" alt="Coverage">
+</a>
+<a href="https://www.codefactor.io/repository/github/collerek/async-orm">
+<img src="https://www.codefactor.io/repository/github/collerek/async-orm/badge" alt="CodeFactor" />
+</a>
+<a href="https://app.codacy.com/manual/collerek/async-orm?utm_source=github.com&utm_medium=referral&utm_content=collerek/async-orm&utm_campaign=Badge_Grade_Dashboard">
+<img src="https://api.codacy.com/project/badge/Grade/62568734f70f49cd8ea7a1a0b2d0c107" alt="Codacy" />
+</a>
+</p>
 
-## Commands
+The `async-orm` package is an async ORM for Python, with support for Postgres,
+MySQL, and SQLite. ORM is built with:
 
-* `mkdocs new [dir-name]` - Create a new project.
+  * [`SQLAlchemy core`][sqlalchemy-core] for query building.
-* `mkdocs serve` - Start the live-reloading docs server.
+  * [`databases`][databases] for cross-database async support.
-* `mkdocs build` - Build the documentation site.
+  * [`pydantic`][pydantic] for data validation.
-* `mkdocs -h` - Print help message and exit.
 
-## Project layout
+Because ORM is built on SQLAlchemy core, you can use [`alembic`][alembic] to provide
+database migrations.
 
-    mkdocs.yml    # The configuration file.
+The goal was to create a simple orm that can be used directly with [`fastapi`][fastapi] that bases it's data validation on pydantic.
-    docs/
+Initial work was inspired by [`encode/orm`][encode/orm].
-        index.md  # The documentation homepage.
+The encode package was too simple (i.e. no ability to join two times to the same table) and used typesystem for data checks.
-        ...       # Other markdown pages, images and other files.
+
+**async-orm is still under development:** We recommend pinning any dependencies with `aorm~=0.0.1`
+
+**Note**: Use `ipython` to try this from the console, since it supports `await`.
+
+```python
+import databases
+import orm
+import sqlalchemy
+
+database = databases.Database("sqlite:///db.sqlite")
+metadata = sqlalchemy.MetaData()
+
+
+class Note(orm.Model):
+    __tablename__ = "notes"
+    __database__ = database
+    __metadata__ = metadata
+
+    # primary keys of type int by dafault are set to autoincrement    
+    id = orm.Integer(primary_key=True)
+    text = orm.String(length=100)
+    completed = orm.Boolean(default=False)
+
+# Create the database
+engine = sqlalchemy.create_engine(str(database.url))
+metadata.create_all(engine)
+
+# .create()
+await Note.objects.create(text="Buy the groceries.", completed=False)
+await Note.objects.create(text="Call Mum.", completed=True)
+await Note.objects.create(text="Send invoices.", completed=True)
+
+# .all()
+notes = await Note.objects.all()
+
+# .filter()
+notes = await Note.objects.filter(completed=True).all()
+
+# exact, iexact, contains, icontains, lt, lte, gt, gte, in
+notes = await Note.objects.filter(text__icontains="mum").all()
+
+# .get()
+note = await Note.objects.get(id=1)
+
+# .update()
+await note.update(completed=True)
+
+# .delete()
+await note.delete()
+
+# 'pk' always refers to the primary key
+note = await Note.objects.get(pk=2)
+note.pk  # 2
+```
+
+ORM supports loading and filtering across foreign keys...
+
+```python
+import databases
+import orm
+import sqlalchemy
+
+database = databases.Database("sqlite:///db.sqlite")
+metadata = sqlalchemy.MetaData()
+
+
+class Album(orm.Model):
+    __tablename__ = "album"
+    __metadata__ = metadata
+    __database__ = database
+
+    id = orm.Integer(primary_key=True)
+    name = orm.String(length=100)
+
+
+class Track(orm.Model):
+    __tablename__ = "track"
+    __metadata__ = metadata
+    __database__ = database
+
+    id = orm.Integer(primary_key=True)
+    album = orm.ForeignKey(Album)
+    title = orm.String(length=100)
+    position = orm.Integer()
+
+
+# Create some records to work with.
+malibu = await Album.objects.create(name="Malibu")
+await Track.objects.create(album=malibu, title="The Bird", position=1)
+await Track.objects.create(album=malibu, title="Heart don't stand a chance", position=2)
+await Track.objects.create(album=malibu, title="The Waters", position=3)
+
+fantasies = await Album.objects.create(name="Fantasies")
+await Track.objects.create(album=fantasies, title="Help I'm Alive", position=1)
+await Track.objects.create(album=fantasies, title="Sick Muse", position=2)
+
+
+# Fetch an instance, without loading a foreign key relationship on it.
+track = await Track.objects.get(title="The Bird")
+
+# We have an album instance, but it only has the primary key populated
+print(track.album)       # Album(id=1) [sparse]
+print(track.album.pk)    # 1
+print(track.album.name)  # Raises AttributeError
+
+# Load the relationship from the database
+await track.album.load()
+assert track.album.name == "Malibu"
+
+# This time, fetch an instance, loading the foreign key relationship.
+track = await Track.objects.select_related("album").get(title="The Bird")
+assert track.album.name == "Malibu"
+
+# By default you also get a second side of the relation 
+# constructed as lowercase source model name +'s' (tracks in this case)
+# you can also provide custom name with parameter related_name
+album = await Album.objects.select_related("tracks").all()
+assert len(album.tracks) == 3
+
+# Fetch instances, with a filter across an FK relationship.
+tracks = Track.objects.filter(album__name="Fantasies")
+assert len(tracks) == 2
+
+# Fetch instances, with a filter and operator across an FK relationship.
+tracks = Track.objects.filter(album__name__iexact="fantasies")
+assert len(tracks) == 2
+
+# Limit a query
+tracks = await Track.objects.limit(1).all()
+assert len(tracks) == 1
+```
+
+## Data types
+
+The following keyword arguments are supported on all field types.
+
+  * `primary_key`
+  * `nullable`
+  * `default`
+  * `server_default`
+  * `index`
+  * `unique`
+
+## Model Fields
+
+### Common parameters
+
+All fields are required unless one of the following is set:
+
+  * `nullable` - Creates a nullable column. Sets the default to `None`.
+  * `default` - Set a default value for the field.
+  * `server_default` - Set a default value for the field on server side (like sqlalchemy's `func.now()`).
+  * `primary key` - Set a primary key on a column.
+  * `autoincrement` - When a column is set to primary key and autoincrement is set on this column. 
+    Autoincrement is set by default on int primary keys. 
+
+### Fields Types
+
+* `orm.String(length)`
+* `orm.Text()`
+* `orm.Boolean()`
+* `orm.Integer()`
+* `orm.Float()`
+* `orm.Date()`
+* `orm.Time()`
+* `orm.DateTime()`
+* `orm.JSON()`
+* `orm.BigInteger()`
+* `orm.Decimal(lenght, precision)`
+
+[sqlalchemy-core]: https://docs.sqlalchemy.org/en/latest/core/
+[databases]: https://github.com/encode/databases
+[pydantic]: https://pydantic-docs.helpmanual.io/
+[encode/orm]: https://github.com/encode/orm/
+[alembic]: https://alembic.sqlalchemy.org/en/latest/
+[fastapi]: https://fastapi.tiangolo.com/

docs_src/fields/docs001.py

@@ -0,0 +1,36 @@
+import databases
+import sqlalchemy
+
+import orm
+
+database = databases.Database("sqlite:///db.sqlite")
+metadata = sqlalchemy.MetaData()
+
+
+class Department(orm.Model):
+    __database__ = database
+    __metadata__ = metadata
+
+    id = orm.Integer(primary_key=True)
+    name = orm.String(length=100)
+
+
+class Course(orm.Model):
+    __database__ = database
+    __metadata__ = metadata
+
+    id = orm.Integer(primary_key=True)
+    name = orm.String(length=100)
+    completed = orm.Boolean(default=False)
+    department = orm.ForeignKey(Department)
+
+
+department = Department(name='Science')
+course = Course(name='Math', completed=False, department=department)
+
+print(department.courses[0])
+# Will produce:
+# Course(id=None,
+#        name='Math',
+#        completed=False,
+#        department=Department(id=None, name='Science'))

docs_src/fields/docs002.py

@@ -0,0 +1,36 @@
+import databases
+import sqlalchemy
+
+import orm
+
+database = databases.Database("sqlite:///db.sqlite")
+metadata = sqlalchemy.MetaData()
+
+
+class Department(orm.Model):
+    __database__ = database
+    __metadata__ = metadata
+
+    id = orm.Integer(primary_key=True)
+    name = orm.String(length=100)
+
+
+class Course(orm.Model):
+    __database__ = database
+    __metadata__ = metadata
+
+    id = orm.Integer(primary_key=True)
+    name = orm.String(length=100)
+    completed = orm.Boolean(default=False)
+    department = orm.ForeignKey(Department, related_name="my_courses")
+
+department = Department(name='Science')
+course = Course(name='Math', completed=False, department=department)
+
+print(department.my_courses[0])
+# Will produce:
+# Course(id=None,
+#        name='Math',
+#        completed=False,
+#        department=Department(id=None, name='Science'))
+

docs_src/fields/docs003.py

@@ -0,0 +1,41 @@
+import orm
+import databases
+import sqlalchemy
+
+database = databases.Database("sqlite:///db.sqlite")
+metadata = sqlalchemy.MetaData()
+
+
+class Album(orm.Model):
+    __tablename__ = "album"
+    __metadata__ = metadata
+    __database__ = database
+
+    id = orm.Integer(primary_key=True)
+    name = orm.String(length=100)
+
+
+class Track(orm.Model):
+    __tablename__ = "track"
+    __metadata__ = metadata
+    __database__ = database
+
+    id = orm.Integer(primary_key=True)
+    album = orm.ForeignKey(Album, nullable=False)
+    title = orm.String(length=100)
+    position = orm.Integer()
+
+
+album = await Album.objects.create(name="Brooklyn")
+await Track.objects.create(album=album, title="The Bird", position=1)
+
+# explicit preload of related Album Model
+track = await Track.objects.select_related("album").get(title="The Bird")
+assert track.album.name == 'Brooklyn'
+# Will produce: True
+
+# even without explicit select_related if ForeignKey is not nullable,
+# the Album Model is still preloaded.
+track2 = await Track.objects.get(title="The Bird")
+assert track2.album.name == 'Brooklyn'
+# Will produce: True

orm/fields/base.py

@@ -35,12 +35,12 @@ class BaseField:
     @property
     def is_required(self) -> bool:
         return (
-            not self.nullable and not self.has_default and not self.is_auto_primary_key
+                not self.nullable and not self.has_default and not self.is_auto_primary_key
         )
 
     @property
     def default_value(self) -> Any:
-        default = self.default if self.default is not None else self.server_default
+        default = self.default
         return default() if callable(default) else default
 
     @property

orm/fields/foreign_key.py

@@ -25,12 +25,12 @@ def create_dummy_instance(fk: Type["Model"], pk: Any = None) -> "Model":
 
 class ForeignKey(BaseField):
     def __init__(
-        self,
+            self,
-        to: Type["Model"],
+            to: Type["Model"],
-        name: str = None,
+            name: str = None,
-        related_name: str = None,
+            related_name: str = None,
-        nullable: bool = True,
+            nullable: bool = True,
-        virtual: bool = False,
+            virtual: bool = False,
     ) -> None:
         super().__init__(nullable=nullable, name=name)
         self.virtual = virtual
@@ -50,7 +50,7 @@ class ForeignKey(BaseField):
         return to_column.get_column_type()
 
     def _extract_model_from_sequence(
-        self, value: List, child: "Model"
+            self, value: List, child: "Model"
     ) -> Union["Model", List["Model"]]:
         return [self.expand_relationship(val, child) for val in value]
 
@@ -75,10 +75,11 @@ class ForeignKey(BaseField):
         return model
 
     def register_relation(self, model: "Model", child: "Model") -> None:
-        model._orm_relationship_manager.add_relation(model, child, virtual=self.virtual)
+        child_model_name = self.related_name or child.get_name()
+        model._orm_relationship_manager.add_relation(model, child, child_model_name, virtual=self.virtual)
 
     def expand_relationship(
-        self, value: Any, child: "Model"
+            self, value: Any, child: "Model"
     ) -> Optional[Union["Model", List["Model"]]]:
 
         if value is None:

orm/models/metaclass.py

@@ -28,8 +28,8 @@ def parse_pydantic_field_from_model_fields(object_dict: dict) -> Dict[str, Tuple
 
 
 def register_relation_on_build(table_name: str, field: ForeignKey, name: str) -> None:
-    child_relation_name = field.to.get_name(title=True) + "_" + name.lower() + "s"
+    child_relation_name = field.to.get_name(title=True) + "_" + (field.related_name or (name.lower() + "s"))
-    reverse_name = field.related_name or child_relation_name
+    reverse_name = child_relation_name
     relation_name = name.lower().title() + "_" + field.to.get_name()
     relationship_manager.add_relation_type(
         relation_name, reverse_name, field, table_name
@@ -43,14 +43,14 @@ def expand_reverse_relationships(model: Type["Model"]) -> None:
             parent_model = model_field.to
             child = model
             if (
-                child_model_name not in parent_model.__fields__
+                    child_model_name not in parent_model.__fields__
-                and child.get_name() not in parent_model.__fields__
+                    and child.get_name() not in parent_model.__fields__
             ):
                 register_reverse_model_fields(parent_model, child, child_model_name)
 
 
 def register_reverse_model_fields(
-    model: Type["Model"], child: Type["Model"], child_model_name: str
+        model: Type["Model"], child: Type["Model"], child_model_name: str
 ) -> None:
     model.__fields__[child_model_name] = ModelField(
         name=child_model_name,
@@ -64,7 +64,7 @@ def register_reverse_model_fields(
 
 
 def sqlalchemy_columns_from_model_fields(
-    name: str, object_dict: Dict, table_name: str
+        name: str, object_dict: Dict, table_name: str
 ) -> Tuple[Optional[str], List[sqlalchemy.Column], Dict[str, BaseField]]:
     columns = []
     pkname = None

orm/relations.py

@@ -37,25 +37,28 @@ class RelationshipManager:
                     del self._relations[rel_type][model._orm_id]
 
     def add_relation(
-        self, parent: "FakePydantic", child: "FakePydantic", virtual: bool = False,
+        self,
+            parent: "FakePydantic",
+            child: "FakePydantic",
+            child_model_name: str,
+            virtual: bool = False,
     ) -> None:
         parent_id, child_id = parent._orm_id, child._orm_id
-        parent_name, child_name = (
+        parent_name =parent.get_name(title=True)
-            parent.get_name(title=True),
+        child_name = child_model_name if child.get_name() != child_model_name else child.get_name()+'s'
-            child.get_name(title=True),
-        )
         if virtual:
-            child_name, parent_name = parent_name, child_name
+            child_name, parent_name = parent_name, child.get_name()
             child_id, parent_id = parent_id, child_id
             child, parent = parent, proxy(child)
+            child_name = child_name.lower()+'s'
         else:
             child = proxy(child)
 
-        parent_relation_name = parent_name + "_" + child_name.lower() + "s"
+        parent_relation_name = parent_name.title() + "_" + child_name
         parents_list = self._relations[parent_relation_name].setdefault(parent_id, [])
         self.append_related_model(parents_list, child)
 
-        child_relation_name = child_name + "_" + parent_name.lower()
+        child_relation_name = child.get_name(title=True) + "_" + parent_name.lower()
         children_list = self._relations[child_relation_name].setdefault(child_id, [])
         self.append_related_model(children_list, parent)
 

tests/test_foreign_keys.py

@@ -3,7 +3,6 @@ import pytest
 import sqlalchemy
 
 import orm
-import orm.fields.foreign_key
 from orm.exceptions import NoMatch, MultipleMatches, RelationshipInstanceError
 from tests.settings import DATABASE_URL
 
@@ -26,11 +25,21 @@ class Track(orm.Model):
     __database__ = database
 
     id = orm.Integer(primary_key=True)
-    album = orm.fields.foreign_key.ForeignKey(Album)
+    album = orm.ForeignKey(Album)
     title = orm.String(length=100)
     position = orm.Integer()
 
 
+class Cover(orm.Model):
+    __tablename__ = "covers"
+    __metadata__ = metadata
+    __database__ = database
+
+    id = orm.Integer(primary_key=True)
+    album = orm.ForeignKey(Album, related_name='cover_pictures')
+    title = orm.String(length=100)
+
+
 class Organisation(orm.Model):
     __tablename__ = "org"
     __metadata__ = metadata
@@ -46,7 +55,7 @@ class Team(orm.Model):
     __database__ = database
 
     id = orm.Integer(primary_key=True)
-    org = orm.fields.foreign_key.ForeignKey(Organisation)
+    org = orm.ForeignKey(Organisation)
     name = orm.String(length=100)
 
 
@@ -56,7 +65,7 @@ class Member(orm.Model):
     __database__ = database
 
     id = orm.Integer(primary_key=True)
-    team = orm.fields.foreign_key.ForeignKey(Team)
+    team = orm.ForeignKey(Team)
     email = orm.String(length=100)
 
 
@@ -81,6 +90,15 @@ async def test_setting_explicitly_empty_relation():
         assert track.album is None
 
 
+@pytest.mark.asyncio
+async def test_related_name():
+    async with database:
+        album = await Album.objects.create(name="Vanilla")
+        await Cover.objects.create(album=album, title="The cover file")
+
+        assert len(album.cover_pictures) == 1
+
+
 @pytest.mark.asyncio
 async def test_model_crud():
     async with database: