Merge pull request #217 from collerek/get_pydantic

Generating pydantic models tree and exclude_parent_fields

docs/api/models/new-basemodel.md

@@ -341,11 +341,11 @@ Calls the pydantic method to evaluate pydantic fields.
 
 `(None)`: None
 
-<a name="models.newbasemodel.NewBaseModel._get_related_not_excluded_fields"></a>
+<a name="models.newbasemodel.NewBaseModel._get_not_excluded_fields"></a>
 #### \_get\_related\_not\_excluded\_fields
 
 ```python
- | _get_related_not_excluded_fields(include: Optional[Dict], exclude: Optional[Dict]) -> List
+ | _get_not_excluded_fields(include: Optional[Dict], exclude: Optional[Dict]) -> List
 ```
 
 Returns related field names applying on them include and exclude set.

docs/fastapi/index.md

@@ -1,3 +1,4 @@
+# Fastapi
 
 The use of ormar with fastapi is quite simple.
 
@@ -14,7 +15,16 @@ Here you can find a very simple sample application code.
     
     It's divided into subsections for clarity.
 
-## Imports and initialization 
+!!!note
+        If you want to read more on how you can use ormar models in fastapi requests and 
+        responses check the [responses](response.md) and [requests](requests.md) documentation.
+
+## Quick Start
+
+!!!note
+        Note that you can find the full quick start script in the [github](https://github.com/collerek/ormar) repo under examples.
+
+### Imports and initialization 
 
 First take care of the imports and initialization 
 ```python
@@ -32,7 +42,7 @@ database = databases.Database("sqlite:///test.db")
 app.state.database = database
 ```
 
-## Database connection 
+### Database connection 
 
 Next define startup and shutdown events (or use middleware)
 - note that this is `databases` specific setting not the ormar one
@@ -54,7 +64,7 @@ async def shutdown() -> None:
 !!!info
     You can read more on connecting to databases in [fastapi][fastapi] documentation
 
-## Models definition 
+### Models definition 
 
 Define ormar models with appropriate fields. 
 
@@ -85,7 +95,7 @@ class Item(ormar.Model):
 !!!tip
     You can read more on defining `Models` in [models][models] section.
 
-## Fastapi endpoints definition
+### Fastapi endpoints definition
 
 Define your desired endpoints, note how `ormar` models are used both 
 as `response_model` and as a requests parameters.
@@ -130,9 +140,9 @@ async def delete_item(item_id: int, item: Item = None):
 !!!note
     Note that you can return a `Model` (or list of `Models`) directly - fastapi will jsonize it for you
 
-## Test the application
+### Test the application
 
-### Run fastapi
+#### Run fastapi
 
 If you want to run this script and play with fastapi swagger install uvicorn first
 
@@ -147,7 +157,7 @@ Now you can navigate to your browser (by default fastapi address is `127.0.0.1:8
 !!!info
     You can read more about running fastapi in [fastapi][fastapi] docs. 
 
-### Test with pytest
+#### Test with pytest
 
 Here you have a sample test that will prove that everything works as intended.
 
@@ -208,6 +218,6 @@ def test_all_endpoints():
     You can read more on testing fastapi in [fastapi][fastapi] docs. 
 
 [fastapi]: https://fastapi.tiangolo.com/
-[models]: ./models/index.md
-[database initialization]:  ./models/migrations.md
+[models]: ../models/index.md
+[database initialization]:  ../models/migrations.md
 [tests]: https://github.com/collerek/ormar/tree/master/tests

docs/fastapi/requests.md

@@ -0,0 +1,143 @@
+# Request
+
+You can use ormar Models in `fastapi` request `Body` parameters instead of pydantic models.
+
+You can of course also mix `ormar.Model`s with `pydantic` ones if you need to.
+
+One of the most common tasks in requests is excluding certain fields that you do not want to include in the payload you send to API.
+
+This can be achieved in several ways in `ormar` so below you can review your options and select the one most suitable for your situation.
+
+## Excluding fields in request
+
+### Optional fields
+
+Note that each field that is optional is not required, that means that Optional fields can be skipped both in response and in requests.
+
+Field is not required if (any/many/all) of following:
+
+* Field is marked with `nullable=True`
+* Field has `default` value or function provided, i.e. `default="Test"`
+* Field has a `server_default` value set
+* Field is an `autoincrement=True` `primary_key` field (note that `ormar.Integer` `primary_key` is `autoincrement` by default)
+
+Example:
+```python
+class User(ormar.Model):
+    class Meta:
+        tablename: str = "users"
+        metadata = metadata
+        database = database
+
+    id: int = ormar.Integer(primary_key=True)
+    email: str = ormar.String(max_length=255)
+    password: str = ormar.String(max_length=255)
+    first_name: str = ormar.String(max_length=255, nullable=True)
+    last_name: str = ormar.String(max_length=255)
+    category: str = ormar.String(max_length=255, default="User")
+```
+
+In above example fields `id` (is an `autoincrement` `Integer`), `first_name` ( has `nullable=True`) and `category` (has `default`) are optional and can be skipped in response and model wil still validate.
+
+If the field is nullable you don't have to include it in payload during creation as well as in response, so given example above you can:
+
+!!!Warning
+        Note that although you do not have to pass the optional field, you still **can** do it.
+        And if someone will pass a value it will be used later unless you take measures to prevent it.
+
+```python
+# note that app is an FastApi app
+@app.post("/users/", response_model=User) # here we use ormar.Model in response
+async def create_user(user: User):  # here we use ormar.Model in request parameter
+    return await user.save()
+```
+
+That means that if you do not pass i.e. `first_name` in request it will validate correctly (as field is optional), `None` will be saved in the database.
+
+### Generate `pydantic` model from `ormar.Model`
+
+Since task of excluding fields is so common `ormar` has a special way to generate `pydantic` models from existing `ormar.Models` without you needing to retype all the fields. 
+
+That method is `get_pydantic()` method available on all models classes.
+
+```python
+# generate a tree of models without password on User and without priority on nested Category
+RequestUser = User.get_pydantic(exclude={"password": ..., "category": {"priority"}})
+@app.post("/users3/", response_model=User) # here you can also use both ormar/pydantic
+async def create_user3(user: RequestUser):  # use the generated model here
+    # note how now user is pydantic and not ormar Model so you need to convert
+    return await User(**user.dict()).save()
+```
+
+!!!Note
+        To see more examples and read more visit [get_pydantic](../models/methods.md#get_pydantic) part of the documentation.
+
+!!!Warning
+        The `get_pydantic` method generates all models in a tree of nested models according to an algorithm that allows to avoid loops in models (same algorithm that is used in `dict()`, `select_all()` etc.)
+        
+        That means that nested models won't have reference to parent model (by default ormar relation is biderectional).
+        
+        Note also that if given model exists in a tree more than once it will be doubled in pydantic models (each occurance will have separate own model). That way you can exclude/include different fields on different leafs of the tree.
+
+#### Mypy and type checking
+
+Note that assigning a function as a python type passes at runtime (as it's not checked) the static type checkers like mypy will complain.
+
+Although result of the function call will always be the same for given model using a dynamically created type is not allowed.
+
+Therefore you have two options:
+
+First one is to simply add `# type: ignore` to skip the type checking
+
+```python
+RequestUser = User.get_pydantic(exclude={"password": ..., "category": {"priority"}})
+@app.post("/users3/", response_model=User)
+async def create_user3(user: RequestUser):  # type: ignore
+    # note how now user is not ormar Model so you need to convert
+    return await User(**user.dict()).save()
+```
+
+The second one is a little bit more hacky and utilizes a way in which fastapi extract function parameters.
+
+You can overwrite the `__annotations__` entry for given param.
+
+```python
+RequestUser = User.get_pydantic(exclude={"password": ..., "category": {"priority"}})
+# do not use the app decorator
+async def create_user3(user: User):  # use ormar model here
+    return await User(**user.dict()).save()
+# overwrite the function annotations entry for user param with generated model 
+create_user3.__annotations__["user"] = RequestUser
+# manually call app functions (app.get, app.post etc.) and pass your function reference
+app.post("/categories/", response_model=User)(create_user3)
+```
+
+Note that this will cause mypy to "think" that user is an ormar model but since in request it doesn't matter that much (you pass jsonized dict anyway and you need to convert before saving).
+
+That still should work fine as generated model will be a subset of fields, so all needed fields will validate, and all not used fields will fail at runtime.
+
+### Separate `pydantic` model
+
+The final solution is to just create separate pydantic model manually. 
+That works exactly the same as with normal fastapi application, so you can have different models for response and requests etc.
+
+Sample:
+```python
+import pydantic
+
+class UserCreate(pydantic.BaseModel):
+    class Config:
+        orm_mode = True
+
+    email: str
+    first_name: str
+    last_name: str
+    password: str
+
+
+@app.post("/users3/", response_model=User) # use ormar model here (but of course you CAN use pydantic also here)
+async def create_user3(user: UserCreate):  # use pydantic model here
+    # note how now request param is a pydantic model and not the ormar one
+    # so you need to parse/convert it to ormar before you can use database
+    return await User(**user.dict()).save()
+```

docs/fastapi/response.md

@@ -0,0 +1,243 @@
+# Response
+
+You can use ormar Models in `fastapi` response_model instead of pydantic models.
+
+You can of course also mix `ormar.Model`s with `pydantic` ones if you need to.
+
+One of the most common tasks in responses is excluding certain fields that you do not want to include in response data.
+
+This can be achieved in several ways in `ormar` so below you can review your options and select the one most suitable for your situation.
+
+## Excluding fields in response
+
+### Optional fields
+Note that each field that is optional is not required, that means that Optional fields can be skipped both in response and in requests.
+
+Field is not required if (any/many/all) of following:
+
+* Field is marked with `nullable=True`
+* Field has `default` value or function provided, i.e. `default="Test"`
+* Field has a `server_default` value set
+* Field is an `autoincrement=True` `primary_key` field (note that `ormar.Integer` `primary_key` is `autoincrement` by default)
+
+Example:
+```python
+class User(ormar.Model):
+    class Meta:
+        tablename: str = "users"
+        metadata = metadata
+        database = database
+
+    id: int = ormar.Integer(primary_key=True)
+    email: str = ormar.String(max_length=255)
+    password: str = ormar.String(max_length=255)
+    first_name: str = ormar.String(max_length=255, nullable=True)
+    last_name: str = ormar.String(max_length=255)
+    category: str = ormar.String(max_length=255, default="User")
+```
+
+In above example fields `id` (is an `autoincrement` `Integer`), `first_name` ( has `nullable=True`) and `category` (has `default`) are optional and can be skipped in response and model wil still validate.
+
+If the field is nullable you don't have to include it in payload during creation as well as in response, so given example above you can:
+
+```python
+# note that app is an FastApi app
+@app.post("/users/", response_model=User) # here we use ormar.Model in response
+async def create_user(user: User):  # here we use ormar.Model in request parameter
+    return await user.save()
+```
+
+That means that if you do not pass i.e. `first_name` in request it will validate correctly (as field is optional), save in the database and return the saved record without this field (which will also pass validation).
+
+!!!Note
+        Note that although you do not pass the **field value**, the **field itself** is still present in the `response_model` that means it **will be present in response data** and set to `None`.
+        
+        If you want to fully exclude the field from the result read on.
+
+### FastApi `response_model_exclude`
+
+Fastapi has `response_model_exclude` that accepts a set (or a list) of field names.
+
+That has it's limitation as `ormar` and `pydantic` accepts also dictionaries in which you can set exclude/include columns also on nested models (more on this below)
+
+!!!Warning
+        Note that you cannot exclude required fields when using `response_model` as it will fail during validation.
+
+```python
+@app.post("/users/", response_model=User, response_model_exclude={"password"})
+async def create_user(user: User):
+    return await user.save()
+```
+
+Above endpoint can be queried like this:
+
+```python
+from starlette.testclient import TestClient
+
+client = TestClient(app)
+
+with client as client:
+        # note there is no pk
+        user = {
+            "email": "test@domain.com",
+            "password": "^*^%A*DA*IAAA",
+            "first_name": "John",
+            "last_name": "Doe",
+        }
+        response = client.post("/users/", json=user)
+        # note that the excluded field is fully gone from response
+        assert "password" not in response.json()
+        # read the response and initialize model out of it
+        created_user = User(**response.json())
+        # note pk is populated by autoincrement
+        assert created_user.pk is not None
+        # note that password is missing in initialized model too
+        assert created_user.password is None
+```
+
+!!!Note
+        Note how in above example `password` field is fully gone from the response data. 
+        
+        Note that you can use this method only for non-required fields.
+
+#### Nested models excludes
+
+Despite the fact that `fastapi` allows passing only set of field names, so simple excludes, when using `response_model_exclude`, ormar is smarter.
+
+In `ormar` you can exclude nested models using two types of notations.
+
+One is a dictionary with nested fields that represents the model tree structure, and the second one is double underscore separated path of field names.
+
+Assume for a second that our user's category is a separate model:
+
+```python
+class BaseMeta(ormar.ModelMeta):
+    metadata = metadata 
+    database = database
+
+class Category(ormar.Model):
+    class Meta(BaseMeta):
+        tablename: str = "categories"
+    
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=255)    
+    priority: int = ormar.Integer(nullable=True)
+
+
+class User(ormar.Model):
+    class Meta(BaseMeta):
+        tablename: str = "users"
+
+    id: int = ormar.Integer(primary_key=True)
+    email: str = ormar.String(max_length=255)
+    password: str = ormar.String(max_length=255)
+    first_name: str = ormar.String(max_length=255, nullable=True)
+    last_name: str = ormar.String(max_length=255)
+    category: Optional[Category] = ormar.ForeignKey(Category, related_name="categories")
+```
+
+If you want to exclude `priority` from category in your response, you can still use fastapi parameter.
+```python
+@app.post("/users/", response_model=User, response_model_exclude={"category__priority"})
+async def create_user(user: User):
+    return await user.save()
+```
+
+Note that you can go in deeper models with double underscore, and if you wan't to exclude multiple fields from nested model you need to prefix them with full path.
+In example `response_model_exclude={"category__priority", "category__other_field", category__nested_model__nested_model_field}` etc.
+
+!!!Note
+        To read more about possible excludes and how to structure your exclude dictionary or set visit [fields](../queries/select-columns.md#fields) section of documentation
+
+!!!Note
+        Note that apart from `response_model_exclude` parameter `fastapi` supports also other parameters inherited from `pydantic`.
+        All of them works also with ormar, but can have some nuances so best to read [dict](../models/methods.md#dict) part of the documentation.
+
+### Exclude in `Model.dict()`
+
+Alternatively you can just return a dict from `ormar.Model` and use . 
+
+Like this you can also set exclude/include as dict and exclude fields on nested models too.
+
+!!!Warning
+        Not using a `response_model` will cause api documentation having no response example and schema since in theory response can have any format.
+
+```python
+@app.post("/users2/", response_model=User)
+async def create_user2(user: User):
+    user = await user.save()
+    return user.dict(exclude={'password'})
+    # could be also something like return user.dict(exclude={'category': {'priority'}}) to exclude category priority
+```
+
+!!!Note
+        Note that above example will nullify the password field even if you pass it in request, but the **field will be still there** as it's part of the response schema, the value will be set to `None`.
+
+If you want to fully exclude the field with this approach simply don't use `response_model` and exclude in Model's dict()
+
+Alternatively you can just return a dict from ormar model. 
+Like this you can also set exclude/include as dict and exclude fields on nested models.
+
+!!!Note
+        In theory you loose validation of response here but since you operate on `ormar.Models` the response data have already been validated after db query (as ormar model is pydantic model).
+
+So if you skip `response_model` altogether you can do something like this:
+
+```python
+@app.post("/users4/") # note no response_model
+async def create_user4(user: User):
+    user = await user.save()
+    return user.dict(exclude={'last_name'})
+```
+
+!!!Note
+        Note that when you skip the response_model you can now **exclude also required fields** as the response is no longer validated after being returned.
+        
+        The cost of this solution is that you loose also api documentation as response schema in unknown from fastapi perspective. 
+
+### Generate `pydantic` model from `ormar.Model`
+
+Since task of excluding fields is so common `ormar` has a special way to generate `pydantic` models from existing `ormar.Models` without you needing to retype all the fields. 
+
+That method is `get_pydantic()` method available on all models classes.
+
+```python
+# generate a tree of models without password on User and without priority on nested Category
+ResponseUser = User.get_pydantic(exclude={"password": ..., "category": {"priority"}})
+@app.post("/users3/", response_model=ResponseUser) # use the generated model here
+async def create_user3(user: User):
+    return await user.save()
+```
+
+!!!Note
+        To see more examples and read more visit [get_pydantic](../models/methods.md#get_pydantic) part of the documentation.
+
+!!!Warning
+        The `get_pydantic` method generates all models in a tree of nested models according to an algorithm that allows to avoid loops in models (same algorithm that is used in `dict()`, `select_all()` etc.)
+        
+        That means that nested models won't have reference to parent model (by default ormar relation is biderectional).
+        
+        Note also that if given model exists in a tree more than once it will be doubled in pydantic models (each occurance will have separate own model). That way you can exclude/include different fields on different leafs of the tree.
+
+### Separate `pydantic` model
+
+The final solution is to just create separate pydantic model manually. 
+That works exactly the same as with normal fastapi application so you can have different models for response and requests etc.
+
+Sample:
+```python
+import pydantic
+
+class UserBase(pydantic.BaseModel):
+    class Config:
+        orm_mode = True
+
+    email: str
+    first_name: str
+    last_name: str
+
+
+@app.post("/users3/", response_model=UserBase) # use pydantic model here
+async def create_user3(user: User): #use ormar model here (but of course you CAN use pydantic also here)
+    return await user.save()
+```

docs/fields/common-parameters.md

@@ -50,6 +50,26 @@ You can pass a static value or a Callable (function etc.)
 
 Used both in sql and pydantic.
 
+Sample usage:
+
+```python
+# note the distinction between passing a value and Callable pointer
+
+# value
+name: str = ormar.String(max_length=200, default="Name")
+
+# note that when you call a function it's not a pointer to Callable
+# a definition like this will call the function at startup and assign
+# the result of the function to the default, so it will be constant value for all instances
+created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now())
+
+# if you want to pass Callable reference (note that it cannot have arguments)
+# note lack of the parenthesis -> ormar will call this function for you on each model
+created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
+
+# Callable can be a function, builtin, class etc.
+```
+
 ## server default
 
 `server_default`: `Any` = `None`  -> defaults to None. 

docs/models/inheritance.md

@@ -461,3 +461,113 @@ abstract parent model you may lose your data on through table if not careful.
 
     That means that each time you define a Child model you need to either manually create
     the table in the database, or run a migration (with alembic).
+
+## exclude_parent_fields
+
+Ormar allows you to skip certain fields in inherited model that are coming from a parent model.
+
+!!!Note
+    Note that the same behaviour can be achieved by splitting the model into more abstract models and mixins - which is a preferred way in normal circumstances.
+
+To skip certain fields from a child model, list all fields that you want to skip in `model.Meta.exclude_parent_fields` parameter like follows:
+
+```python
+metadata = sa.MetaData()
+db = databases.Database(DATABASE_URL)
+
+
+class AuditModel(ormar.Model):
+    class Meta:
+        abstract = True
+
+    created_by: str = ormar.String(max_length=100)
+    updated_by: str = ormar.String(max_length=100, default="Sam")
+
+
+class DateFieldsModel(ormar.Model):
+    class Meta(ormar.ModelMeta):
+        abstract = True
+        metadata = metadata
+        database = db
+
+    created_date: datetime.datetime = ormar.DateTime(
+        default=datetime.datetime.now, name="creation_date"
+    )
+    updated_date: datetime.datetime = ormar.DateTime(
+        default=datetime.datetime.now, name="modification_date"
+    )
+
+
+class Category(DateFieldsModel, AuditModel):
+    class Meta(ormar.ModelMeta):
+        tablename = "categories"
+        # set fields that should be skipped
+        exclude_parent_fields = ["updated_by", "updated_date"]
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=50, unique=True, index=True)
+    code: int = ormar.Integer()
+
+# Note that now the update fields in Category are gone in all places -> ormar fields, pydantic fields and sqlachemy table columns
+# so full list of available fileds in Category is: ["created_by", "created_date", "id", "name", "code"]
+```
+
+Note how you simply need to provide field names and it will exclude the parent field regardless of from which parent model the field is coming from.
+
+!!!Note
+    Note that if you want to overwrite a field in child model you do not have to exclude it, simpy overwrite the field declaration in child model with same field name.
+
+!!!Warning
+    Note that this kind of behavior can confuse mypy and static type checkers, yet accessing the non existing fields will fail at runtime. That's why splitting the base classes is preferred.
+
+The same effect can be achieved by splitting base classes like:
+
+```python
+metadata = sa.MetaData()
+db = databases.Database(DATABASE_URL)
+
+
+class AuditCreateModel(ormar.Model):
+    class Meta:
+        abstract = True
+
+    created_by: str = ormar.String(max_length=100)
+    
+
+class AuditUpdateModel(ormar.Model):
+    class Meta:
+        abstract = True
+
+    updated_by: str = ormar.String(max_length=100, default="Sam")
+
+class CreateDateFieldsModel(ormar.Model):
+    class Meta(ormar.ModelMeta):
+        abstract = True
+        metadata = metadata
+        database = db
+
+    created_date: datetime.datetime = ormar.DateTime(
+        default=datetime.datetime.now, name="creation_date"
+    )
+    
+class UpdateDateFieldsModel(ormar.Model):
+    class Meta(ormar.ModelMeta):
+        abstract = True
+        metadata = metadata
+        database = db
+
+    updated_date: datetime.datetime = ormar.DateTime(
+        default=datetime.datetime.now, name="modification_date"
+    )
+
+
+class Category(CreateDateFieldsModel, AuditCreateModel):
+    class Meta(ormar.ModelMeta):
+        tablename = "categories"
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=50, unique=True, index=True)
+    code: int = ormar.Integer()
+```
+
+That way you can inherit from both create and update classes if needed, and only one of them otherwise.

docs/models/methods.md

@@ -284,6 +284,89 @@ assert item.dict(exclude_through_models=True) == {
 
 Of course the end result is a string with json representation and not a dictionary.
 
+## get_pydantic
+
+`get_pydantic(include: Union[Set, Dict] = None, exclude: Union[Set, Dict] = None)`
+
+This method allows you to generate `pydantic` models from your ormar models without you needing to retype all the fields.
+
+Note that if you have nested models, it **will generate whole tree of pydantic models for you!**
+
+Moreover, you can pass `exclude` and/or `include` parameters to keep only the fields that you want to, including in nested models.
+
+That means that this way you can effortlessly create pydantic models for requests and responses in `fastapi`.
+
+!!!Note
+        To read more about possible excludes/includes and how to structure your exclude dictionary or set visit [fields](../queries/select-columns.md#fields) section of documentation
+
+Given sample ormar models like follows:
+
+```python
+metadata = sqlalchemy.MetaData()
+database = databases.Database(DATABASE_URL, force_rollback=True)
+
+
+class BaseMeta(ormar.ModelMeta):
+    metadata = metadata
+    database = database
+
+class Category(ormar.Model):
+    class Meta(BaseMeta):
+        tablename = "categories"
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=100)
+
+
+class Item(ormar.Model):
+    class Meta(BaseMeta):
+        pass
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=100, default="test")
+    category: Optional[Category] = ormar.ForeignKey(Category, nullable=True)
+```
+
+You can generate pydantic models out of it with a one simple call.
+
+```python
+PydanticCategory = Category.get_pydantic(include={"id", "name"}
+```
+
+Which will generate model equivalent of:
+
+```python
+class Category(BaseModel):
+    id: Optional[int]
+    name: Optional[str] = "test"
+```
+
+!!!warning
+        Note that it's not a good practice to have several classes with same name in one module, as well as it would break `fastapi` docs.
+        Thats's why ormar adds random 3 uppercase letters to the class name. In example above it means that in reality class would be named i.e. `Category_XIP(BaseModel)`.
+
+To exclude or include nested fields you can use dict or double underscores.
+
+```python
+# both calls are equivalent
+PydanticCategory = Category.get_pydantic(include={"id", "items__id"})
+PydanticCategory = Category.get_pydantic(include={"id": ..., "items": {"id"}})
+```
+
+and results in a generated structure as follows:
+```python
+class Item(BaseModel):
+    id: Optional[int]
+    
+class Category(BaseModel):
+    id: Optional[int]
+    items: Optional[List[Item]]
+```
+
+Of course you can use also deeply nested structures and ormar will generate it pydantic equivalent you (in a way that exclude loops).
+
+Note how `Item` model above does not have a reference to `Category` although in ormar the relation is bidirectional (and `ormar.Item` has `categories` field).
+
 ## load
 
 By default when you query a table without prefetching related models, the ormar will still construct

docs/relations/foreign-key.md

@@ -99,7 +99,7 @@ course = Course(name="Math", completed=False) # note - not saved
 await department.courses.add(course)
 assert course.pk is not None # child model was saved
 # relation on child model is set and FK column saved in db
-assert courses.department == department
+assert course.department == department
 # relation on parent model is also set
 assert department.courses[0] == course 
 ```
@@ -112,6 +112,10 @@ assert department.courses[0] == course
     
     That means that in example above the department has to be saved before you can call `department.courses.add()`.
 
+!!!warning
+    This method will not work on `ManyToMany` relations - there, both sides of the relation have to be saved before adding to relation.
+    
+
 ### remove
 
 Removal of the related model one by one.

docs/releases.md

@@ -1,3 +1,21 @@
+# 0.10.10
+
+## ✨ Features
+
+* Add [`get_pydantic`](https://collerek.github.io/ormar/models/methods/#get_pydantic) flag that allows you to auto generate equivalent pydantic models tree from ormar.Model. This newly generated model tree can be used in requests and responses to exclude fields you do not want to include in the data.
+* Add [`exclude_parent_fields`](https://collerek.github.io/ormar/models/inheritance/#exclude_parent_fields) parameter to model Meta that allows you to exclude fields from parent models during inheritance. Note that best practice is to combine models and mixins but if you have many similar models and just one that differs it might be useful tool to achieve that. 
+
+## 🐛 Fixes
+
+* Fix is null filter with pagination and relations (by @erichaydel) [#214](https://github.com/collerek/ormar/issues/214)
+* Fix not saving child object on reverse side of the relation if not saved before [#216](https://github.com/collerek/ormar/issues/216)
+
+
+## 💬 Other
+
+* Expand [fastapi](https://collerek.github.io/ormar/fastapi) part of the documentation to show samples of using ormar in requests and responses in fastapi.
+* Improve the docs in regard of `default`, `ForeignKey.add` etc. 
+
 # 0.10.9
 
 ## Important security fix

examples/fastapi_quick_start.py

@@ -0,0 +1,85 @@
+from typing import List, Optional
+
+import databases
+import sqlalchemy
+import uvicorn
+from fastapi import FastAPI
+
+import ormar
+
+app = FastAPI()
+metadata = sqlalchemy.MetaData()
+database = databases.Database("sqlite:///test.db")
+app.state.database = database
+
+
+@app.on_event("startup")
+async def startup() -> None:
+    database_ = app.state.database
+    if not database_.is_connected:
+        await database_.connect()
+
+
+@app.on_event("shutdown")
+async def shutdown() -> None:
+    database_ = app.state.database
+    if database_.is_connected:
+        await database_.disconnect()
+
+
+class Category(ormar.Model):
+    class Meta:
+        tablename = "categories"
+        metadata = metadata
+        database = database
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=100)
+
+
+class Item(ormar.Model):
+    class Meta:
+        tablename = "items"
+        metadata = metadata
+        database = database
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=100)
+    category: Optional[Category] = ormar.ForeignKey(Category, nullable=True)
+
+
+@app.get("/items/", response_model=List[Item])
+async def get_items():
+    items = await Item.objects.select_related("category").all()
+    return items
+
+
+@app.post("/items/", response_model=Item)
+async def create_item(item: Item):
+    await item.save()
+    return item
+
+
+@app.post("/categories/", response_model=Category)
+async def create_category(category: Category):
+    await category.save()
+    return category
+
+
+@app.put("/items/{item_id}")
+async def get_item(item_id: int, item: Item):
+    item_db = await Item.objects.get(pk=item_id)
+    return await item_db.update(**item.dict())
+
+
+@app.delete("/items/{item_id}")
+async def delete_item(item_id: int, item: Item = None):
+    if item:
+        return {"deleted_rows": await item.delete()}
+    item_db = await Item.objects.get(pk=item_id)
+    return {"deleted_rows": await item_db.delete()}
+
+
+if __name__ == "__main__":
+    # to play with API run the script and visit http://127.0.0.1:8000/docs
+    uvicorn.run(app, host="127.0.0.1", port=8000)

mkdocs.yml

@@ -33,7 +33,10 @@ nav:
     - queries/aggregations.md
   - Signals: signals.md
   - Transactions: transactions.md
-  - Use with Fastapi: fastapi.md
+  - Use with Fastapi:
+    - Quick Start: fastapi/index.md
+    - Using ormar in responses: fastapi/response.md
+    - Using ormar in requests: fastapi/requests.md
   - Use with mypy: mypy.md
   - PyCharm plugin: plugin.md
   - Contributing: contributing.md

ormar/__init__.py

@@ -76,7 +76,7 @@ class UndefinedType:  # pragma no cover
 
 Undefined = UndefinedType()
 
-__version__ = "0.10.9"
+__version__ = "0.10.10"
 __all__ = [
     "Integer",
     "BigInteger",

ormar/models/helpers/__init__.py

@@ -8,6 +8,7 @@ from ormar.models.helpers.pydantic import (
     get_potential_fields,
     get_pydantic_base_orm_config,
     get_pydantic_field,
+    remove_excluded_parent_fields,
 )
 from ormar.models.helpers.relations import (
     alias_manager,
@@ -36,4 +37,5 @@ __all__ = [
     "sqlalchemy_columns_from_model_fields",
     "populate_choices_validators",
     "meta_field_not_set",
+    "remove_excluded_parent_fields",
 ]

ormar/models/helpers/models.py

@@ -54,6 +54,8 @@ def populate_default_options_values(
         new_model.Meta.abstract = False
     if not hasattr(new_model.Meta, "orders_by"):
         new_model.Meta.orders_by = []
+    if not hasattr(new_model.Meta, "exclude_parent_fields"):
+        new_model.Meta.exclude_parent_fields = []
 
     if any(
         is_field_an_forward_ref(field) for field in new_model.Meta.model_fields.values()

ormar/models/helpers/pydantic.py

@@ -117,3 +117,17 @@ def get_potential_fields(attrs: Dict) -> Dict:
         for k, v in attrs.items()
         if (lenient_issubclass(v, BaseField) or isinstance(v, BaseField))
     }
+
+
+def remove_excluded_parent_fields(model: Type["Model"]) -> None:
+    """
+    Removes pydantic fields that should be excluded from parent models
+
+    :param model:
+    :type model: Type["Model"]
+    """
+    excludes = {*model.Meta.exclude_parent_fields} - {*model.Meta.model_fields.keys()}
+    if excludes:
+        model.__fields__ = {
+            k: v for k, v in model.__fields__.items() if k not in excludes
+        }

ormar/models/helpers/validation.py

@@ -131,7 +131,7 @@ def generate_model_example(model: Type["Model"], relation_map: Dict = None) -> D
     :type model: Type["Model"]
     :param relation_map: dict with relations to follow
     :type relation_map: Optional[Dict]
-    :return:
+    :return: dict with example values
     :rtype: Dict[str, int]
     """
     example: Dict[str, Any] = dict()
@@ -141,6 +141,31 @@ def generate_model_example(model: Type["Model"], relation_map: Dict = None) -> D
         else translate_list_to_dict(model._iterate_related_models())
     )
     for name, field in model.Meta.model_fields.items():
+        populates_sample_fields_values(
+            example=example, name=name, field=field, relation_map=relation_map
+        )
+    to_exclude = {name for name in model.Meta.model_fields}
+    pydantic_repr = generate_pydantic_example(pydantic_model=model, exclude=to_exclude)
+    example.update(pydantic_repr)
+
+    return example
+
+
+def populates_sample_fields_values(
+    example: Dict[str, Any], name: str, field: BaseField, relation_map: Dict = None
+) -> None:
+    """
+    Iterates the field and sets fields to sample values
+
+    :param field: ormar field
+    :type field: BaseField
+    :param name: name of the field
+    :type name: str
+    :param example: example dict
+    :type example: Dict[str, Any]
+    :param relation_map: dict with relations to follow
+    :type relation_map: Optional[Dict]
+    """
     if not field.is_relation:
         is_bytes_str = field.__type__ == bytes and field.represent_as_base64_str
         example[name] = field.__sample__ if not is_bytes_str else "string"
@@ -148,11 +173,6 @@ def generate_model_example(model: Type["Model"], relation_map: Dict = None) -> D
         example[name] = get_nested_model_example(
             name=name, field=field, relation_map=relation_map
         )
-    to_exclude = {name for name in model.Meta.model_fields}
-    pydantic_repr = generate_pydantic_example(pydantic_model=model, exclude=to_exclude)
-    example.update(pydantic_repr)
-
-    return example
 
 
 def get_nested_model_example(

ormar/models/metaclass.py

@@ -44,6 +44,7 @@ from ormar.models.helpers import (
     populate_meta_sqlalchemy_table_if_required,
     populate_meta_tablename_columns_and_pk,
     register_relation_in_alias_manager,
+    remove_excluded_parent_fields,
     sqlalchemy_columns_from_model_fields,
 )
 from ormar.models.quick_access_views import quick_access_set
@@ -80,6 +81,7 @@ class ModelMeta:
     abstract: bool
     requires_ref_update: bool
     orders_by: List[str]
+    exclude_parent_fields: List[str]
 
 
 def add_cached_properties(new_model: Type["Model"]) -> None:
@@ -308,7 +310,7 @@ def copy_data_from_parent_model(  # noqa: CCR001
     model_fields: Dict[str, Union[BaseField, ForeignKeyField, ManyToManyField]],
 ) -> Tuple[Dict, Dict]:
     """
-    Copy the key parameters [databse, metadata, property_fields and constraints]
+    Copy the key parameters [database, metadata, property_fields and constraints]
     and fields from parent models. Overwrites them if needed.
 
     Only abstract classes can be subclassed.
@@ -351,6 +353,11 @@ def copy_data_from_parent_model(  # noqa: CCR001
             else attrs.get("__name__", "").lower() + "s"
         )
         for field_name, field in base_class.Meta.model_fields.items():
+            if (
+                hasattr(meta, "exclude_parent_fields")
+                and field_name in meta.exclude_parent_fields
+            ):
+                continue
             if field.is_multi:
                 field = cast(ManyToManyField, field)
                 copy_and_replace_m2m_through_model(
@@ -386,7 +393,7 @@ def extract_from_parents_definition(  # noqa: CCR001
     model_fields: Dict[str, Union[BaseField, ForeignKeyField, ManyToManyField]],
 ) -> Tuple[Dict, Dict]:
     """
-    Extracts fields from base classes if they have valid oramr fields.
+    Extracts fields from base classes if they have valid ormar fields.
 
     If model was already parsed -> fields definitions need to be removed from class
     cause pydantic complains about field re-definition so after first child
@@ -595,6 +602,7 @@ class ModelMetaclass(pydantic.main.ModelMetaclass):
                     )
 
                 new_model.pk = PkDescriptor(name=new_model.Meta.pkname)
+                remove_excluded_parent_fields(new_model)
 
         return new_model
 

ormar/models/mixins/__init__.py

@@ -8,6 +8,7 @@ from ormar.models.mixins.alias_mixin import AliasMixin
 from ormar.models.mixins.excludable_mixin import ExcludableMixin
 from ormar.models.mixins.merge_mixin import MergeModelMixin
 from ormar.models.mixins.prefetch_mixin import PrefetchQueryMixin
+from ormar.models.mixins.pydantic_mixin import PydanticMixin
 from ormar.models.mixins.save_mixin import SavePrepareMixin
 
 __all__ = [
@@ -16,4 +17,5 @@ __all__ = [
     "PrefetchQueryMixin",
     "SavePrepareMixin",
     "ExcludableMixin",
+    "PydanticMixin",
 ]

ormar/models/mixins/pydantic_mixin.py

@@ -0,0 +1,110 @@
+import string
+from random import choices
+from typing import (
+    Any,
+    Callable,
+    Dict,
+    List,
+    Optional,
+    Set,
+    TYPE_CHECKING,
+    Type,
+    Union,
+    cast,
+)
+
+import pydantic
+from pydantic.fields import ModelField
+
+from ormar.models.mixins.relation_mixin import RelationMixin  # noqa: I100, I202
+from ormar.queryset.utils import translate_list_to_dict
+
+
+class PydanticMixin(RelationMixin):
+    if TYPE_CHECKING:  # pragma: no cover
+        __fields__: Dict[str, ModelField]
+        _skip_ellipsis: Callable
+        _get_not_excluded_fields: Callable
+
+    @classmethod
+    def get_pydantic(
+        cls, *, include: Union[Set, Dict] = None, exclude: Union[Set, Dict] = None,
+    ) -> Type[pydantic.BaseModel]:
+        """
+        Returns a pydantic model out of ormar model.
+
+        Converts also nested ormar models into pydantic models.
+
+        Can be used to fully exclude certain fields in fastapi response and requests.
+
+        :param include: fields of own and nested models to include
+        :type include: Union[Set, Dict, None]
+        :param exclude: fields of own and nested models to exclude
+        :type exclude: Union[Set, Dict, None]
+        """
+        relation_map = translate_list_to_dict(cls._iterate_related_models())
+
+        return cls._convert_ormar_to_pydantic(
+            include=include, exclude=exclude, relation_map=relation_map
+        )
+
+    @classmethod
+    def _convert_ormar_to_pydantic(
+        cls,
+        relation_map: Dict[str, Any],
+        include: Union[Set, Dict] = None,
+        exclude: Union[Set, Dict] = None,
+    ) -> Type[pydantic.BaseModel]:
+        if include and isinstance(include, Set):
+            include = translate_list_to_dict(include)
+        if exclude and isinstance(exclude, Set):
+            exclude = translate_list_to_dict(exclude)
+        fields_dict: Dict[str, Any] = dict()
+        defaults: Dict[str, Any] = dict()
+        fields_to_process = cls._get_not_excluded_fields(
+            fields={*cls.Meta.model_fields.keys()}, include=include, exclude=exclude
+        )
+        for name in fields_to_process:
+            field = cls._determine_pydantic_field_type(
+                name=name,
+                defaults=defaults,
+                include=include,
+                exclude=exclude,
+                relation_map=relation_map,
+            )
+            if field is not None:
+                fields_dict[name] = field
+        model = type(
+            f"{cls.__name__}_{''.join(choices(string.ascii_uppercase, k=3))}",
+            (pydantic.BaseModel,),
+            {"__annotations__": fields_dict, **defaults},
+        )
+        return cast(Type[pydantic.BaseModel], model)
+
+    @classmethod
+    def _determine_pydantic_field_type(
+        cls,
+        name: str,
+        defaults: Dict,
+        include: Union[Set, Dict, None],
+        exclude: Union[Set, Dict, None],
+        relation_map: Dict[str, Any],
+    ) -> Any:
+        field = cls.Meta.model_fields[name]
+        target: Any = None
+        if field.is_relation and name in relation_map:  # type: ignore
+            target = field.to._convert_ormar_to_pydantic(
+                include=cls._skip_ellipsis(include, name),
+                exclude=cls._skip_ellipsis(exclude, name),
+                relation_map=cls._skip_ellipsis(
+                    relation_map, field, default_return=dict()
+                ),
+            )
+            if field.is_multi or field.virtual:
+                target = List[target]  # type: ignore
+        elif not field.is_relation:
+            defaults[name] = cls.__fields__[name].field_info
+            target = field.__type__
+        if target is not None and field.nullable:
+            target = Optional[target]
+        return target

ormar/models/mixins/save_mixin.py

@@ -1,5 +1,5 @@
 import uuid
-from typing import Callable, Collection, Dict, Optional, Set, TYPE_CHECKING, cast
+from typing import Callable, Collection, Dict, List, Optional, Set, TYPE_CHECKING, cast
 
 import ormar
 from ormar.exceptions import ModelPersistenceError
@@ -275,9 +275,7 @@ class SavePrepareMixin(RelationMixin, AliasMixin):
         :rtype: int
         """
         for field in fields_list:
-            values = getattr(self, field.name) or []
-            if not isinstance(values, list):
-                values = [values]
+            values = self._get_field_values(name=field.name)
             for value in values:
                 if follow:
                     update_count = await value.save_related(
@@ -299,3 +297,17 @@ class SavePrepareMixin(RelationMixin, AliasMixin):
                         update_count=update_count,
                     )
         return update_count
+
+    def _get_field_values(self, name: str) -> List:
+        """
+        Extract field values and ensures it is a list.
+
+        :param name: name of the field
+        :type name: str
+        :return: list of values
+        :rtype: List
+        """
+        values = getattr(self, name) or []
+        if not isinstance(values, list):
+            values = [values]
+        return values

ormar/models/model.py

@@ -107,7 +107,7 @@ class Model(ModelRow):
         await self.signals.post_save.send(sender=self.__class__, instance=self)
         return self
 
-    async def save_related(  # noqa: CCR001
+    async def save_related(  # noqa: CCR001, CFQ002
         self,
         follow: bool = False,
         save_all: bool = False,

ormar/models/modelproxy.py

@@ -2,12 +2,17 @@ from ormar.models.mixins import (
     ExcludableMixin,
     MergeModelMixin,
     PrefetchQueryMixin,
+    PydanticMixin,
     SavePrepareMixin,
 )
 
 
 class ModelTableProxy(
-    PrefetchQueryMixin, MergeModelMixin, SavePrepareMixin, ExcludableMixin
+    PrefetchQueryMixin,
+    MergeModelMixin,
+    SavePrepareMixin,
+    ExcludableMixin,
+    PydanticMixin,
 ):
     """
     Used to combine all mixins with different set of functionalities.

ormar/models/newbasemodel.py

@@ -454,8 +454,9 @@ class NewBaseModel(pydantic.BaseModel, ModelTableProxy, metaclass=ModelMetaclass
         super().update_forward_refs(**localns)
         cls.Meta.requires_ref_update = False
 
-    def _get_related_not_excluded_fields(
-        self, include: Optional[Dict], exclude: Optional[Dict],
+    @staticmethod
+    def _get_not_excluded_fields(
+        fields: Union[List, Set], include: Optional[Dict], exclude: Optional[Dict],
     ) -> List:
         """
         Returns related field names applying on them include and exclude set.
@@ -467,7 +468,7 @@ class NewBaseModel(pydantic.BaseModel, ModelTableProxy, metaclass=ModelMetaclass
         :return:
         :rtype: List of fields with relations that is not excluded
         """
-        fields = [field for field in self.extract_related_names()]
+        fields = [*fields] if not isinstance(fields, list) else fields
         if include:
             fields = [field for field in fields if field in include]
         if exclude:
@@ -519,8 +520,9 @@ class NewBaseModel(pydantic.BaseModel, ModelTableProxy, metaclass=ModelMetaclass
                 continue
         return result
 
+    @classmethod
     def _skip_ellipsis(
-        self, items: Union[Set, Dict, None], key: str, default_return: Any = None
+        cls, items: Union[Set, Dict, None], key: str, default_return: Any = None
     ) -> Union[Set, Dict, None]:
         """
         Helper to traverse the include/exclude dictionaries.
@@ -534,10 +536,11 @@ class NewBaseModel(pydantic.BaseModel, ModelTableProxy, metaclass=ModelMetaclass
         :return: nested value of the items
         :rtype: Union[Set, Dict, None]
         """
-        result = self.get_child(items, key)
+        result = cls.get_child(items, key)
         return result if result is not Ellipsis else default_return
 
-    def _convert_all(self, items: Union[Set, Dict, None]) -> Union[Set, Dict, None]:
+    @staticmethod
+    def _convert_all(items: Union[Set, Dict, None]) -> Union[Set, Dict, None]:
         """
         Helper to convert __all__ pydantic special index to ormar which does not
         support index based exclusions.
@@ -549,7 +552,7 @@ class NewBaseModel(pydantic.BaseModel, ModelTableProxy, metaclass=ModelMetaclass
             return items.get("__all__")
         return items
 
-    def _extract_nested_models(  # noqa: CCR001
+    def _extract_nested_models(  # noqa: CCR001, CFQ002
         self,
         relation_map: Dict,
         dict_instance: Dict,
@@ -573,8 +576,9 @@ class NewBaseModel(pydantic.BaseModel, ModelTableProxy, metaclass=ModelMetaclass
         :return: current model dict with child models converted to dictionaries
         :rtype: Dict
         """
-
-        fields = self._get_related_not_excluded_fields(include=include, exclude=exclude)
+        fields = self._get_not_excluded_fields(
+            fields=self.extract_related_names(), include=include, exclude=exclude
+        )
 
         for field in fields:
             if not relation_map or field not in relation_map:

ormar/models/quick_access_views.py

@@ -26,7 +26,7 @@ quick_access_set = {
     "_extract_nested_models_from_list",
     "_extract_own_model_fields",
     "_extract_related_model_instead_of_field",
-    "_get_related_not_excluded_fields",
+    "_get_not_excluded_fields",
     "_get_value",
     "_init_private_attributes",
     "_is_conversion_to_json_needed",

ormar/relations/relation_proxy.py

@@ -214,7 +214,7 @@ class RelationProxy(Generic[T], list):
             setattr(self._owner, self.field_name, item)
         else:
             setattr(item, relation_name, self._owner)
-            await item.update()
+            await item.upsert()
         await self._owner.signals.post_relation_add.send(
             sender=self._owner.__class__,
             instance=self._owner,

tests/test_fastapi/test_excludes_with_get_pydantic.py

@@ -0,0 +1,129 @@
+import pytest
+import sqlalchemy
+from fastapi import FastAPI
+from starlette.testclient import TestClient
+
+from tests.settings import DATABASE_URL
+from tests.test_inheritance_and_pydantic_generation.test_geting_the_pydantic_models import (
+    Category,
+    Item,
+    MutualA,
+    MutualB,
+    SelfRef,
+    database,
+    metadata,
+)  # type: ignore
+
+app = FastAPI()
+app.state.database = database
+
+
+@app.on_event("startup")
+async def startup() -> None:
+    database_ = app.state.database
+    if not database_.is_connected:
+        await database_.connect()
+
+
+@app.on_event("shutdown")
+async def shutdown() -> None:
+    database_ = app.state.database
+    if database_.is_connected:
+        await database_.disconnect()
+
+
+@pytest.fixture(autouse=True, scope="module")
+def create_test_database():
+    engine = sqlalchemy.create_engine(DATABASE_URL)
+    metadata.create_all(engine)
+    yield
+    metadata.drop_all(engine)
+
+
+async def create_category(category: Category):
+    return await Category(**category.dict()).save()
+
+
+create_category.__annotations__["category"] = Category.get_pydantic(exclude={"id"})
+app.post("/categories/", response_model=Category)(create_category)
+
+
+@app.post(
+    "/selfrefs/",
+    response_model=SelfRef.get_pydantic(exclude={"parent", "children__name"}),
+)
+async def create_selfref(
+    selfref: SelfRef.get_pydantic(exclude={"children__name"}),  # type: ignore
+):
+    selfr = SelfRef(**selfref.dict())
+    await selfr.save()
+    if selfr.children:
+        for child in selfr.children:
+            await child.upsert()
+    return selfr
+
+
+@app.get("/selfrefs/{ref_id}/")
+async def get_selfref(ref_id: int):
+    selfr = await SelfRef.objects.select_related("children").get(id=ref_id)
+    return selfr
+
+
+def test_read_main():
+    client = TestClient(app)
+    with client as client:
+        test_category = dict(name="Foo", id=12)
+        response = client.post("/categories/", json=test_category)
+        assert response.status_code == 200
+        cat = Category(**response.json())
+        assert cat.name == "Foo"
+        assert cat.id == 1
+        assert cat.items == []
+
+        test_selfref = dict(name="test")
+        test_selfref2 = dict(name="test2", parent={"id": 1})
+        test_selfref3 = dict(name="test3", children=[{"name": "aaa"}])
+
+        response = client.post("/selfrefs/", json=test_selfref)
+        assert response.status_code == 200
+        self_ref = SelfRef(**response.json())
+        assert self_ref.id == 1
+        assert self_ref.name == "test"
+        assert self_ref.parent is None
+        assert self_ref.children == []
+
+        response = client.post("/selfrefs/", json=test_selfref2)
+        assert response.status_code == 200
+        self_ref = SelfRef(**response.json())
+        assert self_ref.id == 2
+        assert self_ref.name == "test2"
+        assert self_ref.parent is None
+        assert self_ref.children == []
+
+        response = client.post("/selfrefs/", json=test_selfref3)
+        assert response.status_code == 200
+        self_ref = SelfRef(**response.json())
+        assert self_ref.id == 3
+        assert self_ref.name == "test3"
+        assert self_ref.parent is None
+        assert self_ref.children[0].dict() == {"id": 4}
+
+        response = client.get("/selfrefs/3/")
+        assert response.status_code == 200
+        check_children = SelfRef(**response.json())
+        assert check_children.children[0].dict() == {
+            "children": [],
+            "id": 4,
+            "name": "selfref",
+            "parent": {"id": 3, "name": "test3"},
+        }
+
+        response = client.get("/selfrefs/2/")
+        assert response.status_code == 200
+        check_children = SelfRef(**response.json())
+        assert check_children.dict() == {
+            "children": [],
+            "id": 2,
+            "name": "test2",
+            "parent": {"id": 1},
+        }

tests/test_fastapi/test_inheritance_concrete_fastapi.py

@@ -7,7 +7,7 @@ from fastapi import FastAPI
 from starlette.testclient import TestClient
 
 from tests.settings import DATABASE_URL
-from tests.test_inheritance.test_inheritance_concrete import (  # type: ignore
+from tests.test_inheritance_and_pydantic_generation.test_inheritance_concrete import (  # type: ignore
     Category,
     Subject,
     Person,

tests/test_fastapi/test_inheritance_mixins_fastapi.py

@@ -6,7 +6,7 @@ from fastapi import FastAPI
 from starlette.testclient import TestClient
 
 from tests.settings import DATABASE_URL
-from tests.test_inheritance.test_inheritance_mixins import Category, Subject, metadata, db as database  # type: ignore
+from tests.test_inheritance_and_pydantic_generation.test_inheritance_mixins import Category, Subject, metadata, db as database  # type: ignore
 
 app = FastAPI()
 app.state.database = database

tests/test_inheritance_and_pydantic_generation/test_excluding_parent_fields_inheritance.py

@@ -0,0 +1,147 @@
+import datetime
+
+import databases
+import pytest
+import sqlalchemy as sa
+from sqlalchemy import create_engine
+
+import ormar
+from tests.settings import DATABASE_URL
+
+metadata = sa.MetaData()
+db = databases.Database(DATABASE_URL)
+engine = create_engine(DATABASE_URL)
+
+
+class User(ormar.Model):
+    class Meta(ormar.ModelMeta):
+        tablename = "users"
+        metadata = metadata
+        database = db
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=50, unique=True, index=True)
+
+
+class RelationalAuditModel(ormar.Model):
+    class Meta:
+        abstract = True
+
+    created_by: User = ormar.ForeignKey(User, nullable=False)
+    updated_by: User = ormar.ForeignKey(User, nullable=False)
+
+
+class AuditModel(ormar.Model):
+    class Meta:
+        abstract = True
+
+    created_by: str = ormar.String(max_length=100)
+    updated_by: str = ormar.String(max_length=100, default="Sam")
+
+
+class DateFieldsModel(ormar.Model):
+    class Meta(ormar.ModelMeta):
+        abstract = True
+        metadata = metadata
+        database = db
+
+    created_date: datetime.datetime = ormar.DateTime(
+        default=datetime.datetime.now, name="creation_date"
+    )
+    updated_date: datetime.datetime = ormar.DateTime(
+        default=datetime.datetime.now, name="modification_date"
+    )
+
+
+class Category(DateFieldsModel, AuditModel):
+    class Meta(ormar.ModelMeta):
+        tablename = "categories"
+        exclude_parent_fields = ["updated_by", "updated_date"]
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=50, unique=True, index=True)
+    code: int = ormar.Integer()
+
+
+class Item(DateFieldsModel, AuditModel):
+    class Meta(ormar.ModelMeta):
+        tablename = "items"
+        exclude_parent_fields = ["updated_by", "updated_date"]
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=50, unique=True, index=True)
+    code: int = ormar.Integer()
+    updated_by: str = ormar.String(max_length=100, default="Bob")
+
+
+class Gun(RelationalAuditModel, DateFieldsModel):
+    class Meta(ormar.ModelMeta):
+        tablename = "guns"
+        exclude_parent_fields = ["updated_by"]
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=50)
+
+
+@pytest.fixture(autouse=True, scope="module")
+def create_test_database():
+    metadata.create_all(engine)
+    yield
+    metadata.drop_all(engine)
+
+
+def test_model_definition():
+    model_fields = Category.Meta.model_fields
+    sqlalchemy_columns = Category.Meta.table.c
+    pydantic_columns = Category.__fields__
+    assert "updated_by" not in model_fields
+    assert "updated_by" not in sqlalchemy_columns
+    assert "updated_by" not in pydantic_columns
+    assert "updated_date" not in model_fields
+    assert "updated_date" not in sqlalchemy_columns
+    assert "updated_date" not in pydantic_columns
+
+    assert "updated_by" not in Gun.Meta.model_fields
+    assert "updated_by" not in Gun.Meta.table.c
+    assert "updated_by" not in Gun.__fields__
+
+
+@pytest.mark.asyncio
+async def test_model_works_as_expected():
+    async with db:
+        async with db.transaction(force_rollback=True):
+            test = await Category(name="Cat", code=2, created_by="Joe").save()
+            assert test.created_date is not None
+
+            test2 = await Category.objects.get(pk=test.pk)
+            assert test2.name == "Cat"
+            assert test2.created_by == "Joe"
+
+
+@pytest.mark.asyncio
+async def test_exclude_with_redefinition():
+    async with db:
+        async with db.transaction(force_rollback=True):
+            test = await Item(name="Item", code=3, created_by="Anna").save()
+            assert test.created_date is not None
+            assert test.updated_by == "Bob"
+
+            test2 = await Item.objects.get(pk=test.pk)
+            assert test2.name == "Item"
+            assert test2.code == 3
+
+
+@pytest.mark.asyncio
+async def test_exclude_with_relation():
+    async with db:
+        async with db.transaction(force_rollback=True):
+            user = await User(name="Michail Kalasznikow").save()
+            test = await Gun(name="AK47", created_by=user).save()
+            assert test.created_date is not None
+
+            with pytest.raises(AttributeError):
+                assert test.updated_by
+
+            test2 = await Gun.objects.select_related("created_by").get(pk=test.pk)
+            assert test2.name == "AK47"
+            assert test2.created_by.name == "Michail Kalasznikow"

tests/test_inheritance_and_pydantic_generation/test_geting_the_pydantic_models.py

@@ -0,0 +1,202 @@
+from typing import List, Optional
+
+import databases
+import pydantic
+import sqlalchemy
+from pydantic import ConstrainedStr
+from pydantic.typing import ForwardRef
+
+import ormar
+from tests.settings import DATABASE_URL
+
+metadata = sqlalchemy.MetaData()
+database = databases.Database(DATABASE_URL, force_rollback=True)
+
+
+class BaseMeta(ormar.ModelMeta):
+    metadata = metadata
+    database = database
+
+
+class SelfRef(ormar.Model):
+    class Meta(BaseMeta):
+        tablename = "self_refs"
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=100, default="selfref")
+    parent = ormar.ForeignKey(ForwardRef("SelfRef"), related_name="children")
+
+
+SelfRef.update_forward_refs()
+
+
+class Category(ormar.Model):
+    class Meta(BaseMeta):
+        tablename = "categories"
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=100)
+
+
+class Item(ormar.Model):
+    class Meta(BaseMeta):
+        pass
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=100, default="test")
+    category: Optional[Category] = ormar.ForeignKey(Category, nullable=True)
+
+
+class MutualA(ormar.Model):
+    class Meta(BaseMeta):
+        tablename = "mutual_a"
+
+    id: int = ormar.Integer(primary_key=True)
+    mutual_b = ormar.ForeignKey(ForwardRef("MutualB"), related_name="mutuals_a")
+
+
+class MutualB(ormar.Model):
+    class Meta(BaseMeta):
+        tablename = "mutual_b"
+
+    id: int = ormar.Integer(primary_key=True)
+    mutual_a = ormar.ForeignKey(MutualA, related_name="mutuals_b")
+
+
+MutualA.update_forward_refs()
+
+
+def test_getting_pydantic_model():
+    PydanticCategory = Category.get_pydantic()
+    assert issubclass(PydanticCategory, pydantic.BaseModel)
+    assert {*PydanticCategory.__fields__.keys()} == {"items", "id", "name"}
+
+    assert not PydanticCategory.__fields__["id"].required
+    assert PydanticCategory.__fields__["id"].outer_type_ == int
+    assert PydanticCategory.__fields__["id"].default is None
+
+    assert PydanticCategory.__fields__["name"].required
+    assert issubclass(PydanticCategory.__fields__["name"].outer_type_, ConstrainedStr)
+    assert PydanticCategory.__fields__["name"].default in [None, Ellipsis]
+
+    PydanticItem = PydanticCategory.__fields__["items"].type_
+    assert PydanticCategory.__fields__["items"].outer_type_ == List[PydanticItem]
+    assert issubclass(PydanticItem, pydantic.BaseModel)
+    assert not PydanticItem.__fields__["name"].required
+    assert PydanticItem.__fields__["name"].default == "test"
+    assert issubclass(PydanticItem.__fields__["name"].outer_type_, ConstrainedStr)
+    assert "category" not in PydanticItem.__fields__
+
+
+def test_initializing_pydantic_model():
+    data = {
+        "id": 1,
+        "name": "test",
+        "items": [{"id": 1, "name": "test_i1"}, {"id": 2, "name": "test_i2"}],
+    }
+    PydanticCategory = Category.get_pydantic()
+    cat = PydanticCategory(**data)
+    assert cat.dict() == data
+
+    data = {"id": 1, "name": "test"}
+    cat = PydanticCategory(**data)
+    assert cat.dict() == {**data, "items": None}
+
+
+def test_getting_pydantic_model_include():
+    PydanticCategory = Category.get_pydantic(include={"id", "name"})
+    assert len(PydanticCategory.__fields__) == 2
+    assert "items" not in PydanticCategory.__fields__
+
+
+def test_getting_pydantic_model_nested_include_set():
+    PydanticCategory = Category.get_pydantic(include={"id", "items__id"})
+    assert len(PydanticCategory.__fields__) == 2
+    assert "name" not in PydanticCategory.__fields__
+    PydanticItem = PydanticCategory.__fields__["items"].type_
+    assert len(PydanticItem.__fields__) == 1
+    assert "id" in PydanticItem.__fields__
+
+
+def test_getting_pydantic_model_nested_include_dict():
+    PydanticCategory = Category.get_pydantic(include={"id": ..., "items": {"id"}})
+    assert len(PydanticCategory.__fields__) == 2
+    assert "name" not in PydanticCategory.__fields__
+    PydanticItem = PydanticCategory.__fields__["items"].type_
+    assert len(PydanticItem.__fields__) == 1
+    assert "id" in PydanticItem.__fields__
+
+
+def test_getting_pydantic_model_nested_include_nested_dict():
+    PydanticCategory = Category.get_pydantic(include={"id": ..., "items": {"id": ...}})
+    assert len(PydanticCategory.__fields__) == 2
+    assert "name" not in PydanticCategory.__fields__
+    PydanticItem = PydanticCategory.__fields__["items"].type_
+    assert len(PydanticItem.__fields__) == 1
+    assert "id" in PydanticItem.__fields__
+
+
+def test_getting_pydantic_model_include_exclude():
+    PydanticCategory = Category.get_pydantic(
+        include={"id": ..., "items": {"id", "name"}}, exclude={"items__name"}
+    )
+    assert len(PydanticCategory.__fields__) == 2
+    assert "name" not in PydanticCategory.__fields__
+    PydanticItem = PydanticCategory.__fields__["items"].type_
+    assert len(PydanticItem.__fields__) == 1
+    assert "id" in PydanticItem.__fields__
+
+
+def test_getting_pydantic_model_exclude():
+    PydanticItem = Item.get_pydantic(exclude={"category__name"})
+    assert len(PydanticItem.__fields__) == 3
+    assert "category" in PydanticItem.__fields__
+    PydanticCategory = PydanticItem.__fields__["category"].type_
+    assert len(PydanticCategory.__fields__) == 1
+    assert "name" not in PydanticCategory.__fields__
+
+
+def test_getting_pydantic_model_exclude_dict():
+    PydanticItem = Item.get_pydantic(exclude={"id": ..., "category": {"name"}})
+    assert len(PydanticItem.__fields__) == 2
+    assert "category" in PydanticItem.__fields__
+    assert "id" not in PydanticItem.__fields__
+    PydanticCategory = PydanticItem.__fields__["category"].type_
+    assert len(PydanticCategory.__fields__) == 1
+    assert "name" not in PydanticCategory.__fields__
+
+
+def test_getting_pydantic_model_self_ref():
+    PydanticSelfRef = SelfRef.get_pydantic()
+    assert len(PydanticSelfRef.__fields__) == 4
+    assert set(PydanticSelfRef.__fields__.keys()) == {
+        "id",
+        "name",
+        "parent",
+        "children",
+    }
+    InnerSelf = PydanticSelfRef.__fields__["parent"].type_
+    assert len(InnerSelf.__fields__) == 2
+    assert set(InnerSelf.__fields__.keys()) == {"id", "name"}
+
+    InnerSelf2 = PydanticSelfRef.__fields__["children"].type_
+    assert len(InnerSelf2.__fields__) == 2
+    assert set(InnerSelf2.__fields__.keys()) == {"id", "name"}
+
+    assert InnerSelf2 != InnerSelf
+
+
+def test_getting_pydantic_model_mutual_rels():
+    MutualAPydantic = MutualA.get_pydantic()
+    assert len(MutualAPydantic.__fields__) == 3
+    assert set(MutualAPydantic.__fields__.keys()) == {"id", "mutual_b", "mutuals_b"}
+
+    MutualB1 = MutualAPydantic.__fields__["mutual_b"].type_
+    MutualB2 = MutualAPydantic.__fields__["mutuals_b"].type_
+    assert MutualB1 != MutualB2
+
+    assert len(MutualB1.__fields__) == 1
+    assert "id" in MutualB1.__fields__
+
+    assert len(MutualB2.__fields__) == 1
+    assert "id" in MutualB2.__fields__

tests/test_inheritance_and_pydantic_generation/test_inheritance_concrete.py

@@ -121,13 +121,6 @@ class Bus(Car):
     max_persons: int = ormar.Integer()
 
 
-# class PersonsCar(ormar.Model):
-#     class Meta:
-#         tablename = "cars_x_persons"
-#         metadata = metadata
-#         database = db
-
-
 class Car2(ormar.Model):
     class Meta:
         abstract = True
@@ -138,9 +131,7 @@ class Car2(ormar.Model):
     name: str = ormar.String(max_length=50)
     owner: Person = ormar.ForeignKey(Person, related_name="owned")
     co_owners: List[Person] = ormar.ManyToMany(
-        Person,
-        # through=PersonsCar,
-        related_name="coowned",
+        Person, related_name="coowned",
     )
     created_date: datetime.datetime = ormar.DateTime(default=datetime.datetime.now)
 

tests/test_queries/test_adding_related.py

@@ -0,0 +1,54 @@
+from typing import Optional
+
+import databases
+import pytest
+import sqlalchemy
+import asyncio
+
+import ormar
+
+from tests.settings import DATABASE_URL
+
+database = databases.Database(DATABASE_URL)
+metadata = sqlalchemy.MetaData()
+
+
+class Department(ormar.Model):
+    class Meta:
+        database = database
+        metadata = metadata
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=100)
+
+
+class Course(ormar.Model):
+    class Meta:
+        database = database
+        metadata = metadata
+
+    id: int = ormar.Integer(primary_key=True)
+    name: str = ormar.String(max_length=100)
+    completed: bool = ormar.Boolean(default=False)
+    department: Optional[Department] = ormar.ForeignKey(Department)
+
+
+@pytest.fixture(autouse=True, scope="module")
+def create_test_database():
+    engine = sqlalchemy.create_engine(DATABASE_URL)
+    metadata.drop_all(engine)
+    metadata.create_all(engine)
+    yield
+    metadata.drop_all(engine)
+
+
+@pytest.mark.asyncio
+async def test_adding_relation_to_reverse_saves_the_child():
+    async with database:
+        department = await Department(name="Science").save()
+        course = Course(name="Math", completed=False)
+
+        await department.courses.add(course)
+        assert course.pk is not None
+        assert course.department == department
+        assert department.courses[0] == course

tests/test_queries/test_isnull_filter.py

@@ -68,8 +68,10 @@ async def test_is_null():
         assert tolkien.books[0].year is None
         assert tolkien.books[0].title == "The Hobbit"
 
-        tolkien = await Author.objects.select_related("books").paginate(1, 10).get(
-            books__year__isnull=True
+        tolkien = (
+            await Author.objects.select_related("books")
+            .paginate(1, 10)
+            .get(books__year__isnull=True)
         )
         assert len(tolkien.books) == 1
         assert tolkien.books[0].year is None