add tests for cross model forward references, add docs for processing forwardrefs, wip on refactoring queries into separate pages based on functionality

2021-01-26 17:29:40 +01:00
parent a2834666fc
commit b710ed9780
39 changed files with 2054 additions and 1004 deletions
--- a/docs/queries/aggregations.md
+++ b/docs/queries/aggregations.md
@ -0,0 +1,28 @@
+# Aggregation functions
+
+`ormar` currently supports 2 aggregation functions:
+
+*  `count() -> int`
+*  `exists() -> bool`
+
+## count
+
+`count() -> int`
+
+Returns number of rows matching the given criteria (i.e. applied with `filter` and `exclude`)
+
+```python
+# returns count of rows in db for Books model
+no_of_books = await Book.objects.count()
+```
+
+## exists
+
+`exists() -> bool`
+
+Returns a bool value to confirm if there are rows matching the given criteria (applied with `filter` and `exclude`)
+
+```python
+# returns a boolean value if given row exists
+has_sample = await Book.objects.filter(title='Sample').exists()
+```
--- a/docs/queries/create.md
+++ b/docs/queries/create.md
@ -0,0 +1,83 @@
+# Create / Insert data into database
+
+* `create(**kwargs): -> Model`
+* `get_or_create(**kwargs) -> Model`
+* `update_or_create(**kwargs) -> Model`
+* `bulk_create(objects: List[Model]) -> None`
+* `Model.save()` method
+* `Model.upsert()` method
+
+## create
+
+`create(**kwargs): -> Model`
+
+Creates the model instance, saves it in a database and returns the updates model
+(with pk populated if not passed and autoincrement is set).
+
+The allowed kwargs are `Model` fields names and proper value types.
+
+```python
+malibu = await Album.objects.create(name="Malibu")
+await Track.objects.create(album=malibu, title="The Bird", position=1)
+```
+
+The alternative is a split creation and persistence of the `Model`.
+
+```python
+malibu = Album(name="Malibu")
+await malibu.save()
+```
+
+!!!tip Check other `Model` methods in [models][models]
+
+## get_or_create
+
+`get_or_create(**kwargs) -> Model`
+
+Combination of create and get methods.
+
+Tries to get a row meeting the criteria and if `NoMatch` exception is raised it creates
+a new one with given kwargs.
+
+```python
+album = await Album.objects.get_or_create(name='The Cat')
+# object is created as it does not exist
+album2 = await Album.objects.get_or_create(name='The Cat')
+assert album == album2
+# return True as the same db row is returned
+```
+
+!!!warning Despite being a equivalent row from database the `album` and `album2` in
+example above are 2 different python objects!
+Updating one of them will not refresh the second one until you excplicitly load() the
+fresh data from db.
+
+!!!note Note that if you want to create a new object you either have to pass pk column
+value or pk column has to be set as autoincrement
+
+## update_or_create
+
+`update_or_create(**kwargs) -> Model`
+
+Updates the model, or in case there is no match in database creates a new one.
+
+```Python hl_lines="26-32"
+--8<-- "../docs_src/queries/docs003.py"
+```
+
+!!!note Note that if you want to create a new object you either have to pass pk column
+value or pk column has to be set as autoincrement
+
+## bulk_create
+
+`bulk_create(objects: List["Model"]) -> None`
+
+Allows you to create multiple objects at once.
+
+A valid list of `Model` objects needs to be passed.
+
+```python hl_lines="21-27"
+--8<-- "../docs_src/queries/docs004.py"
+```
+
+## Model method
--- a/docs/queries/delete.md
+++ b/docs/queries/delete.md
@ -0,0 +1,23 @@
+# Delete/ remove data from database
+
+* `delete(each: bool = False, **kwargs) -> int`
+* `Model.delete()` method
+
+## delete
+
+`delete(each: bool = False, **kwargs) -> int`
+
+QuerySet level delete is used to delete multiple records at once.
+
+You either have to filter the QuerySet first or provide a `each=True` flag to delete
+whole table.
+
+If you do not provide this flag or a filter a `QueryDefinitionError` will be raised.
+
+Return number of rows deleted.
+
+```python hl_lines="26-30"
+--8<-- "../docs_src/queries/docs005.py"
+```
+
+## Model method
--- a/docs/queries/filter-and-sort.md
+++ b/docs/queries/filter-and-sort.md
@ -0,0 +1,151 @@
+# Filtering and sorting data
+
+*  `filter(**kwargs) -> QuerySet`
+*  `exclude(**kwargs) -> QuerySet`
+*  `order_by(columns:Union[List, str]) -> QuerySet`
+
+## filter
+
+`filter(**kwargs) -> QuerySet`
+
+Allows you to filter by any `Model` attribute/field as well as to fetch instances, with
+a filter across an FK relationship.
+
+```python
+track = Track.objects.filter(name="The Bird").get()
+# will return a track with name equal to 'The Bird'
+
+tracks = Track.objects.filter(album__name="Fantasies").all()
+# will return all tracks where the columns album name = 'Fantasies'
+```
+
+You can use special filter suffix to change the filter operands:
+
+* exact - like `album__name__exact='Malibu'` (exact match)
+* iexact - like `album__name__iexact='malibu'` (exact match case insensitive)
+* contains - like `album__name__contains='Mal'` (sql like)
+* icontains - like `album__name__icontains='mal'` (sql like case insensitive)
+* in - like `album__name__in=['Malibu', 'Barclay']` (sql in)
+* gt - like `position__gt=3` (sql >)
+* gte - like `position__gte=3` (sql >=)
+* lt - like `position__lt=3` (sql <)
+* lte - like `position__lte=3` (sql <=)
+* startswith - like `album__name__startswith='Mal'` (exact start match)
+* istartswith - like `album__name__istartswith='mal'` (exact start match case
+  insensitive)
+* endswith - like `album__name__endswith='ibu'` (exact end match)
+* iendswith - like `album__name__iendswith='IBU'` (exact end match case insensitive)
+
+!!!note All methods that do not return the rows explicitly returns a QueySet instance so
+you can chain them together
+
+    So operations like `filter()`, `select_related()`, `limit()` and `offset()` etc. can be chained.
+    
+    Something like `Track.object.select_related("album").filter(album__name="Malibu").offset(1).limit(1).all()`
+
+!!!warning Note that you do not have to specify the `%` wildcard in contains and other
+filters, it's added for you. If you include `%` in your search value it will be escaped
+and treated as literal percentage sign inside the text.
+
+### exclude
+
+`exclude(**kwargs) -> QuerySet`
+
+Works exactly the same as filter and all modifiers (suffixes) are the same, but returns
+a not condition.
+
+So if you use `filter(name='John')` which equals to `where name = 'John'` in SQL,
+the `exclude(name='John')` equals to `where name <> 'John'`
+
+Note that all conditions are joined so if you pass multiple values it becomes a union of
+conditions.
+
+`exclude(name='John', age>=35)` will become `where not (name='John' and age>=35)`
+
+```python
+notes = await Track.objects.exclude(position_gt=3).all()
+# returns all tracks with position < 3
+```
+
+### order_by
+
+`order_by(columns: Union[List, str]) -> QuerySet`
+
+With `order_by()` you can order the results from database based on your choice of
+fields.
+
+You can provide a string with field name or list of strings with different fields.
+
+Ordering in sql will be applied in order of names you provide in order_by.
+
+!!!tip By default if you do not provide ordering `ormar` explicitly orders by all
+primary keys
+
+!!!warning If you are sorting by nested models that causes that the result rows are
+unsorted by the main model
+`ormar` will combine those children rows into one main model.
+
+    Sample raw database rows result (sort by child model desc):
+    ```
+    MODEL: 1 - Child Model - 3
+    MODEL: 2 - Child Model - 2
+    MODEL: 1 - Child Model - 1
+    ```
+    
+    will result in 2 rows of result:
+    ```
+    MODEL: 1 - Child Models: [3, 1] # encountered first in result, all children rows combined
+    MODEL: 2 - Child Modles: [2]
+    ```
+    
+    The main model will never duplicate in the result
+
+Given sample Models like following:
+
+```python
+--8 < -- "../docs_src/queries/docs007.py"
+```
+
+To order by main model field just provide a field name
+
+```python
+toys = await Toy.objects.select_related("owner").order_by("name").all()
+assert [x.name.replace("Toy ", "") for x in toys] == [
+    str(x + 1) for x in range(6)
+]
+assert toys[0].owner == zeus
+assert toys[1].owner == aphrodite
+```
+
+To sort on nested models separate field names with dunder '__'.
+
+You can sort this way across all relation types -> `ForeignKey`, reverse virtual FK
+and `ManyToMany` fields.
+
+```python
+toys = await Toy.objects.select_related("owner").order_by("owner__name").all()
+assert toys[0].owner.name == toys[1].owner.name == "Aphrodite"
+assert toys[2].owner.name == toys[3].owner.name == "Hermes"
+assert toys[4].owner.name == toys[5].owner.name == "Zeus"
+```
+
+To sort in descending order provide a hyphen in front of the field name
+
+```python
+owner = (
+    await Owner.objects.select_related("toys")
+        .order_by("-toys__name")
+        .filter(name="Zeus")
+        .get()
+)
+assert owner.toys[0].name == "Toy 4"
+assert owner.toys[1].name == "Toy 1"
+```
+
+!!!note All methods that do not return the rows explicitly returns a QueySet instance so
+you can chain them together
+
+    So operations like `filter()`, `select_related()`, `limit()` and `offset()` etc. can be chained.
+    
+    Something like `Track.object.select_related("album").filter(album__name="Malibu").offset(1).limit(1).all()`
+
--- a/docs/queries/index.md
+++ b/docs/queries/index.md
@ -0,0 +1,156 @@
+# Querying database with ormar
+
+## QuerySet
+
+Each Model is auto registered with a `QuerySet` that represents the underlying query,
+and it's options.
+
+Most of the methods are also available through many to many relations and on reverse
+foreign key relations through `QuerysetProxy` interface.
+
+!!!info To see which one are supported and how to construct relations
+visit [relations][relations].
+
+For simplicity available methods to fetch and save the data into the database are
+divided into categories according to the function they fulfill.
+
+Note that some functions/methods are in multiple categories.
+
+For complicity also Models and relations methods are listed.
+
+To read more about any specific section or function please refer to the details subpage.
+
+### Create
+
+* `create(**kwargs) -> Model`
+* `get_or_create(**kwargs) -> Model`
+* `update_or_create(**kwargs) -> Model`
+* `bulk_create(objects: List[Model]) -> None`
+
+
+* `Model`
+  * `Model.save()` method
+  * `Model.upsert()` method
+  * `Model.save_related()` method
+
+
+* `QuerysetProxy`
+  * `QuerysetProxy.create(**kwargs)` method
+  * `QuerysetProxy.get_or_create(**kwargs)` method
+  * `QuerysetProxy.update_or_create(**kwargs)` method
+
+### Read
+
+* `get(**kwargs) -> Model`
+* `get_or_create(**kwargs) -> Model`
+* `first() -> Model`
+* `all(**kwargs) -> List[Optional[Model]]`
+
+
+* `Model`
+    * `Model.load()` method
+
+
+* `QuerysetProxy`
+    * `QuerysetProxy.get(**kwargs)` method
+    * `QuerysetProxy.get_or_create(**kwargs)` method
+    * `QuerysetProxy.first()` method
+    * `QuerysetProxy.all(**kwargs)` method
+
+### Update
+
+* `update(each: bool = False, **kwargs) -> int`
+* `update_or_create(**kwargs) -> Model`
+* `bulk_update(objects: List[Model], columns: List[str] = None) -> None`
+
+
+* `Model`
+    * `Model.update()` method
+    * `Model.upsert()` method
+    * `Model.save_related()` method
+
+
+* `QuerysetProxy`
+    * `QuerysetProxy.update_or_create(**kwargs)` method
+
+### Delete
+
+* `delete(each: bool = False, **kwargs) -> int`
+
+
+* `Model`
+    * `Model.delete()` method
+
+
+* `QuerysetProxy`
+    * `QuerysetProxy.remove()` method
+    * `QuerysetProxy.clear()` method
+
+### Joins and subqueries
+
+* `select_related(related: Union[List, str]) -> QuerySet`
+* `prefetch_related(related: Union[List, str]) -> QuerySet`
+
+
+* `Model`
+    * `Model.load()` method
+
+
+* `QuerysetProxy`
+    * `QuerysetProxy.select_related(related: Union[List, str])` method
+    * `QuerysetProxy.prefetch_related(related: Union[List, str])` method
+
+### Filtering and sorting
+
+* `filter(**kwargs) -> QuerySet`
+* `exclude(**kwargs) -> QuerySet`
+* `order_by(columns:Union[List, str]) -> QuerySet`
+* `get(**kwargs) -> Model`
+* `get_or_create(**kwargs) -> Model`
+* `all(**kwargs) -> List[Optional[Model]]`
+
+
+* `QuerysetProxy`
+    * `QuerysetProxy.filter(**kwargs)` method
+    * `QuerysetProxy.exclude(**kwargs)` method
+    * `QuerysetProxy.order_by(columns:Union[List, str])` method
+    * `QuerysetProxy.get(**kwargs)` method
+    * `QuerysetProxy.get_or_create(**kwargs)` method
+    * `QuerysetProxy.all(**kwargs)` method
+
+### Selecting columns
+
+* `fields(columns: Union[List, str, set, dict]) -> QuerySet`
+* `exclude_fields(columns: Union[List, str, set, dict]) -> QuerySet`
+
+
+* `QuerysetProxy`
+    * `QuerysetProxy.fields(columns: Union[List, str, set, dict])` method
+    * `QuerysetProxy.exclude_fields(columns: Union[List, str, set, dict])` method
+
+### Pagination and rows number
+
+* `paginate(page: int) -> QuerySet`
+* `limit(limit_count: int) -> QuerySet`
+* `offset(offset: int) -> QuerySet`
+* `get() -> Model`
+* `first() -> Model`
+
+
+* `QuerysetProxy`
+    * `QuerysetProxy.paginate(page: int)` method
+    * `QuerysetProxy.limit(limit_count: int)` method
+    * `QuerysetProxy.offset(offset: int)` method
+
+### Aggregated functions
+
+* `count() -> int`
+* `exists() -> bool`
+
+
+* `QuerysetProxy`
+    * `QuerysetProxy.count()` method
+    * `QuerysetProxy.exists()` method
+  
+
+[relations]: ./relations/index.md
--- a/docs/queries/joins-and-subqueries.md
+++ b/docs/queries/joins-and-subqueries.md
@ -0,0 +1,223 @@
+# Joins and subqueries
+
+
+
+## select_related
+
+`select_related(related: Union[List, str]) -> QuerySet`
+
+Allows to prefetch related models during the same query.
+
+**With `select_related` always only one query is run against the database**, meaning
+that one (sometimes complicated) join is generated and later nested models are processed in
+python.
+
+To fetch related model use `ForeignKey` names.
+
+To chain related `Models` relation use double underscores between names.
+
+!!!note 
+    If you are coming from `django` note that `ormar` `select_related` differs ->
+    in `django` you can `select_related`
+    only singe relation types, while in `ormar` you can select related across `ForeignKey`
+    relation, reverse side of `ForeignKey` (so virtual auto generated keys) and `ManyToMany`
+    fields (so all relations as of current version).
+
+!!!tip 
+    To control which model fields to select use `fields()`
+    and `exclude_fields()` `QuerySet` methods.
+
+!!!tip 
+    To control order of models (both main or nested) use `order_by()` method.
+
+```python
+album = await Album.objects.select_related("tracks").all()
+# will return album will all columns tracks
+```
+
+You can provide a string or a list of strings
+
+```python
+classes = await SchoolClass.objects.select_related(
+    ["teachers__category", "students"]).all()
+# will return classes with teachers and teachers categories
+# as well as classes students
+```
+
+Exactly the same behavior is for Many2Many fields, where you put the names of Many2Many
+fields and the final `Models` are fetched for you.
+
+!!!warning 
+    If you set `ForeignKey` field as not nullable (so required) during all
+    queries the not nullable `Models` will be auto prefetched, even if you do not include
+    them in select_related.
+
+!!!note 
+    All methods that do not return the rows explicitly returns a QueySet instance so
+    you can chain them together
+
+    So operations like `filter()`, `select_related()`, `limit()` and `offset()` etc. can be chained.
+    
+    Something like `Track.object.select_related("album").filter(album__name="Malibu").offset(1).limit(1).all()`
+
+## prefetch_related
+
+`prefetch_related(related: Union[List, str]) -> QuerySet`
+
+Allows to prefetch related models during query - but opposite to `select_related` each
+subsequent model is fetched in a separate database query.
+
+**With `prefetch_related` always one query per Model is run against the database**,
+meaning that you will have multiple queries executed one after another.
+
+To fetch related model use `ForeignKey` names.
+
+To chain related `Models` relation use double underscores between names.
+
+!!!tip 
+    To control which model fields to select use `fields()`
+    and `exclude_fields()` `QuerySet` methods.
+
+!!!tip 
+    To control order of models (both main or nested) use `order_by()` method.
+
+```python
+album = await Album.objects.prefetch_related("tracks").all()
+# will return album will all columns tracks
+```
+
+You can provide a string or a list of strings
+
+```python
+classes = await SchoolClass.objects.prefetch_related(
+    ["teachers__category", "students"]).all()
+# will return classes with teachers and teachers categories
+# as well as classes students
+```
+
+Exactly the same behavior is for Many2Many fields, where you put the names of Many2Many
+fields and the final `Models` are fetched for you.
+
+!!!warning 
+    If you set `ForeignKey` field as not nullable (so required) during all
+    queries the not nullable `Models` will be auto prefetched, even if you do not include
+    them in select_related.
+
+!!!note 
+    All methods that do not return the rows explicitly returns a QueySet instance so
+    you can chain them together
+
+    So operations like `filter()`, `select_related()`, `limit()` and `offset()` etc. can be chained.
+    
+    Something like `Track.object.select_related("album").filter(album__name="Malibu").offset(1).limit(1).all()`
+
+## select_related vs prefetch_related
+
+Which should you use -> `select_related` or `prefetch_related`?
+
+Well, it really depends on your data. The best answer is try yourself and see which one
+performs faster/better in your system constraints.
+
+What to keep in mind:
+
+### Performance
+
+**Number of queries**:
+`select_related` always executes one query against the database,
+while `prefetch_related` executes multiple queries. Usually the query (I/O) operation is
+the slowest one but it does not have to be.
+
+**Number of rows**:
+Imagine that you have 10 000 object in one table A and each of those objects have 3
+children in table B, and subsequently each object in table B has 2 children in table C.
+Something like this:
+
+```
+                     Model C
+                   /
+           Model B - Model C
+         / 
+Model A  - Model B - Model C
+       \           \ 
+        \            Model C
+         \
+           Model B - Model C
+                   \ 
+                     Model C
+```
+
+That means that `select_related` will always return 60 000 rows (10 000 * 3 * 2) later
+compacted to 10 000 models.
+
+How many rows will return `prefetch_related`?
+
+Well, that depends, if each of models B and C is unique it will return 10 000 rows in
+first query, 30 000 rows
+(each of 3 children of A in table B are unique) in second query and 60 000 rows (each of
+2 children of model B in table C are unique) in 3rd query.
+
+In this case `select_related` seems like a better choice, not only it will run one query
+comparing to 3 of
+`prefetch_related` but will also return 60 000 rows comparing to 100 000
+of `prefetch_related` (10+30+60k).
+
+But what if each Model A has exactly the same 3 models B and each models C has exactly
+same models C? `select_related`
+will still return 60 000 rows, while `prefetch_related` will return 10 000 for model A,
+3 rows for model B and 2 rows for Model C. So in total 10 006 rows. Now depending on the
+structure of models (i.e. if it has long Text() fields etc.) `prefetch_related`
+might be faster despite it needs to perform three separate queries instead of one.
+
+#### Memory
+
+`ormar` is a mini ORM meaning that it does not keep a registry of already loaded models.
+
+That means that in `select_related` example above you will always have 10 000 Models A,
+30 000 Models B
+(even if the unique number of rows in db is 3 - processing of `select_related` spawns **
+new** child models for each parent model). And 60 000 Models C.
+
+If the same Model B is shared by rows 1, 10, 100 etc. and you update one of those, the
+rest of rows that share the same child will **not** be updated on the spot. If you
+persist your changes into the database the change **will be available only after reload
+(either each child separately or the whole query again)**. That means
+that `select_related` will use more memory as each child is instantiated as a new object
+- obviously using it's own space.
+
+!!!note 
+    This might change in future versions if we decide to introduce caching.
+
+!!!warning 
+    By default all children (or event the same models loaded 2+ times) are
+    completely independent, distinct python objects, despite that they represent the same
+    row in db.
+
+    They will evaluate to True when compared, so in example above: 
+    
+    ```python
+    # will return True if child1 of both rows is the same child db row 
+    row1.child1 == row100.child1
+    
+    # same here:
+    model1 = await Model.get(pk=1)
+    model2 = await Model.get(pk=1) # same pk = same row in db
+    # will return `True`
+    model1 == model2
+    ``` 
+    
+    but 
+    
+    ```python
+    # will return False (note that id is a python `builtin` function not ormar one).
+    id(row1.child1) == (ro100.child1)
+    
+    # from above - will also return False
+    id(model1) == id(model2)
+    ``` 
+
+On the contrary - with `prefetch_related` each unique distinct child model is
+instantiated only once and the same child models is shared across all parent models.
+That means that in `prefetch_related` example above if there are 3 distinct models in
+table B and 2 in table C, there will be only 5 children nested models shared between all
+model A instances. That also means that if you update any attribute it will be updated
+on all parents as they share the same child object.
--- a/docs/queries/pagination-and-rows-number.md
+++ b/docs/queries/pagination-and-rows-number.md
@ -0,0 +1,94 @@
+#Pagination and rows number
+
+*  `paginate(page: int) -> QuerySet`
+*  `limit(limit_count: int) -> QuerySet`
+*  `offset(offset: int) -> QuerySet`
+*  `get(**kwargs): -> Model`
+*  `first(): -> Model`
+
+
+## paginate
+
+`paginate(page: int, page_size: int = 20) -> QuerySet`
+
+Combines the `offset` and `limit` methods based on page number and size
+
+```python
+tracks = await Track.objects.paginate(3).all()
+# will return 20 tracks starting at row 41 
+# (with default page size of 20)
+```
+
+Note that `paginate(2)` is equivalent to `offset(20).limit(20)`
+
+## limit
+
+`limit(limit_count: int, limit_raw_sql: bool = None) -> QuerySet`
+
+You can limit the results to desired number of parent models.
+
+To limit the actual number of database query rows instead of number of main models
+use the `limit_raw_sql` parameter flag, and set it to `True`.
+
+```python
+tracks = await Track.objects.limit(1).all()
+# will return just one Track
+```
+
+!!!note
+    All methods that do not return the rows explicitly returns a QueySet instance so you can chain them together
+    
+    So operations like `filter()`, `select_related()`, `limit()` and `offset()` etc. can be chained.
+    
+    Something like `Track.object.select_related("album").filter(album__name="Malibu").offset(1).limit(1).all()`
+
+## offset
+
+`offset(offset: int, limit_raw_sql: bool = None) -> QuerySet`
+
+You can also offset the results by desired number of main models.
+
+To offset the actual number of database query rows instead of number of main models
+use the `limit_raw_sql` parameter flag, and set it to `True`.
+
+```python
+tracks = await Track.objects.offset(1).limit(1).all()
+# will return just one Track, but this time the second one
+```
+
+!!!note
+    All methods that do not return the rows explicitly returns a QueySet instance so you can chain them together
+    
+    So operations like `filter()`, `select_related()`, `limit()` and `offset()` etc. can be chained.
+    
+    Something like `Track.object.select_related("album").filter(album__name="Malibu").offset(1).limit(1).all()`
+
+
+
+## get
+
+`get(**kwargs): -> Model` 
+
+Get's the first row from the db meeting the criteria set by kwargs.
+
+If no criteria set it will return the last row in db sorted by pk.
+
+Passing a criteria is actually calling filter(**kwargs) method described below.
+
+```python
+track = await Track.objects.get(name='The Bird')
+# note that above is equivalent to await Track.objects.filter(name='The Bird').get()
+track2 = track = await Track.objects.get()
+track == track2 # True since it's the only row in db in our example
+```
+
+!!!warning
+    If no row meets the criteria `NoMatch` exception is raised.
+    
+    If there are multiple rows meeting the criteria the `MultipleMatches` exception is raised.
+
+## first
+
+`first(): -> Model` 
+
+Gets the first row from the db ordered by primary key column ascending.
--- a/docs/queries/read.md
+++ b/docs/queries/read.md
@ -0,0 +1,80 @@
+# Read/ Load data from database
+
+*  `get(**kwargs): -> Model`
+*  `get_or_create(**kwargs) -> Model`
+*  `first(): -> Model`
+*  `all(**kwargs) -> List[Optional[Model]]`
+*  `Model.load() method`
+
+## get
+
+`get(**kwargs): -> Model`
+
+Get's the first row from the db meeting the criteria set by kwargs.
+
+If no criteria set it will return the last row in db sorted by pk.
+
+Passing a criteria is actually calling filter(**kwargs) method described below.
+
+```python
+track = await Track.objects.get(name='The Bird')
+# note that above is equivalent to await Track.objects.filter(name='The Bird').get()
+track2 = track = await Track.objects.get()
+track == track2  # True since it's the only row in db in our example
+```
+
+!!!warning If no row meets the criteria `NoMatch` exception is raised.
+
+    If there are multiple rows meeting the criteria the `MultipleMatches` exception is raised.
+
+## get_or_create
+
+`get_or_create(**kwargs) -> Model`
+
+Combination of create and get methods.
+
+Tries to get a row meeting the criteria and if `NoMatch` exception is raised it creates
+a new one with given kwargs.
+
+```python
+album = await Album.objects.get_or_create(name='The Cat')
+# object is created as it does not exist
+album2 = await Album.objects.get_or_create(name='The Cat')
+assert album == album2
+# return True as the same db row is returned
+```
+
+!!!warning Despite being a equivalent row from database the `album` and `album2` in
+example above are 2 different python objects!
+Updating one of them will not refresh the second one until you excplicitly load() the
+fresh data from db.
+
+!!!note Note that if you want to create a new object you either have to pass pk column
+value or pk column has to be set as autoincrement
+
+## first
+
+`first(): -> Model`
+
+Gets the first row from the db ordered by primary key column ascending.
+
+## all
+
+`all(**kwargs) -> List[Optional["Model"]]`
+
+Returns all rows from a database for given model for set filter options.
+
+Passing kwargs is a shortcut and equals to calling `filter(**kwrags).all()`.
+
+If there are no rows meeting the criteria an empty list is returned.
+
+```python
+tracks = await Track.objects.select_related("album").all(title='Sample')
+# will return a list of all Tracks with title Sample
+
+tracks = await Track.objects.all()
+# will return a list of all Tracks in database
+
+```
+
+## Model method
--- a/docs/queries/select-columns.md
+++ b/docs/queries/select-columns.md
@ -0,0 +1,126 @@
+# Selecting subset of columns
+
+*  `fields(columns: Union[List, str, set, dict]) -> QuerySet`
+*  `exclude_fields(columns: Union[List, str, set, dict]) -> QuerySet`
+
+## fields
+
+`fields(columns: Union[List, str, set, dict]) -> QuerySet`
+
+With `fields()` you can select subset of model columns to limit the data load.
+
+!!!note Note that `fields()` and `exclude_fields()` works both for main models (on
+normal queries like `get`, `all` etc.)
+as well as `select_related` and `prefetch_related` models (with nested notation).
+
+Given a sample data like following:
+
+```python
+--8 < -- "../docs_src/queries/docs006.py"
+```
+
+You can select specified fields by passing a `str, List[str], Set[str] or dict` with
+nested definition.
+
+To include related models use
+notation `{related_name}__{column}[__{optional_next} etc.]`.
+
+```python hl_lines="1"
+all_cars = await Car.objects.select_related('manufacturer').fields(['id', 'name', 'manufacturer__name']).all()
+for car in all_cars:
+    # excluded columns will yield None
+    assert all(getattr(car, x) is None for x in ['year', 'gearbox_type', 'gears', 'aircon_type'])
+    # included column on related models will be available, pk column is always included
+    # even if you do not include it in fields list
+    assert car.manufacturer.name == 'Toyota'
+    # also in the nested related models - you cannot exclude pk - it's always auto added
+    assert car.manufacturer.founded is None
+```
+
+`fields()` can be called several times, building up the columns to select.
+
+If you include related models into `select_related()` call but you won't specify columns
+for those models in fields
+
+- implies a list of all fields for those nested models.
+
+```python hl_lines="1"
+all_cars = await Car.objects.select_related('manufacturer').fields('id').fields(
+    ['name']).all()
+# all fiels from company model are selected
+assert all_cars[0].manufacturer.name == 'Toyota'
+assert all_cars[0].manufacturer.founded == 1937
+```
+
+!!!warning Mandatory fields cannot be excluded as it will raise `ValidationError`, to
+exclude a field it has to be nullable.
+
+You cannot exclude mandatory model columns - `manufacturer__name` in this example.
+
+```python
+await Car.objects.select_related('manufacturer').fields(
+    ['id', 'name', 'manufacturer__founded']).all()
+# will raise pydantic ValidationError as company.name is required
+```
+
+!!!tip Pk column cannot be excluded - it's always auto added even if not explicitly
+included.
+
+You can also pass fields to include as dictionary or set.
+
+To mark a field as included in a dictionary use it's name as key and ellipsis as value.
+
+To traverse nested models use nested dictionaries.
+
+To include fields at last level instead of nested dictionary a set can be used.
+
+To include whole nested model specify model related field name and ellipsis.
+
+Below you can see examples that are equivalent:
+
+```python
+--8 < -- "../docs_src/queries/docs009.py"
+```
+
+!!!note All methods that do not return the rows explicitly returns a QueySet instance so
+you can chain them together
+
+    So operations like `filter()`, `select_related()`, `limit()` and `offset()` etc. can be chained.
+    
+    Something like `Track.object.select_related("album").filter(album__name="Malibu").offset(1).limit(1).all()`
+
+## exclude_fields
+
+`exclude_fields(columns: Union[List, str, set, dict]) -> QuerySet`
+
+With `exclude_fields()` you can select subset of model columns that will be excluded to
+limit the data load.
+
+It's the opposite of `fields()` method so check documentation above to see what options
+are available.
+
+Especially check above how you can pass also nested dictionaries and sets as a mask to
+exclude fields from whole hierarchy.
+
+!!!note Note that `fields()` and `exclude_fields()` works both for main models (on
+normal queries like `get`, `all` etc.)
+as well as `select_related` and `prefetch_related` models (with nested notation).
+
+Below you can find few simple examples:
+
+```python hl_lines="47 48 60 61 67"
+--8<-- "../docs_src/queries/docs008.py"
+```
+
+!!!warning Mandatory fields cannot be excluded as it will raise `ValidationError`, to
+exclude a field it has to be nullable.
+
+!!!tip Pk column cannot be excluded - it's always auto added even if explicitly
+excluded.
+
+!!!note All methods that do not return the rows explicitly returns a QueySet instance so
+you can chain them together
+
+    So operations like `filter()`, `select_related()`, `limit()` and `offset()` etc. can be chained.
+    
+    Something like `Track.object.select_related("album").filter(album__name="Malibu").offset(1).limit(1).all()`
--- a/docs/queries/update.md
+++ b/docs/queries/update.md
@ -0,0 +1,71 @@
+# Update
+
+*  `update(each: bool = False, **kwargs) -> int`
+*  `update_or_create(**kwargs) -> Model`
+*  `bulk_update(objects: List[Model], columns: List[str] = None) -> None`
+*  `Model.update() method`
+*  `Model.upsert() method`
+*  `Model.save_related() method`
+
+## update
+
+`update(each: bool = False, **kwargs) -> int`
+
+QuerySet level update is used to update multiple records with the same value at once.
+
+You either have to filter the QuerySet first or provide a `each=True` flag to update
+whole table.
+
+If you do not provide this flag or a filter a `QueryDefinitionError` will be raised.
+
+Return number of rows updated.
+
+```Python hl_lines="26-28"
+--8<-- "../docs_src/queries/docs002.py"
+```
+
+!!!warning Queryset needs to be filtered before updating to prevent accidental
+overwrite.
+
+    To update whole database table `each=True` needs to be provided as a safety switch
+
+## update_or_create
+
+`update_or_create(**kwargs) -> Model`
+
+Updates the model, or in case there is no match in database creates a new one.
+
+```Python hl_lines="26-32"
+--8<-- "../docs_src/queries/docs003.py"
+```
+
+!!!note Note that if you want to create a new object you either have to pass pk column
+value or pk column has to be set as autoincrement
+
+## bulk_update
+
+`bulk_update(objects: List["Model"], columns: List[str] = None) -> None`
+
+Allows to update multiple instance at once.
+
+All `Models` passed need to have primary key column populated.
+
+You can also select which fields to update by passing `columns` list as a list of string
+names.
+
+```python hl_lines="8"
+# continuing the example from bulk_create
+# update objects
+for todo in todoes:
+    todo.completed = False
+
+# perform update of all objects at once
+# objects need to have pk column set, otherwise exception is raised
+await ToDo.objects.bulk_update(todoes)
+
+completed = await ToDo.objects.filter(completed=False).all()
+assert len(completed) == 3
+```
+
+## Model method
+