Prodream/ormar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add tests for cross model forward references, add docs for processing forwardrefs, wip on refactoring queries into separate pages based on functionality

Browse Source
This commit is contained in:
collerek
2021-01-26 17:29:40 +01:00
parent a2834666fc
commit b710ed9780
39 changed files with 2054 additions and 1004 deletions
Download Patch File Download Diff File Expand all files Collapse all files

111
docs/api/query-set/clause.md
View File

@ -8,13 +8,13 @@
class QueryClause()
```
Constructs where clauses from strings passed as arguments
Constructs FilterActions from strings passed as arguments
<a name="queryset.clause.QueryClause.filter"></a>
#### filter
<a name="queryset.clause.QueryClause.prepare_filter"></a>
#### prepare\_filter
```python
| filter(**kwargs: Any) -> Tuple[List[sqlalchemy.sql.expression.TextClause], List[str]]
| prepare_filter(**kwargs: Any) -> Tuple[List[FilterAction], List[str]]
```
Main external access point that processes the clauses into sqlalchemy text
@ -33,7 +33,7 @@ mentioned in select_related strings but not included in select_related.
#### \_populate\_filter\_clauses
```python
| _populate_filter_clauses(**kwargs: Any) -> Tuple[List[sqlalchemy.sql.expression.TextClause], List[str]]
| _populate_filter_clauses(**kwargs: Any) -> Tuple[List[FilterAction], List[str]]
```
Iterates all clauses and extracts used operator and field from related
@ -48,114 +48,59 @@ is determined and the final clause is escaped if needed and compiled.
`(Tuple[List[sqlalchemy.sql.elements.TextClause], List[str]])`: Tuple with list of where clauses and updated select_related list
<a name="queryset.clause.QueryClause._process_column_clause_for_operator_and_value"></a>
#### \_process\_column\_clause\_for\_operator\_and\_value
<a name="queryset.clause.QueryClause._register_complex_duplicates"></a>
#### \_register\_complex\_duplicates
```python
| _process_column_clause_for_operator_and_value(value: Any, op: str, column: sqlalchemy.Column, table: sqlalchemy.Table, table_prefix: str) -> sqlalchemy.sql.expression.TextClause
| _register_complex_duplicates(select_related: List[str]) -> None
```
Escapes characters if it's required.
Substitutes values of the models if value is a ormar Model with its pk value.
Compiles the clause.
Checks if duplicate aliases are presented which can happen in self relation
or when two joins end with the same pair of models.
If there are duplicates, the all duplicated joins are registered as source
model and whole relation key (not just last relation name).
**Arguments**:
- `value (Any)`: value of the filter
- `op (str)`: filter operator
- `column (sqlalchemy.sql.schema.Column)`: column on which filter should be applied
- `table (sqlalchemy.sql.schema.Table)`: table on which filter should be applied
- `table_prefix (str)`: prefix from AliasManager
- `select_related (List[str])`: list of relation strings
**Returns**:
`(sqlalchemy.sql.elements.TextClause)`: complied and escaped clause
`(None)`: None
<a name="queryset.clause.QueryClause._determine_filter_target_table"></a>
#### \_determine\_filter\_target\_table
<a name="queryset.clause.QueryClause._parse_related_prefixes"></a>
#### \_parse\_related\_prefixes
```python
| _determine_filter_target_table(related_parts: List[str], select_related: List[str]) -> Tuple[List[str], str, Type["Model"]]
| _parse_related_prefixes(select_related: List[str]) -> List[Prefix]
```
Adds related strings to select_related list otherwise the clause would fail as
the required columns would not be present. That means that select_related
list is filled with missing values present in filters.
Walks the relation to retrieve the actual model on which the clause should be
constructed, extracts alias based on last relation leading to target model.
Walks all relation strings and parses the target models and prefixes.
**Arguments**:
- `related_parts (List[str])`: list of split parts of related string
- `select_related (List[str])`: list of related models
- `select_related (List[str])`: list of relation strings
**Returns**:
`(Tuple[List[str], str, Type[Model]])`: list of related models, table_prefix, final model class
`(List[Prefix])`: list of parsed prefixes
<a name="queryset.clause.QueryClause._compile_clause"></a>
#### \_compile\_clause
<a name="queryset.clause.QueryClause._switch_filter_action_prefixes"></a>
#### \_switch\_filter\_action\_prefixes
```python
| _compile_clause(clause: sqlalchemy.sql.expression.BinaryExpression, column: sqlalchemy.Column, table: sqlalchemy.Table, table_prefix: str, modifiers: Dict) -> sqlalchemy.sql.expression.TextClause
| _switch_filter_action_prefixes(filter_clauses: List[FilterAction]) -> List[FilterAction]
```
Compiles the clause to str using appropriate database dialect, replace columns
names with aliased names and converts it back to TextClause.
Substitutes aliases for filter action if the complex key (whole relation str) is
present in alias_manager.
**Arguments**:
- `clause (sqlalchemy.sql.elements.BinaryExpression)`: original not compiled clause
- `column (sqlalchemy.sql.schema.Column)`: column on which filter should be applied
- `table (sqlalchemy.sql.schema.Table)`: table on which filter should be applied
- `table_prefix (str)`: prefix from AliasManager
- `modifiers (Dict[str, NoneType])`: sqlalchemy modifiers - used only to escape chars here
- `filter_clauses (List[FilterAction])`: raw list of actions
**Returns**:
`(sqlalchemy.sql.elements.TextClause)`: compiled and escaped clause
<a name="queryset.clause.QueryClause._escape_characters_in_clause"></a>
#### \_escape\_characters\_in\_clause
```python
| @staticmethod
| _escape_characters_in_clause(op: str, value: Any) -> Tuple[Any, bool]
```
Escapes the special characters ["%", "_"] if needed.
Adds `%` for `like` queries.
**Raises**:
- `QueryDefinitionError`: if contains or icontains is used with
ormar model instance
**Arguments**:
- `op (str)`: operator used in query
- `value (Any)`: value of the filter
**Returns**:
`(Tuple[Any, bool])`: escaped value and flag if escaping is needed
<a name="queryset.clause.QueryClause._extract_operator_field_and_related"></a>
#### \_extract\_operator\_field\_and\_related
```python
| @staticmethod
| _extract_operator_field_and_related(parts: List[str]) -> Tuple[str, str, Optional[List]]
```
Splits filter query key and extracts required parts.
**Arguments**:
- `parts (List[str])`: split filter query key
**Returns**:
`(Tuple[str, str, Optional[List]])`: operator, field_name, list of related parts
`(List[FilterAction])`: list of actions with aliases changed if needed

167
docs/api/query-set/join.md
View File

@ -1,15 +1,6 @@
<a name="queryset.join"></a>
# queryset.join
<a name="queryset.join.JoinParameters"></a>
## JoinParameters Objects
```python
class JoinParameters(NamedTuple)
```
Named tuple that holds set of parameters passed during join construction.
<a name="queryset.join.SqlJoin"></a>
## SqlJoin Objects
@ -21,15 +12,11 @@ class SqlJoin()
#### alias\_manager
```python
| @staticmethod
| alias_manager(model_cls: Type["Model"]) -> AliasManager
| @property
| alias_manager() -> AliasManager
```
Shortcut for ormars model AliasManager stored on Meta.
**Arguments**:
- `model_cls (Type[Model])`: ormar Model class
Shortcut for ormar's model AliasManager stored on Meta.
**Returns**:
@ -39,8 +26,7 @@ Shortcut for ormars model AliasManager stored on Meta.
#### on\_clause
```python
| @staticmethod
| on_clause(previous_alias: str, alias: str, from_clause: str, to_clause: str) -> text
| on_clause(previous_alias: str, from_clause: str, to_clause: str) -> text
```
Receives aliases and names of both ends of the join and combines them
@ -49,7 +35,6 @@ into one text clause used in joins.
**Arguments**:
- `previous_alias (str)`: alias of previous table
- `alias (str)`: alias of current table
- `from_clause (str)`: from table name
- `to_clause (str)`: to table name
@ -57,32 +42,11 @@ into one text clause used in joins.
`(sqlalchemy.text)`: clause combining all strings
<a name="queryset.join.SqlJoin.update_inclusions"></a>
#### update\_inclusions
```python
| @staticmethod
| update_inclusions(model_cls: Type["Model"], fields: Optional[Union[Set, Dict]], exclude_fields: Optional[Union[Set, Dict]], nested_name: str) -> Tuple[Optional[Union[Dict, Set]], Optional[Union[Dict, Set]]]
```
Extract nested fields and exclude_fields if applicable.
**Arguments**:
- `model_cls (Type["Model"])`: ormar model class
- `fields (Optional[Union[Set, Dict]])`: fields to include
- `exclude_fields (Optional[Union[Set, Dict]])`: fields to exclude
- `nested_name (str)`: name of the nested field
**Returns**:
`(Tuple[Optional[Union[Dict, Set]], Optional[Union[Dict, Set]]])`: updated exclude and include fields from nested objects
<a name="queryset.join.SqlJoin.build_join"></a>
#### build\_join
```python
| build_join(item: str, join_parameters: JoinParameters) -> Tuple[List, sqlalchemy.sql.select, List, OrderedDict]
| build_join() -> Tuple[List, sqlalchemy.sql.select, List, OrderedDict]
```
Main external access point for building a join.
@ -90,42 +54,96 @@ Splits the join definition, updates fields and exclude_fields if needed,
handles switching to through models for m2m relations, returns updated lists of
used_aliases and sort_orders.
**Arguments**:
- `item (str)`: string with join definition
- `join_parameters (JoinParameters)`: parameters from previous/ current join
**Returns**:
`(Tuple[List[str], Join, List[TextClause], collections.OrderedDict])`: list of used aliases, select from, list of aliased columns, sort orders
<a name="queryset.join.SqlJoin._build_join_parameters"></a>
#### \_build\_join\_parameters
<a name="queryset.join.SqlJoin._forward_join"></a>
#### \_forward\_join
```python
| _build_join_parameters(part: str, join_params: JoinParameters, fields: Optional[Union[Set, Dict]], exclude_fields: Optional[Union[Set, Dict]], is_multi: bool = False) -> JoinParameters
| _forward_join() -> None
```
Updates used_aliases to not join multiple times to the same table.
Updates join parameters with new values.
Process actual join.
Registers complex relation join on encountering of the duplicated alias.
<a name="queryset.join.SqlJoin._process_following_joins"></a>
#### \_process\_following\_joins
```python
| _process_following_joins() -> None
```
Iterates through nested models to create subsequent joins.
<a name="queryset.join.SqlJoin._process_deeper_join"></a>
#### \_process\_deeper\_join
```python
| _process_deeper_join(related_name: str, remainder: Any) -> None
```
Creates nested recurrent instance of SqlJoin for each nested join table,
updating needed return params here as a side effect.
Updated are:
* self.used_aliases,
* self.select_from,
* self.columns,
* self.sorted_orders,
**Arguments**:
- `part (str)`: part of the join str definition
- `join_params (JoinParameters)`: parameters from previous/ current join
- `fields (Optional[Union[Set, Dict]])`: fields to include
- `exclude_fields (Optional[Union[Set, Dict]])`: fields to exclude
- `is_multi (bool)`: flag if the relation is m2m
- `related_name (str)`: name of the relation to follow
- `remainder (Any)`: deeper tables if there are more nested joins
<a name="queryset.join.SqlJoin.process_m2m_through_table"></a>
#### process\_m2m\_through\_table
```python
| process_m2m_through_table() -> None
```
Process Through table of the ManyToMany relation so that source table is
linked to the through table (one additional join)
Replaces needed parameters like:
* self.next_model,
* self.next_alias,
* self.relation_name,
* self.own_alias,
* self.target_field
To point to through model
<a name="queryset.join.SqlJoin.process_m2m_related_name_change"></a>
#### process\_m2m\_related\_name\_change
```python
| process_m2m_related_name_change(reverse: bool = False) -> str
```
Extracts relation name to link join through the Through model declared on
relation field.
Changes the same names in order_by queries if they are present.
**Arguments**:
- `reverse (bool)`: flag if it's on_clause lookup - use reverse fields
**Returns**:
`(ormar.queryset.join.JoinParameters)`: updated join parameters
`(str)`: new relation name switched to through model field
<a name="queryset.join.SqlJoin._process_join"></a>
#### \_process\_join
```python
| _process_join(join_params: JoinParameters, is_multi: bool, model_cls: Type["Model"], part: str, alias: str, fields: Optional[Union[Set, Dict]], exclude_fields: Optional[Union[Set, Dict]]) -> None
| _process_join() -> None
```
Resolves to and from column names and table names.
@ -140,18 +158,8 @@ Updates the used aliases list directly.
Process order_by causes for non m2m relations.
**Arguments**:
- `join_params (JoinParameters)`: parameters from previous/ current join
- `is_multi (bool)`: flag if it's m2m relation
- `model_cls (ormar.models.metaclass.ModelMetaclass)`:
- `part (str)`: name of the field used in join
- `alias (str)`: alias of the current join
- `fields (Optional[Union[Set, Dict]])`: fields to include
- `exclude_fields (Optional[Union[Set, Dict]])`: fields to exclude
<a name="queryset.join.SqlJoin._replace_many_to_many_order_by_columns"></a>
#### \_switch\_many\_to\_many\_order\_columns
#### \_replace\_many\_to\_many\_order\_by\_columns
```python
| _replace_many_to_many_order_by_columns(part: str, new_part: str) -> None
@ -187,7 +195,7 @@ Checks filter conditions to find if they apply to current join.
#### set\_aliased\_order\_by
```python
| set_aliased_order_by(condition: List[str], alias: str, to_table: str, model_cls: Type["Model"]) -> None
| set_aliased_order_by(condition: List[str], to_table: str) -> None
```
Substitute hyphens ('-') with descending order.
@ -196,15 +204,13 @@ Construct actual sqlalchemy text clause using aliased table and column name.
**Arguments**:
- `condition (List[str])`: list of parts of a current condition split by '__'
- `alias (str)`: alias of the table in current join
- `to_table (sqlalchemy.sql.elements.quoted_name)`: target table
- `model_cls (ormar.models.metaclass.ModelMetaclass)`: ormar model class
<a name="queryset.join.SqlJoin.get_order_bys"></a>
#### get\_order\_bys
```python
| get_order_bys(alias: str, to_table: str, pkname_alias: str, part: str, model_cls: Type["Model"]) -> None
| get_order_bys(to_table: str, pkname_alias: str) -> None
```
Triggers construction of order bys if they are given.
@ -212,30 +218,19 @@ Otherwise by default each table is sorted by a primary key column asc.
**Arguments**:
- `alias (str)`: alias of current table in join
- `to_table (sqlalchemy.sql.elements.quoted_name)`: target table
- `pkname_alias (str)`: alias of the primary key column
- `part (str)`: name of the current relation join
- `model_cls (Type[Model])`: ormar model class
<a name="queryset.join.SqlJoin.get_to_and_from_keys"></a>
#### get\_to\_and\_from\_keys
```python
| @staticmethod
| get_to_and_from_keys(join_params: JoinParameters, is_multi: bool, model_cls: Type["Model"], part: str) -> Tuple[str, str]
| get_to_and_from_keys() -> Tuple[str, str]
```
Based on the relation type, name of the relation and previous models and parts
stored in JoinParameters it resolves the current to and from keys, which are
different for ManyToMany relation, ForeignKey and reverse part of relations.
**Arguments**:
- `join_params (JoinParameters)`: parameters from previous/ current join
- `is_multi (bool)`: flag if the relation is of m2m type
- `model_cls (Type[Model])`: ormar model class
- `part (str)`: name of the current relation join
different for ManyToMany relation, ForeignKey and reverse related of relations.
**Returns**:

6
docs/api/query-set/prefetch-query.md
View File

@ -289,7 +289,7 @@ models.
| _get_select_related_if_apply(related: str, select_dict: Dict) -> Dict
```
Extract nested part of select_related dictionary to extract models nested
Extract nested related of select_related dictionary to extract models nested
deeper on related model and already loaded in select related query.
**Arguments**:
@ -299,7 +299,7 @@ deeper on related model and already loaded in select related query.
**Returns**:
`(Dict)`: dictionary with nested part of select related
`(Dict)`: dictionary with nested related of select related
<a name="queryset.prefetch_query.PrefetchQuery._update_already_loaded_rows"></a>
#### \_update\_already\_loaded\_rows
@ -320,7 +320,7 @@ Updates models that are already loaded, usually children of children.
#### \_populate\_rows
```python
| _populate_rows(rows: List, target_field: Type["BaseField"], parent_model: Type["Model"], table_prefix: str, fields: Union[Set[Any], Dict[Any, Any], None], exclude_fields: Union[Set[Any], Dict[Any, Any], None], prefetch_dict: Dict, orders_by: Dict) -> None
| _populate_rows(rows: List, target_field: Type["ForeignKeyField"], parent_model: Type["Model"], table_prefix: str, fields: Union[Set[Any], Dict[Any, Any], None], exclude_fields: Union[Set[Any], Dict[Any, Any], None], prefetch_dict: Dict, orders_by: Dict) -> None
```
Instantiates children models extracted from given relation.

19
docs/api/query-set/utils.md
View File

@ -150,3 +150,22 @@ with all children models under their relation keys.
`(Dict)`: dictionary of lists f related models
<a name="queryset.utils.get_relationship_alias_model_and_str"></a>
#### get\_relationship\_alias\_model\_and\_str
```python
get_relationship_alias_model_and_str(source_model: Type["Model"], related_parts: List) -> Tuple[str, Type["Model"], str]
```
Walks the relation to retrieve the actual model on which the clause should be
constructed, extracts alias based on last relation leading to target model.
**Arguments**:
- `related_parts (Union[List, List[str]])`: list of related names extracted from string
- `source_model (Type[Model])`: model from which relation starts
**Returns**:
`(Tuple[str, Type["Model"], str])`: table prefix, target model and relation string