Compound statements contain (groups of) other statements; they affect or control the execution of those other statements in some way. In general, compound statements span multiple lines, although in simple incarnations a whole compound statement may be contained in one line.
The `if`, `while` and `for` statements implement traditional control flow constructs. `try` specifies exception handlers and/or cleanup code for a group of statements, while the `with` statement allows the execution of initialization and finalization code around a block of code. Function and class definitions are also syntactically compound statements.
single: clause single: suite single: ; (semicolon)
A compound statement consists of one or more 'clauses.' A clause consists of a header and a 'suite.' The clause headers of a particular compound statement are all at the same indentation level. Each clause header begins with a uniquely identifying keyword and ends with a colon. A suite is a group of statements controlled by a clause. A suite can be one or more semicolon-separated simple statements on the same line as the header, following the header's colon, or it can be one or more indented statements on subsequent lines. Only the latter form of a suite can contain nested compound statements; the following is illegal, mostly because it wouldn't be clear to which `if` clause a following `else` clause would belong:
if test1: if test2: print(x)
Also note that the semicolon binds tighter than the colon in this context, so that in the following example, either all or none of the `print` calls are executed:
if x < y < z: print(x); print(y); print(z)
Summarizing:
Note that statements always end in a `NEWLINE` possibly followed by a `DEDENT`. Also note that optional continuation clauses always begin with a keyword that cannot start a statement, thus there are no ambiguities (the 'dangling `else`' problem is solved in Python by requiring nested `if` statements to be indented).
The formatting of the grammar rules in the following sections places each clause on a separate line for clarity.
## The `!if` statement
It selects exactly one of the suites by evaluating the expressions one by one until one is found to be true (see section `booleans` for the definition of true and false); then that suite is executed (and no other part of the `if` statement is executed or evaluated). If all expressions are false, the suite of the `else` clause, if present, is executed.
## The `!while` statement
The `while` statement is used for repeated execution as long as an expression is true:
python-grammar while_stmt: "while" assignment_expression ":" suite : \["else" ":" suite\]
This repeatedly tests the expression and, if it is true, executes the first suite; if the expression is false (which may be the first time it is tested) the suite of the `!else` clause, if present, is executed and the loop terminates.
pair: statement; break pair: statement; continue
A `break` statement executed in the first suite terminates the loop without executing the `!else` clause's suite. A `continue` statement executed in the first suite skips the rest of the suite and goes back to testing the expression.
## The `!for` statement
! pair: statement; for pair: keyword; in pair: keyword; else pair: target; list pair: loop; statement pair: object; sequence single: : (colon); compound statement
The `for` statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other iterable object:
The `~python-grammar:starred_expression_list` expression is evaluated once; it should yield an `iterable` object. An `iterator` is created for that iterable. The first item provided by the iterator is then assigned to the target list using the standard rules for assignments (see `assignment`), and the suite is executed. This repeats for each item provided by the iterator. When the iterator is `exhausted`, the suite in the `!else` clause, if present, is executed, and the loop terminates.
pair: statement; break pair: statement; continue
A `break` statement executed in the first suite terminates the loop without executing the `!else` clause's suite. A `continue` statement executed in the first suite skips the rest of the suite and continues with the next item, or with the `!else` clause if there is no next item.
The for-loop makes assignments to the variables in the target list. This overwrites all previous assignments to those variables including those made in the suite of the for-loop:
for i in range(10):
print(i)
i = 5 # this will not affect the for-loop
# because i will be overwritten with the next
# index in the range
pair: built-in function; range
Names in the target list are not deleted when the loop is finished, but if the sequence is empty, they will not have been assigned to at all by the loop. Hint: the built-in type `range` represents immutable arithmetic sequences of integers. For instance, iterating `range(3)` successively yields 0, 1, and then 2.
3.11 Starred elements are now allowed in the expression list.
Additional information on exceptions can be found in section `exceptions`, and information on using the `raise` statement to generate exceptions may be found in section `raise`.
3.14 Support for optionally dropping grouping parentheses when using multiple exception types. See `758`.
### `!except` clause
The `!except` clause(s) specify one or more exception handlers. When no exception occurs in the `try` clause, no exception handler is executed. When an exception occurs in the `!try` suite, a search for an exception handler is started. This search inspects the `!except` clauses in turn until one is found that matches the exception. An expression-less `!except` clause, if present, must be last; it matches any exception.
For an `!except` clause with an expression, the expression must evaluate to an exception type or a tuple of exception types. Parentheses can be dropped if multiple exception types are provided and the `as` clause is not used. The raised exception matches an `!except` clause whose expression evaluates to the class or a `non-virtual base class ` of the exception object, or to a tuple that contains such a class.
If no `!except` clause matches the exception, the search for an exception handler continues in the surrounding code and on the invocation stack.[^1]
If the evaluation of an expression in the header of an `!except` clause raises an exception, the original search for a handler is canceled and a search starts for the new exception in the surrounding code and on the call stack (it is treated as if the entire `try` statement raised the exception).
single: as; except clause
When a matching `!except` clause is found, the exception is assigned to the target specified after the `!as` keyword in that `!except` clause, if present, and the `!except` clause's suite is executed. All `!except` clauses must have an executable block. When the end of this block is reached, execution continues normally after the entire `try` statement. (This means that if two nested handlers exist for the same exception, and the exception occurs in the `!try` clause of the inner handler, the outer handler will not handle the exception.)
When an exception has been assigned using `as target`, it is cleared at the end of the `!except` clause. This is as if:
except E as N:
foo
was translated to:
except E as N:
try:
foo
finally:
del N
This means the exception must be assigned to a different name to be able to refer to it after the `!except` clause. Exceptions are cleared because with the traceback attached to them, they form a reference cycle with the stack frame, keeping all locals in that frame alive until the next garbage collection occurs.
pair: module; sys pair: object; traceback
Before an `!except` clause's suite is executed, the exception is stored in the `sys` module, where it can be accessed from within the body of the `!except` clause by calling `sys.exception`. When leaving an exception handler, the exception stored in the `sys` module is reset to its previous value:
>>> print(sys.exception())
None
>>> try:
... raise TypeError
... except:
... print(repr(sys.exception()))
... try:
... raise ValueError
... except:
... print(repr(sys.exception()))
... print(repr(sys.exception()))
...
TypeError()
ValueError()
TypeError()
>>> print(sys.exception())
None
pair: keyword; except_star
### `!except*` clause
The `!except*` clause(s) specify one or more handlers for groups of exceptions (`BaseExceptionGroup` instances). A `try` statement can have either `except` or `!except*` clauses, but not both. The exception type for matching is mandatory in the case of `!except*`, so `except*:` is a syntax error. The type is interpreted as in the case of `!except`, but matching is performed on the exceptions contained in the group that is being handled. A `TypeError` is raised if a matching type is a subclass of `!BaseExceptionGroup`, because that would have ambiguous semantics.
When an exception group is raised in the try block, each `!except*` clause splits (see `~BaseExceptionGroup.split`) it into the subgroups of matching and non-matching exceptions. If the matching subgroup is not empty, it becomes the handled exception (the value returned from `sys.exception`) and assigned to the target of the `!except*` clause (if there is one). Then, the body of the `!except*` clause executes. If the non-matching subgroup is not empty, it is processed by the next `!except*` in the same manner. This continues until all exceptions in the group have been matched, or the last `!except*` clause has run.
After all `!except*` clauses execute, the group of unhandled exceptions is merged with any exceptions that were raised or re-raised from within `!except*` clauses. This merged exception group propagates on:
>>> try:
... raise ExceptionGroup("eg",
... [ValueError(1), TypeError(2), OSError(3), OSError(4)])
... except* TypeError as e:
... print(f'caught {type(e)} with nested {e.exceptions}')
... except* OSError as e:
... print(f'caught {type(e)} with nested {e.exceptions}')
...
caught with nested (TypeError(2),)
caught with nested (OSError(3), OSError(4))
+ Exception Group Traceback (most recent call last):
| File "", line 2, in
| raise ExceptionGroup("eg",
| [ValueError(1), TypeError(2), OSError(3), OSError(4)])
| ExceptionGroup: eg (1 sub-exception)
+-+---------------- 1 ----------------
| ValueError: 1
+------------------------------------
If the exception raised from the `try` block is not an exception group and its type matches one of the `!except*` clauses, it is caught and wrapped by an exception group with an empty message string. This ensures that the type of the target `e` is consistently `BaseExceptionGroup`:
>>> try:
... raise BlockingIOError
... except* BlockingIOError as e:
... print(repr(e))
...
ExceptionGroup('', (BlockingIOError(),))
`break`, `continue` and `return` cannot appear in an `!except*` clause.
### `!else` clause
The optional `!else` clause is executed if the control flow leaves the `try` suite, no exception was raised, and no `return`, `continue`, or `break` statement was executed. Exceptions in the `!else` clause are not handled by the preceding `except` clauses.
pair: keyword; finally
### `!finally` clause
If `!finally` is present, it specifies a 'cleanup' handler. The `try` clause is executed, including any `except` and `else ` clauses. If an exception occurs in any of the clauses and is not handled, the exception is temporarily saved. The `!finally` clause is executed. If there is a saved exception it is re-raised at the end of the `!finally` clause. If the `!finally` clause raises another exception, the saved exception is set as the context of the new exception. If the `!finally` clause executes a `return`, `break` or `continue` statement, the saved exception is discarded. For example, this function returns 42.
def f():
try:
1/0
finally:
return 42
The exception information is not available to the program during execution of the `!finally` clause.
When a `return`, `break` or `continue` statement is executed in the `try` suite of a `!try`...`!finally` statement, the `!finally` clause is also executed 'on the way out.'
The return value of a function is determined by the last `return` statement executed. Since the `!finally` clause always executes, a `!return` statement executed in the `!finally` clause will always be the last one executed. The following function returns 'finally'.
def foo():
try:
return 'try'
finally:
return 'finally'
3.8 Prior to Python 3.8, a `continue` statement was illegal in the `!finally` clause due to a problem with the implementation.
3.14 The compiler emits a `SyntaxWarning` when a `return`, `break` or `continue` appears in a `!finally` block (see `765`).
## The `!with` statement
! pair: statement; with pair: keyword; as single: as; with statement single: , (comma); with statement single: : (colon); compound statement
The `with` statement is used to wrap the execution of a block with methods defined by a context manager (see section `context-managers`). This allows common `try`...`except`...`finally` usage patterns to be encapsulated for convenient reuse.
The execution of the `with` statement with one "item" proceeds as follows:
1. The context expression (the expression given in the `~python-grammar:with_item`) is evaluated to obtain a context manager.
2. The context manager's `~object.__enter__` is loaded for later use.
3. The context manager's `~object.__exit__` is loaded for later use.
4. The context manager's `~object.__enter__` method is invoked.
5. If a target was included in the `with` statement, the return value from `~object.__enter__` is assigned to it.
> [!NOTE]
> The `with` statement guarantees that if the `~object.__enter__` method returns without an error, then `~object.__exit__` will always be called. Thus, if an error occurs during the assignment to the target list, it will be treated the same as an error occurring within the suite would be. See step 7 below.
6. The suite is executed.
7. The context manager's `~object.__exit__` method is invoked. If an exception caused the suite to be exited, its type, value, and traceback are passed as arguments to `~object.__exit__`. Otherwise, three `None` arguments are supplied.
If the suite was exited due to an exception, and the return value from the `~object.__exit__` method was false, the exception is reraised. If the return value was true, the exception is suppressed, and execution continues with the statement following the `with` statement.
If the suite was exited for any reason other than an exception, the return value from `~object.__exit__` is ignored, and execution proceeds at the normal location for the kind of exit that was taken.
The following code:
with EXPRESSION as TARGET:
SUITE
is semantically equivalent to:
manager = (EXPRESSION)
enter = manager.__enter__
exit = manager.__exit__
value = enter()
hit_except = False
try:
TARGET = value
SUITE
except:
hit_except = True
if not exit(*sys.exc_info()):
raise
finally:
if not hit_except:
exit(None, None, None)
except that implicit `special method lookup ` is used for `~object.__enter__` and `~object.__exit__`.
With more than one item, the context managers are processed as if multiple `with` statements were nested:
with A() as a, B() as b:
SUITE
is semantically equivalent to:
with A() as a:
with B() as b:
SUITE
You can also write multi-item context managers in multiple lines if the items are surrounded by parentheses. For example:
with (
A() as a,
B() as b,
):
SUITE
3.1 Support for multiple context expressions.
3.10 Support for using grouping parentheses to break the statement in multiple lines.
`343` - The "with" statement
The specification, background, and examples for the Python `with` statement.
## The `!match` statement
! pair: statement; match ! pair: keyword; case ! single: pattern matching pair: keyword; if pair: keyword; as pair: match; case single: as; match statement single: : (colon); compound statement
3.10
The match statement is used for pattern matching. Syntax:
> [!NOTE]
> This section uses single quotes to denote `soft keywords `.
Pattern matching takes a pattern as input (following `case`) and a subject value (following `match`). The pattern (which may contain subpatterns) is matched against the subject value. The outcomes are:
- A match success or failure (also termed a pattern success or failure).
- Possible binding of matched values to a name. The prerequisites for this are further discussed below.
The `match` and `case` keywords are `soft keywords `.
### Overview
Here's an overview of the logical flow of a match statement:
1. The subject expression `subject_expr` is evaluated and a resulting subject value obtained. If the subject expression contains a comma, a tuple is constructed using `the standard rules `.
2. Each pattern in a `case_block` is attempted to match with the subject value. The specific rules for success or failure are described below. The match attempt can also bind some or all of the standalone names within the pattern. The precise pattern binding rules vary per pattern type and are specified below. **Name bindings made during a successful pattern match outlive the executed block and can be used after the match statement**.
> [!NOTE]
> During failed pattern matches, some subpatterns may succeed. Do not rely on bindings being made for a failed match. Conversely, do not rely on variables remaining unchanged after a failed match. The exact behavior is dependent on implementation and may vary. This is an intentional decision made to allow different implementations to add optimizations.
3. If the pattern succeeds, the corresponding guard (if present) is evaluated. In this case all name bindings are guaranteed to have happened.
- If the guard evaluates as true or is missing, the `block` inside `case_block` is executed.
- Otherwise, the next `case_block` is attempted as described above.
- If there are no further case blocks, the match statement is completed.
> [!NOTE]
> Users should generally never rely on a pattern being evaluated. Depending on implementation, the interpreter may cache values or use other optimizations which skip repeated evaluations.
A sample match statement:
>>> flag = False
>>> match (100, 200):
... case (100, 300): # Mismatch: 200 != 300
... print('Case 1')
... case (100, 200) if flag: # Successful match, but guard fails
... print('Case 2')
... case (100, y): # Matches and binds y to 200
... print(f'Case 3, y: {y}')
... case _: # Pattern not attempted
... print('Case 4, I match anything!')
...
Case 3, y: 200
In this case, `if flag` is a guard. Read more about that in the next section.
### Guards
! guard
python-grammar guard: "if" assignment_expression
A `guard` (which is part of the `case`) must succeed for code inside the `case` block to execute. It takes the form: `if` followed by an expression.
The logical flow of a `case` block with a `guard` follows:
1. Check that the pattern in the `case` block succeeded. If the pattern failed, the `guard` is not evaluated and the next `case` block is checked.
2. If the pattern succeeded, evaluate the `guard`.
- If the `guard` condition evaluates as true, the case block is selected.
- If the `guard` condition evaluates as false, the case block is not selected.
- If the `guard` raises an exception during evaluation, the exception bubbles up.
Guards are allowed to have side effects as they are expressions. Guard evaluation must proceed from the first to the last case block, one at a time, skipping case blocks whose pattern(s) don't all succeed. (I.e., guard evaluation must happen in order.) Guard evaluation must stop once a case block is selected.
### Irrefutable Case Blocks
irrefutable case block, case block
An irrefutable case block is a match-all case block. A match statement may have at most one irrefutable case block, and it must be last.
A case block is considered irrefutable if it has no guard and its pattern is irrefutable. A pattern is considered irrefutable if we can prove from its syntax alone that it will always succeed. Only the following patterns are irrefutable:
- `as-patterns` whose left-hand side is irrefutable
- `or-patterns` containing at least one irrefutable pattern
- `capture-patterns`
- `wildcard-patterns`
- parenthesized irrefutable patterns
### Patterns
single: ! patterns single: AS pattern, OR pattern, capture pattern, wildcard pattern
> [!NOTE]
> This section uses grammar notations beyond standard EBNF:
>
> - the notation `SEP.RULE+` is shorthand for `RULE (SEP RULE)*`
> - the notation `!RULE` is shorthand for a negative lookahead assertion
The top-level syntax for `patterns` is:
The descriptions below will include a description "in simple terms" of what a pattern does for illustration purposes (credits to Raymond Hettinger for a document that inspired most of the descriptions). Note that these descriptions are purely for illustration purposes and **may not** reflect the underlying implementation. Furthermore, they do not cover all valid forms.
#### OR Patterns
An OR pattern is two or more patterns separated by vertical bars `|`. Syntax:
Only the final subpattern may be `irrefutable `, and each subpattern must bind the same set of names to avoid ambiguity.
An OR pattern matches each of its subpatterns in turn to the subject value, until one succeeds. The OR pattern is then considered successful. Otherwise, if none of the subpatterns succeed, the OR pattern fails.
In simple terms, `P1 | P2 | ...` will try to match `P1`, if it fails it will try to match `P2`, succeeding immediately if any succeeds, failing otherwise.
#### AS Patterns
An AS pattern matches an OR pattern on the left of the `as` keyword against a subject. Syntax:
If the OR pattern fails, the AS pattern fails. Otherwise, the AS pattern binds the subject to the name on the right of the as keyword and succeeds. `capture_pattern` cannot be a `_`.
In simple terms `P as NAME` will match with `P`, and on success it will set `NAME = `.
#### Literal Patterns
A literal pattern corresponds to most `literals ` in Python. Syntax:
python-grammar literal_pattern: signed_number : \| signed_number "+" NUMBER : \| signed_number "-" NUMBER : \| strings : \| "None" : \| "True" : \| "False" signed_number: \["+" \| "-"\] NUMBER
The rule `strings` and the token `NUMBER` are defined in the `standard Python grammar <./grammar>`. Triple-quoted strings are supported. Raw strings and byte strings are supported. `f-strings` and `t-strings` are not supported.
The forms `signed_number '+' NUMBER` and `signed_number '-' NUMBER` are for expressing `complex numbers `; they require a real number on the left and an imaginary number on the right. E.g. `3 + 4j`.
In simple terms, `LITERAL` will succeed only if ` == LITERAL`. For the singletons `None`, `True` and `False`, the `is` operator is used.
#### Capture Patterns
A capture pattern binds the subject value to a name. Syntax:
python-grammar capture_pattern: !'\_' NAME
A single underscore `_` is not a capture pattern (this is what `!'_'` expresses). It is instead treated as a `~python-grammar:wildcard_pattern`.
In a given pattern, a given name can only be bound once. E.g. `case x, x: ...` is invalid while `case [x] | x: ...` is allowed.
Capture patterns always succeed. The binding follows scoping rules established by the assignment expression operator in `572`; the name becomes a local variable in the closest containing function scope unless there's an applicable `global` or `nonlocal` statement.
In simple terms `NAME` will always succeed and it will set `NAME = `.
#### Wildcard Patterns
A wildcard pattern always succeeds (matches anything) and binds no name. Syntax:
python-grammar wildcard_pattern: '\_'
`_` is a `soft keyword ` within any pattern, but only within patterns. It is an identifier, as usual, even within `match` subject expressions, `guard`s, and `case` blocks.
In simple terms, `_` will always succeed.
#### Value Patterns
A value pattern represents a named value in Python. Syntax:
python-grammar value_pattern: attr attr: name_or_attr "." NAME name_or_attr: attr \| NAME
The dotted name in the pattern is looked up using standard Python `name resolution rules `. The pattern succeeds if the value found compares equal to the subject value (using the `==` equality operator).
In simple terms `NAME1.NAME2` will succeed only if ` == NAME1.NAME2`
> [!NOTE]
> If the same value occurs multiple times in the same match statement, the interpreter may cache the first value found and reuse it rather than repeat the same lookup. This cache is strictly tied to a given execution of a given match statement.
#### Group Patterns
A group pattern allows users to add parentheses around patterns to emphasize the intended grouping. Otherwise, it has no additional syntax. Syntax:
python-grammar group_pattern: "(" pattern ")"
In simple terms `(P)` has the same effect as `P`.
#### Sequence Patterns
A sequence pattern contains several subpatterns to be matched against sequence elements. The syntax is similar to the unpacking of a list or tuple.
There is no difference if parentheses or square brackets are used for sequence patterns (i.e. `(...)` vs `[...]` ).
> [!NOTE]
> A single pattern enclosed in parentheses without a trailing comma (e.g. `(3 | 4)`) is a `group pattern `. While a single pattern enclosed in square brackets (e.g. `[3 | 4]`) is still a sequence pattern.
At most one star subpattern may be in a sequence pattern. The star subpattern may occur in any position. If no star subpattern is present, the sequence pattern is a fixed-length sequence pattern; otherwise it is a variable-length sequence pattern.
The following is the logical flow for matching a sequence pattern against a subject value:
1. If the subject value is not a sequence[^2], the sequence pattern fails.
2. If the subject value is an instance of `str`, `bytes` or `bytearray` the sequence pattern fails.
3. The subsequent steps depend on whether the sequence pattern is fixed or variable-length.
If the sequence pattern is fixed-length:
1. If the length of the subject sequence is not equal to the number of subpatterns, the sequence pattern fails
2. Subpatterns in the sequence pattern are matched to their corresponding items in the subject sequence from left to right. Matching stops as soon as a subpattern fails. If all subpatterns succeed in matching their corresponding item, the sequence pattern succeeds.
Otherwise, if the sequence pattern is variable-length:
1. If the length of the subject sequence is less than the number of non-star subpatterns, the sequence pattern fails.
2. The leading non-star subpatterns are matched to their corresponding items as for fixed-length sequences.
3. If the previous step succeeds, the star subpattern matches a list formed of the remaining subject items, excluding the remaining items corresponding to non-star subpatterns following the star subpattern.
4. Remaining non-star subpatterns are matched to their corresponding subject items, as for a fixed-length sequence.
> [!NOTE]
> The length of the subject sequence is obtained via `len` (i.e. via the `~object.__len__` protocol). This length may be cached by the interpreter in a similar manner as `value patterns `.
In simple terms `[P1, P2, P3,` ... `, P]` matches only if all the following happens:
- check `` is a sequence
- `len(subject) == `
- `P1` matches `[0]` (note that this match can also bind names)
- `P2` matches `[1]` (note that this match can also bind names)
- ... and so on for the corresponding pattern/element.
#### Mapping Patterns
A mapping pattern contains one or more key-value patterns. The syntax is similar to the construction of a dictionary. Syntax:
At most one double star pattern may be in a mapping pattern. The double star pattern must be the last subpattern in the mapping pattern.
Duplicate keys in mapping patterns are disallowed. Duplicate literal keys will raise a `SyntaxError`. Two keys that otherwise have the same value will raise a `ValueError` at runtime.
The following is the logical flow for matching a mapping pattern against a subject value:
1. If the subject value is not a mapping[^3],the mapping pattern fails.
2. If every key given in the mapping pattern is present in the subject mapping, and the pattern for each key matches the corresponding item of the subject mapping, the mapping pattern succeeds.
3. If duplicate keys are detected in the mapping pattern, the pattern is considered invalid. A `SyntaxError` is raised for duplicate literal values; or a `ValueError` for named keys of the same value.
> [!NOTE]
> Key-value pairs are matched using the two-argument form of the mapping subject's `get()` method. Matched key-value pairs must already be present in the mapping, and not created on-the-fly via `~object.__missing__` or `~object.__getitem__`.
In simple terms `{KEY1: P1, KEY2: P2, ... }` matches only if all the following happens:
- check `` is a mapping
- `KEY1 in `
- `P1` matches `[KEY1]`
- ... and so on for the corresponding KEY/pattern pair.
#### Class Patterns
A class pattern represents a class and its positional and keyword arguments (if any). Syntax:
The same keyword should not be repeated in class patterns.
The following is the logical flow for matching a class pattern against a subject value:
1. If `name_or_attr` is not an instance of the builtin `type` , raise `TypeError`.
2. If the subject value is not an instance of `name_or_attr` (tested via `isinstance`), the class pattern fails.
3. If no pattern arguments are present, the pattern succeeds. Otherwise, the subsequent steps depend on whether keyword or positional argument patterns are present.
For a number of built-in types (specified below), a single positional subpattern is accepted which will match the entire subject; for these types keyword patterns also work as for other types.
If only keyword patterns are present, they are processed as follows, one by one:
1. The keyword is looked up as an attribute on the subject.
- If this raises an exception other than `AttributeError`, the exception bubbles up.
- If this raises `AttributeError`, the class pattern has failed.
- Else, the subpattern associated with the keyword pattern is matched against the subject's attribute value. If this fails, the class pattern fails; if this succeeds, the match proceeds to the next keyword.
2. If all keyword patterns succeed, the class pattern succeeds.
If any positional patterns are present, they are converted to keyword patterns using the `~object.__match_args__` attribute on the class `name_or_attr` before matching:
1. The equivalent of `getattr(cls, "__match_args__", ())` is called.
- If this raises an exception, the exception bubbles up.
- If the returned value is not a tuple, the conversion fails and `TypeError` is raised.
- If there are more positional patterns than `len(cls.__match_args__)`, `TypeError` is raised.
- Otherwise, positional pattern `i` is converted to a keyword pattern using `__match_args__[i]` as the keyword. `__match_args__[i]` must be a string; if not `TypeError` is raised.
- If there are duplicate keywords, `TypeError` is raised.
`class-pattern-matching`
2. Once all positional patterns have been converted to keyword patterns, the match proceeds as if there were only keyword patterns.
For the following built-in types the handling of positional subpatterns is different:
- `bool`
- `bytearray`
- `bytes`
- `dict`
- `float`
- `frozendict`
- `frozenset`
- `int`
- `list`
- `set`
- `str`
- `tuple`
These classes accept a single positional argument, and the pattern there is matched against the whole object rather than an attribute. For example `int(0|1)` matches the value `0`, but not the value `0.0`.
In simple terms `CLS(P1, attr=P2)` matches only if the following happens:
- `isinstance(, CLS)`
- convert `P1` to a keyword pattern using `CLS.__match_args__`
- For each keyword argument `attr=P2`:
- `hasattr(, "attr")`
- `P2` matches `.attr`
- ... and so on for the corresponding keyword argument/pattern pair.
A function definition is an executable statement. Its execution binds the function name in the current local namespace to a function object (a wrapper around the executable code for the function). This function object contains a reference to the current global namespace as the global namespace to be used when the function is called.
The function definition does not execute the function body; this gets executed only when the function is called.[^4]
single: @ (at); function definition
A function definition may be wrapped by one or more `decorator` expressions. Decorator expressions are evaluated when the function is defined, in the scope that contains the function definition. The result must be a callable, which is invoked with the function object as the only argument. The returned value is bound to the function name instead of the function object. Multiple decorators are applied in nested fashion. For example, the following code :
@f1(arg)
@f2
def func(): pass
is roughly equivalent to :
def func(): pass
func = f1(arg)(f2(func))
except that the original function is not temporarily bound to the name `func`.
3.9 Functions may be decorated with any valid `~python-grammar:assignment_expression`. Previously, the grammar was much more restrictive; see `614` for details.
A list of `type parameters ` may be given in square brackets between the function's name and the opening parenthesis for its parameter list. This indicates to static type checkers that the function is generic. At runtime, the type parameters can be retrieved from the function's `~function.__type_params__` attribute. See `generic-functions` for more.
3.12 Type parameter lists are new in Python 3.12.
triple: default; parameter; value single: argument; function definition single: = (equals); function definition
When one or more `parameters ` have the form *parameter* `=` *expression*, the function is said to have "default parameter values." For a parameter with a default value, the corresponding `argument` may be omitted from a call, in which case the parameter's default value is substituted. If a parameter has a default value, all following parameters up until the "`*`" must also have a default value --- this is a syntactic restriction that is not expressed by the grammar.
**Default parameter values are evaluated from left to right when the function definition is executed.** This means that the expression is evaluated once, when the function is defined, and that the same "pre-computed" value is used for each call. This is especially important to understand when a default parameter value is a mutable object, such as a list or a dictionary: if the function modifies the object (e.g. by appending an item to a list), the default parameter value is in effect modified. This is generally not what was intended. A way around this is to use `None` as the default, and explicitly test for it in the body of the function, for example:
def whats_on_the_telly(penguin=None):
if penguin is None:
penguin = []
penguin.append("property of the zoo")
return penguin
single: / (slash); function definition single: \* (asterisk); function definition single: \*\*; function definition
Function call semantics are described in more detail in section `calls`. A function call always assigns values to all parameters mentioned in the parameter list, either from positional arguments, from keyword arguments, or from default values. If the form "`*identifier`" is present, it is initialized to a tuple receiving any excess positional parameters, defaulting to the empty tuple. If the form "`**identifier`" is present, it is initialized to a new ordered mapping receiving any excess keyword arguments, defaulting to a new empty mapping of the same type. Parameters after "`*`" or "`*identifier`" are keyword-only parameters and may only be passed by keyword arguments. Parameters before "`/`" are positional-only parameters and may only be passed by positional arguments.
3.8 The `/` function parameter syntax may be used to indicate positional-only parameters. See `570` for details.
pair: function; annotations single: -\>; function annotations single: : (colon); function annotations
Parameters may have an `annotation ` of the form "`: expression`" following the parameter name. Any parameter may have an annotation, even those of the form `*identifier` or `**identifier`. (As a special case, parameters of the form `*identifier` may have an annotation "`: *expression`".) Functions may have "return" annotation of the form "`-> expression`" after the parameter list. These annotations can be any valid Python expression. The presence of annotations does not change the semantics of a function. See `annotations` for more information on annotations.
3.11 Parameters of the form "`*identifier`" may have an annotation "`: *expression`". See `646`.
pair: lambda; expression
It is also possible to create anonymous functions (functions not bound to a name), for immediate use in expressions. This uses lambda expressions, described in section `lambda`. Note that the lambda expression is merely a shorthand for a simplified function definition; a function defined in a "`def`" statement can be passed around or assigned to another name just like a function defined by a lambda expression. The "`!def`" form is actually more powerful since it allows the execution of multiple statements and annotations.
**Programmer's note:** Functions are first-class objects. A "`def`" statement executed inside a function definition defines a local function that can be returned or passed around. Free variables used in the nested function can access the local variables of the function containing the def. See section `naming` for details.
`3107` - Function Annotations
The original specification for function annotations.
`484` - Type Hints
Definition of a standard meaning for annotations: type hints.
`526` - Syntax for Variable Annotations
Ability to type hint variable declarations, including class variables and instance variables.
`563` - Postponed Evaluation of Annotations
Support for forward references within annotations by preserving annotations in a string form at runtime instead of eager evaluation.
`318` - Decorators for Functions and Methods
Function and method decorators were introduced. Class decorators were introduced in `3129`.
## Class definitions
pair: object; class pair: statement; class pair: class; definition pair: class; name pair: name; binding pair: execution; frame single: inheritance single: docstring single: () (parentheses); class definition single: , (comma); expression list single: : (colon); compound statement
A class definition defines a class object (see section `types`):
A class definition is an executable statement. The inheritance list usually gives a list of base classes (see `metaclasses` for more advanced uses), so each item in the list should evaluate to a class object which allows subclassing. Classes without an inheritance list inherit, by default, from the base class `object`; hence, :
class Foo:
pass
is equivalent to :
class Foo(object):
pass
There may be one or more base classes; see `multiple-inheritance` below for more information.
The class's suite is then executed in a new execution frame (see `naming`), using a newly created local namespace and the original global namespace. (Usually, the suite contains mostly function definitions.) When the class's suite finishes execution, its execution frame is discarded but its local namespace is saved.[^5] A class object is then created using the inheritance list for the base classes and the saved local namespace for the attribute dictionary. The class name is bound to this class object in the original local namespace.
The order in which attributes are defined in the class body is preserved in the new class's `~type.__dict__`. Note that this is reliable only right after the class is created and only for classes that were defined using the definition syntax.
Class creation can be customized heavily using `metaclasses `.
single: @ (at); class definition
Classes can also be decorated: just like when decorating functions, :
@f1(arg)
@f2
class Foo: pass
is roughly equivalent to :
class Foo: pass
Foo = f1(arg)(f2(Foo))
The evaluation rules for the decorator expressions are the same as for function decorators. The result is then bound to the class name.
3.9 Classes may be decorated with any valid `~python-grammar:assignment_expression`. Previously, the grammar was much more restrictive; see `614` for details.
A list of `type parameters ` may be given in square brackets immediately after the class's name. This indicates to static type checkers that the class is generic. At runtime, the type parameters can be retrieved from the class's `~type.__type_params__` attribute. See `generic-classes` for more.
3.12 Type parameter lists are new in Python 3.12.
**Programmer's note:** Variables defined in the class definition are class attributes; they are shared by instances. Instance attributes can be set in a method with `self.name = value`. Both class and instance attributes are accessible through the notation "`self.name`", and an instance attribute hides a class attribute with the same name when accessed in this way. Class attributes can be used as defaults for instance attributes, but using mutable values there can lead to unexpected results. `Descriptors ` can be used to create instance variables with different implementation details.
`3115` - Metaclasses in Python 3000
The proposal that changed the declaration of metaclasses to the current syntax, and the semantics for how classes with metaclasses are constructed.
`3129` - Class Decorators
The proposal that added class decorators. Function and method decorators were introduced in `318`.
### Multiple inheritance
Python classes may have multiple base classes, a technique known as *multiple inheritance*. The base classes are specified in the class definition by listing them in parentheses after the class name, separated by commas. For example, the following class definition:
\>\>\> class A: pass \>\>\> class B: pass \>\>\> class C(A, B): pass
defines a class `C` that inherits from classes `A` and `B`.
The `method resolution order` (MRO) is the order in which base classes are searched when looking up an attribute on a class. See `python_2.3_mro` for a description of how Python determines the MRO for a class.
Multiple inheritance is not always allowed. Attempting to define a class with multiple inheritance will raise an error if one of the bases does not allow subclassing, if a consistent MRO cannot be created, if no valid metaclass can be determined, or if there is an instance layout conflict. We'll discuss each of these in turn.
First, all base classes must allow subclassing. While most classes allow subclassing, some built-in classes do not, such as `bool`:
\>\>\> class SubBool(bool): \# TypeError ... pass Traceback (most recent call last): ... TypeError: type 'bool' is not an acceptable base type
In the resolved MRO of a class, the class's bases appear in the order they were specified in the class's bases list. Additionally, the MRO always lists a child class before any of its bases. A class definition will fail if it is impossible to resolve a consistent MRO that satisfies these rules from the list of bases provided:
\>\>\> class Base: pass \>\>\> class Child(Base): pass \>\>\> class Grandchild(Base, Child): pass \# TypeError Traceback (most recent call last): ... TypeError: Cannot create a consistent method resolution order (MRO) for bases Base, Child
In the MRO of `Grandchild`, `Base` must appear before `Child` because it is first in the base class list, but it must also appear after `Child` because it is a parent of `Child`. This is a contradiction, so the class cannot be defined.
If some of the bases have a custom `metaclass`, the metaclass of the resulting class is chosen among the metaclasses of the bases and the explicitly specified metaclass of the child class. It must be a metaclass that is a subclass of all other candidate metaclasses. If no such metaclass exists among the candidates, the class cannot be created, as explained in `metaclass-determination`.
Finally, the instance layouts of the bases must be compatible. This means that it must be possible to compute a *solid base* for the class. Exactly which classes are solid bases depends on the Python implementation.
In CPython, a class is a solid base if it has a nonempty `~object.__slots__` definition. Many but not all classes defined in C are also solid bases, including most builtins (such as `int` or `BaseException`) but excluding most concrete `Exception` classes. Generally, a C class is a solid base if its underlying struct is different in size from its base class.
Every class has a solid base. `object`, the base class, has itself as its solid base. If there is a single base, the child class's solid base is that class if it is a solid base, or else the base class's solid base. If there are multiple bases, we first find the solid base for each base class to produce a list of candidate solid bases. If there is a unique solid base that is a subclass of all others, then that class is the solid base. Otherwise, class creation fails.
Example:
\>\>\> class Solid1: ... \_\_slots\_\_ = ("solid1",) \>\>\> \>\>\> class Solid2: ... \_\_slots\_\_ = ("solid2",) \>\>\> \>\>\> class SolidChild(Solid1): ... \_\_slots\_\_ = ("solid_child",) \>\>\> \>\>\> class C1: \# solid base is object ... pass \>\>\> \>\>\> \# OK: solid bases are Solid1 and object, and Solid1 is a subclass of object. \>\>\> class C2(Solid1, C1): \# solid base is Solid1 ... pass \>\>\> \>\>\> \# OK: solid bases are SolidChild and Solid1, and SolidChild is a subclass of Solid1. \>\>\> class C3(SolidChild, Solid1): \# solid base is SolidChild ... pass \>\>\> \>\>\> \# Error: solid bases are Solid1 and Solid2, but neither is a subclass of the other. \>\>\> class C4(Solid1, Solid2): \# error: no single solid base ... pass Traceback (most recent call last): ... TypeError: multiple bases have instance lay-out conflict
Execution of Python coroutines can be suspended and resumed at many points (see `coroutine`). `await` expressions, `async for` and `async with` can only be used in the body of a coroutine function.
Functions defined with `async def` syntax are always coroutine functions, even if they do not contain `await` or `async` keywords.
It is a `SyntaxError` to use a `yield from` expression inside the body of a coroutine function.
An example of a coroutine function:
async def func(param1, param2):
do_stuff()
await some_coroutine()
3.7 `await` and `async` are now keywords; previously they were only treated as such inside the body of a coroutine function.
pair: statement; async for
### The `!async for` statement
python-grammar async_for_stmt: "async" for_stmt
An `asynchronous iterable` provides an `__aiter__` method that directly returns an `asynchronous iterator`, which can call asynchronous code in its `__anext__` method.
The `async for` statement allows convenient iteration over asynchronous iterables.
The following code:
async for TARGET in ITER:
SUITE
else:
SUITE2
Is semantically equivalent to:
iter = (ITER).__aiter__()
running = True
while running:
try:
TARGET = await iter.__anext__()
except StopAsyncIteration:
running = False
else:
SUITE
else:
SUITE2
except that implicit `special method lookup ` is used for `~object.__aiter__` and `~object.__anext__`.
It is a `SyntaxError` to use an `async for` statement outside the body of a coroutine function.
pair: statement; async with
### The `!async with` statement
python-grammar async_with_stmt: "async" with_stmt
An `asynchronous context manager` is a `context manager` that is able to suspend execution in its *enter* and *exit* methods.
The following code:
async with EXPRESSION as TARGET:
SUITE
is semantically equivalent to:
manager = (EXPRESSION)
aenter = manager.__aenter__
aexit = manager.__aexit__
value = await aenter()
hit_except = False
try:
TARGET = value
SUITE
except:
hit_except = True
if not await aexit(*sys.exc_info()):
raise
finally:
if not hit_except:
await aexit(None, None, None)
except that implicit `special method lookup ` is used for `~object.__aenter__` and `~object.__aexit__`.
It is a `SyntaxError` to use an `async with` statement outside the body of a coroutine function.
`492` - Coroutines with async and await syntax
The proposal that made coroutines a proper standalone concept in Python, and added supporting syntax.
## Type parameter lists
3.12
3.13 Support for default values was added (see `696`).
`Functions ` (including `coroutines `), `classes ` and `type aliases ` may contain a type parameter list:
def max[T](args: list[T]) -> T:
...
async def amax[T](args: list[T]) -> T:
...
class Bag[T]:
def __iter__(self) -> Iterator[T]:
...
def add(self, arg: T) -> None:
...
type ListOrSet[T] = list[T] | set[T]
Semantically, this indicates that the function, class, or type alias is generic over a type variable. This information is primarily used by static type checkers, and at runtime, generic objects behave much like their non-generic counterparts.
Type parameters are declared in square brackets (`[]`) immediately after the name of the function, class, or type alias. The type parameters are accessible within the scope of the generic object, but not elsewhere. Thus, after a declaration `def func[T](): pass`, the name `T` is not available in the module scope. Below, the semantics of generic objects are described with more precision. The scope of type parameters is modeled with a special function (technically, an `annotation scope `) that wraps the creation of the generic object.
Generic functions, classes, and type aliases have a `~definition.__type_params__` attribute listing their type parameters.
Type parameters come in three kinds:
- `typing.TypeVar`, introduced by a plain name (e.g., `T`). Semantically, this represents a single type to a type checker.
- `typing.TypeVarTuple`, introduced by a name prefixed with a single asterisk (e.g., `*Ts`). Semantically, this stands for a tuple of any number of types.
- `typing.ParamSpec`, introduced by a name prefixed with two asterisks (e.g., `**P`). Semantically, this stands for the parameters of a callable.
`typing.TypeVar` declarations can define *bounds* and *constraints* with a colon (`:`) followed by an expression. A single expression after the colon indicates a bound (e.g. `T: int`). Semantically, this means that the `!typing.TypeVar` can only represent types that are a subtype of this bound. A parenthesized tuple of expressions after the colon indicates a set of constraints (e.g. `T: (str, bytes)`). Each member of the tuple should be a type (again, this is not enforced at runtime). Constrained type variables can only take on one of the types in the list of constraints.
For `!typing.TypeVar`s declared using the type parameter list syntax, the bound and constraints are not evaluated when the generic object is created, but only when the value is explicitly accessed through the attributes `__bound__` and `__constraints__`. To accomplish this, the bounds or constraints are evaluated in a separate `annotation scope `.
`typing.TypeVarTuple`s and `typing.ParamSpec`s cannot have bounds or constraints.
All three flavors of type parameters can also have a *default value*, which is used when the type parameter is not explicitly provided. This is added by appending a single equals sign (`=`) followed by an expression. Like the bounds and constraints of type variables, the default value is not evaluated when the object is created, but only when the type parameter's `__default__` attribute is accessed. To this end, the default value is evaluated in a separate `annotation scope `. If no default value is specified for a type parameter, the `__default__` attribute is set to the special sentinel object `typing.NoDefault`.
The following example indicates the full set of allowed type parameter declarations:
def overly_generic[
SimpleTypeVar,
TypeVarWithBound: int,
TypeVarWithConstraints: (str, bytes),
TypeVarWithDefault = int,
*SimpleTypeVarTuple = (int, float),
**SimpleParamSpec = (str, bytearray),
](
a: SimpleTypeVar,
b: TypeVarWithDefault,
c: TypeVarWithBound,
d: Callable[SimpleParamSpec, TypeVarWithConstraints],
*e: SimpleTypeVarTuple,
): ...
### Generic functions
Generic functions are declared as follows:
def func[T](arg: T): ...
This syntax is equivalent to:
annotation-def TYPE_PARAMS_OF_func():
T = typing.TypeVar("T")
def func(arg: T): ...
func.__type_params__ = (T,)
return func
func = TYPE_PARAMS_OF_func()
Here `annotation-def` indicates an `annotation scope `, which is not actually bound to any name at runtime. (One other liberty is taken in the translation: the syntax does not go through attribute access on the `typing` module, but creates an instance of `typing.TypeVar` directly.)
The annotations of generic functions are evaluated within the annotation scope used for declaring the type parameters, but the function's defaults and decorators are not.
The following example illustrates the scoping rules for these cases, as well as for additional flavors of type parameters:
@decorator
def func[T: int, *Ts, **P](*args: *Ts, arg: Callable[P, T] = some_default):
...
Except for the `lazy evaluation ` of the `~typing.TypeVar` bound, this is equivalent to:
DEFAULT_OF_arg = some_default
annotation-def TYPE_PARAMS_OF_func():
annotation-def BOUND_OF_T():
return int
# In reality, BOUND_OF_T() is evaluated only on demand.
T = typing.TypeVar("T", bound=BOUND_OF_T())
Ts = typing.TypeVarTuple("Ts")
P = typing.ParamSpec("P")
def func(*args: *Ts, arg: Callable[P, T] = DEFAULT_OF_arg):
...
func.__type_params__ = (T, Ts, P)
return func
func = decorator(TYPE_PARAMS_OF_func())
The capitalized names like `DEFAULT_OF_arg` are not actually bound at runtime.
### Generic classes
Generic classes are declared as follows:
class Bag[T]: ...
This syntax is equivalent to:
annotation-def TYPE_PARAMS_OF_Bag():
T = typing.TypeVar("T")
class Bag(typing.Generic[T]):
__type_params__ = (T,)
...
return Bag
Bag = TYPE_PARAMS_OF_Bag()
Here again `annotation-def` (not a real keyword) indicates an `annotation scope `, and the name `TYPE_PARAMS_OF_Bag` is not actually bound at runtime.
Generic classes implicitly inherit from `typing.Generic`. The base classes and keyword arguments of generic classes are evaluated within the type scope for the type parameters, and decorators are evaluated outside that scope. This is illustrated by this example:
@decorator
class Bag[T](Base[T], arg=T): ...
This is equivalent to:
annotation-def TYPE_PARAMS_OF_Bag():
T = typing.TypeVar("T")
class Bag(Base[T], typing.Generic[T], arg=T):
__type_params__ = (T,)
...
return Bag
Bag = decorator(TYPE_PARAMS_OF_Bag())
### Generic type aliases
The `type` statement can also be used to create a generic type alias:
type ListOrSet[T] = list[T] | set[T]
Except for the `lazy evaluation ` of the value, this is equivalent to:
annotation-def TYPE_PARAMS_OF_ListOrSet():
T = typing.TypeVar("T")
annotation-def VALUE_OF_ListOrSet():
return list[T] | set[T]
# In reality, the value is lazily evaluated
return typing.TypeAliasType("ListOrSet", VALUE_OF_ListOrSet(), type_params=(T,))
ListOrSet = TYPE_PARAMS_OF_ListOrSet()
Here, `annotation-def` (not a real keyword) indicates an `annotation scope `. The capitalized names like `TYPE_PARAMS_OF_ListOrSet` are not actually bound at runtime.
## Annotations
3.14 Annotations are now lazily evaluated by default.
Variables and function parameters may carry `annotations `, created by adding a colon after the name, followed by an expression:
x: annotation = 1
def f(param: annotation): ...
Functions may also carry a return annotation following an arrow:
def f() -> annotation: ...
Annotations are conventionally used for `type hints `, but this is not enforced by the language, and in general annotations may contain arbitrary expressions. The presence of annotations does not change the runtime semantics of the code, except if some mechanism is used that introspects and uses the annotations (such as `dataclasses` or `functools.singledispatch`).
By default, annotations are lazily evaluated in an `annotation scope `. This means that they are not evaluated when the code containing the annotation is evaluated. Instead, the interpreter saves information that can be used to evaluate the annotation later if requested. The `annotationlib` module provides tools for evaluating annotations.
If the `future statement ` `from __future__ import annotations` is present, all annotations are instead stored as strings:
>>> from __future__ import annotations
>>> def f(param: annotation): ...
>>> f.__annotations__
{'param': 'annotation'}
This future statement will be deprecated and removed in a future version of Python, but not before Python 3.13 reaches its end of life (see `749`). When it is used, introspection tools like `annotationlib.get_annotations` and `typing.get_type_hints` are less likely to be able to resolve annotations at runtime.
**Footnotes**
[^1]: The exception is propagated to the invocation stack unless there is a `finally` clause which happens to raise another exception. That new exception causes the old one to be lost.
[^2]: In pattern matching, a sequence is defined as one of the following:
- a class that inherits from `collections.abc.Sequence`
- a Python class that has been registered as `collections.abc.Sequence`
- a builtin class that has its (CPython) `Py_TPFLAGS_SEQUENCE` bit set
- a class that inherits from any of the above
The following standard library classes are sequences:
- `array.array`
- `collections.deque`
- `list`
- `memoryview`
- `range`
- `tuple`
> [!NOTE]
> Subject values of type `str`, `bytes`, and `bytearray` do not match sequence patterns.
[^3]: In pattern matching, a mapping is defined as one of the following:
- a class that inherits from `collections.abc.Mapping`
- a Python class that has been registered as `collections.abc.Mapping`
- a builtin class that has its (CPython) `Py_TPFLAGS_MAPPING` bit set
- a class that inherits from any of the above
The standard library classes `dict` and `types.MappingProxyType` are mappings.
[^4]: A string literal appearing as the first statement in the function body is transformed into the function's `~function.__doc__` attribute and therefore the function's `docstring`.
[^5]: A string literal appearing as the first statement in the class body is transformed into the namespace's `~type.__doc__` item and therefore the class's `docstring`.
# Data model
## Objects, values and types
single: object single: data
`Objects` are Python's abstraction for data. All data in a Python program is represented by objects or by relations between objects. Even code is represented by objects.
pair: built-in function; id pair: built-in function; type single: identity of an object single: value of an object single: type of an object single: mutable object single: immutable object
Every object has an identity, a type and a value. An object's *identity* never changes once it has been created; you may think of it as the object's address in memory. The `is` operator compares the identity of two objects; the `id` function returns an integer representing its identity.
For CPython, `id(x)` is the memory address where `x` is stored.
An object's type determines the operations that the object supports (e.g., "does it have a length?") and also defines the possible values for objects of that type. The `type` function returns an object's type (which is an object itself). Like its identity, an object's `type` is also unchangeable. [^1]
The *value* of some objects can change. Objects whose value can change are said to be *mutable*; objects whose value is unchangeable once they are created are called *immutable*. (The value of an immutable container object that contains a reference to a mutable object can change when the latter's value is changed; however the container is still considered immutable, because the collection of objects it contains cannot be changed. So, immutability is not strictly the same as having an unchangeable value, it is more subtle.) An object's mutability is determined by its type; for instance, numbers, strings and tuples are immutable, while dictionaries and lists are mutable.
Objects are never explicitly destroyed; however, when they become unreachable they may be garbage-collected. An implementation is allowed to postpone garbage collection or omit it altogether --- it is a matter of implementation quality how garbage collection is implemented, as long as no objects are collected that are still reachable.
CPython currently uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage, which collects most objects as soon as they become unreachable, but is not guaranteed to collect garbage containing circular references. See the documentation of the `gc` module for information on controlling the collection of cyclic garbage. Other implementations act differently and CPython may change. Do not depend on immediate finalization of objects when they become unreachable (so you should always close files explicitly).
Note that the use of the implementation's tracing or debugging facilities may keep objects alive that would normally be collectable. Also note that catching an exception with a `try`...`except` statement may keep objects alive.
Some objects contain references to "external" resources such as open files or windows. It is understood that these resources are freed when the object is garbage-collected, but since garbage collection is not guaranteed to happen, such objects also provide an explicit way to release the external resource, usually a `!close` method. Programs are strongly recommended to explicitly close such objects. The `try`...`finally` statement and the `with` statement provide convenient ways to do this.
single: container
Some objects contain references to other objects; these are called *containers*. Examples of containers are tuples, lists and dictionaries. The references are part of a container's value. In most cases, when we talk about the value of a container, we imply the values, not the identities of the contained objects; however, when we talk about the mutability of a container, only the identities of the immediately contained objects are implied. So, if an immutable container (like a tuple) contains a reference to a mutable object, its value changes if that mutable object is changed.
Types affect almost all aspects of object behavior. Even the importance of object identity is affected in some sense: for immutable types, operations that compute new values may actually return a reference to any existing object with the same type and value, while for mutable objects this is not allowed. For example, after `a = 1; b = 1`, *a* and *b* may or may not refer to the same object with the value one, depending on the implementation. This is because `int` is an immutable type, so the reference to `1` can be reused. This behaviour depends on the implementation used, so should not be relied upon, but is something to be aware of when making use of object identity tests. However, after `c = []; d = []`, *c* and *d* are guaranteed to refer to two different, unique, newly created empty lists. (Note that `e = f = []` assigns the *same* object to both *e* and *f*.)
## The standard type hierarchy
single: type pair: data; type pair: type; hierarchy pair: extension; module pair: C; language
Below is a list of the types that are built into Python. Extension modules (written in C, Java, or other languages, depending on the implementation) can define additional types. Future versions of Python may add types to the type hierarchy (e.g., rational numbers, efficiently stored arrays of integers, etc.), although such additions will often be provided via the standard library instead.
Some of the type descriptions below contain a paragraph listing 'special attributes.' These are attributes that provide access to the implementation and are not intended for general use. Their definition may change in the future.
### None
pair: object; None
This type has a single value. There is a single object with this value. This object is accessed through the built-in name `None`. It is used to signify the absence of a value in many situations, e.g., it is returned from functions that don't explicitly return anything. Its truth value is false.
### NotImplemented
pair: object; NotImplemented
This type has a single value. There is a single object with this value. This object is accessed through the built-in name `NotImplemented`. Numeric methods and rich comparison methods should return this value if they do not implement the operation for the operands provided. (The interpreter will then try the reflected operation, or some other fallback, depending on the operator.) It should not be evaluated in a boolean context.
See `implementing-the-arithmetic-operations` for more details.
3.9 Evaluating `NotImplemented` in a boolean context was deprecated.
3.14 Evaluating `NotImplemented` in a boolean context now raises a `TypeError`. It previously evaluated to `True` and emitted a `DeprecationWarning` since Python 3.9.
This type has a single value. There is a single object with this value. This object is accessed through the literal `...` or the built-in name `Ellipsis`. Its truth value is true.
### `numbers.Number`
pair: object; numeric
These are created by numeric literals and returned as results by arithmetic operators and arithmetic built-in functions. Numeric objects are immutable; once created their value never changes. Python numbers are of course strongly related to mathematical numbers, but subject to the limitations of numerical representation in computers.
The string representations of the numeric classes, computed by `~object.__repr__` and `~object.__str__`, have the following properties:
- They are valid numeric literals which, when passed to their class constructor, produce an object having the value of the original numeric.
- The representation is in base 10, when possible.
- Leading zeros, possibly excepting a single zero before a decimal point, are not shown.
- Trailing zeros, possibly excepting a single zero after a decimal point, are not shown.
- A sign is shown only when the number is negative.
Python distinguishes between integers, floating-point numbers, and complex numbers:
#### `numbers.Integral`
pair: object; integer
These represent elements from the mathematical set of integers (positive and negative).
> [!NOTE]
> .. index:: pair: integer; representation
>
> The rules for integer representation are intended to give the most meaningful interpretation of shift and mask operations involving negative integers.
There are two types of integers:
Integers (`int`)
These represent numbers in an unlimited range, subject to available (virtual) memory only. For the purpose of shift and mask operations, a binary representation is assumed, and negative numbers are represented in a variant of 2's complement which gives the illusion of an infinite string of sign bits extending to the left.
Booleans (`bool`)
pair: object; Boolean single: False single: True
These represent the truth values False and True. The two objects representing the values `False` and `True` are the only Boolean objects. The Boolean type is a subtype of the integer type, and Boolean values behave like the values 0 and 1, respectively, in almost all contexts, the exception being that when converted to a string, the strings `"False"` or `"True"` are returned, respectively.
#### `numbers.Real` (`float`)
pair: object; floating-point pair: floating-point; number pair: C; language pair: Java; language
These represent machine-level double precision floating-point numbers. You are at the mercy of the underlying machine architecture (and C or Java implementation) for the accepted range and handling of overflow. Python does not support single-precision floating-point numbers; the savings in processor and memory usage that are usually the reason for using these are dwarfed by the overhead of using objects in Python, so there is no reason to complicate the language with two kinds of floating-point numbers.
#### `numbers.Complex` (`complex`)
pair: object; complex pair: complex; number
These represent complex numbers as a pair of machine-level double precision floating-point numbers. The same caveats apply as for floating-point numbers. The real and imaginary parts of a complex number `z` can be retrieved through the read-only attributes `z.real` and `z.imag`.
### Sequences
pair: built-in function; len pair: object; sequence single: index operation single: item selection single: subscription
These represent finite ordered sets indexed by non-negative numbers. The built-in function `len` returns the number of items of a sequence. When the length of a sequence is *n*, the index set contains the numbers 0, 1, ..., *n*-1. Item *i* of sequence *a* is selected by `a[i]`. Some sequences, including built-in sequences, interpret negative subscripts by adding the sequence length. For example, `a[-2]` equals `a[n-2]`, the second to last item of sequence a with length `n`.
The resulting value must be a nonnegative integer less than the number of items in the sequence. If it is not, an `IndexError` is raised.
Sequences also support slicing: `a[start:stop]` selects all items with index *k* such that *start* `<=` *k* `<` *stop*. When used as an expression, a slice is a sequence of the same type. The comment above about negative subscripts also applies to negative slice positions. Note that no error is raised if a slice position is less than zero or larger than the length of the sequence.
If *start* is missing or `None`, slicing behaves as if *start* was zero. If *stop* is missing or `None`, slicing behaves as if *stop* was equal to the length of the sequence.
Some sequences also support "extended slicing" with a third "step" parameter: `a[i:j:k]` selects all items of *a* with index *x* where `x = i + n*k`, *n* `>=` `0` and *i* `<=` *x* `<` *j*.
Sequences are distinguished according to their mutability:
#### Immutable sequences
An object of an immutable sequence type cannot change once it is created. (If the object contains references to other objects, these other objects may be mutable and may be changed; however, the collection of objects directly referenced by an immutable object cannot change.)
The following types are immutable sequences:
single: string; immutable sequences
Strings
pair: built-in function; chr pair: built-in function; ord single: character pair: string; item single: Unicode
A string (`str`) is a sequence of values that represent `characters`, or more formally, *Unicode code points*. All the code points in the range `0` to `0x10FFFF` can be represented in a string.
Python doesn't have a dedicated *character* type. Instead, every code point in the string is represented as a string object with length `1`.
The built-in function `ord` converts a code point from its string form to an integer in the range `0` to `0x10FFFF`; `chr` converts an integer in the range `0` to `0x10FFFF` to the corresponding length `1` string object. `str.encode` can be used to convert a `str` to `bytes` using the given text encoding, and `bytes.decode` can be used to achieve the opposite.
Tuples
The items of a `tuple` are arbitrary Python objects. Tuples of two or more items are formed by comma-separated lists of expressions. A tuple of one item (a 'singleton') can be formed by affixing a comma to an expression (an expression by itself does not create a tuple, since parentheses must be usable for grouping of expressions). An empty tuple can be formed by an empty pair of parentheses.
Bytes
bytes, byte
A `bytes` object is an immutable array. The items are 8-bit bytes, represented by integers in the range 0 \<= x \< 256. Bytes literals (like `b'abc'`) and the built-in `bytes` constructor can be used to create bytes objects. Also, bytes objects can be decoded to strings via the `~bytes.decode` method.
#### Mutable sequences
Mutable sequences can be changed after they are created. The subscription and slicing notations can be used as the target of assignment and `del` (delete) statements.
> [!NOTE]
> .. index:: pair: module; array .. index:: pair: module; collections
>
> The `collections` and `array` module provide additional examples of mutable sequence types.
There are currently two intrinsic mutable sequence types:
Lists
pair: object; list
The items of a list are arbitrary Python objects. Lists are formed by placing a comma-separated list of expressions in square brackets. (Note that there are no special cases needed to form lists of length 0 or 1.)
Byte Arrays
bytearray
A bytearray object is a mutable array. They are created by the built-in `bytearray` constructor. Aside from being mutable (and hence unhashable), byte arrays otherwise provide the same interface and functionality as immutable `bytes` objects.
### Set types
pair: built-in function; len pair: object; set type
These represent unordered, finite sets of unique, immutable objects. As such, they cannot be indexed by any subscript. However, they can be iterated over, and the built-in function `len` returns the number of items in a set. Common uses for sets are fast membership testing, removing duplicates from a sequence, and computing mathematical operations such as intersection, union, difference, and symmetric difference.
For set elements, the same immutability rules apply as for dictionary keys. Note that numeric types obey the normal rules for numeric comparison: if two numbers compare equal (e.g., `1` and `1.0`), only one of them can be contained in a set.
There are currently two intrinsic set types:
Sets
pair: object; set
These represent a mutable set. They are created by the built-in `set` constructor and can be modified afterwards by several methods, such as `~set.add`.
Frozen sets
pair: object; frozenset
These represent an immutable set. They are created by the built-in `frozenset` constructor. As a frozenset is immutable and `hashable`, it can be used again as an element of another set, or as a dictionary key.
### Mappings
pair: built-in function; len single: subscription pair: object; mapping
These represent finite sets of objects indexed by arbitrary index sets. The subscript notation `a[k]` selects the item indexed by `k` from the mapping `a`; this can be used in expressions and as the target of assignments or `del` statements. The built-in function `len` returns the number of items in a mapping.
There are two intrinsic mapping types:
#### Dictionaries
pair: object; dictionary
These represent finite sets of objects indexed by nearly arbitrary values. The only types of values not acceptable as keys are values containing lists or dictionaries or other mutable types that are compared by value rather than by object identity, the reason being that the efficient implementation of dictionaries requires a key's hash value to remain constant. Numeric types used for keys obey the normal rules for numeric comparison: if two numbers compare equal (e.g., `1` and `1.0`) then they can be used interchangeably to index the same dictionary entry.
Dictionaries preserve insertion order, meaning that keys will be produced in the same order they were added sequentially over the dictionary. Replacing an existing key does not change the order, however removing a key and re-inserting it will add it to the end instead of keeping its old place.
Dictionaries are mutable; they can be created by the `{}` notation (see section `dict`).
pair: module; dbm.ndbm pair: module; dbm.gnu
The extension modules `dbm.ndbm` and `dbm.gnu` provide additional examples of mapping types, as does the `collections` module.
3.7 Dictionaries did not preserve insertion order in versions of Python before 3.6. In CPython 3.6, insertion order was preserved, but it was considered an implementation detail at that time rather than a language guarantee.
#### Frozen dictionaries
pair: object; frozendict
These represent an immutable dictionary. They are created by the built-in `frozendict` constructor. A frozendict is `hashable` if all of its keys and values are hashable, in which case it can be used as an element of a set, or as a key in another mapping. `!frozendict` is not a subclass of `dict`; it inherits directly from `object`.
These are the types to which the function call operation (see section `calls`) can be applied:
#### User-defined functions
pair: user-defined; function pair: object; function pair: object; user-defined function
A user-defined function object is created by a function definition (see section `function`). It should be called with an argument list containing the same number of items as the function's formal parameter list.
##### Special read-only attributes
A reference to the dictionary <dict> that holds the function's builtins namespace.
3.10
function.__globals__
A reference to the dictionary <dict> that holds the function's global variables <naming> -- the global namespace of the module in which the function was defined.
function.__closure__
None or a tuple of cells that contain bindings for the names specified in the ~codeobject.co_freevars attribute of the function's code object <function.__code__>.
A cell object has the attribute cell_contents. This can be used to get the value of the cell, as well as set the value.
Most of these attributes check the type of the assigned value:
Attribute
Meaning
function.__doc__
The function's documentation string, or None if unavailable.
function.__name__
The function's name. See also: __name__ attributes <definition.__name__>.
function.__qualname__
The function's qualified name. See also: __qualname__ attributes <definition.__qualname__>.
3.3
function.__module__
The name of the module the function was defined in, or None if unavailable.
function.__defaults__
A tuple containing default parameter values for those parameters that have defaults, or None if no parameters have a default value.
function.__code__
The code object <code-objects> representing the compiled function body.
function.__dict__
The namespace supporting arbitrary function attributes. See also: __dict__ attributes <object.__dict__>.
function.__annotations__
A dictionary <dict> containing annotations of parameters <parameter>. The keys of the dictionary are the parameter names, and 'return' for the return annotation, if provided. See also: object.__annotations__.
3.14 Annotations are now lazily evaluated <lazy-evaluation>. See 649.
function.__annotate__
The annotate function for this function, or None if the function has no annotations. See object.__annotate__.
3.14
function.__kwdefaults__
A dictionary <dict> containing defaults for keyword-only parameters <parameter>.
function.__type_params__
A tuple containing the type parameters <type-params> of a generic function <generic-functions>.
3.12
Function objects also support getting and setting arbitrary attributes, which can be used, for example, to attach metadata to functions. Regular attribute dot-notation is used to get and set such attributes.
CPython's current implementation only supports function attributes on user-defined functions. Function attributes on `built-in functions ` may be supported in the future.
Additional information about a function's definition can be retrieved from its `code object ` (accessible via the `~function.__code__` attribute).
#### Instance methods
Refers to the class instance object to which the method is bound <method-binding>
method.__func__
Refers to the original function object <user-defined-funcs>
method.__doc__
The method's documentation (same as method.__func__.__doc__ <function.__doc__>). A string <str> if the original function had a docstring, else None.
method.__name__
The name of the method (same as method.__func__.__name__ <function.__name__>)
method.__module__
The name of the module the method was defined in, or None if unavailable.
Methods also support accessing (but not setting) the arbitrary function attributes on the underlying `function object `.
User-defined method objects may be created when getting an attribute of a class (perhaps via an instance of that class), if that attribute is a user-defined `function object ` or a `classmethod` object.
When an instance method object is created by retrieving a user-defined `function object ` from a class via one of its instances, its `~method.__self__` attribute is the instance, and the method object is said to be *bound*. The new method's `~method.__func__` attribute is the original function object.
When an instance method object is created by retrieving a `classmethod` object from a class or instance, its `~method.__self__` attribute is the class itself, and its `~method.__func__` attribute is the function object underlying the class method.
When an instance method object is called, the underlying function (`~method.__func__`) is called, inserting the class instance (`~method.__self__`) in front of the argument list. For instance, when `!C` is a class which contains a definition for a function `!f`, and `x` is an instance of `!C`, calling `x.f(1)` is equivalent to calling `C.f(x, 1)`.
When an instance method object is derived from a `classmethod` object, the "class instance" stored in `~method.__self__` will actually be the class itself, so that calling either `x.f(1)` or `C.f(1)` is equivalent to calling `f(C,1)` where `f` is the underlying function.
It is important to note that user-defined functions which are attributes of a class instance are not converted to bound methods; this *only* happens when the function is an attribute of the class.
#### Generator functions
single: generator; function single: generator; iterator
A function or method which contains a `yield` expression (see section `yieldexpr`) is called a `generator function`. Such a function, when called, always returns an `iterator` object which can be used to execute the body of the function: calling the iterator's `iterator.__next__` method will cause the function to execute until it provides a value using the `!yield` expression. When the function executes a `return` statement or falls off the end, a `StopIteration` exception is raised and the iterator will have reached the end of the set of values to be returned.
#### Coroutine functions
single: coroutine; function
A function or method which is defined using `async def` is called a `coroutine function`. Such a function, when called, returns a `coroutine` object. It may contain `await` expressions, as well as `async with` and `async for` statements. See also the `coroutine-objects` section.
#### Asynchronous generator functions
single: asynchronous generator; function single: asynchronous generator; asynchronous iterator
A function or method which is defined using `async def` and which contains a `yield` expression is called a `asynchronous generator function`. Such a function, when called, returns an `asynchronous iterator` object which can be used in an `async for` statement to execute the body of the function.
Calling the asynchronous iterator's `aiterator.__anext__ ` method will return an `awaitable` which when awaited will execute until it provides a value using the `yield` expression. When the function executes an empty `return` statement or falls off the end, a `StopAsyncIteration` exception is raised and the asynchronous iterator will have reached the end of the set of values to be yielded.
#### Built-in functions
pair: object; built-in function pair: object; function pair: C; language
A built-in function object is a wrapper around a C function. Examples of built-in functions are `len` and `math.sin` (`math` is a standard built-in module). The number and type of the arguments are determined by the C function. Special read-only attributes:
- `!__doc__` is the function's documentation string, or `None` if unavailable. See `function.__doc__`.
- `!__name__` is the function's name. See `function.__name__`.
- `!__self__` is the module object the function was defined in or `None` if unavailable.
- `!__module__` is the name of the module the function was defined in or `None` if unavailable. See `function.__module__`.
#### Built-in methods
This is really a different disguise of a built-in function, this time containing an object passed to the C function as an implicit extra argument. An example of a built-in method is `alist.append()`, assuming *alist* is a list object. In this case, the special read-only attribute `!__self__` is set to the object denoted by *alist*. (The attribute has the same semantics as it does with `other instance methods `.)
#### Classes
Classes are callable. These objects normally act as factories for new instances of themselves, but variations are possible for class types that override `~object.__new__`. The arguments of the call are passed to `!__new__` and, in the typical case, to `~object.__init__` to initialize the new instance.
#### Class Instances
Instances of arbitrary classes can be made callable by defining a `~object.__call__` method in their class.
### Modules
pair: statement; import pair: object; module
Modules are a basic organizational unit of Python code, and are created by the `import system ` as invoked either by the `import` statement, or by calling functions such as `importlib.import_module` and built-in `__import__`. A module object has a namespace implemented by a `dictionary ` object (this is the dictionary referenced by the `~function.__globals__` attribute of functions defined in the module). Attribute references are translated to lookups in this dictionary, e.g., `m.x` is equivalent to `m.__dict__["x"]`. A module object does not contain the code object used to initialize the module (since it isn't needed once the initialization is done).
Attribute assignment updates the module's namespace dictionary, e.g., `m.x = 1` is equivalent to `m.__dict__["x"] = 1`.
#### Import-related attributes on module objects
Module objects have the following attributes that relate to the `import system `. When a module is created using the machinery associated with the import system, these attributes are filled in based on the module's `spec `, before the `loader` executes and loads the module.
To create a module dynamically rather than using the import system, it's recommended to use `importlib.util.module_from_spec`, which will set the various import-controlled attributes to appropriate values. It's also possible to use the `types.ModuleType` constructor to create modules directly, but this technique is more error-prone, as most attributes must be manually set on the module object after it has been created when using this approach.
> [!CAUTION]
> With the exception of `~module.__name__`, it is **strongly** recommended that you rely on `~module.__spec__` and its attributes instead of any of the other individual attributes listed in this subsection. Note that updating an attribute on `!__spec__` will not update the corresponding attribute on the module itself:
>
>
module.\_\_name\_\_
The name used to uniquely identify the module in the import system. For a directly executed module, this will be set to `"__main__"`.
This attribute must be set to the fully qualified name of the module. It is expected to match the value of `module.__spec__.name `.
module.\_\_spec\_\_
A record of the module's import-system-related state.
Set to the `module spec ` that was used when importing the module. See `module-specs` for more details.
3.4
module.\_\_package\_\_
The `package` a module belongs to.
If the module is top-level (that is, not a part of any specific package) then the attribute should be set to `''` (the empty string). Otherwise, it should be set to the name of the module's package (which can be equal to `module.__name__` if the module itself is a package). See `366` for further details.
This attribute is used instead of `~module.__name__` to calculate explicit relative imports for main modules. It defaults to `None` for modules created dynamically using the `types.ModuleType` constructor; use `importlib.util.module_from_spec` instead to ensure the attribute is set to a `str`.
It is **strongly** recommended that you use `module.__spec__.parent ` instead of `!module.__package__`. `__package__` is now only used as a fallback if `!__spec__.parent` is not set, and this fallback path is deprecated.
3.4 This attribute now defaults to `None` for modules created dynamically using the `types.ModuleType` constructor. Previously the attribute was optional.
3.6 The value of `!__package__` is expected to be the same as `__spec__.parent `. `__package__` is now only used as a fallback during import resolution if `!__spec__.parent` is not defined.
3.10 `ImportWarning` is raised if an import resolution falls back to `!__package__` instead of `__spec__.parent `.
3.12 Raise `DeprecationWarning` instead of `ImportWarning` when falling back to `!__package__` during import resolution.
3.13 3.16 `!__package__` will cease to be set or taken into consideration by the import system or standard library.
module.\_\_loader\_\_
The `loader` object that the import machinery used to load the module.
This attribute is mostly useful for introspection, but can be used for additional loader-specific functionality, for example getting data associated with a loader.
`!__loader__` defaults to `None` for modules created dynamically using the `types.ModuleType` constructor; use `importlib.util.module_from_spec` instead to ensure the attribute is set to a `loader` object.
It is **strongly** recommended that you use `module.__spec__.loader ` instead of `!module.__loader__`.
3.4 This attribute now defaults to `None` for modules created dynamically using the `types.ModuleType` constructor. Previously the attribute was optional.
3.12 3.16 Setting `!__loader__` on a module while failing to set `!__spec__.loader` is deprecated. In Python 3.16, `!__loader__` will cease to be set or taken into consideration by the import system or the standard library.
module.\_\_path\_\_
A (possibly empty) `sequence` of strings enumerating the locations where the package's submodules will be found. Non-package modules should not have a `!__path__` attribute. See `package-path-rules` for more details.
It is **strongly** recommended that you use `module.__spec__.submodule_search_locations ` instead of `!module.__path__`.
module.\_\_file\_\_
`!__file__` is an optional attribute that may or may not be set. Both attributes should be a `str` when they are available.
An optional attribute, `!__file__` indicates the pathname of the file from which the module was loaded (if loaded from a file), or the pathname of the shared library file for extension modules loaded dynamically from a shared library. It might be missing for certain types of modules, such as C modules that are statically linked into the interpreter, and the `import system ` may opt to leave it unset if it has no semantic meaning (for example, a module loaded from a database).
3.15 The `__cached__` attribute is no longer set on modules or taken into consideration by the import system or standard library.
#### Other writable attributes on module objects
As well as the import-related attributes listed above, module objects also have the following writable attributes:
module.\_\_doc\_\_
The module's documentation string, or `None` if unavailable. See also: `__doc__ attributes `.
module.\_\_annotations\_\_
A dictionary containing `variable annotations ` collected during module body execution. For best practices on working with `!__annotations__`, see `annotationlib`.
3.14 Annotations are now `lazily evaluated `. See `649`.
module.\_\_annotate\_\_
The `annotate function` for this module, or `None` if the module has no annotations. See also: `~object.__annotate__` attributes.
3.14
module.\_\_lazy_modules\_\_
A container (an object implementing `~object.__contains__`) of fully qualified module name strings. When defined at module scope, any regular `import` statement in that module whose target module name appears in this container is treated as a `lazy import `, as if the `lazy` keyword had been used. Imports inside functions, class bodies, or `try`/`except`/`finally` blocks are unaffected.
See `lazy-modules-compat` for details and examples.
3.15
#### Module dictionaries
Module objects also have the following special read-only attribute:
single: \_\_dict\_\_ (module attribute)
module.\_\_dict\_\_
The module's namespace as a dictionary object. Uniquely among the attributes listed here, `!__dict__` cannot be accessed as a global variable from within a module; it can only be accessed as an attribute on module objects.
Because of the way CPython clears module dictionaries, the module dictionary will be cleared when the module falls out of scope even if the dictionary still has live references. To avoid this, copy the dictionary or keep the module around while using its dictionary directly.
### Custom classes
Custom class types are typically created by class definitions (see section `class`). A class has a namespace implemented by a dictionary object. Class attribute references are translated to lookups in this dictionary, e.g., `C.x` is translated to `C.__dict__["x"]` (although there are a number of hooks which allow for other means of locating attributes). When the attribute name is not found there, the attribute search continues in the base classes. This search of the base classes uses the C3 method resolution order which behaves correctly even in the presence of 'diamond' inheritance structures where there are multiple inheritance paths leading back to a common ancestor. Additional details on the C3 MRO used by Python can be found at `python_2.3_mro`.
pair: object; class pair: object; class instance pair: object; instance pair: class object; call single: container pair: object; dictionary pair: class; attribute
When a class attribute reference (for class `!C`, say) would yield a class method object, it is transformed into an instance method object whose `~method.__self__` attribute is `!C`. When it would yield a `staticmethod` object, it is transformed into the object wrapped by the static method object. See section `descriptors` for another way in which attributes retrieved from a class may differ from those actually contained in its `~object.__dict__`.
triple: class; attribute; assignment
Class attribute assignments update the class's dictionary, never the dictionary of a base class.
pair: class object; call
A class object can be called (see above) to yield a class instance (see below).
#### Special attributes
The class's name. See also: __name__ attributes <definition.__name__>.
type.__qualname__
The class's qualified name. See also: __qualname__ attributes <definition.__qualname__>.
type.__module__
The name of the module in which the class was defined.
type.__dict__
A mapping proxy <types.MappingProxyType> providing a read-only view of the class's namespace. See also: __dict__ attributes <object.__dict__>.
type.__bases__
A tuple containing the class's bases. In most cases, for a class defined as class X(A, B, C), X.__bases__ will be exactly equal to (A, B, C).
type.__base__
The single base class in the inheritance chain that is responsible for the memory layout of instances. This attribute corresponds to ~PyTypeObject.tp_base at the C level.
type.__doc__
The class's documentation string, or None if undefined. Not inherited by subclasses.
type.__annotations__
A dictionary containing variable annotations <variable annotation> collected during class body execution. See also: __annotations__ attributes <object.__annotations__>.
For best practices on working with ~object.__annotations__, please see annotationlib. Use annotationlib.get_annotations instead of accessing this attribute directly.
Warning
Accessing the !__annotations__ attribute directly on a class object may return annotations for the wrong class, specifically in certain cases where the class, its base class, or a metaclass is defined under from __future__ import annotations. See 749 <749#pep749-metaclasses> for details.
This attribute does not exist on certain builtin classes. On user-defined classes without __annotations__, it is an empty dictionary.
3.14 Annotations are now lazily evaluated <lazy-evaluation>. See 649.
type.__annotate__
The annotate function for this class, or None if the class has no annotations. See also: __annotate__ attributes <object.__annotate__>.
3.14
type.__type_params__
A tuple containing the type parameters <type-params> of a generic class <generic-classes>.
3.12
type.__static_attributes__
A tuple containing names of attributes of this class which are assigned through self.X from any function in its body.
3.13
type.__firstlineno__
The line number of the first line of the class definition, including decorators. Setting the ~type.__module__ attribute removes the !__firstlineno__ item from the type's dictionary.
3.13
type.__mro__
The tuple of classes that are considered when looking for base classes during method resolution.
#### Special methods
In addition to the special attributes described above, all Python classes also have the following two methods available:
type.mro
This method can be overridden by a metaclass to customize the method resolution order for its instances. It is called at class instantiation, and its result is stored in `~type.__mro__`.
type.\_\_subclasses\_\_
Each class keeps a list of weak references to its immediate subclasses. This method returns a list of all those references still alive. The list is in definition order. Example:
\>\>\> class A: pass \>\>\> class B(A): pass \>\>\> A.\_\_subclasses\_\_() \[\\]
### Class instances
pair: object; class instance pair: object; instance pair: class; instance pair: class instance; attribute
A class instance is created by calling a class object (see above). A class instance has a namespace implemented as a dictionary which is the first place in which attribute references are searched. When an attribute is not found there, and the instance's class has an attribute by that name, the search continues with the class attributes. If a class attribute is found that is a user-defined function object, it is transformed into an instance method object whose `~method.__self__` attribute is the instance. Static method and class method objects are also transformed; see above under "Classes". See section `descriptors` for another way in which attributes of a class retrieved via its instances may differ from the objects actually stored in the class's `~object.__dict__`. If no class attribute is found, and the object's class has a `~object.__getattr__` method, that is called to satisfy the lookup.
triple: class instance; attribute; assignment
Attribute assignments and deletions update the instance's dictionary, never a class's dictionary. If the class has a `~object.__setattr__` or `~object.__delattr__` method, this is called instead of updating the instance dictionary directly.
Class instances can pretend to be numbers, sequences, or mappings if they have methods with certain special names. See section `specialnames`.
#### Special attributes
object.\_\_class\_\_
The class to which a class instance belongs.
object.\_\_dict\_\_
A dictionary or other mapping object used to store an object's (writable) attributes. Not all instances have a `!__dict__` attribute; see the section on `slots` for more details.
### I/O objects (also known as file objects)
pair: built-in function; open pair: module; io single: popen() (in module os) single: makefile() (socket method) single: sys.stdin single: sys.stdout single: sys.stderr single: stdio single: stdin (in module sys) single: stdout (in module sys) single: stderr (in module sys)
A `file object` represents an open file. Various shortcuts are available to create file objects: the `open` built-in function, and also `os.popen`, `os.fdopen`, and the `~socket.socket.makefile` method of socket objects (and perhaps by other functions or methods provided by extension modules).
File objects implement common methods, listed below, to simplify usage in generic code. They are expected to be `context-managers`.
The objects `sys.stdin`, `sys.stdout` and `sys.stderr` are initialized to file objects corresponding to the interpreter's standard input, output and error streams; they are all open in text mode and therefore follow the interface defined by the `io.TextIOBase` abstract class.
file.read(size=-1, /)
Retrieve up to *size* data from the file. As a convenience if *size* is unspecified or -1 retrieve all data available.
file.write(data, /)
Store *data* to the file.
file.close()
Flush any buffers and close the underlying file.
### Internal types
single: internal type single: types, internal
A few types used internally by the interpreter are exposed to the user. Their definitions may change with future versions of the interpreter, but they are mentioned here for completeness.
#### Code objects
bytecode, object; code, code object
Code objects represent *byte-compiled* executable Python code, or `bytecode`. The difference between a code object and a function object is that the function object contains an explicit reference to the function's globals (the module in which it was defined), while a code object contains no context; also the default argument values are stored in the function object, not in the code object (because they represent values calculated at run-time). Unlike function objects, code objects are immutable and contain no references (directly or indirectly) to mutable objects.
The total number of positional parameters <parameter> (including positional-only parameters and parameters with default values) that the function has
codeobject.co_posonlyargcount
The number of positional-only parameters <parameter> (including arguments with default values) that the function has
codeobject.co_kwonlyargcount
The number of keyword-only parameters <parameter> (including arguments with default values) that the function has
codeobject.co_nlocals
The number of local variables <naming> used by the function (including parameters)
codeobject.co_varnames
A tuple containing the names of the local variables in the function (starting with the parameter names)
codeobject.co_cellvars
A tuple containing the names of local variables <naming> that are referenced from at least one nested scope inside the function
codeobject.co_freevars
A tuple containing the names of free (closure) variables <closure variable> that a nested scope references in an outer scope. See also function.__closure__.
Note: references to global and builtin names are not included.
codeobject.co_code
A string representing the sequence of bytecode instructions in the function
codeobject.co_consts
A tuple containing the literals used by the bytecode in the function
codeobject.co_names
A tuple containing the names used by the bytecode in the function
codeobject.co_filename
The name of the file from which the code was compiled
codeobject.co_firstlineno
The line number of the first line of the function
codeobject.co_linetable
A bytes object containing encoded source location information. The exact format is an implementation detail and may change between Python versions. Use ~codeobject.co_lines and ~codeobject.co_positions for supported access to line and position information. To create a modified copy of a code object, use ~codeobject.replace.
3.10
codeobject.co_stacksize
The required stack size of the code object
codeobject.co_flags
An integer <int> encoding a number of flags for the interpreter.
pair: object; generator
The following flag bits are defined for `~codeobject.co_flags`: bit `0x04` is set if the function uses the `*arguments` syntax to accept an arbitrary number of positional arguments; bit `0x08` is set if the function uses the `**keywords` syntax to accept arbitrary keyword arguments; bit `0x20` is set if the function is a generator. See `inspect-module-co-flags` for details on the semantics of each flags that might be present.
Future feature declarations (for example, `from __future__ import division`) also use bits in `~codeobject.co_flags` to indicate whether a code object was compiled with a particular feature enabled. See `~__future__._Feature.compiler_flag`.
Other bits in `~codeobject.co_flags` are reserved for internal use.
single: documentation string
If a code object represents a function and has a docstring, the `~inspect.CO_HAS_DOCSTRING` bit is set in `~codeobject.co_flags` and the first item in `~codeobject.co_consts` is the docstring of the function.
##### Methods on code objects
codeobject.co_positions()
Returns an iterable over the source code positions of each `bytecode` instruction in the code object.
The iterator returns `tuple`s containing the `(start_line, end_line, start_column, end_column)`. The *i-th* tuple corresponds to the position of the source code that compiled to the *i-th* code unit. Column information is 0-indexed utf-8 byte offsets on the given source line.
This positional information can be missing. A non-exhaustive lists of cases where this may happen:
- Running the interpreter with `-X` `no_debug_ranges`.
- Loading a pyc file compiled while using `-X` `no_debug_ranges`.
- Position tuples corresponding to artificial instructions.
- Line and column numbers that can't be represented due to implementation specific limitations.
When this occurs, some or all of the tuple elements can be `None`.
3.11
> [!NOTE]
> This feature requires storing column positions in code objects which may result in a small increase of disk usage of compiled Python files or interpreter memory usage. To avoid storing the extra information and/or deactivate printing the extra traceback information, the `-X` `no_debug_ranges` command line flag or the `PYTHONNODEBUGRANGES` environment variable can be used.
codeobject.co_lines()
Returns an iterator that yields information about successive ranges of `bytecode`s. Each item yielded is a `(start, end, lineno)` `tuple`:
- `start` (an `int`) represents the offset (inclusive) of the start of the `bytecode` range
- `end` (an `int`) represents the offset (exclusive) of the end of the `bytecode` range
- `lineno` is an `int` representing the line number of the `bytecode` range, or `None` if the bytecodes in the given range have no line number
The items yielded will have the following properties:
- The first range yielded will have a `start` of 0.
- The `(start, end)` ranges will be non-decreasing and consecutive. That is, for any pair of `tuple`s, the `start` of the second will be equal to the `end` of the first.
- No range will be backwards: `end >= start` for all triples.
- The last `tuple` yielded will have `end` equal to the size of the `bytecode`.
Zero-width ranges, where `start == end`, are allowed. Zero-width ranges are used for lines that are present in the source code, but have been eliminated by the `bytecode` compiler.
3.10
`626` - Precise line numbers for debugging and other tools.
The PEP that introduced the `!co_lines` method.
codeobject.replace(\*\*kwargs)
Return a copy of the code object with new values for the specified fields.
Code objects are also supported by the generic function `copy.replace`.
3.8
#### Frame objects
pair: object; frame
Frame objects represent execution frames. They may occur in `traceback objects `, and are also passed to registered trace functions.
Points to the previous stack frame (towards the caller), or None if this is the bottom stack frame
frame.f_code
The code object <code-objects> being executed in this frame. Accessing this attribute raises an auditing event <auditing>object.__getattr__ with arguments obj and "f_code".
frame.f_locals
The mapping used by the frame to look up local variables <naming>. If the frame refers to an optimized scope, this may return a write-through proxy object.
3.13 Return a proxy for optimized scopes.
frame.f_globals
The dictionary used by the frame to look up global variables <naming>
frame.f_builtins
The dictionary used by the frame to look up built-in (intrinsic) names <naming>
frame.f_lasti
The "precise instruction" of the frame object (this is an index into the bytecode string of the code object <code-objects>)
frame.f_generator
The generator or coroutine object that owns this frame, or None if the frame is a normal function.
If not None, this is a function called for various events during code execution (this is used by debuggers). Normally an event is triggered for each new source line (see ~frame.f_trace_lines).
frame.f_trace_lines
Set this attribute to False to disable triggering a tracing event for each source line.
frame.f_trace_opcodes
Set this attribute to True to allow per-opcode events to be requested. Note that this may lead to undefined interpreter behaviour if exceptions raised by the trace function escape to the function being traced.
frame.f_lineno
The current line number of the frame -- writing to this from within a trace function jumps to the given line (only for the bottom-most frame). A debugger can implement a Jump command (aka Set Next Statement) by writing to this attribute.
##### Frame object methods
Frame objects support one method:
frame.clear()
This method clears all references to `local variables ` held by the frame. Also, if the frame belonged to a `generator`, the generator is finalized. This helps break reference cycles involving frame objects (for example when catching an `exception ` and storing its `traceback ` for later use).
`RuntimeError` is raised if the frame is currently executing or suspended.
3.4
3.13 Attempting to clear a suspended frame raises `RuntimeError` (as has always been the case for executing frames).
Traceback objects represent the stack trace of an `exception `. A traceback object is implicitly created when an exception occurs, and may also be explicitly created by calling `types.TracebackType`.
3.7 Traceback objects can now be explicitly instantiated from Python code.
For implicitly created tracebacks, when the search for an exception handler unwinds the execution stack, at each unwound level a traceback object is inserted in front of the current traceback. When an exception handler is entered, the stack trace is made available to the program. (See section `try`.) It is accessible as the third item of the tuple returned by `sys.exc_info`, and as the `~BaseException.__traceback__` attribute of the caught exception.
When the program contains no suitable handler, the stack trace is written (nicely formatted) to the standard error stream; if the interpreter is interactive, it is also made available to the user as `sys.last_traceback`.
For explicitly created tracebacks, it is up to the creator of the traceback to determine how the `~traceback.tb_next` attributes should be linked to form a full stack trace.
Points to the execution frame <frame-objects> of the current level.
Accessing this attribute raises an auditing event <auditing>object.__getattr__ with arguments obj and "tb_frame".
traceback.tb_lineno
Gives the line number where the exception occurred
traceback.tb_lasti
Indicates the "precise instruction".
The line number and last instruction in the traceback may differ from the line number of its `frame object ` if the exception occurred in a `try` statement with no matching except clause or with a `finally` clause.
single: tb_next (traceback attribute)
traceback.tb_next
The special writable attribute `!tb_next` is the next level in the stack trace (towards the frame where the exception occurred), or `None` if there is no next level.
3.7 This attribute is now writable
#### Slice objects
pair: built-in function; slice
Slice objects are used to represent slices for `~object.__getitem__` methods. They are also created by the built-in `slice` function.
3.15
The `slice` type now supports `subscription `. For example, `slice[float]` may be used in type annotations to indicate a slice containing `float` objects.
Special read-only attributes: `~slice.start` is the lower bound; `~slice.stop` is the upper bound; `~slice.step` is the step value; each is `None` if omitted. These attributes can have any type.
Slice objects support one method:
slice.indices(self, length)
This method takes a single integer argument *length* and computes information about the slice that the slice object would describe if applied to a sequence of *length* items. It returns a tuple of three integers; respectively these are the *start* and *stop* indices and the *step* or stride length of the slice. Missing or out-of-bounds indices are handled in a manner consistent with regular slices.
#### Static method objects
Static method objects provide a way of defeating the transformation of function objects to method objects described above. A static method object is a wrapper around any other object, usually a user-defined method object. When a static method object is retrieved from a class or a class instance, the object actually returned is the wrapped object, which is not subject to any further transformation. Static method objects are also callable. Static method objects are created by the built-in `staticmethod` constructor.
#### Class method objects
A class method object, like a static method object, is a wrapper around another object that alters the way in which that object is retrieved from classes and class instances. The behaviour of class method objects upon such retrieval is described above, under `"instance methods" `. Class method objects are created by the built-in `classmethod` constructor.
## Special method names
A class can implement certain operations that are invoked by special syntax (such as arithmetic operations or subscripting and slicing) by defining methods with special names. This is Python's approach to `operator overloading`, allowing classes to define their own behavior with respect to language operators. For instance, if a class defines a method named `~object.__getitem__`, and `x` is an instance of this class, then `x[i]` is roughly equivalent to `type(x).__getitem__(x, i)`. Except where mentioned, attempts to execute an operation raise an exception when no appropriate method is defined (typically `AttributeError` or `TypeError`).
Setting a special method to `None` indicates that the corresponding operation is not available. For example, if a class sets `~object.__iter__` to `None`, the class is not iterable, so calling `iter` on its instances will raise a `TypeError` (without falling back to `~object.__getitem__`).[^2]
When implementing a class that emulates any built-in type, it is important that the emulation only be implemented to the degree that it makes sense for the object being modelled. For example, some sequences may work well with retrieval of individual elements, but extracting a slice may not make sense. (One example of this is the `NodeList ` interface in the W3C's Document Object Model.)
### Basic customization
object.\_\_new\_\_(cls\[, ...\])
pair: subclassing; immutable types
Called to create a new instance of class *cls*. `__new__` is a static method (special-cased so you need not declare it as such) that takes the class of which an instance was requested as its first argument. The remaining arguments are those passed to the object constructor expression (the call to the class). The return value of `__new__` should be the new object instance (usually an instance of *cls*).
Typical implementations create a new instance of the class by invoking the superclass's `__new__` method using `super().__new__(cls[, ...])` with appropriate arguments and then modifying the newly created instance as necessary before returning it.
If `__new__` is invoked during object construction and it returns an instance of *cls*, then the new instance’s `__init__` method will be invoked like `__init__(self[, ...])`, where *self* is the new instance and the remaining arguments are the same as were passed to the object constructor.
If `__new__` does not return an instance of *cls*, then the new instance's `__init__` method will not be invoked.
`__new__` is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation. It is also commonly overridden in custom metaclasses in order to customize class creation.
object.\_\_init\_\_(self\[, ...\])
pair: class; constructor
Called after the instance has been created (by `__new__`), but before it is returned to the caller. The arguments are those passed to the class constructor expression. If a base class has an `__init__` method, the derived class's `__init__` method, if any, must explicitly call it to ensure proper initialization of the base class part of the instance; for example: `super().__init__([args...])`.
Because `__new__` and `__init__` work together in constructing objects (`__new__` to create it, and `__init__` to customize it), no non-`None` value may be returned by `__init__`; doing so will cause a `TypeError` to be raised at runtime.
object.\_\_del\_\_(self)
single: destructor single: finalizer pair: statement; del
Called when the instance is about to be destroyed. This is also called a finalizer or (improperly) a destructor. If a base class has a `__del__` method, the derived class's `__del__` method, if any, must explicitly call it to ensure proper deletion of the base class part of the instance.
It is possible (though not recommended!) for the `__del__` method to postpone destruction of the instance by creating a new reference to it. This is called object *resurrection*. It is implementation-dependent whether `__del__` is called a second time when a resurrected object is about to be destroyed; the current `CPython` implementation only calls it once.
It is not guaranteed that `__del__` methods are called for objects that still exist when the interpreter exits. `weakref.finalize` provides a straightforward way to register a cleanup function to be called when an object is garbage collected.
> [!NOTE]
> `del x` doesn't directly call `x.__del__()` --- the former decrements the reference count for `x` by one, and the latter is only called when `x`'s reference count reaches zero.
It is possible for a reference cycle to prevent the reference count of an object from going to zero. In this case, the cycle will be later detected and deleted by the `cyclic garbage collector
`. A common cause of reference cycles is when an exception has been caught in a local variable. The frame's locals then reference the exception, which references its own traceback, which references the locals of all frames caught in the traceback.
Documentation for the `gc` module.
> [!WARNING]
> Due to the precarious circumstances under which `__del__` methods are invoked, exceptions that occur during their execution are ignored, and a warning is printed to `sys.stderr` instead. In particular:
>
> - `__del__` can be invoked when arbitrary code is being executed, including from any arbitrary thread. If `__del__` needs to take a lock or invoke any other blocking resource, it may deadlock as the resource may already be taken by the code that gets interrupted to execute `__del__`.
> - `__del__` can be executed during interpreter shutdown. As a consequence, the global variables it needs to access (including other modules) may already have been deleted or set to `None`. Python guarantees that globals whose name begins with a single underscore are deleted from their module before other globals are deleted; if no other references to such globals exist, this may help in assuring that imported modules are still available at the time when the `__del__` method is called.
object.\_\_repr\_\_(self)
Called by the `repr` built-in function to compute the "official" string representation of an object. If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment). If this is not possible, a string of the form `<...some useful description...>` should be returned. The return value must be a string object. If a class defines `__repr__` but not `__str__`, then `__repr__` is also used when an "informal" string representation of instances of that class is required.
This is typically used for debugging, so it is important that the representation is information-rich and unambiguous. A default implementation is provided by the `object` class itself.
object.\_\_str\_\_(self)
Called by `str(object) `, the default `__format__` implementation, and the built-in function `print`, to compute the "informal" or nicely printable string representation of an object. The return value must be a `str ` object.
This method differs from `object.__repr__` in that there is no expectation that `__str__` return a valid Python expression: a more convenient or concise representation can be used.
The default implementation defined by the built-in type `object` calls `object.__repr__`.
object.\_\_bytes\_\_(self)
pair: built-in function; bytes
Called by `bytes ` to compute a byte-string representation of an object. This should return a `bytes` object. The `object` class itself does not provide this method.
object.\_\_format\_\_(self, format_spec)
Called by the `format` built-in function, and by extension, evaluation of `formatted string literals
` and the `str.format` method, to produce a "formatted" string representation of an object. The *format_spec* argument is a string that contains a description of the formatting options desired. The interpretation of the *format_spec* argument is up to the type implementing `__format__`, however most classes will either delegate formatting to one of the built-in types, or use a similar formatting option syntax.
See `formatspec` for a description of the standard formatting syntax.
The return value must be a string object.
The default implementation by the `object` class should be given an empty *format_spec* string. It delegates to `__str__`.
3.4 The \_\_format\_\_ method of `object` itself raises a `TypeError` if passed any non-empty string.
3.7 `object.__format__(x, '')` is now equivalent to `str(x)` rather than `format(str(x), '')`.
>
>
> single: comparisons
>
>
>
> These are the so-called "rich comparison" methods. The correspondence between operator symbols and method names is as follows: `xy` calls `x.__gt__(y)`, and `x>=y` calls `x.__ge__(y)`.
>
> A rich comparison method may return the singleton `NotImplemented` if it does not implement the operation for a given pair of arguments. By convention, `False` and `True` are returned for a successful comparison. However, these methods can return any value, so if the comparison operator is used in a Boolean context (e.g., in the condition of an `if` statement), Python will call `bool` on the value to determine if the result is true or false.
>
> By default, `object` implements `__eq__` by using `is`, returning `NotImplemented` in the case of a false comparison: `True if x is y else NotImplemented`. For `__ne__`, by default it delegates to `__eq__` and inverts the result unless it is `!NotImplemented`. There are no other implied relationships among the comparison operators or default implementations; for example, the truth of `(x
> By default, the `object` class provides implementations consistent with `expressions-value-comparisons`: equality compares according to object identity, and order comparisons raise `TypeError`. Each default method may generate these results directly, but may also return `NotImplemented`.
>
> See the paragraph on `__hash__` for some important notes on creating `hashable` objects which support custom comparison operations and are usable as dictionary keys.
>
> There are no swapped-argument versions of these methods (to be used when the left argument does not support the operation but the right argument does); rather, `__lt__` and `__gt__` are each other's reflection, `__le__` and `__ge__` are each other's reflection, and `__eq__` and `__ne__` are their own reflection. If the operands are of different types, and the right operand's type is a direct or indirect subclass of the left operand's type, the reflected method of the right operand has priority, otherwise the left operand's method has priority. Virtual subclassing is not considered.
>
> When no appropriate method returns any value other than `NotImplemented`, the `==` and `!=` operators will fall back to `is` and `is not`, respectively.
Called by built-in function `hash` and for operations on members of hashed collections including `set`, `frozenset`, `dict`, and `frozendict`. The `__hash__()` method should return an integer. The only required property is that objects which compare equal have the same hash value; it is advised to mix together the hash values of the components of the object that also play a part in comparison of objects by packing them into a tuple and hashing the tuple. Example:
def __hash__(self):
return hash((self.name, self.nick, self.color))
> [!NOTE]
> `hash` truncates the value returned from an object's custom `__hash__` method to the size of a `Py_ssize_t`. This is typically 8 bytes on 64-bit builds and 4 bytes on 32-bit builds. If an object's `__hash__` must interoperate on builds of different bit sizes, be sure to check the width on all supported builds. An easy way to do this is with `python -c "import sys; print(sys.hash_info.width)"`.
If a class does not define an `__eq__` method it should not define a `__hash__` operation either; if it defines `__eq__` but not `__hash__`, its instances will not be usable as items in hashable collections. If a class defines mutable objects and implements an `__eq__` method, it should not implement `__hash__`, since the implementation of `hashable` collections requires that a key's hash value is immutable (if the object's hash value changes, it will be in the wrong hash bucket).
User-defined classes have `__eq__` and `__hash__` methods by default (inherited from the `object` class); with them, all objects compare unequal (except with themselves) and `x.__hash__()` returns an appropriate value such that `x == y` implies both that `x is y` and `hash(x) == hash(y)`.
A class that overrides `__eq__` and does not define `__hash__` will have its `__hash__` implicitly set to `None`. When the `__hash__` method of a class is `None`, instances of the class will raise an appropriate `TypeError` when a program attempts to retrieve their hash value, and will also be correctly identified as unhashable when checking `isinstance(obj, collections.abc.Hashable)`.
If a class that overrides `__eq__` needs to retain the implementation of `__hash__` from a parent class, the interpreter must be told this explicitly by setting `__hash__ = .__hash__`.
If a class that does not override `__eq__` wishes to suppress hash support, it should include `__hash__ = None` in the class definition. A class which defines its own `__hash__` that explicitly raises a `TypeError` would be incorrectly identified as hashable by an `isinstance(obj, collections.abc.Hashable)` call.
> [!NOTE]
> By default, the `__hash__` values of str and bytes objects are "salted" with an unpredictable random value. Although they remain constant within an individual Python process, they are not predictable between repeated invocations of Python.
>
> This is intended to provide protection against a denial-of-service caused by carefully chosen inputs that exploit the worst case performance of a dict insertion, *O*(*n*2) complexity. See for details.
>
> Changing hash values affects the iteration order of sets. Python has never made guarantees about this ordering (and it typically varies between 32-bit and 64-bit builds).
>
> See also `PYTHONHASHSEED`.
3.3 Hash randomization is enabled by default.
object.\_\_bool\_\_(self)
single: \_\_len\_\_() (mapping object method)
Called to implement truth value testing and the built-in operation `bool()`; should return `False` or `True`. When this method is not defined, `~object.__len__` is called, if it is defined, and the object is considered true if its result is nonzero. If a class defines neither `!__len__` nor `!__bool__` (which is true of the `object` class itself), all its instances are considered true.
### Customizing attribute access
The following methods can be defined to customize the meaning of attribute access (use of, assignment to, or deletion of `x.name`) for class instances.
object.\_\_getattr\_\_(self, name)
Called when the default attribute access fails with an `AttributeError` (either `__getattribute__` raises an `AttributeError` because *name* is not an instance attribute or an attribute in the class tree for `self`; or `__get__` of a *name* property raises `AttributeError`). This method should either return the (computed) attribute value or raise an `AttributeError` exception. The `object` class itself does not provide this method.
Note that if the attribute is found through the normal mechanism, `__getattr__` is not called. (This is an intentional asymmetry between `__getattr__` and `__setattr__`.) This is done both for efficiency reasons and because otherwise `__getattr__` would have no way to access other attributes of the instance. Note that at least for instance variables, you can take total control by not inserting any values in the instance attribute dictionary (but instead inserting them in another object). See the `__getattribute__` method below for a way to actually get total control over attribute access.
object.\_\_getattribute\_\_(self, name)
Called unconditionally to implement attribute accesses for instances of the class. If the class also defines `__getattr__`, the latter will not be called unless `__getattribute__` either calls it explicitly or raises an `AttributeError`. This method should return the (computed) attribute value or raise an `AttributeError` exception. In order to avoid infinite recursion in this method, its implementation should always call the base class method with the same name to access any attributes it needs, for example, `object.__getattribute__(self, name)`.
> [!NOTE]
> This method may still be bypassed when looking up special methods as the result of implicit invocation via language syntax or `built-in functions `. See `special-lookup`.
object.\_\_getattr\_\_ obj,name object.\_\_getattribute\_\_
For certain sensitive attribute accesses, raises an `auditing event ` `object.__getattr__` with arguments `obj` and `name`.
object.\_\_setattr\_\_(self, name, value)
Called when an attribute assignment is attempted. This is called instead of the normal mechanism (i.e. store the value in the instance dictionary). *name* is the attribute name, *value* is the value to be assigned to it.
If `__setattr__` wants to assign to an instance attribute, it should call the base class method with the same name, for example, `object.__setattr__(self, name, value)`.
object.\_\_setattr\_\_ obj,name,value object.\_\_setattr\_\_
For certain sensitive attribute assignments, raises an `auditing event ` `object.__setattr__` with arguments `obj`, `name`, `value`.
object.\_\_delattr\_\_(self, name)
Like `__setattr__` but for attribute deletion instead of assignment. This should only be implemented if `del obj.name` is meaningful for the object.
object.\_\_delattr\_\_ obj,name object.\_\_delattr\_\_
For certain sensitive attribute deletions, raises an `auditing event ` `object.__delattr__` with arguments `obj` and `name`.
object.\_\_dir\_\_(self)
Called when `dir` is called on the object. An iterable must be returned. `dir` converts the returned iterable to a list and sorts it.
Special names `__getattr__` and `__dir__` can be also used to customize access to module attributes. The `__getattr__` function at the module level should accept one argument which is the name of an attribute and return the computed value or raise an `AttributeError`. If an attribute is not found on a module object through the normal lookup, i.e. `object.__getattribute__`, then `__getattr__` is searched in the module `__dict__` before raising an `AttributeError`. If found, it is called with the attribute name and the result is returned.
The `__dir__` function should accept no arguments, and return an iterable of strings that represents the names accessible on module. If present, this function overrides the standard `dir` search on a module.
module.\_\_class\_\_
For a more fine grained customization of the module behavior (setting attributes, properties, etc.), one can set the `__class__` attribute of a module object to a subclass of `types.ModuleType`. For example:
import sys
from types import ModuleType
class VerboseModule(ModuleType):
def __repr__(self):
return f'Verbose {self.__name__}'
def __setattr__(self, attr, value):
print(f'Setting {attr}...')
super().__setattr__(attr, value)
sys.modules[__name__].__class__ = VerboseModule
> [!NOTE]
> Defining module `__getattr__` and setting module `__class__` only affect lookups made using the attribute access syntax -- directly accessing the module globals (whether by code within the module, or via a reference to the module's globals dictionary) is unaffected.
3.5 `__class__` module attribute is now writable.
3.7 `__getattr__` and `__dir__` module attributes.
`562` - Module \_\_getattr\_\_ and \_\_dir\_\_
Describes the `__getattr__` and `__dir__` functions on modules.
#### Implementing Descriptors
The following methods only apply when an instance of the class containing the method (a so-called *descriptor* class) appears in an *owner* class (the descriptor must be in either the owner's class dictionary or in the class dictionary for one of its parents). In the examples below, "the attribute" refers to the attribute whose name is the key of the property in the owner class' `~object.__dict__`. The `object` class itself does not implement any of these protocols.
object.\_\_get\_\_(self, instance, owner=None)
Called to get the attribute of the owner class (class attribute access) or of an instance of that class (instance attribute access). The optional *owner* argument is the owner class, while *instance* is the instance that the attribute was accessed through, or `None` when the attribute is accessed through the *owner*.
This method should return the computed attribute value or raise an `AttributeError` exception.
[PEP 252](http://www.python.org/dev/peps/pep-0252/ "PEP 252") specifies that `__get__` is callable with one or two arguments. Python's own built-in descriptors support this specification; however, it is likely that some third-party tools have descriptors that require both arguments. Python's own `__getattribute__` implementation always passes in both arguments whether they are required or not.
object.\_\_set\_\_(self, instance, value)
Called to set the attribute on an instance *instance* of the owner class to a new value, *value*.
Note, adding `__set__` or `__delete__` changes the kind of descriptor to a "data descriptor". See `descriptor-invocation` for more details.
object.\_\_delete\_\_(self, instance)
Called to delete the attribute on an instance *instance* of the owner class.
Instances of descriptors may also have the `!__objclass__` attribute present:
object.\_\_objclass\_\_
The attribute `!__objclass__` is interpreted by the `inspect` module as specifying the class where this object was defined (setting this appropriately can assist in runtime introspection of dynamic class attributes). For callables, it may indicate that an instance of the given type (or a subclass) is expected or required as the first positional argument (for example, CPython sets this attribute for unbound methods that are implemented in C).
#### Invoking Descriptors
In general, a descriptor is an object attribute with "binding behavior", one whose attribute access has been overridden by methods in the descriptor protocol: `~object.__get__`, `~object.__set__`, and `~object.__delete__`. If any of those methods are defined for an object, it is said to be a descriptor.
The default behavior for attribute access is to get, set, or delete the attribute from an object's dictionary. For instance, `a.x` has a lookup chain starting with `a.__dict__['x']`, then `type(a).__dict__['x']`, and continuing through the base classes of `type(a)` excluding metaclasses.
However, if the looked-up value is an object defining one of the descriptor methods, then Python may override the default behavior and invoke the descriptor method instead. Where this occurs in the precedence chain depends on which descriptor methods were defined and how they were called.
The starting point for descriptor invocation is a binding, `a.x`. How the arguments are assembled depends on `a`:
Direct Call
The simplest and least common call is when user code directly invokes a descriptor method: `x.__get__(a)`.
Instance Binding
If binding to an object instance, `a.x` is transformed into the call: `type(a).__dict__['x'].__get__(a, type(a))`.
Class Binding
If binding to a class, `A.x` is transformed into the call: `A.__dict__['x'].__get__(None, A)`.
Super Binding
A dotted lookup such as `super(A, a).x` searches `a.__class__.__mro__` for a base class `B` following `A` and then returns `B.__dict__['x'].__get__(a, A)`. If not a descriptor, `x` is returned unchanged.
class Desc:
def \_\_get\_\_(\*args):
return args
class B:
> x = Desc()
class A(B):
> x = 999
>
> def m(self):
> 'Demonstrate these two descriptor invocations are equivalent' result1 = super(A, self).x result2 = B.\_\_dict\_\_\['x'\].\_\_get\_\_(self, A) return result1 == result2
\>\>\> a = A() \>\>\> a.\_\_class\_\_.\_\_mro\_\_.index(B) \> a.\_\_class\_\_.\_\_mro\_\_.index(A) True \>\>\> super(A, a).x == B.\_\_dict\_\_\['x'\].\_\_get\_\_(a, A) True \>\>\> a.m() True
For instance bindings, the precedence of descriptor invocation depends on which descriptor methods are defined. A descriptor can define any combination of `~object.__get__`, `~object.__set__` and `~object.__delete__`. If it does not define `!__get__`, then accessing the attribute will return the descriptor object itself unless there is a value in the object's instance dictionary. If the descriptor defines `!__set__` and/or `!__delete__`, it is a data descriptor; if it defines neither, it is a non-data descriptor. Normally, data descriptors define both `!__get__` and `!__set__`, while non-data descriptors have just the `!__get__` method. Data descriptors with `!__get__` and `!__set__` (and/or `!__delete__`) defined always override a redefinition in an instance dictionary. In contrast, non-data descriptors can be overridden by instances.
Python methods (including those decorated with `staticmethod` and `classmethod`) are implemented as non-data descriptors. Accordingly, instances can redefine and override methods. This allows individual instances to acquire behaviors that differ from other instances of the same class.
The `property` decorator is implemented as a data descriptor. Accordingly, instances cannot override the behavior of a property.
#### \_\_slots\_\_
*\_\_slots\_\_* allow us to explicitly declare data members (like properties) and deny the creation of `~object.__dict__` and *\_\_weakref\_\_* (unless explicitly declared in *\_\_slots\_\_* or available in a parent.)
The space saved over using `~object.__dict__` can be significant. Attribute lookup speed can be significantly improved as well.
object.\_\_slots\_\_
This class variable can be assigned a string, iterable, or sequence of strings with variable names used by instances. *\_\_slots\_\_* reserves space for the declared variables and prevents the automatic creation of `~object.__dict__` and *\_\_weakref\_\_* for each instance.
Notes on using *\_\_slots\_\_*:
- When inheriting from a class without *\_\_slots\_\_*, the `~object.__dict__` and *\_\_weakref\_\_* attribute of the instances will always be accessible.
- Without a `~object.__dict__` variable, instances cannot be assigned new variables not listed in the *\_\_slots\_\_* definition. Attempts to assign to an unlisted variable name raises `AttributeError`. If dynamic assignment of new variables is desired, then add `'__dict__'` to the sequence of strings in the *\_\_slots\_\_* declaration.
- Without a *\_\_weakref\_\_* variable for each instance, classes defining *\_\_slots\_\_* do not support `weak references ` to its instances. If weak reference support is needed, then add `'__weakref__'` to the sequence of strings in the *\_\_slots\_\_* declaration.
- *\_\_slots\_\_* are implemented at the class level by creating `descriptors ` for each variable name. As a result, class attributes cannot be used to set default values for instance variables defined by *\_\_slots\_\_*; otherwise, the class attribute would overwrite the descriptor assignment.
- The action of a *\_\_slots\_\_* declaration is not limited to the class where it is defined. *\_\_slots\_\_* declared in parents are available in child classes. However, instances of a child subclass will get a `~object.__dict__` and *\_\_weakref\_\_* unless the subclass also defines *\_\_slots\_\_* (which should only contain names of any *additional* slots).
- If a class defines a slot also defined in a base class, the instance variable defined by the base class slot is inaccessible (except by retrieving its descriptor directly from the base class). This renders the meaning of the program undefined. In the future, a check may be added to prevent this.
- `TypeError` will be raised if *\_\_slots\_\_* other than *\_\_dict\_\_* and *\_\_weakref\_\_* are defined for a class derived from a `"variable-length" built-in type ` such as `int`, `bytes`, and `type`, except `tuple`.
- Any non-string `iterable` may be assigned to *\_\_slots\_\_*.
- If a `dictionary ` is used to assign *\_\_slots\_\_*, the dictionary keys will be used as the slot names. The values of the dictionary can be used to provide per-attribute docstrings that will be recognised by `inspect.getdoc` and displayed in the output of `help`.
- `~object.__class__` assignment works only if both classes have the same *\_\_slots\_\_*.
- `Multiple inheritance ` with multiple slotted parent classes can be used, but only one parent is allowed to have attributes created by slots (the other bases must have empty slot layouts) - violations raise `TypeError`.
- If an `iterator` is used for *\_\_slots\_\_* then a `descriptor` is created for each of the iterator's values. However, the *\_\_slots\_\_* attribute will be an empty iterator.
3.15 Allowed defining the *\_\_dict\_\_* and *\_\_weakref\_\_* *\_\_slots\_\_* for any class. Allowed defining any *\_\_slots\_\_* for a class derived from `tuple`.
### Customizing class creation
Whenever a class inherits from another class, `~object.__init_subclass__` is called on the parent class. This way, it is possible to write classes which change the behavior of subclasses. This is closely related to class decorators, but where class decorators only affect the specific class they're applied to, `__init_subclass__` solely applies to future subclasses of the class defining the method.
object.\_\_init_subclass\_\_(cls)
This method is called whenever the containing class is subclassed. *cls* is then the new subclass. If defined as a normal instance method, this method is implicitly converted to a class method.
Keyword arguments which are given to a new class are passed to the parent class's `__init_subclass__`. For compatibility with other classes using `__init_subclass__`, one should take out the needed keyword arguments and pass the others over to the base class, as in:
class Philosopher:
def __init_subclass__(cls, /, default_name, **kwargs):
super().__init_subclass__(**kwargs)
cls.default_name = default_name
class AustralianPhilosopher(Philosopher, default_name="Bruce"):
pass
The default implementation `object.__init_subclass__` does nothing, but raises an error if it is called with any arguments.
> [!NOTE]
> The metaclass hint `metaclass` is consumed by the rest of the type machinery, and is never passed to `__init_subclass__` implementations. The actual metaclass (rather than the explicit hint) can be accessed as `type(cls)`.
3.6
When a class is created, `!type.__new__` scans the class variables and makes callbacks to those with a `~object.__set_name__` hook.
object.\_\_set_name\_\_(self, owner, name)
Automatically called at the time the owning class *owner* is created. The object has been assigned to *name* in that class:
class A:
x = C() # Automatically calls: x.__set_name__(A, 'x')
If the class variable is assigned after the class is created, `__set_name__` will not be called automatically. If needed, `__set_name__` can be called directly:
class A:
pass
c = C()
A.x = c # The hook is not called
c.__set_name__(A, 'x') # Manually invoke the hook
See `class-object-creation` for more details.
3.6
#### Metaclasses
single: metaclass pair: built-in function; type single: = (equals); class definition
By default, classes are constructed using `type`. The class body is executed in a new namespace and the class name is bound locally to the result of `type(name, bases, namespace)`.
The class creation process can be customized by passing the `metaclass` keyword argument in the class definition line, or by inheriting from an existing class that included such an argument. In the following example, both `MyClass` and `MySubclass` are instances of `Meta`:
class Meta(type):
pass
class MyClass(metaclass=Meta):
pass
class MySubclass(MyClass):
pass
Any other keyword arguments that are specified in the class definition are passed through to all metaclass operations described below.
When a class definition is executed, the following steps occur:
- MRO entries are resolved;
- the appropriate metaclass is determined;
- the class namespace is prepared;
- the class body is executed;
- the class object is created.
#### Resolving MRO entries
object.\_\_mro_entries\_\_(self, bases)
If a base that appears in a class definition is not an instance of `type`, then an `!__mro_entries__` method is searched on the base. If an `!__mro_entries__` method is found, the base is substituted with the result of a call to `!__mro_entries__` when creating the class. The method is called with the original bases tuple passed to the *bases* parameter, and must return a tuple of classes that will be used instead of the base. The returned tuple may be empty: in these cases, the original base is ignored.
`types.resolve_bases`
Dynamically resolve bases that are not instances of `type`.
`types.get_original_bases`
Retrieve a class's "original bases" prior to modifications by `~object.__mro_entries__`.
`560`
Core support for typing module and generic types.
#### Determining the appropriate metaclass
single: metaclass hint
The appropriate metaclass for a class definition is determined as follows:
- if no bases and no explicit metaclass are given, then `type` is used;
- if an explicit metaclass is given and it is *not* an instance of `type`, then it is used directly as the metaclass;
- if an instance of `type` is given as the explicit metaclass, or bases are defined, then the most derived metaclass is used.
The most derived metaclass is selected from the explicitly specified metaclass (if any) and the metaclasses (i.e. `type(cls)`) of all specified base classes. The most derived metaclass is one which is a subtype of *all* of these candidate metaclasses. If none of the candidate metaclasses meets that criterion, then the class definition will fail with `TypeError`.
#### Preparing the class namespace
single: \_\_prepare\_\_ (metaclass method)
Once the appropriate metaclass has been identified, then the class namespace is prepared. If the metaclass has a `__prepare__` attribute, it is called as `namespace = metaclass.__prepare__(name, bases, **kwds)` (where the additional keyword arguments, if any, come from the class definition). The `__prepare__` method should be implemented as a `classmethod `. The namespace returned by `__prepare__` is passed in to `__new__`, but when the final class object is created the namespace is copied into a new `dict`.
If the metaclass has no `__prepare__` attribute, then the class namespace is initialised as an empty ordered mapping.
`3115` - Metaclasses in Python 3000
Introduced the `__prepare__` namespace hook
#### Executing the class body
single: class; body
The class body is executed (approximately) as `exec(body, globals(), namespace)`. The key difference from a normal call to `exec` is that lexical scoping allows the class body (including any methods) to reference names from the current and outer scopes when the class definition occurs inside a function.
However, even when the class definition occurs inside the function, methods defined inside the class still cannot see names defined at the class scope. Class variables must be accessed through the first parameter of instance or class methods, or through the implicit lexically scoped `__class__` reference described in the next section.
#### Creating the class object
Once the class namespace has been populated by executing the class body, the class object is created by calling `metaclass(name, bases, namespace, **kwds)` (the additional keywords passed here are the same as those passed to `__prepare__`).
This class object is the one that will be referenced by the zero-argument form of `super`. `__class__` is an implicit closure reference created by the compiler if any methods in a class body refer to either `__class__` or `super`. This allows the zero argument form of `super` to correctly identify the class being defined based on lexical scoping, while the class or instance that was used to make the current call is identified based on the first argument passed to the method.
In CPython 3.6 and later, the `__class__` cell is passed to the metaclass as a `__classcell__` entry in the class namespace. If present, this must be propagated up to the `type.__new__` call in order for the class to be initialised correctly. Failing to do so will result in a `RuntimeError` in Python 3.8.
When using the default metaclass `type`, or any metaclass that ultimately calls `type.__new__`, the following additional customization steps are invoked after creating the class object:
1) The `type.__new__` method collects all of the attributes in the class namespace that define a `~object.__set_name__` method;
2) Those `__set_name__` methods are called with the class being defined and the assigned name of that particular attribute;
3) The `~object.__init_subclass__` hook is called on the immediate parent of the new class in its method resolution order.
After the class object is created, it is passed to the class decorators included in the class definition (if any) and the resulting object is bound in the local namespace as the defined class.
When a new class is created by `type.__new__`, the object provided as the namespace parameter is copied to a new ordered mapping and the original object is discarded. The new copy is wrapped in a read-only proxy, which becomes the `~type.__dict__` attribute of the class object.
`3135` - New super
Describes the implicit `__class__` closure reference
#### Uses for metaclasses
The potential uses for metaclasses are boundless. Some ideas that have been explored include enum, logging, interface checking, automatic delegation, automatic property creation, proxies, frameworks, and automatic resource locking/synchronization.
### Customizing instance and subclass checks
The following methods are used to override the default behavior of the `isinstance` and `issubclass` built-in functions.
In particular, the metaclass `abc.ABCMeta` implements these methods in order to allow the addition of Abstract Base Classes (ABCs) as "virtual base classes" to any class or type (including built-in types), including other ABCs.
type.\_\_instancecheck\_\_(self, instance)
Return true if *instance* should be considered a (direct or indirect) instance of *class*. If defined, called to implement `isinstance(instance, class)`.
type.\_\_subclasscheck\_\_(self, subclass)
Return true if *subclass* should be considered a (direct or indirect) subclass of *class*. If defined, called to implement `issubclass(subclass, class)`.
Note that these methods are looked up on the type (metaclass) of a class. They cannot be defined as class methods in the actual class. This is consistent with the lookup of special methods that are called on instances, only in this case the instance is itself a class.
`3119` - Introducing Abstract Base Classes
Includes the specification for customizing `isinstance` and `issubclass` behavior through `~type.__instancecheck__` and `~type.__subclasscheck__`, with motivation for this functionality in the context of adding Abstract Base Classes (see the `abc` module) to the language.
### Emulating generic types
When using `type annotations`, it is often useful to *parameterize* a `generic type` using Python's square-brackets notation. For example, the annotation `list[int]` might be used to signify a `list` in which all the elements are of type `int`.
`484` - Type Hints
Introducing Python's framework for type annotations
`Generic Alias Types`
Documentation for objects representing parameterized generic classes
`Generics`, `user-defined generics` and `typing.Generic`
Documentation on how to implement generic classes that can be parameterized at runtime and understood by static type-checkers.
A class can *generally* only be parameterized if it defines the special class method `__class_getitem__()`.
object.\_\_class_getitem\_\_(cls, key)
Return an object representing the specialization of a generic class by type arguments found in *key*.
When defined on a class, `__class_getitem__()` is automatically a class method. As such, there is no need for it to be decorated with `classmethod` when it is defined.
#### The purpose of *\_\_class_getitem\_\_*
The purpose of `~object.__class_getitem__` is to allow runtime parameterization of standard-library generic classes in order to more easily apply `type hints` to these classes.
To implement custom generic classes that can be parameterized at runtime and understood by static type-checkers, users should either inherit from a standard library class that already implements `~object.__class_getitem__`, or inherit from `typing.Generic`, which has its own implementation of `__class_getitem__()`.
Custom implementations of `~object.__class_getitem__` on classes defined outside of the standard library may not be understood by third-party type-checkers such as mypy. Using `__class_getitem__()` on any class for purposes other than type hinting is discouraged.
#### *\_\_class_getitem\_\_* versus *\_\_getitem\_\_*
Usually, the `subscription` of an object using square brackets will call the `~object.__getitem__` instance method defined on the object's class. However, if the object being subscribed is itself a class, the class method `~object.__class_getitem__` may be called instead. `__class_getitem__()` should return a `GenericAlias` object if it is properly defined.
Presented with the `expression` `obj[x]`, the Python interpreter follows something like the following process to decide whether `~object.__getitem__` or `~object.__class_getitem__` should be called:
from inspect import isclass
def subscribe(obj, x):
"""Return the result of the expression 'obj[x]'"""
class_of_obj = type(obj)
# If the class of obj defines __getitem__,
# call class_of_obj.__getitem__(obj, x)
if hasattr(class_of_obj, '__getitem__'):
return class_of_obj.__getitem__(obj, x)
# Else, if obj is a class and defines __class_getitem__,
# call obj.__class_getitem__(x)
elif isclass(obj) and hasattr(obj, '__class_getitem__'):
return obj.__class_getitem__(x)
# Else, raise an exception
else:
raise TypeError(
f"'{class_of_obj.__name__}' object is not subscriptable"
)
In Python, all classes are themselves instances of other classes. The class of a class is known as that class's `metaclass`, and most classes have the `type` class as their metaclass. `type` does not define `~object.__getitem__`, meaning that expressions such as `list[int]`, `dict[str, float]` and `tuple[str, bytes]` all result in `~object.__class_getitem__` being called:
>>> # list has class "type" as its metaclass, like most classes:
>>> type(list)
>>> type(dict) == type(list) == type(tuple) == type(str) == type(bytes)
True
>>> # "list[int]" calls "list.__class_getitem__(int)"
>>> list[int]
list[int]
>>> # list.__class_getitem__ returns a GenericAlias object:
>>> type(list[int])
However, if a class has a custom metaclass that defines `~object.__getitem__`, subscribing the class may result in different behaviour. An example of this can be found in the `enum` module:
>>> from enum import Enum
>>> class Menu(Enum):
... """A breakfast menu"""
... SPAM = 'spam'
... BACON = 'bacon'
...
>>> # Enum classes have a custom metaclass:
>>> type(Menu)
>>> # EnumMeta defines __getitem__,
>>> # so __class_getitem__ is not called,
>>> # and the result is not a GenericAlias object:
>>> Menu['SPAM']
>>> type(Menu['SPAM'])
`560` - Core Support for typing module and generic types Introducing `~object.__class_getitem__`, and outlining when a `subscription` results in `__class_getitem__()` being called instead of `~object.__getitem__`
### Emulating callable objects
object.\_\_call\_\_(self\[, args...\])
pair: call; instance
Called when the instance is "called" as a function; if this method is defined, `x(arg1, arg2, ...)` roughly translates to `type(x).__call__(x, arg1, ...)`. The `object` class itself does not provide this method.
### Emulating container types
The following methods can be defined to implement container objects. None of them are provided by the `object` class itself. Containers usually are `sequences ` (such as `lists ` or `tuples `) or `mappings ` (like `dictionaries `), but can represent other containers as well. The first set of methods is used either to emulate a sequence or to emulate a mapping; the difference is that for a sequence, the allowable keys should be the integers *k* for which `0 <= k < N` where *N* is the length of the sequence, or `slice` objects, which define a range of items. It is also recommended that mappings provide the methods `!keys`, `!values`, `!items`, `!get`, `!clear`, `!setdefault`, `!pop`, `!popitem`, `!copy`, and `!update` behaving similar to those for Python's standard `dictionary ` objects. The `collections.abc` module provides a `~collections.abc.MutableMapping` `abstract base class` to help create those methods from a base set of `~object.__getitem__`, `~object.__setitem__`, `~object.__delitem__`, and `!keys`.
Mutable sequences should provide methods `~sequence.append`, `~sequence.clear`, `~sequence.count`, `~sequence.extend`, `~sequence.index`, `~sequence.insert`, `~sequence.pop`, `~sequence.remove`, and `~sequence.reverse`, like Python standard `list` objects. Finally, sequence types should implement addition (meaning concatenation) and multiplication (meaning repetition) by defining the methods `~object.__add__`, `~object.__radd__`, `~object.__iadd__`, `~object.__mul__`, `~object.__rmul__` and `~object.__imul__` described below; they should not define other numerical operators.
It is recommended that both mappings and sequences implement the `~object.__contains__` method to allow efficient use of the `in` operator; for mappings, `in` should search the mapping's keys; for sequences, it should search through the values. It is further recommended that both mappings and sequences implement the `~object.__iter__` method to allow efficient iteration through the container; for mappings, `!__iter__` should iterate through the object's keys; for sequences, it should iterate through the values.
object.\_\_len\_\_(self)
pair: built-in function; len single: \_\_bool\_\_() (object method)
Called to implement the built-in function `len`. Should return the length of the object, an integer `>=` 0. Also, an object that doesn't define a `~object.__bool__` method and whose `!__len__` method returns zero is considered to be false in a Boolean context.
In CPython, the length is required to be at most `sys.maxsize`. If the length is larger than `!sys.maxsize` some features (such as `len`) may raise `OverflowError`. To prevent raising `!OverflowError` by truth value testing, an object must define a `~object.__bool__` method.
object.\_\_length_hint\_\_(self)
Called to implement `operator.length_hint`. Should return an estimated length for the object (which may be greater or less than the actual length). The length must be an integer `>=` 0. The return value may also be `NotImplemented`, which is treated the same as if the `__length_hint__` method didn't exist at all. This method is purely an optimization and is never required for correctness.
3.4
object.\_\_getitem\_\_(self, subscript)
Called to implement *subscription*, that is, `self[subscript]`. See `subscriptions` for details on the syntax.
There are two types of built-in objects that support subscription via `!__getitem__`:
- **sequences**, where *subscript* (also called `index`) should be an integer or a `slice` object. See the `sequence documentation ` for the expected behavior, including handling `slice` objects and negative indices.
- **mappings**, where *subscript* is also called the `key`. See `mapping documentation ` for the expected behavior.
If *subscript* is of an inappropriate type, `!__getitem__` should raise `TypeError`. If *subscript* has an inappropriate value, `!__getitem__` should raise an `LookupError` or one of its subclasses (`IndexError` for sequences; `KeyError` for mappings).
pair: object; slice
> [!NOTE]
> Slicing is handled by `!__getitem__`, `~object.__setitem__`, and `~object.__delitem__`. A call like :
>
> a[1:2] = b
>
> is translated to :
>
> a[slice(1, 2, None)] = b
>
> and so forth. Missing slice items are always filled in with `None`.
> [!NOTE]
> The sequence iteration protocol (used, for example, in `for` loops), expects that an `IndexError` will be raised for illegal indexes to allow proper detection of the end of a sequence.
> [!NOTE]
> When `subscripting ` a *class*, the special class method `~object.__class_getitem__` may be called instead of `!__getitem__`. See `classgetitem-versus-getitem` for more details.
object.\_\_setitem\_\_(self, key, value)
Called to implement assignment to `self[key]`. Same note as for `__getitem__`. This should only be implemented for mappings if the objects support changes to the values for keys, or if new keys can be added, or for sequences if elements can be replaced. The same exceptions should be raised for improper *key* values as for the `__getitem__` method.
object.\_\_delitem\_\_(self, key)
Called to implement deletion of `self[key]`. Same note as for `__getitem__`. This should only be implemented for mappings if the objects support removal of keys, or for sequences if elements can be removed from the sequence. The same exceptions should be raised for improper *key* values as for the `__getitem__` method.
object.\_\_missing\_\_(self, key)
Called by `dict`.`__getitem__` to implement `self[key]` for dict subclasses when key is not in the dictionary.
object.\_\_iter\_\_(self)
This method is called when an `iterator` is required for a container. This method should return a new iterator object that can iterate over all the objects in the container. For mappings, it should iterate over the keys of the container.
object.\_\_reversed\_\_(self)
Called (if present) by the `reversed` built-in to implement reverse iteration. It should return a new iterator object that iterates over all the objects in the container in reverse order.
If the `__reversed__` method is not provided, the `reversed` built-in will fall back to using the sequence protocol (`__len__` and `__getitem__`). Objects that support the sequence protocol should only provide `__reversed__` if they can provide an implementation that is more efficient than the one provided by `reversed`.
The membership test operators (`in` and `not in`) are normally implemented as an iteration through a container. However, container objects can supply the following special method with a more efficient implementation, which also does not require the object be iterable.
object.\_\_contains\_\_(self, item)
Called to implement membership test operators. Should return true if *item* is in *self*, false otherwise. For mapping objects, this should consider the keys of the mapping rather than the values or the key-item pairs.
For objects that don't define `__contains__`, the membership test first tries iteration via `__iter__`, then the old sequence iteration protocol via `__getitem__`, see `this section in the language
reference `.
### Emulating numeric types
The following methods can be defined to emulate numeric objects. Methods corresponding to operations that are not supported by the particular kind of number implemented (e.g., bitwise operations for non-integral numbers) should be left undefined.
These methods are called to implement the binary arithmetic operations (`+`, `-`, `*`, `@`, `/`, `//`, `%`, `divmod`, `pow`, `**`, `<<`, `>>`, `&`, `^`, `|`). For instance, to evaluate the expression `x + y`, where *x* is an instance of a class that has an `__add__` method, `type(x).__add__(x, y)` is called. The `__divmod__` method should be the equivalent to using `__floordiv__` and `__mod__`; it should not be related to `__truediv__`. Note that `__pow__` should be defined to accept an optional third argument if the three-argument version of the built-in `pow` function is to be supported.
If one of those methods does not support the operation with the supplied arguments, it should return `NotImplemented`.
These methods are called to implement the binary arithmetic operations (`+`, `-`, `*`, `@`, `/`, `//`, `%`, `divmod`, `pow`, `**`, `<<`, `>>`, `&`, `^`, `|`) with reflected (swapped) operands. These functions are only called if the operands are of different types, when the left operand does not support the corresponding operation[^3], or the right operand's class is derived from the left operand's class.[^4] For instance, to evaluate the expression `x - y`, where *y* is an instance of a class that has an `__rsub__` method, `type(y).__rsub__(y, x)` is called if `type(x).__sub__(x, y)` returns `NotImplemented` or `type(y)` is a subclass of `type(x)`.[^5]
Note that `__rpow__` should be defined to accept an optional third argument if the three-argument version of the built-in `pow` function is to be supported.
3.14
Three-argument `pow` now try calling `~object.__rpow__` if necessary. Previously it was only called in two-argument `!pow` and the binary power operator.
> [!NOTE]
> If the right operand's type is a subclass of the left operand's type and that subclass provides a different implementation of the reflected method for the operation, this method will be called before the left operand's non-reflected method. This behavior allows subclasses to override their ancestors' operations.
object.\_\_iadd\_\_(self, other) object.\_\_isub\_\_(self, other) object.\_\_imul\_\_(self, other) object.\_\_imatmul\_\_(self, other) object.\_\_itruediv\_\_(self, other) object.\_\_ifloordiv\_\_(self, other) object.\_\_imod\_\_(self, other) object.\_\_ipow\_\_(self, other\[, modulo\]) object.\_\_ilshift\_\_(self, other) object.\_\_irshift\_\_(self, other) object.\_\_iand\_\_(self, other) object.\_\_ixor\_\_(self, other) object.\_\_ior\_\_(self, other)
These methods are called to implement the augmented arithmetic assignments (`+=`, `-=`, `*=`, `@=`, `/=`, `//=`, `%=`, `**=`, `<<=`, `>>=`, `&=`, `^=`, `|=`). These methods should attempt to do the operation in-place (modifying *self*) and return the result (which could be, but does not have to be, *self*). If a specific method is not defined, or if that method returns `NotImplemented`, the augmented assignment falls back to the normal methods. For instance, if *x* is an instance of a class with an `__iadd__` method, `x += y` is equivalent to `x = x.__iadd__(y)` . If `__iadd__` does not exist, or if `x.__iadd__(y)` returns `!NotImplemented`, `x.__add__(y)` and `y.__radd__(x)` are considered, as with the evaluation of `x + y`. In certain situations, augmented assignment can result in unexpected errors (see `faq-augmented-assignment-tuple-error`), but this behavior is in fact part of the data model.
Called to implement the built-in functions `complex`, `int` and `float`. Should return a value of the appropriate type.
object.\_\_index\_\_(self)
Called to implement `operator.index`, and whenever Python needs to losslessly convert the numeric object to an integer object (such as in slicing, or in the built-in `bin`, `hex` and `oct` functions). Presence of this method indicates that the numeric object is an integer type. Must return an integer.
If `__int__`, `__float__` and `__complex__` are not defined then corresponding built-in functions `int`, `float` and `complex` fall back to `__index__`.
Called to implement the built-in function `round` and `math` functions `~math.trunc`, `~math.floor` and `~math.ceil`. Unless *ndigits* is passed to `!__round__` all these methods should return the value of the object truncated to an `~numbers.Integral` (typically an `int`).
3.14 `int` no longer delegates to the `~object.__trunc__` method.
### With Statement Context Managers
A `context manager` is an object that defines the runtime context to be established when executing a `with` statement. The context manager handles the entry into, and the exit from, the desired runtime context for the execution of the block of code. Context managers are normally invoked using the `!with` statement (described in section `with`), but can also be used by directly invoking their methods.
pair: statement; with single: context manager
Typical uses of context managers include saving and restoring various kinds of global state, locking and unlocking resources, closing opened files, etc.
For more information on context managers, see `typecontextmanager`. The `object` class itself does not provide the context manager methods.
object.\_\_enter\_\_(self)
Enter the runtime context related to this object. The `with` statement will bind this method's return value to the target(s) specified in the `!as` clause of the statement, if any.
object.\_\_exit\_\_(self, exc_type, exc_value, traceback)
Exit the runtime context related to this object. The parameters describe the exception that caused the context to be exited. If the context was exited without an exception, all three arguments will be `None`.
If an exception is supplied, and the method wishes to suppress the exception (i.e., prevent it from being propagated), it should return a true value. Otherwise, the exception will be processed normally upon exit from this method.
Note that `~object.__exit__` methods should not reraise the passed-in exception; this is the caller's responsibility.
`343` - The "with" statement
The specification, background, and examples for the Python `with` statement.
### Customizing positional arguments in class pattern matching
When using a class name in a pattern, positional arguments in the pattern are not allowed by default, i.e. `case MyClass(x, y)` is typically invalid without special support in `MyClass`. To be able to use that kind of pattern, the class needs to define a *\_\_match_args\_\_* attribute.
object.\_\_match_args\_\_
This class variable can be assigned a tuple of strings. When this class is used in a class pattern with positional arguments, each positional argument will be converted into a keyword argument, using the corresponding value in *\_\_match_args\_\_* as the keyword. The absence of this attribute is equivalent to setting it to `()`.
For example, if `MyClass.__match_args__` is `("left", "center", "right")` that means that `case MyClass(x, y)` is equivalent to `case MyClass(left=x, center=y)`. Note that the number of arguments in the pattern must be smaller than or equal to the number of elements in *\_\_match_args\_\_*; if it is larger, the pattern match attempt will raise a `TypeError`.
3.10
`634` - Structural Pattern Matching
The specification for the Python `match` statement.
### Emulating buffer types
The `buffer protocol ` provides a way for Python objects to expose efficient access to a low-level memory array. This protocol is implemented by builtin types such as `bytes` and `memoryview`, and third-party libraries may define additional buffer types.
While buffer types are usually implemented in C, it is also possible to implement the protocol in Python.
object.\_\_buffer\_\_(self, flags)
Called when a buffer is requested from *self* (for example, by the `memoryview` constructor). The *flags* argument is an integer representing the kind of buffer requested, affecting for example whether the returned buffer is read-only or writable. `inspect.BufferFlags` provides a convenient way to interpret the flags. The method must return a `memoryview` object.
**Thread safety:** In `free-threaded ` Python, implementations must manage any internal export counter using atomic operations. The method must be safe to call concurrently from multiple threads, and the returned buffer's underlying data must remain valid until the corresponding `~object.__release_buffer__` call completes. See `thread-safety-memoryview` for details.
object.\_\_release_buffer\_\_(self, buffer)
Called when a buffer is no longer needed. The *buffer* argument is a `memoryview` object that was previously returned by `~object.__buffer__`. The method must release any resources associated with the buffer. This method should return `None`.
**Thread safety:** In `free-threaded ` Python, any export counter decrement must use atomic operations. Resource cleanup must be thread-safe, as the final release may race with concurrent releases from other threads.
Buffer objects that do not need to perform any cleanup are not required to implement this method.
3.12
`688` - Making the buffer protocol accessible in Python
Introduces the Python `__buffer__` and `__release_buffer__` methods.
`collections.abc.Buffer`
ABC for buffer types.
### Annotations
Functions, classes, and modules may contain `annotations `, which are a way to associate information (usually `type hints `) with a symbol.
object.\_\_annotations\_\_
This attribute contains the annotations for an object. It is `lazily evaluated `, so accessing the attribute may execute arbitrary code and raise exceptions. If evaluation is successful, the attribute is set to a dictionary mapping from variable names to annotations.
3.14 Annotations are now lazily evaluated.
object.\_\_annotate\_\_(format)
An `annotate function`. Returns a new dictionary object mapping attribute/parameter names to their annotation values.
Takes a format parameter specifying the format in which annotations values should be provided. It must be a member of the `annotationlib.Format` enum, or an integer with a value corresponding to a member of the enum.
If an annotate function doesn't support the requested format, it must raise `NotImplementedError`. Annotate functions must always support `~annotationlib.Format.VALUE` format; they must not raise `NotImplementedError()` when called with this format.
When called with `~annotationlib.Format.VALUE` format, an annotate function may raise `NameError`; it must not raise `!NameError` when called requesting any other format.
If an object does not have any annotations, `~object.__annotate__` should preferably be set to `None` (it can’t be deleted), rather than set to a function that returns an empty dict.
3.14
`649` --- Deferred evaluation of annotation using descriptors
Introduces lazy evaluation of annotations and the `__annotate__` function.
### Special method lookup
For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object's type, not in the object's instance dictionary. That behaviour is the reason why the following code raises an exception:
>>> class C:
... pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
File "", line 1, in
TypeError: object of type 'C' has no len()
The rationale behind this behaviour lies with a number of special methods such as `~object.__hash__` and `~object.__repr__` that are implemented by all objects, including type objects. If the implicit lookup of these methods used the conventional lookup process, they would fail when invoked on the type object itself:
>>> 1 .__hash__() == hash(1)
True
>>> int.__hash__() == hash(int)
Traceback (most recent call last):
File "", line 1, in
TypeError: descriptor '__hash__' of 'int' object needs an argument
Incorrectly attempting to invoke an unbound method of a class in this way is sometimes referred to as 'metaclass confusion', and is avoided by bypassing the instance when looking up special methods:
>>> type(1).__hash__(1) == hash(1)
True
>>> type(int).__hash__(int) == hash(int)
True
In addition to bypassing any instance attributes in the interest of correctness, implicit special method lookup generally also bypasses the `~object.__getattribute__` method even of the object's metaclass:
>>> class Meta(type):
... def __getattribute__(*args):
... print("Metaclass getattribute invoked")
... return type.__getattribute__(*args)
...
>>> class C(object, metaclass=Meta):
... def __len__(self):
... return 10
... def __getattribute__(*args):
... print("Class getattribute invoked")
... return object.__getattribute__(*args)
...
>>> c = C()
>>> c.__len__() # Explicit lookup via instance
Class getattribute invoked
10
>>> type(c).__len__(c) # Explicit lookup via type
Metaclass getattribute invoked
10
>>> len(c) # Implicit lookup
10
Bypassing the `~object.__getattribute__` machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods (the special method *must* be set on the class object itself in order to be consistently invoked by the interpreter).
single: coroutine
## Coroutines
### Awaitable Objects
An `awaitable` object generally implements an `~object.__await__` method. `Coroutine objects ` returned from `async def` functions are awaitable.
> [!NOTE]
> The `generator iterator` objects returned from generators decorated with `types.coroutine` are also awaitable, but they do not implement `~object.__await__`.
object.\_\_await\_\_(self)
Must return an `iterator`. Should be used to implement `awaitable` objects. For instance, `asyncio.Future` implements this method to be compatible with the `await` expression. The `object` class itself is not awaitable and does not provide this method.
> [!NOTE]
> The language doesn't place any restriction on the type or value of the objects yielded by the iterator returned by `__await__`, as this is specific to the implementation of the asynchronous execution framework (e.g. `asyncio`) that will be managing the `awaitable` object.
3.5
`492` for additional information about awaitable objects.
### Coroutine Objects
`Coroutine objects ` are `awaitable` objects. A coroutine's execution can be controlled by calling `~object.__await__` and iterating over the result. When the coroutine has finished executing and returns, the iterator raises `StopIteration`, and the exception's `~StopIteration.value` attribute holds the return value. If the coroutine raises an exception, it is propagated by the iterator. Coroutines should not directly raise unhandled `StopIteration` exceptions.
Coroutines also have the methods listed below, which are analogous to those of generators (see `generator-methods`). However, unlike generators, coroutines do not directly support iteration.
Coroutines are `generic ` over the types of their yield, send, and return values, respectively.
3.5.2 It is a `RuntimeError` to await on a coroutine more than once.
coroutine.send(value)
Starts or resumes execution of the coroutine. If *value* is `None`, this is equivalent to advancing the iterator returned by `~object.__await__`. If *value* is not `None`, this method delegates to the `~generator.send` method of the iterator that caused the coroutine to suspend. The result (return value, `StopIteration`, or other exception) is the same as when iterating over the `!__await__` return value, described above.
coroutine.throw(value) coroutine.throw(type\[, value\[, traceback\]\])
Raises the specified exception in the coroutine. This method delegates to the `~generator.throw` method of the iterator that caused the coroutine to suspend, if it has such a method. Otherwise, the exception is raised at the suspension point. The result (return value, `StopIteration`, or other exception) is the same as when iterating over the `~object.__await__` return value, described above. If the exception is not caught in the coroutine, it propagates back to the caller.
3.12
The second signature (type\[, value\[, traceback\]\]) is deprecated and may be removed in a future version of Python.
coroutine.close()
Causes the coroutine to clean itself up and exit. If the coroutine is suspended, this method first delegates to the `~generator.close` method of the iterator that caused the coroutine to suspend, if it has such a method. Then it raises `GeneratorExit` at the suspension point, causing the coroutine to immediately clean itself up. Finally, the coroutine is marked as having finished executing, even if it was never started.
Coroutine objects are automatically closed using the above process when they are about to be destroyed.
### Asynchronous Iterators
An *asynchronous iterator* can call asynchronous code in its `__anext__` method.
Asynchronous iterators can be used in an `async for` statement.
The `object` class itself does not provide these methods.
object.\_\_aiter\_\_(self)
Must return an *asynchronous iterator* object.
object.\_\_anext\_\_(self)
Must return an *awaitable* resulting in a next value of the iterator. Should raise a `StopAsyncIteration` error when the iteration is over.
An example of an asynchronous iterable object:
class Reader:
async def readline(self):
...
def __aiter__(self):
return self
async def __anext__(self):
val = await self.readline()
if val == b'':
raise StopAsyncIteration
return val
3.5
3.7 Prior to Python 3.7, `~object.__aiter__` could return an *awaitable* that would resolve to an `asynchronous iterator `.
Starting with Python 3.7, `~object.__aiter__` must return an asynchronous iterator object. Returning anything else will result in a `TypeError` error.
### Asynchronous Context Managers
An *asynchronous context manager* is a *context manager* that is able to suspend execution in its `__aenter__` and `__aexit__` methods.
Asynchronous context managers can be used in an `async with` statement.
The `object` class itself does not provide these methods.
object.\_\_aenter\_\_(self)
Semantically similar to `~object.__enter__`, the only difference being that it must return an *awaitable*.
object.\_\_aexit\_\_(self, exc_type, exc_value, traceback)
Semantically similar to `~object.__exit__`, the only difference being that it must return an *awaitable*.
An example of an asynchronous context manager class:
class AsyncContextManager:
async def __aenter__(self):
await log('entering context')
async def __aexit__(self, exc_type, exc, tb):
await log('exiting context')
3.5
**Footnotes**
[^1]: It *is* possible in some cases to change an object's type, under certain controlled conditions. It generally isn't a good idea though, since it can lead to some very strange behaviour if it is handled incorrectly.
[^2]: The `~object.__hash__`, `~object.__iter__`, `~object.__reversed__`, `~object.__contains__`, `~object.__class_getitem__` and `~os.PathLike.__fspath__` methods have special handling for this. Others will still raise a `TypeError`, but may do so by relying on the behavior that `None` is not callable.
[^3]: "Does not support" here means that the class has no such method, or the method returns `NotImplemented`. Do not set the method to `None` if you want to force fallback to the right operand's reflected method—that will instead have the opposite effect of explicitly *blocking* such fallback.
[^4]: For operands of the same type, it is assumed that if the non-reflected method (such as `~object.__add__`) fails then the operation is not supported, which is why the reflected method is not called.
[^5]: If the right operand's type is a subclass of the left operand's type, the reflected method having precedence allows subclasses to override their ancestors' operations.
# Execution model
single: execution model pair: code; block
## Structure of a program
block
A Python program is constructed from code blocks. A `block` is a piece of Python program text that is executed as a unit. The following are blocks: a module, a function body, and a class definition. Each command typed interactively is a block. A script file (a file given as standard input to the interpreter or specified as a command line argument to the interpreter) is a code block. A script command (a command specified on the interpreter command line with the `-c` option) is a code block. A module run as a top level script (as module `__main__`) from the command line using a `-m` argument is also a code block. The string argument passed to the built-in functions `eval` and `exec` is a code block.
pair: execution; frame
A code block is executed in an `execution frame`. A frame contains some administrative information (used for debugging) and determines where and how execution continues after the code block's execution has completed.
## Naming and binding
single: namespace single: scope
### Binding of names
single: name pair: binding; name
`Names` refer to objects. Names are introduced by name binding operations.
single: from; import statement
The following constructs bind names:
- formal parameters to functions,
- class definitions,
- function definitions,
- assignment expressions,
- `targets ` that are identifiers if occurring in an assignment:
- `for` loop header,
- after `!as` in a `with` statement, `except` clause, `except* ` clause, or in the as-pattern in structural pattern matching,
- in a capture pattern in structural pattern matching
- `import` statements.
- `type` statements.
- `type parameter lists `.
The `!import` statement of the form `from ... import *` binds all names defined in the imported module, except those beginning with an underscore. This form may only be used at the module level.
A target occurring in a `del` statement is also considered bound for this purpose (though the actual semantics are to unbind the name).
Each assignment or import statement occurs within a block defined by a class or function definition or at the module level (the top-level code block).
pair: free; variable
If a name is bound in a block, it is a local variable of that block, unless declared as `nonlocal` or `global`. If a name is bound at the module level, it is a global variable. (The variables of the module code block are local and global.) If a variable is used in a code block but not defined there, it is a `free variable`.
Each occurrence of a name in the program text refers to the `binding` of that name established by the following name resolution rules.
### Resolution of names
scope
A `scope` defines the visibility of a name within a block. If a local variable is defined in a block, its scope includes that block. If the definition occurs in a function block, the scope extends to any blocks contained within the defining one, unless a contained block introduces a different binding for the name.
single: environment
When a name is used in a code block, it is resolved using the nearest enclosing scope. The set of all such scopes visible to a code block is called the block's `environment`.
When a name is not found at all, a `NameError` exception is raised. If the current scope is a function scope, and the name refers to a local variable that has not yet been bound to a value at the point where the name is used, an `UnboundLocalError` exception is raised. `UnboundLocalError` is a subclass of `NameError`.
If a name binding operation occurs anywhere within a code block, all uses of the name within the block are treated as references to the current block. This can lead to errors when a name is used within a block before it is bound. This rule is subtle. Python lacks declarations and allows name binding operations to occur anywhere within a code block. The local variables of a code block can be determined by scanning the entire text of the block for name binding operations. See `the FAQ entry on UnboundLocalError ` for examples.
If the `global` statement occurs within a block, all uses of the names specified in the statement refer to the bindings of those names in the top-level namespace. Names are resolved in the top-level namespace by searching the global namespace, i.e. the namespace of the module containing the code block, and the builtins namespace, the namespace of the module `builtins`. The global namespace is searched first. If the names are not found there, the builtins namespace is searched next. If the names are also not found in the builtins namespace, new variables are created in the global namespace. The global statement must precede all uses of the listed names.
The `global` statement has the same scope as a name binding operation in the same block. If the nearest enclosing scope for a free variable contains a global statement, the free variable is treated as a global.
The `nonlocal` statement causes corresponding names to refer to previously bound variables in the nearest enclosing function scope. `SyntaxError` is raised at compile time if the given name does not exist in any enclosing function scope. `Type parameters ` cannot be rebound with the `!nonlocal` statement.
pair: module; \_\_main\_\_
The namespace for a module is automatically created the first time a module is imported. The main module for a script is always called `__main__`.
Class definition blocks and arguments to `exec` and `eval` are special in the context of name resolution. A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution with an exception that unbound local variables are looked up in the global namespace. The namespace of the class definition becomes the attribute dictionary of the class. The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods. This includes comprehensions and generator expressions, but it does not include `annotation scopes `, which have access to their enclosing class scopes. This means that the following will fail:
class A:
a = 42
b = list(a + i for i in range(10))
However, the following will succeed:
class A:
type Alias = Nested
class Nested: pass
print(A.Alias.__value__) #
### Annotation scopes
`Annotations `, `type parameter lists ` and `type` statements introduce *annotation scopes*, which behave mostly like function scopes, but with some exceptions discussed below.
Annotation scopes are used in the following contexts:
- `Function annotations `.
- `Variable annotations `.
- Type parameter lists for `generic type aliases `.
- Type parameter lists for `generic functions `. A generic function's annotations are executed within the annotation scope, but its defaults and decorators are not.
- Type parameter lists for `generic classes `. A generic class's base classes and keyword arguments are executed within the annotation scope, but its decorators are not.
- The bounds, constraints, and default values for type parameters (`lazily evaluated `).
- The value of type aliases (`lazily evaluated `).
Annotation scopes differ from function scopes in the following ways:
- Annotation scopes have access to their enclosing class namespace. If an annotation scope is immediately within a class scope, or within another annotation scope that is immediately within a class scope, the code in the annotation scope can use names defined in the class scope as if it were executed directly within the class body. This contrasts with regular functions defined within classes, which cannot access names defined in the class scope.
- Expressions in annotation scopes cannot contain `yield`, `yield from`, `await`, or `:= ` expressions. (These expressions are allowed in other scopes contained within the annotation scope.)
- Names defined in annotation scopes cannot be rebound with `nonlocal` statements in inner scopes. This includes only type parameters, as no other syntactic elements that can appear within annotation scopes can introduce new names.
- While annotation scopes have an internal name, that name is not reflected in the `qualified name` of objects defined within the scope. Instead, the `~definition.__qualname__` of such objects is as if the object were defined in the enclosing scope.
3.12 Annotation scopes were introduced in Python 3.12 as part of `695`.
3.13 Annotation scopes are also used for type parameter defaults, as introduced by `696`.
3.14 Annotation scopes are now also used for annotations, as specified in `649` and `749`.
### Lazy evaluation
Most annotation scopes are *lazily evaluated*. This includes annotations, the values of type aliases created through the `type` statement, and the bounds, constraints, and default values of type variables created through the `type parameter syntax `. This means that they are not evaluated when the type alias or type variable is created, or when the object carrying annotations is created. Instead, they are only evaluated when necessary, for example when the `__value__` attribute on a type alias is accessed.
Example:
\>\>\> type Alias = 1/0 \>\>\> Alias.\_\_value\_\_ Traceback (most recent call last): ... ZeroDivisionError: division by zero \>\>\> def func\[T: 1/0\](): pass \>\>\> T = func.\_\_type_params\_\_\[0\] \>\>\> T.\_\_bound\_\_ Traceback (most recent call last): ... ZeroDivisionError: division by zero
Here the exception is raised only when the `__value__` attribute of the type alias or the `__bound__` attribute of the type variable is accessed.
This behavior is primarily useful for references to types that have not yet been defined when the type alias or type variable is created. For example, lazy evaluation enables creation of mutually recursive type aliases:
from typing import Literal
type SimpleExpr = int | Parenthesized
type Parenthesized = tuple[Literal["("], Expr, Literal[")"]]
type Expr = SimpleExpr | tuple[SimpleExpr, Literal["+", "-"], Expr]
Lazily evaluated values are evaluated in `annotation scope `, which means that names that appear inside the lazily evaluated value are looked up as if they were used in the immediately enclosing scope.
3.12
### Builtins and restricted execution
pair: restricted; execution
Users should not touch `__builtins__`; it is strictly an implementation detail. Users wanting to override values in the builtins namespace should `import` the `builtins` module and modify its attributes appropriately.
The builtins namespace associated with the execution of a code block is actually found by looking up the name `__builtins__` in its global namespace; this should be a dictionary or a module (in the latter case the module's dictionary is used). By default, when in the `__main__` module, `__builtins__` is the built-in module `builtins`; when in any other module, `__builtins__` is an alias for the dictionary of the `builtins` module itself.
### Interaction with dynamic features
Name resolution of free variables occurs at runtime, not at compile time. This means that the following code will print 42:
i = 10
def f():
print(i)
i = 42
f()
The `eval` and `exec` functions do not have access to the full environment for resolving names. Names may be resolved in the local and global namespaces of the caller. Free variables are not resolved in the nearest enclosing namespace, but in the global namespace.[^1] The `exec` and `eval` functions have optional arguments to override the global and local namespace. If only one namespace is specified, it is used for both.
## Exceptions
single: exception
single: raise an exception single: handle an exception single: exception handler single: errors single: error handling
Exceptions are a means of breaking out of the normal flow of control of a code block in order to handle errors or other exceptional conditions. An exception is *raised* at the point where the error is detected; it may be *handled* by the surrounding code block or by any code block that directly or indirectly invoked the code block where the error occurred.
The Python interpreter raises an exception when it detects a run-time error (such as division by zero). A Python program can also explicitly raise an exception with the `raise` statement. Exception handlers are specified with the `try` ... `except` statement. The `finally` clause of such a statement can be used to specify cleanup code which does not handle the exception, but is executed whether an exception occurred or not in the preceding code.
single: termination model
Python uses the "termination" model of error handling: an exception handler can find out what happened and continue execution at an outer level, but it cannot repair the cause of the error and retry the failing operation (except by re-entering the offending piece of code from the top).
single: SystemExit (built-in exception)
When an exception is not handled at all, the interpreter terminates execution of the program, or returns to its interactive main loop. In either case, it prints a stack traceback, except when the exception is `SystemExit`.
Exceptions are identified by class instances. The `except` clause is selected depending on the class of the instance: it must reference the class of the instance or a `non-virtual base class ` thereof. The instance can be received by the handler and can carry additional information about the exceptional condition.
> [!NOTE]
> Exception messages are not part of the Python API. Their contents may change from one version of Python to the next without warning and should not be relied on by code which will run under multiple versions of the interpreter.
See also the description of the `try` statement in section `try` and `raise` statement in section `raise`.
## Runtime Components
### General Computing Model
Python's execution model does not operate in a vacuum. It runs on a host machine and through that host's runtime environment, including its operating system (OS), if there is one. When a program runs, the conceptual layers of how it runs on the host look something like this:
> **host machine**\
> \*\*process\*\* (global resources)\
> \*\*thread\*\* (runs machine code)
Each process represents a program running on the host. Think of each process itself as the data part of its program. Think of the process' threads as the execution part of the program. This distinction will be important to understand the conceptual Python runtime.
The process, as the data part, is the execution context in which the program runs. It mostly consists of the set of resources assigned to the program by the host, including memory, signals, file handles, sockets, and environment variables.
Processes are isolated and independent from one another. (The same is true for hosts.) The host manages the process' access to its assigned resources, in addition to coordinating between processes.
Each thread represents the actual execution of the program's machine code, running relative to the resources assigned to the program's process. It's strictly up to the host how and when that execution takes place.
From the point of view of Python, a program always starts with exactly one thread. However, the program may grow to run in multiple simultaneous threads. Not all hosts support multiple threads per process, but most do. Unlike processes, threads in a process are not isolated and independent from one another. Specifically, all threads in a process share all of the process' resources.
The fundamental point of threads is that each one does *run* independently, at the same time as the others. That may be only conceptually at the same time ("concurrently") or physically ("in parallel"). Either way, the threads effectively run at a non-synchronized rate.
> [!NOTE]
> That non-synchronized rate means none of the process' memory is guaranteed to stay consistent for the code running in any given thread. Thus multi-threaded programs must take care to coordinate access to intentionally shared resources. Likewise, they must take care to be absolutely diligent about not accessing any *other* resources in multiple threads; otherwise two threads running at the same time might accidentally interfere with each other's use of some shared data. All this is true for both Python programs and the Python runtime.
>
> The cost of this broad, unstructured requirement is the tradeoff for the kind of raw concurrency that threads provide. The alternative to the required discipline generally means dealing with non-deterministic bugs and data corruption.
### Python Runtime Model
The same conceptual layers apply to each Python program, with some extra data layers specific to Python:
> **host machine**\
> \*\*process\*\* (global resources)\
> Python global runtime (*state*)\
> Python interpreter (*state*)\
> \*\*thread\*\* (runs Python bytecode and "C-API")\
> Python thread *state*
At the conceptual level: when a Python program starts, it looks exactly like that diagram, with one of each. The runtime may grow to include multiple interpreters, and each interpreter may grow to include multiple thread states.
> [!NOTE]
> A Python implementation won't necessarily implement the runtime layers distinctly or even concretely. The only exception is places where distinct layers are directly specified or exposed to users, like through the `threading` module.
> [!NOTE]
> The initial interpreter is typically called the "main" interpreter. Some Python implementations, like CPython, assign special roles to the main interpreter.
>
> Likewise, the host thread where the runtime was initialized is known as the "main" thread. It may be different from the process' initial thread, though they are often the same. In some cases "main thread" may be even more specific and refer to the initial thread state. A Python runtime might assign specific responsibilities to the main thread, such as handling signals.
As a whole, the Python runtime consists of the global runtime state, interpreters, and thread states. The runtime ensures all that state stays consistent over its lifetime, particularly when used with multiple host threads.
The global runtime, at the conceptual level, is just a set of interpreters. While those interpreters are otherwise isolated and independent from one another, they may share some data or other resources. The runtime is responsible for managing these global resources safely. The actual nature and management of these resources is implementation-specific. Ultimately, the external utility of the global runtime is limited to managing interpreters.
In contrast, an "interpreter" is conceptually what we would normally think of as the (full-featured) "Python runtime". When machine code executing in a host thread interacts with the Python runtime, it calls into Python in the context of a specific interpreter.
> [!NOTE]
> The term "interpreter" here is not the same as the "bytecode interpreter", which is what regularly runs in threads, executing compiled Python code.
>
> In an ideal world, "Python runtime" would refer to what we currently call "interpreter". However, it's been called "interpreter" at least since introduced in 1997 ([CPython:a027efa5b](https://github.com/python/cpython/commit/a027efa5b)).
Each interpreter completely encapsulates all of the non-process-global, non-thread-specific state needed for the Python runtime to work. Notably, the interpreter's state persists between uses. It includes fundamental data like `sys.modules`. The runtime ensures multiple threads using the same interpreter will safely share it between them.
A Python implementation may support using multiple interpreters at the same time in the same process. They are independent and isolated from one another. For example, each interpreter has its own `sys.modules`.
For thread-specific runtime state, each interpreter has a set of thread states, which it manages, in the same way the global runtime contains a set of interpreters. It can have thread states for as many host threads as it needs. It may even have multiple thread states for the same host thread, though that isn't as common.
Each thread state, conceptually, has all the thread-specific runtime data an interpreter needs to operate in one host thread. The thread state includes the current raised exception and the thread's Python call stack. It may include other thread-specific resources.
> [!NOTE]
> The term "Python thread" can sometimes refer to a thread state, but normally it means a thread created using the `threading` module.
Each thread state, over its lifetime, is always tied to exactly one interpreter and exactly one host thread. It will only ever be used in that thread and with that interpreter.
Multiple thread states may be tied to the same host thread, whether for different interpreters or even the same interpreter. However, for any given host thread, only one of the thread states tied to it can be used by the thread at a time.
Thread states are isolated and independent from one another and don't share any data, except for possibly sharing an interpreter and objects or other resources belonging to that interpreter.
Once a program is running, new Python threads can be created using the `threading` module (on platforms and Python implementations that support threads). Additional processes can be created using the `os`, `subprocess`, and `multiprocessing` modules. Interpreters can be created and used with the `~concurrent.interpreters` module. Coroutines (async) can be run using `asyncio` in each interpreter, typically only in a single thread (often the main thread).
**Footnotes**
[^1]: This limitation occurs because the code that is executed by these operations is not available at the time the module is compiled.
# Expressions
expression, BNF
This chapter explains the meaning of the elements of expressions in Python.
**Syntax Notes:** In this and the following chapters, `grammar notation ` will be used to describe syntax, not lexical analysis.
When (one alternative of) a syntax rule has the form:
python-grammar name: othername
and no semantics are given, the semantics of this form of `name` are the same as for `othername`.
## Arithmetic conversions
pair: arithmetic; conversion
When a description of an arithmetic operator below uses the phrase "the numeric arguments are converted to a common real type", this means that the operator implementation for built-in numeric types works as described in the `Numeric Types ` section of the standard library documentation.
Some additional rules apply for certain operators and non-numeric operands (for example, a string as a left argument to the `%` operator). Extensions must define their own conversion behavior.
## Atoms
atom
Atoms are the most basic elements of expressions. The simplest atoms are `builtin constants `, `names ` and `literals `. More complex atoms are enclosed in paired delimiters:
- `()` (parentheses): `groups `, `tuple displays `, `yield atoms `, and `generator expressions `;
- `[]` (square brackets): `list displays `;
- `{}` (curly braces): `dictionary ` and `set ` displays.
Formally, the syntax for atoms is:
atom:
builtin_constant\
identifier\
literal\
parenthesized_enclosure\
bracketed_enclosure\
braced_enclosure
parenthesized_enclosure:
group\
tuple\
yield_atom\
generator_expression
bracketed_enclosure:
listcomp\
list
braced_enclosure:
dictcomp\
dict\
setcomp\
set
### Built-in constants
The keywords `True`, `False`, and `None` name `built-in constants `. The token `...` names the `Ellipsis` constant.
Evaluation of these atoms yields the corresponding value.
> [!NOTE]
> Several more built-in constants are available as global variables, but only the ones mentioned here are `keywords `. In particular, these names cannot be reassigned or used as attributes:
>
> ```
> >>> False = 123
> File "", line 1
> False = 123
> ^^^^^
> SyntaxError: cannot assign to False
> ```
Formally, the syntax for built-in constants is:
An identifier occurring as an atom is a name. See section `identifiers` for lexical definition and section `naming` for documentation of naming and binding.
pair: exception; NameError
When the name is bound to an object, evaluation of the atom yields that object. When a name is not bound, an attempt to evaluate it raises a `NameError` exception.
pair: name; mangling pair: private; names
#### Private name mangling
When an identifier that textually occurs in a class definition begins with two or more underscore characters and does not end in two or more underscores, it is considered a `private name` of that class.
The `class specifications `.
More precisely, private names are transformed to a longer form before code is generated for them. If the transformed name is longer than 255 characters, implementation-defined truncation may happen.
The transformation is independent of the syntactical context in which the identifier is used but only the following private identifiers are mangled:
- Any name used as the name of a variable that is assigned or read or any name of an attribute being accessed.
The `~definition.__name__` attribute of nested functions, classes, and type aliases is however not mangled.
- The name of imported modules, e.g., `__spam` in `import __spam`. If the module is part of a package (i.e., its name contains a dot), the name is *not* mangled, e.g., the `__foo` in `import __foo.bar` is not mangled.
- The name of an imported member, e.g., `__f` in `from spam import __f`.
The transformation rule is defined as follows:
- The class name, with leading underscores removed and a single leading underscore inserted, is inserted in front of the identifier, e.g., the identifier `__spam` occurring in a class named `Foo`, `_Foo` or `__Foo` is transformed to `_Foo__spam`.
- If the class name consists only of underscores, the transformation is the identity, e.g., the identifier `__spam` occurring in a class named `_` or `__` is left as is.
### Literals
single: literal
A `literal` is a textual representation of a value. Python supports numeric, string and bytes literals. `Format strings ` and `template strings ` are treated as string literals.
Numeric literals consist of a single `NUMBER ` token, which names an integer, floating-point number, or an imaginary number. See the `numbers` section in Lexical analysis documentation for details.
String and bytes literals may consist of several tokens. See section `string-concatenation` for details.
Note that negative and complex numbers, like `-3` or `3+4.2j`, are syntactically not literals, but `unary ` or `binary ` arithmetic operations involving the `-` or `+` operator.
Evaluation of a literal yields an object of the given type (`int`, `float`, `complex`, `str`, `bytes`, or `~string.templatelib.Template`) with the given value. The value may be approximated in the case of floating-point and imaginary literals.
The formal grammar for literals is:
literal: strings \| NUMBER
triple: immutable; data; type pair: immutable; object
#### Literals and object identity
All literals correspond to immutable data types, and hence the object's identity is less important than its value. Multiple evaluations of literals with the same value (either the same occurrence in the program text or a different occurrence) may obtain the same object or a different object with the same value.
CPython implementation detail
For example, in CPython, *small* integers with the same value evaluate to the same object:
>>> x = 7
>>> y = 7
>>> x is y
True
However, large integers evaluate to different objects:
>>> x = 123456789
>>> y = 123456789
>>> x is y
False
This behavior may change in future versions of CPython. In particular, the boundary between "small" and "large" integers has already changed in the past.
CPython will emit a `SyntaxWarning` when you compare literals using `is`:
>>> x = 7
>>> x is 7
:1: SyntaxWarning: "is" with 'int' literal. Did you mean "=="?
True
See `faq-identity-with-is` for more information.
`Template strings ` are immutable but may reference mutable objects as `~string.templatelib.Interpolation` values. For the purposes of this section, two t-strings have the "same value" if both their structure and the *identity* of the values match.
Currently, each evaluation of a template string results in a different object.
#### String literal concatenation
Multiple adjacent string or bytes literals, possibly using different quoting conventions, are allowed, and their meaning is the same as their concatenation:
>>> "hello" 'world'
"helloworld"
This feature is defined at the syntactical level, so it only works with literals. To concatenate string expressions at run time, the '+' operator may be used:
>>> greeting = "Hello"
>>> space = " "
>>> name = "Blaise"
>>> print(greeting + space + name) # not: print(greeting space name)
Hello Blaise
Literal concatenation can freely mix raw strings, triple-quoted strings, and formatted string literals. For example:
>>> "Hello" r', ' f"{name}!"
"Hello, Blaise!"
This feature can be used to reduce the number of backslashes needed, to split long strings conveniently across long lines, or even to add comments to parts of strings. For example:
re.compile("[A-Za-z_]" # letter or underscore
"[A-Za-z0-9_]*" # letter, digit or underscore
)
However, bytes literals may only be combined with other byte literals; not with string literals of any kind. Also, template string literals may only be combined with other template string literals:
>>> t"Hello" t"{name}!"
Template(strings=('Hello', '!'), interpolations=(...))
Formally:
strings: (STRING \| fstring)+ \| tstring+
single: parenthesized form single: () (parentheses)
### Parenthesized groups
A `parenthesized group` is an expression enclosed in parentheses. The group evaluates to the same value as the expression inside.
Groups are used to override or clarify `operator precedence `, in the same way as in math notation. For example:
>>> 3 << 2 | 4
12
>>> 3 << (2 | 4) # Override precedence of the | (bitwise OR)
192
>>> (3 << 2) | 4 # Same as without parentheses (but more clear)
12
Note that not everything in parentheses is a *group*. Specifically, a parenthesized group must include exactly one expression, and cannot end with a comma. See `tuple displays ` and `generator expressions ` for other parenthesized forms.
Formally, the syntax for groups is:
group: '(' assignment_expression ')'
### Container displays
single: comprehensions
For constructing builtin containers (lists, sets, tuples or dictionaries), Python provides special syntax called `displays`. There are subtle differences between the four kinds of displays, detailed in the following sections. All displays, however, consist of comma-separated items enclosed in paired delimiters.
For example, a *list display* is a series of expressions enclosed in square brackets:
>>> ["one", "two", "three"]
['one', 'two', 'three']
>>> [1 + 2, 2 + 3]
[3, 5]
In list, tuple and dictionary (but not set) displays, the series may be empty:
>>> [] # empty list
[]
>>> () # empty tuple
()
>>> {} # empty dictionary
{}
pair: trailing; comma
If the series is not empty, the items may be followed by an additional comma, which has no effect:
>>> ["one", "two", "three",] # note comma after "three"
['one', 'two', 'three']
> [!NOTE]
> The trailing comma is often used for displays that span multiple lines (using `implicit line joining `), so when a future programmer adds a new entry at the end, they do not need to modify an existing line:
>
> >>> [
> ... 'one',
> ... 'two',
> ... 'three',
> ... ]
> ['one', 'two', 'three']
At runtime, when a display is evaluated, the listed items are evaluated from left to right and placed into a new container of the appropriate type.
pair: iterable; unpacking single: \* (asterisk); in expression lists
For tuple, list and set (but not dict) displays, any item in the display may be prefixed with an asterisk (`*`). This denotes `iterable unpacking `. At runtime, the asterisk-prefixed expression must evaluate to an iterable, whose contents are inserted into the container at the location of the unpacking. For example:
>>> numbers = (1, 2)
>>> [*numbers, 'word', *numbers]
[1, 2, 'word', 1, 2]
Dictionary displays use a similar mechanism called *dictionary unpacking*, denoted with a double asterisk (`**`). See `dict` for details.
A more advanced form of displays are `comprehensions`, where items are computed via a set of looping and filtering instructions. See the `comprehensions` section for details.
3.5 Iterable and dictionary unpacking in displays, originally proposed by `448`.
#### List displays
pair: list; display pair: list; comprehensions pair: empty; list pair: object; list single: \[\] (square brackets); list expression single: , (comma); expression list
A `list display` is a possibly empty series of expressions enclosed in square brackets. For example:
>>> ["one", "two", "three"]
['one', 'two', 'three']
>>> ["one"] # One-element list
['one']
>>> [] # empty list
[]
See `displays` for general information on displays.
The formal grammar for list displays is:
list: '\[' \[flexible_expression_list\] '\]'
#### Set displays
pair: set; display pair: set; comprehensions pair: object; set single: {} (curly brackets); set expression single: , (comma); expression list
A `set display` is a *non-empty* series of expressions enclosed in curly braces. For example:
>>> {"one", "two", "three"}
{'one', 'three', 'two'}
>>> {"one"} # One-element set
{'one'}
See `displays` for general information on displays.
There is no special syntax for the empty set. The `{}` literal is a `dictionary display ` that constructs an empty dictionary. Call `set() ` with no arguments to get an empty set.
The formal grammar for set displays is:
#### Tuple displays
A `tuple display` is a series of expressions enclosed in parentheses. For example:
>>> (1, 2)
(1, 2)
>>> () # an empty tuple
()
See `displays` for general information on displays.
To avoid ambiguity, if a tuple display has exactly one element, it requires a trailing comma. Without it, you get a `parenthesized group `:
>>> ('single',) # single-element tuple
('single',)
>>> ('single') # no comma: single string
'single'
To put it in other words, a tuple display is a parenthesized list of either:
- two or more comma-separated expressions, or
- zero or more expressions, each followed by a comma.
Since tuples are immutable, `object identity rules for literals ` also apply to tuples: at runtime, two occurrences of tuples with the same values may or may not yield the same object.
> [!NOTE]
> Python's syntax also includes `expression lists `, where a comma-separated list of expressions is *not* enclosed in parentheses but evaluates to tuple.
>
> In other words, when it comes to tuple syntax, the comma is more important that the use of parentheses. Only the empty tuple is spelled without a comma.
The formal grammar for tuple displays is:
A `dictionary display` is a possibly empty series of `dict items` enclosed in curly braces. Each dict item is a colon-separated pair of expressions: the `key` and its associated `value`. For example:
>>> {1: 'one', 2: 'two'}
{1: 'one', 2: 'two'}
At runtime, when a dictionary comprehension is evaluated, the expressions are evaluated from left to right. Each key object is used as a key into the dictionary to store the corresponding value. This means that you can specify the same key multiple times in the comprehension, and the final dictionary's value for a given key will be the last one given. For example:
>>> {
... 1: 'this will be overridden',
... 2: 'two',
... 1: 'also overridden',
... 1: 'one',
... }
{1: 'one', 2: 'two'}
unpacking; dictionary single: \*\*; in dictionary displays
Instead of a key-value pair, a dict item may be an expression prefixed by a double asterisk `**`. This denotes `dictionary unpacking`. At runtime, the expression must evaluate to a `mapping`; each item of the mapping is added to the new dictionary. As with key-value pairs, later values replace values already set by earlier items and unpackings. This may be used to override a set of defaults:
>>> defaults = {'color': 'blue', 'count': 8}
>>> overrides = {'color': 'yellow'}
>>> {**defaults, **overrides}
{'color': 'yellow', 'count': 8}
3.5 Unpacking into dictionary displays, originally proposed by `448`.
single: comprehensions single: for; in comprehensions
### Comprehensions
List, set and dictionary `comprehensions` are a form of `container displays ` where items are computed via a set of looping and filtering instructions rather than listed explicitly.
In its simplest form, a comprehension consists of a single expression followed by a `!for` clause. The `!for` clause has the same syntax as the header of a `for statement `, without a trailing colon.
For example, a list of the first ten squares is:
>>> [x**2 for x in range(10)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
At run time, a list comprehension creates a new list. The expression after `!in` must evaluate to an `iterable`. For each element of this iterable, the element is bound to the `!for` clause's target as in a `!for` statement, then the expression before `!for` is evaluated with the target in scope and the result is added to the new list. Thus, the example above is roughly equivalent to defining and calling the following function:
def make_list_of_squares(iterable):
result = []
for x in iterable:
result.append(x**2)
return result
make_list_of_squares(range(10))
Set comprehensions work similarly. For example, here is a set of lowercase letters:
>>> {x.lower() for x in ['a', 'A', 'b', 'C']}
{'c', 'a', 'b'}
At run time, this corresponds roughly to calling this function:
def make_lowercase_set(iterable):
result = set()
for x in iterable:
result.add(x.lower())
return result
make_lowercase_set(['a', 'A', 'b', 'C'])
Dictionary comprehensions start with a colon-separated key-value pair instead of an expression. For example:
>>> {func.__name__: func for func in [print, hex, any]}
{'print': ,
'hex': ,
'any': }
At run time, this corresponds roughly to:
def make_dict_mapping_names_to_functions(iterable):
result = {}
for func in iterable:
result[func.__name__] = func
return result
make_dict_mapping_names_to_functions([print, hex, any])
As in other kinds of dictionary displays, the same key may be specified multiple times. Earlier values are overwritten by ones that are evaluated later.
There are no *tuple comprehensions*. A similar syntax is instead used for `generator expressions `, from which you can construct a tuple like this:
>>> tuple(x**2 for x in range(10))
(0, 1, 4, 9, 16, 25, 36, 49, 64, 81)
3.8 Prior to Python 3.8, in dict comprehensions, the evaluation order of key and value was not well-defined. In CPython, the value was evaluated before the key. Starting with 3.8, the key is evaluated before the value, as proposed by `572`.
single: if; in comprehensions
#### Filtering in comprehensions
The `!for` clause may be followed by an `!if` clause with an expression.
For example, a list of names from the `math` module that start with `f` is:
>>> [name for name in vars(math) if name.startswith('f')]
['fabs', 'factorial', 'floor', 'fma', 'fmod', 'frexp', 'fsum']
At run time, the expression after `!if` is evaluated before each element is added to the resulting container, and if it is false, the element is skipped. Thus, the above example roughly corresponds to defining and calling the following function:
def get_math_f_names(iterable):
result = []
for name in iterable:
if name.startswith('f'):
result.append(name)
return result
get_math_f_names(vars(math))
Filtering is a special case of more complex comprehensions. See the next section for a more formal description.
#### Complex comprehensions
Generally, a comprehension's initial `!for` clause may be followed by zero or more additional `!for` or `!if` clauses. For example, here is a list of names exposed by two Python modules, filtered to only include names that start with `a`:
>>> import array
>>> import math
>>> [
... name
... for module in [array, math]
... for name in vars(module)
... if name.startswith('a')
... ]
['array', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh']
At run time, this roughly corresponds to defining and calling:
def get_a_names(iterable):
result = []
for module in iterable:
for name in vars(module):
if name.startswith('a'):
result.append(name)
return result
get_a_names([array, math])
The elements of the new container are those that would be produced by considering each of the `!for` or `!if` clauses a block, nesting from left to right, and evaluating the expression to produce an element (or dictionary entry) each time the innermost block is reached.
Aside from the iterable expression in the leftmost `!for` clause, the comprehension is executed in a separate implicitly nested scope. This ensures that names assigned to in the target list don't "leak" into the enclosing scope. For example:
>>> x = 'old value'
>>> [x**2 for x in range(10)] # this `x` is local to the comprehension
>>> x
'old value'
The iterable expression in the leftmost `!for` clause is evaluated directly in the enclosing scope and then passed as an argument to the implicitly nested scope.
Subsequent `!for` clauses and any filter condition in the leftmost `!for` clause cannot be evaluated in the enclosing scope as they may depend on the values obtained from the leftmost iterable.
To ensure the comprehension always results in a container of the appropriate type, `yield` and `yield from` expressions are prohibited in the implicitly nested scope.
`Assignment expressions ` are not allowed inside comprehension iterable expressions (that is, the expressions after the `!in` keyword), nor anywhere within comprehensions that appear directly in a class definition.
3.8 `yield` and `yield from` prohibited in the implicitly nested scope.
#### Unpacking in comprehensions
If the expression of a list or set comprehension is starred, the result will be `unpacked ` to produce zero or more elements.
This is often used for "flattening" lists, for example:
>>> students = ['Petr', 'Blaise', 'Jarka']
>>> teachers = ['Salim', 'Bartosz']
>>> lists_of_people = [students, teachers]
>>> [*people for people in lists_of_people]
['Petr', 'Blaise', 'Jarka', 'Salim', 'Bartosz']
At run time, this comprehension roughly corresponds to:
def flatten_names(lists_of_people):
result = []
for people in lists_of_people:
result.extend(people)
return result
In dict comprehensions, a double-starred expression will be evaluated and then unpacked using `dictionary unpacking `, inserting zero or more key/value pairs into the new dictionary. As in other kinds of dictionary displays, if the same key is specified multiple times, the associated value in the resulting dictionary will be the last one specified.
For example:
>>> system_defaults = {'color': 'blue', 'count': 8}
>>> user_defaults = {'color': 'yellow'}
>>> overrides = {'count': 5}
>>> configuration_sets = [system_defaults, user_defaults, overrides]
>>> {**d for d in configuration_sets}
{'color': 'yellow', 'count': 5}
3.15
Unpacking in comprehensions using the `*` and `**` operators was introduced in `798`.
single: async for; in comprehensions single: await; in comprehensions
#### Asynchronous comprehensions
In an `async def` function, an `!async for` clause may be used to iterate over a `asynchronous iterator`. A comprehension in an `!async def` function may consist of either a `!for` or `!async for` clause following the leading expression, may contain additional `!for` or `!async for` clauses, and may also use `await` expressions.
If a comprehension contains `!async for` clauses, or if it contains `!await` expressions or other asynchronous comprehensions anywhere except the iterable expression in the leftmost `!for` clause, it is called an `asynchronous comprehension`. An asynchronous comprehension may suspend the execution of the coroutine function in which it appears.
3.6
Asynchronous comprehensions were introduced in `530`.
3.11 Asynchronous comprehensions are now allowed inside comprehensions in asynchronous functions. Outer comprehensions implicitly become asynchronous.
#### Formal grammar for comprehensions
The formal grammar for comprehensions is:
The syntax for `generator expressions` is the same as for list `comprehensions `, except that they are enclosed in parentheses instead of brackets. For example:
>>> iterator = (x ** 2 for x in range(10))
>>> iterator
at ...>
At runtime, a generator expression evaluates to a `generator iterator` which yields the same values as the corresponding list comprehension:
>>> list(iterator)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Thus, the example above is roughly equivalent to defining and calling the following generator function:
def make_generator_of_squares(iterator):
for x in iterator:
yield x ** 2
make_generator_of_squares(iter(range(10)))
The enclosing parentheses can be omitted in calls when the generator expression is the only positional argument and there are no keyword arguments. See the `Calls section ` for details. For example:
# The parentheses after `sum` are part of the call syntax:
>>> sum(x ** 2 for x in range(10))
285
# The generator needs its own parentheses if it's not the only argument:
>>> sum((x ** 2 for x in range(10)), start=1000)
1285
The iterable expression in the leftmost `!for` clause is evaluated immediately, so that an error raised by this expression will be emitted at the point where the generator expression is defined, rather than at the point where the first value is retrieved:
>>> (x ** 2 for x in nonexistent_iterable)
Traceback (most recent call last):
...
NameError: name 'nonexistent_iterable' is not defined
After the expression is evaluated, an iterator is created from the result, as if `iter` was called on it. Any error raised when creating the iterator is also emitted immediately:
>>> (x ** 2 for x in None)
Traceback (most recent call last):
...
TypeError: 'NoneType' object is not iterable
All other expressions are evaluated lazily, in the same fashion as normal generators (that is, when the iterator is asked to yield a value):
>>> iterator = (nonexistent_value for x in range(10))
>>> iterator
at ...>
>>> list(iterator)
Traceback (most recent call last):
...
NameError: name 'nonexistent_value' is not defined
>>> iterator = (x * y for x in range(10) for y in nonexistent_iterable)
>>> iterator
at ...>
>>> list(iterator)
Traceback (most recent call last):
...
NameError: name 'nonexistent_iterable' is not defined
To avoid interfering with the expected operation of the generator expression itself, `yield` and `yield from` expressions are prohibited inside the implicitly nested scope.
If a generator expression contains either `!async for` clauses or `await` expressions it is called an `asynchronous generator expression`. An asynchronous generator expression returns a new asynchronous generator object, which is an asynchronous iterator (see `async-iterators`).
The formal grammar for generator expressions is:
generator_expression: "(" comprehension ")"
3.6 Asynchronous generator expressions were introduced.
3.7 Prior to Python 3.7, asynchronous generator expressions could only appear in `async def` coroutines. Starting with 3.7, any function can use asynchronous generator expressions.
3.8 `yield` and `yield from` prohibited in the implicitly nested scope.
### Yield expressions
pair: keyword; yield pair: keyword; from pair: yield; expression pair: generator; function
The yield expression is used when defining a `generator` function or an `asynchronous generator` function and thus can only be used in the body of a function definition. Using a yield expression in a function's body causes that function to be a generator function, and using it in an `async def` function's body causes that coroutine function to be an asynchronous generator function. For example:
def gen(): # defines a generator function
yield 123
async def agen(): # defines an asynchronous generator function
yield 123
Due to their side effects on the containing scope, `yield` expressions are not permitted as part of the implicitly defined scopes used to implement comprehensions and generator expressions.
3.8 Yield expressions prohibited in the implicitly nested scopes used to implement comprehensions and generator expressions.
Generator functions are described below, while asynchronous generator functions are described separately in section `asynchronous-generator-functions`.
When a generator function is called, it returns an iterator known as a generator. That generator then controls the execution of the generator function. The execution starts when one of the generator's methods is called. At that time, the execution proceeds to the first yield expression, where it is suspended again, returning the value of `~python-grammar:yield_list` to the generator's caller, or `None` if `~python-grammar:yield_list` is omitted. By suspended, we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, the internal evaluation stack, and the state of any exception handling. When the execution is resumed by calling one of the generator's methods, the function can proceed exactly as if the yield expression were just another external call. The value of the yield expression after resuming depends on the method which resumed the execution. If `~generator.__next__` is used (typically via either a `for` or the `next` builtin) then the result is `None`. Otherwise, if `~generator.send` is used, then the result will be the value passed in to that method.
single: coroutine
All of this makes generator functions quite similar to coroutines; they yield multiple times, they have more than one entry point and their execution can be suspended. The only difference is that a generator function cannot control where the execution should continue after it yields; the control is always transferred to the generator's caller.
Yield expressions are allowed anywhere in a `try` construct. If the generator is not resumed before it is finalized (by reaching a zero reference count or by being garbage collected), the generator-iterator's `~generator.close` method will be called, allowing any pending `finally` clauses to execute.
single: from; yield from expression
When `yield from ` is used, the supplied expression must be an iterable. The values produced by iterating that iterable are passed directly to the caller of the current generator's methods. Any values passed in with `~generator.send` and any exceptions passed in with `~generator.throw` are passed to the underlying iterator if it has the appropriate methods. If this is not the case, then `~generator.send` will raise `AttributeError` or `TypeError`, while `~generator.throw` will just raise the passed in exception immediately.
When the underlying iterator is complete, the `~StopIteration.value` attribute of the raised `StopIteration` instance becomes the value of the yield expression. It can be either set explicitly when raising `StopIteration`, or automatically when the subiterator is a generator (by returning a value from the subgenerator).
3.3 Added `yield from ` to delegate control flow to a subiterator.
The parentheses may be omitted when the yield expression is the sole expression on the right hand side of an assignment statement.
`255` - Simple Generators
The proposal for adding generators and the `yield` statement to Python.
`342` - Coroutines via Enhanced Generators
The proposal to enhance the API and syntax of generators, making them usable as simple coroutines.
`380` - Syntax for Delegating to a Subgenerator
The proposal to introduce the `~python-grammar:yield_from` syntax, making delegation to subgenerators easy.
`525` - Asynchronous Generators
The proposal that expanded on `492` by adding generator capabilities to coroutine functions.
single: yield; examples
#### Examples
Here is a simple example that demonstrates the behavior of generators and generator functions:
>>> def echo(value=None):
... print("Execution starts when 'next()' is called for the first time.")
... try:
... while True:
... try:
... value = (yield value)
... except Exception as e:
... value = e
... finally:
... print("Don't forget to clean up when 'close()' is called.")
...
>>> generator = echo(1)
>>> print(next(generator))
Execution starts when 'next()' is called for the first time.
1
>>> print(next(generator))
None
>>> print(generator.send(2))
2
>>> generator.throw(TypeError, "spam")
TypeError('spam',)
>>> generator.close()
Don't forget to clean up when 'close()' is called.
For examples using `yield from`, see `pep-380` in "What's New in Python."
#### Asynchronous generator functions
The presence of a yield expression in a function or method defined using `async def` further defines the function as an `asynchronous generator` function.
When an asynchronous generator function is called, it returns an asynchronous iterator known as an asynchronous generator object. That object then controls the execution of the generator function. An asynchronous generator object is typically used in an `async for` statement in a coroutine function analogously to how a generator object would be used in a `for` statement.
Calling one of the asynchronous generator's methods returns an `awaitable` object, and the execution starts when this object is awaited on. At that time, the execution proceeds to the first yield expression, where it is suspended again, returning the value of `~python-grammar:yield_list` to the awaiting coroutine. As with a generator, suspension means that all local state is retained, including the current bindings of local variables, the instruction pointer, the internal evaluation stack, and the state of any exception handling. When the execution is resumed by awaiting on the next object returned by the asynchronous generator's methods, the function can proceed exactly as if the yield expression were just another external call. The value of the yield expression after resuming depends on the method which resumed the execution. If `~agen.__anext__` is used then the result is `None`. Otherwise, if `~agen.asend` is used, then the result will be the value passed in to that method.
If an asynchronous generator happens to exit early by `break`, the caller task being cancelled, or other exceptions, the generator's async cleanup code will run and possibly raise exceptions or access context variables in an unexpected context--perhaps after the lifetime of tasks it depends, or during the event loop shutdown when the async-generator garbage collection hook is called. To prevent this, the caller must explicitly close the async generator by calling `~agen.aclose` method to finalize the generator and ultimately detach it from the event loop.
In an asynchronous generator function, yield expressions are allowed anywhere in a `try` construct. However, if an asynchronous generator is not resumed before it is finalized (by reaching a zero reference count or by being garbage collected), then a yield expression within a `!try` construct could result in a failure to execute pending `finally` clauses. In this case, it is the responsibility of the event loop or scheduler running the asynchronous generator to call the asynchronous generator-iterator's `~agen.aclose` method and run the resulting coroutine object, thus allowing any pending `!finally` clauses to execute.
To take care of finalization upon event loop termination, an event loop should define a *finalizer* function which takes an asynchronous generator-iterator and presumably calls `~agen.aclose` and executes the coroutine. This *finalizer* may be registered by calling `sys.set_asyncgen_hooks`. When first iterated over, an asynchronous generator-iterator will store the registered *finalizer* to be called upon finalization. For a reference example of a *finalizer* method see the implementation of `asyncio.Loop.shutdown_asyncgens` in `Lib/asyncio/base_events.py`.
The expression `yield from ` is a syntax error when used in an asynchronous generator function.
## Primaries
single: primary
Primaries represent the most tightly bound operations of the language. Their syntax is:
python-grammar primary: atom \| attributeref \| subscription \| call
pair: exception; AttributeError pair: object; module pair: object; list
The primary must evaluate to an object of a type that supports attribute references, which most objects do. This object is then asked to produce the attribute whose name is the identifier. The type and value produced is determined by the object. Multiple evaluations of the same attribute reference may yield different objects.
This production can be customized by overriding the `~object.__getattribute__` method or the `~object.__getattr__` method. The `!__getattribute__` method is called first and either returns a value or raises `AttributeError` if the attribute is not available.
If an `AttributeError` is raised and the object has a `!__getattr__` method, that method is called as a fallback.
### Subscriptions and slicings
The `subscription` syntax is usually used for selecting an element from a `container ` -- for example, to get a value from a `dict`:
>>> digits_by_name = {'one': 1, 'two': 2}
>>> digits_by_name['two'] # Subscripting a dictionary using the key 'two'
2
In the subscription syntax, the object being subscribed -- a `primary ` -- is followed by a `subscript` in square brackets. In the simplest case, the subscript is a single expression.
Depending on the type of the object being subscribed, the subscript is sometimes called a `key` (for mappings), `index` (for sequences), or *type argument* (for `generic types `). Syntactically, these are all equivalent:
>>> colors = ['red', 'blue', 'green', 'black']
>>> colors[3] # Subscripting a list using the index 3
'black'
>>> list[str] # Parameterizing the list type using the type argument str
list[str]
At runtime, the interpreter will evaluate the primary and the subscript, and call the primary's `~object.__getitem__` or `~object.__class_getitem__` `special method` with the subscript as argument. For more details on which of these methods is called, see `classgetitem-versus-getitem`.
To show how subscription works, we can define a custom object that implements `~object.__getitem__` and prints out the value of the subscript:
>>> class SubscriptionDemo:
... def __getitem__(self, key):
... print(f'subscripted with: {key!r}')
...
>>> demo = SubscriptionDemo()
>>> demo[1]
subscripted with: 1
>>> demo['a' * 3]
subscripted with: 'aaa'
See `~object.__getitem__` documentation for how built-in types handle subscription.
Subscriptions may also be used as targets in `assignment ` or `deletion ` statements. In these cases, the interpreter will call the subscripted object's `~object.__setitem__` or `~object.__delitem__` `special method`, respectively, instead of `~object.__getitem__`.
>>> colors = ['red', 'blue', 'green', 'black']
>>> colors[3] = 'white' # Setting item at index
>>> colors
['red', 'blue', 'green', 'white']
>>> del colors[3] # Deleting item at index 3
>>> colors
['red', 'blue', 'green']
All advanced forms of *subscript* documented in the following sections are also usable for assignment and deletion.
#### Slicings
A more advanced form of subscription, `slicing`, is commonly used to extract a portion of a `sequence `. In this form, the subscript is a `slice`: up to three expressions separated by colons. Any of the expressions may be omitted, but a slice must contain at least one colon:
>>> number_names = ['zero', 'one', 'two', 'three', 'four', 'five']
>>> number_names[1:3]
['one', 'two']
>>> number_names[1:]
['one', 'two', 'three', 'four', 'five']
>>> number_names[:3]
['zero', 'one', 'two']
>>> number_names[:]
['zero', 'one', 'two', 'three', 'four', 'five']
>>> number_names[::2]
['zero', 'two', 'four']
>>> number_names[:-3]
['zero', 'one', 'two']
>>> del number_names[4:]
>>> number_names
['zero', 'one', 'two', 'three']
When a slice is evaluated, the interpreter constructs a `slice` object whose `~slice.start`, `~slice.stop` and `~slice.step` attributes, respectively, are the results of the expressions between the colons. Any missing expression evaluates to `None`. This `!slice` object is then passed to the `~object.__getitem__` or `~object.__class_getitem__` `special method`, as above. :
# continuing with the SubscriptionDemo instance defined above:
>>> demo[2:3]
subscripted with: slice(2, 3, None)
>>> demo[::'spam']
subscripted with: slice(None, None, 'spam')
#### Comma-separated subscripts
The subscript can also be given as two or more comma-separated expressions or slices:
# continuing with the SubscriptionDemo instance defined above:
>>> demo[1, 2, 3]
subscripted with: (1, 2, 3)
>>> demo[1:2, 3]
subscripted with: (slice(1, 2, None), 3)
This form is commonly used with numerical libraries for slicing multi-dimensional data. In this case, the interpreter constructs a `tuple` of the results of the expressions or slices, and passes this tuple to the `~object.__getitem__` or `~object.__class_getitem__` `special method`, as above.
The subscript may also be given as a single expression or slice followed by a comma, to specify a one-element tuple:
>>> demo['spam',]
subscripted with: ('spam',)
#### "Starred" subscriptions
3.11 Expressions in *tuple_slices* may be starred. See `646`.
The subscript can also contain a starred expression. In this case, the interpreter unpacks the result into a tuple, and passes this tuple to `~object.__getitem__` or `~object.__class_getitem__`:
# continuing with the SubscriptionDemo instance defined above:
>>> demo[*range(10)]
subscripted with: (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
Starred expressions may be combined with comma-separated expressions and slices:
>>> demo['a', 'b', *range(3), 'c']
subscripted with: ('a', 'b', 0, 1, 2, 'c')
#### Formal subscription grammar
Recall that the `|` operator `denotes ordered choice `. Specifically, in `!subscript`, if both alternatives would match, the first (`!single_subscript`) has priority.
pair: object; callable single: call single: argument; call semantics single: () (parentheses); call single: , (comma); argument list single: = (equals); in function calls
### Calls
A call calls a callable object (e.g., a `function`) with a possibly empty series of `arguments `:
An optional trailing comma may be present after the positional and keyword arguments but does not affect the semantics.
single: parameter; call semantics
The primary must evaluate to a callable object (user-defined functions, built-in functions, methods of built-in objects, class objects, methods of class instances, and all objects having a `~object.__call__` method are callable). All argument expressions are evaluated before the call is attempted. Please refer to section `function` for the syntax of formal `parameter` lists.
If keyword arguments are present, they are first converted to positional arguments, as follows. First, a list of unfilled slots is created for the formal parameters. If there are N positional arguments, they are placed in the first N slots. Next, for each keyword argument, the identifier is used to determine the corresponding slot (if the identifier is the same as the first formal parameter name, the first slot is used, and so on). If the slot is already filled, a `TypeError` exception is raised. Otherwise, the argument is placed in the slot, filling it (even if the expression is `None`, it fills the slot). When all arguments have been processed, the slots that are still unfilled are filled with the corresponding default value from the function definition. (Default values are calculated, once, when the function is defined; thus, a mutable object such as a list or dictionary used as default value will be shared by all calls that don't specify an argument value for the corresponding slot; this should usually be avoided.) If there are any unfilled slots for which no default value is specified, a `TypeError` exception is raised. Otherwise, the list of filled slots is used as the argument list for the call.
An implementation may provide built-in functions whose positional parameters do not have names, even if they are 'named' for the purpose of documentation, and which therefore cannot be supplied by keyword. In CPython, this is the case for functions implemented in C that use `PyArg_ParseTuple` to parse their arguments.
If there are more positional arguments than there are formal parameter slots, a `TypeError` exception is raised, unless a formal parameter using the syntax `*identifier` is present; in this case, that formal parameter receives a tuple containing the excess positional arguments (or an empty tuple if there were no excess positional arguments).
If any keyword argument does not correspond to a formal parameter name, a `TypeError` exception is raised, unless a formal parameter using the syntax `**identifier` is present; in this case, that formal parameter receives a dictionary containing the excess keyword arguments (using the keywords as keys and the argument values as corresponding values), or a (new) empty dictionary if there were no excess keyword arguments.
single: \* (asterisk); in function calls single: unpacking; in function calls
If the syntax `*expression` appears in the function call, `expression` must evaluate to an `iterable`. Elements from these iterables are treated as if they were additional positional arguments. For the call `f(x1, x2, *y, x3, x4)`, if *y* evaluates to a sequence *y1*, ..., *yM*, this is equivalent to a call with M+4 positional arguments *x1*, *x2*, *y1*, ..., *yM*, *x3*, *x4*.
A consequence of this is that although the `*expression` syntax may appear *after* explicit keyword arguments, it is processed *before* the keyword arguments (and any `**expression` arguments -- see below). So:
>>> def f(a, b):
... print(a, b)
...
>>> f(b=1, *(2,))
2 1
>>> f(a=1, *(2,))
Traceback (most recent call last):
File "", line 1, in
TypeError: f() got multiple values for keyword argument 'a'
>>> f(1, *(2,))
1 2
It is unusual for both keyword arguments and the `*expression` syntax to be used in the same call, so in practice this confusion does not often arise.
single: \*\*; in function calls
If the syntax `**expression` appears in the function call, `expression` must evaluate to a `mapping`, the contents of which are treated as additional keyword arguments. If a parameter matching a key has already been given a value (by an explicit keyword argument, or from another unpacking), a `TypeError` exception is raised.
When `**expression` is used, each key in this mapping must be a string. Each value from the mapping is assigned to the first formal parameter eligible for keyword assignment whose name is equal to the key. A key need not be a Python identifier (e.g. `"max-temp °F"` is acceptable, although it will not match any formal parameter that could be declared). If there is no match to a formal parameter the key-value pair is collected by the `**` parameter, if there is one, or if there is not, a `TypeError` exception is raised.
Formal parameters using the syntax `*identifier` or `**identifier` cannot be used as positional argument slots or as keyword argument names.
3.5 Function calls accept any number of `*` and `**` unpackings, positional arguments may follow iterable unpackings (`*`), and keyword arguments may follow dictionary unpackings (`**`). Originally proposed by `448`.
A call always returns some value, possibly `None`, unless it raises an exception. How this value is computed depends on the type of the callable object.
If it is---
a user-defined function:
pair: function; call triple: user-defined; function; call pair: object; user-defined function pair: object; function
The code block for the function is executed, passing it the argument list. The first thing the code block will do is bind the formal parameters to the arguments; this is described in section `function`. When the code block executes a `return` statement, this specifies the return value of the function call. If execution reaches the end of the code block without executing a `return` statement, the return value is `None`.
a built-in function or method:
The result is up to the interpreter; see `built-in-funcs` for the descriptions of built-in functions and methods.
a class object:
pair: object; class pair: class object; call
A new instance of that class is returned.
a class instance method:
pair: object; class instance pair: object; instance pair: class instance; call
The corresponding user-defined function is called, with an argument list that is one longer than the argument list of the call: the instance becomes the first argument.
a class instance:
Thus, in an unparenthesized sequence of power and unary operators, the operators are evaluated from right to left (this does not constrain the evaluation order for the operands): `-1**2` results in `-1`.
The power operator has the same semantics as the built-in `pow` function, when called with two arguments: it yields its left argument raised to the power of its right argument. Numeric arguments are first `converted to a common type `, and the result is of that type.
For int operands, the result has the same type as the operands unless the second argument is negative; in that case, all arguments are converted to float and a float result is delivered. For example, `10**2` returns `100`, but `10**-2` returns `0.01`.
Raising `0.0` to a negative power results in a `ZeroDivisionError`. Raising a negative number to a fractional power results in a `complex` number. (In earlier versions it raised a `ValueError`.)
This operation can be customized using the special `~object.__pow__` and `~object.__rpow__` methods.
## Unary arithmetic and bitwise operations
The unary `+` (plus) operator yields its numeric argument unchanged; the operation can be overridden with the `~object.__pos__` special method.
single: inversion pair: operator; ~ (tilde)
The unary `~` (invert) operator yields the bitwise inversion of its integer argument. The bitwise inversion of `x` is defined as `-(x+1)`. It only applies to integral numbers or to custom objects that override the `~object.__invert__` special method.
pair: exception; TypeError
In all three cases, if the argument does not have the proper type, a `TypeError` exception is raised.
## Binary arithmetic operations
triple: binary; arithmetic; operation
The binary arithmetic operations have the conventional priority levels. Note that some of these operations also apply to certain non-numeric types. Apart from the power operator, there are only two levels, one for multiplicative operators and one for additive operators:
The `*` (multiplication) operator yields the product of its arguments. The arguments must either both be numbers, or one argument must be an integer and the other must be a sequence. In the former case, the numbers are `converted to a common real type ` and then multiplied together. In the latter case, sequence repetition is performed; a negative repetition factor yields an empty sequence.
This operation can be customized using the special `~object.__mul__` and `~object.__rmul__` methods.
3.14 If only one operand is a complex number, the other operand is converted to a floating-point number.
The `@` (at) operator is intended to be used for matrix multiplication. No builtin Python types implement this operator.
This operation can be customized using the special `~object.__matmul__` and `~object.__rmatmul__` methods.
The `/` (division) and `//` (floor division) operators yield the quotient of their arguments. The numeric arguments are first `converted to a common type `. Division of integers yields a float, while floor division of integers results in an integer; the result is that of mathematical division with the 'floor' function applied to the result. Division by zero raises the `ZeroDivisionError` exception.
The division operation can be customized using the special `~object.__truediv__` and `~object.__rtruediv__` methods. The floor division operation can be customized using the special `~object.__floordiv__` and `~object.__rfloordiv__` methods.
single: modulo pair: operator; % (percent)
The `%` (modulo) operator yields the remainder from the division of the first argument by the second. The numeric arguments are first `converted to a common type `. A zero right argument raises the `ZeroDivisionError` exception. The arguments may be floating-point numbers, e.g., `3.14%0.7` equals `0.34` (since `3.14` equals `4*0.7 + 0.34`.) The modulo operator always yields a result with the same sign as its second operand (or zero); the absolute value of the result is strictly smaller than the absolute value of the second operand [^1].
The floor division and modulo operators are connected by the following identity: `x == (x//y)*y + (x%y)`. Floor division and modulo are also connected with the built-in function `divmod`: `divmod(x, y) == (x//y, x%y)`.[^2].
In addition to performing the modulo operation on numbers, the `%` operator is also overloaded by string objects to perform old-style string formatting (also known as interpolation). The syntax for string formatting is described in the Python Library Reference, section `old-string-formatting`.
The *modulo* operation can be customized using the special `~object.__mod__` and `~object.__rmod__` methods.
The floor division operator, the modulo operator, and the `divmod` function are not defined for complex numbers. Instead, convert to a floating-point number using the `abs` function if appropriate.
The `+` (addition) operator yields the sum of its arguments. The arguments must either both be numbers or both be sequences of the same type. In the former case, the numbers are `converted to a common real type ` and then added together. In the latter case, the sequences are concatenated.
This operation can be customized using the special `~object.__add__` and `~object.__radd__` methods.
3.14 If only one operand is a complex number, the other operand is converted to a floating-point number.
The `-` (subtraction) operator yields the difference of its arguments. The numeric arguments are first `converted to a common real type `.
This operation can be customized using the special `~object.__sub__` and `~object.__rsub__` methods.
3.14 If only one operand is a complex number, the other operand is converted to a floating-point number.
These operators accept integers as arguments. They shift the first argument to the left or right by the number of bits given by the second argument.
The left shift operation can be customized using the special `~object.__lshift__` and `~object.__rlshift__` methods. The right shift operation can be customized using the special `~object.__rshift__` and `~object.__rrshift__` methods.
pair: exception; ValueError
A right shift by *n* bits is defined as floor division by `pow(2,n)`. A left shift by *n* bits is defined as multiplication with `pow(2,n)`.
## Binary bitwise operations
triple: binary; bitwise; operation
Each of the three bitwise operations has a different priority level:
The `&` operator yields the bitwise AND of its arguments, which must be integers or one of them must be a custom object overriding `~object.__and__` or `~object.__rand__` special methods.
pair: bitwise; xor pair: exclusive; or pair: operator; ^ (caret)
The `^` operator yields the bitwise XOR (exclusive OR) of its arguments, which must be integers or one of them must be a custom object overriding `~object.__xor__` or `~object.__rxor__` special methods.
pair: bitwise; or pair: inclusive; or pair: operator; \| (vertical bar)
The `|` operator yields the bitwise (inclusive) OR of its arguments, which must be integers or one of them must be a custom object overriding `~object.__or__` or `~object.__ror__` special methods.
## Comparisons
Unlike C, all comparison operations in Python have the same priority, which is lower than that of any arithmetic, shifting or bitwise operation. Also unlike C, expressions like `a < b < c` have the interpretation that is conventional in mathematics:
Comparisons yield boolean values: `True` or `False`. Custom `rich comparison methods` may return non-boolean values. In this case Python will call `bool` on such value in boolean contexts.
pair: chaining; comparisons
Comparisons can be chained arbitrarily, e.g., `x < y <= z` is equivalent to `x < y and y <= z`, except that `y` is evaluated only once (but in both cases `z` is not evaluated at all when `x < y` is found to be false).
Formally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and *op1*, *op2*, ..., *opN* are comparison operators, then `a op1 b op2 c ... y opN z` is equivalent to `a op1 b and b op2 c and ... y opN z`, except that each expression is evaluated at most once.
Note that `a op1 b op2 c` doesn't imply any kind of comparison between *a* and *c*, so that, e.g., `x < y > z` is perfectly legal (though perhaps not pretty).
### Value comparisons
The operators `<`, `>`, `==`, `>=`, `<=`, and `!=` compare the values of two objects. The objects do not need to have the same type.
Chapter `objects` states that objects have a value (in addition to type and identity). The value of an object is a rather abstract notion in Python: For example, there is no canonical access method for an object's value. Also, there is no requirement that the value of an object should be constructed in a particular way, e.g. comprised of all its data attributes. Comparison operators implement a particular notion of what the value of an object is. One can think of them as defining the value of an object indirectly, by means of their comparison implementation.
Because all types are (direct or indirect) subtypes of `object`, they inherit the default comparison behavior from `object`. Types can customize their comparison behavior by implementing `rich comparison methods` like `~object.__lt__`, described in `customization`.
The default behavior for equality comparison (`==` and `!=`) is based on the identity of the objects. Hence, equality comparison of instances with the same identity results in equality, and equality comparison of instances with different identities results in inequality. A motivation for this default behavior is the desire that all objects should be reflexive (i.e. `x is y` implies `x == y`).
A default order comparison (`<`, `>`, `<=`, and `>=`) is not provided; an attempt raises `TypeError`. A motivation for this default behavior is the lack of a similar invariant as for equality.
The behavior of the default equality comparison, that instances with different identities are always unequal, may be in contrast to what types will need that have a sensible definition of object value and value-based equality. Such types will need to customize their comparison behavior, and in fact, a number of built-in types have done that.
The following list describes the comparison behavior of the most important built-in types.
- Numbers of built-in numeric types (`typesnumeric`) and of the standard library types `fractions.Fraction` and `decimal.Decimal` can be compared within and across their types, with the restriction that complex numbers do not support order comparison. Within the limits of the types involved, they compare mathematically (algorithmically) correct without loss of precision.
The not-a-number values `float('NaN')` and `decimal.Decimal('NaN')` are special. Any ordered comparison of a number to a not-a-number value is false. A counter-intuitive implication is that not-a-number values are not equal to themselves. For example, if `x = float('NaN')`, `3 < x`, `x < 3` and `x == x` are all false, while `x != x` is true. This behavior is compliant with IEEE 754.
- `None` and `NotImplemented` are singletons. [PEP 8](http://www.python.org/dev/peps/pep-0008/ "PEP 8") advises that comparisons for singletons should always be done with `is` or `is not`, never the equality operators.
- Binary sequences (instances of `bytes` or `bytearray`) can be compared within and across their types. They compare lexicographically using the numeric values of their elements.
- Strings (instances of `str`) compare lexicographically using the numerical Unicode code points (the result of the built-in function `ord`) of their characters.[^3]
Strings and binary sequences cannot be directly compared.
- Sequences (instances of `tuple`, `list`, or `range`) can be compared only within each of their types, with the restriction that ranges do not support order comparison. Equality comparison across these types results in inequality, and ordering comparison across these types raises `TypeError`.
Sequences compare lexicographically using comparison of corresponding elements. The built-in containers typically assume identical objects are equal to themselves. That lets them bypass equality tests for identical objects to improve performance and to maintain their internal invariants.
Lexicographical comparison between built-in collections works as follows:
- For two collections to compare equal, they must be of the same type, have the same length, and each pair of corresponding elements must compare equal (for example, `[1,2] == (1,2)` is false because the type is not the same).
- Collections that support order comparison are ordered the same as their first unequal elements (for example, `[1,2,x] <= [1,2,y]` has the same value as `x <= y`). If a corresponding element does not exist, the shorter collection is ordered first (for example, `[1,2] < [1,2,3]` is true).
- Mappings (instances of `dict`) compare equal if and only if they have equal `(key, value)` pairs. Equality comparison of the keys and values enforces reflexivity.
Order comparisons (`<`, `>`, `<=`, and `>=`) raise `TypeError`.
- Sets (instances of `set` or `frozenset`) can be compared within and across their types.
They define order comparison operators to mean subset and superset tests. Those relations do not define total orderings (for example, the two sets `{1,2}` and `{2,3}` are not equal, nor subsets of one another, nor supersets of one another). Accordingly, sets are not appropriate arguments for functions which depend on total ordering (for example, `min`, `max`, and `sorted` produce undefined results given a list of sets as inputs).
Comparison of sets enforces reflexivity of its elements.
- Most other built-in types have no comparison methods implemented, so they inherit the default comparison behavior.
User-defined classes that customize their comparison behavior should follow some consistency rules, if possible:
- Equality comparison should be reflexive. In other words, identical objects should compare equal:
> `x is y` implies `x == y`
- Comparison should be symmetric. In other words, the following expressions should have the same result:
> `x == y` and `y == x`
>
> `x != y` and `y != x`
>
> `x < y` and `y > x`
>
> `x <= y` and `y >= x`
- Comparison should be transitive. The following (non-exhaustive) examples illustrate that:
> `x > y and y > z` implies `x > z`
>
> `x < y and y <= z` implies `x < z`
- Inverse comparison should result in the boolean negation. In other words, the following expressions should have the same result:
> `x == y` and `not x != y`
>
> `x < y` and `not x >= y` (for total ordering)
>
> `x > y` and `not x <= y` (for total ordering)
The last two expressions apply to totally ordered collections (e.g. to sequences, but not to sets or mappings). See also the `~functools.total_ordering` decorator.
- The `hash` result should be consistent with equality. Objects that are equal should either have the same hash value, or be marked as unhashable.
Python does not enforce these consistency rules. In fact, the not-a-number values are an example for not following these rules.
### Membership test operations
The operators `in` and `not in` test for membership. `x in s` evaluates to `True` if *x* is a member of *s*, and `False` otherwise. `x not in s` returns the negation of `x in s`. All built-in sequences and set types support this as well as dictionary, for which `!in` tests whether the dictionary has a given key. For container types such as list, tuple, set, frozenset, dict, or collections.deque, the expression `x in y` is equivalent to `any(x is e or x == e for e in y)`.
For the string and bytes types, `x in y` is `True` if and only if *x* is a substring of *y*. An equivalent test is `y.find(x) != -1`. Empty strings are always considered to be a substring of any other string, so `"" in "abc"` will return `True`.
For user-defined classes which define the `~object.__contains__` method, `x in y` returns `True` if `y.__contains__(x)` returns a true value, and `False` otherwise.
For user-defined classes which do not define `~object.__contains__` but do define `~object.__iter__`, `x in y` is `True` if some value `z`, for which the expression `x is z or x == z` is true, is produced while iterating over `y`. If an exception is raised during the iteration, it is as if `in` raised that exception.
Lastly, the old-style iteration protocol is tried: if a class defines `~object.__getitem__`, `x in y` is `True` if and only if there is a non-negative integer index *i* such that `x is y[i] or x == y[i]`, and no lower integer index raises the `IndexError` exception. (If any other exception is raised, it is as if `in` raised that exception).
pair: operator; in pair: operator; not in pair: membership; test pair: object; sequence
The operator `not in` is defined to have the inverse truth value of `in`.
pair: operator; is pair: operator; is not pair: identity; test
### Identity comparisons
The operators `is` and `is not` test for an object's identity: `x is y` is true if and only if *x* and *y* are the same object. An Object's identity is determined using the `id` function. `x is not y` yields the inverse truth value.[^4]
## Boolean operations
In the context of Boolean operations, and also when expressions are used by control flow statements, the following values are interpreted as false: `False`, `None`, zero of any numeric type, and empty strings and containers (including strings, tuples, lists, dictionaries, sets and frozensets). All other values are interpreted as true. User-defined objects can customize their truth value by providing a `~object.__bool__` method.
pair: operator; not
The operator `not` yields `True` if its argument is false, `False` otherwise.
pair: operator; and
The expression `x and y` first evaluates *x*; if *x* is false, its value is returned; otherwise, *y* is evaluated and the resulting value is returned.
pair: operator; or
The expression `x or y` first evaluates *x*; if *x* is true, its value is returned; otherwise, *y* is evaluated and the resulting value is returned.
Note that neither `and` nor `or` restrict the value and type they return to `False` and `True`, but rather return the last evaluated argument. This is sometimes useful, e.g., if `s` is a string that should be replaced by a default value if it is empty, the expression `s or 'foo'` yields the desired value. Because `not` has to create a new value, it returns a boolean value regardless of the type of its argument (for example, `not 'foo'` produces `False` rather than `''`.)
An assignment expression (sometimes also called a "named expression" or "walrus") assigns an `~python-grammar:expression` to an `~python-grammar:identifier`, while also returning the value of the `~python-grammar:expression`.
One common use case is when handling matched regular expressions:
``` python
if matching := pattern.search(data):
do_something(matching)
```
Or, when processing a file stream in chunks:
``` python
while chunk := file.read(9000):
process(chunk)
```
Assignment expressions must be surrounded by parentheses when used as expression statements and when used as sub-expressions in slicing, conditional, lambda, keyword-argument, and comprehension-if expressions and in `assert`, `with`, and `assignment` statements. In all other places where they can be used, parentheses are not required, including in `if` and `while` statements.
3.8 See `572` for more details about assignment expressions.
A conditional expression (sometimes called a "ternary operator") is an alternative to the if-else statement. As it is an expression, it returns a value and can appear as a sub-expression.
The expression `x if C else y` first evaluates the condition, *C* rather than *x*. If *C* is true, *x* is evaluated and its value is returned; otherwise, *y* is evaluated and its value is returned.
See `308` for more details about conditional expressions.
## Lambdas
pair: lambda; expression pair: lambda; form pair: anonymous; function single: : (colon); lambda expression
Lambda expressions (sometimes called lambda forms) are used to create anonymous functions. The expression `lambda parameters: expression` yields a function object. The unnamed object behaves like a function object defined with:
```
def (parameters):
return expression
```
See section `function` for the syntax of parameter lists. Note that functions created with lambda expressions cannot contain statements or annotations.
single: comma single: , (comma)
## Expression lists
pair: expression; list single: , (comma); expression list
Except when part of a list or set display, an expression list containing at least one comma yields a tuple. The length of the tuple is the number of expressions in the list. The expressions are evaluated from left to right.
pair: trailing; comma
A trailing comma is required only to create a one-item tuple, such as `1,`; it is optional in all other cases. A single expression without a trailing comma doesn't create a tuple, but rather yields the value of that expression. (To create an empty tuple, use an empty pair of parentheses: `()`.)
pair: iterable; unpacking single: \* (asterisk); in expression lists
### Iterable unpacking
In an expression list or tuple, list or set display, any expression may be prefixed with an asterisk (`*`). This denotes `iterable unpacking`.
At runtime, the asterisk-prefixed expression must evaluate to an `iterable`. The iterable is expanded into a sequence of items, which are included in the new tuple, list, or set, at the site of the unpacking.
3.5 Iterable unpacking in expression lists, originally proposed by `448`.
3.11 Any item in an expression list may be starred. See `646`.
## Evaluation order
pair: evaluation; order
Python evaluates expressions from left to right. Notice that while evaluating an assignment, the right-hand side is evaluated before the left-hand side.
In the following lines, expressions will be evaluated in the arithmetic order of their suffixes:
expr1, expr2, expr3, expr4
(expr1, expr2, expr3, expr4)
{expr1: expr2, expr3: expr4}
expr1 + expr2 * (expr3 - expr4)
expr1(expr2, expr3, *expr4, **expr5)
expr3, expr4 = expr1, expr2
## Operator precedence
pair: operator; precedence
The following table summarizes the operator precedence in Python, from highest precedence (most binding) to lowest precedence (least binding). Operators in the same box have the same precedence. Unless the syntax is explicitly given, operators are binary. Operators in the same box group left to right (except for exponentiation and conditional expressions, which group from right to left).
Note that comparisons, membership tests, and identity tests, all have the same precedence and have a left-to-right chaining feature as described in the `comparisons` section.
Comparisons, including membership tests and identity tests
not x <not>
Boolean NOT
and
Boolean AND
or
Boolean OR
if <if_expr> -- !else
Conditional expression
lambda
Lambda expression
:=
Assignment expression
The power operator ** binds less tightly than an arithmetic or bitwise unary operator on its right, that is, 2**-1 is 0.5.↩︎
The % operator is also used for string formatting; the same precedence applies.↩︎
**Footnotes**
[^1]: While `abs(x%y) < abs(y)` is true mathematically, for floats it may not be true numerically due to roundoff. For example, and assuming a platform on which a Python float is an IEEE 754 double-precision number, in order that `-1e-100 % 1e100` have the same sign as `1e100`, the computed result is `-1e-100 + 1e100`, which is numerically exactly equal to `1e100`. The function `math.fmod` returns a result whose sign matches the sign of the first argument instead, and so returns `-1e-100` in this case. Which approach is more appropriate depends on the application.
[^2]: If x is very close to an exact integer multiple of y, it's possible for `x//y` to be one larger than `(x-x%y)//y` due to rounding. In such cases, Python returns the latter result, in order to preserve that `divmod(x,y)[0] * y + x % y` be very close to `x`.
[^3]: The Unicode standard distinguishes between `code points` (e.g. U+0041) and `abstract characters` (e.g. "LATIN CAPITAL LETTER A"). While most abstract characters in Unicode are only represented using one code point, there is a number of abstract characters that can in addition be represented using a sequence of more than one code point. For example, the abstract character "LATIN CAPITAL LETTER C WITH CEDILLA" can be represented as a single `precomposed character` at code position U+00C7, or as a sequence of a `base character` at code position U+0043 (LATIN CAPITAL LETTER C), followed by a `combining character` at code position U+0327 (COMBINING CEDILLA).
The comparison operators on strings compare at the level of Unicode code points. This may be counter-intuitive to humans. For example, `"\u00C7" == "\u0043\u0327"` is `False`, even though both strings represent the same abstract character "LATIN CAPITAL LETTER C WITH CEDILLA".
To compare strings at the level of abstract characters (that is, in a way intuitive to humans), use `unicodedata.normalize`.
[^4]: Due to automatic garbage-collection, free lists, and the dynamic nature of descriptors, you may notice seemingly unusual behaviour in certain uses of the `is` operator, like those involving comparisons between instance methods, or constants. Check their documentation for more info.
# Full Grammar specification
This is the full Python grammar, derived directly from the grammar used to generate the CPython parser (see `Grammar/python.gram`). The version here omits details related to code generation and error recovery.
The notation used here is the same as in the preceding docs, and is described in the `notation ` section, except for an extra complication:
- `~` ("cut"): commit to the current alternative; fail the rule if the alternative fails to parse
Python mainly uses cuts for optimizations or improved error messages. They often appear to be useless in the listing below.
Cuts currently don't appear inside parentheses, brackets, lookaheads and similar. Their behavior in these contexts is deliberately left unspecified.
../../Grammar/python.gram
# The import system
single: import machinery
Python code in one `module` gains access to the code in another module by the process of `importing` it. The `import` statement is the most common way of invoking the import machinery, but it is not the only way. Functions such as `importlib.import_module` and built-in `__import__` can also be used to invoke the import machinery.
The `import` statement combines two operations; it searches for the named module, then it binds the results of that search to a name in the local scope. The search operation of the `!import` statement is defined as a call to the `__import__` function, with the appropriate arguments. The return value of `__import__` is used to perform the name binding operation of the `!import` statement. See the `!import` statement for the exact details of that name binding operation.
A direct call to `__import__` performs only the module search and, if found, the module creation operation. While certain side-effects may occur, such as the importing of parent packages, and the updating of various caches (including `sys.modules`), only the `import` statement performs a name binding operation.
When an `import` statement is executed, the standard builtin `__import__` function is called. Other mechanisms for invoking the import system (such as `importlib.import_module`) may choose to bypass `__import__` and use their own solutions to implement import semantics.
When a module is first imported, Python searches for the module and if found, it creates a module object[^1], initializing it. If the named module cannot be found, a `ModuleNotFoundError` is raised. Python implements various strategies to search for the named module when the import machinery is invoked. These strategies can be modified and extended by using various hooks described in the sections below.
3.3 The import system has been updated to fully implement the second phase of `302`. There is no longer any implicit import machinery - the full import system is exposed through `sys.meta_path`. In addition, native namespace package support has been implemented (see `420`).
## `importlib`
The `importlib` module provides a rich API for interacting with the import system. For example `importlib.import_module` provides a recommended, simpler API than built-in `__import__` for invoking the import machinery. Refer to the `importlib` library documentation for additional detail.
## Packages
single: package
Python has only one type of module object, and all modules are of this type, regardless of whether the module is implemented in Python, C, or something else. To help organize modules and provide a naming hierarchy, Python has a concept of `packages `.
You can think of packages as the directories on a file system and modules as files within directories, but don't take this analogy too literally since packages and modules need not originate from the file system. For the purposes of this documentation, we'll use this convenient analogy of directories and files. Like file system directories, packages are organized hierarchically, and packages may themselves contain subpackages, as well as regular modules.
It's important to keep in mind that all packages are modules, but not all modules are packages. Or put another way, packages are just a special kind of module. Specifically, any module that contains a `__path__` attribute is considered a package.
All modules have a name. Subpackage names are separated from their parent package name by a dot, akin to Python's standard attribute access syntax. Thus you might have a package called `email`, which in turn has a subpackage called `email.mime` and a module within that subpackage called `email.mime.text`.
### Regular packages
pair: package; regular
Python defines two types of packages, `regular packages ` and `namespace packages `. Regular packages are traditional packages as they existed in Python 3.2 and earlier. A regular package is typically implemented as a directory containing an `__init__.py` file. When a regular package is imported, this `__init__.py` file is implicitly executed, and the objects it defines are bound to names in the package's namespace. The `__init__.py` file can contain the same Python code that any other module can contain, and Python will add some additional attributes to the module when it is imported.
For example, the following file system layout defines a top level `parent` package with three subpackages:
parent/
__init__.py
one/
__init__.py
two/
__init__.py
three/
__init__.py
Importing `parent.one` will implicitly execute `parent/__init__.py` and `parent/one/__init__.py`. Subsequent imports of `parent.two` or `parent.three` will execute `parent/two/__init__.py` and `parent/three/__init__.py` respectively.
A subdirectory inside a regular package that does not contain an `__init__.py` file is treated as an implicit `namespace package ` (a "namespace subpackage") rooted in that parent. See `420` for the underlying specification.
### Namespace packages
pair: package; namespace pair: package; portion
A namespace package is a composite of various `portions `, where each portion contributes a subpackage to the parent package. Portions may reside in different locations on the file system. Portions may also be found in zip files, on the network, or anywhere else that Python searches during import. Namespace packages may or may not correspond directly to objects on the file system; they may be virtual modules that have no concrete representation.
Namespace packages do not use an ordinary list for their `__path__` attribute. They instead use a custom iterable type which will automatically perform a new search for package portions on the next import attempt within that package if the path of their parent package (or `sys.path` for a top level package) changes.
With namespace packages, there is no `parent/__init__.py` file. In fact, there may be multiple `parent` directories found during import search, where each one is provided by a different portion. Thus `parent/one` may not be physically located next to `parent/two`. In this case, Python will create a namespace package for the top-level `parent` package whenever it or one of its subpackages is imported.
Namespace packages may also be nested inside a regular package. When the import system searches a regular package's `__path__` and encounters a subdirectory that does not contain an `__init__.py` file, that subdirectory becomes a `portion` contributing to a namespace subpackage of the enclosing regular package.
See also `420` for the namespace package specification.
## Searching
To begin the search, Python needs the `fully qualified ` name of the module (or package, but for the purposes of this discussion, the difference is immaterial) being imported. This name may come from various arguments to the `import` statement, or from the parameters to the `importlib.import_module` or `__import__` functions.
This name will be used in various phases of the import search, and it may be the dotted path to a submodule, e.g. `foo.bar.baz`. In this case, Python first tries to import `foo`, then `foo.bar`, and finally `foo.bar.baz`. If any of the intermediate imports fail, a `ModuleNotFoundError` is raised.
### The module cache
single: sys.modules
The first place checked during import search is `sys.modules`. This mapping serves as a cache of all modules that have been previously imported, including the intermediate paths. So if `foo.bar.baz` was previously imported, `sys.modules` will contain entries for `foo`, `foo.bar`, and `foo.bar.baz`. Each key will have as its value the corresponding module object.
During import, the module name is looked up in `sys.modules` and if present, the associated value is the module satisfying the import, and the process completes. However, if the value is `None`, then a `ModuleNotFoundError` is raised. If the module name is missing, Python will continue searching for the module.
`sys.modules` is writable. Deleting a key may not destroy the associated module (as other modules may hold references to it), but it will invalidate the cache entry for the named module, causing Python to search anew for the named module upon its next import. The key can also be assigned to `None`, forcing the next import of the module to result in a `ModuleNotFoundError`.
Beware though, as if you keep a reference to the module object, invalidate its cache entry in `sys.modules`, and then re-import the named module, the two module objects will *not* be the same. By contrast, `importlib.reload` will reuse the *same* module object, and simply reinitialise the module contents by rerunning the module's code.
### Finders and loaders
single: finder single: loader single: module spec
If the named module is not found in `sys.modules`, then Python's import protocol is invoked to find and load the module. This protocol consists of two conceptual objects, `finders ` and `loaders `. A finder's job is to determine whether it can find the named module using whatever strategy it knows about. Objects that implement both of these interfaces are referred to as `importers ` - they return themselves when they find that they can load the requested module.
Python includes a number of default finders and importers. The first one knows how to locate built-in modules, and the second knows how to locate frozen modules. A third default finder searches an `import path` for modules. The `import path` is a list of locations that may name file system paths or zip files. It can also be extended to search for any locatable resource, such as those identified by URLs.
The import machinery is extensible, so new finders can be added to extend the range and scope of module searching.
Finders do not actually load modules. If they can find the named module, they return a `module spec`, an encapsulation of the module's import-related information, which the import machinery then uses when loading the module.
The following sections describe the protocol for finders and loaders in more detail, including how you can create and register new ones to extend the import machinery.
3.4 In previous versions of Python, finders returned `loaders ` directly, whereas now they return module specs which *contain* loaders. Loaders are still used during import but have fewer responsibilities.
### Import hooks
single: import hooks single: meta hooks single: path hooks pair: hooks; import pair: hooks; meta pair: hooks; path
The import machinery is designed to be extensible; the primary mechanism for this are the *import hooks*. There are two types of import hooks: *meta hooks* and *import path hooks*.
Meta hooks are called at the start of import processing, before any other import processing has occurred, other than `sys.modules` cache look up. This allows meta hooks to override `sys.path` processing, frozen modules, or even built-in modules. Meta hooks are registered by adding new finder objects to `sys.meta_path`, as described below.
Import path hooks are called as part of `sys.path` (or `package.__path__`) processing, at the point where their associated path item is encountered. Import path hooks are registered by adding new callables to `sys.path_hooks` as described below.
### The meta path
single: sys.meta_path pair: finder; find_spec
When the named module is not found in `sys.modules`, Python next searches `sys.meta_path`, which contains a list of meta path finder objects. These finders are queried in order to see if they know how to handle the named module. Meta path finders must implement a method called `~importlib.abc.MetaPathFinder.find_spec` which takes three arguments: a name, an import path, and (optionally) a target module. The meta path finder can use any strategy it wants to determine whether it can handle the named module or not.
If the meta path finder knows how to handle the named module, it returns a spec object. If it cannot handle the named module, it returns `None`. If `sys.meta_path` processing reaches the end of its list without returning a spec, then a `ModuleNotFoundError` is raised. Any other exceptions raised are simply propagated up, aborting the import process.
The `~importlib.abc.MetaPathFinder.find_spec` method of meta path finders is called with two or three arguments. The first is the fully qualified name of the module being imported, for example `foo.bar.baz`. The second argument is the path entries to use for the module search. For top-level modules, the second argument is `None`, but for submodules or subpackages, the second argument is the value of the parent package's `__path__` attribute. If the appropriate `__path__` attribute cannot be accessed, a `ModuleNotFoundError` is raised. The third argument is an existing module object that will be the target of loading later. The import system passes in a target module only during reload.
The meta path may be traversed multiple times for a single import request. For example, assuming none of the modules involved has already been cached, importing `foo.bar.baz` will first perform a top level import, calling `mpf.find_spec("foo", None, None)` on each meta path finder (`mpf`). After `foo` has been imported, `foo.bar` will be imported by traversing the meta path a second time, calling `mpf.find_spec("foo.bar", foo.__path__, None)`. Once `foo.bar` has been imported, the final traversal will call `mpf.find_spec("foo.bar.baz", foo.bar.__path__, None)`.
Some meta path finders only support top level imports. These importers will always return `None` when anything other than `None` is passed as the second argument.
Python's default `sys.meta_path` has three meta path finders, one that knows how to import built-in modules, one that knows how to import frozen modules, and one that knows how to import modules from an `import path` (i.e. the `path based finder`).
3.4 The `~importlib.abc.MetaPathFinder.find_spec` method of meta path finders replaced `!find_module`, which is now deprecated. While it will continue to work without change, the import machinery will try it only if the finder does not implement `~importlib.abc.MetaPathFinder.find_spec`.
3.10 Use of `!find_module` by the import system now raises `ImportWarning`.
3.12 `!find_module` has been removed. Use `~importlib.abc.MetaPathFinder.find_spec` instead.
## Loading
If and when a module spec is found, the import machinery will use it (and the loader it contains) when loading the module. Here is an approximation of what happens during the loading portion of import:
module = None
if spec.loader is not None and hasattr(spec.loader, 'create_module'):
# It is assumed 'exec_module' will also be defined on the loader.
module = spec.loader.create_module(spec)
if module is None:
module = ModuleType(spec.name)
# The import-related module attributes get set here:
_init_module_attrs(spec, module)
if spec.loader is None:
# unsupported
raise ImportError
sys.modules[spec.name] = module
try:
spec.loader.exec_module(module)
except BaseException:
try:
del sys.modules[spec.name]
except KeyError:
pass
raise
return sys.modules[spec.name]
Note the following details:
- If there is an existing module object with the given name in `sys.modules`, import will have already returned it.
- The module will exist in `sys.modules` before the loader executes the module code. This is crucial because the module code may (directly or indirectly) import itself; adding it to `sys.modules` beforehand prevents unbounded recursion in the worst case and multiple loading in the best.
- If loading fails, the failing module -- and only the failing module -- gets removed from `sys.modules`. Any module already in the `sys.modules` cache, and any module that was successfully loaded as a side-effect, must remain in the cache. This contrasts with reloading where even the failing module is left in `sys.modules`.
- After the module is created but before execution, the import machinery sets the import-related module attributes ("\_init_module_attrs" in the pseudo-code example above), as summarized in a `later section `.
- Module execution is the key moment of loading in which the module's namespace gets populated. Execution is entirely delegated to the loader, which gets to decide what gets populated and how.
- The module created during loading and passed to exec_module() may not be the one returned at the end of import[^2].
3.4 The import system has taken over the boilerplate responsibilities of loaders. These were previously performed by the `importlib.abc.Loader.load_module` method.
3.15 The `load_module` method is no longer used.
### Loaders
Module loaders provide the critical function of loading: module execution. The import machinery calls the `importlib.abc.Loader.exec_module` method with a single argument, the module object to execute. Any value returned from `~importlib.abc.Loader.exec_module` is ignored.
Loaders must satisfy the following requirements:
- If the module is a Python module (as opposed to a built-in module or a dynamically loaded extension), the loader should execute the module's code in the module's global name space (`module.__dict__`).
- If the loader cannot execute the module, it should raise an `ImportError`, although any other exception raised during `~importlib.abc.Loader.exec_module` will be propagated.
In many cases, the finder and loader can be the same object; in such cases the `~importlib.abc.MetaPathFinder.find_spec` method would just return a spec with the loader set to `self`.
Module loaders may opt in to creating the module object during loading by implementing a `~importlib.abc.Loader.create_module` method. It takes one argument, the module spec, and returns the new module object to use during loading. `create_module()` does not need to set any attributes on the module object. If the method returns `None`, the import machinery will create the new module itself.
3.4 The `~importlib.abc.Loader.create_module` method of loaders.
3.4 The `importlib.abc.Loader.load_module` method was replaced by `~importlib.abc.Loader.exec_module` and the import machinery assumed all the boilerplate responsibilities of loading.
For compatibility with existing loaders, the import machinery will use the `load_module()` method of loaders if it exists and the loader does not also implement `exec_module()`. However, `load_module()` has been deprecated and loaders should implement `exec_module()` instead.
The `load_module()` method must implement all the boilerplate loading functionality described above in addition to executing the module. All the same constraints apply, with some additional clarification:
- If there is an existing module object with the given name in `sys.modules`, the loader must use that existing module. (Otherwise, `importlib.reload` will not work correctly.) If the named module does not exist in `sys.modules`, the loader must create a new module object and add it to `sys.modules`.
- The module *must* exist in `sys.modules` before the loader executes the module code, to prevent unbounded recursion or multiple loading.
- If loading fails, the loader must remove any modules it has inserted into `sys.modules`, but it must remove **only** the failing module(s), and only if the loader itself has loaded the module(s) explicitly.
3.5 A `DeprecationWarning` is raised when `exec_module()` is defined but `create_module()` is not.
3.6 An `ImportError` is raised when `exec_module()` is defined but `create_module()` is not.
3.10 Use of `load_module()` will raise `ImportWarning`.
### Submodules
When a submodule is loaded using any mechanism (e.g. `importlib` APIs, the `import` or `import-from` statements, or built-in `__import__()`) a binding is placed in the parent module's namespace to the submodule object. For example, if package `spam` has a submodule `foo`, after importing `spam.foo`, `spam` will have an attribute `foo` which is bound to the submodule. Let's say you have the following directory structure:
spam/
__init__.py
foo.py
and `spam/__init__.py` has the following line in it:
from .foo import Foo
then executing the following puts name bindings for `foo` and `Foo` in the `spam` module:
>>> import spam
>>> spam.foo
>>> spam.Foo
Given Python's familiar name binding rules this might seem surprising, but it's actually a fundamental feature of the import system. The invariant holding is that if you have `sys.modules['spam']` and `sys.modules['spam.foo']` (as you would after the above import), the latter must appear as the `foo` attribute of the former.
### Module specs
The import machinery uses a variety of information about each module during import, especially before loading. Most of the information is common to all modules. The purpose of a module's spec is to encapsulate this import-related information on a per-module basis.
Using a spec during import allows state to be transferred between import system components, e.g. between the finder that creates the module spec and the loader that executes it. Most importantly, it allows the import machinery to perform the boilerplate operations of loading, whereas without a module spec the loader had that responsibility.
The module's spec is exposed as `module.__spec__`. Setting `!__spec__` appropriately applies equally to `modules initialized during interpreter startup `. The one exception is `__main__`, where `!__spec__` is `set to None in some cases `.
See `~importlib.machinery.ModuleSpec` for details on the contents of the module spec.
3.4
### \_\_path\_\_ attributes on modules
The `~module.__path__` attribute should be a (possibly empty) `sequence` of strings enumerating the locations where the package's submodules will be found. By definition, if a module has a `!__path__` attribute, it is a `package`.
A package's `~module.__path__` attribute is used during imports of its subpackages. Within the import machinery, it functions much the same as `sys.path`, i.e. providing a list of locations to search for modules during import. However, `!__path__` is typically much more constrained than `!sys.path`.
The same rules used for `sys.path` also apply to a package's `!__path__`. `sys.path_hooks` (described below) are consulted when traversing a package's `!__path__`.
A package's `__init__.py` file may set or alter the package's `~module.__path__` attribute, and this was typically the way namespace packages were implemented prior to `420`. With the adoption of `420`, namespace packages no longer need to supply `__init__.py` files containing only `!__path__` manipulation code; the import machinery automatically sets `!__path__` correctly for the namespace package.
### Module reprs
By default, all modules have a usable repr, however depending on the attributes set above, and in the module's spec, you can more explicitly control the repr of module objects.
If the module has a spec (`__spec__`), the import machinery will try to generate a repr from it. If that fails or there is no spec, the import system will craft a default repr using whatever information is available on the module. It will try to use the `module.__name__`, `module.__file__`, and `module.__loader__` as input into the repr, with defaults for whatever information is missing.
Here are the exact rules used:
- If the module has a `__spec__` attribute, the information in the spec is used to generate the repr. The "name", "loader", "origin", and "has_location" attributes are consulted.
- If the module has a `__file__` attribute, this is used as part of the module's repr.
- If the module has no `__file__` but does have a `__loader__` that is not `None`, then the loader's repr is used as part of the module's repr.
- Otherwise, just use the module's `__name__` in the repr.
3.12 Use of `!module_repr`, having been deprecated since Python 3.4, was removed in Python 3.12 and is no longer called during the resolution of a module's repr.
### Cached bytecode invalidation
Before Python loads cached bytecode from a `.pyc` file, it checks whether the cache is up-to-date with the source `.py` file. By default, Python does this by storing the source's last-modified timestamp and size in the cache file when writing it. At runtime, the import system then validates the cache file by checking the stored metadata in the cache file against the source's metadata.
Python also supports "hash-based" cache files, which store a hash of the source file's contents rather than its metadata. There are two variants of hash-based `.pyc` files: checked and unchecked. For checked hash-based `.pyc` files, Python validates the cache file by hashing the source file and comparing the resulting hash with the hash in the cache file. If a checked hash-based cache file is found to be invalid, Python regenerates it and writes a new checked hash-based cache file. For unchecked hash-based `.pyc` files, Python simply assumes the cache file is valid if it exists. Hash-based `.pyc` files validation behavior may be overridden with the `--check-hash-based-pycs` flag.
3.7 Added hash-based `.pyc` files. Previously, Python only supported timestamp-based invalidation of bytecode caches.
## The Path Based Finder
single: path based finder
As mentioned previously, Python comes with several default meta path finders. One of these, called the `path based finder` (`~importlib.machinery.PathFinder`), searches an `import path`, which contains a list of `path entries `. Each path entry names a location to search for modules.
The path based finder itself doesn't know how to import anything. Instead, it traverses the individual path entries, associating each of them with a path entry finder that knows how to handle that particular kind of path.
The default set of path entry finders implement all the semantics for finding modules on the file system, handling special file types such as Python source code (`.py` files), Python byte code (`.pyc` files) and shared libraries (e.g. `.so` files). When supported by the `zipimport` module in the standard library, the default path entry finders also handle loading all of these file types (other than shared libraries) from zipfiles.
Within a single `path entry`, the default path entry finders check for a `regular package` first, then for extension modules, then for source files, and finally for bytecode files. For example, if the same directory contains both `spam/__init__.py` and `spam.py`, `import spam` will import the package from `spam/__init__.py`. A directory without an `__init__.py` file is treated as a `namespace package` portion only if no matching module is found. Note that this does not override the order of the `import path`.
Path entries need not be limited to file system locations. They can refer to URLs, database queries, or any other location that can be specified as a string.
The path based finder provides additional hooks and protocols so that you can extend and customize the types of searchable path entries. For example, if you wanted to support path entries as network URLs, you could write a hook that implements HTTP semantics to find modules on the web. This hook (a callable) would return a `path entry finder` supporting the protocol described below, which was then used to get a loader for the module from the web.
A word of warning: this section and the previous both use the term *finder*, distinguishing between them by using the terms `meta path finder` and `path entry finder`. These two types of finders are very similar, support similar protocols, and function in similar ways during the import process, but it's important to keep in mind that they are subtly different. In particular, meta path finders operate at the beginning of the import process, as keyed off the `sys.meta_path` traversal.
By contrast, path entry finders are in a sense an implementation detail of the path based finder, and in fact, if the path based finder were to be removed from `sys.meta_path`, none of the path entry finder semantics would be invoked.
### Path entry finders
The `path based finder` is responsible for finding and loading Python modules and packages whose location is specified with a string `path entry`. Most path entries name locations in the file system, but they need not be limited to this.
As a meta path finder, the `path based finder` implements the `~importlib.abc.MetaPathFinder.find_spec` protocol previously described, however it exposes additional hooks that can be used to customize how modules are found and loaded from the `import path`.
Three variables are used by the `path based finder`, `sys.path`, `sys.path_hooks` and `sys.path_importer_cache`. The `__path__` attributes on package objects are also used. These provide additional ways that the import machinery can be customized.
`sys.path` contains a list of strings providing search locations for modules and packages. It is initialized from the `PYTHONPATH` environment variable and various other installation- and implementation-specific defaults. Entries in `sys.path` can name directories on the file system, zip files, and potentially other "locations" (see the `site` module) that should be searched for modules, such as URLs, or database queries. Only strings should be present on `sys.path`; all other data types are ignored.
The `path based finder` is a `meta path finder`, so the import machinery begins the `import path` search by calling the path based finder's `~importlib.machinery.PathFinder.find_spec` method as described previously. When the `path` argument to `~importlib.machinery.PathFinder.find_spec` is given, it will be a list of string paths to traverse - typically a package's `__path__` attribute for an import within that package. If the `path` argument is `None`, this indicates a top level import and `sys.path` is used.
The path based finder iterates over every entry in the search path, and for each of these, looks for an appropriate `path entry finder` (`~importlib.abc.PathEntryFinder`) for the path entry. Because this can be an expensive operation (e.g. there may be `stat()` call overheads for this search), the path based finder maintains a cache mapping path entries to path entry finders. This cache is maintained in `sys.path_importer_cache` (despite the name, this cache actually stores finder objects rather than being limited to `importer` objects). In this way, the expensive search for a particular `path entry` location's `path entry finder` need only be done once. User code is free to remove cache entries from `sys.path_importer_cache` forcing the path based finder to perform the path entry search again.
If the path entry is not present in the cache, the path based finder iterates over every callable in `sys.path_hooks`. Each of the `path entry
hooks ` in this list is called with a single argument, the path entry to be searched. This callable may either return a `path
entry finder` that can handle the path entry, or it may raise `ImportError`. An `ImportError` is used by the path based finder to signal that the hook cannot find a `path entry finder` for that `path entry`. The exception is ignored and `import path` iteration continues. The hook should expect either a string or bytes object; the encoding of bytes objects is up to the hook (e.g. it may be a file system encoding, UTF-8, or something else), and if the hook cannot decode the argument, it should raise `ImportError`.
If `sys.path_hooks` iteration ends with no `path entry finder` being returned, then the path based finder's `~importlib.machinery.PathFinder.find_spec` method will store `None` in `sys.path_importer_cache` (to indicate that there is no finder for this path entry) and return `None`, indicating that this `meta path finder` could not find the module.
If a `path entry finder` *is* returned by one of the `path entry
hook` callables on `sys.path_hooks`, then the following protocol is used to ask the finder for a module spec, which is then used when loading the module.
The current working directory -- denoted by an empty string -- is handled slightly differently from other entries on `sys.path`. First, if the current working directory cannot be determined or is found not to exist, no value is stored in `sys.path_importer_cache`. Second, the value for the current working directory is looked up fresh for each module lookup. Third, the path used for `sys.path_importer_cache` and returned by `importlib.machinery.PathFinder.find_spec` will be the actual current working directory and not the empty string.
### Path entry finder protocol
In order to support imports of modules and initialized packages and also to contribute portions to namespace packages, path entry finders must implement the `~importlib.abc.PathEntryFinder.find_spec` method.
`~importlib.abc.PathEntryFinder.find_spec` takes two arguments: the fully qualified name of the module being imported, and the (optional) target module. `find_spec()` returns a fully populated spec for the module. This spec will always have "loader" set (with one exception).
To indicate to the import machinery that the spec represents a namespace `portion`, the path entry finder sets `submodule_search_locations` to a list containing the portion.
3.4 `~importlib.abc.PathEntryFinder.find_spec` replaced `!find_loader` and `!find_module`, both of which are now deprecated, but will be used if `find_spec()` is not defined.
Older path entry finders may implement one of these two deprecated methods instead of `find_spec()`. The methods are still respected for the sake of backward compatibility. However, if `find_spec()` is implemented on the path entry finder, the legacy methods are ignored.
`!find_loader` takes one argument, the fully qualified name of the module being imported. `find_loader()` returns a 2-tuple where the first item is the loader and the second item is a namespace `portion`.
For backwards compatibility with other implementations of the import protocol, many path entry finders also support the same, traditional `find_module()` method that meta path finders support. However path entry finder `find_module()` methods are never called with a `path` argument (they are expected to record the appropriate path information from the initial call to the path hook).
The `find_module()` method on path entry finders is deprecated, as it does not allow the path entry finder to contribute portions to namespace packages. If both `find_loader()` and `find_module()` exist on a path entry finder, the import system will always call `find_loader()` in preference to `find_module()`.
3.10 Calls to `!find_module` and `!find_loader` by the import system will raise `ImportWarning`.
3.12 `find_module()` and `find_loader()` have been removed.
## Replacing the standard import system
The most reliable mechanism for replacing the entire import system is to delete the default contents of `sys.meta_path`, replacing them entirely with a custom meta path hook.
If it is acceptable to only alter the behaviour of import statements without affecting other APIs that access the import system, then replacing the builtin `__import__` function may be sufficient.
To selectively prevent the import of some modules from a hook early on the meta path (rather than disabling the standard import system entirely), it is sufficient to raise `ModuleNotFoundError` directly from `~importlib.abc.MetaPathFinder.find_spec` instead of returning `None`. The latter indicates that the meta path search should continue, while raising an exception terminates it immediately.
## Package Relative Imports
Relative imports use leading dots. A single leading dot indicates a relative import, starting with the current package. Two or more leading dots indicate a relative import to the parent(s) of the current package, one level per dot after the first. For example, given the following package layout:
package/
__init__.py
subpackage1/
__init__.py
moduleX.py
moduleY.py
subpackage2/
__init__.py
moduleZ.py
moduleA.py
In either `subpackage1/moduleX.py` or `subpackage1/__init__.py`, the following are valid relative imports:
from .moduleY import spam
from .moduleY import spam as ham
from . import moduleY
from ..subpackage1 import moduleY
from ..subpackage2.moduleZ import eggs
from ..moduleA import foo
Absolute imports may use either the `import <>` or `from <> import <>` syntax, but relative imports may only use the second form; the reason for this is that:
import XXX.YYY.ZZZ
should expose `XXX.YYY.ZZZ` as a usable expression, but .moduleY is not a valid expression.
## Special considerations for \_\_main\_\_
The `__main__` module is a special case relative to Python's import system. As noted `elsewhere `, the `__main__` module is directly initialized at interpreter startup, much like `sys` and `builtins`. However, unlike those two, it doesn't strictly qualify as a built-in module. This is because the manner in which `__main__` is initialized depends on the flags and other options with which the interpreter is invoked.
### \_\_main\_\_.\_\_spec\_\_
Depending on how `__main__` is initialized, `__main__.__spec__` gets set appropriately or to `None`.
When Python is started with the `-m` option, `__spec__` is set to the module spec of the corresponding module or package. `__spec__` is also populated when the `__main__` module is loaded as part of executing a directory, zipfile or other `sys.path` entry.
In `the remaining cases ` `__main__.__spec__` is set to `None`, as the code used to populate the `__main__` does not correspond directly with an importable module:
- interactive prompt
- `-c` option
- running from stdin
- running directly from a source or bytecode file
Note that `__main__.__spec__` is always `None` in the last case, *even if* the file could technically be imported directly as a module instead. Use the `-m` switch if valid module metadata is desired in `__main__`.
Note also that even when `__main__` corresponds with an importable module and `__main__.__spec__` is set accordingly, they're still considered *distinct* modules. This is due to the fact that blocks guarded by `if __name__ == "__main__":` checks only execute when the module is used to populate the `__main__` namespace, and not during normal import.
## References
The import machinery has evolved considerably since Python's early days. The original [specification for packages](https://www.python.org/doc/essays/packages/) is still available to read, although some details have changed since the writing of that document.
The original specification for `sys.meta_path` was `302`, with subsequent extension in `420`.
`420` introduced `namespace packages ` for Python 3.3. `420` also introduced the `!find_loader` protocol as an alternative to `!find_module`.
`366` describes the addition of the `__package__` attribute for explicit relative imports in main modules.
`328` introduced absolute and explicit relative imports and initially proposed `__name__` for semantics `366` would eventually specify for `__package__`.
`338` defines executing modules as scripts.
`451` adds the encapsulation of per-module import state in spec objects. It also off-loads most of the boilerplate responsibilities of loaders back onto the import machinery. These changes allow the deprecation of several APIs in the import system and also addition of new methods to finders and loaders.
**Footnotes**
[^1]: See `types.ModuleType`.
[^2]: The importlib implementation avoids using the return value directly. Instead, it gets the module object by looking the module name up in `sys.modules`. The indirect effect of this is that an imported module may replace itself in `sys.modules`. This is implementation-specific behavior that is not guaranteed to work in other Python implementations.
# The Python Language Reference
This reference manual describes the syntax and core semantics of the language. It is terse, but attempts to be exact and complete.
Elsewhere, the built-in object types and functions are described in `builtins-index`. Standard library modules are described in `library-index`.
For an informal introduction to the language, see `tutorial-index`. For C or C++ programmers, two additional manuals exist: `extending-index` describes the high-level picture of how to write a Python extension module, and the `c-api-index` describes the interfaces available to C/C++ programmers in detail.
# Introduction
This reference manual describes the Python programming language. It is not intended as a tutorial.
While I am trying to be as precise as possible, I chose to use English rather than formal specifications for everything except syntax and lexical analysis. This should make the document more understandable to the average reader, but will leave room for ambiguities. Consequently, if you were coming from Mars and tried to re-implement Python from this document alone, you might have to guess things and in fact you would probably end up implementing quite a different language. On the other hand, if you are using Python and wonder what the precise rules about a particular area of the language are, you should definitely be able to find them here. If you would like to see a more formal definition of the language, maybe you could volunteer your time --- or invent a cloning machine :-).
It is dangerous to add too many implementation details to a language reference document --- the implementation may change, and other implementations of the same language may work differently. On the other hand, CPython is the one Python implementation in widespread use (although alternate implementations continue to gain support), and its particular quirks are sometimes worth being mentioned, especially where the implementation imposes additional limitations. Therefore, you'll find short "implementation notes" sprinkled throughout the text.
Every Python implementation comes with a number of built-in and standard modules. These are documented in `library-index`. A few built-in modules are mentioned when they interact in a significant way with the language definition.
## Alternate Implementations
Though there is one Python implementation which is by far the most popular, there are some alternate implementations which are of particular interest to different audiences.
Known implementations include:
CPython
This is the original and most-maintained implementation of Python, written in C. New language features generally appear here first.
Jython
Python implemented in Java. This implementation can be used as a scripting language for Java applications, or can be used to create applications using the Java class libraries. It is also often used to create tests for Java libraries. More information can be found at [the Jython website](https://www.jython.org/).
Python for .NET
This implementation actually uses the CPython implementation, but is a managed .NET application and makes .NET libraries available. It was created by Brian Lloyd. For more information, see the [Python for .NET home page](https://pythonnet.github.io/).
IronPython
An alternate Python for .NET. Unlike Python.NET, this is a complete Python implementation that generates IL, and compiles Python code directly to .NET assemblies. It was created by Jim Hugunin, the original creator of Jython. For more information, see [the IronPython website](https://ironpython.net/).
PyPy
An implementation of Python written completely in Python. It supports several advanced features not found in other implementations like stackless support and a Just in Time compiler. One of the goals of the project is to encourage experimentation with the language itself by making it easier to modify the interpreter (since it is written in Python). Additional information is available on [the PyPy project's home page](https://pypy.org/).
Each of these implementations varies in some way from the language as documented in this manual, or introduces specific information beyond what's covered in the standard Python documentation. Please refer to the implementation-specific documentation to determine what else you need to know about the specific implementation you're using.
## Notation
BNF, grammar, syntax, notation
The descriptions of lexical analysis and syntax use a grammar notation that is a mixture of [EBNF](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form) and [PEG](https://en.wikipedia.org/wiki/Parsing_expression_grammar). For example:
In this example, the first line says that a `name` is a `letter` followed by a sequence of zero or more `letter`s, `digit`s, and underscores. A `letter` in turn is any of the single characters `'a'` through `'z'` and `A` through `Z`; a `digit` is a single character from `0` to `9`.
Each rule begins with a name (which identifies the rule that's being defined) followed by a colon, `:`. The definition to the right of the colon uses the following syntax elements:
- `name`: A name refers to another rule. Where possible, it is a link to the rule's definition.
- `TOKEN`: An uppercase name refers to a `token`. For the purposes of grammar definitions, tokens are the same as rules.
- `"text"`, `'text'`: Text in single or double quotes must match literally (without the quotes). The type of quote is chosen according to the meaning of `text`:
- `'if'`: A name in single quotes denotes a `keyword `.
- `"case"`: A name in double quotes denotes a `soft-keyword `.
- `'@'`: A non-letter symbol in single quotes denotes an `~token.OP` token, that is, a `delimiter ` or `operator `.
- `e1 e2`: Items separated only by whitespace denote a sequence. Here, `e1` must be followed by `e2`.
- `e1 | e2`: A vertical bar is used to separate alternatives. It denotes PEG's "ordered choice": if `e1` matches, `e2` is not considered. In traditional PEG grammars, this is written as a slash, `/`, rather than a vertical bar. See `617` for more background and details.
- `e*`: A star means zero or more repetitions of the preceding item.
- `e+`: Likewise, a plus means one or more repetitions.
- `[e]`: A phrase enclosed in square brackets means zero or one occurrences. In other words, the enclosed phrase is optional.
- `e?`: A question mark has exactly the same meaning as square brackets: the preceding item is optional.
- `(e)`: Parentheses are used for grouping.
The following notation is only used in `lexical definitions `.
- `"a"..."z"`: Two literal characters separated by three dots mean a choice of any single character in the given (inclusive) range of ASCII characters.
- `<...>`: A phrase between angular brackets gives an informal description of the matched symbol (for example, ``), or an abbreviation that is defined in nearby text (for example, ``).
Some definitions also use *lookaheads*, which indicate that an element must (or must not) match at a given position, but without consuming any input:
- `&e`: a positive lookahead (that is, `e` is required to match)
- `!e`: a negative lookahead (that is, `e` is required *not* to match)
The unary operators (`*`, `+`, `?`) bind as tightly as possible; the vertical bar (`|`) binds most loosely.
White space is only meaningful to separate tokens.
Rules are normally contained on a single line, but rules that are too long may be wrapped:
Alternatively, rules may be formatted with the first line ending at the colon, and each alternative beginning with a vertical bar on a new line. For example:
This does *not* mean that there is an empty first alternative.
lexical definitions
### Lexical and Syntactic definitions
There is some difference between *lexical* and *syntactic* analysis: the `lexical analyzer` operates on the individual characters of the input source, while the *parser* (syntactic analyzer) operates on the stream of `tokens ` generated by the lexical analysis. However, in some cases the exact boundary between the two phases is a CPython implementation detail.
The practical difference between the two is that in *lexical* definitions, all whitespace is significant. The lexical analyzer `discards ` all whitespace that is not converted to tokens like `token.INDENT` or `~token.NEWLINE`. *Syntactic* definitions then use these tokens, rather than source characters.
This documentation uses the same BNF grammar for both styles of definitions. All uses of BNF in the next chapter (`lexical`) are lexical definitions; uses in subsequent chapters are syntactic definitions.
# Lexical analysis
lexical analysis, parser, token
A Python program is read by a *parser*. Input to the parser is a stream of `tokens `, generated by the *lexical analyzer* (also known as the *tokenizer*). This chapter describes how the lexical analyzer produces these tokens.
The lexical analyzer determines the program text's `encoding ` (UTF-8 by default), and decodes the text into `source characters `. If the text cannot be decoded, a `SyntaxError` is raised.
Next, the lexical analyzer uses the source characters to generate a stream of tokens. The type of a generated token generally depends on the next source character to be processed. Similarly, other special behavior of the analyzer depends on the first source character that hasn't yet been processed. The following table gives a quick summary of these source characters, with links to sections that contain more information.
Character
Next token (or other relevant documentation)
space
tab
formfeed
Whitespace <whitespace>
CR, LF
New line <line-structure>
Indentation <indentation>
backslash (\)
Explicit line joining <explicit-joining>
(Also significant in string escape sequences <escape-sequences>)
hash (#)
Comment <comments>
quote (', ")
String literal <strings>
ASCII letter (a-z, A-Z)
non-ASCII character
Name <identifiers>
Prefixed string or bytes literal <strings>
underscore (_)
Name <identifiers>
(Can also be part of numeric literals <numbers>)
number (0-9)
Numeric literal <numbers>
dot (.)
Numeric literal <numbers>
Operator <operators>
question mark (?)
dollar ($)
.. (the following uses zero-width space characters to render .. a literal backquote)
backquote (`)
control character
Error (outside string literals and comments)
other printing character
Operator or delimiter <operators>
end of file
End marker <endmarker-token>
## Line structure
line structure
A Python program is divided into a number of *logical lines*.
### Logical lines
logical line, physical line, line joining, NEWLINE token
The end of a logical line is represented by the token `~token.NEWLINE`. Statements cannot cross logical line boundaries except where `!NEWLINE` is allowed by the syntax (e.g., between statements in compound statements). A logical line is constructed from one or more *physical lines* by following the `explicit ` or `implicit ` *line joining* rules.
### Physical lines
A physical line is a sequence of characters terminated by one the following end-of-line sequences:
- the Unix form using ASCII LF (linefeed),
- the Windows form using the ASCII sequence CR LF (return followed by linefeed),
- the '\`Classic Mac OS\`\_\_' form using the ASCII CR (return) character.
Regardless of platform, each of these sequences is replaced by a single ASCII LF (linefeed) character. (This is done even inside `string literals `.) Each line can use any of the sequences; they do not need to be consistent within a file.
The end of input also serves as an implicit terminator for the final physical line.
Formally:
newline: \ \| \ \ \| \
### Comments
comment, hash character single: \# (hash); comment
A comment starts with a hash character (`#`) that is not part of a string literal, and ends at the end of the physical line. A comment signifies the end of the logical line unless the implicit line joining rules are invoked. Comments are ignored by the syntax.
### Encoding declarations
If a comment in the first or second line of the Python script matches the regular expression `coding[=:]\s*([-\w.]+)`, this comment is processed as an encoding declaration; the first group of this expression names the encoding of the source code file. The encoding declaration must appear on a line of its own. If it is the second line, the first line must also be a comment-only line. The recommended forms of an encoding expression are :
# -*- coding: -*-
which is recognized also by GNU Emacs, and :
# vim:fileencoding=
which is recognized by Bram Moolenaar's VIM.
If no encoding declaration is found, the default encoding is UTF-8. If the implicit or explicit encoding of a file is UTF-8, an initial UTF-8 byte-order mark (`b'\xef\xbb\xbf'`) is ignored rather than being a syntax error.
If an encoding is declared, the encoding name must be recognized by Python (see `standard-encodings`). The encoding is used for all lexical analysis, including string literals, comments and identifiers.
All lexical analysis, including string literals, comments and identifiers, works on Unicode text decoded using the source encoding. Any Unicode code point, except the NUL control character, can appear in Python source.
source_character: \
### Explicit line joining
physical line, line joining, line continuation, backslash character
Two or more physical lines may be joined into logical lines using backslash characters (`\`), as follows: when a physical line ends in a backslash that is not part of a string literal or comment, it is joined with the following forming a single logical line, deleting the backslash and the following end-of-line character. For example:
if 1900 < year < 2100 and 1 <= month <= 12 \
and 1 <= day <= 31 and 0 <= hour < 24 \
and 0 <= minute < 60 and 0 <= second < 60: # Looks like a valid date
return 1
A line ending in a backslash cannot carry a comment. A backslash does not continue a comment. A backslash does not continue a token except for string literals (i.e., tokens other than string literals cannot be split across physical lines using a backslash). A backslash is illegal elsewhere on a line outside a string literal.
### Implicit line joining
Expressions in parentheses, square brackets or curly braces can be split over more than one physical line without using backslashes. For example:
month_names = ['Januari', 'Februari', 'Maart', # These are the
'April', 'Mei', 'Juni', # Dutch names
'Juli', 'Augustus', 'September', # for the months
'Oktober', 'November', 'December'] # of the year
Implicitly continued lines can carry comments. The indentation of the continuation lines is not important. Blank continuation lines are allowed. There is no NEWLINE token between implicit continuation lines. Implicitly continued lines can also occur within triple-quoted strings (see below); in that case they cannot carry comments.
### Blank lines
single: blank line
A logical line that contains only spaces, tabs, formfeeds and possibly a comment, is ignored (i.e., no `~token.NEWLINE` token is generated). During interactive input of statements, handling of a blank line may differ depending on the implementation of the read-eval-print loop. In the standard interactive interpreter, an entirely blank logical line (that is, one containing not even whitespace or a comment) terminates a multi-line statement.
### Indentation
indentation, leading whitespace, space, tab, grouping, statement grouping
Leading whitespace (spaces and tabs) at the beginning of a logical line is used to compute the indentation level of the line, which in turn is used to determine the grouping of statements.
Tabs are replaced (from left to right) by one to eight spaces such that the total number of characters up to and including the replacement is a multiple of eight (this is intended to be the same rule as used by Unix). The total number of spaces preceding the first non-blank character then determines the line's indentation. Indentation cannot be split over multiple physical lines using backslashes; the whitespace up to the first backslash determines the indentation.
Indentation is rejected as inconsistent if a source file mixes tabs and spaces in a way that makes the meaning dependent on the worth of a tab in spaces; a `TabError` is raised in that case.
**Cross-platform compatibility note:** because of the nature of text editors on non-UNIX platforms, it is unwise to use a mixture of spaces and tabs for the indentation in a single source file. It should also be noted that different platforms may explicitly limit the maximum indentation level.
A formfeed character may be present at the start of the line; it will be ignored for the indentation calculations above. Formfeed characters occurring elsewhere in the leading whitespace have an undefined effect (for instance, they may reset the space count to zero).
INDENT token, DEDENT token
The indentation levels of consecutive lines are used to generate `~token.INDENT` and `~token.DEDENT` tokens, using a stack, as follows.
Before the first line of the file is read, a single zero is pushed on the stack; this will never be popped off again. The numbers pushed on the stack will always be strictly increasing from bottom to top. At the beginning of each logical line, the line's indentation level is compared to the top of the stack. If it is equal, nothing happens. If it is larger, it is pushed on the stack, and one `!INDENT` token is generated. If it is smaller, it *must* be one of the numbers occurring on the stack; all numbers on the stack that are larger are popped off, and for each number popped off a `!DEDENT` token is generated. At the end of the file, a `!DEDENT` token is generated for each number remaining on the stack that is larger than zero.
Here is an example of a correctly (though confusingly) indented piece of Python code:
def perm(l):
# Compute the list of all permutations of l
if len(l) <= 1:
return [l]
r = []
for i in range(len(l)):
s = l[:i] + l[i+1:]
p = perm(s)
for x in p:
r.append(l[i:i+1] + x)
return r
The following example shows various indentation errors:
def perm(l): # error: first line indented
for i in range(len(l)): # error: not indented
s = l[:i] + l[i+1:]
p = perm(l[:i] + l[i+1:]) # error: unexpected indent
for x in p:
r.append(l[i:i+1] + x)
return r # error: inconsistent dedent
(Actually, the first three errors are detected by the parser; only the last error is found by the lexical analyzer --- the indentation of `return r` does not match a level popped off the stack.)
### Whitespace between tokens
Except at the beginning of a logical line or in string literals, the whitespace characters space, tab and formfeed can be used interchangeably to separate tokens:
whitespace: ' ' \| tab \| formfeed
Whitespace is needed between two tokens only if their concatenation could otherwise be interpreted as a different token. For example, `ab` is one token, but `a b` is two tokens. However, `+a` and `+ a` both produce two tokens, `+` and `a`, as `+a` is not a valid token.
### End marker
At the end of non-interactive input, the lexical analyzer generates an `~token.ENDMARKER` token.
## Other tokens
Besides `~token.NEWLINE`, `~token.INDENT` and `~token.DEDENT`, the following categories of tokens exist: *identifiers* and *keywords* (`~token.NAME`), *literals* (such as `~token.NUMBER` and `~token.STRING`), and other symbols (*operators* and *delimiters*, `~token.OP`). Whitespace characters (other than logical line terminators, discussed earlier) are not tokens, but serve to delimit tokens. Where ambiguity exists, a token comprises the longest possible string that forms a legal token, when read from left to right.
## Names (identifiers and keywords)
identifier, name
`~token.NAME` tokens represent *identifiers*, *keywords*, and *soft keywords*.
Names are composed of the following characters:
- uppercase and lowercase letters (`A-Z` and `a-z`),
- the underscore (`_`),
- digits (`0` through `9`), which cannot appear as the first character, and
- non-ASCII characters. Valid names may only contain "letter-like" and "digit-like" characters; see `lexical-names-nonascii` for details.
Names must contain at least one character, but have no upper length limit. Case is significant.
Formally, names are described by the following lexical definitions:
Note that not all names matched by this grammar are valid; see `lexical-names-nonascii` for details.
### Keywords
single: keyword single: reserved word
The following names are used as reserved words, or *keywords* of the language, and cannot be used as ordinary identifiers. They must be spelled exactly as written here:
```
False await else import pass
None break except in raise
True class finally is return
and continue for lambda try
as def from nonlocal while
assert del global not with
async elif if or yield
```
### Soft Keywords
soft keyword, keyword
3.10
Some names are only reserved under specific contexts. These are known as *soft keywords*:
- `match`, `case`, and `_`, when used in the `match` statement.
- `type`, when used in the `type` statement.
- `lazy`, when used before an `import` statement.
These syntactically act as keywords in their specific contexts, but this distinction is done at the parser level, not when tokenizing.
As soft keywords, their use in the grammar is possible while still preserving compatibility with existing code that uses these names as identifier names.
3.12 `type` is now a soft keyword.
3.15 `lazy` is now a soft keyword.
single: \_, identifiers single: \_\_, identifiers
### Reserved classes of identifiers
Certain classes of identifiers (besides keywords) have special meanings. These classes are identified by the patterns of leading and trailing underscore characters:
`_*`
Not imported by `from module import *`.
`_`
In a `case` pattern within a `match` statement, `_` is a `soft keyword ` that denotes a `wildcard `.
Separately, the interactive interpreter makes the result of the last evaluation available in the variable `_`. (It is stored in the `builtins` module, alongside built-in functions like `print`.)
Elsewhere, `_` is a regular identifier. It is often used to name "special" items, but it is not special to Python itself.
> [!NOTE]
> The name `_` is often used in conjunction with internationalization; refer to the documentation for the `gettext` module for more information on this convention.
>
> It is also commonly used for unused variables.
`__*__`
System-defined names, informally known as "dunder" names. These names are defined by the interpreter and its implementation (including the standard library). Current system names are discussed in the `specialnames` section and elsewhere. More will likely be defined in future versions of Python. *Any* use of `__*__` names, in any context, that does not follow explicitly documented use, is subject to breakage without warning.
`__*`
Class-private names. Names in this category, when used within the context of a class definition, are re-written to use a mangled form to help avoid name clashes between "private" attributes of base and derived classes. See section `atom-identifiers`.
### Non-ASCII characters in names
Names that contain non-ASCII characters need additional normalization and validation beyond the rules and grammar explained `above `. For example, `ř_1`, `蛇`, or `साँप` are valid names, but `r〰2`, `€`, or `🐍` are not.
This section explains the exact rules.
All names are converted into the [normalization form](https://www.unicode.org/reports/tr15/#Norm_Forms) NFKC while parsing. This means that, for example, some typographic variants of characters are converted to their "basic" form. For example, `fiⁿₐˡᵢᶻₐᵗᵢᵒₙ` normalizes to `finalization`, so Python treats them as the same name:
>>> fiⁿₐˡᵢᶻₐᵗᵢᵒₙ = 3
>>> finalization
3
> [!NOTE]
> Normalization is done at the lexical level only. Run-time functions that take names as *strings* generally do not normalize their arguments. For example, the variable defined above is accessible at run time in the `globals` dictionary as `globals()["finalization"]` but not `globals()["fiⁿₐˡᵢᶻₐᵗᵢᵒₙ"]`.
Similarly to how ASCII-only names must contain only letters, digits and the underscore, and cannot start with a digit, a valid name must start with a character in the "letter-like" set `xid_start`, and the remaining characters must be in the "letter- and digit-like" set `xid_continue`.
These sets are based on the *XID_Start* and *XID_Continue* sets as defined by the Unicode standard annex [UAX-31](https://www.unicode.org/reports/tr31/). Python's `xid_start` additionally includes the underscore (`_`). Note that Python does not necessarily conform to [UAX-31](https://www.unicode.org/reports/tr31/).
A non-normative listing of characters in the *XID_Start* and *XID_Continue* sets as defined by Unicode is available in the [DerivedCoreProperties.txt](https://www.unicode.org/Public/18.0.0/ucd/DerivedCoreProperties.txt) file in the Unicode Character Database. For reference, the construction rules for the `xid_*` sets are given below.
The set `id_start` is defined as the union of:
- Unicode category `` - uppercase letters (includes `A` to `Z`)
- Unicode category `` - lowercase letters (includes `a` to `z`)
- Unicode category `` - titlecase letters
- Unicode category `` - modifier letters
- Unicode category `` - other letters
- Unicode category `` - letter numbers
- {`"_"`} - the underscore
- `` - an explicit set of characters in [PropList.txt](https://www.unicode.org/Public/18.0.0/ucd/PropList.txt) to support backwards compatibility
The set `xid_start` then closes this set under NFKC normalization, by removing all characters whose normalization is not of the form `id_start id_continue*`.
The set `id_continue` is defined as the union of:
- `id_start` (see above)
- Unicode category `` - decimal numbers (includes `0` to `9`)
- Unicode category `` - connector punctuations
- Unicode category `` - nonspacing marks
- Unicode category `` - spacing combining marks
- `` - another explicit set of characters in [PropList.txt](https://www.unicode.org/Public/18.0.0/ucd/PropList.txt) to support backwards compatibility
Again, `xid_continue` closes this set under NFKC normalization.
Unicode categories use the version of the Unicode Character Database as included in the `unicodedata` module.
Literals are notations for constant values of some built-in types.
In terms of lexical analysis, Python has `string, bytes ` and `numeric ` literals.
Other "literals" are lexically denoted using `keywords ` (`None`, `True`, `False`) and the special `ellipsis token ` (`...`).
## String and Bytes literals
String literals are text enclosed in single quotes (`'`) or double quotes (`"`). For example:
``` python
"spam"
'eggs'
```
The quote used to start the literal also terminates it, so a string literal can only contain the other quote (except with escape sequences, see below). For example:
``` python
'Say "Hello", please.'
"Don't do that!"
```
Except for this limitation, the choice of quote character (`'` or `"`) does not affect how the literal is parsed.
Inside a string literal, the backslash (`\`) character introduces an `escape sequence`, which has special meaning depending on the character after the backslash. For example, `\"` denotes the double quote character, and does *not* end the string:
```
>>> print("Say \"Hello\" to everyone!")
Say "Hello" to everyone!
```
See `escape sequences ` below for a full list of such sequences, and more details.
### Triple-quoted strings
Strings can also be enclosed in matching groups of three single or double quotes. These are generally referred to as `triple-quoted strings`:
"""This is a triple-quoted string."""
In triple-quoted literals, unescaped quotes are allowed (and are retained), except that three unescaped quotes in a row terminate the literal, if they are of the same kind (`'` or `"`) used at the start:
"""This string has "quotes" inside."""
Unescaped newlines are also allowed and retained:
'''This triple-quoted string
continues on the next line.'''
### String prefixes
String literals can have an optional `prefix` that influences how the content of the literal is parsed, for example:
``` python
b"data"
f'{result=}'
```
The allowed prefixes are:
- `b`: `Bytes literal `
- `r`: `Raw string `
- `f`: `Formatted string literal ` ("f-string")
- `t`: `Template string literal ` ("t-string")
- `u`: No effect (allowed for backwards compatibility)
See the linked sections for details on each type.
Prefixes are case-insensitive (for example, '`B`' works the same as '`b`'). The '`r`' prefix can be combined with '`f`', '`t`' or '`b`', so '`fr`', '`rf`', '`tr`', '`rt`', '`br`', and '`rb`' are also valid prefixes.
3.3 The `'rb'` prefix of raw bytes literals has been added as a synonym of `'br'`.
Support for the unicode legacy literal (`u'value'`) was reintroduced to simplify the maintenance of dual Python 2.x and 3.x codebases. See `414` for more information.
### Formal grammar
String literals, except `"f-strings" ` and `"t-strings" `, are described by the following lexical definitions.
These definitions use `negative lookaheads ` (`!`) to indicate that an ending quote ends the literal.
Note that as in all lexical definitions, whitespace is significant. In particular, the prefix (if any) must be immediately followed by the starting quote.