# Library

# `!abc` --- Abstract Base Classes

<div class="module" data-synopsis="Abstract base classes according to :pep:`3119`.">

abc

</div>

**Source code:** `Lib/abc.py`

------------------------------------------------------------------------

This module provides the infrastructure for defining `abstract base
classes <abstract base class>` (ABCs) in Python, as outlined in `3119`; see the PEP for why this was added to Python. (See also `3141` and the `numbers` module regarding a type hierarchy for numbers based on ABCs.)

The `collections` module has some concrete classes that derive from ABCs; these can, of course, be further derived. In addition, the `collections.abc` submodule has some ABCs that can be used to test whether a class or instance provides a particular interface, for example, if it is `hashable` or if it is a `mapping`.

This module provides the metaclass `ABCMeta` for defining ABCs and a helper class `ABC` to alternatively define ABCs through inheritance:

<div class="ABC">

A helper class that has `ABCMeta` as its metaclass. With this class, an abstract base class can be created by simply deriving from `!ABC` avoiding sometimes confusing metaclass usage, for example:

    from abc import ABC

    class MyABC(ABC):
        pass

Note that the type of `!ABC` is still `ABCMeta`, therefore inheriting from `!ABC` requires the usual precautions regarding metaclass usage, as multiple inheritance may lead to metaclass conflicts. One may also define an abstract base class by passing the metaclass keyword and using `!ABCMeta` directly, for example:

    from abc import ABCMeta

    class MyABC(metaclass=ABCMeta):
        pass

<div class="versionadded">

3.4

</div>

</div>

<div class="ABCMeta">

Metaclass for defining Abstract Base Classes (ABCs).

Use this metaclass to create an ABC. An ABC can be subclassed directly, and then acts as a mix-in class. You can also register unrelated concrete classes (even built-in classes) and unrelated ABCs as "virtual subclasses" -- these and their descendants will be considered subclasses of the registering ABC by the built-in `issubclass` function, but the registering ABC won't show up in their MRO (Method Resolution Order) nor will method implementations defined by the registering ABC be callable (not even via `super`).[^1]

Classes created with a metaclass of `!ABCMeta` have the following method:

<div class="method">

register(subclass)

Register *subclass* as a "virtual subclass" of this ABC. For example:

    from abc import ABC

    class MyABC(ABC):
        pass

    MyABC.register(tuple)

    assert issubclass(tuple, MyABC)
    assert isinstance((), MyABC)

<div class="versionchanged">

3.3 Returns the registered subclass, to allow usage as a class decorator.

</div>

<div class="versionchanged">

3.4 To detect calls to `!register`, you can use the `get_cache_token` function.

</div>

</div>

You can also override this method in an abstract base class:

<div class="method">

\_\_subclasshook\_\_(subclass)

(Must be defined as a class method.)

Check whether *subclass* is considered a subclass of this ABC. This means that you can customize the behavior of `issubclass` further without the need to call `register` on every class you want to consider a subclass of the ABC. (This class method is called from the `~type.__subclasscheck__` method of the ABC.)

This method should return `True`, `False` or `NotImplemented`. If it returns `True`, the *subclass* is considered a subclass of this ABC. If it returns `False`, the *subclass* is not considered a subclass of this ABC, even if it would normally be one. If it returns `!NotImplemented`, the subclass check is continued with the usual mechanism.

</div>

For a demonstration of these concepts, look at this example ABC definition:

    class Foo:
        def __getitem__(self, index):
            ...
        def __len__(self):
            ...
        def get_iterator(self):
            return iter(self)

    class MyIterable(ABC):

        @abstractmethod
        def __iter__(self):
            while False:
                yield None

        def get_iterator(self):
            return self.__iter__()

        @classmethod
        def __subclasshook__(cls, C):
            if cls is MyIterable:
                if any("__iter__" in B.__dict__ for B in C.__mro__):
                    return True
            return NotImplemented

    MyIterable.register(Foo)

The ABC `MyIterable` defines the standard iterable method, `~object.__iter__`, as an abstract method. The implementation given here can still be called from subclasses. The `!get_iterator` method is also part of the `MyIterable` abstract base class, but it does not have to be overridden in non-abstract derived classes.

The `__subclasshook__` class method defined here says that any class that has an `~object.__iter__` method in its `~object.__dict__` (or in that of one of its base classes, accessed via the `~type.__mro__` list) is considered a `MyIterable` too.

Finally, the last line makes `Foo` a virtual subclass of `MyIterable`, even though it does not define an `~object.__iter__` method (it uses the old-style iterable protocol, defined in terms of `~object.__len__` and `~object.__getitem__`). Note that this will not make `get_iterator` available as a method of `Foo`, so it is provided separately.

</div>

The `!abc` module also provides the following decorator:

<div class="decorator">

abstractmethod

A decorator indicating abstract methods.

Using this decorator requires that the class's metaclass is `ABCMeta` or is derived from it. A class that has a metaclass derived from `!ABCMeta` cannot be instantiated unless all of its abstract methods and properties are overridden. The abstract methods can be called using any of the normal 'super' call mechanisms. `!abstractmethod` may be used to declare abstract methods for properties and descriptors.

Dynamically adding abstract methods to a class, or attempting to modify the abstraction status of a method or class once it is created, are only supported using the `update_abstractmethods` function. The `!abstractmethod` only affects subclasses derived using regular inheritance; "virtual subclasses" registered with the ABC's `~ABCMeta.register` method are not affected.

When `!abstractmethod` is applied in combination with other method descriptors, it should be applied as the innermost decorator, as shown in the following usage examples:

    class C(ABC):
        @abstractmethod
        def my_abstract_method(self, arg1):
            ...
        @classmethod
        @abstractmethod
        def my_abstract_classmethod(cls, arg2):
            ...
        @staticmethod
        @abstractmethod
        def my_abstract_staticmethod(arg3):
            ...

        @property
        @abstractmethod
        def my_abstract_property(self):
            ...
        @my_abstract_property.setter
        @abstractmethod
        def my_abstract_property(self, val):
            ...

        @abstractmethod
        def _get_x(self):
            ...
        @abstractmethod
        def _set_x(self, val):
            ...
        x = property(_get_x, _set_x)

In order to correctly interoperate with the abstract base class machinery, the descriptor must identify itself as abstract using `!__isabstractmethod__`. In general, this attribute should be `True` if any of the methods used to compose the descriptor are abstract. For example, Python's built-in `property` does the equivalent of:

    class Descriptor:
        ...
        @property
        def __isabstractmethod__(self):
            return any(getattr(f, '__isabstractmethod__', False) for
                       f in (self._fget, self._fset, self._fdel))

> [!NOTE]
> Unlike Java abstract methods, these abstract methods may have an implementation. This implementation can be called via the `super` mechanism from the class that overrides it. This could be useful as an end-point for a super-call in a framework that uses cooperative multiple-inheritance.

</div>

The `!abc` module also supports the following legacy decorators:

<div class="decorator">

abstractclassmethod

<div class="versionadded">

3.2

</div>

<div class="deprecated-removed">

3.3 3.21 It is now possible to use `classmethod` with `abstractmethod`, making this decorator redundant.

</div>

A subclass of the built-in `classmethod`, indicating an abstract classmethod. Otherwise it is similar to `abstractmethod`.

This special case is deprecated, as the `classmethod` decorator is now correctly identified as abstract when applied to an abstract method:

    class C(ABC):
        @classmethod
        @abstractmethod
        def my_abstract_classmethod(cls, arg):
            ...

</div>

<div class="decorator">

abstractstaticmethod

<div class="versionadded">

3.2

</div>

<div class="deprecated-removed">

3.3 3.21 It is now possible to use `staticmethod` with `abstractmethod`, making this decorator redundant.

</div>

A subclass of the built-in `staticmethod`, indicating an abstract staticmethod. Otherwise it is similar to `abstractmethod`.

This special case is deprecated, as the `staticmethod` decorator is now correctly identified as abstract when applied to an abstract method:

    class C(ABC):
        @staticmethod
        @abstractmethod
        def my_abstract_staticmethod(arg):
            ...

</div>

<div class="decorator">

abstractproperty

<div class="deprecated-removed">

3.3 3.21 It is now possible to use `property`, `property.getter`, `property.setter` and `property.deleter` with `abstractmethod`, making this decorator redundant.

</div>

A subclass of the built-in `property`, indicating an abstract property.

This special case is deprecated, as the `property` decorator is now correctly identified as abstract when applied to an abstract method:

    class C(ABC):
        @property
        @abstractmethod
        def my_abstract_property(self):
            ...

The above example defines a read-only property; you can also define a read-write abstract property by appropriately marking one or more of the underlying methods as abstract:

    class C(ABC):
        @property
        def x(self):
            ...

        @x.setter
        @abstractmethod
        def x(self, val):
            ...

If only some components are abstract, only those components need to be updated to create a concrete property in a subclass:

    class D(C):
        @C.x.setter
        def x(self, val):
            ...

</div>

The `!abc` module also provides the following functions:

<div class="function">

get_cache_token()

Returns the current abstract base class cache token.

The token is an opaque object (that supports equality testing) identifying the current version of the abstract base class cache for virtual subclasses. The token changes with every call to `ABCMeta.register` on any ABC.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

update_abstractmethods(cls)

A function to recalculate an abstract class's abstraction status. This function should be called if a class's abstract methods have been implemented or changed after it was created. Usually, this function should be called from within a class decorator.

Returns *cls*, to allow usage as a class decorator.

If *cls* is not an instance of `ABCMeta`, does nothing.

> [!NOTE]
> This function assumes that *cls*'s superclasses are already updated. It does not update any subclasses.

<div class="versionadded">

3.10

</div>

</div>

**Footnotes**

[^1]: C++ programmers should note that Python's virtual base class concept is not the same as C++'s.

# `!aifc` --- Read and write AIFF and AIFC files

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

aifc

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!aifc` module was [Python 3.12](https://docs.python.org/3.12/library/aifc.html).

# Generic Operating System Services

The modules described in this chapter provide interfaces to operating system features that are available on (almost) all operating systems, such as files and a clock. The interfaces are generally modeled after the Unix or C interfaces, but they are available on most other systems as well. Here's an overview:

<div class="toctree">

os.rst io.rst time.rst logging.rst logging.config.rst logging.handlers.rst platform.rst errno.rst ctypes.rst

</div>

# `!annotationlib` --- Functionality for introspecting annotations

<div class="module" data-synopsis="Functionality for introspecting annotations">

annotationlib

</div>

<div class="versionadded">

3.14

</div>

**Source code:** `Lib/annotationlib.py`

<div class="testsetup">

default

import annotationlib from annotationlib import \*

</div>

------------------------------------------------------------------------

The `!annotationlib` module provides tools for introspecting `annotations <annotation>` on modules, classes, and functions.

Annotations are `lazily evaluated <lazy-evaluation>` and often contain forward references to objects that are not yet defined when the annotation is created. This module provides a set of low-level tools that can be used to retrieve annotations in a reliable way, even in the presence of forward references and other edge cases.

This module supports retrieving annotations in three main formats (see `Format`), each of which works best for different use cases:

- `~Format.VALUE` evaluates the annotations and returns their value. This is most straightforward to work with, but it may raise errors, for example if the annotations contain references to undefined names.
- `~Format.FORWARDREF` returns `ForwardRef` objects for annotations that cannot be resolved, allowing you to inspect the annotations without evaluating them. This is useful when you need to work with annotations that may contain unresolved forward references.
- `~Format.STRING` returns the annotations as a string, similar to how it would appear in the source file. This is useful for documentation generators that want to display annotations in a readable way.

The `get_annotations` function is the main entry point for retrieving annotations. Given a function, class, or module, it returns an annotations dictionary in the requested format. This module also provides functionality for working directly with the `annotate function` that is used to evaluate annotations, such as `get_annotate_from_class_namespace` and `call_annotate_function`, as well as the `call_evaluate_function` function for working with `evaluate functions <evaluate function>`.

> [!CAUTION]
> Most functionality in this module can execute arbitrary code; see `the security section <annotationlib-security>` for more information.

<div class="seealso">

`649` proposed the current model for how annotations work in Python.

`749` expanded on various aspects of `649` and introduced the `!annotationlib` module.

`annotations-howto` provides best practices for working with annotations.

`typing-extensions` provides a backport of `get_annotations` that works on earlier versions of Python.

</div>

## Annotation semantics

The way annotations are evaluated has changed over the history of Python 3, and currently still depends on a `future import <future>`. There have been execution models for annotations:

- *Stock semantics* (default in Python 3.0 through 3.13; see `3107` and `526`): Annotations are evaluated eagerly, as they are encountered in the source code.
- *Stringified annotations* (used with `from __future__ import annotations` in Python 3.7 and newer; see `563`): Annotations are stored as strings only.
- *Deferred evaluation* (default in Python 3.14 and newer; see `649` and `749`): Annotations are evaluated lazily, only when they are accessed.

As an example, consider the following program:

    def func(a: Cls) -> None:
        print(a)

    class Cls: pass

    print(func.__annotations__)

This will behave as follows:

- Under stock semantics (Python 3.13 and earlier), it will throw a `NameError` at the line where `func` is defined, because `Cls` is an undefined name at that point.
- Under stringified annotations (if `from __future__ import annotations` is used), it will print `{'a': 'Cls', 'return': 'None'}`.
- Under deferred evaluation (Python 3.14 and later), it will print `{'a': <class 'Cls'>, 'return': None}`.

Stock semantics were used when function annotations were first introduced in Python 3.0 (by `3107`) because this was the simplest, most obvious way to implement annotations. The same execution model was used when variable annotations were introduced in Python 3.6 (by `526`). However, stock semantics caused problems when using annotations as type hints, such as a need to refer to names that are not yet defined when the annotation is encountered. In addition, there were performance problems with executing annotations at module import time. Therefore, in Python 3.7, `563` introduced the ability to store annotations as strings using the `from __future__ import annotations` syntax. The plan at the time was to eventually make this behavior the default, but a problem appeared: stringified annotations are more difficult to process for those who introspect annotations at runtime. An alternative proposal, `649`, introduced the third execution model, deferred evaluation, and was implemented in Python 3.14. Stringified annotations are still used if `from __future__ import annotations` is present, but this behavior will eventually be removed.

## Classes

<div class="Format">

An `~enum.IntEnum` describing the formats in which annotations can be returned. Members of the enum, or their equivalent integer values, can be passed to `get_annotations` and other functions in this module, as well as to `~object.__annotate__` functions.

<div class="attribute" value="1">

VALUE

Values are the result of evaluating the annotation expressions.

</div>

<div class="attribute" value="2">

VALUE_WITH_FAKE_GLOBALS

Special value used to signal that an annotate function is being evaluated in a special environment with fake globals. When passed this value, annotate functions should either return the same value as for the `Format.VALUE` format, or raise `NotImplementedError` to signal that they do not support execution in this environment. This format is only used internally and should not be passed to the functions in this module.

</div>

<div class="attribute" value="3">

FORWARDREF

Values are real annotation values (as per `Format.VALUE` format) for defined values, and `ForwardRef` proxies for undefined values. Real objects may contain references to `ForwardRef` proxy objects.

</div>

<div class="attribute" value="4">

STRING

Values are the text string of the annotation as it appears in the source code, up to modifications including, but not restricted to, whitespace normalizations and constant values optimizations.

The exact values of these strings may change in future versions of Python.

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="ForwardRef">

A proxy object for forward references in annotations.

Instances of this class are returned when the `~Format.FORWARDREF` format is used and annotations contain a name that cannot be resolved. This can happen when a forward reference is used in an annotation, such as when a class is referenced before it is defined.

<div class="attribute">

\_\_forward_arg\_\_

A string containing the code that was evaluated to produce the `~ForwardRef`. The string may not be exactly equivalent to the original source.

</div>

<div class="method">

evaluate(\*, owner=None, globals=None, locals=None, type_params=None, format=Format.VALUE)

Evaluate the forward reference, returning its value.

If the *format* argument is `~Format.VALUE` (the default), this method may throw an exception, such as `NameError`, if the forward reference refers to a name that cannot be resolved. The arguments to this method can be used to provide bindings for names that would otherwise be undefined. If the *format* argument is `~Format.FORWARDREF`, the method will never throw an exception, but may return a `~ForwardRef` instance. For example, if the forward reference object contains the code `list[undefined]`, where `undefined` is a name that is not defined, evaluating it with the `~Format.FORWARDREF` format will return `list[ForwardRef('undefined')]`. If the *format* argument is `~Format.STRING`, the method will return `~ForwardRef.__forward_arg__`.

The *owner* parameter provides the preferred mechanism for passing scope information to this method. The owner of a `~ForwardRef` is the object that contains the annotation from which the `~ForwardRef` derives, such as a module object, type object, or function object.

The *globals*, *locals*, and *type_params* parameters provide a more precise mechanism for influencing the names that are available when the `~ForwardRef` is evaluated. *globals* and *locals* are passed to `eval`, representing the global and local namespaces in which the name is evaluated. The *type_params* parameter is relevant for objects created using the native syntax for `generic classes <generic-classes>` and `functions <generic-functions>`. It is a tuple of `type parameters <type-params>` that are in scope while the forward reference is being evaluated. For example, if evaluating a `~ForwardRef` retrieved from an annotation found in the class namespace of a generic class `C`, *type_params* should be set to `C.__type_params__`.

`~ForwardRef` instances returned by `get_annotations` retain references to information about the scope they originated from, so calling this method with no further arguments may be sufficient to evaluate such objects. `~ForwardRef` instances created by other means may not have any information about their scope, so passing arguments to this method may be necessary to evaluate them successfully.

If no *owner*, *globals*, *locals*, or *type_params* are provided and the `~ForwardRef` does not contain information about its origin, empty globals and locals dictionaries are used.

</div>

<div class="versionadded">

3.14

</div>

</div>

## Functions

<div class="function">

annotations_to_string(annotations)

Convert an annotations dict containing runtime values to a dict containing only strings. If the values are not already strings, they are converted using `type_repr`. This is meant as a helper for user-provided annotate functions that support the `~Format.STRING` format but do not have access to the code creating the annotations.

For example, this is used to implement the `~Format.STRING` format for `typing.TypedDict` classes created through the functional syntax:

<div class="doctest">

\>\>\> from typing import TypedDict \>\>\> Movie = TypedDict("movie", {"name": str, "year": int}) \>\>\> get_annotations(Movie, format=Format.STRING) {'name': 'str', 'year': 'int'}

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

call_annotate_function(annotate, format, \*, owner=None)

Call the `annotate function` *annotate* with the given *format*, a member of the `Format` enum, and return the annotations dictionary produced by the function.

This helper function is required because annotate functions generated by the compiler for functions, classes, and modules only support the `~Format.VALUE` format when called directly. To support other formats, this function calls the annotate function in a special environment that allows it to produce annotations in the other formats. This is a useful building block when implementing functionality that needs to partially evaluate annotations while a class is being constructed.

*owner* is the object that owns the annotation function, usually a function, class, or module. If provided, it is used in the `~Format.FORWARDREF` format to produce a `ForwardRef` object that carries more information.

<div class="seealso">

[PEP PEP 649 \<649#the-stringizer-and-the-fake-globals-environment\>](<http://www.python.org/dev/peps/pep-PEP 649 \<649#the-stringizer-and-the-fake-globals-environment\>/> "PEP PEP 649 <649#the-stringizer-and-the-fake-globals-environment>") contains an explanation of the implementation technique used by this function.

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

call_evaluate_function(evaluate, format, \*, owner=None)

Call the `evaluate function` *evaluate* with the given *format*, a member of the `Format` enum, and return the value produced by the function. This is similar to `call_annotate_function`, but the latter always returns a dictionary mapping strings to annotations, while this function returns a single value.

This is intended for use with the evaluate functions generated for lazily evaluated elements related to type aliases and type parameters:

- `typing.TypeAliasType.evaluate_value`, the value of type aliases
- `typing.TypeVar.evaluate_bound`, the bound of type variables
- `typing.TypeVar.evaluate_constraints`, the constraints of type variables
- `typing.TypeVar.evaluate_default`, the default value of type variables
- `typing.ParamSpec.evaluate_default`, the default value of parameter specifications
- `typing.TypeVarTuple.evaluate_default`, the default value of type variable tuples

*owner* is the object that owns the evaluate function, such as the type alias or type variable object.

*format* can be used to control the format in which the value is returned:

<div class="doctest">

\>\>\> type Alias = undefined \>\>\> call_evaluate_function(Alias.evaluate_value, Format.VALUE) Traceback (most recent call last): ... NameError: name 'undefined' is not defined \>\>\> call_evaluate_function(Alias.evaluate_value, Format.FORWARDREF) ForwardRef('undefined') \>\>\> call_evaluate_function(Alias.evaluate_value, Format.STRING) 'undefined'

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

get_annotate_from_class_namespace(namespace)

Retrieve the `annotate function` from a class namespace dictionary *namespace*. Return `!None` if the namespace does not contain an annotate function. This is primarily useful before the class has been fully created (e.g., in a metaclass); after the class exists, the annotate function can be retrieved with `cls.__annotate__`. See `below <annotationlib-metaclass>` for an example using this function in a metaclass.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

get_annotations(obj, \*, globals=None, locals=None, eval_str=False, format=Format.VALUE)

Compute the annotations dict for an object.

*obj* may be a callable, class, module, or other object with `~object.__annotate__` or `~object.__annotations__` attributes. Passing any other object raises `TypeError`.

The *format* parameter controls the format in which annotations are returned, and must be a member of the `Format` enum or its integer equivalent. The different formats work as follows:

- VALUE: `!object.__annotations__` is tried first; if that does not exist, the `!object.__annotate__` function is called if it exists.
- FORWARDREF: If `!object.__annotations__` exists and can be evaluated successfully, it is used; otherwise, the `!object.__annotate__` function is called. If it does not exist either, `!object.__annotations__` is tried again and any error from accessing it is re-raised.
  - When calling `!object.__annotate__` it is first called with `~Format.FORWARDREF`. If this is not implemented, it will then check if `~Format.VALUE_WITH_FAKE_GLOBALS` is supported and use that in the fake globals environment. If neither of these formats are supported, it will fall back to using `~Format.VALUE`. If `~Format.VALUE` fails, the error from this call will be raised.
- STRING: If `!object.__annotate__` exists, it is called first; otherwise, `!object.__annotations__` is used and stringified using `annotations_to_string`.
  - When calling `!object.__annotate__` it is first called with `~Format.STRING`. If this is not implemented, it will then check if `~Format.VALUE_WITH_FAKE_GLOBALS` is supported and use that in the fake globals environment. If neither of these formats are supported, it will fall back to using `~Format.VALUE` with the result converted using `annotations_to_string`. If `~Format.VALUE` fails, the error from this call will be raised.

Returns a dict. `!get_annotations` returns a new dict every time it's called; calling it twice on the same object will return two different but equivalent dicts.

This function handles several details for you:

- If *eval_str* is true, values of type `!str` will be un-stringized using `eval`. This is intended for use with stringized annotations (`from __future__ import annotations`). It is an error to set *eval_str* to true with formats other than `Format.VALUE`.
- If *obj* doesn't have an annotations dict, returns an empty dict. (Functions and methods always have an annotations dict; classes, modules, and other types of callables may not.)
- Ignores inherited annotations on classes, as well as annotations on metaclasses. If a class doesn't have its own annotations dict, returns an empty dict.
- All accesses to object members and dict values are done using `getattr()` and `dict.get()` for safety.

*eval_str* controls whether or not values of type `!str` are replaced with the result of calling `eval` on those values:

- If eval_str is true, `eval` is called on values of type `!str`. (Note that `!get_annotations` doesn't catch exceptions; if `eval` raises an exception, it will unwind the stack past the `!get_annotations` call.)
- If *eval_str* is false (the default), values of type `!str` are unchanged.

*globals* and *locals* are passed in to `eval`; see the documentation for `eval` for more information. If *globals* or *locals* is `!None`, this function may replace that value with a context-specific default, contingent on `type(obj)`:

- If *obj* is a module, *globals* defaults to `obj.__dict__`.
- If *obj* is a class, *globals* defaults to `sys.modules[obj.__module__].__dict__` and *locals* defaults to the *obj* class namespace.
- If *obj* is a callable, *globals* defaults to `obj.__globals__ <function.__globals__>`, although if *obj* is a wrapped function (using `functools.update_wrapper`) or a `functools.partial` object, it is unwrapped until a non-wrapped function is found.

Calling `!get_annotations` is best practice for accessing the annotations dict of any object. See `annotations-howto` for more information on annotations best practices.

<div class="doctest">

\>\>\> def f(a: int, b: str) -\> float: ... pass \>\>\> get_annotations(f) {'a': \<class 'int'\>, 'b': \<class 'str'\>, 'return': \<class 'float'\>}

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

type_repr(value)

Convert an arbitrary Python value to a format suitable for use by the `~Format.STRING` format. This calls `repr` for most objects, but has special handling for some objects, such as type objects.

This is meant as a helper for user-provided annotate functions that support the `~Format.STRING` format but do not have access to the code creating the annotations. It can also be used to provide a user-friendly string representation for other objects that contain values that are commonly encountered in annotations.

<div class="versionadded">

3.14

</div>

</div>

## Recipes

### Using annotations in a metaclass

A `metaclass <metaclasses>` may want to inspect or even modify the annotations in a class body during class creation. Doing so requires retrieving annotations from the class namespace dictionary. For classes created with `from __future__ import annotations`, the annotations will be in the `__annotations__` key of the dictionary. For other classes with annotations, `get_annotate_from_class_namespace` can be used to get the annotate function, and `call_annotate_function` can be used to call it and retrieve the annotations. Using the `~Format.FORWARDREF` format will usually be best, because this allows the annotations to refer to names that cannot yet be resolved when the class is created.

To modify the annotations, it is best to create a wrapper annotate function that calls the original annotate function, makes any necessary adjustments, and returns the result.

Below is an example of a metaclass that filters out all `typing.ClassVar` annotations from the class and puts them in a separate attribute:

``` python
import annotationlib
import typing

class ClassVarSeparator(type):
   def __new__(mcls, name, bases, ns):
      if "__annotations__" in ns:  # from __future__ import annotations
         annotations = ns["__annotations__"]
         classvar_keys = {
            key for key, value in annotations.items()
            # Use string comparison for simplicity; a more robust solution
            # could use annotationlib.ForwardRef.evaluate
            if value.startswith("ClassVar")
         }
         classvars = {key: annotations[key] for key in classvar_keys}
         ns["__annotations__"] = {
            key: value for key, value in annotations.items()
            if key not in classvar_keys
         }
         wrapped_annotate = None
      elif annotate := annotationlib.get_annotate_from_class_namespace(ns):
         annotations = annotationlib.call_annotate_function(
            annotate, format=annotationlib.Format.FORWARDREF
         )
         classvar_keys = {
            key for key, value in annotations.items()
            if typing.get_origin(value) is typing.ClassVar
         }
         classvars = {key: annotations[key] for key in classvar_keys}

         def wrapped_annotate(format):
            annos = annotationlib.call_annotate_function(annotate, format, owner=typ)
            return {key: value for key, value in annos.items() if key not in classvar_keys}

      else:  # no annotations
         classvars = {}
         wrapped_annotate = None
      typ = super().__new__(mcls, name, bases, ns)

      if wrapped_annotate is not None:
         # Wrap the original __annotate__ with a wrapper that removes ClassVars
         typ.__annotate__ = wrapped_annotate
      typ.classvars = classvars  # Store the ClassVars in a separate attribute
      return typ
```

### Creating a custom callable annotate function

Custom `annotate functions <annotate function>` may be literal functions like those automatically generated for functions, classes, and modules. Or, they may wish to utilise the encapsulation provided by classes, in which case any `callable` can be used as an `annotate function`.

To provide the `~Format.VALUE`, `~Format.STRING`, or `~Format.FORWARDREF` formats directly, an `annotate function` must provide the following attribute:

- A callable `__call__` with signature `__call__(format, /) -> dict`, that does not raise a `NotImplementedError` when called with a supported format.

To provide the `~Format.VALUE_WITH_FAKE_GLOBALS` format, which is used to automatically generate `~Format.STRING` or `~Format.FORWARDREF` if they are not supported directly, `annotate functions <annotate function>` must provide the following attributes:

- A callable `__call__` with signature `__call__(format, /) -> dict`, that does not raise a `NotImplementedError` when called with `~Format.VALUE_WITH_FAKE_GLOBALS`.
- A `code object <code-objects>` `__code__` containing the compiled code for the annotate function.
- Optional: A tuple of the function's positional defaults `__defaults__`, if the function represented by `__code__` uses any positional defaults.
- Optional: A dict of the function's keyword defaults `__kwdefaults__`, if the function represented by `__code__` uses any keyword defaults.
- Optional: All other `function attributes <inspect-types>`.

``` python
class Annotate:
    called_formats = []

    def __call__(self, format=None, /, *, _self=None):
        # When called with fake globals, `_self` will be the
        # actual self value, and `self` will be the format.
        if _self is not None:
            self, format = _self, self

        self.called_formats.append(format)
        if format <= 2:  # VALUE or VALUE_WITH_FAKE_GLOBALS
            return {"x": MyType}
        raise NotImplementedError

    __code__ = __call__.__code__
    __defaults__ = (None,)
    __kwdefaults__ = property(lambda self: dict(_self=self))

    __globals__ = {}
    __builtins__ = {}
    __closure__ = None
```

This can then be called with:

```
>>> from annotationlib import call_annotate_function, Format
>>> call_annotate_function(Annotate(), format=Format.STRING)
{'x': 'MyType'}
```

Or used as the annotate function for an object:

```
>>> from annotationlib import get_annotations, Format
>>> class C:
...   pass
>>> C.__annotate__ = Annotate()
>>> get_annotations(Annotate(), format=Format.STRING)
{'x': 'MyType'}
```

## Limitations of the `STRING` format

The `~Format.STRING` format is meant to approximate the source code of the annotation, but the implementation strategy used means that it is not always possible to recover the exact source code.

First, the stringifier of course cannot recover any information that is not present in the compiled code, including comments, whitespace, parenthesization, and operations that get simplified by the compiler.

Second, the stringifier can intercept almost all operations that involve names looked up in some scope, but it cannot intercept operations that operate fully on constants. As a corollary, this also means it is not safe to request the `STRING` format on untrusted code: Python is powerful enough that it is possible to achieve arbitrary code execution even with no access to any globals or builtins. For example:

```
>>> def f(x: (1).__class__.__base__.__subclasses__()[-1].__init__.__builtins__["print"]("Hello world")): pass
...
>>> annotationlib.get_annotations(f, format=annotationlib.Format.STRING)
Hello world
{'x': 'None'}
```

> [!NOTE]
> This particular example works as of the time of writing, but it relies on implementation details and is not guaranteed to work in the future.

Among the different kinds of expressions that exist in Python, as represented by the `ast` module, some expressions are supported, meaning that the `STRING` format can generally recover the original source code; others are unsupported, meaning that they may result in incorrect output or an error.

The following are supported (sometimes with caveats):

- `ast.BinOp`
- `ast.UnaryOp`
  - `ast.Invert` (`~`), `ast.UAdd` (`+`), and `ast.USub` (`-`) are supported
  - `ast.Not` (`not`) is not supported
- `ast.Dict` (except when using `**` unpacking)
- `ast.Set`
- `ast.Compare`
  - `ast.Eq` and `ast.NotEq` are supported
  - `ast.Lt`, `ast.LtE`, `ast.Gt`, and `ast.GtE` are supported, but the operand may be flipped
  - `ast.Is`, `ast.IsNot`, `ast.In`, and `ast.NotIn` are not supported
- `ast.Call` (except when using `**` unpacking)
- `ast.Constant` (though not the exact representation of the constant; for example, escape sequences in strings are lost; hexadecimal numbers are converted to decimal)
- `ast.Attribute` (assuming the value is not a constant)
- `ast.Subscript` (assuming the value is not a constant)
- `ast.Starred` (`*` unpacking)
- `ast.Name`
- `ast.List`
- `ast.Tuple`
- `ast.Slice`

The following are unsupported, but throw an informative error when encountered by the stringifier:

- `ast.FormattedValue` (f-strings; error is not detected if conversion specifiers like `!r` are used)
- `ast.JoinedStr` (f-strings)

The following are unsupported and result in incorrect output:

- `ast.BoolOp` (`and` and `or`)
- `ast.IfExp`
- `ast.Lambda`
- `ast.ListComp`
- `ast.SetComp`
- `ast.DictComp`
- `ast.GeneratorExp`

The following are disallowed in annotation scopes and therefore not relevant:

- `ast.NamedExpr` (`:=`)
- `ast.Await`
- `ast.Yield`
- `ast.YieldFrom`

## Limitations of the `FORWARDREF` format

The `~Format.FORWARDREF` format aims to produce real values as much as possible, with anything that cannot be resolved replaced with `ForwardRef` objects. It is affected by broadly the same Limitations as the `~Format.STRING` format: annotations that perform operations on literals or that use unsupported expression types may raise exceptions when evaluated using the `~Format.FORWARDREF` format.

Below are a few examples of the behavior with unsupported expressions:

```
>>> from annotationlib import get_annotations, Format
>>> def zerodiv(x: 1 / 0): ...
>>> get_annotations(zerodiv, format=Format.STRING)
Traceback (most recent call last):
  ...
ZeroDivisionError: division by zero
>>> get_annotations(zerodiv, format=Format.FORWARDREF)
Traceback (most recent call last):
  ...
ZeroDivisionError: division by zero
>>> def ifexp(x: 1 if y else 0): ...
>>> get_annotations(ifexp, format=Format.STRING)
{'x': '1'}
```

## Security implications of introspecting annotations

Much of the functionality in this module involves executing code related to annotations, which can then do arbitrary things. For example, `get_annotations` may call an arbitrary `annotate function`, and `ForwardRef.evaluate` may call `eval` on an arbitrary string. Code contained in an annotation might make arbitrary system calls, enter an infinite loop, or perform any other operation. This is also true for any access of the `~object.__annotations__` attribute, and for various functions in the `typing` module that work with annotations, such as `typing.get_type_hints`.

Any security issue arising from this also applies immediately after importing code that may contain untrusted annotations: importing code can always cause arbitrary operations to be performed. However, it is unsafe to accept strings or other input from an untrusted source and pass them to any of the APIs for introspecting annotations, for example by editing an `__annotations__` dictionary or directly creating a `ForwardRef` object.

# Data Compression and Archiving

The modules described in this chapter support data compression with the zlib, gzip, bzip2, lzma, and zstd algorithms, and the creation of ZIP- and tar-format archives. See also `archiving-operations` provided by the `shutil` module.

<div class="toctree">

compression.rst compression.zstd.rst zlib.rst gzip.rst bz2.rst lzma.rst zipfile.rst tarfile.rst

</div>

# `!argparse` --- Parser for command-line options, arguments and subcommands

<div class="module" data-synopsis="Command-line option and argument parsing library.">

argparse

</div>

<div class="versionadded">

3.2

</div>

**Source code:** `Lib/argparse.py`

> [!NOTE]
> While `!argparse` is the default recommended standard library module for implementing basic command line applications, authors with more exacting requirements for exactly how their command line applications behave may find it doesn't provide the necessary level of control. Refer to `choosing-an-argument-parser` for alternatives to consider when `argparse` doesn't support behaviors that the application requires (such as entirely disabling support for interspersed options and positional arguments, or accepting option parameter values that start with `-` even when they correspond to another defined option).

------------------------------------------------------------------------

<div class="sidebar">

**Tutorial**

This page contains the API reference information. For a more gentle introduction to Python command-line parsing, have a look at the `argparse tutorial <argparse-tutorial>`.

</div>

The `!argparse` module makes it easy to write user-friendly command-line interfaces. The program defines what arguments it requires, and `!argparse` will figure out how to parse those out of `sys.argv`. The `!argparse` module also automatically generates help and usage messages. The module will also issue errors when users give the program invalid arguments.

The `!argparse` module's support for command-line interfaces is built around an instance of `argparse.ArgumentParser`. It is a container for argument specifications and has options that apply to the parser as whole:

    parser = argparse.ArgumentParser(
                        prog='ProgramName',
                        description='What the program does',
                        epilog='Text at the bottom of help')

The `ArgumentParser.add_argument` method attaches individual argument specifications to the parser. It supports positional arguments, options that accept values, and on/off flags:

    parser.add_argument('filename')           # positional argument
    parser.add_argument('-c', '--count')      # option that takes a value
    parser.add_argument('-v', '--verbose',
                        action='store_true')  # on/off flag

The `ArgumentParser.parse_args` method runs the parser and places the extracted data in a `argparse.Namespace` object:

    args = parser.parse_args()
    print(args.filename, args.count, args.verbose)

> [!NOTE]
> If you're looking for a guide about how to upgrade `optparse` code to `!argparse`, see `Upgrading Optparse Code <upgrading-optparse-code>`.

## ArgumentParser objects

<div class="ArgumentParser(prog=None, usage=None, description=None, \ epilog=None, parents=[], \ formatter_class=argparse.HelpFormatter, \ prefix_chars='-', fromfile_prefix_chars=None, \ argument_default=None, conflict_handler='error', \ add_help=True, allow_abbrev=True, exit_on_error=True, \ *, suggest_on_error=True, color=True)">

Create a new `ArgumentParser` object. All parameters should be passed as keyword arguments. Each parameter has its own more detailed description below, but in short they are:

- [prog](#prog) - The name of the program (default: generated from the `__main__` module attributes and `sys.argv[0]`)
- [usage](#usage) - The string describing the program usage (default: generated from arguments added to parser)
- [description](#description) - Text to display before the argument help (by default, no text)
- [epilog](#epilog) - Text to display after the argument help (by default, no text)
- [parents](#parents) - A list of `ArgumentParser` objects whose arguments should also be included
- [formatter_class](#formatter_class) - A class for customizing the help output
- [prefix_chars](#prefix_chars) - The set of characters that prefix optional arguments (default: '-')
- [fromfile_prefix_chars](#fromfile_prefix_chars) - The set of characters that prefix files from which additional arguments should be read (default: `None`)
- [argument_default](#argument_default) - The global default value for arguments (default: `None`)
- [conflict_handler](#conflict_handler) - The strategy for resolving conflicting optionals (usually unnecessary)
- [add_help](#add_help) - Add a `-h/--help` option to the parser (default: `True`)
- [allow_abbrev](#allow_abbrev) - Allows long options to be abbreviated if the abbreviation is unambiguous (default: `True`)
- [exit_on_error](#exit_on_error) - Determines whether or not `!ArgumentParser` exits with error info when an error occurs. (default: `True`)
- [suggest_on_error](#suggest_on_error) - Enables suggestions for mistyped argument choices and subparser names (default: `True`)
- [color](#color) - Allow color output (default: `True`)

<div class="versionchanged">

3.5 *allow_abbrev* parameter was added.

</div>

<div class="versionchanged">

3.8 In previous versions, *allow_abbrev* also disabled grouping of short flags such as `-vv` to mean `-v -v`.

</div>

<div class="versionchanged">

3.9 *exit_on_error* parameter was added.

</div>

<div class="versionchanged">

3.14 *suggest_on_error* and *color* parameters were added.

</div>

<div class="versionchanged">

3.15 *suggest_on_error* default changed to `True`.

</div>

</div>

The following sections describe how each of these are used.

### `prog`

By default, `ArgumentParser` calculates the name of the program to display in help messages depending on the way the Python interpreter was run:

- The `base name <os.path.basename>` of `sys.argv[0]` if a file was passed as argument.
- The Python interpreter name followed by `sys.argv[0]` if a directory or a zipfile was passed as argument.
- The Python interpreter name followed by `-m` followed by the module or package name if the `-m` option was used.

This default is almost always desirable because it will make the help messages match the string that was used to invoke the program on the command line. However, to change this default behavior, another value can be supplied using the `prog=` argument to `ArgumentParser`:

    >>> parser = argparse.ArgumentParser(prog='myprogram')
    >>> parser.print_help()
    usage: myprogram [-h]

    options:
     -h, --help  show this help message and exit

Note that the program name, whether determined from `sys.argv[0]`, from the `__main__` module attributes or from the `prog=` argument, is available to help messages using the `%(prog)s` format specifier.

    >>> parser = argparse.ArgumentParser(prog='myprogram')
    >>> parser.add_argument('--foo', help='foo of the %(prog)s program')
    >>> parser.print_help()
    usage: myprogram [-h] [--foo FOO]

    options:
     -h, --help  show this help message and exit
     --foo FOO   foo of the myprogram program

<div class="versionchanged">

3.14 The default `prog` value now reflects how `__main__` was actually executed, rather than always being `os.path.basename(sys.argv[0])`.

</div>

### `usage`

By default, `ArgumentParser` calculates the usage message from the arguments it contains. The default message can be overridden with the `usage=` keyword argument:

    >>> parser = argparse.ArgumentParser(prog='PROG', usage='%(prog)s [options]')
    >>> parser.add_argument('--foo', nargs='?', help='foo help')
    >>> parser.add_argument('bar', nargs='+', help='bar help')
    >>> parser.print_help()
    usage: PROG [options]

    positional arguments:
     bar          bar help

    options:
     -h, --help   show this help message and exit
     --foo [FOO]  foo help

The `%(prog)s` format specifier is available to fill in the program name in your usage messages.

When a custom usage message is specified for the main parser, you may also want to consider passing the `prog` argument to `~ArgumentParser.add_subparsers` or the `prog` and the `usage` arguments to `~_SubParsersAction.add_parser`, to ensure consistent command prefixes and usage information across subparsers.

### `description`

Most calls to the `ArgumentParser` constructor will use the `description=` keyword argument. This argument gives a brief description of what the program does and how it works. In help messages, the description is displayed between the command-line usage string and the help messages for the various arguments.

By default, the description will be line-wrapped so that it fits within the given space. To change this behavior, see the [formatter_class](#formatter_class) argument.

### `epilog`

Some programs like to display additional description of the program after the description of the arguments. Such text can be specified using the `epilog=` argument to `ArgumentParser`:

    >>> parser = argparse.ArgumentParser(
    ...     description='A foo that bars',
    ...     epilog="And that's how you'd foo a bar")
    >>> parser.print_help()
    usage: argparse.py [-h]

    A foo that bars

    options:
     -h, --help  show this help message and exit

    And that's how you'd foo a bar

As with the [description](#description) argument, the `epilog=` text is by default line-wrapped, but this behavior can be adjusted with the [formatter_class](#formatter_class) argument to `ArgumentParser`.

### `parents`

Sometimes, several parsers share a common set of arguments. Rather than repeating the definitions of these arguments, a single parser with all the shared arguments and passed to `parents=` argument to `ArgumentParser` can be used. The `parents=` argument takes a list of `ArgumentParser` objects, collects all the positional and optional actions from them, and adds these actions to the `ArgumentParser` object being constructed:

    >>> parent_parser = argparse.ArgumentParser(add_help=False)
    >>> parent_parser.add_argument('--parent', type=int)

    >>> foo_parser = argparse.ArgumentParser(parents=[parent_parser])
    >>> foo_parser.add_argument('foo')
    >>> foo_parser.parse_args(['--parent', '2', 'XXX'])
    Namespace(foo='XXX', parent=2)

    >>> bar_parser = argparse.ArgumentParser(parents=[parent_parser])
    >>> bar_parser.add_argument('--bar')
    >>> bar_parser.parse_args(['--bar', 'YYY'])
    Namespace(bar='YYY', parent=None)

Note that most parent parsers will specify `add_help=False`. Otherwise, the `ArgumentParser` will see two `-h/--help` options (one in the parent and one in the child) and raise an error.

> [!NOTE]
> You must fully initialize the parsers before passing them via `parents=`. If you change the parent parsers after the child parser, those changes will not be reflected in the child.

### `formatter_class`

`ArgumentParser` objects allow the help formatting to be customized by specifying an alternate formatting class. Currently, there are four such classes:

<div class="RawDescriptionHelpFormatter RawTextHelpFormatter ArgumentDefaultsHelpFormatter MetavarTypeHelpFormatter">

`RawDescriptionHelpFormatter` and `RawTextHelpFormatter` give more control over how textual descriptions are displayed. By default, `ArgumentParser` objects line-wrap the [description](#description) and [epilog](#epilog) texts in command-line help messages:

    >>> parser = argparse.ArgumentParser(
    ...     prog='PROG',
    ...     description='''this description
    ...         was indented weird
    ...             but that is okay''',
    ...     epilog='''
    ...             likewise for this epilog whose whitespace will
    ...         be cleaned up and whose words will be wrapped
    ...         across a couple lines''')
    >>> parser.print_help()
    usage: PROG [-h]

    this description was indented weird but that is okay

    options:
     -h, --help  show this help message and exit

    likewise for this epilog whose whitespace will be cleaned up and whose words
    will be wrapped across a couple lines

</div>

Passing `RawDescriptionHelpFormatter` as `formatter_class=` indicates that [description](#description) and [epilog](#epilog) are already correctly formatted and should not be line-wrapped:

    >>> parser = argparse.ArgumentParser(
    ...     prog='PROG',
    ...     formatter_class=argparse.RawDescriptionHelpFormatter,
    ...     description=textwrap.dedent('''\
    ...         Please do not mess up this text!
    ...         --------------------------------
    ...             I have indented it
    ...             exactly the way
    ...             I want it
    ...         '''))
    >>> parser.print_help()
    usage: PROG [-h]

    Please do not mess up this text!
    --------------------------------
       I have indented it
       exactly the way
       I want it

    options:
     -h, --help  show this help message and exit

`RawTextHelpFormatter` maintains whitespace for all sorts of help text, including argument descriptions. However, multiple newlines are replaced with one. If you wish to preserve multiple blank lines, add spaces between the newlines.

`ArgumentDefaultsHelpFormatter` automatically adds information about default values to each of the argument help messages:

    >>> parser = argparse.ArgumentParser(
    ...     prog='PROG',
    ...     formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    >>> parser.add_argument('--foo', type=int, default=42, help='FOO!')
    >>> parser.add_argument('bar', nargs='*', default=[1, 2, 3], help='BAR!')
    >>> parser.print_help()
    usage: PROG [-h] [--foo FOO] [bar ...]

    positional arguments:
     bar         BAR! (default: [1, 2, 3])

    options:
     -h, --help  show this help message and exit
     --foo FOO   FOO! (default: 42)

`MetavarTypeHelpFormatter` uses the name of the [type](#type) argument for each argument as the display name for its values (rather than using the [dest](#dest) as the regular formatter does):

    >>> parser = argparse.ArgumentParser(
    ...     prog='PROG',
    ...     formatter_class=argparse.MetavarTypeHelpFormatter)
    >>> parser.add_argument('--foo', type=int)
    >>> parser.add_argument('bar', type=float)
    >>> parser.print_help()
    usage: PROG [-h] [--foo int] float

    positional arguments:
      float

    options:
      -h, --help  show this help message and exit
      --foo int

### `prefix_chars`

Most command-line options will use `-` as the prefix, e.g. `-f/--foo`. Parsers that need to support different or additional prefix characters, e.g. for options like `+f` or `/foo`, may specify them using the `prefix_chars=` argument to the `ArgumentParser` constructor:

    >>> parser = argparse.ArgumentParser(prog='PROG', prefix_chars='-+')
    >>> parser.add_argument('+f')
    >>> parser.add_argument('++bar')
    >>> parser.parse_args('+f X ++bar Y'.split())
    Namespace(bar='Y', f='X')

The `prefix_chars=` argument defaults to `'-'`. Supplying a set of characters that does not include `-` will cause `-f/--foo` options to be disallowed.

### `fromfile_prefix_chars`

Sometimes, when dealing with a particularly long argument list, it may make sense to keep the list of arguments in a file rather than typing it out at the command line. If the `fromfile_prefix_chars=` argument is given to the `ArgumentParser` constructor, then arguments that start with any of the specified characters will be treated as files, and will be replaced by the arguments they contain. For example:

    >>> with open('args.txt', 'w', encoding=sys.getfilesystemencoding()) as fp:
    ...     fp.write('-f\nbar')
    ...
    >>> parser = argparse.ArgumentParser(fromfile_prefix_chars='@')
    >>> parser.add_argument('-f')
    >>> parser.parse_args(['-f', 'foo', '@args.txt'])
    Namespace(f='bar')

Arguments read from a file must be one per line by default (but see also `~ArgumentParser.convert_arg_line_to_args`) and are treated as if they were in the same place as the original file referencing argument on the command line. So in the example above, the expression `['-f', 'foo', '@args.txt']` is considered equivalent to the expression `['-f', 'foo', '-f', 'bar']`.

> [!NOTE]
> Each line is treated as a single argument, so an empty line is read as an empty string (`''`).

`ArgumentParser` uses `filesystem encoding and error handler` to read the file containing arguments.

The `fromfile_prefix_chars=` argument defaults to `None`, meaning that arguments will never be treated as file references.

<div class="versionchanged">

3.12 `ArgumentParser` changed encoding and errors to read arguments files from default (e.g. `locale.getpreferredencoding(False) <locale.getpreferredencoding>` and `"strict"`) to the `filesystem encoding and error handler`. Arguments file should be encoded in UTF-8 instead of ANSI Codepage on Windows.

</div>

### `argument_default`

Generally, argument defaults are specified either by passing a default to `~ArgumentParser.add_argument` or by calling the `~ArgumentParser.set_defaults` methods with a specific set of name-value pairs. Sometimes however, it may be useful to specify a single parser-wide default for arguments. This can be accomplished by passing the `argument_default=` keyword argument to `ArgumentParser`. For example, to globally suppress attribute creation on `~ArgumentParser.parse_args` calls, we supply `argument_default=SUPPRESS`:

    >>> parser = argparse.ArgumentParser(argument_default=argparse.SUPPRESS)
    >>> parser.add_argument('--foo')
    >>> parser.add_argument('bar', nargs='?')
    >>> parser.parse_args(['--foo', '1', 'BAR'])
    Namespace(bar='BAR', foo='1')
    >>> parser.parse_args([])
    Namespace()

### `allow_abbrev`

Normally, when you pass an argument list to the `~ArgumentParser.parse_args` method of an `ArgumentParser`, it `recognizes abbreviations <prefix-matching>` of long options.

This feature can be disabled by setting `allow_abbrev` to `False`:

    >>> parser = argparse.ArgumentParser(prog='PROG', allow_abbrev=False)
    >>> parser.add_argument('--foobar', action='store_true')
    >>> parser.add_argument('--foonley', action='store_false')
    >>> parser.parse_args(['--foon'])
    usage: PROG [-h] [--foobar] [--foonley]
    PROG: error: unrecognized arguments: --foon

<div class="versionadded">

3.5

</div>

### `conflict_handler`

`ArgumentParser` objects do not allow two actions with the same option string. By default, `ArgumentParser` objects raise an exception if an attempt is made to create an argument with an option string that is already in use:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('-f', '--foo', help='old foo help')
    >>> parser.add_argument('--foo', help='new foo help')
    Traceback (most recent call last):
     ..
    ArgumentError: argument --foo: conflicting option string(s): --foo

Sometimes (e.g. when using [parents](#parents)) it may be useful to simply override any older arguments with the same option string. To get this behavior, the value `'resolve'` can be supplied to the `conflict_handler=` argument of `ArgumentParser`:

    >>> parser = argparse.ArgumentParser(prog='PROG', conflict_handler='resolve')
    >>> parser.add_argument('-f', '--foo', help='old foo help')
    >>> parser.add_argument('--foo', help='new foo help')
    >>> parser.print_help()
    usage: PROG [-h] [-f FOO] [--foo FOO]

    options:
     -h, --help  show this help message and exit
     -f FOO      old foo help
     --foo FOO   new foo help

Note that `ArgumentParser` objects only remove an action if all of its option strings are overridden. So, in the example above, the old `-f/--foo` action is retained as the `-f` action, because only the `--foo` option string was overridden.

### `add_help`

By default, `ArgumentParser` objects add an option which simply displays the parser's help message. If `-h` or `--help` is supplied at the command line, the `!ArgumentParser` help will be printed.

Occasionally, it may be useful to disable the addition of this help option. This can be achieved by passing `False` as the `add_help=` argument to `ArgumentParser`:

    >>> parser = argparse.ArgumentParser(prog='PROG', add_help=False)
    >>> parser.add_argument('--foo', help='foo help')
    >>> parser.print_help()
    usage: PROG [--foo FOO]

    options:
     --foo FOO  foo help

The help option is typically `-h/--help`. The exception to this is if the `prefix_chars=` is specified and does not include `-`, in which case `-h` and `--help` are not valid options. In this case, the first character in `prefix_chars` is used to prefix the help options:

    >>> parser = argparse.ArgumentParser(prog='PROG', prefix_chars='+/')
    >>> parser.print_help()
    usage: PROG [+h]

    options:
      +h, ++help  show this help message and exit

### `exit_on_error`

Normally, when you pass an invalid argument list to the `~ArgumentParser.parse_args` method of an `ArgumentParser`, it will print a *message* to `sys.stderr` and exit with a status code of 2.

If the user would like to catch errors manually, the feature can be enabled by setting `exit_on_error` to `False`:

    >>> parser = argparse.ArgumentParser(exit_on_error=False)
    >>> parser.add_argument('--integers', type=int)
    _StoreAction(option_strings=['--integers'], dest='integers', nargs=None, const=None, default=None, type=<class 'int'>, choices=None, help=None, metavar=None)
    >>> try:
    ...     parser.parse_args('--integers a'.split())
    ... except argparse.ArgumentError:
    ...     print('Catching an argumentError')
    ...
    Catching an argumentError

<div class="versionadded">

3.9

</div>

### `suggest_on_error`

By default, when a user passes an invalid argument choice or subparser name, `ArgumentParser` will exit with error info and provide suggestions for mistyped arguments. The error message will list the permissible argument choices (if specified) or subparser names, along with a "maybe you meant" suggestion if a close match is found. Note that this only applies for arguments when the choices specified are strings:

    >>> parser = argparse.ArgumentParser(suggest_on_error=True)
    >>> parser.add_argument('--action', choices=['debug', 'dryrun'])
    >>> parser.parse_args(['--action', 'debugg'])
    usage: tester.py [-h] [--action {debug,dryrun}]
    tester.py: error: argument --action: invalid choice: 'debugg', maybe you meant 'debug'? (choose from debug, dryrun)

You can disable suggestions by setting `suggest_on_error` to `False`.

<div class="versionadded">

3.14

</div>

<div class="versionchanged">

3.15 Changed default value of `suggest_on_error` from `False` to `True`.

</div>

### `color`

By default, the help message is printed in color using [ANSI escape sequences](https://en.wikipedia.org/wiki/ANSI_escape_code). If you want plain text help messages, you can disable this `in your local
environment <using-on-controlling-color>`, or in the argument parser itself by setting `color` to `False`:

    >>> parser = argparse.ArgumentParser(description='Process some integers.',
    ...                                  color=False)
    >>> parser.add_argument('--action', choices=['sum', 'max'])
    >>> parser.add_argument('integers', metavar='N', type=int, nargs='+',
    ...                     help='an integer for the accumulator')
    >>> parser.parse_args(['--help'])

Note that when `color=True`, colored output depends on both environment variables and terminal capabilities. However, if `color=False`, colored output is always disabled, even if environment variables like `FORCE_COLOR` are set.

<div class="versionadded">

3.14

</div>

To highlight inline code in your description, epilog, or argument `help` text, you can use single or double backticks:

    >>> parser = argparse.ArgumentParser(
    ...     formatter_class=argparse.RawDescriptionHelpFormatter,
    ...     description='Run ``python -m myapp`` to start.',
    ...     epilog='''Examples:
    ...   `python -m myapp --verbose`
    ...   ``python -m myapp --config settings.json``
    ... ''')
    >>> parser.add_argument('--foo', help='set the `foo` value')

When colors are enabled, the text inside backticks will be displayed in a distinct color to help examples stand out. When colors are disabled, backticks are preserved as-is, which is readable in plain text.

<div class="versionadded">

3.15

</div>

## The add_argument() method

<div class="method">

ArgumentParser.add_argument(name or flags..., \*, \[action\], \[nargs\], \[const\], \[default\], \[type\], \[choices\], \[required\], \[help\], \[metavar\], \[dest\], \[deprecated\])

Define how a single command-line argument should be parsed. Each parameter has its own more detailed description below, but in short they are:

- [name or flags](#name-or-flags) - Either a name or a list of option strings, e.g. `'foo'` or `'-f', '--foo'`.
- [action](#action) - The basic type of action to be taken when this argument is encountered at the command line.
- [nargs](#nargs) - The number of command-line arguments that should be consumed.
- [const](#const) - A constant value required by some [action](#action) and [nargs](#nargs) selections.
- [default](#default) - The value produced if the argument is absent from the command line and if it is absent from the namespace object.
- [type](#type) - The type to which the command-line argument should be converted.
- [choices](#choices) - A sequence of the allowable values for the argument.
- [required](#required) - Whether or not the command-line option may be omitted (optionals only).
- [help](#help) - A brief description of what the argument does.
- [metavar](#metavar) - A name for the argument in usage messages.
- [dest](#dest) - The name of the attribute to be added to the object returned by `parse_args`.
- [deprecated](#deprecated) - Whether or not use of the argument is deprecated.

The method returns an `Action` object representing the argument.

</div>

The following sections describe how each of these are used.

### `name` or `flags`

The `~ArgumentParser.add_argument` method must know whether an optional argument, like `-f` or `--foo`, or a positional argument, like a list of filenames, is expected. The first arguments passed to `~ArgumentParser.add_argument` must therefore be either a series of flags, or a simple argument name.

For example, an optional argument could be created like:

    >>> parser.add_argument('-f', '--foo')

while a positional argument could be created like:

    >>> parser.add_argument('bar')

When `~ArgumentParser.parse_args` is called, optional arguments will be identified by the `-` prefix, and the remaining arguments will be assumed to be positional:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('-f', '--foo')
    >>> parser.add_argument('bar')
    >>> parser.parse_args(['BAR'])
    Namespace(bar='BAR', foo=None)
    >>> parser.parse_args(['BAR', '--foo', 'FOO'])
    Namespace(bar='BAR', foo='FOO')
    >>> parser.parse_args(['--foo', 'FOO'])
    usage: PROG [-h] [-f FOO] bar
    PROG: error: the following arguments are required: bar

By default, `!argparse` automatically handles the internal naming and display names of arguments, simplifying the process without requiring additional configuration. As such, you do not need to specify the [dest](#dest) and [metavar](#metavar) parameters. For optional arguments, the [dest](#dest) parameter defaults to the argument name, with underscores `_` replacing hyphens `-`. The [metavar](#metavar) parameter defaults to the upper-cased name. For example:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('--foo-bar')
    >>> parser.parse_args(['--foo-bar', 'FOO-BAR'])
    Namespace(foo_bar='FOO-BAR')
    >>> parser.print_help()
    usage:  [-h] [--foo-bar FOO-BAR]

    optional arguments:
     -h, --help  show this help message and exit
     --foo-bar FOO-BAR

### `action`

`ArgumentParser` objects associate command-line arguments with actions. These actions can do just about anything with the command-line arguments associated with them, though most actions simply add an attribute to the object returned by `~ArgumentParser.parse_args`. The `action` keyword argument specifies how the command-line arguments should be handled. The supplied actions are:

- `'store'` - This just stores the argument's value. This is the default action.

- `'store_const'` - This stores the value specified by the [const](#const) keyword argument; note that the [const](#const) keyword argument defaults to `None`. The `'store_const'` action is most commonly used with optional arguments that specify some sort of flag. For example:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('--foo', action='store_const', const=42)
      >>> parser.parse_args(['--foo'])
      Namespace(foo=42)

- `'store_true'` and `'store_false'` - These are special cases of `'store_const'` that respectively store the values `True` and `False` with default values of `False` and `True`:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('--foo', action='store_true')
      >>> parser.add_argument('--bar', action='store_false')
      >>> parser.add_argument('--baz', action='store_false')
      >>> parser.parse_args('--foo --bar'.split())
      Namespace(foo=True, bar=False, baz=True)

  These defaults apply when the options are absent. They can be overridden by [default](#default) or [argument_default](#argument_default).

- `'append'` - This appends each argument value to a list. It is useful for allowing an option to be specified multiple times. If the default value is a non-empty list, the parsed value for the option will start with the default list's elements and any values from the command line will be appended after those default values. Example usage:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('--foo', action='append', default=['0'])
      >>> parser.parse_args('--foo 1 --foo 2'.split())
      Namespace(foo=['0', '1', '2'])

- `'append_const'` - This appends the value specified by the [const](#const) keyword argument to a list; note that the [const](#const) keyword argument defaults to `None`. The `'append_const'` action is typically useful when multiple arguments need to store constants to the same list. For example:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('--str', dest='types', action='append_const', const=str)
      >>> parser.add_argument('--int', dest='types', action='append_const', const=int)
      >>> parser.parse_args('--str --int'.split())
      Namespace(types=[<class 'str'>, <class 'int'>])

- `'extend'` - This appends each item from a multi-value argument to a list. The `'extend'` action is typically used with the [nargs](#nargs) keyword argument value `'+'` or `'*'`. Note that when [nargs](#nargs) is `None` (the default) or `'?'`, each character of the argument string will be appended to the list. If the default value is a non-empty list, the parsed value for the option will start with the default list's elements and any values from the command line will be appended after those default values. Example usage:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument("--foo", action="extend", nargs="+", type=str,
      ...                     default=["d1"])
      >>> parser.parse_args(["--foo", "f1", "--foo", "f2", "f3", "f4"])
      Namespace(foo=['d1', 'f1', 'f2', 'f3', 'f4'])

  <div class="versionadded">

  3.8

  </div>

- `'count'` - This counts the number of times an argument occurs. For example, this is useful for increasing verbosity levels:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('--verbose', '-v', action='count', default=0)
      >>> parser.parse_args(['-vvv'])
      Namespace(verbose=3)

  Unless explicitly set, the *default* will be `None`. If the default value is a non-zero number, the count starts from that number rather than from zero.

- `'help'` - This prints a complete help message for all the options in the current parser and then exits. By default a help action is automatically added to the parser. See `ArgumentParser` for details of how the output is created.

- `'version'` - This expects a `version=` keyword argument in the `~ArgumentParser.add_argument` call, and prints version information and exits when invoked:

      >>> import argparse
      >>> parser = argparse.ArgumentParser(prog='PROG')
      >>> parser.add_argument('--version', action='version', version='%(prog)s 2.0')
      >>> parser.parse_args(['--version'])
      PROG 2.0

You may also specify an arbitrary action by passing an `Action` subclass (e.g. `BooleanOptionalAction`) or other object that implements the same interface. Only actions that consume command-line arguments (e.g. `'store'`, `'append'`, `'extend'`, or custom actions with non-zero `nargs`) can be used with positional arguments.

The recommended way to create a custom action is to extend `Action`, overriding the `!__call__` method and optionally the `!__init__` and `!format_usage` methods. You can also register custom actions using the `~ArgumentParser.register` method and reference them by their registered name.

An example of a custom action:

    >>> class FooAction(argparse.Action):
    ...     def __init__(self, option_strings, dest, nargs=None, **kwargs):
    ...         if nargs is not None:
    ...             raise ValueError("nargs not allowed")
    ...         super().__init__(option_strings, dest, **kwargs)
    ...     def __call__(self, parser, namespace, values, option_string=None):
    ...         print('%r %r %r' % (namespace, values, option_string))
    ...         setattr(namespace, self.dest, values)
    ...
    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', action=FooAction)
    >>> parser.add_argument('bar', action=FooAction)
    >>> args = parser.parse_args('1 --foo 2'.split())
    Namespace(bar=None, foo=None) '1' None
    Namespace(bar='1', foo=None) '2' '--foo'
    >>> args
    Namespace(bar='1', foo='2')

For more details, see `Action`.

### `nargs`

`ArgumentParser` objects usually associate a single command-line argument with a single action to be taken. The `nargs` keyword argument associates a different number of command-line arguments with a single action. See also `specifying-ambiguous-arguments`. The supported values are:

- `N` (an integer). `N` arguments from the command line will be gathered together into a list. For example:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('--foo', nargs=2)
      >>> parser.add_argument('bar', nargs=1)
      >>> parser.parse_args('c --foo a b'.split())
      Namespace(bar=['c'], foo=['a', 'b'])

  Note that `nargs=1` produces a list of one item. This is different from the default, in which the item is produced by itself.

<div class="index">

single: ? (question mark); in argparse module

</div>

- `'?'`. One argument will be consumed from the command line if possible, and produced as a single item. If no command-line argument is present, the value from [default](#default) will be produced. Note that for optional arguments, there is an additional case - the option string is present but not followed by a command-line argument. In this case the value from [const](#const) will be produced. Some examples to illustrate this:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('--foo', nargs='?', const='c', default='d')
      >>> parser.add_argument('bar', nargs='?', default='d')
      >>> parser.parse_args(['XX', '--foo', 'YY'])
      Namespace(bar='XX', foo='YY')
      >>> parser.parse_args(['XX', '--foo'])
      Namespace(bar='XX', foo='c')
      >>> parser.parse_args([])
      Namespace(bar='d', foo='d')

  One of the more common uses of `nargs='?'` is to allow optional input and output files:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('infile', nargs='?')
      >>> parser.add_argument('outfile', nargs='?')
      >>> parser.parse_args(['input.txt', 'output.txt'])
      Namespace(infile='input.txt', outfile='output.txt')
      >>> parser.parse_args(['input.txt'])
      Namespace(infile='input.txt', outfile=None)
      >>> parser.parse_args([])
      Namespace(infile=None, outfile=None)

<div class="index">

single: \* (asterisk); in argparse module

</div>

- `'*'`. All command-line arguments present are gathered into a list. Note that it generally doesn't make much sense to have more than one positional argument with `nargs='*'`, but multiple optional arguments with `nargs='*'` is possible. For example:

      >>> parser = argparse.ArgumentParser()
      >>> parser.add_argument('--foo', nargs='*')
      >>> parser.add_argument('--bar', nargs='*')
      >>> parser.add_argument('baz', nargs='*')
      >>> parser.parse_args('a b --foo x y --bar 1 2'.split())
      Namespace(bar=['1', '2'], baz=['a', 'b'], foo=['x', 'y'])

<div class="index">

single: + (plus); in argparse module

</div>

- `'+'`. Just like `'*'`, all command-line arguments present are gathered into a list. Additionally, an error message will be generated if there wasn't at least one command-line argument present. For example:

      >>> parser = argparse.ArgumentParser(prog='PROG')
      >>> parser.add_argument('foo', nargs='+')
      >>> parser.parse_args(['a', 'b'])
      Namespace(foo=['a', 'b'])
      >>> parser.parse_args([])
      usage: PROG [-h] foo [foo ...]
      PROG: error: the following arguments are required: foo

If the `nargs` keyword argument is not provided, the number of arguments consumed is determined by the [action](#action). Generally this means a single command-line argument will be consumed and a single item (not a list) will be produced. Actions that do not consume command-line arguments (e.g. `'store_const'`) set `nargs=0`.

### `const`

The `const` argument of `~ArgumentParser.add_argument` is used to hold constant values that are not read from the command line but are required for the various `ArgumentParser` actions. The two most common uses of it are:

- When `~ArgumentParser.add_argument` is called with `action='store_const'` or `action='append_const'`. These actions add the `const` value to one of the attributes of the object returned by `~ArgumentParser.parse_args`. See the [action](#action) description for examples. If `const` is not provided to `~ArgumentParser.add_argument`, it will receive a default value of `None`.
- When `~ArgumentParser.add_argument` is called with option strings (like `-f` or `--foo`) and `nargs='?'`. This creates an optional argument that can be followed by zero or one command-line arguments. When parsing the command line, if the option string is encountered with no command-line argument following it, the value from `const` will be used. See the [nargs](#nargs) description for examples.

<div class="versionchanged">

3.11 `const=None` by default, including when `action='append_const'` or `action='store_const'`.

</div>

### `default`

All optional arguments and some positional arguments may be omitted at the command line. The `default` keyword argument of `~ArgumentParser.add_argument` specifies what value should be used if the command-line argument is not present. For example:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', default=42)
    >>> parser.parse_args(['--foo', '2'])
    Namespace(foo='2')
    >>> parser.parse_args([])
    Namespace(foo=42)

If `default` is omitted, `~ArgumentParser.add_argument` uses a value previously set for the [dest](#dest) with `~ArgumentParser.set_defaults`, or otherwise a non-`None` [argument_default](#argument_default). If neither is set, the action's own default is used.

If the target namespace already has an attribute set, the action *default* will not overwrite it:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', default=42)
    >>> parser.parse_args([], namespace=argparse.Namespace(foo=101))
    Namespace(foo=101)

If the `default` value is a string, the parser parses the value as if it were a command-line argument. In particular, the parser applies any [type](#type) conversion argument, if provided, before setting the attribute on the `Namespace` return value. Otherwise, the parser uses the value as is:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--length', default='10', type=int)
    >>> parser.add_argument('--width', default=10.5, type=int)
    >>> parser.parse_args()
    Namespace(length=10, width=10.5)

For positional arguments with [nargs](#nargs) equal to `?` or `*`, the `default` value is used when no command-line argument was present:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('foo', nargs='?', default=42)
    >>> parser.parse_args(['a'])
    Namespace(foo='a')
    >>> parser.parse_args([])
    Namespace(foo=42)

Because `nargs='*'` gathers any supplied values into a list, an absent positional argument yields an empty list (`[]`). Only a non-`None` *default* overrides this (so `default=None` still gives `[]`).

For [required](#required) arguments, the `default` value is ignored. For example, this applies to positional arguments with [nargs](#nargs) values other than `?` or `*`, or optional arguments marked as `required=True`.

Providing `default=argparse.SUPPRESS` causes no attribute to be added if the command-line argument was not present:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', default=argparse.SUPPRESS)
    >>> parser.parse_args([])
    Namespace()
    >>> parser.parse_args(['--foo', '1'])
    Namespace(foo='1')

### `type`

By default, the parser reads command-line arguments in as simple strings. However, quite often the command-line string should instead be interpreted as another type, such as a `float` or `int`. The `type` keyword for `~ArgumentParser.add_argument` allows any necessary type-checking and type conversions to be performed.

If the [type](#type) keyword is used with the [default](#default) keyword, the type converter is only applied if the default is a string.

The argument to `type` can be a callable that accepts a single string or the name of a registered type (see `~ArgumentParser.register`) If the function raises `ArgumentTypeError`, `TypeError`, or `ValueError`, the exception is caught and a nicely formatted error message is displayed. Other exception types are not handled.

Common built-in types and functions can be used as type converters:

<div class="testcode">

import argparse import pathlib

parser = argparse.ArgumentParser() parser.add_argument('count', type=int) parser.add_argument('distance', type=float) parser.add_argument('street', type=ascii) parser.add_argument('code_point', type=ord) parser.add_argument('datapath', type=pathlib.Path)

</div>

User defined functions can be used as well:

<div class="doctest">

\>\>\> def hyphenated(string): ... return '-'.join(\[word\[:4\] for word in string.casefold().split()\]) ... \>\>\> parser = argparse.ArgumentParser() \>\>\> \_ = parser.add_argument('short_title', type=hyphenated) \>\>\> parser.parse_args(\['"The Tale of Two Cities"'\]) Namespace(short_title='"the-tale-of-two-citi')

</div>

The `bool` function is not recommended as a type converter. All it does is convert empty strings to `False` and non-empty strings to `True`. This is usually not what is desired:

    >>> parser = argparse.ArgumentParser()
    >>> _ = parser.add_argument('--verbose', type=bool)
    >>> parser.parse_args(['--verbose', 'False'])
    Namespace(verbose=True)

See `BooleanOptionalAction` or `action='store_true'` for common alternatives.

In general, the `type` keyword is a convenience that should only be used for simple conversions that can only raise one of the three supported exceptions. Anything with more interesting error-handling or resource management should be done downstream after the arguments are parsed.

For example, JSON or YAML conversions have complex error cases that require better reporting than can be given by the `type` keyword. A `~json.JSONDecodeError` would not be well formatted and a `FileNotFoundError` exception would not be handled at all.

Even `~argparse.FileType` has its limitations for use with the `type` keyword. If one argument uses `~argparse.FileType` and then a subsequent argument fails, an error is reported but the file is not automatically closed. In this case, it would be better to wait until after the parser has run and then use the `with`-statement to manage the files.

For type checkers that simply check against a fixed set of values, consider using the [choices](#choices) keyword instead.

### `choices`

Some command-line arguments should be selected from a restricted set of values. These can be handled by passing a sequence object as the *choices* keyword argument to `~ArgumentParser.add_argument`. When the command line is parsed, argument values will be checked, and an error message will be displayed if the argument was not one of the acceptable values:

    >>> parser = argparse.ArgumentParser(prog='game.py')
    >>> parser.add_argument('move', choices=['rock', 'paper', 'scissors'])
    >>> parser.parse_args(['rock'])
    Namespace(move='rock')
    >>> parser.parse_args(['fire'])
    usage: game.py [-h] {rock,paper,scissors}
    game.py: error: argument move: invalid choice: 'fire' (choose from 'rock',
    'paper', 'scissors')

Any sequence can be passed as the *choices* value, so `list` objects, `tuple` objects, and custom sequences are all supported.

Use of `enum.Enum` is not recommended because it is difficult to control its appearance in usage, help, and error messages.

Note that *choices* are checked after any [type](#type) conversions have been performed, so objects in *choices* should match the [type](#type) specified. This can make *choices* appear unfamiliar in usage, help, or error messages.

To keep *choices* user-friendly, consider a custom type wrapper that converts and formats values, or omit [type](#type) and handle conversion in your application code.

Formatted choices override the default *metavar* which is normally derived from *dest*. This is usually what you want because the user never sees the *dest* parameter. If this display isn't desirable (perhaps because there are many choices), just specify an explicit [metavar](#metavar).

### `required`

In general, the `!argparse` module assumes that flags like `-f` and `--bar` indicate *optional* arguments, which can always be omitted at the command line. To make an option *required*, `True` can be specified for the `required=` keyword argument to `~ArgumentParser.add_argument`:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', required=True)
    >>> parser.parse_args(['--foo', 'BAR'])
    Namespace(foo='BAR')
    >>> parser.parse_args([])
    usage: [-h] --foo FOO
    : error: the following arguments are required: --foo

As the example shows, if an option is marked as `required`, `~ArgumentParser.parse_args` will report an error if that option is not present at the command line.

> [!NOTE]
> Required options are generally considered bad form because users expect *options* to be *optional*, and thus they should be avoided when possible.

### `help`

The `help` value is a string containing a brief description of the argument. When a user requests help (usually by using `-h` or `--help` at the command line), these `help` descriptions will be displayed with each argument.

The `help` strings can include various format specifiers to avoid repetition of things like the program name or the argument [default](#default). The available specifiers include the program name, `%(prog)s` and most keyword arguments to `~ArgumentParser.add_argument`, e.g. `%(default)s`, `%(type)s`, etc.:

    >>> parser = argparse.ArgumentParser(prog='frobble')
    >>> parser.add_argument('bar', nargs='?', type=int, default=42,
    ...                     help='the bar to %(prog)s (default: %(default)s)')
    >>> parser.print_help()
    usage: frobble [-h] [bar]

    positional arguments:
     bar     the bar to frobble (default: 42)

    options:
     -h, --help  show this help message and exit

As the help string supports %-formatting, if you want a literal `%` to appear in the help string, you must escape it as `%%`.

`!argparse` supports silencing the help entry for certain options, by setting the `help` value to `argparse.SUPPRESS`:

    >>> parser = argparse.ArgumentParser(prog='frobble')
    >>> parser.add_argument('--foo', help=argparse.SUPPRESS)
    >>> parser.print_help()
    usage: frobble [-h]

    options:
      -h, --help  show this help message and exit

### `metavar`

When `ArgumentParser` generates help messages, it needs some way to refer to each expected argument. By default, `!ArgumentParser` objects use the [dest](#dest) value as the "name" of each object. By default, for positional argument actions, the [dest](#dest) value is used directly, and for optional argument actions, the [dest](#dest) value is uppercased. So, a single positional argument with `dest='bar'` will be referred to as `bar`. A single optional argument `--foo` that should be followed by a single command-line argument will be referred to as `FOO`. An example:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo')
    >>> parser.add_argument('bar')
    >>> parser.parse_args('X --foo Y'.split())
    Namespace(bar='X', foo='Y')
    >>> parser.print_help()
    usage:  [-h] [--foo FOO] bar

    positional arguments:
     bar

    options:
     -h, --help  show this help message and exit
     --foo FOO

An alternative name can be specified with `metavar`:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', metavar='YYY')
    >>> parser.add_argument('bar', metavar='XXX')
    >>> parser.parse_args('X --foo Y'.split())
    Namespace(bar='X', foo='Y')
    >>> parser.print_help()
    usage:  [-h] [--foo YYY] XXX

    positional arguments:
     XXX

    options:
     -h, --help  show this help message and exit
     --foo YYY

Note that `metavar` only changes the *displayed* name - the name of the attribute on the `~ArgumentParser.parse_args` object is still determined by the [dest](#dest) value.

Different values of `nargs` may cause the metavar to be used multiple times. Providing a tuple to `metavar` specifies a different display for each of the arguments:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('-x', nargs=2)
    >>> parser.add_argument('--foo', nargs=2, metavar=('bar', 'baz'))
    >>> parser.print_help()
    usage: PROG [-h] [-x X X] [--foo bar baz]

    options:
     -h, --help     show this help message and exit
     -x X X
     --foo bar baz

### `dest`

Most `ArgumentParser` actions add some value as an attribute of the object returned by `~ArgumentParser.parse_args`. The name of this attribute is determined by the `dest` keyword argument of `~ArgumentParser.add_argument`. For positional argument actions, `dest` is normally supplied as the first argument to `~ArgumentParser.add_argument`:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('bar')
    >>> parser.parse_args(['XXX'])
    Namespace(bar='XXX')

For optional argument actions, the value of `dest` is normally inferred from the option strings. `ArgumentParser` generates the value of `dest` by taking the first double-dash long option string and stripping away the initial `-` characters. If no double-dash long option strings were supplied, `dest` will be derived from the first single-dash long option string by stripping the initial `-` character. If no long option strings were supplied, `dest` will be derived from the first short option string by stripping the initial `-` character. Any internal `-` characters will be converted to `_` characters to make sure the string is a valid attribute name. The examples below illustrate this behavior:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('-f', '--foo-bar', '--foo')
    >>> parser.add_argument('-q', '-quz')
    >>> parser.add_argument('-x', '-y')
    >>> parser.parse_args('-f 1 -q 2 -x 3'.split())
    Namespace(foo_bar='1', quz='2', x='3')
    >>> parser.parse_args('--foo 1 -quz 2 -y 3'.split())
    Namespace(foo_bar='1', quz='2', x='2')

`dest` allows a custom attribute name to be provided:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', dest='bar')
    >>> parser.parse_args('--foo XXX'.split())
    Namespace(bar='XXX')

Multiple arguments may share the same `dest`. By default, the value from the last such argument given on the command line wins. Use `action='append'` to collect values from all of them into a list instead. For conflicting *option strings* rather than `dest` names, see [conflict_handler](#conflict_handler).

<div class="versionchanged">

3.15 Single-dash long option now takes precedence over short options.

</div>

### `deprecated`

During a project's lifetime, some arguments may need to be removed from the command line. Before removing them, you should inform your users that the arguments are deprecated and will be removed. The `deprecated` keyword argument of `~ArgumentParser.add_argument`, which defaults to `False`, specifies if the argument is deprecated and will be removed in the future. For arguments, if `deprecated` is `True`, then a warning will be printed to `sys.stderr` when the argument is used:

    >>> import argparse
    >>> parser = argparse.ArgumentParser(prog='snake.py')
    >>> parser.add_argument('--legs', default=0, type=int, deprecated=True)
    >>> parser.parse_args([])
    Namespace(legs=0)
    >>> parser.parse_args(['--legs', '4'])  # doctest: +SKIP
    snake.py: warning: option '--legs' is deprecated
    Namespace(legs=4)

<div class="versionadded">

3.13

</div>

### Action classes

`!Action` classes implement the Action API, a callable which returns a callable which processes arguments from the command-line. Any object which follows this API may be passed as the `action` parameter to `~ArgumentParser.add_argument`.

<div class="Action(option_strings, dest, nargs=None, const=None, default=None, \ type=None, choices=None, required=False, help=None, \ metavar=None)">

`!Action` objects are used by an `ArgumentParser` to represent the information needed to parse a single argument from one or more strings from the command line. The `!Action` class must accept the two positional arguments plus any keyword arguments passed to `ArgumentParser.add_argument` except for the `action` itself.

Instances of `!Action` (or return value of any callable to the `action` parameter) should have attributes `!dest`, `!option_strings`, `!default`, `!type`, `!required`, `!help`, etc. defined. The easiest way to ensure these attributes are defined is to call `!Action.__init__`.

<div class="method">

\_\_call\_\_(parser, namespace, values, option_string=None)

`!Action` instances should be callable, so subclasses must override the `!__call__` method, which should accept four parameters:

- *parser* - The `ArgumentParser` object which contains this action.
- *namespace* - The `Namespace` object that will be returned by `~ArgumentParser.parse_args`. Most actions add an attribute to this object using `setattr`.
- *values* - The associated command-line arguments, with any type conversions applied. Type conversions are specified with the [type](#type) keyword argument to `~ArgumentParser.add_argument`.
- *option_string* - The option string that was used to invoke this action. The `option_string` argument is optional, and will be absent if the action is associated with a positional argument.

The `!__call__` method may perform arbitrary actions, but will typically set attributes on the `namespace` based on `dest` and `values`.

</div>

<div class="method">

format_usage()

`!Action` subclasses can define a `!format_usage` method that takes no argument and return a string which will be used when printing the usage of the program. If such method is not provided, a sensible default will be used.

</div>

</div>

<div class="BooleanOptionalAction">

A subclass of `Action` for handling boolean flags with positive and negative options. Adding a single argument such as `--foo` automatically creates both `--foo` and `--no-foo` options, storing `True` and `False` respectively:

    >>> import argparse
    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', action=argparse.BooleanOptionalAction)
    >>> parser.parse_args(['--no-foo'])
    Namespace(foo=False)

Single-dash long options are also supported. For example, negative option `-nofoo` is automatically added for positive option `-foo`. But no additional options are added for short options such as `-f`.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.15 Added support for single-dash options.

Added support for alternate [prefix_chars](#prefix_chars).

</div>

</div>

## The parse_args() method

<div class="method">

ArgumentParser.parse_args(args=None, namespace=None)

Convert argument strings to objects and assign them as attributes of the namespace. Return the populated namespace.

Previous calls to `add_argument` determine exactly what objects are created and how they are assigned. See the documentation for `!add_argument` for details.

- [args](#args) - List of strings to parse. The default is taken from `sys.argv`.
- [namespace](#namespace) - An object to take the attributes. The default is a new empty `Namespace` object.

</div>

### Option value syntax

The `~ArgumentParser.parse_args` method supports several ways of specifying the value of an option (if it takes one). In the simplest case, the option and its value are passed as two separate arguments:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('-x')
    >>> parser.add_argument('--foo')
    >>> parser.parse_args(['-x', 'X'])
    Namespace(foo=None, x='X')
    >>> parser.parse_args(['--foo', 'FOO'])
    Namespace(foo='FOO', x=None)

For long options (options with names longer than a single character), the option and value can also be passed as a single command-line argument, using `=` to separate them:

    >>> parser.parse_args(['--foo=FOO'])
    Namespace(foo='FOO', x=None)

For short options (options only one character long), the option and its value can be concatenated:

    >>> parser.parse_args(['-xX'])
    Namespace(foo=None, x='X')

Several short options can be joined together, using only a single `-` prefix, as long as only the last option (or none of them) requires a value:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('-x', action='store_true')
    >>> parser.add_argument('-y', action='store_true')
    >>> parser.add_argument('-z')
    >>> parser.parse_args(['-xyzZ'])
    Namespace(x=True, y=True, z='Z')

### Invalid arguments

While parsing the command line, `~ArgumentParser.parse_args` checks for a variety of errors, including ambiguous options, invalid types, invalid options, wrong number of positional arguments, etc. When it encounters such an error, it exits and prints the error along with a usage message:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('--foo', type=int)
    >>> parser.add_argument('bar', nargs='?')

    >>> # invalid type
    >>> parser.parse_args(['--foo', 'spam'])
    usage: PROG [-h] [--foo FOO] [bar]
    PROG: error: argument --foo: invalid int value: 'spam'

    >>> # invalid option
    >>> parser.parse_args(['--bar'])
    usage: PROG [-h] [--foo FOO] [bar]
    PROG: error: unrecognized arguments: --bar

    >>> # wrong number of arguments
    >>> parser.parse_args(['spam', 'badger'])
    usage: PROG [-h] [--foo FOO] [bar]
    PROG: error: unrecognized arguments: badger

### Arguments containing `-`

The `~ArgumentParser.parse_args` method attempts to give errors whenever the user has clearly made a mistake, but some situations are inherently ambiguous. For example, the command-line argument `-1` could either be an attempt to specify an option or an attempt to provide a positional argument. The `~ArgumentParser.parse_args` method is cautious here: positional arguments may only begin with `-` if they look like negative numbers and there are no options in the parser that look like negative numbers:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('-x')
    >>> parser.add_argument('foo', nargs='?')

    >>> # no negative number options, so -1 is a positional argument
    >>> parser.parse_args(['-x', '-1'])
    Namespace(foo=None, x='-1')

    >>> # no negative number options, so -1 and -5 are positional arguments
    >>> parser.parse_args(['-x', '-1', '-5'])
    Namespace(foo='-5', x='-1')

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('-1', dest='one')
    >>> parser.add_argument('foo', nargs='?')

    >>> # negative number options present, so -1 is an option
    >>> parser.parse_args(['-1', 'X'])
    Namespace(foo=None, one='X')

    >>> # negative number options present, so -2 is an option
    >>> parser.parse_args(['-2'])
    usage: PROG [-h] [-1 ONE] [foo]
    PROG: error: unrecognized arguments: -2

    >>> # negative number options present, so both -1s are options
    >>> parser.parse_args(['-1', '-1'])
    usage: PROG [-h] [-1 ONE] [foo]
    PROG: error: argument -1: expected one argument

If you have positional arguments that must begin with `-` and don't look like negative numbers, you can insert the pseudo-argument `'--'` which tells `~ArgumentParser.parse_args` that everything after that is a positional argument:

    >>> parser.parse_args(['--', '-f'])
    Namespace(foo='-f', one=None)

See also `the argparse howto on ambiguous arguments <specifying-ambiguous-arguments>` for more details.

<div class="versionchanged">

3.14 Negative-number matching was expanded to include numbers in scientific notation (`-2.5e-6`), numbers containing underscores (`-1_234.5`), and complex numbers (`-1.2e-3j`).

</div>

### Argument abbreviations (prefix matching)

The `~ArgumentParser.parse_args` method `by default <allow_abbrev>` allows long options to be abbreviated to a prefix, if the abbreviation is unambiguous (the prefix matches a unique option):

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('-bacon')
    >>> parser.add_argument('-badger')
    >>> parser.parse_args('-bac MMM'.split())
    Namespace(bacon='MMM', badger=None)
    >>> parser.parse_args('-bad WOOD'.split())
    Namespace(bacon=None, badger='WOOD')
    >>> parser.parse_args('-ba BA'.split())
    usage: PROG [-h] [-bacon BACON] [-badger BADGER]
    PROG: error: ambiguous option: -ba could match -badger, -bacon

An error is produced for arguments that could produce more than one options. This feature can be disabled by setting `allow_abbrev` to `False`.

### Beyond `sys.argv`

Sometimes it may be useful to have an `ArgumentParser` parse arguments other than those of `sys.argv`. This can be accomplished by passing a list of strings to `~ArgumentParser.parse_args`. This is useful for testing at the interactive prompt:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument(
    ...     'integers', metavar='int', type=int, choices=range(10),
    ...     nargs='+', help='an integer in the range 0..9')
    >>> parser.add_argument(
    ...     '--sum', dest='accumulate', action='store_const', const=sum,
    ...     default=max, help='sum the integers (default: find the max)')
    >>> parser.parse_args(['1', '2', '3', '4'])
    Namespace(accumulate=<built-in function max>, integers=[1, 2, 3, 4])
    >>> parser.parse_args(['1', '2', '3', '4', '--sum'])
    Namespace(accumulate=<built-in function sum>, integers=[1, 2, 3, 4])

### The Namespace object

<div class="Namespace">

Simple class used by default by `~ArgumentParser.parse_args` to create an object holding attributes and return it.

`!Namespace` objects support `copy.replace`, which returns a copy of the object with the specified attributes replaced.

<div class="versionchanged">

3.16 Added support for `copy.replace`.

</div>

This class is deliberately simple, just an `object` subclass with a readable string representation. If you prefer to have dict-like view of the attributes, you can use the standard Python idiom, `vars`:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo')
    >>> args = parser.parse_args(['--foo', 'BAR'])
    >>> vars(args)
    {'foo': 'BAR'}

It may also be useful to have an `ArgumentParser` assign attributes to an already existing object, rather than a new `Namespace` object. This can be achieved by specifying the `namespace=` keyword argument:

    >>> class C:
    ...     pass
    ...
    >>> c = C()
    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo')
    >>> parser.parse_args(args=['--foo', 'BAR'], namespace=c)
    >>> c.foo
    'BAR'

</div>

## Other utilities

### Subcommands

<div class="method">

ArgumentParser.add_subparsers(\*, \[title\], \[description\], \[prog\], \[parser_class\], \[action\], \[dest\], \[required\], \[help\], \[metavar\])

Many programs split up their functionality into a number of subcommands, for example, the `svn` program can invoke subcommands like `svn checkout`, `svn update`, and `svn commit`. Splitting up functionality this way can be a particularly good idea when a program performs several different functions which require different kinds of command-line arguments. `ArgumentParser` supports the creation of such subcommands with the `!add_subparsers` method. The `!add_subparsers` method is normally called with no arguments and returns a special action object. This object has a single method, `~_SubParsersAction.add_parser`, which takes a command name and any `!ArgumentParser` constructor arguments, and returns an `!ArgumentParser` object that can be modified as usual.

Description of parameters:

- *title* - title for the sub-parser group in help output; by default "subcommands" if description is provided, otherwise uses title for positional arguments
- *description* - description for the sub-parser group in help output, by default `None`
- *prog* - usage information that will be displayed with subcommand help, by default the name of the program and any positional arguments before the subparser argument
- *parser_class* - class which will be used to create sub-parser instances, by default the class of the current parser (e.g. `ArgumentParser`)
- [action](#action) - the basic type of action to be taken when this argument is encountered at the command line
- [dest](#dest) - name of the attribute under which subcommand name will be stored; by default `None` and no value is stored
- [required](#required) - Whether or not a subcommand must be provided, by default `False` (added in 3.7)
- [help](#help) - help for sub-parser group in help output, by default `None`
- [metavar](#metavar) - placeholder shown in usage and the subparsers' help entry instead of `{cmd1, cmd2, ...}`; by default `None`. Individual subcommand names are unchanged.

Some example usage:

    >>> # create the top-level parser
    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> parser.add_argument('--foo', action='store_true', help='foo help')
    >>> subparsers = parser.add_subparsers(help='subcommand help')
    >>>
    >>> # create the parser for the "a" command
    >>> parser_a = subparsers.add_parser('a', help='a help')
    >>> parser_a.add_argument('bar', type=int, help='bar help')
    >>>
    >>> # create the parser for the "b" command
    >>> parser_b = subparsers.add_parser('b', help='b help')
    >>> parser_b.add_argument('--baz', choices=('X', 'Y', 'Z'), help='baz help')
    >>>
    >>> # parse some argument lists
    >>> parser.parse_args(['a', '12'])
    Namespace(bar=12, foo=False)
    >>> parser.parse_args(['--foo', 'b', '--baz', 'Z'])
    Namespace(baz='Z', foo=True)

Note that the object returned by `~ArgumentParser.parse_args` will only contain attributes for the main parser and the subparser that was selected by the command line (and not any other subparsers). So in the example above, when the `a` command is specified, only the `foo` and `bar` attributes are present, and when the `b` command is specified, only the `foo` and `baz` attributes are present.

If a subparser defines an argument with the same `dest` as the parent parser, the two share a single namespace attribute, so the parent's value won't be retained. Users should give them distinct `dest` values to keep both.

Similarly, when a help message is requested from a subparser, only the help for that particular parser will be printed. The help message will not include parent parser or sibling parser messages. (A help message for each subparser command, however, can be given by supplying the `help=` argument to `~_SubParsersAction.add_parser` as above.)

    >>> parser.parse_args(['--help'])
    usage: PROG [-h] [--foo] {a,b} ...

    positional arguments:
      {a,b}   subcommand help
        a     a help
        b     b help

    options:
      -h, --help  show this help message and exit
      --foo   foo help

    >>> parser.parse_args(['a', '--help'])
    usage: PROG a [-h] bar

    positional arguments:
      bar     bar help

    options:
      -h, --help  show this help message and exit

    >>> parser.parse_args(['b', '--help'])
    usage: PROG b [-h] [--baz {X,Y,Z}]

    options:
      -h, --help     show this help message and exit
      --baz {X,Y,Z}  baz help

The `~ArgumentParser.add_subparsers` method also supports `title` and `description` keyword arguments. When either is present, the subparser's commands will appear in their own group in the help output. For example:

    >>> parser = argparse.ArgumentParser()
    >>> subparsers = parser.add_subparsers(title='subcommands',
    ...                                    description='valid subcommands',
    ...                                    help='additional help')
    >>> subparsers.add_parser('foo')
    >>> subparsers.add_parser('bar')
    >>> parser.parse_args(['-h'])
    usage:  [-h] {foo,bar} ...

    options:
      -h, --help  show this help message and exit

    subcommands:
      valid subcommands

      {foo,bar}   additional help

One particularly effective way of handling subcommands is to combine the use of the `~ArgumentParser.add_subparsers` method with calls to `~ArgumentParser.set_defaults` so that each subparser knows which Python function it should execute. For example:

    >>> # subcommand functions
    >>> def foo(args):
    ...     print(args.x * args.y)
    ...
    >>> def bar(args):
    ...     print('((%s))' % args.z)
    ...
    >>> # create the top-level parser
    >>> parser = argparse.ArgumentParser()
    >>> subparsers = parser.add_subparsers(required=True)
    >>>
    >>> # create the parser for the "foo" command
    >>> parser_foo = subparsers.add_parser('foo')
    >>> parser_foo.add_argument('-x', type=int, default=1)
    >>> parser_foo.add_argument('y', type=float)
    >>> parser_foo.set_defaults(func=foo)
    >>>
    >>> # create the parser for the "bar" command
    >>> parser_bar = subparsers.add_parser('bar')
    >>> parser_bar.add_argument('z')
    >>> parser_bar.set_defaults(func=bar)
    >>>
    >>> # parse the args and call whatever function was selected
    >>> args = parser.parse_args('foo 1 -x 2'.split())
    >>> args.func(args)
    2.0
    >>>
    >>> # parse the args and call whatever function was selected
    >>> args = parser.parse_args('bar XYZYX'.split())
    >>> args.func(args)
    ((XYZYX))

This way, you can let `~ArgumentParser.parse_args` do the job of calling the appropriate function after argument parsing is complete. Associating functions with actions like this is typically the easiest way to handle the different actions for each of your subparsers. However, if it is necessary to check the name of the subparser that was invoked, the `dest` keyword argument to the `~ArgumentParser.add_subparsers` call will work:

    >>> parser = argparse.ArgumentParser()
    >>> subparsers = parser.add_subparsers(dest='subparser_name')
    >>> subparser1 = subparsers.add_parser('1')
    >>> subparser1.add_argument('-x')
    >>> subparser2 = subparsers.add_parser('2')
    >>> subparser2.add_argument('y')
    >>> parser.parse_args(['2', 'frobble'])
    Namespace(subparser_name='2', y='frobble')

<div class="versionchanged">

3.7 New *required* keyword-only parameter.

</div>

<div class="versionchanged">

3.14 Subparser's *prog* is no longer affected by a custom usage message in the main parser.

</div>

</div>

<div class="method">

<span id="subparsersaction.add_parser">SubParsersAction.add_parser</span>(name, *, help=None, aliases=None, deprecated=False,*\*kwargs)

Create and return a new `ArgumentParser` object for the subcommand *name*.

The *name* argument is the name of the sub-command.

The *help* argument provides a short description for this sub-command.

The *aliases* argument allows providing alternative names for this sub-command. For example:

    >>> parser = argparse.ArgumentParser()
    >>> subparsers = parser.add_subparsers()
    >>> checkout = subparsers.add_parser('checkout', aliases=['co'])
    >>> checkout.add_argument('foo')
    >>> parser.parse_args(['co', 'bar'])
    Namespace(foo='bar')

The *deprecated* argument, if `True`, marks the sub-command as deprecated and will issue a warning when used. For example:

    >>> parser = argparse.ArgumentParser(prog='chicken.py')
    >>> subparsers = parser.add_subparsers()
    >>> fly = subparsers.add_parser('fly', deprecated=True)
    >>> args = parser.parse_args(['fly'])
    chicken.py: warning: command 'fly' is deprecated
    Namespace()

All other keyword arguments are passed directly to the `!ArgumentParser` constructor.

<div class="versionadded">

3.13 Added the *deprecated* parameter.

</div>

</div>

### FileType objects

<div class="FileType(mode='r', bufsize=-1, encoding=None, errors=None)">

The `FileType` factory creates objects that can be passed to the type argument of `ArgumentParser.add_argument`. Arguments that have `FileType` objects as their type will open command-line arguments as files with the requested modes, buffer sizes, encodings and error handling (see the `open` function for more details):

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--raw', type=argparse.FileType('wb', 0))
    >>> parser.add_argument('out', type=argparse.FileType('w', encoding='UTF-8'))
    >>> parser.parse_args(['--raw', 'raw.dat', 'file.txt'])
    Namespace(out=<_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'>, raw=<_io.FileIO name='raw.dat' mode='wb'>)

FileType objects understand the pseudo-argument `'-'` and automatically convert this into `sys.stdin` for readable `FileType` objects and `sys.stdout` for writable `FileType` objects:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('infile', type=argparse.FileType('r'))
    >>> parser.parse_args(['-'])
    Namespace(infile=<_io.TextIOWrapper name='<stdin>' encoding='UTF-8'>)

> [!NOTE]
> If one argument uses *FileType* and then a subsequent argument fails, an error is reported but the file is not automatically closed. This can also clobber the output files. In this case, it would be better to wait until after the parser has run and then use the `with`-statement to manage the files.

<div class="versionchanged">

3.4 Added the *encoding* and *errors* parameters.

</div>

<div class="deprecated">

3.14

</div>

</div>

### Argument groups

<div class="method">

ArgumentParser.add_argument_group(title=None, description=None, \*, \[argument_default\], \[conflict_handler\])

By default, `ArgumentParser` groups command-line arguments into "positional arguments" and "options" when displaying help messages. When there is a better conceptual grouping of arguments than this default one, appropriate groups can be created using the `!add_argument_group` method:

    >>> parser = argparse.ArgumentParser(prog='PROG', add_help=False)
    >>> group = parser.add_argument_group('group')
    >>> group.add_argument('--foo', help='foo help')
    >>> group.add_argument('bar', help='bar help')
    >>> parser.print_help()
    usage: PROG [--foo FOO] bar

    group:
      bar    bar help
      --foo FOO  foo help

The `add_argument_group` method returns an argument group object which has an `~ArgumentParser.add_argument` method just like a regular `ArgumentParser`. When an argument is added to the group, the parser treats it just like a normal argument, but displays the argument in a separate group for help messages. The `!add_argument_group` method accepts *title* and *description* arguments which can be used to customize this display:

    >>> parser = argparse.ArgumentParser(prog='PROG', add_help=False)
    >>> group1 = parser.add_argument_group('group1', 'group1 description')
    >>> group1.add_argument('foo', help='foo help')
    >>> group2 = parser.add_argument_group('group2', 'group2 description')
    >>> group2.add_argument('--bar', help='bar help')
    >>> parser.print_help()
    usage: PROG [--bar BAR] foo

    group1:
      group1 description

      foo    foo help

    group2:
      group2 description

      --bar BAR  bar help

The optional, keyword-only parameters [argument_default](#argument_default) and [conflict_handler](#conflict_handler) allow for finer-grained control of the behavior of the argument group. These parameters have the same meaning as in the `ArgumentParser` constructor, but apply specifically to the argument group rather than the entire parser.

Note that any arguments not in your user-defined groups will end up back in the usual "positional arguments" and "optional arguments" sections.

Within each argument group, arguments are displayed in help output in the order in which they are added.

<div class="deprecated-removed">

3.11 3.14 Calling `add_argument_group` on an argument group now raises an exception. This nesting was never supported, often failed to work correctly, and was unintentionally exposed through inheritance.

</div>

<div class="deprecated">

3.14 Passing [prefix_chars](#prefix_chars) to `add_argument_group` is now deprecated.

</div>

</div>

<div class="ArgumentGroup">

The type of objects returned by `ArgumentParser.add_argument_group`. Its `~ArgumentParser.add_argument` and `~ArgumentParser.add_mutually_exclusive_group` methods work as on `ArgumentParser`. Use `~ArgumentParser.add_argument_group` to create instances. Only the methods documented here are part of the public API; other attributes and methods are implementation details.

<div class="versionadded">

next The private `_ArgumentGroup` name remains as an alias.

</div>

</div>

### Mutual exclusion

<div class="method">

ArgumentParser.add_mutually_exclusive_group(required=False)

Create a mutually exclusive group. `!argparse` will make sure that only one of the arguments in the mutually exclusive group was present on the command line:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> group = parser.add_mutually_exclusive_group()
    >>> group.add_argument('--foo', action='store_true')
    >>> group.add_argument('--bar', action='store_false')
    >>> parser.parse_args(['--foo'])
    Namespace(bar=True, foo=True)
    >>> parser.parse_args(['--bar'])
    Namespace(bar=False, foo=False)
    >>> parser.parse_args(['--foo', '--bar'])
    usage: PROG [-h] [--foo | --bar]
    PROG: error: argument --bar: not allowed with argument --foo

The `add_mutually_exclusive_group` method also accepts a *required* argument, to indicate that at least one of the mutually exclusive arguments is required:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> group = parser.add_mutually_exclusive_group(required=True)
    >>> group.add_argument('--foo', action='store_true')
    >>> group.add_argument('--bar', action='store_false')
    >>> parser.parse_args([])
    usage: PROG [-h] (--foo | --bar)
    PROG: error: one of the following arguments is required: --foo, --bar

Note that currently mutually exclusive argument groups do not support the *title* and *description* arguments of `~ArgumentParser.add_argument_group`. However, a mutually exclusive group can be added to an argument group that has a title and description. For example:

    >>> parser = argparse.ArgumentParser(prog='PROG')
    >>> group = parser.add_argument_group('Group title', 'Group description')
    >>> exclusive_group = group.add_mutually_exclusive_group(required=True)
    >>> exclusive_group.add_argument('--foo', help='foo help')
    >>> exclusive_group.add_argument('--bar', help='bar help')
    >>> parser.print_help()
    usage: PROG [-h] (--foo FOO | --bar BAR)

    options:
      -h, --help  show this help message and exit

    Group title:
      Group description

      --foo FOO   foo help
      --bar BAR   bar help

<div class="deprecated-removed">

3.11 3.14 Calling `add_argument_group` or `add_mutually_exclusive_group` on a mutually exclusive group now raises an exception. This nesting was never supported, often failed to work correctly, and was unintentionally exposed through inheritance.

</div>

</div>

<div class="MutuallyExclusiveGroup">

The type of objects returned by `ArgumentParser.add_mutually_exclusive_group`. Its `~ArgumentParser.add_argument` method works as on `ArgumentParser`. Use `~ArgumentParser.add_mutually_exclusive_group` to create instances. Only the `add_argument` method documented here is part of the public API; other attributes and methods are implementation details.

<div class="versionadded">

next The private `_MutuallyExclusiveGroup` name remains as an alias.

</div>

</div>

### Parser defaults

<div class="method">

ArgumentParser.set_defaults(\*\*kwargs)

Most of the time, the attributes of the object returned by `parse_args` will be fully determined by inspecting the command-line arguments and the argument actions. `set_defaults` allows some additional attributes that are determined without any inspection of the command line to be added:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('foo', type=int)
    >>> parser.set_defaults(bar=42, baz='badger')
    >>> parser.parse_args(['736'])
    Namespace(bar=42, baz='badger', foo=736)

Note that defaults can be set at both the parser level using `set_defaults` and at the argument level using `add_argument`. If both are called for the same argument, the last default set for an argument is used:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', default='bar')
    >>> parser.set_defaults(foo='spam')
    >>> parser.parse_args([])
    Namespace(foo='spam')

Parser-level defaults can be particularly useful when working with multiple parsers. See the `~ArgumentParser.add_subparsers` method for an example of this type.

</div>

<div class="method">

ArgumentParser.get_default(dest)

Get the default value for a namespace attribute, as set by either `~ArgumentParser.add_argument` or by `~ArgumentParser.set_defaults`:

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', default='badger')
    >>> parser.get_default('foo')
    'badger'

</div>

### Printing help

In most typical applications, `~ArgumentParser.parse_args` will take care of formatting and printing any usage or error messages. However, several formatting methods are available:

<div class="method">

ArgumentParser.print_usage(file=None)

Print a brief description of how the `ArgumentParser` should be invoked on the command line. If *file* is `None`, `sys.stdout` is assumed.

</div>

<div class="method">

ArgumentParser.print_help(file=None)

Print a help message, including the program usage and information about the arguments registered with the `ArgumentParser`. If *file* is `None`, `sys.stdout` is assumed.

</div>

There are also variants of these methods that simply return a string instead of printing it:

<div class="method">

ArgumentParser.format_usage()

Return a string containing a brief description of how the `ArgumentParser` should be invoked on the command line.

</div>

<div class="method">

ArgumentParser.format_help()

Return a string containing a help message, including the program usage and information about the arguments registered with the `ArgumentParser`.

</div>

### Partial parsing

<div class="method">

ArgumentParser.parse_known_args(args=None, namespace=None)

Sometimes a script only needs to handle a specific set of command-line arguments, leaving any unrecognized arguments for another script or program. In these cases, the `~ArgumentParser.parse_known_args` method can be useful.

This method works similarly to `~ArgumentParser.parse_args`, but it does not raise an error for extra, unrecognized arguments. Instead, it parses the known arguments and returns a two item tuple that contains the populated namespace and the list of any unrecognized arguments.

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo', action='store_true')
    >>> parser.add_argument('bar')
    >>> parser.parse_known_args(['--foo', '--badger', 'BAR', 'spam'])
    (Namespace(bar='BAR', foo=True), ['--badger', 'spam'])

</div>

> [!WARNING]
> `Prefix matching <prefix-matching>` rules apply to `~ArgumentParser.parse_known_args`. The parser may consume an option even if it's just a prefix of one of its known options, instead of leaving it in the remaining arguments list.

### Customizing file parsing

<div class="method">

ArgumentParser.convert_arg_line_to_args(arg_line)

Arguments that are read from a file (see the *fromfile_prefix_chars* keyword argument to the `ArgumentParser` constructor) are read one argument per line. `convert_arg_line_to_args` can be overridden for fancier reading.

This method takes a single argument *arg_line* which is a string read from the argument file. It returns a list of arguments parsed from this string. The method is called once per line read from the argument file, in order.

A useful override of this method is one that treats each space-separated word as an argument. The following example demonstrates how to do this:

    class MyArgumentParser(argparse.ArgumentParser):
        def convert_arg_line_to_args(self, arg_line):
            return arg_line.split()

Note that with this override an argument can no longer contain spaces, since each space-separated word becomes a separate argument.

</div>

### Exiting methods

<div class="method">

ArgumentParser.exit(status=0, message=None)

This method terminates the program, exiting with the specified *status* and, if given, it prints a *message* to `sys.stderr` before that. The user can override this method to handle these steps differently:

    class ErrorCatchingArgumentParser(argparse.ArgumentParser):
        def exit(self, status=0, message=None):
            if status:
                raise Exception(f'Exiting because of an error: {message}')
            exit(status)

</div>

<div class="method">

ArgumentParser.error(message)

This method prints a usage message, including the *message*, to `sys.stderr` and terminates the program with a status code of 2.

</div>

### Intermixed parsing

<div class="method">

ArgumentParser.parse_intermixed_args(args=None, namespace=None)

</div>

<div class="method">

ArgumentParser.parse_known_intermixed_args(args=None, namespace=None)

A number of Unix commands allow the user to intermix optional arguments with positional arguments. The `~ArgumentParser.parse_intermixed_args` and `~ArgumentParser.parse_known_intermixed_args` methods support this parsing style.

These parsers do not support all the `!argparse` features, and will raise exceptions if unsupported features are used. In particular, subparsers, and mutually exclusive groups that include both optionals and positionals are not supported.

The following example shows the difference between `~ArgumentParser.parse_known_args` and `~ArgumentParser.parse_intermixed_args`: the former returns `['2', '3']` as unparsed arguments, while the latter collects all the positionals into `rest`. :

    >>> parser = argparse.ArgumentParser()
    >>> parser.add_argument('--foo')
    >>> parser.add_argument('cmd')
    >>> parser.add_argument('rest', nargs='*', type=int)
    >>> parser.parse_known_args('doit 1 --foo bar 2 3'.split())
    (Namespace(cmd='doit', foo='bar', rest=[1]), ['2', '3'])
    >>> parser.parse_intermixed_args('doit 1 --foo bar 2 3'.split())
    Namespace(cmd='doit', foo='bar', rest=[1, 2, 3])

`~ArgumentParser.parse_known_intermixed_args` returns a two item tuple containing the populated namespace and the list of remaining argument strings. `~ArgumentParser.parse_intermixed_args` raises an error if there are any remaining unparsed argument strings.

<div class="versionadded">

3.7

</div>

</div>

### Registering custom types or actions

<div class="method">

ArgumentParser.register(registry_name, value, object)

Sometimes it's desirable to use a custom string in error messages to provide more user-friendly output. In these cases, `!register` can be used to register custom actions or types with a parser and allow you to reference the type by their registered name instead of their callable name.

The `!register` method accepts three arguments - a *registry_name*, specifying the internal registry where the object will be stored (e.g., `action`, `type`), *value*, which is the key under which the object will be registered, and object, the callable to be registered.

The following example shows how to register a custom type with a parser:

    >>> import argparse
    >>> parser = argparse.ArgumentParser()
    >>> parser.register('type', 'hexadecimal integer', lambda s: int(s, 16))
    >>> parser.add_argument('--foo', type='hexadecimal integer')
    _StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=None, type='hexadecimal integer', choices=None, required=False, help=None, metavar=None, deprecated=False)
    >>> parser.parse_args(['--foo', '0xFA'])
    Namespace(foo=250)
    >>> parser.parse_args(['--foo', '1.2'])
    usage: PROG [-h] [--foo FOO]
    PROG: error: argument --foo: invalid 'hexadecimal integer' value: '1.2'

</div>

## Exceptions

<div class="exception">

ArgumentError

An error from creating or using an argument (optional or positional).

The string value of this exception is the message, augmented with information about the argument that caused it.

</div>

<div class="exception">

ArgumentTypeError

Raised when something goes wrong converting a command line string to a type.

</div>

**Guides and Tutorials**

<div class="toctree" data-maxdepth="1">

../howto/argparse.rst ../howto/argparse-optparse.rst

</div>

# `!array` --- Efficient arrays of numeric values

<div class="module" data-synopsis="Space efficient arrays of uniformly typed numeric values.">

array

</div>

<div class="index">

single: arrays

</div>

------------------------------------------------------------------------

This module defines an object type which can compactly represent an array of basic values: characters, integers, floating-point numbers, complex numbers. Arrays are mutable `sequence` types and behave very much like lists, except that the type of objects stored in them is constrained. The type is specified at object creation time by using a `type code`. The following type codes are defined:

| Type code | C Type | Python Type | Minimum size in bytes | Notes |
|----|----|----|----|----|
| `'b'` | signed char | int | 1 |  |
| `'B'` | unsigned char | int | 1 |  |
| `'w'` | Py_UCS4 | Unicode character | 4 | \(1\) |
| `'h'` | signed short | int | 2 |  |
| `'H'` | unsigned short | int | 2 |  |
| `'i'` | signed int | int | 2 |  |
| `'I'` | unsigned int | int | 2 |  |
| `'l'` | signed long | int | 4 |  |
| `'L'` | unsigned long | int | 4 |  |
| `'q'` | signed long long | int | 8 |  |
| `'Q'` | unsigned long long | int | 8 |  |
| `'e'` | <span id="float16">Float16</span> | float | 2 | \(2\) |
| `'f'` | float | float | 4 |  |
| `'d'` | double | float | 8 |  |
| `'Zf'` | float complex | complex | 8 | \(3\) |
| `'Zd'` | double complex | complex | 16 | \(3\) |

Notes:

\(1\)  
<div class="versionadded">

3.13

</div>

\(2\)  
The IEEE 754 binary16 "half precision" type was introduced in the 2008 revision of the [IEEE 754 standard](https://en.wikipedia.org/wiki/IEEE_754-2008_revision). This type is not widely supported by C compilers. It's available as `_Float16` type, if the compiler supports the Annex H of the C23 standard.

<div class="versionadded">

3.15

</div>

\(3\)  
Complex types (`Zf` and `Zd`) are available unconditionally, regardless on support for complex types (the Annex G of the C11 standard) by the C compiler. As specified in the C11 standard, each complex type is represented by a two-element C array containing, respectively, the real and imaginary parts.

<div class="versionadded">

3.15

</div>

<div class="seealso">

The `ctypes <ctypes-fundamental-data-types>` and `struct <type-codes>` modules, as well as third-party modules like [numpy](https://numpy.org/doc/stable/reference/arrays.interface.html#object.__array_interface__), use similar -- but slightly different -- type codes.

</div>

The actual representation of values is determined by the machine architecture (strictly speaking, by the C implementation). The actual size can be accessed through the `array.itemsize` attribute.

The module defines the following item:

<div class="data">

typecodes

A tuple with all available type codes.

<div class="versionchanged">

3.15 The type changed from `str` to `tuple`.

</div>

</div>

The module defines the following type:

<div class="array(typecode[, initializer])">

A new array whose items are restricted by *typecode*, and initialized from the optional *initializer* value, which must be a `bytes` or `bytearray` object, a Unicode string, or iterable over elements of the appropriate type.

If given a `bytes` or `bytearray` object, the initializer is passed to the new array's `frombytes` method; if given a Unicode string, the initializer is passed to the `fromunicode` method; otherwise, the initializer's iterator is passed to the `extend` method to add initial items to the array.

Array objects support the ordinary `mutable <typesseq-mutable>` `sequence` operations of indexing, slicing, concatenation, and multiplication. When using slice assignment, the assigned value must be an array object with the same type code; in all other cases, `TypeError` is raised. Array objects also implement the buffer interface, and may be used wherever `bytes-like objects <bytes-like object>` are supported.

Arrays are `generic <generics>` over the type of their contents.

<div class="audit-event">

array.\_\_new\_\_ typecode,initializer array.array

</div>

<div class="attribute">

typecode

The typecode character used to create the array.

</div>

<div class="attribute">

itemsize

The length in bytes of one array item in the internal representation.

</div>

<div class="method">

append(value, /)

Append a new item with the specified value to the end of the array.

</div>

<div class="method">

buffer_info()

Return a tuple `(address, length)` giving the current memory address and the length in elements of the buffer used to hold array's contents. The size of the memory buffer in bytes can be computed as `array.buffer_info()[1] * array.itemsize`. This is occasionally useful when working with low-level (and inherently unsafe) I/O interfaces that require memory addresses, such as certain `!ioctl` operations. The returned numbers are valid as long as the array exists and no length-changing operations are applied to it.

> [!NOTE]
> When using array objects from code written in C or C++ (the only way to effectively make use of this information), it makes more sense to use the buffer interface supported by array objects. This method is maintained for backward compatibility and should be avoided in new code. The buffer interface is documented in `bufferobjects`.

</div>

<div class="method">

byteswap()

"Byteswap" all items of the array. This is only supported for values which are 1, 2, 4, 8 or 16 bytes in size; for other types of values, `RuntimeError` is raised. It is useful when reading data from a file written on a machine with a different byte order. Note, that for complex types the order of components (the real part, followed by imaginary part) is preserved.

</div>

<div class="method">

count(value, /)

Return the number of occurrences of *value* in the array.

</div>

<div class="method">

extend(iterable, /)

Append items from *iterable* to the end of the array. If *iterable* is another array, it must have *exactly* the same type code; if not, `TypeError` will be raised. If *iterable* is not an array, it must be iterable and its elements must be the right type to be appended to the array.

</div>

<div class="method">

frombytes(buffer, /)

Appends items from the `bytes-like object`, interpreting its content as an array of machine values (as if it had been read from a file using the `fromfile` method).

<div class="versionadded">

3.2 `!fromstring` is renamed to `frombytes` for clarity.

</div>

</div>

<div class="method">

fromfile(f, n, /)

Read *n* items (as machine values) from the `file object` *f* and append them to the end of the array. If less than *n* items are available, `EOFError` is raised, but the items that were available are still inserted into the array.

</div>

<div class="method">

fromlist(list, /)

Append items from the list. This is equivalent to `for x in list: a.append(x)` except that if there is a type error, the array is unchanged.

</div>

<div class="method">

fromunicode(ustr, /)

Extends this array with data from the given Unicode string. The array must have type code `'w'`; otherwise a `ValueError` is raised. Use `array.frombytes(unicodestring.encode(enc))` to append Unicode data to an array of some other type.

</div>

<div class="method">

index(value\[, start\[, stop\]\])

Return the smallest *i* such that *i* is the index of the first occurrence of *value* in the array. The optional arguments *start* and *stop* can be specified to search for *value* within a subsection of the array. Raise `ValueError` if *value* is not found.

<div class="versionchanged">

3.10 Added optional *start* and *stop* parameters.

</div>

</div>

<div class="method">

insert(index, value, /)

Insert a new item *value* in the array before position *index*. Negative values are treated as being relative to the end of the array.

</div>

<div class="method">

pop(index=-1, /)

Removes the item with the index *i* from the array and returns it. The optional argument defaults to `-1`, so that by default the last item is removed and returned.

</div>

<div class="method">

remove(value, /)

Remove the first occurrence of *value* from the array.

</div>

<div class="method">

clear()

Remove all elements from the array.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

reverse()

Reverse the order of the items in the array.

</div>

<div class="method">

tobytes()

Convert the array to an array of machine values and return the bytes representation (the same sequence of bytes that would be written to a file by the `tofile` method.)

<div class="versionadded">

3.2 `!tostring` is renamed to `tobytes` for clarity.

</div>

</div>

<div class="method">

tofile(f, /)

Write all items (as machine values) to the `file object` *f*.

</div>

<div class="method">

tolist()

Convert the array to an ordinary list with the same items.

</div>

<div class="method">

tounicode()

Convert the array to a Unicode string. The array must have a type `'w'`; otherwise a `ValueError` is raised. Use `array.tobytes().decode(enc)` to obtain a Unicode string from an array of some other type.

</div>

</div>

The string representation of array objects has the form `array(typecode, initializer)`. The *initializer* is omitted if the array is empty, otherwise it is a Unicode string if the *typecode* is `'w'`, otherwise it is a list of numbers. The string representation is guaranteed to be able to be converted back to an array with the same type and value using `eval`, so long as the `~array.array` class has been imported using `from array import array`. Variables `inf` and `nan` must also be defined if it contains corresponding floating-point values. Examples:

    array('l')
    array('w', 'hello \u2641')
    array('l', [1, 2, 3, 4, 5])
    array('d', [1.0, 2.0, 3.14, -inf, nan])

<div class="seealso">

Module `struct`  
Packing and unpacking of heterogeneous binary data.

[NumPy](https://numpy.org/)  
The NumPy package defines another array type.

</div>

# `!ast` --- Abstract syntax trees

<div class="module" data-synopsis="Abstract Syntax Tree classes and manipulation.">

ast

</div>

<div class="testsetup">

import ast

</div>

**Source code:** `Lib/ast.py`

------------------------------------------------------------------------

The `!ast` module helps Python applications to process trees of the Python abstract syntax grammar. The abstract syntax itself might change with each Python release; this module helps to find out programmatically what the current grammar looks like.

An abstract syntax tree can be generated by passing `ast.PyCF_ONLY_AST` as a flag to the `compile` built-in function, or using the `parse` helper provided in this module. The result will be a tree of objects whose classes all inherit from `ast.AST`. An abstract syntax tree can be compiled into a Python code object using the built-in `compile` function.

## Abstract grammar

The abstract grammar is currently defined as follows:

<div class="literalinclude" data-language="asdl">

../../Parser/Python.asdl

</div>

## Node classes

<div class="AST">

This is the abstract base of all AST node classes. The actual node classes are derived from the `Parser/Python.asdl` file, which is reproduced `above <abstract-grammar>`. They are defined in the `!_ast` C module and re-exported in `!ast`.

There is one class defined for each left-hand side symbol in the abstract grammar (for example, `!ast.stmt` or `!ast.expr`). In addition, there is one class defined for each constructor on the right-hand side; these classes inherit from the classes for the left-hand side trees. For example, `ast.BinOp` inherits from `!ast.expr`. For production rules with alternatives (aka "sums"), the left-hand side class is abstract: only instances of specific constructor nodes are ever created.

<div class="index">

single: ? (question mark); in AST grammar

</div>

<div class="index">

single: \* (asterisk); in AST grammar

</div>

<div class="attribute">

<span id="fields">fields</span>

Each concrete class has an attribute `!_fields` which gives the names of all child nodes.

Each instance of a concrete class has one attribute for each child node, of the type as defined in the grammar. For example, `ast.BinOp` instances have an attribute `!left` of type `!ast.expr`.

If these attributes are marked as optional in the grammar (using a question mark), the value might be `None`. If the attributes can have zero-or-more values (marked with an asterisk), the values are represented as Python lists. All possible attributes must be present and have valid values when compiling an AST with `compile`.

</div>

<div class="attribute">

<span id="field_types">field_types</span>

The `!_field_types` attribute on each concrete class is a dictionary mapping field names (as also listed in `_fields`) to their types.

<div class="doctest">

\>\>\> ast.TypeVar.\_field_types {'name': \<class 'str'\>, 'bound': ast.expr \| None, 'default_value': ast.expr \| None}

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="attribute">

lineno col_offset end_lineno end_col_offset

Instances of `ast.expr` and `ast.stmt` subclasses have `lineno`, `col_offset`, `end_lineno`, and `end_col_offset` attributes. The `lineno` and `end_lineno` are the first and last line numbers of source text span (1-indexed so the first line is line 1) and the `col_offset` and `end_col_offset` are the corresponding UTF-8 byte offsets of the first and last tokens that generated the node. The UTF-8 offset is recorded because the parser uses UTF-8 internally.

Note that the end positions are not required by the compiler and are therefore optional. The end offset is *after* the last symbol, for example one can get the source segment of a one-line expression node using `source_line[node.col_offset : node.end_col_offset]`.

</div>

The constructors of AST node classes parse their arguments as follows:

- If there are positional arguments, there must be as many as there are items in `!_fields`; they will be assigned to attributes with the corresponding names.
- If there are keyword arguments, they will set the attributes of the same names to the given values.

For example, to create and populate an `ast.UnaryOp` node, you could use :

    node = ast.UnaryOp(ast.USub(), ast.Constant(5, lineno=0, col_offset=0),
                       lineno=0, col_offset=0)

If a field that is optional in the grammar is omitted from the constructor, it defaults to `None`. If a list field is omitted, it defaults to the empty list. If a field of type `!ast.expr_context` is omitted, it defaults to `Load() <ast.Load>`. If any other field is omitted, a `DeprecationWarning` is raised and the AST node will not have this field. In Python 3.15, this condition will raise an error.

</div>

<div class="versionchanged">

3.8

Class `ast.Constant` is now used for all constants.

</div>

<div class="versionchanged">

3.9

Simple indices are represented by their value, extended slices are represented as tuples.

</div>

<div class="versionchanged">

3.13

AST node constructors were changed to provide sensible defaults for omitted fields: optional fields now default to `None`, list fields default to an empty list, and fields of type `!ast.expr_context` default to `Load() <ast.Load>`. Previously, omitted attributes would not exist on constructed nodes (accessing them raised `AttributeError`).

</div>

<div class="versionchanged">

3.14

The `~object.__repr__` output of `~ast.AST` nodes includes the values of the node fields.

</div>

<div class="deprecated-removed">

3.8 3.14

Previous versions of Python provided the AST classes `!ast.Num`, `!ast.Str`, `!ast.Bytes`, `!ast.NameConstant` and `!ast.Ellipsis`, which were deprecated in Python 3.8. These classes were removed in Python 3.14, and their functionality has been replaced with `ast.Constant`.

</div>

<div class="deprecated">

3.9

Old classes `!ast.Index` and `!ast.ExtSlice` are still available, but they will be removed in future Python releases. In the meantime, instantiating them will return an instance of a different class.

</div>

<div class="deprecated-removed">

3.13 3.15

Previous versions of Python allowed the creation of AST nodes that were missing required fields. Similarly, AST node constructors allowed arbitrary keyword arguments that were set as attributes of the AST node, even if they did not match any of the fields of the AST node. These cases now raise a `TypeError`.

</div>

<div class="deprecated-removed">

3.15 3.20

In the `grammar above <abstract-grammar>`, the AST node classes that correspond to production rules with variants (aka "sums") are abstract classes. Previous versions of Python allowed for the creation of direct instances of these abstract node classes. This behavior is deprecated and will be removed in Python 3.20.

</div>

> [!NOTE]
> The descriptions of the specific node classes displayed here were initially adapted from the fantastic [Green Tree Snakes](https://greentreesnakes.readthedocs.io/en/latest/) project and all its contributors.

### Root nodes

<div class="Module(body, type_ignores)">

A Python module, as with `file input <file-input>`. Node type generated by `ast.parse` in the default `"exec"` *mode*.

`body` is a `list` of the module's `ast-statements`.

`type_ignores` is a `list` of the module's type ignore comments; see `ast.parse` for more details.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('x = 1'), indent=4)) Module( body=\[ Assign( targets=\[ Name(id='x', ctx=Store())\], value=Constant(value=1))\])

</div>

</div>

<div class="Expression(body)">

A single Python `expression input <expression-input>`. Node type generated by `ast.parse` when *mode* is `"eval"`.

`body` is a single node, one of the `expression types <ast-expressions>`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('123', mode='eval'), indent=4)) Expression( body=Constant(value=123))

</div>

</div>

<div class="Interactive(body)">

A single `interactive input <interactive>`, like in `tut-interac`. Node type generated by `ast.parse` when *mode* is `"single"`.

`body` is a `list` of `statement nodes <ast-statements>`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('x = 1; y = 2', mode='single'), indent=4)) Interactive( body=\[ Assign( targets=\[ Name(id='x', ctx=Store())\], value=Constant(value=1)), Assign( targets=\[ Name(id='y', ctx=Store())\], value=Constant(value=2))\])

</div>

</div>

<div class="FunctionType(argtypes, returns)">

A representation of an old-style type comments for functions, as Python versions prior to 3.5 didn't support `484` annotations. Node type generated by `ast.parse` when *mode* is `"func_type"`.

Such type comments would look like this:

    def sum_two_number(a, b):
        # type: (int, int) -> int
        return a + b

`argtypes` is a `list` of `expression nodes <ast-expressions>`.

`returns` is a single `expression node <ast-expressions>`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('(int, str) -\> List\[int\]', mode='func_type'), indent=4)) FunctionType( argtypes=\[ Name(id='int'), Name(id='str')\], returns=Subscript( value=Name(id='List'), slice=Name(id='int')))

</div>

<div class="versionadded">

3.8

</div>

</div>

### Literals

<div class="Constant(value, kind)">

A constant value. The `value` attribute of the `Constant` literal contains the Python object it represents. The values represented can be instances of `str`, `bytes`, `int`, `float`, `complex`, and `bool`, and the constants `None` and `Ellipsis`.

The `kind` attribute is an optional string. For string literals with a `u` prefix, `kind` is set to `'u'`. For all other constants, `kind` is `None`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('123', mode='eval'), indent=4)) Expression( body=Constant(value=123)) \>\>\> print(ast.dump(ast.parse("u'hello'", mode='eval'), indent=4)) Expression( body=Constant(value='hello', kind='u'))

</div>

</div>

<div class="FormattedValue(value, conversion, format_spec)">

Node representing a single formatting field in an f-string. If the string contains a single formatting field and nothing else the node can be isolated otherwise it appears in `JoinedStr`.

- `value` is any expression node (such as a literal, a variable, or a function call).
- `conversion` is an integer:
  - -1: no formatting
  - 97 (`ord('a')`): `!a` `ASCII <ascii>` formatting
  - 114 (`ord('r')`): `!r` `repr` formatting
  - 115 (`ord('s')`): `!s` `string <str>` formatting
- `format_spec` is a `JoinedStr` node representing the formatting of the value, or `None` if no format was specified. Both `conversion` and `format_spec` can be set at the same time.

</div>

<div class="JoinedStr(values)">

An f-string, comprising a series of `FormattedValue` and `Constant` nodes.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('f"sin({a}) is {sin(a):.3}"', mode='eval'), indent=4)) Expression( body=JoinedStr( values=\[ Constant(value='sin('), FormattedValue( value=Name(id='a'), conversion=-1), Constant(value=') is '), FormattedValue( value=Call( func=Name(id='sin'), args=\[ Name(id='a')\]), conversion=-1, format_spec=JoinedStr( values=\[ Constant(value='.3')\]))\]))

</div>

</div>

<div class="TemplateStr(values, /)">

<div class="versionadded">

3.14

</div>

Node representing a template string literal, comprising a series of `Interpolation` and `Constant` nodes. These nodes may be any order, and do not need to be interleaved.

<div class="doctest">

\>\>\> expr = ast.parse('t"{name} finished {place:ordinal}"', mode='eval') \>\>\> print(ast.dump(expr, indent=4)) Expression( body=TemplateStr( values=\[ Interpolation( value=Name(id='name'), str='name', conversion=-1), Constant(value=' finished '), Interpolation( value=Name(id='place'), str='place', conversion=-1, format_spec=JoinedStr( values=\[ Constant(value='ordinal')\]))\]))

</div>

</div>

<div class="Interpolation(value, str, conversion, format_spec=None)">

<div class="versionadded">

3.14

</div>

Node representing a single interpolation field in a template string literal.

- `value` is any expression node (such as a literal, a variable, or a function call). This has the same meaning as `FormattedValue.value`.

- `str` is a constant containing the text of the interpolation expression.

  If `str` is set to `None`, then `value` is used to generate code when calling `ast.unparse`. This no longer guarantees that the generated code is identical to the original and is intended for code generation.

- `conversion` is an integer:

  - -1: no conversion
  - 97 (`ord('a')`): `!a` `ASCII <ascii>` conversion
  - 114 (`ord('r')`): `!r` `repr` conversion
  - 115 (`ord('s')`): `!s` `string <str>` conversion

  This has the same meaning as `FormattedValue.conversion`.

- `format_spec` is a `JoinedStr` node representing the formatting of the value, or `None` if no format was specified. Both `conversion` and `format_spec` can be set at the same time. This has the same meaning as `FormattedValue.format_spec`.

</div>

<div class="List(elts, ctx) Tuple(elts, ctx)">

A list or tuple. `elts` holds a list of nodes representing the elements. `ctx` is `Store` if the container is an assignment target (i.e. `(x,y)=something`), and `Load` otherwise.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('\[1, 2, 3\]', mode='eval'), indent=4)) Expression( body=List( elts=\[ Constant(value=1), Constant(value=2), Constant(value=3)\])) \>\>\> print(ast.dump(ast.parse('(1, 2, 3)', mode='eval'), indent=4)) Expression( body=Tuple( elts=\[ Constant(value=1), Constant(value=2), Constant(value=3)\]))

</div>

</div>

<div class="Set(elts)">

A set. `elts` holds a list of nodes representing the set's elements.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('{1, 2, 3}', mode='eval'), indent=4)) Expression( body=Set( elts=\[ Constant(value=1), Constant(value=2), Constant(value=3)\]))

</div>

</div>

<div class="Dict(keys, values)">

A dictionary. `keys` and `values` hold lists of nodes representing the keys and the values respectively, in matching order (what would be returned when calling `dictionary.keys()` and `dictionary.values()`).

When doing dictionary unpacking using dictionary literals the expression to be expanded goes in the `values` list, with a `None` at the corresponding position in `keys`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('{"a":1, \*\*d}', mode='eval'), indent=4)) Expression( body=Dict( keys=\[ Constant(value='a'), None\], values=\[ Constant(value=1), Name(id='d')\]))

</div>

</div>

### Variables

<div class="Name(id, ctx)">

A variable name. `id` holds the name as a string, and `ctx` is one of the following types.

</div>

<div class="Load() Store() Del()">

Variable references can be used to load the value of a variable, to assign a new value to it, or to delete it. Variable references are given a context to distinguish these cases.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('a'), indent=4)) Module( body=\[ Expr( value=Name(id='a'))\])

\>\>\> print(ast.dump(ast.parse('a = 1'), indent=4)) Module( body=\[ Assign( targets=\[ Name(id='a', ctx=Store())\], value=Constant(value=1))\])

\>\>\> print(ast.dump(ast.parse('del a'), indent=4)) Module( body=\[ Delete( targets=\[ Name(id='a', ctx=Del())\])\])

</div>

</div>

<div class="Starred(value, ctx)">

A `*var` variable reference. `value` holds the variable, typically a `Name` node. This type must be used when building a `Call` node with `*args`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('a, \*b = it'), indent=4)) Module( body=\[ Assign( targets=\[ Tuple( elts=\[ Name(id='a', ctx=Store()), Starred( value=Name(id='b', ctx=Store()), ctx=Store())\], ctx=Store())\], value=Name(id='it'))\])

</div>

</div>

### Expressions

<div class="Expr(value)">

When an expression, such as a function call, appears as a statement by itself with its return value not used or stored, it is wrapped in this container. `value` holds one of the other nodes in this section, a `Constant`, a `Name`, a `Lambda`, a `Yield` or `YieldFrom` node.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('-a'), indent=4)) Module( body=\[ Expr( value=UnaryOp( op=USub(), operand=Name(id='a')))\])

</div>

</div>

<div class="UnaryOp(op, operand)">

A unary operation. `op` is the operator, and `operand` any expression node.

</div>

<div class="UAdd USub Not Invert">

Unary operator tokens. `Not` is the `not` keyword, `Invert` is the `~` operator.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('not x', mode='eval'), indent=4)) Expression( body=UnaryOp( op=Not(), operand=Name(id='x')))

</div>

</div>

<div class="BinOp(left, op, right)">

A binary operation (like addition or division). `op` is the operator, and `left` and `right` are any expression nodes.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('x + y', mode='eval'), indent=4)) Expression( body=BinOp( left=Name(id='x'), op=Add(), right=Name(id='y')))

</div>

</div>

<div class="Add Sub Mult Div FloorDiv Mod Pow LShift RShift BitOr BitXor BitAnd MatMult">

Binary operator tokens.

</div>

<div class="BoolOp(op, values)">

A boolean operation, 'or' or 'and'. `op` is `Or` or `And`. `values` are the values involved. Consecutive operations with the same operator, such as `a or b or c`, are collapsed into one node with several values.

This doesn't include `not`, which is a `UnaryOp`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('x or y', mode='eval'), indent=4)) Expression( body=BoolOp( op=Or(), values=\[ Name(id='x'), Name(id='y')\]))

</div>

</div>

<div class="And Or">

Boolean operator tokens.

</div>

<div class="Compare(left, ops, comparators)">

A comparison of two or more values. `left` is the first value in the comparison, `ops` the list of operators, and `comparators` the list of values after the first element in the comparison.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('1 \<= a \< 10', mode='eval'), indent=4)) Expression( body=Compare( left=Constant(value=1), ops=\[ LtE(), Lt()\], comparators=\[ Name(id='a'), Constant(value=10)\]))

</div>

</div>

<div class="Eq NotEq Lt LtE Gt GtE Is IsNot In NotIn">

Comparison operator tokens.

</div>

<div class="Call(func, args, keywords)">

A function call. `func` is the function, which will often be a `Name` or `Attribute` object. Of the arguments:

- `args` holds a list of the arguments passed by position.
- `keywords` holds a list of `.keyword` objects representing arguments passed by keyword.

The `args` and `keywords` arguments are optional and default to empty lists.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('func(a, b=c, *d,*\*e)', mode='eval'), indent=4)) Expression( body=Call( func=Name(id='func'), args=\[ Name(id='a'), Starred( value=Name(id='d'))\], keywords=\[ keyword( arg='b', value=Name(id='c')), keyword( value=Name(id='e'))\]))

</div>

</div>

<div class="keyword(arg, value)">

A keyword argument to a function call or class definition. `arg` is a raw string of the parameter name, `value` is a node to pass in.

</div>

<div class="IfExp(test, body, orelse)">

An expression such as `a if b else c`. Each field holds a single node, so in the following example, all three are `Name` nodes.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('a if b else c', mode='eval'), indent=4)) Expression( body=IfExp( test=Name(id='b'), body=Name(id='a'), orelse=Name(id='c')))

</div>

</div>

<div class="Attribute(value, attr, ctx)">

Attribute access, e.g. `d.keys`. `value` is a node, typically a `Name`. `attr` is a bare string giving the name of the attribute, and `ctx` is `Load`, `Store` or `Del` according to how the attribute is acted on.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('snake.colour', mode='eval'), indent=4)) Expression( body=Attribute( value=Name(id='snake'), attr='colour'))

</div>

</div>

<div class="NamedExpr(target, value)">

A named expression. This AST node is produced by the assignment expressions operator (also known as the walrus operator). As opposed to the `Assign` node in which the first argument can be multiple nodes, in this case both `target` and `value` must be single nodes.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('(x := 4)', mode='eval'), indent=4)) Expression( body=NamedExpr( target=Name(id='x', ctx=Store()), value=Constant(value=4)))

</div>

<div class="versionadded">

3.8

</div>

</div>

#### Subscripting

<div class="Subscript(value, slice, ctx)">

A subscript, such as `l[1]`. `value` is the subscripted object (usually sequence or mapping). `slice` is an index, slice or key. It can be a `Tuple` and contain a `Slice`. `ctx` is `Load`, `Store` or `Del` according to the action performed with the subscript.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('l\[1:2, 3\]', mode='eval'), indent=4)) Expression( body=Subscript( value=Name(id='l'), slice=Tuple( elts=\[ Slice( lower=Constant(value=1), upper=Constant(value=2)), Constant(value=3)\])))

</div>

</div>

<div class="Slice(lower, upper, step)">

Regular slicing (on the form `lower:upper` or `lower:upper:step`). Can occur only inside the *slice* field of `Subscript`, either directly or as an element of `Tuple`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('l\[1:2\]', mode='eval'), indent=4)) Expression( body=Subscript( value=Name(id='l'), slice=Slice( lower=Constant(value=1), upper=Constant(value=2))))

</div>

</div>

#### Comprehensions

<div class="ListComp(elt, generators) SetComp(elt, generators) GeneratorExp(elt, generators) DictComp(key, value, generators)">

List and set comprehensions, generator expressions, and dictionary comprehensions. `elt` (or `key` and `value`) is a single node representing the part that will be evaluated for each item.

For dictionary comprehensions using unpacking, for example `{**item for item in items}`, the expression to be expanded goes in `key` and `value` is `None`.

`generators` is a list of `comprehension` nodes.

<div class="doctest">

\>\>\> print(ast.dump( ... ast.parse('\[x for x in numbers\]', mode='eval'), ... indent=4, ... )) Expression( body=ListComp( elt=Name(id='x'), generators=\[ comprehension( target=Name(id='x', ctx=Store()), iter=Name(id='numbers'), is_async=0)\])) \>\>\> print(ast.dump( ... ast.parse('{x: x\*\*2 for x in numbers}', mode='eval'), ... indent=4, ... )) Expression( body=DictComp( key=Name(id='x'), value=BinOp( left=Name(id='x'), op=Pow(), right=Constant(value=2)), generators=\[ comprehension( target=Name(id='x', ctx=Store()), iter=Name(id='numbers'), is_async=0)\])) \>\>\> print(ast.dump( ... ast.parse('{x for x in numbers}', mode='eval'), ... indent=4, ... )) Expression( body=SetComp( elt=Name(id='x'), generators=\[ comprehension( target=Name(id='x', ctx=Store()), iter=Name(id='numbers'), is_async=0)\]))

</div>

</div>

<div class="comprehension(target, iter, ifs, is_async)">

One `for` clause in a comprehension. `target` is the reference to use for each element - typically a `Name` or `Tuple` node. `iter` is the object to iterate over. `ifs` is a list of test expressions: each `for` clause can have multiple `ifs`.

`is_async` indicates a comprehension is asynchronous (using an `async for` instead of `for`). The value is an integer (0 or 1).

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('\[ord(c) for line in file for c in line\]', mode='eval'), ... indent=4)) \# Multiple comprehensions in one. Expression( body=ListComp( elt=Call( func=Name(id='ord'), args=\[ Name(id='c')\]), generators=\[ comprehension( target=Name(id='line', ctx=Store()), iter=Name(id='file'), is_async=0), comprehension( target=Name(id='c', ctx=Store()), iter=Name(id='line'), is_async=0)\]))

\>\>\> print(ast.dump(ast.parse('(n\*\*2 for n in it if n\>5 if n\<10)', mode='eval'), ... indent=4)) \# generator comprehension Expression( body=GeneratorExp( elt=BinOp( left=Name(id='n'), op=Pow(), right=Constant(value=2)), generators=\[ comprehension( target=Name(id='n', ctx=Store()), iter=Name(id='it'), ifs=\[ Compare( left=Name(id='n'), ops=\[ Gt()\], comparators=\[ Constant(value=5)\]), Compare( left=Name(id='n'), ops=\[ Lt()\], comparators=\[ Constant(value=10)\])\], is_async=0)\]))

\>\>\> print(ast.dump(ast.parse('\[i async for i in soc\]', mode='eval'), ... indent=4)) \# Async comprehension Expression( body=ListComp( elt=Name(id='i'), generators=\[ comprehension( target=Name(id='i', ctx=Store()), iter=Name(id='soc'), is_async=1)\]))

</div>

</div>

### Statements

<div class="Assign(targets, value, type_comment)">

An assignment. `targets` is a list of nodes, and `value` is a single node.

Multiple nodes in `targets` represents assigning the same value to each. Unpacking is represented by putting a `Tuple` or `List` within `targets`.

<div class="attribute">

type_comment

`type_comment` is an optional string with the type annotation as a comment.

</div>

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('a = b = 1'), indent=4)) \# Multiple assignment Module( body=\[ Assign( targets=\[ Name(id='a', ctx=Store()), Name(id='b', ctx=Store())\], value=Constant(value=1))\])

\>\>\> print(ast.dump(ast.parse('a,b = c'), indent=4)) \# Unpacking Module( body=\[ Assign( targets=\[ Tuple( elts=\[ Name(id='a', ctx=Store()), Name(id='b', ctx=Store())\], ctx=Store())\], value=Name(id='c'))\])

</div>

</div>

<div class="AnnAssign(target, annotation, value, simple)">

An assignment with a type annotation. `target` is a single node and can be a `Name`, an `Attribute` or a `Subscript`. `annotation` is the annotation, such as a `Constant` or `Name` node. `value` is a single optional node.

`simple` is always either 0 (indicating a "complex" target) or 1 (indicating a "simple" target). A "simple" target consists solely of a `Name` node that does not appear between parentheses; all other targets are considered complex. Only simple targets appear in the `~object.__annotations__` dictionary of modules and classes.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('c: int'), indent=4)) Module( body=\[ AnnAssign( target=Name(id='c', ctx=Store()), annotation=Name(id='int'), simple=1)\])

\>\>\> print(ast.dump(ast.parse('(a): int = 1'), indent=4)) \# Annotation with parenthesis Module( body=\[ AnnAssign( target=Name(id='a', ctx=Store()), annotation=Name(id='int'), value=Constant(value=1), simple=0)\])

\>\>\> print(ast.dump(ast.parse('a.b: int'), indent=4)) \# Attribute annotation Module( body=\[ AnnAssign( target=Attribute( value=Name(id='a'), attr='b', ctx=Store()), annotation=Name(id='int'), simple=0)\])

\>\>\> print(ast.dump(ast.parse('a\[1\]: int'), indent=4)) \# Subscript annotation Module( body=\[ AnnAssign( target=Subscript( value=Name(id='a'), slice=Constant(value=1), ctx=Store()), annotation=Name(id='int'), simple=0)\])

</div>

</div>

<div class="AugAssign(target, op, value)">

Augmented assignment, such as `a += 1`. In the following example, `target` is a `Name` node for `x` (with the `Store` context), `op` is `Add`, and `value` is a `Constant` with value for 1.

The `target` attribute cannot be of class `Tuple` or `List`, unlike the targets of `Assign`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('x += 2'), indent=4)) Module( body=\[ AugAssign( target=Name(id='x', ctx=Store()), op=Add(), value=Constant(value=2))\])

</div>

</div>

<div class="Raise(exc, cause)">

A `raise` statement. `exc` is the exception object to be raised, normally a `Call` or `Name`, or `None` for a standalone `raise`. `cause` is the optional part for `y` in `raise x from y`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('raise x from y'), indent=4)) Module( body=\[ Raise( exc=Name(id='x'), cause=Name(id='y'))\])

</div>

</div>

<div class="Assert(test, msg)">

An assertion. `test` holds the condition, such as a `Compare` node. `msg` holds the failure message.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('assert x,y'), indent=4)) Module( body=\[ Assert( test=Name(id='x'), msg=Name(id='y'))\])

</div>

</div>

<div class="Delete(targets)">

Represents a `del` statement. `targets` is a list of nodes, such as `Name`, `Attribute` or `Subscript` nodes.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('del x,y,z'), indent=4)) Module( body=\[ Delete( targets=\[ Name(id='x', ctx=Del()), Name(id='y', ctx=Del()), Name(id='z', ctx=Del())\])\])

</div>

</div>

<div class="Pass()">

A `pass` statement.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('pass'), indent=4)) Module( body=\[ Pass()\])

</div>

</div>

<div class="TypeAlias(name, type_params, value)">

A `type alias <type-aliases>` created through the `type` statement. `name` is the name of the alias, `type_params` is a list of `type parameters <ast-type-params>`, and `value` is the value of the type alias.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('type Alias = int'), indent=4)) Module( body=\[ TypeAlias( name=Name(id='Alias', ctx=Store()), value=Name(id='int'))\])

</div>

<div class="versionadded">

3.12

</div>

</div>

Other statements which are only applicable inside functions or loops are described in other sections.

#### Imports

<div class="Import(names)">

An import statement. `names` is a list of `alias` nodes.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('import x,y,z'), indent=4)) Module( body=\[ Import( names=\[ alias(name='x'), alias(name='y'), alias(name='z')\], is_lazy=0)\])

</div>

</div>

<div class="ImportFrom(module, names, level)">

Represents `from x import y`. `module` is a raw string of the 'from' name, without any leading dots, or `None` for statements such as `from . import foo`. `level` is an integer holding the level of the relative import (0 means absolute import).

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('from y import x,y,z'), indent=4)) Module( body=\[ ImportFrom( module='y', names=\[ alias(name='x'), alias(name='y'), alias(name='z')\], level=0, is_lazy=0)\])

</div>

</div>

<div class="alias(name, asname)">

Both parameters are raw strings of the names. `asname` can be `None` if the regular name is to be used.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('from ..foo.bar import a as b, c'), indent=4)) Module( body=\[ ImportFrom( module='foo.bar', names=\[ alias(name='a', asname='b'), alias(name='c')\], level=2, is_lazy=0)\])

</div>

</div>

### Control flow

> [!NOTE]
> Optional clauses such as `else` are stored as an empty list if they're not present.

<div class="If(test, body, orelse)">

An `if` statement. `test` holds a single node, such as a `Compare` node. `body` and `orelse` each hold a list of nodes.

`elif` clauses don't have a special representation in the AST, but rather appear as extra `If` nodes within the `orelse` section of the previous one.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... if x: ... ... ... elif y: ... ... ... else: ... ... ... """), indent=4)) Module( body=\[ If( test=Name(id='x'), body=\[ Expr( value=Constant(value=Ellipsis))\], orelse=\[ If( test=Name(id='y'), body=\[ Expr( value=Constant(value=Ellipsis))\], orelse=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

</div>

<div class="For(target, iter, body, orelse, type_comment)">

A `for` loop. `target` holds the variable(s) the loop assigns to, as a single `Name`, `Tuple`, `List`, `Attribute` or `Subscript` node. `iter` holds the item to be looped over, again as a single node. `body` and `orelse` contain lists of nodes to execute. Those in `orelse` are executed if the loop finishes normally, rather than via a `break` statement.

<div class="attribute">

type_comment

`type_comment` is an optional string with the type annotation as a comment.

</div>

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... for x in y: ... ... ... else: ... ... ... """), indent=4)) Module( body=\[ For( target=Name(id='x', ctx=Store()), iter=Name(id='y'), body=\[ Expr( value=Constant(value=Ellipsis))\], orelse=\[ Expr( value=Constant(value=Ellipsis))\])\])

</div>

</div>

<div class="While(test, body, orelse)">

A `while` loop. `test` holds the condition, such as a `Compare` node.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... while x: ... ... ... else: ... ... ... """), indent=4)) Module( body=\[ While( test=Name(id='x'), body=\[ Expr( value=Constant(value=Ellipsis))\], orelse=\[ Expr( value=Constant(value=Ellipsis))\])\])

</div>

</div>

<div class="Break Continue">

The `break` and `continue` statements.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse("""... for a in b: ... if a \> 5: ... break ... else: ... continue ... ... """), indent=4)) Module( body=\[ For( target=Name(id='a', ctx=Store()), iter=Name(id='b'), body=\[ If( test=Compare( left=Name(id='a'), ops=\[ Gt()\], comparators=\[ Constant(value=5)\]), body=\[ Break()\], orelse=\[ Continue()\])\])\])

</div>

</div>

<div class="Try(body, handlers, orelse, finalbody)">

`try` blocks. All attributes are list of nodes to execute, except for `handlers`, which is a list of `ExceptHandler` nodes.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... try: ... ... ... except Exception: ... ... ... except OtherException as e: ... ... ... else: ... ... ... finally: ... ... ... """), indent=4)) Module( body=\[ Try( body=\[ Expr( value=Constant(value=Ellipsis))\], handlers=\[ ExceptHandler( type=Name(id='Exception'), body=\[ Expr( value=Constant(value=Ellipsis))\]), ExceptHandler( type=Name(id='OtherException'), name='e', body=\[ Expr( value=Constant(value=Ellipsis))\])\], orelse=\[ Expr( value=Constant(value=Ellipsis))\], finalbody=\[ Expr( value=Constant(value=Ellipsis))\])\])

</div>

</div>

<div class="TryStar(body, handlers, orelse, finalbody)">

`try` blocks which are followed by `except*` clauses. The attributes are the same as for `Try` but the `ExceptHandler` nodes in `handlers` are interpreted as `except*` blocks rather than `except`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... try: ... ... ... except\* Exception: ... ... ... """), indent=4)) Module( body=\[ TryStar( body=\[ Expr( value=Constant(value=Ellipsis))\], handlers=\[ ExceptHandler( type=Name(id='Exception'), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.11

</div>

</div>

<div class="ExceptHandler(type, name, body)">

A single `except` clause. `type` is the exception type it will match, typically a `Name` node (or `None` for a catch-all `except:` clause). `name` is a raw string for the name to hold the exception, or `None` if the clause doesn't have `as foo`. `body` is a list of nodes.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse("""... try: ... a + 1 ... except TypeError: ... pass ... """), indent=4)) Module( body=\[ Try( body=\[ Expr( value=BinOp( left=Name(id='a'), op=Add(), right=Constant(value=1)))\], handlers=\[ ExceptHandler( type=Name(id='TypeError'), body=\[ Pass()\])\])\])

</div>

</div>

<div class="With(items, body, type_comment)">

A `with` block. `items` is a list of `withitem` nodes representing the context managers, and `body` is the indented block inside the context.

<div class="attribute">

type_comment

`type_comment` is an optional string with the type annotation as a comment.

</div>

</div>

<div class="withitem(context_expr, optional_vars)">

A single context manager in a `with` block. `context_expr` is the context manager, often a `Call` node. `optional_vars` is a `Name`, `Tuple` or `List` for the `as foo` part, or `None` if that isn't used.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse("""... with a as b, c as d: ... something(b, d) ... """), indent=4)) Module( body=\[ With( items=\[ withitem( context_expr=Name(id='a'), optional_vars=Name(id='b', ctx=Store())), withitem( context_expr=Name(id='c'), optional_vars=Name(id='d', ctx=Store()))\], body=\[ Expr( value=Call( func=Name(id='something'), args=\[ Name(id='b'), Name(id='d')\]))\])\])

</div>

</div>

### Pattern matching

<div class="Match(subject, cases)">

A `match` statement. `subject` holds the subject of the match (the object that is being matched against the cases) and `cases` contains an iterable of `match_case` nodes with the different cases.

<div class="versionadded">

3.10

</div>

</div>

<div class="match_case(pattern, guard, body)">

A single case pattern in a `match` statement. `pattern` contains the match pattern that the subject will be matched against. Note that the `AST` nodes produced for patterns differ from those produced for expressions, even when they share the same syntax.

The `guard` attribute contains an expression that will be evaluated if the pattern matches the subject.

`body` contains a list of nodes to execute if the pattern matches and the result of evaluating the guard expression is true.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... match x: ... case \[x\] if x\>0: ... ... ... case tuple(): ... ... ... """), indent=4)) Module( body=\[ Match( subject=Name(id='x'), cases=\[ match_case( pattern=MatchSequence( patterns=\[ MatchAs(name='x')\]), guard=Compare( left=Name(id='x'), ops=\[ Gt()\], comparators=\[ Constant(value=0)\]), body=\[ Expr( value=Constant(value=Ellipsis))\]), match_case( pattern=MatchClass( cls=Name(id='tuple')), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="MatchValue(value)">

A match literal or value pattern that compares by equality. `value` is an expression node. Permitted value nodes are restricted as described in the match statement documentation. This pattern succeeds if the match subject is equal to the evaluated value.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... match x: ... case "Relevant": ... ... ... """), indent=4)) Module( body=\[ Match( subject=Name(id='x'), cases=\[ match_case( pattern=MatchValue( value=Constant(value='Relevant')), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="MatchSingleton(value)">

A match literal pattern that compares by identity. `value` is the singleton to be compared against: `None`, `True`, or `False`. This pattern succeeds if the match subject is the given constant.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... match x: ... case None: ... ... ... """), indent=4)) Module( body=\[ Match( subject=Name(id='x'), cases=\[ match_case( pattern=MatchSingleton(value=None), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="MatchSequence(patterns)">

A match sequence pattern. `patterns` contains the patterns to be matched against the subject elements if the subject is a sequence. Matches a variable length sequence if one of the subpatterns is a `MatchStar` node, otherwise matches a fixed length sequence.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... match x: ... case \[1, 2\]: ... ... ... """), indent=4)) Module( body=\[ Match( subject=Name(id='x'), cases=\[ match_case( pattern=MatchSequence( patterns=\[ MatchValue( value=Constant(value=1)), MatchValue( value=Constant(value=2))\]), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="MatchStar(name)">

Matches the rest of the sequence in a variable length match sequence pattern. If `name` is not `None`, a list containing the remaining sequence elements is bound to that name if the overall sequence pattern is successful.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... match x: ... case \[1, 2, *rest\]: ... ... ... case \[*\_\]: ... ... ... """), indent=4)) Module( body=\[ Match( subject=Name(id='x'), cases=\[ match_case( pattern=MatchSequence( patterns=\[ MatchValue( value=Constant(value=1)), MatchValue( value=Constant(value=2)), MatchStar(name='rest')\]), body=\[ Expr( value=Constant(value=Ellipsis))\]), match_case( pattern=MatchSequence( patterns=\[ MatchStar()\]), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="MatchMapping(keys, patterns, rest)">

A match mapping pattern. `keys` is a sequence of expression nodes. `patterns` is a corresponding sequence of pattern nodes. `rest` is an optional name that can be specified to capture the remaining mapping elements. Permitted key expressions are restricted as described in the match statement documentation.

This pattern succeeds if the subject is a mapping, all evaluated key expressions are present in the mapping, and the value corresponding to each key matches the corresponding subpattern. If `rest` is not `None`, a dict containing the remaining mapping elements is bound to that name if the overall mapping pattern is successful.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... match x: ... case {1: \_, 2: \_}: ... ... ... case {\*\*rest}: ... ... ... """), indent=4)) Module( body=\[ Match( subject=Name(id='x'), cases=\[ match_case( pattern=MatchMapping( keys=\[ Constant(value=1), Constant(value=2)\], patterns=\[ MatchAs(), MatchAs()\]), body=\[ Expr( value=Constant(value=Ellipsis))\]), match_case( pattern=MatchMapping(rest='rest'), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="MatchClass(cls, patterns, kwd_attrs, kwd_patterns)">

A match class pattern. `cls` is an expression giving the nominal class to be matched. `patterns` is a sequence of pattern nodes to be matched against the class defined sequence of pattern matching attributes. `kwd_attrs` is a sequence of additional attributes to be matched (specified as keyword arguments in the class pattern), `kwd_patterns` are the corresponding patterns (specified as keyword values in the class pattern).

This pattern succeeds if the subject is an instance of the nominated class, all positional patterns match the corresponding class-defined attributes, and any specified keyword attributes match their corresponding pattern.

Note: classes may define a property that returns self in order to match a pattern node against the instance being matched. Several builtin types are also matched that way, as described in the match statement documentation.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... match x: ... case Point2D(0, 0): ... ... ... case Point3D(x=0, y=0, z=0): ... ... ... """), indent=4)) Module( body=\[ Match( subject=Name(id='x'), cases=\[ match_case( pattern=MatchClass( cls=Name(id='Point2D'), patterns=\[ MatchValue( value=Constant(value=0)), MatchValue( value=Constant(value=0))\]), body=\[ Expr( value=Constant(value=Ellipsis))\]), match_case( pattern=MatchClass( cls=Name(id='Point3D'), kwd_attrs=\[ 'x', 'y', 'z'\], kwd_patterns=\[ MatchValue( value=Constant(value=0)), MatchValue( value=Constant(value=0)), MatchValue( value=Constant(value=0))\]), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="MatchAs(pattern, name)">

A match "as-pattern", capture pattern or wildcard pattern. `pattern` contains the match pattern that the subject will be matched against. If the pattern is `None`, the node represents a capture pattern (i.e a bare name) and will always succeed.

The `name` attribute contains the name that will be bound if the pattern is successful. If `name` is `None`, `pattern` must also be `None` and the node represents the wildcard pattern.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... match x: ... case \[x\] as y: ... ... ... case \_: ... ... ... """), indent=4)) Module( body=\[ Match( subject=Name(id='x'), cases=\[ match_case( pattern=MatchAs( pattern=MatchSequence( patterns=\[ MatchAs(name='x')\]), name='y'), body=\[ Expr( value=Constant(value=Ellipsis))\]), match_case( pattern=MatchAs(), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="MatchOr(patterns)">

A match "or-pattern". An or-pattern matches each of its subpatterns in turn to the subject, until one succeeds. The or-pattern is then deemed to succeed. If none of the subpatterns succeed the or-pattern fails. The `patterns` attribute contains a list of match pattern nodes that will be matched against the subject.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse(""" ... match x: ... case \[x\] \| (y): ... ... ... """), indent=4)) Module( body=\[ Match( subject=Name(id='x'), cases=\[ match_case( pattern=MatchOr( patterns=\[ MatchSequence( patterns=\[ MatchAs(name='x')\]), MatchAs(name='y')\]), body=\[ Expr( value=Constant(value=Ellipsis))\])\])\])

</div>

<div class="versionadded">

3.10

</div>

</div>

### Type annotations

<div class="TypeIgnore(lineno, tag)">

A `# type: ignore` comment located at *lineno*. *tag* is the optional tag specified by the form `# type: ignore <tag>`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('x = 1 \# type: ignore', type_comments=True), indent=4)) Module( body=\[ Assign( targets=\[ Name(id='x', ctx=Store())\], value=Constant(value=1))\], type_ignores=\[ TypeIgnore(lineno=1, tag='')\]) \>\>\> print(ast.dump(ast.parse('x: bool = 1 \# type: ignore\[assignment\]', type_comments=True), indent=4)) Module( body=\[ AnnAssign( target=Name(id='x', ctx=Store()), annotation=Name(id='bool'), value=Constant(value=1), simple=1)\], type_ignores=\[ TypeIgnore(lineno=1, tag='\[assignment\]')\])

</div>

> [!NOTE]
> `!TypeIgnore` nodes are not generated when the *type_comments* parameter is set to `False` (default). See `ast.parse` for more details.

<div class="versionadded">

3.8

</div>

</div>

### Type parameters

`Type parameters <type-params>` can exist on classes, functions, and type aliases.

<div class="TypeVar(name, bound, default_value)">

A `typing.TypeVar`. `name` is the name of the type variable. `bound` is the bound or constraints, if any. If `bound` is a `Tuple`, it represents constraints; otherwise it represents the bound. `default_value` is the default value; if the `!TypeVar` has no default, this attribute will be set to `None`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse("type Alias\[T: int = bool\] = list\[T\]"), indent=4)) Module( body=\[ TypeAlias( name=Name(id='Alias', ctx=Store()), type_params=\[ TypeVar( name='T', bound=Name(id='int'), default_value=Name(id='bool'))\], value=Subscript( value=Name(id='list'), slice=Name(id='T')))\])

</div>

<div class="versionadded">

3.12

</div>

<div class="versionchanged">

3.13 Added the *default_value* parameter.

</div>

</div>

<div class="ParamSpec(name, default_value)">

A `typing.ParamSpec`. `name` is the name of the parameter specification. `default_value` is the default value; if the `!ParamSpec` has no default, this attribute will be set to `None`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse("type Alias\[\*\*P = \[int, str\]\] = Callable\[P, int\]"), indent=4)) Module( body=\[ TypeAlias( name=Name(id='Alias', ctx=Store()), type_params=\[ ParamSpec( name='P', default_value=List( elts=\[ Name(id='int'), Name(id='str')\]))\], value=Subscript( value=Name(id='Callable'), slice=Tuple( elts=\[ Name(id='P'), Name(id='int')\])))\])

</div>

<div class="versionadded">

3.12

</div>

<div class="versionchanged">

3.13 Added the *default_value* parameter.

</div>

</div>

<div class="TypeVarTuple(name, default_value)">

A `typing.TypeVarTuple`. `name` is the name of the type variable tuple. `default_value` is the default value; if the `!TypeVarTuple` has no default, this attribute will be set to `None`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse("type Alias\[*Ts = ()\] = tuple\[*Ts\]"), indent=4)) Module( body=\[ TypeAlias( name=Name(id='Alias', ctx=Store()), type_params=\[ TypeVarTuple(name='Ts', default_value=Tuple())\], value=Subscript( value=Name(id='tuple'), slice=Tuple( elts=\[ Starred( value=Name(id='Ts'))\])))\])

</div>

<div class="versionadded">

3.12

</div>

<div class="versionchanged">

3.13 Added the *default_value* parameter.

</div>

</div>

### Function and class definitions

<div class="FunctionDef(name, args, body, decorator_list, returns, type_comment, type_params)">

A function definition.

- `name` is a raw string of the function name.
- `args` is an `arguments` node.
- `body` is the list of nodes inside the function.
- `decorator_list` is the list of decorators to be applied, stored outermost first (i.e. the first in the list will be applied last).
- `returns` is the return annotation.
- `type_params` is a list of `type parameters <ast-type-params>`.

<div class="attribute">

type_comment

`type_comment` is an optional string with the type annotation as a comment.

</div>

<div class="versionchanged">

3.12 Added `type_params`.

</div>

</div>

<div class="Lambda(args, body)">

`lambda` is a minimal function definition that can be used inside an expression. Unlike `FunctionDef`, `body` holds a single node.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('lambda x,y: ...'), indent=4)) Module( body=\[ Expr( value=Lambda( args=arguments( args=\[ arg(arg='x'), arg(arg='y')\]), body=Constant(value=Ellipsis)))\])

</div>

</div>

<div class="arguments(posonlyargs, args, vararg, kwonlyargs, kw_defaults, kwarg, defaults)">

The arguments for a function.

- `posonlyargs`, `args` and `kwonlyargs` are lists of `arg` nodes.
- `vararg` and `kwarg` are single `arg` nodes, referring to the `*args, **kwargs` parameters.
- `kw_defaults` is a list of default values for keyword-only arguments. If one is `None`, the corresponding argument is required.
- `defaults` is a list of default values for arguments that can be passed positionally. If there are fewer defaults, they correspond to the last n arguments.

</div>

<div class="arg(arg, annotation, type_comment)">

A single argument in a list. `arg` is a raw string of the argument name; `annotation` is its annotation, such as a `Name` node.

<div class="attribute">

type_comment

`type_comment` is an optional string with the type annotation as a comment

</div>

<div class="doctest">

\>\>\> print(ast.dump(ast.parse("""... @decorator1 ... @decorator2 ... def f(a: 'annotation', b=1, c=2, *d, e, f=3,*\*g) -\> 'return annotation': ... pass ... """), indent=4)) Module( body=\[ FunctionDef( name='f', args=arguments( args=\[ arg( arg='a', annotation=Constant(value='annotation')), arg(arg='b'), arg(arg='c')\], vararg=arg(arg='d'), kwonlyargs=\[ arg(arg='e'), arg(arg='f')\], kw_defaults=\[ None, Constant(value=3)\], kwarg=arg(arg='g'), defaults=\[ Constant(value=1), Constant(value=2)\]), body=\[ Pass()\], decorator_list=\[ Name(id='decorator1'), Name(id='decorator2')\], returns=Constant(value='return annotation'))\])

</div>

</div>

<div class="Return(value)">

A `return` statement.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('return 4'), indent=4)) Module( body=\[ Return( value=Constant(value=4))\])

</div>

</div>

<div class="Yield(value) YieldFrom(value)">

A `yield` or `yield from` expression. Because these are expressions, they must be wrapped in an `Expr` node if the value sent back is not used.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('yield x'), indent=4)) Module( body=\[ Expr( value=Yield( value=Name(id='x')))\])

\>\>\> print(ast.dump(ast.parse('yield from x'), indent=4)) Module( body=\[ Expr( value=YieldFrom( value=Name(id='x')))\])

</div>

</div>

<div class="Global(names) Nonlocal(names)">

`global` and `nonlocal` statements. `names` is a list of raw strings.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse('global x,y,z'), indent=4)) Module( body=\[ Global( names=\[ 'x', 'y', 'z'\])\])

\>\>\> print(ast.dump(ast.parse('nonlocal x,y,z'), indent=4)) Module( body=\[ Nonlocal( names=\[ 'x', 'y', 'z'\])\])

</div>

</div>

<div class="ClassDef(name, bases, keywords, body, decorator_list, type_params)">

A class definition.

- `name` is a raw string for the class name
- `bases` is a list of nodes for explicitly specified base classes.
- `keywords` is a list of `.keyword` nodes, principally for 'metaclass'. Other keywords will be passed to the metaclass, as per `3115`.
- `body` is a list of nodes representing the code within the class definition.
- `decorator_list` is a list of nodes, as in `FunctionDef`.
- `type_params` is a list of `type parameters <ast-type-params>`.

<div class="doctest">

\>\>\> print(ast.dump(ast.parse("""... @decorator1 ... @decorator2 ... class Foo(base1, base2, metaclass=meta): ... pass ... """), indent=4)) Module( body=\[ ClassDef( name='Foo', bases=\[ Name(id='base1'), Name(id='base2')\], keywords=\[ keyword( arg='metaclass', value=Name(id='meta'))\], body=\[ Pass()\], decorator_list=\[ Name(id='decorator1'), Name(id='decorator2')\])\])

</div>

<div class="versionchanged">

3.12 Added `type_params`.

</div>

</div>

### Async and await

<div class="AsyncFunctionDef(name, args, body, decorator_list, returns, type_comment, type_params)">

An `async def` function definition. Has the same fields as `FunctionDef`.

<div class="versionchanged">

3.12 Added `type_params`.

</div>

</div>

<div class="Await(value)">

An `await` expression. `value` is what it waits for. Only valid in the body of an `AsyncFunctionDef`.

</div>

<div class="doctest">

\>\>\> print(ast.dump(ast.parse("""... async def f(): ... await other_func() ... """), indent=4)) Module( body=\[ AsyncFunctionDef( name='f', args=arguments(), body=\[ Expr( value=Await( value=Call( func=Name(id='other_func'))))\])\])

</div>

<div class="AsyncFor(target, iter, body, orelse, type_comment) AsyncWith(items, body, type_comment)">

`async for` loops and `async with` context managers. They have the same fields as `For` and `With`, respectively. Only valid in the body of an `AsyncFunctionDef`.

</div>

> [!NOTE]
> When a string is parsed by `ast.parse`, operator nodes (subclasses of `!ast.operator`, `!ast.unaryop`, `!ast.cmpop`, `!ast.boolop` and `!ast.expr_context`) on the returned tree will be singletons. Changes to one will be reflected in all other occurrences of the same value (for example, `ast.Add`).

## `!ast` helpers

Apart from the node classes, the `!ast` module defines these utility functions and classes for traversing abstract syntax trees:

<div class="function">

parse(source, filename='\<unknown\>', mode='exec', \*, type_comments=False, feature_version=None, optimize=-1, module=None)

Parse the source into an AST node. Equivalent to `compile(source, filename, mode, flags=FLAGS_VALUE, optimize=optimize, module=module)`, where `FLAGS_VALUE` is `ast.PyCF_ONLY_AST` if `optimize <= 0` and `ast.PyCF_OPTIMIZED_AST` otherwise.

If `type_comments=True` is given, the parser is modified to check and return type comments as specified by `484` and `526`. This is equivalent to adding `ast.PyCF_TYPE_COMMENTS` to the flags passed to `compile`. This will report syntax errors for misplaced type comments. Without this flag, type comments will be ignored, and the `type_comment` field on selected AST nodes will always be `None`. In addition, the locations of `# type: ignore` comments will be returned as the `type_ignores` attribute of `Module` (otherwise it is always an empty list).

In addition, if `mode` is `'func_type'`, the input syntax is modified to correspond to `484` "signature type comments", for example `(str, int) -> List[str]`.

Setting `feature_version` to a tuple `(major, minor)` will result in a "best-effort" attempt to parse using that Python version's grammar. For example, setting `feature_version=(3, 9)` will attempt to disallow parsing of `match` statements. Currently `major` must equal to `3`. The lowest supported version is `(3, 7)` (and this may increase in future Python versions); the highest is `sys.version_info[0:2]`. "Best-effort" attempt means there is no guarantee that the parse (or success of the parse) is the same as when run on the Python version corresponding to `feature_version`.

If source contains a null character (`\0`), `ValueError` is raised.

> [!WARNING]
> Note that successfully parsing source code into an AST object doesn't guarantee that the source code provided is valid Python code that can be executed as the compilation step can raise further `SyntaxError` exceptions. For instance, the source `return 42` generates a valid AST node for a return statement, but it cannot be compiled alone (it needs to be inside a function node).
>
> In particular, `ast.parse` won't do any scoping checks, which the compilation step does.

> [!WARNING]
> It is possible to crash the Python interpreter with a sufficiently large/complex string due to stack depth limitations in Python's AST compiler.

<div class="versionchanged">

3.8 Added `type_comments`, `mode='func_type'` and `feature_version`.

</div>

<div class="versionchanged">

3.13 The minimum supported version for `feature_version` is now `(3, 7)`. The `optimize` argument was added.

</div>

<div class="versionadded">

3.15 Added the *module* parameter.

</div>

</div>

<div class="function">

unparse(ast_obj)

Unparse an `ast.AST` object and generate a string with code that would produce an equivalent `ast.AST` object if parsed back with `ast.parse`.

> [!WARNING]
> The produced code string will not necessarily be equal to the original code that generated the `ast.AST` object (without any compiler optimizations, such as constant tuples/frozensets).

> [!WARNING]
> Trying to unparse a highly complex expression would result with `RecursionError`.

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

literal_eval(node_or_string)

Evaluate an expression node or a string containing only a Python literal or container display. The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, `None` and `Ellipsis`.

This can be used for evaluating strings containing Python values without the need to parse the values oneself. It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing.

This function had been documented as "safe" in the past without defining what that meant. That was misleading. This is specifically designed not to execute Python code, unlike the more general `eval`. There is no namespace, no name lookups, or ability to call out. But it is not free from attack: A relatively small input can lead to memory exhaustion or to C stack exhaustion, crashing the process. There is also the possibility for excessive CPU consumption denial of service on some inputs. Calling it on untrusted data is thus not recommended.

> [!WARNING]
> It is possible to crash the Python interpreter due to stack depth limitations in Python's AST compiler.
>
> It can raise `ValueError`, `TypeError`, `SyntaxError`, `MemoryError` and `RecursionError` depending on the malformed input.

<div class="versionchanged">

3.2 Now allows bytes and set literals.

</div>

<div class="versionchanged">

3.9 Now supports creating empty sets with `'set()'`.

</div>

<div class="versionchanged">

3.10 For string inputs, leading spaces and tabs are now stripped.

</div>

</div>

<div class="function">

get_docstring(node, clean=True)

Return the docstring of the given *node* (which must be a `FunctionDef`, `AsyncFunctionDef`, `ClassDef`, or `Module` node), or `None` if it has no docstring. If *clean* is true, clean up the docstring's indentation with `inspect.cleandoc`.

<div class="versionchanged">

3.5 `AsyncFunctionDef` is now supported.

</div>

</div>

<div class="function">

get_source_segment(source, node, \*, padded=False)

Get source code segment of the *source* that generated *node*. If some location information (`~ast.AST.lineno`, `~ast.AST.end_lineno`, `~ast.AST.col_offset`, or `~ast.AST.end_col_offset`) is missing, return `None`.

If *padded* is `True`, the first line of a multi-line statement will be padded with spaces to match its original position.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

fix_missing_locations(node)

When you compile a node tree with `compile`, the compiler expects `~ast.AST.lineno` and `~ast.AST.col_offset` attributes for every node that supports them. This is rather tedious to fill in for generated nodes, so this helper adds these attributes recursively where not already set, by setting them to the values of the parent node. It works recursively starting at *node*.

</div>

<div class="function">

increment_lineno(node, n=1)

Increment the line number and end line number of each node in the tree starting at *node* by *n*. This is useful to "move code" to a different location in a file.

</div>

<div class="function">

copy_location(new_node, old_node)

Copy source location (`~ast.AST.lineno`, `~ast.AST.col_offset`, `~ast.AST.end_lineno`, and `~ast.AST.end_col_offset`) from *old_node* to *new_node* if possible, and return *new_node*.

</div>

<div class="function">

iter_fields(node)

Yield a tuple of `(fieldname, value)` for each field in `node._fields` that is present on *node*.

</div>

<div class="function">

iter_child_nodes(node)

Yield all direct child nodes of *node*, that is, all fields that are nodes and all items of fields that are lists of nodes.

</div>

<div class="function">

walk(node)

Recursively yield all descendant nodes in the tree starting at *node* (including *node* itself), in no specified order. This is useful if you only want to modify nodes in place and don't care about the context.

</div>

<div class="NodeVisitor()">

A node visitor base class that walks the abstract syntax tree and calls a visitor function for every node found. This function may return a value which is forwarded by the `visit` method.

This class is meant to be subclassed, with the subclass adding visitor methods.

<div class="method">

visit(node)

Visit a node. The default implementation calls the method called `self.visit_{classname}` where *classname* is the name of the node class, or `generic_visit` if that method doesn't exist.

</div>

<div class="method">

generic_visit(node)

This visitor calls `visit` on all children of the node.

Note that child nodes of nodes that have a custom visitor method won't be visited unless the visitor calls `generic_visit` or visits them itself.

</div>

<div class="method">

visit_Constant(node)

Handles all constant nodes.

</div>

Don't use the `NodeVisitor` if you want to apply changes to nodes during traversal. For this a special visitor exists (`NodeTransformer`) that allows modifications.

<div class="deprecated-removed">

3.8 3.14

Methods `!visit_Num`, `!visit_Str`, `!visit_Bytes`, `!visit_NameConstant` and `!visit_Ellipsis` will not be called in Python 3.14+. Add the `visit_Constant` method instead to handle all constant nodes.

</div>

</div>

<div class="NodeTransformer()">

A `NodeVisitor` subclass that walks the abstract syntax tree and allows modification of nodes.

The `NodeTransformer` will walk the AST and use the return value of the visitor methods to replace or remove the old node. If the return value of the visitor method is `None`, the node will be removed from its location, otherwise it is replaced with the return value. The return value may be the original node in which case no replacement takes place.

Here is an example transformer that rewrites all occurrences of name lookups (`foo`) to `data['foo']`:

    class RewriteName(NodeTransformer):

        def visit_Name(self, node):
            return Subscript(
                value=Name(id='data'),
                slice=Constant(value=node.id),
                ctx=node.ctx
            )

Keep in mind that if the node you're operating on has child nodes you must either transform the child nodes yourself or call the `~ast.NodeVisitor.generic_visit` method for the node first.

For nodes that were part of a collection of statements (that applies to all statement nodes), the visitor may also return a list of nodes rather than just a single node.

If `NodeTransformer` introduces new nodes (that weren't part of original tree) without giving them location information (such as `~ast.AST.lineno`), `fix_missing_locations` should be called with the new sub-tree to recalculate the location information:

    tree = ast.parse('foo', mode='eval')
    new_tree = fix_missing_locations(RewriteName().visit(tree))

Usually you use the transformer like this:

    node = YourTransformer().visit(node)

</div>

<div class="function">

dump(node, annotate_fields=True, include_attributes=False, \*, color=False, indent=None, show_empty=False)

Return a formatted dump of the tree in *node*. This is mainly useful for debugging purposes. If *annotate_fields* is true (by default), the returned string will show the names and the values for fields. If *annotate_fields* is false, the result string will be more compact by omitting unambiguous field names. Attributes such as line numbers and column offsets are not dumped by default. If this is wanted, *include_attributes* can be set to true.

If *color* is `True`, the returned string is syntax highlighted using ANSI escape sequences. If `False` (the default), colored output is always disabled.

If *indent* is a non-negative integer or string, then the tree will be pretty-printed with that indent level. An indent level of 0, negative, or `""` will only insert newlines. `None` (the default) selects the single line representation. Using a positive integer indent indents that many spaces per level. If *indent* is a string (such as `"\t"`), that string is used to indent each level.

If *show_empty* is false (the default), optional empty lists and `Load()` values will be omitted from the output. Optional `None` values are always omitted.

<div class="doctest">

\>\>\> tree = ast.parse('print(None)', '?', 'eval') \>\>\> print(ast.dump(tree, indent=4)) Expression( body=Call( func=Name(id='print'), args=\[ Constant(value=None)\])) \>\>\> print(ast.dump(tree, indent=4, show_empty=True)) Expression( body=Call( func=Name(id='print', ctx=Load()), args=\[ Constant(value=None)\], keywords=\[\]))

</div>

<div class="versionchanged">

3.9 Added the *indent* option.

</div>

<div class="versionchanged">

3.13 Added the *show_empty* option.

</div>

<div class="versionchanged">

3.15 Omit optional `Load()` values by default.

</div>

<div class="versionchanged">

3.15 Added the *color* parameter.

</div>

</div>

<div class="function">

compare(a, b, /, \*, compare_attributes=False)

Recursively compares two ASTs.

*compare_attributes* affects whether AST attributes are considered in the comparison. If *compare_attributes* is `False` (default), then attributes are ignored. Otherwise they must all be equal. This option is useful to check whether the ASTs are structurally equal but differ in whitespace or similar details. Attributes include line numbers and column offsets.

<div class="versionadded">

3.14

</div>

</div>

## Compiler flags

The following flags may be passed to `compile` in order to change effects on the compilation of a program:

<div class="data">

PyCF_ALLOW_TOP_LEVEL_AWAIT

Enables support for top-level `await`, `async for`, `async with` and async comprehensions.

<div class="versionadded">

3.8

</div>

</div>

<div class="data">

PyCF_ONLY_AST

Generates and returns an abstract syntax tree instead of returning a compiled code object.

</div>

<div class="data">

PyCF_OPTIMIZED_AST

The returned AST is optimized according to the *optimize* argument in `compile` or `ast.parse`.

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

PyCF_TYPE_COMMENTS

Enables support for `484` and `526` style type comments (`# type: <type>`, `# type: ignore <stuff>`).

<div class="versionadded">

3.8

</div>

</div>

## Command-line usage

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.15 The output is now syntax highlighted by default. This can be `controlled using environment variables <using-on-controlling-color>`.

</div>

The `!ast` module can be executed as a script from the command line. It is as simple as:

``` sh
python -m ast [-m <mode>] [-a] [infile]
```

The following options are accepted:

<div class="program">

ast

</div>

<div class="option">

`-h, --help`  

Show the help message and exit.

</div>

<div class="option">

`-m <mode>`  

`--mode <mode>`  

Specify what kind of code must be compiled, like the *mode* argument in `parse`.

</div>

<div class="option">

`--no-type-comments`  

Don't parse type comments.

</div>

<div class="option">

`-a, --include-attributes`  

Include attributes such as line numbers and column offsets.

</div>

<div class="option">

`-i <indent>`  

`--indent <indent>`  

Indentation of nodes in AST (number of spaces).

</div>

<div class="option">

`--feature-version <version>`  

Python version in the format 3.x (for example, 3.10). Defaults to the current version of the interpreter.

<div class="versionadded">

3.14

</div>

</div>

<div class="option">

`-O <level>`  

`--optimize <level>`  

Optimization level for parser. Defaults to no optimization.

<div class="versionadded">

3.14

</div>

</div>

<div class="option">

`--show-empty`  

Show empty lists and fields that are `None`. Defaults to not showing empty objects.

<div class="versionadded">

3.14

</div>

</div>

If `infile` is specified its contents are parsed to AST and dumped to stdout. Otherwise, the content is read from stdin.

<div class="seealso">

[Green Tree Snakes](https://greentreesnakes.readthedocs.io/), an external documentation resource, has good details on working with Python ASTs.

[ASTTokens](https://asttokens.readthedocs.io/en/latest/user-guide.html) annotates Python ASTs with the positions of tokens and text in the source code that generated them. This is helpful for tools that make source code transformations.

[leoAst.py](https://leo-editor.github.io/leo-editor/appendices.html#leoast-py) unifies the token-based and parse-tree-based views of python programs by inserting two-way links between tokens and ast nodes.

[LibCST](https://libcst.readthedocs.io/) parses code as a Concrete Syntax Tree that looks like an ast tree and keeps all formatting details. It's useful for building automated refactoring (codemod) applications and linters.

[Parso](https://parso.readthedocs.io) is a Python parser that supports error recovery and round-trip parsing for different Python versions (in multiple Python versions). Parso is also able to list multiple syntax errors in your Python file.

</div>

# `!asynchat` --- Asynchronous socket command/response handler

<div class="module" data-synopsis="Removed in 3.12." data-deprecated="">

asynchat

</div>

<div class="deprecated-removed">

3.6 3.12

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.12 <whatsnew312-removed>` after being deprecated in Python 3.6. The removal was decided in `594`.

Applications should use the `asyncio` module instead.

The last version of Python that provided the `!asynchat` module was [Python 3.11](https://docs.python.org/3.11/library/asynchat.html).

# asyncio-api-index

<div class="currentmodule">

asyncio

</div>

# High-level API Index

This page lists all high-level async/await enabled asyncio APIs.

## Tasks

Utilities to run asyncio programs, create Tasks, and await on multiple things with timeouts.

|  |  |
|----|----|
| `run` | Create event loop, run a coroutine, close the loop. |
| `Runner` | A context manager that simplifies multiple async function calls. |
| `Task` | Task object. |
| `TaskGroup` | A context manager that holds a group of tasks. Provides a convenient and reliable way to wait for all tasks in the group to finish. |
| `create_task` | Start an asyncio Task, then returns it. |
| `current_task` | Return the current Task. |
| `all_tasks` | Return all tasks that are not yet finished for an event loop. |
| `await` `sleep` | Sleep for a number of seconds. |
| `await` `gather` | Schedule and wait for things concurrently. |
| `await` `wait_for` | Run with a timeout. |
| `await` `shield` | Shield from cancellation. |
| `await` `wait` | Monitor for completion. |
| `timeout` | Run with a timeout. Useful in cases when `wait_for` is not suitable. |
| `to_thread` | Asynchronously run a function in a separate OS thread. |
| `run_coroutine_threadsafe` | Schedule a coroutine from another OS thread. |
| `for in` `as_completed` | Monitor for completion with a `for` loop. |

**Examples**

- `Using asyncio.gather() to run things in parallel
  <asyncio_example_gather>`.
- `Using asyncio.wait_for() to enforce a timeout
  <asyncio_example_waitfor>`.
- `Cancellation <asyncio_example_task_cancel>`.
- `Using asyncio.sleep() <asyncio_example_sleep>`.
- See also the main `Tasks documentation page <coroutine>`.

## Queues

Queues should be used to distribute work amongst multiple asyncio Tasks, implement connection pools, and pub/sub patterns.

|                 |                   |
|-----------------|-------------------|
| `Queue`         | A FIFO queue.     |
| `PriorityQueue` | A priority queue. |
| `LifoQueue`     | A LIFO queue.     |

**Examples**

- `Using asyncio.Queue to distribute workload between several
  Tasks <asyncio_example_queue_dist>`.
- See also the `Queues documentation page <asyncio-queues>`.

## Subprocesses

Utilities to spawn subprocesses and run shell commands.

|                                   |                      |
|-----------------------------------|----------------------|
| `await` `create_subprocess_exec`  | Create a subprocess. |
| `await` `create_subprocess_shell` | Run a shell command. |

**Examples**

- `Executing a shell command <asyncio_example_subprocess_shell>`.
- See also the `subprocess APIs <asyncio-subprocess>` documentation.

## Streams

High-level APIs to work with network IO.

|  |  |
|----|----|
| `await` `open_connection` | Establish a TCP connection. |
| `await` `open_unix_connection` | Establish a Unix socket connection. |
| `await` `start_server` | Start a TCP server. |
| `await` `start_unix_server` | Start a Unix socket server. |
| `StreamReader` | High-level async/await object to receive network data. |
| `StreamWriter` | High-level async/await object to send network data. |

**Examples**

- `Example TCP client <asyncio_example_stream>`.
- See also the `streams APIs <asyncio-streams>` documentation.

## Synchronization

Threading-like synchronization primitives that can be used in Tasks.

|                    |                      |
|--------------------|----------------------|
| `Lock`             | A mutex lock.        |
| `Event`            | An event object.     |
| `Condition`        | A condition object.  |
| `Semaphore`        | A semaphore.         |
| `BoundedSemaphore` | A bounded semaphore. |
| `Barrier`          | A barrier object.    |

**Examples**

- `Using asyncio.Event <asyncio_example_sync_event>`.
- `Using asyncio.Barrier <asyncio_example_barrier>`.
- See also the documentation of asyncio `synchronization primitives <asyncio-sync>`.

## Exceptions

|  |  |
|----|----|
| `asyncio.CancelledError` | Raised when a Task is cancelled. See also `Task.cancel`. |
| `asyncio.BrokenBarrierError` | Raised when a Barrier is broken. See also `Barrier.wait`. |

**Examples**

- `Handling CancelledError to run code on cancellation request
  <asyncio_example_task_cancel>`.
- See also the full list of `asyncio-specific exceptions <asyncio-exceptions>`.

# asyncio-dev

<div class="currentmodule">

asyncio

</div>

# Developing with asyncio

Asynchronous programming is different from classic "sequential" programming.

This page lists common mistakes and traps and explains how to avoid them.

## Debug Mode

By default asyncio runs in production mode. In order to ease the development asyncio has a *debug mode*.

There are several ways to enable asyncio debug mode:

- Setting the `PYTHONASYNCIODEBUG` environment variable to `1`.
- Using the `Python Development Mode <devmode>`.
- Passing `debug=True` to `asyncio.run`.
- Calling `loop.set_debug`.

In addition to enabling the debug mode, consider also:

- setting the log level of the `asyncio logger <asyncio-logger>` to `logging.DEBUG`, for example the following snippet of code can be run at startup of the application:

      logging.basicConfig(level=logging.DEBUG)

- configuring the `warnings` module to display `ResourceWarning` warnings. One way of doing that is by using the `-W` `default` command line option.

When the debug mode is enabled:

- Many non-threadsafe asyncio APIs (such as `loop.call_soon` and `loop.call_at` methods) raise an exception if they are called from a wrong thread.
- The execution time of the I/O selector is logged if it takes too long to perform an I/O operation.
- Callbacks taking longer than 100 milliseconds are logged. The `loop.slow_callback_duration` attribute can be used to set the minimum execution duration in seconds that is considered "slow".

## Concurrency and Multithreading

An event loop runs in a thread (typically the main thread) and executes all callbacks and Tasks in its thread. While a Task is running in the event loop, no other Tasks can run in the same thread. When a Task executes an `await` expression, the running Task gets suspended, and the event loop executes the next Task.

To schedule a `callback` from another OS thread, the `loop.call_soon_threadsafe` method should be used. Example:

    loop.call_soon_threadsafe(callback, *args)

Almost all asyncio objects are not thread safe, which is typically not a problem unless there is code that works with them from outside of a Task or a callback. If there's a need for such code to call a low-level asyncio API, the `loop.call_soon_threadsafe` method should be used, e.g.:

    loop.call_soon_threadsafe(fut.cancel)

To schedule a coroutine object from a different OS thread, the `run_coroutine_threadsafe` function should be used. It returns a `concurrent.futures.Future` to access the result:

    async def coro_func():
         return await asyncio.sleep(1, 42)

    # Later in another OS thread:

    future = asyncio.run_coroutine_threadsafe(coro_func(), loop)
    # Wait for the result:
    result = future.result()

To handle signals the event loop must be run in the main thread.

The `loop.run_in_executor` method can be used with a `concurrent.futures.ThreadPoolExecutor` or `~concurrent.futures.InterpreterPoolExecutor` to execute blocking code in a different OS thread without blocking the OS thread that the event loop runs in.

There is currently no way to schedule coroutines or callbacks directly from a different process (such as one started with `multiprocessing`). The `asyncio-event-loop-methods` section lists APIs that can read from pipes and watch file descriptors without blocking the event loop. In addition, asyncio's `Subprocess <asyncio-subprocess>` APIs provide a way to start a process and communicate with it from the event loop. Lastly, the aforementioned `loop.run_in_executor` method can also be used with a `concurrent.futures.ProcessPoolExecutor` to execute code in a different process.

## Running Blocking Code

Blocking (CPU-bound) code should not be called directly. For example, if a function performs a CPU-intensive calculation for 1 second, all concurrent asyncio Tasks and IO operations would be delayed by 1 second.

An executor can be used to run a task in a different thread, including in a different interpreter, or even in a different process to avoid blocking the OS thread with the event loop. See the `loop.run_in_executor` method for more details.

## Logging

asyncio uses the `logging` module and all logging is performed via the `"asyncio"` logger.

The default log level is `logging.INFO`, which can be easily adjusted:

    logging.getLogger("asyncio").setLevel(logging.WARNING)

Network logging can block the event loop. It is recommended to use a separate thread for handling logs or use non-blocking IO. For example, see `blocking-handlers`.

## Detect never-awaited coroutines

When a coroutine function is called, but not awaited (e.g. `coro()` instead of `await coro()`) or the coroutine is not scheduled with `asyncio.create_task`, asyncio will emit a `RuntimeWarning`:

    import asyncio

    async def test():
        print("never scheduled")

    async def main():
        test()

    asyncio.run(main())

Output:

    test.py:7: RuntimeWarning: coroutine 'test' was never awaited
      test()

Output in debug mode:

    test.py:7: RuntimeWarning: coroutine 'test' was never awaited
    Coroutine created at (most recent call last)
      File "../t.py", line 9, in <module>
        asyncio.run(main(), debug=True)

      < .. >

      File "../t.py", line 7, in main
        test()
      test()

The usual fix is to either await the coroutine or call the `asyncio.create_task` function:

    async def main():
        await test()

## Detect never-retrieved exceptions

If a `Future.set_exception` is called but the Future object is never awaited on, the exception would never be propagated to the user code. In this case, asyncio would emit a log message when the Future object is garbage collected.

Example of an unhandled exception:

    import asyncio

    async def bug():
        raise Exception("not consumed")

    async def main():
        asyncio.create_task(bug())

    asyncio.run(main())

Output:

    Task exception was never retrieved
    future: <Task finished coro=<bug() done, defined at test.py:3>
      exception=Exception('not consumed')>

    Traceback (most recent call last):
      File "test.py", line 4, in bug
        raise Exception("not consumed")
    Exception: not consumed

`Enable the debug mode <asyncio-debug-mode>` to get the traceback where the task was created:

    asyncio.run(main(), debug=True)

Output in debug mode:

    Task exception was never retrieved
    future: <Task finished coro=<bug() done, defined at test.py:3>
        exception=Exception('not consumed') created at asyncio/tasks.py:321>

    source_traceback: Object created at (most recent call last):
      File "../t.py", line 9, in <module>
        asyncio.run(main(), debug=True)

    < .. >

    Traceback (most recent call last):
      File "../t.py", line 4, in bug
        raise Exception("not consumed")
    Exception: not consumed

## Asynchronous generators best practices

Writing correct and efficient asyncio code requires awareness of certain pitfalls. This section outlines essential best practices that can save you hours of debugging.

### Close asynchronous generators explicitly

It is recommended to manually close the `asynchronous generator <asynchronous generator iterator>`. If a generator exits early - for example, due to an exception raised in the body of an `async for` loop - its asynchronous cleanup code may run in an unexpected context. This can occur after the tasks it depends on have completed, or during the event loop shutdown when the async-generator's garbage collection hook is called.

To avoid this, explicitly close the generator by calling its `~agen.aclose` method, or use the `contextlib.aclosing` context manager:

    import asyncio
    import contextlib

    async def gen():
      yield 1
      yield 2

    async def func():
      async with contextlib.aclosing(gen()) as g:
        async for x in g:
          break  # Don't iterate until the end

    asyncio.run(func())

As noted above, the cleanup code for these asynchronous generators is deferred. The following example demonstrates that the finalization of an asynchronous generator can occur in an unexpected order:

    import asyncio
    work_done = False

    async def cursor():
        try:
            yield 1
        finally:
            assert work_done

    async def rows():
        global work_done
        try:
            yield 2
        finally:
            await asyncio.sleep(0.1) # imitate some async work
            work_done = True


    async def main():
        async for c in cursor():
            async for r in rows():
                break
            break

    asyncio.run(main())

For this example, we get the following output:

    unhandled exception during asyncio.run() shutdown
    task: <Task finished name='Task-3' coro=<<async_generator_athrow without __name__>()> exception=AssertionError()>
    Traceback (most recent call last):
      File "example.py", line 6, in cursor
        yield 1
    asyncio.exceptions.CancelledError

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "example.py", line 8, in cursor
        assert work_done
               ^^^^^^^^^
    AssertionError

The `cursor()` asynchronous generator was finalized before the `rows` generator - an unexpected behavior.

The example can be fixed by explicitly closing the `cursor` and `rows` async-generators:

    async def main():
        async with contextlib.aclosing(cursor()) as cursor_gen:
            async for c in cursor_gen:
                async with contextlib.aclosing(rows()) as rows_gen:
                    async for r in rows_gen:
                        break
                break

### Create asynchronous generators only when the event loop is running

It is recommended to create `asynchronous generators <asynchronous generator iterator>` only after the event loop has been created.

To ensure that asynchronous generators close reliably, the event loop uses the `sys.set_asyncgen_hooks` function to register callback functions. These callbacks update the list of running asynchronous generators to keep it in a consistent state.

When the `loop.shutdown_asyncgens() <asyncio.loop.shutdown_asyncgens>` function is called, the running generators are stopped gracefully and the list is cleared.

The asynchronous generator invokes the corresponding system hook during its first iteration. At the same time, the generator records that the hook has been called and does not call it again.

Therefore, if iteration begins before the event loop is created, the event loop will not be able to add the generator to its list of active generators because the hooks are set after the generator attempts to call them. Consequently, the event loop will not be able to terminate the generator if necessary.

Consider the following example:

    import asyncio

    async def agenfn():
        try:
            yield 10
        finally:
            await asyncio.sleep(0)


    with asyncio.Runner() as runner:
        agen = agenfn()
        print(runner.run(anext(agen)))
        del agen

Output:

    10
    Exception ignored while closing generator <async_generator object agenfn at 0x000002F71CD10D70>:
    Traceback (most recent call last):
      File "example.py", line 13, in <module>
        del agen
            ^^^^
    RuntimeError: async generator ignored GeneratorExit

This example can be fixed as follows:

    import asyncio

    async def agenfn():
        try:
            yield 10
        finally:
            await asyncio.sleep(0)

    async def main():
        agen = agenfn()
        print(await anext(agen))
        del agen

    asyncio.run(main())

### Avoid concurrent iteration and closure of the same generator

Async generators may be reentered while another `~agen.__anext__` / `~agen.athrow` / `~agen.aclose` call is in progress. This may lead to an inconsistent state of the async generator and can cause errors.

Let's consider the following example:

    import asyncio

    async def consumer():
        for idx in range(100):
            await asyncio.sleep(0)
            message = yield idx
            print('received', message)

    async def amain():
        agenerator = consumer()
        await agenerator.asend(None)

        fa = asyncio.create_task(agenerator.asend('A'))
        fb = asyncio.create_task(agenerator.asend('B'))
        await fa
        await fb

    asyncio.run(amain())

Output:

    received A
    Traceback (most recent call last):
      File "test.py", line 38, in <module>
        asyncio.run(amain())
        ~~~~~~~~~~~^^^^^^^^^
      File "Lib/asyncio/runners.py", line 204, in run
        return runner.run(main)
               ~~~~~~~~~~^^^^^^
      File "Lib/asyncio/runners.py", line 127, in run
        return self._loop.run_until_complete(task)
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
      File "Lib/asyncio/base_events.py", line 719, in run_until_complete
        return future.result()
               ~~~~~~~~~~~~~^^
      File "test.py", line 36, in amain
        await fb
    RuntimeError: anext(): asynchronous generator is already running

Therefore, it is recommended to avoid using asynchronous generators in parallel tasks or across multiple event loops.

# asyncio-eventloop

<div class="currentmodule">

asyncio

</div>

# Event loop

**Source code:** `Lib/asyncio/events.py`, `Lib/asyncio/base_events.py`

------------------------------------------------------------------------

**Preface**

The event loop is the core of every asyncio application. Event loops run asynchronous tasks and callbacks, perform network IO operations, and run subprocesses.

Application developers should typically use the high-level asyncio functions, such as `asyncio.run`, and should rarely need to reference the loop object or call its methods. This section is intended mostly for authors of lower-level code, libraries, and frameworks, who need finer control over the event loop behavior.

**Obtaining the Event Loop**

The following low-level functions can be used to get, set, or create an event loop:

<div class="function">

get_running_loop()

Return the running event loop in the current OS thread.

Raise a `RuntimeError` if there is no running event loop.

This function can only be called from a coroutine or a callback.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

get_event_loop()

Get the current event loop.

When called from a coroutine or a callback (e.g. scheduled with call_soon or similar API), this function will always return the running event loop.

If there is no running event loop set, the function will return the loop set by `set_event_loop`, or raise a `RuntimeError` if no loop has been set.

Because this function has rather complex behavior, using the `get_running_loop` function is preferred to `get_event_loop` in coroutines and callbacks.

As noted above, consider using the higher-level `asyncio.run` function, instead of using these lower level functions to manually create and close an event loop.

<div class="versionchanged">

3.14 Raises a `RuntimeError` if there is no current event loop.

</div>

</div>

<div class="function">

set_event_loop(loop)

Set *loop* as the current event loop for the current OS thread.

</div>

<div class="function">

new_event_loop()

Create and return a new event loop object.

</div>

**Contents**

This documentation page contains the following sections:

- The [Event Loop Methods](#event-loop-methods) section is the reference documentation of the event loop APIs;
- The [Callback Handles](#callback-handles) section documents the `Handle` and `TimerHandle` instances which are returned from scheduling methods such as `loop.call_soon` and `loop.call_later`;
- The [Server Objects](#server-objects) section documents types returned from event loop methods like `loop.create_server`;
- The [Event Loop Implementations](#event-loop-implementations) section documents the `SelectorEventLoop` and `ProactorEventLoop` classes;
- The [Examples](#examples) section showcases how to work with some event loop APIs.

## Event loop methods

Event loops have **low-level** APIs for the following:

<div class="contents" data-depth="1" data-local="">

</div>

### Running and stopping the loop

<div class="method">

loop.run_until_complete(future)

Run until the *future* (an instance of `Future`) has completed.

If the argument is a `coroutine object <coroutine>` it is implicitly scheduled to run as a `asyncio.Task`.

Return the Future's result or raise its exception.

</div>

<div class="method">

loop.run_forever()

Run the event loop until `stop` is called.

If `stop` is called before `run_forever` is called, the loop will poll the I/O selector once with a timeout of zero, run all callbacks scheduled in response to I/O events (and those that were already scheduled), and then exit.

If `stop` is called while `run_forever` is running, the loop will run the current batch of callbacks and then exit. Note that new callbacks scheduled by callbacks will not run in this case; instead, they will run the next time `run_forever` or `run_until_complete` is called.

</div>

<div class="method">

loop.stop()

Stop the event loop.

</div>

<div class="method">

loop.is_running()

Return `True` if the event loop is currently running.

</div>

<div class="method">

loop.is_closed()

Return `True` if the event loop was closed.

</div>

<div class="method">

loop.close()

Close the event loop.

The loop must not be running when this function is called. Any pending callbacks will be discarded.

This method clears all queues and shuts down the executor, but does not wait for the executor to finish.

This method is idempotent and irreversible. No other methods should be called after the event loop is closed.

</div>

<div class="method" async="">

loop.shutdown_asyncgens()

Schedule all currently open `asynchronous generator` objects to close with an `~agen.aclose` call. After calling this method, the event loop will issue a warning if a new asynchronous generator is iterated. This should be used to reliably finalize all scheduled asynchronous generators.

Note that there is no need to call this function when `asyncio.run` is used.

Example:

    try:
        loop.run_forever()
    finally:
        loop.run_until_complete(loop.shutdown_asyncgens())
        loop.close()

<div class="versionadded">

3.6

</div>

</div>

<div class="method" async="">

loop.shutdown_default_executor(timeout=None)

Schedule the closure of the default executor and wait for it to join all of the threads in the `~concurrent.futures.ThreadPoolExecutor`. Once this method has been called, using the default executor with `loop.run_in_executor` will raise a `RuntimeError`.

The *timeout* parameter specifies the amount of time (in `float` seconds) the executor will be given to finish joining. With the default, `None`, the executor is allowed an unlimited amount of time.

If the *timeout* is reached, a `RuntimeWarning` is emitted and the default executor is terminated without waiting for its threads to finish joining.

> [!NOTE]
> Do not call this method when using `asyncio.run`, as the latter handles default executor shutdown automatically.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.12 Added the *timeout* parameter.

</div>

</div>

### Scheduling callbacks

<div class="method">

loop.call_soon(callback, \*args, context=None)

Schedule the *callback* `callback` to be called with *args* arguments at the next iteration of the event loop.

Return an instance of `asyncio.Handle`, which can be used later to cancel the callback.

Callbacks are called in the order in which they are registered. Each callback will be called exactly once.

The optional keyword-only *context* argument specifies a custom `contextvars.Context` for the *callback* to run in. Callbacks use the current context when no *context* is provided.

Unlike `call_soon_threadsafe`, this method is not thread-safe.

</div>

<div class="method">

loop.call_soon_threadsafe(callback, \*args, context=None)

A thread-safe variant of `call_soon`. When scheduling callbacks from another thread, this function *must* be used, since `call_soon` is not thread-safe.

This function is safe to be called from a reentrant context or signal handler, however, it is not safe or fruitful to use the returned handle in such contexts.

Raises `RuntimeError` if called on a loop that's been closed. This can happen on a secondary thread when the main application is shutting down.

See the `concurrency and multithreading <asyncio-multithreading>` section of the documentation.

<div class="versionchanged">

3.7 The *context* keyword-only parameter was added. See `567` for more details.

</div>

</div>

<div id="asyncio-pass-keywords">

> [!NOTE]
> Most `asyncio` scheduling functions don't allow passing keyword arguments. To do that, use `functools.partial`:
>
>     # will schedule "print("Hello", flush=True)"
>     loop.call_soon(
>         functools.partial(print, "Hello", flush=True))
>
> Using partial objects is usually more convenient than using lambdas, as asyncio can render partial objects better in debug and error messages.

</div>

### Scheduling delayed callbacks

Event loop provides mechanisms to schedule callback functions to be called at some point in the future. Event loop uses monotonic clocks to track time.

<div class="method">

loop.call_later(delay, callback, \*args, context=None)

Schedule *callback* to be called after the given *delay* number of seconds (can be either an int or a float).

An instance of `asyncio.TimerHandle` is returned which can be used to cancel the callback.

*callback* will be called exactly once. If two callbacks are scheduled for exactly the same time, the order in which they are called is undefined.

The optional positional *args* will be passed to the callback when it is called. Use `functools.partial` `to pass keyword arguments <asyncio-pass-keywords>` to *callback*.

An optional keyword-only *context* argument allows specifying a custom `contextvars.Context` for the *callback* to run in. The current context is used when no *context* is provided.

> [!NOTE]
> For performance, callbacks scheduled with `loop.call_later` may run up to one clock-resolution early (see `time.get_clock_info('monotonic').resolution`).

<div class="versionchanged">

3.7 The *context* keyword-only parameter was added. See `567` for more details.

</div>

<div class="versionchanged">

3.8 In Python 3.7 and earlier with the default event loop implementation, the *delay* could not exceed one day. This has been fixed in Python 3.8.

</div>

</div>

<div class="method">

loop.call_at(when, callback, \*args, context=None)

Schedule *callback* to be called at the given absolute timestamp *when* (an int or a float), using the same time reference as `loop.time`.

This method's behavior is the same as `call_later`.

An instance of `asyncio.TimerHandle` is returned which can be used to cancel the callback.

> [!NOTE]
> For performance, callbacks scheduled with `loop.call_at` may run up to one clock-resolution early (see `time.get_clock_info('monotonic').resolution`).

<div class="versionchanged">

3.7 The *context* keyword-only parameter was added. See `567` for more details.

</div>

<div class="versionchanged">

3.8 In Python 3.7 and earlier with the default event loop implementation, the difference between *when* and the current time could not exceed one day. This has been fixed in Python 3.8.

</div>

</div>

<div class="method">

loop.time()

Return the current time, as a `float` value, according to the event loop's internal monotonic clock.

</div>

> [!NOTE]
> .. versionchanged:: 3.8 In Python 3.7 and earlier timeouts (relative *delay* or absolute *when*) should not exceed one day. This has been fixed in Python 3.8.

<div class="seealso">

The `asyncio.sleep` function.

</div>

### Creating futures and tasks

<div class="method">

loop.create_future()

Create an `asyncio.Future` object attached to the event loop.

This is the preferred way to create Futures in asyncio. This lets third-party event loops provide alternative implementations of the Future object (with better performance or instrumentation).

<div class="versionadded">

3.5.2

</div>

</div>

<div class="method">

loop.create_task(coro, *, name=None, context=None, eager_start=None,*\*kwargs)

Schedule the execution of `coroutine <coroutine>` *coro*. Return a `Task` object.

Third-party event loops can use their own subclass of `Task` for interoperability. In this case, the result type is a subclass of `Task`.

The full function signature is largely the same as that of the `Task` constructor (or factory) - all of the keyword arguments to this function are passed through to that interface.

If the *name* argument is provided and not `None`, it is set as the name of the task using `Task.set_name`.

An optional keyword-only *context* argument allows specifying a custom `contextvars.Context` for the *coro* to run in. The current context copy is created when no *context* is provided.

An optional keyword-only *eager_start* argument allows specifying if the task should execute eagerly during the call to create_task, or be scheduled later. If *eager_start* is not passed the mode set by `loop.set_task_factory` will be used.

<div class="versionchanged">

3.8 Added the *name* parameter.

</div>

<div class="versionchanged">

3.11 Added the *context* parameter.

</div>

<div class="versionchanged">

3.13.3 Added `kwargs` which passes on arbitrary extra parameters, including `name` and `context`.

</div>

<div class="versionchanged">

3.13.4 Rolled back the change that passes on *name* and *context* (if it is None), while still passing on other arbitrary keyword arguments (to avoid breaking backwards compatibility with 3.13.3).

</div>

<div class="versionchanged">

3.14 All *kwargs* are now passed on. The *eager_start* parameter works with eager task factories.

</div>

</div>

<div class="method">

loop.set_task_factory(factory)

Set a task factory that will be used by `loop.create_task`.

If *factory* is `None` the default task factory will be set. Otherwise, *factory* must be a *callable* with the signature matching `(loop, coro, **kwargs)`, where *loop* is a reference to the active event loop, and *coro* is a coroutine object. The callable must pass on all *kwargs*, and return a `asyncio.Task`-compatible object.

<div class="versionchanged">

3.13.3 Required that all *kwargs* are passed on to `asyncio.Task`.

</div>

<div class="versionchanged">

3.13.4 *name* is no longer passed to task factories. *context* is no longer passed to task factories if it is `None`.

<div class="versionchanged">

3.14 *name* and *context* are now unconditionally passed on to task factories again.

</div>

</div>

</div>

<div class="method">

loop.get_task_factory()

Return a task factory or `None` if the default one is in use.

</div>

### Opening network connections

<div class="method" async="">

loop.create_connection(protocol_factory, host=None, port=None, \*, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None, happy_eyeballs_delay=None, interleave=None, all_errors=False)

Open a streaming transport connection to a given address specified by *host* and *port*.

The socket family can be either `~socket.AF_INET` or `~socket.AF_INET6` depending on *host* (or the *family* argument, if provided).

The socket type will be `~socket.SOCK_STREAM`.

*protocol_factory* must be a callable returning an `asyncio protocol <asyncio-protocol>` implementation.

This method will try to establish the connection in the background. When successful, it returns a `(transport, protocol)` pair.

The chronological synopsis of the underlying operation is as follows:

1.  The connection is established and a `transport <asyncio-transport>` is created for it.
2.  *protocol_factory* is called without arguments and is expected to return a `protocol <asyncio-protocol>` instance.
3.  The protocol instance is coupled with the transport by calling its `~BaseProtocol.connection_made` method.
4.  A `(transport, protocol)` tuple is returned on success.

The created transport is an implementation-dependent bidirectional stream.

Other arguments:

- *ssl*: if given and not false, a SSL/TLS transport is created (by default a plain TCP transport is created). If *ssl* is a `ssl.SSLContext` object, this context is used to create the transport; if *ssl* is `True`, a default context returned from `ssl.create_default_context` is used.

  <div class="seealso">

  `SSL/TLS security considerations <ssl-security>`

  </div>

- *server_hostname* sets or overrides the hostname that the target server's certificate will be matched against. Should only be passed if *ssl* is not `None`. By default the value of the *host* argument is used. If *host* is empty, there is no default and you must pass a value for *server_hostname*. If *server_hostname* is an empty string, hostname matching is disabled (which is a serious security risk, allowing for potential man-in-the-middle attacks).

- *family*, *proto*, *flags* are the optional address family, protocol and flags to be passed through to getaddrinfo() for *host* resolution. If given, these should all be integers from the corresponding `socket` module constants.

- *happy_eyeballs_delay*, if given, enables Happy Eyeballs for this connection. It should be a floating-point number representing the amount of time in seconds to wait for a connection attempt to complete, before starting the next attempt in parallel. This is the "Connection Attempt Delay" as defined in `8305`. A sensible default value recommended by the RFC is `0.25` (250 milliseconds).

- *interleave* controls address reordering when a host name resolves to multiple IP addresses. If `0` or unspecified, no reordering is done, and addresses are tried in the order returned by `getaddrinfo`. If a positive integer is specified, the addresses are interleaved by address family, and the given integer is interpreted as "First Address Family Count" as defined in `8305`. The default is `0` if *happy_eyeballs_delay* is not specified, and `1` if it is.

- *sock*, if given, should be an existing, already connected `socket.socket` object to be used by the transport. If *sock* is given, none of *host*, *port*, *family*, *proto*, *flags*, *happy_eyeballs_delay*, *interleave* and *local_addr* should be specified.

  > [!NOTE]
  > The *sock* argument transfers ownership of the socket to the transport created. To close the socket, call the transport's `~asyncio.BaseTransport.close` method.

- *local_addr*, if given, is a `(local_host, local_port)` tuple used to bind the socket locally. The *local_host* and *local_port* are looked up using `getaddrinfo()`, similarly to *host* and *port*.

- *ssl_handshake_timeout* is (for a TLS connection) the time in seconds to wait for the TLS handshake to complete before aborting the connection. `60.0` seconds if `None` (default).

- *ssl_shutdown_timeout* is the time in seconds to wait for the SSL shutdown to complete before aborting the connection. `30.0` seconds if `None` (default).

- *all_errors* determines what exceptions are raised when a connection cannot be created. By default, only a single `Exception` is raised: the first exception if there is only one or all errors have same message, or a single `OSError` with the error messages combined. When `all_errors` is `True`, an `ExceptionGroup` will be raised containing all exceptions (even if there is only one).

<div class="versionchanged">

3.5

Added support for SSL/TLS in `ProactorEventLoop`.

</div>

<div class="versionchanged">

3.6

The socket option `socket.TCP_NODELAY <socket-unix-constants>` is set by default for all TCP connections.

</div>

<div class="versionchanged">

3.7

Added the *ssl_handshake_timeout* parameter.

</div>

<div class="versionchanged">

3.8

Added the *happy_eyeballs_delay* and *interleave* parameters.

Happy Eyeballs Algorithm: Success with Dual-Stack Hosts. When a server's IPv4 path and protocol are working, but the server's IPv6 path and protocol are not working, a dual-stack client application experiences significant connection delay compared to an IPv4-only client. This is undesirable because it causes the dual-stack client to have a worse user experience. This document specifies requirements for algorithms that reduce this user-visible delay and provides an algorithm.

For more information: <https://datatracker.ietf.org/doc/html/rfc6555>

</div>

<div class="versionchanged">

3.11

Added the *ssl_shutdown_timeout* parameter.

</div>

<div class="versionchanged">

3.12 *all_errors* was added.

</div>

<div class="versionchanged">

3.16 Raises a `ValueError` if `ssl.check_hostname` is `True` and `server_hostname` is not supplied.

</div>

<div class="seealso">

The `open_connection` function is a high-level alternative API. It returns a pair of (`StreamReader`, `StreamWriter`) that can be used directly in async/await code.

</div>

</div>

<div class="method" async="">

loop.create_datagram_endpoint(protocol_factory, local_addr=None, remote_addr=None, \*, family=0, proto=0, flags=0, reuse_port=None, allow_broadcast=None, sock=None)

Create a datagram connection.

The socket family can be either `~socket.AF_INET`, `~socket.AF_INET6`, or `~socket.AF_UNIX`, depending on *host* (or the *family* argument, if provided).

The socket type will be `~socket.SOCK_DGRAM`.

*protocol_factory* must be a callable returning a `protocol <asyncio-protocol>` implementation.

A tuple of `(transport, protocol)` is returned on success.

Other arguments:

- *local_addr*, if given, is a `(local_host, local_port)` tuple used to bind the socket locally. The *local_host* and *local_port* are looked up using `getaddrinfo`.

  > [!NOTE]
  > On Windows, when using the proactor event loop with `local_addr=None`, an `OSError` with `!errno.WSAEINVAL` will be raised when running it.

- *remote_addr*, if given, is a `(remote_host, remote_port)` tuple used to connect the socket to a remote address. The *remote_host* and *remote_port* are looked up using `getaddrinfo`.

- *family*, *proto*, *flags* are the optional address family, protocol and flags to be passed through to `getaddrinfo` for *host* resolution. If given, these should all be integers from the corresponding `socket` module constants.

- *reuse_port* tells the kernel to allow this endpoint to be bound to the same port as other existing endpoints are bound to, so long as they all set this flag when being created. This option is not supported on Windows and some Unixes. If the `socket.SO_REUSEPORT <socket-unix-constants>` constant is not defined then this capability is unsupported.

- *allow_broadcast* tells the kernel to allow this endpoint to send messages to the broadcast address.

- *sock* can optionally be specified in order to use a preexisting, already connected, `socket.socket` object to be used by the transport. If specified, *local_addr* and *remote_addr* should be omitted (must be `None`).

  > [!NOTE]
  > The *sock* argument transfers ownership of the socket to the transport created. To close the socket, call the transport's `~asyncio.BaseTransport.close` method.

See `UDP echo client protocol <asyncio-udp-echo-client-protocol>` and `UDP echo server protocol <asyncio-udp-echo-server-protocol>` examples.

<div class="versionchanged">

3.4.4 The *family*, *proto*, *flags*, *reuse_address*, *reuse_port*, *allow_broadcast*, and *sock* parameters were added.

</div>

<div class="versionchanged">

3.8 Added support for Windows.

</div>

<div class="versionchanged">

3.8.1 The *reuse_address* parameter is no longer supported, as using `socket.SO_REUSEADDR <socket-unix-constants>` poses a significant security concern for UDP. Explicitly passing `reuse_address=True` will raise an exception.

When multiple processes with differing UIDs assign sockets to an identical UDP socket address with `SO_REUSEADDR`, incoming packets can become randomly distributed among the sockets.

For supported platforms, *reuse_port* can be used as a replacement for similar functionality. With *reuse_port*, `socket.SO_REUSEPORT <socket-unix-constants>` is used instead, which specifically prevents processes with differing UIDs from assigning sockets to the same socket address.

</div>

<div class="versionchanged">

3.11 The *reuse_address* parameter, disabled since Python 3.8.1, 3.7.6 and 3.6.10, has been entirely removed.

</div>

</div>

<div class="method" async="">

loop.create_unix_connection(protocol_factory, path=None, \*, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None)

Create a Unix connection.

The socket family will be `~socket.AF_UNIX`; socket type will be `~socket.SOCK_STREAM`.

A tuple of `(transport, protocol)` is returned on success.

*path* is the name of a Unix domain socket and is required, unless a *sock* parameter is specified. Abstract Unix sockets, `str`, `bytes`, and `~pathlib.Path` paths are supported.

See the documentation of the `loop.create_connection` method for information about arguments to this method.

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.7 Added the *ssl_handshake_timeout* parameter. The *path* parameter can now be a `path-like object`.

</div>

<div class="versionchanged">

3.11

Added the *ssl_shutdown_timeout* parameter.

</div>

</div>

### Creating network servers

<div id="loop_create_server">

<div class="method" async="">

loop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, keep_alive=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None, start_serving=True)

Create a TCP server (socket type `~socket.SOCK_STREAM`) listening on *port* of the *host* address.

Returns a `Server` object.

Arguments:

- *protocol_factory* must be a callable returning a `protocol <asyncio-protocol>` implementation.

- The *host* parameter can be set to several types which determine where the server would be listening:

  - If *host* is a string, the TCP server is bound to a single network interface specified by *host*.
  - If *host* is a sequence of strings, the TCP server is bound to all network interfaces specified by the sequence.
  - If *host* is an empty string or `None`, all interfaces are assumed and a list of multiple sockets will be returned (most likely one for IPv4 and another one for IPv6).

- The *port* parameter can be set to specify which port the server should listen on. If `0` or `None` (the default), a random unused port will be selected (note that if *host* resolves to multiple network interfaces, a different random port will be selected for each interface).

- *family* can be set to either `socket.AF_INET` or `~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set, the *family* will be determined from host name (defaults to `~socket.AF_UNSPEC`).

- *flags* is a bitmask for `getaddrinfo`.

- *sock* can optionally be specified in order to use a preexisting socket object. If specified, *host* and *port* must not be specified.

  > [!NOTE]
  > The *sock* argument transfers ownership of the socket to the server created. To close the socket, call the server's `~asyncio.Server.close` method.

- *backlog* is the maximum number of queued connections passed to `~socket.socket.listen` (defaults to 100).

- *ssl* can be set to an `~ssl.SSLContext` instance to enable TLS over the accepted connections.

- *reuse_address* tells the kernel to reuse a local socket in `TIME_WAIT` state, without waiting for its natural timeout to expire. If not specified will automatically be set to `True` on Unix.

- *reuse_port* tells the kernel to allow this endpoint to be bound to the same port as other existing endpoints are bound to, so long as they all set this flag when being created. This option is not supported on Windows.

- *keep_alive* set to `True` keeps connections active by enabling the periodic transmission of messages.

<div class="versionchanged">

3.13

Added the *keep_alive* parameter.

</div>

- *ssl_handshake_timeout* is (for a TLS server) the time in seconds to wait for the TLS handshake to complete before aborting the connection. `60.0` seconds if `None` (default).
- *ssl_shutdown_timeout* is the time in seconds to wait for the SSL shutdown to complete before aborting the connection. `30.0` seconds if `None` (default).
- *start_serving* set to `True` (the default) causes the created server to start accepting connections immediately. When set to `False`, the user should await on `Server.start_serving` or `Server.serve_forever` to make the server to start accepting connections.

<div class="versionchanged">

3.5

Added support for SSL/TLS in `ProactorEventLoop`.

</div>

<div class="versionchanged">

3.5.1

The *host* parameter can be a sequence of strings.

</div>

<div class="versionchanged">

3.6

Added *ssl_handshake_timeout* and *start_serving* parameters. The socket option `socket.TCP_NODELAY <socket-unix-constants>` is set by default for all TCP connections.

</div>

<div class="versionchanged">

3.11

Added the *ssl_shutdown_timeout* parameter.

</div>

<div class="seealso">

The `start_server` function is a higher-level alternative API that returns a pair of `StreamReader` and `StreamWriter` that can be used in an async/await code.

</div>

</div>

</div>

<div class="method" async="">

loop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None, start_serving=True, cleanup_socket=True, mode=None)

Similar to `loop.create_server` but works with the `~socket.AF_UNIX` socket family.

*path* is the name of a Unix domain socket, and is required, unless a *sock* argument is provided. Abstract Unix sockets, `str`, `bytes`, and `~pathlib.Path` paths are supported.

If *cleanup_socket* is true then the Unix socket will automatically be removed from the filesystem when the server is closed, unless the socket has been replaced after the server has been created.

If *mode* is not `None`, the permissions of the socket file created for *path* are changed to *mode* (as accepted by `os.chmod`) right after binding, before the server starts accepting connections, so a connection can never be accepted while the default, umask-derived permissions are still in effect. *mode* cannot be combined with *sock* and is not supported for abstract Unix sockets.

See the documentation of the `loop.create_server` method for information about arguments to this method.

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.7

Added the *ssl_handshake_timeout* and *start_serving* parameters. The *path* parameter can now be a `~pathlib.Path` object.

</div>

<div class="versionchanged">

3.11

Added the *ssl_shutdown_timeout* parameter.

</div>

<div class="versionchanged">

3.13

Added the *cleanup_socket* parameter.

</div>

<div class="versionchanged">

3.16

Added the *mode* parameter.

</div>

</div>

<div class="method" async="">

loop.connect_accepted_socket(protocol_factory, sock, \*, ssl=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None)

Wrap an already accepted connection into a transport/protocol pair.

This method can be used by servers that accept connections outside of asyncio but that use asyncio to handle them.

Parameters:

- *protocol_factory* must be a callable returning a `protocol <asyncio-protocol>` implementation.

- *sock* is a preexisting socket object returned from `socket.accept <socket.socket.accept>`.

  > [!NOTE]
  > The *sock* argument transfers ownership of the socket to the transport created. To close the socket, call the transport's `~asyncio.BaseTransport.close` method.

- *ssl* can be set to an `~ssl.SSLContext` to enable SSL over the accepted connections.

- *ssl_handshake_timeout* is (for an SSL connection) the time in seconds to wait for the SSL handshake to complete before aborting the connection. `60.0` seconds if `None` (default).

- *ssl_shutdown_timeout* is the time in seconds to wait for the SSL shutdown to complete before aborting the connection. `30.0` seconds if `None` (default).

Returns a `(transport, protocol)` pair.

<div class="versionadded">

3.5.3

</div>

<div class="versionchanged">

3.7

Added the *ssl_handshake_timeout* parameter.

</div>

<div class="versionchanged">

3.11

Added the *ssl_shutdown_timeout* parameter.

</div>

</div>

### Transferring files

<div class="method" async="">

loop.sendfile(transport, file, offset=0, count=None, \*, fallback=True)

Send a *file* over a *transport*. Return the total number of bytes sent.

The method uses high-performance `os.sendfile` if available.

*file* must be a regular file object opened in binary mode.

*offset* tells from where to start reading the file. If specified, *count* is the total number of bytes to transmit as opposed to sending the file until EOF is reached. File position is always updated, even when this method raises an error, and `file.tell() <io.IOBase.tell>` can be used to obtain the actual number of bytes sent.

*fallback* set to `True` makes asyncio to manually read and send the file when the platform does not support the sendfile system call (e.g. Windows or SSL socket on Unix).

Raise `SendfileNotAvailableError` if the system does not support the *sendfile* syscall and *fallback* is `False`.

<div class="versionadded">

3.7

</div>

</div>

### TLS upgrade

<div class="method" async="">

loop.start_tls(transport, protocol, sslcontext, \*, server_side=False, server_hostname=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None)

Upgrade an existing transport-based connection to TLS.

Create a TLS coder/decoder instance and insert it between the *transport* and the *protocol*. The coder/decoder implements both *transport*-facing protocol and *protocol*-facing transport.

Return the created two-interface instance. After *await*, the *protocol* must stop using the original *transport* and communicate with the returned object only because the coder caches *protocol*-side data and sporadically exchanges extra TLS session packets with *transport*.

In some situations (e.g. when the passed transport is already closing) this may return `None`.

Parameters:

- *transport* and *protocol* instances that methods like `~loop.create_server` and `~loop.create_connection` return.
- *sslcontext*: a configured instance of `~ssl.SSLContext`.
- *server_side* pass `True` when a server-side connection is being upgraded (like the one created by `~loop.create_server`).
- *server_hostname*: sets or overrides the host name that the target server's certificate will be matched against.
- *ssl_handshake_timeout* is (for a TLS connection) the time in seconds to wait for the TLS handshake to complete before aborting the connection. `60.0` seconds if `None` (default).
- *ssl_shutdown_timeout* is the time in seconds to wait for the SSL shutdown to complete before aborting the connection. `30.0` seconds if `None` (default).

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.11

Added the *ssl_shutdown_timeout* parameter.

</div>

</div>

### Watching file descriptors

<div class="method">

loop.add_reader(fd, callback, \*args)

Start monitoring the *fd* file descriptor for read availability and invoke *callback* with the specified arguments once *fd* is available for reading.

Any preexisting callback registered for *fd* is cancelled and replaced by *callback*.

</div>

<div class="method">

loop.remove_reader(fd)

Stop monitoring the *fd* file descriptor for read availability. Returns `True` if *fd* was previously being monitored for reads.

</div>

<div class="method">

loop.add_writer(fd, callback, \*args)

Start monitoring the *fd* file descriptor for write availability and invoke *callback* with the specified arguments *args* once *fd* is available for writing.

Any preexisting callback registered for *fd* is cancelled and replaced by *callback*.

Use `functools.partial` `to pass keyword arguments
<asyncio-pass-keywords>` to *callback*.

</div>

<div class="method">

loop.remove_writer(fd)

Stop monitoring the *fd* file descriptor for write availability. Returns `True` if *fd* was previously being monitored for writes.

</div>

See also `Platform Support <asyncio-platform-support>` section for some limitations of these methods.

### Working with socket objects directly

In general, protocol implementations that use transport-based APIs such as `loop.create_connection` and `loop.create_server` are faster than implementations that work with sockets directly. However, there are some use cases when performance is not critical, and working with `~socket.socket` objects directly is more convenient.

<div class="method" async="">

loop.sock_recv(sock, nbytes)

Receive up to *nbytes* from *sock*. Asynchronous version of `socket.recv() <socket.socket.recv>`.

Return the received data as a bytes object.

*sock* must be a non-blocking socket.

<div class="versionchanged">

3.7 Even though this method was always documented as a coroutine method, releases before Python 3.7 returned a `Future`. Since Python 3.7 this is an `async def` method.

</div>

</div>

<div class="method" async="">

loop.sock_recv_into(sock, buf)

Receive data from *sock* into the *buf* buffer. Modeled after the blocking `socket.recv_into() <socket.socket.recv_into>` method.

Return the number of bytes written to the buffer.

*sock* must be a non-blocking socket.

<div class="versionadded">

3.7

</div>

</div>

<div class="method" async="">

loop.sock_recvfrom(sock, bufsize)

Receive a datagram of up to *bufsize* from *sock*. Asynchronous version of `socket.recvfrom() <socket.socket.recvfrom>`.

Return a tuple of (received data, remote address).

*sock* must be a non-blocking socket.

<div class="versionadded">

3.11

</div>

</div>

<div class="method" async="">

loop.sock_recvfrom_into(sock, buf, nbytes=0)

Receive a datagram of up to *nbytes* from *sock* into *buf*. Asynchronous version of `socket.recvfrom_into() <socket.socket.recvfrom_into>`.

Return a tuple of (number of bytes received, remote address).

*sock* must be a non-blocking socket.

<div class="versionadded">

3.11

</div>

</div>

<div class="method" async="">

loop.sock_sendall(sock, data)

Send *data* to the *sock* socket. Asynchronous version of `socket.sendall() <socket.socket.sendall>`.

This method continues to send to the socket until either all data in *data* has been sent or an error occurs. `None` is returned on success. On error, an exception is raised. Additionally, there is no way to determine how much data, if any, was successfully processed by the receiving end of the connection.

*sock* must be a non-blocking socket.

<div class="versionchanged">

3.7 Even though the method was always documented as a coroutine method, before Python 3.7 it returned a `Future`. Since Python 3.7, this is an `async def` method.

</div>

</div>

<div class="method" async="">

loop.sock_sendto(sock, data, address)

Send a datagram from *sock* to *address*. Asynchronous version of `socket.sendto() <socket.socket.sendto>`.

Return the number of bytes sent.

*sock* must be a non-blocking socket.

<div class="versionadded">

3.11

</div>

</div>

<div class="method" async="">

loop.sock_connect(sock, address)

Connect *sock* to a remote socket at *address*.

Asynchronous version of `socket.connect() <socket.socket.connect>`.

*sock* must be a non-blocking socket.

With `SelectorEventLoop`, *address* does not need to be resolved: for `~socket.AF_INET` and `~socket.AF_INET6` sockets, `sock_connect` first checks whether *address* is already resolved by calling `socket.inet_pton`, and uses `loop.getaddrinfo` to resolve it if it is not.

`ProactorEventLoop`, the default event loop on Windows, does not resolve *address*. The host must already be a numeric IP address; passing a host name raises `OSError`. Resolve the address with `loop.getaddrinfo` first, or use `loop.create_connection`, which resolves the address on every platform.

<div class="versionchanged">

3.5.2 With `SelectorEventLoop`, `address` no longer needs to be resolved.

</div>

<div class="seealso">

`loop.create_connection` and `asyncio.open_connection() <open_connection>`.

</div>

</div>

<div class="method" async="">

loop.sock_accept(sock)

Accept a connection. Modeled after the blocking `socket.accept() <socket.socket.accept>` method.

The socket must be bound to an address and listening for connections. The return value is a pair `(conn, address)` where *conn* is a *new* socket object usable to send and receive data on the connection, and *address* is the address bound to the socket on the other end of the connection.

*sock* must be a non-blocking socket.

<div class="versionchanged">

3.7 Even though the method was always documented as a coroutine method, before Python 3.7 it returned a `Future`. Since Python 3.7, this is an `async def` method.

</div>

<div class="seealso">

`loop.create_server` and `start_server`.

</div>

</div>

<div class="method" async="">

loop.sock_sendfile(sock, file, offset=0, count=None, \*, fallback=True)

Send a file using high-performance `os.sendfile` if possible. Return the total number of bytes sent.

Asynchronous version of `socket.sendfile() <socket.socket.sendfile>`.

*sock* must be a non-blocking `socket.SOCK_STREAM` `~socket.socket`.

*file* must be a regular file object open in binary mode.

*offset* tells from where to start reading the file. If specified, *count* is the total number of bytes to transmit as opposed to sending the file until EOF is reached. File position is always updated, even when this method raises an error, and `file.tell() <io.IOBase.tell>` can be used to obtain the actual number of bytes sent.

*fallback*, when set to `True`, makes asyncio manually read and send the file when the platform does not support the sendfile syscall (e.g. Windows or SSL socket on Unix).

Raise `SendfileNotAvailableError` if the system does not support *sendfile* syscall and *fallback* is `False`.

*sock* must be a non-blocking socket.

<div class="versionadded">

3.7

</div>

</div>

### DNS

<div class="method" async="">

loop.getaddrinfo(host, port, \*, family=0, type=0, proto=0, flags=0)

Asynchronous version of `socket.getaddrinfo`.

</div>

<div class="method" async="">

loop.getnameinfo(sockaddr, flags=0)

Asynchronous version of `socket.getnameinfo`.

</div>

> [!NOTE]
> Both *getaddrinfo* and *getnameinfo* internally utilize their synchronous versions through the loop's default thread pool executor. When this executor is saturated, these methods may experience delays, which higher-level networking libraries may report as increased timeouts. To mitigate this, consider using a custom executor for other user tasks, or setting a default executor with a larger number of workers.

<div class="versionchanged">

3.7 Both *getaddrinfo* and *getnameinfo* methods were always documented to return a coroutine, but prior to Python 3.7 they were, in fact, returning `asyncio.Future` objects. Starting with Python 3.7 both methods are coroutines.

</div>

### Working with pipes

<div class="method" async="">

loop.connect_read_pipe(protocol_factory, pipe)

Register the read end of *pipe* in the event loop.

*protocol_factory* must be a callable returning an `asyncio protocol <asyncio-protocol>` implementation.

*pipe* is a `file-like object <file object>`. See `Supported pipe objects <asyncio-pipe-objects>` for the objects supported as *pipe*.

Return pair `(transport, protocol)`, where *transport* supports the `ReadTransport` interface and *protocol* is an object instantiated by the *protocol_factory*.

With `SelectorEventLoop` event loop, the *pipe* is set to non-blocking mode.

</div>

<div class="method" async="">

loop.connect_write_pipe(protocol_factory, pipe)

Register the write end of *pipe* in the event loop.

*protocol_factory* must be a callable returning an `asyncio protocol <asyncio-protocol>` implementation.

*pipe* is a `file-like object <file object>`. See `Supported pipe objects <asyncio-pipe-objects>` for the objects supported as *pipe*.

Return pair `(transport, protocol)`, where *transport* supports `WriteTransport` interface and *protocol* is an object instantiated by the *protocol_factory*.

With `SelectorEventLoop` event loop, the *pipe* is set to non-blocking mode.

</div>

<div id="asyncio-pipe-objects">

**Supported pipe objects**

</div>

These methods only work with objects the operating system can poll for readiness or perform overlapped I/O on. Regular files on disk are **not** supported on any platform. There is no asynchronous file I/O in asyncio; use `loop.run_in_executor` to read and write regular files without blocking the event loop.

On Unix, with `SelectorEventLoop`, *pipe* must wrap one of the following:

- a pipe, such as an end of an `os.pipe` pair or a FIFO created with `os.mkfifo`;
- a socket;
- a character device, such as a terminal.

On Windows, where only `ProactorEventLoop` implements these methods, *pipe* must wrap a handle opened for overlapped I/O (that is, created with the `FILE_FLAG_OVERLAPPED` flag), since the handle has to be associated with an I/O completion port. Handles that were not opened for overlapped I/O are rejected. In particular, the standard streams (`sys.stdin`, `sys.stdout` and `sys.stderr`), console handles, and the pipes created by `os.pipe` are **not** opened for overlapped I/O and therefore cannot be used with these methods.

> [!NOTE]
> `SelectorEventLoop` does not support the above methods on Windows. Use `ProactorEventLoop` instead for Windows.

<div class="seealso">

The `loop.subprocess_exec` and `loop.subprocess_shell` methods.

</div>

### Unix signals

<div id="loop_add_signal_handler">

<div class="method">

loop.add_signal_handler(signum, callback, \*args)

Set *callback* as the handler for the *signum* signal, passing *args* as positional arguments.

The callback will be invoked by *loop*, along with other queued callbacks and runnable coroutines of that event loop. Unlike signal handlers registered using `signal.signal`, a callback registered with this function is allowed to interact with the event loop.

Raise `ValueError` if the signal number is invalid or uncatchable. Raise `RuntimeError` if there is a problem setting up the handler.

Use `functools.partial` `to pass keyword arguments
<asyncio-pass-keywords>` to *callback*.

Like `signal.signal`, this function must be invoked in the main thread.

</div>

</div>

<div class="method">

loop.remove_signal_handler(sig)

Remove the handler for the *sig* signal.

Return `True` if the signal handler was removed, or `False` if no handler was set for the given signal.

<div class="availability">

Unix.

</div>

</div>

<div class="seealso">

The `signal` module.

</div>

### Executing code in thread or process pools

<div class="awaitablemethod">

loop.run_in_executor(executor, func, \*args)

Arrange for *func* to be called in the specified executor passing *args* as positional arguments.

The *executor* argument should be an `concurrent.futures.Executor` instance. The default executor is used if *executor* is `None`. The default executor can be set by `loop.set_default_executor`, otherwise, a `concurrent.futures.ThreadPoolExecutor` will be lazy-initialized and used by `run_in_executor` if needed.

Example:

    import asyncio
    import concurrent.futures

    def blocking_io():
        # File operations (such as logging) can block the
        # event loop: run them in a thread pool.
        with open('/dev/urandom', 'rb') as f:
            return f.read(100)

    def cpu_bound():
        # CPU-bound operations will block the event loop:
        # in general it is preferable to run them in a
        # process pool.
        return sum(i * i for i in range(10 ** 7))

    async def main():
        loop = asyncio.get_running_loop()

        ## Options:

        # 1. Run in the default loop's executor:
        result = await loop.run_in_executor(
            None, blocking_io)
        print('default thread pool', result)

        # 2. Run in a custom thread pool:
        with concurrent.futures.ThreadPoolExecutor() as pool:
            result = await loop.run_in_executor(
                pool, blocking_io)
            print('custom thread pool', result)

        # 3. Run in a custom process pool:
        with concurrent.futures.ProcessPoolExecutor() as pool:
            result = await loop.run_in_executor(
                pool, cpu_bound)
            print('custom process pool', result)

        # 4. Run in a custom interpreter pool:
        with concurrent.futures.InterpreterPoolExecutor() as pool:
            result = await loop.run_in_executor(
                pool, cpu_bound)
            print('custom interpreter pool', result)

    if __name__ == '__main__':
        asyncio.run(main())

Note that the entry point guard (`if __name__ == '__main__'`) is required for option 3 due to the peculiarities of `multiprocessing`, which is used by `~concurrent.futures.ProcessPoolExecutor`. See `Safe importing of main module <multiprocessing-safe-main-import>`.

This method returns a `asyncio.Future` object.

Use `functools.partial` `to pass keyword arguments
<asyncio-pass-keywords>` to *func*.

<div class="versionchanged">

3.5.3 `loop.run_in_executor` no longer configures the `max_workers` of the thread pool executor it creates, instead leaving it up to the thread pool executor (`~concurrent.futures.ThreadPoolExecutor`) to set the default.

</div>

</div>

<div class="method">

loop.set_default_executor(executor)

Set *executor* as the default executor used by `run_in_executor`. *executor* must be an instance of `~concurrent.futures.ThreadPoolExecutor`, which includes `~concurrent.futures.InterpreterPoolExecutor`.

<div class="versionchanged">

3.11 *executor* must be an instance of `~concurrent.futures.ThreadPoolExecutor`.

</div>

</div>

### Error handling API

Allows customizing how exceptions are handled in the event loop.

<div class="method">

loop.set_exception_handler(handler)

Set *handler* as the new event loop exception handler.

If *handler* is `None`, the default exception handler will be set. Otherwise, *handler* must be a callable with the signature matching `(loop, context)`, where `loop` is a reference to the active event loop, and `context` is a `dict` object containing the details of the exception (see `call_exception_handler` documentation for details about context).

If the handler is called on behalf of a `~asyncio.Task` or `~asyncio.Handle`, it is run in the `contextvars.Context` of that task or callback handle.

<div class="versionchanged">

3.12

The handler may be called in the `~contextvars.Context` of the task or handle where the exception originated.

</div>

</div>

<div class="method">

loop.get_exception_handler()

Return the current exception handler, or `None` if no custom exception handler was set.

<div class="versionadded">

3.5.2

</div>

</div>

<div class="method">

loop.default_exception_handler(context)

Default exception handler.

This is called when an exception occurs and no exception handler is set. This can be called by a custom exception handler that wants to defer to the default handler behavior.

*context* parameter has the same meaning as in `call_exception_handler`.

</div>

<div class="method">

loop.call_exception_handler(context)

Call the current event loop exception handler.

*context* is a `dict` object containing the following keys (new keys may be introduced in future Python versions):

- 'message': Error message;
- 'exception' (optional): Exception object;
- 'future' (optional): `asyncio.Future` instance;
- 'task' (optional): `asyncio.Task` instance;
- 'handle' (optional): `asyncio.Handle` instance;
- 'protocol' (optional): `Protocol <asyncio-protocol>` instance;
- 'transport' (optional): `Transport <asyncio-transport>` instance;
- 'socket' (optional): `socket.socket` instance;
- 'source_traceback' (optional): Traceback of the source;
- 'handle_traceback' (optional): Traceback of the handle;
- 'asyncgen' (optional): Asynchronous generator that caused the exception.

> [!NOTE]
> This method should not be overloaded in subclassed event loops. For custom exception handling, use the `set_exception_handler` method.

</div>

### Enabling debug mode

<div class="method">

loop.get_debug()

Get the debug mode (`bool`) of the event loop.

The default value is `True` if the environment variable `PYTHONASYNCIODEBUG` is set to a non-empty string, `False` otherwise.

</div>

<div class="method">

loop.set_debug(enabled: bool)

Set the debug mode of the event loop.

<div class="versionchanged">

3.7

The new `Python Development Mode <devmode>` can now also be used to enable the debug mode.

</div>

</div>

<div class="attribute">

loop.slow_callback_duration

This attribute can be used to set the minimum execution duration in seconds that is considered "slow". When debug mode is enabled, "slow" callbacks are logged.

Default value is 100 milliseconds.

</div>

<div class="seealso">

The `debug mode of asyncio <asyncio-debug-mode>`.

</div>

### Running subprocesses

Methods described in this subsections are low-level. In regular async/await code consider using the high-level `asyncio.create_subprocess_shell` and `asyncio.create_subprocess_exec` convenience functions instead.

> [!NOTE]
> On Windows, the default event loop `ProactorEventLoop` supports subprocesses, whereas `SelectorEventLoop` does not. See `Subprocess Support on Windows <asyncio-windows-subprocess>` for details.

<div id="loop_subprocess_exec">

<div class="method" async="">

loop.subprocess_exec(protocol_factory, *args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,*\*kwargs)

Create a subprocess from one or more string arguments specified by *args*.

*args* must be a list of strings represented by:

- `str`;
- or `bytes`, encoded to the `filesystem encoding <filesystem-encoding>`.

The first string specifies the program executable, and the remaining strings specify the arguments. Together, string arguments form the `argv` of the program.

This is similar to the standard library `subprocess.Popen` class called with `shell=False` and the list of strings passed as the first argument; however, where `~subprocess.Popen` takes a single argument which is list of strings, *subprocess_exec* takes multiple string arguments.

The *protocol_factory* must be a callable returning a subclass of the `asyncio.SubprocessProtocol` class.

Other parameters:

- *stdin* can be any of these:

  - a file-like object
  - an existing file descriptor (a positive integer), for example those created with `os.pipe`
  - the `subprocess.PIPE` constant (default) which will create a new pipe and connect it,
  - the value `None` which will make the subprocess inherit the file descriptor from this process
  - the `subprocess.DEVNULL` constant which indicates that the special `os.devnull` file will be used

- *stdout* can be any of these:

  - a file-like object
  - the `subprocess.PIPE` constant (default) which will create a new pipe and connect it,
  - the value `None` which will make the subprocess inherit the file descriptor from this process
  - the `subprocess.DEVNULL` constant which indicates that the special `os.devnull` file will be used

- *stderr* can be any of these:

  - a file-like object
  - the `subprocess.PIPE` constant (default) which will create a new pipe and connect it,
  - the value `None` which will make the subprocess inherit the file descriptor from this process
  - the `subprocess.DEVNULL` constant which indicates that the special `os.devnull` file will be used
  - the `subprocess.STDOUT` constant which will connect the standard error stream to the process' standard output stream

- All other keyword arguments are passed to `subprocess.Popen` without interpretation, except for *bufsize*, *universal_newlines*, *shell*, *text*, *encoding* and *errors*, which should not be specified at all.

  The `asyncio` subprocess API does not support decoding the streams as text. `bytes.decode` can be used to convert the bytes returned from the stream to text.

If a file-like object passed as *stdin*, *stdout* or *stderr* represents a pipe, then the other side of this pipe should be registered with `~loop.connect_write_pipe` or `~loop.connect_read_pipe` for use with the event loop.

See the constructor of the `subprocess.Popen` class for documentation on other arguments.

Returns a pair of `(transport, protocol)`, where *transport* conforms to the `asyncio.SubprocessTransport` base class and *protocol* is an object instantiated by the *protocol_factory*.

If the transport is closed or is garbage collected, the child process is killed if it is still running.

</div>

</div>

<div class="method" async="">

loop.subprocess_shell(protocol_factory, cmd, *, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,*\*kwargs)

Create a subprocess from *cmd*, which can be a `str` or a `bytes` string encoded to the `filesystem encoding <filesystem-encoding>`, using the platform's "shell" syntax.

This is similar to the standard library `subprocess.Popen` class called with `shell=True`.

The *protocol_factory* must be a callable returning a subclass of the `SubprocessProtocol` class.

See `~loop.subprocess_exec` for more details about the remaining arguments.

Returns a pair of `(transport, protocol)`, where *transport* conforms to the `SubprocessTransport` base class and *protocol* is an object instantiated by the *protocol_factory*.

If the transport is closed or is garbage collected, the child process is killed if it is still running.

</div>

> [!NOTE]
> It is the application's responsibility to ensure that all whitespace and special characters are quoted appropriately to avoid [shell injection](https://en.wikipedia.org/wiki/Shell_injection#Shell_injection) vulnerabilities. The `shlex.quote` function can be used to properly escape whitespace and special characters in strings that are going to be used to construct shell commands.

## Callback handles

<div class="Handle">

A callback wrapper object returned by `loop.call_soon`, `loop.call_soon_threadsafe`.

<div class="method">

get_context()

Return the `contextvars.Context` object associated with the handle.

<div class="versionadded">

3.12

</div>

</div>

<div class="method">

cancel()

Cancel the callback. If the callback has already been canceled or executed, this method has no effect.

</div>

<div class="method">

cancelled()

Return `True` if the callback was cancelled.

<div class="versionadded">

3.7

</div>

</div>

</div>

<div class="TimerHandle">

A callback wrapper object returned by `loop.call_later`, and `loop.call_at`.

This class is a subclass of `Handle`.

<div class="method">

when()

Return a scheduled callback time as `float` seconds.

The time is an absolute timestamp, using the same time reference as `loop.time`.

<div class="versionadded">

3.7

</div>

</div>

</div>

## Server objects

Server objects are created by `loop.create_server`, `loop.create_unix_server`, `start_server`, and `start_unix_server` functions.

Do not instantiate the `Server` class directly.

<div class="Server">

*Server* objects are asynchronous context managers. When used in an `async with` statement, it's guaranteed that the Server object is closed and not accepting new connections when the `async with` statement is completed:

    srv = await loop.create_server(...)

    async with srv:
        # some code

    # At this point, srv is closed and no longer accepts new connections.

<div class="versionchanged">

3.7 Server object is an asynchronous context manager since Python 3.7.

</div>

<div class="versionchanged">

3.11 This class was exposed publicly as `asyncio.Server` in Python 3.9.11, 3.10.3 and 3.11.

</div>

<div class="method">

close()

Stop serving: close listening sockets and set the `sockets` attribute to `None`.

The sockets that represent existing incoming client connections are left open.

The server is closed asynchronously; use the `wait_closed` coroutine to wait until the server is closed (and no more connections are active).

</div>

<div class="method">

close_clients()

Close all existing incoming client connections.

Calls `~asyncio.BaseTransport.close` on all associated transports.

`close` should be called before `close_clients` when closing the server to avoid races with new clients connecting.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

abort_clients()

Close all existing incoming client connections immediately, without waiting for pending operations to complete.

Calls `~asyncio.WriteTransport.abort` on all associated transports.

`close` should be called before `abort_clients` when closing the server to avoid races with new clients connecting.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

get_loop()

Return the event loop associated with the server object.

<div class="versionadded">

3.7

</div>

</div>

<div class="method" async="">

start_serving()

Start accepting connections.

This method is idempotent, so it can be called when the server is already serving.

The *start_serving* keyword-only parameter to `loop.create_server` and `asyncio.start_server` allows creating a Server object that is not accepting connections initially. In this case `Server.start_serving()`, or `Server.serve_forever` can be used to make the Server start accepting connections.

<div class="versionadded">

3.7

</div>

</div>

<div class="method" async="">

serve_forever()

Start accepting connections until the coroutine is cancelled. Cancellation of `serve_forever` task causes the server to be closed.

This method can be called if the server is already accepting connections. Only one `serve_forever` task can exist per one *Server* object.

Example:

    async def client_connected(reader, writer):
        # Communicate with the client with
        # reader/writer streams.  For example:
        await reader.readline()

    async def main(host, port):
        srv = await asyncio.start_server(
            client_connected, host, port)
        await srv.serve_forever()

    asyncio.run(main('127.0.0.1', 0))

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

is_serving()

Return `True` if the server is accepting new connections.

<div class="versionadded">

3.7

</div>

</div>

<div class="method" async="">

wait_closed()

Wait until the `close` method completes and all active connections have finished.

<div class="versionchanged">

3.12 `wait_closed()` now waits until the server is closed and all active connections have finished. Previously, it returned immediately if the server was already closed, even if connections were still active.

</div>

</div>

<div class="attribute">

sockets

List of socket-like objects, `asyncio.trsock.TransportSocket`, which the server is listening on.

<div class="versionchanged">

3.7 Prior to Python 3.7 `Server.sockets` used to return an internal list of server sockets directly. In 3.7 a copy of that list is returned.

</div>

</div>

</div>

## Event loop implementations<span id="asyncio-event-loops"></span>

asyncio ships with two different event loop implementations: `SelectorEventLoop` and `ProactorEventLoop`.

By default asyncio is configured to use `EventLoop`.

<div class="SelectorEventLoop">

A subclass of `AbstractEventLoop` based on the `selectors` module.

Uses the most efficient *selector* available for the given platform. It is also possible to manually configure the exact selector implementation to be used:

    import asyncio
    import selectors

    async def main():
       ...

    loop_factory = lambda: asyncio.SelectorEventLoop(selectors.SelectSelector())
    asyncio.run(main(), loop_factory=loop_factory)

<div class="availability">

Unix, Windows.

</div>

</div>

<div class="ProactorEventLoop">

A subclass of `AbstractEventLoop` for Windows that uses "I/O Completion Ports" (IOCP).

<div class="availability">

Windows.

</div>

<div class="seealso">

[MSDN documentation on I/O Completion Ports](https://learn.microsoft.com/windows/win32/fileio/i-o-completion-ports).

</div>

</div>

<div class="EventLoop">

An alias to the most efficient available subclass of `AbstractEventLoop` for the given platform.

It is an alias to `SelectorEventLoop` on Unix and `ProactorEventLoop` on Windows.

<div class="versionadded">

3.13

</div>

</div>

<div class="AbstractEventLoop">

Abstract base class for asyncio-compliant event loops.

The `asyncio-event-loop-methods` section lists all methods that an alternative implementation of `AbstractEventLoop` should have defined.

</div>

<div class="BaseEventLoop">

A base class that partially implements `AbstractEventLoop` to simplify writing custom event loops.

It provides default implementations of many `AbstractEventLoop` methods, so subclasses have fewer methods to implement themselves. In return, a subclass is expected to implement a number of internal, protected methods. See `writing_custom_event_loop` for more information.

</div>

## Examples

Note that all examples in this section **purposefully** show how to use the low-level event loop APIs, such as `loop.run_forever` and `loop.call_soon`. Modern asyncio applications rarely need to be written this way; consider using the high-level functions like `asyncio.run`.

### Hello World with call_soon()

An example using the `loop.call_soon` method to schedule a callback. The callback displays `"Hello World"` and then stops the event loop:

    import asyncio

    def hello_world(loop):
        """A callback to print 'Hello World' and stop the event loop"""
        print('Hello World')
        loop.stop()

    loop = asyncio.new_event_loop()

    # Schedule a call to hello_world()
    loop.call_soon(hello_world, loop)

    # Blocking call interrupted by loop.stop()
    try:
        loop.run_forever()
    finally:
        loop.close()

<div class="seealso">

A similar `Hello World <coroutine>` example created with a coroutine and the `run` function.

</div>

### Display the current date with call_later()

An example of a callback displaying the current date every second. The callback uses the `loop.call_later` method to reschedule itself after 5 seconds, and then stops the event loop:

    import asyncio
    import datetime as dt

    def display_date(end_time, loop):
        print(dt.datetime.now())
        if (loop.time() + 1.0) < end_time:
            loop.call_later(1, display_date, end_time, loop)
        else:
            loop.stop()

    loop = asyncio.new_event_loop()

    # Schedule the first call to display_date()
    end_time = loop.time() + 5.0
    loop.call_soon(display_date, end_time, loop)

    # Blocking call interrupted by loop.stop()
    try:
        loop.run_forever()
    finally:
        loop.close()

<div class="seealso">

A similar `current date <asyncio_example_sleep>` example created with a coroutine and the `run` function.

</div>

### Watch a file descriptor for read events

Wait until a file descriptor received some data using the `loop.add_reader` method and then close the event loop:

    import asyncio
    from socket import socketpair

    # Create a pair of connected file descriptors
    rsock, wsock = socketpair()

    loop = asyncio.new_event_loop()

    def reader():
        data = rsock.recv(100)
        print("Received:", data.decode())

        # We are done: unregister the file descriptor
        loop.remove_reader(rsock)

        # Stop the event loop
        loop.stop()

    # Register the file descriptor for read event
    loop.add_reader(rsock, reader)

    # Simulate the reception of data from the network
    loop.call_soon(wsock.send, 'abc'.encode())

    try:
        # Run the event loop
        loop.run_forever()
    finally:
        # We are done. Close sockets and the event loop.
        rsock.close()
        wsock.close()
        loop.close()

<div class="seealso">

- A similar `example <asyncio_example_create_connection>` using transports, protocols, and the `loop.create_connection` method.
- Another similar `example <asyncio_example_create_connection-streams>` using the high-level `asyncio.open_connection` function and streams.

</div>

### Set signal handlers for SIGINT and SIGTERM

(This `signal` example only works on Unix.)

Register handlers for signals `~signal.SIGINT` and `~signal.SIGTERM` using the `loop.add_signal_handler` method:

    import asyncio
    import functools
    import os
    import signal

    def ask_exit(signame, loop):
        print("got signal %s: exit" % signame)
        loop.stop()

    async def main():
        loop = asyncio.get_running_loop()

        for signame in {'SIGINT', 'SIGTERM'}:
            loop.add_signal_handler(
                getattr(signal, signame),
                functools.partial(ask_exit, signame, loop))

        await asyncio.sleep(3600)

    print("Event loop running for 1 hour, press Ctrl+C to interrupt.")
    print(f"pid {os.getpid()}: send SIGINT or SIGTERM to exit.")

    asyncio.run(main())

# asyncio-exceptions

<div class="currentmodule">

asyncio

</div>

# Exceptions

**Source code:** `Lib/asyncio/exceptions.py`

------------------------------------------------------------------------

<div class="exception">

TimeoutError

A deprecated alias of `TimeoutError`, raised when the operation has exceeded the given deadline.

<div class="versionchanged">

3.11

This class was made an alias of `TimeoutError`.

</div>

</div>

<div class="exception">

CancelledError

The operation has been cancelled.

This exception can be caught to perform custom operations when asyncio Tasks are cancelled. In almost all situations the exception must be re-raised.

<div class="versionchanged">

3.8

`CancelledError` is now a subclass of `BaseException` rather than `Exception`.

</div>

</div>

<div class="exception">

InvalidStateError

Invalid internal state of `Task` or `Future`.

Can be raised in situations like setting a result value for a *Future* object that already has a result value set.

</div>

<div class="exception">

SendfileNotAvailableError

The "sendfile" syscall is not available for the given socket or file type.

A subclass of `RuntimeError`.

</div>

<div class="exception">

IncompleteReadError

The requested read operation did not complete fully.

Raised by the `asyncio stream APIs<asyncio-streams>`.

This exception is a subclass of `EOFError`.

<div class="attribute">

expected

The total number (`int`) of expected bytes.

</div>

<div class="attribute">

partial

A string of `bytes` read before the end of stream was reached.

</div>

</div>

<div class="exception">

LimitOverrunError

Reached the buffer size limit while looking for a separator.

Raised by the `asyncio stream APIs <asyncio-streams>`.

<div class="attribute">

consumed

The total number of to be consumed bytes.

</div>

</div>

# asyncio-extending

<div class="currentmodule">

asyncio

</div>

# Extending

The main direction for `asyncio` extending is writing custom *event loop* classes. Asyncio has helpers that could be used to simplify this task.

> [!NOTE]
> Third-parties should reuse existing asyncio code with caution, a new Python version is free to break backward compatibility in the *internal* part of the API.

## Writing a custom event loop

`asyncio.AbstractEventLoop` declares very many methods. See `asyncio-event-loop-methods` for the full list. Implementing them all from scratch is a tedious job.

A loop can get many common methods implementation for free by inheriting from `asyncio.BaseEventLoop`.

In turn, the successor should implement a bunch of *private* methods declared but not implemented in `asyncio.BaseEventLoop`.

For example, `loop.create_connection()` checks arguments, resolves DNS addresses, and calls `loop._make_socket_transport()` that should be implemented by inherited class. The `_make_socket_transport()` method is not documented and is considered as an *internal* API.

## Future and Task private constructors

`asyncio.Future` and `asyncio.Task` should be never created directly, please use corresponding `loop.create_future` and `loop.create_task`, or `asyncio.create_task` factories instead.

However, third-party *event loops* may *reuse* built-in future and task implementations for the sake of getting a complex and highly optimized code for free.

For this purpose the following, *private* constructors are listed:

<div class="method">

Future.\_\_init\_\_(\*, loop=None)

Create a built-in future instance.

*loop* is an optional event loop instance.

</div>

<div class="method">

Task.\_\_init\_\_(coro, \*, loop=None, name=None, context=None)

Create a built-in task instance.

*loop* is an optional event loop instance. The rest of arguments are described in `loop.create_task` description.

<div class="versionchanged">

3.11

*context* argument is added.

</div>

</div>

## Task lifetime support

A third party task implementation should call the following functions to keep a task visible by `asyncio.all_tasks` and `asyncio.current_task`:

<div class="function">

<span id="register_task">register_task</span>(task)

Register a new *task* as managed by *asyncio*.

Call the function from a task constructor.

</div>

<div class="function">

<span id="unregister_task">unregister_task</span>(task)

Unregister a *task* from *asyncio* internal structures.

The function should be called when a task is about to finish.

</div>

<div class="function">

<span id="enter_task">enter_task</span>(loop, task)

Switch the current task to the *task* argument.

Call the function just before executing a portion of embedded *coroutine* (`coroutine.send` or `coroutine.throw`).

</div>

<div class="function">

<span id="leave_task">leave_task</span>(loop, task)

Switch the current task back from *task* to `None`.

Call the function just after `coroutine.send` or `coroutine.throw` execution.

</div>

# asyncio-future

<div class="currentmodule">

asyncio

</div>

# Futures

**Source code:** `Lib/asyncio/futures.py`, `Lib/asyncio/base_futures.py`

------------------------------------------------------------------------

*Future* objects are used to bridge **low-level callback-based code** with high-level async/await code.

## Future Functions

<div class="function">

isfuture(obj)

Return `True` if *obj* is either of:

- an instance of `asyncio.Future`,
- an instance of `asyncio.Task`,
- a Future-like object with a `_asyncio_future_blocking` attribute.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

ensure_future(obj, \*, loop=None)

Return:

- *obj* argument as is, if *obj* is a `Future`, a `Task`, or a Future-like object (`isfuture` is used for the test.)
- a `Task` object wrapping *obj*, if *obj* is a coroutine (`iscoroutine` is used for the test); in this case the coroutine will be scheduled by `ensure_future()`.
- a `Task` object that would await on *obj*, if *obj* is an awaitable (`inspect.isawaitable` is used for the test.)

If *obj* is neither of the above a `TypeError` is raised.

> [!IMPORTANT]
> Save a reference to the result of this function, to avoid a task disappearing mid-execution.
>
> See also the `create_task` function which is the preferred way for creating new tasks or use `asyncio.TaskGroup` which keeps reference to the task internally.

<div class="versionchanged">

3.5.1 The function accepts any `awaitable` object.

</div>

<div class="deprecated">

3.10 Deprecation warning is emitted if *obj* is not a Future-like object and *loop* is not specified and there is no running event loop.

</div>

</div>

<div class="function">

wrap_future(future, \*, loop=None)

Wrap a `concurrent.futures.Future` object in a `asyncio.Future` object.

<div class="deprecated">

3.10 Deprecation warning is emitted if *future* is not a Future-like object and *loop* is not specified and there is no running event loop.

</div>

</div>

## Future Object

<div class="Future(*, loop=None)">

A Future represents an eventual result of an asynchronous operation. Not thread-safe.

Future is an `awaitable` object. Coroutines can await on Future objects until they either have a result or an exception set, or until they are cancelled. A Future can be awaited multiple times and the result is same.

Typically Futures are used to enable low-level callback-based code (e.g. in protocols implemented using asyncio `transports <asyncio-transports-protocols>`) to interoperate with high-level async/await code.

The rule of thumb is to never expose Future objects in user-facing APIs, and the recommended way to create a Future object is to call `loop.create_future`. This way alternative event loop implementations can inject their own optimized implementations of a Future object.

Futures are `generic <generics>` over the type of their results.

<div class="versionchanged">

3.7 Added support for the `contextvars` module.

</div>

<div class="deprecated">

3.10 Deprecation warning is emitted if *loop* is not specified and there is no running event loop.

</div>

<div class="method">

result()

Return the result of the Future.

If the Future is *done* and has a result set by the `set_result` method, the result value is returned.

If the Future is *done* and has an exception set by the `set_exception` method, this method raises the exception.

If the Future has been *cancelled*, this method raises a `CancelledError` exception.

If the Future's result isn't yet available, this method raises an `InvalidStateError` exception.

</div>

<div class="method">

set_result(result)

Mark the Future as *done* and set its result.

Raises an `InvalidStateError` error if the Future is already *done*.

</div>

<div class="method">

set_exception(exception)

Mark the Future as *done* and set an exception.

Raises an `InvalidStateError` error if the Future is already *done*.

</div>

<div class="method">

done()

Return `True` if the Future is *done*.

A Future is *done* if it was *cancelled* or if it has a result or an exception set with `set_result` or `set_exception` calls.

</div>

<div class="method">

cancelled()

Return `True` if the Future was *cancelled*.

The method is usually used to check if a Future is not *cancelled* before setting a result or an exception for it:

    if not fut.cancelled():
        fut.set_result(42)

</div>

<div class="method">

add_done_callback(callback, \*, context=None)

Add a callback to be run when the Future is *done*.

The *callback* is called with the Future object as its only argument.

If the Future is already *done* when this method is called, the callback is scheduled with `loop.call_soon`.

An optional keyword-only *context* argument allows specifying a custom `contextvars.Context` for the *callback* to run in. The current context is used when no *context* is provided.

`functools.partial` can be used to pass parameters to the callback, e.g.:

    # Call 'print("Future:", fut)' when "fut" is done.
    fut.add_done_callback(
        functools.partial(print, "Future:"))

<div class="versionchanged">

3.7 The *context* keyword-only parameter was added. See `567` for more details.

</div>

</div>

<div class="method">

remove_done_callback(callback)

Remove *callback* from the callbacks list.

Returns the number of callbacks removed, which is typically 1, unless a callback was added more than once.

</div>

<div class="method">

cancel(msg=None)

Cancel the Future and schedule callbacks.

If the Future is already *done* or *cancelled*, return `False`. Otherwise, change the Future's state to *cancelled*, schedule the callbacks, and return `True`.

The optional string argument *msg* is passed as the argument to the `CancelledError` exception raised when a cancelled Future is awaited.

<div class="versionchanged">

3.9 Added the *msg* parameter.

</div>

</div>

<div class="method">

exception()

Return the exception that was set on this Future.

The exception (or `None` if no exception was set) is returned only if the Future is *done*.

If the Future has been *cancelled*, this method raises a `CancelledError` exception.

If the Future isn't *done* yet, this method raises an `InvalidStateError` exception.

</div>

<div class="method">

get_loop()

Return the event loop the Future object is bound to.

<div class="versionadded">

3.7

</div>

</div>

</div>

<div id="asyncio_example_future">

This example creates a Future object, creates and schedules an asynchronous Task to set result for the Future, and waits until the Future has a result:

    async def set_after(fut, delay, value):
        # Sleep for *delay* seconds.
        await asyncio.sleep(delay)

        # Set *value* as a result of *fut* Future.
        fut.set_result(value)

    async def main():
        # Get the current event loop.
        loop = asyncio.get_running_loop()

        # Create a new Future object.
        fut = loop.create_future()

        # Run "set_after()" coroutine in a parallel Task.
        # We are using the low-level "loop.create_task()" API here because
        # we already have a reference to the event loop at hand.
        # Otherwise we could have just used "asyncio.create_task()".
        loop.create_task(
            set_after(fut, 1, '... world'))

        print('hello ...')

        # Wait until *fut* has a result (1 second) and print it.
        print(await fut)

    asyncio.run(main())

</div>

> [!IMPORTANT]
> The Future object was designed to mimic `concurrent.futures.Future`. Key differences include:
>
> - unlike asyncio Futures, `concurrent.futures.Future` instances cannot be awaited.
> - `asyncio.Future.result` and `asyncio.Future.exception` do not accept the *timeout* argument.
> - `asyncio.Future.result` and `asyncio.Future.exception` raise an `InvalidStateError` exception when the Future is not *done*.
> - Callbacks registered with `asyncio.Future.add_done_callback` are not called immediately. They are scheduled with `loop.call_soon` instead.
> - asyncio Future is not compatible with the `concurrent.futures.wait` and `concurrent.futures.as_completed` functions.
> - `asyncio.Future.cancel` accepts an optional `msg` argument, but `concurrent.futures.Future.cancel` does not.

# asyncio-graph

<div class="currentmodule">

asyncio

</div>

# Call graph introspection

**Source code:** `Lib/asyncio/graph.py`

------------------------------------------------------------------------

asyncio has powerful runtime call graph introspection utilities to trace the entire call graph of a running *coroutine* or *task*, or a suspended *future*. These utilities and the underlying machinery can be used from within a Python program or by external profilers and debuggers.

<div class="seealso">

`asyncio-introspection-tools`  
Command-line tools for inspecting tasks in another running Python process.

</div>

<div class="versionadded">

3.14

</div>

<div class="function">

print_call_graph(future=None, /, \*, file=None, depth=1, limit=None)

Print the async call graph for the current task or the provided `Task` or `Future`.

This function prints entries starting from the top frame and going down towards the invocation point.

The function receives an optional *future* argument. If not passed, the current running task will be used.

If the function is called on *the current task*, the optional keyword-only *depth* argument can be used to skip the specified number of frames from top of the stack.

If the optional keyword-only *limit* argument is provided, each call stack in the resulting graph is truncated to include at most `abs(limit)` entries. If *limit* is positive, the entries left are the closest to the invocation point. If *limit* is negative, the topmost entries are left. If *limit* is omitted or `None`, all entries are present. If *limit* is `0`, the call stack is not printed at all, only "awaited by" information is printed.

If *file* is omitted or `None`, the function will print to `sys.stdout`.

**Example:**

The following Python code:

``` python
import asyncio

async def test():
    asyncio.print_call_graph()

async def main():
    async with asyncio.TaskGroup() as g:
        g.create_task(test(), name='test')

asyncio.run(main())
```

will print:

    * Task(name='test', id=0x1039f0fe0)
    + Call stack:
    |   File 't2.py', line 4, in async test()
    + Awaited by:
       * Task(name='Task-1', id=0x103a5e060)
          + Call stack:
          |   File 'taskgroups.py', line 107, in async TaskGroup.__aexit__()
          |   File 't2.py', line 7, in async main()

</div>

<div class="function">

format_call_graph(future=None, /, \*, depth=1, limit=None)

Like `print_call_graph`, but returns a string. If *future* is `None` and there's no current task, the function returns an empty string.

</div>

<div class="function">

capture_call_graph(future=None, /, \*, depth=1, limit=None)

Capture the async call graph for the current task or the provided `Task` or `Future`.

The function receives an optional *future* argument. If not passed, the current running task will be used. If there's no current task, the function returns `None`.

If the function is called on *the current task*, the optional keyword-only *depth* argument can be used to skip the specified number of frames from top of the stack.

Returns a `FutureCallGraph` data class object:

- `FutureCallGraph(future, call_stack, awaited_by)`

  > Where *future* is a reference to a `Future` or a `Task` (or their subclasses.)
  >
  > `call_stack` is a tuple of `FrameCallGraphEntry` objects.
  >
  > `awaited_by` is a tuple of `FutureCallGraph` objects.

- `FrameCallGraphEntry(frame)`

  > Where *frame* is a frame object of a regular Python function in the call stack.

</div>

## Low level utility functions

To introspect an async call graph asyncio requires cooperation from control flow structures, such as `shield` or `TaskGroup`. Any time an intermediate `Future` object with low-level APIs like `Future.add_done_callback() <asyncio.Future.add_done_callback>` is involved, the following two functions should be used to inform asyncio about how exactly such intermediate future objects are connected with the tasks they wrap or control.

<div class="function">

future_add_to_awaited_by(future, waiter, /)

Record that *future* is awaited on by *waiter*.

Both *future* and *waiter* must be instances of `Future` or `Task` or their subclasses, otherwise the call would have no effect.

A call to `future_add_to_awaited_by()` must be followed by an eventual call to the `future_discard_from_awaited_by` function with the same arguments.

</div>

<div class="function">

future_discard_from_awaited_by(future, waiter, /)

Record that *future* is no longer awaited on by *waiter*.

Both *future* and *waiter* must be instances of `Future` or `Task` or their subclasses, otherwise the call would have no effect.

</div>

# asyncio-llapi-index

<div class="currentmodule">

asyncio

</div>

# Low-level API Index

This page lists all low-level asyncio APIs.

## Obtaining the Event Loop

|  |  |
|----|----|
| `asyncio.get_running_loop` | The **preferred** function to get the running event loop. |
| `asyncio.get_event_loop` | Get the running event loop or the event loop set for the current thread. |
| `asyncio.set_event_loop` | Set the event loop for the current thread. |
| `asyncio.new_event_loop` | Create a new event loop. |

**Examples**

- `Using asyncio.get_running_loop() <asyncio_example_future>`.

## Event Loop Methods

See also the main documentation section about the `asyncio-event-loop-methods`.

**Lifecycle**

|  |  |
|----|----|
| `loop.run_until_complete` | Run a Future/Task/awaitable until complete. |
| `loop.run_forever` | Run the event loop forever. |
| `loop.stop` | Stop the event loop. |
| `loop.close` | Close the event loop. |
| `loop.is_running` | Return `True` if the event loop is running. |
| `loop.is_closed` | Return `True` if the event loop is closed. |
| `await` `loop.shutdown_asyncgens` | Close asynchronous generators. |

**Debugging**

|                  |                                   |
|------------------|-----------------------------------|
| `loop.set_debug` | Enable or disable the debug mode. |
| `loop.get_debug` | Get the current debug mode.       |

**Scheduling Callbacks**

|                             |                                            |
|-----------------------------|--------------------------------------------|
| `loop.call_soon`            | Invoke a callback soon.                    |
| `loop.call_soon_threadsafe` | A thread-safe variant of `loop.call_soon`. |
| `loop.call_later`           | Invoke a callback *after* the given time.  |
| `loop.call_at`              | Invoke a callback *at* the given time.     |

**Thread/Interpreter/Process Pool**

|  |  |
|----|----|
| `await` `loop.run_in_executor` | Run a CPU-bound or other blocking function in a `concurrent.futures` executor. |
| `loop.set_default_executor` | Set the default executor for `loop.run_in_executor`. |

**Tasks and Futures**

|  |  |
|----|----|
| `loop.create_future` | Create a `Future` object. |
| `loop.create_task` | Schedule coroutine as a `Task`. |
| `loop.set_task_factory` | Set a factory used by `loop.create_task` to create `Tasks <Task>`. |
| `loop.get_task_factory` | Get the factory `loop.create_task` uses to create `Tasks <Task>`. |

**DNS**

|                            |                                               |
|----------------------------|-----------------------------------------------|
| `await` `loop.getaddrinfo` | Asynchronous version of `socket.getaddrinfo`. |
| `await` `loop.getnameinfo` | Asynchronous version of `socket.getnameinfo`. |

**Networking and IPC**

|  |  |
|----|----|
| `await` `loop.create_connection` | Open a TCP connection. |
| `await` `loop.create_server` | Create a TCP server. |
| `await` `loop.create_unix_connection` | Open a Unix socket connection. |
| `await` `loop.create_unix_server` | Create a Unix socket server. |
| `await` `loop.connect_accepted_socket` | Wrap a `~socket.socket` into a `(transport, protocol)` pair. |
| `await` `loop.create_datagram_endpoint` | Open a datagram (UDP) connection. |
| `await` `loop.sendfile` | Send a file over a transport. |
| `await` `loop.start_tls` | Upgrade an existing connection to TLS. |
| `await` `loop.connect_read_pipe` | Wrap a read end of a pipe into a `(transport, protocol)` pair. |
| `await` `loop.connect_write_pipe` | Wrap a write end of a pipe into a `(transport, protocol)` pair. |

**Sockets**

|  |  |
|----|----|
| `await` `loop.sock_recv` | Receive data from the `~socket.socket`. |
| `await` `loop.sock_recv_into` | Receive data from the `~socket.socket` into a buffer. |
| `await` `loop.sock_recvfrom` | Receive a datagram from the `~socket.socket`. |
| `await` `loop.sock_recvfrom_into` | Receive a datagram from the `~socket.socket` into a buffer. |
| `await` `loop.sock_sendall` | Send data to the `~socket.socket`. |
| `await` `loop.sock_sendto` | Send a datagram via the `~socket.socket` to the given address. |
| `await` `loop.sock_connect` | Connect the `~socket.socket`. |
| `await` `loop.sock_accept` | Accept a `~socket.socket` connection. |
| `await` `loop.sock_sendfile` | Send a file over the `~socket.socket`. |
| `loop.add_reader` | Start watching a file descriptor for read availability. |
| `loop.remove_reader` | Stop watching a file descriptor for read availability. |
| `loop.add_writer` | Start watching a file descriptor for write availability. |
| `loop.remove_writer` | Stop watching a file descriptor for write availability. |

**Unix Signals**

|                              |                                  |
|------------------------------|----------------------------------|
| `loop.add_signal_handler`    | Add a handler for a `signal`.    |
| `loop.remove_signal_handler` | Remove a handler for a `signal`. |

**Subprocesses**

|                         |                                          |
|-------------------------|------------------------------------------|
| `loop.subprocess_exec`  | Spawn a subprocess.                      |
| `loop.subprocess_shell` | Spawn a subprocess from a shell command. |

**Error Handling**

|  |  |
|----|----|
| `loop.call_exception_handler` | Call the exception handler. |
| `loop.set_exception_handler` | Set a new exception handler. |
| `loop.get_exception_handler` | Get the current exception handler. |
| `loop.default_exception_handler` | The default exception handler implementation. |

**Examples**

- `Using asyncio.new_event_loop() and loop.run_forever()
  <asyncio_example_lowlevel_helloworld>`.
- `Using loop.call_later() <asyncio_example_call_later>`.
- Using `loop.create_connection()` to implement `an echo-client <asyncio_example_tcp_echo_client_protocol>`.
- Using `loop.create_connection()` to `connect a socket <asyncio_example_create_connection>`.
- `Using add_reader() to watch an FD for read events
  <asyncio_example_watch_fd>`.
- `Using loop.add_signal_handler() <asyncio_example_unix_signals>`.
- `Using loop.subprocess_exec() <asyncio_example_subprocess_proto>`.

## Transports

All transports implement the following methods:

|  |  |
|----|----|
| `transport.close() <BaseTransport.close>` | Close the transport. |
| `transport.is_closing() <BaseTransport.is_closing>` | Return `True` if the transport is closing or is closed. |
| `transport.get_extra_info() <BaseTransport.get_extra_info>` | Request for information about the transport. |
| `transport.set_protocol() <BaseTransport.set_protocol>` | Set a new protocol. |
| `transport.get_protocol() <BaseTransport.get_protocol>` | Return the current protocol. |

Transports that can receive data (TCP and Unix connections, pipes, etc). Returned from methods like `loop.create_connection`, `loop.create_unix_connection`, `loop.connect_read_pipe`, etc:

**Read Transports**

|  |  |
|----|----|
| `transport.is_reading() <ReadTransport.is_reading>` | Return `True` if the transport is receiving. |
| `transport.pause_reading() <ReadTransport.pause_reading>` | Pause receiving. |
| `transport.resume_reading() <ReadTransport.resume_reading>` | Resume receiving. |

Transports that can Send data (TCP and Unix connections, pipes, etc). Returned from methods like `loop.create_connection`, `loop.create_unix_connection`, `loop.connect_write_pipe`, etc:

**Write Transports**

|  |  |
|----|----|
| `transport.write() <WriteTransport.write>` | Write data to the transport. |
| `transport.writelines() <WriteTransport.writelines>` | Write buffers to the transport. |
| `transport.can_write_eof() <WriteTransport.can_write_eof>` | Return `True` if the transport supports sending EOF. |
| `transport.write_eof() <WriteTransport.write_eof>` | Close and send EOF after flushing buffered data. |
| `transport.abort() <WriteTransport.abort>` | Close the transport immediately. |
| `transport.get_write_buffer_size()
<WriteTransport.get_write_buffer_size>` | Return the current size of the output buffer. |
| `transport.get_write_buffer_limits()
<WriteTransport.get_write_buffer_limits>` | Return high and low water marks for write flow control. |
| `transport.set_write_buffer_limits()
<WriteTransport.set_write_buffer_limits>` | Set new high and low water marks for write flow control. |

Transports returned by `loop.create_datagram_endpoint`:

**Datagram Transports**

|  |  |
|----|----|
| `transport.sendto() <DatagramTransport.sendto>` | Send data to the remote peer. |
| `transport.abort() <DatagramTransport.abort>` | Close the transport immediately. |

Low-level transport abstraction over subprocesses. Returned by `loop.subprocess_exec` and `loop.subprocess_shell`:

**Subprocess Transports**

|  |  |
|----|----|
| `transport.get_pid() <SubprocessTransport.get_pid>` | Return the subprocess process id. |
| `transport.get_pipe_transport()
<SubprocessTransport.get_pipe_transport>` | Return the transport for the requested communication pipe (*stdin*, *stdout*, or *stderr*). |
| `transport.get_returncode() <SubprocessTransport.get_returncode>` | Return the subprocess return code. |
| `transport.kill() <SubprocessTransport.kill>` | Kill the subprocess. |
| `transport.send_signal() <SubprocessTransport.send_signal>` | Send a signal to the subprocess. |
| `transport.terminate() <SubprocessTransport.terminate>` | Stop the subprocess. |
| `transport.close() <SubprocessTransport.close>` | Kill the subprocess and close all pipes. |

## Protocols

Protocol classes can implement the following **callback methods**:

|  |  |
|----|----|
| `callback` `connection_made() <BaseProtocol.connection_made>` | Called when a connection is made. |
| `callback` `connection_lost() <BaseProtocol.connection_lost>` | Called when the connection is lost or closed. |
| `callback` `pause_writing() <BaseProtocol.pause_writing>` | Called when the transport's buffer goes over the high water mark. |
| `callback` `resume_writing() <BaseProtocol.resume_writing>` | Called when the transport's buffer drains below the low water mark. |

**Streaming Protocols (TCP, Unix Sockets, Pipes)**

|  |  |
|----|----|
| `callback` `data_received() <Protocol.data_received>` | Called when some data is received. |
| `callback` `eof_received() <Protocol.eof_received>` | Called when an EOF is received. |

**Buffered Streaming Protocols**

|  |  |
|----|----|
| `callback` `get_buffer() <BufferedProtocol.get_buffer>` | Called to allocate a new receive buffer. |
| `callback` `buffer_updated() <BufferedProtocol.buffer_updated>` | Called when the buffer was updated with the received data. |
| `callback` `eof_received() <BufferedProtocol.eof_received>` | Called when an EOF is received. |

**Datagram Protocols**

|  |  |
|----|----|
| `callback` `datagram_received()
<DatagramProtocol.datagram_received>` | Called when a datagram is received. |
| `callback` `error_received() <DatagramProtocol.error_received>` | Called when a previous send or receive operation raises an `OSError`. |

**Subprocess Protocols**

|  |  |
|----|----|
| `callback` `~SubprocessProtocol.pipe_data_received` | Called when the child process writes data into its *stdout* or *stderr* pipe. |
| `callback` `~SubprocessProtocol.pipe_connection_lost` | Called when one of the pipes communicating with the child process is closed. |
| `callback` `process_exited()
<SubprocessProtocol.process_exited>` | Called when the child process has exited. It can be called before `~SubprocessProtocol.pipe_data_received` and `~SubprocessProtocol.pipe_connection_lost` methods. |

# asyncio-platforms

<div class="currentmodule">

asyncio

</div>

# Platform Support

The `asyncio` module is designed to be portable, but some platforms have subtle differences and limitations due to the platforms' underlying architecture and capabilities.

## All Platforms

- `loop.add_reader` and `loop.add_writer` cannot be used to monitor file I/O.
- `loop.connect_read_pipe` and `loop.connect_write_pipe` cannot be used with regular files. See `Supported pipe objects
  <asyncio-pipe-objects>` for the objects that are accepted on each platform.

## Windows

**Source code:** `Lib/asyncio/proactor_events.py`, `Lib/asyncio/windows_events.py`, `Lib/asyncio/windows_utils.py`

------------------------------------------------------------------------

<div class="versionchanged">

3.8

On Windows, `ProactorEventLoop` is now the default event loop.

</div>

All event loops on Windows do not support the following methods:

- `loop.create_unix_connection` and `loop.create_unix_server` are not supported. The `socket.AF_UNIX` socket family is specific to Unix.
- `loop.add_signal_handler` and `loop.remove_signal_handler` are not supported.

`SelectorEventLoop` has the following limitations:

- `~selectors.SelectSelector` is used to wait on socket events: it supports sockets and is limited to 512 sockets.
- `loop.add_reader` and `loop.add_writer` only accept socket handles (e.g. pipe file descriptors are not supported).
- Pipes are not supported, so the `loop.connect_read_pipe` and `loop.connect_write_pipe` methods are not implemented.
- `Subprocesses <asyncio-subprocess>` are not supported, i.e. `loop.subprocess_exec` and `loop.subprocess_shell` methods are not implemented.

`ProactorEventLoop` has the following limitations:

- The `loop.add_reader` and `loop.add_writer` methods are not supported.
- `loop.connect_read_pipe` and `loop.connect_write_pipe` only accept a handle opened for overlapped I/O. See `Supported pipe objects <asyncio-pipe-objects>` for which objects are supported.

The resolution of the monotonic clock on Windows is usually around 15.6 milliseconds. The best resolution is 0.5 milliseconds. The resolution depends on the hardware (availability of [HPET](https://en.wikipedia.org/wiki/High_Precision_Event_Timer)) and on the Windows configuration.

### Subprocess Support on Windows

On Windows, the default event loop `ProactorEventLoop` supports subprocesses, whereas `SelectorEventLoop` does not.

## macOS

Modern macOS versions are fully supported.

**macOS \<= 10.8**

On macOS 10.6, 10.7 and 10.8, the default event loop uses `selectors.KqueueSelector`, which does not support character devices on these versions. The `SelectorEventLoop` can be manually configured to use `~selectors.SelectSelector` or `~selectors.PollSelector` to support character devices on these older versions of macOS. Example:

    import asyncio
    import selectors

    selector = selectors.SelectSelector()
    loop = asyncio.SelectorEventLoop(selector)
    asyncio.set_event_loop(loop)

# asyncio-protocol

<div class="currentmodule">

asyncio

</div>

# Transports and Protocols

**Preface**

Transports and Protocols are used by the **low-level** event loop APIs such as `loop.create_connection`. They use callback-based programming style and enable high-performance implementations of network or IPC protocols (e.g. HTTP).

Essentially, transports and protocols should only be used in libraries and frameworks and never in high-level asyncio applications.

This documentation page covers both [Transports](#transports) and [Protocols](#protocols).

**Introduction**

At the highest level, the transport is concerned with *how* bytes are transmitted, while the protocol determines *which* bytes to transmit (and to some extent when).

A different way of saying the same thing: a transport is an abstraction for a socket (or similar I/O endpoint) while a protocol is an abstraction for an application, from the transport's point of view.

Yet another view is the transport and protocol interfaces together define an abstract interface for using network I/O and interprocess I/O.

There is always a 1:1 relationship between transport and protocol objects: the protocol calls transport methods to send data, while the transport calls protocol methods to pass it data that has been received.

Most of connection oriented event loop methods (such as `loop.create_connection`) usually accept a *protocol_factory* argument used to create a *Protocol* object for an accepted connection, represented by a *Transport* object. Such methods usually return a tuple of `(transport, protocol)`.

**Contents**

This documentation page contains the following sections:

- The [Transports](#transports) section documents asyncio `BaseTransport`, `ReadTransport`, `WriteTransport`, `Transport`, `DatagramTransport`, and `SubprocessTransport` classes.
- The [Protocols](#protocols) section documents asyncio `BaseProtocol`, `Protocol`, `BufferedProtocol`, `DatagramProtocol`, and `SubprocessProtocol` classes.
- The [Examples](#examples) section showcases how to work with transports, protocols, and low-level event loop APIs.

## Transports

**Source code:** `Lib/asyncio/transports.py`

------------------------------------------------------------------------

Transports are classes provided by `asyncio` in order to abstract various kinds of communication channels.

Transport objects are always instantiated by an `asyncio event loop <asyncio-event-loop>`.

asyncio implements transports for TCP, UDP, SSL, and subprocess pipes. The methods available on a transport depend on the transport's kind.

The transport classes are `not thread safe <asyncio-multithreading>`.

### Transports Hierarchy

<div class="BaseTransport">

Base class for all transports. Contains methods that all asyncio transports share.

</div>

<div class="WriteTransport(BaseTransport)">

A base transport for write-only connections.

Instances of the *WriteTransport* class are returned from the `loop.connect_write_pipe` event loop method and are also used by subprocess-related methods like `loop.subprocess_exec`.

</div>

<div class="ReadTransport(BaseTransport)">

A base transport for read-only connections.

Instances of the *ReadTransport* class are returned from the `loop.connect_read_pipe` event loop method and are also used by subprocess-related methods like `loop.subprocess_exec`.

</div>

<div class="Transport(WriteTransport, ReadTransport)">

Interface representing a bidirectional transport, such as a TCP connection.

The user does not instantiate a transport directly; they call a utility function, passing it a protocol factory and other information necessary to create the transport and protocol.

Instances of the *Transport* class are returned from or used by event loop methods like `loop.create_connection`, `loop.create_unix_connection`, `loop.create_server`, `loop.sendfile`, etc.

</div>

<div class="DatagramTransport(BaseTransport)">

A transport for datagram (UDP) connections.

Instances of the *DatagramTransport* class are returned from the `loop.create_datagram_endpoint` event loop method.

</div>

<div class="SubprocessTransport(BaseTransport)">

An abstraction to represent a connection between a parent and its child OS process.

Instances of the *SubprocessTransport* class are returned from event loop methods `loop.subprocess_shell` and `loop.subprocess_exec`.

</div>

### Base Transport

<div class="method">

BaseTransport.close()

Close the transport.

If the transport has a buffer for outgoing data, buffered data will be flushed asynchronously. No more data will be received. After all buffered data is flushed, the protocol's `protocol.connection_lost()
<BaseProtocol.connection_lost>` method will be called with `None` as its argument. The transport should not be used once it is closed.

</div>

<div class="method">

BaseTransport.is_closing()

Return `True` if the transport is closing or is closed.

</div>

<div class="method">

BaseTransport.get_extra_info(name, default=None)

Return information about the transport or underlying resources it uses.

*name* is a string representing the piece of transport-specific information to get.

*default* is the value to return if the information is not available, or if the transport does not support querying it with the given third-party event loop implementation or on the current platform.

For example, the following code attempts to get the underlying socket object of the transport:

    sock = transport.get_extra_info('socket')
    if sock is not None:
        print(sock.getsockopt(...))

Categories of information that can be queried on some transports:

- socket:
  - `'peername'`: the remote address to which the socket is connected, result of `socket.socket.getpeername` (`None` on error)
  - `'socket'`: `socket.socket` instance
  - `'sockname'`: the socket's own address, result of `socket.socket.getsockname`
- SSL socket:
  - `'compression'`: the compression algorithm being used as a string, or `None` if the connection isn't compressed; result of `ssl.SSLSocket.compression`
  - `'cipher'`: a three-value tuple containing the name of the cipher being used, the version of the SSL protocol that defines its use, and the number of secret bits being used; result of `ssl.SSLSocket.cipher`
  - `'peercert'`: peer certificate; result of `ssl.SSLSocket.getpeercert`
  - `'sslcontext'`: `ssl.SSLContext` instance
  - `'ssl_object'`: `ssl.SSLObject` or `ssl.SSLSocket` instance
- pipe:
  - `'pipe'`: pipe object
- subprocess:
  - `'subprocess'`: `subprocess.Popen` instance

</div>

<div class="method">

BaseTransport.set_protocol(protocol)

Set a new protocol.

Switching protocol should only be done when both protocols are documented to support the switch.

</div>

<div class="method">

BaseTransport.get_protocol()

Return the current protocol.

</div>

### Read-only Transports

<div class="method">

ReadTransport.is_reading()

Return `True` if the transport is receiving new data.

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

ReadTransport.pause_reading()

Pause the receiving end of the transport. No data will be passed to the protocol's `protocol.data_received() <Protocol.data_received>` method until `resume_reading` is called.

<div class="versionchanged">

3.7 The method is idempotent, i.e. it can be called when the transport is already paused or closed.

</div>

</div>

<div class="method">

ReadTransport.resume_reading()

Resume the receiving end. The protocol's `protocol.data_received() <Protocol.data_received>` method will be called once again if some data is available for reading.

<div class="versionchanged">

3.7 The method is idempotent, i.e. it can be called when the transport is already reading.

</div>

</div>

### Write-only Transports

<div class="method">

WriteTransport.abort()

Close the transport immediately, without waiting for pending operations to complete. Buffered data will be lost. No more data will be received. The protocol's `protocol.connection_lost()
<BaseProtocol.connection_lost>` method will eventually be called with `None` as its argument.

</div>

<div class="method">

WriteTransport.can_write_eof()

Return `True` if the transport supports `~WriteTransport.write_eof`, `False` if not.

</div>

<div class="method">

WriteTransport.get_write_buffer_size()

Return the current size of the output buffer used by the transport.

</div>

<div class="method">

WriteTransport.get_write_buffer_limits()

Get the *high* and *low* watermarks for write flow control. Return a tuple `(low, high)` where *low* and *high* are positive number of bytes.

Use `set_write_buffer_limits` to set the limits.

<div class="versionadded">

3.4.2

</div>

</div>

<div class="method">

WriteTransport.set_write_buffer_limits(high=None, low=None)

Set the *high* and *low* watermarks for write flow control.

These two values (measured in number of bytes) control when the protocol's `protocol.pause_writing() <BaseProtocol.pause_writing>` and `protocol.resume_writing() <BaseProtocol.resume_writing>` methods are called. If specified, the low watermark must be less than or equal to the high watermark. Neither *high* nor *low* can be negative.

`~BaseProtocol.pause_writing` is called when the buffer size becomes greater than or equal to the *high* value. If writing has been paused, `~BaseProtocol.resume_writing` is called when the buffer size becomes less than or equal to the *low* value.

The defaults are implementation-specific. If only the high watermark is given, the low watermark defaults to an implementation-specific value less than or equal to the high watermark. Setting *high* to zero forces *low* to zero as well, and causes `~BaseProtocol.pause_writing` to be called whenever the buffer becomes non-empty. Setting *low* to zero causes `~BaseProtocol.resume_writing` to be called only once the buffer is empty. Use of zero for either limit is generally sub-optimal as it reduces opportunities for doing I/O and computation concurrently.

Use `~WriteTransport.get_write_buffer_limits` to get the limits.

</div>

<div class="method">

WriteTransport.write(data)

Write some *data* bytes to the transport.

This method does not block; it buffers the data and arranges for it to be sent out asynchronously.

The transport keeps a reference to *data* rather than copying. *data* must not be mutated while being sent.

</div>

<div class="method">

WriteTransport.writelines(list_of_data)

Write a list (or any iterable) of data bytes to the transport. This is functionally equivalent to calling `write` on each element yielded by the iterable, but may be implemented more efficiently.

</div>

<div class="method">

WriteTransport.write_eof()

Close the write end of the transport after flushing all buffered data. Data may still be received.

This method can raise `NotImplementedError` if the transport (e.g. SSL) doesn't support half-closed connections.

</div>

### Datagram Transports

<div class="method">

DatagramTransport.sendto(data, addr=None)

Send the *data* bytes to the remote peer given by *addr* (a transport-dependent target address). If *addr* is `None`, the data is sent to the target address given on transport creation.

This method does not block; it buffers the data and arranges for it to be sent out asynchronously.

<div class="versionchanged">

3.13 This method can be called with an empty bytes object to send a zero-length datagram. The buffer size calculation used for flow control is also updated to account for the datagram header.

</div>

</div>

<div class="method">

DatagramTransport.abort()

Close the transport immediately, without waiting for pending operations to complete. Buffered data will be lost. No more data will be received. The protocol's `protocol.connection_lost() <BaseProtocol.connection_lost>` method will eventually be called with `None` as its argument.

</div>

### Subprocess Transports

<div class="method">

SubprocessTransport.get_pid()

Return the subprocess process id as an integer.

</div>

<div class="method">

SubprocessTransport.get_pipe_transport(fd)

Return the transport for the communication pipe corresponding to the integer file descriptor *fd*:

- `0`: writable streaming transport of the standard input (*stdin*), or `None` if the subprocess was not created with `stdin=PIPE`
- `1`: readable streaming transport of the standard output (*stdout*), or `None` if the subprocess was not created with `stdout=PIPE`
- `2`: readable streaming transport of the standard error (*stderr*), or `None` if the subprocess was not created with `stderr=PIPE`
- other *fd*: `None`

</div>

<div class="method">

SubprocessTransport.get_returncode()

Return the subprocess return code as an integer or `None` if it hasn't returned, which is similar to the `subprocess.Popen.returncode` attribute.

</div>

<div class="method">

SubprocessTransport.kill()

Kill the subprocess.

On POSIX systems, the function sends SIGKILL to the subprocess. On Windows, this method is an alias for `terminate`.

See also `subprocess.Popen.kill`.

</div>

<div class="method">

SubprocessTransport.send_signal(signal)

Send the *signal* number to the subprocess, as in `subprocess.Popen.send_signal`.

</div>

<div class="method">

SubprocessTransport.terminate()

Stop the subprocess.

On POSIX systems, this method sends `~signal.SIGTERM` to the subprocess. On Windows, the Windows API function `!TerminateProcess` is called to stop the subprocess.

See also `subprocess.Popen.terminate`.

</div>

<div class="method">

SubprocessTransport.close()

Kill the subprocess by calling the `kill` method.

If the subprocess hasn't returned yet, and close transports of *stdin*, *stdout*, and *stderr* pipes.

</div>

## Protocols

**Source code:** `Lib/asyncio/protocols.py`

------------------------------------------------------------------------

asyncio provides a set of abstract base classes that should be used to implement network protocols. Those classes are meant to be used together with `transports <asyncio-transport>`.

Subclasses of abstract base protocol classes may implement some or all methods. All these methods are callbacks: they are called by transports on certain events, for example when some data is received. A base protocol method should be called by the corresponding transport.

### Base Protocols

<div class="BaseProtocol">

Base protocol with methods that all protocols share.

</div>

<div class="Protocol(BaseProtocol)">

The base class for implementing streaming protocols (TCP, Unix sockets, etc).

</div>

<div class="BufferedProtocol(BaseProtocol)">

A base class for implementing streaming protocols with manual control of the receive buffer.

</div>

<div class="DatagramProtocol(BaseProtocol)">

The base class for implementing datagram (UDP) protocols.

</div>

<div class="SubprocessProtocol(BaseProtocol)">

The base class for implementing protocols communicating with child processes (unidirectional pipes).

</div>

### Base Protocol

All asyncio protocols can implement Base Protocol callbacks.

**Connection Callbacks**

Connection callbacks are called on all protocols, exactly once per a successful connection. All other protocol callbacks can only be called between those two methods.

<div class="method">

BaseProtocol.connection_made(transport)

Called when a connection is made.

The *transport* argument is the transport representing the connection. The protocol is responsible for storing the reference to its transport.

</div>

<div class="method">

BaseProtocol.connection_lost(exc)

Called when the connection is lost or closed.

The argument is either an exception object or `None`. The latter means a regular EOF is received, or the connection was aborted or closed by this side of the connection.

</div>

**Flow Control Callbacks**

Flow control callbacks can be called by transports to pause or resume writing performed by the protocol.

See the documentation of the `~WriteTransport.set_write_buffer_limits` method for more details.

<div class="method">

BaseProtocol.pause_writing()

Called when the transport's buffer goes over the high watermark.

</div>

<div class="method">

BaseProtocol.resume_writing()

Called when the transport's buffer drains below the low watermark.

</div>

If the buffer size equals the high watermark, `~BaseProtocol.pause_writing` is not called: the buffer size must go strictly over.

Conversely, `~BaseProtocol.resume_writing` is called when the buffer size is equal or lower than the low watermark. These end conditions are important to ensure that things go as expected when either mark is zero.

### Streaming Protocols

Event methods, such as `loop.create_server`, `loop.create_unix_server`, `loop.create_connection`, `loop.create_unix_connection`, `loop.connect_accepted_socket`, `loop.connect_read_pipe`, and `loop.connect_write_pipe` accept factories that return streaming protocols.

<div class="method">

Protocol.data_received(data)

Called when some data is received. *data* is a non-empty bytes object containing the incoming data.

Whether the data is buffered, chunked or reassembled depends on the transport. In general, you shouldn't rely on specific semantics and instead make your parsing generic and flexible. However, data is always received in the correct order.

The method can be called an arbitrary number of times while a connection is open.

However, `protocol.eof_received() <Protocol.eof_received>` is called at most once. Once `eof_received()` is called, `data_received()` is not called anymore.

</div>

<div class="method">

Protocol.eof_received()

Called when the other end signals it won't send any more data (for example by calling `transport.write_eof()
<WriteTransport.write_eof>`, if the other end also uses asyncio).

This method may return a false value (including `None`), in which case the transport will close itself. Conversely, if this method returns a true value, the protocol used determines whether to close the transport. Since the default implementation returns `None`, it implicitly closes the connection.

Some transports, including SSL, don't support half-closed connections, in which case returning true from this method will result in the connection being closed.

</div>

State machine:

```
start -> connection_made
    [-> data_received]*
    [-> eof_received]?
-> connection_lost -> end
```

### Buffered Streaming Protocols

<div class="versionadded">

3.7

</div>

Buffered Protocols can be used with any event loop method that supports [Streaming Protocols](#streaming-protocols).

`BufferedProtocol` implementations allow explicit manual allocation and control of the receive buffer. Event loops can then use the buffer provided by the protocol to avoid unnecessary data copies. This can result in noticeable performance improvement for protocols that receive big amounts of data. Sophisticated protocol implementations can significantly reduce the number of buffer allocations.

The following callbacks are called on `BufferedProtocol` instances:

<div class="method">

BufferedProtocol.get_buffer(sizehint)

Called to allocate a new receive buffer.

*sizehint* is the recommended minimum size for the returned buffer. It is acceptable to return smaller or larger buffers than what *sizehint* suggests. When set to -1, the buffer size can be arbitrary. It is an error to return a buffer with a zero size.

`get_buffer()` must return an object implementing the `buffer protocol <bufferobjects>`.

</div>

<div class="method">

BufferedProtocol.buffer_updated(nbytes)

Called when the buffer was updated with the received data.

*nbytes* is the total number of bytes that were written to the buffer.

</div>

<div class="method">

BufferedProtocol.eof_received()

See the documentation of the `protocol.eof_received()
<Protocol.eof_received>` method.

</div>

`~BufferedProtocol.get_buffer` can be called an arbitrary number of times during a connection. However, `protocol.eof_received()
<Protocol.eof_received>` is called at most once and, if called, `~BufferedProtocol.get_buffer` and `~BufferedProtocol.buffer_updated` won't be called after it.

State machine:

```
start -> connection_made
    [-> get_buffer
        [-> buffer_updated]?
    ]*
    [-> eof_received]?
-> connection_lost -> end
```

### Datagram Protocols

Datagram Protocol instances should be constructed by protocol factories passed to the `loop.create_datagram_endpoint` method.

<div class="method">

DatagramProtocol.datagram_received(data, addr)

Called when a datagram is received. *data* is a bytes object containing the incoming data. *addr* is the address of the peer sending the data; the exact format depends on the transport.

</div>

<div class="method">

DatagramProtocol.error_received(exc)

Called when a previous send or receive operation raises an `OSError`. *exc* is the `OSError` instance.

This method is called in rare conditions, when the transport (e.g. UDP) detects that a datagram could not be delivered to its recipient. In many conditions though, undeliverable datagrams will be silently dropped.

</div>

> [!NOTE]
> On BSD systems (macOS, FreeBSD, etc.) flow control is not supported for datagram protocols, because there is no reliable way to detect send failures caused by writing too many packets.
>
> The socket always appears 'ready' and excess packets are dropped. An `OSError` with `errno` set to `errno.ENOBUFS` may or may not be raised; if it is raised, it will be reported to `DatagramProtocol.error_received` but otherwise ignored.

### Subprocess Protocols

Subprocess Protocol instances should be constructed by protocol factories passed to the `loop.subprocess_exec` and `loop.subprocess_shell` methods.

<div class="method">

SubprocessProtocol.pipe_data_received(fd, data)

Called when the child process writes data into its stdout or stderr pipe.

*fd* is the integer file descriptor of the pipe.

*data* is a non-empty bytes object containing the received data.

</div>

<div class="method">

SubprocessProtocol.pipe_connection_lost(fd, exc)

Called when one of the pipes communicating with the child process is closed.

*fd* is the integer file descriptor that was closed.

</div>

<div class="method">

SubprocessProtocol.process_exited()

Called when the child process has exited.

It can be called before `~SubprocessProtocol.pipe_data_received` and `~SubprocessProtocol.pipe_connection_lost` methods.

</div>

## Examples

### TCP Echo Server

Create a TCP echo server using the `loop.create_server` method, send back received data, and close the connection:

    import asyncio


    class EchoServerProtocol(asyncio.Protocol):
        def connection_made(self, transport):
            peername = transport.get_extra_info('peername')
            print('Connection from {}'.format(peername))
            self.transport = transport

        def data_received(self, data):
            message = data.decode()
            print('Data received: {!r}'.format(message))

            print('Send: {!r}'.format(message))
            self.transport.write(data)

            print('Close the client socket')
            self.transport.close()


    async def main():
        # Get a reference to the event loop as we plan to use
        # low-level APIs.
        loop = asyncio.get_running_loop()

        server = await loop.create_server(
            EchoServerProtocol,
            '127.0.0.1', 8888)

        async with server:
            await server.serve_forever()


    asyncio.run(main())

<div class="seealso">

The `TCP echo server using streams <asyncio-tcp-echo-server-streams>` example uses the high-level `asyncio.start_server` function.

</div>

### TCP Echo Client

A TCP echo client using the `loop.create_connection` method, sends data, and waits until the connection is closed:

    import asyncio


    class EchoClientProtocol(asyncio.Protocol):
        def __init__(self, message, on_con_lost):
            self.message = message
            self.on_con_lost = on_con_lost

        def connection_made(self, transport):
            transport.write(self.message.encode())
            print('Data sent: {!r}'.format(self.message))

        def data_received(self, data):
            print('Data received: {!r}'.format(data.decode()))

        def connection_lost(self, exc):
            print('The server closed the connection')
            self.on_con_lost.set_result(True)


    async def main():
        # Get a reference to the event loop as we plan to use
        # low-level APIs.
        loop = asyncio.get_running_loop()

        on_con_lost = loop.create_future()
        message = 'Hello World!'

        transport, protocol = await loop.create_connection(
            lambda: EchoClientProtocol(message, on_con_lost),
            '127.0.0.1', 8888)

        # Wait until the protocol signals that the connection
        # is lost and close the transport.
        try:
            await on_con_lost
        finally:
            transport.close()


    asyncio.run(main())

<div class="seealso">

The `TCP echo client using streams <asyncio-tcp-echo-client-streams>` example uses the high-level `asyncio.open_connection` function.

</div>

### UDP Echo Server

A UDP echo server, using the `loop.create_datagram_endpoint` method, sends back received data:

    import asyncio


    class EchoServerProtocol:
        def connection_made(self, transport):
            self.transport = transport

        def datagram_received(self, data, addr):
            message = data.decode()
            print('Received %r from %s' % (message, addr))
            print('Send %r to %s' % (message, addr))
            self.transport.sendto(data, addr)


    async def main():
        print("Starting UDP server")

        # Get a reference to the event loop as we plan to use
        # low-level APIs.
        loop = asyncio.get_running_loop()

        # One protocol instance will be created to serve all
        # client requests.
        transport, protocol = await loop.create_datagram_endpoint(
            EchoServerProtocol,
            local_addr=('127.0.0.1', 9999))

        try:
            await asyncio.sleep(3600)  # Serve for 1 hour.
        finally:
            transport.close()


    asyncio.run(main())

### UDP Echo Client

A UDP echo client, using the `loop.create_datagram_endpoint` method, sends data and closes the transport when it receives the answer:

    import asyncio


    class EchoClientProtocol:
        def __init__(self, message, on_con_lost):
            self.message = message
            self.on_con_lost = on_con_lost
            self.transport = None

        def connection_made(self, transport):
            self.transport = transport
            print('Send:', self.message)
            self.transport.sendto(self.message.encode())

        def datagram_received(self, data, addr):
            print("Received:", data.decode())

            print("Close the socket")
            self.transport.close()

        def error_received(self, exc):
            print('Error received:', exc)

        def connection_lost(self, exc):
            print("Connection closed")
            self.on_con_lost.set_result(True)


    async def main():
        # Get a reference to the event loop as we plan to use
        # low-level APIs.
        loop = asyncio.get_running_loop()

        on_con_lost = loop.create_future()
        message = "Hello World!"

        transport, protocol = await loop.create_datagram_endpoint(
            lambda: EchoClientProtocol(message, on_con_lost),
            remote_addr=('127.0.0.1', 9999))

        try:
            await on_con_lost
        finally:
            transport.close()


    asyncio.run(main())

### Connecting Existing Sockets

Wait until a socket receives data using the `loop.create_connection` method with a protocol:

    import asyncio
    import socket


    class MyProtocol(asyncio.Protocol):

        def __init__(self, on_con_lost):
            self.transport = None
            self.on_con_lost = on_con_lost

        def connection_made(self, transport):
            self.transport = transport

        def data_received(self, data):
            print("Received:", data.decode())

            # We are done: close the transport;
            # connection_lost() will be called automatically.
            self.transport.close()

        def connection_lost(self, exc):
            # The socket has been closed
            self.on_con_lost.set_result(True)


    async def main():
        # Get a reference to the event loop as we plan to use
        # low-level APIs.
        loop = asyncio.get_running_loop()
        on_con_lost = loop.create_future()

        # Create a pair of connected sockets
        rsock, wsock = socket.socketpair()

        # Register the socket to wait for data.
        transport, protocol = await loop.create_connection(
            lambda: MyProtocol(on_con_lost), sock=rsock)

        # Simulate the reception of data from the network.
        loop.call_soon(wsock.send, 'abc'.encode())

        try:
            await protocol.on_con_lost
        finally:
            transport.close()
            wsock.close()

    asyncio.run(main())

<div class="seealso">

The `watch a file descriptor for read events
<asyncio_example_watch_fd>` example uses the low-level `loop.add_reader` method to register an FD.

The `register an open socket to wait for data using streams
<asyncio_example_create_connection-streams>` example uses high-level streams created by the `open_connection` function in a coroutine.

</div>

### loop.subprocess_exec() and SubprocessProtocol

An example of a subprocess protocol used to get the output of a subprocess and to wait for the subprocess exit.

The subprocess is created by the `loop.subprocess_exec` method:

    import asyncio
    import sys

    class DateProtocol(asyncio.SubprocessProtocol):
        def __init__(self, exit_future):
            self.exit_future = exit_future
            self.output = bytearray()
            self.pipe_closed = False
            self.exited = False

        def pipe_connection_lost(self, fd, exc):
            self.pipe_closed = True
            self.check_for_exit()

        def pipe_data_received(self, fd, data):
            self.output.extend(data)

        def process_exited(self):
            self.exited = True
            # process_exited() method can be called before
            # pipe_connection_lost() method: wait until both methods are
            # called.
            self.check_for_exit()

        def check_for_exit(self):
            if self.pipe_closed and self.exited:
                self.exit_future.set_result(True)

    async def get_date():
        # Get a reference to the event loop as we plan to use
        # low-level APIs.
        loop = asyncio.get_running_loop()

        code = 'import datetime as dt; print(dt.datetime.now())'
        exit_future = asyncio.Future(loop=loop)

        # Create the subprocess controlled by DateProtocol;
        # redirect the standard output into a pipe.
        transport, protocol = await loop.subprocess_exec(
            lambda: DateProtocol(exit_future),
            sys.executable, '-c', code,
            stdin=None, stderr=None)

        # Wait for the subprocess exit using the process_exited()
        # method of the protocol.
        await exit_future

        # Close the stdout pipe.
        transport.close()

        # Read the output which was collected by the
        # pipe_data_received() method of the protocol.
        data = bytes(protocol.output)
        return data.decode('ascii').rstrip()

    date = asyncio.run(get_date())
    print(f"Current date: {date}")

See also the `same example <asyncio_example_create_subprocess_exec>` written using high-level APIs.

# asyncio-queue

<div class="currentmodule">

asyncio

</div>

# Queues

**Source code:** `Lib/asyncio/queues.py`

------------------------------------------------------------------------

asyncio queues are designed to be similar to classes of the `queue` module. Although asyncio queues are not thread-safe, they are designed to be used specifically in async/await code.

Note that methods of asyncio queues don't have a *timeout* parameter; use `asyncio.wait_for` function to do queue operations with a timeout.

See also the [Examples](#examples) section below.

## Queue

<div class="Queue(maxsize=0)">

A first in, first out (FIFO) queue.

If *maxsize* is less than or equal to zero, the queue size is infinite. If it is an integer greater than `0`, then `await put()` blocks when the queue reaches *maxsize* until an item is removed by `get`.

Unlike the standard library threading `queue`, the size of the queue is always known and can be returned by calling the `qsize` method.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

This class is `not thread safe <asyncio-multithreading>`.

<div class="attribute">

maxsize

Number of items allowed in the queue.

</div>

<div class="method">

empty()

Return `True` if the queue is empty, `False` otherwise.

</div>

<div class="method">

full()

Return `True` if there are `maxsize` items in the queue.

If the queue was initialized with `maxsize=0` (the default), then `full` never returns `True`.

</div>

<div class="method" async="">

get()

Remove and return an item from the queue. If queue is empty, wait until an item is available.

Raises `QueueShutDown` if the queue has been shut down and is empty, or if the queue has been shut down immediately.

</div>

<div class="method">

get_nowait()

Return an item if one is immediately available, else raise `QueueEmpty`.

Raises `QueueShutDown` if the queue has been shut down and is empty.

</div>

<div class="method" async="">

join()

Block until all items in the queue have been received and processed.

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer coroutine calls `task_done` to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, `join` unblocks.

</div>

<div class="method" async="">

put(item)

Put an item into the queue. If the queue is full, wait until a free slot is available before adding the item.

Raises `QueueShutDown` if the queue has been shut down.

</div>

<div class="method">

put_nowait(item)

Put an item into the queue without blocking.

If no free slot is immediately available, raise `QueueFull`.

Raises `QueueShutDown` if the queue has been shut down.

</div>

<div class="method">

qsize()

Return the number of items in the queue.

</div>

<div class="method">

shutdown(immediate=False)

Put a `Queue` instance into a shutdown mode.

The queue can no longer grow. Future calls to `~Queue.put` raise `QueueShutDown`. Currently blocked callers of `~Queue.put` will be unblocked and will raise `QueueShutDown` in the formerly awaiting task.

If *immediate* is false (the default), the queue can be wound down normally with `~Queue.get` calls to extract tasks that have already been loaded.

And if `~Queue.task_done` is called for each remaining task, a pending `~Queue.join` will be unblocked normally.

Once the queue is empty, future calls to `~Queue.get` will raise `QueueShutDown`.

If *immediate* is true, the queue is terminated immediately. The queue is drained to be completely empty and the count of unfinished tasks is reduced by the number of tasks drained. If unfinished tasks is zero, callers of `~Queue.join` are unblocked. Also, blocked callers of `~Queue.get` are unblocked and will raise `QueueShutDown` because the queue is empty.

Use caution when using `~Queue.join` with *immediate* set to true. This unblocks the join even when no work has been done on the tasks, violating the usual invariant for joining a queue.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

task_done()

Indicate that a formerly enqueued work item is complete.

Used by queue consumers. For each `~Queue.get` used to fetch a work item, a subsequent call to `task_done` tells the queue that the processing on the work item is complete.

If a `join` is currently blocking, it will resume when all items have been processed (meaning that a `task_done` call was received for every item that had been `~Queue.put` into the queue).

Raises `ValueError` if called more times than there were items placed in the queue.

</div>

</div>

## Priority Queue

<div class="PriorityQueue">

A variant of `Queue`; retrieves entries in priority order (lowest first).

Entries are typically tuples of the form `(priority_number, data)`.

</div>

## LIFO Queue

<div class="LifoQueue">

A variant of `Queue` that retrieves most recently added entries first (last in, first out).

</div>

## Exceptions

<div class="exception">

QueueEmpty

This exception is raised when the `~Queue.get_nowait` method is called on an empty queue.

</div>

<div class="exception">

QueueFull

Exception raised when the `~Queue.put_nowait` method is called on a queue that has reached its *maxsize*.

</div>

<div class="exception">

QueueShutDown

Exception raised when `~Queue.put`, `~Queue.put_nowait`, `~Queue.get` or `~Queue.get_nowait` is called on a queue which has been shut down.

<div class="versionadded">

3.13

</div>

</div>

## Examples

<div id="asyncio_example_queue_dist">

Queues can be used to distribute workload between several concurrent tasks:

    import asyncio
    import random
    import time


    async def worker(name, queue):
        while True:
            # Get a "work item" out of the queue.
            sleep_for = await queue.get()

            # Sleep for the "sleep_for" seconds.
            await asyncio.sleep(sleep_for)

            # Notify the queue that the "work item" has been processed.
            queue.task_done()

            print(f'{name} has slept for {sleep_for:.2f} seconds')


    async def main():
        # Create a queue that we will use to store our "workload".
        queue = asyncio.Queue()

        # Generate random timings and put them into the queue.
        total_sleep_time = 0
        for _ in range(20):
            sleep_for = random.uniform(0.05, 1.0)
            total_sleep_time += sleep_for
            queue.put_nowait(sleep_for)

        # Create three worker tasks to process the queue concurrently.
        tasks = []
        for i in range(3):
            task = asyncio.create_task(worker(f'worker-{i}', queue))
            tasks.append(task)

        # Wait until the queue is fully processed.
        started_at = time.monotonic()
        await queue.join()
        total_slept_for = time.monotonic() - started_at

        # Cancel our worker tasks.
        for task in tasks:
            task.cancel()
        # Wait until all worker tasks are cancelled.
        await asyncio.gather(*tasks, return_exceptions=True)

        print('====')
        print(f'3 workers slept in parallel for {total_slept_for:.2f} seconds')
        print(f'total expected sleep time: {total_sleep_time:.2f} seconds')


    asyncio.run(main())

</div>

# asyncio-runner

<div class="currentmodule">

asyncio

</div>

# Runners

**Source code:** `Lib/asyncio/runners.py`

This section outlines high-level asyncio primitives to run asyncio code.

They are built on top of an `event loop <asyncio-event-loop>` with the aim to simplify async code usage for common wide-spread scenarios.

<div class="contents" data-depth="1" data-local="">

</div>

## Running an asyncio Program

<div class="function">

run(coro, \*, debug=None, loop_factory=None, eager_tasks=False)

Execute *coro* in an asyncio event loop and return the result.

The argument can be any awaitable object.

This function runs the awaitable, taking care of managing the asyncio event loop, *finalizing asynchronous generators*, and closing the executor.

This function cannot be called when another asyncio event loop is running in the same thread.

If *debug* is `True`, the event loop will be run in debug mode. `False` disables debug mode explicitly. `None` is used to respect the global `asyncio-debug-mode` settings.

If *loop_factory* is not `None`, it is used to create a new event loop; otherwise `asyncio.new_event_loop` is used. The loop is closed at the end. This function should be used as a main entry point for asyncio programs, and should ideally only be called once. It is recommended to use *loop_factory* to configure the event loop.

If *eager_tasks* is `True`, the created loop is configured to use `eager-task-factory` by default.

The executor is given a timeout duration of 5 minutes to shutdown. If the executor hasn't finished within that duration, a warning is emitted and the executor is closed.

> [!NOTE]
> Users concerned about performance are encouraged to experiment with `eager_tasks=True` in their code.

Example:

    async def main():
        await asyncio.sleep(1)
        print('hello')

    asyncio.run(main())

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.9 Updated to use `loop.shutdown_default_executor`.

</div>

<div class="versionchanged">

3.10

*debug* is `None` by default to respect the global debug mode settings.

</div>

<div class="versionchanged">

3.12

Added *loop_factory* parameter.

</div>

<div class="versionchanged">

3.14

*coro* can be any awaitable object.

</div>

<div class="versionchanged">

3.16

Added *eager_tasks* parameter.

</div>

</div>

## Runner context manager

<div class="Runner(*, debug=None, loop_factory=None, eager_tasks=False)">

A context manager that simplifies *multiple* async function calls in the same context.

Sometimes several top-level async functions should be called in the same `event
loop <asyncio-event-loop>` and `contextvars.Context`.

If *debug* is `True`, the event loop will be run in debug mode. `False` disables debug mode explicitly. `None` is used to respect the global `asyncio-debug-mode` settings.

*loop_factory* could be used for overriding the loop creation. It is the responsibility of the *loop_factory* to set the created loop as the current one. By default `asyncio.new_event_loop` is used and set as current event loop with `asyncio.set_event_loop` if *loop_factory* is `None`.

If *eager_tasks* is `True`, the created loop is configured to use `eager-task-factory` by default.

Basically, `asyncio.run` example can be rewritten with the runner usage:

    async def main():
        await asyncio.sleep(1)
        print('hello')

    with asyncio.Runner() as runner:
        runner.run(main())

> [!NOTE]
> Users concerned about performance are encouraged to experiment with `eager_tasks=True` in their code.

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.16

Added *eager_tasks* parameter.

</div>

<div class="method">

run(coro, \*, context=None)

Execute *coro* in the embedded event loop.

The argument can be any awaitable object.

If the argument is a coroutine, it is wrapped in a Task.

An optional keyword-only *context* argument allows specifying a custom `contextvars.Context` for the code to run in. The runner's default context is used if context is `None`.

Returns the awaitable's result or raises an exception.

This function cannot be called when another asyncio event loop is running in the same thread.

<div class="versionchanged">

3.14

*coro* can be any awaitable object.

</div>

</div>

<div class="method">

close()

Close the runner.

Finalize asynchronous generators, shutdown default executor, close the event loop and release embedded `contextvars.Context`.

</div>

<div class="method">

get_loop()

Return the event loop associated with the runner instance.

</div>

> [!NOTE]
> `Runner` uses the lazy initialization strategy, its constructor doesn't initialize underlying low-level structures.
>
> Embedded *loop* and *context* are created at the `with` body entering or the first call of `run` or `get_loop`.

</div>

## Handling Keyboard Interruption

<div class="versionadded">

3.11

</div>

When `signal.SIGINT` is raised by `Ctrl-C`, `KeyboardInterrupt` exception is raised in the main thread by default. However this doesn't work with `asyncio` because it can interrupt asyncio internals and can hang the program from exiting.

To mitigate this issue, `asyncio` handles `signal.SIGINT` as follows:

1.  `asyncio.Runner.run` installs a custom `signal.SIGINT` handler before any user code is executed and removes it when exiting from the function.
2.  The `~asyncio.Runner` creates the main task for the passed coroutine for its execution.
3.  When `signal.SIGINT` is raised by `Ctrl-C`, the custom signal handler cancels the main task by calling `asyncio.Task.cancel` which raises `asyncio.CancelledError` inside the main task. This causes the Python stack to unwind, `try/except` and `try/finally` blocks can be used for resource cleanup. After the main task is cancelled, `asyncio.Runner.run` raises `KeyboardInterrupt`.
4.  A user could write a tight loop which cannot be interrupted by `asyncio.Task.cancel`, in which case the second following `Ctrl-C` immediately raises the `KeyboardInterrupt` without cancelling the main task.

# asyncio-stream

<div class="currentmodule">

asyncio

</div>

# Streams

**Source code:** `Lib/asyncio/streams.py`

------------------------------------------------------------------------

Streams are high-level async/await-ready primitives to work with network connections. Streams allow sending and receiving data without using callbacks or low-level protocols and transports.

<div id="asyncio_example_stream">

Here is an example of a TCP echo client written using asyncio streams:

    import asyncio

    async def tcp_echo_client(message):
        reader, writer = await asyncio.open_connection(
            '127.0.0.1', 8888)

        print(f'Send: {message!r}')
        writer.write(message.encode())
        await writer.drain()

        data = await reader.read(100)
        print(f'Received: {data.decode()!r}')

        print('Close the connection')
        writer.close()
        await writer.wait_closed()

    asyncio.run(tcp_echo_client('Hello World!'))

</div>

See also the [Examples](#examples) section below.

**Stream Functions**

The following top-level asyncio functions can be used to create and work with streams:

<div class="function" async="">

open_connection(host=None, port=None, \*, limit=65536, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None, happy_eyeballs_delay=None, interleave=None)

Establish a network connection and return a pair of `(reader, writer)` objects.

The returned *reader* and *writer* objects are instances of `StreamReader` and `StreamWriter` classes.

*limit* determines the buffer size limit used by the returned `StreamReader` instance. By default the *limit* is set to 64 KiB.

The rest of the arguments are passed directly to `loop.create_connection`.

> [!NOTE]
> The *sock* argument transfers ownership of the socket to the `StreamWriter` created. To close the socket, call its `~asyncio.StreamWriter.close` method.

<div class="versionchanged">

3.7 Added the *ssl_handshake_timeout* parameter.

</div>

<div class="versionchanged">

3.8 Added the *happy_eyeballs_delay* and *interleave* parameters.

</div>

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="versionchanged">

3.11 Added the *ssl_shutdown_timeout* parameter.

</div>

</div>

<div class="function" async="">

start_server(client_connected_cb, host=None, port=None, \*, limit=65536, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, keep_alive=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None, start_serving=True)

Start a socket server.

The *client_connected_cb* callback is called whenever a new client connection is established. It receives a `(reader, writer)` pair as two arguments, instances of the `StreamReader` and `StreamWriter` classes.

*client_connected_cb* can be a plain callable or a `coroutine function <coroutine>`; if it is a coroutine function, it will be automatically scheduled as a `Task`.

*limit* determines the buffer size limit used by the returned `StreamReader` instance. By default the *limit* is set to 64 KiB.

The rest of the arguments are passed directly to `loop.create_server`.

> [!NOTE]
> The *sock* argument transfers ownership of the socket to the server created. To close the socket, call the server's `~asyncio.Server.close` method.

<div class="versionchanged">

3.7 Added the *ssl_handshake_timeout* and *start_serving* parameters.

</div>

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="versionchanged">

3.11 Added the *ssl_shutdown_timeout* parameter.

</div>

<div class="versionchanged">

3.13 Added the *keep_alive* parameter.

</div>

</div>

**Unix Sockets**

<div class="function" async="">

open_unix_connection(path=None, \*, limit=65536, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None)

Establish a Unix socket connection and return a pair of `(reader, writer)`.

Similar to `open_connection` but operates on Unix sockets.

See also the documentation of `loop.create_unix_connection`.

> [!NOTE]
> The *sock* argument transfers ownership of the socket to the `StreamWriter` created. To close the socket, call its `~asyncio.StreamWriter.close` method.

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.7 Added the *ssl_handshake_timeout* parameter. The *path* parameter can now be a `path-like object`

</div>

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="versionchanged">

3.11 Added the *ssl_shutdown_timeout* parameter.

</div>

</div>

<div class="function" async="">

start_unix_server(client_connected_cb, path=None, \*, limit=65536, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None, start_serving=True, cleanup_socket=True, mode=None)

Start a Unix socket server.

Similar to `start_server` but works with Unix sockets.

If *cleanup_socket* is true then the Unix socket will automatically be removed from the filesystem when the server is closed, unless the socket has been replaced after the server has been created.

If *mode* is not `None`, the permissions of the Unix socket file are set to *mode* before the server starts accepting connections.

See also the documentation of `loop.create_unix_server`.

> [!NOTE]
> The *sock* argument transfers ownership of the socket to the server created. To close the socket, call the server's `~asyncio.Server.close` method.

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.7 Added the *ssl_handshake_timeout* and *start_serving* parameters. The *path* parameter can now be a `path-like object`.

</div>

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="versionchanged">

3.11 Added the *ssl_shutdown_timeout* parameter.

</div>

<div class="versionchanged">

3.13 Added the *cleanup_socket* parameter.

</div>

<div class="versionchanged">

3.16 Added the *mode* parameter.

</div>

</div>

## StreamReader

<div class="StreamReader">

Represents a reader object that provides APIs to read data from the IO stream. As an `asynchronous iterable`, the object supports the `async for` statement.

It is not recommended to instantiate *StreamReader* objects directly; use `open_connection` and `start_server` instead.

<div class="method">

feed_eof()

Acknowledge the EOF.

</div>

<div class="method" async="">

read(n=-1)

Read up to *n* bytes from the stream.

If *n* is not provided or set to `-1`, read until EOF, then return all read `bytes`. If EOF was received and the internal buffer is empty, return an empty `bytes` object.

If *n* is `0`, return an empty `bytes` object immediately.

If *n* is positive, return at most *n* available `bytes` as soon as at least 1 byte is available in the internal buffer. If EOF is received before any byte is read, return an empty `bytes` object.

</div>

<div class="method" async="">

readline()

Read one line, where "line" is a sequence of bytes ending with `\n`.

If EOF is received and `\n` was not found, the method returns partially read data.

If EOF is received and the internal buffer is empty, return an empty `bytes` object.

</div>

<div class="method" async="">

readexactly(n)

Read exactly *n* bytes.

Raise an `IncompleteReadError` if EOF is reached before *n* can be read. Use the `IncompleteReadError.partial` attribute to get the partially read data.

</div>

<div class="method" async="">

readuntil(separator=b'n')

Read data from the stream until *separator* is found.

On success, the data and separator will be removed from the internal buffer (consumed). Returned data will include the separator at the end.

If the amount of data read exceeds the configured stream limit, a `LimitOverrunError` exception is raised, and the data is left in the internal buffer and can be read again.

If EOF is reached before the complete separator is found, an `IncompleteReadError` exception is raised, and the internal buffer is reset. The `IncompleteReadError.partial` attribute may contain a portion of the separator.

The *separator* may also be a tuple of separators. In this case the return value will be the shortest possible that has any separator as the suffix. For the purposes of `LimitOverrunError`, the shortest possible separator is considered to be the one that matched.

<div class="versionadded">

3.5.2

</div>

<div class="versionchanged">

3.13

The *separator* parameter may now be a `tuple` of separators.

</div>

</div>

<div class="method">

at_eof()

Return `True` if the buffer is empty and `feed_eof` was called.

</div>

</div>

## StreamWriter

<div class="StreamWriter">

Represents a writer object that provides APIs to write data to the IO stream.

It is not recommended to instantiate *StreamWriter* objects directly; use `open_connection` and `start_server` instead.

<div class="method">

write(data)

The method attempts to write the *data* to the underlying socket immediately. If that fails, the data is queued in an internal write buffer until it can be sent.

The *data* buffer should be a bytes, bytearray, or C-contiguous one-dimensional memoryview object.

The method should be used along with the `drain()` method:

    stream.write(data)
    await stream.drain()

</div>

<div class="method">

writelines(data)

The method writes a list (or any iterable) of bytes to the underlying socket immediately. If that fails, the data is queued in an internal write buffer until it can be sent.

The method should be used along with the `drain()` method:

    stream.writelines(lines)
    await stream.drain()

</div>

<div class="method">

close()

The method closes the stream and the underlying socket.

The method should be used, though not mandatory, along with the `wait_closed()` method:

    stream.close()
    await stream.wait_closed()

</div>

<div class="method">

can_write_eof()

Return `True` if the underlying transport supports the `write_eof` method, `False` otherwise.

</div>

<div class="method">

write_eof()

Close the write end of the stream after the buffered write data is flushed.

</div>

<div class="attribute">

transport

Return the underlying asyncio transport.

</div>

<div class="method">

get_extra_info(name, default=None)

Access optional transport information; see `BaseTransport.get_extra_info` for details.

</div>

<div class="method" async="">

drain()

Wait until it is appropriate to resume writing to the stream. Example:

    writer.write(data)
    await writer.drain()

This is a flow control method that interacts with the underlying IO write buffer. When the size of the buffer reaches the high watermark, *drain()* blocks until the size of the buffer is drained down to the low watermark and writing can be resumed. When there is nothing to wait for, the `drain` returns immediately.

> [!NOTE]
> When the write buffer is below the high watermark, `drain` returns immediately without yielding to the event loop. As a result, code which repeatedly calls `write()` followed by `await drain()` may prevent other tasks from running. To prevent blocking behavior, yield to the event loop explicitly with `await asyncio.sleep(0)` (see `asyncio.sleep`).

</div>

<div class="method" async="">

start_tls(sslcontext, \*, server_hostname=None, ssl_handshake_timeout=None, ssl_shutdown_timeout=None)

Upgrade an existing stream-based connection to TLS.

Parameters:

- *sslcontext*: a configured instance of `~ssl.SSLContext`.
- *server_hostname*: sets or overrides the host name that the target server's certificate will be matched against.
- *ssl_handshake_timeout* is the time in seconds to wait for the TLS handshake to complete before aborting the connection. `60.0` seconds if `None` (default).
- *ssl_shutdown_timeout* is the time in seconds to wait for the SSL shutdown to complete before aborting the connection. `30.0` seconds if `None` (default).

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.12 Added the *ssl_shutdown_timeout* parameter.

</div>

<div class="versionchanged">

3.16 Raises a `ValueError` if `sslcontext.check_hostname` is `True` and `server_hostname` is not supplied.

</div>

</div>

<div class="method">

is_closing()

Return `True` if the stream is closed or in the process of being closed.

<div class="versionadded">

3.7

</div>

</div>

<div class="method" async="">

wait_closed()

Wait until the stream is closed.

Should be called after `close` to wait until the underlying connection is closed, ensuring that all data has been flushed before e.g. exiting the program.

<div class="versionadded">

3.7

</div>

</div>

</div>

## Examples

### TCP echo client using streams

TCP echo client using the `asyncio.open_connection` function:

    import asyncio

    async def tcp_echo_client(message):
        reader, writer = await asyncio.open_connection(
            '127.0.0.1', 8888)

        print(f'Send: {message!r}')
        writer.write(message.encode())
        await writer.drain()

        data = await reader.read(100)
        print(f'Received: {data.decode()!r}')

        print('Close the connection')
        writer.close()
        await writer.wait_closed()

    asyncio.run(tcp_echo_client('Hello World!'))

<div class="seealso">

The `TCP echo client protocol <asyncio_example_tcp_echo_client_protocol>` example uses the low-level `loop.create_connection` method.

</div>

### TCP echo server using streams

TCP echo server using the `asyncio.start_server` function:

    import asyncio

    async def handle_echo(reader, writer):
        data = await reader.read(100)
        message = data.decode()
        addr = writer.get_extra_info('peername')

        print(f"Received {message!r} from {addr!r}")

        print(f"Send: {message!r}")
        writer.write(data)
        await writer.drain()

        print("Close the connection")
        writer.close()
        await writer.wait_closed()

    async def main():
        server = await asyncio.start_server(
            handle_echo, '127.0.0.1', 8888)

        addrs = ', '.join(str(sock.getsockname()) for sock in server.sockets)
        print(f'Serving on {addrs}')

        async with server:
            await server.serve_forever()

    asyncio.run(main())

<div class="seealso">

The `TCP echo server protocol <asyncio_example_tcp_echo_server_protocol>` example uses the `loop.create_server` method.

</div>

### Get HTTP headers

Simple example querying HTTP headers of the URL passed on the command line:

    import asyncio
    import urllib.parse
    import sys

    async def print_http_headers(url):
        url = urllib.parse.urlsplit(url)
        if url.scheme == 'https':
            reader, writer = await asyncio.open_connection(
                url.hostname, 443, ssl=True)
        else:
            reader, writer = await asyncio.open_connection(
                url.hostname, 80)

        query = (
            f"HEAD {url.path or '/'} HTTP/1.0\r\n"
            f"Host: {url.hostname}\r\n"
            f"\r\n"
        )

        writer.write(query.encode('latin-1'))
        while True:
            line = await reader.readline()
            if not line:
                break

            line = line.decode('latin1').rstrip()
            if line:
                print(f'HTTP header> {line}')

        # Ignore the body, close the socket
        writer.close()
        await writer.wait_closed()

    url = sys.argv[1]
    asyncio.run(print_http_headers(url))

Usage:

    python example.py http://example.com/path/page.html

or with HTTPS:

    python example.py https://example.com/path/page.html

### Register an open socket to wait for data using streams

Coroutine waiting until a socket receives data using the `open_connection` function:

    import asyncio
    import socket

    async def wait_for_data():
        # Get a reference to the current event loop because
        # we want to access low-level APIs.
        loop = asyncio.get_running_loop()

        # Create a pair of connected sockets.
        rsock, wsock = socket.socketpair()

        # Register the open socket to wait for data.
        reader, writer = await asyncio.open_connection(sock=rsock)

        # Simulate the reception of data from the network
        loop.call_soon(wsock.send, 'abc'.encode())

        # Wait for data
        data = await reader.read(100)

        # Got data, we are done: close the socket
        print("Received:", data.decode())
        writer.close()
        await writer.wait_closed()

        # Close the second socket
        wsock.close()

    asyncio.run(wait_for_data())

<div class="seealso">

The `register an open socket to wait for data using a protocol
<asyncio_example_create_connection>` example uses a low-level protocol and the `loop.create_connection` method.

The `watch a file descriptor for read events
<asyncio_example_watch_fd>` example uses the low-level `loop.add_reader` method to watch a file descriptor.

</div>

# asyncio-subprocess

<div class="currentmodule">

asyncio

</div>

# Subprocesses

**Source code:** `Lib/asyncio/subprocess.py`, `Lib/asyncio/base_subprocess.py`

------------------------------------------------------------------------

This section describes high-level async/await asyncio APIs to create and manage subprocesses.

<div id="asyncio_example_subprocess_shell">

Here's an example of how asyncio can run a shell command and obtain its result:

    import asyncio

    async def run(cmd):
        proc = await asyncio.create_subprocess_shell(
            cmd,
            stdout=asyncio.subprocess.PIPE,
            stderr=asyncio.subprocess.PIPE)

        stdout, stderr = await proc.communicate()

        print(f'[{cmd!r} exited with {proc.returncode}]')
        if stdout:
            print(f'[stdout]\n{stdout.decode()}')
        if stderr:
            print(f'[stderr]\n{stderr.decode()}')

    asyncio.run(run('ls /zzz'))

</div>

will print:

    ['ls /zzz' exited with 1]
    [stderr]
    ls: /zzz: No such file or directory

Because all asyncio subprocess functions are asynchronous and asyncio provides many tools to work with such functions, it is easy to execute and monitor multiple subprocesses in parallel. It is indeed trivial to modify the above example to run several commands simultaneously:

    async def main():
        await asyncio.gather(
            run('ls /zzz'),
            run('sleep 1; echo "hello"'))

    asyncio.run(main())

See also the [Examples](#examples) subsection.

## Creating Subprocesses

<div class="function" async="">

create_subprocess_exec(program, *args, stdin=None, stdout=None, stderr=None, limit=65536,*\*kwds)

Create a subprocess.

The *limit* argument sets the buffer limit for `StreamReader` wrappers for `~asyncio.subprocess.Process.stdout` and `~asyncio.subprocess.Process.stderr` (if `subprocess.PIPE` is passed to *stdout* and *stderr* arguments).

Return a `~asyncio.subprocess.Process` instance.

See the documentation of `loop.subprocess_exec` for other parameters.

If the process object is garbage collected while the process is still running, the child process will be killed.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

</div>

<div class="function" async="">

create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, limit=65536, \*\*kwds)

Run the *cmd* shell command.

The *limit* argument sets the buffer limit for `StreamReader` wrappers for `~asyncio.subprocess.Process.stdout` and `~asyncio.subprocess.Process.stderr` (if `subprocess.PIPE` is passed to *stdout* and *stderr* arguments).

Return a `~asyncio.subprocess.Process` instance.

See the documentation of `loop.subprocess_shell` for other parameters.

If the process object is garbage collected while the process is still running, the child process will be killed.

> [!IMPORTANT]
> It is the application's responsibility to ensure that all whitespace and special characters are quoted appropriately to avoid [shell injection](https://en.wikipedia.org/wiki/Shell_injection#Shell_injection) vulnerabilities. The `shlex.quote` function can be used to properly escape whitespace and special shell characters in strings that are going to be used to construct shell commands.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

</div>

> [!NOTE]
> Subprocesses are available for Windows if a `ProactorEventLoop` is used. See `Subprocess Support on Windows <asyncio-windows-subprocess>` for details.

<div class="seealso">

asyncio also has the following *low-level* APIs to work with subprocesses: `loop.subprocess_exec`, `loop.subprocess_shell`, `loop.connect_read_pipe`, `loop.connect_write_pipe`, as well as the `Subprocess Transports <asyncio-subprocess-transports>` and `Subprocess Protocols <asyncio-subprocess-protocols>`.

</div>

## Constants

<div class="data" data-module="">

asyncio.subprocess.PIPE

Can be passed to the *stdin*, *stdout* or *stderr* parameters.

If *PIPE* is passed to *stdin* argument, the `Process.stdin <asyncio.subprocess.Process.stdin>` attribute will point to a `~asyncio.StreamWriter` instance.

If *PIPE* is passed to *stdout* or *stderr* arguments, the `Process.stdout <asyncio.subprocess.Process.stdout>` and `Process.stderr <asyncio.subprocess.Process.stderr>` attributes will point to `~asyncio.StreamReader` instances.

</div>

<div class="data" data-module="">

asyncio.subprocess.STDOUT

Special value that can be used as the *stderr* argument and indicates that standard error should be redirected into standard output.

</div>

<div class="data" data-module="">

asyncio.subprocess.DEVNULL

Special value that can be used as the *stdin*, *stdout* or *stderr* argument to process creation functions. It indicates that the special file `os.devnull` will be used for the corresponding subprocess stream.

</div>

## Interacting with Subprocesses

Both `create_subprocess_exec` and `create_subprocess_shell` functions return instances of the *Process* class. *Process* is a high-level wrapper that allows communicating with subprocesses and watching for their completion.

<div class="asyncio.subprocess.Process" data-module="">

An object that wraps OS processes created by the `~asyncio.create_subprocess_exec` and `~asyncio.create_subprocess_shell` functions.

This class is designed to have a similar API to the `subprocess.Popen` class, but there are some notable differences:

- unlike Popen, Process instances do not have an equivalent to the `~subprocess.Popen.poll` method;
- the `~asyncio.subprocess.Process.communicate` and `~asyncio.subprocess.Process.wait` methods don't have a *timeout* parameter: use the `~asyncio.wait_for` function;
- the `Process.wait() <asyncio.subprocess.Process.wait>` method is asynchronous, whereas `subprocess.Popen.wait` method is implemented as a blocking busy loop;
- the *universal_newlines* parameter is not supported.

This class is `not thread safe <asyncio-multithreading>`.

See also the `Subprocess and Threads <asyncio-subprocess-threads>` section.

<div class="method" async="">

wait()

Wait for the child process to terminate.

Set and return the `returncode` attribute.

> [!NOTE]
> This method can deadlock when using `stdout=PIPE` or `stderr=PIPE` and the child process generates so much output that it blocks waiting for the OS pipe buffer to accept more data. Use the `communicate` method when using pipes to avoid this condition.

</div>

<div class="method" async="">

communicate(input=None)

Interact with process:

1.  send data to *stdin* (if *input* is not `None`);
2.  closes *stdin*;
3.  read data from *stdout* and *stderr*, until EOF is reached;
4.  wait for process to terminate.

The optional *input* argument is the data (`bytes` object) that will be sent to the child process.

Return a tuple `(stdout_data, stderr_data)`.

If either `BrokenPipeError` or `ConnectionResetError` exception is raised when writing *input* into *stdin*, the exception is ignored. This condition occurs when the process exits before all data are written into *stdin*.

If it is desired to send data to the process' *stdin*, the process needs to be created with `stdin=PIPE`. Similarly, to get anything other than `None` in the result tuple, the process has to be created with `stdout=PIPE` and/or `stderr=PIPE` arguments.

Note, that the data read is buffered in memory, so do not use this method if the data size is large or unlimited.

If this coroutine is cancelled (for example, when a timeout is set with `~asyncio.wait_for`), the output that was already read is not lost: call `!communicate` again to read the remaining output and get the complete data:

    try:
        stdout, stderr = await asyncio.wait_for(
            proc.communicate(), timeout=5.0)
    except TimeoutError:
        proc.kill()
        stdout, stderr = await proc.communicate()

Passing *input* after a previous `!communicate` call was cancelled raises `ValueError`; pass `input=None` to continue the communication, the original *input* is used.

<div class="versionchanged">

3.12

*stdin* gets closed when `input=None` too.

</div>

<div class="versionchanged">

3.16

If `!communicate` is cancelled, the output that was already read is now preserved and returned by a subsequent `!communicate` call. Passing *input* to a `!communicate` call following a cancelled one now raises `ValueError`.

</div>

</div>

<div class="method">

send_signal(signal)

Sends the signal *signal* to the child process.

> [!NOTE]
> On Windows, `~signal.SIGTERM` is an alias for `terminate`. `CTRL_C_EVENT` and `CTRL_BREAK_EVENT` can be sent to processes started with a *creationflags* parameter which includes `CREATE_NEW_PROCESS_GROUP`.

</div>

<div class="method">

terminate()

Stop the child process.

On POSIX systems this method sends `~signal.SIGTERM` to the child process.

On Windows the Win32 API function `!TerminateProcess` is called to stop the child process.

</div>

<div class="method">

kill()

Kill the child process.

On POSIX systems this method sends `~signal.SIGKILL` to the child process.

On Windows this method is an alias for `terminate`.

</div>

<div class="attribute">

stdin

Standard input stream (`~asyncio.StreamWriter`) or `None` if the process was created with `stdin=None`.

</div>

<div class="attribute">

stdout

Standard output stream (`~asyncio.StreamReader`) or `None` if the process was created with `stdout=None`.

</div>

<div class="attribute">

stderr

Standard error stream (`~asyncio.StreamReader`) or `None` if the process was created with `stderr=None`.

</div>

> [!WARNING]
> Use the `communicate` method rather than `process.stdin.write() <stdin>`, `await process.stdout.read() <stdout>` or `await process.stderr.read() <stderr>`. This avoids deadlocks due to streams pausing reading or writing and blocking the child process.

<div class="attribute">

pid

Process identification number (PID).

Note that for processes created by the `~asyncio.create_subprocess_shell` function, this attribute is the PID of the spawned shell.

</div>

<div class="attribute">

returncode

Return code of the process when it exits.

A `None` value indicates that the process has not terminated yet.

For processes created with `~asyncio.create_subprocess_exec`, a negative value `-N` indicates that the child was terminated by signal `N` (POSIX only).

For processes created with `~asyncio.create_subprocess_shell`, the return code reflects the exit status of the shell itself (e.g. `/bin/sh`), which may map signals to codes such as `128+N`. See the documentation of the shell (for example, the Bash manual's Exit Status) for details.

</div>

</div>

### Subprocess and Threads

Standard asyncio event loop supports running subprocesses from different threads by default.

On Windows subprocesses are provided by `ProactorEventLoop` only (default), `SelectorEventLoop` has no subprocess support.

Note that alternative event loop implementations might have own limitations; please refer to their documentation.

<div class="seealso">

The `Concurrency and multithreading in asyncio
<asyncio-multithreading>` section.

</div>

### Examples

An example using the `~asyncio.subprocess.Process` class to control a subprocess and the `StreamReader` class to read from its standard output.

<div id="asyncio_example_create_subprocess_exec">

The subprocess is created by the `create_subprocess_exec` function:

    import asyncio
    import sys

    async def get_date():
        code = 'import datetime as dt; print(dt.datetime.now())'

        # Create the subprocess; redirect the standard output
        # into a pipe.
        proc = await asyncio.create_subprocess_exec(
            sys.executable, '-c', code,
            stdout=asyncio.subprocess.PIPE)

        # Read one line of output.
        data = await proc.stdout.readline()
        line = data.decode('ascii').rstrip()

        # Wait for the subprocess exit.
        await proc.wait()
        return line

    date = asyncio.run(get_date())
    print(f"Current date: {date}")

</div>

See also the `same example <asyncio_example_subprocess_proto>` written using low-level APIs.

# asyncio-sync

<div class="currentmodule">

asyncio

</div>

# Synchronization Primitives

**Source code:** `Lib/asyncio/locks.py`

------------------------------------------------------------------------

asyncio synchronization primitives are designed to be similar to those of the `threading` module with two important caveats:

- asyncio primitives are not thread-safe, therefore they should not be used for OS thread synchronization (use `threading` for that);
- methods of these synchronization primitives do not accept the *timeout* argument; use the `asyncio.wait_for` function to perform operations with timeouts.

asyncio has the following basic synchronization primitives:

- `Lock`
- `Event`
- `Condition`
- `Semaphore`
- `BoundedSemaphore`
- `Barrier`

------------------------------------------------------------------------

## Lock

<div class="Lock()">

Implements a mutex lock for asyncio tasks. Not thread-safe.

An asyncio lock can be used to guarantee exclusive access to a shared resource.

The preferred way to use a Lock is an `async with` statement:

    lock = asyncio.Lock()

    # ... later
    async with lock:
        # access shared state

which is equivalent to:

    lock = asyncio.Lock()

    # ... later
    await lock.acquire()
    try:
        # access shared state
    finally:
        lock.release()

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="method" async="">

acquire()

Acquire the lock.

This method waits until the lock is *unlocked*, sets it to *locked* and returns `True`.

When more than one coroutine is blocked in `acquire` waiting for the lock to be unlocked, only one coroutine eventually proceeds.

Acquiring a lock is *fair*: the coroutine that proceeds will be the first coroutine that started waiting on the lock.

</div>

<div class="method">

release()

Release the lock.

When the lock is *locked*, reset it to *unlocked* and return.

If the lock is *unlocked*, a `RuntimeError` is raised.

</div>

<div class="method">

locked()

Return `True` if the lock is *locked*.

</div>

</div>

## Event

<div class="Event()">

An event object. Not thread-safe.

An asyncio event can be used to notify multiple asyncio tasks that some event has happened.

An Event object manages an internal flag that can be set to *true* with the `~Event.set` method and reset to *false* with the `clear` method. The `~Event.wait` method blocks until the flag is set to *true*. The flag is set to *false* initially.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div id="asyncio_example_sync_event">

Example:

    async def waiter(event):
        print('waiting for it ...')
        await event.wait()
        print('... got it!')

    async def main():
        # Create an Event object.
        event = asyncio.Event()

        # Spawn a Task to wait until 'event' is set.
        waiter_task = asyncio.create_task(waiter(event))

        # Sleep for 1 second and set the event.
        await asyncio.sleep(1)
        event.set()

        # Wait until the waiter task is finished.
        await waiter_task

    asyncio.run(main())

</div>

<div class="method" async="">

wait()

Wait until the event is set.

If the event is set, return `True` immediately. Otherwise block until another task calls `~Event.set`.

</div>

<div class="method">

set()

Set the event.

All tasks waiting for event to be set will be immediately awakened.

</div>

<div class="method">

clear()

Clear (unset) the event.

Subsequent tasks awaiting on `~Event.wait` will now block until the `~Event.set` method is called again.

</div>

<div class="method">

is_set()

Return `True` if the event is set.

</div>

</div>

## Condition

<div class="Condition(lock=None)">

A Condition object. Not thread-safe.

An asyncio condition primitive can be used by a task to wait for some event to happen and then get exclusive access to a shared resource.

In essence, a Condition object combines the functionality of an `Event` and a `Lock`. It is possible to have multiple Condition objects share one Lock, which allows coordinating exclusive access to a shared resource between different tasks interested in particular states of that shared resource.

The optional *lock* argument must be a `Lock` object or `None`. In the latter case a new Lock object is created automatically.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

The preferred way to use a Condition is an `async with` statement:

    cond = asyncio.Condition()

    # ... later
    async with cond:
        await cond.wait()

which is equivalent to:

    cond = asyncio.Condition()

    # ... later
    await cond.acquire()
    try:
        await cond.wait()
    finally:
        cond.release()

<div class="method" async="">

acquire()

Acquire the underlying lock.

This method waits until the underlying lock is *unlocked*, sets it to *locked* and returns `True`.

</div>

<div class="method">

notify(n=1)

Wake up *n* tasks (1 by default) waiting on this condition. If fewer than *n* tasks are waiting they are all awakened.

The lock must be acquired before this method is called and released shortly after. If called with an *unlocked* lock a `RuntimeError` error is raised.

</div>

<div class="method">

locked()

Return `True` if the underlying lock is acquired.

</div>

<div class="method">

notify_all()

Wake up all tasks waiting on this condition.

This method acts like `notify`, but wakes up all waiting tasks.

The lock must be acquired before this method is called and released shortly after. If called with an *unlocked* lock a `RuntimeError` error is raised.

</div>

<div class="method">

release()

Release the underlying lock.

When invoked on an unlocked lock, a `RuntimeError` is raised.

</div>

<div class="method" async="">

wait()

Wait until notified.

If the calling task has not acquired the lock when this method is called, a `RuntimeError` is raised.

This method releases the underlying lock, and then blocks until it is awakened by a `notify` or `notify_all` call. Once awakened, the Condition re-acquires its lock and this method returns `True`.

Note that a task *may* return from this call spuriously, which is why the caller should always re-check the state and be prepared to `~Condition.wait` again. For this reason, you may prefer to use `~Condition.wait_for` instead.

</div>

<div class="method" async="">

wait_for(predicate)

Wait until a predicate becomes *true*.

The predicate must be a callable which result will be interpreted as a boolean value. The method will repeatedly `~Condition.wait` until the predicate evaluates to *true*. The final value is the return value.

</div>

</div>

## Semaphore

<div class="Semaphore(value=1)">

A Semaphore object. Not thread-safe.

A semaphore manages an internal counter which is decremented by each `acquire` call and incremented by each `release` call. The counter can never go below zero; when `acquire` finds that it is zero, it blocks, waiting until some task calls `release`.

The optional *value* argument gives the initial value for the internal counter (`1` by default). If the given value is less than `0` a `ValueError` is raised.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

The preferred way to use a Semaphore is an `async with` statement:

    sem = asyncio.Semaphore(10)

    # ... later
    async with sem:
        # work with shared resource

which is equivalent to:

    sem = asyncio.Semaphore(10)

    # ... later
    await sem.acquire()
    try:
        # work with shared resource
    finally:
        sem.release()

<div class="method" async="">

acquire()

Acquire a semaphore.

If the internal counter is greater than zero, decrement it by one and return `True` immediately. If it is zero, wait until a `release` is called and return `True`.

</div>

<div class="method">

locked()

Returns `True` if semaphore can not be acquired immediately.

</div>

<div class="method">

release()

Release a semaphore, incrementing the internal counter by one. Can wake up a task waiting to acquire the semaphore.

Unlike `BoundedSemaphore`, `Semaphore` allows making more `release()` calls than `acquire()` calls.

</div>

</div>

## BoundedSemaphore

<div class="BoundedSemaphore(value=1)">

A bounded semaphore object. Not thread-safe.

Bounded Semaphore is a version of `Semaphore` that raises a `ValueError` in `~Semaphore.release` if it increases the internal counter above the initial *value*.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

</div>

## Barrier

<div class="Barrier(parties)">

A barrier object. Not thread-safe.

A barrier is a simple synchronization primitive that allows to block until *parties* number of tasks are waiting on it. Tasks can wait on the `~Barrier.wait` method and would be blocked until the specified number of tasks end up waiting on `~Barrier.wait`. At that point all of the waiting tasks would unblock simultaneously.

`async with` can be used as an alternative to awaiting on `~Barrier.wait`.

The barrier can be reused any number of times.

<div id="asyncio_example_barrier">

Example:

    async def example_barrier():
       # barrier with 3 parties
       b = asyncio.Barrier(3)

       # create 2 new waiting tasks
       asyncio.create_task(b.wait())
       asyncio.create_task(b.wait())

       await asyncio.sleep(0)
       print(b)

       # The third .wait() call passes the barrier
       await b.wait()
       print(b)
       print("barrier passed")

       await asyncio.sleep(0)
       print(b)

    asyncio.run(example_barrier())

</div>

Result of this example is:

    <asyncio.locks.Barrier object at 0x... [filling, waiters:2/3]>
    <asyncio.locks.Barrier object at 0x... [draining, waiters:0/3]>
    barrier passed
    <asyncio.locks.Barrier object at 0x... [filling, waiters:0/3]>

<div class="versionadded">

3.11

</div>

<div class="method" async="">

wait()

Pass the barrier. When all the tasks party to the barrier have called this function, they are all unblocked simultaneously.

When a waiting or blocked task in the barrier is cancelled, this task exits the barrier which stays in the same state. If the state of the barrier is "filling", the number of waiting task decreases by 1.

The return value is an integer in the range of 0 to `parties-1`, different for each task. This can be used to select a task to do some special housekeeping, e.g.:

    ...
    async with barrier as position:
       if position == 0:
          # Only one task prints this
          print('End of *draining phase*')

This method may raise a `BrokenBarrierError` exception if the barrier is broken or reset while a task is waiting. It could raise a `CancelledError` if a task is cancelled.

</div>

<div class="method" async="">

reset()

Return the barrier to the default, empty state. Any tasks waiting on it will receive the `BrokenBarrierError` exception.

If a barrier is broken it may be better to just leave it and create a new one.

</div>

<div class="method" async="">

abort()

Put the barrier into a broken state. This causes any active or future calls to `~Barrier.wait` to fail with the `BrokenBarrierError`. Use this for example if one of the tasks needs to abort, to avoid infinite waiting tasks.

</div>

<div class="attribute">

parties

The number of tasks required to pass the barrier.

</div>

<div class="attribute">

n_waiting

The number of tasks currently waiting in the barrier while filling.

</div>

<div class="attribute">

broken

A boolean that is `True` if the barrier is in the broken state.

</div>

</div>

<div class="exception">

BrokenBarrierError

This exception, a subclass of `RuntimeError`, is raised when the `Barrier` object is reset or broken.

</div>

------------------------------------------------------------------------

<div class="versionchanged">

3.9

Acquiring a lock using `await lock` or `yield from lock` and/or `with` statement (`with await lock`, `with (yield from lock)`) was removed. Use `async with lock` instead.

</div>

# asyncio-task

<div class="currentmodule">

asyncio

</div>

# Coroutines and tasks

This section outlines high-level asyncio APIs to work with coroutines and Tasks.

<div class="contents" data-depth="1" data-local="">

</div>

## Coroutines

**Source code:** `Lib/asyncio/coroutines.py`

------------------------------------------------------------------------

`Coroutines <coroutine>` declared with the async/await syntax is the preferred way of writing asyncio applications. For example, the following snippet of code prints "hello", waits 1 second, and then prints "world":

    >>> import asyncio

    >>> async def main():
    ...     print('hello')
    ...     await asyncio.sleep(1)
    ...     print('world')

    >>> asyncio.run(main())
    hello
    world

Note that simply calling a coroutine will not schedule it to be executed:

    >>> main()
    <coroutine object main at 0x1053bb7c8>

To actually run a coroutine, asyncio provides the following mechanisms:

- The `asyncio.run` function to run the top-level entry point "main()" function (see the above example.)

- Awaiting on a coroutine. The following snippet of code will print "hello" after waiting for 1 second, and then print "world" after waiting for *another* 2 seconds:

      import asyncio
      import time

      async def say_after(delay, what):
          await asyncio.sleep(delay)
          print(what)

      async def main():
          print(f"started at {time.strftime('%X')}")

          await say_after(1, 'hello')
          await say_after(2, 'world')

          print(f"finished at {time.strftime('%X')}")

      asyncio.run(main())

  Expected output:

      started at 17:13:52
      hello
      world
      finished at 17:13:55

- The `asyncio.create_task` function to run coroutines concurrently as asyncio `Tasks <Task>`.

  Let's modify the above example and run two `say_after` coroutines *concurrently*:

      async def main():
          task1 = asyncio.create_task(
              say_after(1, 'hello'))

          task2 = asyncio.create_task(
              say_after(2, 'world'))

          print(f"started at {time.strftime('%X')}")

          # Wait until both tasks are completed (should take
          # around 2 seconds.)
          await task1
          await task2

          print(f"finished at {time.strftime('%X')}")

  Note that expected output now shows that the snippet runs 1 second faster than before:

      started at 17:14:32
      hello
      world
      finished at 17:14:34

- The `asyncio.TaskGroup` class provides a more modern alternative to `create_task`. Using this API, the last example becomes:

      async def main():
          async with asyncio.TaskGroup() as tg:
              task1 = tg.create_task(
                  say_after(1, 'hello'))

              task2 = tg.create_task(
                  say_after(2, 'world'))

              print(f"started at {time.strftime('%X')}")

          # The await is implicit when the context manager exits.

          print(f"finished at {time.strftime('%X')}")

  The timing and output should be the same as for the previous version.

  <div class="versionadded">

  3.11 `asyncio.TaskGroup`.

  </div>

## Awaitables

We say that an object is an **awaitable** object if it can be used in an `await` expression. Many asyncio APIs are designed to accept awaitables.

There are three main types of *awaitable* objects: **coroutines**, **Tasks**, and **Futures**.

**Coroutines**

Python coroutines are *awaitables* and therefore can be awaited from other coroutines:

    import asyncio

    async def nested():
        return 42

    async def main():
        # Nothing happens if we just call "nested()".
        # A coroutine object is created but not awaited,
        # so it *won't run at all*.
        nested()  # will raise a "RuntimeWarning".

        # Let's do it differently now and await it:
        print(await nested())  # will print "42".

    asyncio.run(main())

> [!IMPORTANT]
> In this documentation the term "coroutine" can be used for two closely related concepts:
>
> - a *coroutine function*: an `async def` function;
> - a *coroutine object*: an object returned by calling a *coroutine function*.
>
> Generator-based coroutines, created with `types.coroutine`, are covered by neither term and are not supported by asyncio.

**Tasks**

*Tasks* are used to schedule coroutines *concurrently*.

When a coroutine is wrapped into a *Task* with functions like `asyncio.create_task` the coroutine is automatically scheduled to run soon:

    import asyncio

    async def nested():
        return 42

    async def main():
        # Schedule nested() to run soon concurrently
        # with "main()".
        task = asyncio.create_task(nested())

        # "task" can now be used to cancel "nested()", or
        # can simply be awaited to wait until it is complete:
        await task

    asyncio.run(main())

**Futures**

A `Future` is a special **low-level** awaitable object that represents an **eventual result** of an asynchronous operation.

When a Future object is *awaited* it means that the coroutine will wait until the Future is resolved in some other place.

Future objects in asyncio are needed to allow callback-based code to be used with async/await.

Normally **there is no need** to create Future objects at the application level code.

Future objects, sometimes exposed by libraries and some asyncio APIs, can be awaited:

    async def main():
        await function_that_returns_a_future_object()

        # this is also valid:
        await asyncio.gather(
            function_that_returns_a_future_object(),
            some_python_coroutine()
        )

A good example of a low-level function that returns a Future object is `loop.run_in_executor`.

## Creating tasks

**Source code:** `Lib/asyncio/tasks.py`

------------------------------------------------------------------------

<div class="function">

create_task(coro, *, name=None, context=None, eager_start=None,*\*kwargs)

Wrap the *coro* `coroutine <coroutine>` into a `Task` and schedule its execution. Return the Task object.

The full function signature is largely the same as that of the `Task` constructor (or factory) - all of the keyword arguments to this function are passed through to that interface.

An optional keyword-only *context* argument allows specifying a custom `contextvars.Context` for the *coro* to run in. The current context copy is created when no *context* is provided.

An optional keyword-only *eager_start* argument allows specifying if the task should execute eagerly during the call to create_task, or be scheduled later. If *eager_start* is not passed the mode set by `loop.set_task_factory` will be used.

The task is executed in the loop returned by `get_running_loop`, `RuntimeError` is raised if there is no running loop in current thread.

> [!NOTE]
> `asyncio.TaskGroup.create_task` is a new alternative leveraging structural concurrency; it allows for waiting for a group of related tasks with strong safety guarantees.

> [!IMPORTANT]
> Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn't referenced elsewhere may get garbage collected at any time, even before it's done. For reliable "fire-and-forget" background tasks, gather them in a collection:
>
>     background_tasks = set()
>
>     for i in range(10):
>         task = asyncio.create_task(some_coro(param=i))
>
>         # Add task to the set. This creates a strong reference.
>         background_tasks.add(task)
>
>         # To prevent keeping references to finished tasks forever,
>         # make each task remove its own reference from the set after
>         # completion:
>         task.add_done_callback(background_tasks.discard)
>
> Note that this approach never awaits the tasks, so if a task fails, its exception is never retrieved and asyncio logs a "Task exception was never retrieved" message when the task is garbage collected. To avoid this, use `asyncio.TaskGroup` which keeps a strong reference to each task, awaits them and propagates their exceptions:
>
>     async with asyncio.TaskGroup() as tg:
>         for i in range(10):
>             tg.create_task(some_coro(param=i))

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.8 Added the *name* parameter.

</div>

<div class="versionchanged">

3.11 Added the *context* parameter.

</div>

<div class="versionchanged">

3.14 Added the *eager_start* parameter by passing on all *kwargs*.

</div>

</div>

## Task cancellation

Tasks can easily and safely be cancelled. When a task is cancelled, `asyncio.CancelledError` will be raised in the task at the next opportunity.

It is recommended that coroutines use `try/finally` blocks to robustly perform clean-up logic. In case `asyncio.CancelledError` is explicitly caught, it should generally be propagated when clean-up is complete. `asyncio.CancelledError` directly subclasses `BaseException` so most code will not need to be aware of it.

The asyncio components that enable structured concurrency, like `asyncio.TaskGroup` and `asyncio.timeout`, are implemented using cancellation internally and might misbehave if a coroutine swallows `asyncio.CancelledError`. Similarly, user code should not generally call `uncancel <asyncio.Task.uncancel>`. However, in cases when suppressing `asyncio.CancelledError` is truly desired, it is necessary to also call `uncancel()` to completely remove the cancellation state.

## Task groups

Task groups combine a task creation API with a convenient and reliable way to wait for all tasks in the group to finish.

<div class="TaskGroup()">

An `asynchronous context manager <async-context-managers>` holding a group of tasks. Tasks can be added to the group using `create_task`. All tasks are awaited when the context manager exits.

<div class="versionadded">

3.11

</div>

<div class="method">

create_task(coro, *, name=None, context=None, eager_start=None,*\*kwargs)

Create a task in this task group. The signature matches that of `asyncio.create_task`. If the task group is inactive (e.g. not yet entered, already finished, or in the process of shutting down), we will close the given `coro` and raise `RuntimeError`.

<div class="versionchanged">

3.13

Close the given coroutine if the task group is not active.

</div>

<div class="versionchanged">

3.14

Passes on all *kwargs* to `loop.create_task`

</div>

</div>

<div class="method">

cancel()

Cancel the task group. This is a non-exceptional, early exit of the task group's lifetime -- useful once the group's goal has been met or its services no longer needed.

`~asyncio.Task.cancel` will be called on any tasks in the group that aren't yet done, as well as the parent (body) of the group. The task group context manager will exit *without* `asyncio.CancelledError` being raised.

If `cancel` is called before entering the task group, the group will be cancelled upon entry. This is useful for patterns where one piece of code passes an unused `asyncio.TaskGroup` instance to another in order to have the ability to cancel anything run within the group.

`cancel` is idempotent and may be called after the task group has already exited.

Some ways to use `cancel`:

- call it from the task group body based on some condition or event
- pass the task group instance to child tasks via `create_task`, allowing a child task to conditionally cancel the entire group
- pass the task group instance or bound `cancel` method to some other task *before* opening the task group, allowing remote cancellation

<div class="versionadded">

3.15

</div>

</div>

</div>

Example:

    async def main():
        async with asyncio.TaskGroup() as tg:
            task1 = tg.create_task(some_coro(...))
            task2 = tg.create_task(another_coro(...))
        print(f"Both tasks have completed now: {task1.result()}, {task2.result()}")

A few points to keep in mind when using task groups:

- The `async with` statement will wait for all tasks in the group to finish. While waiting, new tasks may still be added to the group (for example, by passing `tg` into one of the coroutines and calling `tg.create_task()` in that coroutine); once the last task has finished and the `async with` block is exited, no new tasks may be added.

- Termination of the entire task group may be requested with `tg.cancel()`, based on some condition.

- If the group is shut down (e.g. because another task failed) before a newly created task has started running, the task is cancelled without its coroutine executing at all, not even to its first `await`. To guarantee that the coroutine starts, create the task eagerly with `eager_start=True` or use `asyncio.eager_task_factory`. For example:

      async def job():
          print("job started")  # never printed
          try:
              await asyncio.sleep(1)
          finally:
              print("job cleaned up")  # never printed

      async def main():
          async with asyncio.TaskGroup() as tg:
              tg.create_task(job())
              raise RuntimeError  # shuts down the group before job() runs

  With `tg.create_task(job(), eager_start=True)`, `job()` runs up to the `await`, is cancelled there, and both messages are printed.

When any of the tasks belonging to the group fails with an exception other than `asyncio.CancelledError` (or the body of the `async with` statement exits with an exception, which is treated the same way):

- The first time this happens, the remaining tasks in the group are cancelled and then waited for, and no further tasks can be added to the group. If the body of the `async with` statement is still active (i.e., `~object.__aexit__` hasn't been called yet), the task directly containing the `async with` statement is also cancelled. The resulting `asyncio.CancelledError` will interrupt an `await`, but it will not bubble out of the containing `async with` statement.
- Once all tasks have finished, the non-cancellation exceptions -- including the exception the body exited with, unless it is `asyncio.CancelledError` -- are combined in an `ExceptionGroup` or `BaseExceptionGroup` (as appropriate; see their documentation), which is then raised.
- Some exceptions are treated specially: if any task fails with `KeyboardInterrupt` or `SystemExit`, the task group still cancels the remaining tasks and waits for them, but then the initial `KeyboardInterrupt` or `SystemExit` is re-raised instead of `ExceptionGroup` or `BaseExceptionGroup`. Additionally, if the body of the `async with` statement raises `GeneratorExit` and none of the other tasks raise exceptions that would be reported, the `GeneratorExit` is re-raised.

Task groups are careful not to mix up the internal cancellation used to "wake up" their `~object.__aexit__` with cancellation requests for the task in which they are running made by other parties. In particular, when one task group is syntactically nested in another, and both experience an exception in one of their child tasks simultaneously, the inner task group will process its exceptions, and then the outer task group will receive another cancellation and process its own exceptions.

In the case where a task group is cancelled externally and also must raise an `ExceptionGroup`, it will call the parent task's `~asyncio.Task.cancel` method. This ensures that a `asyncio.CancelledError` will be raised at the next `await`, so the cancellation is not lost. Task groups also preserve the cancellation count reported by `asyncio.Task.cancelling`.

<div class="versionchanged">

3.13

Improved handling of simultaneous internal and external cancellations and correct preservation of cancellation counts.

</div>

<div class="versionchanged">

3.15

Addition of the special case for `GeneratorExit`.

</div>

## Sleeping

<div class="function" async="">

sleep(delay, result=None)

Block for *delay* seconds.

If *result* is provided, it is returned to the caller when the coroutine completes.

`sleep()` always suspends the current task, allowing other tasks to run.

Setting the delay to 0 provides an optimized path to allow other tasks to run. This can be used by long-running functions to avoid blocking the event loop for the full duration of the function call.

<div id="asyncio_example_sleep">

Example of coroutine displaying the current date every second for 5 seconds:

    import asyncio
    import datetime as dt

    async def display_date():
        loop = asyncio.get_running_loop()
        end_time = loop.time() + 5.0
        while True:
            print(dt.datetime.now())
            if (loop.time() + 1.0) >= end_time:
                break
            await asyncio.sleep(1)

    asyncio.run(display_date())

</div>

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="versionchanged">

3.13 Raises `ValueError` if *delay* is `~math.nan`.

</div>

</div>

## Running tasks concurrently

<div class="awaitablefunction">

gather(\*aws, return_exceptions=False)

Run `awaitable objects <asyncio-awaitables>` in the *aws* sequence *concurrently*.

If any awaitable in *aws* is a coroutine, it is automatically scheduled as a Task.

If all awaitables are completed successfully, the result is an aggregate list of returned values. The order of result values corresponds to the order of awaitables in *aws*.

If *return_exceptions* is `False` (default), the first raised exception is immediately propagated to the task that awaits on `gather()`. Other awaitables in the *aws* sequence **won't be cancelled** and will continue to run.

If *return_exceptions* is `True`, exceptions are treated the same as successful results, and aggregated in the result list.

If `gather()` is *cancelled*, all submitted awaitables (that have not completed yet) are also *cancelled*.

If any Task or Future from the *aws* sequence is *cancelled*, it is treated as if it raised `CancelledError` -- the `gather()` call is **not** cancelled in this case. This is to prevent the cancellation of one submitted Task/Future to cause other Tasks/Futures to be cancelled.

> [!NOTE]
> A new alternative to create and run tasks concurrently and wait for their completion is `asyncio.TaskGroup`. *TaskGroup* provides stronger safety guarantees than *gather* for scheduling a nesting of subtasks: if a task (or a subtask, a task scheduled by a task) raises an exception, *TaskGroup* will, while *gather* will not, cancel the remaining scheduled tasks.

<div id="asyncio_example_gather">

Example:

    import asyncio

    async def factorial(name, number):
        f = 1
        for i in range(2, number + 1):
            print(f"Task {name}: Compute factorial({number}), currently i={i}...")
            await asyncio.sleep(1)
            f *= i
        print(f"Task {name}: factorial({number}) = {f}")
        return f

    async def main():
        # Schedule three calls *concurrently*:
        L = await asyncio.gather(
            factorial("A", 2),
            factorial("B", 3),
            factorial("C", 4),
        )
        print(L)

    asyncio.run(main())

    # Expected output:
    #
    #     Task A: Compute factorial(2), currently i=2...
    #     Task B: Compute factorial(3), currently i=2...
    #     Task C: Compute factorial(4), currently i=2...
    #     Task A: factorial(2) = 2
    #     Task B: Compute factorial(3), currently i=3...
    #     Task C: Compute factorial(4), currently i=3...
    #     Task B: factorial(3) = 6
    #     Task C: Compute factorial(4), currently i=4...
    #     Task C: factorial(4) = 24
    #     [2, 6, 24]

</div>

> [!NOTE]
> If *return_exceptions* is false, cancelling gather() after it has been marked done won't cancel any submitted awaitables. For instance, gather can be marked done after propagating an exception to the caller, therefore, calling `gather.cancel()` after catching an exception (raised by one of the awaitables) from gather won't cancel any other awaitables.

<div class="versionchanged">

3.7 If the *gather* itself is cancelled, the cancellation is propagated regardless of *return_exceptions*.

</div>

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="deprecated">

3.10 Deprecation warning is emitted if no positional arguments are provided or not all positional arguments are Future-like objects and there is no running event loop.

</div>

</div>

## Eager task factory

<div class="function">

eager_task_factory(loop, coro, \*, name=None, context=None)

A task factory for eager task execution.

When using this factory (via `loop.set_task_factory(asyncio.eager_task_factory) <loop.set_task_factory>`), coroutines begin execution synchronously during `Task` construction. Tasks are only scheduled on the event loop if they block. This can be a performance improvement as the overhead of loop scheduling is avoided for coroutines that complete synchronously.

A common example where this is beneficial is coroutines which employ caching or memoization to avoid actual I/O when possible.

> [!NOTE]
> Immediate execution of the coroutine is a semantic change. If the coroutine returns or raises, the task is never scheduled to the event loop. If the coroutine execution blocks, the task is scheduled to the event loop. This change may introduce behavior changes to existing applications. For example, the application's task execution order is likely to change.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

create_eager_task_factory(custom_task_constructor)

Create an eager task factory, similar to `eager_task_factory`, using the provided *custom_task_constructor* when creating a new task instead of the default `Task`.

*custom_task_constructor* must be a *callable* with the signature matching the signature of `Task.__init__ <Task>`. The callable must return a `asyncio.Task`-compatible object.

This function returns a *callable* intended to be used as a task factory of an event loop via `loop.set_task_factory(factory) <loop.set_task_factory>`).

<div class="versionadded">

3.12

</div>

</div>

## Shielding from cancellation

<div class="awaitablefunction">

shield(arg)

Protect an `awaitable object <asyncio-awaitables>` from being `cancelled <Task.cancel>`.

If *arg* is a coroutine it is automatically scheduled as a Task.

The statement:

    task = asyncio.create_task(something())
    res = await shield(task)

is equivalent to:

    res = await something()

*except* that if the coroutine containing it is cancelled, the Task running in `something()` is not cancelled. From the point of view of `something()`, the cancellation did not happen. Although its caller is still cancelled, so the "await" expression still raises a `CancelledError`.

If `something()` is cancelled by other means (i.e. from within itself) that would also cancel `shield()`.

If it is desired to completely ignore cancellation (not recommended) the `shield()` function should be combined with a try/except clause, as follows:

    task = asyncio.create_task(something())
    try:
        res = await shield(task)
    except CancelledError:
        res = None

> [!IMPORTANT]
> Save a reference to tasks passed to this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn't referenced elsewhere may get garbage collected at any time, even before it's done.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="deprecated">

3.10 Deprecation warning is emitted if *arg* is not Future-like object and there is no running event loop.

</div>

</div>

## Timeouts

<div class="function">

timeout(delay)

Return an `asynchronous context manager <async-context-managers>` that can be used to limit the amount of time spent waiting on something.

*delay* can either be `None`, or a float/int number of seconds to wait. If *delay* is `None`, no time limit will be applied; this can be useful if the delay is unknown when the context manager is created.

In either case, the context manager can be rescheduled after creation using `Timeout.reschedule`.

Example:

    async def main():
        async with asyncio.timeout(10):
            await long_running_task()

If `long_running_task` takes more than 10 seconds to complete, the context manager will cancel the current task and handle the resulting `asyncio.CancelledError` internally, transforming it into a `TimeoutError` which can be caught and handled.

> [!NOTE]
> The `asyncio.timeout` context manager is what transforms the `asyncio.CancelledError` into a `TimeoutError`, which means the `TimeoutError` can only be caught *outside* of the context manager.

Example of catching `TimeoutError`:

    async def main():
        try:
            async with asyncio.timeout(10):
                await long_running_task()
        except TimeoutError:
            print("The long operation timed out, but we've handled it.")

        print("This statement will run regardless.")

The context manager produced by `asyncio.timeout` can be rescheduled to a different deadline and inspected.

<div class="Timeout(when)">

An `asynchronous context manager <async-context-managers>` for cancelling overdue coroutines.

Prefer using `asyncio.timeout` or `asyncio.timeout_at` rather than instantiating `!Timeout` directly.

`when` should be an absolute time at which the context should time out, as measured by the event loop's clock:

- If `when` is `None`, the timeout will never trigger.
- If `when < loop.time()`, the timeout will trigger on the next iteration of the event loop.

> <div class="method">
>
> when() -\> float \| None
>
> Return the current deadline, or `None` if the current deadline is not set.
>
> </div>
>
> <div class="method">
>
> reschedule(when: float \| None)
>
> Reschedule the timeout.
>
> </div>
>
> <div class="method">
>
> expired() -\> bool
>
> Return whether the context manager has exceeded its deadline (expired).
>
> </div>

</div>

Example:

    async def main():
        try:
            # We do not know the timeout when starting, so we pass ``None``.
            async with asyncio.timeout(None) as cm:
                # We know the timeout now, so we reschedule it.
                new_deadline = get_running_loop().time() + 10
                cm.reschedule(new_deadline)

                await long_running_task()
        except TimeoutError:
            pass

        if cm.expired():
            print("Looks like we haven't finished on time.")

Timeout context managers can be safely nested.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

timeout_at(when)

Similar to `asyncio.timeout`, except *when* is the absolute time to stop waiting, or `None`.

Example:

    async def main():
        loop = get_running_loop()
        deadline = loop.time() + 20
        try:
            async with asyncio.timeout_at(deadline):
                await long_running_task()
        except TimeoutError:
            print("The long operation timed out, but we've handled it.")

        print("This statement will run regardless.")

<div class="versionadded">

3.11

</div>

</div>

<div class="function" async="">

wait_for(fut, timeout)

Wait for the *fut* `awaitable <asyncio-awaitables>` to complete with a timeout.

*timeout* can either be `None` or a float or int number of seconds to wait for. If *timeout* is `None`, block until the future completes.

If a timeout occurs, it cancels *fut* and raises `TimeoutError`.

To prevent *fut* from being cancelled, wrap it in `shield`.

The function will wait until the future is actually cancelled, so the total wait time may exceed the *timeout*. If an exception happens during cancellation, it is propagated.

If the wait is cancelled, the future *fut* is also cancelled.

<div id="asyncio_example_waitfor">

Example:

    async def eternity():
        # Sleep for one hour
        await asyncio.sleep(3600)
        print('yay!')

    async def main():
        # Wait for at most 1 second
        try:
            await asyncio.wait_for(eternity(), timeout=1.0)
        except TimeoutError:
            print('timeout!')

    asyncio.run(main())

    # Expected output:
    #
    #     timeout!

</div>

<div class="versionchanged">

3.7 When *fut* is cancelled due to a timeout, `wait_for` waits for *fut* to be cancelled. Previously, it raised `TimeoutError` immediately.

</div>

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="versionchanged">

3.11 Raises `TimeoutError` instead of `asyncio.TimeoutError`.

</div>

<div class="versionchanged">

3.12 Implemented using `asyncio.timeout`, a coroutine passed as *fut* is no longer wrapped in a `Task` when *timeout* is positive.

</div>

</div>

## Waiting primitives

<div class="function" async="">

wait(fs, \*, timeout=None, return_when=ALL_COMPLETED)

Run `~asyncio.Future` and `~asyncio.Task` instances in the *fs* iterable concurrently and block until the condition specified by *return_when*.

The *fs* iterable must not be empty.

Returns two sets of Tasks/Futures: `(done, pending)`.

Usage:

    done, pending = await asyncio.wait(fs)

*timeout* (a float or int), if specified, can be used to control the maximum number of seconds to wait before returning.

Note that this function does not raise `TimeoutError`. Futures or Tasks that aren't done when the timeout occurs are simply returned in the second set.

*return_when* indicates when this function should return. It must be one of the following constants:

<table>
<thead>
<tr>
<th>Constant</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>FIRST_COMPLETED</p>
</div></td>
<td>The function will return when any future finishes or is cancelled.</td>
</tr>
<tr>
<td><div class="data">
<p>FIRST_EXCEPTION</p>
</div></td>
<td>The function will return when any future finishes by raising an exception. If no future raises an exception then it is equivalent to <code class="interpreted-text" role="const">ALL_COMPLETED</code>.</td>
</tr>
<tr>
<td><div class="data">
<p>ALL_COMPLETED</p>
</div></td>
<td>The function will return when all futures finish or are cancelled.</td>
</tr>
</tbody>
</table>

Unlike `~asyncio.wait_for`, `wait()` does not cancel the futures when a timeout occurs.

If `wait()` is cancelled, the futures in *fs* are not cancelled and continue to run.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="versionchanged">

3.11 Passing coroutine objects to `wait()` directly is forbidden.

</div>

<div class="versionchanged">

3.12 Added support for generators yielding tasks.

</div>

</div>

<div class="function">

as_completed(fs, \*, timeout=None)

Run `awaitable objects <asyncio-awaitables>` in the *fs* iterable concurrently. The returned object can be iterated to obtain the results of the awaitables as they finish.

The object returned by `as_completed()` can be iterated as an `asynchronous iterator` or a plain `iterator`. When asynchronous iteration is used, the originally-supplied awaitables are yielded if they are tasks or futures. This makes it easy to correlate previously-scheduled tasks with their results. Example:

    ipv4_connect = create_task(open_connection("127.0.0.1", 80))
    ipv6_connect = create_task(open_connection("::1", 80))
    tasks = [ipv4_connect, ipv6_connect]

    async for earliest_connect in as_completed(tasks):
        # earliest_connect is done. The result can be obtained by
        # awaiting it or calling earliest_connect.result()
        reader, writer = await earliest_connect

        if earliest_connect is ipv6_connect:
            print("IPv6 connection established.")
        else:
            print("IPv4 connection established.")

During asynchronous iteration, implicitly-created tasks will be yielded for supplied awaitables that aren't tasks or futures.

When used as a plain iterator, each iteration yields a new coroutine that returns the result or raises the exception of the next completed awaitable. This pattern is compatible with Python versions older than 3.13:

    ipv4_connect = create_task(open_connection("127.0.0.1", 80))
    ipv6_connect = create_task(open_connection("::1", 80))
    tasks = [ipv4_connect, ipv6_connect]

    for next_connect in as_completed(tasks):
        # next_connect is not one of the original task objects. It must be
        # awaited to obtain the result value or raise the exception of the
        # awaitable that finishes next.
        reader, writer = await next_connect

A `TimeoutError` is raised if the timeout occurs before all awaitables are done. This is raised by the `async for` loop during asynchronous iteration or by the coroutines yielded during plain iteration.

`as_completed()` does not cancel the tasks running the supplied awaitables: if a timeout occurs or the iteration is cancelled, the remaining tasks continue to run.

<div class="versionchanged">

3.10 Removed the *loop* parameter.

</div>

<div class="deprecated">

3.10 Deprecation warning is emitted if not all awaitable objects in the *fs* iterable are Future-like objects and there is no running event loop.

</div>

<div class="versionchanged">

3.12 Added support for generators yielding tasks.

</div>

<div class="versionchanged">

3.13 The result can now be used as either an `asynchronous iterator` or as a plain `iterator` (previously it was only a plain iterator).

</div>

</div>

## Running in threads

<div class="function" async="">

to_thread(func, /, *args,*\*kwargs)

Asynchronously run function *func* in a separate thread.

Any \*args and \*\*kwargs supplied for this function are directly passed to *func*. Also, the current `contextvars.Context` is propagated, allowing context variables from the event loop thread to be accessed in the separate thread.

Return a coroutine that can be awaited to get the eventual result of *func*.

This coroutine function is primarily intended to be used for executing IO-bound functions/methods that would otherwise block the event loop if they were run in the main thread. For example:

    def blocking_io():
        print(f"start blocking_io at {time.strftime('%X')}")
        # Note that time.sleep() can be replaced with any blocking
        # IO-bound operation, such as file operations.
        time.sleep(1)
        print(f"blocking_io complete at {time.strftime('%X')}")

    async def main():
        print(f"started main at {time.strftime('%X')}")

        await asyncio.gather(
            asyncio.to_thread(blocking_io),
            asyncio.sleep(1))

        print(f"finished main at {time.strftime('%X')}")


    asyncio.run(main())

    # Expected output:
    #
    # started main at 19:50:53
    # start blocking_io at 19:50:53
    # blocking_io complete at 19:50:54
    # finished main at 19:50:54

Directly calling `blocking_io()` in any coroutine would block the event loop for its duration, resulting in an additional 1 second of run time. Instead, by using `asyncio.to_thread()`, we can run it in a separate thread without blocking the event loop.

> [!NOTE]
> Due to the `GIL`, `asyncio.to_thread()` can typically only be used to make IO-bound functions non-blocking. However, for extension modules that release the GIL or alternative Python implementations that don't have one, `asyncio.to_thread()` can also be used for CPU-bound functions.

<div class="versionadded">

3.9

</div>

</div>

## Scheduling from other threads

<div class="function">

run_coroutine_threadsafe(coro, loop)

Submit a coroutine to the given event loop. Thread-safe.

Return a `concurrent.futures.Future` to wait for the result from another OS thread.

This function is meant to be called from a different OS thread than the one where the event loop is running. Example:

    def in_thread(loop: asyncio.AbstractEventLoop) -> None:
        # Run some blocking IO
        pathlib.Path("example.txt").write_text("hello world", encoding="utf8")

        # Create a coroutine
        coro = asyncio.sleep(1, result=3)

        # Submit the coroutine to a given loop
        future = asyncio.run_coroutine_threadsafe(coro, loop)

        # Wait for the result with an optional timeout argument
        assert future.result(timeout=2) == 3

    async def amain() -> None:
        # Get the running loop
        loop = asyncio.get_running_loop()

        # Run something in a thread
        await asyncio.to_thread(in_thread, loop)

It's also possible to run the other way around. Example:

    @contextlib.contextmanager
    def loop_in_thread() -> Generator[asyncio.AbstractEventLoop]:
        loop_fut = concurrent.futures.Future[asyncio.AbstractEventLoop]()
        stop_event = asyncio.Event()

        async def main() -> None:
            loop_fut.set_result(asyncio.get_running_loop())
            await stop_event.wait()

        with concurrent.futures.ThreadPoolExecutor(1) as tpe:
            complete_fut = tpe.submit(asyncio.run, main())
            for fut in concurrent.futures.as_completed((loop_fut, complete_fut)):
                if fut is loop_fut:
                    loop = loop_fut.result()
                    try:
                        yield loop
                    finally:
                        loop.call_soon_threadsafe(stop_event.set)
                else:
                    fut.result()

    # Create a loop in another thread
    with loop_in_thread() as loop:
        # Create a coroutine
        coro = asyncio.sleep(1, result=3)

        # Submit the coroutine to a given loop
        future = asyncio.run_coroutine_threadsafe(coro, loop)

        # Wait for the result with an optional timeout argument
        assert future.result(timeout=2) == 3

If an exception is raised in the coroutine, the returned Future will be notified. It can also be used to cancel the task in the event loop:

    try:
        result = future.result(timeout)
    except TimeoutError:
        print('The coroutine took too long, cancelling the task...')
        future.cancel()
    except Exception as exc:
        print(f'The coroutine raised an exception: {exc!r}')
    else:
        print(f'The coroutine returned: {result!r}')

See the `concurrency and multithreading <asyncio-multithreading>` section of the documentation.

Unlike other asyncio functions this function requires the *loop* argument to be passed explicitly.

<div class="versionadded">

3.5.1

</div>

</div>

## Introspection

<div class="function">

current_task(loop=None)

Return the currently running `Task` instance, or `None` if no task is running.

If *loop* is `None` `get_running_loop` is used to get the current loop.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

all_tasks(loop=None)

Return a set of not yet finished `Task` objects run by the loop.

If *loop* is `None`, `get_running_loop` is used for getting current loop.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

iscoroutine(obj)

Return `True` if *obj* is a coroutine object.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.12 Generator-based coroutines are no longer supported, and `False` is returned for them.

</div>

</div>

## Task object

<div class="Task(coro, *, loop=None, name=None, context=None, eager_start=False)">

A `Future-like <Future>` object that runs a Python `coroutine <coroutine>`. Not thread-safe.

Tasks are used to run coroutines in event loops. If a coroutine awaits on a Future, the Task suspends the execution of the coroutine and waits for the completion of the Future. When the Future is *done*, the execution of the wrapped coroutine resumes.

Event loops use cooperative scheduling: an event loop runs one Task at a time. While a Task awaits for the completion of a Future, the event loop runs other Tasks, callbacks, or performs IO operations.

Use the high-level `asyncio.create_task` function to create Tasks, or the low-level `loop.create_task` or `ensure_future` functions. Manual instantiation of Tasks is discouraged.

To cancel a running Task use the `cancel` method. Calling it will cause the Task to throw a `CancelledError` exception into the wrapped coroutine. If a coroutine is awaiting on a future-like object during cancellation, the awaited object will be cancelled.

`cancelled` can be used to check if the Task was cancelled. The method returns `True` if the wrapped coroutine did not suppress the `CancelledError` exception and was actually cancelled.

`asyncio.Task` inherits from `Future` all of its APIs except `Future.set_result` and `Future.set_exception`.

An optional keyword-only *context* argument allows specifying a custom `contextvars.Context` for the *coro* to run in. If no *context* is provided, the Task copies the current context and later runs its coroutine in the copied context.

An optional keyword-only *eager_start* argument allows eagerly starting the execution of the `asyncio.Task` at task creation time. If set to `True` and the event loop is running, the task will start executing the coroutine immediately, until the first time the coroutine blocks. If the coroutine returns or raises without blocking, the task will be finished eagerly and will skip scheduling to the event loop.

Tasks are `generic <generics>` over the return type of their wrapped coroutines.

<div class="versionchanged">

3.7 Added support for the `contextvars` module.

</div>

<div class="versionchanged">

3.8 Added the *name* parameter.

</div>

<div class="deprecated">

3.10 Deprecation warning is emitted if *loop* is not specified and there is no running event loop.

</div>

<div class="versionchanged">

3.11 Added the *context* parameter.

</div>

<div class="versionchanged">

3.12 Added the *eager_start* parameter.

</div>

<div class="method">

done()

Return `True` if the Task is *done*.

A Task is *done* when the wrapped coroutine either returned a value, raised an exception, or the Task was cancelled.

</div>

<div class="method">

result()

Return the result of the Task.

If the Task is *done*, the result of the wrapped coroutine is returned (or if the coroutine raised an exception, that exception is re-raised.)

If the Task has been *cancelled*, this method raises a `CancelledError` exception.

If the Task's result isn't yet available, this method raises an `InvalidStateError` exception.

</div>

<div class="method">

exception()

Return the exception of the Task.

If the wrapped coroutine raised an exception that exception is returned. If the wrapped coroutine returned normally this method returns `None`.

If the Task has been *cancelled*, this method raises a `CancelledError` exception.

If the Task isn't *done* yet, this method raises an `InvalidStateError` exception.

</div>

<div class="method">

add_done_callback(callback, \*, context=None)

Add a callback to be run when the Task is *done*.

This method should only be used in low-level callback-based code.

See the documentation of `Future.add_done_callback` for more details.

</div>

<div class="method">

remove_done_callback(callback)

Remove *callback* from the callbacks list.

This method should only be used in low-level callback-based code.

See the documentation of `Future.remove_done_callback` for more details.

</div>

<div class="method">

get_stack(\*, limit=None)

Return the list of stack frames for this Task.

If the wrapped coroutine is not done, this returns the stack where it is suspended. If the coroutine has completed successfully or was cancelled, this returns an empty list. If the coroutine was terminated by an exception, this returns the list of traceback frames.

The frames are always ordered from oldest to newest.

Only one stack frame is returned for a suspended coroutine.

The optional *limit* argument sets the maximum number of frames to return; by default all available frames are returned. The ordering of the returned list differs depending on whether a stack or a traceback is returned: the newest frames of a stack are returned, but the oldest frames of a traceback are returned. (This matches the behavior of the traceback module.)

</div>

<div class="method">

print_stack(\*, limit=None, file=None)

Print the stack or traceback for this Task.

This produces output similar to that of the traceback module for the frames retrieved by `get_stack`.

The *limit* argument is passed to `get_stack` directly.

The *file* argument is an I/O stream to which the output is written; by default output is written to `sys.stdout`.

</div>

<div class="method">

get_coro()

Return the coroutine object wrapped by the `Task`.

> [!NOTE]
> This will return `None` for Tasks which have already completed eagerly. See the `Eager Task Factory <eager-task-factory>`.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.12

Newly added eager task execution means result may be `None`.

</div>

</div>

<div class="method">

get_context()

Return the `contextvars.Context` object associated with the task.

<div class="versionadded">

3.12

</div>

</div>

<div class="method">

get_name()

Return the name of the Task.

If no name has been explicitly assigned to the Task, the default asyncio Task implementation generates a default name during instantiation.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

set_name(value)

Set the name of the Task.

The *value* argument can be any object, which is then converted to a string.

In the default Task implementation, the name will be visible in the `repr` output of a task object.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

cancel(msg=None)

Request the Task to be cancelled.

If the Task is already *done* or *cancelled*, return `False`, otherwise, return `True`.

The method arranges for a `CancelledError` exception to be thrown into the wrapped coroutine on the next cycle of the event loop.

The coroutine then has a chance to clean up or even deny the request by suppressing the exception with a `try` ... ... `except CancelledError` ... `finally` block. Therefore, unlike `Future.cancel`, `Task.cancel` does not guarantee that the Task will be cancelled, although suppressing cancellation completely is not common and is actively discouraged. Should the coroutine nevertheless decide to suppress the cancellation, it needs to call `Task.uncancel` in addition to catching the exception.

If the Task being cancelled is currently awaiting on a future-like object, that awaited object will also be cancelled. This cancellation propagates down the entire chain of awaited objects.

<div class="versionchanged">

3.9 Added the *msg* parameter.

</div>

<div class="versionchanged">

3.11 The `msg` parameter is propagated from cancelled task to its awaiter.

</div>

<div id="asyncio_example_task_cancel">

The following example illustrates how coroutines can intercept the cancellation request:

    async def cancel_me():
        print('cancel_me(): before sleep')

        try:
            # Wait for 1 hour
            await asyncio.sleep(3600)
        except asyncio.CancelledError:
            print('cancel_me(): cancel sleep')
            raise
        finally:
            print('cancel_me(): after sleep')

    async def main():
        # Create a "cancel_me" Task
        task = asyncio.create_task(cancel_me())

        # Wait for 1 second
        await asyncio.sleep(1)

        task.cancel()
        try:
            await task
        except asyncio.CancelledError:
            print("main(): cancel_me is cancelled now")

    asyncio.run(main())

    # Expected output:
    #
    #     cancel_me(): before sleep
    #     cancel_me(): cancel sleep
    #     cancel_me(): after sleep
    #     main(): cancel_me is cancelled now

</div>

</div>

<div class="method">

cancelled()

Return `True` if the Task is *cancelled*.

The Task is *cancelled* when the cancellation was requested with `cancel` and the wrapped coroutine propagated the `CancelledError` exception thrown into it.

</div>

<div class="method">

uncancel()

Decrement the count of cancellation requests to this Task.

Returns the remaining number of cancellation requests.

Note that once execution of a cancelled task completed, further calls to `uncancel` are ineffective.

<div class="versionadded">

3.11

</div>

This method is used by asyncio's internals and isn't expected to be used by end-user code. In particular, if a Task gets successfully uncancelled, this allows for elements of structured concurrency like `taskgroups` and `asyncio.timeout` to continue running, isolating cancellation to the respective structured block. For example:

    async def make_request_with_timeout():
        try:
            async with asyncio.timeout(1):
                # Structured block affected by the timeout:
                await make_request()
                await make_another_request()
        except TimeoutError:
            log("There was a timeout")
        # Outer code not affected by the timeout:
        await unrelated_code()

While the block with `make_request()` and `make_another_request()` might get cancelled due to the timeout, `unrelated_code()` should continue running even in case of the timeout. This is implemented with `uncancel`. `TaskGroup` context managers use `uncancel` in a similar fashion.

If end-user code is, for some reason, suppressing cancellation by catching `CancelledError`, it needs to call this method to remove the cancellation state.

When this method decrements the cancellation count to zero, the method checks if a previous `cancel` call had arranged for `CancelledError` to be thrown into the task. If it hasn't been thrown yet, that arrangement will be rescinded (by resetting the internal `_must_cancel` flag).

</div>

<div class="versionchanged">

3.13 Changed to rescind pending cancellation requests upon reaching zero.

</div>

<div class="method">

cancelling()

Return the number of pending cancellation requests to this Task, i.e., the number of calls to `cancel` less the number of `uncancel` calls.

Note that if this number is greater than zero but the Task is still executing, `cancelled` will still return `False`. This is because this number can be lowered by calling `uncancel`, which can lead to the task not being cancelled after all if the cancellation requests go down to zero.

This method is used by asyncio's internals and isn't expected to be used by end-user code. See `uncancel` for more details.

<div class="versionadded">

3.11

</div>

</div>

</div>

# asyncio-threading

<div class="currentmodule">

asyncio

</div>

# asyncio and free-threaded Python

asyncio uses an event loop as a scheduler to enable highly efficient concurrency by switching between tasks to allow non-blocking I/O operations. This results in better performance for I/O-bound use cases. It also allows off-loading CPU-bound work to a thread or process pool, but that is still limited by the `global
interpreter lock` in CPython.

However, in `free-threaded Python <freethreading-python-howto>`, the GIL is disabled and Python can run true multi-threaded code. This means that asyncio can now take advantage of multiple CPU cores without the limitations imposed by the GIL.

Since Python 3.14, asyncio has first-class support for free-threaded Python, and the implementation of asyncio is safe to use in a multi-threaded environment.

A single event loop on one core can handle many connections concurrently, but the Python code that runs to handle each one still executes serially. Once requests involve a non-trivial amount of per-request computation, that handling becomes the bottleneck, and a single core can no longer keep up. Combining asyncio with threads is most useful here: by running an event loop per thread, the handling of different requests can run in parallel across multiple CPU cores. It is also useful when you need to run blocking or CPU-bound code from an asyncio application.

<div class="seealso">

[Scaling asyncio on Free-Threaded Python](https://labs.quansight.org/blog/scaling-asyncio-on-free-threaded-python), a blog post by Kumar Aditya which explains the internal changes that make asyncio safe and efficient under free-threaded Python, together with benchmarks of the resulting improvements.

</div>

## Thread safety considerations

While asyncio is designed to be thread-safe in a free-threaded Python environment, there are still some considerations to keep in mind when using asyncio with threads:

1.  **Event loop**: Each thread should have its own event loop which should not be shared across threads. This ensures that the event loop can manage its own tasks and callbacks without interference from other threads.
2.  **Task management**: Tasks and futures created in one thread should not be awaited or manipulated from another thread.
3.  **Thread-safe APIs**: When interacting with asyncio from multiple threads, it's important to use thread-safe APIs provided by asyncio, such as `asyncio.run_coroutine_threadsafe` for submitting coroutines to an event loop from another thread. If you need to call a callback from a different thread, you can use `loop.call_soon_threadsafe` to schedule it safely.
4.  **Synchronization**: The synchronization primitives provided by asyncio (like `asyncio.Lock` and `asyncio.Event`) are not designed to be used across threads. If you need to synchronize between threads, you should use the synchronization primitives from the `threading` module instead.

## Using asyncio with threads

asyncio supports running one event loop per thread, which allows you to take advantage of multiple CPU cores in a free-threaded Python environment. Each thread can run its own event loop, and tasks can be scheduled on those loops independently.

Here's an example of how to use asyncio with threads:

    import asyncio
    import threading

    async def worker(name: str) -> None:
        print(f"Worker {name} starting")
        await asyncio.sleep(1)
        print(f"Worker {name} done")

    def run_loop(name: str) -> None:
        asyncio.run(worker(name))

    threads = [
        threading.Thread(target=run_loop, args=(f"T{i}",))
        for i in range(4)
    ]
    for t in threads:
        t.start()
    for t in threads:
        t.join()

In this example, each thread creates its own event loop with `asyncio.run` and runs a coroutine on it. The threads execute concurrently, and in a free-threaded build they can run on separate CPU cores in parallel.

## Producer/consumer across threads

When a regular (non-asyncio) thread needs to hand work to an asyncio event loop running in another thread, use a thread-safe primitive such as `queue.Queue` rather than `asyncio.Queue`, which is only safe within a single event loop.:

    import asyncio
    import queue
    import threading

    def producer(q: queue.Queue[int]) -> None:
        for i in range(5):
            print(f"Producing {i}")
            q.put(i)
        q.shutdown()

    async def consumer(q: queue.Queue[int]) -> None:
        while True:
            try:
                item = q.get_nowait()
            except queue.Empty:
                await asyncio.sleep(0.1)
                continue
            except queue.ShutDown:
                break
            print(f"Consumed {item}")
            await asyncio.sleep(item)

    q: queue.Queue[int] = queue.Queue()
    consumer_thread = threading.Thread(
        target=lambda: asyncio.run(consumer(q))
    )
    consumer_thread.start()
    producer(q)
    consumer_thread.join()

The producer runs on the main thread while the consumer runs inside an event loop on its own thread, yet they communicate safely through `queue.Queue`. When the queue is empty the consumer sleeps briefly and tries again. When the producer is done it calls `~queue.Queue.shutdown`, which causes subsequent `~queue.Queue.get_nowait` calls to raise `queue.ShutDown` so the consumer can exit cleanly.

# asyncio-tools

<div class="currentmodule">

asyncio

</div>

# Command-line introspection tools

**Source code:** `Lib/asyncio/tools.py`

------------------------------------------------------------------------

The `!asyncio` module can be invoked as a script via `python -m asyncio` to inspect the task graph of another running Python process without modifying it or restarting it. The `!asyncio.tools` submodule implements this interface.

The following commands inspect the process identified by `PID`:

```
$ python -m asyncio pstree [--retries N] PID
$ python -m asyncio ps [--retries N] PID
```

The commands read the target process state without executing any code in it. They are only available on supported platforms and may require permission to inspect another process. See the `permission requirements <permission-requirements>` for details.

<div class="seealso">

`asyncio-graph`  
Programmatic APIs for inspecting the async call graph of a task or future in the current process.

</div>

The command examples below use this program, which creates a task hierarchy suitable for inspection and prints its process ID:

``` python
import asyncio
import os

async def play(track):
    await asyncio.sleep(3600)
    print(f"🎵 Finished: {track}")

async def album(name, tracks):
    async with asyncio.TaskGroup() as tg:
        for track in tracks:
            tg.create_task(play(track), name=track)

async def main():
    print(f"PID: {os.getpid()}")
    async with asyncio.TaskGroup() as tg:
        tg.create_task(
            album("Sundowning", ["TNDNBTG", "Levitate"]),
            name="Sundowning",
        )
        tg.create_task(
            album("TMBTE", ["DYWTYLM", "Aqua Regia"]),
            name="TMBTE",
        )

asyncio.run(main())
```

Run the program in one terminal and leave it running:

```
$ python example.py
PID: 12345
```

Then pass the printed process ID to the commands from another terminal. Thread IDs, task IDs, file paths, and line numbers vary between runs and source layouts.

<div class="versionadded">

3.14

</div>

## Command-line options

<div class="option">

pstree PID

Display task and coroutine relationships as a tree. Each task is shown with its full coroutine stack, nested under the task (if any) that is awaiting it. This subcommand is useful for quickly identifying which branch of a task hierarchy is blocked and where in its coroutine stack execution has paused:

```
$ python -m asyncio pstree 12345
└── (T) Task-1
    └──  main example.py:12
        └──  TaskGroup.__aexit__ Lib/asyncio/taskgroups.py:75
            └──  TaskGroup._aexit Lib/asyncio/taskgroups.py:124
                ├── (T) Sundowning
                │   └──  album example.py:7
                │       └──  TaskGroup.__aexit__ Lib/asyncio/taskgroups.py:75
                │           └──  TaskGroup._aexit Lib/asyncio/taskgroups.py:124
                │               ├── (T) TNDNBTG
                │               │   └──  play example.py:4
                │               │       └──  sleep Lib/asyncio/tasks.py:702
                │               └── (T) Levitate
                │                   └──  play example.py:4
                │                       └──  sleep Lib/asyncio/tasks.py:702
                └── (T) TMBTE
                    └──  album example.py:7
                        └──  TaskGroup.__aexit__ Lib/asyncio/taskgroups.py:75
                            └──  TaskGroup._aexit Lib/asyncio/taskgroups.py:124
                                ├── (T) DYWTYLM
                                │   └──  play example.py:4
                                │       └──  sleep Lib/asyncio/tasks.py:702
                                └── (T) Aqua Regia
                                    └──  play example.py:4
                                        └──  sleep Lib/asyncio/tasks.py:702
```

If the await graph contains a cycle, `pstree` reports an error instead of printing a tree. A cycle in the await graph is unusual and typically indicates a programming error:

```
$ python -m asyncio pstree 12345
ERROR: await-graph contains cycles - cannot print a tree!

cycle: Task-2 → Task-3 → Task-2
```

</div>

<div class="option">

ps PID

Display a flat table of all pending tasks in the process *PID*. Each row shows the event-loop thread ID, task ID and name, coroutine stack, and the awaiting task's stack, name, and ID, if any.

This subcommand prints all tasks regardless of whether the await graph contains cycles:

```
$ python -m asyncio ps 12345
tid        task id              task name            coroutine stack                                    awaiter chain                                      awaiter name    awaiter id
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
18445801   0x10a456060          Task-1               TaskGroup._aexit -> TaskGroup.__aexit__ -> main                                                                       0x0
18445801   0x10a439f60          Sundowning           TaskGroup._aexit -> TaskGroup.__aexit__ -> album   TaskGroup._aexit -> TaskGroup.__aexit__ -> main    Task-1          0x10a456060
18445801   0x10a439d70          TMBTE                TaskGroup._aexit -> TaskGroup.__aexit__ -> album   TaskGroup._aexit -> TaskGroup.__aexit__ -> main    Task-1          0x10a456060
18445801   0x10a2a3a80          TNDNBTG              sleep -> play                                      TaskGroup._aexit -> TaskGroup.__aexit__ -> album   Sundowning      0x10a439f60
18445801   0x10a2a38a0          Levitate             sleep -> play                                      TaskGroup._aexit -> TaskGroup.__aexit__ -> album   Sundowning      0x10a439f60
18445801   0x10a2d7150          DYWTYLM              sleep -> play                                      TaskGroup._aexit -> TaskGroup.__aexit__ -> album   TMBTE           0x10a439d70
18445801   0x10a6bdaa0          Aqua Regia           sleep -> play                                      TaskGroup._aexit -> TaskGroup.__aexit__ -> album   TMBTE           0x10a439d70
```

</div>

<div class="option">

`--retries N`  

Retry failed attempts to inspect the target process up to *N* times. This can help when the target process changes while its state is being read.

<div class="versionadded">

3.15

</div>

</div>

# `!asyncio` --- Asynchronous I/O

<div class="module" data-synopsis="Asynchronous I/O.">

asyncio

</div>

------------------------------------------------------------------------

<div class="sidebar">

**Hello World!**

    import asyncio

    async def main():
        print('Hello ...')
        await asyncio.sleep(1)
        print('... World!')

    asyncio.run(main())

</div>

asyncio is a library to write **concurrent** code using the **async/await** syntax.

asyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc.

asyncio is often a perfect fit for IO-bound and high-level **structured** network code.

<div class="seealso">

`a-conceptual-overview-of-asyncio`  
Explanation of the fundamentals of asyncio.

</div>

asyncio provides a set of **high-level** APIs to:

- `run Python coroutines <coroutine>` concurrently and have full control over their execution;
- perform `network IO and IPC <asyncio-streams>`;
- control `subprocesses <asyncio-subprocess>`;
- distribute tasks via `queues <asyncio-queues>`;
- `synchronize <asyncio-sync>` concurrent code;

For **introspection**, asyncio provides APIs and tools for:

- inspecting the `async call graph <asyncio-graph>` of tasks and futures;
- inspecting tasks in another running Python process with `command-line tools <asyncio-introspection-tools>`;

Additionally, there are **low-level** APIs for *library and framework developers* to:

- create and manage `event loops <asyncio-event-loop>`, which provide asynchronous APIs for `networking <loop_create_server>`, running `subprocesses <loop_subprocess_exec>`, handling `OS signals <loop_add_signal_handler>`, etc;
- implement efficient protocols using `transports <asyncio-transports-protocols>`;
- `bridge <asyncio-futures>` callback-based libraries and code with async/await syntax.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

<div id="asyncio-cli">

**asyncio REPL**

</div>

You can experiment with an `asyncio` concurrent context in the `REPL`:

```
$ python -m asyncio
asyncio REPL ...
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> await asyncio.sleep(10, result='hello')
'hello'
```

This REPL provides limited compatibility with `PYTHON_BASIC_REPL`. It is recommended that the default REPL is used for full functionality and the latest features.

<div class="audit-event">

cpython.run_stdin "" ""

</div>

<div class="versionchanged">

3.12.5 (also 3.11.10, 3.10.15, 3.9.20, and 3.8.20) Emits audit events.

</div>

<div class="versionchanged">

3.13 Uses PyREPL if possible, in which case `PYTHONSTARTUP` is also executed. Emits audit events.

</div>

**Reference**

<div class="toctree" data-caption="High-level APIs" data-maxdepth="1">

asyncio-runner.rst asyncio-task.rst asyncio-stream.rst asyncio-sync.rst asyncio-subprocess.rst asyncio-queue.rst asyncio-exceptions.rst

</div>

<div class="toctree" data-caption="Introspection APIs" data-maxdepth="1">

asyncio-graph.rst asyncio-tools.rst

</div>

<div class="toctree" data-caption="Low-level APIs" data-maxdepth="1">

asyncio-eventloop.rst asyncio-future.rst asyncio-protocol.rst asyncio-platforms.rst asyncio-extending.rst

</div>

<div class="toctree" data-caption="Guides and Tutorials" data-maxdepth="1">

asyncio-api-index.rst asyncio-llapi-index.rst asyncio-dev.rst asyncio-threading.rst

</div>

> [!NOTE]
> The source code for asyncio can be found in `Lib/asyncio/`.

# `!asyncore` --- Asynchronous socket handler

<div class="module" data-synopsis="Removed in 3.12." data-deprecated="">

asyncore

</div>

<div class="deprecated-removed">

3.6 3.12

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.12 <whatsnew312-removed>` after being deprecated in Python 3.6. The removal was decided in `594`.

Applications should use the `asyncio` module instead.

The last version of Python that provided the `!asyncore` module was [Python 3.11](https://docs.python.org/3.11/library/asyncore.html).

# `!atexit` --- Exit handlers

<div class="module" data-synopsis="Register and execute cleanup functions.">

atexit

</div>

------------------------------------------------------------------------

The `!atexit` module defines functions to register and unregister `exit handlers`: functions that are automatically executed "at exit", that is, upon normal program termination (for instance, if `sys.exit` is called or the main module's execution completes) or, more generally, upon `interpreter shutdown`.

At exit, all registered exit handlers are called in the *reverse* order in which they were registered. If you register `A`, `B`, and `C`, at interpreter shutdown time they will be run in the order `C`, `B`, `A`. The assumption is that lower level modules will normally be imported before higher level modules and thus must be cleaned up later.

If an exception is raised during execution of an exit handler, a traceback is printed (unless `SystemExit` is raised) and the exception information is saved. After all exit handlers have had a chance to run, the last exception to be raised is re-raised.

In programs that use multiple interpreters, each interpreter has its own stack of exit handlers, which are executed when the interpreter shuts down (for example, with `concurrent.interpreters.Interpreter.close` or the C API `Py_EndInterpreter`). Registration functions in this module only affect the interpreter they are called from.

**Note:** Exit handlers are not called when the program is killed by a signal not handled by Python, when a Python fatal internal error is detected, or when `os._exit` is called.

**Note:** The effect of registering or unregistering functions from within a cleanup function is undefined.

> [!WARNING]
> When writing exit handlers, especially in C API extensions, keep in mind that other exit handlers may still run arbitrary Python code after you clean up. Such code should succeed or fail with an exception, rather than crash.

<div class="versionchanged">

3.12 Attempts to start a new thread or `os.fork` a new process in an exit handler now leads to `RuntimeError`. Previously, this could cause race conditions between the main Python runtime thread freeing thread states while internal `threading` routines or the new process try to use that state, which could lead to crashes rather than clean shutdown.

</div>

<div class="versionchanged">

3.7 When used with subinterpreters, registered functions are local to the interpreter they were registered in.

</div>

<div class="function">

register(func, *args,*\*kwargs)

Register *func* as an exit handler. Any optional arguments that are to be passed to *func* must be passed as arguments to `register`. It is possible to register the same function and arguments more than once.

This function returns *func*, which makes it possible to use it as a decorator.

</div>

<div class="function">

unregister(func)

Remove *func* from the list of exit handlers. `unregister` silently does nothing if *func* was not previously registered. If *func* has been registered more than once, every occurrence of that function in the `!atexit` call stack will be removed. Equality comparisons (`==`) are used internally during unregistration, so function references do not need to have matching identities.

</div>

<div class="seealso">

Module `readline`  
Useful example of `!atexit` to read and write `readline` history files.

</div>

## `!atexit` Example

The following simple example demonstrates how a module can initialize a counter from a file when it is imported and save the counter's updated value automatically when the program terminates without relying on the application making an explicit call into this module at termination. :

    try:
        with open('counterfile') as infile:
            _count = int(infile.read())
    except FileNotFoundError:
        _count = 0

    def incrcounter(n):
        global _count
        _count = _count + n

    def savecounter():
        with open('counterfile', 'w') as outfile:
            outfile.write('%d' % _count)

    import atexit

    atexit.register(savecounter)

Positional and keyword arguments may also be passed to `register` to be passed along to the registered function when it is called:

    def goodbye(name, adjective):
        print('Goodbye %s, it was %s to meet you.' % (name, adjective))

    import atexit

    atexit.register(goodbye, 'Donny', 'nice')
    # or:
    atexit.register(goodbye, adjective='nice', name='Donny')

Usage as a `decorator`:

    import atexit

    @atexit.register
    def goodbye():
        print('You are now leaving the Python sector.')

This only works with functions that can be called without arguments.

# `!audioop` --- Manipulate raw audio data

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

audioop

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!audioop` module was [Python 3.12](https://docs.python.org/3.12/library/audioop.html).

# audit_events

<div id="audit-events">

<div class="index">

single: audit events

</div>

</div>

# Audit events table

This table contains all events raised by `sys.audit` or `PySys_Audit` calls throughout the CPython runtime and the standard library. These calls were added in 3.8 or later (see `578`).

See `sys.addaudithook` and `PySys_AddAuditHook` for information on handling these events.

<div class="impl-detail">

This table is generated from the CPython documentation, and may not represent events raised by other implementations. See your runtime specific documentation for actual events raised.

</div>

<div class="audit-event-table">

</div>

The following events are raised internally and do not correspond to any public API of CPython:

| Audit event | Arguments |
|----|----|
| <span id="winapi.createfile">winapi.CreateFile</span> | `file_name`, `desired_access`, `share_mode`, `creation_disposition`, `flags_and_attributes` |
| <span id="winapi.createjunction">winapi.CreateJunction</span> | `src_path`, `dst_path` |
| <span id="winapi.createnamedpipe">winapi.CreateNamedPipe</span> | `name`, `open_mode`, `pipe_mode` |
| <span id="winapi.createpipe">winapi.CreatePipe</span> |  |
| <span id="winapi.createprocess">winapi.CreateProcess</span> | `application_name`, `command_line`, `current_directory` |
| <span id="winapi.openprocess">winapi.OpenProcess</span> | `process_id`, `desired_access` |
| <span id="winapi.terminateprocess">winapi.TerminateProcess</span> | `handle`, `exit_code` |
| <span id="posixsubprocess.fork_exec">posixsubprocess.fork_exec</span> | `exec_list`, `args`, `env` |
| ctypes.PyObj_FromPtr | `obj` |

<div class="versionadded">

3.14 The `_posixsubprocess.fork_exec` internal audit event.

</div>

# `!base64` --- Base16, Base32, Base64, Base85 Data Encodings

<div class="module" data-synopsis="RFC 4648: Base16, Base32, Base64 Data Encodings;
Base85 and Ascii85">

base64

</div>

**Source code:** `Lib/base64.py`

<div class="index">

pair: base64; encoding single: MIME; base64 encoding

</div>

------------------------------------------------------------------------

This module provides functions for encoding binary data to printable ASCII characters and decoding such encodings back to binary data. This includes the `encodings specified in <base64-rfc-4648>` `4648` (Base64, Base32 and Base16), the `Base85 encoding
<base64-base-85>` specified in [PDF 2.0](https://pdfa.org/resource/iso-32000-2/), and non-standard variants of Base85 used elsewhere.

There are two interfaces provided by this module. The modern interface supports encoding `bytes-like objects <bytes-like object>` to ASCII `bytes`, and decoding `bytes-like objects <bytes-like object>` or strings containing ASCII to `bytes`. Both base-64 alphabets defined in `4648` (normal, and URL- and filesystem-safe) are supported.

The `legacy interface <base64-legacy>` does not support decoding from strings, but it does provide functions for encoding and decoding to and from `file objects
<file object>`. It only supports the Base64 standard alphabet, and it adds newlines every 76 characters as per `2045`. Note that if you are looking for `2045` support you probably want to be looking at the `email` package instead.

<div class="versionchanged">

3.3 ASCII-only Unicode strings are now accepted by the decoding functions of the modern interface.

</div>

<div class="versionchanged">

3.4 Any `bytes-like objects <bytes-like object>` are now accepted by all encoding and decoding functions in this module. Ascii85/Base85 support added.

</div>

## RFC 4648 Encodings

The `4648` encodings are suitable for encoding binary data so that it can be safely sent by email, used as parts of URLs, or included as part of an HTTP POST request.

<div class="function">

b64encode(s, altchars=None, \*, padded=True, wrapcol=0)

Encode the `bytes-like object` *s* using Base64 and return the encoded `bytes`.

Optional *altchars* must be a `bytes-like object` of length 2 which specifies an alternative alphabet for the `+` and `/` characters. This allows an application to e.g. generate URL or filesystem safe Base64 strings. The default is `None`, for which the standard Base64 alphabet is used.

If *padded* is true (default), pad the encoded data with the '=' character to a size multiple of 4. If *padded* is false, do not add the pad characters.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not insert any newlines.

<div class="versionchanged">

3.15 Added the *padded* and *wrapcol* parameters.

</div>

</div>

<div class="function">

b64decode(s, altchars=None, validate=False, *, padded=True, canonical=False) b64decode(s, altchars=None, validate=True,*, ignorechars, padded=True, canonical=False)

Decode the Base64 encoded `bytes-like object` or ASCII string *s* and return the decoded `bytes`.

Optional *altchars* must be a `bytes-like object` or ASCII string of length 2 which specifies the alternative alphabet used instead of the `+` and `/` characters.

If *padded* is true, the last group of 4 base 64 alphabet characters must be padded with the '=' character. If *padded* is false, padding is neither required nor recognized: the '=' character is not treated as padding but as a non-alphabet character, which means it is silently discarded when *validate* is false, or causes an `~binascii.Error` when *validate* is true unless b'=' is included in *ignorechars*.

A `binascii.Error` exception is raised if *s* is incorrectly padded.

If *ignorechars* is specified, it should be a `bytes-like object` containing characters to ignore from the input when *validate* is true. If *ignorechars* contains the pad character `'='`, the pad characters presented before the end of the encoded data and the excess pad characters will be ignored. The default value of *validate* is `True` if *ignorechars* is specified, `False` otherwise.

If *validate* is false, characters that are neither in the normal base-64 alphabet nor (if *ignorechars* is not specified) the alternative alphabet are discarded prior to the padding check, but the `+` and `/` characters keep their meaning if they are not in *altchars* (they will be discarded in future Python versions).

If *validate* is true, these non-alphabet characters in the input result in a `binascii.Error`.

If *canonical* is true, non-zero padding bits are rejected. See `binascii.a2b_base64` for details.

For more information about the strict base64 check, see `binascii.a2b_base64`

<div class="versionchanged">

3.15 Added the *canonical*, *ignorechars*, and *padded* parameters.

</div>

<div class="deprecated">

3.15 Accepting the `+` and `/` characters with an alternative alphabet is now deprecated.

</div>

</div>

<div class="function">

standard_b64encode(s)

Encode `bytes-like object` *s* using the standard Base64 alphabet and return the encoded `bytes`.

</div>

<div class="function">

standard_b64decode(s)

Decode `bytes-like object` or ASCII string *s* using the standard Base64 alphabet and return the decoded `bytes`.

</div>

<div class="function">

urlsafe_b64encode(s, \*, padded=True)

Encode `bytes-like object` *s* using the URL- and filesystem-safe alphabet, which substitutes `-` instead of `+` and `_` instead of `/` in the standard Base64 alphabet, and return the encoded `bytes`. The result can still contain `=` if *padded* is true (default).

<div class="versionchanged">

3.15 Added the *padded* parameter.

</div>

</div>

<div class="function">

urlsafe_b64decode(s, \*, padded=False)

Decode `bytes-like object` or ASCII string *s* using the URL- and filesystem-safe alphabet, which substitutes `-` instead of `+` and `_` instead of `/` in the standard Base64 alphabet, and return the decoded `bytes`.

<div class="versionchanged">

3.15 Added the *padded* parameter. Padding of input is no longer required by default.

</div>

<div class="deprecated">

3.15 Accepting the `+` and `/` characters is now deprecated.

</div>

</div>

<div class="function">

b32encode(s, \*, padded=True, wrapcol=0)

Encode the `bytes-like object` *s* using Base32 and return the encoded `bytes`.

If *padded* is true (default), pad the encoded data with the '=' character to a size multiple of 8. If *padded* is false, do not add the pad characters.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not add any newlines.

<div class="versionchanged">

3.15 Added the *padded* and *wrapcol* parameters.

</div>

</div>

<div class="function">

b32decode(s, casefold=False, map01=None, \*, padded=True, ignorechars=b'', canonical=False)

Decode the Base32 encoded `bytes-like object` or ASCII string *s* and return the decoded `bytes`.

Optional *casefold* is a flag specifying whether a lowercase alphabet is acceptable as input. For security purposes, the default is `False`.

`4648` allows for optional mapping of the digit 0 (zero) to the letter O (oh), and for optional mapping of the digit 1 (one) to either the letter I (eye) or letter L (el). The optional argument *map01* when not `None`, specifies which letter the digit 1 should be mapped to (when *map01* is not `None`, the digit 0 is always mapped to the letter O). For security purposes the default is `None`, so that 0 and 1 are not allowed in the input.

If *padded* is true, the last group of 8 base 32 alphabet characters must be padded with the '=' character. If *padded* is false, padding is neither required nor recognized: the '=' character is not treated as padding but as a non-alphabet character, which means it raises an `~binascii.Error` unless b'=' is included in *ignorechars*.

*ignorechars* should be a `bytes-like object` containing characters to ignore from the input.

If *canonical* is true, non-zero padding bits are rejected. See `binascii.a2b_base32` for details.

A `binascii.Error` is raised if *s* is incorrectly padded or if there are non-alphabet characters present in the input.

<div class="versionchanged">

3.15 Added the *canonical*, *ignorechars*, and *padded* parameters.

</div>

</div>

<div class="function">

b32hexencode(s, \*, padded=True, wrapcol=0)

Similar to `b32encode` but uses the Extended Hex Alphabet, as defined in `4648`.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.15 Added the *padded* and *wrapcol* parameters.

</div>

</div>

<div class="function">

b32hexdecode(s, casefold=False, \*, padded=True, ignorechars=b'', canonical=False)

Similar to `b32decode` but uses the Extended Hex Alphabet, as defined in `4648`.

This version does not allow the digit 0 (zero) to the letter O (oh) and digit 1 (one) to either the letter I (eye) or letter L (el) mappings, all these characters are included in the Extended Hex Alphabet and are not interchangeable.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.15 Added the *canonical*, *ignorechars*, and *padded* parameters.

</div>

</div>

<div class="function">

b16encode(s, \*, wrapcol=0)

Encode the `bytes-like object` *s* using Base16 and return the encoded `bytes`.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not add any newlines.

<div class="versionchanged">

3.15 Added the *wrapcol* parameter.

</div>

</div>

<div class="function">

b16decode(s, casefold=False, \*, ignorechars=b'')

Decode the Base16 encoded `bytes-like object` or ASCII string *s* and return the decoded `bytes`.

Optional *casefold* is a flag specifying whether a lowercase alphabet is acceptable as input. For security purposes, the default is `False`.

*ignorechars* should be a `bytes-like object` containing characters to ignore from the input.

A `binascii.Error` is raised if *s* is incorrectly padded or if there are non-alphabet characters present in the input.

<div class="versionchanged">

3.15 Added the *ignorechars* parameter.

</div>

</div>

## Base85 Encodings

Base85 encoding is a family of algorithms which represent four bytes using five ASCII characters. Originally implemented in the Unix `btoa(1)` utility, a version of it was later adopted by Adobe in the PostScript language and is standardized in PDF 2.0 (ISO 32000-2). This version, in both its `btoa` and PDF variants, is implemented by `a85encode`.

A separate version, using a different output character set, was defined as an April Fool's joke in `1924` but is now used by Git and other software. This version is implemented by `b85encode`.

Finally, a third version, using yet another output character set designed for safe inclusion in programming language strings, is defined by ZeroMQ and implemented here by `z85encode`.

The functions present in this module differ in how they handle the following:

- Whether to include and expect enclosing `<~` and `~>` markers.
- Whether to fold the input into multiple lines.
- The set of ASCII characters used for encoding.
- Compact encodings of sequences of spaces and null bytes.
- The encoding of zero-padding bytes applied to the input.

Refer to the documentation of the individual functions for more information.

<div class="function">

a85encode(b, \*, foldspaces=False, wrapcol=0, pad=False, adobe=False)

Encode the `bytes-like object` *b* using Ascii85 and return the encoded `bytes`.

*foldspaces* is an optional flag that uses the special short sequence 'y' instead of 4 consecutive spaces (ASCII 0x20) as supported by 'btoa'. This feature is not supported by the standard encoding used in PDF.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not insert any newlines.

*pad* controls whether zero-padding applied to the end of the input is fully retained in the output encoding, as done by `btoa`, producing an exact multiple of 5 bytes of output. This is not part of the standard encoding used in PDF, as it does not preserve the length of the data.

*adobe* controls whether the encoded byte sequence is framed with `<~` and `~>`, as in a PostScript base-85 string literal. Note that while ASCII85Decode streams in PDF documents *must* be terminated with `~>`, they *must not* use a leading `<~`.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

a85decode(b, \*, foldspaces=False, adobe=False, ignorechars=b' tnrv', canonical=False)

Decode the Ascii85 encoded `bytes-like object` or ASCII string *b* and return the decoded `bytes`.

*foldspaces* is a flag that specifies whether the 'y' short sequence should be accepted as shorthand for 4 consecutive spaces (ASCII 0x20). This feature is not supported by the standard Ascii85 encoding used in PDF and PostScript.

*adobe* controls whether the `<~` and `~>` markers are present. While the leading `<~` is not required, the input must end with `~>`, or a `ValueError` is raised.

*ignorechars* should be a `bytes-like object` containing characters to ignore from the input. This should only contain whitespace characters, and by default contains all whitespace characters in ASCII.

If *canonical* is true, non-canonical encodings are rejected. See `binascii.a2b_ascii85` for details.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.15 Added the *canonical* parameter. Single-character final groups are now always rejected as encoding violations.

</div>

</div>

<div class="function">

b85encode(b, pad=False, \*, wrapcol=0)

Encode the `bytes-like object` *b* using base85 (as used in e.g. git-style binary diffs) and return the encoded `bytes`.

The input is padded with `b'\0'` so its length is a multiple of 4 bytes before encoding. If *pad* is true, all the resulting characters are retained in the output, which will always be a multiple of 5 bytes, and thus the length of the data may not be preserved on decoding.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not add any newlines.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.15 Added the *wrapcol* parameter.

</div>

</div>

<div class="function">

b85decode(b, \*, ignorechars=b'', canonical=False)

Decode the base85-encoded `bytes-like object` or ASCII string *b* and return the decoded `bytes`.

*ignorechars* should be a `bytes-like object` containing characters to ignore from the input.

If *canonical* is true, non-canonical encodings are rejected. See `binascii.a2b_base85` for details.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.15 Added the *canonical* and *ignorechars* parameters. Single-character final groups are now always rejected as encoding violations.

</div>

</div>

<div class="function">

z85encode(s, pad=False, \*, wrapcol=0)

Encode the `bytes-like object` *s* using Z85 (as used in ZeroMQ) and return the encoded `bytes`.

The input is padded with `b'\0'` so its length is a multiple of 4 bytes before encoding. If *pad* is true, all the resulting characters are retained in the output, which will always be a multiple of 5 bytes, as required by the ZeroMQ standard.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not add any newlines.

<div class="versionadded">

3.13

</div>

<div class="versionchanged">

3.15 The *pad* parameter was added.

</div>

<div class="versionchanged">

3.15 Added the *wrapcol* parameter.

</div>

</div>

<div class="function">

z85decode(s, \*, ignorechars=b'', canonical=False)

Decode the Z85-encoded `bytes-like object` or ASCII string *s* and return the decoded `bytes`.

*ignorechars* should be a `bytes-like object` containing characters to ignore from the input.

If *canonical* is true, non-canonical encodings are rejected. See `binascii.a2b_base85` for details.

<div class="versionadded">

3.13

</div>

<div class="versionchanged">

3.15 Added the *canonical* and *ignorechars* parameters. Single-character final groups are now always rejected as encoding violations.

</div>

</div>

## Legacy Interface

<div class="function">

decode(input, output)

Decode the contents of the binary *input* file and write the resulting binary data to the *output* file. *input* and *output* must be `file objects
<file object>`. *input* will be read until `input.readline()` returns an empty bytes object.

</div>

<div class="function">

decodebytes(s)

Decode the `bytes-like object` *s*, which must contain one or more lines of base64 encoded data, and return the decoded `bytes`.

<div class="versionadded">

3.1

</div>

</div>

<div class="function">

encode(input, output)

Encode the contents of the binary *input* file and write the resulting base64 encoded data to the *output* file. *input* and *output* must be `file
objects <file object>`. *input* will be read until `input.read()` returns an empty bytes object. `encode` inserts a newline character (`b'\n'`) after every 76 bytes of the output, as well as ensuring that the output always ends with a newline, as per `2045` (MIME).

</div>

<div class="function">

encodebytes(s)

Encode the `bytes-like object` *s*, which can contain arbitrary binary data, and return `bytes` containing the base64-encoded data, with newlines (`b'\n'`) inserted after every 76 bytes of output, and ensuring that there is a trailing newline, as per `2045` (MIME).

<div class="versionadded">

3.1

</div>

</div>

An example usage of the module:

> \>\>\> import base64 \>\>\> encoded = base64.b64encode(b'data to be encoded') \>\>\> encoded b'ZGF0YSB0byBiZSBlbmNvZGVk' \>\>\> data = base64.b64decode(encoded) \>\>\> data b'data to be encoded'

## Security Considerations

A new security considerations section was added to `4648` (section 12); it's recommended to review the security section for any code deployed to production.

<div class="seealso">

Module `binascii`  
Support module containing ASCII-to-binary and binary-to-ASCII conversions.

`1521` - MIME (Multipurpose Internet Mail Extensions) Part One: Mechanisms for Specifying and Describing the Format of Internet Message Bodies  
Section 5.2, "Base64 Content-Transfer-Encoding," provides the definition of the base64 encoding.

[ISO 32000-2 Portable document format - Part 2: PDF 2.0](https://pdfa.org/resource/iso-32000-2/)  
Section 7.4.3, "ASCII85Decode Filter," provides the definition of the Ascii85 encoding used in PDF and PostScript, including the output character set and the details of data length preservation using zero-padding and partial output groups.

[ZeroMQ RFC 32/Z85](https://rfc.zeromq.org/spec/32/)  
The "Formal Specification" section provides the character set used in Z85.

</div>

# `!bdb` --- Debugger framework

<div class="module" data-synopsis="Debugger framework.">

bdb

</div>

**Source code:** `Lib/bdb.py`

------------------------------------------------------------------------

The `!bdb` module handles basic debugger functions, like setting breakpoints or managing execution via the debugger.

The following exception is defined:

<div class="exception">

BdbQuit

Exception raised by the `Bdb` class for quitting the debugger.

<div class="versionchanged">

3.16 `BdbQuit` now inherits from `BaseException` instead of `Exception`.

</div>

</div>

The `!bdb` module also defines two classes:

<div class="Breakpoint(self, file, line, temporary=False, cond=None, funcname=None)">

This class implements temporary breakpoints, ignore counts, disabling and (re-)enabling, and conditionals.

Breakpoints are indexed by number through a list called `bpbynumber` and by `(file, line)` pairs through `bplist`. The former points to a single instance of class `Breakpoint`. The latter points to a list of such instances since there may be more than one breakpoint per line.

When creating a breakpoint, its associated `file name <file>` should be in canonical form. If a `funcname` is defined, a breakpoint `hit <hits>` will be counted when the first line of that function is executed. A `conditional <cond>` breakpoint always counts a `hit <hits>`.

`Breakpoint` instances have the following methods:

<div class="method">

deleteMe()

Delete the breakpoint from the list associated to a file/line. If it is the last breakpoint in that position, it also deletes the entry for the file/line.

</div>

<div class="method">

enable()

Mark the breakpoint as enabled.

</div>

<div class="method">

disable()

Mark the breakpoint as disabled.

</div>

<div class="method">

bpformat()

Return a string with all the information about the breakpoint, nicely formatted:

- Breakpoint number.
- Temporary status (del or keep).
- File/line position.
- Break condition.
- Number of times to ignore.
- Number of times hit.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

bpprint(out=None)

Print the output of `bpformat` to the file *out*, or if it is `None`, to standard output.

</div>

`Breakpoint` instances have the following attributes:

<div class="attribute">

file

File name of the `Breakpoint`.

</div>

<div class="attribute">

line

Line number of the `Breakpoint` within `file`.

</div>

<div class="attribute">

temporary

`True` if a `Breakpoint` at (file, line) is temporary.

</div>

<div class="attribute">

cond

Condition for evaluating a `Breakpoint` at (file, line).

</div>

<div class="attribute">

funcname

Function name that defines whether a `Breakpoint` is hit upon entering the function.

</div>

<div class="attribute">

enabled

`True` if `Breakpoint` is enabled.

</div>

<div class="attribute">

bpbynumber

Numeric index for a single instance of a `Breakpoint`.

</div>

<div class="attribute">

bplist

Dictionary of `Breakpoint` instances indexed by (`file`, `line`) tuples.

</div>

<div class="attribute">

ignore

Number of times to ignore a `Breakpoint`.

</div>

<div class="attribute">

hits

Count of the number of times a `Breakpoint` has been hit.

</div>

</div>

<div class="Bdb(skip=None, backend='settrace')">

The `Bdb` class acts as a generic Python debugger base class.

This class takes care of the details of the trace facility; a derived class should implement user interaction. The standard debugger class (`pdb.Pdb`) is an example.

The *skip* argument, if given, must be an iterable of glob-style module name patterns. The debugger will not step into frames that originate in a module that matches one of these patterns. Whether a frame is considered to originate in a certain module is determined by the `__name__` in the frame globals.

The *backend* argument specifies the backend to use for `Bdb`. It can be either `'settrace'` or `'monitoring'`. `'settrace'` uses `sys.settrace` which has the best backward compatibility. The `'monitoring'` backend uses the new `sys.monitoring` that was introduced in Python 3.12, which can be much more efficient because it can disable unused events. We are trying to keep the exact interfaces for both backends, but there are some differences. The debugger developers are encouraged to use the `'monitoring'` backend to achieve better performance.

<div class="versionchanged">

3.1 Added the *skip* parameter.

</div>

<div class="versionchanged">

3.14 Added the *backend* parameter.

</div>

The following methods of `Bdb` normally don't need to be overridden.

<div class="method">

canonic(filename)

Return canonical form of *filename*.

For real file names, the canonical form is an operating-system-dependent, `case-normalized <os.path.normcase>` `absolute path
<os.path.abspath>`. A *filename* with angle brackets, such as `"<stdin>"` generated in interactive mode, is returned unchanged.

</div>

<div class="method">

start_trace(self)

Start tracing. For `'settrace'` backend, this method is equivalent to `sys.settrace(self.trace_dispatch)`

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

stop_trace(self)

Stop tracing. For `'settrace'` backend, this method is equivalent to `sys.settrace(None)`

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

reset()

Set the `!botframe`, `!stopframe`, `!returnframe` and `quitting <Bdb.set_quit>` attributes with values ready to start debugging.

</div>

<div class="method">

trace_dispatch(frame, event, arg)

This function is installed as the trace function of debugged frames. Its return value is the new trace function (in most cases, that is, itself).

The default implementation decides how to dispatch a frame, depending on the type of event (passed as a string) that is about to be executed. *event* can be one of the following:

- `"line"`: A new line of code is going to be executed.
- `"call"`: A function is about to be called, or another code block entered.
- `"return"`: A function or other code block is about to return.
- `"exception"`: An exception has occurred.

For all the events, specialized functions (see below) are called.

The *arg* parameter depends on the previous event.

See the documentation for `sys.settrace` for more information on the trace function. For more information on code and frame objects, refer to `types`.

</div>

<div class="method">

dispatch_line(frame)

If the debugger should stop on the current line, invoke the `user_line` method (which should be overridden in subclasses). Raise a `BdbQuit` exception if the `quitting  <Bdb.set_quit>` flag is set (which can be set from `user_line`). Return a reference to the `trace_dispatch` method for further tracing in that scope.

</div>

<div class="method">

dispatch_call(frame, arg)

If the debugger should stop on this function call, invoke the `user_call` method (which should be overridden in subclasses). Raise a `BdbQuit` exception if the `quitting  <Bdb.set_quit>` flag is set (which can be set from `user_call`). Return a reference to the `trace_dispatch` method for further tracing in that scope.

</div>

<div class="method">

dispatch_return(frame, arg)

If the debugger should stop on this function return, invoke the `user_return` method (which should be overridden in subclasses). Raise a `BdbQuit` exception if the `quitting  <Bdb.set_quit>` flag is set (which can be set from `user_return`). Return a reference to the `trace_dispatch` method for further tracing in that scope.

</div>

<div class="method">

dispatch_exception(frame, arg)

If the debugger should stop at this exception, invokes the `user_exception` method (which should be overridden in subclasses). Raise a `BdbQuit` exception if the `quitting  <Bdb.set_quit>` flag is set (which can be set from `user_exception`). Return a reference to the `trace_dispatch` method for further tracing in that scope.

</div>

Normally derived classes don't override the following methods, but they may if they want to redefine the definition of stopping and breakpoints.

<div class="method">

is_skipped_module(module_name)

Return `True` if *module_name* matches any skip pattern.

</div>

<div class="method">

stop_here(frame)

Return `True` if *frame* is below the starting frame in the stack.

</div>

<div class="method">

break_here(frame)

Return `True` if there is an effective breakpoint for this line.

Check whether a line or function breakpoint exists and is in effect. Delete temporary breakpoints based on information from `effective`.

</div>

<div class="method">

break_anywhere(frame)

Return `True` if any breakpoint exists for *frame*'s filename.

</div>

Derived classes should override these methods to gain control over debugger operation.

<div class="method">

user_call(frame, argument_list)

Called from `dispatch_call` if a break might stop inside the called function.

*argument_list* is not used anymore and will always be `None`. The argument is kept for backwards compatibility.

</div>

<div class="method">

user_line(frame)

Called from `dispatch_line` when either `stop_here` or `break_here` returns `True`.

</div>

<div class="method">

user_return(frame, return_value)

Called from `dispatch_return` when `stop_here` returns `True`.

</div>

<div class="method">

user_exception(frame, exc_info)

Called from `dispatch_exception` when `stop_here` returns `True`.

</div>

<div class="method">

do_clear(arg)

Handle how a breakpoint must be removed when it is a temporary one.

This method must be implemented by derived classes.

</div>

Derived classes and clients can call the following methods to affect the stepping state.

<div class="method">

set_step()

Stop after one line of code.

</div>

<div class="method">

set_next(frame)

Stop on the next line in or below the given frame.

</div>

<div class="method">

set_return(frame)

Stop when returning from the given frame.

</div>

<div class="method">

set_until(frame, lineno=None)

Stop when the line with the *lineno* greater than the current one is reached or when returning from current frame.

</div>

<div class="method">

set_trace(\[frame\])

Start debugging from *frame*. If *frame* is not specified, debugging starts from caller's frame.

<div class="versionchanged">

3.13 `set_trace` will enter the debugger immediately, rather than on the next line of code to be executed.

</div>

</div>

<div class="method">

set_continue()

Stop only at breakpoints or when finished. If there are no breakpoints, set the system trace function to `None`.

</div>

<div class="method">

set_quit()

<div class="index">

single: quitting (bdb.Bdb attribute)

</div>

Set the `!quitting` attribute to `True`. This raises `BdbQuit` in the next call to one of the `!dispatch_\*` methods.

</div>

Derived classes and clients can call the following methods to manipulate breakpoints. These methods return a string containing an error message if something went wrong, or `None` if all is well.

<div class="method">

set_break(filename, lineno, temporary=False, cond=None, funcname=None)

Set a new breakpoint. If the *lineno* line doesn't exist for the *filename* passed as argument, return an error message. The *filename* should be in canonical form, as described in the `canonic` method.

</div>

<div class="method">

clear_break(filename, lineno)

Delete the breakpoints in *filename* and *lineno*. If none were set, return an error message.

</div>

<div class="method">

clear_bpbynumber(arg)

Delete the breakpoint which has the index *arg* in the `Breakpoint.bpbynumber`. If *arg* is not numeric or out of range, return an error message.

</div>

<div class="method">

clear_all_file_breaks(filename)

Delete all breakpoints in *filename*. If none were set, return an error message.

</div>

<div class="method">

clear_all_breaks()

Delete all existing breakpoints. If none were set, return an error message.

</div>

<div class="method">

get_bpbynumber(arg)

Return a breakpoint specified by the given number. If *arg* is a string, it will be converted to a number. If *arg* is a non-numeric string, if the given breakpoint never existed or has been deleted, a `ValueError` is raised.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

get_break(filename, lineno)

Return `True` if there is a breakpoint for *lineno* in *filename*.

</div>

<div class="method">

get_breaks(filename, lineno)

Return all breakpoints for *lineno* in *filename*, or an empty list if none are set.

</div>

<div class="method">

get_file_breaks(filename)

Return all breakpoints in *filename*, or an empty list if none are set.

</div>

<div class="method">

get_all_breaks()

Return all breakpoints that are set.

</div>

Derived classes and clients can call the following methods to disable and restart events to achieve better performance. These methods only work when using the `'monitoring'` backend.

<div class="method">

disable_current_event()

Disable the current event until the next time `restart_events` is called. This is helpful when the debugger is not interested in the current line.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

restart_events()

Restart all the disabled events. This function is automatically called in `dispatch_*` methods after `user_*` methods are called. If the `dispatch_*` methods are not overridden, the disabled events will be restarted after each user interaction.

<div class="versionadded">

3.14

</div>

</div>

Derived classes and clients can call the following methods to get a data structure representing a stack trace.

<div class="method">

get_stack(f, t)

Return a list of (frame, lineno) tuples in a stack trace, and a size.

The most recently called frame is last in the list. The size is the number of frames below the frame where the debugger was invoked.

</div>

<div class="method">

format_stack_entry(frame_lineno, lprefix=': ')

Return a string with information about a stack entry, which is a `(frame, lineno)` tuple. The return string contains:

- The canonical filename which contains the frame.
- The function name or `"<lambda>"`.
- The input arguments.
- The return value.
- The line of code (if it exists).

</div>

The following two methods can be called by clients to use a debugger to debug a `statement`, given as a string.

<div class="method">

run(cmd, globals=None, locals=None)

Debug a statement executed via the `exec` function. *globals* defaults to `!__main__.__dict__`, *locals* defaults to *globals*.

</div>

<div class="method">

runeval(expr, globals=None, locals=None)

Debug an expression executed via the `eval` function. *globals* and *locals* have the same meaning as in `run`.

</div>

<div class="method">

runctx(cmd, globals, locals)

For backwards compatibility. Calls the `run` method.

</div>

<div class="method">

runcall(func, /, *args,*\*kwds)

Debug a single function call, and return its result.

</div>

</div>

Finally, the module defines the following functions:

<div class="function">

checkfuncname(b, frame)

Return `True` if we should break here, depending on the way the `Breakpoint` *b* was set.

If it was set via line number, it checks if `b.line <bdb.Breakpoint.line>` is the same as the one in *frame*. If the breakpoint was set via `function name <bdb.Breakpoint.funcname>`, we have to check we are in the right *frame* (the right function) and if we are on its first executable line.

</div>

<div class="function">

effective(file, line, frame)

Return `(active breakpoint, delete temporary flag)` or `(None, None)` as the breakpoint to act upon.

The *active breakpoint* is the first entry in `bplist <bdb.Breakpoint.bplist>` for the (`file <bdb.Breakpoint.file>`, `line <bdb.Breakpoint.line>`) (which must exist) that is `enabled <bdb.Breakpoint.enabled>`, for which `checkfuncname` is true, and that has neither a false `condition <bdb.Breakpoint.cond>` nor positive `ignore <bdb.Breakpoint.ignore>` count. The *flag*, meaning that a temporary breakpoint should be deleted, is `False` only when the `cond <bdb.Breakpoint.cond>` cannot be evaluated (in which case, `ignore <bdb.Breakpoint.ignore>` count is ignored).

If no such entry exists, then `(None, None)` is returned.

</div>

<div class="function">

set_trace()

Start debugging with a `Bdb` instance from caller's frame.

</div>

# Binary Data Services

The modules described in this chapter provide some basic services operations for manipulation of binary data. Other operations on binary data, specifically in relation to file formats and network protocols, are described in the relevant sections.

Some libraries described under `textservices` also work with either ASCII-compatible binary formats (for example, `re`) or all binary data (for example, `difflib`).

In addition, see the documentation for Python's built-in binary data types in `binaryseq`.

<div class="toctree">

struct.rst codecs.rst

</div>

# `!binascii` --- Convert between binary and ASCII

<div class="module" data-synopsis="Tools for converting between binary and various ASCII-encoded binary
representations.">

binascii

</div>

<div class="index">

pair: module; base64

</div>

------------------------------------------------------------------------

The `!binascii` module contains a number of methods to convert between binary and various ASCII-encoded binary representations. Normally, you will not use these functions directly but use wrapper modules like `base64` instead. The `!binascii` module contains low-level functions written in C for greater speed that are used by the higher-level modules.

> [!NOTE]
> `a2b_*` functions accept Unicode strings containing only ASCII characters. Other functions only accept `bytes-like objects <bytes-like object>` (such as `bytes`, `bytearray` and other objects that support the buffer protocol).
>
> <div class="versionchanged">
>
> 3.3 ASCII-only unicode strings are now accepted by the `a2b_*` functions.
>
> </div>

The `!binascii` module defines the following functions:

<div class="function">

a2b_uu(string)

Convert a single line of uuencoded data back to binary and return the binary data. Lines normally contain 45 (binary) bytes, except for the last line. Line data may be followed by whitespace.

</div>

<div class="function">

b2a_uu(data, \*, backtick=False)

Convert binary data to a line of ASCII characters, the return value is the converted line, including a newline char. The length of *data* should be at most 45. If *backtick* is true, zeros are represented by `` '`' `` instead of spaces.

<div class="versionchanged">

3.7 Added the *backtick* parameter.

</div>

</div>

<div class="function">

a2b_base64(string, /, *, padded=True, alphabet=BASE64_ALPHABET, strict_mode=False, canonical=False) a2b_base64(string, /,*, ignorechars, padded=True, alphabet=BASE64_ALPHABET, strict_mode=True, canonical=False)

Convert a block of base64 data back to binary and return the binary data. More than one line may be passed at a time.

Optional *alphabet* must be a `bytes` object of length 64 which specifies an alternative alphabet.

If *padded* is true, the last group of 4 base 64 alphabet characters must be padded with the '=' character. If *padded* is false, padding is neither required nor recognized: the '=' character is not treated as padding but as a non-alphabet character, which means it is silently discarded when *strict_mode* is false, or causes an `~binascii.Error` when *strict_mode* is true unless b'=' is included in *ignorechars*.

If *ignorechars* is specified, it should be a `bytes-like object` containing characters to ignore from the input when *strict_mode* is true. If *ignorechars* contains the pad character `'='`, the pad characters presented before the end of the encoded data and the excess pad characters will be ignored. The default value of *strict_mode* is `True` if *ignorechars* is specified, `False` otherwise.

If *strict_mode* is true, only valid base64 data will be converted. Invalid base64 data will raise `binascii.Error`.

Valid base64:

- Conforms to `4648`.
- Contains only characters from the base64 alphabet.
- Contains no excess data after padding (including excess padding, newlines, etc.).
- Does not start with a padding.

If *canonical* is true, non-zero padding bits in the last group are rejected with `binascii.Error`, enforcing canonical encoding as defined in `4648` section 3.5. This check is independent of *strict_mode*.

<div class="versionchanged">

3.11 Added the *strict_mode* parameter.

</div>

<div class="versionchanged">

3.15 Added the *alphabet*, *canonical*, *ignorechars*, and *padded* parameters.

</div>

</div>

<div class="function">

b2a_base64(data, \*, padded=True, alphabet=BASE64_ALPHABET, wrapcol=0, newline=True)

Convert binary data to a line(s) of ASCII characters in base64 coding, as specified in `4648`.

If *padded* is true (default), pad the encoded data with the '=' character to a size multiple of 4. If *padded* is false, do not add the pad characters.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not insert any newlines.

If *newline* is true (default), a newline character will be added at the end of the output.

<div class="versionchanged">

3.6 Added the *newline* parameter.

</div>

<div class="versionchanged">

3.15 Added the *alphabet*, *padded* and *wrapcol* parameters.

</div>

</div>

<div class="function">

a2b_ascii85(string, /, \*, foldspaces=False, adobe=False, ignorechars=b'', canonical=False)

Convert Ascii85 data back to binary and return the binary data.

Valid Ascii85 data contains characters from the Ascii85 alphabet in groups of five (except for the final group, which may have from two to five characters). Each group encodes 32 bits of binary data in the range from `0` to `2 ** 32 - 1`, inclusive. The special character `z` is accepted as a short form of the group `!!!!!`, which encodes four consecutive null bytes. A single-character final group is always rejected as an encoding violation.

*foldspaces* is a flag that specifies whether the 'y' short sequence should be accepted as shorthand for 4 consecutive spaces (ASCII 0x20). This feature is not supported by the "standard" Ascii85 encoding.

*adobe* controls whether the encoded byte sequence is framed with `<~` and `~>`, as in a PostScript base-85 string literal. If *adobe* is true, a leading `<~` is optionally accepted, while a trailing `~>` is *required*, and `binascii.Error` is raised if it is not found.

*ignorechars* should be a `bytes-like object` containing characters to ignore from the input. This should only contain whitespace characters.

If *canonical* is true, non-canonical encodings are rejected with `binascii.Error`. Here "canonical" means the encoding that `b2a_ascii85` would produce: the `z` abbreviation must be used for all-zero groups (rather than `!!!!!`), and partial final groups must use the same padding digits as the encoder.

Invalid Ascii85 data will raise `binascii.Error`.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

b2a_ascii85(data, /, \*, foldspaces=False, wrapcol=0, pad=False, adobe=False)

Convert binary data to a formatted sequence of ASCII characters in Ascii85 coding. The return value is the converted data.

*foldspaces* is an optional flag that uses the special short sequence 'y' instead of 4 consecutive spaces (ASCII 0x20) as supported by 'btoa'. This feature is not supported by the "standard" Ascii85 encoding.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not insert any newlines.

If *pad* is true, the zero-padding applied to the end of the input is fully retained in the output encoding, as done by `btoa`, producing an exact multiple of 5 bytes of output. This is not part of the standard encoding used in PDF, as it does not preserve the length of the data.

*adobe* controls whether the encoded byte sequence is framed with `<~` and `~>`, as in a PostScript base-85 string literal. Note that while ASCII85Decode streams in PDF documents *must* be terminated with `~>`, they *must not* use a leading `<~`.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

a2b_base85(string, /, \*, alphabet=BASE85_ALPHABET, ignorechars=b'', canonical=False)

Convert Base85 data back to binary and return the binary data. More than one line may be passed at a time.

Valid Base85 data contains characters from the Base85 alphabet in groups of five (except for the final group, which may have from two to five characters). Each group encodes 32 bits of binary data in the range from `0` to `2 ** 32 - 1`, inclusive. A single-character final group is always rejected as an encoding violation.

Optional *alphabet* must be a `bytes` object of length 85 which specifies an alternative alphabet.

*ignorechars* should be a `bytes-like object` containing characters to ignore from the input.

If *canonical* is true, non-canonical encodings are rejected with `binascii.Error`. Here "canonical" means the encoding that `b2a_base85` would produce: partial final groups must use the same padding digits as the encoder.

Invalid Base85 data will raise `binascii.Error`.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

b2a_base85(data, /, \*, alphabet=BASE85_ALPHABET, wrapcol=0, pad=False)

Convert binary data to a line of ASCII characters in Base85 coding. The return value is the converted line.

Optional *alphabet* must be a `bytes-like object` of length 85 which specifies an alternative alphabet.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not insert any newlines.

If *pad* is true, the zero-padding applied to the end of the input is retained in the output, which will always be a multiple of 5 bytes, and thus the length of the data may not be preserved on decoding.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

a2b_base32(string, /, \*, padded=True, alphabet=BASE32_ALPHABET, ignorechars=b'', canonical=False)

Convert base32 data back to binary and return the binary data.

Valid base32 data contains characters from the base32 alphabet specified in `4648` in groups of eight (if necessary, the final group is padded to eight characters with `=`). Each group encodes 40 bits of binary data in the range from `0` to `2 ** 40 - 1`, inclusive.

> [!NOTE]
> This function does not map lowercase characters (which are invalid in standard base32) to their uppercase counterparts, nor does it contextually map `0` to `O` and `1` to `I`/`L` as `4648` allows.

Optional *alphabet* must be a `bytes` object of length 32 which specifies an alternative alphabet.

If *padded* is true, the last group of 8 base 32 alphabet characters must be padded with the '=' character. If *padded* is false, the '=' character is treated as other non-alphabet characters (depending on the value of *ignorechars*).

*ignorechars* should be a `bytes-like object` containing characters to ignore from the input. If *ignorechars* contains the pad character `'='`, the pad characters presented before the end of the encoded data and the excess pad characters will be ignored.

If *canonical* is true, non-zero padding bits in the last group are rejected with `binascii.Error`, enforcing canonical encoding as defined in `4648` section 3.5.

Invalid base32 data will raise `binascii.Error`.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

b2a_base32(data, /, \*, padded=True, alphabet=BASE32_ALPHABET, wrapcol=0)

Convert binary data to a line of ASCII characters in base32 coding, as specified in `4648`. The return value is the converted line.

Optional *alphabet* must be a `bytes-like object` of length 32 which specifies an alternative alphabet.

If *padded* is true (default), pad the encoded data with the '=' character to a size multiple of 8. If *padded* is false, do not add the pad characters.

If *wrapcol* is non-zero, insert a newline (`b'\n'`) character after at most every *wrapcol* characters. If *wrapcol* is zero (default), do not insert any newlines.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

a2b_qp(data, header=False, \*, strip_ws=False)

Convert a block of quoted-printable data back to binary and return the binary data. More than one line may be passed at a time. If the optional argument *header* is present and true, underscores will be decoded as spaces. If the optional argument *strip_ws* is true, trailing whitespace is stripped from each line, as required by `2045`.

<div class="versionchanged">

next Added the *strip_ws* parameter.

</div>

</div>

<div class="function">

b2a_qp(data, quotetabs=False, istext=True, header=False)

Convert binary data to a line(s) of ASCII characters in quoted-printable encoding. The return value is the converted line(s). If the optional argument *quotetabs* is present and true, all tabs and spaces will be encoded. If the optional argument *istext* is present and true, newlines are not encoded but trailing whitespace will be encoded. If the optional argument *header* is present and true, spaces will be encoded as underscores per `1522`. If the optional argument *header* is present and false, newline characters will be encoded as well; otherwise linefeed conversion might corrupt the binary data stream.

</div>

<div class="function">

crc_hqx(data, value)

Compute a 16-bit CRC value of *data*, starting with *value* as the initial CRC, and return the result. This uses the CRC-CCITT polynomial *x*<sup>16</sup> + *x*<sup>12</sup> + *x*<sup>5</sup> + 1, often represented as 0x1021. This CRC is used in the binhex4 format.

</div>

<div class="function">

crc32(data\[, value\])

Compute CRC-32, the unsigned 32-bit checksum of *data*, starting with an initial CRC of *value*. The default initial CRC is zero. The algorithm is consistent with the ZIP file checksum. Since the algorithm is designed for use as a checksum algorithm, it is not suitable for use as a general hash algorithm. Use as follows:

    print(binascii.crc32(b"hello world"))
    # Or, in two pieces:
    crc = binascii.crc32(b"hello")
    crc = binascii.crc32(b" world", crc)
    print('crc32 = {:#010x}'.format(crc))

<div class="versionchanged">

3.0 The result is always unsigned.

</div>

</div>

<div class="function">

b2a_hex(data\[, sep\[, bytes_per_sep=1\]\]) hexlify(data\[, sep\[, bytes_per_sep=1\]\])

Return the hexadecimal representation of the binary *data*. Every byte of *data* is converted into the corresponding 2-digit hex representation. The returned bytes object is therefore twice as long as the length of *data*.

Similar functionality (but returning a text string) is also conveniently accessible using the `bytes.hex` method.

If *sep* is specified, it must be a single character str or bytes object. It will be inserted in the output after every *bytes_per_sep* input bytes. Separator placement is counted from the right end of the output by default, if you wish to count from the left, supply a negative *bytes_per_sep* value.

> \>\>\> import binascii \>\>\> binascii.b2a_hex(b'xb9x01xef') b'b901ef' \>\>\> binascii.hexlify(b'xb9x01xef', '-') b'b9-01-ef' \>\>\> binascii.b2a_hex(b'xb9x01xef', b'\_', 2) b'b9_01ef' \>\>\> binascii.b2a_hex(b'xb9x01xef', b' ', -2) b'b901 ef'

<div class="versionchanged">

3.8 The *sep* and *bytes_per_sep* parameters were added.

</div>

</div>

<div class="function">

a2b_hex(hexstr, *, ignorechars=b'') unhexlify(hexstr,*, ignorechars=b'')

Return the binary data represented by the hexadecimal string *hexstr*. This function is the inverse of `b2a_hex`. *hexstr* must contain an even number of hexadecimal digits (which can be upper or lower case), otherwise an `Error` exception is raised.

*ignorechars* should be a `bytes-like object` containing characters to ignore from the input.

Similar functionality (but more liberal towards whitespace) is also accessible using the `bytes.fromhex` class method.

<div class="versionchanged">

3.15 Added the *ignorechars* parameter.

</div>

</div>

<div class="exception">

Error

Exception raised on errors. These are usually programming errors.

</div>

<div class="exception">

Incomplete

Exception raised on incomplete data. These are usually not programming errors, but may be handled by reading a little more data and trying again.

</div>

<div class="data">

BASE64_ALPHABET

The Base 64 alphabet according to `4648`.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

URLSAFE_BASE64_ALPHABET

The "URL and filename safe" Base 64 alphabet according to `4648`.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

UU_ALPHABET

The uuencoding alphabet.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

CRYPT_ALPHABET

The Base 64 alphabet used in the `crypt(3)` routine and in the GEDCOM format.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

BINHEX_ALPHABET

The Base 64 alphabet used in BinHex 4 (HQX) within the classic Mac OS.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

BASE85_ALPHABET

The Base85 alphabet.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

ASCII85_ALPHABET

The Ascii85 alphabet.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

Z85_ALPHABET

The [Z85](https://rfc.zeromq.org/spec/32/) alphabet.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

BASE32_ALPHABET

The Base 32 alphabet according to `4648`.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

BASE32HEX_ALPHABET

The "Extended Hex" Base 32 alphabet according to `4648`. Data encoded with this alphabet maintains its sort order during bitwise comparisons.

<div class="versionadded">

3.15

</div>

</div>

<div class="seealso">

Module `base64`  
Support for RFC compliant base64-style encoding in base 16, 32, 64, and 85.

Module `quopri`  
Support for quoted-printable encoding used in MIME email messages.

</div>

# `!bisect` --- Array bisection algorithm

<div class="module" data-synopsis="Array bisection algorithms for binary searching.">

bisect

</div>

**Source code:** `Lib/bisect.py`

------------------------------------------------------------------------

This module provides support for maintaining a list in sorted order without having to sort the list after each insertion. For long lists of items with expensive comparison operations, this can be an improvement over linear searches or frequent resorting.

The module is called `!bisect` because it uses a basic bisection algorithm to do its work. Unlike other bisection tools that search for a specific value, the functions in this module are designed to locate an insertion point. Accordingly, the functions never call an `~object.__eq__` method to determine whether a value has been found. Instead, the functions only call the `~object.__lt__` method and will return an insertion point between values in an array.

> [!NOTE]
> The functions in this module are not thread-safe. If multiple threads concurrently use `!bisect` functions on the same sequence, this may result in undefined behaviour. Likewise, if the provided sequence is mutated by a different thread while a `!bisect` function is operating on it, the result is undefined. For example, using `~bisect.insort_left` on the same list from multiple threads may result in the list becoming unsorted.

<div id="bisect functions">

The following functions are provided:

</div>

<div class="function">

bisect_left(a, x, lo=0, hi=len(a), \*, key=None)

Locate the insertion point for *x* in *a* to maintain sorted order. The parameters *lo* and *hi* may be used to specify a subset of the list which should be considered; by default the entire list is used. If *x* is already present in *a*, the insertion point will be before (to the left of) any existing entries. The return value is suitable for use as the first parameter to `list.insert()` assuming that *a* is already sorted.

The returned insertion point *ip* partitions the array *a* into two slices such that `all(elem < x for elem in a[lo : ip])` is true for the left slice and `all(elem >= x for elem in a[ip : hi])` is true for the right slice.

*key* specifies a `key function` of one argument that is used to extract a comparison key from each element in the array. To support searching complex records, the key function is not applied to the *x* value.

If *key* is `None`, the elements are compared directly and no key function is called.

<div class="versionchanged">

3.10 Added the *key* parameter.

</div>

</div>

<div class="function">

bisect_right(a, x, lo=0, hi=len(a), *, key=None) bisect(a, x, lo=0, hi=len(a),*, key=None)

Similar to `~bisect.bisect_left`, but returns an insertion point which comes after (to the right of) any existing entries of *x* in *a*.

The returned insertion point *ip* partitions the array *a* into two slices such that `all(elem <= x for elem in a[lo : ip])` is true for the left slice and `all(elem > x for elem in a[ip : hi])` is true for the right slice.

<div class="versionchanged">

3.10 Added the *key* parameter.

</div>

</div>

<div class="function">

insort_left(a, x, lo=0, hi=len(a), \*, key=None)

Insert *x* in *a* in sorted order.

This function first runs `~bisect.bisect_left` to locate an insertion point. Next, it runs the `~sequence.insert` method on *a* to insert *x* at the appropriate position to maintain sort order.

To support inserting records in a table, the *key* function (if any) is applied to *x* for the search step but not for the insertion step.

Keep in mind that the *O*(log *n*) search is dominated by the slow *O*(*n*) insertion step.

<div class="versionchanged">

3.10 Added the *key* parameter.

</div>

</div>

<div class="function">

insort_right(a, x, lo=0, hi=len(a), *, key=None) insort(a, x, lo=0, hi=len(a),*, key=None)

Similar to `~bisect.insort_left`, but inserting *x* in *a* after any existing entries of *x*.

This function first runs `~bisect.bisect_right` to locate an insertion point. Next, it runs the `~sequence.insert` method on *a* to insert *x* at the appropriate position to maintain sort order.

To support inserting records in a table, the *key* function (if any) is applied to *x* for the search step but not for the insertion step.

Keep in mind that the *O*(log *n*) search is dominated by the slow *O*(*n*) insertion step.

<div class="versionchanged">

3.10 Added the *key* parameter.

</div>

</div>

## Performance Notes

When writing time sensitive code using *bisect()* and *insort()*, keep these thoughts in mind:

- Bisection is effective for searching ranges of values. For locating specific values, dictionaries are more performant.
- The *insort()* functions are *O*(*n*) because the logarithmic search step is dominated by the linear time insertion step.
- The search functions are stateless and discard key function results after they are used. Consequently, if the search functions are used in a loop, the key function may be called again and again on the same array elements. If the key function isn't fast, consider wrapping it with `functools.cache` to avoid duplicate computations. Alternatively, consider searching an array of precomputed keys to locate the insertion point (as shown in the examples section below).

<div class="seealso">

- [Sorted Collections](https://grantjenks.com/docs/sortedcollections/) is a high performance module that uses *bisect* to managed sorted collections of data.
- The [SortedCollection recipe](https://code.activestate.com/recipes/577197-sortedcollection/) uses bisect to build a full-featured collection class with straight-forward search methods and support for a key-function. The keys are precomputed to save unnecessary calls to the key function during searches.

</div>

## Searching Sorted Lists

The above [bisect functions](<#bisect functions>) are useful for finding insertion points but can be tricky or awkward to use for common searching tasks. The following five functions show how to transform them into the standard lookups for sorted lists:

    def index(a, x):
        'Locate the leftmost value exactly equal to x'
        i = bisect_left(a, x)
        if i != len(a) and a[i] == x:
            return i
        raise ValueError

    def find_lt(a, x):
        'Find rightmost value less than x'
        i = bisect_left(a, x)
        if i:
            return a[i-1]
        raise ValueError

    def find_le(a, x):
        'Find rightmost value less than or equal to x'
        i = bisect_right(a, x)
        if i:
            return a[i-1]
        raise ValueError

    def find_gt(a, x):
        'Find leftmost value greater than x'
        i = bisect_right(a, x)
        if i != len(a):
            return a[i]
        raise ValueError

    def find_ge(a, x):
        'Find leftmost item greater than or equal to x'
        i = bisect_left(a, x)
        if i != len(a):
            return a[i]
        raise ValueError

## Examples

<div id="bisect-example">

The `~bisect.bisect` function can be useful for numeric table lookups. This example uses `~bisect.bisect` to look up a letter grade for an exam score (say) based on a set of ordered numeric breakpoints: 90 and up is an 'A', 80 to 89 is a 'B', and so on:

    >>> def grade(score):
    ...     i = bisect([60, 70, 80, 90], score)
    ...     return "FDCBA"[i]
    ...
    >>> [grade(score) for score in [33, 99, 77, 70, 89, 90, 100]]
    ['F', 'A', 'C', 'C', 'B', 'A', 'A']

</div>

The `~bisect.bisect` and `~bisect.insort` functions also work with lists of tuples. The *key* argument can serve to extract the field used for ordering records in a table:

    >>> from collections import namedtuple
    >>> from operator import attrgetter
    >>> from bisect import bisect, insort
    >>> from pprint import pprint

    >>> Movie = namedtuple('Movie', ('name', 'released', 'director'))

    >>> movies = [
    ...     Movie('Jaws', 1975, 'Spielberg'),
    ...     Movie('Titanic', 1997, 'Cameron'),
    ...     Movie('The Birds', 1963, 'Hitchcock'),
    ...     Movie('Aliens', 1986, 'Cameron')
    ... ]

    >>> # Find the first movie released after 1960
    >>> by_year = attrgetter('released')
    >>> movies.sort(key=by_year)
    >>> movies[bisect(movies, 1960, key=by_year)]
    Movie(name='The Birds', released=1963, director='Hitchcock')

    >>> # Insert a movie while maintaining sort order
    >>> romance = Movie('Love Story', 1970, 'Hiller')
    >>> insort(movies, romance, key=by_year)
    >>> pprint(movies)
    [Movie(name='The Birds', released=1963, director='Hitchcock'),
     Movie(name='Love Story', released=1970, director='Hiller'),
     Movie(name='Jaws', released=1975, director='Spielberg'),
     Movie(name='Aliens', released=1986, director='Cameron'),
     Movie(name='Titanic', released=1997, director='Cameron')]

If the key function is expensive, it is possible to avoid repeated function calls by searching a list of precomputed keys to find the index of a record:

    >>> data = [('red', 5), ('blue', 1), ('yellow', 8), ('black', 0)]
    >>> data.sort(key=lambda r: r[1])       # Or use operator.itemgetter(1).
    >>> keys = [r[1] for r in data]         # Precompute a list of keys.
    >>> data[bisect_left(keys, 0)]
    ('black', 0)
    >>> data[bisect_left(keys, 1)]
    ('blue', 1)
    >>> data[bisect_left(keys, 5)]
    ('red', 5)
    >>> data[bisect_left(keys, 8)]
    ('yellow', 8)

# `!builtins` --- Built-in objects

<div class="module" data-synopsis="The module that provides the built-in namespace.">

builtins

</div>

------------------------------------------------------------------------

This module provides direct access to all 'built-in' identifiers of Python; for example, `builtins.open` is the full name for the built-in function `open`.

This module is not normally accessed explicitly by most applications, but can be useful in modules that provide objects with the same name as a built-in value, but in which the built-in of that name is also needed. For example, in a module that wants to implement an `open` function that wraps the built-in `open`, this module can be used directly:

    import builtins

    def open(path):
        f = builtins.open(path, 'r')
        return UpperCaser(f)

    class UpperCaser:
        '''Wrapper around a file that converts output to uppercase.'''

        def __init__(self, f):
            self._f = f

        def read(self, count=-1):
            return self._f.read(count).upper()

        # ...

As an implementation detail, most modules have the name `__builtins__` made available as part of their globals. The value of `__builtins__` is normally either this module or the value of this module's `~object.__dict__` attribute. Since this is an implementation detail, it may not be used by alternate implementations of Python.

<div class="seealso">

- `built-in-consts`
- `bltin-exceptions`
- `built-in-funcs`
- `bltin-types`

</div>

# `!bz2` --- Support for `bzip2` compression

<div class="module" data-synopsis="Interfaces for bzip2 compression and decompression.">

bz2

</div>

**Source code:** `Lib/bz2.py`

------------------------------------------------------------------------

This module provides a comprehensive interface for compressing and decompressing data using the bzip2 compression algorithm.

The `!bz2` module contains:

- The `.open` function and `BZ2File` class for reading and writing compressed files.
- The `BZ2Compressor` and `BZ2Decompressor` classes for incremental (de)compression.
- The `compress` and `decompress` functions for one-shot (de)compression.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

## (De)compression of files

<div class="function">

open(filename, mode='rb', compresslevel=9, encoding=None, errors=None, newline=None)

Open a bzip2-compressed file in binary or text mode, returning a `file
object`.

As with the constructor for `BZ2File`, the *filename* argument can be an actual filename (a `str` or `bytes` object), or an existing file object to read from or write to.

The *mode* argument can be any of `'r'`, `'rb'`, `'w'`, `'wb'`, `'x'`, `'xb'`, `'a'` or `'ab'` for binary mode, or `'rt'`, `'wt'`, `'xt'`, or `'at'` for text mode. The default is `'rb'`.

The *compresslevel* argument is an integer from 1 to 9, as for the `BZ2File` constructor.

For binary mode, this function is equivalent to the `BZ2File` constructor: `BZ2File(filename, mode, compresslevel=compresslevel)`. In this case, the *encoding*, *errors* and *newline* arguments must not be provided.

For text mode, a `BZ2File` object is created, and wrapped in an `io.TextIOWrapper` instance with the specified encoding, error handling behavior, and line ending(s).

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.4 The `'x'` (exclusive creation) mode was added.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="BZ2File(filename, mode='r', *, compresslevel=9)">

Open a bzip2-compressed file in binary mode.

If *filename* is a `str` or `bytes` object, open the named file directly. Otherwise, *filename* should be a `file object`, which will be used to read or write the compressed data.

The *mode* argument can be either `'r'` for reading (default), `'w'` for overwriting, `'x'` for exclusive creation, or `'a'` for appending. These can equivalently be given as `'rb'`, `'wb'`, `'xb'` and `'ab'` respectively.

If *filename* is a file object (rather than an actual file name), a mode of `'w'` does not truncate the file, and is instead equivalent to `'a'`.

If *mode* is `'w'` or `'a'`, *compresslevel* can be an integer between `1` and `9` specifying the level of compression: `1` produces the least compression, and `9` (default) produces the most compression.

If *mode* is `'r'`, the input file may be the concatenation of multiple compressed streams.

`BZ2File` provides all of the members specified by the `io.BufferedIOBase`, except for `~io.BufferedIOBase.detach` and `~io.IOBase.truncate`. Iteration and the `with` statement are supported.

`BZ2File` also provides the following methods and attributes:

<div class="method">

peek(\[n\])

Return buffered data without advancing the file position. At least one byte of data will be returned (unless at EOF). The exact number of bytes returned is unspecified.

> [!NOTE]
> While calling `peek` does not change the file position of the `BZ2File`, it may change the position of the underlying file object (e.g. if the `BZ2File` was constructed by passing a file object for *filename*).

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

fileno()

Return the file descriptor for the underlying file.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

readable()

Return whether the file was opened for reading.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

seekable()

Return whether the file supports seeking.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

writable()

Return whether the file was opened for writing.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

read1(size=-1)

Read up to *size* uncompressed bytes, while trying to avoid making multiple reads from the underlying stream. Reads up to a buffer's worth of data if size is negative.

Returns `b''` if the file is at EOF.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

readinto(b)

Read bytes into *b*.

Returns the number of bytes read (0 for EOF).

<div class="versionadded">

3.3

</div>

</div>

<div class="attribute">

mode

`'rb'` for reading and `'wb'` for writing.

<div class="versionadded">

3.13

</div>

</div>

<div class="attribute">

name

The bzip2 file name. Equivalent to the `~io.FileIO.name` attribute of the underlying `file object`.

<div class="versionadded">

3.13

</div>

</div>

<div class="versionchanged">

3.1 Support for the `with` statement was added.

</div>

<div class="versionchanged">

3.3 Support was added for *filename* being a `file object` instead of an actual filename.

The `'a'` (append) mode was added, along with support for reading multi-stream files.

</div>

<div class="versionchanged">

3.4 The `'x'` (exclusive creation) mode was added.

</div>

<div class="versionchanged">

3.5 The `~io.BufferedIOBase.read` method now accepts an argument of `None`.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.9 The *buffering* parameter has been removed. It was ignored and deprecated since Python 3.0. Pass an open file object to control how the file is opened.

The *compresslevel* parameter became keyword-only.

</div>

<div class="versionchanged">

3.10 This class is thread unsafe in the face of multiple simultaneous readers or writers, just like its equivalent classes in `gzip` and `lzma` have always been.

</div>

</div>

## Incremental (de)compression

<div class="BZ2Compressor(compresslevel=9)">

Create a new compressor object. This object may be used to compress data incrementally. For one-shot compression, use the `compress` function instead.

*compresslevel*, if given, must be an integer between `1` and `9`. The default is `9`.

<div class="method">

compress(data)

Provide data to the compressor object. Returns a chunk of compressed data if possible, or an empty byte string otherwise.

When you have finished providing data to the compressor, call the `flush` method to finish the compression process.

</div>

<div class="method">

flush()

Finish the compression process. Returns the compressed data left in internal buffers.

The compressor object may not be used after this method has been called.

</div>

</div>

<div class="BZ2Decompressor()">

Create a new decompressor object. This object may be used to decompress data incrementally. For one-shot compression, use the `decompress` function instead.

> [!NOTE]
> This class does not transparently handle inputs containing multiple compressed streams, unlike `decompress` and `BZ2File`. If you need to decompress a multi-stream input with `BZ2Decompressor`, you must use a new decompressor for each stream.

<div class="method">

decompress(data, max_length=-1)

Decompress *data* (a `bytes-like object`), returning uncompressed data as bytes. Some of *data* may be buffered internally, for use in later calls to `decompress`. The returned data should be concatenated with the output of any previous calls to `decompress`.

If *max_length* is nonnegative, returns at most *max_length* bytes of decompressed data. If this limit is reached and further output can be produced, the `~.needs_input` attribute will be set to `False`. In this case, the next call to `~.decompress` may provide *data* as `b''` to obtain more of the output.

If all of the input data was decompressed and returned (either because this was less than *max_length* bytes, or because *max_length* was negative), the `~.needs_input` attribute will be set to `True`.

Attempting to decompress data after the end of stream is reached raises an `EOFError`. Any data found after the end of the stream is ignored and saved in the `~.unused_data` attribute.

<div class="versionchanged">

3.5 Added the *max_length* parameter.

</div>

</div>

<div class="attribute">

eof

`True` if the end-of-stream marker has been reached.

<div class="versionadded">

3.3

</div>

</div>

<div class="attribute">

unused_data

Data found after the end of the compressed stream.

If this attribute is accessed before the end of the stream has been reached, its value will be `b''`.

</div>

<div class="attribute">

needs_input

`False` if the `.decompress` method can provide more decompressed data before requiring new uncompressed input.

<div class="versionadded">

3.5

</div>

</div>

</div>

## One-shot (de)compression

<div class="function">

compress(data, compresslevel=9)

Compress *data*, a `bytes-like object <bytes-like object>`.

*compresslevel*, if given, must be an integer between `1` and `9`. The default is `9`.

For incremental compression, use a `BZ2Compressor` instead.

</div>

<div class="function">

decompress(data)

Decompress *data*, a `bytes-like object <bytes-like object>`.

If *data* is the concatenation of multiple compressed streams, decompress all of the streams.

For incremental decompression, use a `BZ2Decompressor` instead.

<div class="versionchanged">

3.3 Support for multi-stream inputs was added.

</div>

</div>

## Miscellaneous

Information about the version of the bzip2 compression library actually loaded by the interpreter is available through the following constants. The version used for building the module is not available, because the bzip2 library does not provide it.

<div class="data">

bzlib_version

The version string of the bzip2 compression library actually loaded by the interpreter.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

bzlib_version_info

A named tuple containing the three components of the bzip2 compression library version actually loaded by the interpreter: *major*, *minor*, and *patch*. All values are integers. The components can also be accessed by name, so `bz2.bzlib_version_info[0]` is equivalent to `bz2.bzlib_version_info.major` and so on.

<div class="versionadded">

3.16

</div>

</div>

## Examples of usage

Below are some examples of typical usage of the `!bz2` module.

Using `compress` and `decompress` to demonstrate round-trip compression:

> \>\>\> import bz2 \>\>\> data = b"""... Donec rhoncus quis sapien sit amet molestie. Fusce scelerisque vel augue ... nec ullamcorper. Nam rutrum pretium placerat. Aliquam vel tristique lorem, ... sit amet cursus ante. In interdum laoreet mi, sit amet ultrices purus ... pulvinar a. Nam gravida euismod magna, non varius justo tincidunt feugiat. ... Aliquam pharetra lacus non risus vehicula rutrum. Maecenas aliquam leo ... felis. Pellentesque semper nunc sit amet nibh ullamcorper, ac elementum ... dolor luctus. Curabitur lacinia mi ornare consectetur vestibulum.""" \>\>\> c = bz2.compress(data) \>\>\> len(data) / len(c) \# Data compression ratio 1.513595166163142 \>\>\> d = bz2.decompress(c) \>\>\> data == d \# Check equality to original object after round-trip True

Using `BZ2Compressor` for incremental compression:

> \>\>\> import bz2 \>\>\> def gen_data(chunks=10, chunksize=1000): ... """Yield incremental blocks of chunksize bytes.""" ... for \_ in range(chunks): ... yield b"z" \* chunksize ... \>\>\> comp = bz2.BZ2Compressor() \>\>\> out = b"" \>\>\> for chunk in gen_data(): ... \# Provide data to the compressor object ... out = out + comp.compress(chunk) ... \>\>\> \# Finish the compression process. Call this once you have \>\>\> \# finished providing data to the compressor. \>\>\> out = out + comp.flush()

The example above uses a very "nonrandom" stream of data (a stream of `b"z"` chunks). Random data tends to compress poorly, while ordered, repetitive data usually yields a high compression ratio.

Writing and reading a bzip2-compressed file in binary mode:

> \>\>\> import bz2 \>\>\> data = b"""... Donec rhoncus quis sapien sit amet molestie. Fusce scelerisque vel augue ... nec ullamcorper. Nam rutrum pretium placerat. Aliquam vel tristique lorem, ... sit amet cursus ante. In interdum laoreet mi, sit amet ultrices purus ... pulvinar a. Nam gravida euismod magna, non varius justo tincidunt feugiat. ... Aliquam pharetra lacus non risus vehicula rutrum. Maecenas aliquam leo ... felis. Pellentesque semper nunc sit amet nibh ullamcorper, ac elementum ... dolor luctus. Curabitur lacinia mi ornare consectetur vestibulum.""" \>\>\> with bz2.open("myfile.bz2", "wb") as f: ... \# Write compressed data to file ... unused = f.write(data) ... \>\>\> with bz2.open("myfile.bz2", "rb") as f: ... \# Decompress data from file ... content = f.read() ... \>\>\> content == data \# Check equality to original object after round-trip True

<div class="testcleanup">

import os os.remove("myfile.bz2")

</div>

# `!calendar` --- General calendar-related functions

<div class="module" data-synopsis="Functions for working with calendars, including some emulation
of the Unix cal program.">

calendar

</div>

**Source code:** `Lib/calendar.py`

------------------------------------------------------------------------

This module allows you to output calendars like the Unix `cal` program, and provides additional useful functions related to the calendar. By default, these calendars have Monday as the first day of the week, and Sunday as the last (the European convention). Use `setfirstweekday` to set the first day of the week to Sunday (6) or to any other weekday. Parameters that specify dates are given as integers. For related functionality, see also the `datetime` and `time` modules.

The functions and classes defined in this module use an idealized calendar, the current Gregorian calendar extended indefinitely in both directions. This matches the definition of the "proleptic Gregorian" calendar in Dershowitz and Reingold's book "Calendrical Calculations", where it's the base calendar for all computations. Zero and negative years are interpreted as prescribed by the ISO 8601 standard. Year 0 is 1 BC, year -1 is 2 BC, and so on.

<div class="Calendar(firstweekday=0)">

Creates a `Calendar` object. *firstweekday* is an integer specifying the first day of the week. `MONDAY` is `0` (the default), `SUNDAY` is `6`.

A `Calendar` object provides several methods that can be used for preparing the calendar data for formatting. This class doesn't do any formatting itself. This is the job of subclasses.

`Calendar` instances have the following methods and attributes:

<div class="attribute">

firstweekday

The first weekday as an integer (0--6).

This property can also be set and read using `~Calendar.setfirstweekday` and `~Calendar.getfirstweekday` respectively.

</div>

<div class="method">

getfirstweekday()

Return an `int` for the current first weekday (0--6).

Identical to reading the `~Calendar.firstweekday` property.

</div>

<div class="method">

setfirstweekday(firstweekday)

Set the first weekday to *firstweekday*, passed as an `int` (0--6).

Identical to setting the `~Calendar.firstweekday` property.

</div>

<div class="method">

iterweekdays()

Return an iterator for the weekday numbers that will be used for one week. The first value from the iterator will be the same as the value of the `~Calendar.firstweekday` property.

</div>

<div class="method">

itermonthdates(year, month)

Return an iterator for the month *month* (1--12) in the year *year*. This iterator will return all days (as `datetime.date` objects) for the month and all days before the start of the month or after the end of the month that are required to get a complete week.

</div>

<div class="method">

itermonthdays(year, month)

Return an iterator for the month *month* in the year *year* similar to `itermonthdates`, but not restricted by the `datetime.date` range. Days returned will simply be day of the month numbers. For the days outside of the specified month, the day number is `0`.

</div>

<div class="method">

itermonthdays2(year, month)

Return an iterator for the month *month* in the year *year* similar to `itermonthdates`, but not restricted by the `datetime.date` range. Days returned will be tuples consisting of a day of the month number and a weekday number.

</div>

<div class="method">

itermonthdays3(year, month)

Return an iterator for the month *month* in the year *year* similar to `itermonthdates`, but not restricted by the `datetime.date` range. Days returned will be tuples consisting of a year, a month and a day of the month numbers.

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

itermonthdays4(year, month)

Return an iterator for the month *month* in the year *year* similar to `itermonthdates`, but not restricted by the `datetime.date` range. Days returned will be tuples consisting of a year, a month, a day of the month, and a day of the week numbers.

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

monthdatescalendar(year, month)

Return a list of the weeks in the month *month* of the *year* as full weeks. Weeks are lists of seven `datetime.date` objects.

</div>

<div class="method">

monthdays2calendar(year, month)

Return a list of the weeks in the month *month* of the *year* as full weeks. Weeks are lists of seven tuples of day numbers and weekday numbers.

</div>

<div class="method">

monthdayscalendar(year, month)

Return a list of the weeks in the month *month* of the *year* as full weeks. Weeks are lists of seven day numbers.

</div>

<div class="method">

yeardatescalendar(year, width=3)

Return the data for the specified year ready for formatting. The return value is a list of month rows. Each month row contains up to *width* months (defaulting to 3). Each month contains between 4 and 6 weeks and each week contains 1--7 days. Days are `datetime.date` objects.

</div>

<div class="method">

yeardays2calendar(year, width=3)

Return the data for the specified year ready for formatting (similar to `yeardatescalendar`). Entries in the week lists are tuples of day numbers and weekday numbers. Day numbers outside this month are zero.

</div>

<div class="method">

yeardayscalendar(year, width=3)

Return the data for the specified year ready for formatting (similar to `yeardatescalendar`). Entries in the week lists are day numbers. Day numbers outside this month are zero.

</div>

</div>

<div class="TextCalendar(firstweekday=0)">

This class can be used to generate plain text calendars.

`TextCalendar` instances have the following methods:

<div class="method">

prweek(theweek, width)

Print a week's calendar as returned by `formatweek` and without a final newline.

</div>

<div class="method">

formatday(theday, weekday, width)

Return a string representing a single day formatted with the given *width*. If *theday* is `0`, return a string of spaces of the specified width, representing an empty day. The *weekday* parameter is unused.

</div>

<div class="method">

formatweek(theweek, width)

Return a single week in a string with no newline. *width* specifies the width of the date columns, which are centered. Depends on the first weekday as specified in the constructor or set by the `setfirstweekday` method.

</div>

<div class="method">

formatweekday(weekday, width)

Return a string representing the name of a single weekday formatted to the specified *width*. The *weekday* parameter is an integer representing the day of the week, where `0` is Monday and `6` is Sunday.

</div>

<div class="method">

formatweekheader(width)

Return a string containing the header row of weekday names, formatted with the given *width* for each column. The names depend on the locale settings and are padded to the specified width.

</div>

<div class="method">

formatmonth(theyear, themonth, w=0, l=0)

Return a month's calendar in a multi-line string. If *w* is provided, it specifies the width of the date columns, which are centered. If *l* is given, it specifies the number of lines that each week will use. Depends on the first weekday as specified in the constructor or set by the `setfirstweekday` method.

</div>

<div class="method">

formatmonthname(theyear, themonth, width, withyear=True)

Return a string representing the month's name centered within the specified *width*. If *withyear* is `True`, include the year in the output. The *theyear* and *themonth* parameters specify the year and month for the name to be formatted respectively.

</div>

<div class="method">

prmonth(theyear, themonth, w=0, l=0)

Print a month's calendar as returned by `formatmonth`.

</div>

<div class="method">

formatyear(theyear, w=2, l=1, c=6, m=3)

Return a *m*-column calendar for an entire year as a multi-line string. Optional parameters *w*, *l*, and *c* are for date column width, lines per week, and number of spaces between month columns, respectively. Depends on the first weekday as specified in the constructor or set by the `setfirstweekday` method. The earliest year for which a calendar can be generated is platform-dependent.

</div>

<div class="method">

pryear(theyear, w=2, l=1, c=6, m=3)

Print the calendar for an entire year as returned by `formatyear`.

</div>

</div>

<div class="HTMLCalendar(firstweekday=0)">

This class can be used to generate HTML calendars.

`!HTMLCalendar` instances have the following methods:

<div class="method">

formatmonth(theyear, themonth, withyear=True)

Return a month's calendar as an HTML table. If *withyear* is true the year will be included in the header, otherwise just the month name will be used.

</div>

<div class="method">

formatyear(theyear, width=3)

Return a year's calendar as an HTML table. *width* (defaulting to 3) specifies the number of months per row.

</div>

<div class="method">

formatyearpage(theyear, width=3, css='calendar.css', encoding=None)

Return a year's calendar as a complete HTML page. *width* (defaulting to 3) specifies the number of months per row. *css* is the name for the cascading style sheet to be used. `None` can be passed if no style sheet should be used. *encoding* specifies the encoding to be used for the output (defaulting to `'utf-8'`).

</div>

<div class="method">

formatmonthname(theyear, themonth, withyear=True)

Return a month name as an HTML table row. If *withyear* is true the year will be included in the row, otherwise just the month name will be used.

</div>

`!HTMLCalendar` has the following attributes you can override to customize the CSS classes used by the calendar:

<div class="attribute">

cssclasses

A list of CSS classes used for each weekday. The default class list is:

    cssclasses = ["mon", "tue", "wed", "thu", "fri", "sat", "sun"]

more styles can be added for each day:

    cssclasses = ["mon text-bold", "tue", "wed", "thu", "fri", "sat", "sun red"]

Note that the length of this list must be seven items.

</div>

<div class="attribute">

cssclass_noday

The CSS class for a weekday occurring in the previous or coming month.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

cssclasses_weekday_head

A list of CSS classes used for weekday names in the header row. The default is the same as `cssclasses`.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

cssclass_month_head

The month's head CSS class (used by `formatmonthname`). The default value is `"month"`.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

cssclass_month

The CSS class for the whole month's table (used by `formatmonth`). The default value is `"month"`.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

cssclass_year

The CSS class for the whole year's table of tables (used by `formatyear`). The default value is `"year"`.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

cssclass_year_head

The CSS class for the table head for the whole year (used by `formatyear`). The default value is `"year"`.

<div class="versionadded">

3.7

</div>

</div>

Note that although the naming for the above described class attributes is singular (e.g. `cssclass_month` `cssclass_noday`), one can replace the single CSS class with a space separated list of CSS classes, for example:

    "text-bold text-red"

Here is an example how `!HTMLCalendar` can be customized:

    class CustomHTMLCal(calendar.HTMLCalendar):
        cssclasses = [style + " text-nowrap" for style in
                      calendar.HTMLCalendar.cssclasses]
        cssclass_month_head = "text-center month-head"
        cssclass_month = "text-center month"
        cssclass_year = "text-italic lead"

</div>

<div class="LocaleTextCalendar(firstweekday=0, locale=None)">

This subclass of `TextCalendar` can be passed a locale name in the constructor and will return month and weekday names in the specified locale.

</div>

<div class="LocaleHTMLCalendar(firstweekday=0, locale=None)">

This subclass of `HTMLCalendar` can be passed a locale name in the constructor and will return month and weekday names in the specified locale.

</div>

> [!NOTE]
> The constructor, `!formatweekday` and `!formatmonthname` methods of these two classes temporarily change the `LC_TIME` locale to the given *locale*. Because the current locale is a process-wide setting, they are not thread-safe.

For simple text calendars this module provides the following functions.

<div class="function">

setfirstweekday(firstweekday)

Sets the weekday (`0` is Monday, `6` is Sunday) to start each week. The values `MONDAY`, `TUESDAY`, `WEDNESDAY`, `THURSDAY`, `FRIDAY`, `SATURDAY`, and `SUNDAY` are provided for convenience. For example, to set the first weekday to Sunday:

    import calendar
    calendar.setfirstweekday(calendar.SUNDAY)

</div>

<div class="function">

firstweekday()

Returns the current setting for the weekday to start each week.

</div>

<div class="function">

isleap(year)

Returns `True` if *year* is a leap year, otherwise `False`.

</div>

<div class="function">

leapdays(y1, y2)

Returns the number of leap years in the range from *y1* to *y2* (exclusive), where *y1* and *y2* are years.

This function works for ranges spanning a century change.

</div>

<div class="function">

weekday(year, month, day)

Returns the day of the week (`0` is Monday) for *year* (`1970`--...), *month* (`1`--`12`), *day* (`1`--`31`).

</div>

<div class="function">

weekheader(width)

Return a header containing abbreviated weekday names. *width* specifies the width in characters for one weekday.

</div>

<div class="function">

monthrange(year, month)

Returns weekday of first day of the month and number of days in month, for the specified *year* and *month*.

</div>

<div class="function">

monthcalendar(year, month)

Returns a matrix representing a month's calendar. Each row represents a week; days outside of the month are represented by zeros. Each week begins with Monday unless set by `setfirstweekday`.

</div>

<div class="function">

prmonth(theyear, themonth, w=0, l=0)

Prints a month's calendar as returned by `month`.

</div>

<div class="function">

month(theyear, themonth, w=0, l=0)

Returns a month's calendar in a multi-line string using the `~TextCalendar.formatmonth` of the `TextCalendar` class.

</div>

<div class="function">

prcal(theyear, w=0, l=0, c=6, m=3)

Prints the calendar for an entire year as returned by `~calendar.calendar`.

</div>

<div class="function">

calendar(theyear, w=2, l=1, c=6, m=3)

Returns a 3-column calendar for an entire year as a multi-line string using the `~TextCalendar.formatyear` of the `TextCalendar` class.

</div>

<div class="function">

timegm(tuple)

An unrelated but handy function that takes a time tuple such as returned by the `~time.gmtime` function in the `time` module, and returns the corresponding Unix timestamp value, assuming an epoch of 1970, and the POSIX encoding. In fact, `time.gmtime` and `timegm` are each other's inverse.

</div>

The `!calendar` module exports the following data attributes:

<div class="data">

day_name

A sequence that represents the days of the week in the current locale, where Monday is day number 0.

> \>\>\> import calendar \>\>\> list(calendar.day_name) \['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday'\]

</div>

<div class="data">

day_abbr

A sequence that represents the abbreviated days of the week in the current locale, where Mon is day number 0.

> \>\>\> import calendar \>\>\> list(calendar.day_abbr) \['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'\]

</div>

<div class="data">

MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY SUNDAY

Aliases for the days of the week, where `MONDAY` is `0` and `SUNDAY` is `6`.

<div class="versionadded">

3.12

</div>

</div>

<div class="Day">

Enumeration defining days of the week as integer constants. The members of this enumeration are exported to the module scope as `MONDAY` through `SUNDAY`.

<div class="versionadded">

3.12

</div>

</div>

<div class="data">

month_name

A sequence that represents the months of the year in the current locale. This follows normal convention of January being month number 1, so it has a length of 13 and `month_name[0]` is the empty string.

> \>\>\> import calendar \>\>\> list(calendar.month_name) \['', 'January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'\]

> [!CAUTION]
> In locales with alternative month names forms, the `!month_name` sequence may not be suitable when a month name stands by itself and not as part of a date. For instance, in Greek and in many Slavic and Baltic languages, `!month_name` will produce the month in genitive case. Use `standalone_month_name` for a form suitable for standalone use.

</div>

<div class="data">

month_abbr

A sequence that represents the abbreviated months of the year in the current locale. This follows normal convention of January being month number 1, so it has a length of 13 and `month_abbr[0]` is the empty string.

> \>\>\> import calendar \>\>\> list(calendar.month_abbr) \['', 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'\]

> [!CAUTION]
> In locales with alternative month names forms, the `!month_abbr` sequence may not be suitable when a month name stands by itself and not as part of a date. Use `standalone_month_abbr` for a form suitable for standalone use.

</div>

<div class="data">

standalone_month_name

A sequence that represents the months of the year in the current locale in the standalone form if the locale provides one. Else it is equivalent to `month_name`.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

standalone_month_abbr

A sequence that represents the abbreviated months of the year in the current locale in the standalone form if the locale provides one. Else it is equivalent to `month_abbr`.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

JANUARY FEBRUARY MARCH APRIL MAY JUNE JULY AUGUST SEPTEMBER OCTOBER NOVEMBER DECEMBER

Aliases for the months of the year, where `JANUARY` is `1` and `DECEMBER` is `12`.

<div class="versionadded">

3.12

</div>

</div>

<div class="Month">

Enumeration defining months of the year as integer constants. The members of this enumeration are exported to the module scope as `JANUARY` through `DECEMBER`.

<div class="versionadded">

3.12

</div>

</div>

The `!calendar` module defines the following exceptions:

<div class="exception">

IllegalMonthError(month)

A subclass of `ValueError` and `IndexError`, raised when the given month number is outside of the range 1-12 (inclusive).

<div class="versionchanged">

3.12 `IllegalMonthError` is now also a subclass of `ValueError`. New code should avoid catching `IndexError`.

</div>

<div class="attribute">

month

The invalid month number.

</div>

</div>

<div class="exception">

IllegalWeekdayError(weekday)

A subclass of `ValueError`, raised when the given weekday number is outside of the range 0-6 (inclusive).

<div class="attribute">

weekday

The invalid weekday number.

</div>

</div>

<div class="seealso">

Module `datetime`  
Object-oriented interface to dates and times with similar functionality to the `time` module.

Module `time`  
Low-level time related functions.

</div>

## Command-line usage

<div class="versionadded">

2.5

</div>

The `!calendar` module can be executed as a script from the command line to interactively print a calendar.

```
python -m calendar [-h] [-L LOCALE] [-e ENCODING] [-t {text,html}]
                   [-w WIDTH] [-l LINES] [-s SPACING] [-m MONTHS] [-c CSS]
                   [-f FIRST_WEEKDAY] [year] [month]
```

For example, to print a calendar for the year 2000:

```
$ python -m calendar 2000
                                  2000

      January                   February                   March
Mo Tu We Th Fr Sa Su      Mo Tu We Th Fr Sa Su      Mo Tu We Th Fr Sa Su
                1  2          1  2  3  4  5  6             1  2  3  4  5
 3  4  5  6  7  8  9       7  8  9 10 11 12 13       6  7  8  9 10 11 12
10 11 12 13 14 15 16      14 15 16 17 18 19 20      13 14 15 16 17 18 19
17 18 19 20 21 22 23      21 22 23 24 25 26 27      20 21 22 23 24 25 26
24 25 26 27 28 29 30      28 29                     27 28 29 30 31
31

       April                      May                       June
Mo Tu We Th Fr Sa Su      Mo Tu We Th Fr Sa Su      Mo Tu We Th Fr Sa Su
                1  2       1  2  3  4  5  6  7                1  2  3  4
 3  4  5  6  7  8  9       8  9 10 11 12 13 14       5  6  7  8  9 10 11
10 11 12 13 14 15 16      15 16 17 18 19 20 21      12 13 14 15 16 17 18
17 18 19 20 21 22 23      22 23 24 25 26 27 28      19 20 21 22 23 24 25
24 25 26 27 28 29 30      29 30 31                  26 27 28 29 30

        July                     August                  September
Mo Tu We Th Fr Sa Su      Mo Tu We Th Fr Sa Su      Mo Tu We Th Fr Sa Su
                1  2          1  2  3  4  5  6                   1  2  3
 3  4  5  6  7  8  9       7  8  9 10 11 12 13       4  5  6  7  8  9 10
10 11 12 13 14 15 16      14 15 16 17 18 19 20      11 12 13 14 15 16 17
17 18 19 20 21 22 23      21 22 23 24 25 26 27      18 19 20 21 22 23 24
24 25 26 27 28 29 30      28 29 30 31               25 26 27 28 29 30
31

      October                   November                  December
Mo Tu We Th Fr Sa Su      Mo Tu We Th Fr Sa Su      Mo Tu We Th Fr Sa Su
                   1             1  2  3  4  5                   1  2  3
 2  3  4  5  6  7  8       6  7  8  9 10 11 12       4  5  6  7  8  9 10
 9 10 11 12 13 14 15      13 14 15 16 17 18 19      11 12 13 14 15 16 17
16 17 18 19 20 21 22      20 21 22 23 24 25 26      18 19 20 21 22 23 24
23 24 25 26 27 28 29      27 28 29 30               25 26 27 28 29 30 31
30 31
```

The following options are accepted:

<div class="program">

calendar

</div>

<div class="option">

`--help, -h`  

Show the help message and exit.

</div>

<div class="option">

`--locale LOCALE, -L LOCALE`  

The locale to use for month and weekday names. Defaults to English.

</div>

<div class="option">

`--encoding ENCODING, -e ENCODING`  

The encoding to use for output. `--encoding` is required if `--locale` is set.

</div>

<div class="option">

--type {text,html}, -t {text,html}

Print the calendar to the terminal as text, or as an HTML document.

</div>

<div class="option">

`--first-weekday FIRST_WEEKDAY, -f FIRST_WEEKDAY`  

The weekday to start each week. Must be a number between 0 (Monday) and 6 (Sunday). Defaults to 0.

<div class="versionadded">

3.13

</div>

</div>

<div class="option">

year

The year to print the calendar for. Defaults to the current year.

</div>

<div class="option">

month

The month of the specified `year` to print the calendar for. Must be a number between 1 and 12. Defaults to printing a calendar for the full year.

</div>

*Text-mode options:*

<div class="option">

`--width WIDTH, -w WIDTH`  

The width of the date column in terminal columns. The date is printed centred in the column. Any value lower than 2 is ignored. Defaults to 2.

</div>

<div class="option">

`--lines LINES, -l LINES`  

The number of lines for each week in terminal rows. The date is printed top-aligned. Any value lower than 1 is ignored. Defaults to 1.

</div>

<div class="option">

`--spacing SPACING, -s SPACING`  

The space between months in columns. Any value lower than 2 is ignored. Defaults to 6.

</div>

<div class="option">

`--months MONTHS, -m MONTHS`  

The number of months printed per row. Defaults to 3.

</div>

<div class="versionchanged">

3.14 By default, today's date is highlighted in color and can be `controlled using environment variables <using-on-controlling-color>`.

</div>

<div class="versionchanged">

3.15 By default, the month is now also highlighted in color, and the days of the week are also in color. This behavior can be `controlled using environment variables <using-on-controlling-color>`.

</div>

*HTML-mode options:*

<div class="option">

`--css CSS, -c CSS`  

The path of a CSS stylesheet to use for the calendar. This must either be relative to the generated HTML, or an absolute HTTP or `file:///` URL.

</div>

# `!cgi` --- Common Gateway Interface support

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

cgi

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

A fork of the module on PyPI can be used instead: `legacy-cgi`. This is a copy of the cgi module, no longer maintained or supported by the core Python team.

The last version of Python that provided the `!cgi` module was [Python 3.12](https://docs.python.org/3.12/library/cgi.html).

# `!cgitb` --- Traceback manager for CGI scripts

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

cgitb

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

A fork of the module on PyPI can now be used instead: `legacy-cgi`. This is a copy of the cgi module, no longer maintained or supported by the core Python team.

The last version of Python that provided the `!cgitb` module was [Python 3.12](https://docs.python.org/3.12/library/cgitb.html).

# `!chunk` --- Read IFF chunked data

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

chunk

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!chunk` module was [Python 3.12](https://docs.python.org/3.12/library/chunk.html).

# `!cmath` --- Mathematical functions for complex numbers

<div class="module" data-synopsis="Mathematical functions for complex numbers.">

cmath

</div>

------------------------------------------------------------------------

This module provides access to mathematical functions for complex numbers. The functions in this module accept integers, floating-point numbers or complex numbers as arguments. They will also accept any Python object that has either a `~object.__complex__` or a `~object.__float__` method: these methods are used to convert the object to a complex or floating-point number, respectively, and the function is then applied to the result of the conversion.

> [!NOTE]
> For functions involving branch cuts, we have the problem of deciding how to define those functions on the cut itself. Following Kahan's "Branch cuts for complex elementary functions" paper, as well as Annex G of C99 and later C standards, we use the sign of zero to distinguish one side of the branch cut from the other: for a branch cut along (a portion of) the real axis we look at the sign of the imaginary part, while for a branch cut along the imaginary axis we look at the sign of the real part.
>
> For example, the `cmath.sqrt` function has a branch cut along the negative real axis. An argument of `-2-0j` is treated as though it lies *below* the branch cut, and so gives a result on the negative imaginary axis:
>
>     >>> cmath.sqrt(-2-0j)
>     -1.4142135623730951j
>
> But an argument of `-2+0j` is treated as though it lies above the branch cut:
>
>     >>> cmath.sqrt(-2+0j)
>     1.4142135623730951j

<table>
<tbody>
<tr>
<td colspan="2"><strong>Conversions to and from polar coordinates</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">phase(z) &lt;phase&gt;</code></td>
<td>Return the phase of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">polar(z) &lt;polar&gt;</code></td>
<td>Return the representation of <em>z</em> in polar coordinates</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">rect(r, phi) &lt;rect&gt;</code></td>
<td>Return the complex number <em>z</em> with polar coordinates <em>r</em> and <em>phi</em></td>
</tr>
<tr>
<td colspan="2"><strong>Power and logarithmic functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">exp(z) &lt;exp&gt;</code></td>
<td>Return <em>e</em> raised to the power <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">log(z[, base]) &lt;log&gt;</code></td>
<td>Return the logarithm of <em>z</em> to the given <em>base</em> (<em>e</em> by default)</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">log10(z) &lt;log10&gt;</code></td>
<td>Return the base-10 logarithm of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">sqrt(z) &lt;sqrt&gt;</code></td>
<td>Return the square root of <em>z</em></td>
</tr>
<tr>
<td colspan="2"><strong>Trigonometric functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">acos(z) &lt;acos&gt;</code></td>
<td>Return the arc cosine of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">asin(z) &lt;asin&gt;</code></td>
<td>Return the arc sine of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">atan(z) &lt;atan&gt;</code></td>
<td>Return the arc tangent of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">cos(z) &lt;cos&gt;</code></td>
<td>Return the cosine of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">sin(z) &lt;sin&gt;</code></td>
<td>Return the sine of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">tan(z) &lt;tan&gt;</code></td>
<td>Return the tangent of <em>z</em></td>
</tr>
<tr>
<td colspan="2"><strong>Hyperbolic functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">acosh(z) &lt;acosh&gt;</code></td>
<td>Return the inverse hyperbolic cosine of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">asinh(z) &lt;asinh&gt;</code></td>
<td>Return the inverse hyperbolic sine of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">atanh(z) &lt;atanh&gt;</code></td>
<td>Return the inverse hyperbolic tangent of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">cosh(z) &lt;cosh&gt;</code></td>
<td>Return the hyperbolic cosine of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">sinh(z) &lt;sinh&gt;</code></td>
<td>Return the hyperbolic sine of <em>z</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">tanh(z) &lt;tanh&gt;</code></td>
<td>Return the hyperbolic tangent of <em>z</em></td>
</tr>
<tr>
<td colspan="2"><strong>Classification functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isfinite(z) &lt;isfinite&gt;</code></td>
<td>Check if all components of <em>z</em> are finite</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isinf(z) &lt;isinf&gt;</code></td>
<td>Check if any component of <em>z</em> is infinite</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isnan(z) &lt;isnan&gt;</code></td>
<td>Check if any component of <em>z</em> is a NaN</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isclose(a, b, *, rel_tol, abs_tol) &lt;isclose&gt;</code></td>
<td>Check if the values <em>a</em> and <em>b</em> are close to each other</td>
</tr>
<tr>
<td colspan="2"><strong>Constants</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">pi</code></td>
<td><em>π</em> = 3.141592...</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">e</code></td>
<td><em>e</em> = 2.718281...</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">tau</code></td>
<td><em>τ</em> = 2<em>π</em> = 6.283185...</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">inf</code></td>
<td>Positive infinity</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">infj</code></td>
<td>Pure imaginary infinity</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">nan</code></td>
<td>"Not a number" (NaN)</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">nanj</code></td>
<td>Pure imaginary NaN</td>
</tr>
</tbody>
</table>

## Conversions to and from polar coordinates

A Python complex number `z` is stored internally using *rectangular* or *Cartesian* coordinates. It is completely determined by its *real part* `z.real` and its *imaginary part* `z.imag`.

*Polar coordinates* give an alternative way to represent a complex number. In polar coordinates, a complex number *z* is defined by the modulus *r* and the phase angle *phi*. The modulus *r* is the distance from *z* to the origin, while the phase *phi* is the counterclockwise angle, measured in radians, from the positive x-axis to the line segment that joins the origin to *z*.

The following functions can be used to convert from the native rectangular coordinates to polar coordinates and back.

<div class="function">

phase(z)

Return the phase of *z* (also known as the *argument* of *z*), as a float. `phase(z)` is equivalent to `math.atan2(z.imag, z.real)`. The result lies in the range \[-*π*, *π*\], and the branch cut for this operation lies along the negative real axis. The sign of the result is the same as the sign of `z.imag`, even when `z.imag` is zero:

    >>> phase(-1+0j)
    3.141592653589793
    >>> phase(-1-0j)
    -3.141592653589793

</div>

> [!NOTE]
> The modulus (absolute value) of a complex number *z* can be computed using the built-in `abs` function. There is no separate `!cmath` module function for this operation.

<div class="function">

polar(z)

Return the representation of *z* in polar coordinates. Returns a pair `(r, phi)` where *r* is the modulus of *z* and *phi* is the phase of *z*. `polar(z)` is equivalent to `(abs(z), phase(z))`.

</div>

<div class="function">

rect(r, phi)

Return the complex number *z* with polar coordinates *r* and *phi*. Equivalent to `complex(r * math.cos(phi), r * math.sin(phi))`.

</div>

## Power and logarithmic functions

<div class="function">

exp(z)

Return *e* raised to the power *z*, where *e* is the base of natural logarithms.

</div>

<div class="function">

log(z\[, base\])

Return the logarithm of *z* to the given *base*. If the *base* is not specified, returns the natural logarithm of *z*. There is one branch cut, from 0 along the negative real axis to -∞.

</div>

<div class="function">

log10(z)

Return the base-10 logarithm of *z*. This has the same branch cut as `log`.

</div>

<div class="function">

sqrt(z)

Return the square root of *z*. This has the same branch cut as `log`.

</div>

## Trigonometric functions

<div class="function">

acos(z)

Return the arc cosine of *z*. There are two branch cuts: One extends right from 1 along the real axis to ∞. The other extends left from -1 along the real axis to -∞.

</div>

<div class="function">

asin(z)

Return the arc sine of *z*. This has the same branch cuts as `acos`.

</div>

<div class="function">

atan(z)

Return the arc tangent of *z*. There are two branch cuts: One extends from `1j` along the imaginary axis to `∞j`. The other extends from `-1j` along the imaginary axis to `-∞j`.

</div>

<div class="function">

cos(z)

Return the cosine of *z*.

</div>

<div class="function">

sin(z)

Return the sine of *z*.

</div>

<div class="function">

tan(z)

Return the tangent of *z*.

</div>

## Hyperbolic functions

<div class="function">

acosh(z)

Return the inverse hyperbolic cosine of *z*. There is one branch cut, extending left from 1 along the real axis to -∞.

</div>

<div class="function">

asinh(z)

Return the inverse hyperbolic sine of *z*. There are two branch cuts: One extends from `1j` along the imaginary axis to `∞j`. The other extends from `-1j` along the imaginary axis to `-∞j`.

</div>

<div class="function">

atanh(z)

Return the inverse hyperbolic tangent of *z*. There are two branch cuts: One extends from `1` along the real axis to `∞`. The other extends from `-1` along the real axis to `-∞`.

</div>

<div class="function">

cosh(z)

Return the hyperbolic cosine of *z*.

</div>

<div class="function">

sinh(z)

Return the hyperbolic sine of *z*.

</div>

<div class="function">

tanh(z)

Return the hyperbolic tangent of *z*.

</div>

## Classification functions

<div class="function">

isfinite(z)

Return `True` if both the real and imaginary parts of *z* are finite, and `False` otherwise.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

isinf(z)

Return `True` if either the real or the imaginary part of *z* is an infinity, and `False` otherwise.

</div>

<div class="function">

isnan(z)

Return `True` if either the real or the imaginary part of *z* is a NaN, and `False` otherwise.

</div>

<div class="function">

isclose(a, b, \*, rel_tol=1e-09, abs_tol=0.0)

Return `True` if the values *a* and *b* are close to each other and `False` otherwise.

Whether or not two values are considered close is determined according to given absolute and relative tolerances. If no errors occur, the result will be: `abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)`.

*rel_tol* is the relative tolerance -- it is the maximum allowed difference between *a* and *b*, relative to the larger absolute value of *a* or *b*. For example, to set a tolerance of 5%, pass `rel_tol=0.05`. The default tolerance is `1e-09`, which assures that the two values are the same within about 9 decimal digits. *rel_tol* must be nonnegative and less than `1.0`.

*abs_tol* is the absolute tolerance; it defaults to `0.0` and it must be nonnegative. When comparing `x` to `0.0`, `isclose(x, 0)` is computed as `abs(x) <= rel_tol  * abs(x)`, which is `False` for any `x` and rel_tol less than `1.0`. So add an appropriate positive abs_tol argument to the call.

The IEEE 754 special values of `NaN`, `inf`, and `-inf` will be handled according to IEEE rules. Specifically, `NaN` is not considered close to any other value, including `NaN`. `inf` and `-inf` are only considered close to themselves.

<div class="versionadded">

3.5

</div>

<div class="seealso">

`485` -- A function for testing approximate equality

</div>

</div>

## Constants

<div class="data">

pi

The mathematical constant *π*, as a float.

</div>

<div class="data">

e

The mathematical constant *e*, as a float.

</div>

<div class="data">

tau

The mathematical constant *τ*, as a float.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

inf

Floating-point positive infinity. Equivalent to `float('inf')`.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

infj

Complex number with zero real part and positive infinity imaginary part. Equivalent to `complex(0.0, float('inf'))`.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

nan

A floating-point "not a number" (NaN) value. Equivalent to `float('nan')`. See also `math.nan`.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

nanj

Complex number with zero real part and NaN imaginary part. Equivalent to `complex(0.0, float('nan'))`.

<div class="versionadded">

3.6

</div>

</div>

<div class="index">

pair: module; math

</div>

Note that the selection of functions is similar, but not identical, to that in module `math`. The reason for having two modules is that some users aren't interested in complex numbers, and perhaps don't even know what they are. They would rather have `math.sqrt(-1)` raise an exception than return a complex number. Also note that the functions defined in `!cmath` always return a complex number, even if the answer can be expressed as a real number (in which case the complex number has an imaginary part of zero).

A note on branch cuts: They are curves along which the given function fails to be continuous. They are a necessary feature of many complex functions. It is assumed that if you need to compute with complex functions, you will understand about branch cuts. Consult almost any (not too elementary) book on complex variables for enlightenment. For information of the proper choice of branch cuts for numerical purposes, a good reference should be the following:

<div class="seealso">

Kahan, W: Branch cuts for complex elementary functions; or, Much ado about nothing's sign bit. In Iserles, A., and Powell, M. (eds.), The state of the art in numerical analysis. Clarendon Press (1987) pp165--211.

</div>

# `!cmd` --- Support for line-oriented command interpreters

<div class="module" data-synopsis="Build line-oriented command interpreters.">

cmd

</div>

**Source code:** `Lib/cmd.py`

------------------------------------------------------------------------

The `Cmd` class provides a simple framework for writing line-oriented command interpreters. These are often useful for test harnesses, administrative tools, and prototypes that will later be wrapped in a more sophisticated interface.

A command interpreter built with `Cmd` typically presents the user with an interactive prompt, reads input one line at a time, and dispatches each command to a method named `do_` followed by the command name. For example, the `help` command is handled by the `~Cmd.do_help` method. Apart from the built-in `help` command, most of the interpreter's behavior is defined by the `do_*` methods you add in your own subclass.

<div class="Cmd(completekey='tab', stdin=None, stdout=None)">

A `Cmd` instance or subclass instance is a line-oriented interpreter framework. There is no good reason to instantiate `Cmd` itself; rather, it's useful as a superclass of an interpreter class you define yourself in order to inherit `Cmd`'s methods and encapsulate action methods.

The optional argument *completekey* is the `readline` name of a completion key; it defaults to `Tab`. If *completekey* is not `None` and `readline` is available, command completion is done automatically.

The default, `'tab'`, is treated specially, so that it refers to the `Tab` key on every `readline.backend`. Specifically, if `readline.backend` is `editline`, `Cmd` will use `'^I'` instead of `'tab'`. Note that other values are not treated this way, and might only work with a specific backend.

The optional arguments *stdin* and *stdout* specify the input and output file objects that the Cmd instance or subclass instance will use for input and output. If not specified, they will default to `sys.stdin` and `sys.stdout`.

If you want a given *stdin* to be used, make sure to set the instance's `use_rawinput` attribute to `False`, otherwise *stdin* will be ignored.

<div class="versionchanged">

3.13 `completekey='tab'` is replaced by `'^I'` for `editline`.

</div>

</div>

## Cmd Objects

A `Cmd` instance has the following methods:

<div class="method">

Cmd.cmdloop(intro=None)

Repeatedly issue a prompt, accept input, parse an initial prefix off the received input, and dispatch to action methods, passing them the remainder of the line as argument.

The optional argument is a banner or intro string to be issued before the first prompt (this overrides the `intro` class attribute).

If the `readline` module is loaded, input will automatically inherit `bash`-like history-list editing (e.g. `Control-P` scrolls back to the last command, `Control-N` forward to the next one, `Control-F` moves the cursor to the right non-destructively, `Control-B` moves the cursor to the left non-destructively, etc.).

An end-of-file on input is passed back as the string `'EOF'`.

<div class="index">

single: ? (question mark); in a command interpreter single: ! (exclamation); in a command interpreter

</div>

An interpreter instance will recognize a command name `foo` if and only if it has a method `!do_foo`. As a special case, a line beginning with the character `'?'` is dispatched to the method `do_help`. As another special case, a line beginning with the character `'!'` is dispatched to the method `!do_shell` (if such a method is defined).

This method will return when the `postcmd` method returns a true value. The *stop* argument to `postcmd` is the return value from the command's corresponding `!do_\*` method.

If completion is enabled, completing commands will be done automatically, and completing of commands args is done by calling `!complete_foo` with arguments *text*, *line*, *begidx*, and *endidx*. *text* is the string prefix we are attempting to match: all returned matches must begin with it. *line* is the current input line with leading whitespace removed, *begidx* and *endidx* are the beginning and ending indexes of the prefix text, which could be used to provide different completion depending upon which position the argument is in.

</div>

<div class="method">

Cmd.do_help(arg)

All subclasses of `Cmd` inherit a predefined `!do_help`. This method, called with an argument `'bar'`, invokes the corresponding method `!help_bar`, and if that is not present, prints the docstring of `!do_bar`, if available. With no argument, `!do_help` lists all available help topics (that is, all commands with corresponding `!help_\*` methods or commands that have docstrings), and also lists any undocumented commands.

</div>

<div class="method">

Cmd.onecmd(str)

Interpret the argument as though it had been typed in response to the prompt. This may be overridden, but should not normally need to be; see the `precmd` and `postcmd` methods for useful execution hooks. The return value is a flag indicating whether interpretation of commands by the interpreter should stop. If there is a `!do_\*` method for the command *str*, the return value of that method is returned, otherwise the return value from the `default` method is returned.

</div>

<div class="method">

Cmd.emptyline()

Method called when an empty line is entered in response to the prompt. If this method is not overridden, it repeats the last nonempty command entered.

</div>

<div class="method">

Cmd.default(line)

Method called on an input line when the command prefix is not recognized. If this method is not overridden, it prints an error message and returns.

</div>

<div class="method">

Cmd.completedefault(text, line, begidx, endidx)

Method called to complete an input line when no command-specific `!complete_\*` method is available. By default, it returns an empty list.

</div>

<div class="method">

Cmd.columnize(list, displaywidth=80)

Method called to display a list of strings as a compact set of columns. Each column is only as wide as necessary. Columns are separated by two spaces for readability.

</div>

<div class="method">

Cmd.precmd(line)

Hook method executed just before the command line *line* is interpreted, but after the input prompt is generated and issued. This method is a stub in `Cmd`; it exists to be overridden by subclasses. The return value is used as the command which will be executed by the `onecmd` method; the `precmd` implementation may re-write the command or simply return *line* unchanged.

</div>

<div class="method">

Cmd.postcmd(stop, line)

Hook method executed just after a command dispatch is finished. This method is a stub in `Cmd`; it exists to be overridden by subclasses. *line* is the command line which was executed, and *stop* is a flag which indicates whether execution will be terminated after the call to `postcmd`; this will be the return value of the `onecmd` method. The return value of this method will be used as the new value for the internal flag which corresponds to *stop*; returning false will cause interpretation to continue.

</div>

<div class="method">

Cmd.preloop()

Hook method executed once when `cmdloop` is called. This method is a stub in `Cmd`; it exists to be overridden by subclasses.

</div>

<div class="method">

Cmd.postloop()

Hook method executed once when `cmdloop` is about to return. This method is a stub in `Cmd`; it exists to be overridden by subclasses.

</div>

Instances of `Cmd` subclasses have some public instance variables:

<div class="attribute">

Cmd.prompt

The prompt issued to solicit input.

</div>

<div class="attribute">

Cmd.identchars

The string of characters accepted for the command prefix.

</div>

<div class="attribute">

Cmd.lastcmd

The last nonempty command prefix seen.

</div>

<div class="attribute">

Cmd.cmdqueue

A list of queued input lines. The cmdqueue list is checked in `cmdloop` when new input is needed; if it is nonempty, its elements will be processed in order, as if entered at the prompt.

</div>

<div class="attribute">

Cmd.intro

A string to issue as an intro or banner. May be overridden by giving the `cmdloop` method an argument.

</div>

<div class="attribute">

Cmd.doc_header

The header to issue if the help output has a section for documented commands.

</div>

<div class="attribute">

Cmd.misc_header

The header to issue if the help output has a section for miscellaneous help topics (that is, there are `!help_\*` methods without corresponding `!do_\*` methods).

</div>

<div class="attribute">

Cmd.undoc_header

The header to issue if the help output has a section for undocumented commands (that is, there are `!do_\*` methods without corresponding `!help_\*` methods).

</div>

<div class="attribute">

Cmd.ruler

The character used to draw separator lines under the help-message headers. If empty, no ruler line is drawn. It defaults to `'='`.

</div>

<div class="attribute">

Cmd.use_rawinput

A flag, defaulting to true. If true, `cmdloop` uses `input` to display a prompt and read the next command; if false, `sys.stdout.write() <sys.stdout>` and `sys.stdin.readline() <sys.stdin>` are used. (This means that by importing `readline`, on systems that support it, the interpreter will automatically support `Emacs`-like line editing and command-history keystrokes.)

</div>

## Cmd Example

The `!cmd` module is mainly useful for building custom shells that let a user work with a program interactively.

This section presents a simple example of how to build a shell around a few of the commands in the `turtle` module.

Basic turtle commands such as `~turtle.forward` are added to a `Cmd` subclass with method named `!do_forward`. The argument is converted to a number and dispatched to the turtle module. The docstring is used in the help utility provided by the shell.

The example also includes a basic record and playback facility implemented with the `~Cmd.precmd` method which is responsible for converting the input to lowercase and writing the commands to a file. The `!do_playback` method reads the file and adds the recorded commands to the `~Cmd.cmdqueue` for immediate playback:

    import cmd, sys
    from turtle import *

    class TurtleShell(cmd.Cmd):
        intro = 'Welcome to the turtle shell.   Type help or ? to list commands.\n'
        prompt = '(turtle) '
        file = None

        # ----- basic turtle commands -----
        def do_forward(self, arg):
            'Move the turtle forward by the specified distance:  FORWARD 10'
            forward(*parse(arg))
        def do_right(self, arg):
            'Turn turtle right by given number of degrees:  RIGHT 20'
            right(*parse(arg))
        def do_left(self, arg):
            'Turn turtle left by given number of degrees:  LEFT 90'
            left(*parse(arg))
        def do_goto(self, arg):
            'Move turtle to an absolute position with changing orientation.  GOTO 100 200'
            goto(*parse(arg))
        def do_home(self, arg):
            'Return turtle to the home position:  HOME'
            home()
        def do_circle(self, arg):
            'Draw circle with given radius an options extent and steps:  CIRCLE 50'
            circle(*parse(arg))
        def do_position(self, arg):
            'Print the current turtle position:  POSITION'
            print('Current position is %d %d\n' % position())
        def do_heading(self, arg):
            'Print the current turtle heading in degrees:  HEADING'
            print('Current heading is %d\n' % (heading(),))
        def do_color(self, arg):
            'Set the color:  COLOR BLUE'
            color(arg.lower())
        def do_undo(self, arg):
            'Undo (repeatedly) the last turtle action(s):  UNDO'
        def do_reset(self, arg):
            'Clear the screen and return turtle to center:  RESET'
            reset()
        def do_bye(self, arg):
            'Stop recording, close the turtle window, and exit:  BYE'
            print('Thank you for using Turtle')
            self.close()
            bye()
            return True

        # ----- record and playback -----
        def do_record(self, arg):
            'Save future commands to filename:  RECORD rose.cmd'
            self.file = open(arg, 'w')
        def do_playback(self, arg):
            'Playback commands from a file:  PLAYBACK rose.cmd'
            self.close()
            with open(arg) as f:
                self.cmdqueue.extend(f.read().splitlines())
        def precmd(self, line):
            line = line.lower()
            if self.file and 'playback' not in line:
                print(line, file=self.file)
            return line
        def close(self):
            if self.file:
                self.file.close()
                self.file = None

    def parse(arg):
        'Convert a series of zero or more numbers to an argument tuple'
        return tuple(map(int, arg.split()))

    if __name__ == '__main__':
        TurtleShell().cmdloop()

Here is a sample session with the turtle shell showing the help functions, using blank lines to repeat commands, and the simple record and playback facility:

```
Welcome to the turtle shell.   Type help or ? to list commands.

(turtle) ?

Documented commands (type help <topic>):
========================================
bye     color    goto     home  playback  record  right
circle  forward  heading  left  position  reset   undo

(turtle) help forward
Move the turtle forward by the specified distance:  FORWARD 10
(turtle) record spiral.cmd
(turtle) position
Current position is 0 0

(turtle) heading
Current heading is 0

(turtle) reset
(turtle) circle 20
(turtle) right 30
(turtle) circle 40
(turtle) right 30
(turtle) circle 60
(turtle) right 30
(turtle) circle 80
(turtle) right 30
(turtle) circle 100
(turtle) right 30
(turtle) circle 120
(turtle) right 30
(turtle) circle 120
(turtle) heading
Current heading is 180

(turtle) forward 100
(turtle)
(turtle) right 90
(turtle) forward 100
(turtle)
(turtle) right 90
(turtle) forward 400
(turtle) right 90
(turtle) forward 500
(turtle) right 90
(turtle) forward 400
(turtle) right 90
(turtle) forward 300
(turtle) playback spiral.cmd
Current position is 0 0

Current heading is 0

Current heading is 180

(turtle) bye
Thank you for using Turtle
```

# Modules command-line interface (CLI)

The following modules have a command-line interface.

- `ast <ast-cli>`
- `asyncio <asyncio-cli>`
- `base64`
- `calendar <calendar-cli>`
- `code`
- `compileall <compileall-cli>`
- `cProfile`: see `profiling.tracing <profiling-tracing-cli>`
- `dis <dis-cli>`
- `doctest <doctest-cli>`
- `!encodings.rot_13`
- `ensurepip <ensurepip-cli>`
- `filecmp`
- `fileinput`
- `ftplib`
- `gzip <gzip-cli>`
- `http.server <http-server-cli>`
- `idlelib <idlelib-cli>`
- `inspect <inspect-module-cli>`
- `json <json-commandline>`
- `mimetypes <mimetypes-cli>`
- `pdb <pdb-cli>`
- `pickle <pickle-cli>`
- `pickletools <pickletools-cli>`
- `platform <platform-cli>`
- `poplib`
- `profiling.sampling <profiling-sampling>`
- `profiling.tracing <profiling-tracing-cli>`
- `pstats <pstats-cli>`
- `py_compile <py_compile-cli>`
- `pyclbr`
- `pydoc`
- `quopri`
- `random <random-cli>`
- `runpy`
- `site <site-commandline>`
- `sqlite3 <sqlite3-cli>`
- `symtable <symtable-cli>`
- `sysconfig <sysconfig-cli>`
- `tabnanny`
- `tarfile <tarfile-commandline>`
- `!this`
- `timeit <timeit-command-line-interface>`
- `tokenize <tokenize-cli>`
- `trace <trace-cli>`
- `turtledemo`
- `unittest <unittest-command-line-interface>`
- `uuid <uuid-cli>`
- `venv <venv-cli>`
- `webbrowser <webbrowser-cli>`
- `zipapp <zipapp-command-line-interface>`
- `zipfile <zipfile-commandline>`

See also the `Python command-line interface <using-on-general>`.

# Command-line interface libraries

The modules described in this chapter assist with implementing command line and terminal interfaces for applications.

Here's an overview:

<div class="toctree" data-maxdepth="1">

argparse.rst optparse.rst getpass.rst fileinput.rst curses.rst curses.ascii.rst curses.panel.rst cmd.rst

</div>

# `!code` --- Interpreter base classes

<div class="module" data-synopsis="Facilities to implement read-eval-print loops.">

code

</div>

**Source code:** `Lib/code.py`

------------------------------------------------------------------------

The `code` module provides facilities to implement read-eval-print loops in Python. Two classes and convenience functions are included which can be used to build applications which provide an interactive interpreter prompt.

<div class="InteractiveInterpreter(locals=None)">

This class deals with parsing and interpreter state (the user's namespace); it does not deal with input buffering or prompting or input file naming (the filename is always passed in explicitly). The optional *locals* argument specifies a mapping to use as the namespace in which code will be executed; it defaults to a newly created dictionary with key `'__name__'` set to `'__console__'` and key `'__doc__'` set to `None`.

Note that functions and classes objects created under an `!InteractiveInterpreter` instance will belong to the namespace specified by *locals*. They are only pickleable if *locals* is the namespace of an existing module.

</div>

<div class="InteractiveConsole(locals=None, filename=&quot;&lt;console&gt;&quot;, *, local_exit=False)">

Closely emulate the behavior of the interactive Python interpreter. This class builds on `InteractiveInterpreter` and adds prompting using the familiar `sys.ps1` and `sys.ps2`, and input buffering. If *local_exit* is true, `exit()` and `quit()` in the console will not raise `SystemExit`, but instead return to the calling code.

<div class="versionchanged">

3.13 Added *local_exit* parameter.

</div>

</div>

<div class="function">

interact(banner=None, readfunc=None, local=None, exitmsg=None, local_exit=False)

Convenience function to run a read-eval-print loop. This creates a new instance of `InteractiveConsole` and sets *readfunc* to be used as the `InteractiveConsole.raw_input` method, if provided. If *local* is provided, it is passed to the `InteractiveConsole` constructor for use as the default namespace for the interpreter loop. If *local_exit* is provided, it is passed to the `InteractiveConsole` constructor. The `~InteractiveConsole.interact` method of the instance is then run with *banner* and *exitmsg* passed as the banner and exit message to use, if provided. The console object is discarded after use.

<div class="versionchanged">

3.6 Added *exitmsg* parameter.

</div>

<div class="versionchanged">

3.13 Added *local_exit* parameter.

</div>

</div>

<div class="function">

compile_command(source, filename="\<input\>", symbol="single")

This function is useful for programs that want to emulate Python's interpreter main loop (a.k.a. the read-eval-print loop). The tricky part is to determine when the user has entered an incomplete command that can be completed by entering more text (as opposed to a complete command or a syntax error). This function *almost* always makes the same decision as the real interpreter main loop.

*source* is the source string; *filename* is the optional filename from which source was read, defaulting to `'<input>'`; and *symbol* is the optional grammar start symbol, which should be `'single'` (the default), `'eval'` or `'exec'`.

Returns a code object (the same as `compile(source, filename, symbol)`) if the command is complete and valid; `None` if the command is incomplete; raises `SyntaxError` if the command is complete and contains a syntax error, or raises `OverflowError` or `ValueError` if the command contains an invalid literal.

</div>

## Interactive Interpreter Objects

<div class="method">

InteractiveInterpreter.runsource(source, filename="\<input\>", symbol="single")

Compile and run some source in the interpreter. Arguments are the same as for `compile_command`; the default for *filename* is `'<input>'`, and for *symbol* is `'single'`. One of several things can happen:

- The input is incorrect; `compile_command` raised an exception (usually `SyntaxError`). A syntax traceback will be printed by calling the `showsyntaxerror` method. `runsource` returns `False`.
- The input is incomplete, and more input is required; `compile_command` returned `None`. `runsource` returns `True`.
- The input is complete; `compile_command` returned a code object. The code is executed by calling the `runcode` (which also handles run-time exceptions, except for `SystemExit`). `runsource` returns `False`.

The return value can be used to decide whether to use `sys.ps1` or `sys.ps2` to prompt the next line.

</div>

<div class="method">

InteractiveInterpreter.runcode(code)

Execute a code object. When an exception occurs, `showtraceback` is called to display a traceback. All exceptions are caught except `SystemExit`, which is allowed to propagate.

A note about `KeyboardInterrupt`: this exception may occur elsewhere in this code, and may not always be caught. The caller should be prepared to deal with it.

</div>

<div class="method">

InteractiveInterpreter.showsyntaxerror(filename=None)

Display the syntax error that just occurred. This does not display a stack trace because there isn't one for syntax errors. If *filename* is given, it is stuffed into the exception instead of the default filename provided by Python's parser, because it always uses `'<string>'` when reading from a string. The output is written by the `write` method.

</div>

<div class="method">

InteractiveInterpreter.showtraceback()

Display the exception that just occurred. We remove the first stack item because it is within the interpreter object implementation. The output is written by the `write` method.

<div class="versionchanged">

3.5 The full chained traceback is displayed instead of just the primary traceback.

</div>

</div>

<div class="method">

InteractiveInterpreter.write(data)

Write a string to the standard error stream (`sys.stderr`). Derived classes should override this to provide the appropriate output handling as needed.

</div>

## Interactive Console Objects

The `InteractiveConsole` class is a subclass of `InteractiveInterpreter`, and so offers all the methods of the interpreter objects as well as the following additions.

<div class="method">

InteractiveConsole.interact(banner=None, exitmsg=None)

Closely emulate the interactive Python console. The optional *banner* argument specify the banner to print before the first interaction; by default it prints a banner similar to the one printed by the standard Python interpreter, followed by the class name of the console object in parentheses (so as not to confuse this with the real interpreter -- since it's so close!).

The optional *exitmsg* argument specifies an exit message printed when exiting. Pass the empty string to suppress the exit message. If *exitmsg* is not given or `None`, a default message is printed.

<div class="versionchanged">

3.4 To suppress printing any banner, pass an empty string.

</div>

<div class="versionchanged">

3.6 Print an exit message when exiting.

</div>

</div>

<div class="method">

InteractiveConsole.push(line)

Push a line of source text to the interpreter. The line should not have a trailing newline; it may have internal newlines. The line is appended to a buffer and the interpreter's `~InteractiveInterpreter.runsource` method is called with the concatenated contents of the buffer as source. If this indicates that the command was executed or invalid, the buffer is reset; otherwise, the command is incomplete, and the buffer is left as it was after the line was appended. The return value is `True` if more input is required, `False` if the line was dealt with in some way (this is the same as `!runsource`).

</div>

<div class="method">

InteractiveConsole.resetbuffer()

Remove any unhandled source text from the input buffer.

</div>

<div class="method">

InteractiveConsole.raw_input(prompt="")

Write a prompt and read a line. The returned line does not include the trailing newline. When the user enters the EOF key sequence, `EOFError` is raised. The base implementation reads from `sys.stdin`; a subclass may replace this with a different implementation.

</div>

# `!codecs` --- Codec registry and base classes

<div class="module" data-synopsis="Encode and decode data and streams.">

codecs

</div>

**Source code:** `Lib/codecs.py`

<div class="index">

single: Unicode single: Codecs pair: Codecs; encode pair: Codecs; decode single: streams pair: stackable; streams

</div>

------------------------------------------------------------------------

This module defines base classes for standard Python codecs (encoders and decoders) and provides access to the internal Python codec registry, which manages the codec and error handling lookup process. Most standard codecs are `text encodings <text encoding>`, which encode text to bytes (and decode bytes to text), but there are also codecs provided that encode text to text, and bytes to bytes. Custom codecs may encode and decode between arbitrary types, but some module features are restricted to be used specifically with `text encodings <text encoding>` or with codecs that encode to `bytes`.

The module defines the following functions for encoding and decoding with any codec:

<div class="function">

encode(obj, encoding='utf-8', errors='strict')

Encodes *obj* using the codec registered for *encoding*.

*Errors* may be given to set the desired error handling scheme. The default error handler is `'strict'` meaning that encoding errors raise `ValueError` (or a more codec specific subclass, such as `UnicodeEncodeError`). Refer to `codec-base-classes` for more information on codec error handling.

</div>

<div class="function">

decode(obj, encoding='utf-8', errors='strict')

Decodes *obj* using the codec registered for *encoding*.

*Errors* may be given to set the desired error handling scheme. The default error handler is `'strict'` meaning that decoding errors raise `ValueError` (or a more codec specific subclass, such as `UnicodeDecodeError`). Refer to `codec-base-classes` for more information on codec error handling.

</div>

<div class="function">

charmap_build(string)

Return a mapping suitable for encoding with a custom single-byte encoding. Given a `str` *string* of up to 256 characters representing a decoding table, returns either a compact internal mapping object `EncodingMap` or a `dictionary <dict>` mapping character ordinals to byte values. Raises a `TypeError` on invalid input.

</div>

The full details for each codec can also be looked up directly:

<div class="function">

lookup(encoding, /)

Looks up the codec info in the Python codec registry and returns a `CodecInfo` object as defined below.

This function first normalizes the *encoding*: all ASCII letters are converted to lower case, spaces are replaced with hyphens. Then encoding is looked up in the registry's cache. If not found, the list of registered search functions is scanned. If no `CodecInfo` object is found, a `LookupError` is raised. Otherwise, the `CodecInfo` object is stored in the cache and returned to the caller.

<div class="versionchanged">

3.9 Any characters except ASCII letters and digits and a dot are converted to underscore.

</div>

<div class="versionchanged">

3.15 No characters are converted to underscore anymore. Spaces are converted to hyphens.

</div>

</div>

<div class="CodecInfo(encode, decode, streamreader=None, streamwriter=None, incrementalencoder=None, incrementaldecoder=None, name=None)">

Codec details when looking up the codec registry. The constructor arguments are stored in attributes of the same name:

<div class="attribute">

name

The name of the encoding.

</div>

<div class="attribute">

encode decode

The stateless encoding and decoding functions. These must be functions or methods which have the same interface as the `~Codec.encode` and `~Codec.decode` methods of Codec instances (see `Codec Interface <codec-objects>`). The functions or methods are expected to work in a stateless mode.

</div>

<div class="attribute">

incrementalencoder incrementaldecoder

Incremental encoder and decoder classes or factory functions. These have to provide the interface defined by the base classes `IncrementalEncoder` and `IncrementalDecoder`, respectively. Incremental codecs can maintain state.

</div>

<div class="attribute">

streamwriter streamreader

Stream writer and reader classes or factory functions. These have to provide the interface defined by the base classes `StreamWriter` and `StreamReader`, respectively. Stream codecs can maintain state.

</div>

</div>

To simplify access to the various codec components, the module provides these additional functions which use `lookup` for the codec lookup:

<div class="function">

getencoder(encoding)

Look up the codec for the given encoding and return its encoder function.

Raises a `LookupError` in case the encoding cannot be found.

</div>

<div class="function">

getdecoder(encoding)

Look up the codec for the given encoding and return its decoder function.

Raises a `LookupError` in case the encoding cannot be found.

</div>

<div class="function">

getincrementalencoder(encoding)

Look up the codec for the given encoding and return its incremental encoder class or factory function.

Raises a `LookupError` in case the encoding cannot be found or the codec doesn't support an incremental encoder.

</div>

<div class="function">

getincrementaldecoder(encoding)

Look up the codec for the given encoding and return its incremental decoder class or factory function.

Raises a `LookupError` in case the encoding cannot be found or the codec doesn't support an incremental decoder.

</div>

<div class="function">

getreader(encoding)

Look up the codec for the given encoding and return its `StreamReader` class or factory function.

Raises a `LookupError` in case the encoding cannot be found.

</div>

<div class="function">

getwriter(encoding)

Look up the codec for the given encoding and return its `StreamWriter` class or factory function.

Raises a `LookupError` in case the encoding cannot be found.

</div>

Custom codecs are made available by registering a suitable codec search function:

<div class="function">

register(search_function, /)

Register a codec search function. Search functions are expected to take one argument, being the encoding name in all lower case letters with spaces converted to hyphens, and return a `CodecInfo` object. In case a search function cannot find a given encoding, it should return `None`.

</div>

<div class="function">

unregister(search_function, /)

Unregister a codec search function and clear the registry's cache. If the search function is not registered, do nothing.

<div class="versionadded">

3.10

</div>

</div>

While the builtin `open` and the associated `io` module are the recommended approach for working with encoded text files, this module provides additional utility functions and classes that allow the use of a wider range of codecs when working with binary files:

<div class="function">

open(filename, mode='r', encoding=None, errors='strict', buffering=-1)

Open an encoded file using the given *mode* and return an instance of `StreamReaderWriter`, providing transparent encoding/decoding. The default file mode is `'r'`, meaning to open the file in read mode.

> [!NOTE]
> If *encoding* is not `None`, then the underlying encoded files are always opened in binary mode. No automatic conversion of `'\n'` is done on reading and writing. The *mode* argument may be any binary mode acceptable to the built-in `open` function; the `'b'` is automatically added.

*encoding* specifies the encoding which is to be used for the file. Any encoding that encodes to and decodes from bytes is allowed, and the data types supported by the file methods depend on the codec used.

*errors* may be given to define the error handling. It defaults to `'strict'` which causes a `ValueError` to be raised in case an encoding error occurs.

*buffering* has the same meaning as for the built-in `open` function. It defaults to -1 which means that the default buffer size will be used.

<div class="versionchanged">

3.11 The `'U'` mode has been removed.

</div>

<div class="deprecated">

3.14

`codecs.open` has been superseded by `open`.

</div>

</div>

<div class="function">

EncodedFile(file, data_encoding, file_encoding=None, errors='strict')

Return a `StreamRecoder` instance, a wrapped version of *file* which provides transparent transcoding. The original file is closed when the wrapped version is closed.

Data written to the wrapped file is decoded according to the given *data_encoding* and then written to the original file as bytes using *file_encoding*. Bytes read from the original file are decoded according to *file_encoding*, and the result is encoded using *data_encoding*.

If *file_encoding* is not given, it defaults to *data_encoding*.

*errors* may be given to define the error handling. It defaults to `'strict'`, which causes `ValueError` to be raised in case an encoding error occurs.

</div>

<div class="function">

iterencode(iterator, encoding, errors='strict', \*\*kwargs)

Uses an incremental encoder to iteratively encode the input provided by *iterator*. *iterator* must yield `str` objects. This function is a `generator`. The *errors* argument (as well as any other keyword argument) is passed through to the incremental encoder.

This function requires that the codec accept text `str` objects to encode. Therefore it does not support bytes-to-bytes encoders such as `base64_codec`.

</div>

<div class="function">

iterdecode(iterator, encoding, errors='strict', \*\*kwargs)

Uses an incremental decoder to iteratively decode the input provided by *iterator*. *iterator* must yield `bytes` objects. This function is a `generator`. The *errors* argument (as well as any other keyword argument) is passed through to the incremental decoder.

This function requires that the codec accept `bytes` objects to decode. Therefore it does not support text-to-text encoders such as `rot_13`, although `rot_13` may be used equivalently with `iterencode`.

</div>

<div class="function">

readbuffer_encode(buffer, errors=None, /)

Return a `tuple` containing the raw bytes of *buffer*, a `buffer-compatible object <bufferobjects>` or `str` (encoded to UTF-8 before processing), and their length in bytes.

The *errors* argument is ignored.

```
>>> codecs.readbuffer_encode(b"Zito")
(b'Zito', 4)
```

</div>

The module also provides the following constants which are useful for reading and writing to platform dependent files:

<div class="data">

BOM BOM_BE BOM_LE BOM_UTF8 BOM_UTF16 BOM_UTF16_BE BOM_UTF16_LE BOM_UTF32 BOM_UTF32_BE BOM_UTF32_LE

These constants define various byte sequences, being Unicode byte order marks (BOMs) for several encodings. They are used in UTF-16 and UTF-32 data streams to indicate the byte order used, and in UTF-8 as a Unicode signature. `BOM_UTF16` is either `BOM_UTF16_BE` or `BOM_UTF16_LE` depending on the platform's native byte order, `BOM` is an alias for `BOM_UTF16`, `BOM_LE` for `BOM_UTF16_LE` and `BOM_BE` for `BOM_UTF16_BE`. The others represent the BOM in UTF-8 and UTF-32 encodings.

</div>

## Codec Base Classes

The `!codecs` module defines a set of base classes which define the interfaces for working with codec objects, and can also be used as the basis for custom codec implementations.

Each codec has to define four interfaces to make it usable as codec in Python: stateless encoder, stateless decoder, stream reader and stream writer. The stream reader and writers typically reuse the stateless encoder/decoder to implement the file protocols. Codec authors also need to define how the codec will handle encoding and decoding errors.

### Error Handlers<span id="surrogateescape"></span>

To simplify and standardize error handling, codecs may implement different error handling schemes by accepting the *errors* string argument:

> \>\>\> 'German ß, ♬'.encode(encoding='ascii', errors='backslashreplace') b'German \xdf, \u266c' \>\>\> 'German ß, ♬'.encode(encoding='ascii', errors='xmlcharrefreplace') b'German \&#223;, \&#9836;'

<div class="index">

pair: strict; error handler's name pair: ignore; error handler's name pair: replace; error handler's name pair: backslashreplace; error handler's name pair: surrogateescape; error handler's name single: ? (question mark); replacement character single: (backslash); escape sequence single: x; escape sequence single: u; escape sequence single: U; escape sequence

</div>

The following error handlers can be used with all Python `standard-encodings` codecs:

<div class="tabularcolumns">

\|l\|L\|

</div>

| Value | Meaning |
|----|----|
| `'strict'` | Raise `UnicodeError` (or a subclass), this is the default. Implemented in `strict_errors`. |
| `'ignore'` | Ignore the malformed data and continue without further notice. Implemented in `ignore_errors`. |
| `'replace'` | Replace with a replacement marker. On encoding, use `?` (ASCII character). On decoding, use `�` (U+FFFD, the official REPLACEMENT CHARACTER). Implemented in `replace_errors`. |
| `'backslashreplace'` | Replace with backslashed escape sequences. On encoding, use hexadecimal form of Unicode code point with formats `\\x{hh}` `\\u{xxxx}` `\\U{xxxxxxxx}`. On decoding, use hexadecimal form of byte value with format `\\x{hh}`. Implemented in `backslashreplace_errors`. |
| `'surrogateescape'` | On decoding, replace byte with individual surrogate code ranging from `U+DC80` to `U+DCFF`. This code will then be turned back into the same byte when the `'surrogateescape'` error handler is used when encoding the data. (See `383` for more.) |

<div class="index">

pair: xmlcharrefreplace; error handler's name pair: namereplace; error handler's name single: N; escape sequence

</div>

The following error handlers are only applicable to encoding (within `text encodings <text encoding>`):

| Value | Meaning |
|----|----|
| `'xmlcharrefreplace'` | Replace with XML/HTML numeric character reference, which is a decimal form of Unicode code point with format `&#{num};`. Implemented in `xmlcharrefreplace_errors`. |
| `'namereplace'` | Replace with `\N{...}` escape sequences, what appears in the braces is the Name property from Unicode Character Database. Implemented in `namereplace_errors`. |

<div class="index">

pair: surrogatepass; error handler's name

</div>

In addition, the following error handler is specific to the given codecs:

| Value | Codecs | Meaning |
|----|----|----|
| `'surrogatepass'` | utf-8, utf-16, utf-32, utf-16-be, utf-16-le, utf-32-be, utf-32-le | Allow encoding and decoding surrogate code point (`U+D800` - `U+DFFF`) as normal code point. Otherwise these codecs treat the presence of surrogate code point in `str` as an error. |

<div class="versionadded">

3.1 The `'surrogateescape'` and `'surrogatepass'` error handlers.

</div>

<div class="versionchanged">

3.4 The `'surrogatepass'` error handler now works with utf-16\* and utf-32\* codecs.

</div>

<div class="versionadded">

3.5 The `'namereplace'` error handler.

</div>

<div class="versionchanged">

3.5 The `'backslashreplace'` error handler now works with decoding and translating.

</div>

The set of allowed values can be extended by registering a new named error handler:

<div class="function">

register_error(name, error_handler, /)

Register the error handling function *error_handler* under the name *name*. The *error_handler* argument will be called during encoding and decoding in case of an error, when *name* is specified as the errors parameter.

For encoding, *error_handler* will be called with a `UnicodeEncodeError` instance, which contains information about the location of the error. The error handler must either raise this or a different exception, or return a tuple with a replacement for the unencodable part of the input and a position where encoding should continue. The replacement may be either `str` or `bytes`. If the replacement is bytes, the encoder will simply copy them into the output buffer. If the replacement is a string, the encoder will encode the replacement. Encoding continues on original input at the specified position. Negative position values will be treated as being relative to the end of the input string. If the resulting position is out of bound an `IndexError` will be raised.

Decoding and translating works similarly, except `UnicodeDecodeError` or `UnicodeTranslateError` will be passed to the handler and that the replacement from the error handler will be put into the output directly.

</div>

Previously registered error handlers (including the standard error handlers) can be looked up by name:

<div class="function">

lookup_error(name, /)

Return the error handler previously registered under the name *name*.

Raises a `LookupError` in case the handler cannot be found.

</div>

The following standard error handlers are also made available as module level functions:

<div class="function">

strict_errors(exception)

Implements the `'strict'` error handling.

Each encoding or decoding error raises a `UnicodeError`.

</div>

<div class="function">

ignore_errors(exception)

Implements the `'ignore'` error handling.

Malformed data is ignored; encoding or decoding is continued without further notice.

</div>

<div class="function">

replace_errors(exception)

Implements the `'replace'` error handling.

Substitutes `?` (ASCII character) for encoding errors or `�` (U+FFFD, the official REPLACEMENT CHARACTER) for decoding errors.

</div>

<div class="function">

backslashreplace_errors(exception)

Implements the `'backslashreplace'` error handling.

Malformed data is replaced by a backslashed escape sequence. On encoding, use the hexadecimal form of Unicode code point with formats `\\x{hh}` `\\u{xxxx}` `\\U{xxxxxxxx}`. On decoding, use the hexadecimal form of byte value with format `\\x{hh}`.

<div class="versionchanged">

3.5 Works with decoding and translating.

</div>

</div>

<div class="function">

xmlcharrefreplace_errors(exception)

Implements the `'xmlcharrefreplace'` error handling (for encoding within `text encoding` only).

The unencodable character is replaced by an appropriate XML/HTML numeric character reference, which is a decimal form of Unicode code point with format `&#{num};` .

</div>

<div class="function">

namereplace_errors(exception)

Implements the `'namereplace'` error handling (for encoding within `text encoding` only).

The unencodable character is replaced by a `\N{...}` escape sequence. The set of characters that appear in the braces is the Name property from Unicode Character Database. For example, the German lowercase letter `'ß'` will be converted to byte sequence `\N{LATIN SMALL LETTER SHARP S}` .

<div class="versionadded">

3.5

</div>

</div>

### Stateless Encoding and Decoding

The base `Codec` class defines these methods which also define the function interfaces of the stateless encoder and decoder:

<div class="Codec">

<div class="method">

encode(input, errors='strict')

Encodes the object *input* and returns a tuple (output object, length consumed). For instance, `text encoding` converts a string object to a bytes object using a particular character set encoding (e.g., `cp1252` or `iso-8859-1`).

The *errors* argument defines the error handling to apply. It defaults to `'strict'` handling.

The method may not store state in the `Codec` instance. Use `StreamWriter` for codecs which have to keep state in order to make encoding efficient.

The encoder must be able to handle zero length input and return an empty object of the output object type in this situation.

</div>

<div class="method">

decode(input, errors='strict')

Decodes the object *input* and returns a tuple (output object, length consumed). For instance, for a `text encoding`, decoding converts a bytes object encoded using a particular character set encoding to a string object.

For text encodings and bytes-to-bytes codecs, *input* must be a bytes object or one which provides the read-only buffer interface -- for example, buffer objects and memory mapped files.

The *errors* argument defines the error handling to apply. It defaults to `'strict'` handling.

The method may not store state in the `Codec` instance. Use `StreamReader` for codecs which have to keep state in order to make decoding efficient.

The decoder must be able to handle zero length input and return an empty object of the output object type in this situation.

</div>

</div>

### Incremental Encoding and Decoding

The `IncrementalEncoder` and `IncrementalDecoder` classes provide the basic interface for incremental encoding and decoding. Encoding/decoding the input isn't done with one call to the stateless encoder/decoder function, but with multiple calls to the `~IncrementalEncoder.encode`/`~IncrementalDecoder.decode` method of the incremental encoder/decoder. The incremental encoder/decoder keeps track of the encoding/decoding process during method calls.

The joined output of calls to the `~IncrementalEncoder.encode`/`~IncrementalDecoder.decode` method is the same as if all the single inputs were joined into one, and this input was encoded/decoded with the stateless encoder/decoder.

#### IncrementalEncoder Objects

The `IncrementalEncoder` class is used for encoding an input in multiple steps. It defines the following methods which every incremental encoder must define in order to be compatible with the Python codec registry.

<div class="IncrementalEncoder(errors='strict')">

Constructor for an `IncrementalEncoder` instance.

All incremental encoders must provide this constructor interface. They are free to add additional keyword arguments, but only the ones defined here are used by the Python codec registry.

The `IncrementalEncoder` may implement different error handling schemes by providing the *errors* keyword argument. See `error-handlers` for possible values.

The *errors* argument will be assigned to an attribute of the same name. Assigning to this attribute makes it possible to switch between different error handling strategies during the lifetime of the `IncrementalEncoder` object.

<div class="method">

encode(object, final=False)

Encodes *object* (taking the current state of the encoder into account) and returns the resulting encoded object. If this is the last call to `encode` *final* must be true (the default is false).

</div>

<div class="method">

reset()

Reset the encoder to the initial state. The output is discarded: call `.encode(object, final=True)`, passing an empty byte or text string if necessary, to reset the encoder and to get the output.

</div>

<div class="method">

getstate()

Return the current state of the encoder which must be an integer. The implementation should make sure that `0` is the most common state. (States that are more complicated than integers can be converted into an integer by marshaling/pickling the state and encoding the bytes of the resulting string into an integer.)

</div>

<div class="method">

setstate(state)

Set the state of the encoder to *state*. *state* must be an encoder state returned by `getstate`.

</div>

</div>

#### IncrementalDecoder Objects

The `IncrementalDecoder` class is used for decoding an input in multiple steps. It defines the following methods which every incremental decoder must define in order to be compatible with the Python codec registry.

<div class="IncrementalDecoder(errors='strict')">

Constructor for an `IncrementalDecoder` instance.

All incremental decoders must provide this constructor interface. They are free to add additional keyword arguments, but only the ones defined here are used by the Python codec registry.

The `IncrementalDecoder` may implement different error handling schemes by providing the *errors* keyword argument. See `error-handlers` for possible values.

The *errors* argument will be assigned to an attribute of the same name. Assigning to this attribute makes it possible to switch between different error handling strategies during the lifetime of the `IncrementalDecoder` object.

<div class="method">

decode(object, final=False)

Decodes *object* (taking the current state of the decoder into account) and returns the resulting decoded object. If this is the last call to `decode` *final* must be true (the default is false). If *final* is true the decoder must decode the input completely and must flush all buffers. If this isn't possible (e.g. because of incomplete byte sequences at the end of the input) it must initiate error handling just like in the stateless case (which might raise an exception).

</div>

<div class="method">

reset()

Reset the decoder to the initial state.

</div>

<div class="method">

getstate()

Return the current state of the decoder. This must be a tuple with two items, the first must be the buffer containing the still undecoded input. The second must be an integer and can be additional state info. (The implementation should make sure that `0` is the most common additional state info.) If this additional state info is `0` it must be possible to set the decoder to the state which has no input buffered and `0` as the additional state info, so that feeding the previously buffered input to the decoder returns it to the previous state without producing any output. (Additional state info that is more complicated than integers can be converted into an integer by marshaling/pickling the info and encoding the bytes of the resulting string into an integer.)

</div>

<div class="method">

setstate(state)

Set the state of the decoder to *state*. *state* must be a decoder state returned by `getstate`.

</div>

</div>

### Stream Encoding and Decoding

The `StreamWriter` and `StreamReader` classes provide generic working interfaces which can be used to implement new encoding submodules very easily. See `!encodings.utf_8` for an example of how this is done.

#### StreamWriter Objects

The `StreamWriter` class is a subclass of `Codec` and defines the following methods which every stream writer must define in order to be compatible with the Python codec registry.

<div class="StreamWriter(stream, errors='strict')">

Constructor for a `StreamWriter` instance.

All stream writers must provide this constructor interface. They are free to add additional keyword arguments, but only the ones defined here are used by the Python codec registry.

The *stream* argument must be a file-like object open for writing text or binary data, as appropriate for the specific codec.

The `StreamWriter` may implement different error handling schemes by providing the *errors* keyword argument. See `error-handlers` for the standard error handlers the underlying stream codec may support.

The *errors* argument will be assigned to an attribute of the same name. Assigning to this attribute makes it possible to switch between different error handling strategies during the lifetime of the `StreamWriter` object.

<div class="method">

write(object)

Writes the object's contents encoded to the stream.

</div>

<div class="method">

writelines(list)

Writes the concatenated iterable of strings to the stream (possibly by reusing the `write` method). Infinite or very large iterables are not supported. The standard bytes-to-bytes codecs do not support this method.

</div>

<div class="method">

reset()

Resets the codec buffers used for keeping internal state.

Calling this method should ensure that the data on the output is put into a clean state that allows appending of new fresh data without having to rescan the whole stream to recover state.

</div>

</div>

In addition to the above methods, the `StreamWriter` must also inherit all other methods and attributes from the underlying stream.

#### StreamReader Objects

The `StreamReader` class is a subclass of `Codec` and defines the following methods which every stream reader must define in order to be compatible with the Python codec registry.

<div class="StreamReader(stream, errors='strict')">

Constructor for a `StreamReader` instance.

All stream readers must provide this constructor interface. They are free to add additional keyword arguments, but only the ones defined here are used by the Python codec registry.

The *stream* argument must be a file-like object open for reading text or binary data, as appropriate for the specific codec.

The `StreamReader` may implement different error handling schemes by providing the *errors* keyword argument. See `error-handlers` for the standard error handlers the underlying stream codec may support.

The *errors* argument will be assigned to an attribute of the same name. Assigning to this attribute makes it possible to switch between different error handling strategies during the lifetime of the `StreamReader` object.

The set of allowed values for the *errors* argument can be extended with `register_error`.

<div class="method">

read(size=-1, chars=-1, firstline=False)

Decodes data from the stream and returns the resulting object.

The *chars* argument indicates the number of decoded code points or bytes to return. The `read` method will never return more data than requested, but it might return less, if there is not enough available.

The *size* argument indicates the approximate maximum number of encoded bytes or code points to read for decoding. The decoder can modify this setting as appropriate. The default value -1 indicates to read and decode as much as possible. This parameter is intended to prevent having to decode huge files in one step.

The *firstline* flag indicates that it would be sufficient to only return the first line, if there are decoding errors on later lines.

The method should use a greedy read strategy meaning that it should read as much data as is allowed within the definition of the encoding and the given size, e.g. if optional encoding endings or state markers are available on the stream, these should be read too.

</div>

<div class="method">

readline(size=None, keepends=True)

Read one line from the input stream and return the decoded data.

*size*, if given, is passed as size argument to the stream's `read` method.

If *keepends* is false line-endings will be stripped from the lines returned.

</div>

<div class="method">

readlines(sizehint=None, keepends=True)

Read all lines available on the input stream and return them as a list of lines.

Line-endings are implemented using the codec's `decode` method and are included in the list entries if *keepends* is true.

*sizehint*, if given, is passed as the *size* argument to the stream's `read` method.

</div>

<div class="method">

reset()

Resets the codec buffers used for keeping internal state.

Note that no stream repositioning should take place. This method is primarily intended to be able to recover from decoding errors.

</div>

</div>

In addition to the above methods, the `StreamReader` must also inherit all other methods and attributes from the underlying stream.

#### StreamReaderWriter Objects

The `StreamReaderWriter` is a convenience class that allows wrapping streams which work in both read and write modes.

The design is such that one can use the factory functions returned by the `lookup` function to construct the instance.

<div class="StreamReaderWriter(stream, Reader, Writer, errors='strict')">

Creates a `StreamReaderWriter` instance. *stream* must be a file-like object. *Reader* and *Writer* must be factory functions or classes providing the `StreamReader` and `StreamWriter` interface resp. Error handling is done in the same way as defined for the stream readers and writers.

</div>

`StreamReaderWriter` instances define the combined interfaces of `StreamReader` and `StreamWriter` classes. They inherit all other methods and attributes from the underlying stream.

#### StreamRecoder Objects

The `StreamRecoder` translates data from one encoding to another, which is sometimes useful when dealing with different encoding environments.

The design is such that one can use the factory functions returned by the `lookup` function to construct the instance.

<div class="StreamRecoder(stream, encode, decode, Reader, Writer, errors='strict')">

Creates a `StreamRecoder` instance which implements a two-way conversion: *encode* and *decode* work on the frontend — the data visible to code calling `~StreamReader.read` and `~StreamWriter.write`, while *Reader* and *Writer* work on the backend — the data in *stream*.

You can use these objects to do transparent transcodings, e.g., from Latin-1 to UTF-8 and back.

The *stream* argument must be a file-like object.

The *encode* and *decode* arguments must adhere to the `Codec` interface. *Reader* and *Writer* must be factory functions or classes providing objects of the `StreamReader` and `StreamWriter` interface respectively.

Error handling is done in the same way as defined for the stream readers and writers.

</div>

`StreamRecoder` instances define the combined interfaces of `StreamReader` and `StreamWriter` classes. They inherit all other methods and attributes from the underlying stream.

## Encodings and Unicode

Strings are stored internally as sequences of code points in range `U+0000`--`U+10FFFF`. (See `393` for more details about the implementation.) Once a string object is used outside of CPU and memory, endianness and how these arrays are stored as bytes become an issue. As with other codecs, serialising a string into a sequence of bytes is known as *encoding*, and recreating the string from the sequence of bytes is known as *decoding*. There are a variety of different text serialisation codecs, which are collectivity referred to as `text encodings <text encoding>`.

The simplest text encoding (called `'latin-1'` or `'iso-8859-1'`) maps the code points 0--255 to the bytes `0x0`--`0xff`, which means that a string object that contains code points above `U+00FF` can't be encoded with this codec. Doing so will raise a `UnicodeEncodeError` that looks like the following (although the details of the error message may differ): `UnicodeEncodeError: 'latin-1' codec can't encode character '\u1234' in position 3: ordinal not in range(256)`.

There's another group of encodings (the so called charmap encodings) that choose a different subset of all Unicode code points and how these code points are mapped to the bytes `0x0`--`0xff`. To see how this is done simply open e.g. `encodings/cp1252.py` (which is an encoding that is used primarily on Windows). There's a string constant with 256 characters that shows you which character is mapped to which byte value.

All of these encodings can only encode 256 of the 1114112 code points defined in Unicode. A simple and straightforward way that can store each Unicode code point, is to store each code point as four consecutive bytes. There are two possibilities: store the bytes in big endian or in little endian order. These two encodings are called `UTF-32-BE` and `UTF-32-LE` respectively. Their disadvantage is that if, for example, you use `UTF-32-BE` on a little endian machine you will always have to swap bytes on encoding and decoding. Python's `UTF-16` and `UTF-32` codecs avoid this problem by using the platform's native byte order when no BOM is present. Python follows prevailing platform practice, so native-endian data round-trips without redundant byte swapping, even though the Unicode Standard defaults to big-endian when the byte order is unspecified. When these bytes are read by a CPU with a different endianness, the bytes have to be swapped. To be able to detect the endianness of a `UTF-16` or `UTF-32` byte sequence, a BOM ("Byte Order Mark") is used. This is the Unicode character `U+FEFF`. This character can be prepended to every `UTF-16` or `UTF-32` byte sequence. The byte swapped version of this character (`0xFFFE`) is an illegal character that may not appear in a Unicode text. When the first character of a `UTF-16` or `UTF-32` byte sequence is `U+FFFE`, the bytes have to be swapped on decoding.

Unfortunately the character `U+FEFF` had a second purpose as a `ZERO WIDTH NO-BREAK SPACE`: a character that has no width and doesn't allow a word to be split. It can e.g. be used to give hints to a ligature algorithm. With Unicode 4.0 using `U+FEFF` as a `ZERO WIDTH NO-BREAK SPACE` has been deprecated (with `U+2060` (`WORD JOINER`) assuming this role). Nevertheless Unicode software still must be able to handle `U+FEFF` in both roles: as a BOM it's a device to determine the storage layout of the encoded bytes, and vanishes once the byte sequence has been decoded into a string; as a `ZERO WIDTH NO-BREAK SPACE` it's a normal character that will be decoded like any other.

There's another encoding that is able to encode the full range of Unicode characters: UTF-8. UTF-8 is an 8-bit encoding, which means there are no issues with byte order in UTF-8. Each byte in a UTF-8 byte sequence consists of two parts: marker bits (the most significant bits) and payload bits. The marker bits are a sequence of zero to four `1` bits followed by a `0` bit. Unicode characters are encoded like this (with x being payload bits, which when concatenated give the Unicode character):

| Range                         | Encoding                            |
|-------------------------------|-------------------------------------|
| `U-00000000` ... `U-0000007F` | 0xxxxxxx                            |
| `U-00000080` ... `U-000007FF` | 110xxxxx 10xxxxxx                   |
| `U-00000800` ... `U-0000FFFF` | 1110xxxx 10xxxxxx 10xxxxxx          |
| `U-00010000` ... `U-0010FFFF` | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx |

The least significant bit of the Unicode character is the rightmost x bit.

As UTF-8 is an 8-bit encoding no BOM is required and any `U+FEFF` character in the decoded string (even if it's the first character) is treated as a `ZERO WIDTH NO-BREAK SPACE`.

Without external information it's impossible to reliably determine which encoding was used for encoding a string. Each charmap encoding can decode any random byte sequence. However that's not possible with UTF-8, as UTF-8 byte sequences have a structure that doesn't allow arbitrary byte sequences. To increase the reliability with which a UTF-8 encoding can be detected, Microsoft invented a variant of UTF-8 (that Python calls `"utf-8-sig"`) for its Notepad program: Before any of the Unicode characters is written to the file, a UTF-8 encoded BOM (which looks like this as a byte sequence: `0xef`, `0xbb`, `0xbf`) is written. As it's rather improbable that any charmap encoded file starts with these byte values (which would e.g. map to

> LATIN SMALL LETTER I WITH DIAERESIS\
> RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK\
> INVERTED QUESTION MARK

in iso-8859-1), this increases the probability that a `utf-8-sig` encoding can be correctly guessed from the byte sequence. So here the BOM is not used to be able to determine the byte order used for generating the byte sequence, but as a signature that helps in guessing the encoding. On encoding the utf-8-sig codec will write `0xef`, `0xbb`, `0xbf` as the first three bytes to the file. On decoding `utf-8-sig` will skip those three bytes if they appear as the first three bytes in the file. In UTF-8, the use of the BOM is discouraged and should generally be avoided.

## Standard Encodings

Python comes with a number of codecs built-in, either implemented as C functions or with dictionaries as mapping tables. The following table lists the codecs by name, together with a few common aliases, and the languages for which the encoding is likely used. Neither the list of aliases nor the list of languages is meant to be exhaustive. Notice that spelling alternatives that only differ in case or use a space or a hyphen instead of an underscore are also valid aliases because they are equivalent when normalized by `~encodings.normalize_encoding`. For example, `'utf-8'` is a valid alias for the `'utf_8'` codec.

> [!NOTE]
> The below table lists the most common aliases, for a complete list refer to the source `aliases.py <Lib/encodings/aliases.py>` file.

On Windows, `cpXXX` codecs are available for all code pages. But only codecs listed in the following table are guaranteed to exist on other platforms.

On platforms that provide the C library's `iconv(3)` function (such as those using the GNU C Library), every encoding known to `iconv` for which Python has no built-in codec is available as well. Such an encoding is looked up by its `iconv` name (for example `cp1133`). Prefixing the name with `iconv:` forces the use of the `iconv`-based codec even when a built-in codec of the same name exists (for example `iconv:latin1`), which is mostly useful for testing.

<div class="versionchanged">

3.16 Added support for encodings provided by the C library's `iconv`.

</div>

<div class="impl-detail">

Some common encodings can bypass the codecs lookup machinery to improve performance. These optimization opportunities are only recognized by CPython for a limited set of (case insensitive) aliases: utf-8, utf8, latin-1, latin1, iso-8859-1, iso8859-1, mbcs (Windows only), ascii, us-ascii, utf-16, utf16, utf-32, utf32, and the same using underscores instead of dashes. Using alternative aliases for these encodings may result in slower execution.

<div class="versionchanged">

3.6 Optimization opportunity recognized for us-ascii.

</div>

</div>

Many of the character sets support the same languages. They vary in individual characters (e.g. whether the EURO SIGN is supported or not), and in the assignment of characters to code positions. For the European languages in particular, the following variants typically exist:

- an ISO 8859 codeset
- a Microsoft Windows code page, which is typically derived from an 8859 codeset, but replaces control characters with additional graphic characters
- an IBM EBCDIC code page
- an IBM PC code page, which is ASCII compatible

<div class="tabularcolumns">

\|l\|p{0.3linewidth}\|p{0.3\linewidth}\|

</div>

<table style="width:98%;">
<colgroup>
<col style="width: 21%" />
<col style="width: 38%" />
<col style="width: 38%" />
</colgroup>
<thead>
<tr>
<th><p>Codec</p></th>
<th><p>Aliases</p></th>
<th><p>Languages</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p>ascii</p></td>
<td><p>646, us-ascii</p></td>
<td><p>English</p></td>
</tr>
<tr>
<td><p>big5</p></td>
<td><p>big5-tw, csbig5</p></td>
<td><p>Traditional Chinese</p></td>
</tr>
<tr>
<td><p>big5hkscs</p></td>
<td><p>big5-hkscs, hkscs</p></td>
<td><p>Traditional Chinese</p></td>
</tr>
<tr>
<td><p>cp037</p></td>
<td><p>IBM037, IBM039</p></td>
<td><p>English</p></td>
</tr>
<tr>
<td><p>cp273</p></td>
<td><p>273, IBM273, csIBM273</p></td>
<td><p>German</p>
<div class="versionadded">
<p>3.4</p>
</div></td>
</tr>
<tr>
<td><p>cp424</p></td>
<td><p>EBCDIC-CP-HE, IBM424</p></td>
<td><p>Hebrew</p></td>
</tr>
<tr>
<td><p>cp437</p></td>
<td><p>437, IBM437</p></td>
<td><p>English</p></td>
</tr>
<tr>
<td><p>cp500</p></td>
<td><p>EBCDIC-CP-BE, EBCDIC-CP-CH, IBM500</p></td>
<td><p>Western Europe</p></td>
</tr>
<tr>
<td><p>cp720</p></td>
<td></td>
<td><p>Arabic</p></td>
</tr>
<tr>
<td><p>cp737</p></td>
<td></td>
<td><p>Greek</p></td>
</tr>
<tr>
<td><p>cp775</p></td>
<td><p>IBM775</p></td>
<td><p>Baltic languages</p></td>
</tr>
<tr>
<td><p>cp850</p></td>
<td><p>850, IBM850</p></td>
<td><p>Western Europe</p></td>
</tr>
<tr>
<td><p>cp852</p></td>
<td><p>852, IBM852</p></td>
<td><p>Central and Eastern Europe</p></td>
</tr>
<tr>
<td><p>cp855</p></td>
<td><p>855, IBM855</p></td>
<td><p>Belarusian, Bulgarian, Macedonian, Russian, Serbian</p></td>
</tr>
<tr>
<td><p>cp856</p></td>
<td></td>
<td><p>Hebrew</p></td>
</tr>
<tr>
<td><p>cp857</p></td>
<td><p>857, IBM857</p></td>
<td><p>Turkish</p></td>
</tr>
<tr>
<td><p>cp858</p></td>
<td><p>858, IBM00858</p></td>
<td><p>Western Europe</p></td>
</tr>
<tr>
<td><p>cp860</p></td>
<td><p>860, IBM860</p></td>
<td><p>Portuguese</p></td>
</tr>
<tr>
<td><p>cp861</p></td>
<td><p>861, CP-IS, IBM861</p></td>
<td><p>Icelandic</p></td>
</tr>
<tr>
<td><p>cp862</p></td>
<td><p>862, IBM862</p></td>
<td><p>Hebrew</p></td>
</tr>
<tr>
<td><p>cp863</p></td>
<td><p>863, IBM863</p></td>
<td><p>Canadian</p></td>
</tr>
<tr>
<td><p>cp864</p></td>
<td><p>IBM864</p></td>
<td><p>Arabic</p></td>
</tr>
<tr>
<td><p>cp865</p></td>
<td><p>865, IBM865</p></td>
<td><p>Danish, Norwegian</p></td>
</tr>
<tr>
<td><p>cp866</p></td>
<td><p>866, IBM866</p></td>
<td><p>Russian</p></td>
</tr>
<tr>
<td><p>cp869</p></td>
<td><p>869, CP-GR, IBM869</p></td>
<td><p>Greek</p></td>
</tr>
<tr>
<td><p>cp874</p></td>
<td></td>
<td><p>Thai</p></td>
</tr>
<tr>
<td><p>cp875</p></td>
<td></td>
<td><p>Greek</p></td>
</tr>
<tr>
<td><p>cp932</p></td>
<td><p>932, ms932, mskanji, ms-kanji, windows-31j</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>cp949</p></td>
<td><p>949, ms949, uhc</p></td>
<td><p>Korean</p></td>
</tr>
<tr>
<td><p>cp950</p></td>
<td><p>950, ms950</p></td>
<td><p>Traditional Chinese</p></td>
</tr>
<tr>
<td><p>cp1006</p></td>
<td></td>
<td><p>Urdu</p></td>
</tr>
<tr>
<td><p>cp1026</p></td>
<td><p>ibm1026</p></td>
<td><p>Turkish</p></td>
</tr>
<tr>
<td><p>cp1125</p></td>
<td><p>1125, ibm1125, cp866u, ruscii</p></td>
<td><p>Ukrainian</p>
<div class="versionadded">
<p>3.4</p>
</div></td>
</tr>
<tr>
<td><p>cp1140</p></td>
<td><p>IBM01140</p></td>
<td><p>Western Europe</p></td>
</tr>
<tr>
<td><p>cp1250</p></td>
<td><p>windows-1250</p></td>
<td><p>Central and Eastern Europe</p></td>
</tr>
<tr>
<td><p>cp1251</p></td>
<td><p>windows-1251</p></td>
<td><p>Belarusian, Bulgarian, Macedonian, Russian, Serbian</p></td>
</tr>
<tr>
<td><p>cp1252</p></td>
<td><p>windows-1252</p></td>
<td><p>Western Europe</p></td>
</tr>
<tr>
<td><p>cp1253</p></td>
<td><p>windows-1253</p></td>
<td><p>Greek</p></td>
</tr>
<tr>
<td><p>cp1254</p></td>
<td><p>windows-1254</p></td>
<td><p>Turkish</p></td>
</tr>
<tr>
<td><p>cp1255</p></td>
<td><p>windows-1255</p></td>
<td><p>Hebrew</p></td>
</tr>
<tr>
<td><p>cp1256</p></td>
<td><p>windows-1256</p></td>
<td><p>Arabic</p></td>
</tr>
<tr>
<td><p>cp1257</p></td>
<td><p>windows-1257</p></td>
<td><p>Baltic languages</p></td>
</tr>
<tr>
<td><p>cp1258</p></td>
<td><p>windows-1258</p></td>
<td><p>Vietnamese</p></td>
</tr>
<tr>
<td><p>euc_jp</p></td>
<td><p>eucjp, ujis, u-jis</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>euc_jis_2004</p></td>
<td><p>jisx0213, eucjis2004</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>euc_jisx0213</p></td>
<td><p>eucjisx0213</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>euc_kr</p></td>
<td><p>euckr, korean, ksc5601, ks_c-5601, ks_c-5601-1987, ksx1001, ks_x-1001</p></td>
<td><p>Korean</p></td>
</tr>
<tr>
<td><p>gb2312</p></td>
<td><p>chinese, csiso58gb231280, euc-cn, euccn, eucgb2312-cn, gb2312-1980, gb2312-80, iso-ir-58</p></td>
<td><p>Simplified Chinese</p></td>
</tr>
<tr>
<td><p>gbk</p></td>
<td><p>936, cp936, ms936</p></td>
<td><p>Unified Chinese</p></td>
</tr>
<tr>
<td><p>gb18030</p></td>
<td><p>gb18030-2000</p></td>
<td><p>Unified Chinese</p></td>
</tr>
<tr>
<td><p>hz</p></td>
<td><p>hzgb, hz-gb, hz-gb-2312</p></td>
<td><p>Simplified Chinese</p></td>
</tr>
<tr>
<td><p>iso2022_jp</p></td>
<td><p>csiso2022jp, iso2022jp, iso-2022-jp</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>iso2022_jp_1</p></td>
<td><p>iso2022jp-1, iso-2022-jp-1</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>iso2022_jp_2</p></td>
<td><p>iso2022jp-2, iso-2022-jp-2</p></td>
<td><p>Japanese, Korean, Simplified Chinese, Western Europe, Greek</p></td>
</tr>
<tr>
<td><p>iso2022_jp_2004</p></td>
<td><p>iso2022jp-2004, iso-2022-jp-2004</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>iso2022_jp_3</p></td>
<td><p>iso2022jp-3, iso-2022-jp-3</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>iso2022_jp_ext</p></td>
<td><p>iso2022jp-ext, iso-2022-jp-ext</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>iso2022_kr</p></td>
<td><p>csiso2022kr, iso2022kr, iso-2022-kr</p></td>
<td><p>Korean</p></td>
</tr>
<tr>
<td><p>latin_1</p></td>
<td><p>iso-8859-1, iso8859-1, 8859, cp819, latin, latin1, L1</p></td>
<td><p>Western Europe</p></td>
</tr>
<tr>
<td><p>iso8859_2</p></td>
<td><p>iso-8859-2, latin2, L2</p></td>
<td><p>Central and Eastern Europe</p></td>
</tr>
<tr>
<td><p>iso8859_3</p></td>
<td><p>iso-8859-3, latin3, L3</p></td>
<td><p>Esperanto, Maltese</p></td>
</tr>
<tr>
<td><p>iso8859_4</p></td>
<td><p>iso-8859-4, latin4, L4</p></td>
<td><p>Northern Europe</p></td>
</tr>
<tr>
<td><p>iso8859_5</p></td>
<td><p>iso-8859-5, cyrillic</p></td>
<td><p>Belarusian, Bulgarian, Macedonian, Russian, Serbian</p></td>
</tr>
<tr>
<td><p>iso8859_6</p></td>
<td><p>iso-8859-6, arabic</p></td>
<td><p>Arabic</p></td>
</tr>
<tr>
<td><p>iso8859_7</p></td>
<td><p>iso-8859-7, greek, greek8</p></td>
<td><p>Greek</p></td>
</tr>
<tr>
<td><p>iso8859_8</p></td>
<td><p>iso-8859-8, hebrew</p></td>
<td><p>Hebrew</p></td>
</tr>
<tr>
<td><p>iso8859_9</p></td>
<td><p>iso-8859-9, latin5, L5</p></td>
<td><p>Turkish</p></td>
</tr>
<tr>
<td><p>iso8859_10</p></td>
<td><p>iso-8859-10, latin6, L6</p></td>
<td><p>Nordic languages</p></td>
</tr>
<tr>
<td><p>iso8859_11</p></td>
<td><p>iso-8859-11, thai</p></td>
<td><p>Thai languages</p></td>
</tr>
<tr>
<td><p>iso8859_13</p></td>
<td><p>iso-8859-13, latin7, L7</p></td>
<td><p>Baltic languages</p></td>
</tr>
<tr>
<td><p>iso8859_14</p></td>
<td><p>iso-8859-14, latin8, L8</p></td>
<td><p>Celtic languages</p></td>
</tr>
<tr>
<td><p>iso8859_15</p></td>
<td><p>iso-8859-15, latin9, L9</p></td>
<td><p>Western Europe</p></td>
</tr>
<tr>
<td><p>iso8859_16</p></td>
<td><p>iso-8859-16, latin10, L10</p></td>
<td><p>South-Eastern Europe</p></td>
</tr>
<tr>
<td><p>johab</p></td>
<td><p>cp1361, ms1361</p></td>
<td><p>Korean</p></td>
</tr>
<tr>
<td><p>koi8_r</p></td>
<td></td>
<td><p>Russian</p></td>
</tr>
<tr>
<td><p>koi8_t</p></td>
<td></td>
<td><p>Tajik</p>
<div class="versionadded">
<p>3.5</p>
</div></td>
</tr>
<tr>
<td><p>koi8_u</p></td>
<td></td>
<td><p>Ukrainian</p></td>
</tr>
<tr>
<td><p>kz1048</p></td>
<td><p>kz_1048, strk1048_2002, rk1048</p></td>
<td><p>Kazakh</p>
<div class="versionadded">
<p>3.5</p>
</div></td>
</tr>
<tr>
<td><p>mac_cyrillic</p></td>
<td><p>maccyrillic</p></td>
<td><p>Belarusian, Bulgarian, Macedonian, Russian, Serbian</p></td>
</tr>
<tr>
<td><p>mac_greek</p></td>
<td><p>macgreek</p></td>
<td><p>Greek</p></td>
</tr>
<tr>
<td><p>mac_iceland</p></td>
<td><p>maciceland</p></td>
<td><p>Icelandic</p></td>
</tr>
<tr>
<td><p>mac_latin2</p></td>
<td><p>maclatin2, maccentraleurope, mac_centeuro</p></td>
<td><p>Central and Eastern Europe</p></td>
</tr>
<tr>
<td><p>mac_roman</p></td>
<td><p>macroman, macintosh</p></td>
<td><p>Western Europe</p></td>
</tr>
<tr>
<td><p>mac_turkish</p></td>
<td><p>macturkish</p></td>
<td><p>Turkish</p></td>
</tr>
<tr>
<td><p>ptcp154</p></td>
<td><p>csptcp154, pt154, cp154, cyrillic-asian</p></td>
<td><p>Kazakh</p></td>
</tr>
<tr>
<td><p>shift_jis</p></td>
<td><p>csshiftjis, shiftjis, sjis, s_jis</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>shift_jis_2004</p></td>
<td><p>shiftjis2004, sjis_2004, sjis2004</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>shift_jisx0213</p></td>
<td><p>shiftjisx0213, sjisx0213, s_jisx0213</p></td>
<td><p>Japanese</p></td>
</tr>
<tr>
<td><p>utf_32</p></td>
<td><p>U32, utf32</p></td>
<td><p>all languages</p></td>
</tr>
<tr>
<td><p>utf_32_be</p></td>
<td><p>UTF-32BE</p></td>
<td><p>all languages</p></td>
</tr>
<tr>
<td><p>utf_32_le</p></td>
<td><p>UTF-32LE</p></td>
<td><p>all languages</p></td>
</tr>
<tr>
<td><p>utf_16</p></td>
<td><p>U16, utf16</p></td>
<td><p>all languages</p></td>
</tr>
<tr>
<td><p>utf_16_be</p></td>
<td><p>UTF-16BE</p></td>
<td><p>all languages</p></td>
</tr>
<tr>
<td><p>utf_16_le</p></td>
<td><p>UTF-16LE</p></td>
<td><p>all languages</p></td>
</tr>
<tr>
<td><p>utf_7</p></td>
<td><p>U7, unicode-1-1-utf-7</p></td>
<td><p>all languages</p></td>
</tr>
<tr>
<td><p>utf_8</p></td>
<td><p>U8, UTF, utf8, cp65001</p></td>
<td><p>all languages</p></td>
</tr>
<tr>
<td><p>utf_8_sig</p></td>
<td><p>utf8-sig</p></td>
<td><p>all languages</p></td>
</tr>
</tbody>
</table>

<div class="versionchanged">

3.4 The utf-16\* and utf-32\* encoders no longer allow surrogate code points (`U+D800`--`U+DFFF`) to be encoded. The utf-32\* decoders no longer decode byte sequences that correspond to surrogate code points.

</div>

<div class="versionchanged">

3.8 `cp65001` is now an alias to `utf_8`.

</div>

<div class="versionchanged">

3.14 On Windows, `cpXXX` codecs are now available for all code pages.

</div>

## Python Specific Encodings

A number of predefined codecs are specific to Python, so their codec names have no meaning outside Python. These are listed in the tables below based on the expected input and output types (note that while text encodings are the most common use case for codecs, the underlying codec infrastructure supports arbitrary data transforms rather than just text encodings). For asymmetric codecs, the stated meaning describes the encoding direction.

### Text Encodings

The following codecs provide `str` to `bytes` encoding and `bytes-like object` to `str` decoding, similar to the Unicode text encodings.

<div class="tabularcolumns">

\|l\|p{0.3linewidth}\|p{0.3\linewidth}\|

</div>

<table style="width:81%;">
<colgroup>
<col style="width: 29%" />
<col style="width: 13%" />
<col style="width: 38%" />
</colgroup>
<thead>
<tr>
<th><p>Codec</p></th>
<th><p>Aliases</p></th>
<th><p>Meaning</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p>idna</p></td>
<td></td>
<td><p>Implement <code class="interpreted-text" role="rfc">3490</code>, see also <code class="interpreted-text" role="mod">encodings.idna</code>. Only <code>errors='strict'</code> is supported.</p></td>
</tr>
<tr>
<td><p>mbcs</p></td>
<td><p>ansi, dbcs</p></td>
<td><p>Windows only: Encode the operand according to the ANSI codepage (CP_ACP).</p></td>
</tr>
<tr>
<td><p>oem</p></td>
<td></td>
<td><p>Windows only: Encode the operand according to the OEM codepage (CP_OEMCP).</p>
<div class="versionadded">
<p>3.6</p>
</div></td>
</tr>
<tr>
<td><p>palmos</p></td>
<td></td>
<td><p>Encoding of PalmOS 3.5.</p></td>
</tr>
<tr>
<td><p>punycode</p></td>
<td></td>
<td><p>Implement <code class="interpreted-text" role="rfc">3492</code>. Stateful codecs are not supported.</p>
<div class="warning">
<div class="title">
<p>Warning</p>
</div>
<p>The decoding and encoding algorithms scale poorly, so limit the length of untrusted input.</p>
</div></td>
</tr>
<tr>
<td><p>raw_unicode_escape</p></td>
<td></td>
<td><p>Latin-1 encoding with <code class="interpreted-text" role="samp">\\u{XXXX}</code> and <code class="interpreted-text" role="samp">\\U{XXXXXXXX}</code> for other code points. Existing backslashes are not escaped in any way. It is used in the Python pickle protocol.</p></td>
</tr>
<tr>
<td><p>undefined</p></td>
<td></td>
<td><p>This Codec should only be used for testing purposes.</p>
<p>Raise an exception for all conversions, even empty strings. The error handler is ignored.</p></td>
</tr>
<tr>
<td><p>unicode_escape</p></td>
<td></td>
<td><p>Encoding suitable as the contents of a Unicode literal in ASCII-encoded Python source code, except that quotes are not escaped. Decode from Latin-1 source code. Beware that Python source code actually uses UTF-8 by default.</p></td>
</tr>
<tr>
<td><p>utf-7-imap</p></td>
<td><p>mUTF-7</p></td>
<td><p>Modified UTF-7 encoding of <code class="interpreted-text" role="rfc">3501</code> for IMAP4 mailbox names. Only <code>errors='strict'</code> is supported.</p>
<div class="versionadded">
<p>next</p>
</div></td>
</tr>
</tbody>
</table>

<div class="versionchanged">

3.8 "unicode_internal" codec is removed.

</div>

### Binary Transforms

The following codecs provide binary transforms: `bytes-like object` to `bytes` mappings. They are not supported by `bytes.decode` (which only produces `str` output).

<div class="tabularcolumns">

\|l\|L\|L\|L\|

</div>

<table style="width:98%;">
<colgroup>
<col style="width: 21%" />
<col style="width: 17%" />
<col style="width: 29%" />
<col style="width: 29%" />
</colgroup>
<thead>
<tr>
<th><p>Codec</p></th>
<th><p>Aliases</p></th>
<th><p>Meaning</p></th>
<th><p>Encoder / decoder</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p>base64_codec<a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a></p></td>
<td><p>base64, base_64</p></td>
<td><p>Convert the operand to multiline MIME base64 (the result always includes a trailing <code>'\n'</code>).</p>
<div class="versionchanged">
<p>3.4 accepts any <code class="interpreted-text" role="term">bytes-like object</code> as input for encoding and decoding</p>
</div></td>
<td><p><code class="interpreted-text" role="meth">base64.encodebytes</code> / <code class="interpreted-text" role="meth">base64.decodebytes</code></p></td>
</tr>
<tr>
<td><p>bz2_codec</p></td>
<td><p>bz2</p></td>
<td><p>Compress the operand using bz2.</p></td>
<td><p><code class="interpreted-text" role="meth">bz2.compress</code> / <code class="interpreted-text" role="meth">bz2.decompress</code></p></td>
</tr>
<tr>
<td><p>hex_codec</p></td>
<td><p>hex</p></td>
<td><p>Convert the operand to hexadecimal representation, with two digits per byte.</p></td>
<td><p><code class="interpreted-text" role="meth">binascii.b2a_hex</code> / <code class="interpreted-text" role="meth">binascii.a2b_hex</code></p></td>
</tr>
<tr>
<td><p>quopri_codec</p></td>
<td><p>quopri, quotedprintable, quoted_printable</p></td>
<td><p>Convert the operand to MIME quoted printable.</p></td>
<td><p><code class="interpreted-text" role="meth">quopri.encode</code> with <code>quotetabs=True</code> / <code class="interpreted-text" role="meth">quopri.decode</code></p></td>
</tr>
<tr>
<td><p>uu_codec</p></td>
<td><p>uu</p></td>
<td><p>Convert the operand using uuencode.</p></td>
<td></td>
</tr>
<tr>
<td><p>zlib_codec</p></td>
<td><p>zip, zlib</p></td>
<td><p>Compress the operand using gzip.</p></td>
<td><p><code class="interpreted-text" role="meth">zlib.compress</code> / <code class="interpreted-text" role="meth">zlib.decompress</code></p></td>
</tr>
</tbody>
</table>
<section id="footnotes" class="footnotes footnotes-end-of-document" role="doc-endnotes">
<hr />
<ol>
<li id="fn1"><p>In addition to <code class="interpreted-text" role="term">bytes-like objects &lt;bytes-like object&gt;</code>, <code>'base64_codec'</code> also accepts ASCII-only instances of <code class="interpreted-text" role="class">str</code> for decoding<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
</ol>
</section>

<div class="versionadded">

3.2 Restoration of the binary transforms.

</div>

<div class="versionchanged">

3.4 Restoration of the aliases for the binary transforms.

</div>

### Standalone Codec Functions

The following functions provide encoding and decoding functionality similar to codecs, but are not available as named codecs through `codecs.encode` or `codecs.decode`. They are used internally (for example, by `pickle`) and behave similarly to the `string_escape` codec that was removed in Python 3.

<div class="function">

codecs.escape_encode(input, errors=None)

Encode *input* using escape sequences. Similar to how `repr` on bytes produces escaped byte values.

*input* must be a `bytes` object.

Returns a tuple `(output, length)` where *output* is a `bytes` object and *length* is the number of bytes consumed.

</div>

<div class="function">

codecs.escape_decode(input, errors=None)

Decode *input* from escape sequences back to the original bytes.

*input* must be a `bytes-like object`.

Returns a tuple `(output, length)` where *output* is a `bytes` object and *length* is the number of bytes consumed.

</div>

### Text Transforms

The following codec provides a text transform: a `str` to `str` mapping. It is not supported by `str.encode` (which only produces `bytes` output).

<div class="tabularcolumns">

\|l\|l\|L\|

</div>

| Codec  | Aliases | Meaning                                             |
|--------|---------|-----------------------------------------------------|
| rot_13 | rot13   | Return the Caesar-cypher encryption of the operand. |

<div class="versionadded">

3.2 Restoration of the `rot_13` text transform.

</div>

<div class="versionchanged">

3.4 Restoration of the `rot13` alias.

</div>

## `!encodings` --- Encodings package

<div class="module" data-synopsis="Encodings package">

encodings

</div>

This module implements the following functions:

<div class="function">

normalize_encoding(encoding)

Normalize encoding name *encoding*.

Normalization works as follows: all non-alphanumeric characters except the dot used for Python package names are collapsed and replaced with a single underscore, leading and trailing underscores are removed. For example, `'  -;#'` becomes `'_'`.

Note that *encoding* should be ASCII only.

</div>

> [!NOTE]
> The following functions should not be used directly, except for testing purposes; `codecs.lookup` should be used instead.

<div class="function">

search_function(encoding)

Search for the codec module corresponding to the given encoding name *encoding*.

This function first normalizes the *encoding* using `normalize_encoding`, then looks for a corresponding alias. It attempts to import a codec module from the encodings package using either the alias or the normalized name. If the module is found and defines a valid `getregentry()` function that returns a `codecs.CodecInfo` object, the codec is cached and returned.

If the codec module defines a `getaliases()` function any returned aliases are registered for future use.

</div>

<div class="function">

win32_code_page_search_function(encoding)

Search for a Windows code page encoding *encoding* of the form `cpXXXX`.

If the code page is valid and supported, return a `codecs.CodecInfo` object for it.

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.14

</div>

</div>

This module implements the following exception:

<div class="exception">

CodecRegistryError

Raised when a codec is invalid or incompatible.

</div>

## `!encodings.idna` --- Internationalized Domain Names in Applications

<div class="module" data-synopsis="Internationalized Domain Names implementation">

encodings.idna

</div>

This module implements `3490` (Internationalized Domain Names in Applications) and `3492` (Nameprep: A Stringprep Profile for Internationalized Domain Names (IDN)). It builds upon the `punycode` encoding and `stringprep`.

If you need the IDNA 2008 standard from `5891` and `5895`, use the third-party `idna` module.

These RFCs together define a protocol to support non-ASCII characters in domain names. A domain name containing non-ASCII characters (such as `www.Alliancefrançaise.nu`) is converted into an ASCII-compatible encoding (ACE, such as `www.xn--alliancefranaise-npb.nu`). The ACE form of the domain name is then used in all places where arbitrary characters are not allowed by the protocol, such as DNS queries, HTTP `Host` fields, and so on. This conversion is carried out in the application; if possible invisible to the user: The application should transparently convert Unicode domain labels to IDNA on the wire, and convert back ACE labels to Unicode before presenting them to the user.

Python supports this conversion in several ways: the `idna` codec performs conversion between Unicode and ACE, separating an input string into labels based on the separator characters defined in `section 3.1 of RFC 3490 <3490#section-3.1>` and converting each label to ACE as required, and conversely separating an input byte string into labels based on the `.` separator and converting any ACE labels found into unicode. Furthermore, the `socket` module transparently converts Unicode host names to ACE, so that applications need not be concerned about converting host names themselves when they pass them to the socket module. On top of that, modules that have host names as function parameters, such as `http.client` and `ftplib`, accept Unicode host names (`http.client` then also transparently sends an IDNA hostname in the `Host` field if it sends that field at all).

When receiving host names from the wire (such as in reverse name lookup), no automatic conversion to Unicode is performed: applications wishing to present such host names to the user should decode them to Unicode.

The module `!encodings.idna` also implements the nameprep procedure, which performs certain normalizations on host names, to achieve case-insensitivity of international domain names, and to unify similar characters. The nameprep functions can be used directly if desired.

<div class="function">

nameprep(label, \*, limit=None)

Return the nameprepped version of *label*. The implementation currently assumes query strings, so `AllowUnassigned` is true.

Raise `UnicodeEncodeError` if the nameprep algorithm emits an error.

If the *limit* argument is given, it should be set to the maximum size of an encoded A-label (that is, 63 for IDNA). `!nameprep` will raise `UnicodeEncodeError` if the label is **much** larger than *limit*. Note that this is only a rough check meant to skip expensive processing of extremely large input; the caller should check any exact limits separately.

> [!WARNING]
> For backwards compatibility, label size is unlimited by default. This may cause issues when processing the result with the `punycode` encoding, whose algorithms scale poorly.

<div class="versionchanged">

3.16

Added the *limit* parameter.

</div>

</div>

<div class="function">

ToASCII(label)

Convert a label to ASCII, as specified in `3490`. `UseSTD3ASCIIRules` is assumed to be false.

</div>

<div class="function">

ToUnicode(label)

Convert a label to Unicode, as specified in `3490`.

</div>

## `!encodings.mbcs` --- Windows ANSI codepage

<div class="module" data-synopsis="Windows ANSI codepage">

encodings.mbcs

</div>

This module implements the ANSI codepage (CP_ACP).

<div class="availability">

Windows.

</div>

<div class="versionchanged">

3.2 Before 3.2, the *errors* argument was ignored; `'replace'` was always used to encode, and `'ignore'` to decode.

</div>

<div class="versionchanged">

3.3 Support any error handler.

</div>

## `!encodings.utf_8_sig` --- UTF-8 codec with BOM signature

<div class="module" data-synopsis="UTF-8 codec with BOM signature">

encodings.utf_8_sig

</div>

This module implements a variant of the UTF-8 codec. On encoding, a UTF-8 encoded BOM will be prepended to the UTF-8 encoded bytes. For the stateful encoder this is only done once (on the first write to the byte stream). On decoding, an optional UTF-8 encoded BOM at the start of the data will be skipped.

# `!codeop` --- Compile Python code

<div class="module" data-synopsis="Compile (possibly incomplete) Python code.">

codeop

</div>

**Source code:** `Lib/codeop.py`

------------------------------------------------------------------------

The `!codeop` module provides utilities upon which the Python read-eval-print loop can be emulated, as is done in the `code` module. As a result, you probably don't want to use the module directly; if you want to include such a loop in your program you probably want to use the `code` module instead.

There are two parts to this job:

1.  Being able to tell if a line of input completes a Python statement: in short, telling whether to print '`>>>`' or '`...`' next.
2.  Remembering which future statements the user has entered, so subsequent input can be compiled with these in effect.

The `!codeop` module provides a way of doing each of these things, and a way of doing them both.

To do just the former:

<div class="function">

compile_command(source, filename="\<input\>", symbol="single")

Tries to compile *source*, which should be a string of Python code and return a code object if *source* is valid Python code. In that case, the filename attribute of the code object will be *filename*, which defaults to `'<input>'`. Returns `None` if *source* is *not* valid Python code, but is a prefix of valid Python code.

If there is a problem with *source*, an exception will be raised. `SyntaxError` is raised if there is invalid Python syntax, and `OverflowError` or `ValueError` if there is an invalid literal.

The *symbol* argument determines whether *source* is compiled as a statement (`'single'`, the default), as a sequence of `statement` (`'exec'`) or as an `expression` (`'eval'`). Any other value will cause `ValueError` to be raised.

> [!NOTE]
> It is possible (but not likely) that the parser stops parsing with a successful outcome before reaching the end of the source; in this case, trailing symbols may be ignored instead of causing an error. For example, a backslash followed by two newlines may be followed by arbitrary garbage. This will be fixed once the API for the parser is better.

</div>

<div class="Compile()">

Instances of this class have `~object.__call__` methods identical in signature to the built-in function `compile`, but with the difference that if the instance compiles program text containing a `__future__` statement, the instance 'remembers' and compiles all subsequent program texts with the statement in force.

</div>

<div class="CommandCompiler()">

Instances of this class have `~object.__call__` methods identical in signature to `compile_command`; the difference is that if the instance compiles program text containing a `__future__` statement, the instance 'remembers' and compiles all subsequent program texts with the statement in force.

</div>

# `!collections.abc` --- Abstract Base Classes for Containers

<div class="module" data-synopsis="Abstract base classes for containers">

collections.abc

</div>

<div class="versionadded">

3.3 Formerly, this module was part of the `collections` module.

</div>

**Source code:** `Lib/_collections_abc.py`

<div class="testsetup">

- 

from collections.abc import \* import itertools \_\_name\_\_ = '\<doctest\>'

</div>

------------------------------------------------------------------------

This module provides `abstract base classes <abstract base class>` that can be used to test whether a class provides a particular interface; for example, whether it is `hashable` or whether it is a `mapping`.

An `issubclass` or `isinstance` test for an interface works in one of three ways.

1)  A newly written class can inherit directly from one of the abstract base classes. The class must supply the required abstract methods. The remaining mixin methods come from inheritance and can be overridden if desired. Other methods may be added as needed:

    <div class="testcode">

    class C(Sequence): \# Direct inheritance  
    def \_\_init\_\_(self): ... \# Extra method not required by the ABC def \_\_getitem\_\_(self, index): ... \# Required abstract method def \_\_len\_\_(self): ... \# Required abstract method def count(self, value): ... \# Optionally override a mixin method

    </div>

    <div class="doctest">

    \>\>\> issubclass(C, Sequence) True \>\>\> isinstance(C(), Sequence) True

    </div>

2)  Existing classes and built-in classes can be registered as "virtual subclasses" of the ABCs. Those classes should define the full API including all of the abstract methods and all of the mixin methods. This lets users rely on `issubclass` or `isinstance` tests to determine whether the full interface is supported. The exception to this rule is for methods that are automatically inferred from the rest of the API:

    <div class="testcode">

    class D: \# No inheritance  
    def \_\_init\_\_(self): ... \# Extra method not required by the ABC def \_\_getitem\_\_(self, index): ... \# Abstract method def \_\_len\_\_(self): ... \# Abstract method def count(self, value): ... \# Mixin method def index(self, value): ... \# Mixin method

    Sequence.register(D) \# Register instead of inherit

    </div>

    <div class="doctest">

    \>\>\> issubclass(D, Sequence) True \>\>\> isinstance(D(), Sequence) True

    </div>

    In this example, class `!D` does not need to define `__contains__`, `__iter__`, and `__reversed__` because the `in-operator <comparisons>`, the `iteration <iterable>` logic, and the `reversed` function automatically fall back to using `__getitem__` and `__len__`.

3)  Some simple interfaces are directly recognizable by the presence of the required methods (unless those methods have been set to `None`):

    <div class="testcode">

    class E:  
    def \_\_iter\_\_(self): ... def \_\_next\_\_(self): ...

    </div>

    <div class="doctest">

    \>\>\> issubclass(E, Iterable) True \>\>\> isinstance(E(), Iterable) True

    </div>

    Complex interfaces do not support this last technique because an interface is more than just the presence of method names. Interfaces specify semantics and relationships between methods that cannot be inferred solely from the presence of specific method names. For example, knowing that a class supplies `__getitem__`, `__len__`, and `__iter__` is insufficient for distinguishing a `Sequence` from a `Mapping`.

<div class="versionadded">

3.9 These abstract classes now support `[]`. See `types-genericalias` and `585`.

</div>

## Collections Abstract Base Classes

The collections module offers the following `ABCs <abstract base class>`:

<div class="tabularcolumns">

\|l\|L\|L\|L\|

</div>

| ABC | Inherits from | Abstract Methods | Mixin Methods |
|----|----|----|----|
| `Container`[^1] |  | `__contains__` |  |
| `Hashable`[^2] |  | `__hash__` |  |
| `Iterable`[^3][^4] |  | `__iter__` |  |
| `Iterator`[^5] | `Iterable` | `__next__` | `__iter__` |
| `Reversible`[^6] | `Iterable` | `__reversed__` |  |
| `Generator`[^7] | `Iterator` | `send`, `throw` | `close`, `__iter__`, `__next__` |
| `Sized`[^8] |  | `__len__` |  |
| `Callable`[^9] |  | `__call__` |  |
| `Collection`[^10] | `Sized`, `Iterable`, `Container` | `__contains__`, `__iter__`, `__len__` |  |
| `Sequence` | `Reversible`, `Collection` | `__getitem__`, `__len__` | `__contains__`, `__iter__`, `__reversed__`, `index`, and `count` |
| `MutableSequence` | `Sequence` | `__getitem__`, `__setitem__`, `__delitem__`, `__len__`, `insert` | Inherited `Sequence` methods and `append`, `clear`, `reverse`, `extend`, `pop`, `remove`, and `__iadd__` |
| `ByteString` | `Sequence` | `__getitem__`, `__len__` | Inherited `Sequence` methods |
| `Set` | `Collection` | `__contains__`, `__iter__`, `__len__` | `__le__`, `__lt__`, `__eq__`, `__ne__`, `__gt__`, `__ge__`, `__and__`, `__or__`, `__sub__`, `__rsub__`, `__xor__`, `__rxor__` and `isdisjoint` |
| `MutableSet` | `Set` | `__contains__`, `__iter__`, `__len__`, `add`, `discard` | Inherited `Set` methods and `clear`, `pop`, `remove`, `__ior__`, `__iand__`, `__ixor__`, and `__isub__` |
| `Mapping` | `Collection` | `__getitem__`, `__iter__`, `__len__` | `__contains__`, `keys`, `items`, `values`, `get`, `__eq__`, and `__ne__` |
| `MutableMapping` | `Mapping` | `__getitem__`, `__setitem__`, `__delitem__`, `__iter__`, `__len__` | Inherited `Mapping` methods and `pop`, `popitem`, `clear`, `update`, and `setdefault` |
| `MappingView` | `Sized` |  | `__init__`, `__len__` and `__repr__` |
| `ItemsView` | `MappingView`, `Set` |  | `__contains__`, `__iter__` |
| `KeysView` | `MappingView`, `Set` |  | `__contains__`, `__iter__` |
| `ValuesView` | `MappingView`, `Collection` |  | `__contains__`, `__iter__` |
| `Awaitable`[^11] |  | `__await__` |  |
| `Coroutine`[^12] | `Awaitable` | `send`, `throw` | `close` |
| `AsyncIterable`[^13] |  | `__aiter__` |  |
| `AsyncIterator`[^14] | `AsyncIterable` | `__anext__` | `__aiter__` |
| `AsyncGenerator`[^15] | `AsyncIterator` | `asend`, `athrow` | `aclose`, `__aiter__`, `__anext__` |
| `Buffer`[^16] |  | `__buffer__` |  |

**Footnotes**

## Collections Abstract Base Classes -- Detailed Descriptions

<div class="Container">

ABC for classes that provide the `~object.__contains__` method.

</div>

<div class="Hashable">

ABC for classes that provide the `~object.__hash__` method.

</div>

<div class="Sized">

ABC for classes that provide the `~object.__len__` method.

</div>

<div class="Callable">

ABC for classes that provide the `~object.__call__` method.

See `annotating-callables` for details on how to use `!Callable` in type annotations.

</div>

<div class="Iterable">

ABC for classes that provide the `~container.__iter__` method.

Checking `isinstance(obj, Iterable)` detects classes that are registered as `Iterable` or that have an `~container.__iter__` method, but it does not detect classes that iterate with the `~object.__getitem__` method. The only reliable way to determine whether an object is `iterable` is to call `iter(obj)`.

</div>

<div class="Collection">

ABC for sized iterable container classes.

<div class="versionadded">

3.6

</div>

</div>

<div class="Iterator">

ABC for classes that provide the `~iterator.__iter__` and `~iterator.__next__` methods. See also the definition of `iterator`.

</div>

<div class="Reversible">

ABC for iterable classes that also provide the `~object.__reversed__` method.

<div class="versionadded">

3.6

</div>

</div>

<div class="Generator">

ABC for `generator` classes that implement the protocol defined in `342` that extends `iterators <iterator>` with the `~generator.send`, `~generator.throw` and `~generator.close` methods.

See `annotating-generators-and-coroutines` for details on using `!Generator` in type annotations.

<div class="versionadded">

3.5

</div>

</div>

<div class="Sequence MutableSequence ByteString">

ABCs for read-only and mutable `sequences <sequence>`.

Implementation note: Some of the mixin methods, such as `~container.__iter__`, `~object.__reversed__`, and `~sequence.index` make repeated calls to the underlying `~object.__getitem__` method. Consequently, if `~object.__getitem__` is implemented with constant access speed, the mixin methods will have linear performance; however, if the underlying method is linear (as it would be with a linked list), the mixins will have quadratic performance and will likely need to be overridden.

<div class="method">

index(value, start=0, stop=None)

Return first index of *value*.

Raises `ValueError` if the value is not present.

Supporting the *start* and *stop* arguments is optional, but recommended.

<div class="versionchanged">

3.5 The `~sequence.index` method gained support for the *stop* and *start* arguments.

</div>

</div>

<div class="deprecated-removed">

3.12 3.17 The `ByteString` ABC has been deprecated.

Use `isinstance(obj, collections.abc.Buffer)` to test if `obj` implements the `buffer protocol <bufferobjects>` at runtime. For use in type annotations, either use `Buffer` or a union that explicitly specifies the types your code supports (e.g., `bytes | bytearray | memoryview`).

`!ByteString` was originally intended to be an abstract class that would serve as a supertype of both `bytes` and `bytearray`. However, since the ABC never had any methods, knowing that an object was an instance of `!ByteString` never actually told you anything useful about the object. Other common buffer types such as `memoryview` were also never understood as subtypes of `!ByteString` (either at runtime or by static type checkers).

See `PEP 688 <688#current-options>` for more details.

</div>

</div>

<div class="Set MutableSet">

ABCs for read-only and mutable `sets <types-set>`.

</div>

<div class="Mapping MutableMapping">

ABCs for read-only and mutable `mappings <mapping>`.

</div>

<div class="MappingView ItemsView KeysView ValuesView">

ABCs for mapping, items, keys, and values `views <dictionary view>`.

</div>

<div class="Awaitable">

ABC for `awaitable` objects, which can be used in `await` expressions. Custom implementations must provide the `~object.__await__` method.

`Coroutine <coroutine>` objects and instances of the `~collections.abc.Coroutine` ABC are all instances of this ABC.

> [!NOTE]
> In CPython, generator-based coroutines (`generators <generator>` decorated with `types.coroutine`) are *awaitables*, even though they do not have an `~object.__await__` method. Using `isinstance(gencoro, Awaitable)` for them will return `False`. Use `inspect.isawaitable` to detect them.

<div class="versionadded">

3.5

</div>

</div>

<div class="Coroutine">

ABC for `coroutine` compatible classes. These implement the following methods, defined in `coroutine-objects`: `~coroutine.send`, `~coroutine.throw`, and `~coroutine.close`. Custom implementations must also implement `~object.__await__`. All `Coroutine` instances are also instances of `Awaitable`.

> [!NOTE]
> In CPython, generator-based coroutines (`generators <generator>` decorated with `types.coroutine`) are *awaitables*, even though they do not have an `~object.__await__` method. Using `isinstance(gencoro, Coroutine)` for them will return `False`. Use `inspect.isawaitable` to detect them.

See `annotating-generators-and-coroutines` for details on using `!Coroutine` in type annotations. The variance and order of type parameters correspond to those of `Generator`.

<div class="versionadded">

3.5

</div>

</div>

<div class="AsyncIterable">

ABC for classes that provide an `__aiter__` method. See also the definition of `asynchronous iterable`.

<div class="versionadded">

3.5

</div>

</div>

<div class="AsyncIterator">

ABC for classes that provide `__aiter__` and `__anext__` methods. See also the definition of `asynchronous iterator`.

<div class="versionadded">

3.5

</div>

</div>

<div class="AsyncGenerator">

ABC for `asynchronous generator` classes that implement the protocol defined in `525` and `492`.

See `annotating-generators-and-coroutines` for details on using `!AsyncGenerator` in type annotations.

<div class="versionadded">

3.6

</div>

</div>

<div class="Buffer">

ABC for classes that provide the `~object.__buffer__` method, implementing the `buffer protocol <bufferobjects>`. See `688`.

<div class="versionadded">

3.12

</div>

</div>

## Examples and Recipes

ABCs allow us to ask classes or instances if they provide particular functionality, for example:

    size = None
    if isinstance(myvar, collections.abc.Sized):
        size = len(myvar)

Several of the ABCs are also useful as mixins that make it easier to develop classes supporting container APIs. For example, to write a class supporting the full `Set` API, it is only necessary to supply the three underlying abstract methods: `~object.__contains__`, `~container.__iter__`, and `~object.__len__`. The ABC supplies the remaining methods such as `!__and__` and `~frozenset.isdisjoint`:

    class ListBasedSet(collections.abc.Set):
        ''' Alternate set implementation favoring space over speed
            and not requiring the set elements to be hashable. '''
        def __init__(self, iterable):
            self.elements = lst = []
            for value in iterable:
                if value not in lst:
                    lst.append(value)

        def __iter__(self):
            return iter(self.elements)

        def __contains__(self, value):
            return value in self.elements

        def __len__(self):
            return len(self.elements)

    s1 = ListBasedSet('abcdef')
    s2 = ListBasedSet('defghi')
    overlap = s1 & s2            # The __and__() method is supported automatically

Notes on using `Set` and `MutableSet` as a mixin:

\(1\)  
Since some set operations create new sets, the default mixin methods need a way to create new instances from an `iterable`. The class constructor is assumed to have a signature in the form `ClassName(iterable)`. That assumption is factored-out to an internal `classmethod` called `!_from_iterable` which calls `cls(iterable)` to produce a new set. If the `Set` mixin is being used in a class with a different constructor signature, you will need to override `!_from_iterable` with a classmethod or regular method that can construct new instances from an iterable argument.

\(2\)  
To override the comparisons (presumably for speed, as the semantics are fixed), redefine `~object.__le__` and `~object.__ge__`, then the other operations will automatically follow suit.

\(3\)  
The `Set` mixin provides a `!_hash` method to compute a hash value for the set; however, `~object.__hash__` is not defined because not all sets are `hashable` or immutable. To add set hashability using mixins, inherit from both `Set` and `Hashable`, then define `__hash__ = Set._hash`.

<div class="seealso">

- [OrderedSet recipe](https://code.activestate.com/recipes/576694/) for an example built on `MutableSet`.
- For more about ABCs, see the `abc` module and `3119`.

</div>

[^1]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^2]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^3]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^4]: Checking `isinstance(obj, Iterable)` detects classes that are registered as `Iterable` or that have an `~container.__iter__` method, but it does not detect classes that iterate with the `~object.__getitem__` method. The only reliable way to determine whether an object is `iterable` is to call `iter(obj)`.

[^5]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^6]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^7]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^8]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^9]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^10]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^11]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^12]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^13]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^14]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^15]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

[^16]: These ABCs override `~abc.ABCMeta.__subclasshook__` to support testing an interface by verifying the required methods are present and have not been set to `None`. This only works for simple interfaces. More complex interfaces require registration or direct subclassing.

# `!collections` --- Container datatypes

<div class="module" data-synopsis="Container datatypes">

collections

</div>

**Source code:** `Lib/collections/__init__.py`

<div class="testsetup">

- 

from collections import \* import itertools \_\_name\_\_ = '\<doctest\>'

</div>

------------------------------------------------------------------------

This module implements specialized container datatypes providing alternatives to Python's general purpose built-in containers, `dict`, `list`, `set`, and `tuple`.

|  |  |
|----|----|
| `namedtuple` | factory function for creating tuple subclasses with named fields |
| `deque` | list-like container with fast appends and pops on either end |
| `ChainMap` | dict-like class for creating a single view of multiple mappings |
| `Counter` | dict subclass for counting `hashable` objects |
| `OrderedDict` | dict subclass that remembers the order entries were added |
| `defaultdict` | dict subclass that calls a factory function to supply missing values |
| `UserDict` | wrapper around dictionary objects for easier dict subclassing |
| `UserList` | wrapper around list objects for easier list subclassing |
| `UserString` | wrapper around string objects for easier string subclassing |

## `ChainMap` objects

<div class="versionadded">

3.3

</div>

A `ChainMap` class is provided for quickly linking a number of mappings so they can be treated as a single unit. It is often much faster than creating a new dictionary and running multiple `~dict.update` calls.

The class can be used to simulate nested scopes and is useful in templating.

<div class="ChainMap(*maps)">

A `ChainMap` groups multiple dicts or other mappings together to create a single, updateable view. If no *maps* are specified, a single empty dictionary is provided so that a new chain always has at least one mapping.

The underlying mappings are stored in a list. That list is public and can be accessed or updated using the *maps* attribute. There is no other state.

Lookups search the underlying mappings successively until a key is found. In contrast, writes, updates, and deletions only operate on the first mapping.

A `ChainMap` incorporates the underlying mappings by reference. So, if one of the underlying mappings gets updated, those changes will be reflected in `ChainMap`.

All of the usual dictionary methods are supported. In addition, there is a *maps* attribute, a method for creating new subcontexts, and a property for accessing all but the first mapping:

<div class="attribute">

maps

A user updateable list of mappings. The list is ordered from first-searched to last-searched. It is the only stored state and can be modified to change which mappings are searched. The list should always contain at least one mapping.

</div>

<div class="method">

new_child(m=None, \*\*kwargs)

Returns a new `ChainMap` containing a new map followed by all of the maps in the current instance. If `m` is specified, it becomes the new map at the front of the list of mappings; if not specified, an empty dict is used, so that a call to `d.new_child()` is equivalent to: `ChainMap({}, *d.maps)`. If any keyword arguments are specified, they update passed map or new empty dict. This method is used for creating subcontexts that can be updated without altering values in any of the parent mappings.

<div class="versionchanged">

3.4 The optional `m` parameter was added.

</div>

<div class="versionchanged">

3.10 Keyword arguments support was added.

</div>

</div>

<div class="attribute">

parents

Property returning a new `ChainMap` containing all of the maps in the current instance except the first one. This is useful for skipping the first map in the search. Use cases are similar to those for the `nonlocal` keyword used in `nested scopes <nested
scope>`. The use cases also parallel those for the built-in `super` function. A reference to `d.parents` is equivalent to: `ChainMap(*d.maps[1:])`.

</div>

Note, the iteration order of a `ChainMap` is determined by scanning the mappings last to first:

    >>> baseline = {'music': 'bach', 'art': 'rembrandt'}
    >>> adjustments = {'art': 'van gogh', 'opera': 'carmen'}
    >>> list(ChainMap(adjustments, baseline))
    ['music', 'art', 'opera']

This gives the same ordering as a series of `dict.update` calls starting with the last mapping:

    >>> combined = baseline.copy()
    >>> combined.update(adjustments)
    >>> list(combined)
    ['music', 'art', 'opera']

<div class="versionchanged">

3.9 Added support for `|` and `|=` operators, specified in `584`.

</div>

</div>

<div class="seealso">

- The [MultiContext class](https://github.com/enthought/codetools/blob/4.0.0/codetools/contexts/multi_context.py) in the Enthought [CodeTools package](https://github.com/enthought/codetools) has options to support writing to any mapping in the chain.
- Django's [Context class](https://github.com/django/django/blob/main/django/template/context.py) for templating is a read-only chain of mappings. It also features pushing and popping of contexts similar to the `~collections.ChainMap.new_child` method and the `~collections.ChainMap.parents` property.
- The [Nested Contexts recipe](https://code.activestate.com/recipes/577434-nested-contexts-a-chain-of-mapping-objects/) has options to control whether writes and other mutations apply only to the first mapping or to any mapping in the chain.
- A [greatly simplified read-only version of Chainmap](https://code.activestate.com/recipes/305268/).

</div>

### `ChainMap` Examples and Recipes

This section shows various approaches to working with chained maps.

Example of simulating Python's internal lookup chain:

    import builtins
    pylookup = ChainMap(locals(), globals(), vars(builtins))

Example of letting user specified command-line arguments take precedence over environment variables which in turn take precedence over default values:

    import os, argparse

    defaults = {'color': 'red', 'user': 'guest'}

    parser = argparse.ArgumentParser()
    parser.add_argument('-u', '--user')
    parser.add_argument('-c', '--color')
    namespace = parser.parse_args()
    command_line_args = {k: v for k, v in vars(namespace).items() if v is not None}

    combined = ChainMap(command_line_args, os.environ, defaults)
    print(combined['color'])
    print(combined['user'])

Example patterns for using the `ChainMap` class to simulate nested contexts:

    c = ChainMap()        # Create root context
    d = c.new_child()     # Create nested child context
    e = c.new_child()     # Child of c, independent from d
    e.maps[0]             # Current context dictionary -- like Python's locals()
    e.maps[-1]            # Root context -- like Python's globals()
    e.parents             # Enclosing context chain -- like Python's nonlocals

    d['x'] = 1            # Set value in current context
    d['x']                # Get first key in the chain of contexts
    del d['x']            # Delete from current context
    list(d)               # All nested values
    k in d                # Check all nested values
    len(d)                # Number of nested values
    d.items()             # All nested items
    dict(d)               # Flatten into a regular dictionary

The `ChainMap` class only makes updates (writes and deletions) to the first mapping in the chain while lookups will search the full chain. However, if deep writes and deletions are desired, it is easy to make a subclass that updates keys found deeper in the chain:

    class DeepChainMap(ChainMap):
        'Variant of ChainMap that allows direct updates to inner scopes'

        def __setitem__(self, key, value):
            for mapping in self.maps:
                if key in mapping:
                    mapping[key] = value
                    return
            self.maps[0][key] = value

        def __delitem__(self, key):
            for mapping in self.maps:
                if key in mapping:
                    del mapping[key]
                    return
            raise KeyError(key)

    >>> d = DeepChainMap({'zebra': 'black'}, {'elephant': 'blue'}, {'lion': 'yellow'})
    >>> d['lion'] = 'orange'         # update an existing key two levels down
    >>> d['snake'] = 'red'           # new keys get added to the topmost dict
    >>> del d['elephant']            # remove an existing key one level down
    >>> d                            # display result
    DeepChainMap({'zebra': 'black', 'snake': 'red'}, {}, {'lion': 'orange'})

## `Counter` objects

A counter tool is provided to support convenient and rapid tallies. For example:

    >>> # Tally occurrences of words in a list
    >>> cnt = Counter()
    >>> for word in ['red', 'blue', 'red', 'green', 'blue', 'blue']:
    ...     cnt[word] += 1
    ...
    >>> cnt
    Counter({'blue': 3, 'red': 2, 'green': 1})

    >>> # Find the ten most common words in Hamlet
    >>> import re
    >>> words = re.findall(r'\w+', open('hamlet.txt').read().lower())
    >>> Counter(words).most_common(10)
    [('the', 1143), ('and', 966), ('to', 762), ('of', 669), ('i', 631),
     ('you', 554),  ('a', 546), ('my', 514), ('hamlet', 471), ('in', 451)]

<div class="Counter(**kwargs) Counter(iterable, /, **kwargs) Counter(mapping, /, **kwargs)">

A `Counter` is a `dict` subclass for counting `hashable` objects. It is a collection where elements are stored as dictionary keys and their counts are stored as dictionary values. Counts are allowed to be any integer value including zero or negative counts. The `Counter` class is similar to bags or multisets in other languages.

Elements are counted from an *iterable* or initialized from another *mapping* (or counter):

> \>\>\> c = Counter() \# a new, empty counter \>\>\> c = Counter('gallahad') \# a new counter from an iterable \>\>\> c = Counter({'red': 4, 'blue': 2}) \# a new counter from a mapping \>\>\> c = Counter(cats=4, dogs=8) \# a new counter from keyword args

Counter objects have a dictionary interface except that they return a zero count for missing items instead of raising a `KeyError`:

> \>\>\> c = Counter(\['eggs', 'ham'\]) \>\>\> c\['bacon'\] \# count of a missing element is zero 0

Setting a count to zero does not remove an element from a counter. Use `del` to remove it entirely:

> \>\>\> c\['sausage'\] = 0 \# counter entry with a zero count \>\>\> del c\['sausage'\] \# del actually removes the entry

Counters maintain insertion order internally but display from most common to least common when possible:

> \>\>\> c = Counter(a=1, b=2, c=3) \>\>\> c \# display most common to least Counter({'c': 3, 'b': 2, 'a': 1}) \>\>\> list(c.items()) \# original insertion order \[('a', 1), ('b', 2), ('c', 3)\]

<div class="versionadded">

3.1

</div>

<div class="versionchanged">

3.7 As a `dict` subclass, `Counter` inherited the capability to remember insertion order. Math operations on *Counter* objects also preserve order. Results are ordered according to when an element is first encountered in the left operand and then by the order encountered in the right operand.

</div>

Counter objects support additional methods beyond those available for all dictionaries:

<div class="method">

elements()

Return an iterator over elements repeating each as many times as its count. Elements are returned in the order first encountered. If an element's count is less than one, `elements` will ignore it.

> \>\>\> c = Counter(a=4, b=2, c=0, d=-2) \>\>\> sorted(c.elements()) \['a', 'a', 'a', 'a', 'b', 'b'\]

</div>

<div class="method">

most_common(n=None)

Return a list of the *n* most common elements and their counts from the most common to the least. If *n* is omitted or `None`, `most_common` returns *all* elements in the counter. Elements with equal counts are ordered in the order first encountered:

> \>\>\> Counter('abracadabra').most_common(3) \[('a', 5), ('b', 2), ('r', 2)\]

</div>

<div class="method">

subtract(**kwargs) subtract(iterable, /,**kwargs) subtract(mapping, /, \*\*kwargs)

Elements are subtracted from an *iterable* or from another *mapping* (or counter). Like `dict.update` but subtracts counts instead of replacing them. Both inputs and outputs may be zero or negative.

> \>\>\> c = Counter(a=4, b=2, c=0, d=-2) \>\>\> d = Counter(a=1, b=2, c=3, d=4) \>\>\> c.subtract(d) \>\>\> c Counter({'a': 3, 'b': 0, 'c': -3, 'd': -6})

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

total()

Compute the sum of the counts.

> \>\>\> c = Counter(a=10, b=5, c=0) \>\>\> c.total() 15

<div class="versionadded">

3.10

</div>

</div>

The usual dictionary methods are available for `Counter` objects except for these two which work differently for counters:

<div class="method">

fromkeys(iterable)

This class method is not implemented for `Counter` objects.

</div>

<div class="method">

update(**kwargs) update(iterable, /,**kwargs) update(mapping, /, \*\*kwargs)

Elements are counted from an *iterable* or added-in from another *mapping* (or counter). Like `dict.update` but adds counts instead of replacing them. Also, the *iterable* is expected to be a sequence of elements, not a sequence of `(key, value)` pairs.

</div>

</div>

Counters support rich comparison operators for equality, subset, and superset relationships: `==`, `!=`, `<`, `<=`, `>`, `>=`. All of those tests treat missing elements as having zero counts so that `Counter(a=1) == Counter(a=1, b=0)` returns true.

<div class="versionchanged">

3.10 Rich comparison operations were added.

</div>

<div class="versionchanged">

3.10 In equality tests, missing elements are treated as having zero counts. Formerly, `Counter(a=3)` and `Counter(a=3, b=0)` were considered distinct.

</div>

Common patterns for working with `Counter` objects:

    c.total()                       # total of all counts
    c.clear()                       # reset all counts
    list(c)                         # list unique elements
    set(c)                          # convert to a set
    dict(c)                         # convert to a regular dictionary
    c.items()                       # access the (elem, cnt) pairs
    Counter(dict(list_of_pairs))    # convert from a list of (elem, cnt) pairs
    c.most_common()[:-n-1:-1]       # n least common elements
    +c                              # remove zero and negative counts

Several mathematical operations are provided for combining `Counter` objects to produce multisets (counters that have counts greater than zero). Addition and subtraction combine counters by adding or subtracting the counts of corresponding elements. Intersection and union return the minimum and maximum of corresponding counts. Symmetric difference returns the difference between the maximum and minimum of the corresponding counts. Equality and inclusion compare corresponding counts. Each operation can accept inputs with signed counts, but the output will exclude results with counts of zero or below.

<div class="doctest">

\>\>\> c = Counter(a=3, b=1) \>\>\> d = Counter(a=1, b=2) \>\>\> c + d \# add two counters together: c\[x\] + d\[x\] Counter({'a': 4, 'b': 3}) \>\>\> c - d \# subtract (keeping only positive counts) Counter({'a': 2}) \>\>\> c & d \# intersection: min(c\[x\], d\[x\]) Counter({'a': 1, 'b': 1}) \>\>\> c \| d \# union: max(c\[x\], d\[x\]) Counter({'a': 3, 'b': 2}) \>\>\> c ^ d \# max(c\[x\], d\[x\]) - min(c\[x\], d\[x\]) Counter({'a': 2, 'b': 1}) \>\>\> c == d \# equality: c\[x\] == d\[x\] False \>\>\> c \<= d \# inclusion: c\[x\] \<= d\[x\] False

</div>

Unary addition and subtraction are shortcuts for adding an empty counter or subtracting from an empty counter.

> \>\>\> c = Counter(a=2, b=-4) \>\>\> +c Counter({'a': 2}) \>\>\> -c Counter({'b': 4})

<div class="versionadded">

3.3 Added support for unary plus, unary minus, and in-place multiset operations.

</div>

<div class="versionadded">

3.15 Added support for the symmetric difference multiset operation, `c ^ d`.

</div>

> [!NOTE]
> Counters were primarily designed to work with positive integers to represent running counts; however, care was taken to not unnecessarily preclude use cases needing other types or negative values. To help with those use cases, this section documents the minimum range and type restrictions.
>
> - The `Counter` class itself is a dictionary subclass with no restrictions on its keys and values. The values are intended to be numbers representing counts, but you *could* store anything in the value field.
> - The `~Counter.most_common` method requires only that the values be orderable.
> - For in-place operations such as `c[key] += 1`, the value type need only support addition and subtraction. So fractions, floats, and decimals would work and negative values are supported. The same is also true for `~Counter.update` and `~Counter.subtract` which allow negative and zero values for both inputs and outputs.
> - The multiset methods are designed only for use cases with positive values. The inputs may be negative or zero, but only outputs with positive values are created. There are no type restrictions, but the value type needs to support addition, subtraction, and comparison.
> - The `~Counter.elements` method requires integer counts. It ignores zero and negative counts.

<div class="seealso">

- [Bag class](https://www.gnu.org/software/smalltalk/manual-base/html_node/Bag.html) in Smalltalk.

- Wikipedia entry for [Multisets](https://en.wikipedia.org/wiki/Multiset).

- [C++ multisets](http://www.java2s.com/Tutorial/Cpp/0380__set-multiset/Catalog0380__set-multiset.htm) tutorial with examples.

- For mathematical operations on multisets and their use cases, see *Knuth, Donald. The Art of Computer Programming Volume II, Section 4.6.3, Exercise 19*.

- To enumerate all distinct multisets of a given size over a given set of elements, see `itertools.combinations_with_replacement`:

      map(Counter, combinations_with_replacement('ABC', 2)) # --> AA AB AC BB BC CC

</div>

## `deque` objects

<div class="deque([iterable, [maxlen]])">

Returns a new deque object initialized left-to-right (using `append`) with data from *iterable*. If *iterable* is not specified, the new deque is empty.

Deques are a generalization of stacks and queues (the name is pronounced "deck" and is short for "double-ended queue"). Deques support thread-safe, memory efficient appends and pops from either side of the deque with approximately the same *O*(1) performance in either direction.

Though `list` objects support similar operations, they are optimized for fast fixed-length operations and incur *O*(*n*) memory movement costs for `pop(0)` and `insert(0, v)` operations which change both the size and position of the underlying data representation.

If *maxlen* is not specified or is `None`, deques may grow to an arbitrary length. Otherwise, the deque is bounded to the specified maximum length. Once a bounded length deque is full, when new items are added, a corresponding number of items are discarded from the opposite end. Bounded length deques provide functionality similar to the `tail` filter in Unix. They are also useful for tracking transactions and other pools of data where only the most recent activity is of interest.

Deques are `generic <generics>` over the type of their contents.

Deque objects support the following methods:

<div class="method">

append(item, /)

Add *item* to the right side of the deque.

</div>

<div class="method">

appendleft(item, /)

Add *item* to the left side of the deque.

</div>

<div class="method">

clear()

Remove all elements from the deque leaving it with length 0.

</div>

<div class="method">

copy()

Create a shallow copy of the deque.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

count(value, /)

Count the number of deque elements equal to *value*.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

extend(iterable, /)

Extend the right side of the deque by appending elements from the iterable argument.

</div>

<div class="method">

extendleft(iterable, /)

Extend the left side of the deque by appending elements from *iterable*. Note, the series of left appends results in reversing the order of elements in the iterable argument.

</div>

<div class="method">

index(value\[, start\[, stop\]\])

Return the position of *value* in the deque (at or after index *start* and before index *stop*). Returns the first match or raises `ValueError` if not found.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

insert(index, value, /)

Insert *value* into the deque at position *index*.

If the insertion would cause a bounded deque to grow beyond *maxlen*, an `IndexError` is raised.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

pop()

Remove and return an element from the right side of the deque. If no elements are present, raises an `IndexError`.

</div>

<div class="method">

popleft()

Remove and return an element from the left side of the deque. If no elements are present, raises an `IndexError`.

</div>

<div class="method">

remove(value, /)

Remove the first occurrence of *value*. If not found, raises a `ValueError`.

</div>

<div class="method">

reverse()

Reverse the elements of the deque in-place and then return `None`.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

rotate(n=1, /)

Rotate the deque *n* steps to the right. If *n* is negative, rotate to the left.

When the deque is not empty, rotating one step to the right is equivalent to `d.appendleft(d.pop())`, and rotating one step to the left is equivalent to `d.append(d.popleft())`.

</div>

Deque objects also provide one read-only attribute:

<div class="attribute">

maxlen

Maximum size of a deque or `None` if unbounded.

<div class="versionadded">

3.1

</div>

</div>

</div>

In addition to the above, deques support iteration, pickling, `len(d)`, `reversed(d)`, `copy.copy(d)`, `copy.deepcopy(d)`, membership testing with the `in` operator, and subscript references such as `d[0]` to access the first element. Indexed access is *O*(1) at both ends but slows to *O*(*n*) in the middle. For fast random access, use lists instead.

Starting in version 3.5, deques support `__add__()`, `__mul__()`, and `__imul__()`.

Example:

<div class="doctest">

\>\>\> from collections import deque \>\>\> d = deque('ghi') \# make a new deque with three items \>\>\> for elem in d: \# iterate over the deque's elements ... print(elem.upper()) G H I

\>\>\> d.append('j') \# add a new entry to the right side \>\>\> d.appendleft('f') \# add a new entry to the left side \>\>\> d \# show the representation of the deque deque(\['f', 'g', 'h', 'i', 'j'\])

\>\>\> d.pop() \# return and remove the rightmost item 'j' \>\>\> d.popleft() \# return and remove the leftmost item 'f' \>\>\> list(d) \# list the contents of the deque \['g', 'h', 'i'\] \>\>\> d\[0\] \# peek at leftmost item 'g' \>\>\> d\[-1\] \# peek at rightmost item 'i'

\>\>\> list(reversed(d)) \# list the contents of a deque in reverse \['i', 'h', 'g'\] \>\>\> 'h' in d \# search the deque True \>\>\> d.extend('jkl') \# add multiple elements at once \>\>\> d deque(\['g', 'h', 'i', 'j', 'k', 'l'\]) \>\>\> d.rotate(1) \# right rotation \>\>\> d deque(\['l', 'g', 'h', 'i', 'j', 'k'\]) \>\>\> d.rotate(-1) \# left rotation \>\>\> d deque(\['g', 'h', 'i', 'j', 'k', 'l'\])

\>\>\> deque(reversed(d)) \# make a new deque in reverse order deque(\['l', 'k', 'j', 'i', 'h', 'g'\]) \>\>\> d.clear() \# empty the deque \>\>\> d.pop() \# cannot pop from an empty deque Traceback (most recent call last): File "\<pyshell#6\>", line 1, in -toplevel- d.pop() IndexError: pop from an empty deque

\>\>\> d.extendleft('abc') \# extendleft() reverses the input order \>\>\> d deque(\['c', 'b', 'a'\])

</div>

### `deque` Recipes

This section shows various approaches to working with deques.

Bounded length deques provide functionality similar to the `tail` filter in Unix:

    def tail(filename, n=10):
        'Return the last n lines of a file'
        with open(filename) as f:
            return deque(f, n)

Another approach to using deques is to maintain a sequence of recently added elements by appending to the right and popping to the left:

    def moving_average(iterable, n=3):
        # moving_average([40, 30, 50, 46, 39, 44]) --> 40.0 42.0 45.0 43.0
        # https://en.wikipedia.org/wiki/Moving_average
        it = iter(iterable)
        d = deque(itertools.islice(it, n-1))
        d.appendleft(0)
        s = sum(d)
        for elem in it:
            s += elem - d.popleft()
            d.append(elem)
            yield s / n

A [round-robin scheduler](https://en.wikipedia.org/wiki/Round-robin_scheduling) can be implemented with input iterators stored in a `deque`. Values are yielded from the active iterator in position zero. If that iterator is `exhausted`, it can be removed with `~deque.popleft`; otherwise, it can be cycled back to the end with the `~deque.rotate` method:

    def roundrobin(*iterables):
        "roundrobin('ABC', 'D', 'EF') --> A D E B F C"
        iterators = deque(map(iter, iterables))
        while iterators:
            try:
                while True:
                    yield next(iterators[0])
                    iterators.rotate(-1)
            except StopIteration:
                # Remove an exhausted iterator.
                iterators.popleft()

The `~deque.rotate` method provides a way to implement `deque` slicing and deletion. For example, a pure Python implementation of `del d[n]` relies on the `rotate()` method to position elements to be popped:

    def delete_nth(d, n):
        d.rotate(-n)
        d.popleft()
        d.rotate(n)

To implement `deque` slicing, use a similar approach applying `~deque.rotate` to bring a target element to the left side of the deque. Remove old entries with `~deque.popleft`, add new entries with `~deque.extend`, and then reverse the rotation. With minor variations on that approach, it is easy to implement Forth style stack manipulations such as `dup`, `drop`, `swap`, `over`, `pick`, `rot`, and `roll`.

## `defaultdict` objects

<div class="defaultdict(default_factory=None, /, **kwargs) defaultdict(default_factory, mapping, /, **kwargs) defaultdict(default_factory, iterable, /, **kwargs)">

Return a new dictionary-like object. `defaultdict` is a subclass of the built-in `dict` class. It overrides one method and adds one writable instance variable. The remaining functionality is the same as for the `dict` class and is not documented here.

The first argument provides the initial value for the `default_factory` attribute; it defaults to `None`. All remaining arguments are treated the same as if they were passed to the `dict` constructor, including keyword arguments.

`!defaultdict`s are `generic <generics>` over two types, signifying (respectively) the types of the dictionary's keys and values.

`defaultdict` objects support the following method in addition to the standard `dict` operations:

<div class="method">

\_\_missing\_\_(key, /)

If the `default_factory` attribute is `None`, this raises a `KeyError` exception with the *key* as argument.

If `default_factory` is not `None`, it is called without arguments to provide a default value for the given *key*, this value is inserted in the dictionary for the *key*, and returned.

If calling `default_factory` raises an exception this exception is propagated unchanged.

This method is called by the `~object.__getitem__` method of the `dict` class when the requested key is not found; whatever it returns or raises is then returned or raised by `~object.__getitem__`.

Note that `__missing__` is *not* called for any operations besides `~object.__getitem__`. This means that `~dict.get` will, like normal dictionaries, return `None` as a default rather than using `default_factory`.

</div>

`defaultdict` objects support the following instance variable:

<div class="attribute">

default_factory

This attribute is used by the `~defaultdict.__missing__` method; it is initialized from the first argument to the constructor, if present, or to `None`, if absent.

</div>

<div class="versionchanged">

3.9 Added merge (`|`) and update (`|=`) operators, specified in `584`.

</div>

</div>

### `defaultdict` Examples

Using `list` as the `~defaultdict.default_factory`, it is easy to group a sequence of key-value pairs into a dictionary of lists:

> \>\>\> s = \[('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)\] \>\>\> d = defaultdict(list) \>\>\> for k, v in s: ... d\[k\].append(v) ... \>\>\> sorted(d.items()) \[('blue', \[2, 4\]), ('red', \[1\]), ('yellow', \[1, 3\])\]

When each key is encountered for the first time, it is not already in the mapping; so an entry is automatically created using the `~defaultdict.default_factory` function which returns an empty `list`. The `list.append` operation then attaches the value to the new list. When keys are encountered again, the look-up proceeds normally (returning the list for that key) and the `list.append` operation adds another value to the list. This technique is simpler and faster than an equivalent technique using `dict.setdefault`:

> \>\>\> d = {} \>\>\> for k, v in s: ... d.setdefault(k, \[\]).append(v) ... \>\>\> sorted(d.items()) \[('blue', \[2, 4\]), ('red', \[1\]), ('yellow', \[1, 3\])\]

Setting the `~defaultdict.default_factory` to `int` makes the `defaultdict` useful for counting (like a bag or multiset in other languages):

> \>\>\> s = 'mississippi' \>\>\> d = defaultdict(int) \>\>\> for k in s: ... d\[k\] += 1 ... \>\>\> sorted(d.items()) \[('i', 4), ('m', 1), ('p', 2), ('s', 4)\]

When a letter is first encountered, it is missing from the mapping, so the `~defaultdict.default_factory` function calls `int` to supply a default count of zero. The increment operation then builds up the count for each letter.

The function `int` which always returns zero is just a special case of constant functions. A faster and more flexible way to create constant functions is to use a lambda function which can supply any constant value (not just zero):

> \>\>\> def constant_factory(value): ... return lambda: value ... \>\>\> d = defaultdict(constant_factory('\<missing\>')) \>\>\> d.update(name='John', action='ran') \>\>\> '%(name)s %(action)s to %(object)s' % d 'John ran to \<missing\>'

Setting the `~defaultdict.default_factory` to `set` makes the `defaultdict` useful for building a dictionary of sets:

> \>\>\> s = \[('red', 1), ('blue', 2), ('red', 3), ('blue', 4), ('red', 1), ('blue', 4)\] \>\>\> d = defaultdict(set) \>\>\> for k, v in s: ... d\[k\].add(v) ... \>\>\> sorted(d.items()) \[('blue', {2, 4}), ('red', {1, 3})\]

## `namedtuple` Factory Function for Tuples with Named Fields

Named tuples assign meaning to each position in a tuple and allow for more readable, self-documenting code. They can be used wherever regular tuples are used, and they add the ability to access fields by name instead of position index.

<div class="function">

namedtuple(typename, field_names, \*, rename=False, defaults=None, module=None)

Returns a new tuple subclass named *typename*. The new subclass is used to create tuple-like objects that have fields accessible by attribute lookup as well as being indexable and iterable. Instances of the subclass also have a helpful docstring (with *typename* and *field_names*) and a helpful `~object.__repr__` method which lists the tuple contents in a `name=value` format.

The *field_names* are a sequence of strings such as `['x', 'y']`. Alternatively, *field_names* can be a single string with each fieldname separated by whitespace and/or commas, for example `'x y'` or `'x, y'`.

Any valid Python identifier may be used for a fieldname except for names starting with an underscore. Valid identifiers consist of letters, digits, and underscores but do not start with a digit or underscore and cannot be a `keyword` such as *class*, *for*, *return*, *global*, *pass*, or *raise*.

If *rename* is true, invalid fieldnames are automatically replaced with positional names. For example, `['abc', 'def', 'ghi', 'abc']` is converted to `['abc', '_1', 'ghi', '_3']`, eliminating the keyword `def` and the duplicate fieldname `abc`.

*defaults* can be `None` or an `iterable` of default values. Since fields with a default value must come after any fields without a default, the *defaults* are applied to the rightmost parameters. For example, if the fieldnames are `['x', 'y', 'z']` and the defaults are `(1, 2)`, then `x` will be a required argument, `y` will default to `1`, and `z` will default to `2`.

If *module* is defined, the `~type.__module__` attribute of the named tuple is set to that value.

Named tuple instances do not have per-instance dictionaries, so they are lightweight and require no more memory than regular tuples.

To support pickling, the named tuple class should be assigned to a variable that matches *typename*.

<div class="versionchanged">

3.1 Added support for *rename*.

</div>

<div class="versionchanged">

3.6 The *verbose* and *rename* parameters became `keyword-only arguments <keyword-only_parameter>`.

</div>

<div class="versionchanged">

3.6 Added the *module* parameter.

</div>

<div class="versionchanged">

3.7 Removed the *verbose* parameter and the `!_source` attribute.

</div>

<div class="versionchanged">

3.7 Added the *defaults* parameter and the `~somenamedtuple._field_defaults` attribute.

</div>

</div>

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> \# Basic example \>\>\> Point = namedtuple('Point', \['x', 'y'\]) \>\>\> p = Point(11, y=22) \# instantiate with positional or keyword arguments \>\>\> p\[0\] + p\[1\] \# indexable like the plain tuple (11, 22) 33 \>\>\> x, y = p \# unpack like a regular tuple \>\>\> x, y (11, 22) \>\>\> p.x + p.y \# fields also accessible by name 33 \>\>\> p \# readable \_\_repr\_\_ with a name=value style Point(x=11, y=22)

</div>

Named tuples are especially useful for assigning field names to result tuples returned by the `csv` or `sqlite3` modules:

    EmployeeRecord = namedtuple('EmployeeRecord', 'name, age, title, department, paygrade')

    import csv
    for emp in map(EmployeeRecord._make, csv.reader(open("employees.csv", "rb"))):
        print(emp.name, emp.title)

    import sqlite3
    conn = sqlite3.connect('/companydata')
    cursor = conn.cursor()
    cursor.execute('SELECT name, age, title, department, paygrade FROM employees')
    for emp in map(EmployeeRecord._make, cursor.fetchall()):
        print(emp.name, emp.title)

In addition to the methods inherited from tuples, named tuples support three additional methods and two attributes. To prevent conflicts with field names, the method and attribute names start with an underscore.

<div class="classmethod">

somenamedtuple.\_make(iterable, /)

Class method that makes a new instance from an existing sequence or iterable.

<div class="doctest">

\>\>\> t = \[11, 22\] \>\>\> Point.\_make(t) Point(x=11, y=22)

</div>

</div>

<div class="method">

somenamedtuple.\_asdict()

Return a new `dict` which maps field names to their corresponding values:

<div class="doctest">

\>\>\> p = Point(x=11, y=22) \>\>\> p.\_asdict() {'x': 11, 'y': 22}

</div>

<div class="versionchanged">

3.1 Returns an `OrderedDict` instead of a regular `dict`.

</div>

<div class="versionchanged">

3.8 Returns a regular `dict` instead of an `OrderedDict`. As of Python 3.7, regular dicts are guaranteed to be ordered. If the extra features of `OrderedDict` are required, the suggested remediation is to cast the result to the desired type: `OrderedDict(nt._asdict())`.

</div>

</div>

<div class="method">

somenamedtuple.\_replace(\*\*kwargs)

Return a new instance of the named tuple replacing specified fields with new values:

    >>> p = Point(x=11, y=22)
    >>> p._replace(x=33)
    Point(x=33, y=22)

    >>> for partnum, record in inventory.items():
    ...     inventory[partnum] = record._replace(price=newprices[partnum], timestamp=time.now())

Named tuples are also supported by generic function `copy.replace`.

<div class="versionchanged">

3.13 Raise `TypeError` instead of `ValueError` for invalid keyword arguments.

</div>

</div>

<div class="attribute">

somenamedtuple.\_fields

Tuple of strings listing the field names. Useful for introspection and for creating new named tuple types from existing named tuples.

<div class="doctest">

\>\>\> p.\_fields \# view the field names ('x', 'y')

\>\>\> Color = namedtuple('Color', 'red green blue') \>\>\> Pixel = namedtuple('Pixel', Point.\_fields + Color.\_fields) \>\>\> Pixel(11, 22, 128, 255, 0) Pixel(x=11, y=22, red=128, green=255, blue=0)

</div>

</div>

<div class="attribute">

somenamedtuple.\_field_defaults

Dictionary mapping field names to default values.

<div class="doctest">

\>\>\> Account = namedtuple('Account', \['type', 'balance'\], defaults=\[0\]) \>\>\> Account.\_field_defaults {'balance': 0} \>\>\> Account('premium') Account(type='premium', balance=0)

</div>

</div>

To retrieve a field whose name is stored in a string, use the `getattr` function:

> \>\>\> getattr(p, 'x') 11

To convert a dictionary to a named tuple, use the double-star-operator (as described in `tut-unpacking-arguments`):

> \>\>\> d = {'x': 11, 'y': 22} \>\>\> Point(\*\*d) Point(x=11, y=22)

Since a named tuple is a regular Python class, it is easy to add or change functionality with a subclass. Here is how to add a calculated field and a fixed-width print format:

<div class="doctest">

\>\>\> class Point(namedtuple('Point', \['x', 'y'\])): ... \_\_slots\_\_ = () ... @property ... def hypot(self): ... return (self.x \*\* 2 + self.y \*\* 2) \*\* 0.5 ... def \_\_str\_\_(self): ... return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot)

\>\>\> for p in Point(3, 4), Point(14, 5/7): ... print(p) Point: x= 3.000 y= 4.000 hypot= 5.000 Point: x=14.000 y= 0.714 hypot=14.018

</div>

The subclass shown above sets `__slots__` to an empty tuple. This helps keep memory requirements low by preventing the creation of instance dictionaries.

Subclassing is not useful for adding new, stored fields. Instead, simply create a new named tuple type from the `~somenamedtuple._fields` attribute:

> \>\>\> Point3D = namedtuple('Point3D', Point.\_fields + ('z',))

Docstrings can be customized by making direct assignments to the `__doc__` fields:

> \>\>\> Book = namedtuple('Book', \['id', 'title', 'authors'\]) \>\>\> Book.\_\_doc\_\_ += ': Hardcover book in active collection' \>\>\> Book.id.\_\_doc\_\_ = '13-digit ISBN' \>\>\> Book.title.\_\_doc\_\_ = 'Title of first printing' \>\>\> Book.authors.\_\_doc\_\_ = 'List of authors sorted by last name'

<div class="versionchanged">

3.5 Property docstrings became writeable.

</div>

<div class="seealso">

- See `typing.NamedTuple` for a way to add type hints for named tuples. It also provides an elegant notation using the `class` keyword:

      class Component(NamedTuple):
          part_number: int
          weight: float
          description: Optional[str] = None

- See `types.SimpleNamespace` for a mutable namespace based on an underlying dictionary instead of a tuple.

- The `dataclasses` module provides a decorator and functions for automatically adding generated special methods to user-defined classes.

</div>

## `OrderedDict` objects

Ordered dictionaries are just like regular dictionaries but have some extra capabilities relating to ordering operations. They have become less important now that the built-in `dict` class gained the ability to remember insertion order (this new behavior became guaranteed in Python 3.7).

Some differences from `dict` still remain:

- The regular `dict` was designed to be very good at mapping operations. Tracking insertion order was secondary.

- The `OrderedDict` was designed to be good at reordering operations. Space efficiency, iteration speed, and the performance of update operations were secondary.

- The `OrderedDict` algorithm can handle frequent reordering operations better than `dict`. As shown in the recipes below, this makes it suitable for implementing various kinds of LRU caches.

- The equality operation for `OrderedDict` checks for matching order.

  A regular `dict` can emulate the order sensitive equality test with `p == q and all(k1 == k2 for k1, k2 in zip(p, q))`.

- The `~OrderedDict.popitem` method of `OrderedDict` has a different signature. It accepts an optional argument to specify which item is popped.

  A regular `dict` can emulate OrderedDict's `od.popitem(last=True)` with `d.popitem()` which is guaranteed to pop the rightmost (last) item.

  A regular `dict` can emulate OrderedDict's `od.popitem(last=False)` with `(k := next(iter(d)), d.pop(k))` which will return and remove the leftmost (first) item if it exists.

- `OrderedDict` has a `~OrderedDict.move_to_end` method to efficiently reposition an element to an endpoint.

  A regular `dict` can emulate OrderedDict's `od.move_to_end(k, last=True)` with `d[k] = d.pop(k)` which will move the key and its associated value to the rightmost (last) position.

  A regular `dict` does not have an efficient equivalent for OrderedDict's `od.move_to_end(k, last=False)` which moves the key and its associated value to the leftmost (first) position.

- Until Python 3.8, `dict` lacked a `~object.__reversed__` method.

<div class="OrderedDict(**kwargs) OrderedDict(mapping, /, **kwargs) OrderedDict(iterable, /, **kwargs)">

Return an instance of a `dict` subclass that has methods specialized for rearranging dictionary order.

<div class="versionadded">

3.1

</div>

<div class="method">

popitem(last=True)

The `popitem` method for ordered dictionaries returns and removes a (key, value) pair. The pairs are returned in `LIFO (last-in, first-out)` order if *last* is true or `FIFO (first-in, first-out)` order if false.

</div>

<div class="method">

move_to_end(key, last=True)

Move an existing *key* to either end of an ordered dictionary. The item is moved to the right end if *last* is true (the default) or to the beginning if *last* is false. Raises `KeyError` if the *key* does not exist:

<div class="doctest">

\>\>\> d = OrderedDict.fromkeys('abcde') \>\>\> d.move_to_end('b') \>\>\> ''.join(d) 'acdeb' \>\>\> d.move_to_end('b', last=False) \>\>\> ''.join(d) 'bacde'

</div>

<div class="versionadded">

3.2

</div>

</div>

</div>

In addition to the usual mapping methods, ordered dictionaries also support reverse iteration using `reversed`.

<div id="collections_OrderedDict__eq__">

Equality tests between `OrderedDict` objects are order-sensitive and are roughly equivalent to `list(od1.items())==list(od2.items())`.

</div>

Equality tests between `OrderedDict` objects and other `~collections.abc.Mapping` objects are order-insensitive like regular dictionaries. This allows `OrderedDict` objects to be substituted anywhere a regular dictionary is used.

<div class="versionchanged">

3.5 The items, keys, and values `views <dictionary view>` of `OrderedDict` now support reverse iteration using `reversed`.

</div>

<div class="versionchanged">

3.6 With the acceptance of `468`, order is retained for keyword arguments passed to the `OrderedDict` constructor and its `~dict.update` method.

</div>

<div class="versionchanged">

3.9 Added merge (`|`) and update (`|=`) operators, specified in `584`.

</div>

### `OrderedDict` Examples and Recipes

It is straightforward to create an ordered dictionary variant that remembers the order the keys were *last* inserted. If a new entry overwrites an existing entry, the original insertion position is changed and moved to the end:

    class LastUpdatedOrderedDict(OrderedDict):
        'Store items in the order that the keys were last updated.'

        def __setitem__(self, key, value):
            super().__setitem__(key, value)
            self.move_to_end(key)

An `OrderedDict` would also be useful for implementing variants of `functools.lru_cache`:

<div class="testcode">

from collections import OrderedDict from time import monotonic

class TimeBoundedLRU:  
"LRU Cache that invalidates and refreshes old entries."

def \_\_init\_\_(self, func, maxsize=128, maxage=30):  
self.cache = OrderedDict() \# { args : (timestamp, result)} self.func = func self.maxsize = maxsize self.maxage = maxage

def \_\_call\_\_(self, *args): if args in self.cache: self.cache.move_to_end(args) timestamp, result = self.cache\[args\] if monotonic() - timestamp \<= self.maxage: return result result = self.func(*args)  
self.cache\[args\] = monotonic(), result if len(self.cache) \> self.maxsize: self.cache.popitem(last=False) return result

</div>

<div class="testcode">

class MultiHitLRUCache:  
""" LRU cache that defers caching a result until  
it has been requested multiple times.

To avoid flushing the LRU cache with one-time requests, we don't cache until a request has been made more than once.

"""

def \_\_init\_\_(self, func, maxsize=128, maxrequests=4096, cache_after=1):  
self.requests = OrderedDict() \# { uncached_key : request_count } self.cache = OrderedDict() \# { cached_key : function_result } self.func = func self.maxrequests = maxrequests \# max number of uncached requests self.maxsize = maxsize \# max number of stored return values self.cache_after = cache_after

def \_\_call\_\_(self, *args): if args in self.cache: self.cache.move_to_end(args) return self.cache\[args\] result = self.func(*args)  
self.requests\[args\] = self.requests.get(args, 0) + 1 if self.requests\[args\] \<= self.cache_after: self.requests.move_to_end(args) if len(self.requests) \> self.maxrequests: self.requests.popitem(last=False) else: self.requests.pop(args, None) self.cache\[args\] = result if len(self.cache) \> self.maxsize: self.cache.popitem(last=False) return result

</div>

<div class="doctest" data-hide="">

\>\>\> def square(x): ... return x \* x ... \>\>\> f = MultiHitLRUCache(square, maxsize=4, maxrequests=6) \>\>\> list(map(f, range(10))) \# First requests, don't cache \[0, 1, 4, 9, 16, 25, 36, 49, 64, 81\] \>\>\> f(4) \# Cache the second request 16 \>\>\> f(6) \# Cache the second request 36 \>\>\> f(2) \# The first request aged out, so don't cache 4 \>\>\> f(6) \# Cache hit 36 \>\>\> f(4) \# Cache hit and move to front 16 \>\>\> list(f.cache.values()) \[36, 16\] \>\>\> set(f.requests).isdisjoint(f.cache) True \>\>\> list(map(f, \[9, 8, 7\])) \# Cache these second requests \[81, 64, 49\] \>\>\> list(map(f, \[7, 9\])) \# Cache hits \[49, 81\] \>\>\> list(f.cache.values()) \[16, 64, 49, 81\] \>\>\> set(f.requests).isdisjoint(f.cache) True

</div>

## `UserDict` objects

The class, `UserDict` acts as a wrapper around dictionary objects. The need for this class has been partially supplanted by the ability to subclass directly from `dict`; however, this class can be easier to work with because the underlying dictionary is accessible as an attribute.

<div class="UserDict(**kwargs) UserDict(mapping, /, **kwargs) UserDict(iterable, /, **kwargs)">

Class that simulates a dictionary. The instance's contents are kept in a regular dictionary, which is accessible via the `data` attribute of `!UserDict` instances. If arguments are provided, they are used to initialize `data`, like a regular dictionary.

In addition to supporting the methods and operations of mappings, `!UserDict` instances provide the following attribute:

<div class="attribute">

data

A real dictionary used to store the contents of the `UserDict` class.

</div>

`!UserDict` instances also override the following method:

<div class="method">

popitem

Remove and return a `(key, value)` pair from the wrapped dictionary. Pairs are returned in the same order as `data.popitem()`. (For the default `dict.popitem`, this order is `LIFO (last-in, first-out)`.) If the dictionary is empty, raises a `KeyError`.

</div>

</div>

## `UserList` objects

This class acts as a wrapper around list objects. It is a useful base class for your own list-like classes which can inherit from them and override existing methods or add new ones. In this way, one can add new behaviors to lists.

The need for this class has been partially supplanted by the ability to subclass directly from `list`; however, this class can be easier to work with because the underlying list is accessible as an attribute.

<div class="UserList([list])">

Class that simulates a list. The instance's contents are kept in a regular list, which is accessible via the `data` attribute of `UserList` instances. The instance's contents are initially set to a copy of *list*, defaulting to the empty list `[]`. *list* can be any iterable, for example a real Python list or a `UserList` object.

In addition to supporting the methods and operations of mutable sequences, `UserList` instances provide the following attribute:

<div class="attribute">

data

A real `list` object used to store the contents of the `UserList` class.

</div>

</div>

**Subclassing requirements:** Subclasses of `UserList` are expected to offer a constructor which can be called with either no arguments or one argument. List operations which return a new sequence attempt to create an instance of the actual implementation class. To do so, it assumes that the constructor can be called with a single parameter, which is a sequence object used as a data source.

If a derived class does not wish to comply with this requirement, all of the special methods supported by this class will need to be overridden; please consult the sources for information about the methods which need to be provided in that case.

## `UserString` objects

The class, `UserString` acts as a wrapper around string objects. The need for this class has been partially supplanted by the ability to subclass directly from `str`; however, this class can be easier to work with because the underlying string is accessible as an attribute.

<div class="UserString(seq)">

Class that simulates a string object. The instance's content is kept in a regular string object, which is accessible via the `data` attribute of `UserString` instances. The instance's contents are initially set to a copy of *seq*. The *seq* argument can be any object which can be converted into a string using the built-in `str` function.

In addition to supporting the methods and operations of strings, `UserString` instances provide the following attribute:

<div class="attribute">

data

A real `str` object used to store the contents of the `UserString` class.

</div>

<div class="versionchanged">

3.5 New methods `__getnewargs__`, `__rmod__`, `casefold`, `format_map`, `isprintable`, and `maketrans`.

</div>

</div>

# `!colorsys` --- Conversions between color systems

<div class="module" data-synopsis="Conversion functions between RGB and other color systems.">

colorsys

</div>

**Source code:** `Lib/colorsys.py`

------------------------------------------------------------------------

The `!colorsys` module defines bidirectional conversions of color values between colors expressed in the RGB (Red Green Blue) color space used in computer monitors and three other coordinate systems: YIQ, HLS (Hue Lightness Saturation) and HSV (Hue Saturation Value). Coordinates in all of these color spaces are floating-point values. In the YIQ space, the Y coordinate is between 0 and 1, but the I and Q coordinates can be positive or negative. In all other spaces, the coordinates are all between 0 and 1.

<div class="seealso">

More information about color spaces can be found at <https://www.poynton.ca/pdf/ColourFAQ.pdf> and <https://www.cambridgeincolour.com/tutorials/color-spaces.htm>.

</div>

The `!colorsys` module defines the following functions:

<div class="function">

rgb_to_yiq(r, g, b)

Convert the color from RGB coordinates to YIQ coordinates.

</div>

<div class="function">

yiq_to_rgb(y, i, q)

Convert the color from YIQ coordinates to RGB coordinates.

</div>

<div class="function">

rgb_to_hls(r, g, b)

Convert the color from RGB coordinates to HLS coordinates.

</div>

<div class="function">

hls_to_rgb(h, l, s)

Convert the color from HLS coordinates to RGB coordinates.

</div>

<div class="function">

rgb_to_hsv(r, g, b)

Convert the color from RGB coordinates to HSV coordinates.

</div>

<div class="function">

hsv_to_rgb(h, s, v)

Convert the color from HSV coordinates to RGB coordinates.

</div>

Example:

    >>> import colorsys
    >>> colorsys.rgb_to_hsv(0.2, 0.4, 0.4)
    (0.5, 0.5, 0.4)
    >>> colorsys.hsv_to_rgb(0.5, 0.5, 0.4)
    (0.2, 0.4, 0.4)

# `!compileall` --- Byte-compile Python libraries

<div class="module" data-synopsis="Tools for byte-compiling all Python source files in a directory tree.">

compileall

</div>

**Source code:** `Lib/compileall.py`

------------------------------------------------------------------------

This module provides some utility functions to support installing Python libraries. These functions compile Python source files in a directory tree. This module can be used to create the cached byte-code files at library installation time, which makes them available for use even by users who don't have write permission to the library directories.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

## Command-line use

This module can work as a script (using `python -m compileall`) to compile Python sources.

<div class="program">

compileall

</div>

<div class="option">

directory ... file ...

Positional arguments are files to compile or directories that contain source files, traversed recursively. If no argument is given, behave as if the command line was `-l {<directories from sys.path>}`.

</div>

<div class="option">

`-l`  

Do not recurse into subdirectories, only compile source code files directly contained in the named or implied directories.

</div>

<div class="option">

`-f`  

Force rebuild even if timestamps are up-to-date.

</div>

<div class="option">

`-q`  

Do not print the list of files compiled. If passed once, error messages will still be printed. If passed twice (`-qq`), all output is suppressed.

</div>

<div class="option">

`-d destdir`  

Directory prepended to the path to each file being compiled. This will appear in compilation time tracebacks, and is also compiled in to the byte-code file, where it will be used in tracebacks and other messages in cases where the source file does not exist at the time the byte-code file is executed.

</div>

<div class="option">

`-s strip_prefix`  

Remove the given prefix from paths recorded in the `.pyc` files. Paths are made relative to the prefix.

This option can be used with `-p` but not with `-d`.

</div>

<div class="option">

`-p prepend_prefix`  

Prepend the given prefix to paths recorded in the `.pyc` files. Use `-p /` to make the paths absolute.

This option can be used with `-s` but not with `-d`.

</div>

<div class="option">

`-x regex`  

regex is used to search the full path to each file considered for compilation, and if the regex produces a match, the file is skipped.

</div>

<div class="option">

`-i list`  

Read the file `list` and add each line that it contains to the list of files and directories to compile. If `list` is `-`, read lines from `stdin`.

</div>

<div class="option">

`-b`  

Write the byte-code files to their legacy locations and names, which may overwrite byte-code files created by another version of Python. The default is to write files to their `3147` locations and names, which allows byte-code files from multiple versions of Python to coexist.

</div>

<div class="option">

`-r`  

Control the maximum recursion level for subdirectories. If this is given, then `-l` option will not be taken into account. `python -m compileall <directory> -r 0` is equivalent to `python -m compileall <directory> -l`.

</div>

<div class="option">

`-j N`  

Use *N* workers to compile the files within the given directory. If `0` is used, then the result of `os.process_cpu_count` will be used.

</div>

<div class="option">

--invalidation-mode \[timestamp\|checked-hash\|unchecked-hash\]

Control how the generated byte-code files are invalidated at runtime. The `timestamp` value, means that `.pyc` files with the source timestamp and size embedded will be generated. The `checked-hash` and `unchecked-hash` values cause hash-based pycs to be generated. Hash-based pycs embed a hash of the source file contents rather than a timestamp. See `pyc-invalidation` for more information on how Python validates bytecode cache files at runtime. The default is `timestamp` if the `SOURCE_DATE_EPOCH` environment variable is not set, and `checked-hash` if the `SOURCE_DATE_EPOCH` environment variable is set.

</div>

<div class="option">

`-o level`  

Compile with the given optimization level. May be used multiple times to compile for multiple levels at a time (for example, `compileall -o 1 -o 2`).

</div>

<div class="option">

`-e dir`  

Ignore symlinks pointing outside the given directory.

</div>

<div class="option">

`--hardlink-dupes`  

If two `.pyc` files with different optimization level have the same content, use hard links to consolidate duplicate files.

</div>

<div class="versionchanged">

3.2 Added the `-i`, `-b` and `-h` options.

</div>

<div class="versionchanged">

3.5 Added the `-j`, `-r`, and `-qq` options. `-q` option was changed to a multilevel value. `-b` will always produce a byte-code file ending in `.pyc`, never `.pyo`.

</div>

<div class="versionchanged">

3.7 Added the `--invalidation-mode` option.

</div>

<div class="versionchanged">

3.9 Added the `-s`, `-p`, `-e` and `--hardlink-dupes` options. Raised the default recursion limit from 10 to `sys.getrecursionlimit()`. Added the possibility to specify the `-o` option multiple times.

</div>

There is no command-line option to control the optimization level used by the `compile` function, because the Python interpreter itself already provides the option: `python -O -m compileall`.

Similarly, the `compile` function respects the `sys.pycache_prefix` setting. The generated bytecode cache will only be useful if `compile` is run with the same `sys.pycache_prefix` (if any) that will be used at runtime.

## Public functions

<div class="function">

compile_dir(dir, maxlevels=sys.getrecursionlimit(), ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None, \*, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False)

Recursively descend the directory tree named by *dir*, compiling all `.py` files along the way. Return a true value if all the files compiled successfully, and a false value otherwise.

The *maxlevels* parameter is used to limit the depth of the recursion; it defaults to `sys.getrecursionlimit()`.

If *ddir* is given, it is prepended to the path to each file being compiled for use in compilation time tracebacks, and is also compiled in to the byte-code file, where it will be used in tracebacks and other messages in cases where the source file does not exist at the time the byte-code file is executed.

If *force* is true, modules are re-compiled even if the timestamps are up to date.

If *rx* is given, its `search` method is called on the complete path to each file considered for compilation, and if it returns a true value, the file is skipped. This can be used to exclude files matching a regular expression, given as a `re.Pattern <re-objects>` object.

If *quiet* is `False` or `0` (the default), the filenames and other information are printed to standard out. Set to `1`, only errors are printed. Set to `2`, all output is suppressed.

If *legacy* is true, byte-code files are written to their legacy locations and names, which may overwrite byte-code files created by another version of Python. The default is to write files to their `3147` locations and names, which allows byte-code files from multiple versions of Python to coexist.

*optimize* specifies the optimization level for the compiler. It is passed to the built-in `compile` function. Accepts also a sequence of optimization levels which lead to multiple compilations of one `.py` file in one call.

The argument *workers* specifies how many workers are used to compile files in parallel. The default is to not use multiple workers. If the platform can't use multiple workers and *workers* argument is given, then sequential compilation will be used as a fallback. If *workers* is 0, the number of cores in the system is used. If *workers* is lower than `0`, a `ValueError` will be raised.

*invalidation_mode* should be a member of the `py_compile.PycInvalidationMode` enum and controls how the generated pycs are invalidated at runtime.

The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to the `-s`, `-p` and `-e` options described above. They may be specified as `str` or `os.PathLike`.

If *hardlink_dupes* is true and two `.pyc` files with different optimization level have the same content, use hard links to consolidate duplicate files.

<div class="versionchanged">

3.2 Added the *legacy* and *optimize* parameter.

</div>

<div class="versionchanged">

3.5 Added the *workers* parameter.

</div>

<div class="versionchanged">

3.5 *quiet* parameter was changed to a multilevel value.

</div>

<div class="versionchanged">

3.5 The *legacy* parameter only writes out `.pyc` files, not `.pyo` files no matter what the value of *optimize* is.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.7 The *invalidation_mode* parameter was added.

</div>

<div class="versionchanged">

3.7.2 The *invalidation_mode* parameter's default value is updated to `None`.

</div>

<div class="versionchanged">

3.8 Setting *workers* to 0 now chooses the optimal number of cores.

</div>

<div class="versionchanged">

3.9 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments. Default value of *maxlevels* was changed from `10` to `sys.getrecursionlimit()`

</div>

</div>

<div class="function">

compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None, \*, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False)

Compile the file with path *fullname*. Return a true value if the file compiled successfully, and a false value otherwise.

If *ddir* is given, it is prepended to the path to the file being compiled for use in compilation time tracebacks, and is also compiled in to the byte-code file, where it will be used in tracebacks and other messages in cases where the source file does not exist at the time the byte-code file is executed.

If *rx* is given, its `search` method is passed the full path name to the file being compiled, and if it returns a true value, the file is not compiled and `True` is returned. This can be used to exclude files matching a regular expression, given as a `re.Pattern <re-objects>` object.

If *quiet* is `False` or `0` (the default), the filenames and other information are printed to standard out. Set to `1`, only errors are printed. Set to `2`, all output is suppressed.

If *legacy* is true, byte-code files are written to their legacy locations and names, which may overwrite byte-code files created by another version of Python. The default is to write files to their `3147` locations and names, which allows byte-code files from multiple versions of Python to coexist.

*optimize* specifies the optimization level for the compiler. It is passed to the built-in `compile` function. Accepts also a sequence of optimization levels which lead to multiple compilations of one `.py` file in one call.

*invalidation_mode* should be a member of the `py_compile.PycInvalidationMode` enum and controls how the generated pycs are invalidated at runtime.

The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to the `-s`, `-p` and `-e` options described above. They may be specified as `str` or `os.PathLike`.

If *hardlink_dupes* is true and two `.pyc` files with different optimization level have the same content, use hard links to consolidate duplicate files.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.5 *quiet* parameter was changed to a multilevel value.

</div>

<div class="versionchanged">

3.5 The *legacy* parameter only writes out `.pyc` files, not `.pyo` files no matter what the value of *optimize* is.

</div>

<div class="versionchanged">

3.7 The *invalidation_mode* parameter was added.

</div>

<div class="versionchanged">

3.7.2 The *invalidation_mode* parameter's default value is updated to `None`.

</div>

<div class="versionchanged">

3.9 Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments.

</div>

</div>

<div class="function">

compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None)

Byte-compile all the `.py` files found along `sys.path`. Return a true value if all the files compiled successfully, and a false value otherwise.

If *skip_curdir* is true (the default), the current directory is not included in the search. All other parameters are passed to the `compile_dir` function. Note that unlike the other compile functions, `maxlevels` defaults to `0`.

<div class="versionchanged">

3.2 Added the *legacy* and *optimize* parameter.

</div>

<div class="versionchanged">

3.5 *quiet* parameter was changed to a multilevel value.

</div>

<div class="versionchanged">

3.5 The *legacy* parameter only writes out `.pyc` files, not `.pyo` files no matter what the value of *optimize* is.

</div>

<div class="versionchanged">

3.7 The *invalidation_mode* parameter was added.

</div>

<div class="versionchanged">

3.7.2 The *invalidation_mode* parameter's default value is updated to `None`.

</div>

</div>

To force a recompile of all the `.py` files in the `Lib/` subdirectory and all its subdirectories:

    import compileall

    compileall.compile_dir('Lib/', force=True)

    # Perform same compilation, excluding files in .svn directories.
    import re
    compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True)

    # pathlib.Path objects can also be used.
    import pathlib
    compileall.compile_dir(pathlib.Path('Lib/'), force=True)

<div class="seealso">

Module `py_compile`  
Byte-compile a single source file.

</div>

# The `!compression` package

<div class="module">

compression

</div>

<div class="versionadded">

3.14

</div>

The `!compression` package contains the canonical compression modules containing interfaces to several different compression algorithms. Some of these modules have historically been available as separate modules; those will continue to be available under their original names for compatibility reasons, and will not be removed without a deprecation cycle. The use of modules in `!compression` is encouraged where practical.

- `!compression.bz2` -- Re-exports `bz2`
- `!compression.gzip` -- Re-exports `gzip`
- `!compression.lzma` -- Re-exports `lzma`
- `!compression.zlib` -- Re-exports `zlib`
- `compression.zstd` -- Wrapper for the Zstandard compression library

# `!compression.zstd` --- Compression compatible with the Zstandard format

<div class="module" data-synopsis="Low-level interface to compression and decompression routines in
the zstd library.">

compression.zstd

</div>

<div class="versionadded">

3.14

</div>

**Source code:** `Lib/compression/zstd/__init__.py`

------------------------------------------------------------------------

This module provides classes and functions for compressing and decompressing data using the Zstandard (or *zstd*) compression algorithm. The [zstd manual](https://facebook.github.io/zstd/doc/api_manual_latest.html) describes Zstandard as "a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios." Also included is a file interface that supports reading and writing the contents of `.zst` files created by the `zstd` utility, as well as raw zstd compressed streams.

The `!compression.zstd` module contains:

- The `.open` function and `ZstdFile` class for reading and writing compressed files.
- The `ZstdCompressor` and `ZstdDecompressor` classes for incremental (de)compression.
- The `compress` and `decompress` functions for one-shot (de)compression.
- The `train_dict` and `finalize_dict` functions and the `ZstdDict` class to train and manage Zstandard dictionaries.
- The `CompressionParameter`, `DecompressionParameter`, and `Strategy` classes for setting advanced (de)compression parameters.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

## Exceptions

<div class="exception">

ZstdError

This exception is raised when an error occurs during compression or decompression, or while initializing the (de)compressor state.

</div>

## Reading and writing compressed files

<div class="function">

open(file, /, mode='rb', \*, level=None, options=None, zstd_dict=None, encoding=None, errors=None, newline=None)

Open a Zstandard-compressed file in binary or text mode, returning a `file object`.

The *file* argument can be either a file name (given as a `str`, `bytes` or `path-like <path-like object>` object), in which case the named file is opened, or it can be an existing file object to read from or write to.

The mode argument can be either `'rb'` for reading (default), `'wb'` for overwriting, `'ab'` for appending, or `'xb'` for exclusive creation. These can equivalently be given as `'r'`, `'w'`, `'a'`, and `'x'` respectively. You may also open in text mode with `'rt'`, `'wt'`, `'at'`, and `'xt'` respectively.

When reading, the *options* argument can be a dictionary providing advanced decompression parameters; see `DecompressionParameter` for detailed information about supported parameters. The *zstd_dict* argument is a `ZstdDict` instance to be used during decompression. When reading, if the *level* argument is not None, a `!TypeError` will be raised.

When writing, the *options* argument can be a dictionary providing advanced compression parameters; see `CompressionParameter` for detailed information about supported parameters. The *level* argument is the compression level to use when writing compressed data. Only one of *level* or *options* may be non-None. The *zstd_dict* argument is a `ZstdDict` instance to be used during compression.

In binary mode, this function is equivalent to the `ZstdFile` constructor: `ZstdFile(file, mode, ...)`. In this case, the *encoding*, *errors*, and *newline* parameters must not be provided.

In text mode, a `ZstdFile` object is created, and wrapped in an `io.TextIOWrapper` instance with the specified encoding, error handling behavior, and line endings.

</div>

<div class="ZstdFile(file, /, mode='rb', *, level=None, options=None, \ zstd_dict=None)">

Open a Zstandard-compressed file in binary mode.

A `ZstdFile` can wrap an already-open `file object`, or operate directly on a named file. The *file* argument specifies either the file object to wrap, or the name of the file to open (as a `str`, `bytes` or `path-like <path-like object>` object). If wrapping an existing file object, the wrapped file will not be closed when the `ZstdFile` is closed.

The *mode* argument can be either `'rb'` for reading (default), `'wb'` for overwriting, `'xb'` for exclusive creation, or `'ab'` for appending. These can equivalently be given as `'r'`, `'w'`, `'x'` and `'a'` respectively.

If *file* is a file object (rather than an actual file name), a mode of `'w'` does not truncate the file, and is instead equivalent to `'a'`.

When reading, the *options* argument can be a dictionary providing advanced decompression parameters; see `DecompressionParameter` for detailed information about supported parameters. The *zstd_dict* argument is a `ZstdDict` instance to be used during decompression. When reading, if the *level* argument is not None, a `!TypeError` will be raised.

When writing, the *options* argument can be a dictionary providing advanced compression parameters; see `CompressionParameter` for detailed information about supported parameters. The *level* argument is the compression level to use when writing compressed data. Only one of *level* or *options* may be passed. The *zstd_dict* argument is a `ZstdDict` instance to be used during compression.

`!ZstdFile` supports all the members specified by `io.BufferedIOBase`, except for `~io.BufferedIOBase.detach` and `~io.IOBase.truncate`. Iteration and the `with` statement are supported.

The following method and attributes are also provided:

<div class="method">

peek(size=-1)

Return buffered data without advancing the file position. At least one byte of data will be returned, unless EOF has been reached. The exact number of bytes returned is unspecified (the *size* argument is ignored).

> [!NOTE]
> While calling `peek` does not change the file position of the `ZstdFile`, it may change the position of the underlying file object (for example, if the `ZstdFile` was constructed by passing a file object for *file*).

</div>

<div class="attribute">

mode

`'rb'` for reading and `'wb'` for writing.

</div>

<div class="attribute">

name

The name of the Zstandard file. Equivalent to the `~io.FileIO.name` attribute of the underlying `file object`.

</div>

</div>

## Compressing and decompressing data in memory

<div class="function">

compress(data, level=None, options=None, zstd_dict=None)

Compress *data* (a `bytes-like object`), returning the compressed data as a `bytes` object.

The *level* argument is an integer controlling the level of compression. *level* is an alternative to setting `CompressionParameter.compression_level` in *options*. Use `~CompressionParameter.bounds` on `~CompressionParameter.compression_level` to get the values that can be passed for *level*. If advanced compression options are needed, the *level* argument must be omitted and in the *options* dictionary the `!CompressionParameter.compression_level` parameter should be set.

The *options* argument is a Python dictionary containing advanced compression parameters. The valid keys and values for compression parameters are documented as part of the `CompressionParameter` documentation.

The *zstd_dict* argument is an instance of `ZstdDict` containing trained data to improve compression efficiency. The function `train_dict` can be used to generate a Zstandard dictionary.

</div>

<div class="function">

decompress(data, zstd_dict=None, options=None)

Decompress *data* (a `bytes-like object`), returning the uncompressed data as a `bytes` object.

The *options* argument is a Python dictionary containing advanced decompression parameters. The valid keys and values for compression parameters are documented as part of the `DecompressionParameter` documentation.

The *zstd_dict* argument is an instance of `ZstdDict` containing trained data used during compression. This must be the same Zstandard dictionary used during compression.

If *data* is the concatenation of multiple distinct compressed frames, decompress all of these frames, and return the concatenation of the results.

</div>

<div class="ZstdCompressor(level=None, options=None, zstd_dict=None)">

Create a compressor object, which can be used to compress data incrementally.

For a more convenient way of compressing a single chunk of data, see the module-level function `compress`.

The *level* argument is an integer controlling the level of compression. *level* is an alternative to setting `CompressionParameter.compression_level` in *options*. Use `~CompressionParameter.bounds` on `~CompressionParameter.compression_level` to get the values that can be passed for *level*. If advanced compression options are needed, the *level* argument must be omitted and in the *options* dictionary the `!CompressionParameter.compression_level` parameter should be set.

The *options* argument is a Python dictionary containing advanced compression parameters. The valid keys and values for compression parameters are documented as part of the `CompressionParameter` documentation.

The *zstd_dict* argument is an optional instance of `ZstdDict` containing trained data to improve compression efficiency. The function `train_dict` can be used to generate a Zstandard dictionary.

<div class="method">

compress(data, mode=ZstdCompressor.CONTINUE)

Compress *data* (a `bytes-like object`), returning a `bytes` object with compressed data if possible, or otherwise an empty `!bytes` object. Some of *data* may be buffered internally, for use in later calls to `!compress` and `~.flush`. The returned data should be concatenated with the output of any previous calls to `~.compress`.

The *mode* argument is a `ZstdCompressor` attribute, either `~.CONTINUE`, `~.FLUSH_BLOCK`, or `~.FLUSH_FRAME`.

When all data has been provided to the compressor, call the `~.flush` method to finish the compression process. If `~.compress` is called with *mode* set to `~.FLUSH_FRAME`, `~.flush` should not be called, as it would write out a new empty frame.

</div>

<div class="method">

flush(mode=ZstdCompressor.FLUSH_FRAME)

Finish the compression process, returning a `bytes` object containing any data stored in the compressor's internal buffers.

The *mode* argument is a `ZstdCompressor` attribute, either `~.FLUSH_BLOCK`, or `~.FLUSH_FRAME`.

</div>

<div class="method">

set_pledged_input_size(size)

Specify the amount of uncompressed data *size* that will be provided for the next frame. *size* will be written into the frame header of the next frame unless `CompressionParameter.content_size_flag` is `False` or `0`. A size of `0` means that the frame is empty. If *size* is `None`, the frame header will omit the frame size. Frames that include the uncompressed data size require less memory to decompress, especially at higher compression levels.

If `last_mode` is not `FLUSH_FRAME`, a `ValueError` is raised as the compressor is not at the start of a frame. If the pledged size does not match the actual size of data provided to `.compress`, future calls to `!compress` or `flush` may raise `ZstdError` and the last chunk of data may be lost.

After `flush` or `.compress` are called with mode `FLUSH_FRAME`, the next frame will not include the frame size into the header unless `!set_pledged_input_size` is called again.

</div>

<div class="attribute">

CONTINUE

Collect more data for compression, which may or may not generate output immediately. This mode optimizes the compression ratio by maximizing the amount of data per block and frame.

</div>

<div class="attribute">

FLUSH_BLOCK

Complete and write a block to the data stream. The data returned so far can be immediately decompressed. Past data can still be referenced in future blocks generated by calls to `~.compress`, improving compression.

</div>

<div class="attribute">

FLUSH_FRAME

Complete and write out a frame. Future data provided to `~.compress` will be written into a new frame and *cannot* reference past data.

</div>

<div class="attribute">

last_mode

The last mode passed to either `~.compress` or `~.flush`. The value can be one of `~.CONTINUE`, `~.FLUSH_BLOCK`, or `~.FLUSH_FRAME`. The initial value is `~.FLUSH_FRAME`, signifying that the compressor is at the start of a new frame.

</div>

</div>

<div class="ZstdDecompressor(zstd_dict=None, options=None)">

Create a decompressor object, which can be used to decompress data incrementally.

For a more convenient way of decompressing an entire compressed stream at once, see the module-level function `decompress`.

The *options* argument is a Python dictionary containing advanced decompression parameters. The valid keys and values for compression parameters are documented as part of the `DecompressionParameter` documentation.

The *zstd_dict* argument is an instance of `ZstdDict` containing trained data used during compression. This must be the same Zstandard dictionary used during compression.

> [!NOTE]
> This class does not transparently handle inputs containing multiple compressed frames, unlike the `decompress` function and `ZstdFile` class. To decompress a multi-frame input, you should use `decompress`, `ZstdFile` if working with a `file object`, or multiple `!ZstdDecompressor` instances.

<div class="method">

decompress(data, max_length=-1)

Decompress *data* (a `bytes-like object`), returning uncompressed data as bytes. Some of *data* may be buffered internally, for use in later calls to `!decompress`. The returned data should be concatenated with the output of any previous calls to `!decompress`.

If *max_length* is non-negative, the method returns at most *max_length* bytes of decompressed data. If this limit is reached and further output can be produced (or EOF is reached), the `~.needs_input` attribute will be set to `False`. In this case, the next call to `~.decompress` may provide *data* as `b''` to obtain more of the output. The full content can thus be read like:

    process_output(d.decompress(data, max_length))
    while not d.eof and not d.needs_input:
        process_output(d.decompress(b"", max_length))

If all of the input data was decompressed and returned (either because this was less than *max_length* bytes, or because *max_length* was negative), the `~.needs_input` attribute will be set to `True`.

Attempting to decompress data after the end of a frame will raise a `EOFError`. Any data found after the end of the frame is ignored and saved in the `~.unused_data` attribute.

</div>

<div class="attribute">

eof

`True` if the end-of-stream marker has been reached.

</div>

<div class="attribute">

unused_data

Data found after the end of the compressed stream.

Before the end of the stream is reached, this will be `b''`.

</div>

<div class="attribute">

needs_input

`False` if the `.decompress` method can provide more decompressed data before requiring new compressed input.

</div>

</div>

## Zstandard dictionaries

<div class="function">

train_dict(samples, dict_size)

Train a Zstandard dictionary, returning a `ZstdDict` instance. Zstandard dictionaries enable more efficient compression of smaller sizes of data, which is traditionally difficult to compress due to less repetition. If you are compressing multiple similar groups of data (such as similar files), Zstandard dictionaries can improve compression ratios and speed significantly.

The *samples* argument (an iterable of `bytes` objects), is the population of samples used to train the Zstandard dictionary.

The *dict_size* argument, an integer, is the maximum size (in bytes) the Zstandard dictionary should be. The Zstandard documentation suggests an absolute maximum of no more than 100 KB, but the maximum can often be smaller depending on the data. Larger dictionaries generally slow down compression, but improve compression ratios. Smaller dictionaries lead to faster compression, but reduce the compression ratio.

</div>

<div class="function">

finalize_dict(zstd_dict, /, samples, dict_size, level)

An advanced function for converting a "raw content" Zstandard dictionary into a regular Zstandard dictionary. "Raw content" dictionaries are a sequence of bytes that do not need to follow the structure of a normal Zstandard dictionary.

The *zstd_dict* argument is a `ZstdDict` instance with the `~ZstdDict.dict_content` containing the raw dictionary contents.

The *samples* argument (an iterable of `bytes` objects), contains sample data for generating the Zstandard dictionary.

The *dict_size* argument, an integer, is the maximum size (in bytes) the Zstandard dictionary should be. See `train_dict` for suggestions on the maximum dictionary size.

The *level* argument (an integer) is the compression level expected to be passed to the compressors using this dictionary. The dictionary information varies for each compression level, so tuning for the proper compression level can make compression more efficient.

</div>

<div class="ZstdDict(dict_content, /, *, is_raw=False)">

A wrapper around Zstandard dictionaries. Dictionaries can be used to improve the compression of many small chunks of data. Use `train_dict` if you need to train a new dictionary from sample data.

The *dict_content* argument (a `bytes-like object`), is the already trained dictionary information.

The *is_raw* argument, a boolean, is an advanced parameter controlling the meaning of *dict_content*. `True` means *dict_content* is a "raw content" dictionary, without any format restrictions. `False` means *dict_content* is an ordinary Zstandard dictionary, created from Zstandard functions, for example, `train_dict` or the external `zstd` CLI.

When passing a `!ZstdDict` to a function, the `!as_digested_dict` and `!as_undigested_dict` attributes can control how the dictionary is loaded by passing them as the `zstd_dict` argument, for example, `compress(data, zstd_dict=zd.as_digested_dict)`. Digesting a dictionary is a costly operation that occurs when loading a Zstandard dictionary. When making multiple calls to compression or decompression, passing a digested dictionary will reduce the overhead of loading the dictionary.

> |  | Digested dictionary | Undigested dictionary |
> |----|----|----|
> | Advanced parameters of the compressor which may be overridden by the dictionary's parameters | `window_log`, `hash_log`, `chain_log`, `search_log`, `min_match`, `target_length`, `strategy`, `enable_long_distance_matching`, `ldm_hash_log`, `ldm_min_match`, `ldm_bucket_size_log`, `ldm_hash_rate_log`, and some non-public parameters. | None |
> | `!ZstdDict` internally caches the dictionary | Yes. It's faster when loading a digested dictionary again with the same compression level. | No. If you wish to load an undigested dictionary multiple times, consider reusing a compressor object. |
>
> Difference for compression

If passing a `!ZstdDict` without any attribute, an undigested dictionary is passed by default when compressing and a digested dictionary is generated if necessary and passed by default when decompressing.

> <div class="attribute">
>
> dict_content
>
> The content of the Zstandard dictionary, a `bytes` object. It's the same as the *dict_content* argument in the `__init__` method. It can be used with other programs, such as the `zstd` CLI program.
>
> </div>
>
> <div class="attribute">
>
> dict_id
>
> Identifier of the Zstandard dictionary, a non-negative int value.
>
> Non-zero means the dictionary is ordinary, created by Zstandard functions and following the Zstandard format.
>
> `0` means a "raw content" dictionary, free of any format restriction, used for advanced users.
>
> > [!NOTE]
> > The meaning of `0` for `!ZstdDict.dict_id` is different from the `dictionary_id` attribute to the `get_frame_info` function.
>
> </div>
>
> <div class="attribute">
>
> as_digested_dict
>
> Load as a digested dictionary.
>
> </div>
>
> <div class="attribute">
>
> as_undigested_dict
>
> Load as an undigested dictionary.
>
> </div>

</div>

## Advanced parameter control

<div class="CompressionParameter()">

An `~enum.IntEnum` containing the advanced compression parameter keys that can be used when compressing data.

The `~.bounds` method can be used on any attribute to get the valid values for that parameter.

Parameters are optional; any omitted parameter will have its value selected automatically.

Example getting the lower and upper bound of `~.compression_level`:

    lower, upper = CompressionParameter.compression_level.bounds()

Example setting the `~.window_log` to the maximum size:

    _lower, upper = CompressionParameter.window_log.bounds()
    options = {CompressionParameter.window_log: upper}
    compress(b'venezuelan beaver cheese', options=options)

<div class="method">

bounds()

Return the tuple of int bounds, `(lower, upper)`, of a compression parameter. This method should be called on the attribute you wish to retrieve the bounds of. For example, to get the valid values for `~.compression_level`, one may check the result of `CompressionParameter.compression_level.bounds()`.

Both the lower and upper bounds are inclusive.

</div>

<div class="attribute">

compression_level

A high-level means of setting other compression parameters that affect the speed and ratio of compressing data.

Regular compression levels are greater than `0`. Values greater than `20` are considered "ultra" compression and require more memory than other levels. Negative values can be used to trade off faster compression for worse compression ratios.

Setting the level to zero uses `COMPRESSION_LEVEL_DEFAULT`.

</div>

<div class="attribute">

window_log

Maximum allowed back-reference distance the compressor can use when compressing data, expressed as power of two, `1 << window_log` bytes. This parameter greatly influences the memory usage of compression. Higher values require more memory but gain better compression values.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

hash_log

Size of the initial probe table, as a power of two. The resulting memory usage is `1 << (hash_log+2)` bytes. Larger tables improve compression ratio of strategies \<= `~Strategy.dfast`, and improve compression speed of strategies \> `~Strategy.dfast`.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

chain_log

Size of the multi-probe search table, as a power of two. The resulting memory usage is `1 << (chain_log+2)` bytes. Larger tables result in better and slower compression. This parameter has no effect for the `~Strategy.fast` strategy. It's still useful when using `~Strategy.dfast` strategy, in which case it defines a secondary probe table.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

search_log

Number of search attempts, as a power of two. More attempts result in better and slower compression. This parameter is useless for `~Strategy.fast` and `~Strategy.dfast` strategies.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

min_match

Minimum size of searched matches. Larger values increase compression and decompression speed, but decrease ratio. Note that Zstandard can still find matches of smaller size, it just tweaks its search algorithm to look for this size and larger. For all strategies \< `~Strategy.btopt`, the effective minimum is `4`; for all strategies \> `~Strategy.fast`, the effective maximum is `6`.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

target_length

The impact of this field depends on the selected `Strategy`.

For strategies `~Strategy.btopt`, `~Strategy.btultra` and `~Strategy.btultra2`, the value is the length of a match considered "good enough" to stop searching. Larger values make compression ratios better, but compresses slower.

For strategy `~Strategy.fast`, it is the distance between match sampling. Larger values make compression faster, but with a worse compression ratio.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

strategy

The higher the value of selected strategy, the more complex the compression technique used by zstd, resulting in higher compression ratios but slower compression.

<div class="seealso">

`Strategy`

</div>

</div>

<div class="attribute">

enable_long_distance_matching

Long distance matching can be used to improve compression for large inputs by finding large matches at greater distances. It increases memory usage and window size.

`True` or `1` enable long distance matching while `False` or `0` disable it.

Enabling this parameter increases default `~CompressionParameter.window_log` to 128 MiB except when expressly set to a different value. This setting is enabled by default if `!window_log` \>= 128 MiB and the compression strategy \>= `~Strategy.btopt` (compression level 16+).

</div>

<div class="attribute">

ldm_hash_log

Size of the table for long distance matching, as a power of two. Larger values increase memory usage and compression ratio, but decrease compression speed.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

ldm_min_match

Minimum match size for long distance matcher. Larger or too small values can often decrease the compression ratio.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

ldm_bucket_size_log

Log size of each bucket in the long distance matcher hash table for collision resolution. Larger values improve collision resolution but decrease compression speed.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

ldm_hash_rate_log

Frequency of inserting/looking up entries into the long distance matcher hash table. Larger values improve compression speed. Deviating far from the default value will likely result in a compression ratio decrease.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

content_size_flag

Write the size of the data to be compressed into the Zstandard frame header when known prior to compressing.

This flag only takes effect under the following scenarios:

- Calling `compress` for one-shot compression
- Providing all of the data to be compressed in the frame in a single `ZstdCompressor.compress` call, with the `ZstdCompressor.FLUSH_FRAME` mode.
- Calling `ZstdCompressor.set_pledged_input_size` with the exact amount of data that will be provided to the compressor prior to any calls to `ZstdCompressor.compress` for the current frame. `!ZstdCompressor.set_pledged_input_size` must be called for each new frame.

All other compression calls may not write the size information into the frame header.

`True` or `1` enable the content size flag while `False` or `0` disable it.

</div>

<div class="attribute">

checksum_flag

A four-byte checksum using XXHash64 of the uncompressed content is written at the end of each frame. Zstandard's decompression code verifies the checksum. If there is a mismatch a `ZstdError` exception is raised.

`True` or `1` enable checksum generation while `False` or `0` disable it.

</div>

<div class="attribute">

dict_id_flag

When compressing with a `ZstdDict`, the dictionary's ID is written into the frame header.

`True` or `1` enable storing the dictionary ID while `False` or `0` disable it.

</div>

<div class="attribute">

nb_workers

Select how many threads will be spawned to compress in parallel. When `!nb_workers` \> 0, enables multi-threaded compression, a value of `1` means "one-thread multi-threaded mode". More workers improve speed, but also increase memory usage and slightly reduce compression ratio.

A value of zero disables multi-threading.

</div>

<div class="attribute">

job_size

Size of a compression job, in bytes. This value is enforced only when `~CompressionParameter.nb_workers` \>= 1. Each compression job is completed in parallel, so this value can indirectly impact the number of active threads.

A value of zero causes the value to be selected automatically.

</div>

<div class="attribute">

overlap_log

Sets how much data is reloaded from previous jobs (threads) for new jobs to be used by the look behind window during compression. This value is only used when `~CompressionParameter.nb_workers` \>= 1. Acceptable values vary from 0 to 9.

> - 0 means dynamically set the overlap amount
> - 1 means no overlap
> - 9 means use a full window size from the previous job

Each increment halves/doubles the overlap size. "8" means an overlap of `window_size/2`, "7" means an overlap of `window_size/4`, etc.

</div>

</div>

<div class="DecompressionParameter()">

An `~enum.IntEnum` containing the advanced decompression parameter keys that can be used when decompressing data. Parameters are optional; any omitted parameter will have its value selected automatically.

The `~.bounds` method can be used on any attribute to get the valid values for that parameter.

Example setting the `~.window_log_max` to the maximum size:

    data = compress(b'Some very long buffer of bytes...')

    _lower, upper = DecompressionParameter.window_log_max.bounds()

    options = {DecompressionParameter.window_log_max: upper}
    decompress(data, options=options)

<div class="method">

bounds()

Return the tuple of int bounds, `(lower, upper)`, of a decompression parameter. This method should be called on the attribute you wish to retrieve the bounds of.

Both the lower and upper bounds are inclusive.

</div>

<div class="attribute">

window_log_max

The base-two logarithm of the maximum size of the window used during decompression. This can be useful to limit the amount of memory used when decompressing data. A larger maximum window size leads to faster decompression.

A value of zero causes the value to be selected automatically.

</div>

</div>

<div class="Strategy()">

An `~enum.IntEnum` containing strategies for compression. Higher-numbered strategies correspond to more complex and slower compression.

> [!NOTE]
> The values of attributes of `!Strategy` are not necessarily stable across zstd versions. Only the ordering of the attributes may be relied upon. The attributes are listed below in order.

The following strategies are available:

<div class="attribute">

fast

</div>

<div class="attribute">

dfast

</div>

<div class="attribute">

greedy

</div>

<div class="attribute">

lazy

</div>

<div class="attribute">

lazy2

</div>

<div class="attribute">

btlazy2

</div>

<div class="attribute">

btopt

</div>

<div class="attribute">

btultra

</div>

<div class="attribute">

btultra2

</div>

</div>

## Miscellaneous

<div class="function">

get_frame_info(frame_buffer)

Retrieve a `FrameInfo` object containing metadata about a Zstandard frame. Frames contain metadata related to the compressed data they hold.

</div>

<div class="FrameInfo">

Metadata related to a Zstandard frame.

<div class="attribute">

decompressed_size

The size of the decompressed contents of the frame.

</div>

<div class="attribute">

dictionary_id

An integer representing the Zstandard dictionary ID needed for decompressing the frame. `0` means the dictionary ID was not recorded in the frame header. This may mean that a Zstandard dictionary is not needed, or that the ID of a required dictionary was not recorded.

</div>

</div>

<div class="attribute">

COMPRESSION_LEVEL_DEFAULT

The default compression level for Zstandard: `3`.

</div>

Information about the version of the zstd library in use is available through the following constants:

<div class="data">

ZSTD_VERSION

The version string of the zstd library that was used for building the module. This may be different from the zstd library actually used at runtime, which is available as `zstd_version`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

zstd_version

The version string of the zstd library actually loaded by the interpreter.

</div>

<div class="data">

ZSTD_VERSION_INFO

A named tuple containing the three components of the zstd library version that was used for building the module: *major*, *minor*, and *patch*. All values are integers. The components can also be accessed by name, so `zstd.ZSTD_VERSION_INFO[0]` is equivalent to `zstd.ZSTD_VERSION_INFO.major` and so on. This may be different from the zstd library actually used at runtime, which is available as `zstd_version_info`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

zstd_version_info

A named tuple containing the version of the zstd library actually loaded by the interpreter, with the same fields as `ZSTD_VERSION_INFO`.

<div class="versionchanged">

3.16 It is now a named tuple.

</div>

</div>

## Examples

Reading in a compressed file:

``` python
from compression import zstd

with zstd.open("file.zst") as f:
    file_content = f.read()
```

Creating a compressed file:

``` python
from compression import zstd

data = b"Insert Data Here"
with zstd.open("file.zst", "w") as f:
    f.write(data)
```

Compressing data in memory:

``` python
from compression import zstd

data_in = b"Insert Data Here"
data_out = zstd.compress(data_in)
```

Incremental compression:

``` python
from compression import zstd

comp = zstd.ZstdCompressor()
out1 = comp.compress(b"Some data\n")
out2 = comp.compress(b"Another piece of data\n")
out3 = comp.compress(b"Even more data\n")
out4 = comp.flush()
# Concatenate all the partial results:
result = b"".join([out1, out2, out3, out4])
```

Writing compressed data to an already-open file:

``` python
from compression import zstd

with open("myfile", "wb") as f:
    f.write(b"This data will not be compressed\n")
    with zstd.open(f, "w") as zstf:
        zstf.write(b"This *will* be compressed\n")
    f.write(b"Not compressed\n")
```

Creating a compressed file using compression parameters:

``` python
from compression import zstd

options = {
   zstd.CompressionParameter.checksum_flag: 1
}
with zstd.open("file.zst", "w", options=options) as f:
    f.write(b"Mind if I squeeze in?")
```

# Concurrent Execution

The modules described in this chapter provide support for concurrent execution of code. The appropriate choice of tool will depend on the task to be executed (CPU bound vs IO bound) and preferred style of development (event driven cooperative multitasking vs preemptive multitasking). Here's an overview:

<div class="toctree">

threading.rst multiprocessing.rst multiprocessing.shared_memory.rst concurrent.rst concurrent.futures.rst concurrent.interpreters.rst subprocess.rst sched.rst queue.rst contextvars.rst

</div>

The following are support modules for some of the above services:

<div class="toctree">

<span id="thread.rst">thread.rst</span>

</div>

# `!concurrent.futures` --- Launching parallel tasks

<div class="module" data-synopsis="Execute computations concurrently using threads or processes.">

concurrent.futures

</div>

<div class="versionadded">

3.2

</div>

**Source code:** `Lib/concurrent/futures/thread.py`, `Lib/concurrent/futures/process.py`, and `Lib/concurrent/futures/interpreter.py`

------------------------------------------------------------------------

The `!concurrent.futures` module provides a high-level interface for asynchronously executing callables.

The asynchronous execution can be performed with threads, using `ThreadPoolExecutor` or `InterpreterPoolExecutor`, or separate processes, using `ProcessPoolExecutor`. Each implements the same interface, which is defined by the abstract `Executor` class.

`concurrent.futures.Future` must not be confused with `asyncio.Future`, which is designed for use with `asyncio` tasks and coroutines. See the `asyncio's Future <asyncio-future>` documentation for a detailed comparison of the two.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

## Executor Objects

<div class="Executor">

An abstract class that provides methods to execute calls asynchronously. It should not be used directly, but through its concrete subclasses.

<div class="method">

submit(fn, /, *args,*\*kwargs)

Schedules the callable, *fn*, to be executed as `fn(*args, **kwargs)` and returns a `Future` object representing the execution of the callable. :

    with ThreadPoolExecutor(max_workers=1) as executor:
        future = executor.submit(pow, 323, 1235)
        print(future.result())

</div>

<div class="method">

map(fn, \*iterables, timeout=None, chunksize=1, buffersize=None)

Similar to `map(fn, *iterables) <map>` except:

- The *iterables* are collected immediately rather than lazily, unless a *buffersize* is specified to limit the number of submitted tasks whose results have not yet been yielded. If the buffer is full, iteration over the *iterables* pauses until a result is yielded from the buffer.
- *fn* is executed asynchronously and several calls to *fn* may be made concurrently.

The returned iterator raises a `TimeoutError` if `~iterator.__next__` is called and the result isn't available after *timeout* seconds from the original call to `Executor.map`. *timeout* can be an int or a float. It cancels all future calls of *fn* and closes the iterator. If *timeout* is not specified or `None`, there is no limit to the wait time.

If a *fn* call raises an exception, then that exception will be raised when its value is retrieved from the iterator. It does not cancel future calls of *fn*.

The returned iterator has method `!close` which cancels all future calls of *fn* and discards the results of already finished calls if they are available.

When using `ProcessPoolExecutor`, this method chops *iterables* into a number of chunks which it submits to the pool as separate tasks. The (approximate) size of these chunks can be specified by setting *chunksize* to a positive integer. For very long iterables, using a large value for *chunksize* can significantly improve performance compared to the default size of 1. With `ThreadPoolExecutor` and `InterpreterPoolExecutor`, *chunksize* has no effect.

<div class="versionchanged">

3.5 Added the *chunksize* parameter.

</div>

<div class="versionchanged">

3.14 Added the *buffersize* parameter.

</div>

<div class="versionchanged">

3.16 The returned iterator is no longer automatically closed if a *fn* call raises an exception.

</div>

</div>

<div class="method">

shutdown(wait=True, \*, cancel_futures=False)

Signal the executor that it should free any resources that it is using when the currently pending futures are done executing. Calls to `Executor.submit` and `Executor.map` made after shutdown will raise `RuntimeError`.

If *wait* is `True` then this method will not return until all the pending futures are done executing and the resources associated with the executor have been freed. If *wait* is `False` then this method will return immediately and the resources associated with the executor will be freed when all pending futures are done executing. Regardless of the value of *wait*, the entire Python program will not exit until all pending futures are done executing.

If *cancel_futures* is `True`, this method will cancel all pending futures that the executor has not started running. Any futures that are completed or running won't be cancelled, regardless of the value of *cancel_futures*.

If both *cancel_futures* and *wait* are `True`, all futures that the executor has started running will be completed prior to this method returning. The remaining futures are cancelled.

You can avoid having to call this method explicitly if you use the executor as a `context manager` via the `with` statement, which will shutdown the `Executor` (waiting as if `Executor.shutdown` were called with *wait* set to `True`):

    import shutil
    with ThreadPoolExecutor(max_workers=4) as e:
        e.submit(shutil.copy, 'src1.txt', 'dest1.txt')
        e.submit(shutil.copy, 'src2.txt', 'dest2.txt')
        e.submit(shutil.copy, 'src3.txt', 'dest3.txt')
        e.submit(shutil.copy, 'src4.txt', 'dest4.txt')

<div class="versionchanged">

3.9 Added the *cancel_futures* parameter.

</div>

</div>

</div>

## ThreadPoolExecutor

`ThreadPoolExecutor` is an `Executor` subclass that uses a pool of threads to execute calls asynchronously.

Deadlocks can occur when the callable associated with a `Future` waits on the results of another `Future`. For example:

    import time
    def wait_on_b():
        time.sleep(5)
        print(b.result())  # b will never complete because it is waiting on a.
        return 5

    def wait_on_a():
        time.sleep(5)
        print(a.result())  # a will never complete because it is waiting on b.
        return 6


    executor = ThreadPoolExecutor(max_workers=2)
    a = executor.submit(wait_on_b)
    b = executor.submit(wait_on_a)

And:

    def wait_on_future():
        f = executor.submit(pow, 5, 2)
        # This will never complete because there is only one worker thread and
        # it is executing this function.
        print(f.result())

    executor = ThreadPoolExecutor(max_workers=1)
    future = executor.submit(wait_on_future)
    # Note: calling future.result() would also cause a deadlock because
    # the single worker thread is already waiting for wait_on_future().

<div class="ThreadPoolExecutor(max_workers=None, thread_name_prefix='', \ initializer=None, initargs=(), **ctxkwargs)">

An `Executor` subclass that uses a pool of at most *max_workers* threads to execute calls asynchronously.

All threads enqueued to `ThreadPoolExecutor` will be joined before the interpreter can exit. Note that the exit handler which does this is executed *before* any exit handlers added using `atexit`. This means exceptions in the main thread must be caught and handled in order to signal threads to exit gracefully. For this reason, it is recommended that `ThreadPoolExecutor` not be used for long-running tasks.

*initializer* is an optional callable that is called at the start of each worker thread; *initargs* is a tuple of arguments passed to the initializer. Should *initializer* raise an exception, all currently pending jobs will raise a `~concurrent.futures.thread.BrokenThreadPool`, as well as any attempt to submit more jobs to the pool.

*ctxkwargs* is a mapping of additional keyword arguments passed to `prepare_context`, allowing customization of the worker execution context.

<div class="classmethod">

prepare_context(initializer, initargs)

Setting up the necessary context for creating worker instances in a pool-based executor (for example, in a concurrent environment like threads or interpreters).

<div class="versionadded">

3.14

</div>

</div>

<div class="versionchanged">

3.5 If *max_workers* is `None` or not given, it will default to the number of processors on the machine, multiplied by `5`, assuming that `ThreadPoolExecutor` is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for `ProcessPoolExecutor`.

</div>

<div class="versionchanged">

3.6 Added the *thread_name_prefix* parameter to allow users to control the `threading.Thread` names for worker threads created by the pool for easier debugging.

</div>

<div class="versionchanged">

3.7 Added the *initializer* and *initargs* arguments.

</div>

<div class="versionchanged">

3.8 Default value of *max_workers* is changed to `min(32, os.cpu_count() + 4)`. This default value preserves at least 5 workers for I/O bound tasks. It utilizes at most 32 CPU cores for CPU bound tasks which release the GIL. And it avoids using very large resources implicitly on many-core machines.

ThreadPoolExecutor now reuses idle worker threads before starting *max_workers* worker threads too.

</div>

<div class="versionchanged">

3.13 Default value of *max_workers* is changed to `min(32, (os.process_cpu_count() or 1) + 4)`.

</div>

<div class="versionchanged">

3.14 Added *ctxkwargs* to pass additional arguments to `prepare_context` class method.

</div>

</div>

### ThreadPoolExecutor Example

    import concurrent.futures
    import urllib.request

    URLS = ['http://www.foxnews.com/',
            'http://www.cnn.com/',
            'http://europe.wsj.com/',
            'http://www.bbc.co.uk/',
            'http://nonexistent-subdomain.python.org/']

    # Retrieve a single page and report the URL and contents
    def load_url(url, timeout):
        with urllib.request.urlopen(url, timeout=timeout) as conn:
            return conn.read()

    # We can use a with statement to ensure threads are cleaned up promptly
    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        # Start the load operations and mark each future with its URL
        future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
        for future in concurrent.futures.as_completed(future_to_url):
            url = future_to_url[future]
            try:
                data = future.result()
            except Exception as exc:
                print('%r generated an exception: %s' % (url, exc))
            else:
                print('%r page is %d bytes' % (url, len(data)))

## InterpreterPoolExecutor

<div class="versionadded">

3.14

</div>

The `InterpreterPoolExecutor` class uses a pool of interpreters to execute calls asynchronously. It is a `ThreadPoolExecutor` subclass, which means each worker is running in its own thread. The difference here is that each worker has its own interpreter, and runs each task using that interpreter.

The biggest benefit to using interpreters instead of only threads is true multi-core parallelism. Each interpreter has its own `Global Interpreter Lock <global interpreter lock>`, so code running in one interpreter can run on one CPU core, while code in another interpreter runs unblocked on a different core.

The tradeoff is that writing concurrent code for use with multiple interpreters can take extra effort. However, this is because it forces you to be deliberate about how and when interpreters interact, and to be explicit about what data is shared between interpreters. This results in several benefits that help balance the extra effort, including true multi-core parallelism, For example, code written this way can make it easier to reason about concurrency. Another major benefit is that you don't have to deal with several of the big pain points of using threads, like race conditions.

Each worker's interpreter is isolated from all the other interpreters. "Isolated" means each interpreter has its own runtime state and operates completely independently. For example, if you redirect `sys.stdout` in one interpreter, it will not be automatically redirected to any other interpreter. If you import a module in one interpreter, it is not automatically imported in any other. You would need to import the module separately in interpreter where you need it. In fact, each module imported in an interpreter is a completely separate object from the same module in a different interpreter, including `sys`, `builtins`, and even `__main__`.

Isolation means a mutable object, or other data, cannot be used by more than one interpreter at the same time. That effectively means interpreters cannot actually share such objects or data. Instead, each interpreter must have its own copy, and you will have to synchronize any changes between the copies manually. Immutable objects and data, like the builtin singletons, strings, and tuples of immutable objects, don't have these limitations.

Communicating and synchronizing between interpreters is most effectively done using dedicated tools, like those proposed in `734`. One less efficient alternative is to serialize with `pickle` and then send the bytes over a shared `socket <socket>` or `pipe <os.pipe>`.

<div class="InterpreterPoolExecutor(max_workers=None, thread_name_prefix='', \ initializer=None, initargs=())">

A `ThreadPoolExecutor` subclass that executes calls asynchronously using a pool of at most *max_workers* threads. Each thread runs tasks in its own interpreter. The worker interpreters are isolated from each other, which means each has its own runtime state and that they can't share any mutable objects or other data. Each interpreter has its own `Global Interpreter Lock <global interpreter lock>`, which means code run with this executor has true multi-core parallelism.

The optional *initializer* and *initargs* arguments have the same meaning as for `!ThreadPoolExecutor`: the initializer is run when each worker is created, though in this case it is run in the worker's interpreter. The executor serializes the *initializer* and *initargs* using `pickle` when sending them to the worker's interpreter.

> [!NOTE]
> The executor may replace uncaught exceptions from *initializer* with `~concurrent.interpreters.ExecutionFailed`.

Other caveats from parent `ThreadPoolExecutor` apply here.

</div>

`~Executor.submit` and `~Executor.map` work like normal, except the worker serializes the callable and arguments using `pickle` when sending them to its interpreter. The worker likewise serializes the return value when sending it back.

When a worker's current task raises an uncaught exception, the worker always tries to preserve the exception as-is. If that is successful then it also sets the `__cause__` to a corresponding `~concurrent.interpreters.ExecutionFailed` instance, which contains a summary of the original exception. In the uncommon case that the worker is not able to preserve the original as-is then it directly preserves the corresponding `~concurrent.interpreters.ExecutionFailed` instance instead.

## ProcessPoolExecutor

The `ProcessPoolExecutor` class is an `Executor` subclass that uses a pool of processes to execute calls asynchronously. `ProcessPoolExecutor` uses the `multiprocessing` module, which allows it to side-step the `Global Interpreter Lock
<global interpreter lock>` but also means that only picklable objects can be executed and returned.

The `__main__` module must be importable by worker subprocesses. This means that `ProcessPoolExecutor` will not work in the interactive interpreter.

Calling `Executor` or `Future` methods from a callable submitted to a `ProcessPoolExecutor` will result in deadlock.

Note that the restrictions on functions and arguments needing to picklable as per `multiprocessing.Process` apply when using `~Executor.submit` and `~Executor.map` on a `ProcessPoolExecutor`. A function defined in a REPL or a lambda should not be expected to work.

<div class="ProcessPoolExecutor(max_workers=None, mp_context=None, \ initializer=None, initargs=(), \ max_tasks_per_child=None)">

An `Executor` subclass that executes calls asynchronously using a pool of at most *max_workers* processes. If *max_workers* is `None` or not given, it will default to `os.process_cpu_count`. If *max_workers* is less than or equal to `0`, then a `ValueError` will be raised. On Windows, *max_workers* must be less than or equal to `61`. If it is not then `ValueError` will be raised. If *max_workers* is `None`, then the default chosen will be at most `61`, even if more processors are available. *mp_context* can be a `multiprocessing` context or `None`. It will be used to launch the workers. If *mp_context* is `None` or not given, the default `multiprocessing` context is used. See `multiprocessing-start-methods`.

*initializer* is an optional callable that is called at the start of each worker process; *initargs* is a tuple of arguments passed to the initializer. Should *initializer* raise an exception, all currently pending jobs will raise a `~concurrent.futures.process.BrokenProcessPool`, as well as any attempt to submit more jobs to the pool.

*max_tasks_per_child* is an optional argument that specifies the maximum number of tasks a single process can execute before it will exit and be replaced with a fresh worker process. By default *max_tasks_per_child* is `None` which means worker processes will live as long as the pool. When a max is specified, the "spawn" multiprocessing start method will be used by default in absence of a *mp_context* parameter. This feature is incompatible with the "fork" start method.

<div class="versionchanged">

3.3 When one of the worker processes terminates abruptly, a `~concurrent.futures.process.BrokenProcessPool` error is now raised. Previously, behaviour was undefined but operations on the executor or its futures would often freeze or deadlock.

</div>

<div class="versionchanged">

3.7 The *mp_context* argument was added to allow users to control the start_method for worker processes created by the pool.

Added the *initializer* and *initargs* arguments.

</div>

<div class="versionchanged">

3.11 The *max_tasks_per_child* argument was added to allow users to control the lifetime of workers in the pool.

</div>

<div class="versionchanged">

3.12 On POSIX systems, if your application has multiple threads and the `multiprocessing` context uses the `"fork"` start method: The `os.fork` function called internally to spawn workers may raise a `DeprecationWarning`. Pass a *mp_context* configured to use a different start method. See the `os.fork` documentation for further explanation.

</div>

<div class="versionchanged">

3.13 *max_workers* uses `os.process_cpu_count` by default, instead of `os.cpu_count`.

</div>

<div class="versionchanged">

3.14 The default process start method (see `multiprocessing-start-methods`) changed away from *fork*. If you require the *fork* start method for `ProcessPoolExecutor` you must explicitly pass `mp_context=multiprocessing.get_context("fork")`.

</div>

<div class="versionchanged">

3.16 Fixed a deadlock (`115634`) where the executor could hang after a worker process exited upon reaching its *max_tasks_per_child* limit while tasks remained queued.

</div>

<div class="method">

terminate_workers()

Attempt to terminate all living worker processes immediately by calling `Process.terminate <multiprocessing.Process.terminate>` on each of them. Internally, it will also call `Executor.shutdown` to ensure that all other resources associated with the executor are freed.

After calling this method the caller should no longer submit tasks to the executor.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

kill_workers()

Attempt to kill all living worker processes immediately by calling `Process.kill <multiprocessing.Process.kill>` on each of them. Internally, it will also call `Executor.shutdown` to ensure that all other resources associated with the executor are freed.

After calling this method the caller should no longer submit tasks to the executor.

<div class="versionadded">

3.14

</div>

</div>

</div>

### ProcessPoolExecutor Example

    import concurrent.futures
    import math

    PRIMES = [
        112272535095293,
        112582705942171,
        112272535095293,
        115280095190773,
        115797848077099,
        1099726899285419]

    def is_prime(n):
        if n < 2:
            return False
        if n == 2:
            return True
        if n % 2 == 0:
            return False

        sqrt_n = int(math.floor(math.sqrt(n)))
        for i in range(3, sqrt_n + 1, 2):
            if n % i == 0:
                return False
        return True

    def main():
        with concurrent.futures.ProcessPoolExecutor() as executor:
            for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
                print('%d is prime: %s' % (number, prime))

    if __name__ == '__main__':
        main()

## Future Objects

The `Future` class encapsulates the asynchronous execution of a callable. `Future` instances are created by `Executor.submit`.

<div class="Future">

Encapsulates the asynchronous execution of a callable. `Future` instances are created by `Executor.submit` and should not be created directly except for testing.

<div class="method">

cancel()

Attempt to cancel the call. If the call is currently being executed or finished running and cannot be cancelled then the method will return `False`, otherwise the call will be cancelled and the method will return `True`.

</div>

<div class="method">

cancelled()

Return `True` if the call was successfully cancelled.

</div>

<div class="method">

running()

Return `True` if the call is currently being executed and cannot be cancelled.

</div>

<div class="method">

done()

Return `True` if the call was successfully cancelled or finished running.

</div>

<div class="method">

result(timeout=None)

Return the value returned by the call. If the call hasn't yet completed then this method will wait up to *timeout* seconds. If the call hasn't completed in *timeout* seconds, then a `TimeoutError` will be raised. *timeout* can be an int or float. If *timeout* is not specified or `None`, there is no limit to the wait time.

If the future is cancelled before completing then `.CancelledError` will be raised.

If the call raised an exception, this method will raise the same exception.

</div>

<div class="method">

exception(timeout=None)

Return the exception raised by the call. If the call hasn't yet completed then this method will wait up to *timeout* seconds. If the call hasn't completed in *timeout* seconds, then a `TimeoutError` will be raised. *timeout* can be an int or float. If *timeout* is not specified or `None`, there is no limit to the wait time.

If the future is cancelled before completing then `.CancelledError` will be raised.

If the call completed without raising, `None` is returned.

</div>

<div class="method">

add_done_callback(fn)

Attaches the callable *fn* to the future. *fn* will be called, with the future as its only argument, when the future is cancelled or finishes running.

Added callables are called in the order that they were added and are always called in a thread belonging to the process that added them. If the callable raises an `Exception` subclass, it will be logged and ignored. If the callable raises a `BaseException` subclass, the behavior is undefined.

If the future has already completed or been cancelled, *fn* will be called immediately.

</div>

The following `Future` methods are meant for use in unit tests and `Executor` implementations.

<div class="method">

set_running_or_notify_cancel()

This method should only be called by `Executor` implementations before executing the work associated with the `Future` and by unit tests.

If the method returns `False` then the `Future` was cancelled, i.e. `Future.cancel` was called and returned `True`. Any threads waiting on the `Future` completing (i.e. through `as_completed` or `wait`) will be woken up.

If the method returns `True` then the `Future` was not cancelled and has been put in the running state, i.e. calls to `Future.running` will return `True`.

This method can only be called once and cannot be called after `Future.set_result` or `Future.set_exception` have been called.

</div>

<div class="method">

set_result(result)

Sets the result of the work associated with the `Future` to *result*.

This method should only be used by `Executor` implementations and unit tests.

<div class="versionchanged">

3.8 This method raises `concurrent.futures.InvalidStateError` if the `Future` is already done.

</div>

</div>

<div class="method">

set_exception(exception)

Sets the result of the work associated with the `Future` to the `Exception` *exception*.

This method should only be used by `Executor` implementations and unit tests.

<div class="versionchanged">

3.8 This method raises `concurrent.futures.InvalidStateError` if the `Future` is already done.

</div>

</div>

</div>

## Module Functions

<div class="function">

wait(fs, timeout=None, return_when=ALL_COMPLETED)

Wait for the `Future` instances (possibly created by different `Executor` instances) given by *fs* to complete. Duplicate futures given to *fs* are removed and will be returned only once. Returns a named 2-tuple of sets. The first set, named `done`, contains the futures that completed (finished or cancelled futures) before the wait completed. The second set, named `not_done`, contains the futures that did not complete (pending or running futures).

*timeout* can be used to control the maximum number of seconds to wait before returning. *timeout* can be an int or float. If *timeout* is not specified or `None`, there is no limit to the wait time.

*return_when* indicates when this function should return. It must be one of the following constants:

<table>
<thead>
<tr>
<th>Constant</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>FIRST_COMPLETED</p>
</div></td>
<td>The function will return when any future finishes or is cancelled.</td>
</tr>
<tr>
<td><div class="data">
<p>FIRST_EXCEPTION</p>
</div></td>
<td>The function will return when any future finishes by raising an exception. If no future raises an exception then it is equivalent to <code class="interpreted-text" role="const">ALL_COMPLETED</code>.</td>
</tr>
<tr>
<td><div class="data">
<p>ALL_COMPLETED</p>
</div></td>
<td>The function will return when all futures finish or are cancelled.</td>
</tr>
</tbody>
</table>

</div>

<div class="function">

as_completed(fs, timeout=None)

Returns an iterator over the `Future` instances (possibly created by different `Executor` instances) given by *fs* that yields futures as they complete (finished or cancelled futures). Any futures given by *fs* that are duplicated will be returned once. Any futures that completed before `as_completed` is called will be yielded first. The returned iterator raises a `TimeoutError` if `~iterator.__next__` is called and the result isn't available after *timeout* seconds from the original call to `as_completed`. *timeout* can be an int or float. If *timeout* is not specified or `None`, there is no limit to the wait time.

</div>

<div class="seealso">

`3148` -- futures - execute computations asynchronously  
The proposal which described this feature for inclusion in the Python standard library.

</div>

## Exception classes

<div class="currentmodule">

concurrent.futures

</div>

<div class="exception">

CancelledError

Raised when a future is cancelled.

</div>

<div class="exception">

TimeoutError

A deprecated alias of `TimeoutError`, raised when a future operation exceeds the given timeout.

<div class="versionchanged">

3.11

This class was made an alias of `TimeoutError`.

</div>

</div>

<div class="exception">

BrokenExecutor

Derived from `RuntimeError`, this exception class is raised when an executor is broken for some reason, and cannot be used to submit or execute new tasks.

<div class="versionadded">

3.7

</div>

</div>

<div class="exception">

InvalidStateError

Raised when an operation is performed on a future that is not allowed in the current state.

<div class="versionadded">

3.8

</div>

</div>

<div class="currentmodule">

concurrent.futures.thread

</div>

<div class="exception">

BrokenThreadPool

Derived from `~concurrent.futures.BrokenExecutor`, this exception class is raised when one of the workers of a `~concurrent.futures.ThreadPoolExecutor` has failed initializing.

<div class="versionadded">

3.7

</div>

</div>

<div class="currentmodule">

concurrent.futures.interpreter

</div>

<div class="exception">

BrokenInterpreterPool

Derived from `~concurrent.futures.thread.BrokenThreadPool`, this exception class is raised when one of the workers of a `~concurrent.futures.InterpreterPoolExecutor` has failed initializing.

<div class="versionadded">

3.14

</div>

</div>

<div class="currentmodule">

concurrent.futures.process

</div>

<div class="exception">

BrokenProcessPool

Derived from `~concurrent.futures.BrokenExecutor` (formerly `RuntimeError`), this exception class is raised when one of the workers of a `~concurrent.futures.ProcessPoolExecutor` has terminated in a non-clean fashion (for example, if it was killed from the outside).

<div class="versionadded">

3.3

</div>

</div>

# `!concurrent.interpreters` --- Multiple interpreters in the same process

<div class="module" data-synopsis="Multiple interpreters in the same process">

concurrent.interpreters

</div>

<div class="versionadded">

3.14

</div>

**Source code:** `Lib/concurrent/interpreters`

------------------------------------------------------------------------

The `!concurrent.interpreters` module constructs higher-level interfaces on top of the lower level `!_interpreters` module.

The module is primarily meant to provide a basic API for managing interpreters (AKA "subinterpreters") and running things in them. Running mostly involves switching to an interpreter (in the current thread) and calling a function in that execution context.

For concurrency, interpreters themselves (and this module) don't provide much more than isolation, which on its own isn't useful. Actual concurrency is available separately through `threads <threading>` -- see [below](#interp-concurrency).

<div class="seealso">

`~concurrent.futures.InterpreterPoolExecutor`  
Combines threads with interpreters in a familiar interface.

<!-- -->

`isolating-extensions-howto`  
How to update an extension module to support multiple interpreters.

`554`

`734`

`684`

</div>

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

## Key details

Before we dive in further, there are a small number of details to keep in mind about using multiple interpreters:

- [isolated](#interp-isolation), by default
- no implicit threads
- not all PyPI packages support use in multiple interpreters yet

## Introduction

An "interpreter" is effectively the execution context of the Python runtime. It contains all of the state the runtime needs to execute a program. This includes things like the import state and builtins. (Each thread, even if there's only the main thread, has some extra runtime state, in addition to the current interpreter, related to the current exception and the bytecode eval loop.)

The concept and functionality of the interpreter have been a part of Python since version 2.2, but the feature was only available through the C-API and not well known, and the [isolation](#interp-isolation) was relatively incomplete until version 3.12.

### Multiple Interpreters and Isolation

A Python implementation may support using multiple interpreters in the same process. CPython has this support. Each interpreter is effectively isolated from the others (with a limited number of carefully managed process-global exceptions to the rule).

That isolation is primarily useful as a strong separation between distinct logical components of a program, where you want to have careful control of how those components interact.

> [!NOTE]
> Interpreters in the same process can technically never be strictly isolated from one another since there are few restrictions on memory access within the same process. The Python runtime makes a best effort at isolation but extension modules may easily violate that. Therefore, do not use multiple interpreters in security-sensitive situations, where they shouldn't have access to each other's data.

### Running in an Interpreter

Running in a different interpreter involves switching to it in the current thread and then calling some function. The runtime will execute the function using the current interpreter's state. The `!concurrent.interpreters` module provides a basic API for creating and managing interpreters, as well as the switch-and-call operation.

No other threads are automatically started for the operation. There is [a helper](#interp-call-in-thread) for that though. There is another dedicated helper for calling the builtin `exec` in an interpreter.

When `exec` (or `eval`) are called in an interpreter, they run using the interpreter's `!__main__` module as the "globals" namespace. The same is true for functions that aren't associated with any module. This is the same as how scripts invoked from the command-line run in the `!__main__` module.

### Concurrency and Parallelism

As noted earlier, interpreters do not provide any concurrency on their own. They strictly represent the isolated execution context the runtime will use *in the current thread*. That isolation makes them similar to processes, but they still enjoy in-process efficiency, like threads.

All that said, interpreters do naturally support certain flavors of concurrency. There's a powerful side effect of that isolation. It enables a different approach to concurrency than you can take with async or threads. It's a similar concurrency model to CSP or the actor model, a model which is relatively easy to reason about.

You can take advantage of that concurrency model in a single thread, switching back and forth between interpreters, Stackless-style. However, this model is more useful when you combine interpreters with multiple threads. This mostly involves starting a new thread, where you switch to another interpreter and run what you want there.

Each actual thread in Python, even if you're only running in the main thread, has its own *current* execution context. Multiple threads can use the same interpreter or different ones.

At a high level, you can think of the combination of threads and interpreters as threads with opt-in sharing.

As a significant bonus, interpreters are sufficiently isolated that they do not share the `GIL`, which means combining threads with multiple interpreters enables full multi-core parallelism. (This has been the case since Python 3.12.)

### Communication Between Interpreters

In practice, multiple interpreters are useful only if we have a way to communicate between them. This usually involves some form of message passing, but can even mean sharing data in some carefully managed way.

With this in mind, the `!concurrent.interpreters` module provides a `queue.Queue` implementation, available through `create_queue`.

### "Sharing" Objects

Any data actually shared between interpreters loses the thread-safety provided by the `GIL`. There are various options for dealing with this in extension modules. However, from Python code the lack of thread-safety means objects can't actually be shared, with a few exceptions. Instead, a copy must be created, which means mutable objects won't stay in sync.

By default, most objects are copied with `pickle` when they are passed to another interpreter. Nearly all of the immutable builtin objects are either directly shared or copied efficiently. For example:

- `None`
- `bool` (`True` and `False`)
- `bytes`
- `str`
- `int`
- `float`
- `tuple` (of similarly supported objects)

There are a small number of Python types that actually share mutable data between interpreters:

- `memoryview`
- `Queue`

## Reference

This module defines the following functions:

<div class="function">

list_all()

Return a `list` of `Interpreter` objects, one for each existing interpreter.

</div>

<div class="function">

get_current()

Return an `Interpreter` object for the currently running interpreter.

</div>

<div class="function">

get_main()

Return an `Interpreter` object for the main interpreter. This is the interpreter the runtime created to run the `REPL` or the script given at the command-line. It is usually the only one.

</div>

<div class="function">

create()

Initialize a new (idle) Python interpreter and return a `Interpreter` object for it.

</div>

<div class="function">

create_queue(maxsize=0)

Initialize a new cross-interpreter queue and return a `Queue` object for it.

*maxsize* is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will be blocked once this size has been reached, until queue items are consumed. If *maxsize* is less than or equal to zero, the queue size is infinite.

</div>

### Interpreter objects

<div class="Interpreter(id)">

A single interpreter in the current process.

Generally, `Interpreter` shouldn't be called directly. Instead, use `create` or one of the other module functions.

`!Interpreter` objects support the `context manager` protocol. Exiting the `with` block calls `close`:

    with interpreters.create() as interp:
        interp.exec('print("hello")')

<div class="versionchanged">

next Added support for using `!Interpreter` objects in a `context manager`.

</div>

<div class="attribute">

id

(read-only)

The underlying interpreter's ID.

</div>

<div class="attribute">

whence

(read-only)

A string describing where the interpreter came from.

</div>

<div class="method">

is_running()

Return `True` if the interpreter is currently executing code in its `!__main__` module and `False` otherwise.

</div>

<div class="method">

close()

Finalize and destroy the interpreter.

</div>

<div class="method">

prepare_main(ns=None, \*\*kwargs)

Bind objects in the interpreter's `!__main__` module.

Some objects are actually shared and some are copied efficiently, but most are copied via `pickle`. See `interp-object-sharing`.

</div>

<div class="method">

exec(code, /)

Run the given source code in the interpreter (in the current thread).

</div>

<div class="method">

call(callable, /, *args,*\*kwargs)

Return the result of running the given function in the interpreter (in the current thread).

</div>

<div id="interp-call-in-thread">

<div class="method">

call_in_thread(callable, /, *args,*\*kwargs)

Run the given function in the interpreter (in a new thread).

</div>

</div>

</div>

### Exceptions

<div class="exception">

InterpreterError

This exception, a subclass of `Exception`, is raised when an interpreter-related error happens.

</div>

<div class="exception">

InterpreterNotFoundError

This exception, a subclass of `InterpreterError`, is raised when the targeted interpreter no longer exists.

</div>

<div class="exception">

ExecutionFailed

This exception, a subclass of `InterpreterError`, is raised when the running code raised an uncaught exception.

<div class="attribute">

excinfo

A basic snapshot of the exception raised in the other interpreter.

</div>

</div>

<div class="exception">

NotShareableError

This exception, a subclass of `TypeError`, is raised when an object cannot be sent to another interpreter.

</div>

### Communicating Between Interpreters

<div class="Queue(id)">

A wrapper around a low-level, cross-interpreter queue, which implements all the methods of `queue.Queue` except for `~queue.Queue.task_done`, `~queue.Queue.join` and `~queue.Queue.shutdown`. The underlying queue can only be created through `create_queue`.

Some objects are actually shared and some are copied efficiently, but most are copied via `pickle`. See `interp-object-sharing`.

<div class="attribute">

id

(read-only)

The queue's ID.

</div>

<div class="method">

qsize()

Return the number of items in the queue.

</div>

<div class="method">

empty()

Return `True` if the queue is empty, `False` otherwise.

</div>

<div class="method">

full()

Return `True` if the queue is full, `False` otherwise.

</div>

<div class="method">

put(obj, block=True, timeout=None)

Put *obj* into the queue. If optional args *block* is true and *timeout* is `None` (the default), block if necessary until a free slot is available. If *timeout* is a positive number, it blocks at most *timeout* seconds and raises the `QueueFull` exception if no free slot was available within that time. Otherwise (*block* is false), put an object on the queue if a free slot is immediately available, else raise the `QueueFull` exception (*timeout* is ignored in that case).

</div>

<div class="method">

put_nowait(obj)

Equivalent to `put(obj, block=False)`.

</div>

<div class="method">

get(block=True, timeout=None)

Remove and return an object from the queue. If optional args *block* is true and *timeout* is `None` (the default), block if necessary until an object is available. If *timeout* is a positive number, it blocks at most *timeout* seconds and raises the `QueueEmpty` exception if no object was available within that time. Otherwise (*block* is false), return an object if one is immediately available, else raise the `QueueEmpty` exception (*timeout* is ignored in that case).

</div>

<div class="method">

get_nowait()

Equivalent to `get(block=False)`.

</div>

</div>

<div class="exception">

QueueEmpty

This exception, a subclass of `queue.Empty`, is raised from `Queue.get` and `Queue.get_nowait` when the queue is empty.

</div>

<div class="exception">

QueueFull

This exception, a subclass of `queue.Full`, is raised from `Queue.put` and `Queue.put_nowait` when the queue is full.

</div>

## Basic usage

Creating an interpreter and running code in it:

    from concurrent import interpreters

    interp = interpreters.create()

    # Run in the current OS thread.

    interp.exec('print("spam!")')

    interp.exec("""if True:
        print('spam!')
        """)

    from textwrap import dedent
    interp.exec(dedent("""
        print('spam!')
        """))

    def run(arg):
        return arg

    res = interp.call(run, 'spam!')
    print(res)

    def run():
        print('spam!')

    interp.call(run)

    # Run in new OS thread.

    t = interp.call_in_thread(run)
    t.join()

Communicating between interpreters with a queue:

    from concurrent import interpreters

    interp = interpreters.create()
    queue = interpreters.create_queue()

    def worker(queue):
        queue.put('spam!')

    t = interp.call_in_thread(worker, queue)
    print(queue.get())
    t.join()

# The `!concurrent` package

This package contains the following modules:

- `concurrent.futures` -- Launching parallel tasks
- `concurrent.interpreters` -- Multiple interpreters in the same process

# `!configparser` --- Configuration file parser

<div class="module" data-synopsis="Configuration file parser.">

configparser

</div>

**Source code:** `Lib/configparser.py`

<div class="index">

pair: .ini; file pair: configuration; file single: ini file single: Windows ini file

</div>

------------------------------------------------------------------------

This module provides the `ConfigParser` class which implements a basic configuration language which provides a structure similar to what's found in Microsoft Windows INI files. You can use this to write Python programs which can be customized by end users easily.

> [!NOTE]
> This library does *not* interpret or write the value-type prefixes used in the Windows Registry extended version of INI syntax.

> [!WARNING]
> Be cautious when parsing data from untrusted sources. A malicious INI file may cause the decoder to consume considerable CPU and memory resources. Limiting the size of data to be parsed is recommended.

<div class="seealso">

Module `tomllib`  
TOML is a well-specified format for application configuration files. It is specifically designed to be an improved version of INI.

Module `shlex`  
Support for creating Unix shell-like mini-languages which can also be used for application configuration files.

Module `json`  
The `json` module implements a subset of JavaScript syntax which is sometimes used for configuration, but does not support comments.

</div>

<div class="testsetup">

import configparser

</div>

<div class="testcleanup">

import os os.remove("example.ini") os.remove("override.ini")

</div>

## Quick Start

Let's take a very basic configuration file that looks like this:

``` ini
[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes

[forge.example]
User = hg

[topsecret.server.example]
Port = 50022
ForwardX11 = no
```

The structure of INI files is described [in the following section](#supported-ini-file-structure). Essentially, the file consists of sections, each of which contains keys with values. `!configparser` classes can read and write such files. Let's start by creating the above configuration file programmatically.

<div class="doctest">

\>\>\> import configparser \>\>\> config = configparser.ConfigParser() \>\>\> config\['DEFAULT'\] = {'ServerAliveInterval': '45', ... 'Compression': 'yes', ... 'CompressionLevel': '9'} \>\>\> config\['forge.example'\] = {} \>\>\> config\['forge.example'\]\['User'\] = 'hg' \>\>\> config\['topsecret.server.example'\] = {} \>\>\> topsecret = config\['topsecret.server.example'\] \>\>\> topsecret\['Port'\] = '50022' \# mutates the parser \>\>\> topsecret\['ForwardX11'\] = 'no' \# same here \>\>\> config\['DEFAULT'\]\['ForwardX11'\] = 'yes' \>\>\> with open('example.ini', 'w') as configfile: ... config.write(configfile) ...

</div>

As you can see, we can treat a config parser much like a dictionary. There are differences, [outlined later](#mapping-protocol-access), but the behavior is very close to what you would expect from a dictionary.

Now that we have created and saved a configuration file, let's read it back and explore the data it holds.

<div class="doctest">

\>\>\> config = configparser.ConfigParser() \>\>\> config.sections() \[\] \>\>\> config.read('example.ini') \['example.ini'\] \>\>\> config.sections() \['forge.example', 'topsecret.server.example'\] \>\>\> 'forge.example' in config True \>\>\> 'python.org' in config False \>\>\> config\['forge.example'\]\['User'\] 'hg' \>\>\> config\['DEFAULT'\]\['Compression'\] 'yes' \>\>\> topsecret = config\['topsecret.server.example'\] \>\>\> topsecret\['ForwardX11'\] 'no' \>\>\> topsecret\['Port'\] '50022' \>\>\> for key in config\['forge.example'\]: \# doctest: +SKIP ... print(key) user compressionlevel serveraliveinterval compression forwardx11 \>\>\> config\['forge.example'\]\['ForwardX11'\] 'yes'

</div>

As we can see above, the API is pretty straightforward. The only bit of magic involves the `DEFAULT` section which provides default values for all other sections[^1]. Note also that keys in sections are case-insensitive and stored in lowercase[^2].

It is possible to read several configurations into a single `ConfigParser`, where the most recently added configuration has the highest priority. Any conflicting keys are taken from the more recent configuration while the previously existing keys are retained. The example below reads in an `override.ini` file, which will override any conflicting keys from the `example.ini` file.

``` ini
[DEFAULT]
ServerAliveInterval = -1
```

<div class="doctest">

\>\>\> config_override = configparser.ConfigParser() \>\>\> config_override\['DEFAULT'\] = {'ServerAliveInterval': '-1'} \>\>\> with open('override.ini', 'w') as configfile: ... config_override.write(configfile) ... \>\>\> config_override = configparser.ConfigParser() \>\>\> config_override.read(\['example.ini', 'override.ini'\]) \['example.ini', 'override.ini'\] \>\>\> print(config_override.get('DEFAULT', 'ServerAliveInterval')) -1

</div>

This behaviour is equivalent to a `ConfigParser.read` call with several files passed to the *filenames* parameter.

## Supported Datatypes

Config parsers do not guess datatypes of values in configuration files, always storing them internally as strings. This means that if you need other datatypes, you should convert on your own:

<div class="doctest">

\>\>\> int(topsecret\['Port'\]) 50022 \>\>\> float(topsecret\['CompressionLevel'\]) 9.0

</div>

Since this task is so common, config parsers provide a range of handy getter methods to handle integers, floats and booleans. The last one is the most interesting because simply passing the value to `bool()` would do no good since `bool('False')` is still `True`. This is why config parsers also provide `~ConfigParser.getboolean`. This method is case-insensitive and recognizes Boolean values from `'yes'`/`'no'`, `'on'`/`'off'`, `'true'`/`'false'` and `'1'`/`'0'`[^3]. For example:

<div class="doctest">

\>\>\> topsecret.getboolean('ForwardX11') False \>\>\> config\['forge.example'\].getboolean('ForwardX11') True \>\>\> config.getboolean('forge.example', 'Compression') True

</div>

Apart from `~ConfigParser.getboolean`, config parsers also provide equivalent `~ConfigParser.getint` and `~ConfigParser.getfloat` methods. You can register your own converters and customize the provided ones.[^4]

## Fallback Values

As with a dictionary, you can use a section's `~ConfigParser.get` method to provide fallback values:

<div class="doctest">

\>\>\> topsecret.get('Port') '50022' \>\>\> topsecret.get('CompressionLevel') '9' \>\>\> topsecret.get('Cipher') \>\>\> topsecret.get('Cipher', '3des-cbc') '3des-cbc'

</div>

Please note that default values have precedence over fallback values. For instance, in our example the `'CompressionLevel'` key was specified only in the `'DEFAULT'` section. If we try to get it from the section `'topsecret.server.example'`, we will always get the default, even if we specify a fallback:

<div class="doctest">

\>\>\> topsecret.get('CompressionLevel', '3') '9'

</div>

One more thing to be aware of is that the parser-level `~ConfigParser.get` method provides a custom, more complex interface, maintained for backwards compatibility. When using this method, a fallback value can be provided via the `fallback` keyword-only argument:

<div class="doctest">

\>\>\> config.get('forge.example', 'monster', ... fallback='No such things as monsters') 'No such things as monsters'

</div>

The same `fallback` argument can be used with the `~ConfigParser.getint`, `~ConfigParser.getfloat` and `~ConfigParser.getboolean` methods, for example:

<div class="doctest">

\>\>\> 'BatchMode' in topsecret False \>\>\> topsecret.getboolean('BatchMode', fallback=True) True \>\>\> config\['DEFAULT'\]\['BatchMode'\] = 'no' \>\>\> topsecret.getboolean('BatchMode', fallback=True) False

</div>

## Supported INI File Structure

A configuration file consists of sections, each led by a `[section]` header, followed by key/value entries separated by a specific string (`=` or `:` by default[^5]). By default, section names are case sensitive but keys are not [^6]. Leading and trailing whitespace is removed from keys and values. Values can be omitted if the parser is configured to allow it[^7], in which case the key/value delimiter may also be left out. Values can also span multiple lines, as long as they are indented deeper than the first line of the value. Depending on the parser's mode, blank lines may be treated as parts of multiline values or ignored.

By default, a valid section name can be any string that does not contain '\n'. To change this, see `ConfigParser.SECTCRE`.

The first section name may be omitted if the parser is configured to allow an unnamed top level section with `allow_unnamed_section=True`. In this case, the keys/values may be retrieved by `UNNAMED_SECTION` as in `config[UNNAMED_SECTION]`.

Configuration files may include comments, prefixed by specific characters (`#` and `;` by default[^8]). Comments may appear on their own on an otherwise empty line, possibly indented.[^9]

For example:

``` ini
[Simple Values]
key=value
spaces in keys=allowed
spaces in values=allowed as well
spaces around the delimiter = obviously
you can also use : to delimit keys from values

[All Values Are Strings]
values like this: 1000000
or this: 3.14159265359
are they treated as numbers? : no
integers, floats and booleans are held as: strings
can use the API to get converted values directly: true

[Multiline Values]
chorus: I'm a lumberjack, and I'm okay
    I sleep all night and I work all day

[No Values]
key_without_value
empty string value here =

[You can use comments]
# like this
; or this

# By default only in an empty line.
# Inline comments can be harmful because they prevent users
# from using the delimiting characters as parts of values.
# That being said, this can be customized.

    [Sections Can Be Indented]
        can_values_be_as_well = True
        does_that_mean_anything_special = False
        purpose = formatting for readability
        multiline_values = are
            handled just fine as
            long as they are indented
            deeper than the first line
            of a value
        # Did I mention we can indent comments, too?
```

## Unnamed Sections

The name of the first section (or unique) may be omitted and values retrieved by the `UNNAMED_SECTION` attribute.

<div class="doctest">

\>\>\> config = """ ... option = value ... ... \[ Section 2 \] ... another = val ... """ \>\>\> unnamed = configparser.ConfigParser(allow_unnamed_section=True) \>\>\> unnamed.read_string(config) \>\>\> unnamed.get(configparser.UNNAMED_SECTION, 'option') 'value'

</div>

## Interpolation of values

On top of the core functionality, `ConfigParser` supports interpolation. This means values can be preprocessed before returning them from `get()` calls.

<div class="index">

single: % (percent); interpolation in configuration files

</div>

<div class="BasicInterpolation()">

The default implementation used by `ConfigParser`. It enables values to contain format strings which refer to other values in the same section, or values in the special default section[^10]. Additional default values can be provided on initialization.

For example:

``` ini
[Paths]
home_dir: /Users
my_dir: %(home_dir)s/lumberjack
my_pictures: %(my_dir)s/Pictures

[Escape]
# use a %% to escape the % sign (% is the only character that needs to be escaped):
gain: 80%%
```

In the example above, `ConfigParser` with *interpolation* set to `BasicInterpolation()` would resolve `%(home_dir)s` to the value of `home_dir` (`/Users` in this case). `%(my_dir)s` in effect would resolve to `/Users/lumberjack`. All interpolations are done on demand so keys used in the chain of references do not have to be specified in any specific order in the configuration file.

With `interpolation` set to `None`, the parser would simply return `%(my_dir)s/Pictures` as the value of `my_pictures` and `%(home_dir)s/lumberjack` as the value of `my_dir`.

</div>

<div class="index">

single: \$ (dollar); interpolation in configuration files

</div>

<div class="ExtendedInterpolation()">

An alternative handler for interpolation which implements a more advanced syntax, used for instance in `zc.buildout`. Extended interpolation is using `${section:option}` to denote a value from a foreign section. Interpolation can span multiple levels. For convenience, if the `section:` part is omitted, interpolation defaults to the current section (and possibly the default values from the special section).

For example, the configuration specified above with basic interpolation, would look like this with extended interpolation:

``` ini
[Paths]
home_dir: /Users
my_dir: ${home_dir}/lumberjack
my_pictures: ${my_dir}/Pictures

[Escape]
# use a $$ to escape the $ sign ($ is the only character that needs to be escaped):
cost: $$80
```

Values from other sections can be fetched as well:

``` ini
[Common]
home_dir: /Users
library_dir: /Library
system_dir: /System
macports_dir: /opt/local

[Frameworks]
Python: 3.2
path: ${Common:system_dir}/Library/Frameworks/

[Arthur]
nickname: Two Sheds
last_name: Jackson
my_dir: ${Common:home_dir}/twosheds
my_pictures: ${my_dir}/Pictures
python_dir: ${Frameworks:path}/Python/Versions/${Frameworks:Python}
```

</div>

## Mapping Protocol Access

<div class="versionadded">

3.2

</div>

Mapping protocol access is a generic name for functionality that enables using custom objects as if they were dictionaries. In case of `!configparser`, the mapping interface implementation is using the `parser['section']['option']` notation.

`parser['section']` in particular returns a proxy for the section's data in the parser. This means that the values are not copied but they are taken from the original parser on demand. What's even more important is that when values are changed on a section proxy, they are actually mutated in the original parser.

`!configparser` objects behave as close to actual dictionaries as possible. The mapping interface is complete and adheres to the `~collections.abc.MutableMapping` ABC. However, there are a few differences that should be taken into account:

- By default, all keys in sections are accessible in a case-insensitive manner [^11]. E.g. `for option in parser["section"]` yields only `optionxform`'ed option key names. This means lowercased keys by default. At the same time, for a section that holds the key `'a'`, both expressions return `True`:

      "a" in parser["section"]
      "A" in parser["section"]

- All sections include `DEFAULTSECT` values as well which means that `.clear()` on a section may not leave the section visibly empty. This is because default values cannot be deleted from the section (because technically they are not there). If they are overridden in the section, deleting causes the default value to be visible again. Trying to delete a default value causes a `KeyError`.

- `DEFAULTSECT` cannot be removed from the parser:

  - trying to delete it raises `ValueError`,
  - `parser.clear()` leaves it intact,
  - `parser.popitem()` never returns it.

- `parser.get(section, option, **kwargs)` - the second argument is **not** a fallback value. Note however that the section-level `get()` methods are compatible both with the mapping protocol and the classic configparser API.

- `parser.items()` is compatible with the mapping protocol (returns a list of *section_name*, *section_proxy* pairs including the DEFAULTSECT). However, this method can also be invoked with arguments: `parser.items(section, raw, vars)`. The latter call returns a list of *option*, *value* pairs for a specified `section`, with all interpolations expanded (unless `raw=True` is provided).

The mapping protocol is implemented on top of the existing legacy API so that subclasses overriding the original interface still should have mappings working as expected.

## Customizing Parser Behaviour

There are nearly as many INI format variants as there are applications using it. `!configparser` goes a long way to provide support for the largest sensible set of INI styles available. The default functionality is mainly dictated by historical background and it's very likely that you will want to customize some of the features.

The most common way to change the way a specific config parser works is to use the `!__init__` options:

- *defaults*, default value: `None`

  This option accepts a dictionary of key-value pairs which will be initially put in the `DEFAULT` section. This makes for an elegant way to support concise configuration files that don't specify values which are the same as the documented default.

  Hint: if you want to specify default values for a specific section, use `~ConfigParser.read_dict` before you read the actual file.

- *dict_type*, default value: `dict`

  This option has a major impact on how the mapping protocol will behave and how the written configuration files look. With the standard dictionary, every section is stored in the order they were added to the parser. Same goes for options within sections.

  An alternative dictionary type can be used for example to sort sections and options on write-back.

  Please note: there are ways to add a set of key-value pairs in a single operation. When you use a regular dictionary in those operations, the order of the keys will be ordered. For example:

  <div class="doctest">

  \>\>\> parser = configparser.ConfigParser() \>\>\> parser.read_dict({'section1': {'key1': 'value1', ... 'key2': 'value2', ... 'key3': 'value3'}, ... 'section2': {'keyA': 'valueA', ... 'keyB': 'valueB', ... 'keyC': 'valueC'}, ... 'section3': {'foo': 'x', ... 'bar': 'y', ... 'baz': 'z'} ... }) \>\>\> parser.sections() \['section1', 'section2', 'section3'\] \>\>\> \[option for option in parser\['section3'\]\] \['foo', 'bar', 'baz'\]

  </div>

- *allow_no_value*, default value: `False`

  Some configuration files are known to include settings without values, but which otherwise conform to the syntax supported by `!configparser`. The *allow_no_value* parameter to the constructor can be used to indicate that such values should be accepted:

  <div class="doctest">

  \>\>\> import configparser

  \>\>\> sample_config = """ ... \[mysqld\] ... user = mysql ... pid-file = /var/run/mysqld/mysqld.pid ... skip-external-locking ... old_passwords = 1 ... skip-bdb ... \# we don't need ACID today ... skip-innodb ... """ \>\>\> config = configparser.ConfigParser(allow_no_value=True) \>\>\> config.read_string(sample_config)

  \>\>\> \# Settings with values are treated as before: \>\>\> config\["mysqld"\]\["user"\] 'mysql'

  \>\>\> \# Settings without values provide None: \>\>\> config\["mysqld"\]\["skip-bdb"\]

  \>\>\> \# Settings which aren't specified still raise an error: \>\>\> config\["mysqld"\]\["does-not-exist"\] Traceback (most recent call last): ... KeyError: 'does-not-exist'

  </div>

- *delimiters*, default value: `('=', ':')`

  Delimiters are substrings that delimit keys from values within a section. The first occurrence of a delimiting substring on a line is considered a delimiter. This means values (but not keys) can contain the delimiters.

  See also the *space_around_delimiters* argument to `ConfigParser.write`.

- *comment_prefixes*, default value: `('#', ';')`

- *inline_comment_prefixes*, default value: `None`

  Comment prefixes are strings that indicate the start of a valid comment within a config file. *comment_prefixes* are used only on otherwise empty lines (optionally indented) whereas *inline_comment_prefixes* can be used after every valid value (e.g. section names, options and empty lines as well). By default inline comments are disabled and `'#'` and `';'` are used as prefixes for whole line comments.

  <div class="versionchanged">

  3.2 In previous versions of `!configparser` behaviour matched `comment_prefixes=('#',';')` and `inline_comment_prefixes=(';',)`.

  </div>

  Please note that config parsers don't support escaping of comment prefixes so using *inline_comment_prefixes* may prevent users from specifying option values with characters used as comment prefixes. When in doubt, avoid setting *inline_comment_prefixes*. In any circumstances, the only way of storing comment prefix characters at the beginning of a line in multiline values is to interpolate the prefix, for example:

      >>> from configparser import ConfigParser, ExtendedInterpolation
      >>> parser = ConfigParser(interpolation=ExtendedInterpolation())
      >>> # the default BasicInterpolation could be used as well
      >>> parser.read_string("""
      ... [DEFAULT]
      ... hash = #
      ...
      ... [hashes]
      ... shebang =
      ...   ${hash}!/usr/bin/env python
      ...   ${hash} -*- coding: utf-8 -*-
      ...
      ... extensions =
      ...   enabled_extension
      ...   another_extension
      ...   #disabled_by_comment
      ...   yet_another_extension
      ...
      ... interpolation not necessary = if # is not at line start
      ... even in multiline values = line #1
      ...   line #2
      ...   line #3
      ... """)
      >>> print(parser['hashes']['shebang'])
      <BLANKLINE>
      #!/usr/bin/env python
      # -*- coding: utf-8 -*-
      >>> print(parser['hashes']['extensions'])
      <BLANKLINE>
      enabled_extension
      another_extension
      yet_another_extension
      >>> print(parser['hashes']['interpolation not necessary'])
      if # is not at line start
      >>> print(parser['hashes']['even in multiline values'])
      line #1
      line #2
      line #3

- *strict*, default value: `True`

  When set to `True`, the parser will not allow for any section or option duplicates while reading from a single source (using `~ConfigParser.read_file`, `~ConfigParser.read_string` or `~ConfigParser.read_dict`). It is recommended to use strict parsers in new applications.

  <div class="versionchanged">

  3.2 In previous versions of `!configparser` behaviour matched `strict=False`.

  </div>

- *empty_lines_in_values*, default value: `True`

  In config parsers, values can span multiple lines as long as they are indented more than the key that holds them. By default parsers also let empty lines to be parts of values. At the same time, keys can be arbitrarily indented themselves to improve readability. In consequence, when configuration files get big and complex, it is easy for the user to lose track of the file structure. Take for instance:

  ``` ini
  [Section]
  key = multiline
    value with a gotcha

   this = is still a part of the multiline value of 'key'
  ```

  This can be especially problematic for the user to see if she's using a proportional font to edit the file. That is why when your application does not need values with empty lines, you should consider disallowing them. This will make empty lines split keys every time. In the example above, it would produce two keys, `key` and `this`.

- *default_section*, default value: `configparser.DEFAULTSECT` (that is: `"DEFAULT"`)

  The convention of allowing a special section of default values for other sections or interpolation purposes is a powerful concept of this library, letting users create complex declarative configurations. This section is normally called `"DEFAULT"` but this can be customized to point to any other valid section name. Some typical values include: `"general"` or `"common"`. The name provided is used for recognizing default sections when reading from any source and is used when writing configuration back to a file. Its current value can be retrieved using the `parser_instance.default_section` attribute and may be modified at runtime (i.e. to convert files from one format to another).

- *interpolation*, default value: `configparser.BasicInterpolation`

  Interpolation behaviour may be customized by providing a custom handler through the *interpolation* argument. `None` can be used to turn off interpolation completely, `ExtendedInterpolation()` provides a more advanced variant inspired by `zc.buildout`. More on the subject in the [dedicated documentation section](#interpolation-of-values). `RawConfigParser` has a default value of `None`.

- *converters*, default value: not set

  Config parsers provide option value getters that perform type conversion. By default `~ConfigParser.getint`, `~ConfigParser.getfloat`, and `~ConfigParser.getboolean` are implemented. Should other getters be desirable, users may define them in a subclass or pass a dictionary where each key is a name of the converter and each value is a callable implementing said conversion. For instance, passing `{'decimal': decimal.Decimal}` would add `!getdecimal` on both the parser object and all section proxies. In other words, it will be possible to write both `parser_instance.getdecimal('section', 'key', fallback=0)` and `parser_instance['section'].getdecimal('key', 0)`.

  If the converter needs to access the state of the parser, it can be implemented as a method on a config parser subclass. If the name of this method starts with `get`, it will be available on all section proxies, in the dict-compatible form (see the `getdecimal()` example above).

More advanced customization may be achieved by overriding default values of these parser attributes. The defaults are defined on the classes, so they may be overridden by subclasses or by attribute assignment.

<div class="attribute">

ConfigParser.BOOLEAN_STATES

By default when using `~ConfigParser.getboolean`, config parsers consider the following values `True`: `'1'`, `'yes'`, `'true'`, `'on'` and the following values `False`: `'0'`, `'no'`, `'false'`, `'off'`. You can override this by specifying a custom dictionary of strings and their Boolean outcomes. For example:

<div class="doctest">

\>\>\> custom = configparser.ConfigParser() \>\>\> custom\['section1'\] = {'funky': 'nope'} \>\>\> custom\['section1'\].getboolean('funky') Traceback (most recent call last): ... ValueError: Not a boolean: nope \>\>\> custom.BOOLEAN_STATES = {'sure': True, 'nope': False} \>\>\> custom\['section1'\].getboolean('funky') False

</div>

Other typical Boolean pairs include `accept`/`reject` or `enabled`/`disabled`.

</div>

<div class="method" data-noindex="">

ConfigParser.optionxform(option)

This method transforms option names on every read, get, or set operation. The default converts the name to lowercase. This also means that when a configuration file gets written, all keys will be lowercase. Override this method if that's unsuitable. For example:

<div class="doctest">

\>\>\> config = """ ... \[Section1\] ... Key = Value ... ... \[Section2\] ... AnotherKey = Value ... """ \>\>\> typical = configparser.ConfigParser() \>\>\> typical.read_string(config) \>\>\> list(typical\['Section1'\].keys()) \['key'\] \>\>\> list(typical\['Section2'\].keys()) \['anotherkey'\] \>\>\> custom = configparser.RawConfigParser() \>\>\> custom.optionxform = lambda option: option \>\>\> custom.read_string(config) \>\>\> list(custom\['Section1'\].keys()) \['Key'\] \>\>\> list(custom\['Section2'\].keys()) \['AnotherKey'\]

</div>

> [!NOTE]
> The optionxform function transforms option names to a canonical form. This should be an idempotent function: if the name is already in canonical form, it should be returned unchanged.

</div>

<div class="attribute">

ConfigParser.SECTCRE

A compiled regular expression used to parse section headers. The default matches `[section]` to the name `"section"`. Whitespace is considered part of the section name, thus `[  larch  ]` will be read as a section of name `"  larch  "`. Override this attribute if that's unsuitable. For example:

<div class="doctest">

\>\>\> import re \>\>\> config = """ ... \[Section 1\] ... option = value ... ... \[ Section 2 \] ... another = val ... """ \>\>\> typical = configparser.ConfigParser() \>\>\> typical.read_string(config) \>\>\> typical.sections() \['Section 1', ' Section 2 '\] \>\>\> custom = configparser.ConfigParser() \>\>\> custom.SECTCRE = re.compile(r"\[ *(?P\<header\>\[^\]\]+?)*\]") \>\>\> custom.read_string(config) \>\>\> custom.sections() \['Section 1', 'Section 2'\]

</div>

> [!NOTE]
> While ConfigParser objects also use an `OPTCRE` attribute for recognizing option lines, it's not recommended to override it because that would interfere with constructor options *allow_no_value* and *delimiters*.

</div>

## Legacy API Examples

Mainly because of backwards compatibility concerns, `!configparser` provides also a legacy API with explicit `get`/`set` methods. While there are valid use cases for the methods outlined below, mapping protocol access is preferred for new projects. The legacy API is at times more advanced, low-level and downright counterintuitive.

An example of writing to a configuration file:

    import configparser

    config = configparser.RawConfigParser()

    # Please note that using RawConfigParser's set functions, you can assign
    # non-string values to keys internally, but will receive an error when
    # attempting to write to a file or when you get it in non-raw mode. Setting
    # values using the mapping protocol or ConfigParser's set() does not allow
    # such assignments to take place.
    config.add_section('Section1')
    config.set('Section1', 'an_int', '15')
    config.set('Section1', 'a_bool', 'true')
    config.set('Section1', 'a_float', '3.1415')
    config.set('Section1', 'baz', 'fun')
    config.set('Section1', 'bar', 'Python')
    config.set('Section1', 'foo', '%(bar)s is %(baz)s!')

    # Writing our configuration file to 'example.cfg'
    with open('example.cfg', 'w') as configfile:
        config.write(configfile)

An example of reading the configuration file again:

    import configparser

    config = configparser.RawConfigParser()
    config.read('example.cfg')

    # getfloat() raises an exception if the value is not a float
    # getint() and getboolean() also do this for their respective types
    a_float = config.getfloat('Section1', 'a_float')
    an_int = config.getint('Section1', 'an_int')
    print(a_float + an_int)

    # Notice that the next output does not interpolate '%(bar)s' or '%(baz)s'.
    # This is because we are using a RawConfigParser().
    if config.getboolean('Section1', 'a_bool'):
        print(config.get('Section1', 'foo'))

To get interpolation, use `ConfigParser`:

    import configparser

    cfg = configparser.ConfigParser()
    cfg.read('example.cfg')

    # Set the optional *raw* argument of get() to True if you wish to disable
    # interpolation in a single get operation.
    print(cfg.get('Section1', 'foo', raw=False))  # -> "Python is fun!"
    print(cfg.get('Section1', 'foo', raw=True))   # -> "%(bar)s is %(baz)s!"

    # The optional *vars* argument is a dict with members that will take
    # precedence in interpolation.
    print(cfg.get('Section1', 'foo', vars={'bar': 'Documentation',
                                           'baz': 'evil'}))

    # The optional *fallback* argument can be used to provide a fallback value
    print(cfg.get('Section1', 'foo'))
          # -> "Python is fun!"

    print(cfg.get('Section1', 'foo', fallback='Monty is not.'))
          # -> "Python is fun!"

    print(cfg.get('Section1', 'monster', fallback='No such things as monsters.'))
          # -> "No such things as monsters."

    # A bare print(cfg.get('Section1', 'monster')) would raise NoOptionError
    # but we can also use:

    print(cfg.get('Section1', 'monster', fallback=None))
          # -> None

Default values are available in both types of ConfigParsers. They are used in interpolation if an option used is not defined elsewhere. :

    import configparser

    # New instance with 'bar' and 'baz' defaulting to 'Life' and 'hard' each
    config = configparser.ConfigParser({'bar': 'Life', 'baz': 'hard'})
    config.read('example.cfg')

    print(config.get('Section1', 'foo'))     # -> "Python is fun!"
    config.remove_option('Section1', 'bar')
    config.remove_option('Section1', 'baz')
    print(config.get('Section1', 'foo'))     # -> "Life is hard!"

## ConfigParser Objects

<div class="ConfigParser(defaults=None, dict_type=dict, allow_no_value=False, *, \ delimiters=('=', ':'), comment_prefixes=('#', ';'), \ inline_comment_prefixes=None, strict=True, \ empty_lines_in_values=True, \ default_section=configparser.DEFAULTSECT, \ interpolation=BasicInterpolation(), converters={}, \ allow_unnamed_section=False)">

The main configuration parser. When *defaults* is given, it is initialized into the dictionary of intrinsic defaults. When *dict_type* is given, it will be used to create the dictionary objects for the list of sections, for the options within a section, and for the default values.

When *delimiters* is given, it is used as the set of substrings that divide keys from values. When *comment_prefixes* is given, it will be used as the set of substrings that prefix comments in otherwise empty lines. Comments can be indented. When *inline_comment_prefixes* is given, it will be used as the set of substrings that prefix comments in non-empty lines.

When *strict* is `True` (the default), the parser won't allow for any section or option duplicates while reading from a single source (file, string or dictionary), raising `DuplicateSectionError` or `DuplicateOptionError`. When *empty_lines_in_values* is `False` (default: `True`), each empty line marks the end of an option. Otherwise, internal empty lines of a multiline option are kept as part of the value. When *allow_no_value* is `True` (default: `False`), options without values are accepted; the value held for these is `None` and they are serialized without the trailing delimiter.

When *default_section* is given, it specifies the name for the special section holding default values for other sections and interpolation purposes (normally named `"DEFAULT"`). This value can be retrieved and changed at runtime using the `default_section` instance attribute. This won't re-evaluate an already parsed config file, but will be used when writing parsed settings to a new config file.

Interpolation behaviour may be customized by providing a custom handler through the *interpolation* argument. `None` can be used to turn off interpolation completely, `ExtendedInterpolation()` provides a more advanced variant inspired by `zc.buildout`. More on the subject in the [dedicated documentation section](#interpolation-of-values).

All option names used in interpolation will be passed through the `optionxform` method just like any other option name reference. For example, using the default implementation of `optionxform` (which converts option names to lower case), the values `foo %(bar)s` and `foo %(BAR)s` are equivalent.

When *converters* is given, it should be a dictionary where each key represents the name of a type converter and each value is a callable implementing the conversion from string to the desired datatype. Every converter gets its own corresponding `!get*` method on the parser object and section proxies.

When *allow_unnamed_section* is `True` (default: `False`), the first section name can be omitted. See the ["Unnamed Sections" section](#unnamed-sections).

It is possible to read several configurations into a single `ConfigParser`, where the most recently added configuration has the highest priority. Any conflicting keys are taken from the more recent configuration while the previously existing keys are retained. The example below reads in an `override.ini` file, which will override any conflicting keys from the `example.ini` file.

``` ini
[DEFAULT]
ServerAliveInterval = -1
```

<div class="doctest">

\>\>\> config_override = configparser.ConfigParser() \>\>\> config_override\['DEFAULT'\] = {'ServerAliveInterval': '-1'} \>\>\> with open('override.ini', 'w') as configfile: ... config_override.write(configfile) ... \>\>\> config_override = configparser.ConfigParser() \>\>\> config_override.read(\['example.ini', 'override.ini'\]) \['example.ini', 'override.ini'\] \>\>\> print(config_override.get('DEFAULT', 'ServerAliveInterval')) -1

</div>

<div class="versionchanged">

3.1 The default *dict_type* is `collections.OrderedDict`.

</div>

<div class="versionchanged">

3.2 *allow_no_value*, *delimiters*, *comment_prefixes*, *strict*, *empty_lines_in_values*, *default_section* and *interpolation* were added.

</div>

<div class="versionchanged">

3.5 The *converters* argument was added.

</div>

<div class="versionchanged">

3.7 The *defaults* argument is read with `read_dict`, providing consistent behavior across the parser: non-string keys and values are implicitly converted to strings.

</div>

<div class="versionchanged">

3.8 The default *dict_type* is `dict`, since it now preserves insertion order.

</div>

<div class="versionchanged">

3.13 Raise a `MultilineContinuationError` when *allow_no_value* is `True`, and a key without a value is continued with an indented line.

</div>

<div class="versionchanged">

3.13 The *allow_unnamed_section* argument was added.

</div>

<div class="method">

defaults()

Return a dictionary containing the instance-wide defaults.

</div>

<div class="method">

sections()

Return a list of the sections available; the *default section* is not included in the list.

</div>

<div class="method">

add_section(section)

Add a section named *section* to the instance. If a section by the given name already exists, `DuplicateSectionError` is raised. If the *default section* name is passed, `ValueError` is raised. The name of the section must be a string; if not, `TypeError` is raised.

<div class="versionchanged">

3.2 Non-string section names raise `TypeError`.

</div>

</div>

<div class="method">

has_section(section)

Indicates whether the named *section* is present in the configuration. The *default section* is not acknowledged.

</div>

<div class="method">

options(section)

Return a list of options available in the specified *section*.

</div>

<div class="method">

has_option(section, option)

If the given *section* exists, and contains the given *option*, return `True`; otherwise return `False`. If the specified *section* is `None` or an empty string, DEFAULT is assumed.

</div>

<div class="method">

read(filenames, encoding=None)

Attempt to read and parse an iterable of filenames, returning a list of filenames which were successfully parsed.

If *filenames* is a string, a `bytes` object or a `path-like object`, it is treated as a single filename. If a file named in *filenames* cannot be opened, that file will be ignored. This is designed so that you can specify an iterable of potential configuration file locations (for example, the current directory, the user's home directory, and some system-wide directory), and all existing configuration files in the iterable will be read.

If none of the named files exist, the `ConfigParser` instance will contain an empty dataset. An application which requires initial values to be loaded from a file should load the required file or files using `read_file` before calling `read` for any optional files:

    import configparser, os

    config = configparser.ConfigParser()
    config.read_file(open('defaults.cfg'))
    config.read(['site.cfg', os.path.expanduser('~/.myapp.cfg')],
                encoding='cp1250')

<div class="versionchanged">

3.2 Added the *encoding* parameter. Previously, all files were read using the default encoding for `open`.

</div>

<div class="versionchanged">

3.6.1 The *filenames* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.7 The *filenames* parameter accepts a `bytes` object.

</div>

</div>

<div class="method">

read_file(f, source=None)

Read and parse configuration data from *f* which must be an iterable yielding Unicode strings (for example files opened in text mode).

Optional argument *source* specifies the name of the file being read. If not given and *f* has a `!name` attribute, that is used for *source*; the default is `'<???>'`.

<div class="versionadded">

3.2 Replaces `!readfp`.

</div>

</div>

<div class="method">

read_string(string, source='\<string\>')

Parse configuration data from a string.

Optional argument *source* specifies a context-specific name of the string passed. If not given, `'<string>'` is used. This should commonly be a filesystem path or a URL.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

read_dict(dictionary, source='\<dict\>')

Load configuration from any object that provides a dict-like `items()` method. Keys are section names, values are dictionaries with keys and values that should be present in the section. If the used dictionary type preserves order, sections and their keys will be added in order. Values are automatically converted to strings.

Optional argument *source* specifies a context-specific name of the dictionary passed. If not given, `<dict>` is used.

This method can be used to copy state between parsers.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

get(section, option, \*, raw=False, vars=None\[, fallback\])

Get an *option* value for the named *section*. If *vars* is provided, it must be a dictionary. The *option* is looked up in *vars* (if provided), *section*, and in *DEFAULTSECT* in that order. If the key is not found and *fallback* is provided, it is used as a fallback value. `None` can be provided as a *fallback* value.

All the `'%'` interpolations are expanded in the return values, unless the *raw* argument is true. Values for interpolation keys are looked up in the same manner as the option.

<div class="versionchanged">

3.2 Arguments *raw*, *vars* and *fallback* are keyword only to protect users from trying to use the third argument as the *fallback* fallback (especially when using the mapping protocol).

</div>

</div>

<div class="method">

getint(section, option, \*, raw=False, vars=None\[, fallback\])

A convenience method which coerces the *option* in the specified *section* to an integer. See `get` for explanation of *raw*, *vars* and *fallback*.

</div>

<div class="method">

getfloat(section, option, \*, raw=False, vars=None\[, fallback\])

A convenience method which coerces the *option* in the specified *section* to a floating-point number. See `get` for explanation of *raw*, *vars* and *fallback*.

</div>

<div class="method">

getboolean(section, option, \*, raw=False, vars=None\[, fallback\])

A convenience method which coerces the *option* in the specified *section* to a Boolean value. Note that the accepted values for the option are `'1'`, `'yes'`, `'true'`, and `'on'`, which cause this method to return `True`, and `'0'`, `'no'`, `'false'`, and `'off'`, which cause it to return `False`. These string values are checked in a case-insensitive manner. Any other value will cause it to raise `ValueError`. See `get` for explanation of *raw*, *vars* and *fallback*.

</div>

<div class="method">

items(raw=False, vars=None) items(section, raw=False, vars=None)

When *section* is not given, return a list of *section_name*, *section_proxy* pairs, including DEFAULTSECT.

Otherwise, return a list of *name*, *value* pairs for the options in the given *section*. Optional arguments have the same meaning as for the `get` method.

<div class="versionchanged">

3.8 Items present in *vars* no longer appear in the result. The previous behaviour mixed actual parser options with variables provided for interpolation.

</div>

</div>

<div class="method">

set(section, option, value)

If the given section exists, set the given option to the specified value; otherwise raise `NoSectionError`. *option* and *value* must be strings; if not, `TypeError` is raised.

</div>

<div class="method">

write(fileobject, space_around_delimiters=True)

Write a representation of the configuration to the specified `file
object`, which must be opened in text mode (accepting strings). This representation can be parsed by a future `read` call. If *space_around_delimiters* is true, delimiters between keys and values are surrounded by spaces.

<div class="versionchanged">

3.14 Raises InvalidWriteError if this would write a representation which cannot be accurately parsed by a future `read` call from this parser.

</div>

</div>

> [!NOTE]
> Comments in the original configuration file are not preserved when writing the configuration back. What is considered a comment, depends on the given values for *comment_prefix* and *inline_comment_prefix*.

<div class="method">

remove_option(section, option)

Remove the specified *option* from the specified *section*. If the section does not exist, raise `NoSectionError`. If the option existed to be removed, return `True`; otherwise return `False`.

</div>

<div class="method">

remove_section(section)

Remove the specified *section* from the configuration. If the section in fact existed, return `True`. Otherwise return `False`.

</div>

<div class="method">

optionxform(option)

Transforms the option name *option* as found in an input file or as passed in by client code to the form that should be used in the internal structures. The default implementation returns a lower-case version of *option*; subclasses may override this or client code can set an attribute of this name on instances to affect this behavior.

You don't need to subclass the parser to use this method, you can also set it on an instance, to a function that takes a string argument and returns a string. Setting it to `str`, for example, would make option names case sensitive:

    cfgparser = ConfigParser()
    cfgparser.optionxform = str

Note that when reading configuration files, whitespace around the option names is stripped before `optionxform` is called.

</div>

</div>

<div class="data">

UNNAMED_SECTION

A special object representing a section name used to reference the unnamed section (see `unnamed-sections`).

</div>

<div class="data">

MAX_INTERPOLATION_DEPTH

The maximum depth for recursive interpolation for `~configparser.ConfigParser.get` when the *raw* parameter is false. This is relevant only when the default *interpolation* is used.

</div>

## RawConfigParser Objects

<div class="RawConfigParser(defaults=None, dict_type=dict, \ allow_no_value=False, *, delimiters=('=', ':'), \ comment_prefixes=('#', ';'), \ inline_comment_prefixes=None, strict=True, \ empty_lines_in_values=True, \ default_section=configparser.DEFAULTSECT, \ interpolation=BasicInterpolation(), converters={}, \ allow_unnamed_section=False)">

Legacy variant of the `ConfigParser`. It has interpolation disabled by default and allows for non-string section names, option names, and values via its unsafe `add_section` and `set` methods, as well as the legacy `defaults=` keyword argument handling.

<div class="versionchanged">

3.2 *allow_no_value*, *delimiters*, *comment_prefixes*, *strict*, *empty_lines_in_values*, *default_section* and *interpolation* were added.

</div>

<div class="versionchanged">

3.5 The *converters* argument was added.

</div>

<div class="versionchanged">

3.8 The default *dict_type* is `dict`, since it now preserves insertion order.

</div>

<div class="versionchanged">

3.13 The *allow_unnamed_section* argument was added.

</div>

> [!NOTE]
> Consider using `ConfigParser` instead which checks types of the values to be stored internally. If you don't want interpolation, you can use `ConfigParser(interpolation=None)`.

<div class="method">

add_section(section)

Add a section named *section* or `UNNAMED_SECTION` to the instance.

If the given section already exists, `DuplicateSectionError` is raised. If the *default section* name is passed, `ValueError` is raised. If `UNNAMED_SECTION` is passed and support is disabled, `UnnamedSectionDisabledError` is raised.

Type of *section* is not checked which lets users create non-string named sections. This behaviour is unsupported and may cause internal errors.

</div>

<div class="versionchanged">

3.14 Added support for `UNNAMED_SECTION`.

</div>

<div class="method">

set(section, option, value)

If the given section exists, set the given option to the specified value; otherwise raise `NoSectionError`. While it is possible to use `RawConfigParser` (or `ConfigParser` with *raw* parameters set to true) for *internal* storage of non-string values, full functionality (including interpolation and output to files) can only be achieved using string values.

This method lets users assign non-string values to keys internally. This behaviour is unsupported and will cause errors when attempting to write to a file or get it in non-raw mode. **Use the mapping protocol API** which does not allow such assignments to take place.

</div>

</div>

## Exceptions

<div class="exception">

Error

Base class for all other `!configparser` exceptions.

</div>

<div class="exception">

NoSectionError

Exception raised when a specified section is not found.

</div>

<div class="exception">

DuplicateSectionError

Exception raised if `~ConfigParser.add_section` is called with the name of a section that is already present or in strict parsers when a section if found more than once in a single input file, string or dictionary.

<div class="versionchanged">

3.2 Added the optional *source* and *lineno* attributes and parameters to `!__init__`.

</div>

</div>

<div class="exception">

DuplicateOptionError

Exception raised by strict parsers if a single option appears twice during reading from a single file, string or dictionary. This catches misspellings and case sensitivity-related errors, e.g. a dictionary may have two keys representing the same case-insensitive configuration key.

</div>

<div class="exception">

NoOptionError

Exception raised when a specified option is not found in the specified section.

</div>

<div class="exception">

InterpolationError

Base class for exceptions raised when problems occur performing string interpolation.

</div>

<div class="exception">

InterpolationDepthError

Exception raised when string interpolation cannot be completed because the number of iterations exceeds `MAX_INTERPOLATION_DEPTH`. Subclass of `InterpolationError`.

</div>

<div class="exception">

InterpolationMissingOptionError

Exception raised when an option referenced from a value does not exist. Subclass of `InterpolationError`.

</div>

<div class="exception">

InterpolationSyntaxError

Exception raised when the source text into which substitutions are made does not conform to the required syntax. Subclass of `InterpolationError`.

</div>

<div class="exception">

MissingSectionHeaderError

Exception raised when attempting to parse a file which has no section headers.

</div>

<div class="exception">

ParsingError

Exception raised when errors occur attempting to parse a file.

<div class="versionchanged">

3.12 The `filename` attribute and `!__init__` constructor argument were removed. They have been available using the name `source` since 3.2.

</div>

</div>

<div class="exception">

MultilineContinuationError

Exception raised when a key without a corresponding value is continued with an indented line.

<div class="versionadded">

3.13

</div>

</div>

<div class="exception">

UnnamedSectionDisabledError

Exception raised when attempting to use the `UNNAMED_SECTION` without enabling it.

> <div class="versionadded">
>
> 3.14
>
> </div>

</div>

<div class="exception">

InvalidWriteError

Exception raised when an attempted `ConfigParser.write` would not be parsed accurately with a future `ConfigParser.read` call.

Ex: Writing a key beginning with the `ConfigParser.SECTCRE` pattern would parse as a section header when read. Attempting to write this will raise this exception.

<div class="versionadded">

3.14

</div>

</div>

**Footnotes**

[^1]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^2]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^3]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^4]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^5]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^6]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^7]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^8]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^9]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^10]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

[^11]: Config parsers allow for heavy customization. If you are interested in changing the behaviour outlined by the footnote reference, consult the [Customizing Parser Behaviour](<##REF##Customizing Parser Behaviour>) section.

# `!contextlib` --- Utilities for `!with`-statement contexts

<div class="module" data-synopsis="Utilities for with-statement contexts.">

contextlib

</div>

**Source code:** `Lib/contextlib.py`

------------------------------------------------------------------------

This module provides utilities for common tasks involving the `with` statement. For more information see also `typecontextmanager` and `context-managers`.

## Utilities

Functions and classes provided:

<div class="AbstractContextManager">

An `abstract base class` for classes that implement `~object.__enter__` and `~object.__exit__`. A default implementation for `~object.__enter__` is provided which returns `self` while `~object.__exit__` is an abstract method which by default returns `None`. See also the definition of `typecontextmanager`.

<div class="versionadded">

3.6

</div>

</div>

<div class="AbstractAsyncContextManager">

An `abstract base class` for classes that implement `~object.__aenter__` and `~object.__aexit__`. A default implementation for `~object.__aenter__` is provided which returns `self` while `~object.__aexit__` is an abstract method which by default returns `None`. See also the definition of `async-context-managers`.

<div class="versionadded">

3.7

</div>

</div>

<div class="decorator">

contextmanager

This function is a `decorator` that can be used to define a factory function for `with` statement context managers, without needing to create a class or separate `~object.__enter__` and `~object.__exit__` methods.

While many objects natively support use in with statements, sometimes a resource needs to be managed that isn't a context manager in its own right, and doesn't implement a `close()` method for use with `contextlib.closing`.

An abstract example would be the following to ensure correct resource management:

    from contextlib import contextmanager

    @contextmanager
    def managed_resource(*args, **kwds):
        # Code to acquire resource, e.g.:
        resource = acquire_resource(*args, **kwds)
        try:
            yield resource
        finally:
            # Code to release resource, e.g.:
            release_resource(resource)

The function can then be used like this:

    >>> with managed_resource(timeout=3600) as resource:
    ...     # Resource is released at the end of this block,
    ...     # even if code in the block raises an exception

The function being decorated must return a `generator`-iterator when called. This iterator must yield exactly one value, which will be bound to the targets in the `with` statement's `!as` clause, if any.

At the point where the generator yields, the block nested in the `with` statement is executed. The generator is then resumed after the block is exited. If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred. Thus, you can use a `try`...`except`...`finally` statement to trap the error (if any), or ensure that some cleanup takes place. If an exception is trapped merely in order to log it or to perform some action (rather than to suppress it entirely), the generator must reraise that exception. Otherwise the generator context manager will indicate to the `!with` statement that the exception has been handled, and execution will resume with the statement immediately following the `!with` statement.

`contextmanager` uses `ContextDecorator` so the context managers it creates can be used as decorators as well as in `with` statements. When used as a decorator, a new generator instance is implicitly created on each function call (this allows the otherwise "one-shot" context managers created by `contextmanager` to meet the requirement that context managers support multiple invocations in order to be used as decorators).

<div class="versionchanged">

3.2 Use of `ContextDecorator`.

</div>

</div>

<div class="decorator">

asynccontextmanager

Similar to `~contextlib.contextmanager`, but creates an `asynchronous context manager <async-context-managers>`.

This function is a `decorator` that can be used to define a factory function for `async with` statement asynchronous context managers, without needing to create a class or separate `~object.__aenter__` and `~object.__aexit__` methods. It must be applied to an `asynchronous
generator` function.

A simple example:

    from contextlib import asynccontextmanager

    @asynccontextmanager
    async def get_connection():
        conn = await acquire_db_connection()
        try:
            yield conn
        finally:
            await release_db_connection(conn)

    async def get_all_users():
        async with get_connection() as conn:
            return conn.query('SELECT ...')

<div class="versionadded">

3.7

</div>

Context managers defined with `asynccontextmanager` can be used either as decorators or with `async with` statements:

    import time
    from contextlib import asynccontextmanager

    @asynccontextmanager
    async def timeit():
        now = time.monotonic()
        try:
            yield
        finally:
            print(f'it took {time.monotonic() - now}s to run')

    @timeit()
    async def main():
        # ... async code ...

When used as a decorator, a new generator instance is implicitly created on each function call. This allows the otherwise "one-shot" context managers created by `asynccontextmanager` to meet the requirement that context managers support multiple invocations in order to be used as decorators.

<div class="versionchanged">

3.10 Async context managers created with `asynccontextmanager` can be used as decorators.

</div>

</div>

<div class="function">

closing(thing)

Return a context manager that closes *thing* upon completion of the block. This is basically equivalent to:

    from contextlib import contextmanager

    @contextmanager
    def closing(thing):
        try:
            yield thing
        finally:
            thing.close()

And lets you write code like this:

    from contextlib import closing
    from urllib.request import urlopen

    with closing(urlopen('https://www.python.org')) as page:
        for line in page:
            print(line)

without needing to explicitly close `page`. Even if an error occurs, `page.close()` will be called when the `with` block is exited.

> [!NOTE]
> Most types managing resources support the `context manager` protocol, which closes *thing* on leaving the `with` statement. As such, `!closing` is most useful for third party types that don't support context managers. This example is purely for illustration purposes, as `~urllib.request.urlopen` would normally be used in a context manager.

</div>

<div class="function">

aclosing(thing)

Return an async context manager that calls the `aclose()` method of *thing* upon completion of the block. This is basically equivalent to:

    from contextlib import asynccontextmanager

    @asynccontextmanager
    async def aclosing(thing):
        try:
            yield thing
        finally:
            await thing.aclose()

Significantly, `aclosing()` supports deterministic cleanup of async generators when they happen to exit early by `break` or an exception. For example:

    from contextlib import aclosing

    async with aclosing(my_generator()) as values:
        async for value in values:
            if value == 42:
                break

This pattern ensures that the generator's async exit code is executed in the same context as its iterations (so that exceptions and context variables work as expected, and the exit code isn't run after the lifetime of some task it depends on).

<div class="versionadded">

3.10

</div>

</div>

<div id="simplifying-support-for-single-optional-context-managers">

<div class="function">

nullcontext(enter_result=None)

Return a context manager that returns *enter_result* from `~object.__enter__`, but otherwise does nothing. It is intended to be used as a stand-in for an optional context manager, for example:

    def myfunction(arg, ignore_exceptions=False):
        if ignore_exceptions:
            # Use suppress to ignore all exceptions.
            cm = contextlib.suppress(Exception)
        else:
            # Do not ignore any exceptions, cm has no effect.
            cm = contextlib.nullcontext()
        with cm:
            # Do something

An example using *enter_result*:

    def process_file(file_or_path):
        if isinstance(file_or_path, str):
            # If string, open file
            cm = open(file_or_path)
        else:
            # Caller is responsible for closing file
            cm = nullcontext(file_or_path)

        with cm as file:
            # Perform processing on the file

It can also be used as a stand-in for `asynchronous context managers <async-context-managers>`:

    async def send_http(session=None):
        if not session:
            # If no http session, create it with aiohttp
            cm = aiohttp.ClientSession()
        else:
            # Caller is responsible for closing the session
            cm = nullcontext(session)

        async with cm as session:
            # Send http requests with session

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.10 `asynchronous context manager` support was added.

</div>

</div>

</div>

<div class="function">

suppress(\*exceptions)

Return a context manager that suppresses any of the specified exceptions if they occur in the body of a `!with` statement and then resumes execution with the first statement following the end of the `!with` statement.

As with any other mechanism that completely suppresses exceptions, this context manager should be used only to cover very specific errors where silently continuing with program execution is known to be the right thing to do.

For example:

    from contextlib import suppress

    with suppress(FileNotFoundError):
        os.remove('somefile.tmp')

    with suppress(FileNotFoundError):
        os.remove('someotherfile.tmp')

This code is equivalent to:

    try:
        os.remove('somefile.tmp')
    except FileNotFoundError:
        pass

    try:
        os.remove('someotherfile.tmp')
    except FileNotFoundError:
        pass

This context manager is `reentrant <reentrant-cms>`.

If the code within the `!with` block raises a `BaseExceptionGroup`, suppressed exceptions are removed from the group. Any exceptions of the group which are not suppressed are re-raised in a new group which is created using the original group's `~BaseExceptionGroup.derive` method.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.12 `suppress` now supports suppressing exceptions raised as part of a `BaseExceptionGroup`.

</div>

</div>

<div class="function">

redirect_stdout(new_target)

Context manager for temporarily redirecting `sys.stdout` to another `file object`.

This tool adds flexibility to existing functions or classes whose output is hardwired to `sys.stdout`.

For example, the output of `help` normally is sent to *sys.stdout*. You can capture that output in a string by redirecting the output to an `io.StringIO` object. The replacement stream is returned from the `~object.__enter__` method and so is available as the target of the `with` statement:

    with redirect_stdout(io.StringIO()) as f:
        help(pow)
    s = f.getvalue()

To send the output of `help` to a file on disk, redirect the output to a regular file:

    with open('help.txt', 'w') as f:
        with redirect_stdout(f):
            help(pow)

To send the output of `help` to *sys.stderr*:

    with redirect_stdout(sys.stderr):
        help(pow)

Note that the global side effect on `sys.stdout` means that this context manager is not suitable for use in library code and most threaded applications. It also has no effect on the output of subprocesses. However, it is still a useful approach for many utility scripts.

This context manager is `reentrant <reentrant-cms>`.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

redirect_stderr(new_target)

Similar to `~contextlib.redirect_stdout` but redirecting the global `sys.stderr` to another `file object`.

This context manager is `reentrant <reentrant-cms>`.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

chdir(path)

Non parallel-safe context manager to change the current working directory. As this changes a global state, the working directory, it is not suitable for use in most threaded or async contexts. It is also not suitable for most non-linear code execution, like generators, where the program execution is temporarily relinquished -- unless explicitly desired, you should not yield when this context manager is active.

This is a simple wrapper around `~os.chdir`, it changes the current working directory upon entering and restores the old one on exit.

This context manager is `reentrant <reentrant-cms>`.

<div class="versionadded">

3.11

</div>

</div>

<div class="ContextDecorator()">

A base class that enables a context manager to also be used as a decorator.

Context managers inheriting from `ContextDecorator` have to implement `~object.__enter__` and `~object.__exit__` as normal. `__exit__` retains its optional exception handling even when used as a decorator.

`ContextDecorator` is used by `contextmanager`, so you get this functionality automatically.

Example of `ContextDecorator`:

    from contextlib import ContextDecorator

    class mycontext(ContextDecorator):
        def __enter__(self):
            print('Starting')
            return self

        def __exit__(self, *exc):
            print('Finishing')
            return False

The class can then be used like this:

    >>> @mycontext()
    ... def function():
    ...     print('The bit in the middle')
    ...
    >>> function()
    Starting
    The bit in the middle
    Finishing

    >>> with mycontext():
    ...     print('The bit in the middle')
    ...
    Starting
    The bit in the middle
    Finishing

This change is just syntactic sugar for any construct of the following form:

    def f():
        with cm():
            # Do stuff

`ContextDecorator` lets you instead write:

    @cm()
    def f():
        # Do stuff

It makes it clear that the `cm` applies to the whole function, rather than just a piece of it (and saving an indentation level is nice, too).

Existing context managers that already have a base class can be extended by using `ContextDecorator` as a mixin class:

    from contextlib import ContextDecorator

    class mycontext(ContextBaseClass, ContextDecorator):
        def __enter__(self):
            return self

        def __exit__(self, *exc):
            return False

> [!NOTE]
> As the decorated function must be able to be called multiple times, the underlying context manager must support use in multiple `with` statements. If this is not the case, then the original construct with the explicit `!with` statement inside the function should be used.

When the decorated callable is a generator function, coroutine function, or asynchronous generator function, the returned wrapper is of the same kind and keeps the context manager open for the lifetime of the iteration or await rather than only for the call that creates the generator or coroutine object. Wrapped generators and asynchronous generators are explicitly closed when iteration ends, as if by `closing` or `aclosing`.

> [!NOTE]
> For asynchronous generators the wrapper re-yields each value with `async for`; values sent with `~agen.asend` and exceptions thrown with `~agen.athrow` are not forwarded to the wrapped generator.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.15 Decorating a generator function, coroutine function, or asynchronous generator function now keeps the context manager open across iteration or await. Previously the context manager exited as soon as the generator or coroutine object was created.

</div>

</div>

<div class="AsyncContextDecorator">

Similar to `ContextDecorator`, but the context manager is entered and exited with `async with`. Decorate coroutine functions and asynchronous generator functions with this class; the returned wrapper is of the same kind.

> [!NOTE]
> Synchronous functions and generators are accepted, but the wrapper is always asynchronous, so the decorated callable must then be awaited or iterated with `async for`. If that change of calling convention is not intended, use `ContextDecorator` instead.

Example of `AsyncContextDecorator`:

    from asyncio import run
    from contextlib import AsyncContextDecorator

    class mycontext(AsyncContextDecorator):
        async def __aenter__(self):
            print('Starting')
            return self

        async def __aexit__(self, *exc):
            print('Finishing')
            return False

The class can then be used like this:

    >>> @mycontext()
    ... async def function():
    ...     print('The bit in the middle')
    ...
    >>> run(function())
    Starting
    The bit in the middle
    Finishing

    >>> async def function():
    ...    async with mycontext():
    ...         print('The bit in the middle')
    ...
    >>> run(function())
    Starting
    The bit in the middle
    Finishing

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.15 Decorating an asynchronous generator function now keeps the context manager open across iteration. Previously the context manager exited as soon as the generator object was created. Synchronous functions and synchronous generator functions are also now accepted, with an asynchronous wrapper returned.

</div>

</div>

<div class="ExitStack()">

A context manager that is designed to make it easy to programmatically combine other context managers and cleanup functions, especially those that are optional or otherwise driven by input data.

For example, a set of files may easily be handled in a single with statement as follows:

    with ExitStack() as stack:
        files = [stack.enter_context(open(fname)) for fname in filenames]
        # All opened files will automatically be closed at the end of
        # the with statement, even if attempts to open files later
        # in the list raise an exception

The `~object.__enter__` method returns the `ExitStack` instance, and performs no additional operations.

Each instance maintains a stack of registered callbacks that are called in reverse order when the instance is closed (either explicitly or implicitly at the end of a `with` statement). Note that callbacks are *not* invoked implicitly when the context stack instance is garbage collected.

This stack model is used so that context managers that acquire their resources in their `__init__` method (such as file objects) can be handled correctly.

Since registered callbacks are invoked in the reverse order of registration, this ends up behaving as if multiple nested `with` statements had been used with the registered set of callbacks. This even extends to exception handling - if an inner callback suppresses or replaces an exception, then outer callbacks will be passed arguments based on that updated state.

This is a relatively low level API that takes care of the details of correctly unwinding the stack of exit callbacks. It provides a suitable foundation for higher level context managers that manipulate the exit stack in application specific ways.

<div class="versionadded">

3.3

</div>

<div class="method">

enter_context(cm)

Enters a new context manager and adds its `~object.__exit__` method to the callback stack. The return value is the result of the context manager's own `~object.__enter__` method.

These context managers may suppress exceptions just as they normally would if used directly as part of a `with` statement.

<div class="versionchanged">

3.11 Raises `TypeError` instead of `AttributeError` if *cm* is not a context manager.

</div>

<div class="versionchanged">

3.15 Added support for arbitrary descriptors `!__enter__` and `!__exit__`.

</div>

</div>

<div class="method">

push(exit)

Adds a context manager's `~object.__exit__` method to the callback stack.

As `__enter__` is *not* invoked, this method can be used to cover part of an `~object.__enter__` implementation with a context manager's own `~object.__exit__` method.

If passed an object that is not a context manager, this method assumes it is a callback with the same signature as a context manager's `~object.__exit__` method and adds it directly to the callback stack.

By returning true values, these callbacks can suppress exceptions the same way context manager `~object.__exit__` methods can.

The passed in object is returned from the function, allowing this method to be used as a function decorator.

<div class="versionchanged">

3.15 Added support for arbitrary descriptors `!__exit__`.

</div>

</div>

<div class="method">

callback(callback, /, *args,*\*kwds)

Accepts an arbitrary callback function and arguments and adds it to the callback stack.

Unlike the other methods, callbacks added this way cannot suppress exceptions (as they are never passed the exception details).

The passed in callback is returned from the function, allowing this method to be used as a function decorator.

</div>

<div class="method">

pop_all()

Transfers the callback stack to a fresh `ExitStack` instance and returns it. No callbacks are invoked by this operation - instead, they will now be invoked when the new stack is closed (either explicitly or implicitly at the end of a `with` statement).

For example, a group of files can be opened as an "all or nothing" operation as follows:

    with ExitStack() as stack:
        files = [stack.enter_context(open(fname)) for fname in filenames]
        # Hold onto the close method, but don't call it yet.
        close_files = stack.pop_all().close
        # If opening any file fails, all previously opened files will be
        # closed automatically. If all files are opened successfully,
        # they will remain open even after the with statement ends.
        # close_files() can then be invoked explicitly to close them all.

</div>

<div class="method">

close()

Immediately unwinds the callback stack, invoking callbacks in the reverse order of registration. For any context managers and exit callbacks registered, the arguments passed in will indicate that no exception occurred.

</div>

</div>

<div class="AsyncExitStack()">

An `asynchronous context manager <async-context-managers>`, similar to `ExitStack`, that supports combining both synchronous and asynchronous context managers, as well as having coroutines for cleanup logic.

The `~ExitStack.close` method is not implemented; `aclose` must be used instead.

<div class="method" async="">

enter_async_context(cm)

Similar to `ExitStack.enter_context` but expects an asynchronous context manager.

<div class="versionchanged">

3.11 Raises `TypeError` instead of `AttributeError` if *cm* is not an asynchronous context manager.

</div>

<div class="versionchanged">

3.15 Added support for arbitrary descriptors `!__aenter__` and `!__aexit__`.

</div>

</div>

<div class="method">

push_async_exit(exit)

Similar to `ExitStack.push` but expects either an asynchronous context manager or a coroutine function.

<div class="versionchanged">

3.15 Added support for arbitrary descriptors `!__aexit__`.

</div>

</div>

<div class="method">

push_async_callback(callback, /, *args,*\*kwds)

Similar to `ExitStack.callback` but expects a coroutine function.

</div>

<div class="method" async="">

aclose()

Similar to `ExitStack.close` but properly handles awaitables.

</div>

Continuing the example for `asynccontextmanager`:

    async with AsyncExitStack() as stack:
        connections = [await stack.enter_async_context(get_connection())
            for i in range(5)]
        # All opened connections will automatically be released at the end of
        # the async with statement, even if attempts to open a connection
        # later in the list raise an exception.

<div class="versionadded">

3.7

</div>

</div>

## Examples and Recipes

This section describes some examples and recipes for making effective use of the tools provided by `!contextlib`.

### Supporting a variable number of context managers

The primary use case for `ExitStack` is the one given in the class documentation: supporting a variable number of context managers and other cleanup operations in a single `with` statement. The variability may come from the number of context managers needed being driven by user input (such as opening a user specified collection of files), or from some of the context managers being optional:

    with ExitStack() as stack:
        for resource in resources:
            stack.enter_context(resource)
        if need_special_resource():
            special = acquire_special_resource()
            stack.callback(release_special_resource, special)
        # Perform operations that use the acquired resources

As shown, `ExitStack` also makes it quite easy to use `with` statements to manage arbitrary resources that don't natively support the context management protocol.

### Catching exceptions from `__enter__` methods

It is occasionally desirable to catch exceptions from an `~object.__enter__` method implementation, *without* inadvertently catching exceptions from the `with` statement body or the context manager's `~object.__exit__` method. By using `ExitStack` the steps in the context management protocol can be separated slightly in order to allow this:

    stack = ExitStack()
    try:
        x = stack.enter_context(cm)
    except Exception:
        # handle __enter__ exception
    else:
        with stack:
            # Handle normal case

Actually needing to do this is likely to indicate that the underlying API should be providing a direct resource management interface for use with `try`/`except`/`finally` statements, but not all APIs are well designed in that regard. When a context manager is the only resource management API provided, then `ExitStack` can make it easier to handle various situations that can't be handled directly in a `with` statement.

### Cleaning up in an `__enter__` implementation

As noted in the documentation of `ExitStack.push`, this method can be useful in cleaning up an already allocated resource if later steps in the `~object.__enter__` implementation fail.

Here's an example of doing this for a context manager that accepts resource acquisition and release functions, along with an optional validation function, and maps them to the context management protocol:

    from contextlib import contextmanager, AbstractContextManager, ExitStack

    class ResourceManager(AbstractContextManager):

        def __init__(self, acquire_resource, release_resource, check_resource_ok=None):
            self.acquire_resource = acquire_resource
            self.release_resource = release_resource
            if check_resource_ok is None:
                def check_resource_ok(resource):
                    return True
            self.check_resource_ok = check_resource_ok

        @contextmanager
        def _cleanup_on_error(self):
            with ExitStack() as stack:
                stack.push(self)
                yield
                # The validation check passed and didn't raise an exception
                # Accordingly, we want to keep the resource, and pass it
                # back to our caller
                stack.pop_all()

        def __enter__(self):
            resource = self.acquire_resource()
            with self._cleanup_on_error():
                if not self.check_resource_ok(resource):
                    msg = "Failed validation for {!r}"
                    raise RuntimeError(msg.format(resource))
            return resource

        def __exit__(self, *exc_details):
            # We don't need to duplicate any of our resource release logic
            self.release_resource()

### Replacing any use of `try-finally` and flag variables

A pattern you will sometimes see is a `try-finally` statement with a flag variable to indicate whether or not the body of the `finally` clause should be executed. In its simplest form (that can't already be handled just by using an `except` clause instead), it looks something like this:

    cleanup_needed = True
    try:
        result = perform_operation()
        if result:
            cleanup_needed = False
    finally:
        if cleanup_needed:
            cleanup_resources()

As with any `try` statement based code, this can cause problems for development and review, because the setup code and the cleanup code can end up being separated by arbitrarily long sections of code.

`ExitStack` makes it possible to instead register a callback for execution at the end of a `with` statement, and then later decide to skip executing that callback:

    from contextlib import ExitStack

    with ExitStack() as stack:
        stack.callback(cleanup_resources)
        result = perform_operation()
        if result:
            stack.pop_all()

This allows the intended cleanup behaviour to be made explicit up front, rather than requiring a separate flag variable.

If a particular application uses this pattern a lot, it can be simplified even further by means of a small helper class:

    from contextlib import ExitStack

    class Callback(ExitStack):
        def __init__(self, callback, /, *args, **kwds):
            super().__init__()
            self.callback(callback, *args, **kwds)

        def cancel(self):
            self.pop_all()

    with Callback(cleanup_resources) as cb:
        result = perform_operation()
        if result:
            cb.cancel()

If the resource cleanup isn't already neatly bundled into a standalone function, then it is still possible to use the decorator form of `ExitStack.callback` to declare the resource cleanup in advance:

    from contextlib import ExitStack

    with ExitStack() as stack:
        @stack.callback
        def cleanup_resources():
            ...
        result = perform_operation()
        if result:
            stack.pop_all()

Due to the way the decorator protocol works, a callback function declared this way cannot take any parameters. Instead, any resources to be released must be accessed as closure variables.

### Using a context manager as a function decorator

`ContextDecorator` makes it possible to use a context manager in both an ordinary `with` statement and also as a function decorator.

For example, it is sometimes useful to wrap functions or groups of statements with a logger that can track the time of entry and time of exit. Rather than writing both a function decorator and a context manager for the task, inheriting from `ContextDecorator` provides both capabilities in a single definition:

    from contextlib import ContextDecorator
    import logging

    logging.basicConfig(level=logging.INFO)

    class track_entry_and_exit(ContextDecorator):
        def __init__(self, name):
            self.name = name

        def __enter__(self):
            logging.info('Entering: %s', self.name)

        def __exit__(self, exc_type, exc, exc_tb):
            logging.info('Exiting: %s', self.name)

Instances of this class can be used as both a context manager:

    with track_entry_and_exit('widget loader'):
        print('Some time consuming activity goes here')
        load_widget()

And also as a function decorator:

    @track_entry_and_exit('widget loader')
    def activity():
        print('Some time consuming activity goes here')
        load_widget()

Note that there is one additional limitation when using context managers as function decorators: there's no way to access the return value of `~object.__enter__`. If that value is needed, then it is still necessary to use an explicit `with` statement.

<div class="seealso">

`343` - The "with" statement  
The specification, background, and examples for the Python `with` statement.

</div>

## Single use, reusable and reentrant context managers

Most context managers are written in a way that means they can only be used effectively in a `with` statement once. These single use context managers must be created afresh each time they're used - attempting to use them a second time will trigger an exception or otherwise not work correctly.

This common limitation means that it is generally advisable to create context managers directly in the header of the `with` statement where they are used (as shown in all of the usage examples above).

Files are an example of effectively single use context managers, since the first `with` statement will close the file, preventing any further IO operations using that file object.

Context managers created using `contextmanager` are also single use context managers, and will complain about the underlying generator failing to yield if an attempt is made to use them a second time:

    >>> from contextlib import contextmanager
    >>> @contextmanager
    ... def singleuse():
    ...     print("Before")
    ...     yield
    ...     print("After")
    ...
    >>> cm = singleuse()
    >>> with cm:
    ...     pass
    ...
    Before
    After
    >>> with cm:
    ...     pass
    ...
    Traceback (most recent call last):
        ...
    RuntimeError: generator didn't yield

### Reentrant context managers

More sophisticated context managers may be "reentrant". These context managers can not only be used in multiple `with` statements, but may also be used *inside* a `!with` statement that is already using the same context manager.

`threading.RLock` is an example of a reentrant context manager, as are `suppress`, `redirect_stdout`, and `chdir`. Here's a very simple example of reentrant use:

    >>> from contextlib import redirect_stdout
    >>> from io import StringIO
    >>> stream = StringIO()
    >>> write_to_stream = redirect_stdout(stream)
    >>> with write_to_stream:
    ...     print("This is written to the stream rather than stdout")
    ...     with write_to_stream:
    ...         print("This is also written to the stream")
    ...
    >>> print("This is written directly to stdout")
    This is written directly to stdout
    >>> print(stream.getvalue())
    This is written to the stream rather than stdout
    This is also written to the stream

Real world examples of reentrancy are more likely to involve multiple functions calling each other and hence be far more complicated than this example.

Note also that being reentrant is *not* the same thing as being thread safe. `redirect_stdout`, for example, is definitely not thread safe, as it makes a global modification to the system state by binding `sys.stdout` to a different stream.

### Reusable context managers

Distinct from both single use and reentrant context managers are "reusable" context managers (or, to be completely explicit, "reusable, but not reentrant" context managers, since reentrant context managers are also reusable). These context managers support being used multiple times, but will fail (or otherwise not work correctly) if the specific context manager instance has already been used in a containing with statement.

`threading.Lock` is an example of a reusable, but not reentrant, context manager (for a reentrant lock, it is necessary to use `threading.RLock` instead).

Another example of a reusable, but not reentrant, context manager is `ExitStack`, as it invokes *all* currently registered callbacks when leaving any with statement, regardless of where those callbacks were added:

    >>> from contextlib import ExitStack
    >>> stack = ExitStack()
    >>> with stack:
    ...     stack.callback(print, "Callback: from first context")
    ...     print("Leaving first context")
    ...
    Leaving first context
    Callback: from first context
    >>> with stack:
    ...     stack.callback(print, "Callback: from second context")
    ...     print("Leaving second context")
    ...
    Leaving second context
    Callback: from second context
    >>> with stack:
    ...     stack.callback(print, "Callback: from outer context")
    ...     with stack:
    ...         stack.callback(print, "Callback: from inner context")
    ...         print("Leaving inner context")
    ...     print("Leaving outer context")
    ...
    Leaving inner context
    Callback: from inner context
    Callback: from outer context
    Leaving outer context

As the output from the example shows, reusing a single stack object across multiple with statements works correctly, but attempting to nest them will cause the stack to be cleared at the end of the innermost with statement, which is unlikely to be desirable behaviour.

Using separate `ExitStack` instances instead of reusing a single instance avoids that problem:

    >>> from contextlib import ExitStack
    >>> with ExitStack() as outer_stack:
    ...     outer_stack.callback(print, "Callback: from outer context")
    ...     with ExitStack() as inner_stack:
    ...         inner_stack.callback(print, "Callback: from inner context")
    ...         print("Leaving inner context")
    ...     print("Leaving outer context")
    ...
    Leaving inner context
    Callback: from inner context
    Leaving outer context
    Callback: from outer context

# `!contextvars` --- Context Variables

<div class="module" data-synopsis="Context Variables">

contextvars

</div>

------------------------------------------------------------------------

This module provides APIs to manage, store, and access context-local state. The `~contextvars.ContextVar` class is used to declare and work with *Context Variables*. The `~contextvars.copy_context` function and the `~contextvars.Context` class should be used to manage the current context in asynchronous frameworks.

Context managers that have state should use Context Variables instead of `threading.local` to prevent their state from bleeding to other code unexpectedly, when used in concurrent code.

See also `567` for additional details.

<div class="versionadded">

3.7

</div>

## Context Variables

<div class="ContextVar(name, [*, default])">

This class is used to declare a new Context Variable, e.g.:

    var: ContextVar[int] = ContextVar('var', default=42)

The required *name* parameter is used for introspection and debug purposes.

The optional keyword-only *default* parameter is returned by `ContextVar.get` when no value for the variable is found in the current context.

**Important:** Context Variables should be created at the top module level and never in closures. `Context` objects hold strong references to context variables which prevents context variables from being properly garbage collected.

`!ContextVar`s are `generic <generics>` over the type of their contained value.

<div class="attribute">

ContextVar.name

The name of the variable. This is a read-only property.

<div class="versionadded">

3.7.1

</div>

</div>

<div class="method">

get(\[default\])

Return a value for the context variable for the current context.

If there is no value for the variable in the current context, the method will:

- return the value of the *default* argument of the method, if provided; or
- return the default value for the context variable, if it was created with one; or
- raise a `LookupError`.

</div>

<div class="method">

set(value)

Call to set a new value for the context variable in the current context.

The required *value* argument is the new value for the context variable.

Returns a `~contextvars.Token` object that can be used to restore the variable to its previous value via the `ContextVar.reset` method.

For convenience, the token object can be used as a context manager to avoid calling `ContextVar.reset` manually:

    var = ContextVar('var', default='default value')

    with var.set('new value'):
        assert var.get() == 'new value'

    assert var.get() == 'default value'

It is a shorthand for:

    var = ContextVar('var', default='default value')

    token = var.set('new value')
    try:
        assert var.get() == 'new value'
    finally:
        var.reset(token)

    assert var.get() == 'default value'

<div class="versionadded">

3.14

Added support for using tokens as context managers.

</div>

</div>

<div class="method">

reset(token)

Reset the context variable to the value it had before the `ContextVar.set` that created the *token* was used.

For example:

    var = ContextVar('var')

    token = var.set('new value')
    # code that uses 'var'; var.get() returns 'new value'.
    var.reset(token)

    # After the reset call the var has no value again, so
    # var.get() would raise a LookupError.

The same *token* cannot be used twice.

</div>

</div>

<div class="Token">

*Token* objects are returned by the `ContextVar.set` method. They can be passed to the `ContextVar.reset` method to revert the value of the variable to what it was before the corresponding *set*. A single token cannot reset a context variable more than once.

Tokens support the `context manager protocol <context-managers>` to automatically reset context variables. See `ContextVar.set`.

Tokens are `generic <generics>` over the same type as the `ContextVar` which created them.

<div class="versionadded">

3.14

Added support for usage as a context manager.

</div>

<div class="attribute">

Token.var

A read-only property. Points to the `ContextVar` object that created the token.

</div>

<div class="attribute">

Token.old_value

A read-only property. Set to the value the variable had before the `ContextVar.set` method call that created the token. It points to `Token.MISSING` if the variable was not set before the call.

</div>

<div class="attribute">

Token.MISSING

A marker object used by `Token.old_value`.

</div>

</div>

## Manual Context Management

<div class="function">

copy_context()

Returns a copy of the current `~contextvars.Context` object.

The following snippet gets a copy of the current context and prints all variables and their values that are set in it:

    ctx: Context = copy_context()
    print(list(ctx.items()))

The function has an *O*(1) complexity, i.e. works equally fast for contexts with a few context variables and for contexts that have a lot of them.

</div>

<div class="Context()">

A mapping of `ContextVars <ContextVar>` to their values.

`Context()` creates an empty context with no values in it. To get a copy of the current context use the `~contextvars.copy_context` function.

Each thread has its own effective stack of `!Context` objects. The `current context` is the `!Context` object at the top of the current thread's stack. All `!Context` objects in the stacks are considered to be *entered*.

*Entering* a context, which can be done by calling its `~Context.run` method, makes the context the current context by pushing it onto the top of the current thread's context stack.

*Exiting* from the current context, which can be done by returning from the callback passed to the `~Context.run` method, restores the current context to what it was before the context was entered by popping the context off the top of the context stack.

Since each thread has its own context stack, `ContextVar` objects behave in a similar fashion to `threading.local` when values are assigned in different threads.

Attempting to enter an already entered context, including contexts entered in other threads, raises a `RuntimeError`.

After exiting a context, it can later be re-entered (from any thread).

Any changes to `ContextVar` values via the `ContextVar.set` method are recorded in the current context. The `ContextVar.get` method returns the value associated with the current context. Exiting a context effectively reverts any changes made to context variables while the context was entered (if needed, the values can be restored by re-entering the context).

Context implements the `collections.abc.Mapping` interface.

<div class="method">

run(callable, *args,*\*kwargs)

Enters the Context, executes `callable(*args, **kwargs)`, then exits the Context. Returns *callable*'s return value, or propagates an exception if one occurred.

Example:

<div class="testcode">

import contextvars

var = contextvars.ContextVar('var') var.set('spam') print(var.get()) \# 'spam'

ctx = contextvars.copy_context()

def main():  
\# 'var' was set to 'spam' before \# calling 'copy_context()' and 'ctx.run(main)', so: print(var.get()) \# 'spam' print(ctx\[var\]) \# 'spam'

var.set('ham')

\# Now, after setting 'var' to 'ham': print(var.get()) \# 'ham' print(ctx\[var\]) \# 'ham'

\# Any changes that the 'main' function makes to 'var' \# will be contained in 'ctx'. ctx.run(main)

\# The 'main()' function was run in the 'ctx' context, \# so changes to 'var' are contained in it: print(ctx\[var\]) \# 'ham'

\# However, outside of 'ctx', 'var' is still set to 'spam': print(var.get()) \# 'spam'

</div>

<div class="testoutput" data-hide="">

spam spam spam ham ham ham spam

</div>

</div>

<div class="method">

copy()

Return a shallow copy of the context object.

</div>

<div class="describe">

var in context

Return `True` if the *context* has a value for *var* set; return `False` otherwise.

</div>

<div class="describe">

context\[var\]

Return the value of the *var* `ContextVar` variable. If the variable is not set in the context object, a `KeyError` is raised.

</div>

<div class="method">

get(var, \[default\])

Return the value for *var* if *var* has the value in the context object. Return *default* otherwise. If *default* is not given, return `None`.

</div>

<div class="describe">

iter(context)

Return an iterator over the variables stored in the context object.

</div>

<div class="describe">

len(proxy)

Return the number of variables set in the context object.

</div>

<div class="method">

keys()

Return a list of all variables in the context object.

</div>

<div class="method">

values()

Return a list of all variables' values in the context object.

</div>

<div class="method">

items()

Return a list of 2-tuples containing all variables and their values in the context object.

</div>

</div>

## asyncio support

Context variables are natively supported in `asyncio` and are ready to be used without any extra configuration. For example, here is a simple echo server, that uses a context variable to make the address of a remote client available in the Task that handles that client:

    import asyncio
    import contextvars

    client_addr_var = contextvars.ContextVar('client_addr')

    def render_goodbye():
        # The address of the currently handled client can be accessed
        # without passing it explicitly to this function.

        client_addr = client_addr_var.get()
        return f'Good bye, client @ {client_addr}\r\n'.encode()

    async def handle_request(reader, writer):
        addr = writer.transport.get_extra_info('socket').getpeername()
        client_addr_var.set(addr)

        # In any code that we call, it is now possible to get the
        # client's address by calling 'client_addr_var.get()'.

        while True:
            line = await reader.readline()
            print(line)
            if not line.strip():
                break

        writer.write(b'HTTP/1.1 200 OK\r\n')  # status line
        writer.write(b'\r\n')  # headers
        writer.write(render_goodbye())  # body
        writer.close()

    async def main():
        srv = await asyncio.start_server(
            handle_request, '127.0.0.1', 8081)

        async with srv:
            await srv.serve_forever()

    asyncio.run(main())

    # To test it you can use telnet or curl:
    #     telnet 127.0.0.1 8081
    #     curl 127.0.0.1:8081

# `!copy` --- Shallow and deep copy operations

<div class="module" data-synopsis="Shallow and deep copy operations.">

copy

</div>

**Source code:** `Lib/copy.py`

------------------------------------------------------------------------

Assignment statements in Python do not copy objects, they create bindings between a target and an object. For collections that are mutable or contain mutable items, a copy is sometimes needed so one can change one copy without changing the other. This module provides generic shallow and deep copy operations (explained below).

Interface summary:

<div class="function">

copy(obj)

Return a shallow copy of *obj*.

</div>

<div class="function">

deepcopy(obj\[, memo\])

Return a deep copy of *obj*.

</div>

<div class="function">

replace(obj, /, \*\*changes)

Creates a new object of the same type as *obj*, replacing fields with values from *changes*.

<div class="versionadded">

3.13

</div>

</div>

<div class="exception">

Error

Raised for module specific errors.

</div>

<div id="shallow_vs_deep_copy">

The difference between shallow and deep copying is only relevant for compound objects (objects that contain other objects, like lists or class instances):

</div>

- A *shallow copy* constructs a new compound object and then (to the extent possible) inserts *references* into it to the objects found in the original.
- A *deep copy* constructs a new compound object and then, recursively, inserts *copies* into it of the objects found in the original.

Two problems often exist with deep copy operations that don't exist with shallow copy operations:

- Recursive objects (compound objects that, directly or indirectly, contain a reference to themselves) may cause a recursive loop.
- Because deep copy copies everything it may copy too much, such as data which is intended to be shared between copies.

The `deepcopy` function avoids these problems by:

- keeping a `memo` dictionary of objects already copied during the current copying pass; and
- letting user-defined classes override the copying operation or the set of components copied.

This module does not copy types like module, method, stack trace, stack frame, file, socket, window, or any similar types. It does "copy" functions and classes (shallow and deeply), by returning the original object unchanged; this is compatible with the way these are treated by the `pickle` module.

Shallow copies of many collections can be made using the corresponding `!copy` method (such as `list.copy`, `dict.copy` or `set.copy`), and of sequences (such as lists or bytearrays) by making a slice of the entire sequence (`sequence[:]`). However, these methods and slicing can create an instance of the base type when copying an instance of a subclass, whereas `copy.copy` normally returns an instance of the same type.

<div class="index">

pair: module; pickle

</div>

Classes can use the same interfaces to control copying that they use to control pickling. See the description of module `pickle` for information on these methods. In fact, the `!copy` module uses the registered pickle functions from the `copyreg` module.

<div class="index">

single: \_\_copy\_\_() (copy protocol) single: \_\_deepcopy\_\_() (copy protocol)

</div>

<div class="currentmodule">

None

</div>

In order for a class to define its own copy implementation, it can define special methods `~object.__copy__` and `~object.__deepcopy__`.

<div class="method" data-noindexentry="">

object.\_\_copy\_\_(self)

Called to implement the shallow copy operation; no additional arguments are passed.

</div>

<div class="method" data-noindexentry="">

object.\_\_deepcopy\_\_(self, memo)

Called to implement the deep copy operation; it is passed one argument, the *memo* dictionary. If the `__deepcopy__` implementation needs to make a deep copy of a component, it should call the `~copy.deepcopy` function with the component as first argument and the *memo* dictionary as second argument. The *memo* dictionary should be treated as an opaque object.

</div>

<div class="index">

single: \_\_replace\_\_() (replace protocol)

</div>

Function `!copy.replace` is more limited than `~copy.copy` and `~copy.deepcopy`, and only supports named tuples created by `~collections.namedtuple`, `dataclasses`, and other classes which define method `~object.__replace__`.

<div class="method" data-noindexentry="">

object.\_\_replace\_\_(self, /, \*\*changes)

This method should create a new object of the same type, replacing fields with values from *changes*.

<div class="versionadded">

3.13

</div>

</div>

<div class="seealso">

Module `pickle`  
Discussion of the special methods used to support object state retrieval and restoration.

</div>

# `!copyreg` --- Register `!pickle` support functions

<div class="module" data-synopsis="Register pickle support functions.">

copyreg

</div>

**Source code:** `Lib/copyreg.py`

<div class="index">

pair: module; pickle pair: module; copy

</div>

------------------------------------------------------------------------

The `!copyreg` module offers a way to define functions used while pickling specific objects. The `pickle` and `copy` modules use those functions when pickling/copying those objects. The module provides configuration information about object constructors which are not classes. Such constructors may be factory functions or class instances.

<div class="function">

constructor(object)

Declares *object* to be a valid constructor. If *object* is not callable (and hence not valid as a constructor), raises `TypeError`.

</div>

<div class="function">

pickle(type, function, constructor_ob=None)

Declares that *function* should be used as a "reduction" function for objects of type *type*. *function* must return either a string or a tuple containing between two and six elements. See the `~pickle.Pickler.dispatch_table` for more details on the interface of *function*.

The *constructor_ob* parameter is a legacy feature and is now ignored, but if passed it must be a callable.

Note that the `~pickle.Pickler.dispatch_table` attribute of a pickler object or subclass of `pickle.Pickler` can also be used for declaring reduction functions.

</div>

## Example

The example below would like to show how to register a pickle function and how it will be used:

> \>\>\> import copyreg, copy, pickle \>\>\> class C: ... def \_\_init\_\_(self, a): ... self.a = a ... \>\>\> def pickle_c(c): ... print("pickling a C instance...") ... return C, (c.a,) ... \>\>\> copyreg.pickle(C, pickle_c) \>\>\> c = C(1) \>\>\> d = copy.copy(c) \# doctest: +SKIP pickling a C instance... \>\>\> p = pickle.dumps(c) \# doctest: +SKIP pickling a C instance...

# `!crypt` --- Function to check Unix passwords

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

crypt

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

Applications can use the `hashlib` module from the standard library. Other possible replacements are third-party libraries from PyPI: `legacycrypt`, `bcrypt`, or `argon2-cffi`. These are not supported or maintained by the Python core team.

The last version of Python that provided the `!crypt` module was [Python 3.12](https://docs.python.org/3.12/library/crypt.html).

# Cryptographic Services

<div class="index">

single: cryptography

</div>

The modules described in this chapter implement various algorithms of a cryptographic nature. They are available at the discretion of the installation. Here's an overview:

<div class="toctree">

hashlib.rst hmac.rst secrets.rst

</div>

# `!csv` --- CSV file reading and writing

<div class="module" data-synopsis="Write and read tabular data to and from delimited files.">

csv

</div>

**Source code:** `Lib/csv.py`

<div class="index">

single: csv pair: data; tabular

</div>

------------------------------------------------------------------------

The so-called CSV (Comma Separated Values) format is the most common import and export format for spreadsheets and databases. CSV format was used for many years prior to attempts to describe the format in a standardized way in `4180`. The lack of a well-defined standard means that subtle differences often exist in the data produced and consumed by different applications. These differences can make it annoying to process CSV files from multiple sources. Still, while the delimiters and quoting characters vary, the overall format is similar enough that it is possible to write a single module which can efficiently manipulate such data, hiding the details of reading and writing the data from the programmer.

The `!csv` module implements classes to read and write tabular data in CSV format. It allows programmers to say, "write this data in the format preferred by Excel," or "read data from this file which was generated by Excel," without knowing the precise details of the CSV format used by Excel. Programmers can also describe the CSV formats understood by other applications or define their own special-purpose CSV formats.

The `!csv` module's `reader` and `writer` objects read and write sequences. Programmers can also read and write data in dictionary form using the `DictReader` and `DictWriter` classes.

<div class="seealso">

`305` - CSV File API  
The Python Enhancement Proposal which proposed this addition to Python.

</div>

## Module contents

The `!csv` module defines the following functions:

<div class="index">

single: universal newlines; csv.reader function

</div>

<div class="function">

reader(csvfile, /, dialect='excel', \*\*fmtparams)

Return a `reader object <reader-objects>` that will process lines from the given *csvfile*. A csvfile must be an iterable of strings, each in the reader's defined csv format. A csvfile is most commonly a file-like object or list. If *csvfile* is a file object, it should be opened with `newline=''`.[^1] An optional *dialect* parameter can be given which is used to define a set of parameters specific to a particular CSV dialect. It may be an instance of a subclass of the `Dialect` class or one of the strings returned by the `list_dialects` function. The other optional *fmtparams* keyword arguments can be given to override individual formatting parameters in the current dialect. For full details about the dialect and formatting parameters, see section `csv-fmt-params`.

Each row read from the csv file is returned as a list of strings. No automatic data type conversion is performed unless the `QUOTE_NONNUMERIC` format option is specified (in which case unquoted fields are transformed into floats).

A short usage example:

    >>> import csv
    >>> with open('eggs.csv', newline='') as csvfile:
    ...     spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
    ...     for row in spamreader:
    ...         print(', '.join(row))
    Spam, Spam, Spam, Spam, Spam, Baked Beans
    Spam, Lovely Spam, Wonderful Spam

where `eggs.csv` contains:

```
Spam Spam Spam Spam Spam |Baked Beans|
Spam |Lovely Spam| |Wonderful Spam|
```

</div>

<div class="function">

writer(csvfile, /, dialect='excel', \*\*fmtparams)

Return a writer object responsible for converting the user's data into delimited strings on the given file-like object. *csvfile* can be any object with a `~io.TextIOBase.write` method. If *csvfile* is a file object, it should be opened with `newline=''`[^2]. An optional *dialect* parameter can be given which is used to define a set of parameters specific to a particular CSV dialect. It may be an instance of a subclass of the `Dialect` class or one of the strings returned by the `list_dialects` function. The other optional *fmtparams* keyword arguments can be given to override individual formatting parameters in the current dialect. For full details about dialects and formatting parameters, see the `csv-fmt-params` section. To make it as easy as possible to interface with modules which implement the DB API, the value `None` is written as the empty string. While this isn't a reversible transformation, it makes it easier to dump SQL NULL data values to CSV files without preprocessing the data returned from a `cursor.fetch*` call. All other non-string data are stringified with `str` before being written.

A short usage example:

    import csv
    with open('eggs.csv', 'w', newline='') as csvfile:
        spamwriter = csv.writer(csvfile, delimiter=' ',
                                quotechar='|', quoting=csv.QUOTE_MINIMAL)
        spamwriter.writerow(['Spam'] * 5 + ['Baked Beans'])
        spamwriter.writerow(['Spam', 'Lovely Spam', 'Wonderful Spam'])

which writes `eggs.csv` containing:

```
Spam Spam Spam Spam Spam |Baked Beans|
Spam |Lovely Spam| |Wonderful Spam|
```

</div>

<div class="function">

register_dialect(name, /, dialect='excel', \*\*fmtparams)

Associate *dialect* with *name*. *name* must be a string. The dialect can be specified either by passing a sub-class of `Dialect`, or by *fmtparams* keyword arguments, or both, with keyword arguments overriding parameters of the dialect. For full details about dialects and formatting parameters, see section `csv-fmt-params`.

</div>

<div class="function">

unregister_dialect(name)

Delete the dialect associated with *name* from the dialect registry. An `Error` is raised if *name* is not a registered dialect name.

</div>

<div class="function">

get_dialect(name)

Return the dialect associated with *name*. An `Error` is raised if *name* is not a registered dialect name. This function returns an immutable `Dialect`.

</div>

<div class="function">

list_dialects()

Return the names of all registered dialects.

</div>

<div class="function">

field_size_limit() field_size_limit(new_limit)

Returns the current maximum field size allowed by the parser. If *new_limit* is given, this becomes the new limit.

</div>

The `!csv` module defines the following classes:

<div class="DictReader(f, fieldnames=None, restkey=None, restval=None, \ dialect='excel', *args, **kwds)">

Create an object that operates like a regular reader but maps the information in each row to a `dict` whose keys are given by the optional *fieldnames* parameter.

The *fieldnames* parameter is a `sequence`. If *fieldnames* is omitted, the values in the first row of file *f* will be used as the fieldnames and will be omitted from the results. If *fieldnames* is provided, they will be used and the first row will be included in the results. Regardless of how the fieldnames are determined, the dictionary preserves their original ordering.

If a row has more fields than fieldnames, the remaining data is put in a list and stored with the fieldname specified by *restkey* (which defaults to `None`). If a non-blank row has fewer fields than fieldnames, the missing values are filled-in with the value of *restval* (which defaults to `None`).

All other optional or keyword arguments are passed to the underlying `reader` instance.

If the argument passed to *fieldnames* is an iterator, it will be coerced to a `list`.

<div class="versionchanged">

3.6 Returned rows are now of type `OrderedDict`.

</div>

<div class="versionchanged">

3.8 Returned rows are now of type `dict`.

</div>

A short usage example:

    >>> import csv
    >>> with open('names.csv', newline='') as csvfile:
    ...     reader = csv.DictReader(csvfile)
    ...     for row in reader:
    ...         print(row['first_name'], row['last_name'])
    ...
    Eric Idle
    John Cleese

    >>> print(row)
    {'first_name': 'John', 'last_name': 'Cleese'}

where `names.csv` contains:

```
first_name,last_name
Eric,Idle
John,Cleese
```

</div>

<div class="DictWriter(f, fieldnames, restval='', extrasaction='raise', \ dialect='excel', *args, **kwds)">

Create an object which operates like a regular writer but maps dictionaries onto output rows. The *fieldnames* parameter is a `sequence
<collections.abc>` of keys that identify the order in which values in the dictionary passed to the `~csvwriter.writerow` method are written to file *f*. The optional *restval* parameter specifies the value to be written if the dictionary is missing a key in *fieldnames*. If the dictionary passed to the `~csvwriter.writerow` method contains a key not found in *fieldnames*, the optional *extrasaction* parameter indicates what action to take. If it is set to `'raise'`, the default value, a `ValueError` is raised. If it is set to `'ignore'`, extra values in the dictionary are ignored. Any other optional or keyword arguments are passed to the underlying `writer` instance.

Note that unlike the `DictReader` class, the *fieldnames* parameter of the `DictWriter` class is not optional.

If the argument passed to *fieldnames* is an iterator, it will be coerced to a `list`.

A short usage example:

    import csv

    with open('names.csv', 'w', newline='') as csvfile:
        fieldnames = ['first_name', 'last_name']
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

        writer.writeheader()
        writer.writerow({'first_name': 'Baked', 'last_name': 'Beans'})
        writer.writerow({'first_name': 'Lovely', 'last_name': 'Spam'})
        writer.writerow({'first_name': 'Wonderful', 'last_name': 'Spam'})

which writes `names.csv` containing:

```
first_name,last_name
Baked,Beans
Lovely,Spam
Wonderful,Spam
```

</div>

<div class="Dialect">

The `Dialect` class is a container class whose attributes contain information for how to handle doublequotes, whitespace, delimiters, etc. Due to the lack of a strict CSV specification, different applications produce subtly different CSV data. `Dialect` instances define how `reader` and `writer` instances behave.

All available `Dialect` names are returned by `list_dialects`, and they can be registered with specific `reader` and `writer` classes through their initializer (`__init__`) functions like this:

    import csv

    with open('students.csv', 'w', newline='') as csvfile:
        writer = csv.writer(csvfile, dialect='unix')

</div>

<div class="excel()">

The `excel` class defines the usual properties of an Excel-generated CSV file. It is registered with the dialect name `'excel'`.

</div>

<div class="excel_tab()">

The `excel_tab` class defines the usual properties of an Excel-generated TAB-delimited file. It is registered with the dialect name `'excel-tab'`.

</div>

<div class="unix_dialect()">

The `unix_dialect` class defines the usual properties of a CSV file generated on UNIX systems, i.e. using `'\n'` as line terminator and quoting all fields. It is registered with the dialect name `'unix'`.

<div class="versionadded">

3.2

</div>

</div>

<div class="Sniffer()">

The `Sniffer` class is used to deduce the format of a CSV file.

The `Sniffer` class provides two methods:

<div class="method">

sniff(sample, delimiters=None)

Analyze the given *sample* and return a `Dialect` subclass reflecting the parameters found. If the optional *delimiters* parameter is given, it is interpreted as a string containing possible valid delimiter characters.

The dialect is deduced by parsing the sample with every plausible combination of parameters and choosing the combination which splits the sample into rows with the most consistent number of fields, so the returned dialect is consistent with how `reader` will parse the sample. Raise `Error` if no combination fits the sample, in particular if it is a single column, so there is no delimiter to find.

If several combinations fit the sample equally well --- for example if both `','` and `';'` split every row consistently --- the delimiters listed in the `~Sniffer.preferred` attribute are preferred, in that order, no matter how many times each of them occurs.

The *lineterminator* parameter is deduced separately, by a majority vote among the line endings of the sample. A tie is broken in the order `'\r\n'`, `'\n'`, `'\r'`, so a sample without a complete line gives `'\r\n'`.

<div class="versionchanged">

3.16 The dialect is now deduced by trial parsing and the results may differ from those of earlier Python versions. The *escapechar* and *lineterminator* parameters can now be detected, and the requested *delimiters* are not restricted to ASCII.

</div>

</div>

<div class="method">

has_header(sample)

Analyze the sample text (presumed to be in CSV format) and return `True` if the first row appears to be a series of column headers. Inspecting each column, one of two key criteria will be considered to estimate if the sample contains a header:

- the second through n-th rows contain numeric values
- the second through n-th rows contain strings where at least one value's length differs from that of the putative header of that column.

Twenty-one rows after the header are sampled; if more than half of the columns + rows meet the criteria, `True` is returned.

</div>

> [!NOTE]
> This method is a rough heuristic and may produce both false positives and negatives.

The `Sniffer` class has the following attribute:

<div class="attribute">

preferred

The list of the delimiters preferred for breaking ties, in the order of preference. It can be modified. Its initial value is `[',', '\t', ';', ' ', ':']`.

</div>

</div>

An example for `Sniffer` use:

    with open('example.csv', newline='') as csvfile:
        dialect = csv.Sniffer().sniff(csvfile.read(1024))
        csvfile.seek(0)
        reader = csv.reader(csvfile, dialect)
        # ... process CSV file contents here ...

<div id="csv-constants">

The `!csv` module defines the following constants:

</div>

<div class="data">

QUOTE_ALL

Instructs `writer` objects to quote all fields.

</div>

<div class="data">

QUOTE_MINIMAL

Instructs `writer` objects to only quote those fields which contain special characters such as *delimiter*, *quotechar*, `'\r'`, `'\n'` or any of the characters in *lineterminator*. If *doublequote* is `False` and *escapechar* is set, the *quotechar* is escaped instead of causing the field to be quoted.

</div>

<div class="data">

QUOTE_NONNUMERIC

Instructs `writer` objects to quote all non-numeric fields.

Instructs `reader` objects to convert all non-quoted fields to type `float`.

> [!NOTE]
> Some numeric types, such as `bool`, `~fractions.Fraction`, or `~enum.IntEnum`, have a string representation that cannot be converted to `float`. They cannot be read in the `QUOTE_NONNUMERIC` and `QUOTE_STRINGS` modes.

</div>

<div class="data">

QUOTE_NONE

Instructs `writer` objects to never quote fields. When the current *delimiter*, *quotechar*, *escapechar*, `'\r'`, `'\n'` or any of the characters in *lineterminator* occurs in output data it is preceded by the current *escapechar* character. If *escapechar* is not set, the writer will raise `Error` if any characters that require escaping are encountered. Set *quotechar* to `None` to prevent its escaping.

Instructs `reader` objects to perform no special processing of quote characters.

</div>

<div class="data">

QUOTE_NOTNULL

Instructs `writer` objects to quote all fields which are not `None`. This is similar to `QUOTE_ALL`, except that if a field value is `None` an empty (unquoted) string is written.

Instructs `reader` objects to interpret an empty (unquoted) field as `None` and to otherwise behave as `QUOTE_ALL`.

<div class="versionadded">

3.12

</div>

</div>

<div class="data">

QUOTE_STRINGS

Instructs `writer` objects to always place quotes around fields which are strings. This is similar to `QUOTE_NONNUMERIC`, except that if a field value is `None` an empty (unquoted) string is written.

Instructs `reader` objects to interpret an empty (unquoted) string as `None` and to otherwise behave as `QUOTE_NONNUMERIC`.

<div class="versionadded">

3.12

</div>

</div>

The `!csv` module defines the following exception:

<div class="exception">

Error

Raised by any of the functions when an error is detected.

</div>

## Dialects and formatting parameters

To make it easier to specify the format of input and output records, specific formatting parameters are grouped together into dialects. A dialect is a subclass of the `Dialect` class containing various attributes describing the format of the CSV file. When creating `reader` or `writer` objects, the programmer can specify a string or a subclass of the `Dialect` class as the dialect parameter. In addition to, or instead of, the *dialect* parameter, the programmer can also specify individual formatting parameters, which have the same names as the attributes defined below for the `Dialect` class.

Dialects support the following attributes:

<div class="attribute">

Dialect.delimiter

A one-character string used to separate fields. It defaults to `','`.

</div>

<div class="attribute">

Dialect.doublequote

Controls how instances of *quotechar* appearing inside a field should themselves be quoted. When `True`, the character is doubled. When `False`, the *escapechar* is used as a prefix to the *quotechar*. It defaults to `True`.

On output, if *doublequote* is `False` and no *escapechar* is set, `Error` is raised if a *quotechar* is found in a field.

</div>

<div class="attribute">

Dialect.escapechar

A one-character string used by the writer to escape characters that require escaping:

> - the *delimiter*, the *quotechar*, `'\r'`, `'\n'` and any of the characters in *lineterminator* are escaped if *quoting* is set to `QUOTE_NONE`;
> - the *quotechar* is escaped if *doublequote* is `False`;
> - the *escapechar* itself.

On reading, the *escapechar* removes any special meaning from the following character. It defaults to `None`, which disables escaping.

<div class="versionchanged">

3.10 Previously the *escapechar* itself was not escaped, which lost it on reading.

</div>

<div class="versionchanged">

3.11 An empty *escapechar* is not allowed.

</div>

</div>

<div class="attribute">

Dialect.lineterminator

The string used to terminate lines produced by the `writer`. It defaults to `'\r\n'`.

> [!NOTE]
> The `reader` is hard-coded to recognise either `'\r'` or `'\n'` as end-of-line, and ignores *lineterminator*. This behavior may change in the future.

</div>

<div class="attribute">

Dialect.quotechar

A one-character string used to quote fields containing special characters, such as the *delimiter* or the *quotechar*, or which contain new-line characters (`'\r'`, `'\n'` or any of the characters in *lineterminator*). It defaults to `'"'`. Can be set to `None` to prevent escaping `'"'` if *quoting* is set to `QUOTE_NONE`.

<div class="versionchanged">

3.11 An empty *quotechar* is not allowed.

</div>

</div>

<div class="attribute">

Dialect.quoting

Controls when quotes should be generated by the writer and recognised by the reader. It can take on any of the `QUOTE_\* constants <csv-constants>` and defaults to `QUOTE_MINIMAL` if *quotechar* is not `None`, and `QUOTE_NONE` otherwise.

</div>

<div class="attribute">

Dialect.skipinitialspace

When `True`, spaces immediately following the *delimiter* are ignored. The default is `False`. When combining `delimiter=' '` with `skipinitialspace=True`, unquoted empty fields are not allowed.

</div>

<div class="attribute">

Dialect.strict

When `True`, raise exception `Error` on bad CSV input. The default is `False`.

</div>

Dialects support `copy.replace`, which returns a copy of the dialect with the specified formatting parameters replaced.

<div class="versionchanged">

3.16 Added support for `copy.replace`.

</div>

## Reader objects

Reader objects (`DictReader` instances and objects returned by the `reader` function) have the following public methods:

<div class="method">

csvreader.\_\_next\_\_()

Return the next row of the reader's iterable object as a list (if the object was returned from `reader`) or a dict (if it is a `DictReader` instance), parsed according to the current `Dialect`. Usually you should call this as `next(reader)`.

</div>

Reader objects have the following public attributes:

<div class="attribute">

csvreader.dialect

A read-only description of the dialect in use by the parser.

</div>

<div class="attribute">

csvreader.line_num

The number of lines read from the source iterator. This is not the same as the number of records returned, as records can span multiple lines.

</div>

DictReader objects have the following public attribute:

<div class="attribute">

DictReader.fieldnames

If not passed as a parameter when creating the object, this attribute is initialized upon first access or when the first record is read from the file.

</div>

## Writer objects

`writer` objects (`DictWriter` instances and objects returned by the `writer` function) have the following public methods. A *row* must be an iterable of strings or numbers for `writer` objects and a dictionary mapping fieldnames to strings or numbers (by passing them through `str` first) for `DictWriter` objects. Note that complex numbers are written out surrounded by parens. This may cause some problems for other programs which read CSV files (assuming they support complex numbers at all).

<div class="method">

csvwriter.writerow(row, /)

Write the *row* parameter to the writer's file object, formatted according to the current `Dialect`. Return the return value of the call to the *write* method of the underlying file object.

<div class="versionchanged">

3.5 Added support of arbitrary iterables.

</div>

</div>

<div class="method">

csvwriter.writerows(rows, /)

Write all elements in *rows* (an iterable of *row* objects as described above) to the writer's file object, formatted according to the current dialect.

</div>

Writer objects have the following public attribute:

<div class="attribute">

csvwriter.dialect

A read-only description of the dialect in use by the writer.

</div>

DictWriter objects have the following public method:

<div class="method">

DictWriter.writeheader()

Write a row with the field names (as specified in the constructor) to the writer's file object, formatted according to the current dialect. Return the return value of the `csvwriter.writerow` call used internally.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.8 `writeheader` now also returns the value returned by the `csvwriter.writerow` method it uses internally.

</div>

</div>

## Examples

The simplest example of reading a CSV file:

    import csv
    with open('some.csv', newline='') as f:
        reader = csv.reader(f)
        for row in reader:
            print(row)

Reading a file with an alternate format:

    import csv
    with open('passwd', newline='') as f:
        reader = csv.reader(f, delimiter=':', quoting=csv.QUOTE_NONE)
        for row in reader:
            print(row)

The corresponding simplest possible writing example is:

    import csv
    with open('some.csv', 'w', newline='') as f:
        writer = csv.writer(f)
        writer.writerows(someiterable)

Since `open` is used to open a CSV file for reading, the file will by default be decoded into Unicode using UTF-8. To decode a file using a different encoding, use the `encoding` argument of open:

    import csv
    with open('some.csv', newline='', encoding='latin-1') as f:
        reader = csv.reader(f)
        for row in reader:
            print(row)

The same applies to writing in something other than the default encoding: specify the encoding argument when opening the output file.

Registering a new dialect:

    import csv
    csv.register_dialect('unixpwd', delimiter=':', quoting=csv.QUOTE_NONE)
    with open('passwd', newline='') as f:
        reader = csv.reader(f, 'unixpwd')

A slightly more advanced use of the reader --- catching and reporting errors:

    import csv, sys
    filename = 'some.csv'
    with open(filename, newline='') as f:
        reader = csv.reader(f)
        try:
            for row in reader:
                print(row)
        except csv.Error as e:
            sys.exit(f'file {filename}, line {reader.line_num}: {e}')

And while the module doesn't directly support parsing strings, it can easily be done:

    import csv
    for row in csv.reader(['one,two,three']):
        print(row)

**Footnotes**

[^1]: If `newline=''` is not specified, newlines embedded inside quoted fields will not be interpreted correctly, and on platforms that use `\r\n` line endings on write an extra `\r` will be added. It should always be safe to specify `newline=''`, since the csv module does its own (`universal <universal newlines>`) newline handling.

[^2]: If `newline=''` is not specified, newlines embedded inside quoted fields will not be interpreted correctly, and on platforms that use `\r\n` line endings on write an extra `\r` will be added. It should always be safe to specify `newline=''`, since the csv module does its own (`universal <universal newlines>`) newline handling.

# `!ctypes` --- A foreign function library for Python

<div class="module" data-synopsis="A foreign function library for Python.">

ctypes

</div>

**Source code:** `Lib/ctypes`

------------------------------------------------------------------------

`!ctypes` is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or shared libraries. It can be used to wrap these libraries in pure Python.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

> [!WARNING]
> `!ctypes` provides low-level access to native libraries and the process's memory, bypassing Python's safety mechanisms and allowing execution of arbitrary native code. Incorrect use can corrupt data and objects, reveal sensitive information, cause crashes, or otherwise compromise the running process.

## ctypes tutorial

Note: Some code samples reference the ctypes `c_int` type. On platforms where `sizeof(long) == sizeof(int)` it is an alias to `c_long`. So, you should not be confused if `c_long` is printed if you would expect `c_int` --- they are actually the same type.

### Loading dynamic link libraries

`!ctypes` exports the `~ctypes.cdll`, and on Windows `~ctypes.windll` and `~ctypes.oledll` objects, for loading dynamic link libraries.

You load libraries by accessing them as attributes of these objects. `!cdll` loads libraries which export functions using the standard `cdecl` calling convention, while `!windll` libraries call functions using the `stdcall` calling convention. `~oledll` also uses the `stdcall` calling convention, and assumes the functions return a Windows `!HRESULT` error code. The error code is used to automatically raise an `OSError` exception when the function call fails.

<div class="versionchanged">

3.3 Windows errors used to raise `WindowsError`, which is now an alias of `OSError`.

</div>

Here are some examples for Windows. Note that `msvcrt` is the MS standard C library containing most standard C functions, and uses the `cdecl` calling convention:

    >>> from ctypes import *
    >>> print(windll.kernel32)  # doctest: +WINDOWS
    <WinDLL 'kernel32', handle ... at ...>
    >>> print(cdll.msvcrt)      # doctest: +WINDOWS
    <CDLL 'msvcrt', handle ... at ...>
    >>> libc = cdll.msvcrt      # doctest: +WINDOWS
    >>>

Windows appends the usual `.dll` file suffix automatically.

> [!NOTE]
> Accessing the standard C library through `cdll.msvcrt` will use an outdated version of the library that may be incompatible with the one being used by Python. Where possible, use native Python functionality, or else import and use the `msvcrt` module.

Other systems require the filename *including* the extension to load a library, so attribute access can not be used to load libraries. Either the `~LibraryLoader.LoadLibrary` method of the dll loaders should be used, or you should load the library by creating an instance of `CDLL` by calling the constructor.

For example, on Linux:

    >>> cdll.LoadLibrary("libc.so.6")  # doctest: +LINUX
    <CDLL 'libc.so.6', handle ... at ...>
    >>> libc = CDLL("libc.so.6")       # doctest: +LINUX
    >>> libc                           # doctest: +LINUX
    <CDLL 'libc.so.6', handle ... at ...>
    >>>

On macOS:

    >>> cdll.LoadLibrary("libc.dylib")  # doctest: +MACOS
    <CDLL 'libc.dylib', handle ... at ...>
    >>> libc = CDLL("libc.dylib")       # doctest: +MACOS
    >>> libc                            # doctest: +MACOS
    <CDLL 'libc.dylib', handle ... at ...>

### Accessing functions from loaded dlls

Functions are accessed as attributes of dll objects:

    >>> libc.printf
    <_FuncPtr object at 0x...>
    >>> print(windll.kernel32.GetModuleHandleA)  # doctest: +WINDOWS
    <_FuncPtr object at 0x...>
    >>> print(windll.kernel32.MyOwnFunction)     # doctest: +WINDOWS
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "ctypes.py", line 239, in __getattr__
        func = _StdcallFuncPtr(name, self)
    AttributeError: function 'MyOwnFunction' not found
    >>>

Note that win32 system dlls like `kernel32` and `user32` often export ANSI as well as UNICODE versions of a function. The UNICODE version is exported with a `W` appended to the name, while the ANSI version is exported with an `A` appended to the name. The win32 `GetModuleHandle` function, which returns a *module handle* for a given module name, has the following C prototype, and a macro is used to expose one of them as `GetModuleHandle` depending on whether UNICODE is defined or not:

    /* ANSI version */
    HMODULE GetModuleHandleA(LPCSTR lpModuleName);
    /* UNICODE version */
    HMODULE GetModuleHandleW(LPCWSTR lpModuleName);

*windll* does not try to select one of them by magic, you must access the version you need by specifying `GetModuleHandleA` or `GetModuleHandleW` explicitly, and then call it with bytes or string objects respectively.

Sometimes, dlls export functions with names which aren't valid Python identifiers, like `"??2@YAPAXI@Z"`. In this case you have to use `getattr` to retrieve the function:

    >>> getattr(cdll.msvcrt, "??2@YAPAXI@Z")  # doctest: +WINDOWS
    <_FuncPtr object at 0x...>
    >>>

On Windows, some dlls export functions not by name but by ordinal. These functions can be accessed by indexing the dll object with the ordinal number:

    >>> cdll.kernel32[1]  # doctest: +WINDOWS
    <_FuncPtr object at 0x...>
    >>> cdll.kernel32[0]  # doctest: +WINDOWS
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "ctypes.py", line 310, in __getitem__
        func = _StdcallFuncPtr(name, self)
    AttributeError: function ordinal 0 not found
    >>>

### Calling functions

You can call these functions like any other Python callable. This example uses the `rand()` function, which takes no arguments and returns a pseudo-random integer:

    >>> print(libc.rand())  # doctest: +SKIP
    1804289383

On Windows, you can call the `GetModuleHandleA()` function, which returns a win32 module handle (passing `None` as single argument to call it with a `NULL` pointer):

    >>> print(hex(windll.kernel32.GetModuleHandleA(None)))  # doctest: +WINDOWS
    0x1d000000
    >>>

`!ctypes` cannot determine a function's calling convention or signature. Consult the function's C header file or documentation to choose the correct calling convention and set the `~_CFuncPtr.argtypes` attribute, as described in `ctypes-specifying-required-argument-types`. Without `~_CFuncPtr.argtypes`, the number of arguments is not checked. Calling a function with an incorrect calling convention or with incorrect arguments can corrupt memory or crash Python.

On Windows, `!ctypes` uses win32 structured exception handling to prevent crashes from general protection faults when functions are called with invalid argument values:

    >>> windll.kernel32.GetModuleHandleA(32)  # doctest: +WINDOWS
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: exception: access violation reading 0x00000020
    >>>

The `faulthandler` module can help debug crashes, such as segmentation faults produced by erroneous C library calls.

`None`, integers, bytes objects and (unicode) strings are the only native Python objects that can directly be used as parameters in these function calls. `None` is passed as a C `NULL` pointer, bytes objects and strings are passed as pointer to the memory block that contains their data (`char *` or `wchar_t *`). Python integers are passed as the platform's default C `int` type, their value is masked to fit into the C type.

Before we move on calling functions with other parameter types, we have to learn more about `!ctypes` data types.

### Fundamental data types

`!ctypes` defines a number of primitive C compatible data types:

| ctypes type | C type | Python type | `~_SimpleCData._type_` |
|----|----|----|----|
| `c_bool` | `_Bool` | `bool` | `'?'` |
| `c_char` | `char` | 1-character `bytes` | `'c'` |
| `c_wchar` | `wchar_t` | 1-character `str` | `'u'` |
| `c_byte` | `char` | `int` | `'b'` |
| `c_ubyte` | `unsigned char` | `int` | `'B'` |
| `c_short` | `short` | `int` | `'h'` |
| `c_ushort` | `unsigned short` | `int` | `'H'` |
| `c_int` | `int` | `int` | `'i'` \* |
| `c_int8` | `int8_t` | `int` | \* |
| `c_int16` | `int16_t` | `int` | \* |
| `c_int32` | `int32_t` | `int` | \* |
| `c_int64` | `int64_t` | `int` | \* |
| `c_uint` | `unsigned int` | `int` | `'I'` \* |
| `c_uint8` | `uint8_t` | `int` | \* |
| `c_uint16` | `uint16_t` | `int` | \* |
| `c_uint32` | `uint32_t` | `int` | \* |
| `c_uint64` | `uint64_t` | `int` | \* |
| `c_long` | `long` | `int` | `'l'` |
| `c_ulong` | `unsigned long` | `int` | `'L'` |
| `c_longlong` | `long long` | `int` | `'q'` \* |
| `c_ulonglong` | `unsigned long long` | `int` | `'Q'` \* |
| `c_size_t` | `size_t` | `int` | \* |
| `c_ssize_t` | `Py_ssize_t` | `int` | \* |
| `c_time_t` | `time_t` | `int` | \* |
| `c_float` | `float` | `float` | `'f'` |
| `c_double` | `double` | `float` | `'d'` |
| `c_longdouble` | `long double` | `float` | `'g'` \* |
| `c_char_p` | `char *` (NUL terminated) | `bytes` or `None` | `'z'` |
| `c_wchar_p` | `wchar_t *` (NUL terminated) | `str` or `None` | `'Z'` |
| `c_void_p` | `void *` | `int` or `None` | `'P'` |
| `py_object` | `PyObject *` | `object` | `'O'` |
| `VARIANT_BOOL <ctypes-wintypes>` | `short int` | `bool` | `'v'` |

Additionally, if IEC 60559 compatible complex arithmetic (Annex G) is supported in both C and `libffi`, the following complex types are available:

| ctypes type | C type | Python type | `~_SimpleCData._type_` |
|----|----|----|----|
| `c_float_complex` | `float complex` | `complex` | `'Zf'` |
| `c_double_complex` | `double complex` | `complex` | `'Zd'` |
| `c_longdouble_complex` | `long double complex` | `complex` | `'Zg'` |

<div class="versionchanged">

3.15 The `~_SimpleCData._type_` types `F`, `D` and `G` have been replaced with `Zf`, `Zd` and `Zg`.

</div>

All these types can be created by calling them with an optional initializer of the correct type and value:

    >>> c_int()
    c_long(0)
    >>> c_wchar_p("Hello, World")
    c_wchar_p(140018365411392)
    >>> c_ushort(-3)
    c_ushort(65533)
    >>>

The constructors for numeric types will convert input using `~object.__bool__`, `~object.__index__` (for `int`), `~object.__float__` or `~object.__complex__`. This means `~ctypes.c_bool` accepts any object with a truth value:

    >>> empty_list = []
    >>> c_bool(empty_list)
    c_bool(False)

Since these types are mutable, their value can also be changed afterwards:

    >>> i = c_int(42)
    >>> print(i)
    c_long(42)
    >>> print(i.value)
    42
    >>> i.value = -99
    >>> print(i.value)
    -99
    >>>

Assigning a new value to instances of the pointer types `c_char_p`, `c_wchar_p`, and `c_void_p` changes the *memory location* they point to, *not the contents* of the memory block (of course not, because Python string objects are immutable):

    >>> s = "Hello, World"
    >>> c_s = c_wchar_p(s)
    >>> print(c_s)
    c_wchar_p(139966785747344)
    >>> print(c_s.value)
    Hello World
    >>> c_s.value = "Hi, there"
    >>> print(c_s)              # the memory location has changed
    c_wchar_p(139966783348904)
    >>> print(c_s.value)
    Hi, there
    >>> print(s)                # first object is unchanged
    Hello, World
    >>>

You should be careful, however, not to pass them to functions expecting pointers to mutable memory. If you need mutable memory blocks, ctypes has a `create_string_buffer` function which creates these in various ways. The current memory block contents can be accessed (or changed) with the `raw` property; if you want to access it as NUL terminated string, use the `value` property:

    >>> from ctypes import *
    >>> p = create_string_buffer(3)            # create a 3 byte buffer, initialized to NUL bytes
    >>> print(sizeof(p), repr(p.raw))
    3 b'\x00\x00\x00'
    >>> p = create_string_buffer(b"Hello")     # create a buffer containing a NUL terminated string
    >>> print(sizeof(p), repr(p.raw))
    6 b'Hello\x00'
    >>> print(repr(p.value))
    b'Hello'
    >>> p = create_string_buffer(b"Hello", 10) # create a 10 byte buffer
    >>> print(sizeof(p), repr(p.raw))
    10 b'Hello\x00\x00\x00\x00\x00'
    >>> p.value = b"Hi"
    >>> print(sizeof(p), repr(p.raw))
    10 b'Hi\x00lo\x00\x00\x00\x00\x00'
    >>>

The `create_string_buffer` function replaces the old `!c_buffer` function (which is still available as an alias). To create a mutable memory block containing unicode characters of the C type `wchar_t`, use the `create_unicode_buffer` function.

### Calling functions, continued

Note that printf prints to the real standard output channel, *not* to `sys.stdout`, so these examples will only work at the console prompt, not from within *IDLE* or *PythonWin*:

    >>> printf = libc.printf
    >>> printf(b"Hello, %s\n", b"World!")
    Hello, World!
    14
    >>> printf(b"Hello, %S\n", "World!")
    Hello, World!
    14
    >>> printf(b"%d bottles of beer\n", 42)
    42 bottles of beer
    19
    >>> printf(b"%f bottles of beer\n", 42.5)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ctypes.ArgumentError: argument 2: TypeError: Don't know how to convert parameter 2
    >>>

As has been mentioned before, all Python types except integers, strings, and bytes objects have to be wrapped in their corresponding `!ctypes` type, so that they can be converted to the required C data type:

    >>> printf(b"An int %d, a double %f\n", 1234, c_double(3.14))
    An int 1234, a double 3.140000
    31
    >>>

### Calling variadic functions

On a lot of platforms calling variadic functions through ctypes is exactly the same as calling functions with a fixed number of parameters. On some platforms, and in particular ARM64 for Apple Platforms, the calling convention for variadic functions is different than that for regular functions.

On those platforms it is required to specify the `~_CFuncPtr.argtypes` attribute for the regular, non-variadic, function arguments:

```
libc.printf.argtypes = [ctypes.c_char_p]
```

Because specifying the attribute does not inhibit portability it is advised to always specify `~_CFuncPtr.argtypes` for all variadic functions.

### Calling functions with your own custom data types

You can also customize `!ctypes` argument conversion to allow instances of your own classes be used as function arguments. `!ctypes` looks for an `!_as_parameter_` attribute and uses this as the function argument. The attribute must be an integer, string, bytes, a `!ctypes` instance, or an object with an `!_as_parameter_` attribute:

    >>> class Bottles:
    ...     def __init__(self, number):
    ...         self._as_parameter_ = number
    ...
    >>> bottles = Bottles(42)
    >>> printf(b"%d bottles of beer\n", bottles)
    42 bottles of beer
    19
    >>>

If you don't want to store the instance's data in the `!_as_parameter_` instance variable, you could define a `property` which makes the attribute available on request.

### Specifying the required argument types (function prototypes)

It is possible to specify the required argument types of functions exported from DLLs by setting the `~_CFuncPtr.argtypes` attribute.

`~_CFuncPtr.argtypes` must be a sequence of C data types (the `!printf` function is probably not a good example here, because it takes a variable number and different types of parameters depending on the format string, on the other hand this is quite handy to experiment with this feature):

    >>> printf.argtypes = [c_char_p, c_char_p, c_int, c_double]
    >>> printf(b"String '%s', Int %d, Double %f\n", b"Hi", 10, 2.2)
    String 'Hi', Int 10, Double 2.200000
    37
    >>>

Specifying a format protects against incompatible argument types (just as a prototype for a C function), and tries to convert the arguments to valid types:

    >>> printf(b"%d %d %d", 1, 2, 3)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ctypes.ArgumentError: argument 2: TypeError: 'int' object cannot be interpreted as ctypes.c_char_p
    >>> printf(b"%s %d %f\n", b"X", 2, 3)
    X 2 3.000000
    13
    >>>

Setting `~_CFuncPtr.argtypes` also causes `!ctypes` to raise `TypeError` if too few arguments are passed:

    >>> printf(b"%s %d %f\n", b"X", 2)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: this function takes at least 4 arguments (3 given)

Functions using the `stdcall` calling convention require exactly as many arguments as there are items in `~_CFuncPtr.argtypes`. Functions using the `cdecl` calling convention accept additional arguments, to support variadic functions. The declaration must match the function's actual signature; ctypes cannot verify this.

If you have defined your own classes which you pass to function calls, you have to implement a `~_CData.from_param` class method for them to be able to use them in the `~_CFuncPtr.argtypes` sequence. The `~_CData.from_param` class method receives the Python object passed to the function call, it should do a typecheck or whatever is needed to make sure this object is acceptable, and then return the object itself, its `!_as_parameter_` attribute, or whatever you want to pass as the C function argument in this case. Again, the result should be an integer, string, bytes, a `!ctypes` instance, or an object with an `!_as_parameter_` attribute.

### Return types

<div class="testsetup">

from ctypes import CDLL, c_char, c_char_p from ctypes.util import find_library libc = CDLL(find_library('c')) strchr = libc.strchr

</div>

By default functions are assumed to return the C `int` type. Other return types can be specified by setting the `~_CFuncPtr.restype` attribute of the function object.

The C prototype of `time` is `time_t time(time_t *)`. Because `time_t` might be of a different type than the default return type `int`, you should specify the `!restype` attribute:

    >>> libc.time.restype = c_time_t

The argument types can be specified using `~_CFuncPtr.argtypes`:

    >>> libc.time.argtypes = (POINTER(c_time_t),)

To call the function with a `NULL` pointer as first argument, use `None`:

    >>> print(libc.time(None))  # doctest: +SKIP
    1150640792

Here is a more advanced example, it uses the `!strchr` function, which expects a string pointer and a char, and returns a pointer to a string:

    >>> strchr = libc.strchr
    >>> strchr(b"abcdef", ord("d"))  # doctest: +SKIP
    8059983
    >>> strchr.restype = c_char_p    # c_char_p is a pointer to a string
    >>> strchr(b"abcdef", ord("d"))
    b'def'
    >>> print(strchr(b"abcdef", ord("x")))
    None
    >>>

If you want to avoid the `ord("x") <ord>` calls above, you can set the `~_CFuncPtr.argtypes` attribute, and the second argument will be converted from a single character Python bytes object into a C char:

<div class="doctest">

\>\>\> strchr.restype = c_char_p \>\>\> strchr.argtypes = \[c_char_p, c_char\] \>\>\> strchr(b"abcdef", b"d") b'def' \>\>\> strchr(b"abcdef", b"def") Traceback (most recent call last): ctypes.ArgumentError: argument 2: TypeError: one character bytes, bytearray or integer expected \>\>\> print(strchr(b"abcdef", b"x")) None \>\>\> strchr(b"abcdef", b"d") b'def' \>\>\>

</div>

You can also use a callable Python object (a function or a class for example) as the `~_CFuncPtr.restype` attribute, if the foreign function returns an integer. The callable will be called with the *integer* the C function returns, and the result of this call will be used as the result of your function call. This is useful to check for error return values and automatically raise an exception:

    >>> GetModuleHandle = windll.kernel32.GetModuleHandleA  # doctest: +WINDOWS
    >>> def ValidHandle(value):
    ...     if value == 0:
    ...         raise WinError()
    ...     return value
    ...
    >>>
    >>> GetModuleHandle.restype = ValidHandle  # doctest: +WINDOWS
    >>> GetModuleHandle(None)  # doctest: +WINDOWS
    486539264
    >>> GetModuleHandle("something silly")  # doctest: +WINDOWS
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 3, in ValidHandle
    OSError: [Errno 126] The specified module could not be found.
    >>>

`WinError` is a function which will call Windows `FormatMessage()` api to get the string representation of an error code, and *returns* an exception. `WinError` takes an optional error code parameter, if no one is used, it calls `GetLastError` to retrieve it.

Please note that a much more powerful error checking mechanism is available through the `~_CFuncPtr.errcheck` attribute; see the reference manual for details.

### Specifying function pointers using type annotations

<div class="decorator" data-module="ctypes.util">

wrap_dll_function(dll)

A `decorator` that generates `~ctypes._CFuncPtr.argtypes` and `~ctypes._CFuncPtr.restype` from a function signature, using the `~function.__name__` of the function and its `type annotations <annotation>`.

The decorated function should look like this:

    @wrap_dll_function(dll_to_wrap)
    def function_ptr_name(arg_name: ctypes_type, ...) -> ctypes_type:
        """Optional docstring. There should be no function body."""

The body of the decorated function is ignored, and any parameters that are missing type annotations are skipped. The names of the parameters are ignored and do not have to match the underlying C implementation.

If the decorated function does not have a return type annotation, a `ValueError` is raised. A `ValueError` is also raised if it has a keyword-only, `*args`, or `**kwargs` parameter, since `~ctypes._CFuncPtr.argtypes` describes positional arguments only. If the name of the function does not exist in *dll*, an `AttributeError` is raised.

For example:

    import ctypes
    from ctypes.util import wrap_dll_function

    @wrap_dll_function(ctypes.pythonapi)
    def PyObject_GetAttrString(op: ctypes.py_object, attr: ctypes.c_char_p) -> ctypes.py_object:
        pass

    PyObject_GetAttrString(42, b"real")

<div class="versionadded">

3.16

</div>

</div>

### Passing pointers (or: passing parameters by reference)

Sometimes a C api function expects a *pointer* to a data type as parameter, probably to write into the corresponding location, or if the data is too large to be passed by value. This is also known as *passing parameters by reference*.

`!ctypes` exports the `byref` function which is used to pass parameters by reference. The same effect can be achieved with the `pointer` function, although `pointer` does a lot more work since it constructs a real pointer object, so it is faster to use `byref` if you don't need the pointer object in Python itself:

    >>> i = c_int()
    >>> f = c_float()
    >>> s = create_string_buffer(b'\000' * 32)
    >>> print(i.value, f.value, repr(s.value))
    0 0.0 b''
    >>> libc.sscanf(b"1 3.14 Hello", b"%d %f %s",
    ...             byref(i), byref(f), s)
    3
    >>> print(i.value, f.value, repr(s.value))
    1 3.1400001049 b'Hello'
    >>>

### Structures and unions

Structures and unions must derive from the `Structure` and `Union` base classes which are defined in the `!ctypes` module. Each subclass must define a `~Structure._fields_` attribute. `!_fields_` must be a list of *2-tuples*, containing a *field name* and a *field type*.

The field type must be a `!ctypes` type like `c_int`, or any other derived `!ctypes` type: structure, union, array, pointer.

Here is a simple example of a POINT structure, which contains two integers named *x* and *y*, and also shows how to initialize a structure in the constructor:

    >>> from ctypes import *
    >>> class POINT(Structure):
    ...     _fields_ = [("x", c_int),
    ...                 ("y", c_int)]
    ...
    >>> point = POINT(10, 20)
    >>> print(point.x, point.y)
    10 20
    >>> point = POINT(y=5)
    >>> print(point.x, point.y)
    0 5
    >>> POINT(1, 2, 3)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: too many initializers
    >>>

You can, however, build much more complicated structures. A structure can itself contain other structures by using a structure as a field type.

Here is a RECT structure which contains two POINTs named *upperleft* and *lowerright*:

    >>> class RECT(Structure):
    ...     _fields_ = [("upperleft", POINT),
    ...                 ("lowerright", POINT)]
    ...
    >>> rc = RECT(point)
    >>> print(rc.upperleft.x, rc.upperleft.y)
    0 5
    >>> print(rc.lowerright.x, rc.lowerright.y)
    0 0
    >>>

Nested structures can also be initialized in the constructor in several ways:

    >>> r = RECT(POINT(1, 2), POINT(3, 4))
    >>> r = RECT((1, 2), (3, 4))

Field `descriptor`s can be retrieved from the *class*, they are useful for debugging because they can provide useful information. See `CField`:

    >>> POINT.x
    <ctypes.CField 'x' type=c_int, ofs=0, size=4>
    >>> POINT.y
    <ctypes.CField 'y' type=c_int, ofs=4, size=4>
    >>>

<div id="ctypes-structureunion-alignment-byte-order">

> [!WARNING]
> `!ctypes` does not support passing unions or structures with bit-fields to functions by value. While this may work on 32-bit x86, it's not guaranteed by the library to work in the general case. Unions and structures with bit-fields should always be passed to functions by pointer.

</div>

### Structure/union layout, alignment and byte order

By default, Structure and Union fields are laid out in the same way the C compiler does it. It is possible to override this behavior entirely by specifying a `~Structure._layout_` class attribute in the subclass definition; see the attribute documentation for details.

It is possible to specify the maximum alignment for the fields and/or for the structure itself by setting the class attributes `~Structure._pack_` and/or `~Structure._align_`, respectively. See the attribute documentation for details.

`!ctypes` uses the native byte order for Structures and Unions. To build structures with non-native byte order, you can use one of the `BigEndianStructure`, `LittleEndianStructure`, `BigEndianUnion`, and `LittleEndianUnion` base classes. These classes cannot contain pointer fields.

### Bit fields in structures and unions

It is possible to create structures and unions containing bit fields. Bit fields are only possible for integer fields, the bit width is specified as the third item in the `~Structure._fields_` tuples:

    >>> class Int(Structure):
    ...     _fields_ = [("first_16", c_int, 16),
    ...                 ("second_16", c_int, 16)]
    ...
    >>> print(Int.first_16)
    <ctypes.CField 'first_16' type=c_int, ofs=0, bit_size=16, bit_offset=0>
    >>> print(Int.second_16)
    <ctypes.CField 'second_16' type=c_int, ofs=0, bit_size=16, bit_offset=16>

It is important to note that bit field allocation and layout in memory are not defined as a C standard; their implementation is compiler-specific. By default, Python will attempt to match the behavior of a "native" compiler for the current platform. See the `~Structure._layout_` attribute for details on the default behavior and how to change it.

### Arrays

Arrays are sequences, containing a fixed number of instances of the same type.

The recommended way to create array types is by multiplying a data type with a positive integer:

    TenPointsArrayType = POINT * 10

Here is an example of a somewhat artificial data type, a structure containing 4 POINTs among other stuff:

    >>> from ctypes import *
    >>> class POINT(Structure):
    ...     _fields_ = ("x", c_int), ("y", c_int)
    ...
    >>> class MyStruct(Structure):
    ...     _fields_ = [("a", c_int),
    ...                 ("b", c_float),
    ...                 ("point_array", POINT * 4)]
    >>>
    >>> print(len(MyStruct().point_array))
    4
    >>>

Instances are created in the usual way, by calling the class:

    arr = TenPointsArrayType()
    for pt in arr:
        print(pt.x, pt.y)

The above code print a series of `0 0` lines, because the array contents is initialized to zeros.

Initializers of the correct type can also be specified:

    >>> from ctypes import *
    >>> TenIntegers = c_int * 10
    >>> ii = TenIntegers(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
    >>> print(ii)
    <c_long_Array_10 object at 0x...>
    >>> for i in ii: print(i, end=" ")
    ...
    1 2 3 4 5 6 7 8 9 10
    >>>

### Pointers

Pointer instances are created by calling the `pointer` function on a `!ctypes` type:

    >>> from ctypes import *
    >>> i = c_int(42)
    >>> pi = pointer(i)
    >>>

Pointer instances have a `~_Pointer.contents` attribute which returns the object to which the pointer points, the `i` object above:

    >>> pi.contents
    c_long(42)
    >>>

Note that `!ctypes` does not have OOR (original object return), it constructs a new, equivalent object each time you retrieve an attribute:

    >>> pi.contents is i
    False
    >>> pi.contents is pi.contents
    False
    >>>

Assigning another `c_int` instance to the pointer's contents attribute would cause the pointer to point to the memory location where this is stored:

    >>> i = c_int(99)
    >>> pi.contents = i
    >>> pi.contents
    c_long(99)
    >>>

Pointer instances can also be indexed with integers:

    >>> pi[0]
    99
    >>>

Assigning to an integer index changes the pointed to value:

    >>> print(i)
    c_long(99)
    >>> pi[0] = 22
    >>> print(i)
    c_long(22)
    >>>

It is also possible to use indexes different from 0, but you must know what you're doing, just as in C: You can access or change arbitrary memory locations. Generally you only use this feature if you receive a pointer from a C function, and you *know* that the pointer actually points to an array instead of a single item.

> [!WARNING]
> Because pointer objects support subscription, they implicitly support `iteration <iterator>`. Unless doing this in a controlled manner, such as by manually calling `next` on a `pointer` iterator, this will typically lead to infinite loops or crashes, because ctypes has no way of knowing when to stop iteration. In other words, a `pointer` iterator will infinitely yield arbitrary memory.

Behind the scenes, the `pointer` function does more than simply create pointer instances, it has to create pointer *types* first. This is done with the `POINTER` function, which accepts any `!ctypes` type, and returns a new type:

    >>> PI = POINTER(c_int)
    >>> PI
    <class 'ctypes.LP_c_long'>
    >>> PI(42)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: expected c_long instead of int
    >>> PI(c_int(42))
    <ctypes.LP_c_long object at 0x...>
    >>>

Calling the pointer type without an argument creates a `NULL` pointer. `NULL` pointers have a `False` boolean value:

    >>> null_ptr = POINTER(c_int)()
    >>> print(bool(null_ptr))
    False
    >>>

`!ctypes` checks for `NULL` when dereferencing pointers (but dereferencing invalid non-`NULL` pointers would crash Python):

    >>> null_ptr[0]
    Traceback (most recent call last):
        ....
    ValueError: NULL pointer access
    >>>

    >>> null_ptr[0] = 1234
    Traceback (most recent call last):
        ....
    ValueError: NULL pointer access
    >>>

### Thread safety without the GIL

From Python 3.13 onward, the `GIL` can be disabled on the `free-threaded build`. In ctypes, reads and writes to a single object concurrently is safe, but not across multiple objects:

> ```
> >>> number = c_int(42)
> >>> pointer_a = pointer(number)
> >>> pointer_b = pointer(number)
> ```

In the above, it's only safe for one object to read and write to the address at once if the GIL is disabled. So, `pointer_a` can be shared and written to across multiple threads, but only if `pointer_b` is not also attempting to do the same. If this is an issue, consider using a `threading.Lock` to synchronize access to memory:

> ```
> >>> import threading
> >>> lock = threading.Lock()
> >>> # Thread 1
> >>> with lock:
> ...    pointer_a.contents = 24
> >>> # Thread 2
> >>> with lock:
> ...    pointer_b.contents = 42
> ```

### Type conversions

Usually, ctypes does strict type checking. This means, if you have `POINTER(c_int)` in the `~_CFuncPtr.argtypes` list of a function or as the type of a member field in a structure definition, only instances of exactly the same type are accepted. There are some exceptions to this rule, where ctypes accepts other objects. For example, you can pass compatible array instances instead of pointer types. So, for `POINTER(c_int)`, ctypes accepts an array of c_int:

    >>> class Bar(Structure):
    ...     _fields_ = [("count", c_int), ("values", POINTER(c_int))]
    ...
    >>> bar = Bar()
    >>> bar.values = (c_int * 3)(1, 2, 3)
    >>> bar.count = 3
    >>> for i in range(bar.count):
    ...     print(bar.values[i])
    ...
    1
    2
    3
    >>>

In addition, if a function argument is explicitly declared to be a pointer type (such as `POINTER(c_int)`) in `~_CFuncPtr.argtypes`, an object of the pointed type (`c_int` in this case) can be passed to the function. ctypes will apply the required `byref` conversion in this case automatically.

These conversions are also applied to the `!_as_parameter_` attribute of an object passed for a pointer argument (see `ctypes-calling-functions-with-own-custom-data-types`). So, for a `POINTER(c_int)` argument, an object whose `!_as_parameter_` is a `c_int` instance is passed by reference, and one whose `!_as_parameter_` is a `c_int` array is accepted as well.

<div class="versionchanged">

next Objects with an `!_as_parameter_` attribute now receive the same implicit conversions for pointer arguments as the object they wrap.

</div>

To set a POINTER type field to `NULL`, you can assign `None`:

    >>> bar.values = None
    >>>

Sometimes you have instances of incompatible types. In C, you can cast one type into another type. `!ctypes` provides a `cast` function which can be used in the same way. The `Bar` structure defined above accepts `POINTER(c_int)` pointers or `c_int` arrays for its `values` field, but not instances of other types:

    >>> bar.values = (c_byte * 4)()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: incompatible types, c_byte_Array_4 instance instead of LP_c_long instance
    >>>

For these cases, the `cast` function is handy.

The `cast` function can be used to cast a ctypes instance into a pointer to a different ctypes data type. `cast` takes two parameters, a ctypes object that is or can be converted to a pointer of some kind, and a ctypes pointer type. It returns an instance of the second argument, which references the same memory block as the first argument:

    >>> a = (c_byte * 4)()
    >>> cast(a, POINTER(c_int))
    <ctypes.LP_c_long object at ...>
    >>>

So, `cast` can be used to assign to the `values` field of `Bar` the structure:

    >>> bar = Bar()
    >>> bar.values = cast((c_byte * 4)(), POINTER(c_int))
    >>> print(bar.values[0])
    0
    >>>

### Incomplete Types

*Incomplete Types* are structures, unions or arrays whose members are not yet specified. In C, they are specified by forward declarations, which are defined later:

    struct cell; /* forward declaration */

    struct cell {
        char *name;
        struct cell *next;
    };

The straightforward translation into ctypes code would be this, but it does not work:

    >>> class cell(Structure):
    ...     _fields_ = [("name", c_char_p),
    ...                 ("next", POINTER(cell))]
    ...
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 2, in cell
    NameError: name 'cell' is not defined
    >>>

because the new `class cell` is not available in the class statement itself. In `!ctypes`, we can define the `cell` class and set the `~Structure._fields_` attribute later, after the class statement:

    >>> from ctypes import *
    >>> class cell(Structure):
    ...     pass
    ...
    >>> cell._fields_ = [("name", c_char_p),
    ...                  ("next", POINTER(cell))]
    >>>

Let's try it. We create two instances of `cell`, and let them point to each other, and finally follow the pointer chain a few times:

    >>> c1 = cell()
    >>> c1.name = b"foo"
    >>> c2 = cell()
    >>> c2.name = b"bar"
    >>> c1.next = pointer(c2)
    >>> c2.next = pointer(c1)
    >>> p = c1
    >>> for i in range(8):
    ...     print(p.name, end=" ")
    ...     p = p.next[0]
    ...
    foo bar foo bar foo bar foo bar
    >>>

### Callback functions

`!ctypes` allows creating C callable function pointers from Python callables. These are sometimes called *callback functions*.

First, you must create a class for the callback function. The class knows the calling convention, the return type, and the number and types of arguments this function will receive.

The `CFUNCTYPE` factory function creates types for callback functions using the `cdecl` calling convention. On Windows, the `WINFUNCTYPE` factory function creates types for callback functions using the `stdcall` calling convention.

Both of these factory functions are called with the result type as first argument, and the callback functions expected argument types as the remaining arguments.

I will present an example here which uses the standard C library's `!qsort` function, that is used to sort items with the help of a callback function. `!qsort` will be used to sort an array of integers:

    >>> IntArray5 = c_int * 5
    >>> ia = IntArray5(5, 1, 7, 33, 99)
    >>> qsort = libc.qsort
    >>> qsort.restype = None
    >>>

`!qsort` must be called with a pointer to the data to sort, the number of items in the data array, the size of one item, and a pointer to the comparison function, the callback. The callback will then be called with two pointers to items, and it must return a negative integer if the first item is smaller than the second, a zero if they are equal, and a positive integer otherwise.

So our callback function receives pointers to integers, and must return an integer. First we create the `type` for the callback function:

    >>> CMPFUNC = CFUNCTYPE(c_int, POINTER(c_int), POINTER(c_int))
    >>>

To get started, here is a simple callback that shows the values it gets passed:

    >>> def py_cmp_func(a, b):
    ...     print("py_cmp_func", a[0], b[0])
    ...     return 0
    ...
    >>> cmp_func = CMPFUNC(py_cmp_func)
    >>>

The result:

    >>> qsort(ia, len(ia), sizeof(c_int), cmp_func)  # doctest: +LINUX
    py_cmp_func 5 1
    py_cmp_func 33 99
    py_cmp_func 7 33
    py_cmp_func 5 7
    py_cmp_func 1 7
    >>>

Now we can actually compare the two items and return a useful result:

    >>> def py_cmp_func(a, b):
    ...     print("py_cmp_func", a[0], b[0])
    ...     return a[0] - b[0]
    ...
    >>>
    >>> qsort(ia, len(ia), sizeof(c_int), CMPFUNC(py_cmp_func)) # doctest: +LINUX
    py_cmp_func 5 1
    py_cmp_func 33 99
    py_cmp_func 7 33
    py_cmp_func 1 7
    py_cmp_func 5 7
    >>>

As we can easily check, our array is sorted now:

    >>> for i in ia: print(i, end=" ")
    ...
    1 5 7 33 99
    >>>

The function factories can be used as decorator factories, so we may as well write:

    >>> @CFUNCTYPE(c_int, POINTER(c_int), POINTER(c_int))
    ... def py_cmp_func(a, b):
    ...     print("py_cmp_func", a[0], b[0])
    ...     return a[0] - b[0]
    ...
    >>> qsort(ia, len(ia), sizeof(c_int), py_cmp_func)
    py_cmp_func 5 1
    py_cmp_func 33 99
    py_cmp_func 7 33
    py_cmp_func 1 7
    py_cmp_func 5 7
    >>>

> [!NOTE]
> Make sure you keep references to `CFUNCTYPE` objects as long as they are used from C code. `!ctypes` doesn't, and if you don't, they may be garbage collected, crashing your program when a callback is made.
>
> Also, note that if the callback function is called in a thread created outside of Python's control (e.g. by the foreign code that calls the callback), ctypes creates a new dummy Python thread on every invocation. This behavior is correct for most purposes, but it means that values stored with `threading.local` will *not* survive across different callbacks, even when those calls are made from the same C thread.

Callback functions can return structures and unions by value, in addition to the simple types:

``` python
class Point(Structure):
    _fields_ = [("x", c_int), ("y", c_int)]

@CFUNCTYPE(Point)
def get_origin():
    return Point(0, 0)
```

> [!NOTE]
> A structure or union returned from a callback is copied *by value*, and `!ctypes` does not keep the returned object alive after the callback returns. If the structure contains a pointer field (such as `c_char_p` or a `POINTER` type), you must ensure that the memory it refers to stays valid for as long as the C code uses it -- the same requirement C itself imposes. Returning a pointer to memory owned by a temporary Python object will leave the C caller with a dangling pointer.

<div class="versionchanged">

next Callback functions can now return structures and unions.

</div>

### Accessing values exported from dlls

Some shared libraries not only export functions, they also export variables. An example in the Python library itself is the `Py_Version`, Python runtime version number encoded in a single constant integer.

`!ctypes` can access values like this with the `~_CData.in_dll` class methods of the type. *pythonapi* is a predefined symbol giving access to the Python C api:

    >>> version = ctypes.c_int.in_dll(ctypes.pythonapi, "Py_Version")
    >>> print(hex(version.value))
    0x30c00a0

An extended example which also demonstrates the use of pointers accesses the `PyImport_FrozenModules` pointer exported by Python.

Quoting the docs for that value:

> This pointer is initialized to point to an array of `_frozen` records, terminated by one whose members are all `NULL` or zero. When a frozen module is imported, it is searched in this table. Third-party code could play tricks with this to provide a dynamically created collection of frozen modules.

So manipulating this pointer could even prove useful. To restrict the example size, we show only how this table can be read with `!ctypes`:

    >>> from ctypes import *
    >>>
    >>> class struct_frozen(Structure):
    ...     _fields_ = [("name", c_char_p),
    ...                 ("code", POINTER(c_ubyte)),
    ...                 ("size", c_int),
    ...                 ("get_code", POINTER(c_ubyte)),  # Function pointer
    ...                ]
    ...
    >>>

We have defined the `_frozen` data type, so we can get the pointer to the table:

    >>> FrozenTable = POINTER(struct_frozen)
    >>> table = FrozenTable.in_dll(pythonapi, "_PyImport_FrozenBootstrap")
    >>>

Since `table` is a `pointer` to the array of `struct_frozen` records, we can iterate over it, but we just have to make sure that our loop terminates, because pointers have no size. Sooner or later it would probably crash with an access violation or whatever, so it's better to break out of the loop when we hit the `NULL` entry:

    >>> for item in table:
    ...     if item.name is None:
    ...         break
    ...     print(item.name.decode("ascii"), item.size)
    ...
    _frozen_importlib 31764
    _frozen_importlib_external 41499
    zipimport 12345
    >>>

The fact that standard Python has a frozen module and a frozen package (indicated by the negative `size` member) is not well known, it is only used for testing. Try it out with `import __hello__` for example.

### Surprises

There are some edges in `!ctypes` where you might expect something other than what actually happens.

Consider the following example:

    >>> from ctypes import *
    >>> class POINT(Structure):
    ...     _fields_ = ("x", c_int), ("y", c_int)
    ...
    >>> class RECT(Structure):
    ...     _fields_ = ("a", POINT), ("b", POINT)
    ...
    >>> p1 = POINT(1, 2)
    >>> p2 = POINT(3, 4)
    >>> rc = RECT(p1, p2)
    >>> print(rc.a.x, rc.a.y, rc.b.x, rc.b.y)
    1 2 3 4
    >>> # now swap the two points
    >>> rc.a, rc.b = rc.b, rc.a
    >>> print(rc.a.x, rc.a.y, rc.b.x, rc.b.y)
    3 4 3 4
    >>>

Hm. We certainly expected the last statement to print `3 4 1 2`. What happened? Here are the steps of the `rc.a, rc.b = rc.b, rc.a` line above:

    >>> temp0, temp1 = rc.b, rc.a
    >>> rc.a = temp0
    >>> rc.b = temp1
    >>>

Note that `temp0` and `temp1` are objects still using the internal buffer of the `rc` object above. So executing `rc.a = temp0` copies the buffer contents of `temp0` into `rc` 's buffer. This, in turn, changes the contents of `temp1`. So, the last assignment `rc.b = temp1`, doesn't have the expected effect.

Keep in mind that retrieving sub-objects from Structure, Unions, and Arrays doesn't *copy* the sub-object, instead it retrieves a wrapper object accessing the root-object's underlying buffer.

Another example that may behave differently from what one would expect is this:

    >>> s = c_char_p()
    >>> s.value = b"abc def ghi"
    >>> s.value
    b'abc def ghi'
    >>> s.value is s.value
    False
    >>>

> [!NOTE]
> Objects instantiated from `c_char_p` can only have their value set to bytes or integers.

Why is it printing `False`? ctypes instances are objects containing a memory block plus some `descriptor`s accessing the contents of the memory. Storing a Python object in the memory block does not store the object itself, instead the `contents` of the object is stored. Accessing the contents again constructs a new Python object each time!

### Variable-sized data types

`!ctypes` provides some support for variable-sized arrays and structures.

The `resize` function can be used to resize the memory buffer of an existing ctypes object. The function takes the object as first argument, and the requested size in bytes as the second argument. The memory block cannot be made smaller than the natural memory block specified by the objects type, a `ValueError` is raised if this is tried:

    >>> short_array = (c_short * 4)()
    >>> print(sizeof(short_array))
    8
    >>> resize(short_array, 4)
    Traceback (most recent call last):
        ...
    ValueError: minimum size is 8
    >>> resize(short_array, 32)
    >>> sizeof(short_array)
    32
    >>> sizeof(type(short_array))
    8
    >>>

This is nice and fine, but how would one access the additional elements contained in this array? Since the type still only knows about 4 elements, we get errors accessing other elements:

    >>> short_array[:]
    [0, 0, 0, 0]
    >>> short_array[7]
    Traceback (most recent call last):
        ...
    IndexError: invalid index
    >>>

Another way to use variable-sized data types with `!ctypes` is to use the dynamic nature of Python, and (re-)define the data type after the required size is already known, on a case by case basis.

## ctypes reference

### Loading shared libraries

There are several ways to load shared libraries into the Python process. One way is to instantiate `CDLL` or one of its subclasses:

<div class="CDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, use_last_error=False, winmode=None)">

Represents a loaded shared library.

Functions in this library use the standard C calling convention, and are assumed to return `int`. The Python `global interpreter lock` is released before calling any function exported by these libraries, and reacquired afterwards. For different function behavior, use a subclass: `~ctypes.OleDLL`, `~ctypes.WinDLL`, or `~ctypes.PyDLL`.

If you have an existing `handle <ctypes.CDLL._handle>` to an already loaded shared library, it can be passed as the *handle* argument to wrap the opened library in a new `!CDLL` object. In this case, *name* is only used to set the `~ctypes.CDLL._name` attribute, but it may be adjusted and/or validated.

If *handle* is `None`, the underlying platform's `dlopen(3)` or [LoadLibraryExW](https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexw) function is used to load the library into the process, and to get a handle to it.

*name* is the pathname of the shared library to open. If *name* does not contain a path separator, the library is found in a platform-specific way.

On Windows, the `.DLL` suffix may be missing. (For details, see [LoadLibraryExW](https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexw) documentation.) Other platform-specific prefixes and suffixes (for example, `lib`, `.so`, `.dylib`, or version numbers) must be present in *name*; they are not added automatically. See `ctypes-finding-shared-libraries` for more information.

On non-Windows systems, *name* can be `None`. In this case, `!dlopen` is called with `NULL`, which opens the main program as a "library". (Some systems do the same if *name* is empty; `None`/`NULL` is more portable.)

<div class="admonition">

CPython implementation detail

Since CPython is linked to `libc`, a `None` *name* is often used to access the C standard library:

    >>> printf = ctypes.CDLL(None).printf
    >>> printf.argtypes = [ctypes.c_char_p]
    >>> printf(b"hello\n")
    hello
    6

To access the Python C API, prefer `ctypes.pythonapi` which works across platforms.

</div>

The *mode* parameter can be used to specify how the library is loaded. For details, consult the `dlopen(3)` manpage. On Windows, *mode* is ignored. On posix systems, RTLD_NOW is always added, and is not configurable.

The *use_errno* parameter, when set to true, enables a ctypes mechanism that allows accessing the system `errno` error number in a safe way. `!ctypes` maintains a thread-local copy of the system's `errno` variable; if you call foreign functions created with `use_errno=True` then the `errno` value before the function call is swapped with the ctypes private copy, the same happens immediately after the function call. This mechanism only works for libraries that use the same C runtime library as the interpreter itself. A library linked against a different C runtime (for example, a different CRT on Windows) has its own `errno`, which `!ctypes` cannot observe.

The function `ctypes.get_errno` returns the value of the ctypes private copy, and the function `ctypes.set_errno` changes the ctypes private copy to a new value and returns the former value.

The *use_last_error* parameter, when set to true, enables the same mechanism for the Windows error code which is managed by the `GetLastError` and `!SetLastError` Windows API functions; `ctypes.get_last_error` and `ctypes.set_last_error` are used to request and change the ctypes private copy of the windows error code.

The *winmode* parameter is used on Windows to specify how the library is loaded (since *mode* is ignored). It takes any value that is valid for the Win32 API [LoadLibraryExW](https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexw) flags parameter. When omitted, the default is to use the flags that result in the most secure DLL load, which avoids issues such as DLL hijacking. Passing the full path to the DLL is the safest way to ensure the correct library and dependencies are loaded.

On Windows creating a `CDLL` instance may fail even if the DLL name exists. When a dependent DLL of the loaded DLL is not found, a `OSError` error is raised with the message *"\[WinError 126\] The specified module could not be found".* This error message does not contain the name of the missing DLL because the Windows API does not return this information making this error hard to diagnose. To resolve this error and determine which DLL is not found, you need to find the list of dependent DLLs and determine which one is not found using Windows debugging and tracing tools.

<div class="seealso">

[Microsoft DUMPBIN tool](https://learn.microsoft.com/en-us/cpp/build/reference/dumpbin-reference?view=msvc-170) -- A tool to find DLL dependents.

</div>

<div class="versionchanged">

3.8 Added *winmode* parameter.

</div>

<div class="versionchanged">

3.12

The *name* parameter can now be a `path-like object`.

</div>

Instances of this class have no public methods. Functions exported by the shared library can be accessed as attributes or by index. Please note that accessing the function through an attribute caches the result and therefore accessing it repeatedly returns the same object each time. On the other hand, accessing it through an index returns a new object each time:

    >>> from ctypes import CDLL
    >>> libc = CDLL("libc.so.6")  # On Linux
    >>> libc.time == libc.time
    True
    >>> libc['time'] == libc['time']
    False

The following public attributes are available. Their name starts with an underscore to not clash with exported function names:

<div class="attribute">

<span id="handle">handle</span>

The system handle used to access the library.

</div>

<div class="attribute">

<span id="name">name</span>

The name of the library passed in the constructor.

</div>

</div>

<div class="OleDLL">

See `~ctypes.CDLL`, the superclass, for common information.

Functions in this library use the `stdcall` calling convention, and are assumed to return the windows specific `HRESULT` code. `HRESULT` values contain information specifying whether the function call failed or succeeded, together with additional error code. If the return value signals a failure, an `OSError` is automatically raised.

<div class="availability">

Windows

</div>

<div class="versionchanged">

3.3 `WindowsError` used to be raised, which is now an alias of `OSError`.

</div>

</div>

<div class="WinDLL">

See `~ctypes.CDLL`, the superclass, for common information.

Functions in these libraries use the `stdcall` calling convention, and are assumed to return `int` by default.

<div class="availability">

Windows

</div>

</div>

<div class="PyDLL">

See `~ctypes.CDLL`, the superclass, for common information.

When functions in this library are called, the Python GIL is *not* released during the function call, and after the function execution the Python error flag is checked. If the error flag is set, a Python exception is raised.

Thus, this is only useful to call Python C API functions directly.

</div>

<div class="data">

RTLD_GLOBAL

Flag to use as *mode* parameter. On platforms where this flag is not available, it is defined as the integer zero.

</div>

<div class="data">

RTLD_LOCAL

Flag to use as *mode* parameter. On platforms where this is not available, it is the same as *RTLD_GLOBAL*.

</div>

<div class="data">

DEFAULT_MODE

The default mode which is used to load shared libraries. On OSX 10.3, this is *RTLD_GLOBAL*, otherwise it is the same as *RTLD_LOCAL*.

</div>

Shared libraries can also be loaded by using one of the prefabricated objects, which are instances of the `LibraryLoader` class, either by calling the `~LibraryLoader.LoadLibrary` method, or by retrieving the library as attribute of the loader instance.

<div class="LibraryLoader(dlltype)">

Class which loads shared libraries. *dlltype* should be one of the `CDLL`, `PyDLL`, `WinDLL`, or `OleDLL` types.

`!__getattr__` has special behavior: It allows loading a shared library by accessing it as attribute of a library loader instance. The result is cached, so repeated attribute accesses return the same library each time.

<div class="method">

LoadLibrary(name)

Load a shared library into the process and return it. This method always returns a new instance of the library.

</div>

</div>

These prefabricated library loaders are available:

> <div class="data">
>
> cdll
>
> Creates `CDLL` instances.
>
> </div>
>
> <div class="data">
>
> windll
>
> Creates `WinDLL` instances.
>
> <div class="availability">
>
> Windows
>
> </div>
>
> </div>
>
> <div class="data">
>
> oledll
>
> Creates `OleDLL` instances.
>
> <div class="availability">
>
> Windows
>
> </div>
>
> </div>
>
> <div class="data">
>
> pydll
>
> Creates `PyDLL` instances.
>
> </div>
>
> <div class="data">
>
> pythonapi
>
> An instance of `PyDLL` that exposes Python C API functions as attributes. Note that all these functions are assumed to return C `int`, which is of course not always the truth, so you have to assign the correct `!restype` attribute to use these functions.
>
> > [!NOTE]
> > If the Python interpreter is statically linked, this may be `None`.
>
> </div>

<div class="audit-event">

ctypes.dlopen name ctypes.LibraryLoader

Loading a library through any of these objects raises an `auditing event <auditing>` `ctypes.dlopen` with string argument `name`, the name used to load the library.

</div>

<div class="audit-event">

ctypes.dlsym library,name ctypes.LibraryLoader

Accessing a function on a loaded library raises an auditing event `ctypes.dlsym` with arguments `library` (the library object) and `name` (the symbol's name as a string or integer).

</div>

<div class="audit-event">

ctypes.dlsym/handle handle,name ctypes.LibraryLoader

In cases when only the library handle is available rather than the object, accessing a function raises an auditing event `ctypes.dlsym/handle` with arguments `handle` (the raw library handle) and `name`.

</div>

### Finding shared libraries

When programming in a compiled language, shared libraries are accessed when compiling/linking a program, and when the program is run. The programmer specifies a short name; the C compiler, linker, and runtime dynamic library loader then interact in system-specific ways to find the filename of the library to load.

While the mapping from short names to filenames is not consistently exposed by platforms, the `!ctypes.util` module provides a function, `!find_library`, that attempts to match it. However, as backwards compatibility concerns make it difficult to adjust its behavior for new platforms and configurations, the function is `soft deprecated`.

If wrapping a shared library with `!ctypes`, consider determining the shared library name at development time, and hardcoding it into the wrapper module instead of using `!find_library` to locate the library at runtime. Also consider adding a configuration option or environment variable to let users select a library to use, and then perhaps use `!find_library` as a default or fallback.

<div class="function" data-module="ctypes.util">

find_library(name)

Try to find a library and return a pathname.

*name* is the "short" library name without any prefix like `lib`, suffix like `.so`, `.dylib` or version number. (This is the form used for the posix linker option `!-l`.) The result is in a format suitable for passing to `~ctypes.CDLL`.

If no library can be found, return `None`.

The exact functionality is system dependent, and is *not guaranteed* to match the behavior of the compiler, linker, and loader used for (or by) Python. It is recommended to only use this function as a default or fallback,

<div class="soft-deprecated">

3.15

This function is kept for use in cases where it works, but not expected to be updated for additional platforms and configurations.

</div>

</div>

On Linux, `!find_library` tries to run external programs (`/sbin/ldconfig`, `gcc`, `objdump` and `ld`) to find the library file. If the output of these programs does not correspond to the dynamic linker used by Python, the result of this function may be misleading.

<div class="versionchanged">

3.6 On Linux, the value of the environment variable `LD_LIBRARY_PATH` is used when searching for libraries, if a library cannot be found by any other means.

</div>

Here are some examples:

    >>> from ctypes.util import find_library
    >>> find_library("m")
    'libm.so.6'
    >>> find_library("c")
    'libc.so.6'
    >>> find_library("bz2")
    'libbz2.so.1.0'
    >>>

On macOS and Android, `!find_library` uses the system's standard naming schemes and paths to locate the library, and returns a full pathname if successful:

    >>> from ctypes.util import find_library
    >>> find_library("c")
    '/usr/lib/libc.dylib'
    >>> find_library("m")
    '/usr/lib/libm.dylib'
    >>> find_library("bz2")
    '/usr/lib/libbz2.dylib'
    >>> find_library("AGL")
    '/System/Library/Frameworks/AGL.framework/AGL'
    >>>

On Windows, `!find_library` searches along the system search path, and returns the full pathname, but since there is no predefined naming scheme a call like `find_library("c")` will fail and return `None`.

<div class="function" data-module="ctypes.util">

find_msvcrt()

Returns the filename of the VC runtime library used by Python, and by the extension modules.

If the name of the library cannot be determined, `None` is returned. Notably, this will happen for recent versions of the VC runtime library, which are not directly loadable.

If you need to free memory, for example, allocated by an extension module with a call to the `free(void *)`, it is important that you use the function in the same library that allocated the memory.

<div class="availability">

Windows

</div>

<div class="soft-deprecated">

3.16 This function now always returns `None`, as there are no more VC runtime DLLs that are a single file and supported by Microsoft.

</div>

</div>

### Listing loaded shared libraries

When writing code that relies on code loaded from shared libraries, it can be useful to know which shared libraries have already been loaded into the current process.

The `!ctypes.util` module provides the `~ctypes.util.dllist` function, which calls the different APIs provided by the various platforms to help determine which shared libraries have already been loaded into the current process.

The exact output of this function will be system dependent. On most platforms, the first entry of this list represents the current process itself, which may be an empty string. For example, on glibc-based Linux, the return may look like:

    >>> from ctypes.util import dllist
    >>> dllist()
    ['', 'linux-vdso.so.1', '/lib/x86_64-linux-gnu/libm.so.6', '/lib/x86_64-linux-gnu/libc.so.6', ... ]

### Foreign functions

As explained in the previous section, foreign functions can be accessed as attributes of loaded shared libraries. The function objects created in this way by default accept any number of arguments, accept any ctypes data instances as arguments, and return the default result type specified by the library loader.

They are instances of a private local class `!_FuncPtr` (not exposed in `!ctypes`) which inherits from the private `_CFuncPtr` class:

<div class="doctest">

\>\>\> import ctypes \>\>\> lib = ctypes.CDLL(None) \>\>\> issubclass(lib.\_FuncPtr, ctypes.\_CFuncPtr) True \>\>\> lib.\_FuncPtr is ctypes.\_CFuncPtr False

</div>

<div class="_CFuncPtr">

Base class for C callable foreign functions.

Instances of foreign functions are also C compatible data types; they represent C function pointers.

This behavior can be customized by assigning to special attributes of the foreign function object.

<div class="attribute">

restype

Assign a ctypes type to specify the result type of the foreign function. Use `None` for `void`, a function not returning anything.

It is possible to assign a callable Python object that is not a ctypes type, in this case the function is assumed to return a C `int`, and the callable will be called with this integer, allowing further processing or error checking. Using this is deprecated, for more flexible post processing or error checking use a ctypes data type as `!restype` and assign a callable to the `errcheck` attribute.

</div>

<div class="attribute">

argtypes

Assign a tuple of ctypes types to specify the argument types that the function accepts. `TypeError` is raised if the function is called with fewer arguments than the length of this tuple. Functions using the `stdcall` calling convention also raise `TypeError` if called with more arguments; functions using the `cdecl` calling convention accept additional, unspecified arguments as well.

When a foreign function is called, each actual argument is passed to the `~_CData.from_param` class method of the items in the `argtypes` tuple, this method allows adapting the actual argument to an object that the foreign function accepts. For example, a `c_char_p` item in the `argtypes` tuple will convert a string passed as argument into a bytes object using ctypes conversion rules.

New: It is now possible to put items in argtypes which are not ctypes types, but each item must have a `~_CData.from_param` method which returns a value usable as argument (integer, string, ctypes instance). This allows defining adapters that can adapt custom objects as function parameters.

</div>

<div class="attribute">

errcheck

Assign a Python function or another callable to this attribute. The callable will be called with three or more arguments:

<div class="function" data-noindex="" data-module="">

callable(result, func, arguments)

*result* is what the foreign function returns, as specified by the `!restype` attribute.

*func* is the foreign function object itself, this allows reusing the same callable object to check or post process the results of several functions.

*arguments* is a tuple containing the parameters passed to the function call, including any output parameters that `!ctypes` created for the call; this allows specializing the behavior on the arguments used.

</div>

If the callable returns the *arguments* tuple it received, unchanged, `!ctypes` continues its normal processing of the result, which includes returning output parameters (see `ctypes-function-prototypes`). Any other object returned by the callable is returned from the foreign function call instead. The callable can also check the result value and raise an exception if the foreign function call failed.

For example, this callable raises an exception when the foreign function signals failure with a negative return value, and otherwise leaves the result to `!ctypes`:

    def errcheck(result, func, arguments):
        if result < 0:
            raise RuntimeError(f"call failed with error {result}")
        return arguments

This one instead replaces the result, converting a C `int` used as a boolean into a `bool`:

    def errcheck(result, func, arguments):
        return result != 0

</div>

</div>

<div class="audit-event">

ctypes.set_exception code foreign-functions

On Windows, when a foreign function call raises a system exception (for example, due to an access violation), it will be captured and replaced with a suitable Python exception. Further, an auditing event `ctypes.set_exception` with argument `code` will be raised, allowing an audit hook to replace the exception with its own.

</div>

<div class="audit-event">

ctypes.call_function func_pointer,arguments foreign-functions

Some ways to invoke foreign function calls as well as some of the functions in this module may raise an auditing event `ctypes.call_function` with arguments `function pointer` and `arguments`.

</div>

### Function prototypes

Foreign functions can also be created by instantiating function prototypes. Function prototypes are similar to function prototypes in C; they describe a function (return type, argument types, calling convention) without defining an implementation. The factory functions must be called with the desired result type and the argument types of the function, and can be used as decorator factories, and as such, be applied to functions through the `@wrapper` syntax. See `ctypes-callback-functions` for examples.

<div class="function">

CFUNCTYPE(restype, \*argtypes, use_errno=False, use_last_error=False)

The returned function prototype creates functions that use the standard C calling convention. The function will release the GIL during the call. If *use_errno* is set to true, the ctypes private copy of the system `errno` variable is exchanged with the real `errno` value before and after the call; *use_last_error* does the same for the Windows error code.

</div>

<div class="function">

WINFUNCTYPE(restype, \*argtypes, use_errno=False, use_last_error=False)

The returned function prototype creates functions that use the `stdcall` calling convention. The function will release the GIL during the call. *use_errno* and *use_last_error* have the same meaning as above.

<div class="availability">

Windows

</div>

</div>

<div class="function">

PYFUNCTYPE(restype, \*argtypes)

The returned function prototype creates functions that use the Python calling convention. Unlike `CFUNCTYPE` and `WINFUNCTYPE`, ctypes does *not* release the GIL before calling the foreign function, so the function may safely use the Python C API. Note that this does not prevent the GIL from being released *during* the call.

</div>

Function prototypes created by these factory functions can be instantiated in different ways, depending on the type and number of the parameters in the call:

<div class="function" data-noindex="" data-module="">

prototype(address)

Returns a foreign function at the specified address which must be an integer.

</div>

<div class="function" data-noindex="" data-module="">

prototype(callable)

Create a C callable function (a callback function) from a Python *callable*.

</div>

<div class="function" data-noindex="" data-module="">

prototype(func_spec\[, paramflags\])

Returns a foreign function exported by a shared library. *func_spec* must be a 2-tuple `(name_or_ordinal, library)`. The first item is the name of the exported function as string, or the ordinal of the exported function as small integer. The second item is the shared library instance.

</div>

<div class="function" data-noindex="" data-module="">

prototype(vtbl_index, name\[, paramflags\[, iid\]\])

Returns a foreign function that will call a COM method. *vtbl_index* is the index into the virtual function table, a small non-negative integer. *name* is name of the COM method. *iid* is an optional pointer to the interface identifier which is used in extended error reporting.

If *iid* is not specified, an `OSError` is raised if the COM method call fails. If *iid* is specified, a `~ctypes.COMError` is raised instead.

COM methods use a special calling convention: They require a pointer to the COM interface as first argument, in addition to those parameters that are specified in the `!argtypes` tuple.

<div class="availability">

Windows

</div>

</div>

The optional *paramflags* parameter creates foreign function wrappers with much more functionality than the features described above.

*paramflags* must be a tuple of the same length as `~_CFuncPtr.argtypes`.

Each item in this tuple contains further information about a parameter, it must be a tuple containing one, two, or three items.

The first item is an integer containing a combination of direction flags for the parameter:

> 1  
> Specifies an input parameter to the function.
>
> 2  
> Output parameter. The foreign function fills in a value.
>
> 4  
> Input parameter which defaults to the integer zero.

The optional second item is the parameter name as string. If this is specified, the foreign function can be called with named parameters.

The optional third item is the default value for this parameter.

The following example demonstrates how to wrap the Windows `MessageBoxW` function so that it supports default parameters and named arguments. The C declaration from the windows header file is this:

    WINUSERAPI int WINAPI
    MessageBoxW(
        HWND hWnd,
        LPCWSTR lpText,
        LPCWSTR lpCaption,
        UINT uType);

Here is the wrapping with `!ctypes`:

    >>> from ctypes import c_int, WINFUNCTYPE, windll
    >>> from ctypes.wintypes import HWND, LPCWSTR, UINT
    >>> prototype = WINFUNCTYPE(c_int, HWND, LPCWSTR, LPCWSTR, UINT)
    >>> paramflags = (1, "hwnd", 0), (1, "text", "Hi"), (1, "caption", "Hello from ctypes"), (1, "flags", 0)
    >>> MessageBox = prototype(("MessageBoxW", windll.user32), paramflags)

The `MessageBox` foreign function can now be called in these ways:

    >>> MessageBox()
    >>> MessageBox(text="Spam, spam, spam")
    >>> MessageBox(flags=2, text="foo bar")

A second example demonstrates output parameters. The win32 `GetWindowRect` function retrieves the dimensions of a specified window by copying them into `RECT` structure that the caller has to supply. Here is the C declaration:

    WINUSERAPI BOOL WINAPI
    GetWindowRect(
         HWND hWnd,
         LPRECT lpRect);

Here is the wrapping with `!ctypes`:

    >>> from ctypes import POINTER, WINFUNCTYPE, windll, WinError
    >>> from ctypes.wintypes import BOOL, HWND, RECT
    >>> prototype = WINFUNCTYPE(BOOL, HWND, POINTER(RECT))
    >>> paramflags = (1, "hwnd"), (2, "lprect")
    >>> GetWindowRect = prototype(("GetWindowRect", windll.user32), paramflags)
    >>>

Functions with output parameters will automatically return the output parameter value if there is a single one, or a tuple containing the output parameter values when there are more than one, so the GetWindowRect function now returns a RECT instance, when called.

Output parameters can be combined with the `~_CFuncPtr.errcheck` protocol to do further output processing and error checking. The win32 `GetWindowRect` api function returns a `BOOL` to signal success or failure, so this function could do the error checking, and raises an exception when the api call failed:

    >>> def errcheck(result, func, args):
    ...     if not result:
    ...         raise WinError()
    ...     return args
    ...
    >>> GetWindowRect.errcheck = errcheck
    >>>

If the `~_CFuncPtr.errcheck` function returns the argument tuple it receives unchanged, `!ctypes` continues the normal processing it does on the output parameters. If you want to return a tuple of window coordinates instead of a `RECT` instance, you can retrieve the fields in the function and return them instead, the normal processing will no longer take place:

    >>> def errcheck(result, func, args):
    ...     if not result:
    ...         raise WinError()
    ...     rc = args[1]
    ...     return rc.left, rc.top, rc.bottom, rc.right
    ...
    >>> GetWindowRect.errcheck = errcheck
    >>>

### Utility functions

<div class="function">

addressof(obj)

Returns the address of the memory buffer as integer. *obj* must be an instance of a ctypes type.

<div class="audit-event">

ctypes.addressof obj ctypes.addressof

</div>

</div>

<div class="function">

alignment(obj_or_type)

Returns the alignment requirements of a ctypes type. *obj_or_type* must be a ctypes type or instance.

</div>

<div class="function">

byref(obj\[, offset\])

Returns a light-weight pointer to *obj*, which must be an instance of a ctypes type. *offset* defaults to zero, and must be an integer that will be added to the internal pointer value.

`byref(obj, offset)` corresponds to this C code:

    (((char *)&obj) + offset)

The returned object can only be used as a foreign function call parameter. It behaves similar to `pointer(obj)`, but the construction is a lot faster.

</div>

<div class="function">

CopyComPointer(src, dst)

Copies a COM pointer from *src* to *dst* and returns the Windows specific `!HRESULT` value.

If *src* is not `NULL`, its `AddRef` method is called, incrementing the reference count.

In contrast, the reference count of *dst* will not be decremented before assigning the new value. Unless *dst* is `NULL`, the caller is responsible for decrementing the reference count by calling its `Release` method when necessary.

<div class="availability">

Windows

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

cast(obj, type)

This function is similar to the cast operator in C. It returns a new instance of *type* which points to the same memory block as *obj*. *type* must be a pointer type, and *obj* must be an object that can be interpreted as a pointer.

</div>

<div class="function">

create_string_buffer(init, size=None) create_string_buffer(size)

This function creates a mutable character buffer. The returned object is a ctypes array of `c_char`.

If *size* is given (and not `None`), it must be an `int`. It specifies the size of the returned array.

If the *init* argument is given, it must be `bytes`. It is used to initialize the array items. Bytes not initialized this way are set to zero (NUL).

If *size* is not given (or if it is `None`), the buffer is made one element larger than *init*, effectively adding a NUL terminator.

If both arguments are given, *size* must not be less than `len(init)`.

> [!WARNING]
> If *size* is equal to `len(init)`, a NUL terminator is not added. Do not treat such a buffer as a C string.

For example:

    >>> bytes(create_string_buffer(2))
    b'\x00\x00'
    >>> bytes(create_string_buffer(b'ab'))
    b'ab\x00'
    >>> bytes(create_string_buffer(b'ab', 2))
    b'ab'
    >>> bytes(create_string_buffer(b'ab', 4))
    b'ab\x00\x00'
    >>> bytes(create_string_buffer(b'abcdef', 2))
    Traceback (most recent call last):
       ...
    ValueError: byte string too long

<div class="audit-event">

ctypes.create_string_buffer init,size ctypes.create_string_buffer

</div>

</div>

<div class="function">

create_unicode_buffer(init, size=None) create_unicode_buffer(size)

This function creates a mutable unicode character buffer. The returned object is a ctypes array of `c_wchar`.

The function takes the same arguments as `~create_string_buffer` except *init* must be a string and *size* counts `c_wchar`.

<div class="audit-event">

ctypes.create_unicode_buffer init,size ctypes.create_unicode_buffer

</div>

</div>

<div class="function">

DllCanUnloadNow()

This function is a hook which allows implementing in-process COM servers with ctypes. It is called from the DllCanUnloadNow function that the <span id="ctypes">ctypes</span> extension dll exports.

<div class="availability">

Windows

</div>

</div>

<div class="function">

DllGetClassObject()

This function is a hook which allows implementing in-process COM servers with ctypes. It is called from the DllGetClassObject function that the `_ctypes` extension dll exports.

<div class="availability">

Windows

</div>

</div>

<div class="function" data-module="ctypes.util">

dllist()

Try to provide a list of paths of the shared libraries loaded into the current process. These paths are not normalized or processed in any way. The function can raise `OSError` if the underlying platform APIs fail. The exact functionality is system dependent.

On most platforms, the first element of the list represents the current executable file. It may be an empty string.

<div class="availability">

Windows, macOS, iOS, glibc, BSD libc, musl

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

FormatError(\[code\])

Returns a textual description of the error code *code*. If no error code is specified, the last error code is used by calling the Windows API function `GetLastError`.

<div class="availability">

Windows

</div>

</div>

<div class="function">

GetLastError()

Returns the last error code set by Windows in the calling thread. This function calls the Windows `GetLastError()` function directly, it does not return the ctypes-private copy of the error code.

<div class="availability">

Windows

</div>

</div>

<div class="function">

get_errno()

Returns the current value of the ctypes-private copy of the system `errno` variable in the calling thread.

<div class="audit-event">

ctypes.get_errno "" ctypes.get_errno

</div>

</div>

<div class="function">

get_last_error()

Returns the current value of the ctypes-private copy of the system `!LastError` variable in the calling thread.

<div class="availability">

Windows

</div>

<div class="audit-event">

ctypes.get_last_error "" ctypes.get_last_error

</div>

</div>

<div class="function">

memmove(dst, src, count)

Same as the standard C memmove library function: copies *count* bytes from *src* to *dst*. *dst* and *src* must be integers or ctypes instances that can be converted to pointers.

</div>

<div class="function">

memset(dst, c, count)

Same as the standard C memset library function: fills the memory block at address *dst* with *count* bytes of value *c*. *dst* must be an integer specifying an address, or a ctypes instance.

</div>

<div class="function">

POINTER(type, /)

Create or return a ctypes pointer type. Pointer types are cached and reused internally, so calling this function repeatedly is cheap. *type* must be a ctypes type.

<div class="impl-detail">

The resulting pointer type is cached in the `__pointer_type__` attribute of *type*. It is possible to set this attribute before the first call to `POINTER` in order to set a custom pointer type. However, doing this is discouraged: manually creating a suitable pointer type is difficult without relying on implementation details that may change in future Python versions.

</div>

</div>

<div class="function">

pointer(obj, /)

Create a new pointer instance, pointing to *obj*. The returned object is of the type `POINTER(type(obj))`.

Note: If you just want to pass a pointer to an object to a foreign function call, you should use `byref(obj)` which is much faster.

</div>

<div class="function">

resize(obj, size)

This function resizes the internal memory buffer of *obj*, which must be an instance of a ctypes type. It is not possible to make the buffer smaller than the native size of the objects type, as given by `sizeof(type(obj))`, but it is possible to enlarge the buffer.

</div>

<div class="function">

set_errno(value)

Set the current value of the ctypes-private copy of the system `errno` variable in the calling thread to *value* and return the previous value.

<div class="audit-event">

ctypes.set_errno errno ctypes.set_errno

</div>

</div>

<div class="function">

set_last_error(value)

Sets the current value of the ctypes-private copy of the system `!LastError` variable in the calling thread to *value* and return the previous value.

<div class="availability">

Windows

</div>

<div class="audit-event">

ctypes.set_last_error error ctypes.set_last_error

</div>

</div>

<div class="function">

sizeof(obj_or_type)

Returns the size in bytes of a ctypes type or instance memory buffer. Does the same as the C `sizeof` operator.

</div>

<div class="function">

string_at(ptr, size=-1)

Return the byte string at *void \*ptr*. If *size* is specified, it is used as size, otherwise the string is assumed to be zero-terminated.

<div class="audit-event">

ctypes.string_at ptr,size ctypes.string_at

</div>

</div>

<div class="function">

WinError(code=None, descr=None)

Creates an instance of `OSError`. If *code* is not specified, `GetLastError` is called to determine the error code. If *descr* is not specified, `FormatError` is called to get a textual description of the error.

<div class="availability">

Windows

</div>

<div class="versionchanged">

3.3 An instance of `WindowsError` used to be created, which is now an alias of `OSError`.

</div>

</div>

<div class="function">

wstring_at(ptr, size=-1)

Return the wide-character string at *void \*ptr*. If *size* is specified, it is used as the number of characters of the string, otherwise the string is assumed to be zero-terminated.

<div class="audit-event">

ctypes.wstring_at ptr,size ctypes.wstring_at

</div>

</div>

<div class="function">

memoryview_at(ptr, size, readonly=False)

Return a `memoryview` object of length *size* that references memory starting at *void \*ptr*.

If *readonly* is true, the returned `!memoryview` object can not be used to modify the underlying memory. (Changes made by other means will still be reflected in the returned object.)

This function is similar to `string_at` with the key difference of not making a copy of the specified memory. It is a semantically equivalent (but more efficient) alternative to `memoryview((c_byte * size).from_address(ptr))`. (While `~_CData.from_address` only takes integers, *ptr* can also be given as a `ctypes.POINTER` or a `~ctypes.byref` object.)

<div class="audit-event">

ctypes.memoryview_at address,size,readonly

</div>

<div class="versionadded">

3.14

</div>

</div>

### Data types

<div class="_CData">

This non-public class is the common base class of all ctypes data types. Among other things, all ctypes type instances contain a memory block that hold C compatible data; the address of the memory block is returned by the `addressof` helper function. Another instance variable is exposed as `_objects`; this contains other Python objects that need to be kept alive in case the memory block contains pointers.

Common methods of ctypes data types, these are all class methods (to be exact, they are methods of the `metaclass`):

<div class="method">

<span id="cdata.from_buffer">CData.from_buffer</span>(source\[, offset\])

This method returns a ctypes instance that shares the buffer of the *source* object. The *source* object must support the writeable buffer interface. The optional *offset* parameter specifies an offset into the source buffer in bytes; the default is zero. If the source buffer is not large enough a `ValueError` is raised.

<div class="audit-event">

ctypes.cdata/buffer pointer,size,offset ctypes.\_CData.from_buffer

</div>

</div>

<div class="method">

<span id="cdata.from_buffer_copy">CData.from_buffer_copy</span>(source\[, offset\])

This method creates a ctypes instance, copying the buffer from the *source* object buffer which must be readable. The optional *offset* parameter specifies an offset into the source buffer in bytes; the default is zero. If the source buffer is not large enough a `ValueError` is raised.

<div class="audit-event">

ctypes.cdata/buffer pointer,size,offset ctypes.\_CData.from_buffer_copy

</div>

</div>

<div class="method">

from_address(address)

This method returns a ctypes type instance using the memory specified by *address* which must be an integer.

<div class="audit-event">

ctypes.cdata address ctypes.\_CData.from_address

This method, and others that indirectly call this method, raises an `auditing event <auditing>` `ctypes.cdata` with argument `address`.

</div>

</div>

<div class="method">

from_param(obj)

This method adapts *obj* to a ctypes type. It is called with the actual object used in a foreign function call when the type is present in the foreign function's `~_CFuncPtr.argtypes` tuple; it must return an object that can be used as a function call parameter.

All ctypes data types have a default implementation of this classmethod that normally returns *obj* if that is an instance of the type. Some types accept other objects as well.

</div>

<div class="method">

in_dll(library, name)

This method returns a ctypes type instance exported by a shared library. *name* is the name of the symbol that exports the data, *library* is the loaded shared library.

</div>

Common class variables of ctypes data types:

<div class="attribute">

\_\_pointer_type\_\_

The pointer type that was created by calling `POINTER` for corresponding ctypes data type. If a pointer type was not yet created, the attribute is missing.

<div class="versionadded">

3.14

</div>

</div>

Common instance variables of ctypes data types:

<div class="attribute">

<span id="b_base">b_base</span>\_

Sometimes ctypes data instances do not own the memory block they contain, instead they share part of the memory block of a base object. The `_b_base_` read-only member is the root ctypes object that owns the memory block.

</div>

<div class="attribute">

<span id="b_needsfree">b_needsfree</span>\_

This read-only variable is true when the ctypes data instance has allocated the memory block itself, false otherwise.

</div>

<div class="attribute">

<span id="objects">objects</span>

This member is either `None` or a dictionary containing Python objects that need to be kept alive so that the memory block contents is kept valid. This object is only exposed for debugging; never modify the contents of this dictionary.

</div>

</div>

### Fundamental data types

<div class="_SimpleCData">

This non-public class is the base class of all fundamental ctypes data types. It is mentioned here because it contains the common attributes of the fundamental ctypes data types. `_SimpleCData` is a subclass of `_CData`, so it inherits their methods and attributes. ctypes data types that are not and do not contain pointers can now be pickled.

Instances have a single attribute:

<div class="attribute">

value

This attribute contains the actual value of the instance. For integer and pointer types, it is an integer, for character types, it is a single character bytes object or string, for character pointer types it is a Python bytes object or string.

When the `value` attribute is retrieved from a ctypes instance, usually a new object is returned each time. `!ctypes` does *not* implement original object return, always a new object is constructed. The same is true for all other ctypes object instances.

</div>

Each subclass has a class attribute:

<div class="attribute">

<span id="type">type</span>\_

Class attribute that contains an internal type code, as a string. See `ctypes-fundamental-data-types` for a summary.

Types marked \* in the summary may be (or always are) aliases of a different `_SimpleCData` subclass, and will not necessarily use the listed type code. For example, if the platform's `long`, `long long` and `time_t` C types are the same, then `c_long`, `c_longlong` and `c_time_t` all refer to a single class, `c_long`, whose `_type_` code is `'l'`. The `'L'` code will be unused.

<div class="seealso">

The `array` and `struct <type-codes>` modules, as well as third-party modules like [numpy](https://numpy.org/doc/stable/reference/arrays.interface.html#object.__array_interface__), use similar -- but slightly different -- type codes.

</div>

</div>

</div>

Fundamental data types, when returned as foreign function call results, or, for example, by retrieving structure field members or array items, are transparently converted to native Python types. In other words, if a foreign function has a `~_CFuncPtr.restype` of `c_char_p`, you will always receive a Python bytes object, *not* a `c_char_p` instance.

Subclasses of fundamental data types do *not* inherit this behavior. So, if a foreign functions `!restype` is a subclass of `c_void_p`, you will receive an instance of this subclass from the function call. Of course, you can get the value of the pointer by accessing the `value` attribute.

These are the fundamental ctypes data types:

<div class="c_byte">

Represents the C `signed char` datatype, and interprets the value as small integer. The constructor accepts an optional integer initializer; no overflow checking is done.

</div>

<div class="c_char">

Represents the C `char` datatype, and interprets the value as a single character. The constructor accepts an optional string initializer, the length of the string must be exactly one character.

</div>

<div class="c_char_p">

Represents the C `char *` datatype when it points to a zero-terminated string. For a general character pointer that may also point to binary data, `POINTER(c_char)` must be used. The constructor accepts an integer address, or a bytes object.

</div>

<div class="c_double">

Represents the C `double` datatype. The constructor accepts an optional float initializer.

</div>

<div class="c_longdouble">

Represents the C `long double` datatype. The constructor accepts an optional float initializer. On platforms where `sizeof(long double) == sizeof(double)` it is an alias to `c_double`.

</div>

<div class="c_float">

Represents the C `float` datatype. The constructor accepts an optional float initializer.

</div>

<div class="c_double_complex">

Represents the C `double complex` datatype, if available. The constructor accepts an optional `complex` initializer.

<div class="versionadded">

3.14

</div>

</div>

<div class="c_float_complex">

Represents the C `float complex` datatype, if available. The constructor accepts an optional `complex` initializer.

<div class="versionadded">

3.14

</div>

</div>

<div class="c_longdouble_complex">

Represents the C `long double complex` datatype, if available. The constructor accepts an optional `complex` initializer.

<div class="versionadded">

3.14

</div>

</div>

<div class="c_int">

Represents the C `signed int` datatype. The constructor accepts an optional integer initializer; no overflow checking is done. On platforms where `sizeof(int) == sizeof(long)` it is an alias to `c_long`.

</div>

<div class="c_int8">

Represents the C 8-bit `signed int` datatype. It is an alias for `c_byte`.

</div>

<div class="c_int16">

Represents the C 16-bit `signed int` datatype. Usually an alias for `c_short`.

</div>

<div class="c_int32">

Represents the C 32-bit `signed int` datatype. Usually an alias for `c_int`.

</div>

<div class="c_int64">

Represents the C 64-bit `signed int` datatype. Usually an alias for `c_longlong`.

</div>

<div class="c_long">

Represents the C `signed long` datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

</div>

<div class="c_longlong">

Represents the C `signed long long` datatype. The constructor accepts an optional integer initializer; no overflow checking is done. On platforms where `sizeof(long long) == sizeof(long)` it is an alias to `c_long`.

</div>

<div class="c_short">

Represents the C `signed short` datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

</div>

<div class="c_size_t">

Represents the C `size_t` datatype. Usually an alias for another unsigned integer type.

</div>

<div class="c_ssize_t">

Represents the `Py_ssize_t` datatype. This is a signed version of `size_t`; that is, the POSIX `ssize_t` type. Usually an alias for another integer type.

<div class="versionadded">

3.2

</div>

</div>

<div class="c_time_t">

Represents the C `time_t` datatype. Usually an alias for another integer type.

<div class="versionadded">

3.12

</div>

</div>

<div class="c_ubyte">

Represents the C `unsigned char` datatype, it interprets the value as small integer. The constructor accepts an optional integer initializer; no overflow checking is done.

</div>

<div class="c_uint">

Represents the C `unsigned int` datatype. The constructor accepts an optional integer initializer; no overflow checking is done. On platforms where `sizeof(int) == sizeof(long)` it is an alias for `c_ulong`.

</div>

<div class="c_uint8">

Represents the C 8-bit `unsigned int` datatype. It is an alias for `c_ubyte`.

</div>

<div class="c_uint16">

Represents the C 16-bit `unsigned int` datatype. Usually an alias for `c_ushort`.

</div>

<div class="c_uint32">

Represents the C 32-bit `unsigned int` datatype. Usually an alias for `c_uint`.

</div>

<div class="c_uint64">

Represents the C 64-bit `unsigned int` datatype. Usually an alias for `c_ulonglong`.

</div>

<div class="c_ulong">

Represents the C `unsigned long` datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

</div>

<div class="c_ulonglong">

Represents the C `unsigned long long` datatype. The constructor accepts an optional integer initializer; no overflow checking is done. On platforms where `sizeof(long long) == sizeof(long)` it is an alias to `c_long`.

</div>

<div class="c_ushort">

Represents the C `unsigned short` datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

</div>

<div class="c_void_p">

Represents the C `void *` type. The value is represented as integer. The constructor accepts an optional integer initializer.

</div>

<div class="c_wchar">

Represents the C `wchar_t` datatype, and interprets the value as a single character unicode string. The constructor accepts an optional string initializer, the length of the string must be exactly one character.

</div>

<div class="c_wchar_p">

Represents the C `wchar_t *` datatype, which must be a pointer to a zero-terminated wide character string. The constructor accepts an integer address, or a string.

</div>

<div class="c_bool">

Represent the C `bool` datatype (more accurately, `_Bool` from C99). Its value can be `True` or `False`, and the constructor accepts any object that has a truth value.

</div>

<div class="HRESULT">

Represents a `!HRESULT` value, which contains success or error information for a function or method call.

<div class="availability">

Windows

</div>

</div>

<div class="py_object">

Represents the C `PyObject *` datatype. Calling this without an argument creates a `NULL` `PyObject *` pointer.

<div class="versionchanged">

3.14 `!py_object` is now a `generic type`.

</div>

</div>

### Windows data types

<div class="module" data-synopsis="Windows data types for ctypes.">

ctypes.wintypes

</div>

**Source code:** `Lib/ctypes/wintypes.py`

The `!ctypes.wintypes` module provides types for use with Windows APIs. Most are aliases for types defined in `ctypes`. See Microsoft's [Windows data types](https://learn.microsoft.com/en-us/windows/win32/winprog/windows-data-types) documentation for the corresponding C definitions. On non-Windows platforms, the sizes and layouts of these types may differ from their Windows counterparts.

<div class="versionchanged">

3.14 Added `HCONV`, `HCONVLIST`, `HCURSOR`, `HDDEDATA`, `HDROP`, `HFILE`, `HRESULT`, and `HSZ`.

</div>

For example:

    >>> from ctypes import POINTER
    >>> from ctypes.wintypes import LPRECT, RECT, SHORT
    >>> SHORT(-1).value
    -1
    >>> rect = RECT(0, 0, 100, 50)
    >>> rect.right, rect.bottom
    (100, 50)
    >>> LPRECT is POINTER(RECT)
    True

#### Simple types

The following names are aliases for fundamental `ctypes` types:

| Windows types | `ctypes` type         |
|---------------|-----------------------|
|               | `~ctypes.c_ubyte`     |
|               | `~ctypes.c_char`      |
|               | `~ctypes.c_wchar`     |
|               | `~ctypes.c_ushort`    |
|               | `~ctypes.c_ulong`     |
|               | `~ctypes.c_long`      |
|               | `~ctypes.c_int`       |
|               | `~ctypes.c_uint`      |
|               | `~ctypes.c_short`     |
|               | `~ctypes.c_double`    |
|               | `~ctypes.c_float`     |
|               | `~ctypes.c_longlong`  |
|               | `~ctypes.c_ulonglong` |
|               | `~ctypes.c_char_p`    |
|               | `~ctypes.c_wchar_p`   |
|               | `~ctypes.c_void_p`    |

`LARGE_INTEGER` and `ULARGE_INTEGER` expose an integer `!value` attribute, with no `LowPart`, `HighPart`, or `QuadPart` fields.

<div class="WPARAM">

An unsigned integer type with the same size as `~ctypes.c_void_p`.

</div>

<div class="LPARAM">

A signed integer type with the same size as `~ctypes.c_void_p`.

</div>

<div class="VARIANT_BOOL">

A 16-bit Boolean type whose `!value` attribute is a Python `bool`. Assigning to `!value` stores `-1` for a true value and `0` for a false value.

</div>

#### Handle types

<div class="HANDLE">

An alias for `~ctypes.c_void_p`, used to represent Windows handles.

</div>

<div class="HACCEL HBITMAP HBRUSH HCOLORSPACE HCONV HCONVLIST HCURSOR HDC HDDEDATA HDESK HDROP HDWP HENHMETAFILE HFONT HGDIOBJ HGLOBAL HHOOK HICON HINSTANCE HKEY HKL HLOCAL HMENU HMETAFILE HMODULE HMONITOR HPALETTE HPEN HRGN HRSRC HSTR HSZ HTASK HWINSTA HWND SC_HANDLE SERVICE_STATUS_HANDLE">

Aliases for `HANDLE` for use with specific Windows APIs. `HFILE` is an alias for `INT` and is listed among the simple types above.

</div>

#### Structures

The following types are subclasses of `ctypes.Structure`.

<div class="RECT RECTL tagRECT">

A rectangle with `left`, `top`, `right`, and `bottom` fields, all of type `LONG`.

</div>

<div class="SMALL_RECT">

A rectangle with `Left`, `Top`, `Right`, and `Bottom` fields, all of type `SHORT`.

</div>

<div class="POINT POINTL tagPOINT">

A point with `x` and `y` fields, both of type `LONG`.

</div>

<div class="SIZE SIZEL tagSIZE">

Dimensions with `cx` and `cy` fields, both of type `LONG`.

</div>

<div class="FILETIME">

A file time with `dwLowDateTime` and `dwHighDateTime` fields, both of type `DWORD`.

</div>

<div class="MSG tagMSG">

A Windows message with the following fields:

| Field     | Type     |
|-----------|----------|
| `hWnd`    | `HWND`   |
| `message` | `UINT`   |
| `wParam`  | `WPARAM` |
| `lParam`  | `LPARAM` |
| `time`    | `DWORD`  |
| `pt`      | `POINT`  |

</div>

<div class="WIN32_FIND_DATAA WIN32_FIND_DATAW">

Structures used by Windows file-search functions, with the following fields. `WIN32_FIND_DATAA` uses `CHAR` for its file names, and `WIN32_FIND_DATAW` uses `WCHAR`.

| Fields | Type |
|----|----|
| `dwFileAttributes` | `DWORD` |
| `ftCreationTime`, `ftLastAccessTime`, `ftLastWriteTime` | `FILETIME` |
| `nFileSizeHigh`, `nFileSizeLow` | `DWORD` |
| `dwReserved0`, `dwReserved1` | `DWORD` |
| `cFileName` | An array of `MAX_PATH` characters |
| `cAlternateFileName` | An array of 14 characters |

</div>

#### Pointer types

The following names are aliases for `ctypes.POINTER(T)`, where *T* is the type in the second column:

| Pointer types | Referenced type    |
|---------------|--------------------|
|               | `BOOL`             |
|               | `BOOLEAN`          |
|               | `BYTE`             |
|               | `CHAR`             |
|               | `COLORREF`         |
|               | `DWORD`            |
|               | `FILETIME`         |
|               | `FLOAT`            |
|               | `HANDLE`           |
|               | `HKEY`             |
|               | `HKL`              |
|               | `INT`              |
|               | `LARGE_INTEGER`    |
|               | `LCID`             |
|               | `LONG`             |
|               | `MSG`              |
|               | `POINT`            |
|               | `POINTL`           |
|               | `RECT`             |
|               | `RECTL`            |
|               | `SC_HANDLE`        |
|               | `SHORT`            |
|               | `SIZE`             |
|               | `SIZEL`            |
|               | `SMALL_RECT`       |
|               | `UINT`             |
|               | `ULARGE_INTEGER`   |
|               | `ULONG`            |
|               | `USHORT`           |
|               | `WCHAR`            |
|               | `WIN32_FIND_DATAA` |
|               | `WIN32_FIND_DATAW` |
|               | `WORD`             |

#### Constants and functions

<div class="data">

MAX_PATH

The value `260`, used as the size of the file-name arrays in `WIN32_FIND_DATAA` and `WIN32_FIND_DATAW`.

</div>

<div class="function">

RGB(red, green, blue)

Return a `COLORREF` value as a Python integer. The *red*, *green*, and *blue* arguments are integers from `0` to `255`, specifying the intensity of each color component:

    >>> from ctypes.wintypes import RGB
    >>> hex(RGB(255, 128, 0))
    '0x80ff'

</div>

<div class="currentmodule">

ctypes

</div>

### Structured data types

<div class="Union(*args, **kw)">

Abstract base class for unions in native byte order.

Unions share common attributes and behavior with structures; see `Structure` documentation for details.

</div>

<div class="BigEndianUnion(*args, **kw)">

Abstract base class for unions in *big endian* byte order.

<div class="versionadded">

3.11

</div>

</div>

<div class="LittleEndianUnion(*args, **kw)">

Abstract base class for unions in *little endian* byte order.

<div class="versionadded">

3.11

</div>

</div>

<div class="BigEndianStructure(*args, **kw)">

Abstract base class for structures in *big endian* byte order.

</div>

<div class="LittleEndianStructure(*args, **kw)">

Abstract base class for structures in *little endian* byte order.

</div>

Structures and unions with non-native byte order cannot contain pointer type fields, or any other data types containing pointer type fields.

<div class="Structure(*args, **kw)">

Abstract base class for structures in *native* byte order.

Concrete structure and union types must be created by subclassing one of these types, and at least define a `_fields_` class variable. `!ctypes` will create `descriptor`s which allow reading and writing the fields by direct attribute accesses. These are the

<div class="attribute">

<span id="fields">fields</span>\_

A sequence defining the structure fields. The items must be 2-tuples or 3-tuples. The first item is the name of the field, the second item specifies the type of the field; it can be any ctypes data type.

For integer type fields like `c_int`, a third optional item can be given. It must be a small positive integer defining the bit width of the field.

Field names must be unique within one structure or union. This is not checked, only one field can be accessed when names are repeated.

It is possible to define the `_fields_` class variable *after* the class statement that defines the Structure subclass, this allows creating data types that directly or indirectly reference themselves:

    class List(Structure):
        pass
    List._fields_ = [("pnext", POINTER(List)),
                     ...
                    ]

The `!_fields_` class variable can only be set once. Later assignments will raise an `AttributeError`.

Additionally, the `!_fields_` class variable must be defined before the structure or union type is first used: an instance or subclass is created, `sizeof` is called on it, and so on. Later assignments to `!_fields_` will raise an `AttributeError`. If `!_fields_` has not been set before such use, the structure or union will have no own fields, as if `!_fields_` was empty.

Sub-subclasses of structure types inherit the fields of the base class plus the `_fields_` defined in the sub-subclass, if any.

</div>

<div class="attribute">

<span id="pack">pack</span>\_

An optional small integer that allows overriding the alignment of structure fields in the instance.

This is only implemented for the MSVC-compatible memory layout (see `_layout_`).

Setting `!_pack_` to 0 is the same as not setting it at all. Otherwise, the value must be a positive power of two. The effect is equivalent to `#pragma pack(N)` in C, except `!ctypes` may allow larger *n* than what the compiler accepts.

`!_pack_` must already be defined when `_fields_` is assigned, otherwise it will have no effect.

<div class="deprecated-removed">

3.14 3.19

For historical reasons, if `!_pack_` is non-zero, the MSVC-compatible layout will be used by default. On non-Windows platforms, this default is deprecated and is slated to become an error in Python 3.19. If it is intended, set `~Structure._layout_` to `'ms'` explicitly.

</div>

</div>

<div class="attribute">

<span id="align">align</span>\_

An optional small integer that allows increasing the alignment of the structure when being packed or unpacked to/from memory.

The value must not be negative. The effect is equivalent to `__attribute__((aligned(N)))` on GCC or `#pragma align(N)` on MSVC, except `!ctypes` may allow values that the compiler would reject.

`!_align_` can only *increase* a structure's alignment requirements. Setting it to 0 or 1 has no effect.

Using values that are not powers of two is discouraged and may lead to surprising behavior.

`!_align_` must already be defined when `_fields_` is assigned, otherwise it will have no effect.

<div class="versionadded">

3.13

</div>

</div>

<div class="attribute">

<span id="layout">layout</span>\_

An optional string naming the struct/union layout. It can currently be set to:

- `"ms"`: the layout used by the Microsoft compiler (MSVC). On GCC and Clang, this layout can be selected with `__attribute__((ms_struct))`.
- `"gcc-sysv"`: the layout used by GCC with the System V or “SysV-like” data model, as used on Linux and macOS. With this layout, `~Structure._pack_` must be unset or zero.

If not set explicitly, `ctypes` will use a default that matches the platform conventions. This default may change in future Python releases (for example, when a new platform gains official support, or when a difference between similar platforms is found). Currently the default will be:

- On Windows: `"ms"`
- When `~Structure._pack_` is specified: `"ms"`. (This is deprecated; see `~Structure._pack_` documentation.)
- Otherwise: `"gcc-sysv"`

`!_layout_` must already be defined when `~Structure._fields_` is assigned, otherwise it will have no effect.

<div class="versionadded">

3.14

</div>

</div>

<div class="attribute">

<span id="anonymous">anonymous</span>\_

An optional sequence that lists the names of unnamed (anonymous) fields. `_anonymous_` must be already defined when `_fields_` is assigned, otherwise it will have no effect.

The fields listed in this variable must be structure or union type fields. `!ctypes` will create descriptors in the structure type that allows accessing the nested fields directly, without the need to create the structure or union field.

Here is an example type (Windows):

    class _U(Union):
        _fields_ = [("lptdesc", POINTER(TYPEDESC)),
                    ("lpadesc", POINTER(ARRAYDESC)),
                    ("hreftype", HREFTYPE)]

    class TYPEDESC(Structure):
        _anonymous_ = ("u",)
        _fields_ = [("u", _U),
                    ("vt", VARTYPE)]

The `TYPEDESC` structure describes a COM data type, the `vt` field specifies which one of the union fields is valid. Since the `u` field is defined as anonymous field, it is now possible to access the members directly off the TYPEDESC instance. `td.lptdesc` and `td.u.lptdesc` are equivalent, but the former is faster since it does not need to create a temporary union instance:

    td = TYPEDESC()
    td.vt = VT_PTR
    td.lptdesc = POINTER(some_type)
    td.u.lptdesc = POINTER(some_type)

</div>

It is possible to define sub-subclasses of structures, they inherit the fields of the base class. If the subclass definition has a separate `_fields_` variable, the fields specified in this are appended to the fields of the base class.

Structure and union constructors accept both positional and keyword arguments. Positional arguments are used to initialize member fields in the same order as they are appear in `_fields_`. Keyword arguments in the constructor are interpreted as attribute assignments, so they will initialize `_fields_` with the same name, or create new attributes for names not present in `_fields_`.

</div>

<div class="CField(*args, **kw)">

Descriptor for fields of a `Structure` and `Union`. For example:

    >>> class Color(Structure):
    ...     _fields_ = (
    ...         ('red', c_uint8),
    ...         ('green', c_uint8),
    ...         ('blue', c_uint8),
    ...         ('intense', c_bool, 1),
    ...         ('blinking', c_bool, 1),
    ...    )
    ...
    >>> Color.red
    <ctypes.CField 'red' type=c_ubyte, ofs=0, size=1>
    >>> Color.green.type
    <class 'ctypes.c_ubyte'>
    >>> Color.blue.byte_offset
    2
    >>> Color.intense
    <ctypes.CField 'intense' type=c_bool, ofs=3, bit_size=1, bit_offset=0>
    >>> Color.blinking.bit_offset
    1

All attributes are read-only.

`!CField` objects are created via `~Structure._fields_`; do not instantiate the class directly.

<div class="versionadded">

3.14

Previously, descriptors only had `offset` and `size` attributes and a readable string representation; the `!CField` class was not available directly.

</div>

<div class="attribute">

name

Name of the field, as a string.

</div>

<div class="attribute">

type

Type of the field, as a `ctypes class <ctypes-data-types>`.

</div>

<div class="attribute">

offset byte_offset

Offset of the field, in bytes.

For bitfields, this is the offset of the underlying byte-aligned *storage unit*; see `~CField.bit_offset`.

</div>

<div class="attribute">

byte_size

Size of the field, in bytes.

For bitfields, this is the size of the underlying *storage unit*. Typically, it has the same size as the bitfield's type.

</div>

<div class="attribute">

size

For non-bitfields, equivalent to `~CField.byte_size`.

For bitfields, this contains a backwards-compatible bit-packed value that combines `~CField.bit_size` and `~CField.bit_offset`. Prefer using the explicit attributes instead.

</div>

<div class="attribute">

is_bitfield

True if this is a bitfield.

</div>

<div class="attribute">

bit_offset bit_size

The location of a bitfield within its *storage unit*, that is, within `~CField.byte_size` bytes of memory starting at `~CField.byte_offset`.

To get the field's value, read the storage unit as an integer, `shift left <shifting>` by `!bit_offset` and take the `!bit_size` least significant bits.

For non-bitfields, `!bit_offset` is zero and `!bit_size` is equal to `byte_size * 8`.

</div>

<div class="attribute">

is_anonymous

True if this field is anonymous, that is, it contains nested sub-fields that should be merged into a containing structure or union.

</div>

</div>

<div class="decorator" data-module="ctypes.util">

struct(\*, align=None, layout=None, endian='native', pack=None)

A `decorator` that allows generating structure types using an annotation-based syntax, similar to the `dataclasses` module.

For example:

``` python
from ctypes.util import struct
from ctypes import c_int

@struct
class Point:
    x: c_int
    y: c_int

point = Point(1, 2)
```

> [!NOTE]
> Calling `super` without arguments in methods of the decorated class raises a `TypeError`. Use the two-argument form instead, for example `super(Point, self).__init__(x, y)`.

*align*, *layout*, and *pack* supply the value for the `~ctypes.Structure._align_`, `~ctypes.Structure._layout_`, and `~ctypes.Structure._pack_` attributes, respectively.

*endian* controls which structure class will be used as the base.

- If *endian* is `'native'`, `~ctypes.Structure` will be used.
- If *endian* is `'big'`, `~ctypes.BigEndianStructure` will be used.
- If *endian* is `'little'`, `~ctypes.LittleEndianStructure` will be used.

Any other value will raise a `ValueError`.

For controlling field-specific data, wrap the annotation in `typing.Annotated` with `CFieldInfo` as the second argument, like so:

``` python
from typing import Annotated
from ctypes import c_ssize_t, c_void_p
from ctypes.util import struct, CFieldInfo

@struct
class PyObject:
    ob_refcnt: c_ssize_t
    ob_type: c_void_p

@struct
class PyHovercraftObject:
    ob_base: Annotated[PyObject, CFieldInfo(anonymous=True)]
```

<div class="versionadded">

3.16

</div>

</div>

<div class="CFieldInfo(anonymous=False, bit_width=None)" data-module="ctypes.util">

Information regarding a structure field defined by the `~ctypes.util.struct` decorator. This should be used in the second argument of a `typing.Annotated` wrapping a ctypes type.

*anonymous* specifies whether the field will be present in the `~ctypes.Structure._anonymous_` attribute of the generated class.

If *bit_width* is non-`None`, the annotated field will be *bit_width* number of bits in the generated structure. This is equivalent to passing a third item in `~ctypes.Structure._fields_`.

<div class="versionadded">

3.16

</div>

</div>

### Arrays and pointers

<div class="Array(*args)">

Abstract base class for arrays.

The recommended way to create concrete array types is by multiplying any `!ctypes` data type with a non-negative integer. Alternatively, you can subclass this type and define `_length_` and `_type_` class variables. Array elements can be read and written using standard subscript and slice accesses; for slice reads, the resulting object is *not* itself an `Array`.

Arrays are `generic <generics>` over the type of their elements.

<div class="attribute">

<span id="length">length</span>\_

A positive integer specifying the number of elements in the array. Out-of-range subscripts result in an `IndexError`. Will be returned by `len`.

</div>

<div class="attribute">

<span id="type">type</span>\_

Specifies the type of each element in the array.

</div>

Array subclass constructors accept positional arguments, used to initialize the elements in order.

</div>

<div class="function">

ARRAY(type, length)

Create an array. Equivalent to `type * length`, where *type* is a `!ctypes` data type and *length* an integer.

<div class="soft-deprecated">

3.14 In favor of multiplication.

</div>

</div>

<div class="_Pointer">

Private, abstract base class for pointers.

Concrete pointer types are created by calling `POINTER` with the type that will be pointed to; this is done automatically by `pointer`.

If a pointer points to an array, its elements can be read and written using standard subscript and slice accesses. Pointer objects have no size, so `len` will raise `TypeError`. Negative subscripts will read from the memory *before* the pointer (as in C), and out-of-range subscripts will probably crash with an access violation (if you're lucky).

<div class="attribute">

<span id="type">type</span>\_

Specifies the type pointed to.

</div>

<div class="attribute">

contents

Returns the object to which to pointer points. Assigning to this attribute changes the pointer to point to the assigned object.

</div>

</div>

### Exceptions

<div class="exception">

ArgumentError

This exception is raised when a foreign function call cannot convert one of the passed arguments.

</div>

<div class="exception">

COMError(hresult, text, details)

This exception is raised when a COM method call failed.

<div class="attribute">

hresult

The integer value representing the error code.

</div>

<div class="attribute">

text

The error message.

</div>

<div class="attribute">

details

The 5-tuple `(descr, source, helpfile, helpcontext, progid)`.

*descr* is the textual description. *source* is the language-dependent `ProgID` for the class or application that raised the error. *helpfile* is the path of the help file. *helpcontext* is the help context identifier. *progid* is the `ProgID` of the interface that defined the error.

</div>

<div class="availability">

Windows

</div>

<div class="versionadded">

3.14

</div>

</div>

### Library version

The following constants are only available if `!ctypes` was built with libffi 3.5 or later, which is the first version providing this information.

<div class="data">

LIBFFI_VERSION

The version string of the libffi library that was used for building the module, like `'3.5.2'`. This may be different from the libffi library actually used at runtime, which is available as `libffi_version`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

libffi_version

The version string of the libffi library actually loaded by the interpreter.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

LIBFFI_VERSION_INFO

A named tuple containing the three components of the libffi library version that was used for building the module: *major*, *minor*, and *patch*. All values are integers. The components can also be accessed by name, so `ctypes.LIBFFI_VERSION_INFO[0]` is equivalent to `ctypes.LIBFFI_VERSION_INFO.major` and so on. This may be different from the libffi library actually used at runtime, which is available as `libffi_version_info`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

libffi_version_info

A named tuple containing the version of the libffi library actually loaded by the interpreter, with the same fields as `LIBFFI_VERSION_INFO`.

<div class="versionadded">

3.16

</div>

</div>

# `!curses.ascii` --- Utilities for ASCII characters

<div class="module" data-synopsis="Constants and set-membership functions for ASCII characters.">

curses.ascii

</div>

**Source code:** `Lib/curses/ascii.py`

------------------------------------------------------------------------

The `!curses.ascii` module supplies name constants for ASCII characters and functions to test membership in various ASCII character classes. The constants supplied are names for control characters as follows:

<table style="width:87%;">
<colgroup>
<col style="width: 22%" />
<col style="width: 65%" />
</colgroup>
<thead>
<tr>
<th><p>Name</p></th>
<th><p>Meaning</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>NUL</p>
</div></td>
<td></td>
</tr>
<tr>
<td><div class="data">
<p>SOH</p>
</div></td>
<td><p>Start of heading, console interrupt</p></td>
</tr>
<tr>
<td><div class="data">
<p>STX</p>
</div></td>
<td><p>Start of text</p></td>
</tr>
<tr>
<td><div class="data">
<p>ETX</p>
</div></td>
<td><p>End of text</p></td>
</tr>
<tr>
<td><div class="data">
<p>EOT</p>
</div></td>
<td><p>End of transmission</p></td>
</tr>
<tr>
<td><div class="data">
<p>ENQ</p>
</div></td>
<td><p>Enquiry, goes with <code class="interpreted-text" role="const">ACK</code> flow control</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACK</p>
</div></td>
<td><p>Acknowledgement</p></td>
</tr>
<tr>
<td><div class="data">
<p>BEL</p>
</div></td>
<td><p>Bell</p></td>
</tr>
<tr>
<td><div class="data">
<p>BS</p>
</div></td>
<td><p>Backspace</p></td>
</tr>
<tr>
<td><div class="data">
<p>TAB</p>
</div></td>
<td><p>Tab</p></td>
</tr>
<tr>
<td><div class="data">
<p>HT</p>
</div></td>
<td><p>Alias for <code class="interpreted-text" role="const">TAB</code>: "Horizontal tab"</p></td>
</tr>
<tr>
<td><div class="data">
<p>LF</p>
</div></td>
<td><p>Line feed</p></td>
</tr>
<tr>
<td><div class="data">
<p>NL</p>
</div></td>
<td><p>Alias for <code class="interpreted-text" role="const">LF</code>: "New line"</p></td>
</tr>
<tr>
<td><div class="data">
<p>VT</p>
</div></td>
<td><p>Vertical tab</p></td>
</tr>
<tr>
<td><div class="data">
<p>FF</p>
</div></td>
<td><p>Form feed</p></td>
</tr>
<tr>
<td><div class="data">
<p>CR</p>
</div></td>
<td><p>Carriage return</p></td>
</tr>
<tr>
<td><div class="data">
<p>SO</p>
</div></td>
<td><p>Shift-out, begin alternate character set</p></td>
</tr>
<tr>
<td><div class="data">
<p>SI</p>
</div></td>
<td><p>Shift-in, resume default character set</p></td>
</tr>
<tr>
<td><div class="data">
<p>DLE</p>
</div></td>
<td><p>Data-link escape</p></td>
</tr>
<tr>
<td><div class="data">
<p>DC1</p>
</div></td>
<td><p>XON, for flow control</p></td>
</tr>
<tr>
<td><div class="data">
<p>DC2</p>
</div></td>
<td><p>Device control 2, block-mode flow control</p></td>
</tr>
<tr>
<td><div class="data">
<p>DC3</p>
</div></td>
<td><p>XOFF, for flow control</p></td>
</tr>
<tr>
<td><div class="data">
<p>DC4</p>
</div></td>
<td><p>Device control 4</p></td>
</tr>
<tr>
<td><div class="data">
<p>NAK</p>
</div></td>
<td><p>Negative acknowledgement</p></td>
</tr>
<tr>
<td><div class="data">
<p>SYN</p>
</div></td>
<td><p>Synchronous idle</p></td>
</tr>
<tr>
<td><div class="data">
<p>ETB</p>
</div></td>
<td><p>End transmission block</p></td>
</tr>
<tr>
<td><div class="data">
<p>CAN</p>
</div></td>
<td><p>Cancel</p></td>
</tr>
<tr>
<td><div class="data">
<p>EM</p>
</div></td>
<td><p>End of medium</p></td>
</tr>
<tr>
<td><div class="data">
<p>SUB</p>
</div></td>
<td><p>Substitute</p></td>
</tr>
<tr>
<td><div class="data">
<p>ESC</p>
</div></td>
<td><p>Escape</p></td>
</tr>
<tr>
<td><div class="data">
<p>FS</p>
</div></td>
<td><p>File separator</p></td>
</tr>
<tr>
<td><div class="data">
<p>GS</p>
</div></td>
<td><p>Group separator</p></td>
</tr>
<tr>
<td><div class="data">
<p>RS</p>
</div></td>
<td><p>Record separator, block-mode terminator</p></td>
</tr>
<tr>
<td><div class="data">
<p>US</p>
</div></td>
<td><p>Unit separator</p></td>
</tr>
<tr>
<td><div class="data">
<p>SP</p>
</div></td>
<td><p>Space</p></td>
</tr>
<tr>
<td><div class="data">
<p>DEL</p>
</div></td>
<td><p>Delete</p></td>
</tr>
</tbody>
</table>

Note that many of these have little practical significance in modern usage. The mnemonics derive from teleprinter conventions that predate digital computers.

The module supplies the following functions, patterned on those in the standard C library:

<div class="function">

isalnum(c)

Checks for an ASCII alphanumeric character; it is equivalent to `isalpha(c) or isdigit(c)`.

</div>

<div class="function">

isalpha(c)

Checks for an ASCII alphabetic character; it is equivalent to `isupper(c) or islower(c)`.

</div>

<div class="function">

isascii(c)

Checks for a character value that fits in the 7-bit ASCII set.

</div>

<div class="function">

isblank(c)

Checks for an ASCII blank character; space or horizontal tab.

</div>

<div class="function">

iscntrl(c)

Checks for an ASCII control character (in the range 0x00 to 0x1f or 0x7f).

</div>

<div class="function">

isdigit(c)

Checks for an ASCII decimal digit, `'0'` through `'9'`. This is equivalent to `c in string.digits`.

</div>

<div class="function">

isgraph(c)

Checks for any ASCII printable character except space.

</div>

<div class="function">

islower(c)

Checks for an ASCII lower-case character.

</div>

<div class="function">

isprint(c)

Checks for any ASCII printable character including space.

</div>

<div class="function">

ispunct(c)

Checks for any ASCII printable character which is not a space or an alphanumeric character.

</div>

<div class="function">

isspace(c)

Checks for ASCII white-space characters; space, line feed, carriage return, form feed, horizontal tab, vertical tab.

</div>

<div class="function">

isupper(c)

Checks for an ASCII uppercase letter.

</div>

<div class="function">

isxdigit(c)

Checks for an ASCII hexadecimal digit. This is equivalent to `c in string.hexdigits`.

</div>

<div class="function">

isctrl(c)

Checks for an ASCII control character (ordinal values 0 to 31). Unlike `iscntrl`, this does not include the delete character (0x7f).

</div>

<div class="function">

ismeta(c)

Checks for a non-ASCII character (ordinal values 0x80 and above).

</div>

These functions accept an integer, a single-character string, or a `curses.complexchar`. A string is converted using the built-in function `ord`, and a complexchar by the code of its single character; a complexchar that holds combining characters is not a single character and matches no class.

Note that all these functions check ordinal bit values derived from the character of the string you pass in; they do not actually know anything about the host machine's character encoding.

The following three functions take either a single-character string or an integer byte value; they return a value of the same type.

<div class="function">

ascii(c)

Return the ASCII value corresponding to the low 7 bits of *c*.

</div>

<div class="function">

ctrl(c)

Return the control character corresponding to the given ASCII character (the character bit value is bitwise-anded with 0x1f). A non-ASCII character has no control character and is returned unchanged.

<div class="versionchanged">

3.16 A non-ASCII argument is now returned unchanged instead of masked to a control character.

</div>

</div>

<div class="function">

alt(c)

Return the 8-bit character corresponding to the given ASCII character (the character bit value is bitwise-ored with 0x80).

</div>

The following function takes a single-character string, an integer value, or a `curses.complexchar`; it returns a string.

<div class="index">

single: ^ (caret); in curses module single: ! (exclamation); in curses module

</div>

<div class="function">

unctrl(c)

Return a string representation of the ASCII character *c*. If *c* is printable, this string is the character itself. If the character is a control character (0x00--0x1f) the string consists of a caret (`'^'`) followed by the corresponding uppercase letter. If the character is an ASCII delete (0x7f) the string is `'^?'`. If the character has its meta bit (0x80) set, the meta bit is stripped, the preceding rules applied, and `'!'` prepended to the result.

</div>

<div class="data">

controlnames

A 33-element string array that contains the ASCII mnemonics for the thirty-two ASCII control characters from 0 (NUL) to 0x1f (US), in order, plus the mnemonic `SP` for the space character.

</div>

# `!curses` --- Terminal handling for character-cell displays

<div class="module" data-synopsis="An interface to the curses library, providing portable
terminal handling.">

curses

</div>

**Source code:** `Lib/curses`

------------------------------------------------------------------------

The `!curses` module provides an interface to the curses library, the de-facto standard for portable advanced terminal handling.

While curses is most widely used in the Unix environment, versions are available for Windows, DOS, and possibly other systems as well. This extension module is designed to match the API of ncurses, an open-source curses library hosted on Linux and the BSD variants of Unix.

<div class="availability">

not Android, not iOS, not WASI.

This module is not supported on `mobile platforms <mobile-availability>` or `WebAssembly platforms <wasm-availability>`.

</div>

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

<div class="availability">

Unix.

</div>

> [!NOTE]
> Whenever the documentation mentions a *character* it can be specified as an integer, a one-character Unicode string or a one-byte byte string. An integer is the code of a single encoded byte, optionally combined with attributes and a color pair, as returned by `window.inch`. Methods that write to a window accept also a character cell: a Unicode string of a spacing character followed by combining characters, or a `complexchar`.
>
> Whenever the documentation mentions a *character string* it can be specified as a Unicode string or a byte string. Methods that write to a window accept also a `complexstr`.

> [!NOTE]
> Whether curses may be used from several threads depends on the underlying library and how it was built. In many implementations, including the default build of ncurses, the screen state is shared and not thread-safe; since the blocking and refresh methods (such as `~window.getch` and `~window.refresh`) release the `GIL`, unsynchronized use from several threads can then crash the interpreter. Serialize the calls, or wrap them in `window.use` and `screen.use`.

<div class="seealso">

Module `curses.ascii`  
Utilities for working with ASCII characters, regardless of your locale settings.

Module `curses.panel`  
A panel stack extension that adds depth to curses windows.

Module `curses.textpad`  
Editable text widget for curses supporting `Emacs`-like bindings.

`curses-howto`  
Tutorial material on using curses with Python, by Andrew Kuchling and Eric Raymond.

</div>

## Functions

The module `!curses` defines the following exception:

<div class="exception">

error

Exception raised when a curses library function returns an error.

</div>

> [!NOTE]
> Whenever *x* or *y* arguments to a function or a method are optional, they default to the current cursor location. Whenever *attr* is optional, it defaults to `A_NORMAL`.

The module `!curses` defines the following functions:

### Initialization and termination

<div class="function">

initscr()

Initialize the library. Return a `window <curses-window-objects>` object which represents the whole screen.

See `setupterm` for a caveat about calling it before this function.

> [!NOTE]
> If there is an error opening the terminal, the underlying curses library may cause the interpreter to exit.

</div>

<div class="function">

endwin()

De-initialize the library, and return terminal to normal status.

</div>

<div class="function">

isendwin()

Return `True` if `endwin` has been called (that is, the curses library has been deinitialized).

</div>

<div class="function">

newterm(type=None, fd=None, infd=None, /)

Initialize a new terminal in addition to the one initialized by `initscr`, and return a `screen <curses-screen-objects>` for it. This allows a program to drive more than one terminal.

*type* is the terminal name, as in `setupterm`; if `None`, the value of the `TERM` environment variable is used. *fd* and *infd* are the output and input files for the terminal: either a file object or a file descriptor. They default to `sys.stdout` and `sys.stdin`.

The new screen becomes the current one. Use `set_term` to switch between screens.

See `setupterm` for a caveat about calling it before this function.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

new_prescr()

Return a new `screen <curses-screen-objects>` that can be used to call functions that affect global state before `initscr` or `newterm` is called.

Availability: if the underlying curses library provides `new_prescr()`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

set_term(screen, /)

Make *screen*, a `screen <curses-screen-objects>` returned by `newterm`, the current terminal, and return the previously current screen. Returns `None` if the previous screen was the one created by `initscr`. Raises `error` if *screen* has no terminal, as is the case for a screen returned by `new_prescr`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

wrapper(func, /, *args,*\*kwargs)

Initialize curses and call another callable object, *func*, which should be the rest of your curses-using application. If the application raises an exception, this function will restore the terminal to a sane state before re-raising the exception and generating a traceback. The callable object *func* is then passed the main window 'stdscr' as its first argument, followed by any other arguments passed to `!wrapper`. Before calling *func*, `!wrapper` turns on cbreak mode, turns off echo, enables the terminal keypad, and initializes colors if the terminal has color support. On exit (whether normally or by exception) it restores cooked mode, turns on echo, and disables the terminal keypad.

</div>

### Terminal mode control

<div class="function">

def_prog_mode()

Save the current terminal mode as the "program" mode, the mode when the running program is using curses. (Its counterpart is the "shell" mode, for when the program is not in curses.) Subsequent calls to `reset_prog_mode` will restore this mode.

</div>

<div class="function">

def_shell_mode()

Save the current terminal mode as the "shell" mode, the mode when the running program is not using curses. (Its counterpart is the "program" mode, when the program is using curses capabilities.) Subsequent calls to `reset_shell_mode` will restore this mode.

</div>

<div class="function">

reset_prog_mode()

Restore the terminal to "program" mode, as previously saved by `def_prog_mode`.

</div>

<div class="function">

reset_shell_mode()

Restore the terminal to "shell" mode, as previously saved by `def_shell_mode`.

</div>

<div class="function">

savetty()

Save the current state of the terminal modes in a buffer, usable by `resetty`.

</div>

<div class="function">

resetty()

Restore the state of the terminal modes to what it was at the last call to `savetty`.

</div>

<div class="function">

curs_set(visibility)

Set the cursor state. *visibility* can be set to `0`, `1`, or `2`, for invisible, normal, or very visible. If the terminal supports the visibility requested, return the previous cursor state; otherwise raise an exception. On many terminals, the "visible" mode is an underline cursor and the "very visible" mode is a block cursor.

</div>

<div class="function">

getsyx()

Return the current coordinates of the virtual screen cursor as a tuple `(y, x)`. If `leaveok <window.leaveok>` is currently `True`, then return `(-1, -1)`.

</div>

<div class="function">

setsyx(y, x)

Set the virtual screen cursor to *y*, *x*. If *y* and *x* are both `-1`, then `leaveok <window.leaveok>` is set `True`.

</div>

<div class="function">

doupdate()

Update the physical screen. The curses library keeps two data structures, one representing the current physical screen contents and a virtual screen representing the desired next state. The `doupdate` function updates the physical screen to match the virtual screen.

The virtual screen may be updated by a `~window.noutrefresh` call after write operations such as `~window.addstr` have been performed on a window. The normal `~window.refresh` call is simply `!noutrefresh` followed by `!doupdate`; if you have to update multiple windows, you can speed performance and perhaps reduce screen flicker by issuing `!noutrefresh` calls on all windows, followed by a single `!doupdate`.

</div>

### Input options

<div class="function">

cbreak()

Enter cbreak mode. In cbreak mode (sometimes called "rare" mode) normal tty line buffering is turned off and characters are available to be read one by one. However, unlike raw mode, special characters (interrupt, quit, suspend, and flow control) retain their effects on the tty driver and calling program. Calling first `raw` then `cbreak` leaves the terminal in cbreak mode.

</div>

<div class="function">

nocbreak()

Leave cbreak mode. Return to normal "cooked" mode with line buffering.

</div>

<div class="function">

echo()

Enter echo mode. In echo mode, each character input is echoed to the screen as it is entered.

</div>

<div class="function">

noecho()

Leave echo mode. Echoing of input characters is turned off.

</div>

<div class="function">

raw()

Enter raw mode. In raw mode, normal line buffering and processing of interrupt, quit, suspend, and flow control keys are turned off; characters are presented to curses input functions one by one.

</div>

<div class="function">

noraw()

Leave raw mode. Return to normal "cooked" mode with line buffering.

</div>

<div class="function">

halfdelay(tenths)

Used for half-delay mode, which is similar to cbreak mode in that characters typed by the user are immediately available to the program. However, after blocking for *tenths* tenths of seconds, if nothing has been typed, `~window.getch` returns `-1`, while `~window.getkey` and `~window.get_wch` raise `curses.error`. The value of *tenths* must be a number between `1` and `255`. Use `nocbreak` to leave half-delay mode.

</div>

<div class="function">

meta(flag)

If *flag* is `True`, allow 8-bit characters to be input. If *flag* is `False`, allow only 7-bit chars.

</div>

<div class="function">

nl(flag=True)

Enter newline mode. This mode translates the return key into newline on input, and translates newline into return and line-feed on output. Newline mode is initially on.

If *flag* is `False`, the effect is the same as calling `nonl`.

</div>

<div class="function">

nonl()

Leave newline mode. Disable translation of return into newline on input, and disable low-level translation of newline into newline/return on output (but this does not change the behavior of `addch('\n')`, which always does the equivalent of return and line feed on the virtual screen). With translation off, curses can sometimes speed up vertical motion a little; also, it will be able to detect the return key on input.

</div>

<div class="function">

intrflush(flag)

If *flag* is `True`, pressing an interrupt key (interrupt, break, or quit) will flush all output in the terminal driver queue. If *flag* is `False`, no flushing is done.

</div>

<div class="function">

qiflush(\[flag\])

If *flag* is `False`, the effect is the same as calling `noqiflush`. If *flag* is `True`, or no argument is provided, the queues will be flushed when these control characters are read.

</div>

<div class="function">

noqiflush()

When the `!noqiflush` routine is used, normal flush of input and output queues associated with the `INTR`, `QUIT` and `SUSP` characters will not be done. You may want to call `!noqiflush` in a signal handler if you want output to continue as though the interrupt had not occurred, after the handler exits.

</div>

<div class="function">

typeahead(fd)

Specify that the file descriptor *fd* be used for typeahead checking. If *fd* is `-1`, then no typeahead checking is done.

The curses library does "line-breakout optimization" by looking for typeahead periodically while updating the screen. If input is found, and it is coming from a tty, the current update is postponed until `~window.refresh` or `doupdate` is called again, allowing faster response to commands typed in advance. This function allows specifying a different file descriptor for typeahead checking.

</div>

<div class="function">

is_cbreak()

Return `True` if cbreak mode (see `cbreak`) is enabled, `False` otherwise. Availability: ncurses 6.5 or later.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

is_echo()

Return `True` if echo mode (see `echo`) is enabled, `False` otherwise. Availability: ncurses 6.5 or later.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

is_nl()

Return `True` if nl mode (see `nl`) is enabled, `False` otherwise. Availability: ncurses 6.5 or later.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

is_raw()

Return `True` if raw mode (see `raw`) is enabled, `False` otherwise. Availability: ncurses 6.5 or later.

<div class="versionadded">

3.16

</div>

</div>

### Keyboard input

<div class="function">

ungetch(ch)

Push *ch* so the next `~window.getch` or `~window.get_wch` will return it.

*ch* may be an integer (a key code or the code of an encoded byte), a byte, or a string of length 1. A one-character string is pushed like `unget_wch`; on a narrow build it must encode to a single byte.

> [!NOTE]
> Only one *ch* can be pushed before `!getch` is called.

<div class="versionchanged">

3.16 A one-character string argument is no longer required to encode to a single byte, except on a narrow build.

</div>

</div>

<div class="function">

unget_wch(ch)

Push *ch* so the next `~window.get_wch` will return it.

*ch* may be an integer (a character code, not a key code) or a string of length 1.

> [!NOTE]
> Only one *ch* can be pushed before `!get_wch` is called.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.16 Also available on a narrow build, where *ch* must encode to a single byte (an 8-bit locale).

</div>

</div>

<div class="function">

flushinp()

Flush all input buffers. This throws away any typeahead that has been typed by the user and has not yet been processed by the program.

</div>

<div class="function">

has_key(ch)

Take a key value *ch*, and return `True` if the current terminal type recognizes a key with that value.

</div>

<div class="function">

keyname(k)

Return the name of the key numbered *k* as a bytes object. The name of a key generating printable ASCII character is the key's character. The name of a control-key combination is a two-byte bytes object consisting of a caret (`b'^'`) followed by the corresponding printable ASCII character. The name of an alt-key combination (128--255) is a bytes object consisting of the prefix `b'M-'` followed by the name of the corresponding ASCII character.

Raise a `ValueError` if *k* is negative.

</div>

<div class="function">

define_key(definition, keycode)

Define an escape sequence *definition*, a string, as a key that generates the key code *keycode*, so that `curses` interprets it like one of the keys predefined in the terminal database.

If *definition* is `None`, any existing binding for *keycode* is removed. If *keycode* is zero or negative, any existing binding for *definition* is removed.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

key_defined(definition)

Return the key code bound to the escape sequence *definition*, a string, `0` if no key code is bound to it, or `-1` if *definition* is a prefix of a longer bound sequence (and so is ambiguous).

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

keyok(keycode, enable)

Enable (if *enable* is true) or disable (otherwise) interpretation of the key code *keycode*. Unlike `window.keypad`, this affects a single key code rather than all of them.

<div class="versionadded">

3.16

</div>

</div>

### Mouse

<div class="function">

getmouse()

After `~window.getch` returns `KEY_MOUSE` to signal a mouse event, this method should be called to retrieve the queued mouse event, represented as a 5-tuple `(id, x, y, z, bstate)`. *id* is an ID value used to distinguish multiple devices, and *x*, *y*, *z* are the event's coordinates. (*z* is currently unused.) *bstate* is an integer value whose bits will be set to indicate the type of event, and will be the bitwise OR of one or more of the following constants, where *n* is the button number from 1 to 5: `BUTTONn_PRESSED`, `BUTTONn_RELEASED`, `BUTTONn_CLICKED`, `BUTTONn_DOUBLE_CLICKED`, `BUTTONn_TRIPLE_CLICKED`, `BUTTON_SHIFT`, `BUTTON_CTRL`, `BUTTON_ALT`.

<div class="versionchanged">

3.10 The `BUTTON5_*` constants are now exposed if they are provided by the underlying curses library.

</div>

</div>

<div class="function">

ungetmouse(id, x, y, z, bstate)

Push a `KEY_MOUSE` event onto the input queue, associating the given state data with it.

</div>

<div class="function">

mousemask(mousemask)

Set the mouse events to be reported, and return a tuple `(availmask, oldmask)`. *availmask* indicates which of the specified mouse events can be reported; on complete failure it returns `0`. *oldmask* is the previous value of the mouse event mask. If this function is never called, no mouse events are ever reported.

</div>

<div class="function">

mouseinterval(interval)

Set the maximum time in milliseconds that can elapse between press and release events in order for them to be recognized as a click, and return the previous interval value. The default value is 166 milliseconds, or one sixth of a second. Use a negative *interval* to obtain the interval value without changing it.

</div>

<div class="function">

has_mouse()

Return `True` if the mouse driver has been successfully initialized.

Availability: if the underlying curses library provides `has_mouse()`.

<div class="versionadded">

3.16

</div>

</div>

### Color

<div class="function">

start_color()

Must be called if the programmer wants to use colors, and before any other color manipulation routine is called. It is good practice to call this routine right after `initscr`.

`start_color` initializes eight basic colors (black, red, green, yellow, blue, magenta, cyan, and white), and two global variables in the `!curses` module, `COLORS` and `COLOR_PAIRS`, containing the maximum number of colors and color-pairs the terminal can support. It also restores the colors on the terminal to the values they had when the terminal was just turned on.

</div>

<div class="function">

has_colors()

Return `True` if the terminal can display colors; otherwise, return `False`.

</div>

<div class="function">

has_extended_color_support()

Return `True` if the module supports extended colors; otherwise, return `False`. Extended color support allows more than 256 color pairs for terminals that support more than 16 colors (for example, xterm-256color).

Extended color support requires ncurses version 6.1 or later.

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

can_change_color()

Return `True` or `False`, depending on whether the programmer can change the colors displayed by the terminal.

</div>

<div class="function">

init_color(color_number, r, g, b)

Change the definition of a color, taking the number of the color to be changed followed by three RGB values (for the amounts of red, green, and blue components). The value of *color_number* must be between `0` and `COLORS - 1`. Each of *r*, *g*, *b*, must be a value between `0` and `1000`. When `init_color` is used, all occurrences of that color on the screen immediately change to the new definition. This function is a no-op on most terminals; it is active only if `can_change_color` returns `True`.

</div>

<div class="function">

color_content(color_number)

Return the intensity of the red, green, and blue (RGB) components in the color *color_number*, which must be between `0` and `COLORS - 1`. Return a 3-tuple, containing the R,G,B values for the given color, which will be between `0` (no component) and `1000` (maximum amount of component). Raise an exception if the color is not supported.

</div>

<div class="function">

init_pair(pair_number, fg, bg)

Change the definition of a color-pair. It takes three arguments: the number of the color-pair to be changed, the foreground color number, and the background color number. The value of *pair_number* must be between `1` and `COLOR_PAIRS - 1` (the `0` color pair can only be changed by `use_default_colors` and `assume_default_colors`). The value of *fg* and *bg* arguments must be between `0` and `COLORS - 1`, or, after calling `!use_default_colors` or `!assume_default_colors`, `-1`. If the color-pair was previously initialized, the screen is refreshed and all occurrences of that color-pair are changed to the new definition.

</div>

<div class="function">

pair_content(pair_number)

Return a tuple `(fg, bg)` containing the colors for the requested color pair. The value of *pair_number* must be between `0` and `COLOR_PAIRS - 1`.

</div>

<div class="function">

color_pair(pair_number)

Return the attribute value for displaying text in the specified color pair. Only color pairs that fit in the color-pair field of the returned value can be represented (usually the first 256); a larger *pair_number* raises `OverflowError` rather than being silently masked to a different pair. Use `~window.color_set` or `~window.attr_set` to display higher pairs. This attribute value can be combined with `A_STANDOUT`, `A_REVERSE`, and the other `!A_\*` attributes. `pair_number` is the counterpart to this function.

</div>

<div class="function">

pair_number(attr)

Return the number of the color-pair set by the attribute value *attr*. `color_pair` is the counterpart to this function.

</div>

<div class="function">

use_default_colors()

Equivalent to `assume_default_colors(-1, -1)`.

</div>

<div class="function">

assume_default_colors(fg, bg, /)

Allow use of default values for colors on terminals supporting this feature. Use this to support transparency in your application.

- Assign terminal default foreground/background colors to color number `-1`. So `init_pair(x, COLOR_RED, -1)` will initialize pair *x* as red on default background and `init_pair(x, -1, COLOR_BLUE)` will initialize pair *x* as default foreground on blue.
- Change the definition of the color-pair `0` to `(fg, bg)`.

This is an ncurses extension.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

alloc_pair(fg, bg)

Allocate a color pair for foreground color *fg* and background color *bg*, and return its number. If a color pair for the same combination of colors already exists, return its number. Otherwise allocate a new color pair and return its number.

This function is only available if Python was built against a wide-character version of the underlying curses library with extended-color support (see `has_extended_color_support`).

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

free_pair(pair_number)

Free the color pair *pair_number*, which must have been allocated by `alloc_pair`. The pair must not be in use.

This function is only available if Python was built against a wide-character version of the underlying curses library with extended-color support (see `has_extended_color_support`).

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

find_pair(fg, bg)

Return the number of a color pair for foreground color *fg* and background color *bg*, or `-1` if no color pair for this combination of colors has been allocated.

This function is only available if Python was built against a wide-character version of the underlying curses library with extended-color support (see `has_extended_color_support`).

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

reset_color_pairs()

Discard all color-pair definitions, releasing the color pairs allocated by `init_pair` and `alloc_pair`.

This function is only available if Python was built against a wide-character version of the underlying curses library with extended-color support (see `has_extended_color_support`).

<div class="versionadded">

3.16

</div>

</div>

### Windows and pads

<div class="function">

newwin(nlines, ncols) newwin(nlines, ncols, begin_y, begin_x)

Return a new `window <curses-window-objects>`, whose left-upper corner is at `(begin_y, begin_x)`, and whose height/width is *nlines*/\*ncols\*.

By default, the window will extend from the specified position to the lower right corner of the screen.

</div>

<div class="function">

newpad(nlines, ncols)

Create and return a pointer to a new pad data structure with the given number of lines and columns. Return a pad as a window object.

A pad is like a window, except that it is not restricted by the screen size, and is not necessarily associated with a particular part of the screen. Pads can be used when a large window is needed, and only a part of the window will be on the screen at one time. Automatic refreshes of pads (such as from scrolling or echoing of input) do not occur. The `~window.refresh` and `~window.noutrefresh` methods of a pad require 6 arguments to specify the part of the pad to be displayed and the location on the screen to be used for the display. The arguments are *pminrow*, *pmincol*, *sminrow*, *smincol*, *smaxrow*, *smaxcol*; the *p* arguments refer to the upper-left corner of the pad region to be displayed and the *s* arguments define a clipping box on the screen within which the pad region is to be displayed.

</div>

### Soft labels

<div id="curses-slk">

The following functions manage *soft-label keys*, a row of labels displayed along the bottom line of the screen, typically used to label a row of function keys. `slk_init` must be called before `initscr` or `newterm`; it takes one screen line away from the standard window for the labels.

</div>

<div class="function">

slk_init(fmt=0)

Reserve a screen line for the soft labels and choose their layout. *fmt* selects the arrangement: `0` for 3-2-3 (eight labels), `1` for 4-4 (eight labels). Where the underlying curses library supports them, `2` gives 4-4-4 (twelve labels) and `3` gives 4-4-4 with an index line.

Must be called before `initscr` or `newterm`, and affects only the screen created next.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_set(labnum, label, justify)

Set the text of soft label number *labnum*, in the range `1` through `8` (or `12` in a twelve-label layout). *justify* controls how *label* is placed within the label: `0` for left, `1` for centered, `2` for right.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_label(labnum)

Return the current text of soft label number *labnum*, justified as it was set, or an empty string if it has no label.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_refresh()

Update the soft labels on the physical screen, like `~curses.window.refresh` for a window.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_noutrefresh()

Update the soft labels on the virtual screen, like `window.noutrefresh`. Use it together with `doupdate` to batch screen updates.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_clear()

Remove the soft labels from the screen.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_restore()

Restore the soft labels to the screen after a `slk_clear`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_touch()

Force all the soft labels to be redrawn by the next `slk_refresh` or `slk_noutrefresh`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_attron(attr) slk_attroff(attr) slk_attrset(attr)

Add, remove, or set the attributes used to display the soft labels, given as packed `A_*` attributes.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_attr()

Return the current attributes of the soft labels as packed `A_*` attributes. Availability depends on the underlying curses library.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_attr_on(attr) slk_attr_off(attr)

Turn the given attributes on or off without affecting any others. Like the `attr_*` window methods, these work with the `WA_* attributes <curses-wa-constants>` rather than packed `A_*` attributes.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_attr_set(attr, pair=0)

Set the attributes and color pair of the soft labels. *attr* is given as `WA_* attributes <curses-wa-constants>` and *pair* as a color pair number.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

slk_color(pair)

Set the color pair of the soft labels to color pair number *pair*.

<div class="versionadded">

3.16

</div>

</div>

### Saving and restoring

<div class="function">

getwin(file)

Read window-related data stored in the file by an earlier `window.putwin` call. The routine then creates and initializes a new window using that data, returning the new window object. The *file* argument must be a file object opened for reading in binary mode.

</div>

<div class="function">

scr_dump(filename)

Write the current contents of the virtual screen to *filename*, which may be a string or a `path-like object`. The file can later be read by `scr_restore`, `scr_init` or `scr_set`. This is the whole-screen counterpart of `window.putwin`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

scr_restore(filename)

Set the virtual screen to the contents of *filename*, which must have been written by `scr_dump`. The next call to `doupdate` or `window.refresh` restores the screen to those contents.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

scr_init(filename)

Initialize the assumed contents of the terminal from *filename*, which must have been written by `scr_dump`. Use it when the terminal already displays those contents, for example after another program has drawn the screen, so that curses does not redraw what is already there.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

scr_set(filename)

Use *filename*, which must have been written by `scr_dump`, as both the virtual screen and the assumed terminal contents. This combines the effects of `scr_restore` and `scr_init`.

<div class="versionadded">

3.16

</div>

</div>

### Querying the terminal

<div class="function">

baudrate()

Return the output speed of the terminal in bits per second. On software terminal emulators it will have a fixed high value. Included for historical reasons; in former times, it was used to write output loops for time delays and occasionally to change interfaces depending on the line speed.

</div>

<div class="function">

erasechar()

Return the user's current erase character as a raw byte, a `bytes` object of length 1. See also `erasewchar`.

</div>

<div class="function">

erasewchar()

Return the user's current erase character as a one-character `str`. Under Unix operating systems this is a property of the controlling tty of the curses program, and is not set by the curses library itself.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

killchar()

Return the user's current line kill character as a raw byte, a `bytes` object of length 1. See also `killwchar`.

</div>

<div class="function">

killwchar()

Return the user's current line kill character as a one-character `str`. Under Unix operating systems this is a property of the controlling tty of the curses program, and is not set by the curses library itself.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

termname()

Return the name of the terminal in use, as a bytes object. This is the name passed to `newterm` or `setupterm`, or the value of the `TERM` environment variable if none was given. Some curses libraries truncate it to 14 characters.

</div>

<div class="function">

longname()

Return a bytes object containing the terminfo long name field describing the current terminal. The maximum length of a verbose description is 128 characters. It is defined only after the call to `initscr`.

</div>

<div class="function">

termattrs()

Return a logical OR of all video attributes supported by the terminal. This information is useful when a curses program needs complete control over the appearance of the screen.

</div>

<div class="function">

term_attrs()

Like `termattrs`, but return the attributes as `WA_*
<curses-wa-constants>` values rather than `A_*` values.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

has_ic()

Return `True` if the terminal has insert- and delete-character capabilities. This function is included for historical reasons only, as all modern software terminal emulators have such capabilities.

</div>

<div class="function">

has_il()

Return `True` if the terminal has insert- and delete-line capabilities, or can simulate them using scrolling regions. This function is included for historical reasons only, as all modern software terminal emulators have such capabilities.

</div>

### Bell and screen flash

<div class="function">

beep()

Emit a short attention sound.

</div>

<div class="function">

flash()

Flash the screen. That is, change it to reverse-video and then change it back in a short interval. Some people prefer such as 'visible bell' to the audible attention signal produced by `beep`.

</div>

### Terminal resizing

<div class="function">

resizeterm(nlines, ncols)

Resize the standard and current windows to the specified dimensions, and adjusts other bookkeeping data used by the curses library that record the window dimensions (in particular the SIGWINCH handler).

</div>

<div class="function">

resize_term(nlines, ncols)

Backend function used by `resizeterm`, performing most of the work; when resizing the windows, `resize_term` blank-fills the areas that are extended. The calling application should fill in these areas with appropriate data. The `!resize_term` function attempts to resize all windows. However, due to the calling convention of pads, it is not possible to resize these without additional interaction with the application.

</div>

<div class="function">

is_term_resized(nlines, ncols)

Return `True` if `resize_term` would modify the window structure, `False` otherwise.

</div>

<div class="function">

update_lines_cols()

Update the `LINES` and `COLS` module variables. Useful for detecting manual screen resize.

<div class="versionadded">

3.5

</div>

</div>

### Terminfo database

<div class="function">

setupterm(term=None, fd=-1)

Initialize the terminal. *term* is a string giving the terminal name, or `None`; if omitted or `None`, the value of the `TERM` environment variable will be used. *fd* is the file descriptor to which any initialization sequences will be sent; if not supplied or `-1`, the file descriptor for `sys.stdout` will be used.

Raise a `curses.error` if the terminal could not be found or its terminfo database entry could not be read. If the terminal has already been initialized, this function has no effect.

> [!NOTE]
> Calling `initscr` or `newterm` after `setupterm` leaks the terminal that `setupterm` allocated: the curses library keeps only a single current terminal and does not free the previously allocated one.

</div>

<div class="function">

tigetflag(capname)

Return the value of the Boolean capability corresponding to the terminfo capability name *capname* as an integer. Return the value `-1` if *capname* is not a Boolean capability, or `0` if it is canceled or absent from the terminal description.

`setupterm` (or `initscr`) must be called first.

</div>

<div class="function">

tigetnum(capname)

Return the value of the numeric capability corresponding to the terminfo capability name *capname* as an integer. Return the value `-2` if *capname* is not a numeric capability, or `-1` if it is canceled or absent from the terminal description.

`setupterm` (or `initscr`) must be called first.

</div>

<div class="function">

tigetstr(capname)

Return the value of the string capability corresponding to the terminfo capability name *capname* as a bytes object. Return `None` if *capname* is not a terminfo "string capability", or is canceled or absent from the terminal description.

`setupterm` (or `initscr`) must be called first.

</div>

<div class="function">

tparm(str\[, ...\])

Instantiate the bytes object *str* with the supplied parameters, where *str* should be a parameterized byte string obtained from the terminfo database. For example, `tparm(tigetstr("cup"), 5, 3)` could result in `b'\033[6;4H'`, the exact result depending on terminal type. Up to nine integer parameters may be supplied.

`setupterm` (or `initscr`) must be called first.

</div>

<div class="function">

putp(str)

Equivalent to `tputs(str, 1, putchar)`; emit the value of a specified terminfo capability, a bytes object, for the current terminal. Note that the output of `putp` always goes to standard output.

`setupterm` (or `initscr`) must be called first.

</div>

### Utilities

<div class="function">

unctrl(ch)

Return a bytes object which is a printable representation of the character *ch*; any attributes and color pair are ignored. Control characters are represented as a caret followed by a character, for example as `b'^C'`. Printing characters are left as they are. The representation of other characters is defined by the underlying curses library.

*ch* must fit in a single byte; use `wunctrl` for other characters.

</div>

<div class="function">

wunctrl(ch)

Return a string which is a printable representation of the character *ch*; any attributes and color pair are ignored. ASCII control characters are represented as a caret followed by a character, for example as `'^C'`. Printing characters, including non-ASCII characters printable in the locale, are left as they are. The representation of other characters is defined by the underlying curses library.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

filter()

The `.filter` routine, if used, must be called before `initscr` or `newterm` is called, and affects every screen created afterwards. The effect is that, during the initialization, `LINES` is set to `1`; the capabilities `clear`, `cup`, `cud`, `cud1`, `cuu1`, `cuu`, `vpa` are disabled; and the `home` string is set to the value of `cr`. The effect is that the cursor is confined to the current line, and so are screen updates. This may be used for enabling character-at-a-time line editing without touching the rest of the screen.

</div>

<div class="function">

nofilter()

Undo the effect of a previous `.filter` call. Like `.filter`, it must be called before `initscr` (or `newterm`) so that the next initialization uses the full screen again.

Availability: if the underlying curses library provides `nofilter()`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

use_env(flag)

If used, this function should be called before `initscr` or `newterm` are called, and affects every screen created afterwards. When *flag* is `False`, the values of lines and columns specified in the terminfo database will be used, even if environment variables `LINES` and `COLUMNS` (used by default) are set, or if curses is running in a window (in which case default behavior would be to use the window size if `LINES` and `COLUMNS` are not set).

</div>

<div class="function">

get_escdelay()

Retrieves the value set by `set_escdelay`.

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

set_escdelay(ms)

Sets the number of milliseconds to wait after reading an escape character, to distinguish between an individual escape character entered on the keyboard from escape sequences sent by cursor and function keys.

Depending on the curses library, the setting may apply to all screens, not only to the current one.

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

get_tabsize()

Retrieves the value set by `set_tabsize`.

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

set_tabsize(size)

Sets the number of columns used by the curses library when converting a tab character to spaces as it adds the tab to a window.

Depending on the curses library, the setting may apply to all screens, not only to the current one, and creating a screen may reset it.

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

napms(ms)

Sleep for *ms* milliseconds.

</div>

<div class="function">

delay_output(ms)

Insert an *ms* millisecond pause in output.

</div>

## Window objects

<div class="window">

Window objects, as returned by `initscr` and `newwin` above, have the following methods and attributes:

</div>

### Adding and inserting text

<div class="method">

window.addch(ch\[, attr\]) window.addch(y, x, ch\[, attr\])

Paint character *ch* at `(y, x)` with attributes *attr*, overwriting any character previously painted at that location. By default, the character position and attributes are the current settings for the window object.

*ch* may be a single character, optionally followed by combining characters, that together occupy one character cell.

> [!NOTE]
> Writing outside the window, subwindow, or pad raises a `curses.error`. Attempting to write to the lower-right corner of a window, subwindow, or pad will cause an exception to be raised after the character is printed.

<div class="versionchanged">

3.16 A character may now be given as a string of a base character followed by combining characters, instead of only a single character, or as a `complexchar` cell.

</div>

</div>

<div class="method">

window.addstr(str\[, attr\]) window.addstr(y, x, str\[, attr\])

Paint the character string *str* at `(y, x)` with attributes *attr*, overwriting anything previously on the display.

*str* may also be a `complexstr`, in which case each cell carries its own attributes and color pair, so *attr* must not be given. A `complexstr` obtained from `in_wchstr` is written back unchanged.

> [!NOTE]
> - Writing outside the window, subwindow, or pad raises `curses.error`. Attempting to write to the lower-right corner of a window, subwindow, or pad will cause an exception to be raised after the string is printed.
> - A bug in ncurses, the backend for this Python module, could cause segfaults when resizing windows. This was fixed in ncurses-6.1-20190511. If you are stuck with an earlier ncurses, you can avoid triggering it by not calling `!addstr` with a *str* that has embedded newlines; instead, call `!addstr` separately for each line.

<div class="versionchanged">

3.16 *str* may now also be a `complexstr`, as described above.

</div>

</div>

<div class="method">

window.addnstr(str, n\[, attr\]) window.addnstr(y, x, str, n\[, attr\])

Paint at most *n* characters of the character string *str* at `(y, x)` with attributes *attr*, overwriting anything previously on the display.

<div class="versionchanged">

3.16 *str* may now also be a `complexstr`; see `addstr`.

</div>

</div>

<div class="method">

window.echochar(ch\[, attr\])

Add character *ch* with attribute *attr*, and immediately call `refresh` on the window.

<div class="versionchanged">

3.16 Wide and combining characters, and `complexchar` cells, are now accepted.

</div>

</div>

<div class="method">

window.insch(ch\[, attr\]) window.insch(y, x, ch\[, attr\])

Insert character *ch* with attributes *attr* before the character under the cursor, or at `(y, x)` if specified. All characters to the right of the cursor are shifted one position right, with the rightmost character on the line being lost. The cursor position does not change (after moving to *y*, *x*, if specified).

<div class="versionchanged">

3.16 Wide and combining characters, and `complexchar` cells, are now accepted.

</div>

</div>

<div class="method">

window.insstr(str\[, attr\]) window.insstr(y, x, str\[, attr\])

Insert a character string (as many characters as will fit on the line) before the character under the cursor. All characters to the right of the cursor are shifted right, with the rightmost characters on the line being lost. The cursor position does not change (after moving to *y*, *x*, if specified).

*str* may also be a `complexstr`, in which case each cell carries its own attributes and color pair, so *attr* must not be given.

<div class="versionchanged">

3.16 *str* may now also be a `complexstr`, as described above.

</div>

</div>

<div class="method">

window.insnstr(str, n\[, attr\]) window.insnstr(y, x, str, n\[, attr\])

Insert a character string (as many characters as will fit on the line) before the character under the cursor, up to *n* characters. If *n* is negative, the entire string is inserted. All characters to the right of the cursor are shifted right, with the rightmost characters on the line being lost. The cursor position does not change (after moving to *y*, *x*, if specified).

<div class="versionchanged">

3.16 *str* may now also be a `complexstr`; see `insstr`.

</div>

</div>

### Deleting and inserting lines

<div class="method">

window.delch(\[y, x\])

Delete the character under the cursor, or at `(y, x)` if specified. All characters to the right on the same line are shifted one position left.

</div>

<div class="method">

window.deleteln()

Delete the line under the cursor. All following lines are moved up by one line.

</div>

<div class="method">

window.insertln()

Insert a blank line under the cursor. All following lines are moved down by one line.

</div>

<div class="method">

window.insdelln(nlines)

Insert *nlines* lines into the specified window above the current line. The *nlines* bottom lines are lost. For negative *nlines*, delete *nlines* lines starting with the one under the cursor, and move the remaining lines up. The bottom *nlines* lines are cleared. The current cursor position remains the same.

</div>

### Reading input

<div class="method">

window.getch(\[y, x\])

Read a key press, after moving the cursor to *y*, *x* if specified, and return it as an integer. The window is refreshed first if it is not a pad and was modified since the last refresh. Wait until a key is pressed, or return `-1` if the read is non-blocking or times out (see `nodelay` and `timeout`).

An ordinary key is returned as the code of a single byte of its encoding in the current locale, so a character encoded with several bytes takes several calls. For example, in a UTF-8 locale `'é'` is read as `195`, then `169`. Use `get_wch` to read it as a single character.

In keypad mode (see `keypad`) function keys and other special keys are returned as one of the `KEY_* constants <curses-key-constants>`, which cannot be mistaken for an ordinary key. Otherwise, or if their escape sequence does not arrive in time (see `notimeout` and `set_escdelay`), their bytes are returned one at a time.

In echo mode (see `echo`) the key is added to the window as by `addch`; special keys are not echoed.

</div>

<div class="method">

window.get_wch(\[y, x\])

Read a key press, after moving the cursor to *y*, *x* if specified, and return it as a one-character `str`. The window is refreshed first if it is not a pad and was modified since the last refresh. Wait until a key is pressed, or raise `error` if the read is non-blocking or times out (see `nodelay` and `timeout`).

In keypad mode (see `keypad`) function keys and other special keys are returned as one of the `KEY_* constants <curses-key-constants>`, an integer. Otherwise, or if their escape sequence does not arrive in time (see `notimeout` and `set_escdelay`), their characters are returned one at a time.

In echo mode (see `echo`) the key is added to the window as by `addch`; special keys are not echoed.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.16 Also available on a narrow build, where only a character representable as a single byte (an 8-bit locale) can be returned.

</div>

</div>

<div class="method">

window.getkey(\[y, x\])

Read a key press as `getch` does, but return it as a `str`: an ordinary key as a one-character string, the byte decoded as Latin-1, and a special key as its name, such as `'KEY_UP'` (see `keyname`). Raise `error` instead of returning `-1` if there is no input.

</div>

<div class="method">

window.getstr() window.getstr(n) window.getstr(y, x) window.getstr(y, x, n)

Read a line of input from the user, with primitive line editing capacity, after moving the cursor to *y*, *x* if specified. Return it as a bytes object, in the encoding of the current locale and without the terminating newline. At most *n* bytes are read; *n* defaults to and cannot exceed 2047.

Use `get_wstr` to read the input as a `str`.

<div class="versionchanged">

3.14 The maximum value for *n* was increased from 1023 to 2047.

</div>

</div>

<div class="method">

window.get_wstr() window.get_wstr(n) window.get_wstr(y, x) window.get_wstr(y, x, n)

Read a line of input from the user, with primitive line editing capacity, after moving the cursor to *y*, *x* if specified. Return it as a `str`, without the terminating newline. At most *n* characters are read; *n* defaults to and cannot exceed 2047.

This is the wide-character variant of `getstr`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.getdelay()

Return the window's read timeout in milliseconds, as set by `nodelay` or `timeout`: `-1` for blocking, `0` for non-blocking, or a positive number of milliseconds.

<div class="versionadded">

3.16

</div>

</div>

### Reading window contents

<div class="method">

window.inch(\[y, x\])

Return the character at the given position in the window. The bottom 8 bits are the character proper and the upper bits are the attributes; extract them with the `A_CHARTEXT` and `A_ATTRIBUTES` bit-masks, and the color pair with `pair_number`. The character byte is the locale-encoded byte of the cell's character, consistent with `instr`. It cannot represent a cell holding combining characters, a character that does not fit in a single byte, or a color pair outside the `color_pair` range; use `in_wch` for those, which returns it as a `complexchar`.

</div>

<div class="method">

window.in_wch(\[y, x\])

Return the complex character at the given position in the window as a `complexchar`. Unlike `inch`, the returned object carries the cell's text (a spacing character optionally followed by combining characters) together with its attributes and color pair.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.instr(\[n\]) window.instr(y, x\[, n\])

Read the text of the window from the current cursor position, or from *y*, *x* if specified, to the end of the line or at most *n* bytes if *n* is specified, and return it as a bytes object, in the encoding of the current locale. Attributes and color pairs are stripped; use `in_wchstr` to read them too. A character not representable in the encoding cannot be returned; use `in_wstr` for those.

<div class="versionchanged">

3.14 The maximum value for *n* was increased from 1023 to 2047.

</div>

<div class="versionchanged">

3.16 *n* is no longer limited to 2047.

</div>

</div>

<div class="method">

window.in_wstr(\[n\]) window.in_wstr(y, x\[, n\])

Read the text of the window from the current cursor position, or from *y*, *x* if specified, to the end of the line or at most *n* characters if *n* is specified, and return it as a `str`. Attributes and color pairs are stripped; use `in_wchstr` to read them too.

This is the wide-character variant of `instr`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.in_wchstr(\[n\]) window.in_wchstr(y, x\[, n\])

Read the styled cells of the window from the current cursor position, or from *y*, *x* if specified, to the end of the line or at most *n* cells if *n* is specified, and return them as a `complexstr`. Unlike `instr` and `in_wstr`, each cell keeps its attributes and color pair, so the result can be written back unchanged with `addstr`.

<div class="versionadded">

3.16

</div>

</div>

### Attributes

<div class="method">

window.attroff(attr)

Remove attribute *attr* from the "background" set applied to all writes to the current window.

</div>

<div class="method">

window.attron(attr)

Add attribute *attr* to the "background" set applied to all writes to the current window.

</div>

<div class="method">

window.attrset(attr)

Set the "background" set of attributes to *attr*. This set is initially `0` (no attributes).

</div>

<div class="method">

window.attr_get()

Return the window's current rendition as a `(attrs, pair)` tuple, where *attrs* is the set of attributes and *pair* is the color pair number.

Unlike `attron` and friends, which take packed `A_*` attributes, this method and the other `attr_*` methods work with the `WA_* attributes <curses-wa-constants>` and keep the color pair as a separate number, which lets them use color pairs that do not fit alongside the attributes in a single value.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.attr_set(attr, pair=0)

Set the window's rendition to the attributes *attr* and the color pair *pair*.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.attr_on(attr)

Turn on the attributes *attr* without affecting any others.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.attr_off(attr)

Turn off the attributes *attr* without affecting any others.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.color_set(pair)

Set the window's color pair to *pair*, leaving the other attributes unchanged.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.getattrs()

Return the window's current attributes.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.standout()

Turn on attribute *A_STANDOUT*.

</div>

<div class="method">

window.standend()

Turn off the standout attribute. On some terminals this has the side effect of turning off all attributes.

</div>

<div class="method">

window.chgat(attr) window.chgat(num, attr) window.chgat(y, x, attr) window.chgat(y, x, num, attr)

Set the attributes of *num* characters at the current cursor position, or at position `(y, x)` if supplied. If *num* is not given or is `-1`, the attribute will be set on all the characters to the end of the line. This function moves cursor to position `(y, x)` if supplied. The changed line will be touched using the `touchline` method so that the contents will be redisplayed by the next window refresh.

</div>

### Background

<div class="method">

window.bkgd(ch\[, attr\])

Set the background property of the window to the character *ch*, with attributes *attr*. The change is then applied to every character position in that window:

- The attribute of every character in the window is changed to the new background attribute.
- Wherever the former background character appears, it is changed to the new background character.

<div class="versionchanged">

3.16 Wide and combining characters, and `complexchar` cells, are now accepted.

</div>

</div>

<div class="method">

window.bkgdset(ch\[, attr\])

Set the window's background. A window's background consists of a character and any combination of attributes. The attribute part of the background is combined (OR'ed) with all non-blank characters that are written into the window. Both the character and attribute parts of the background are combined with the blank characters. The background becomes a property of the character and moves with the character through any scrolling and insert/delete line/character operations.

<div class="versionchanged">

3.16 Wide and combining characters, and `complexchar` cells, are now accepted.

</div>

</div>

<div class="method">

window.getbkgd()

Return the given window's current background character/attribute pair. Its components can be extracted like those of `inch`. It cannot represent a background set with a wide character or with a color pair outside the `color_pair` range; use `getbkgrnd` for those.

</div>

<div class="method">

window.getbkgrnd()

Return the given window's current background as a `complexchar`. Unlike `getbkgd`, the returned object carries the background character together with its attributes and color pair, and the color pair is not limited to the value that fits in a `color_pair`.

<div class="versionadded">

3.16

</div>

</div>

### Clearing and erasing

<div class="method">

window.erase()

Clear the window.

</div>

<div class="method">

window.clear()

Like `erase`, but also cause the whole window to be repainted upon next call to `refresh`.

</div>

<div class="method">

window.clrtobot()

Erase from cursor to the end of the window: all lines below the cursor are deleted, and then the equivalent of `clrtoeol` is performed.

</div>

<div class="method">

window.clrtoeol()

Erase from cursor to the end of the line.

</div>

### Borders and lines

<div class="method">

window.border(\[ls\[, rs\[, ts\[, bs\[, tl\[, tr\[, bl\[, br\]\]\]\]\]\]\]\])

Draw a border around the edges of the window. Each parameter specifies the character to use for a specific part of the border; see the table below for more details.

> [!NOTE]
> A `0` value for any parameter will cause the default character to be used for that parameter. Keyword parameters can *not* be used. The defaults are listed in this table:

| Parameter | Description         | Default value  | Wide default value |
|-----------|---------------------|----------------|--------------------|
| *ls*      | Left side           | `ACS_VLINE`    | `WACS_VLINE`       |
| *rs*      | Right side          | `ACS_VLINE`    | `WACS_VLINE`       |
| *ts*      | Top                 | `ACS_HLINE`    | `WACS_HLINE`       |
| *bs*      | Bottom              | `ACS_HLINE`    | `WACS_HLINE`       |
| *tl*      | Upper-left corner   | `ACS_ULCORNER` | `WACS_ULCORNER`    |
| *tr*      | Upper-right corner  | `ACS_URCORNER` | `WACS_URCORNER`    |
| *bl*      | Bottom-left corner  | `ACS_LLCORNER` | `WACS_LLCORNER`    |
| *br*      | Bottom-right corner | `ACS_LRCORNER` | `WACS_LRCORNER`    |

The wide default value is used when the border is drawn from string characters or `complexchar` cells.

If any parameter is a byte character or an integer other than `0`, the border is drawn from byte characters, and every string character must be encodable as a single byte.

<div class="versionchanged">

3.16 Wide and combining characters, and `complexchar` cells, are now accepted. A single call cannot mix `complexchar` cells with integer or byte characters.

</div>

</div>

<div class="method">

window.box(\[vertch, horch\])

Similar to `border`, but both *ls* and *rs* are *vertch* and both *ts* and *bs* are *horch*. The default corner characters are always used by this function.

<div class="versionchanged">

3.16 Wide and combining characters, and `complexchar` cells, are now accepted. A single call cannot mix `complexchar` cells with integer or byte characters.

</div>

</div>

<div class="method">

window.hline(ch, n\[, attr\]) window.hline(y, x, ch, n\[, attr\])

Display a horizontal line starting at `(y, x)` with length *n* consisting of the character *ch* with attributes *attr*. The line stops at the right edge of the window if fewer than *n* cells are available.

<div class="versionchanged">

3.16 Wide and combining characters, and `complexchar` cells, are now accepted.

</div>

</div>

<div class="method">

window.vline(ch, n\[, attr\]) window.vline(y, x, ch, n\[, attr\])

Display a vertical line starting at `(y, x)` with length *n* consisting of the character *ch* with attributes *attr*.

<div class="versionchanged">

3.16 Wide and combining characters, and `complexchar` cells, are now accepted.

</div>

</div>

### Cursor position and window geometry

<div class="method">

window.move(new_y, new_x)

Move cursor to `(new_y, new_x)`.

</div>

<div class="method">

window.getyx()

Return a tuple `(y, x)` of current cursor position relative to the window's upper-left corner.

</div>

<div class="method">

window.getbegyx()

Return a tuple `(y, x)` of coordinates of upper-left corner.

</div>

<div class="method">

window.getmaxyx()

Return a tuple `(y, x)` of the height and width of the window.

</div>

<div class="method">

window.getparyx()

Return the beginning coordinates of this window relative to its parent window as a tuple `(y, x)`. Return `(-1, -1)` if this window has no parent.

</div>

<div class="method">

window.getparent()

Return the parent window of this subwindow, or `None` if this window is not a subwindow.

<div class="versionadded">

3.16

</div>

</div>

### Creating and resizing windows

<div class="method">

window.subwin(begin_y, begin_x) window.subwin(nlines, ncols, begin_y, begin_x)

Return a sub-window, whose upper-left corner is at the screen-relative coordinates `(begin_y, begin_x)`, and whose width/height is *ncols*/\*nlines\*.

By default, the sub-window will extend from the specified position to the lower right corner of the window.

</div>

<div class="method">

window.derwin(begin_y, begin_x) window.derwin(nlines, ncols, begin_y, begin_x)

An abbreviation for "derive window", `derwin` is the same as calling `subwin`, except that *begin_y* and *begin_x* are relative to the origin of the window, rather than relative to the entire screen. Return a window object for the derived window.

</div>

<div class="method">

window.subpad(begin_y, begin_x) window.subpad(nlines, ncols, begin_y, begin_x)

Return a sub-pad, whose upper-left corner is at `(begin_y, begin_x)`, and whose width/height is *ncols*/\*nlines\*. The coordinates are relative to the parent pad (unlike `subwin`, which uses screen coordinates). This method is only available for pads created with `newpad`.

</div>

<div class="method">

window.dupwin()

Return a new window that is an exact duplicate of the window: it has the same size, position, contents and attributes. Unlike a window created by `subwin` or `derwin`, the duplicate is independent of the original -- it has its own cell buffer, so later changes to one do not affect the other.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.mvwin(new_y, new_x)

Move the window so its upper-left corner is at `(new_y, new_x)`.

Moving the window so that any part of it would be off the screen is an error: the window is not moved and `curses.error` is raised.

</div>

<div class="method">

window.mvderwin(y, x)

Move the window inside its parent window. The screen-relative parameters of the window are not changed. This routine is used to display different parts of the parent window at the same physical position on the screen.

</div>

<div class="method">

window.resize(nlines, ncols)

Reallocate storage for a curses window to adjust its dimensions to the specified values. If either dimension is larger than the current values, the window's data is filled with blanks that have the current background rendition (as set by `bkgdset`) merged into them.

</div>

### Refreshing and redrawing

<div class="method">

window.refresh(\[pminrow, pmincol, sminrow, smincol, smaxrow, smaxcol\])

Update the display immediately (sync actual screen with previous drawing/deleting methods).

The 6 arguments can only be specified, and are then required, when the window is a pad created with `newpad`. The additional parameters are needed to indicate what part of the pad and screen are involved. *pminrow* and *pmincol* specify the upper-left corner of the rectangle to be displayed in the pad. *sminrow*, *smincol*, *smaxrow*, and *smaxcol* specify the edges of the rectangle to be displayed on the screen. The lower-right corner of the rectangle to be displayed in the pad is calculated from the screen coordinates, since the rectangles must be the same size. Both rectangles must be entirely contained within their respective structures. Negative values of *pminrow*, *pmincol*, *sminrow*, or *smincol* are treated as if they were zero.

</div>

<div class="method">

window.noutrefresh() window.noutrefresh(pminrow, pmincol, sminrow, smincol, smaxrow, smaxcol)

Mark for refresh but wait. This function updates the data structure representing the desired state of the window, but does not force an update of the physical screen. To accomplish that, call `doupdate`.

The 6 arguments can only be specified, and are then required, when the window is a pad created with `newpad`; they have the same meaning as for `refresh`.

</div>

<div class="method">

window.redrawln(beg, num)

Indicate that the *num* screen lines, starting at line *beg*, are corrupted and should be completely redrawn on the next `refresh` call.

</div>

<div class="method">

window.redrawwin()

Touch the entire window, causing it to be completely redrawn on the next `refresh` call.

</div>

### Output options

<div class="method">

window.clearok(flag)

If *flag* is `True`, the next call to `refresh` will clear the window completely.

</div>

<div class="method">

window.idlok(flag)

If *flag* is `True`, `!curses` will try to use hardware line editing facilities. Otherwise, curses will not use them.

</div>

<div class="method">

window.idcok(flag)

If *flag* is `False`, curses no longer considers using the hardware insert/delete character feature of the terminal; if *flag* is `True`, use of character insertion and deletion is enabled. When curses is first initialized, use of character insert/delete is enabled by default.

</div>

<div class="method">

window.immedok(flag)

If *flag* is `True`, any change in the window image automatically causes the window to be refreshed; you no longer have to call `refresh` yourself. However, it may degrade performance considerably, due to repeated calls to wrefresh. This option is disabled by default.

</div>

<div class="method">

window.leaveok(flag)

If *flag* is `True`, cursor is left where it is on update, instead of being at "cursor position." This reduces cursor movement where possible.

If *flag* is `False`, cursor will always be at "cursor position" after an update.

</div>

<div class="method">

window.scrollok(flag)

Control what happens when the cursor of a window is moved off the edge of the window or scrolling region, either as a result of a newline action on the bottom line, or typing the last character of the last line. If *flag* is `False`, the cursor is left on the bottom line. If *flag* is `True`, the window is scrolled up one line. Note that in order to get the physical scrolling effect on the terminal, it is also necessary to call `idlok`.

</div>

<div class="method">

window.scroll(\[lines=1\])

Scroll the screen or scrolling region. Scroll upward by *lines* lines if *lines* is positive, or downward if it is negative. Raise `curses.error` unless scrolling has been enabled for the window with `scrollok`.

</div>

<div class="method">

window.setscrreg(top, bottom)

Set the scrolling region from line *top* to line *bottom*. All scrolling actions will take place in this region.

</div>

<div class="method">

window.getscrreg()

Return a tuple `(top, bottom)` of the window's current scrolling region, as set by `setscrreg`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.syncok(flag)

If *flag* is `True`, then `syncup` is called automatically whenever there is a change in the window.

</div>

### Input options

<div class="method">

window.keypad(flag)

If *flag* is `True`, escape sequences generated by some keys (keypad, function keys) will be interpreted by `!curses`. If *flag* is `False`, escape sequences will be left as is in the input stream. Keypad mode is disabled by default, but `wrapper` enables it for the main window.

</div>

<div class="method">

window.nodelay(flag)

If *flag* is `True`, `getch` will be non-blocking.

</div>

<div class="method">

window.notimeout(flag)

If *flag* is `True`, escape sequences will not be timed out.

If *flag* is `False`, after a few milliseconds, an escape sequence will not be interpreted, and will be left in the input stream as is.

</div>

<div class="method">

window.timeout(delay)

Set blocking or non-blocking read behavior for the window. If *delay* is negative, blocking read is used (which will wait indefinitely for input). If *delay* is zero, then non-blocking read is used, and `getch` will return `-1` if no input is waiting. If *delay* is positive, then `getch` will block for *delay* milliseconds, and return `-1` if there is still no input at the end of that time.

</div>

### Overlapping and touch

<div class="method">

window.overlay(destwin\[, sminrow, smincol, dminrow, dmincol, dmaxrow, dmaxcol\])

Overlay the window on top of *destwin*. The windows need not be the same size, only the overlapping region is copied. This copy is non-destructive, which means that the current background character does not overwrite the old contents of *destwin*.

To get fine-grained control over the copied region, the second form of `overlay` can be used. *sminrow* and *smincol* are the upper-left coordinates of the source window, and the other variables mark a rectangle in the destination window.

</div>

<div class="method">

window.overwrite(destwin\[, sminrow, smincol, dminrow, dmincol, dmaxrow, dmaxcol\])

Overwrite the window on top of *destwin*. The windows need not be the same size, in which case only the overlapping region is copied. This copy is destructive, which means that the current background character overwrites the old contents of *destwin*.

To get fine-grained control over the copied region, the second form of `overwrite` can be used. *sminrow* and *smincol* are the upper-left coordinates of the source window, the other variables mark a rectangle in the destination window.

</div>

<div class="method">

window.touchline(start, count\[, changed\])

Pretend *count* lines have been changed, starting with line *start*. If *changed* is supplied, it specifies whether the affected lines are marked as having been changed (*changed*`=True`) or unchanged (*changed*`=False`).

</div>

<div class="method">

window.touchwin()

Pretend the whole window has been changed, for purposes of drawing optimizations.

</div>

<div class="method">

window.untouchwin()

Mark all lines in the window as unchanged since the last call to `refresh`.

</div>

<div class="method">

window.syncup()

Touch all locations in ancestors of the window that have been changed in the window.

</div>

<div class="method">

window.syncdown()

Touch each location in the window that has been touched in any of its ancestor windows. This routine is called by `refresh`, so it should almost never be necessary to call it manually.

</div>

<div class="method">

window.cursyncup()

Update the current cursor position of all the ancestors of the window to reflect the current cursor position of the window.

</div>

### Coordinate conversion

<div class="method">

window.enclose(y, x)

Test whether the given pair of screen-relative character-cell coordinates are enclosed by the given window, returning `True` or `False`. It is useful for determining what subset of the screen windows enclose the location of a mouse event.

<div class="versionchanged">

3.10 Previously it returned `1` or `0` instead of `True` or `False`.

</div>

</div>

<div class="method">

window.mouse_trafo(y, x, to_screen)

Convert between window-relative and screen-relative (`stdscr`-relative) character-cell coordinates. If *to_screen* is true, convert the window-relative coordinates *y*, *x* to screen-relative coordinates; otherwise convert in the opposite direction. The two coordinate systems differ when lines are reserved on the screen, for example for soft labels.

Return the converted coordinates as a `(y, x)` tuple, or `None` if they lie outside the window.

<div class="versionadded">

3.16

</div>

</div>

### Other

<div class="attribute">

window.encoding

Encoding used to encode the string arguments of the methods and to decode their results on a build without wide-character support. The encoding attribute is inherited from the parent window when a subwindow is created, for example with `window.subwin`. By default, current locale encoding is used (see `locale.getencoding`).

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

window.putwin(file)

Write all data associated with the window into the provided file object. This information can be later retrieved using the `getwin` function.

</div>

<div class="method">

window.use(func, /, *args,*\*kwargs)

Call `func(window, *args, **kwargs)` with the lock of the window held, and return its result. This provides automatic protection for the window against concurrent access from another thread.

Availability: if the underlying curses library provides `use_window()`.

<div class="versionadded">

3.16

</div>

</div>

### State queries

<div class="method">

window.is_cleared()

Return the current value set by `clearok`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_idcok()

Return the current value set by `idcok`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_idlok()

Return the current value set by `idlok`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_immedok()

Return the current value set by `immedok`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_keypad()

Return the current value set by `keypad`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_leaveok()

Return the current value set by `leaveok`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_linetouched(line)

Return `True` if the specified line was modified since the last call to `refresh`; otherwise return `False`. Raise a `curses.error` exception if *line* is not valid for the given window.

</div>

<div class="method">

window.is_nodelay()

Return the current value set by `nodelay`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_notimeout()

Return the current value set by `notimeout`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_pad()

Return `True` if the window is a pad created by `newpad`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_scrollok()

Return the current value set by `scrollok`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_subwin()

Return `True` if the window is a subwindow created by `subwin` or `derwin`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_syncok()

Return the current value set by `syncok`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

window.is_wintouched()

Return `True` if the specified window was modified since the last call to `refresh`; otherwise return `False`.

</div>

## Screen objects

<div class="screen">

A *screen* object represents a terminal initialized by `newterm` (or `new_prescr`), in addition to the default screen created by `initscr`. Screen objects are returned by those functions; they cannot be instantiated directly.

A screen is freed automatically once it is no longer referenced, either directly or through one of its windows. Each window keeps its screen alive, so a screen remains valid as long as any of its windows does.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

screen.close()

Detach the screen's standard window, breaking the reference cycle between them so the screen can be reclaimed promptly instead of waiting for a garbage collection. Afterwards `~screen.stdscr` is `None` and the window it returned earlier can no longer be used. The screen's resources are released once it and all its windows are no longer referenced.

<div class="versionadded">

3.16

</div>

</div>

<div class="attribute">

screen.stdscr

The standard `window <curses-window-objects>` of the screen, covering the whole terminal, or `None` for a screen created by `new_prescr`.

</div>

<div class="method">

screen.use(func, /, *args,*\*kwargs)

Call `func(screen, *args, **kwargs)` with the lock of the screen held, and return its result. This provides automatic protection for the screen against concurrent access from another thread.

Availability: if the underlying curses library provides `use_screen()`.

<div class="versionadded">

3.16

</div>

</div>

## Complex character objects

<div class="complexchar(text, /, attr=0, pair=0)">

A *complex character* (or *complexchar*) is an immutable styled character cell: a spacing character optionally followed by combining characters, together with a set of attributes and a color pair.

*text* is the cell's text, *attr* a combination of the `WA_* attributes <curses-wa-constants>` (equivalent to the matching `A_*` constants), and *pair* a color pair number. Unlike the packed `chtype <int>` used by `~window.inch` and the `A_*` methods, the color pair is stored separately and is not limited to the value that fits in a `color_pair`.

Complex characters are returned by `window.in_wch` and `window.getbkgrnd`, and are accepted (along with an integer, a byte or a string) by the character-cell methods such as `window.addch`, `window.insch`, `window.bkgd`, `window.border`, `window.hline` and `window.vline`. A complex character already carries its own rendition, so it cannot be combined with an explicit *attr* argument.

`str` returns the cell's text; two complex characters are equal when their text, attributes and color pair all match.

The same code works on both wide- and narrow-character builds. On a narrow build a cell holds a single character (no combining marks) that must encode to one byte in the window's encoding (8-bit locales only), and *pair* is limited to the value that fits in a `color_pair`.

<div class="attribute">

attr

The attributes of the character cell (read-only).

</div>

<div class="attribute">

pair

The color pair number of the character cell (read-only).

</div>

<div class="versionadded">

3.16

</div>

</div>

<div class="complexstr(cells[, attr[, pair]])">

A *complex character string* (or *complexstr*) is an immutable sequence of styled character cells -- the string counterpart of `complexchar` (as `str` is to a single character).

If *cells* is a string, it is split into character cells (each a spacing character optionally followed by combining characters), and *attr* (a combination of the `WA_* attributes <curses-wa-constants>`) and *pair* (a color pair number), if given, are applied to every cell.

Otherwise *cells* is an iterable whose items are themselves cells, each a `complexchar` or a string; each item then carries its own rendition, and *attr* and *pair* must be omitted.

It is returned by `window.in_wchstr`, and accepted by `window.addstr`, `~window.addnstr`, `~window.insstr` and `~window.insnstr`, so a run read from a window can be written back unchanged.

It behaves like an immutable sequence: `len(s)` is the number of cells, `s[i]` is the *i*-th cell as a `complexchar`, slicing and concatenation produce new `!complexstr` instances, and iterating yields the cells. `str` returns the cells' text joined together, and two complex character strings are equal when their cells all match. It is hashable.

To build or edit a run of cells, use an ordinary `list` of `complexchar` (or strings); a `!complexstr` is the immutable form returned by a read.

Like `complexchar`, this type works on both wide- and narrow-character builds, with the same per-cell limitations on a narrow build.

<div class="versionadded">

3.16

</div>

</div>

## Constants

### General

The `!curses` module defines the following data members:

<div class="data">

ERR

Some curses routines that return an integer, such as `~window.getch`, return `ERR` upon failure.

</div>

<div class="data">

OK

Some curses routines that return an integer, such as `napms`, return `OK` upon success.

</div>

<div class="data">

version

A bytes object representing the current version of the module.

</div>

<div class="data">

ncurses_version

A named tuple containing the three components of the ncurses library version: *major*, *minor*, and *patch*. All values are integers. The components can also be accessed by name, so `curses.ncurses_version[0]` is equivalent to `curses.ncurses_version.major` and so on.

Availability: if the ncurses library is used.

<div class="versionadded">

3.8

</div>

</div>

<div class="data">

COLORS

The maximum number of colors the terminal can support. It is defined only after the call to `start_color`.

</div>

<div class="data">

COLOR_PAIRS

The maximum number of color pairs the terminal can support. It is defined only after the call to `start_color`.

</div>

<div class="data">

COLS

The width of the screen, that is, the number of columns. It is defined only after the call to `initscr`. Updated by `update_lines_cols`, `resizeterm` and `resize_term`.

</div>

<div class="data">

LINES

The height of the screen, that is, the number of lines. It is defined only after the call to `initscr`. Updated by `update_lines_cols`, `resizeterm` and `resize_term`.

</div>

### Attributes

Some constants are available to specify character cell attributes. The exact constants available are system dependent.

<div id="curses-wa-constants">

<table style="width:98%;">
<colgroup>
<col style="width: 26%" />
<col style="width: 27%" />
<col style="width: 45%" />
</colgroup>
<thead>
<tr>
<th><p>Attribute</p></th>
<th><p>Wide attribute</p></th>
<th><p>Meaning</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>A_ALTCHARSET</p>
</div></td>
<td><div class="data">
<p>WA_ALTCHARSET</p>
</div></td>
<td><p>Alternate character set mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_BLINK</p>
</div></td>
<td><div class="data">
<p>WA_BLINK</p>
</div></td>
<td><p>Blink mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_BOLD</p>
</div></td>
<td><div class="data">
<p>WA_BOLD</p>
</div></td>
<td><p>Bold mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_DIM</p>
</div></td>
<td><div class="data">
<p>WA_DIM</p>
</div></td>
<td><p>Dim mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_INVIS</p>
</div></td>
<td><div class="data">
<p>WA_INVIS</p>
</div></td>
<td><p>Invisible or blank mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_ITALIC</p>
</div></td>
<td><div class="data">
<p>WA_ITALIC</p>
</div></td>
<td><p>Italic mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_NORMAL</p>
</div></td>
<td><div class="data">
<p>WA_NORMAL</p>
</div></td>
<td><p>Normal attribute</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_PROTECT</p>
</div></td>
<td><div class="data">
<p>WA_PROTECT</p>
</div></td>
<td><p>Protected mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_REVERSE</p>
</div></td>
<td><div class="data">
<p>WA_REVERSE</p>
</div></td>
<td><p>Reverse background and foreground colors</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_STANDOUT</p>
</div></td>
<td><div class="data">
<p>WA_STANDOUT</p>
</div></td>
<td><p>Standout mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_UNDERLINE</p>
</div></td>
<td><div class="data">
<p>WA_UNDERLINE</p>
</div></td>
<td><p>Underline mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_HORIZONTAL</p>
</div></td>
<td><div class="data">
<p>WA_HORIZONTAL</p>
</div></td>
<td><p>Horizontal highlight</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_LEFT</p>
</div></td>
<td><div class="data">
<p>WA_LEFT</p>
</div></td>
<td><p>Left highlight</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_LOW</p>
</div></td>
<td><div class="data">
<p>WA_LOW</p>
</div></td>
<td><p>Low highlight</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_RIGHT</p>
</div></td>
<td><div class="data">
<p>WA_RIGHT</p>
</div></td>
<td><p>Right highlight</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_TOP</p>
</div></td>
<td><div class="data">
<p>WA_TOP</p>
</div></td>
<td><p>Top highlight</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_VERTICAL</p>
</div></td>
<td><div class="data">
<p>WA_VERTICAL</p>
</div></td>
<td><p>Vertical highlight</p></td>
</tr>
</tbody>
</table>

</div>

<div class="versionadded">

3.7 `A_ITALIC` was added.

</div>

The `~window.attr_get`, `~window.attr_set`, `~window.attr_on` and `~window.attr_off` methods use the parallel set of `WA_*` constants listed above. Each has the same meaning as the corresponding `A_*` attribute (`WA_BOLD` like `A_BOLD`, and so on), but belongs to the `attr_t` type rather than being packed into a character. In ncurses the two sets share the same values, but other curses implementations may give them different ones, so use the `WA_*` constants with the `attr_*` methods.

<div class="versionadded">

3.16 The `WA_*` constants were added.

</div>

Several constants are available to extract corresponding attributes returned by some methods.

<table style="width:99%;">
<colgroup>
<col style="width: 24%" />
<col style="width: 25%" />
<col style="width: 48%" />
</colgroup>
<thead>
<tr>
<th><p>Bit-mask</p></th>
<th><p>Wide bit-mask</p></th>
<th><p>Meaning</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>A_ATTRIBUTES</p>
</div></td>
<td><div class="data">
<p>WA_ATTRIBUTES</p>
</div></td>
<td><p>Bit-mask to extract attributes</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_CHARTEXT</p>
</div></td>
<td></td>
<td><p>Bit-mask to extract a character</p></td>
</tr>
<tr>
<td><div class="data">
<p>A_COLOR</p>
</div></td>
<td></td>
<td><p>Bit-mask to extract color-pair field information</p></td>
</tr>
</tbody>
</table>

### Keys

Keys are referred to by integer constants with names starting with `KEY_`. The exact keycaps available are system dependent.

<table style="width:98%;">
<colgroup>
<col style="width: 36%" />
<col style="width: 62%" />
</colgroup>
<thead>
<tr>
<th><p>Key constant</p></th>
<th><p>Key</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>KEY_MIN</p>
</div></td>
<td><p>Minimum key value</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_BREAK</p>
</div></td>
<td><p>Break key (unreliable)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_DOWN</p>
</div></td>
<td><p>Down-arrow</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_UP</p>
</div></td>
<td><p>Up-arrow</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_LEFT</p>
</div></td>
<td><p>Left-arrow</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_RIGHT</p>
</div></td>
<td><p>Right-arrow</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_HOME</p>
</div></td>
<td><p>Home key (upward+left arrow)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_BACKSPACE</p>
</div></td>
<td><p>Backspace (unreliable)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_F0</p>
</div></td>
<td><p>Function keys. Up to 64 function keys are supported.</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_Fn</p>
</div></td>
<td><p>Value of function key <em>n</em></p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_DL</p>
</div></td>
<td><p>Delete line</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_IL</p>
</div></td>
<td><p>Insert line</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_DC</p>
</div></td>
<td><p>Delete character</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_IC</p>
</div></td>
<td><p>Insert char or enter insert mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_EIC</p>
</div></td>
<td><p>Exit insert char mode</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_CLEAR</p>
</div></td>
<td><p>Clear screen</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_EOS</p>
</div></td>
<td><p>Clear to end of screen</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_EOL</p>
</div></td>
<td><p>Clear to end of line</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SF</p>
</div></td>
<td><p>Scroll 1 line forward</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SR</p>
</div></td>
<td><p>Scroll 1 line backward (reverse)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_NPAGE</p>
</div></td>
<td><p>Next page</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_PPAGE</p>
</div></td>
<td><p>Previous page</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_STAB</p>
</div></td>
<td><p>Set tab</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_CTAB</p>
</div></td>
<td><p>Clear tab</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_CATAB</p>
</div></td>
<td><p>Clear all tabs</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_ENTER</p>
</div></td>
<td><p>Enter or send (unreliable)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SRESET</p>
</div></td>
<td><p>Soft (partial) reset (unreliable)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_RESET</p>
</div></td>
<td><p>Reset or hard reset (unreliable)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_PRINT</p>
</div></td>
<td><p>Print</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_LL</p>
</div></td>
<td><p>Home down or bottom (lower left)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_A1</p>
</div></td>
<td><p>Upper left of keypad</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_A3</p>
</div></td>
<td><p>Upper right of keypad</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_B2</p>
</div></td>
<td><p>Center of keypad</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_C1</p>
</div></td>
<td><p>Lower left of keypad</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_C3</p>
</div></td>
<td><p>Lower right of keypad</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_BTAB</p>
</div></td>
<td><p>Back tab</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_BEG</p>
</div></td>
<td><p>Beg (beginning)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_CANCEL</p>
</div></td>
<td><p>Cancel</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_CLOSE</p>
</div></td>
<td><p>Close</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_COMMAND</p>
</div></td>
<td><p>Cmd (command)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_COPY</p>
</div></td>
<td><p>Copy</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_CREATE</p>
</div></td>
<td><p>Create</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_END</p>
</div></td>
<td><p>End</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_EXIT</p>
</div></td>
<td><p>Exit</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_FIND</p>
</div></td>
<td><p>Find</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_HELP</p>
</div></td>
<td><p>Help</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_MARK</p>
</div></td>
<td><p>Mark</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_MESSAGE</p>
</div></td>
<td><p>Message</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_MOVE</p>
</div></td>
<td><p>Move</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_NEXT</p>
</div></td>
<td><p>Next</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_OPEN</p>
</div></td>
<td><p>Open</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_OPTIONS</p>
</div></td>
<td><p>Options</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_PREVIOUS</p>
</div></td>
<td><p>Prev (previous)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_REDO</p>
</div></td>
<td><p>Redo</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_REFERENCE</p>
</div></td>
<td><p>Ref (reference)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_REFRESH</p>
</div></td>
<td><p>Refresh</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_REPLACE</p>
</div></td>
<td><p>Replace</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_RESTART</p>
</div></td>
<td><p>Restart</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_RESUME</p>
</div></td>
<td><p>Resume</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SAVE</p>
</div></td>
<td><p>Save</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SBEG</p>
</div></td>
<td><p>Shifted Beg (beginning)</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SCANCEL</p>
</div></td>
<td><p>Shifted Cancel</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SCOMMAND</p>
</div></td>
<td><p>Shifted Command</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SCOPY</p>
</div></td>
<td><p>Shifted Copy</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SCREATE</p>
</div></td>
<td><p>Shifted Create</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SDC</p>
</div></td>
<td><p>Shifted Delete char</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SDL</p>
</div></td>
<td><p>Shifted Delete line</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SELECT</p>
</div></td>
<td><p>Select</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SEND</p>
</div></td>
<td><p>Shifted End</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SEOL</p>
</div></td>
<td><p>Shifted Clear line</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SEXIT</p>
</div></td>
<td><p>Shifted Exit</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SFIND</p>
</div></td>
<td><p>Shifted Find</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SHELP</p>
</div></td>
<td><p>Shifted Help</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SHOME</p>
</div></td>
<td><p>Shifted Home</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SIC</p>
</div></td>
<td><p>Shifted Input</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SLEFT</p>
</div></td>
<td><p>Shifted Left arrow</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SMESSAGE</p>
</div></td>
<td><p>Shifted Message</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SMOVE</p>
</div></td>
<td><p>Shifted Move</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SNEXT</p>
</div></td>
<td><p>Shifted Next</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SOPTIONS</p>
</div></td>
<td><p>Shifted Options</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SPREVIOUS</p>
</div></td>
<td><p>Shifted Prev</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SPRINT</p>
</div></td>
<td><p>Shifted Print</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SREDO</p>
</div></td>
<td><p>Shifted Redo</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SREPLACE</p>
</div></td>
<td><p>Shifted Replace</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SRIGHT</p>
</div></td>
<td><p>Shifted Right arrow</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SRSUME</p>
</div></td>
<td><p>Shifted Resume</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SSAVE</p>
</div></td>
<td><p>Shifted Save</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SSUSPEND</p>
</div></td>
<td><p>Shifted Suspend</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SUNDO</p>
</div></td>
<td><p>Shifted Undo</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_SUSPEND</p>
</div></td>
<td><p>Suspend</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_UNDO</p>
</div></td>
<td><p>Undo</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_MOUSE</p>
</div></td>
<td><p>Mouse event has occurred</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_RESIZE</p>
</div></td>
<td><p>Terminal resize event</p></td>
</tr>
<tr>
<td><div class="data">
<p>KEY_MAX</p>
</div></td>
<td><p>Maximum key value</p></td>
</tr>
</tbody>
</table>

On VT100s and their software emulations, such as X terminal emulators, there are normally at least four function keys (`KEY_F1 <KEY_Fn>`, `KEY_F2 <KEY_Fn>`, `KEY_F3 <KEY_Fn>`, `KEY_F4 <KEY_Fn>`) available, and the arrow keys mapped to `KEY_UP`, `KEY_DOWN`, `KEY_LEFT` and `KEY_RIGHT` in the obvious way. If your machine has a PC keyboard, it is safe to expect arrow keys and twelve function keys (older PC keyboards may have only ten function keys); also, the following keypad mappings are standard:

| Keycap      | Constant  |
|-------------|-----------|
| `Insert`    | KEY_IC    |
| `Delete`    | KEY_DC    |
| `Home`      | KEY_HOME  |
| `End`       | KEY_END   |
| `Page Up`   | KEY_PPAGE |
| `Page Down` | KEY_NPAGE |

### Alternate character set

<div id="curses-acs-codes">

The following table lists characters from the alternate character set. These are inherited from the VT100 terminal, and will generally be available on software emulations such as X terminals. When there is no graphic available, curses falls back on a crude printable ASCII approximation.

</div>

Every character has two names. The `ACS_*` code is an integer character, restricted to the 8-bit alternate character set of the terminal. The `WACS_*` code is the same character as a `complexchar` cell, which is not restricted to the alternate character set.

> [!NOTE]
> These are available only after `initscr` has been called. The `WACS_*` codes are only available if Python is built with wide character support.

<div class="versionadded">

3.16 The `WACS_*` codes.

</div>

<table style="width:98%;">
<colgroup>
<col style="width: 26%" />
<col style="width: 27%" />
<col style="width: 45%" />
</colgroup>
<thead>
<tr>
<th><p>ACS code</p></th>
<th><p>WACS code</p></th>
<th><p>Meaning</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>ACS_BBSS</p>
</div></td>
<td><div class="data">
<p>WACS_BBSS</p>
</div></td>
<td><p>alternate name for upper-right corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_BLOCK</p>
</div></td>
<td><div class="data">
<p>WACS_BLOCK</p>
</div></td>
<td><p>solid square block</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_BOARD</p>
</div></td>
<td><div class="data">
<p>WACS_BOARD</p>
</div></td>
<td><p>board of squares</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_BSBS</p>
</div></td>
<td><div class="data">
<p>WACS_BSBS</p>
</div></td>
<td><p>alternate name for horizontal line</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_BSSB</p>
</div></td>
<td><div class="data">
<p>WACS_BSSB</p>
</div></td>
<td><p>alternate name for upper-left corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_BSSS</p>
</div></td>
<td><div class="data">
<p>WACS_BSSS</p>
</div></td>
<td><p>alternate name for top tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_BTEE</p>
</div></td>
<td><div class="data">
<p>WACS_BTEE</p>
</div></td>
<td><p>bottom tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_BULLET</p>
</div></td>
<td><div class="data">
<p>WACS_BULLET</p>
</div></td>
<td><p>bullet</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_CKBOARD</p>
</div></td>
<td><div class="data">
<p>WACS_CKBOARD</p>
</div></td>
<td><p>checker board (stipple)</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_DARROW</p>
</div></td>
<td><div class="data">
<p>WACS_DARROW</p>
</div></td>
<td><p>arrow pointing down</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_DEGREE</p>
</div></td>
<td><div class="data">
<p>WACS_DEGREE</p>
</div></td>
<td><p>degree symbol</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_DIAMOND</p>
</div></td>
<td><div class="data">
<p>WACS_DIAMOND</p>
</div></td>
<td><p>diamond</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_GEQUAL</p>
</div></td>
<td><div class="data">
<p>WACS_GEQUAL</p>
</div></td>
<td><p>greater-than-or-equal-to</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_HLINE</p>
</div></td>
<td><div class="data">
<p>WACS_HLINE</p>
</div></td>
<td><p>horizontal line</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_LANTERN</p>
</div></td>
<td><div class="data">
<p>WACS_LANTERN</p>
</div></td>
<td><p>lantern symbol</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_LARROW</p>
</div></td>
<td><div class="data">
<p>WACS_LARROW</p>
</div></td>
<td><p>left arrow</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_LEQUAL</p>
</div></td>
<td><div class="data">
<p>WACS_LEQUAL</p>
</div></td>
<td><p>less-than-or-equal-to</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_LLCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_LLCORNER</p>
</div></td>
<td><p>lower-left corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_LRCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_LRCORNER</p>
</div></td>
<td><p>lower-right corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_LTEE</p>
</div></td>
<td><div class="data">
<p>WACS_LTEE</p>
</div></td>
<td><p>left tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_NEQUAL</p>
</div></td>
<td><div class="data">
<p>WACS_NEQUAL</p>
</div></td>
<td><p>not-equal sign</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_PI</p>
</div></td>
<td><div class="data">
<p>WACS_PI</p>
</div></td>
<td><p>letter pi</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_PLMINUS</p>
</div></td>
<td><div class="data">
<p>WACS_PLMINUS</p>
</div></td>
<td><p>plus-or-minus sign</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_PLUS</p>
</div></td>
<td><div class="data">
<p>WACS_PLUS</p>
</div></td>
<td><p>big plus sign</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_RARROW</p>
</div></td>
<td><div class="data">
<p>WACS_RARROW</p>
</div></td>
<td><p>right arrow</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_RTEE</p>
</div></td>
<td><div class="data">
<p>WACS_RTEE</p>
</div></td>
<td><p>right tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_S1</p>
</div></td>
<td><div class="data">
<p>WACS_S1</p>
</div></td>
<td><p>scan line 1</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_S3</p>
</div></td>
<td><div class="data">
<p>WACS_S3</p>
</div></td>
<td><p>scan line 3</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_S7</p>
</div></td>
<td><div class="data">
<p>WACS_S7</p>
</div></td>
<td><p>scan line 7</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_S9</p>
</div></td>
<td><div class="data">
<p>WACS_S9</p>
</div></td>
<td><p>scan line 9</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_SBBS</p>
</div></td>
<td><div class="data">
<p>WACS_SBBS</p>
</div></td>
<td><p>alternate name for lower-right corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_SBSB</p>
</div></td>
<td><div class="data">
<p>WACS_SBSB</p>
</div></td>
<td><p>alternate name for vertical line</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_SBSS</p>
</div></td>
<td><div class="data">
<p>WACS_SBSS</p>
</div></td>
<td><p>alternate name for right tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_SSBB</p>
</div></td>
<td><div class="data">
<p>WACS_SSBB</p>
</div></td>
<td><p>alternate name for lower-left corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_SSBS</p>
</div></td>
<td><div class="data">
<p>WACS_SSBS</p>
</div></td>
<td><p>alternate name for bottom tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_SSSB</p>
</div></td>
<td><div class="data">
<p>WACS_SSSB</p>
</div></td>
<td><p>alternate name for left tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_SSSS</p>
</div></td>
<td><div class="data">
<p>WACS_SSSS</p>
</div></td>
<td><p>alternate name for crossover or big plus</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_STERLING</p>
</div></td>
<td><div class="data">
<p>WACS_STERLING</p>
</div></td>
<td><p>pound sterling</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_TTEE</p>
</div></td>
<td><div class="data">
<p>WACS_TTEE</p>
</div></td>
<td><p>top tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_UARROW</p>
</div></td>
<td><div class="data">
<p>WACS_UARROW</p>
</div></td>
<td><p>up arrow</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_ULCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_ULCORNER</p>
</div></td>
<td><p>upper-left corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_URCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_URCORNER</p>
</div></td>
<td><p>upper-right corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>ACS_VLINE</p>
</div></td>
<td><div class="data">
<p>WACS_VLINE</p>
</div></td>
<td><p>vertical line</p></td>
</tr>
</tbody>
</table>

The following table lists the double-line and thick-line characters. They have no `ACS_*` counterpart, and are not provided by every implementation. As in the table above, the alternate name spells out the four sides of the character, clockwise from the top: `B` for a blank side, `S` for a single line, `D` for a double line and `T` for a thick line.

<table style="width:98%;">
<colgroup>
<col style="width: 33%" />
<col style="width: 26%" />
<col style="width: 39%" />
</colgroup>
<thead>
<tr>
<th><p>WACS code</p></th>
<th><p>Alternate name</p></th>
<th><p>Meaning</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>WACS_D_BTEE</p>
</div></td>
<td><div class="data">
<p>WACS_DDBD</p>
</div></td>
<td><p>double-line bottom tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_HLINE</p>
</div></td>
<td><div class="data">
<p>WACS_BDBD</p>
</div></td>
<td><p>double-line horizontal line</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_LLCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_DDBB</p>
</div></td>
<td><p>double-line lower-left corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_LRCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_DBBD</p>
</div></td>
<td><p>double-line lower-right corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_LTEE</p>
</div></td>
<td><div class="data">
<p>WACS_DDDB</p>
</div></td>
<td><p>double-line left tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_PLUS</p>
</div></td>
<td><div class="data">
<p>WACS_DDDD</p>
</div></td>
<td><p>double-line big plus sign</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_RTEE</p>
</div></td>
<td><div class="data">
<p>WACS_DBDD</p>
</div></td>
<td><p>double-line right tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_TTEE</p>
</div></td>
<td><div class="data">
<p>WACS_BDDD</p>
</div></td>
<td><p>double-line top tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_ULCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_BDDB</p>
</div></td>
<td><p>double-line upper-left corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_URCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_BBDD</p>
</div></td>
<td><p>double-line upper-right corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_D_VLINE</p>
</div></td>
<td><div class="data">
<p>WACS_DBDB</p>
</div></td>
<td><p>double-line vertical line</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_BTEE</p>
</div></td>
<td><div class="data">
<p>WACS_TTBT</p>
</div></td>
<td><p>thick-line bottom tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_HLINE</p>
</div></td>
<td><div class="data">
<p>WACS_BTBT</p>
</div></td>
<td><p>thick-line horizontal line</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_LLCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_TTBB</p>
</div></td>
<td><p>thick-line lower-left corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_LRCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_TBBT</p>
</div></td>
<td><p>thick-line lower-right corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_LTEE</p>
</div></td>
<td><div class="data">
<p>WACS_TTTB</p>
</div></td>
<td><p>thick-line left tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_PLUS</p>
</div></td>
<td><div class="data">
<p>WACS_TTTT</p>
</div></td>
<td><p>thick-line big plus sign</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_RTEE</p>
</div></td>
<td><div class="data">
<p>WACS_TBTT</p>
</div></td>
<td><p>thick-line right tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_TTEE</p>
</div></td>
<td><div class="data">
<p>WACS_BTTT</p>
</div></td>
<td><p>thick-line top tee</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_ULCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_BTTB</p>
</div></td>
<td><p>thick-line upper-left corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_URCORNER</p>
</div></td>
<td><div class="data">
<p>WACS_BBTT</p>
</div></td>
<td><p>thick-line upper-right corner</p></td>
</tr>
<tr>
<td><div class="data">
<p>WACS_T_VLINE</p>
</div></td>
<td><div class="data">
<p>WACS_TBTB</p>
</div></td>
<td><p>thick-line vertical line</p></td>
</tr>
</tbody>
</table>

### Mouse buttons

The following table lists mouse button constants used by `getmouse`:

<table>
<colgroup>
<col style="width: 43%" />
<col style="width: 56%" />
</colgroup>
<thead>
<tr>
<th><p>Mouse button constant</p></th>
<th><p>Meaning</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>BUTTONn_PRESSED</p>
</div></td>
<td><p>Mouse button <em>n</em> pressed</p></td>
</tr>
<tr>
<td><div class="data">
<p>BUTTONn_RELEASED</p>
</div></td>
<td><p>Mouse button <em>n</em> released</p></td>
</tr>
<tr>
<td><div class="data">
<p>BUTTONn_CLICKED</p>
</div></td>
<td><p>Mouse button <em>n</em> clicked</p></td>
</tr>
<tr>
<td><div class="data">
<p>BUTTONn_DOUBLE_CLICKED</p>
</div></td>
<td><p>Mouse button <em>n</em> double clicked</p></td>
</tr>
<tr>
<td><div class="data">
<p>BUTTONn_TRIPLE_CLICKED</p>
</div></td>
<td><p>Mouse button <em>n</em> triple clicked</p></td>
</tr>
<tr>
<td><div class="data">
<p>BUTTON_SHIFT</p>
</div></td>
<td><p>Shift was down during button state change</p></td>
</tr>
<tr>
<td><div class="data">
<p>BUTTON_CTRL</p>
</div></td>
<td><p>Control was down during button state change</p></td>
</tr>
<tr>
<td><div class="data">
<p>BUTTON_ALT</p>
</div></td>
<td><p>Alt was down during button state change</p></td>
</tr>
</tbody>
</table>

<div class="versionchanged">

3.10 The `BUTTON5_*` constants are now exposed if they are provided by the underlying curses library.

</div>

### Colors

The following table lists the predefined colors:

<table style="width:76%;">
<colgroup>
<col style="width: 36%" />
<col style="width: 40%" />
</colgroup>
<thead>
<tr>
<th><p>Constant</p></th>
<th><p>Color</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>COLOR_BLACK</p>
</div></td>
<td><p>Black</p></td>
</tr>
<tr>
<td><div class="data">
<p>COLOR_BLUE</p>
</div></td>
<td><p>Blue</p></td>
</tr>
<tr>
<td><div class="data">
<p>COLOR_CYAN</p>
</div></td>
<td><p>Cyan (light greenish blue)</p></td>
</tr>
<tr>
<td><div class="data">
<p>COLOR_GREEN</p>
</div></td>
<td><p>Green</p></td>
</tr>
<tr>
<td><div class="data">
<p>COLOR_MAGENTA</p>
</div></td>
<td><p>Magenta (purplish red)</p></td>
</tr>
<tr>
<td><div class="data">
<p>COLOR_RED</p>
</div></td>
<td><p>Red</p></td>
</tr>
<tr>
<td><div class="data">
<p>COLOR_WHITE</p>
</div></td>
<td><p>White</p></td>
</tr>
<tr>
<td><div class="data">
<p>COLOR_YELLOW</p>
</div></td>
<td><p>Yellow</p></td>
</tr>
</tbody>
</table>

# `!curses.textpad` --- Text input widget for curses programs

<div class="module" data-synopsis="Emacs-like input editing in a curses window.">

curses.textpad

</div>

The `!curses.textpad` module provides a `Textbox` class that handles elementary text editing in a curses window, supporting a set of keybindings resembling those of Emacs (thus, also of Netscape Navigator, BBedit 6.x, FrameMaker, and many other programs). The module also provides a rectangle-drawing function useful for framing text boxes or for other purposes.

The module `!curses.textpad` defines the following function:

<div class="function">

rectangle(win, uly, ulx, lry, lrx)

Draw a rectangle. The first argument must be a window object; the remaining arguments are coordinates relative to that window. The second and third arguments are the y and x coordinates of the upper-left corner of the rectangle to be drawn; the fourth and fifth arguments are the y and x coordinates of the lower-right corner. The rectangle will be drawn using VT100/IBM PC forms characters on terminals that make this possible (including xterm and most other software terminal emulators). Otherwise it will be drawn with ASCII dashes, vertical bars, and plus signs.

</div>

## Textbox objects

You can instantiate a `Textbox` object as follows:

<div class="Textbox(win, insert_mode=False)">

Return a textbox widget object. The *win* argument should be a curses `window <curses-window-objects>` object in which the textbox is to be contained. If *insert_mode* is true, the textbox inserts typed characters, shifting existing text to the right, rather than overwriting it. The edit cursor of the textbox is initially located at the upper-left corner of the containing window, with coordinates `(0, 0)`. The instance's `stripspaces` flag is initially on.

<div class="versionchanged">

3.16 Entering and reading back the full Unicode range, including combining characters, is now supported when curses is built with wide-character support.

</div>

`Textbox` objects have the following methods:

<div class="method">

edit(validate=None)

This is the entry point you will normally use. It accepts editing keystrokes until one of the termination keystrokes is entered. If *validate* is supplied, it must be a function. It will be called for each keystroke entered with the keystroke as a parameter; command dispatch is done on the result. If it returns a false value, the keystroke is ignored. This method returns the window contents as a string; whether blanks in the window are included is affected by the `stripspaces` attribute.

<div class="versionchanged">

3.16 *validate* is now called with a non-ASCII character as a string; other keystrokes are still passed as an integer.

</div>

</div>

<div class="method">

do_command(ch)

Process a single command keystroke. Returns `1` to continue editing, or `0` if a termination keystroke was processed. Here are the supported special keystrokes:

| Keystroke | Action |
|----|----|
| `Control-A` | Go to left edge of window. |
| `Control-B` | Cursor left, wrapping to previous line if appropriate. |
| `Control-D` | Delete character under cursor. |
| `Control-E` | Go to right edge (stripspaces off) or end of line (stripspaces on). |
| `Control-F` | Cursor right, wrapping to next line when appropriate. |
| `Control-G` | Terminate, returning the window contents. |
| `Control-H` | Delete character backward. |
| `Control-J` | Terminate if the window is 1 line, otherwise move to the start of the next line. |
| `Control-K` | If line is blank, delete it, otherwise clear to end of line. |
| `Control-L` | Refresh screen. |
| `Control-N` | Cursor down; move down one line. |
| `Control-O` | Insert a blank line at cursor location. |
| `Control-P` | Cursor up; move up one line. |

Move operations do nothing if the cursor is at an edge where the movement is not possible. The following synonyms are supported where possible:

| Constant                | Keystroke   |
|-------------------------|-------------|
| `~curses.KEY_LEFT`      | `Control-B` |
| `~curses.KEY_RIGHT`     | `Control-F` |
| `~curses.KEY_UP`        | `Control-P` |
| `~curses.KEY_DOWN`      | `Control-N` |
| `~curses.KEY_BACKSPACE` | `Control-h` |

All other keystrokes are treated as a command to insert the given character and move right (with line wrapping).

</div>

<div class="method">

gather()

Return the window contents as a string; whether blanks in the window are included is affected by the `stripspaces` member.

</div>

<div class="attribute">

stripspaces

This attribute is a flag which controls the interpretation of blanks in the window. When it is on, trailing blanks on each line are ignored; any cursor motion that would land the cursor on a trailing blank goes to the end of that line instead, and trailing blanks are stripped when the window contents are gathered.

</div>

</div>

# `!curses.panel` --- A panel stack extension for curses

<div class="module" data-synopsis="A panel stack extension that adds depth to  curses windows.">

curses.panel

</div>

------------------------------------------------------------------------

Panels are windows with the added feature of depth, so they can be stacked on top of each other, and only the visible portions of each window will be displayed. Panels can be added, moved up or down in the stack, and removed.

## Functions

The module `!curses.panel` defines the following exception:

<div class="exception">

error

Exception raised when a curses panel library function returns an error.

</div>

The module `!curses.panel` defines the following functions:

<div class="function">

bottom_panel()

Returns the bottom panel in the panel stack.

</div>

<div class="function">

new_panel(win)

Returns a panel object, associating it with the given window *win* and placing the new panel on top of the panel stack. Be aware that you need to keep the returned panel object referenced explicitly. If you don't, the panel object is garbage collected and removed from the panel stack.

</div>

<div class="function">

top_panel()

Returns the top panel in the panel stack.

</div>

<div class="function">

update_panels()

Updates the virtual screen after changes in the panel stack. This does not call `curses.doupdate`, so you'll have to do this yourself.

</div>

## Panel objects

<!-- Keep the old URL fragments working (see gh-89554) -->
<span id='curses.panel.Panel.above'></span>
<span id='curses.panel.Panel.below'></span>
<span id='curses.panel.Panel.bottom'></span>
<span id='curses.panel.Panel.hidden'></span>
<span id='curses.panel.Panel.hide'></span>
<span id='curses.panel.Panel.move'></span>
<span id='curses.panel.Panel.replace'></span>
<span id='curses.panel.Panel.set_userptr'></span>
<span id='curses.panel.Panel.show'></span>
<span id='curses.panel.Panel.top'></span>
<span id='curses.panel.Panel.userptr'></span>
<span id='curses.panel.Panel.window'></span>

<div class="panel">

Panel objects, as returned by `new_panel` above, are windows with a stacking order. There's always a window associated with a panel which determines the content, while the panel methods are responsible for the window's depth in the panel stack.

Panel objects have the following methods:

</div>

<div class="method">

panel.above()

Returns the panel above the current panel.

</div>

<div class="method">

panel.below()

Returns the panel below the current panel.

</div>

<div class="method">

panel.bottom()

Push the panel to the bottom of the stack.

</div>

<div class="method">

panel.hidden()

Returns `True` if the panel is hidden (not visible), `False` otherwise.

</div>

<div class="method">

panel.hide()

Hide the panel. This does not delete the object, it just makes the window on screen invisible.

</div>

<div class="method">

panel.move(y, x)

Move the panel to the screen coordinates `(y, x)`.

</div>

<div class="method">

panel.replace(win)

Change the window associated with the panel to the window *win*. Raise `curses.panel.error` if *win* has been detached from its screen by `screen.close() <curses.screen.close>`.

</div>

<div class="method">

panel.set_userptr(obj)

Set the panel's user pointer to *obj*. This is used to associate an arbitrary piece of data with the panel, and can be any Python object.

</div>

<div class="method">

panel.show()

Display the panel (which might have been hidden), placing it on top of the panel stack.

</div>

<div class="method">

panel.top()

Push panel to the top of the stack.

</div>

<div class="method">

panel.userptr()

Returns the user pointer for the panel. This might be any Python object.

</div>

<div class="method">

panel.window()

Returns the window object associated with the panel.

</div>

# Custom Python Interpreters

The modules described in this chapter allow writing interfaces similar to Python's interactive interpreter. If you want a Python interpreter that supports some special feature in addition to the Python language, you should look at the `code` module. (The `codeop` module is lower-level, used to support compiling a possibly incomplete chunk of Python code.)

The full list of modules described in this chapter is:

<div class="toctree">

code.rst codeop.rst

</div>

# `!dataclasses` --- Data Classes

<div class="module" data-synopsis="Generate special methods on user-defined classes.">

dataclasses

</div>

**Source code:** `Lib/dataclasses.py`

------------------------------------------------------------------------

This module provides a decorator and functions for automatically adding generated `special methods <special method>` such as `~object.__init__` and `~object.__repr__` to user-defined classes. It was originally described in `557`.

The member variables to use in these generated methods are defined using `526` type annotations. For example, this code:

    from dataclasses import dataclass

    @dataclass
    class InventoryItem:
        """Class for keeping track of an item in inventory."""
        name: str
        unit_price: float
        quantity_on_hand: int = 0

        def total_cost(self) -> float:
            return self.unit_price * self.quantity_on_hand

will add, among other things, a `!__init__` that looks like:

    def __init__(self, name: str, unit_price: float, quantity_on_hand: int = 0):
        self.name = name
        self.unit_price = unit_price
        self.quantity_on_hand = quantity_on_hand

Note that this method is automatically added to the class: it is not directly specified in the `!InventoryItem` definition shown above.

<div class="versionadded">

3.7

</div>

## Module contents

<div class="decorator">

dataclass(\*, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False, weakref_slot=False)

This function is a `decorator` that is used to add generated `special methods <special method>` to classes, as described below.

The `@dataclass` decorator examines the class to find `field`s. A `field` is defined as a class variable that has a `type annotation <variable annotation>`. With two exceptions described below, nothing in `@dataclass` examines the type specified in the variable annotation.

The order of the fields in all of the generated methods is the order in which they appear in the class definition.

The `@dataclass` decorator will add various "dunder" methods to the class, described below. If any of the added methods already exist in the class, the behavior depends on the parameter, as documented below. The decorator returns the same class that it is called on; no new class is created.

If `@dataclass` is used just as a simple decorator with no parameters, it acts as if it has the default values documented in this signature. That is, these three uses of `@dataclass` are equivalent:

    @dataclass
    class C:
        ...

    @dataclass()
    class C:
        ...

    @dataclass(init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False,
               match_args=True, kw_only=False, slots=False, weakref_slot=False)
    class C:
        ...

The parameters to `@dataclass` are:

- *init*: If true (the default), a `~object.__init__` method will be generated.

  If the class already defines `!__init__`, this parameter is ignored.

- *repr*: If true (the default), a `~object.__repr__` method will be generated. The generated repr string will have the class name and the name and repr of each field, in the order they are defined in the class. Fields that are marked as being excluded from the repr are not included. For example: `InventoryItem(name='widget', unit_price=3.0, quantity_on_hand=10)`.

  If the class already defines `!__repr__`, this parameter is ignored.

- *eq*: If true (the default), an `~object.__eq__` method will be generated.

  This method compares the class by comparing each field in order. Both instances in the comparison must be of the identical type.

  If the class already defines `!__eq__`, this parameter is ignored.

  <div class="versionchanged">

  3.13 The generated `__eq__` method now compares each field individually (for example, `self.a == other.a and self.b == other.b`), rather than comparing tuples of fields as in previous versions.

  This change makes the comparison faster but it may alter results in cases where attributes compare equal by identity but not by value (such as `float('nan')`).

  In Python 3.12 and earlier, the comparison was performed by creating tuples of the fields and comparing them (for example, `(self.a, self.b) == (other.a, other.b)`).

  </div>

- *order*: If true (the default is `False`), `~object.__lt__`, `~object.__le__`, `~object.__gt__`, and `~object.__ge__` methods will be generated. These compare the class as if it were a tuple of its fields, in order. Both instances in the comparison must be of the identical type. If *order* is true and *eq* is false, a `ValueError` is raised.

  If the class already defines any of `!__lt__`, `!__le__`, `!__gt__`, or `!__ge__`, then `TypeError` is raised.

- *unsafe_hash*: If true, force `dataclasses` to create a `~object.__hash__` method, even though it may not be safe to do so. Otherwise, generate a `~object.__hash__` method according to how *eq* and *frozen* are set. The default value is `False`.

  `!__hash__` is used by built-in `hash`, and when objects are added to hashed collections such as dictionaries and sets. Having a `!__hash__` implies that instances of the class are immutable. Mutability is a complicated property that depends on the programmer's intent, the existence and behavior of `!__eq__`, and the values of the *eq* and *frozen* flags in the `@dataclass` decorator.

  By default, `@dataclass` will not implicitly add a `~object.__hash__` method unless it is safe to do so. Neither will it add or change an existing explicitly defined `!__hash__` method. Setting the class attribute `__hash__ = None` has a specific meaning to Python, as described in the `!__hash__` documentation.

  If `!__hash__` is not explicitly defined, or if it is set to `None`, then `@dataclass` *may* add an implicit `!__hash__` method. Although not recommended, you can force `@dataclass` to create a `!__hash__` method with `unsafe_hash=True`. This might be the case if your class is logically immutable but can still be mutated. This is a specialized use case and should be considered carefully.

  Here are the rules governing implicit creation of a `!__hash__` method. Note that you cannot both have an explicit `!__hash__` method in your dataclass and set `unsafe_hash=True`; this will result in a `TypeError`.

  If *eq* and *frozen* are both true, by default `@dataclass` will generate a `!__hash__` method for you. If *eq* is true and *frozen* is false, `!__hash__` will be set to `None`, marking it unhashable (which it is, since it is mutable). If *eq* is false, `!__hash__` will be left untouched meaning the `!__hash__` method of the superclass will be used (if the superclass is `object`, this means it will fall back to id-based hashing).

- *frozen*: If true (the default is `False`), assigning to fields will generate an exception. This emulates read-only frozen instances. See the `discussion <dataclasses-frozen>` below.

  If `~object.__setattr__` or `~object.__delattr__` is defined in the class and *frozen* is true, then `TypeError` is raised.

- *match_args*: If true (the default is `True`), the `~object.__match_args__` tuple will be created from the list of non keyword-only parameters to the generated `~object.__init__` method (even if `!__init__` is not generated, see above). If false, or if `!__match_args__` is already defined in the class, then `!__match_args__` will not be generated.

> <div class="versionadded">
>
> 3.10
>
> </div>

- *kw_only*: If true (the default value is `False`), then all fields will be marked as keyword-only. If a field is marked as keyword-only, then the only effect is that the `~object.__init__` parameter generated from a keyword-only field must be specified with a keyword when `!__init__` is called. See the `parameter` glossary entry for details. Also see the `KW_ONLY` section.

  Keyword-only fields are not included in `!__match_args__`.

> <div class="versionadded">
>
> 3.10
>
> </div>

- *slots*: If true (the default is `False`), `~object.__slots__` attribute will be generated and new class will be returned instead of the original one. If `!__slots__` is already defined in the class, then `TypeError` is raised.

> > [!WARNING]
> > Passing parameters to a base class `~object.__init_subclass__` when using `slots=True` will result in a `TypeError`. Either use `__init_subclass__` with no parameters or use default values as a workaround. See `91126` for full details.
>
> <div class="versionadded">
>
> 3.10
>
> </div>
>
> <div class="versionchanged">
>
> 3.11 If a field name is already included in the `!__slots__` of a base class, it will not be included in the generated `!__slots__` to prevent `overriding them <datamodel-note-slots>`. Therefore, do not use `!__slots__` to retrieve the field names of a dataclass. Use `fields` instead. To be able to determine inherited slots, base class `!__slots__` may be any iterable, but *not* an iterator.
>
> </div>

- *weakref_slot*: If true (the default is `False`), add a slot named "\_\_weakref\_\_", which is required to make an instance `weakref-able <weakref.ref>`. It is an error to specify `weakref_slot=True` without also specifying `slots=True`.

> <div class="versionadded">
>
> 3.11
>
> </div>

`field`s may optionally specify a default value, using normal Python syntax:

    @dataclass
    class C:
        a: int       # 'a' has no default value
        b: int = 0   # assign a default value for 'b'

In this example, both `!a` and `!b` will be included in the added `~object.__init__` method, which will be defined as:

    def __init__(self, a: int, b: int = 0):

`TypeError` will be raised if a field without a default value follows a field with a default value. This is true whether this occurs in a single class, or as a result of class inheritance.

</div>

<div class="function">

field(\*, default=MISSING, default_factory=MISSING, init=True, repr=True, hash=None, compare=True, metadata=None, kw_only=MISSING, doc=None)

For common and simple use cases, no other functionality is required. There are, however, some dataclass features that require additional per-field information. To satisfy this need for additional information, you can replace the default field value with a call to the provided `!field` function. For example:

    @dataclass
    class C:
        mylist: list[int] = field(default_factory=list)

    c = C()
    c.mylist += [1, 2, 3]

As shown above, the `MISSING` value is a `sentinel` object used to detect if some parameters are provided by the user. This sentinel is used because `None` is a valid value for some parameters with a distinct meaning. No code should directly use the `MISSING` value.

The parameters to `!field` are:

- *default*: If provided, this will be the default value for this field. This is needed because the `!field` call itself replaces the normal position of the default value.

- *default_factory*: If provided, it must be a zero-argument callable that will be called when a default value is needed for this field. Among other purposes, this can be used to specify fields with mutable default values, as discussed below. It is an error to specify both *default* and *default_factory*.

- *init*: If true (the default), this field is included as a parameter to the generated `~object.__init__` method.

- *repr*: If true (the default), this field is included in the string returned by the generated `~object.__repr__` method.

- *hash*: This can be a bool or `None`. If true, this field is included in the generated `~object.__hash__` method. If false, this field is excluded from the generated `~object.__hash__`. If `None` (the default), use the value of *compare*: this would normally be the expected behavior, since a field should be included in the hash if it's used for comparisons. Setting this value to anything other than `None` is discouraged.

  One possible reason to set `hash=False` but `compare=True` would be if a field is expensive to compute a hash value for, that field is needed for equality testing, and there are other fields that contribute to the type's hash value. Even if a field is excluded from the hash, it will still be used for comparisons.

- *compare*: If true (the default), this field is included in the generated equality and comparison methods (`~object.__eq__`, `~object.__gt__`, et al.).

- *metadata*: This can be a mapping or `None`. `None` is treated as an empty dict. This value is wrapped in `~types.MappingProxyType` to make it read-only, and exposed on the `Field` object. It is not used at all by Data Classes, and is provided as a third-party extension mechanism. Multiple third-parties can each have their own key, to use as a namespace in the metadata.

- *kw_only*: If true, this field will be marked as keyword-only. This is used when the generated `~object.__init__` method's parameters are computed.

  Keyword-only fields are also not included in `!__match_args__`.

> <div class="versionadded">
>
> 3.10
>
> </div>

- *doc*: optional docstring for this field.

> <div class="versionadded">
>
> 3.14
>
> </div>

If the default value of a field is specified by a call to `!field`, then the class attribute for this field will be replaced by the specified *default* value. If *default* is not provided, then the class attribute will be deleted. The intent is that after the `dataclass` decorator runs, the class attributes will all contain the default values for the fields, just as if the default value itself were specified. For example, after:

    @dataclass
    class C:
        x: int
        y: int = field(repr=False)
        z: int = field(repr=False, default=10)
        t: int = 20

The class attribute `!C.z` will be `10`, the class attribute `!C.t` will be `20`, and the class attributes `!C.x` and `!C.y` will not be set.

<div class="versionchanged">

3.15 If *metadata* is `None`, use an empty `frozendict`, instead of a `~types.MappingProxyType` of an empty `dict`.

</div>

</div>

<div class="Field">

`!Field` objects describe each defined field. These objects are created internally, and are returned by the `fields` module-level method (see below). Users should never instantiate a `!Field` object directly. Its documented attributes are:

- `!name`: The name of the field.
- `!type`: The type of the field.
- `!default`, `!default_factory`, `!init`, `!repr`, `!hash`, `!compare`, `!metadata`, and `!kw_only` have the identical meaning and values as they do in the `field` function.

Other attributes may exist, but they are private and must not be inspected or relied on.

</div>

<div class="InitVar">

`InitVar[T]` type annotations describe variables that are `init-only
<dataclasses-init-only-variables>`. Fields annotated with `!InitVar` are considered pseudo-fields, and thus are neither returned by the `fields` function nor used in any way except adding them as parameters to `~object.__init__` and an optional `__post_init__`.

</div>

<div class="function">

fields(class_or_instance)

Returns a tuple of `Field` objects that define the fields for this dataclass. Accepts either a dataclass, or an instance of a dataclass. Raises `TypeError` if not passed a dataclass or instance of one. Does not return pseudo-fields which are `ClassVar` or `InitVar`.

</div>

<div class="function">

asdict(obj, \*, dict_factory=dict)

Converts the dataclass *obj* to a dict (by using the factory function *dict_factory*). Each dataclass is converted to a dict of its fields, as `name: value` pairs. dataclasses, dicts, frozendicts, lists, and tuples are recursed into. Other objects are copied with `copy.deepcopy`.

Example of using `!asdict` on nested dataclasses:

    @dataclass
    class Point:
         x: int
         y: int

    @dataclass
    class C:
         mylist: list[Point]

    p = Point(10, 20)
    assert asdict(p) == {'x': 10, 'y': 20}

    c = C([Point(0, 0), Point(10, 4)])
    assert asdict(c) == {'mylist': [{'x': 0, 'y': 0}, {'x': 10, 'y': 4}]}

To create a shallow copy, the following workaround may be used:

    {field.name: getattr(obj, field.name) for field in fields(obj)}

`!asdict` raises `TypeError` if *obj* is not a dataclass instance.

</div>

<div class="function">

astuple(obj, \*, tuple_factory=tuple)

Converts the dataclass *obj* to a tuple (by using the factory function *tuple_factory*). Each dataclass is converted to a tuple of its field values. dataclasses, dicts, frozendicts, lists, and tuples are recursed into. Other objects are copied with `copy.deepcopy`.

Continuing from the previous example:

    assert astuple(p) == (10, 20)
    assert astuple(c) == ([(0, 0), (10, 4)],)

To create a shallow copy, the following workaround may be used:

    tuple(getattr(obj, field.name) for field in dataclasses.fields(obj))

`!astuple` raises `TypeError` if *obj* is not a dataclass instance.

</div>

<div class="function">

make_dataclass(cls_name, fields, \*, bases=(), namespace=None, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False, weakref_slot=False, module=None, qualname=None, decorator=dataclass)

Creates a new dataclass with name *cls_name*, fields as defined in *fields*, base classes as given in *bases*, and initialized with a namespace as given in *namespace*. *fields* is an iterable whose elements are each either `name`, `(name, type)`, or `(name, type, Field)`. If just `name` is supplied, `typing.Any` is used for `type`. The values of *init*, *repr*, *eq*, *order*, *unsafe_hash*, *frozen*, *match_args*, *kw_only*, *slots*, and *weakref_slot* have the same meaning as they do in `dataclass`.

If *module* is defined, the `!__module__` attribute of the dataclass is set to that value. By default, it is set to the module name of the caller.

If *qualname* is defined, the `~type.__qualname__` attribute of the dataclass is set to that value. By default, it is set to the value passed to *cls_name*.

The *decorator* parameter is a callable that will be used to create the dataclass. It should take the class object as a first argument and the same keyword arguments as `dataclass`. By default, the `dataclass` function is used.

This function is not strictly required, because any Python mechanism for creating a new class with `~object.__annotations__` can then apply the `dataclass` function to convert that class to a dataclass. This function is provided as a convenience. For example:

    C = make_dataclass('C',
                       [('x', int),
                         'y',
                        ('z', int, field(default=5))],
                       namespace={'add_one': lambda self: self.x + 1})

Is equivalent to:

    @dataclass
    class C:
        x: int
        y: 'typing.Any'
        z: int = 5

        def add_one(self):
            return self.x + 1

<div class="versionadded">

3.14 Added the *decorator* parameter.

</div>

<div class="versionadded">

3.16 Added the *qualname* parameter.

</div>

</div>

<div class="function">

replace(obj, /, \*\*changes)

Creates a new object of the same type as *obj*, replacing fields with values from *changes*. If *obj* is not a Data Class, raises `TypeError`. If keys in *changes* are not field names of the given dataclass, raises `TypeError`.

The newly returned object is created by calling the `~object.__init__` method of the dataclass. This ensures that `__post_init__`, if present, is also called.

Init-only variables without default values, if any exist, must be specified on the call to `!replace` so that they can be passed to `!__init__` and `__post_init__`.

It is an error for *changes* to contain any fields that are defined as having `init=False`. A `ValueError` will be raised in this case.

Be forewarned about how `init=False` fields work during a call to `!replace`. They are not copied from the source object, but rather are initialized in `__post_init__`, if they're initialized at all. It is expected that `init=False` fields will be rarely and judiciously used. If they are used, it might be wise to have alternate class constructors, or perhaps a custom `!replace` (or similarly named) method which handles instance copying.

Dataclass instances are also supported by generic function `copy.replace`.

</div>

<div class="function">

is_dataclass(obj)

Return `True` if its parameter is a dataclass (including subclasses of a dataclass, but not including `generic aliases <types-genericalias>`) or an instance of one, otherwise return `False`.

If you need to know if a class is an instance of a dataclass (and not a dataclass itself), then add a further check for `not isinstance(obj, type)`:

    def is_dataclass_instance(obj):
        return is_dataclass(obj) and not isinstance(obj, type)

</div>

<div class="data">

MISSING

A `sentinel` object signifying a missing default or *default_factory*.

<div class="versionchanged">

3.15 `!MISSING` is now a `sentinel` object.

</div>

</div>

<div class="data">

KW_ONLY

A `sentinel` object used as a type annotation. Any fields after a pseudo-field with the type of `!KW_ONLY` are marked as keyword-only fields. Note that a pseudo-field of type `!KW_ONLY` is otherwise completely ignored. This includes the name of such a field. By convention, a name of `_` is used for a `!KW_ONLY` field. Keyword-only fields signify `~object.__init__` parameters that must be specified as keywords when the class is instantiated.

In this example, the fields `y` and `z` will be marked as keyword-only fields:

    @dataclass
    class Point:
        x: float
        _: KW_ONLY
        y: float
        z: float

    p = Point(0, y=1.5, z=2.0)

In a single dataclass, it is an error to specify more than one field whose type is `!KW_ONLY`.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.15 `!KW_ONLY` is now a `sentinel` object.

</div>

</div>

<div class="exception">

FrozenInstanceError

Raised when an implicitly defined `~object.__setattr__` or `~object.__delattr__` is called on a dataclass which was defined with `frozen=True`. It is a subclass of `AttributeError`.

</div>

## Post-init processing

<div class="function">

\_\_post_init\_\_()

When defined on the class, it will be called by the generated `~object.__init__`, normally as `!self.__post_init__`. However, if any `InitVar` fields are defined, they will also be passed to `!__post_init__` in the order they were defined in the class. If no `!__init__` method is generated, then `!__post_init__` will not automatically be called.

Among other uses, this allows for initializing field values that depend on one or more other fields. For example:

    @dataclass
    class C:
        a: float
        b: float
        c: float = field(init=False)

        def __post_init__(self):
            self.c = self.a + self.b

</div>

The `~object.__init__` method generated by `dataclass` does not call base class `!__init__` methods. If the base class has an `!__init__` method that has to be called, it is common to call this method in a `__post_init__` method:

    class Rectangle:
        def __init__(self, height, width):
            self.height = height
            self.width = width

    @dataclass
    class Square(Rectangle):
        side: float

        def __post_init__(self):
            super().__init__(self.side, self.side)

Note, however, that in general the dataclass-generated `!__init__` methods don't need to be called, since the derived dataclass will take care of initializing all fields of any base class that is a dataclass itself.

See the section below on init-only variables for ways to pass parameters to `!__post_init__`. Also see the warning about how `replace` handles `init=False` fields.

## Class variables

One of the few places where `dataclass` actually inspects the type of a field is to determine if a field is a class variable as defined in `526`. It does this by checking if the type of the field is `typing.ClassVar`. If a field is a `ClassVar`, it is excluded from consideration as a field and is ignored by the dataclass mechanisms. Such `ClassVar` pseudo-fields are not returned by the module-level `fields` function.

## Init-only variables

Another place where `dataclass` inspects a type annotation is to determine if a field is an init-only variable. It does this by seeing if the type of a field is of type `InitVar`. If a field is an `InitVar`, it is considered a pseudo-field called an init-only field. As it is not a true field, it is not returned by the module-level `fields` function. Init-only fields are added as parameters to the generated `~object.__init__` method, and are passed to the optional `__post_init__` method. They are not otherwise used by dataclasses.

For example, suppose a field will be initialized from a database, if a value is not provided when creating the class:

    @dataclass
    class C:
        i: int
        j: int | None = None
        database: InitVar[DatabaseType | None] = None

        def __post_init__(self, database):
            if self.j is None and database is not None:
                self.j = database.lookup('j')

    c = C(10, database=my_database)

In this case, `fields` will return `Field` objects for `!i` and `!j`, but not for `!database`.

## Frozen instances

It is not possible to create truly immutable Python objects. However, by passing `frozen=True` to the `dataclass` decorator you can emulate immutability. In that case, dataclasses will add `~object.__setattr__` and `~object.__delattr__` methods to the class. These methods will raise a `FrozenInstanceError` when invoked.

There is a tiny performance penalty when using `frozen=True`: `~object.__init__` cannot use simple assignment to initialize fields, and must use `!object.__setattr__`.

## Inheritance

When the dataclass is being created by the `dataclass` decorator, it looks through all of the class's base classes in reverse MRO (that is, starting at `object`) and, for each dataclass that it finds, adds the fields from that base class to an ordered mapping of fields. After all of the base class fields are added, it adds its own fields to the ordered mapping. All of the generated methods will use this combined, calculated ordered mapping of fields. Because the fields are in insertion order, derived classes override base classes. An example:

    @dataclass
    class Base:
        x: Any = 15.0
        y: int = 0

    @dataclass
    class C(Base):
        z: int = 10
        x: int = 15

The final list of fields is, in order, `!x`, `!y`, `!z`. The final type of `!x` is `int`, as specified in class `!C`.

The generated `~object.__init__` method for `!C` will look like:

    def __init__(self, x: int = 15, y: int = 0, z: int = 10):

## Re-ordering of keyword-only parameters in `!__init__`

After the parameters needed for `~object.__init__` are computed, any keyword-only parameters are moved to come after all regular (non-keyword-only) parameters. This is a requirement of how keyword-only parameters are implemented in Python: they must come after non-keyword-only parameters.

In this example, `!Base.y`, `!Base.w`, and `!D.t` are keyword-only fields, and `!Base.x` and `!D.z` are regular fields:

    @dataclass
    class Base:
        x: Any = 15.0
        _: KW_ONLY
        y: int = 0
        w: int = 1

    @dataclass
    class D(Base):
        z: int = 10
        t: int = field(kw_only=True, default=0)

The generated `!__init__` method for `!D` will look like:

    def __init__(self, x: Any = 15.0, z: int = 10, *, y: int = 0, w: int = 1, t: int = 0):

Note that the parameters have been re-ordered from how they appear in the list of fields: parameters derived from regular fields are followed by parameters derived from keyword-only fields.

The relative ordering of keyword-only parameters is maintained in the re-ordered `!__init__` parameter list.

## Default factory functions

If a `field` specifies a *default_factory*, it is called with zero arguments when a default value for the field is needed. For example, to create a new instance of a list, use:

    mylist: list = field(default_factory=list)

If a field is excluded from `~object.__init__` (using `init=False`) and the field also specifies *default_factory*, then the default factory function will always be called from the generated `!__init__` function. This happens because there is no other way to give the field an initial value.

## Mutable default values

Python stores default member variable values in class attributes. Consider this example, not using dataclasses:

    class C:
        x = []
        def add(self, element):
            self.x.append(element)

    o1 = C()
    o2 = C()
    o1.add(1)
    o2.add(2)
    assert o1.x == [1, 2]
    assert o1.x is o2.x

Note that the two instances of class `!C` share the same class variable `!x`, as expected.

Using dataclasses, *if* this code was valid:

    @dataclass
    class D:
        x: list = []      # This code raises ValueError
        def add(self, element):
            self.x.append(element)

it would generate code similar to:

    class D:
        x = []
        def __init__(self, x=x):
            self.x = x
        def add(self, element):
            self.x.append(element)

    assert D().x is D().x

This has the same issue as the original example using class `!C`. That is, two instances of class `!D` that do not specify a value for `!x` when creating a class instance will share the same copy of `!x`. Because dataclasses just use normal Python class creation they also share this behavior. There is no general way for Data Classes to detect this condition. Instead, the `dataclass` decorator will raise a `ValueError` if it detects an unhashable default parameter. The assumption is that if a value is unhashable, it is mutable. This is a partial solution, but it does protect against many common errors.

Using default factory functions is a way to create new instances of mutable types as default values for fields:

    @dataclass
    class D:
        x: list = field(default_factory=list)

    assert D().x is not D().x

<div class="versionchanged">

3.11 Instead of looking for and disallowing objects of type `list`, `dict`, or `set`, unhashable objects are now not allowed as default values. Unhashability is used to approximate mutability.

</div>

## Descriptor-typed fields

Fields that are assigned `descriptor objects <descriptors>` as their default value have the following special behaviors:

- The value for the field passed to the dataclass's `~object.__init__` method is passed to the descriptor's `~object.__set__` method rather than overwriting the descriptor object.
- Similarly, when getting or setting the field, the descriptor's `~object.__get__` or `!__set__` method is called rather than returning or overwriting the descriptor object.
- To determine whether a field contains a default value, `dataclass` will call the descriptor's `!__get__` method using its class access form: `descriptor.__get__(obj=None, type=cls)`. If the descriptor returns a value in this case, it will be used as the field's default. On the other hand, if the descriptor raises `AttributeError` in this situation, no default value will be provided for the field.

<!-- -->

    class IntConversionDescriptor:
        def __init__(self, *, default):
            self._default = default

        def __set_name__(self, owner, name):
            self._name = "_" + name

        def __get__(self, obj, type):
            if obj is None:
                return self._default

            return getattr(obj, self._name, self._default)

        def __set__(self, obj, value):
            setattr(obj, self._name, int(value))

    @dataclass
    class InventoryItem:
        quantity_on_hand: IntConversionDescriptor = IntConversionDescriptor(default=100)

    i = InventoryItem()
    print(i.quantity_on_hand)   # 100
    i.quantity_on_hand = 2.5    # calls __set__ with 2.5
    print(i.quantity_on_hand)   # 2

Note that if a field is annotated with a descriptor type, but is not assigned a descriptor object as its default value, the field will act like a normal field.

# Data Types

The modules described in this chapter provide a variety of specialized data types such as dates and times, fixed-type arrays, heap queues, double-ended queues, and enumerations.

Python also provides `some built-in data types <bltin-types>`, in particular, `list`, `tuple`, `dict`, `frozendict`, `set`, and `frozenset`. The `str` class is used to hold Unicode strings, and the `bytes` and `bytearray` classes are used to hold binary data.

The following modules are documented in this chapter:

<div class="toctree">

datetime.rst zoneinfo.rst calendar.rst collections.rst collections.abc.rst heapq.rst bisect.rst array.rst weakref.rst types.rst copy.rst pprint.rst reprlib.rst enum.rst graphlib.rst

</div>

# `!datetime` --- Basic date and time types

<div class="module" data-synopsis="Basic date and time types.">

datetime

</div>

**Source code:** `Lib/datetime.py`

------------------------------------------------------------------------

The `!datetime` module supplies classes for manipulating dates and times.

While date and time arithmetic is supported, the focus of the implementation is on efficient attribute extraction for output formatting and manipulation.

> [!TIP]
> Skip to `the format codes <format-codes>`.

<div class="seealso">

Module `calendar`  
General calendar related functions.

Module `time`  
Time access and conversions.

Module `zoneinfo`  
Concrete time zones representing the IANA time zone database.

Package [dateutil](https://dateutil.readthedocs.io/en/stable/)  
Third-party library with expanded time zone and parsing support.

Package `DateType`  
Third-party library that introduces distinct static types to for example, allow `static type checkers <static type checker>` to differentiate between naive and aware datetimes.

</div>

## Aware and naive objects

Date and time objects may be categorized as "aware" or "naive" depending on whether or not they include time zone information.

With sufficient knowledge of applicable algorithmic and political time adjustments, such as time zone and daylight saving time information, an **aware** object can locate itself relative to other aware objects. An aware object represents a specific moment in time that is not open to interpretation.[^1]

A **naive** object does not contain enough information to unambiguously locate itself relative to other date/time objects. Whether a naive object represents Coordinated Universal Time (UTC), local time, or time in some other time zone is purely up to the program, just like it is up to the program whether a particular number represents metres, miles, or mass. Naive objects are easy to understand and to work with, at the cost of ignoring some aspects of reality.

For applications requiring aware objects, `.datetime` and `.time` objects have an optional time zone information attribute, `!tzinfo`, that can be set to an instance of a subclass of the abstract `!tzinfo` class. These `tzinfo` objects capture information about the offset from UTC time, the time zone name, and whether daylight saving time is in effect.

Only one concrete `tzinfo` class, the `timezone` class, is supplied by the `!datetime` module. The `!timezone` class can represent simple time zones with fixed offsets from UTC, such as UTC itself or North American EST and EDT time zones. Supporting time zones at deeper levels of detail is up to the application. The rules for time adjustment across the world are more political than rational, change frequently, and there is no standard suitable for every application aside from UTC.

## Constants

The `!datetime` module exports the following constants:

<div class="data">

MINYEAR

The smallest year number allowed in a `date` or `.datetime` object. `MINYEAR` is 1.

</div>

<div class="data">

MAXYEAR

The largest year number allowed in a `date` or `.datetime` object. `MAXYEAR` is 9999.

</div>

<div class="data">

UTC

Alias for the UTC time zone singleton `datetime.timezone.utc`.

<div class="versionadded">

3.11

</div>

</div>

## Available types

<div class="date" data-noindex="">

An idealized naive date, assuming the current Gregorian calendar always was, and always will be, in effect. Attributes: `year`, `month`, and `day`.

</div>

<div class="time" data-noindex="">

An idealized time, independent of any particular day, assuming that every day has exactly 24\*60\*60 seconds. (There is no notion of "leap seconds" here.) Attributes: `hour`, `minute`, `second`, `microsecond`, and `.tzinfo`.

</div>

<div class="datetime" data-noindex="">

A combination of a date and a time. Attributes: `year`, `month`, `day`, `hour`, `minute`, `second`, `microsecond`, and `.tzinfo`.

</div>

<div class="timedelta" data-noindex="">

A duration expressing the difference between two `.datetime` or `date` instances to microsecond resolution.

</div>

<div class="tzinfo" data-noindex="">

An abstract base class for time zone information objects. These are used by the `.datetime` and `.time` classes to provide a customizable notion of time adjustment (for example, to account for time zone and/or daylight saving time).

</div>

<div class="timezone" data-noindex="">

A class that implements the `tzinfo` abstract base class as a fixed offset from the UTC.

<div class="versionadded">

3.2

</div>

</div>

Objects of these types are immutable.

Subclass relationships:

<figure class="invert-in-dark-modealign-center">
<img src="datetime-inheritance.svg" class="invert-in-dark-mode" alt="timedelta, tzinfo, time, and date inherit from object; timezone inherits from tzinfo; and datetime inherits from date." />
</figure>

### Common properties

The `date`, `.datetime`, `.time`, and `timezone` types share these common features:

- Objects of these types are immutable.
- Objects of these types are `hashable`, meaning that they can be used as dictionary keys.
- Objects of these types support efficient pickling via the `pickle` module.

### Determining if an object is aware or naive

Objects of the `date` type are always naive.

An object of type `.time` or `.datetime` may be aware or naive.

A `.datetime` object `d` is aware if `d.utcoffset() <datetime.utcoffset>` does not return `None`. Otherwise, `d` is naive.

A `.time` object `t` is aware if `t.utcoffset() <time.utcoffset>` does not return `None`. Otherwise, `t` is naive.

The distinction between aware and naive doesn't apply to `timedelta` objects.

## `!timedelta` objects

A `timedelta` object represents a duration, the difference between two `.datetime` or `date` instances.

<div class="timedelta(days=0, seconds=0, microseconds=0, milliseconds=0, minutes=0, hours=0, weeks=0)">

All arguments are optional and default to 0. Arguments may be integers or floats, and may be positive or negative.

Only *days*, *seconds* and *microseconds* are stored internally. Arguments are converted to those units:

- A millisecond is converted to 1000 microseconds.
- A minute is converted to 60 seconds.
- An hour is converted to 3600 seconds.
- A week is converted to 7 days.

and days, seconds and microseconds are then normalized so that the representation is unique, with

- `0 <= microseconds < 1000000`
- `0 <= seconds < 3600*24` (the number of seconds in one day)
- `-999999999 <= days <= 999999999`

The following example illustrates how any arguments besides *days*, *seconds* and *microseconds* are "merged" and normalized into those three resulting attributes:

    >>> import datetime as dt
    >>> delta = dt.timedelta(
    ...     days=50,
    ...     seconds=27,
    ...     microseconds=10,
    ...     milliseconds=29000,
    ...     minutes=5,
    ...     hours=8,
    ...     weeks=2
    ... )
    >>> # Only days, seconds, and microseconds remain
    >>> delta
    datetime.timedelta(days=64, seconds=29156, microseconds=10)

> [!TIP]
> `import datetime as dt` instead of `import datetime` or `from datetime import datetime` to avoid confusion between the module and the class. See [How I Import Python’s datetime Module](https://adamj.eu/tech/2019/09/12/how-i-import-pythons-datetime-module/).

If any argument is a float and there are fractional microseconds, the fractional microseconds left over from all arguments are combined and their sum is rounded to the nearest microsecond using round-half-to-even tiebreaker. If no argument is a float, the conversion and normalization processes are exact (no information is lost).

If the normalized value of days lies outside the indicated range, `OverflowError` is raised.

Note that normalization of negative values may be surprising at first. For example:

    >>> import datetime as dt
    >>> d = dt.timedelta(microseconds=-1)
    >>> (d.days, d.seconds, d.microseconds)
    (-1, 86399, 999999)

Since the string representation of `!timedelta` objects can be confusing, use the following recipe to produce a more readable format:

```
>>> def pretty_timedelta(td):
...     if td.days >= 0:
...         return str(td)
...     return f'-({-td!s})'
...
>>> d = timedelta(hours=-1)
>>> str(d)  # not human-friendly
'-1 day, 23:00:00'
>>> pretty_timedelta(d)
'-(1:00:00)'
```

</div>

Class attributes:

<div class="attribute">

timedelta.min

The most negative `timedelta` object, `timedelta(-999999999)`.

</div>

<div class="attribute">

timedelta.max

The most positive `timedelta` object, `timedelta(days=999999999, hours=23, minutes=59, seconds=59, microseconds=999999)`.

</div>

<div class="attribute">

timedelta.resolution

The smallest possible difference between non-equal `timedelta` objects, `timedelta(microseconds=1)`.

</div>

Note that, because of normalization, `timedelta.max` is greater than `-timedelta.min`. `-timedelta.max` is not representable as a `timedelta` object.

Instance attributes (read-only):

<div class="attribute">

timedelta.days

Between -999,999,999 and 999,999,999 inclusive.

</div>

<div class="attribute">

timedelta.seconds

Between 0 and 86,399 inclusive.

> [!CAUTION]
> It is a somewhat common bug for code to unintentionally use this attribute when it is actually intended to get a `~timedelta.total_seconds` value instead:
>
> <div class="doctest">
>
> \>\>\> import datetime as dt \>\>\> duration = dt.timedelta(seconds=11235813) \>\>\> duration.days, duration.seconds (130, 3813) \>\>\> duration.total_seconds() 11235813.0
>
> </div>

</div>

<div class="attribute">

timedelta.microseconds

Between 0 and 999,999 inclusive.

</div>

Supported operations:

| Operation | Result |
|----|----|
| `t1 = t2 + t3` | Sum of `t2` and `t3`. Afterwards `t1 - t2 == t3` and `t1 - t3 == t2` are true. (1) |
| `t1 = t2 - t3` | Difference of `t2` and `t3`. Afterwards `t1 == t2 - t3` and `t2 == t1 + t3` are true. (1)(6) |
| `t1 = t2 * i or t1 = i * t2` | Delta multiplied by an integer. Afterwards `t1 // i == t2` is true, provided `i != 0`. |
|  | In general, `t1  * i == t1 * (i-1) + t1` is true. (1) |
| `t1 = t2 * f or t1 = f * t2` | Delta multiplied by a float. The result is rounded to the nearest multiple of timedelta.resolution using round-half-to-even. |
| `f = t2 / t3` | Division (3) of overall duration `t2` by interval unit `t3`. Returns a `float` object. |
| `t1 = t2 / f or t1 = t2 / i` | Delta divided by a float or an int. The result is rounded to the nearest multiple of timedelta.resolution using round-half-to-even. |
| `t1 = t2 // i` or `t1 = t2 // t3` | The floor is computed and the remainder (if any) is thrown away. In the second case, an integer is returned. (3) |
| `t1 = t2 % t3` | The remainder is computed as a `timedelta` object. (3) |
| `q, r = divmod(t1, t2)` | Computes the quotient and the remainder: `q = t1 // t2` (3) and `r = t1 % t2`. `q` is an integer and `r` is a `timedelta` object. |
| `+t1` | Returns a `timedelta` object with the same value. (2) |
| `-t1` | Equivalent to `timedelta(-t1.days, -t1.seconds, -t1.microseconds)`, and to `t1 * -1`. (1)(4) |
| `abs(t)` | Equivalent to `+t` when `t.days >= 0`, and to `-t` when `t.days < 0`. (2) |
| `str(t)` | Returns a string in the form `[D day[s], ][H]H:MM:SS[.UUUUUU]`, where D is negative for negative `t`. (5) |
| `repr(t)` | Returns a string representation of the `timedelta` object as a constructor call with canonical attribute values. |

Notes:

\(1\)  
This is exact but may overflow.

\(2\)  
This is exact and cannot overflow.

\(3\)  
Division by zero raises `ZeroDivisionError`.

\(4\)  
`-timedelta.max` is not representable as a `timedelta` object.

\(5\)  
String representations of `timedelta` objects are normalized similarly to their internal representation. This leads to somewhat unusual results for negative timedeltas. For example:

    >>> timedelta(hours=-5)
    datetime.timedelta(days=-1, seconds=68400)
    >>> print(_)
    -1 day, 19:00:00

\(6\)  
The expression `t2 - t3` will always be equal to the expression `t2 + (-t3)` except when t3 is equal to `timedelta.max`; in that case the former will produce a result while the latter will overflow.

In addition to the operations listed above, `timedelta` objects support certain additions and subtractions with `date` and `.datetime` objects (see below).

<div class="versionchanged">

3.2 Floor division and true division of a `timedelta` object by another `!timedelta` object are now supported, as are remainder operations and the `divmod` function. True division and multiplication of a `!timedelta` object by a `float` object are now supported.

</div>

`timedelta` objects support equality and order comparisons.

In Boolean contexts, a `timedelta` object is considered to be true if and only if it isn't equal to `timedelta(0)`.

Instance methods:

<div class="method">

timedelta.total_seconds()

Return the total number of seconds contained in the duration. Equivalent to `td / timedelta(seconds=1)`. For interval units other than seconds, use the division form directly (for example, `td / timedelta(microseconds=1)`).

Note that for very large time intervals (greater than 270 years on most platforms) this method will lose microsecond accuracy.

<div class="versionadded">

3.2

</div>

</div>

### Examples of usage: `!timedelta`

An additional example of normalization:

    >>> # Components of another_year add up to exactly 365 days
    >>> import datetime as dt
    >>> year = dt.timedelta(days=365)
    >>> another_year = dt.timedelta(weeks=40, days=84, hours=23,
    ...                             minutes=50, seconds=600)
    >>> year == another_year
    True
    >>> year.total_seconds()
    31536000.0

Examples of `timedelta` arithmetic:

    >>> import datetime as dt
    >>> year = dt.timedelta(days=365)
    >>> ten_years = 10 * year
    >>> ten_years
    datetime.timedelta(days=3650)
    >>> ten_years.days // 365
    10
    >>> nine_years = ten_years - year
    >>> nine_years
    datetime.timedelta(days=3285)
    >>> three_years = nine_years // 3
    >>> three_years, three_years.days // 365
    (datetime.timedelta(days=1095), 3)

## `!date` objects

A `date` object represents a date (year, month and day) in an idealized calendar, the current Gregorian calendar indefinitely extended in both directions.

January 1 of year 1 is called day number 1, January 2 of year 1 is called day number 2, and so on.[^2]

<div class="date(year, month, day)">

All arguments are required. Arguments must be integers, in the following ranges:

- `MINYEAR <= year <= MAXYEAR`
- `1 <= month <= 12`
- `1 <= day <= number of days in the given month and year`

If an argument outside those ranges is given, `ValueError` is raised.

</div>

Other constructors, all class methods:

<div class="classmethod">

date.today()

Return the current local date.

This is equivalent to `date.fromtimestamp(time.time())`.

</div>

<div class="classmethod">

date.fromtimestamp(timestamp)

Return the local date corresponding to the POSIX *timestamp*, such as is returned by `time.time`.

This may raise `OverflowError`, if the timestamp is out of the range of values supported by the platform C `localtime` function, and `OSError` on `localtime` failure. It's common for this to be restricted to years from 1970 through 2038. Note that on non-POSIX systems that include leap seconds in their notion of a timestamp, leap seconds are ignored by `fromtimestamp`.

<div class="versionchanged">

3.3 Raise `OverflowError` instead of `ValueError` if the timestamp is out of the range of values supported by the platform C `localtime` function. Raise `OSError` instead of `ValueError` on `localtime` failure.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timestamp*, not only integer or float.

</div>

</div>

<div class="classmethod">

date.fromordinal(ordinal)

Return the date corresponding to the proleptic Gregorian *ordinal*, where January 1 of year 1 has ordinal 1.

`ValueError` is raised unless `1 <= ordinal <= date.max.toordinal()`. For any date `d`, `date.fromordinal(d.toordinal()) == d`.

</div>

<div class="classmethod">

date.fromisoformat(date_string)

Return a `date` corresponding to a *date_string* given in any valid ISO 8601 format, with the following exceptions:

1.  Reduced precision dates are not currently supported (`YYYY-MM`, `YYYY`).
2.  Extended date representations are not currently supported (`±YYYYYY-MM-DD`).
3.  Ordinal dates are not currently supported (`YYYY-OOO`).

Examples:

    >>> import datetime as dt
    >>> dt.date.fromisoformat('2019-12-04')
    datetime.date(2019, 12, 4)
    >>> dt.date.fromisoformat('20191204')
    datetime.date(2019, 12, 4)
    >>> dt.date.fromisoformat('2021-W01-1')
    datetime.date(2021, 1, 4)

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.11 Previously, this method only supported the format `YYYY-MM-DD`.

</div>

</div>

<div class="classmethod">

date.fromisocalendar(year, week, day)

Return a `date` corresponding to the ISO calendar date specified by *year*, *week* and *day*. This is the inverse of the function `date.isocalendar`.

<div class="versionadded">

3.8

</div>

</div>

<div class="classmethod">

date.strptime(date_string, format)

Return a `.date` corresponding to *date_string*, parsed according to *format*. This is equivalent to:

    date(*(time.strptime(date_string, format)[0:3]))

`ValueError` is raised if the date_string and format can't be parsed by `time.strptime` or if it returns a value which isn't a time tuple. See also `strftime-strptime-behavior` and `date.fromisoformat`.

> [!NOTE]
> If *format* specifies a day of month (`%d`) without a year, `ValueError` is raised. This is to avoid a quadrennial leap year bug in code seeking to parse only a month and day as the default year used in absence of one in the format is not a leap year. The workaround is to always include a year in your *format*. If parsing *date_string* values that do not have a year, explicitly add a year that is a leap year before parsing:
>
> <div class="doctest">
>
> \>\>\> import datetime as dt \>\>\> date_string = "02/29" \>\>\> when = dt.date.strptime(f"{date_string};1984", "%m/%d;%Y") \# Avoids leap year bug. \>\>\> when.strftime("%B %d") \# doctest: +SKIP 'February 29'
>
> </div>

<div class="versionadded">

3.14

</div>

</div>

Class attributes:

<div class="attribute">

date.min

The earliest representable date, `date(MINYEAR, 1, 1)`.

</div>

<div class="attribute">

date.max

The latest representable date, `date(MAXYEAR, 12, 31)`.

</div>

<div class="attribute">

date.resolution

The smallest possible difference between non-equal date objects, `timedelta(days=1)`.

</div>

Instance attributes (read-only):

<div class="attribute">

date.year

Between `MINYEAR` and `MAXYEAR` inclusive.

</div>

<div class="attribute">

date.month

Between 1 and 12 inclusive.

</div>

<div class="attribute">

date.day

Between 1 and the number of days in the given month of the given year.

</div>

Supported operations:

<table>
<colgroup>
<col style="width: 40%" />
<col style="width: 59%" />
</colgroup>
<thead>
<tr>
<th><p>Operation</p></th>
<th><p>Result</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p><code>date2 = date1 + timedelta</code></p></td>
<td><p><code>date2</code> will be <code>timedelta.days</code> days after <code>date1</code>. (1)</p></td>
</tr>
<tr>
<td><p><code>date2 = date1 - timedelta</code></p></td>
<td><p>Computes <code>date2</code> such that <code>date2 + timedelta == date1</code>. (2)</p></td>
</tr>
<tr>
<td><p><code>timedelta = date1 - date2</code></p></td>
<td><p>(3)</p></td>
</tr>
<tr>
<td><div class="line-block"><code>date1 == date2</code><br />
<code>date1 != date2</code></div></td>
<td><p>Equality comparison. (4)</p></td>
</tr>
<tr>
<td><div class="line-block"><code>date1 &lt; date2</code><br />
<code>date1 &gt; date2</code><br />
<code>date1 &lt;= date2</code><br />
<code>date1 &gt;= date2</code></div></td>
<td><p>Order comparison. (5)</p></td>
</tr>
</tbody>
</table>

Notes:

\(1\)  
*date2* is moved forward in time if `timedelta.days > 0`, or backward if `timedelta.days < 0`. Afterward `date2 - date1 == timedelta.days`. `timedelta.seconds` and `timedelta.microseconds` are ignored. `OverflowError` is raised if `date2.year` would be smaller than `MINYEAR` or larger than `MAXYEAR`.

\(2\)  
`timedelta.seconds` and `timedelta.microseconds` are ignored.

\(3\)  
This is exact, and cannot overflow. `timedelta.seconds` and `timedelta.microseconds` are 0, and `date2 + timedelta == date1` after.

\(4\)  
`date` objects are equal if they represent the same date.

`!date` objects that are not also `.datetime` instances are never equal to `!datetime` objects, even if they represent the same date.

\(5\)  
*date1* is considered less than *date2* when *date1* precedes *date2* in time. In other words, `date1 < date2` if and only if `date1.toordinal() < date2.toordinal()`.

Order comparison between a `date` object that is not also a `.datetime` instance and a `!datetime` object raises `TypeError`.

<div class="versionchanged">

3.13 Comparison between `.datetime` object and an instance of the `date` subclass that is not a `!datetime` subclass no longer converts the latter to `!date`, ignoring the time part and the time zone. The default behavior can be changed by overriding the special comparison methods in subclasses.

</div>

In Boolean contexts, all `date` objects are considered to be true.

Instance methods:

<div class="method">

date.replace(year=self.year, month=self.month, day=self.day)

Return a new `date` object with the same values, but with specified parameters updated.

Example:

    >>> import datetime as dt
    >>> d = dt.date(2002, 12, 31)
    >>> d.replace(day=26)
    datetime.date(2002, 12, 26)

The generic function `copy.replace` also supports `date` objects.

</div>

<div class="method">

date.timetuple()

Return a `time.struct_time` such as returned by `time.localtime`.

The hours, minutes and seconds are 0, and the DST flag is -1.

`d.timetuple()` is equivalent to:

    time.struct_time((d.year, d.month, d.day, 0, 0, 0, d.weekday(), yday, -1))

where `yday = d.toordinal() - date(d.year, 1, 1).toordinal() + 1` is the day number within the current year starting with 1 for January 1st.

</div>

<div class="method">

date.toordinal()

Return the proleptic Gregorian ordinal of the date, where January 1 of year 1 has ordinal 1. For any `date` object `d`, `date.fromordinal(d.toordinal()) == d`.

</div>

<div class="method">

date.weekday()

Return the day of the week as an integer, where Monday is 0 and Sunday is 6. For example, `date(2002, 12, 4).weekday() == 2`, a Wednesday. See also `isoweekday`.

</div>

<div class="method">

date.isoweekday()

Return the day of the week as an integer, where Monday is 1 and Sunday is 7. For example, `date(2002, 12, 4).isoweekday() == 3`, a Wednesday. See also `weekday`, `isocalendar`.

</div>

<div class="method">

date.isocalendar()

Return a `named tuple` object with three components: `year`, `week` and `weekday`.

The ISO calendar is a widely used variant of the Gregorian calendar.[^3]

The ISO year consists of 52 or 53 full weeks, and where a week starts on a Monday and ends on a Sunday. The first week of an ISO year is the first (Gregorian) calendar week of a year containing a Thursday. This is called week number 1, and the ISO year of that Thursday is the same as its Gregorian year.

For example, 2004 begins on a Thursday, so the first week of ISO year 2004 begins on Monday, 29 Dec 2003 and ends on Sunday, 4 Jan 2004:

    >>> import datetime as dt
    >>> dt.date(2003, 12, 29).isocalendar()
    datetime.IsoCalendarDate(year=2004, week=1, weekday=1)
    >>> dt.date(2004, 1, 4).isocalendar()
    datetime.IsoCalendarDate(year=2004, week=1, weekday=7)

<div class="versionchanged">

3.9 Result changed from a tuple to a `named tuple`.

</div>

</div>

<div class="method">

date.isoformat()

Return a string representing the date in ISO 8601 format, `YYYY-MM-DD`:

    >>> import datetime as dt
    >>> dt.date(2002, 12, 4).isoformat()
    '2002-12-04'

</div>

<div class="method">

date.\_\_str\_\_()

For a date `d`, `str(d)` is equivalent to `d.isoformat()`.

</div>

<div class="method">

date.ctime()

Return a string representing the date:

    >>> import datetime as dt
    >>> dt.date(2002, 12, 4).ctime()
    'Wed Dec  4 00:00:00 2002'

`d.ctime()` is equivalent to:

    time.ctime(time.mktime(d.timetuple()))

on platforms where the native C `ctime` function (which `time.ctime` invokes, but which `date.ctime` does not invoke) conforms to the C standard.

</div>

<div class="method">

date.strftime(format)

Return a string representing the date, controlled by an explicit format string. Format codes referring to hours, minutes or seconds will see 0 values. See also `strftime-strptime-behavior` and `date.isoformat`.

</div>

<div class="method">

date.\_\_format\_\_(format)

Same as `.date.strftime`. This makes it possible to specify a format string for a `.date` object in `formatted string
literals <f-strings>` and when using `str.format`. See also `strftime-strptime-behavior` and `date.isoformat`.

</div>

### Examples of usage: `!date`

Example of counting days to an event:

    >>> import time
    >>> import datetime as dt
    >>> today = dt.date.today()
    >>> today
    datetime.date(2007, 12, 5)
    >>> today == dt.date.fromtimestamp(time.time())
    True
    >>> my_birthday = dt.date(today.year, 6, 24)
    >>> if my_birthday < today:
    ...     my_birthday = my_birthday.replace(year=today.year + 1)
    ...
    >>> my_birthday
    datetime.date(2008, 6, 24)
    >>> time_to_birthday = abs(my_birthday - today)
    >>> time_to_birthday.days
    202

More examples of working with `date`:

<div class="doctest">

\>\>\> import datetime as dt \>\>\> d = dt.date.fromordinal(730920) \# 730920th day after 1. 1. 0001 \>\>\> d datetime.date(2002, 3, 11)

\>\>\> \# Methods related to formatting string output \>\>\> d.isoformat() '2002-03-11' \>\>\> d.strftime("%d/%m/%y") '11/03/02' \>\>\> d.strftime("%A %d. %B %Y") 'Monday 11. March 2002' \>\>\> d.ctime() 'Mon Mar 11 00:00:00 2002' \>\>\> 'The {1} is {0:%d}, the {2} is {0:%B}.'.format(d, "day", "month") 'The day is 11, the month is March.'

\>\>\> \# Methods for extracting 'components' under different calendars \>\>\> t = d.timetuple() \>\>\> for i in t: \# doctest: +SKIP ... print(i) 2002 \# year 3 \# month 11 \# day 0 0 0 0 \# weekday (0 = Monday) 70 \# 70th day in the year -1 \>\>\> ic = d.isocalendar() \>\>\> for i in ic: \# doctest: +SKIP ... print(i) 2002 \# ISO year 11 \# ISO week number 1 \# ISO day number ( 1 = Monday )

\>\>\> \# A date object is immutable; all operations produce a new object \>\>\> d.replace(year=2005) datetime.date(2005, 3, 11)

</div>

## `!datetime` objects

A `.datetime` object is a single object containing all the information from a `date` object and a `.time` object.

Like a `date` object, `.datetime` assumes the current Gregorian calendar extended in both directions; like a `.time` object, `!datetime` assumes there are exactly 3600\*24 seconds in every day.

Constructor:

<div class="datetime(year, month, day, hour=0, minute=0, second=0, microsecond=0, tzinfo=None, *, fold=0)">

The *year*, *month* and *day* arguments are required. *tzinfo* may be `None`, or an instance of a `tzinfo` subclass. The remaining arguments must be integers in the following ranges:

- `MINYEAR <= year <= MAXYEAR`,
- `1 <= month <= 12`,
- `1 <= day <= number of days in the given month and year`,
- `0 <= hour < 24`,
- `0 <= minute < 60`,
- `0 <= second < 60`,
- `0 <= microsecond < 1000000`,
- `fold in [0, 1]`.

If an argument outside those ranges is given, `ValueError` is raised.

<div class="versionchanged">

3.6 Added the *fold* parameter.

</div>

</div>

Other constructors, all class methods:

<div class="classmethod">

datetime.today()

Return the current local date and time, with `.tzinfo` `None`.

Equivalent to:

    datetime.fromtimestamp(time.time())

See also `now`, `fromtimestamp`.

This method is functionally equivalent to `now`, but without a `tz` parameter.

</div>

<div class="classmethod">

datetime.now(tz=None)

Return the current local date and time.

If optional argument *tz* is `None` or not specified, this is like `today`, but, if possible, supplies more precision than can be gotten from going through a `time.time` timestamp (for example, this may be possible on platforms supplying the C `gettimeofday` function).

If *tz* is not `None`, it must be an instance of a `tzinfo` subclass, and the current date and time are converted to *tz*’s time zone.

This function is preferred over `today` and `utcnow`.

> [!NOTE]
> Subsequent calls to `!datetime.now` may return the same instant depending on the precision of the underlying clock.

</div>

<div class="classmethod">

datetime.utcnow()

Return the current UTC date and time, with `.tzinfo` `None`.

This is like `now`, but returns the current UTC date and time, as a naive `.datetime` object. An aware current UTC datetime can be obtained by calling `datetime.now(timezone.utc)`. See also `now`.

> [!WARNING]
> Because naive `datetime` objects are treated by many `datetime` methods as local times, it is preferred to use aware datetimes to represent times in UTC. As such, the recommended way to create an object representing the current time in UTC is by calling `datetime.now(timezone.utc)`.

<div class="deprecated">

3.12

Use `datetime.now` with `UTC` instead.

</div>

</div>

<div class="classmethod">

datetime.fromtimestamp(timestamp, tz=None)

Return the local date and time corresponding to the POSIX timestamp, such as is returned by `time.time`. If optional argument *tz* is `None` or not specified, the timestamp is converted to the platform's local date and time, and the returned `.datetime` object is naive.

If *tz* is not `None`, it must be an instance of a `tzinfo` subclass, and the timestamp is converted to *tz*’s time zone.

`fromtimestamp` may raise `OverflowError`, if the timestamp is out of the range of values supported by the platform C `localtime` or `gmtime` functions, and `OSError` on `localtime` or `gmtime` failure. It's common for this to be restricted to years in 1970 through 2038. Note that on non-POSIX systems that include leap seconds in their notion of a timestamp, leap seconds are ignored by `fromtimestamp`, and then it's possible to have two timestamps differing by a second that yield identical `.datetime` objects. This method is preferred over `utcfromtimestamp`.

<div class="versionchanged">

3.3 Raise `OverflowError` instead of `ValueError` if the timestamp is out of the range of values supported by the platform C `localtime` or `gmtime` functions. Raise `OSError` instead of `ValueError` on `localtime` or `gmtime` failure.

</div>

<div class="versionchanged">

3.6 `fromtimestamp` may return instances with `.fold` set to 1.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timestamp*, not only integer or float.

</div>

</div>

<div class="classmethod">

datetime.utcfromtimestamp(timestamp)

Return the UTC `.datetime` corresponding to the POSIX timestamp, with `.tzinfo` `None`. (The resulting object is naive.)

This may raise `OverflowError`, if the timestamp is out of the range of values supported by the platform C `gmtime` function, and `OSError` on `gmtime` failure. It's common for this to be restricted to years in 1970 through 2038.

To get an aware `.datetime` object, call `fromtimestamp`:

    datetime.fromtimestamp(timestamp, timezone.utc)

On the POSIX compliant platforms, it is equivalent to the following expression:

    datetime(1970, 1, 1, tzinfo=timezone.utc) + timedelta(seconds=timestamp)

except the latter formula always supports the full years range: between `MINYEAR` and `MAXYEAR` inclusive.

> [!WARNING]
> Because naive `datetime` objects are treated by many `datetime` methods as local times, it is preferred to use aware datetimes to represent times in UTC. As such, the recommended way to create an object representing a specific timestamp in UTC is by calling `datetime.fromtimestamp(timestamp, tz=timezone.utc)`.

<div class="versionchanged">

3.3 Raise `OverflowError` instead of `ValueError` if the timestamp is out of the range of values supported by the platform C `gmtime` function. Raise `OSError` instead of `ValueError` on `gmtime` failure.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timestamp*, not only integer or float.

</div>

<div class="deprecated">

3.12

Use `datetime.fromtimestamp` with `UTC` instead.

</div>

</div>

<div class="classmethod">

datetime.fromordinal(ordinal)

Return the `.datetime` corresponding to the proleptic Gregorian ordinal, where January 1 of year 1 has ordinal 1. `ValueError` is raised unless `1 <= ordinal <= datetime.max.toordinal()`. The hour, minute, second and microsecond of the result are all 0, and `.tzinfo` is `None`.

</div>

<div class="classmethod">

datetime.combine(date, time, tzinfo=time.tzinfo)

Return a new `.datetime` object whose date components are equal to the given `date` object's, and whose time components are equal to the given `.time` object's. If the *tzinfo* argument is provided, its value is used to set the `.tzinfo` attribute of the result, otherwise the `~.time.tzinfo` attribute of the *time* argument is used. If the *date* argument is a `!datetime` object, its time components and `.tzinfo` attributes are ignored.

For any `.datetime` object `d`, `d == datetime.combine(d.date(), d.time(), d.tzinfo)`.

<div class="versionchanged">

3.6 Added the *tzinfo* argument.

</div>

</div>

<div class="classmethod">

datetime.fromisoformat(date_string)

Return a `.datetime` corresponding to a *date_string* in any valid ISO 8601 format, with the following exceptions:

1.  Time zone offsets may have fractional seconds.
2.  The `T` separator may be replaced by any single unicode character.
3.  Fractional hours and minutes are not supported.
4.  Reduced precision dates are not currently supported (`YYYY-MM`, `YYYY`).
5.  Extended date representations are not currently supported (`±YYYYYY-MM-DD`).
6.  Ordinal dates are not currently supported (`YYYY-OOO`).

Examples:

    >>> import datetime as dt
    >>> dt.datetime.fromisoformat('2011-11-04')
    datetime.datetime(2011, 11, 4, 0, 0)
    >>> dt.datetime.fromisoformat('20111104')
    datetime.datetime(2011, 11, 4, 0, 0)
    >>> dt.datetime.fromisoformat('2011-11-04T00:05:23')
    datetime.datetime(2011, 11, 4, 0, 5, 23)
    >>> dt.datetime.fromisoformat('2011-11-04T00:05:23Z')
    datetime.datetime(2011, 11, 4, 0, 5, 23, tzinfo=datetime.timezone.utc)
    >>> dt.datetime.fromisoformat('20111104T000523')
    datetime.datetime(2011, 11, 4, 0, 5, 23)
    >>> dt.datetime.fromisoformat('2011-W01-2T00:05:23.283')
    datetime.datetime(2011, 1, 4, 0, 5, 23, 283000)
    >>> dt.datetime.fromisoformat('2011-11-04 00:05:23.283')
    datetime.datetime(2011, 11, 4, 0, 5, 23, 283000)
    >>> dt.datetime.fromisoformat('2011-11-04 00:05:23.283+00:00')
    datetime.datetime(2011, 11, 4, 0, 5, 23, 283000, tzinfo=datetime.timezone.utc)
    >>> dt.datetime.fromisoformat('2011-11-04T00:05:23+04:00')   # doctest: +NORMALIZE_WHITESPACE
    datetime.datetime(2011, 11, 4, 0, 5, 23,
        tzinfo=datetime.timezone(datetime.timedelta(seconds=14400)))

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.11 Previously, this method only supported formats that could be emitted by `date.isoformat` or `datetime.isoformat`.

</div>

</div>

<div class="classmethod">

datetime.fromisocalendar(year, week, day)

Return a `.datetime` corresponding to the ISO calendar date specified by *year*, *week* and *day*. The non-date components of the datetime are populated with their normal default values. This is the inverse of the function `datetime.isocalendar`.

<div class="versionadded">

3.8

</div>

</div>

<div class="classmethod">

datetime.strptime(date_string, format)

Return a `.datetime` corresponding to *date_string*, parsed according to *format*.

If *format* does not contain microseconds or time zone information, this is equivalent to:

    datetime(*(time.strptime(date_string, format)[0:6]))

`ValueError` is raised if the date_string and format can't be parsed by `time.strptime` or if it returns a value which isn't a time tuple. See also `strftime-strptime-behavior` and `datetime.fromisoformat`.

<div class="versionchanged">

3.15

If *format* specifies a day of month (`%d`) without a year, `ValueError` is raised. This is to avoid a quadrennial leap year bug in code seeking to parse only a month and day as the default year used in absence of one in the format is not a leap year. The workaround is to always include a year in your *format*. If parsing *date_string* values that do not have a year, explicitly add a year that is a leap year before parsing:

<div class="doctest">

\>\>\> import datetime as dt \>\>\> date_string = "02/29" \>\>\> when = dt.datetime.strptime(f"{date_string};1984", "%m/%d;%Y") \# Avoids leap year bug. \>\>\> when.strftime("%B %d") \# doctest: +SKIP 'February 29'

</div>

</div>

</div>

Class attributes:

<div class="attribute">

datetime.min

The earliest representable `.datetime`, `datetime(MINYEAR, 1, 1, tzinfo=None)`.

</div>

<div class="attribute">

datetime.max

The latest representable `.datetime`, `datetime(MAXYEAR, 12, 31, 23, 59, 59, 999999, tzinfo=None)`.

</div>

<div class="attribute">

datetime.resolution

The smallest possible difference between non-equal `.datetime` objects, `timedelta(microseconds=1)`.

</div>

Instance attributes (read-only):

<div class="attribute">

datetime.year

Between `MINYEAR` and `MAXYEAR` inclusive.

</div>

<div class="attribute">

datetime.month

Between 1 and 12 inclusive.

</div>

<div class="attribute">

datetime.day

Between 1 and the number of days in the given month of the given year.

</div>

<div class="attribute">

datetime.hour

In `range(24)`.

</div>

<div class="attribute">

datetime.minute

In `range(60)`.

</div>

<div class="attribute">

datetime.second

In `range(60)`.

</div>

<div class="attribute">

datetime.microsecond

In `range(1000000)`.

</div>

<div class="attribute">

datetime.tzinfo

The object passed as the *tzinfo* argument to the `.datetime` constructor, or `None` if none was passed.

</div>

<div class="attribute">

datetime.fold

In `[0, 1]`. Used to disambiguate wall times during a repeated interval. (A repeated interval occurs when clocks are rolled back at the end of daylight saving time or when the UTC offset for the current zone is decreased for political reasons.) The values 0 and 1 represent, respectively, the earlier and later of the two moments with the same wall time representation.

<div class="versionadded">

3.6

</div>

</div>

Supported operations:

<table>
<colgroup>
<col style="width: 54%" />
<col style="width: 45%" />
</colgroup>
<thead>
<tr>
<th><p>Operation</p></th>
<th><p>Result</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p><code>datetime2 = datetime1 + timedelta</code></p></td>
<td><p>(1)</p></td>
</tr>
<tr>
<td><p><code>datetime2 = datetime1 - timedelta</code></p></td>
<td><p>(2)</p></td>
</tr>
<tr>
<td><p><code>timedelta = datetime1 - datetime2</code></p></td>
<td><p>(3)</p></td>
</tr>
<tr>
<td><div class="line-block"><code>datetime1 == datetime2</code><br />
<code>datetime1 != datetime2</code></div></td>
<td><p>Equality comparison. (4)</p></td>
</tr>
<tr>
<td><div class="line-block"><code>datetime1 &lt; datetime2</code><br />
<code>datetime1 &gt; datetime2</code><br />
<code>datetime1 &lt;= datetime2</code><br />
<code>datetime1 &gt;= datetime2</code></div></td>
<td><p>Order comparison. (5)</p></td>
</tr>
</tbody>
</table>

\(1\)  
`datetime2` is a duration of `timedelta` removed from `datetime1`, moving forward in time if `timedelta.days > 0`, or backward if `timedelta.days < 0`. The result has the same `~.datetime.tzinfo` attribute as the input datetime, and `datetime2 - datetime1 == timedelta` after. `OverflowError` is raised if `datetime2.year` would be smaller than `MINYEAR` or larger than `MAXYEAR`. Note that no time zone adjustments are done even if the input is an aware object.

\(2\)  
Computes the `datetime2` such that `datetime2 + timedelta == datetime1`. As for addition, the result has the same `~.datetime.tzinfo` attribute as the input datetime, and no time zone adjustments are done even if the input is aware.

\(3\)  
Subtraction of a `.datetime` from a `!datetime` is defined only if both operands are naive, or if both are aware. If one is aware and the other is naive, `TypeError` is raised.

If both are naive, or both are aware and have the same `~.datetime.tzinfo` attribute, the `~.datetime.tzinfo` attributes are ignored, and the result is a `timedelta` object `t` such that `datetime2 + t == datetime1`. No time zone adjustments are done in this case.

If both are aware and have different `~.datetime.tzinfo` attributes, `a-b` acts as if `a` and `b` were first converted to naive UTC datetimes. The result is `(a.replace(tzinfo=None) - a.utcoffset()) - (b.replace(tzinfo=None) - b.utcoffset())` except that the implementation never overflows.

\(4\)  
`.datetime` objects are equal if they represent the same date and time, taking into account the time zone.

Naive and aware `.datetime` objects are never equal.

If both comparands are aware, and have the same `!tzinfo` attribute, the `!tzinfo` and `~.datetime.fold` attributes are ignored and the base datetimes are compared. If both comparands are aware and have different `~.datetime.tzinfo` attributes, the comparison acts as comparands were first converted to UTC datetimes except that the implementation never overflows. `.datetime` instances in a repeated interval are never equal to `!datetime` instances in other time zone.

\(5\)  
*datetime1* is considered less than *datetime2* when *datetime1* precedes *datetime2* in time, taking into account the time zone.

Order comparison between naive and aware `.datetime` objects raises `TypeError`.

If both comparands are aware, and have the same `!tzinfo` attribute, the `!tzinfo` and `~.datetime.fold` attributes are ignored and the base datetimes are compared. If both comparands are aware and have different `~.datetime.tzinfo` attributes, the comparison acts as comparands were first converted to UTC datetimes except that the implementation never overflows.

<div class="versionchanged">

3.3 Equality comparisons between aware and naive `.datetime` instances don't raise `TypeError`.

</div>

<div class="versionchanged">

3.13 Comparison between `.datetime` object and an instance of the `date` subclass that is not a `!datetime` subclass no longer converts the latter to `!date`, ignoring the time part and the time zone. The default behavior can be changed by overriding the special comparison methods in subclasses.

</div>

Instance methods:

<div class="method">

datetime.date()

Return `date` object with same year, month and day.

</div>

<div class="method">

datetime.time()

Return `.time` object with same hour, minute, second, microsecond and fold. `.tzinfo` is `None`. See also method `timetz`.

<div class="versionchanged">

3.6 The fold value is copied to the returned `.time` object.

</div>

</div>

<div class="method">

datetime.timetz()

Return `.time` object with same hour, minute, second, microsecond, fold, and tzinfo attributes. See also method `time`.

<div class="versionchanged">

3.6 The fold value is copied to the returned `.time` object.

</div>

</div>

<div class="method">

datetime.replace(year=self.year, month=self.month, day=self.day, hour=self.hour, minute=self.minute, second=self.second, microsecond=self.microsecond, tzinfo=self.tzinfo, \*, fold=0)

Return a new `~datetime.datetime` object with the same attributes, but with specified parameters updated. Note that `tzinfo=None` can be specified to create a naive datetime from an aware datetime with no conversion of date and time data.

`.datetime` objects are also supported by generic function `copy.replace`.

<div class="versionchanged">

3.6 Added the *fold* parameter.

</div>

</div>

<div class="method">

datetime.astimezone(tz=None)

Return a `.datetime` object with new `.tzinfo` attribute *tz*, adjusting the date and time data so the result is the same UTC time as *self*, but in *tz*'s local time.

If provided, *tz* must be an instance of a `tzinfo` subclass, and its `utcoffset` and `dst` methods must not return `None`. If *self* is naive, it is presumed to represent time in the system time zone.

If called without arguments (or with `tz=None`) the system local time zone is assumed for the target time zone. The `.tzinfo` attribute of the converted datetime instance will be set to an instance of `timezone` with the zone name and offset obtained from the OS.

If `self.tzinfo` is *tz*, `self.astimezone(tz)` is equal to *self*: no adjustment of date or time data is performed. Else the result is local time in the time zone *tz*, representing the same UTC time as *self*: after `astz = dt.astimezone(tz)`, `astz - astz.utcoffset()` will have the same date and time data as `dt - dt.utcoffset()`.

If you merely want to attach a `timezone` object *tz* to a datetime *dt* without adjustment of date and time data, use `dt.replace(tzinfo=tz)`. If you merely want to remove the `!timezone` object from an aware datetime *dt* without conversion of date and time data, use `dt.replace(tzinfo=None)`.

Note that the default `tzinfo.fromutc` method can be overridden in a `tzinfo` subclass to affect the result returned by `astimezone`. Ignoring error cases, `astimezone` acts like:

    def astimezone(self, tz):
        if self.tzinfo is tz:
            return self
        # Convert self to UTC, and attach the new timezone object.
        utc = (self - self.utcoffset()).replace(tzinfo=tz)
        # Convert from UTC to tz's local time.
        return tz.fromutc(utc)

<div class="versionchanged">

3.3 *tz* now can be omitted.

</div>

<div class="versionchanged">

3.6 The `astimezone` method can now be called on naive instances that are presumed to represent system local time.

</div>

</div>

<div class="method">

datetime.utcoffset()

If `.tzinfo` is `None`, returns `None`, else returns `self.tzinfo.utcoffset(self)`, and raises an exception if the latter doesn't return `None` or a `timedelta` object with magnitude less than one day.

<div class="versionchanged">

3.7 The UTC offset is not restricted to a whole number of minutes.

</div>

</div>

<div class="method">

datetime.dst()

If `.tzinfo` is `None`, returns `None`, else returns `self.tzinfo.dst(self)`, and raises an exception if the latter doesn't return `None` or a `timedelta` object with magnitude less than one day.

<div class="versionchanged">

3.7 The DST offset is not restricted to a whole number of minutes.

</div>

</div>

<div class="method">

datetime.tzname()

If `.tzinfo` is `None`, returns `None`, else returns `self.tzinfo.tzname(self)`, raises an exception if the latter doesn't return `None` or a string object,

</div>

<div class="method">

datetime.timetuple()

Return a `time.struct_time` such as returned by `time.localtime`.

`d.timetuple()` is equivalent to:

    time.struct_time((d.year, d.month, d.day,
                      d.hour, d.minute, d.second,
                      d.weekday(), yday, dst))

where `yday = d.toordinal() - date(d.year, 1, 1).toordinal() + 1` is the day number within the current year starting with 1 for January 1st. The `~time.struct_time.tm_isdst` flag of the result is set according to the `dst` method: `.tzinfo` is `None` or `dst` returns `None`, `!tm_isdst` is set to `-1`; else if `dst` returns a non-zero value, `!tm_isdst` is set to 1; else `!tm_isdst` is set to 0.

</div>

<div class="method">

datetime.utctimetuple()

If `.datetime` instance `d` is naive, this is the same as `d.timetuple()` except that `~.time.struct_time.tm_isdst` is forced to 0 regardless of what `d.dst()` returns. DST is never in effect for a UTC time.

If `d` is aware, `d` is normalized to UTC time, by subtracting `d.utcoffset()`, and a `time.struct_time` for the normalized time is returned. `!tm_isdst` is forced to 0. Note that an `OverflowError` may be raised if `d.year` was `MINYEAR` or `MAXYEAR` and UTC adjustment spills over a year boundary.

> [!WARNING]
> Because naive `datetime` objects are treated by many `datetime` methods as local times, it is preferred to use aware datetimes to represent times in UTC; as a result, using `datetime.utctimetuple` may give misleading results. If you have a naive `datetime` representing UTC, use `datetime.replace(tzinfo=timezone.utc)` to make it aware, at which point you can use `.datetime.timetuple`.

</div>

<div class="method">

datetime.toordinal()

Return the proleptic Gregorian ordinal of the date. The same as `self.date().toordinal()`.

</div>

<div class="method">

datetime.timestamp()

Return POSIX timestamp corresponding to the `.datetime` instance. The return value is a `float` similar to that returned by `time.time`.

Naive `.datetime` instances are assumed to represent local time and this method relies on platform C functions to perform the conversion. Since `!datetime` supports a wider range of values than the platform C functions on many platforms, this method may raise `OverflowError` or `OSError` for times far in the past or far in the future.

For aware `.datetime` instances, the return value is computed as:

    (dt - datetime(1970, 1, 1, tzinfo=timezone.utc)).total_seconds()

> [!NOTE]
> There is no method to obtain the POSIX timestamp directly from a naive `.datetime` instance representing UTC time. If your application uses this convention and your system time zone is not set to UTC, you can obtain the POSIX timestamp by supplying `tzinfo=timezone.utc`:
>
>     timestamp = dt.replace(tzinfo=timezone.utc).timestamp()
>
> or by calculating the timestamp directly:
>
>     timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.6 The `timestamp` method uses the `.fold` attribute to disambiguate the times during a repeated interval.

</div>

<div class="versionchanged">

3.6 This method no longer relies on the platform C `mktime` function to perform conversions.

</div>

</div>

<div class="method">

datetime.weekday()

Return the day of the week as an integer, where Monday is 0 and Sunday is 6. The same as `self.date().weekday()`. See also `isoweekday`.

</div>

<div class="method">

datetime.isoweekday()

Return the day of the week as an integer, where Monday is 1 and Sunday is 7. The same as `self.date().isoweekday()`. See also `weekday`, `isocalendar`.

</div>

<div class="method">

datetime.isocalendar()

Return a `named tuple` with three components: `year`, `week` and `weekday`. The same as `self.date().isocalendar()`.

</div>

<div class="method">

datetime.isoformat(sep='T', timespec='auto')

Return a string representing the date and time in ISO 8601 format:

- `YYYY-MM-DDTHH:MM:SS.ffffff`, if `microsecond` is not 0
- `YYYY-MM-DDTHH:MM:SS`, if `microsecond` is 0

If `utcoffset` does not return `None`, a string is appended, giving the UTC offset:

- `YYYY-MM-DDTHH:MM:SS.ffffff+HH:MM[:SS[.ffffff]]`, if `microsecond` is not 0
- `YYYY-MM-DDTHH:MM:SS+HH:MM[:SS[.ffffff]]`, if `microsecond` is 0

Examples:

    >>> import datetime as dt
    >>> dt.datetime(2019, 5, 18, 15, 17, 8, 132263).isoformat()
    '2019-05-18T15:17:08.132263'
    >>> dt.datetime(2019, 5, 18, 15, 17, tzinfo=dt.timezone.utc).isoformat()
    '2019-05-18T15:17:00+00:00'

The optional argument *sep* (default `'T'`) is a one-character separator, placed between the date and time portions of the result. For example:

    >>> import datetime as dt
    >>> class TZ(dt.tzinfo):
    ...     """A time zone with an arbitrary, constant -06:39 offset."""
    ...     def utcoffset(self, when):
    ...         return dt.timedelta(hours=-6, minutes=-39)
    ...
    >>> dt.datetime(2002, 12, 25, tzinfo=TZ()).isoformat(' ')
    '2002-12-25 00:00:00-06:39'
    >>> dt.datetime(2009, 11, 27, microsecond=100, tzinfo=TZ()).isoformat()
    '2009-11-27T00:00:00.000100-06:39'

The optional argument *timespec* specifies the number of additional components of the time to include (the default is `'auto'`). It can be one of the following:

- `'auto'`: Same as `'seconds'` if `microsecond` is 0, same as `'microseconds'` otherwise.
- `'hours'`: Include the `hour` in the two-digit `HH` format.
- `'minutes'`: Include `hour` and `minute` in `HH:MM` format.
- `'seconds'`: Include `hour`, `minute`, and `second` in `HH:MM:SS` format.
- `'milliseconds'`: Include full time, but truncate fractional second part to milliseconds. `HH:MM:SS.sss` format.
- `'microseconds'`: Include full time in `HH:MM:SS.ffffff` format.

> [!NOTE]
> Excluded time components are truncated, not rounded.

`ValueError` will be raised on an invalid *timespec* argument:

    >>> import datetime as dt
    >>> dt.datetime.now().isoformat(timespec='minutes')   # doctest: +SKIP
    '2002-12-25T00:00'
    >>> my_datetime = dt.datetime(2015, 1, 1, 12, 30, 59, 0)
    >>> my_datetime.isoformat(timespec='microseconds')
    '2015-01-01T12:30:59.000000'

<div class="versionchanged">

3.6 Added the *timespec* parameter.

</div>

</div>

<div class="method">

datetime.\_\_str\_\_()

For a `.datetime` instance `d`, `str(d)` is equivalent to `d.isoformat(' ')`.

</div>

<div class="method">

datetime.ctime()

Return a string representing the date and time:

    >>> import datetime as dt
    >>> dt.datetime(2002, 12, 4, 20, 30, 40).ctime()
    'Wed Dec  4 20:30:40 2002'

The output string will *not* include time zone information, regardless of whether the input is aware or naive.

`d.ctime()` is equivalent to:

    time.ctime(time.mktime(d.timetuple()))

on platforms where the native C `ctime` function (which `time.ctime` invokes, but which `datetime.ctime` does not invoke) conforms to the C standard.

</div>

<div class="method">

datetime.strftime(format)

Return a string representing the date and time, controlled by an explicit format string. See also `strftime-strptime-behavior` and `datetime.isoformat`.

</div>

<div class="method">

datetime.\_\_format\_\_(format)

Same as `.datetime.strftime`. This makes it possible to specify a format string for a `.datetime` object in `formatted string
literals <f-strings>` and when using `str.format`. See also `strftime-strptime-behavior` and `datetime.isoformat`.

</div>

### Examples of usage: `!datetime`

Examples of working with `.datetime` objects:

<div class="doctest">

\>\>\> import datetime as dt

\>\>\> \# Using datetime.combine() \>\>\> d = dt.date(2005, 7, 14) \>\>\> t = dt.time(12, 30) \>\>\> dt.datetime.combine(d, t) datetime.datetime(2005, 7, 14, 12, 30)

\>\>\> \# Using datetime.now() \>\>\> dt.datetime.now() \# doctest: +SKIP datetime.datetime(2007, 12, 6, 16, 29, 43, 79043) \# GMT +1 \>\>\> dt.datetime.now(dt.timezone.utc) \# doctest: +SKIP datetime.datetime(2007, 12, 6, 15, 29, 43, 79060, tzinfo=datetime.timezone.utc)

\>\>\> \# Using datetime.strptime() \>\>\> my_datetime = dt.datetime.strptime("21/11/06 16:30", "%d/%m/%y %H:%M") \>\>\> my_datetime datetime.datetime(2006, 11, 21, 16, 30)

\>\>\> \# Using datetime.timetuple() to get tuple of all attributes \>\>\> tt = my_datetime.timetuple() \>\>\> for it in tt: \# doctest: +SKIP ... print(it) ... 2006 \# year 11 \# month 21 \# day 16 \# hour 30 \# minute 0 \# second 1 \# weekday (0 = Monday) 325 \# number of days since 1st January -1 \# dst - method tzinfo.dst() returned None

\>\>\> \# Date in ISO format \>\>\> ic = my_datetime.isocalendar() \>\>\> for it in ic: \# doctest: +SKIP ... print(it) ... 2006 \# ISO year 47 \# ISO week 2 \# ISO weekday

\>\>\> \# Formatting a datetime \>\>\> my_datetime.strftime("%A, %d. %B %Y %I:%M%p") 'Tuesday, 21. November 2006 04:30PM' \>\>\> 'The {1} is {0:%d}, the {2} is {0:%B}, the {3} is {0:%I:%M%p}.'.format(my_datetime, "day", "month", "time") 'The day is 21, the month is November, the time is 04:30PM.'

</div>

The example below defines a `tzinfo` subclass capturing time zone information for Kabul, Afghanistan, which used +4 UTC until 1945 and then +4:30 UTC thereafter:

    import datetime as dt

    class KabulTz(dt.tzinfo):
        # Kabul used +4 until 1945, when they moved to +4:30
        UTC_MOVE_DATE = dt.datetime(1944, 12, 31, 20, tzinfo=dt.timezone.utc)

        def utcoffset(self, when):
            if when.year < 1945:
                return dt.timedelta(hours=4)
            elif (1945, 1, 1, 0, 0) <= when.timetuple()[:5] < (1945, 1, 1, 0, 30):
                # An ambiguous ("imaginary") half-hour range representing
                # a 'fold' in time due to the shift from +4 to +4:30.
                # If when falls in the imaginary range, use fold to decide how
                # to resolve. See PEP 495.
                return dt.timedelta(hours=4, minutes=(30 if when.fold else 0))
            else:
                return dt.timedelta(hours=4, minutes=30)

        def fromutc(self, when):
            # Follow same validations as in datetime.tzinfo
            if not isinstance(when, dt.datetime):
                raise TypeError("fromutc() requires a datetime argument")
            if when.tzinfo is not self:
                raise ValueError("when.tzinfo is not self")

            # A custom implementation is required for fromutc as
            # the input to this function is a datetime with utc values
            # but with a tzinfo set to self.
            # See datetime.astimezone or fromtimestamp.
            if when.replace(tzinfo=dt.timezone.utc) >= self.UTC_MOVE_DATE:
                return when + dt.timedelta(hours=4, minutes=30)
            else:
                return when + dt.timedelta(hours=4)

        def dst(self, when):
            # Kabul does not observe daylight saving time.
            return dt.timedelta(0)

        def tzname(self, when):
            if when >= self.UTC_MOVE_DATE:
                return "+04:30"
            return "+04"

Usage of `KabulTz` from above:

    >>> tz1 = KabulTz()

    >>> # Datetime before the change
    >>> dt1 = dt.datetime(1900, 11, 21, 16, 30, tzinfo=tz1)
    >>> print(dt1.utcoffset())
    4:00:00

    >>> # Datetime after the change
    >>> dt2 = dt.datetime(2006, 6, 14, 13, 0, tzinfo=tz1)
    >>> print(dt2.utcoffset())
    4:30:00

    >>> # Convert datetime to another time zone
    >>> dt3 = dt2.astimezone(dt.timezone.utc)
    >>> dt3
    datetime.datetime(2006, 6, 14, 8, 30, tzinfo=datetime.timezone.utc)
    >>> dt2
    datetime.datetime(2006, 6, 14, 13, 0, tzinfo=KabulTz())
    >>> dt2 == dt3
    True

## `!time` objects

A `.time` object represents a (local) time of day, independent of any particular day, and subject to adjustment via a `tzinfo` object.

<div class="time(hour=0, minute=0, second=0, microsecond=0, tzinfo=None, *, fold=0)">

All arguments are optional. *tzinfo* may be `None`, or an instance of a `tzinfo` subclass. The remaining arguments must be integers in the following ranges:

- `0 <= hour < 24`,
- `0 <= minute < 60`,
- `0 <= second < 60`,
- `0 <= microsecond < 1000000`,
- `fold in [0, 1]`.

If an argument outside those ranges is given, `ValueError` is raised. All default to 0 except *tzinfo*, which defaults to `None`.

</div>

Class attributes:

<div class="attribute">

time.min

The earliest representable `.time`, `time(0, 0, 0, 0)`.

</div>

<div class="attribute">

time.max

The latest representable `.time`, `time(23, 59, 59, 999999)`.

</div>

<div class="attribute">

time.resolution

The smallest possible difference between non-equal `.time` objects, `timedelta(microseconds=1)`, although note that arithmetic on `.time` objects is not supported.

</div>

Instance attributes (read-only):

<div class="attribute">

time.hour

In `range(24)`.

</div>

<div class="attribute">

time.minute

In `range(60)`.

</div>

<div class="attribute">

time.second

In `range(60)`.

</div>

<div class="attribute">

time.microsecond

In `range(1000000)`.

</div>

<div class="attribute">

time.tzinfo

The object passed as the tzinfo argument to the `.time` constructor, or `None` if none was passed.

</div>

<div class="attribute">

time.fold

In `[0, 1]`. Used to disambiguate wall times during a repeated interval. (A repeated interval occurs when clocks are rolled back at the end of daylight saving time or when the UTC offset for the current zone is decreased for political reasons.) The values 0 and 1 represent, respectively, the earlier and later of the two moments with the same wall time representation.

<div class="versionadded">

3.6

</div>

</div>

`.time` objects support equality and order comparisons, where `a` is considered less than `b` when `a` precedes `b` in time.

Naive and aware `!time` objects are never equal. Order comparison between naive and aware `!time` objects raises `TypeError`.

If both comparands are aware, and have the same `~.time.tzinfo` attribute, the `!tzinfo` and `!fold` attributes are ignored and the base times are compared. If both comparands are aware and have different `!tzinfo` attributes, the comparands are first adjusted by subtracting their UTC offsets (obtained from `self.utcoffset()`).

<div class="versionchanged">

3.3 Equality comparisons between aware and naive `.time` instances don't raise `TypeError`.

</div>

In Boolean contexts, a `.time` object is always considered to be true.

<div class="versionchanged">

3.5 Before Python 3.5, a `.time` object was considered to be false if it represented midnight in UTC. This behavior was considered obscure and error-prone and has been removed in Python 3.5. See `13936` for more information.

</div>

Other constructors:

<div class="classmethod">

time.fromisoformat(time_string)

Return a `.time` corresponding to a *time_string* in any valid ISO 8601 format, with the following exceptions:

1.  Time zone offsets may have fractional seconds.
2.  The leading `T`, normally required in cases where there may be ambiguity between a date and a time, is not required.
3.  Fractional seconds may have any number of digits (anything beyond 6 will be truncated).
4.  Fractional hours and minutes are not supported.

Examples:

<div class="doctest">

\>\>\> import datetime as dt \>\>\> dt.time.fromisoformat('04:23:01') datetime.time(4, 23, 1) \>\>\> dt.time.fromisoformat('T04:23:01') datetime.time(4, 23, 1) \>\>\> dt.time.fromisoformat('T042301') datetime.time(4, 23, 1) \>\>\> dt.time.fromisoformat('04:23:01.000384') datetime.time(4, 23, 1, 384) \>\>\> dt.time.fromisoformat('04:23:01,000384') datetime.time(4, 23, 1, 384) \>\>\> dt.time.fromisoformat('04:23:01+04:00') datetime.time(4, 23, 1, tzinfo=datetime.timezone(datetime.timedelta(seconds=14400))) \>\>\> dt.time.fromisoformat('04:23:01Z') datetime.time(4, 23, 1, tzinfo=datetime.timezone.utc) \>\>\> dt.time.fromisoformat('04:23:01+00:00') datetime.time(4, 23, 1, tzinfo=datetime.timezone.utc)

</div>

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.11 Previously, this method only supported formats that could be emitted by `time.isoformat`.

</div>

</div>

<div class="classmethod">

time.strptime(date_string, format)

Return a `.time` corresponding to *date_string*, parsed according to *format*.

If *format* does not contain microseconds or timezone information, this is equivalent to:

    time(*(time.strptime(date_string, format)[3:6]))

`ValueError` is raised if the *date_string* and *format* cannot be parsed by `time.strptime` or if it returns a value which is not a time tuple. See also `strftime-strptime-behavior` and `time.fromisoformat`.

<div class="versionadded">

3.14

</div>

</div>

Instance methods:

<div class="method">

time.replace(hour=self.hour, minute=self.minute, second=self.second, microsecond=self.microsecond, tzinfo=self.tzinfo, \*, fold=0)

Return a new `.time` with the same values, but with specified parameters updated. Note that `tzinfo=None` can be specified to create a naive `!time` from an aware `!time`, without conversion of the time data.

`.time` objects are also supported by generic function `copy.replace`.

<div class="versionchanged">

3.6 Added the *fold* parameter.

</div>

</div>

<div class="method">

time.isoformat(timespec='auto')

Return a string representing the time in ISO 8601 format, one of:

- `HH:MM:SS.ffffff`, if `microsecond` is not 0
- `HH:MM:SS`, if `microsecond` is 0
- `HH:MM:SS.ffffff+HH:MM[:SS[.ffffff]]`, if `utcoffset` does not return `None`
- `HH:MM:SS+HH:MM[:SS[.ffffff]]`, if `microsecond` is 0 and `utcoffset` does not return `None`

The optional argument *timespec* specifies the number of additional components of the time to include (the default is `'auto'`). It can be one of the following:

- `'auto'`: Same as `'seconds'` if `microsecond` is 0, same as `'microseconds'` otherwise.
- `'hours'`: Include the `hour` in the two-digit `HH` format.
- `'minutes'`: Include `hour` and `minute` in `HH:MM` format.
- `'seconds'`: Include `hour`, `minute`, and `second` in `HH:MM:SS` format.
- `'milliseconds'`: Include full time, but truncate fractional second part to milliseconds. `HH:MM:SS.sss` format.
- `'microseconds'`: Include full time in `HH:MM:SS.ffffff` format.

> [!NOTE]
> Excluded time components are truncated, not rounded.

`ValueError` will be raised on an invalid *timespec* argument.

Example:

    >>> import datetime as dt
    >>> dt.time(hour=12, minute=34, second=56, microsecond=123456).isoformat(timespec='minutes')
    '12:34'
    >>> my_time = dt.time(hour=12, minute=34, second=56, microsecond=0)
    >>> my_time.isoformat(timespec='microseconds')
    '12:34:56.000000'
    >>> my_time.isoformat(timespec='auto')
    '12:34:56'

<div class="versionchanged">

3.6 Added the *timespec* parameter.

</div>

</div>

<div class="method">

time.\_\_str\_\_()

For a time `t`, `str(t)` is equivalent to `t.isoformat()`.

</div>

<div class="method">

time.strftime(format)

Return a string representing the time, controlled by an explicit format string. See also `strftime-strptime-behavior` and `time.isoformat`.

</div>

<div class="method">

time.\_\_format\_\_(format)

Same as `.time.strftime`. This makes it possible to specify a format string for a `.time` object in `formatted string
literals <f-strings>` and when using `str.format`. See also `strftime-strptime-behavior` and `time.isoformat`.

</div>

<div class="method">

time.utcoffset()

If `.tzinfo` is `None`, returns `None`, else returns `self.tzinfo.utcoffset(None)`, and raises an exception if the latter doesn't return `None` or a `timedelta` object with magnitude less than one day.

<div class="versionchanged">

3.7 The UTC offset is not restricted to a whole number of minutes.

</div>

</div>

<div class="method">

time.dst()

If `.tzinfo` is `None`, returns `None`, else returns `self.tzinfo.dst(None)`, and raises an exception if the latter doesn't return `None`, or a `timedelta` object with magnitude less than one day.

<div class="versionchanged">

3.7 The DST offset is not restricted to a whole number of minutes.

</div>

</div>

<div class="method">

time.tzname()

If `.tzinfo` is `None`, returns `None`, else returns `self.tzinfo.tzname(None)`, or raises an exception if the latter doesn't return `None` or a string object.

</div>

### Examples of usage: `!time`

Examples of working with a `.time` object:

    >>> import datetime as dt
    >>> class TZ1(dt.tzinfo):
    ...     def utcoffset(self, when):
    ...         return dt.timedelta(hours=1)
    ...     def dst(self, when):
    ...         return dt.timedelta(0)
    ...     def tzname(self, when):
    ...         return "+01:00"
    ...     def  __repr__(self):
    ...         return f"{self.__class__.__name__}()"
    ...
    >>> t = dt.time(12, 10, 30, tzinfo=TZ1())
    >>> t
    datetime.time(12, 10, 30, tzinfo=TZ1())
    >>> t.isoformat()
    '12:10:30+01:00'
    >>> t.dst()
    datetime.timedelta(0)
    >>> t.tzname()
    '+01:00'
    >>> t.strftime("%H:%M:%S %Z")
    '12:10:30 +01:00'
    >>> 'The {} is {:%H:%M}.'.format("time", t)
    'The time is 12:10.'

## `!tzinfo` objects

<div class="tzinfo()">

This is an `abstract base class`, meaning that this class should not be instantiated directly. Define a subclass of `tzinfo` to capture information about a particular time zone.

An instance of (a concrete subclass of) `tzinfo` can be passed to the constructors for `.datetime` and `.time` objects. The latter objects view their attributes as being in local time, and the `!tzinfo` object supports methods revealing offset of local time from UTC, the name of the time zone, and DST offset, all relative to a date or time object passed to them.

You need to derive a concrete subclass, and (at least) supply implementations of the standard `tzinfo` methods needed by the `.datetime` methods you use. The `!datetime` module provides `timezone`, a simple concrete subclass of `!tzinfo` which can represent time zones with fixed offset from UTC such as UTC itself or North American EST and EDT.

Special requirement for pickling: A `tzinfo` subclass must have an `~object.__init__` method that can be called with no arguments, otherwise it can be pickled but possibly not unpickled again. This is a technical requirement that may be relaxed in the future.

A concrete subclass of `tzinfo` may need to implement the following methods. Exactly which methods are needed depends on the uses made of aware `!datetime` objects. If in doubt, simply implement all of them.

</div>

<div class="method">

tzinfo.utcoffset(dt)

Return offset of local time from UTC, as a `timedelta` object that is positive east of UTC. If local time is west of UTC, this should be negative.

This represents the *total* offset from UTC; for example, if a `tzinfo` object represents both time zone and DST adjustments, `utcoffset` should return their sum. If the UTC offset isn't known, return `None`. Else the value returned must be a `timedelta` object strictly between `-timedelta(hours=24)` and `timedelta(hours=24)` (the magnitude of the offset must be less than one day). Most implementations of `utcoffset` will probably look like one of these two:

    return CONSTANT                 # fixed-offset class
    return CONSTANT + self.dst(dt)  # daylight-aware class

If `utcoffset` does not return `None`, `dst` should not return `None` either.

The default implementation of `utcoffset` raises `NotImplementedError`.

<div class="versionchanged">

3.7 The UTC offset is not restricted to a whole number of minutes.

</div>

</div>

<div class="method">

tzinfo.dst(dt)

Return the daylight saving time (DST) adjustment, as a `timedelta` object or `None` if DST information isn't known.

Return `timedelta(0)` if DST is not in effect. If DST is in effect, return the offset as a `timedelta` object (see `utcoffset` for details). Note that DST offset, if applicable, has already been added to the UTC offset returned by `utcoffset`, so there's no need to consult `dst` unless you're interested in obtaining DST info separately. For example, `datetime.timetuple` calls its `~.datetime.tzinfo` attribute's `dst` method to determine how the `~time.struct_time.tm_isdst` flag should be set, and `tzinfo.fromutc` calls `dst` to account for DST changes when crossing time zones.

An instance *tz* of a `tzinfo` subclass that models both standard and daylight times must be consistent in this sense:

`tz.utcoffset(dt) - tz.dst(dt)`

must return the same result for every `.datetime` *dt* with `dt.tzinfo == tz`. For sane `!tzinfo` subclasses, this expression yields the time zone's "standard offset", which should not depend on the date or the time, but only on geographic location. The implementation of `datetime.astimezone` relies on this, but cannot detect violations; it's the programmer's responsibility to ensure it. If a `!tzinfo` subclass cannot guarantee this, it may be able to override the default implementation of `tzinfo.fromutc` to work correctly with `~.datetime.astimezone` regardless.

Most implementations of `dst` will probably look like one of these two:

    import datetime as dt

    def dst(self, when):
        # a fixed-offset class:  doesn't account for DST
        return dt.timedelta(0)

or:

    import datetime as dt

    def dst(self, when):
        # Code to set dston and dstoff to the time zone's DST
        # transition times based on the input when.year, and expressed
        # in standard local time.

        if dston <= when.replace(tzinfo=None) < dstoff:
            return dt.timedelta(hours=1)
        else:
            return dt.timedelta(0)

The default implementation of `dst` raises `NotImplementedError`.

<div class="versionchanged">

3.7 The DST offset is not restricted to a whole number of minutes.

</div>

</div>

<div class="method">

tzinfo.tzname(dt)

Return the time zone name corresponding to the `.datetime` object *dt*, as a string. Nothing about string names is defined by the `!datetime` module, and there's no requirement that it mean anything in particular. For example, `"GMT"`, `"UTC"`, `"-500"`, `"-5:00"`, `"EDT"`, `"US/Eastern"`, `"America/New York"` are all valid replies. Return `None` if a string name isn't known. Note that this is a method rather than a fixed string primarily because some `tzinfo` subclasses will wish to return different names depending on the specific value of *dt* passed, especially if the `!tzinfo` class is accounting for daylight time.

The default implementation of `tzname` raises `NotImplementedError`.

</div>

These methods are called by a `.datetime` or `.time` object, in response to their methods of the same names. A `!datetime` object passes itself as the argument, and a `!time` object passes `None` as the argument. A `tzinfo` subclass's methods should therefore be prepared to accept a *dt* argument of `None`, or of class `!datetime`.

When `None` is passed, it's up to the class designer to decide the best response. For example, returning `None` is appropriate if the class wishes to say that time objects don't participate in the `tzinfo` protocols. It may be more useful for `utcoffset(None)` to return the standard UTC offset, as there is no other convention for discovering the standard offset.

When a `.datetime` object is passed in response to a `!datetime` method, `dt.tzinfo` is the same object as *self*. `tzinfo` methods can rely on this, unless user code calls `!tzinfo` methods directly. The intent is that the `!tzinfo` methods interpret *dt* as being in local time, and not need worry about objects in other time zones.

There is one more `tzinfo` method that a subclass may wish to override:

<div class="method">

tzinfo.fromutc(dt)

This is called from the default `datetime.astimezone` implementation. When called from that, `dt.tzinfo` is *self*, and *dt*'s date and time data are to be viewed as expressing a UTC time. The purpose of `fromutc` is to adjust the date and time data, returning an equivalent datetime in *self*'s local time.

Most `tzinfo` subclasses should be able to inherit the default `fromutc` implementation without problems. It's strong enough to handle fixed-offset time zones, and time zones accounting for both standard and daylight time, and the latter even if the DST transition times differ in different years. An example of a time zone the default `fromutc` implementation may not handle correctly in all cases is one where the standard offset (from UTC) depends on the specific date and time passed, which can happen for political reasons. The default implementations of `~.datetime.astimezone` and `fromutc` may not produce the result you want if the result is one of the hours straddling the moment the standard offset changes.

Skipping code for error cases, the default `fromutc` implementation acts like:

    import datetime as dt

    def fromutc(self, when):
        # raise ValueError error if when.tzinfo is not self
        dtoff = when.utcoffset()
        dtdst = when.dst()
        # raise ValueError if dtoff is None or dtdst is None
        delta = dtoff - dtdst  # this is self's standard offset
        if delta:
            when += delta   # convert to standard local time
            dtdst = when.dst()
            # raise ValueError if dtdst is None
        if dtdst:
            return when + dtdst
        else:
            return when

</div>

In the following `tzinfo_examples.py
<../includes/tzinfo_examples.py>` file there are some examples of `tzinfo` classes:

<div class="literalinclude">

../includes/tzinfo_examples.py

</div>

Note that there are unavoidable subtleties twice per year in a `tzinfo` subclass accounting for both standard and daylight time, at the DST transition points. For concreteness, consider US Eastern (UTC -0500), where EDT begins the minute after 1:59 (EST) on the second Sunday in March, and ends the minute after 1:59 (EDT) on the first Sunday in November:

    UTC   3:MM  4:MM  5:MM  6:MM  7:MM  8:MM
    EST  22:MM 23:MM  0:MM  1:MM  2:MM  3:MM
    EDT  23:MM  0:MM  1:MM  2:MM  3:MM  4:MM

    start  22:MM 23:MM  0:MM  1:MM  3:MM  4:MM

    end  23:MM  0:MM  1:MM  1:MM  2:MM  3:MM

When DST starts (the "start" line), the local wall clock leaps from 1:59 to 3:00. A wall time of the form 2:MM doesn't really make sense on that day, so `astimezone(Eastern)` won't deliver a result with `hour == 2` on the day DST begins. For example, at the Spring forward transition of 2016, we get:

    >>> import datetime as dt
    >>> from tzinfo_examples import HOUR, Eastern
    >>> u0 = dt.datetime(2016, 3, 13, 5, tzinfo=dt.timezone.utc)
    >>> for i in range(4):
    ...     u = u0 + i*HOUR
    ...     t = u.astimezone(Eastern)
    ...     print(u.time(), 'UTC =', t.time(), t.tzname())
    ...
    05:00:00 UTC = 00:00:00 EST
    06:00:00 UTC = 01:00:00 EST
    07:00:00 UTC = 03:00:00 EDT
    08:00:00 UTC = 04:00:00 EDT

When DST ends (the "end" line), there's a potentially worse problem: there's an hour that can't be spelled unambiguously in local wall time: the last hour of daylight time. In Eastern, that's times of the form 5:MM UTC on the day daylight time ends. The local wall clock leaps from 1:59 (daylight time) back to 1:00 (standard time) again. Local times of the form 1:MM are ambiguous. `~.datetime.astimezone` mimics the local clock's behavior by mapping two adjacent UTC hours into the same local hour then. In the Eastern example, UTC times of the form 5:MM and 6:MM both map to 1:MM when converted to Eastern, but earlier times have the `~.datetime.fold` attribute set to 0 and the later times have it set to 1. For example, at the Fall back transition of 2016, we get:

    >>> import datetime as dt
    >>> from tzinfo_examples import HOUR, Eastern
    >>> u0 = dt.datetime(2016, 11, 6, 4, tzinfo=dt.timezone.utc)
    >>> for i in range(4):
    ...     u = u0 + i*HOUR
    ...     t = u.astimezone(Eastern)
    ...     print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold)
    ...
    04:00:00 UTC = 00:00:00 EDT 0
    05:00:00 UTC = 01:00:00 EDT 0
    06:00:00 UTC = 01:00:00 EST 1
    07:00:00 UTC = 02:00:00 EST 0

Note that the `.datetime` instances that differ only by the value of the `~.datetime.fold` attribute are considered equal in comparisons.

Applications that can't bear wall-time ambiguities should explicitly check the value of the `~.datetime.fold` attribute or avoid using hybrid `tzinfo` subclasses; there are no ambiguities when using `timezone`, or any other fixed-offset `!tzinfo` subclass (such as a class representing only EST (fixed offset -5 hours), or only EDT (fixed offset -4 hours)).

<div class="seealso">

`zoneinfo`  
The `!datetime` module has a basic `timezone` class (for handling arbitrary fixed offsets from UTC) and its `timezone.utc` attribute (a UTC `!timezone` instance).

`zoneinfo` brings the *IANA time zone database* (also known as the Olson database) to Python, and its usage is recommended.

[IANA time zone database](https://www.iana.org/time-zones)  
The Time Zone Database (often called tz, tzdata or zoneinfo) contains code and data that represent the history of local time for many representative locations around the globe. It is updated periodically to reflect changes made by political bodies to time zone boundaries, UTC offsets, and daylight-saving rules.

</div>

## `!timezone` objects

The `timezone` class is a subclass of `tzinfo`, each instance of which represents a time zone defined by a fixed offset from UTC.

Objects of this class cannot be used to represent time zone information in the locations where different offsets are used in different days of the year or where historical changes have been made to civil time.

<div class="timezone(offset[, name])">

The *offset* argument must be specified as a `timedelta` object representing the difference between the local time and UTC. It must be strictly between `-timedelta(hours=24)` and `timedelta(hours=24)`, otherwise `ValueError` is raised.

The *name* argument is optional. If specified it must be a string that will be used as the value returned by the `datetime.tzname` method.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.7 The UTC offset is not restricted to a whole number of minutes.

</div>

</div>

<div class="method">

timezone.utcoffset(dt)

Return the fixed value specified when the `timezone` instance is constructed.

The *dt* argument is ignored. The return value is a `timedelta` instance equal to the difference between the local time and UTC.

<div class="versionchanged">

3.7 The UTC offset is not restricted to a whole number of minutes.

</div>

</div>

<div class="method">

timezone.tzname(dt)

Return the fixed value specified when the `timezone` instance is constructed.

If *name* is not provided in the constructor, the name returned by `tzname(dt)` is generated from the value of the `offset` as follows. If *offset* is `timedelta(0)`, the name is "UTC", otherwise it is a string in the format `UTC±HH:MM`, where ± is the sign of `offset`, HH and MM are two digits of `offset.hours` and `offset.minutes` respectively.

<div class="versionchanged">

3.6 Name generated from `offset=timedelta(0)` is now plain `'UTC'`, not `'UTC+00:00'`.

</div>

</div>

<div class="method">

timezone.dst(dt)

Always returns `None`.

</div>

<div class="method">

timezone.fromutc(dt)

Return `dt + offset`. The *dt* argument must be an aware `.datetime` instance, with `tzinfo` set to `self`.

</div>

<div class="attribute">

timezone.name

The *name* passed to the constructor, or `None` if no name was passed.

<div class="versionadded">

next

</div>

</div>

Class attributes:

<div class="attribute">

timezone.utc

The UTC time zone, `timezone(timedelta(0))`.

</div>

<div class="index">

single: % (percent); datetime format

</div>

## `!strftime` and `!strptime` behavior

`date`, `.datetime`, and `.time` objects all support a `strftime(format)` method, to create a string representing the time under the control of an explicit format string.

Conversely, the `date.strptime`, `datetime.strptime` and `time.strptime` class methods create an object from a string representing the time and a corresponding format string.

The table below provides a high-level comparison of `~.datetime.strftime` versus `~.datetime.strptime`:

|  | `strftime` | `strptime` |
|----|----|----|
| Usage | Convert object to a string according to a given format | Parse a string into an object given a corresponding format |
| Type of method | Instance method | Class method |
| Signature | `strftime(format)` | `strptime(date_string, format)` |

> 

### `!strftime` and `!strptime` format codes

These methods accept format codes that can be used to parse and format dates:

    >>> import datetime as dt
    >>> dt.datetime.strptime('31/01/22 23:59:59.999999',
    ...                      '%d/%m/%y %H:%M:%S.%f')
    datetime.datetime(2022, 1, 31, 23, 59, 59, 999999)
    >>> _.strftime('%a %d %b %Y, %I:%M%p')
    'Mon 31 Jan 2022, 11:59PM'

The following is a list of all the format codes that the 2011 C standard requires, and these work on all supported platforms.

<table style="width:97%;">
<colgroup>
<col style="width: 15%" />
<col style="width: 41%" />
<col style="width: 31%" />
<col style="width: 10%" />
</colgroup>
<thead>
<tr>
<th><p>Directive</p></th>
<th><blockquote>
<p>Meaning</p>
</blockquote></th>
<th><blockquote>
<p>Example</p>
</blockquote></th>
<th><p>Notes</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><blockquote>
<p><code>%a</code></p>
</blockquote></td>
<td><p>Weekday as locale's abbreviated name.</p></td>
<td><div class="line-block">Sun, Mon, ..., Sat (en_US);<br />
So, Mo, ..., Sa (de_DE)</div></td>
<td><p>(1)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%A</code></p>
</blockquote></td>
<td><p>Weekday as locale's full name.</p></td>
<td><div class="line-block">Sunday, Monday, ..., Saturday (en_US);<br />
Sonntag, Montag, ..., Samstag (de_DE)</div></td>
<td><p>(1)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%b</code></p>
</blockquote></td>
<td><p>Month as locale's abbreviated name.</p></td>
<td><div class="line-block">Jan, Feb, ..., Dec (en_US);<br />
Jan, Feb, ..., Dez (de_DE)</div></td>
<td><p>(1)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%B</code></p>
</blockquote></td>
<td><p>Month as locale's full name.</p></td>
<td><div class="line-block">January, February, ..., December (en_US);<br />
Januar, Februar, ..., Dezember (de_DE)</div></td>
<td><p>(1)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%c</code></p>
</blockquote></td>
<td><p>Locale's appropriate date and time representation.</p></td>
<td><div class="line-block">Tue Aug 16 21:30:00 1988 (en_US);<br />
Di 16 Aug 21:30:00 1988 (de_DE)</div></td>
<td><p>(1)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%C</code></p>
</blockquote></td>
<td><p>The year divided by 100 and truncated to an integer as a zero-padded decimal number.</p></td>
<td><p>01, 02, ..., 99</p></td>
<td><p>(0)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%d</code></p>
</blockquote></td>
<td><p>Day of the month as a zero-padded decimal number.</p></td>
<td><p>01, 02, ..., 31</p></td>
<td><p>(9), (10)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%D</code></p>
</blockquote></td>
<td><p>Equivalent to <code>%m/%d/%y</code>.</p></td>
<td><p>11/28/25</p></td>
<td><p>(9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%e</code></p>
</blockquote></td>
<td><p>The day of the month as a space-padded decimal number.</p></td>
<td><p>␣1, ␣2, ..., 31</p></td>
<td><p>(10)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%F</code></p>
</blockquote></td>
<td><p>Equivalent to <code>%Y-%m-%d</code>, the ISO 8601 format.</p></td>
<td><p>2025-10-11, 1001-12-30</p></td>
<td></td>
</tr>
<tr>
<td><blockquote>
<p><code>%g</code></p>
</blockquote></td>
<td><p>Last 2 digits of ISO 8601 year representing the year that contains the greater part of the ISO week (<code>%V</code>).</p></td>
<td><p>00, 01, ..., 99</p></td>
<td><p>(0)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%G</code></p>
</blockquote></td>
<td><p>ISO 8601 year with century representing the year that contains the greater part of the ISO week (<code>%V</code>).</p></td>
<td><p>0001, 0002, ..., 2013, 2014, ..., 9998, 9999</p></td>
<td><p>(8)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%h</code></p>
</blockquote></td>
<td><p>Equivalent to <code>%b</code>.</p></td>
<td><p>See <code>%b</code>.</p></td>
<td><p>(0)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%H</code></p>
</blockquote></td>
<td><p>Hour (24-hour clock) as a zero-padded decimal number.</p></td>
<td><p>00, 01, ..., 23</p></td>
<td><p>(9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%I</code></p>
</blockquote></td>
<td><p>Hour (12-hour clock) as a zero-padded decimal number.</p></td>
<td><p>01, 02, ..., 12</p></td>
<td><p>(9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%j</code></p>
</blockquote></td>
<td><p>Day of the year as a zero-padded decimal number.</p></td>
<td><p>001, 002, ..., 366</p></td>
<td><p>(9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%m</code></p>
</blockquote></td>
<td><p>Month as a zero-padded decimal number.</p></td>
<td><p>01, 02, ..., 12</p></td>
<td><p>(9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%M</code></p>
</blockquote></td>
<td><p>Minute as a zero-padded decimal number.</p></td>
<td><p>00, 01, ..., 59</p></td>
<td><p>(9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%n</code></p>
</blockquote></td>
<td><p>The newline character (<code>'\n'</code>). For <code class="interpreted-text" role="meth">!strptime</code>, zero or more whitespace.</p></td>
<td><p><code>\n</code></p></td>
<td></td>
</tr>
<tr>
<td><blockquote>
<p><code>%p</code></p>
</blockquote></td>
<td><p>Locale's equivalent of either AM or PM.</p></td>
<td><div class="line-block">AM, PM (en_US);<br />
am, pm (de_DE)</div></td>
<td><p>(1), (3)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%r</code></p>
</blockquote></td>
<td><p>Locale's 12-hour clock time.</p></td>
<td><p>12:00:00 AM</p></td>
<td><p>(1), (0)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%R</code></p>
</blockquote></td>
<td><p>Equivalent to <code>%H:%M</code>.</p></td>
<td><p>10:01</p></td>
<td></td>
</tr>
<tr>
<td><blockquote>
<p><code>%S</code></p>
</blockquote></td>
<td><p>Second as a zero-padded decimal number.</p></td>
<td><p>00, 01, ..., 59</p></td>
<td><p>(4), (9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%t</code></p>
</blockquote></td>
<td><p>The tab character (<code>'\t'</code>). For <code class="interpreted-text" role="meth">!strptime</code>, zero or more whitespace.</p></td>
<td><p><code>\t</code></p></td>
<td></td>
</tr>
<tr>
<td><blockquote>
<p><code>%T</code></p>
</blockquote></td>
<td><p>ISO 8601 time format, equivalent to <code>%H:%M:%S</code>.</p></td>
<td><p>10:01:59</p></td>
<td></td>
</tr>
<tr>
<td><blockquote>
<p><code>%u</code></p>
</blockquote></td>
<td><p>ISO 8601 weekday as a decimal number where 1 is Monday.</p></td>
<td><p>1, 2, ..., 7</p></td>
<td></td>
</tr>
<tr>
<td><blockquote>
<p><code>%U</code></p>
</blockquote></td>
<td><p>Week number of the year (Sunday as the first day of the week) as a zero-padded decimal number. All days in a new year preceding the first Sunday are considered to be in week 0.</p></td>
<td><p>00, 01, ..., 53</p></td>
<td><p>(7), (9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%V</code></p>
</blockquote></td>
<td><p>ISO 8601 week as a decimal number with Monday as the first day of the week. Week 01 is the week containing Jan 4.</p></td>
<td><p>01, 02, ..., 53</p></td>
<td><p>(8), (9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%w</code></p>
</blockquote></td>
<td><p>Weekday as a decimal number, where 0 is Sunday and 6 is Saturday.</p></td>
<td><p>0, 1, ..., 6</p></td>
<td></td>
</tr>
<tr>
<td><blockquote>
<p><code>%W</code></p>
</blockquote></td>
<td><p>Week number of the year (Monday as the first day of the week) as a zero-padded decimal number. All days in a new year preceding the first Monday are considered to be in week 0.</p></td>
<td><p>00, 01, ..., 53</p></td>
<td><p>(7), (9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%x</code></p>
</blockquote></td>
<td><p>Locale's appropriate date representation.</p></td>
<td><div class="line-block">08/16/88 (None);<br />
08/16/1988 (en_US);<br />
16.08.1988 (de_DE)</div></td>
<td><p>(1)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%X</code></p>
</blockquote></td>
<td><p>Locale's appropriate time representation.</p></td>
<td><div class="line-block">21:30:00 (en_US);<br />
21:30:00 (de_DE)</div></td>
<td><p>(1)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%y</code></p>
</blockquote></td>
<td><p>Year without century as a zero-padded decimal number.</p></td>
<td><p>00, 01, ..., 99</p></td>
<td><p>(9)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%Y</code></p>
</blockquote></td>
<td><p>Year with century as a decimal number.</p></td>
<td><p>0001, 0002, ..., 2013, 2014, ..., 9998, 9999</p></td>
<td><p>(2)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%z</code></p>
</blockquote></td>
<td><p>UTC offset in the form <code>±HHMM[SS[.ffffff]]</code> (empty string if the object is naive).</p></td>
<td><p>(empty), +0000, -0400, +1030, +063415, -030712.345216</p></td>
<td><p>(6)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%Z</code></p>
</blockquote></td>
<td><p>Time zone name (empty string if the object is naive).</p></td>
<td><p>(empty), UTC, GMT</p></td>
<td><p>(6)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%%</code></p>
</blockquote></td>
<td><p>A literal <code>'%'</code> character.</p></td>
<td><p>%</p></td>
<td></td>
</tr>
</tbody>
</table>

The ISO 8601 year and ISO 8601 week directives are not interchangeable with the year and week number directives above. Calling `~.datetime.strptime` with incomplete or ambiguous ISO 8601 directives will raise a `ValueError`.

Several additional directives not required by the C11 standard are included for convenience.

<table style="width:97%;">
<colgroup>
<col style="width: 15%" />
<col style="width: 41%" />
<col style="width: 31%" />
<col style="width: 10%" />
</colgroup>
<thead>
<tr>
<th><p>Directive</p></th>
<th><p>Meaning</p></th>
<th><p>Example</p></th>
<th><p>Notes</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><blockquote>
<p><code>%f</code></p>
</blockquote></td>
<td><p>Microsecond as a decimal number, zero-padded to 6 digits.</p></td>
<td><p>000000, 000001, ..., 999999</p></td>
<td><p>(5)</p></td>
</tr>
<tr>
<td><blockquote>
<p><code>%Nf</code></p>
</blockquote></td>
<td><p>Fraction of a second truncated to <em>N</em> digits, where <em>N</em> is from 1 to 6.</p></td>
<td><p>000, 001, ..., 999 (<code>%3f</code>)</p></td>
<td><p>(5)</p></td>
</tr>
<tr>
<td><p><code>%:z</code></p></td>
<td><p>UTC offset in the form <code>±HH:MM[:SS[.ffffff]]</code> (empty string if the object is naive).</p></td>
<td><p>(empty), +00:00, -04:00, +10:30, +06:34:15, -03:07:12.345216</p></td>
<td><p>(6)</p></td>
</tr>
</tbody>
</table>

The full set of format codes supported varies across platforms, because Python calls the platform C library's `strftime` function, and platform variations are common. To see the full set of format codes supported on your platform, consult the `strftime(3)` documentation. There are also differences between platforms in handling of unsupported format specifiers.

<div class="versionadded">

3.6 `%G`, `%u` and `%V` were added.

</div>

<div class="versionadded">

3.12 `%:z` was added for `~.datetime.strftime`.

</div>

<div class="versionadded">

3.15 `%D`, `%F`, `%n`, `%t`, and `%:z` were added for `~.datetime.strptime`.

</div>

<div class="versionadded">

next `%Nf` was added.

</div>

### Technical detail

Broadly speaking, `d.strftime(fmt)` acts like the `time` module's `time.strftime(fmt, d.timetuple())` although not all objects support a `~date.timetuple` method.

For the `.datetime.strptime` and `.date.strptime` class methods, the default value is `1900-01-01T00:00:00.000`: any components not specified in the format string will be pulled from the default value.

> [!NOTE]
> Format strings without separators can be ambiguous for parsing. For example, with `%Y%m%d`, the string `2026111` may be parsed either as `2026-11-01` or as `2026-01-11`. Use separators to ensure the input is parsed as intended.

> [!NOTE]
> When used to parse partial dates lacking a year, `.datetime.strptime` and `.date.strptime` will raise when encountering February 29 because the default year of 1900 is *not* a leap year. Always add a default leap year to partial date strings before parsing.

> [!NOTE]
> If a `~.datetime.strptime` format string contains more than one directive for the same field, for example both `%Y` and `%y`, every directive must still match the input, but only the value of the last one is used. For example, parsing `200099` with `%Y%y` gives the year 1999. Repeating the same directive, like `%Y%Y`, raises an error.

<div class="testsetup">

\# doctest seems to turn the warning into an error which makes it \# show up and require matching and prevents the actual interesting \# exception from being raised. \# Manually apply the catch_warnings context manager import warnings catch_warnings = warnings.catch_warnings() catch_warnings.\_\_enter\_\_() warnings.simplefilter("ignore")

</div>

<div class="testcleanup">

catch_warnings.\_\_exit\_\_()

</div>

<div class="doctest">

\>\>\> import datetime as dt \>\>\> value = "2/29" \>\>\> dt.datetime.strptime(value, "%m/%d") Traceback (most recent call last): ... ValueError: day 29 must be in range 1..28 for month 2 in year 1900 \>\>\> dt.datetime.strptime(f"1904 {value}", "%Y %m/%d") datetime.datetime(1904, 2, 29, 0, 0)

</div>

Using `datetime.strptime(date_string, format)` is equivalent to:

    datetime(*(time.strptime(date_string, format)[0:6]))

except when the format includes sub-second components or time zone offset information, which are supported in `datetime.strptime` but are discarded by `time.strptime`.

For `.time` objects, the format codes for year, month, and day should not be used, as `!time` objects have no such values. If they're used anyway, 1900 is substituted for the year, and 1 for the month and day.

For `date` objects, the format codes for hours, minutes, seconds, and microseconds should not be used, as `date` objects have no such values. If they're used anyway, 0 is substituted for them.

For the same reason, handling of format strings containing Unicode code points that can't be represented in the charset of the current locale is also platform-dependent. On some platforms such code points are preserved intact in the output, while on others `strftime` may raise `UnicodeError` or return an empty string instead.

Notes:

\(0\)  
This format code is currently unsupported by `~.datetime.strptime`.

\(1\)  
Because the format depends on the current locale, care should be taken when making assumptions about the output value. Field orderings will vary (for example, "month/day/year" versus "day/month/year"), and the output may contain non-ASCII characters.

\(2\)  
The `~.datetime.strptime` method can parse years in the full \[1, 9999\] range, but years \< 1000 must be zero-filled to 4-digit width.

<div class="versionchanged">

3.2 In previous versions, `~.datetime.strftime` method was restricted to years \>= 1900.

</div>

<div class="versionchanged">

3.3 In version 3.2, `~.datetime.strftime` method was restricted to years \>= 1000.

</div>

\(3\)  
When used with the `~.datetime.strptime` method, the `%p` directive only affects the output hour field if the `%I` directive is used to parse the hour.

\(4\)  
Unlike the `time` module, the `!datetime` module does not support leap seconds.

\(5\)  
When used with the `~.datetime.strptime` method, the `%f` directive accepts from one to six digits and zero pads on the right, while `%Nf` accepts exactly *N* digits. `%f` is an extension to the set of format characters in the C standard (but implemented separately in datetime objects, and therefore always available).

\(6\)  
For a naive object, the `%z`, `%:z` and `%Z` format codes are replaced by empty strings.

For an aware object:

`%z`  
`~.datetime.utcoffset` is transformed into a string of the form `±HHMM[SS[.ffffff]]`, where `HH` is a 2-digit string giving the number of UTC offset hours, `MM` is a 2-digit string giving the number of UTC offset minutes, `SS` is a 2-digit string giving the number of UTC offset seconds and `ffffff` is a 6-digit string giving the number of UTC offset microseconds. The `ffffff` part is omitted when the offset is a whole number of seconds and both the `ffffff` and the `SS` part is omitted when the offset is a whole number of minutes. For example, if `~.datetime.utcoffset` returns `timedelta(hours=-3, minutes=-30)`, `%z` is replaced with the string `'-0330'`.

<div class="versionchanged">

3.7 The UTC offset is not restricted to a whole number of minutes.

</div>

<div class="versionchanged">

3.7 When the `%z` directive is provided to the `~.datetime.strptime` method, the UTC offsets can have a colon as a separator between hours, minutes and seconds. For example, both `'+010000'` and `'+01:00:00'` will be parsed as an offset of one hour. In addition, providing `'Z'` is identical to `'+00:00'`.

</div>

`%:z`  
When used with `~.datetime.strftime`, behaves exactly as `%z`, except that a colon separator is added between hours, minutes and seconds.

When used with `~.datetime.strptime`, the UTC offset is *required* to have a colon as a separator between hours, minutes and seconds. For example, `'+01:00:00'` (but *not* `'+010000'`) will be parsed as an offset of one hour. In addition, providing `'Z'` is identical to `'+00:00'`.

`%Z`  
In `~.datetime.strftime`, `%Z` is replaced by an empty string if `~.datetime.tzname` returns `None`; otherwise `%Z` is replaced by the returned value, which must be a string.

`~.datetime.strptime` only accepts certain values for `%Z`:

1.  any value in `time.tzname` for your machine's locale
2.  the hard-coded values `UTC` and `GMT`

So someone living in Japan may have `JST`, `UTC`, and `GMT` as valid values, but probably not `EST`. It will raise `ValueError` for invalid values.

<div class="versionchanged">

3.2 When the `%z` directive is provided to the `~.datetime.strptime` method, an aware `.datetime` object will be produced. The `tzinfo` of the result will be set to a `timezone` instance.

</div>

\(7\)  
When used with the `~.datetime.strptime` method, `%U` and `%W` are only used in calculations when the day of the week and the calendar year (`%Y`) are specified.

\(8\)  
Similar to `%U` and `%W`, `%V` is only used in calculations when the day of the week and the ISO year (`%G`) are specified in a `~.datetime.strptime` format string. Also note that `%G` and `%Y` are not interchangeable.

\(9\)  
When used with the `~.datetime.strptime` method, the leading zero is optional for formats `%d`, `%m`, `%H`, `%I`, `%M`, `%S`, `%j`, `%U`, `%W`, and `%V`. Format `%y` does require a leading zero.

\(10\)  
When parsing a month and day using `~.datetime.strptime`, always include a year in the format. If the value you need to parse lacks a year, append an explicit dummy leap year. Otherwise your code will raise an exception when it encounters leap day because the default year used by the parser (1900) is not a leap year. Users run into that bug every leap year.

<div class="doctest">

\>\>\> month_day = "02/29" \>\>\> dt.datetime.strptime(f"{month_day};1984", "%m/%d;%Y") \# No leap year bug. datetime.datetime(1984, 2, 29, 0, 0)

</div>

<div class="versionchanged">

3.15 Using `%d` without a year now raises `ValueError`.

</div>

<div class="deprecated-removed">

3.15 3.17 `~.datetime.strptime` calls using a format string containing `%e` without a year now emit a `DeprecationWarning`.

</div>

**Footnotes**

[^1]: If, that is, we ignore the effects of relativity.

[^2]: This matches the definition of the "proleptic Gregorian" calendar in Dershowitz and Reingold's book *Calendrical Calculations*, where it's the base calendar for all computations. See the book for algorithms for converting between proleptic Gregorian ordinals and many other calendar systems.

[^3]: See R. H. van Gent's [guide to the mathematics of the ISO 8601 calendar](https://web.archive.org/web/20220531051136/https://webspace.science.uu.nl/~gent0113/calendar/isocalendar.htm) for a good explanation.

# `!dbm` --- Interfaces to Unix "databases"

<div class="module" data-synopsis="Interfaces to various Unix &quot;database&quot; formats.">

dbm

</div>

**Source code:** `Lib/dbm/__init__.py`

------------------------------------------------------------------------

`!dbm` is a generic interface to variants of the DBM database:

- `dbm.sqlite3`
- `dbm.gnu`
- `dbm.ndbm`

If none of these modules are installed, the slow-but-simple implementation in module `dbm.dumb` will be used. There is a [third party interface](https://www.jcea.es/programacion/pybsddb.htm) to the Oracle Berkeley DB.

> [!NOTE]
> None of the underlying modules will automatically shrink the disk space used by the database file. However, `dbm.sqlite3`, `dbm.gnu` and `dbm.dumb` provide a `!reorganize` method that can be used for this purpose.

<div class="exception">

error

A tuple containing the exceptions that can be raised by each of the supported modules, with a unique exception also named `dbm.error` as the first item --- the latter is used when `dbm.error` is raised.

</div>

<div class="function">

whichdb(filename)

This function attempts to guess which of the several simple database modules available --- `dbm.sqlite3`, `dbm.gnu`, `dbm.ndbm`, or `dbm.dumb` --- should be used to open a given file.

Return one of the following values:

- `None` if the file can't be opened because it's unreadable or doesn't exist
- the empty string (`''`) if the file's format can't be guessed
- a string containing the required module name, such as `'dbm.ndbm'` or `'dbm.gnu'`

<div class="versionchanged">

3.11 *filename* accepts a `path-like object`.

</div>

</div>

<div class="function">

open(file, flag='r', mode=0o666)

Open a database and return the corresponding database object.

param file  
The database file to open.

If the database file already exists, the `whichdb` function is used to determine its type and the appropriate module is used; if it does not exist, the first submodule listed above that can be imported is used.

type file  
`path-like object`

param str flag  
- `'r'` (default): Open existing database for reading only.
- `'w'`: Open existing database for reading and writing.
- `'c'`: Open database for reading and writing, creating it if it doesn't exist.
- `'n'`: Always create a new, empty database, open for reading and writing.

param int mode  
The Unix file access mode of the file (default: octal `0o666`), used only when the database has to be created.

<div class="versionchanged">

3.11 *file* accepts a `path-like object`.

</div>

</div>

The object returned by `~dbm.open` supports the basic functionality of mutable `mappings <mapping>`; keys and their corresponding values can be stored, retrieved, and deleted, and iteration, the `in` operator and methods `!keys`, `!get`, `!setdefault` and `!clear` are available. The `!keys` method returns a list instead of a view object. The `!setdefault` method requires two arguments.

Key and values are always stored as `bytes`. This means that when strings are used they are implicitly converted to the default encoding before being stored. For detailed information about accepted types, see `dbm-key-value-types`.

These objects also support being used in a `with` statement, which will automatically close them when done.

<div class="versionchanged">

3.2 `!get` and `!setdefault` methods are now available for all `!dbm` backends.

</div>

<div class="versionchanged">

3.4 Added native support for the context management protocol to the objects returned by `~dbm.open`.

</div>

<div class="versionchanged">

3.8 Deleting a key from a read-only database raises a database module specific exception instead of `KeyError`.

</div>

<div class="versionchanged">

3.13 `!clear` methods are now available for all `!dbm` backends.

</div>

## Key and value types

The accepted types for keys and values vary by backend. Keys and values are handled the same way, except where noted below:

- **Traditional backends:**
  - `dbm.gnu` and `dbm.ndbm`: Accept `str` and `bytes` objects.
  - `dbm.dumb`: Accepts `str` and `bytes` objects; `bytearray` is acceptable as a value, but not as a key.
- **SQLite backend** (`dbm.sqlite3`): Accepts `str`, `bytes`, `int`, `float`, `bool`, `bytearray`, `memoryview`, and `array.array` objects.

**Storage format:**

All keys and values are stored as `bytes` objects in the database. When retrieving, you'll always get bytes back, regardless of the original type stored.

**Type conversion examples:**

- `db['key'] = 'string'` stored as `b'string'`
- `db['key'] = bytearray(b'data')` stored as `b'data'` (`dbm.dumb` and `dbm.sqlite3` only)
- `db['key'] = 42` stored as `b'42'` (`dbm.sqlite3` only)
- `db['key'] = 3.14` stored as `b'3.14'` (`dbm.sqlite3` only)
- `db['key'] = True` stored as `b'1'` (`dbm.sqlite3` only)
- `db['key'] = False` stored as `b'0'` (`dbm.sqlite3` only)
- `db['key'] = memoryview(b'data')` stored as `b'data'` (`dbm.sqlite3` only)
- `db['key'] = array.array('i', [1, 2, 3])` stored as bytes (for example, on little-endian: `b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00'`) (`dbm.sqlite3` only)

The following example records some hostnames and a corresponding title, and then prints out the contents of the database:

    import dbm

    # Open database, creating it if necessary.
    with dbm.open('cache', 'c') as db:

        # Record some values
        db[b'hello'] = b'there'
        db['www.python.org'] = 'Python Website'
        db['www.cnn.com'] = 'Cable News Network'

        # Note that the keys are considered bytes now.
        assert db[b'www.python.org'] == b'Python Website'
        # Notice how the value is now in bytes.
        assert db['www.cnn.com'] == b'Cable News Network'

        # Often-used methods of the dict interface work too.
        print(db.get('python.org', b'not present'))

        # Storing a non-string key or value behavior depends on the backend:
        # - Traditional backends (ndbm, gnu, dumb) will raise a TypeError
        # - The sqlite3 backend accepts it and converts to bytes
        db['www.yahoo.com'] = 4

    # db is automatically closed when leaving the with statement.

<div class="seealso">

Module `shelve`  
Persistence module which stores non-string data.

</div>

The individual submodules are described in the following sections.

## `!dbm.sqlite3` --- SQLite backend for dbm

<div class="module" data-synopsis="SQLite backend for dbm">

dbm.sqlite3

</div>

<div class="versionadded">

3.13

</div>

**Source code:** `Lib/dbm/sqlite3.py`

------------------------------------------------------------------------

This module uses the standard library `sqlite3` module to provide an SQLite backend for the `!dbm` module. The files created by `!dbm.sqlite3` can thus be opened by `sqlite3`, or any other SQLite browser, including the SQLite CLI.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

<div class="function">

open(filename, /, flag="r", mode=0o666)

Open an SQLite database.

param filename  
The path to the database to be opened.

type filename  
`path-like object`

param str flag  

> - `'r'` (default): Open existing database for reading only.
> - `'w'`: Open existing database for reading and writing.
> - `'c'`: Open database for reading and writing, creating it if it doesn't exist.
> - `'n'`: Always create a new, empty database, open for reading and writing.

param mode  
The Unix file access mode of the file (default: octal `0o666`), used only when the database has to be created.

The returned database object behaves similar to a mutable `mapping`, but the `!keys` method returns a list, and the `!setdefault` method requires two arguments. It also supports a "closing" context manager via the `with` keyword.

The following methods are also provided:

<div class="method">

sqlite3.close()

Close the SQLite database.

</div>

<div class="method">

sqlite3.reorganize()

If you have carried out a lot of deletions and would like to shrink the space used on disk, this method will reorganize the database; otherwise, deleted file space will be kept and reused as new (key, value) pairs are added.

> [!NOTE]
> While reorganizing, as much as two times the size of the original database is required in free disk space. However, be aware that this factor changes for each `!dbm` submodule.

<div class="versionadded">

3.15

</div>

</div>

</div>

## `!dbm.gnu` --- GNU database manager

<div class="module" data-synopsis="GNU database manager">

dbm.gnu

</div>

**Source code:** `Lib/dbm/gnu.py`

------------------------------------------------------------------------

The `!dbm.gnu` module provides an interface to the `GDBM (GNU dbm)` library, similar to the `dbm.ndbm` module, but with additional functionality like crash tolerance.

> [!NOTE]
> The file formats created by `!dbm.gnu` and `dbm.ndbm` are incompatible and can not be used interchangeably.

<div class="availability">

not Android, not iOS, not WASI.

This module is not supported on `mobile platforms <mobile-availability>` or `WebAssembly platforms <wasm-availability>`.

</div>

<div class="availability">

Unix.

</div>

<div class="exception">

error

Raised on `!dbm.gnu`-specific errors, such as I/O errors. `KeyError` is raised for general mapping errors like specifying an incorrect key.

</div>

<div class="data">

open_flags

A string of characters the *flag* parameter of `~dbm.gnu.open` supports.

</div>

<div class="data">

GDBM_VERSION_INFO

A named tuple containing the three components of the GDBM library version that was used for building the module: *major*, *minor*, and *patch*. All values are integers. The components can also be accessed by name, so `dbm.gnu.GDBM_VERSION_INFO[0]` is equivalent to `dbm.gnu.GDBM_VERSION_INFO.major` and so on. This may be different from the GDBM library actually used at runtime, which is available as `gdbm_version_info`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

gdbm_version_info

A named tuple containing the version of the GDBM library actually loaded by the interpreter, with the same fields as `GDBM_VERSION_INFO`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

gdbm_version

The release string of the GDBM library actually loaded by the interpreter, like `'GDBM version 1.26. 30/07/2025'`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

open(filename, flag="r", mode=0o666, /)

Open a GDBM database and return a `!gdbm` object.

param filename  
The database file to open.

type filename  
`path-like object`

param str flag  
- `'r'` (default): Open existing database for reading only.
- `'w'`: Open existing database for reading and writing.
- `'c'`: Open database for reading and writing, creating it if it doesn't exist.
- `'n'`: Always create a new, empty database, open for reading and writing.

The following additional characters may be appended to control how the database is opened:

- `'f'`: Open the database in fast mode. Writes to the database will not be synchronized.
- `'s'`: Synchronized mode. Changes to the database will be written immediately to the file.
- `'u'`: Do not lock database.

Not all flags are valid for all versions of GDBM. See the `open_flags` member for a list of supported flag characters.

param int mode  
The Unix file access mode of the file (default: octal `0o666`), used only when the database has to be created.

raises error  
If an invalid *flag* argument is passed.

<div class="versionchanged">

3.11 *filename* accepts a `path-like object`.

</div>

`!gdbm` objects behave similar to mutable `mappings <mapping>`, but methods `!items`, `!values`, `!pop`, `!popitem`, and `!update` are not supported, the `!keys` method returns a list, and the `!setdefault` method requires two arguments. It also supports a "closing" context manager via the `with` keyword.

<div class="versionchanged">

3.2 Added the `!get` and `!setdefault` methods.

</div>

<div class="versionchanged">

3.13 Added the `!clear` method.

</div>

The following methods are also provided:

<div class="method">

gdbm.close()

Close the GDBM database.

</div>

<div class="method">

gdbm.firstkey()

It's possible to loop over every key in the database using this method and the `nextkey` method. The traversal is ordered by GDBM's internal hash values, and won't be sorted by the key values. This method returns the starting key.

</div>

<div class="method">

gdbm.nextkey(key)

Returns the key that follows *key* in the traversal. The following code prints every key in the database `db`, without having to create a list in memory that contains them all:

    k = db.firstkey()
    while k is not None:
        print(k)
        k = db.nextkey(k)

</div>

<div class="method">

gdbm.reorganize()

If you have carried out a lot of deletions and would like to shrink the space used by the GDBM file, this routine will reorganize the database. `!gdbm` objects will not shorten the length of a database file except by using this reorganization; otherwise, deleted file space will be kept and reused as new (key, value) pairs are added.

> [!NOTE]
> While reorganizing, as much as one time the size of the original database is required in free disk space. However, be aware that this factor changes for each `!dbm` submodule.

</div>

<div class="method">

gdbm.sync()

When the database has been opened in fast mode, this method forces any unwritten data to be written to the disk.

</div>

</div>

## `!dbm.ndbm` --- New Database Manager

<div class="module" data-synopsis="The New Database Manager">

dbm.ndbm

</div>

**Source code:** `Lib/dbm/ndbm.py`

------------------------------------------------------------------------

The `!dbm.ndbm` module provides an interface to the `NDBM (New Database Manager)` library. This module can be used with the "classic" NDBM interface or the `GDBM (GNU dbm)` compatibility interface.

> [!NOTE]
> The file formats created by `dbm.gnu` and `!dbm.ndbm` are incompatible and can not be used interchangeably.

> [!WARNING]
> The NDBM library shipped as part of macOS has an undocumented limitation on the size of values, which can result in corrupted database files when storing values larger than this limit. Reading such corrupted files can result in a hard crash (segmentation fault).

<div class="availability">

not Android, not iOS, not WASI.

This module is not supported on `mobile platforms <mobile-availability>` or `WebAssembly platforms <wasm-availability>`.

</div>

<div class="availability">

Unix.

</div>

<div class="exception">

error

Raised on `!dbm.ndbm`-specific errors, such as I/O errors. `KeyError` is raised for general mapping errors like specifying an incorrect key.

</div>

<div class="data">

library

Name of the NDBM implementation library used: `'GNU gdbm'`, `'Berkeley DB'` or `'ndbm'`.

<div class="versionchanged">

3.16 The value is `'ndbm'` for a classic NDBM library. It was `'GNU gdbm'` before.

</div>

</div>

<div class="data">

GDBM_VERSION_INFO gdbm_version_info gdbm_version

Information about the GDBM library in use, with the same meaning as the constants of the same names in `dbm.gnu`. Only available if `library` is `'GNU gdbm'`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

BDB_VERSION

The version string of the Berkeley DB library that was used for building the module, like `'Berkeley DB 5.3.28: (September  9, 2013)'`. This may be different from the Berkeley DB library actually used at runtime, which is available as `bdb_version`. Only available if `library` is `'Berkeley DB'`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

bdb_version

The version string of the Berkeley DB library actually loaded by the interpreter. Only available if `library` is `'Berkeley DB'`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

BDB_VERSION_INFO

A named tuple containing the three components of the Berkeley DB library version that was used for building the module: *major*, *minor*, and *patch*. All values are integers. The components can also be accessed by name, so `dbm.ndbm.BDB_VERSION_INFO[0]` is equivalent to `dbm.ndbm.BDB_VERSION_INFO.major` and so on. This may be different from the Berkeley DB library actually used at runtime, which is available as `bdb_version_info`. Only available if `library` is `'Berkeley DB'`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

bdb_version_info

A named tuple containing the version of the Berkeley DB library actually loaded by the interpreter, with the same fields as `BDB_VERSION_INFO`. Only available if `library` is `'Berkeley DB'`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

open(filename, flag="r", mode=0o666, /)

Open an NDBM database and return an `!ndbm` object.

param filename  
The basename of the database file (without the `.dir` or `.pag` extensions).

type filename  
`path-like object`

param str flag  
- `'r'` (default): Open existing database for reading only.
- `'w'`: Open existing database for reading and writing.
- `'c'`: Open database for reading and writing, creating it if it doesn't exist.
- `'n'`: Always create a new, empty database, open for reading and writing.

param int mode  
The Unix file access mode of the file (default: octal `0o666`), used only when the database has to be created.

<div class="versionchanged">

3.11 Accepts `path-like object` for filename.

</div>

`!ndbm` objects behave similar to mutable `mappings <mapping>`, but methods `!items`, `!values`, `!pop`, `!popitem`, and `!update` are not supported, the `!keys` method returns a list, and the `!setdefault` method requires two arguments. It also supports a "closing" context manager via the `with` keyword.

<div class="versionchanged">

3.2 Added the `!get` and `!setdefault` methods.

</div>

<div class="versionchanged">

3.13 Added the `!clear` method.

</div>

The following method is also provided:

<div class="method">

ndbm.close()

Close the NDBM database.

</div>

</div>

## `!dbm.dumb` --- Portable DBM implementation

<div class="module" data-synopsis="Portable implementation of the simple DBM interface.">

dbm.dumb

</div>

**Source code:** `Lib/dbm/dumb.py`

<div class="index">

single: databases

</div>

> [!NOTE]
> The `!dbm.dumb` module is intended as a last resort fallback for the `!dbm` module when a more robust module is not available. The `!dbm.dumb` module is not written for speed and is not nearly as heavily used as the other database modules.

------------------------------------------------------------------------

The `!dbm.dumb` module provides a persistent `dict`-like interface which is written entirely in Python. Unlike other `!dbm` backends, such as `dbm.gnu`, no external library is required.

The `!dbm.dumb` module defines the following:

<div class="exception">

error

Raised on `!dbm.dumb`-specific errors, such as I/O errors. `KeyError` is raised for general mapping errors like specifying an incorrect key.

</div>

<div class="function">

open(filename, flag="c", mode=0o666)

Open a `!dbm.dumb` database.

param filename  
The basename of the database file (without extensions). A new database creates the following files:

- `{filename}.dat`
- `{filename}.dir`

type database  
`path-like object`

param str flag  
- `'r'`: Open existing database for reading only.
- `'w'`: Open existing database for reading and writing.
- `'c'` (default): Open database for reading and writing, creating it if it doesn't exist.
- `'n'`: Always create a new, empty database, open for reading and writing.

param int mode  
The Unix file access mode of the file (default: octal `0o666`), used only when the database has to be created.

> [!WARNING]
> It is possible to crash the Python interpreter when loading a database with a sufficiently large/complex entry due to stack depth limitations in Python's AST compiler.

> [!WARNING]
> `!dbm.dumb` does not support concurrent read/write access. (Multiple simultaneous read accesses are safe.) When a program has the database open for writing, no other program should have it open for reading or writing.

<div class="versionchanged">

3.5 `~dbm.dumb.open` always creates a new database when *flag* is `'n'`.

</div>

<div class="versionchanged">

3.8 A database opened read-only if *flag* is `'r'`. A database is not created if it does not exist if *flag* is `'r'` or `'w'`.

</div>

<div class="versionchanged">

3.11 *filename* accepts a `path-like object`.

</div>

The returned database object behaves similar to a mutable `mapping`, but the `!keys` and `!items` methods return lists, and the `!setdefault` method requires two arguments. It also supports a "closing" context manager via the `with` keyword.

The following methods are also provided:

<div class="method">

dumbdbm.close()

Close the database.

</div>

<div class="method">

dumbdbm.reorganize()

If you have carried out a lot of deletions and would like to shrink the space used on disk, this method will reorganize the database; otherwise, deleted file space will not be reused.

> [!NOTE]
> While reorganizing, no additional free disk space is required. However, be aware that this factor changes for each `!dbm` submodule.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

dumbdbm.sync()

Synchronize the on-disk directory and data files. This method is called by the `shelve.Shelf.sync` method.

</div>

</div>

# Debugging and profiling

These libraries help you with Python development: the debugger enables you to step through code, analyze stack frames and set breakpoints etc., and the profilers run code and give you a detailed breakdown of execution times, allowing you to identify bottlenecks in your programs. Auditing events provide visibility into runtime behaviors that would otherwise require intrusive debugging or patching.

<div class="toctree">

audit_events.rst bdb.rst faulthandler.rst pdb.rst profiling.rst pstats.rst timeit.rst trace.rst tracemalloc.rst

</div>

# `!decimal` --- Decimal fixed-point and floating-point arithmetic

<div class="module" data-synopsis="Implementation of the General Decimal Arithmetic Specification.">

decimal

</div>

**Source code:** `Lib/decimal.py`

<div class="testsetup">

- 

import decimal import math from decimal import \* \# make sure each group gets a fresh context setcontext(Context())

</div>

<div class="testcleanup">

- 

\# make sure other tests (outside this file) get a fresh context setcontext(Context())

</div>

------------------------------------------------------------------------

The `!decimal` module provides support for fast correctly rounded decimal floating-point arithmetic. It offers several advantages over the `float` datatype:

- Decimal "is based on a [floating-point model](https://speleotrove.com/decimal/damodel.html#refnumber) which was designed with people in mind, and necessarily has a paramount guiding principle -- computers must provide an arithmetic that works in the same way as the arithmetic that people learn at school." -- excerpt from the decimal arithmetic specification.

- Decimal numbers can be represented exactly. In contrast, numbers like `1.1` and `2.2` do not have exact representations in binary floating point. End users typically would not expect `1.1 + 2.2` to display as `3.3000000000000003` as it does with binary floating point.

- The exactness carries over into arithmetic. In decimal floating point, `0.1 + 0.1 + 0.1 - 0.3` is exactly equal to zero. In binary floating point, the result is `5.5511151231257827e-017`. While near to zero, the differences prevent reliable equality testing and differences can accumulate. For this reason, decimal is preferred in accounting applications which have strict equality invariants.

- The decimal module incorporates a notion of significant places so that `1.30 + 1.20` is `2.50`. The trailing zero is kept to indicate significance. This is the customary presentation for monetary applications. For multiplication, the "schoolbook" approach uses all the figures in the multiplicands. For instance, `1.3 * 1.2` gives `1.56` while `1.30 * 1.20` gives `1.5600`.

- Unlike hardware based binary floating point, the decimal module has a user alterable precision (defaulting to 28 places) which can be as large as needed for a given problem:

  > \>\>\> from decimal import \* \>\>\> getcontext().prec = 6 \>\>\> Decimal(1) / Decimal(7) Decimal('0.142857') \>\>\> getcontext().prec = 28 \>\>\> Decimal(1) / Decimal(7) Decimal('0.1428571428571428571428571429')

- Both binary and decimal floating point are implemented in terms of published standards. While the built-in float type exposes only a modest portion of its capabilities, the decimal module exposes all required parts of the standard. When needed, the programmer has full control over rounding and signal handling. This includes an option to enforce exact arithmetic by using exceptions to block any inexact operations.

- The decimal module was designed to support "without prejudice, both exact unrounded decimal arithmetic (sometimes called fixed-point arithmetic) and rounded floating-point arithmetic." -- excerpt from the decimal arithmetic specification.

The module design is centered around three concepts: the decimal number, the context for arithmetic, and signals.

A decimal number is immutable. It has a sign, coefficient digits, and an exponent. To preserve significance, the coefficient digits do not truncate trailing zeros. Decimals also include special values such as `Infinity`, `-Infinity`, and `NaN`. The standard also differentiates `-0` from `+0`.

The context for arithmetic is an environment specifying precision, rounding rules, limits on exponents, flags indicating the results of operations, and trap enablers which determine whether signals are treated as exceptions. Rounding options include `ROUND_CEILING`, `ROUND_DOWN`, `ROUND_FLOOR`, `ROUND_HALF_DOWN`, `ROUND_HALF_EVEN`, `ROUND_HALF_UP`, `ROUND_UP`, and `ROUND_05UP`.

Signals are groups of exceptional conditions arising during the course of computation. Depending on the needs of the application, signals may be ignored, considered as informational, or treated as exceptions. The signals in the decimal module are: `Clamped`, `InvalidOperation`, `DivisionByZero`, `Inexact`, `Rounded`, `Subnormal`, `Overflow`, `Underflow` and `FloatOperation`.

For each signal there is a flag and a trap enabler. When a signal is encountered, its flag is set to one, then, if the trap enabler is set to one, an exception is raised. Flags are sticky, so the user needs to reset them before monitoring a calculation.

<div class="seealso">

- IBM's General Decimal Arithmetic Specification, [The General Decimal Arithmetic Specification](https://speleotrove.com/decimal/decarith.html).

</div>

## Quick-start tutorial

The usual start to using decimals is importing the module, viewing the current context with `getcontext` and, if necessary, setting new values for precision, rounding, or enabled traps:

    >>> from decimal import *
    >>> getcontext()
    Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999, Emax=999999,
            capitals=1, clamp=0, flags=[], traps=[Overflow, DivisionByZero,
            InvalidOperation])

    >>> getcontext().prec = 7       # Set a new precision

Decimal instances can be constructed from integers, strings, floats, or tuples. Construction from an integer or a float performs an exact conversion of the value of that integer or float. Decimal numbers include special values such as `NaN` which stands for "Not a number", positive and negative `Infinity`, and `-0`:

    >>> getcontext().prec = 28
    >>> Decimal(10)
    Decimal('10')
    >>> Decimal('3.14')
    Decimal('3.14')
    >>> Decimal(3.14)
    Decimal('3.140000000000000124344978758017532527446746826171875')
    >>> Decimal((0, (3, 1, 4), -2))
    Decimal('3.14')
    >>> Decimal(str(2.0 ** 0.5))
    Decimal('1.4142135623730951')
    >>> Decimal(2) ** Decimal('0.5')
    Decimal('1.414213562373095048801688724')
    >>> Decimal('NaN')
    Decimal('NaN')
    >>> Decimal('-Infinity')
    Decimal('-Infinity')

If the `FloatOperation` signal is trapped, accidental mixing of decimals and floats in constructors or ordering comparisons raises an exception:

    >>> c = getcontext()
    >>> c.traps[FloatOperation] = True
    >>> Decimal(3.14)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    decimal.FloatOperation: [<class 'decimal.FloatOperation'>]
    >>> Decimal('3.5') < 3.7
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    decimal.FloatOperation: [<class 'decimal.FloatOperation'>]
    >>> Decimal('3.5') == 3.5
    True

<div class="versionadded">

3.3

</div>

The significance of a new Decimal is determined solely by the number of digits input. Context precision and rounding only come into play during arithmetic operations.

<div class="doctest">

newcontext

\>\>\> getcontext().prec = 6 \>\>\> Decimal('3.0') Decimal('3.0') \>\>\> Decimal('3.1415926535') Decimal('3.1415926535') \>\>\> Decimal('3.1415926535') + Decimal('2.7182818285') Decimal('5.85987') \>\>\> getcontext().rounding = ROUND_UP \>\>\> Decimal('3.1415926535') + Decimal('2.7182818285') Decimal('5.85988')

</div>

If the internal limits of the C version are exceeded, constructing a decimal raises `InvalidOperation`:

    >>> Decimal("1e9999999999999999999")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]

<div class="versionchanged">

3.3

</div>

Decimals interact well with much of the rest of Python. Here is a small decimal floating-point flying circus:

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> data = list(map(Decimal, '1.34 1.87 3.45 2.35 1.00 0.03 9.25'.split())) \>\>\> max(data) Decimal('9.25') \>\>\> min(data) Decimal('0.03') \>\>\> sorted(data) \[Decimal('0.03'), Decimal('1.00'), Decimal('1.34'), Decimal('1.87'), Decimal('2.35'), Decimal('3.45'), Decimal('9.25')\] \>\>\> sum(data) Decimal('19.29') \>\>\> a,b,c = data\[:3\] \>\>\> str(a) '1.34' \>\>\> float(a) 1.34 \>\>\> round(a, 1) Decimal('1.3') \>\>\> int(a) 1 \>\>\> a \* 5 Decimal('6.70') \>\>\> a \* b Decimal('2.5058') \>\>\> c % a Decimal('0.77')

</div>

Decimals can be formatted (with `format` built-in or `f-strings`) in fixed-point or scientific notation, using the same formatting syntax (see `formatspec`) as builtin `float` type:

<div class="doctest">

\>\>\> format(Decimal('2.675'), "f") '2.675' \>\>\> format(Decimal('2.675'), ".2f") '2.68' \>\>\> f"{Decimal('2.675'):.2f}" '2.68' \>\>\> format(Decimal('2.675'), ".2e") '2.68e+0' \>\>\> with localcontext() as ctx: ... ctx.rounding = ROUND_DOWN ... print(format(Decimal('2.675'), ".2f")) ... 2.67

</div>

And some mathematical functions are also available to Decimal:

> \>\>\> getcontext().prec = 28 \>\>\> Decimal(2).sqrt() Decimal('1.414213562373095048801688724') \>\>\> Decimal(1).exp() Decimal('2.718281828459045235360287471') \>\>\> Decimal('10').ln() Decimal('2.302585092994045684017991455') \>\>\> Decimal('10').log10() Decimal('1')

The `~Decimal.quantize` method rounds a number to a fixed exponent. This method is useful for monetary applications that often round results to a fixed number of places:

> \>\>\> Decimal('7.325').quantize(Decimal('.01'), rounding=ROUND_DOWN) Decimal('7.32') \>\>\> Decimal('7.325').quantize(Decimal('1.'), rounding=ROUND_UP) Decimal('8')

As shown above, the `getcontext` function accesses the current context and allows the settings to be changed. This approach meets the needs of most applications.

For more advanced work, it may be useful to create alternate contexts using the `Context` constructor. To make an alternate active, use the `setcontext` function.

In accordance with the standard, the `!decimal` module provides two ready to use standard contexts, `BasicContext` and `ExtendedContext`. The former is especially useful for debugging because many of the traps are enabled:

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

newcontext

\>\>\> myothercontext = Context(prec=60, rounding=ROUND_HALF_DOWN) \>\>\> setcontext(myothercontext) \>\>\> Decimal(1) / Decimal(7) Decimal('0.142857142857142857142857142857142857142857142857142857142857')

\>\>\> ExtendedContext Context(prec=9, rounding=ROUND_HALF_EVEN, Emin=-999999, Emax=999999, capitals=1, clamp=0, flags=\[\], traps=\[\]) \>\>\> setcontext(ExtendedContext) \>\>\> Decimal(1) / Decimal(7) Decimal('0.142857143') \>\>\> Decimal(42) / Decimal(0) Decimal('Infinity')

\>\>\> setcontext(BasicContext) \>\>\> Decimal(42) / Decimal(0) Traceback (most recent call last): File "\<pyshell#143\>", line 1, in -toplevel- Decimal(42) / Decimal(0) DivisionByZero: x / 0

</div>

Contexts also have signal flags for monitoring exceptional conditions encountered during computations. The flags remain set until explicitly cleared, so it is best to clear the flags before each set of monitored computations by using the `~Context.clear_flags` method. :

    >>> setcontext(ExtendedContext)
    >>> getcontext().clear_flags()
    >>> Decimal(355) / Decimal(113)
    Decimal('3.14159292')
    >>> getcontext()
    Context(prec=9, rounding=ROUND_HALF_EVEN, Emin=-999999, Emax=999999,
            capitals=1, clamp=0, flags=[Inexact, Rounded], traps=[])

The *flags* entry shows that the rational approximation to pi was rounded (digits beyond the context precision were thrown away) and that the result is inexact (some of the discarded digits were non-zero).

Individual traps are set using the dictionary in the `~Context.traps` attribute of a context:

<div class="doctest">

newcontext

\>\>\> setcontext(ExtendedContext) \>\>\> Decimal(1) / Decimal(0) Decimal('Infinity') \>\>\> getcontext().traps\[DivisionByZero\] = 1 \>\>\> Decimal(1) / Decimal(0) Traceback (most recent call last): File "\<pyshell#112\>", line 1, in -toplevel- Decimal(1) / Decimal(0) DivisionByZero: x / 0

</div>

Most programs adjust the current context only once, at the beginning of the program. And, in many applications, data is converted to `Decimal` with a single cast inside a loop. With context set and decimals created, the bulk of the program manipulates the data no differently than with other Python numeric types.

## Decimal objects

<div class="Decimal(value=&quot;0&quot;, context=None)">

Construct a new `Decimal` object based from *value*.

*value* can be an integer, string, tuple, `float`, or another `Decimal` object. If no *value* is given, returns `Decimal('0')`. If *value* is a string, it should conform to the decimal numeric string syntax after leading and trailing whitespace characters, as well as underscores throughout, are removed:

    sign           ::=  '+' | '-'
    digit          ::=  '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9'
    indicator      ::=  'e' | 'E'
    digits         ::=  digit [digit]...
    decimal-part   ::=  digits '.' [digits] | ['.'] digits
    exponent-part  ::=  indicator [sign] digits
    infinity       ::=  'Infinity' | 'Inf'
    nan            ::=  'NaN' [digits] | 'sNaN' [digits]
    numeric-value  ::=  decimal-part [exponent-part] | infinity
    numeric-string ::=  [sign] numeric-value | [sign] nan

Other Unicode decimal digits are also permitted where `digit` appears above. These include decimal digits from various other alphabets (for example, Arabic-Indic and Devanāgarī digits) along with the fullwidth digits `'\uff10'` through `'\uff19'`. Case is not significant, so, for example, `inf`, `Inf`, `INFINITY`, and `iNfINity` are all acceptable spellings for positive infinity.

If *value* is a `tuple`, it should have three components, a sign (`0` for positive or `1` for negative), a `tuple` of digits, and an integer exponent. For example, `Decimal((0, (1, 4, 1, 4), -3))` returns `Decimal('1.414')`.

If *value* is a `float`, the binary floating-point value is losslessly converted to its exact decimal equivalent. This conversion can often require 53 or more digits of precision. For example, `Decimal(float('1.1'))` converts to `Decimal('1.100000000000000088817841970012523233890533447265625')`.

The *context* precision does not affect how many digits are stored. That is determined exclusively by the number of digits in *value*. For example, `Decimal('3.00000')` records all five zeros even if the context precision is only three.

The purpose of the *context* argument is determining what to do if *value* is a malformed string. If the context traps `InvalidOperation`, an exception is raised; otherwise, the constructor returns a new Decimal with the value of `NaN`.

Once constructed, `Decimal` objects are immutable.

<div class="versionchanged">

3.2 The argument to the constructor is now permitted to be a `float` instance.

</div>

<div class="versionchanged">

3.3 `float` arguments raise an exception if the `FloatOperation` trap is set. By default the trap is off.

</div>

<div class="versionchanged">

3.6 Underscores are allowed for grouping, as with integral and floating-point literals in code.

</div>

Decimal floating-point objects share many properties with the other built-in numeric types such as `float` and `int`. All of the usual math operations and special methods apply. Likewise, decimal objects can be copied, pickled, printed, used as dictionary keys, used as set elements, compared, sorted, and coerced to another type (such as `float` or `int`).

There are some small differences between arithmetic on Decimal objects and arithmetic on integers and floats. When the remainder operator `%` is applied to Decimal objects, the sign of the result is the sign of the *dividend* rather than the sign of the divisor:

    >>> (-7) % 4
    1
    >>> Decimal(-7) % Decimal(4)
    Decimal('-3')

The integer division operator `//` behaves analogously, returning the integer part of the true quotient (truncating towards zero) rather than its floor, so as to preserve the usual identity `x == (x // y) * y + x % y`:

    >>> -7 // 4
    -2
    >>> Decimal(-7) // Decimal(4)
    Decimal('-1')

The `%` and `//` operators implement the `remainder` and `divide-integer` operations (respectively) as described in the specification.

Decimal objects cannot generally be combined with floats or instances of `fractions.Fraction` in arithmetic operations: an attempt to add a `Decimal` to a `float`, for example, will raise a `TypeError`. However, it is possible to use Python's comparison operators to compare a `Decimal` instance `x` with another number `y`. This avoids confusing results when doing equality comparisons between numbers of different types.

<div class="versionchanged">

3.2 Mixed-type comparisons between `Decimal` instances and other numeric types are now fully supported.

</div>

In addition to the standard numeric properties, decimal floating-point objects also have a number of specialized methods:

<div class="method">

adjusted()

Return the adjusted exponent after shifting out the coefficient's rightmost digits until only the lead digit remains: `Decimal('321e+5').adjusted()` returns seven. Used for determining the position of the most significant digit with respect to the decimal point.

</div>

<div class="method">

as_integer_ratio()

Return a pair `(n, d)` of integers that represent the given `Decimal` instance as a fraction, in lowest terms and with a positive denominator:

    >>> Decimal('-3.14').as_integer_ratio()
    (-157, 50)

The conversion is exact. Raise OverflowError on infinities and ValueError on NaNs.

</div>

<div class="versionadded">

3.6

</div>

<div class="method">

as_tuple()

Return a `named tuple` representation of the number: `DecimalTuple(sign, digits, exponent)`.

</div>

<div class="method">

canonical()

Return the canonical encoding of the argument. Currently, the encoding of a `Decimal` instance is always canonical, so this operation returns its argument unchanged.

</div>

<div class="method">

compare(other, context=None)

Compare the values of two Decimal instances. `compare` returns a Decimal instance, and if either operand is a NaN then the result is a NaN:

    a or b is a NaN  ==> Decimal('NaN')
    a < b            ==> Decimal('-1')
    a == b           ==> Decimal('0')
    a > b            ==> Decimal('1')

</div>

<div class="method">

compare_signal(other, context=None)

This operation is identical to the `compare` method, except that all NaNs signal. That is, if neither operand is a signaling NaN then any quiet NaN operand is treated as though it were a signaling NaN.

</div>

<div class="method">

compare_total(other, context=None)

Compare two operands using their abstract representation rather than their numerical value. Similar to the `compare` method, but the result gives a total ordering on `Decimal` instances. Two `Decimal` instances with the same numeric value but different representations compare unequal in this ordering:

> \>\>\> Decimal('12.0').compare_total(Decimal('12')) Decimal('-1')

Quiet and signaling NaNs are also included in the total ordering. The result of this function is `Decimal('0')` if both operands have the same representation, `Decimal('-1')` if the first operand is lower in the total order than the second, and `Decimal('1')` if the first operand is higher in the total order than the second operand. See the specification for details of the total order.

This operation is unaffected by context and is quiet: no flags are changed and no rounding is performed. As an exception, the C version may raise InvalidOperation if the second operand cannot be converted exactly.

</div>

<div class="method">

compare_total_mag(other, context=None)

Compare two operands using their abstract representation rather than their value as in `compare_total`, but ignoring the sign of each operand. `x.compare_total_mag(y)` is equivalent to `x.copy_abs().compare_total(y.copy_abs())`.

This operation is unaffected by context and is quiet: no flags are changed and no rounding is performed. As an exception, the C version may raise InvalidOperation if the second operand cannot be converted exactly.

</div>

<div class="method">

conjugate()

Just returns self, this method is only to comply with the Decimal Specification.

</div>

<div class="method">

copy_abs()

Return the absolute value of the argument. This operation is unaffected by the context and is quiet: no flags are changed and no rounding is performed.

</div>

<div class="method">

copy_negate()

Return the negation of the argument. This operation is unaffected by the context and is quiet: no flags are changed and no rounding is performed.

</div>

<div class="method">

copy_sign(other, context=None)

Return a copy of the first operand with the sign set to be the same as the sign of the second operand. For example:

> \>\>\> Decimal('2.3').copy_sign(Decimal('-1.5')) Decimal('-2.3')

This operation is unaffected by context and is quiet: no flags are changed and no rounding is performed. As an exception, the C version may raise InvalidOperation if the second operand cannot be converted exactly.

</div>

<div class="method">

exp(context=None)

Return the value of the (natural) exponential function `e**x` at the given number. The result is correctly rounded using the `ROUND_HALF_EVEN` rounding mode.

\>\>\> Decimal(1).exp() Decimal('2.718281828459045235360287471') \>\>\> Decimal(321).exp() Decimal('2.561702493119680037517373933E+139')

</div>

<div class="classmethod">

from_float(f, /)

Alternative constructor that only accepts instances of `float` or `int`.

Note `Decimal.from_float(0.1)` is not the same as `Decimal('0.1')`. Since 0.1 is not exactly representable in binary floating point, the value is stored as the nearest representable value which is `0x1.999999999999ap-4`. That equivalent value in decimal is `0.1000000000000000055511151231257827021181583404541015625`.

> [!NOTE]
> From Python 3.2 onwards, a `Decimal` instance can also be constructed directly from a `float`.

<div class="doctest">

\>\>\> Decimal.from_float(0.1) Decimal('0.1000000000000000055511151231257827021181583404541015625') \>\>\> Decimal.from_float(float('nan')) Decimal('NaN') \>\>\> Decimal.from_float(float('inf')) Decimal('Infinity') \>\>\> Decimal.from_float(float('-inf')) Decimal('-Infinity')

</div>

<div class="versionadded">

3.1

</div>

</div>

<div class="classmethod">

from_number(number, /)

Alternative constructor that only accepts instances of `float`, `int` or `Decimal`, but not strings or tuples.

<div class="doctest">

\>\>\> Decimal.from_number(314) Decimal('314') \>\>\> Decimal.from_number(0.1) Decimal('0.1000000000000000055511151231257827021181583404541015625') \>\>\> Decimal.from_number(Decimal('3.14')) Decimal('3.14')

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

fma(other, third, context=None)

Fused multiply-add. Return self\*other+third with no rounding of the intermediate product self\*other.

\>\>\> Decimal(2).fma(3, 5) Decimal('11')

</div>

<div class="method">

is_canonical()

Return `True` if the argument is canonical and `False` otherwise. Currently, a `Decimal` instance is always canonical, so this operation always returns `True`.

</div>

<div class="method">

is_finite()

Return `True` if the argument is a finite number, and `False` if the argument is an infinity or a NaN.

</div>

<div class="method">

is_infinite()

Return `True` if the argument is either positive or negative infinity and `False` otherwise.

</div>

<div class="method">

is_nan()

Return `True` if the argument is a (quiet or signaling) NaN and `False` otherwise.

</div>

<div class="method">

is_normal(context=None)

Return `True` if the argument is a *normal* finite number. Return `False` if the argument is zero, subnormal, infinite or a NaN.

</div>

<div class="method">

is_qnan()

Return `True` if the argument is a quiet NaN, and `False` otherwise.

</div>

<div class="method">

is_signed()

Return `True` if the argument has a negative sign and `False` otherwise. Note that zeros and NaNs can both carry signs.

</div>

<div class="method">

is_snan()

Return `True` if the argument is a signaling NaN and `False` otherwise.

</div>

<div class="method">

is_subnormal(context=None)

Return `True` if the argument is subnormal, and `False` otherwise.

</div>

<div class="method">

is_zero()

Return `True` if the argument is a (positive or negative) zero and `False` otherwise.

</div>

<div class="method">

ln(context=None)

Return the natural (base e) logarithm of the operand. The result is correctly rounded using the `ROUND_HALF_EVEN` rounding mode.

</div>

<div class="method">

log10(context=None)

Return the base ten logarithm of the operand. The result is correctly rounded using the `ROUND_HALF_EVEN` rounding mode.

</div>

<div class="method">

logb(context=None)

For a nonzero number, return the adjusted exponent of its operand as a `Decimal` instance. If the operand is a zero then `Decimal('-Infinity')` is returned and the `DivisionByZero` flag is raised. If the operand is an infinity then `Decimal('Infinity')` is returned.

</div>

<div class="method">

logical_and(other, context=None)

`logical_and` is a logical operation which takes two *logical operands* (see `logical_operands_label`). The result is the digit-wise `and` of the two operands.

</div>

<div class="method">

logical_invert(context=None)

`logical_invert` is a logical operation. The result is the digit-wise inversion of the operand.

</div>

<div class="method">

logical_or(other, context=None)

`logical_or` is a logical operation which takes two *logical operands* (see `logical_operands_label`). The result is the digit-wise `or` of the two operands.

</div>

<div class="method">

logical_xor(other, context=None)

`logical_xor` is a logical operation which takes two *logical operands* (see `logical_operands_label`). The result is the digit-wise exclusive or of the two operands.

</div>

<div class="method">

max(other, context=None)

Like `max(self, other)` except that the context rounding rule is applied before returning and that `NaN` values are either signaled or ignored (depending on the context and whether they are signaling or quiet).

</div>

<div class="method">

max_mag(other, context=None)

Similar to the `.max` method, but the comparison is done using the absolute values of the operands.

</div>

<div class="method">

min(other, context=None)

Like `min(self, other)` except that the context rounding rule is applied before returning and that `NaN` values are either signaled or ignored (depending on the context and whether they are signaling or quiet).

</div>

<div class="method">

min_mag(other, context=None)

Similar to the `.min` method, but the comparison is done using the absolute values of the operands.

</div>

<div class="method">

next_minus(context=None)

Return the largest number representable in the given context (or in the current thread's context if no context is given) that is smaller than the given operand.

</div>

<div class="method">

next_plus(context=None)

Return the smallest number representable in the given context (or in the current thread's context if no context is given) that is larger than the given operand.

</div>

<div class="method">

next_toward(other, context=None)

If the two operands are unequal, return the number closest to the first operand in the direction of the second operand. If both operands are numerically equal, return a copy of the first operand with the sign set to be the same as the sign of the second operand.

</div>

<div class="method">

normalize(context=None)

Used for producing canonical values of an equivalence class within either the current context or the specified context.

This has the same semantics as the unary plus operation, except that if the final result is finite it is reduced to its simplest form, with all trailing zeros removed and its sign preserved. That is, while the coefficient is non-zero and a multiple of ten the coefficient is divided by ten and the exponent is incremented by 1. Otherwise (the coefficient is zero) the exponent is set to 0. In all cases the sign is unchanged.

For example, `Decimal('32.100')` and `Decimal('0.321000e+2')` both normalize to the equivalent value `Decimal('32.1')`.

Note that rounding is applied *before* reducing to simplest form.

In the latest versions of the specification, this operation is also known as `reduce`.

</div>

<div class="method">

number_class(context=None)

Return a string describing the *class* of the operand. The returned value is one of the following ten strings.

- `"-Infinity"`, indicating that the operand is negative infinity.
- `"-Normal"`, indicating that the operand is a negative normal number.
- `"-Subnormal"`, indicating that the operand is negative and subnormal.
- `"-Zero"`, indicating that the operand is a negative zero.
- `"+Zero"`, indicating that the operand is a positive zero.
- `"+Subnormal"`, indicating that the operand is positive and subnormal.
- `"+Normal"`, indicating that the operand is a positive normal number.
- `"+Infinity"`, indicating that the operand is positive infinity.
- `"NaN"`, indicating that the operand is a quiet NaN (Not a Number).
- `"sNaN"`, indicating that the operand is a signaling NaN.

</div>

<div class="method">

quantize(exp, rounding=None, context=None)

Return a value equal to the first operand after rounding and having the exponent of the second operand.

\>\>\> Decimal('1.41421356').quantize(Decimal('1.000')) Decimal('1.414')

Unlike other operations, if the length of the coefficient after the quantize operation would be greater than precision, then an `InvalidOperation` is signaled. This guarantees that, unless there is an error condition, the quantized exponent is always equal to that of the right-hand operand.

Also unlike other operations, quantize never signals Underflow, even if the result is subnormal and inexact.

If the exponent of the second operand is larger than that of the first then rounding may be necessary. In this case, the rounding mode is determined by the `rounding` argument if given, else by the given `context` argument; if neither argument is given the rounding mode of the current thread's context is used.

An error is returned whenever the resulting exponent is greater than `~Context.Emax` or less than `~Context.Etiny`.

</div>

<div class="method">

radix()

Return `Decimal(10)`, the radix (base) in which the `Decimal` class does all its arithmetic. Included for compatibility with the specification.

</div>

<div class="method">

remainder_near(other, context=None)

Return the remainder from dividing *self* by *other*. This differs from `self % other` in that the sign of the remainder is chosen so as to minimize its absolute value. More precisely, the return value is `self - n * other` where `n` is the integer nearest to the exact value of `self / other`, and if two integers are equally near then the even one is chosen.

If the result is zero then its sign will be the sign of *self*.

\>\>\> Decimal(18).remainder_near(Decimal(10)) Decimal('-2') \>\>\> Decimal(25).remainder_near(Decimal(10)) Decimal('5') \>\>\> Decimal(35).remainder_near(Decimal(10)) Decimal('-5')

</div>

<div class="method">

rotate(other, context=None)

Return the result of rotating the digits of the first operand by an amount specified by the second operand. The second operand must be an integer in the range -precision through precision. The absolute value of the second operand gives the number of places to rotate. If the second operand is positive then rotation is to the left; otherwise rotation is to the right. The coefficient of the first operand is padded on the left with zeros to length precision if necessary. The sign and exponent of the first operand are unchanged.

</div>

<div class="method">

same_quantum(other, context=None)

Test whether self and other have the same exponent or whether both are `NaN`.

This operation is unaffected by context and is quiet: no flags are changed and no rounding is performed. As an exception, the C version may raise InvalidOperation if the second operand cannot be converted exactly.

</div>

<div class="method">

scaleb(other, context=None)

Return the first operand with exponent adjusted by the second. Equivalently, return the first operand multiplied by `10**other`. The second operand must be an integer.

</div>

<div class="method">

shift(other, context=None)

Return the result of shifting the digits of the first operand by an amount specified by the second operand. The second operand must be an integer in the range -precision through precision. The absolute value of the second operand gives the number of places to shift. If the second operand is positive then the shift is to the left; otherwise the shift is to the right. Digits shifted into the coefficient are zeros. The sign and exponent of the first operand are unchanged.

</div>

<div class="method">

sqrt(context=None)

Return the square root of the argument to full precision.

</div>

<div class="method">

to_eng_string(context=None)

Convert to a string, using engineering notation if an exponent is needed.

Engineering notation has an exponent which is a multiple of 3. This can leave up to 3 digits to the left of the decimal place and may require the addition of either one or two trailing zeros.

For example, this converts `Decimal('123E+1')` to `Decimal('1.23E+3')`.

</div>

<div class="method">

to_integral(rounding=None, context=None)

Identical to the `to_integral_value` method. The `to_integral` name has been kept for compatibility with older versions.

</div>

<div class="method">

to_integral_exact(rounding=None, context=None)

Round to the nearest integer, signaling `Inexact` or `Rounded` as appropriate if rounding occurs. The rounding mode is determined by the `rounding` parameter if given, else by the given `context`. If neither parameter is given then the rounding mode of the current context is used.

</div>

<div class="method">

to_integral_value(rounding=None, context=None)

Round to the nearest integer without signaling `Inexact` or `Rounded`. If given, applies *rounding*; otherwise, uses the rounding method in either the supplied *context* or the current context.

</div>

Decimal numbers can be rounded using the `.round` function:

<div class="describe">

round(number)

</div>

<div class="describe">

round(number, ndigits)

If *ndigits* is not given or `None`, returns the nearest `int` to *number*, rounding ties to even, and ignoring the rounding mode of the `Decimal` context. Raises `OverflowError` if *number* is an infinity or `ValueError` if it is a (quiet or signaling) NaN.

If *ndigits* is an `int`, the context's rounding mode is respected and a `Decimal` representing *number* rounded to the nearest multiple of `Decimal('1E-ndigits')` is returned; in this case, `round(number, ndigits)` is equivalent to `self.quantize(Decimal('1E-ndigits'))`. Returns `Decimal('NaN')` if *number* is a quiet NaN. Raises `InvalidOperation` if *number* is an infinity, a signaling NaN, or if the length of the coefficient after the quantize operation would be greater than the current context's precision. In other words, for the non-corner cases:

- if *ndigits* is positive, return *number* rounded to *ndigits* decimal places;
- if *ndigits* is zero, return *number* rounded to the nearest integer;
- if *ndigits* is negative, return *number* rounded to the nearest multiple of `10**abs(ndigits)`.

For example:

    >>> from decimal import Decimal, getcontext, ROUND_DOWN
    >>> getcontext().rounding = ROUND_DOWN
    >>> round(Decimal('3.75'))     # context rounding ignored
    4
    >>> round(Decimal('3.5'))      # round-ties-to-even
    4
    >>> round(Decimal('3.75'), 0)  # uses the context rounding
    Decimal('3')
    >>> round(Decimal('3.75'), 1)
    Decimal('3.7')
    >>> round(Decimal('3.75'), -1)
    Decimal('0E+1')

</div>

</div>

### Logical operands

The `~Decimal.logical_and`, `~Decimal.logical_invert`, `~Decimal.logical_or`, and `~Decimal.logical_xor` methods expect their arguments to be *logical operands*. A *logical operand* is a `Decimal` instance whose exponent and sign are both zero, and whose digits are all either `0` or `1`.

## Context objects

Contexts are environments for arithmetic operations. They govern precision, set rules for rounding, determine which signals are treated as exceptions, and limit the range for exponents.

Each thread has its own current context which is accessed or changed using the `getcontext` and `setcontext` functions:

<div class="function">

getcontext()

Return the current context for the active thread.

</div>

<div class="function">

setcontext(c, /)

Set the current context for the active thread to *c*.

</div>

You can also use the `with` statement and the `localcontext` function to temporarily change the active context.

<div class="function">

localcontext(ctx=None, \*\*kwargs)

Return a context manager that will set the current context for the active thread to a copy of *ctx* on entry to the with-statement and restore the previous context when exiting the with-statement. If no context is specified, a copy of the current context is used. The *kwargs* argument is used to set the attributes of the new context.

For example, the following code sets the current decimal precision to 42 places, performs a calculation, and then automatically restores the previous context:

    from decimal import localcontext

    with localcontext() as ctx:
        ctx.prec = 42   # Perform a high precision calculation
        s = calculate_something()
    s = +s  # Round the final result back to the default precision

Using keyword arguments, the code would be the following:

    from decimal import localcontext

    with localcontext(prec=42) as ctx:
        s = calculate_something()
    s = +s

Raises `TypeError` if *kwargs* supplies an attribute that `Context` doesn't support. Raises either `TypeError` or `ValueError` if *kwargs* supplies an invalid value for an attribute.

<div class="versionchanged">

3.11 `localcontext` now supports setting context attributes through the use of keyword arguments.

</div>

</div>

<div class="function">

IEEEContext(bits)

Return a context object initialized to the proper values for one of the IEEE interchange formats. The argument must be a multiple of 32 and less than `IEEE_CONTEXT_MAX_BITS`.

<div class="versionadded">

3.14

</div>

</div>

New contexts can also be created using the `Context` constructor described below. In addition, the module provides three pre-made contexts:

<div class="data">

BasicContext

This is a standard context defined by the General Decimal Arithmetic Specification. Precision is set to nine. Rounding is set to `ROUND_HALF_UP`. All flags are cleared. All traps are enabled (treated as exceptions) except `Inexact`, `Rounded`, and `Subnormal`.

Because many of the traps are enabled, this context is useful for debugging.

</div>

<div class="data">

ExtendedContext

This is a standard context defined by the General Decimal Arithmetic Specification. Precision is set to nine. Rounding is set to `ROUND_HALF_EVEN`. All flags are cleared. No traps are enabled (so that exceptions are not raised during computations).

Because the traps are disabled, this context is useful for applications that prefer to have result value of `NaN` or `Infinity` instead of raising exceptions. This allows an application to complete a run in the presence of conditions that would otherwise halt the program.

</div>

<div class="data">

DefaultContext

This context is used by the `Context` constructor as a prototype for new contexts. Changing a field (such a precision) has the effect of changing the default for new contexts created by the `Context` constructor.

This context is most useful in multi-threaded environments. Changing one of the fields before threads are started has the effect of setting system-wide defaults. Changing the fields after threads have started is not recommended as it would require thread synchronization to prevent race conditions.

In single threaded environments, it is preferable to not use this context at all. Instead, simply create contexts explicitly as described below.

The default values are `Context.prec`=`28`, `Context.rounding`=`ROUND_HALF_EVEN`, and enabled traps for `Overflow`, `InvalidOperation`, and `DivisionByZero`.

</div>

In addition to the three supplied contexts, new contexts can be created with the `Context` constructor.

<div class="Context(prec=None, rounding=None, Emin=None, Emax=None, capitals=None, clamp=None, flags=None, traps=None)">

Creates a new context. If a field is not specified or is `None`, the default values are copied from the `DefaultContext`. If the *flags* field is not specified or is `None`, all flags are cleared.

<div class="attribute">

prec

An integer in the range \[`1`, `MAX_PREC`\] that sets the precision for arithmetic operations in the context.

</div>

<div class="attribute">

rounding

One of the constants listed in the section [Rounding Modes](#rounding-modes).

</div>

<div class="attribute">

traps flags

Lists of any signals to be set. Generally, new contexts should only set traps and leave the flags clear.

</div>

<div class="attribute">

Emin Emax

Integers specifying the outer limits allowable for exponents. *Emin* must be in the range \[`MIN_EMIN`, `0`\], *Emax* in the range \[`0`, `MAX_EMAX`\].

</div>

<div class="attribute">

capitals

Either `0` or `1` (the default). If set to `1`, exponents are printed with a capital `E`; otherwise, a lowercase `e` is used: `Decimal('6.02e+23')`.

</div>

<div class="attribute">

clamp

Either `0` (the default) or `1`. If set to `1`, the exponent `e` of a `Decimal` instance representable in this context is strictly limited to the range `Emin - prec + 1 <= e <= Emax - prec + 1`. If *clamp* is `0` then a weaker condition holds: the adjusted exponent of the `Decimal` instance is at most `~Context.Emax`. When *clamp* is `1`, a large normal number will, where possible, have its exponent reduced and a corresponding number of zeros added to its coefficient, in order to fit the exponent constraints; this preserves the value of the number but loses information about significant trailing zeros. For example:

    >>> Context(prec=6, Emax=999, clamp=1).create_decimal('1.23e999')
    Decimal('1.23000E+999')

A *clamp* value of `1` allows compatibility with the fixed-width decimal interchange formats specified in IEEE 754.

</div>

The `Context` class defines several general purpose methods as well as a large number of methods for doing arithmetic directly in a given context. In addition, for each of the `Decimal` methods described above (with the exception of the `~Decimal.adjusted` and `~Decimal.as_tuple` methods) there is a corresponding `Context` method. For example, for a `Context` instance `C` and `Decimal` instance `x`, `C.exp(x)` is equivalent to `x.exp(context=C)`. Each `Context` method accepts a Python integer (an instance of `int`) anywhere that a Decimal instance is accepted.

<div class="method">

clear_flags()

Resets all of the flags to `0`.

</div>

<div class="method">

clear_traps()

Resets all of the traps to `0`.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

copy()

Return a duplicate of the context.

`!Context` objects also support `copy.replace`, which returns a duplicate with the specified fields replaced. Fields which are not specified keep the values they have in the original context.

<div class="versionchanged">

3.16 Added support for `copy.replace`.

</div>

</div>

<div class="method">

copy_decimal(num, /)

Return a copy of the Decimal instance num.

</div>

<div class="method">

create_decimal(num='0', /)

Creates a new Decimal instance from *num* but using *self* as context. Unlike the `Decimal` constructor, the context precision, rounding method, flags, and traps are applied to the conversion.

This is useful because constants are often given to a greater precision than is needed by the application. Another benefit is that rounding immediately eliminates unintended effects from digits beyond the current precision. In the following example, using unrounded inputs means that adding zero to a sum can change the result:

<div class="doctest">

newcontext

\>\>\> getcontext().prec = 3 \>\>\> Decimal('3.4445') + Decimal('1.0023') Decimal('4.45') \>\>\> Decimal('3.4445') + Decimal(0) + Decimal('1.0023') Decimal('4.44')

</div>

This method implements the to-number operation of the IBM specification. If the argument is a string, no leading or trailing whitespace or underscores are permitted.

</div>

<div class="method">

create_decimal_from_float(f, /)

Creates a new Decimal instance from a float *f* but rounding using *self* as the context. Unlike the `Decimal.from_float` class method, the context precision, rounding method, flags, and traps are applied to the conversion.

<div class="doctest">

\>\>\> context = Context(prec=5, rounding=ROUND_DOWN) \>\>\> context.create_decimal_from_float(math.pi) Decimal('3.1415') \>\>\> context = Context(prec=5, traps=\[Inexact\]) \>\>\> context.create_decimal_from_float(math.pi) Traceback (most recent call last): ... decimal.Inexact: None

</div>

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

Etiny()

Returns a value equal to `Emin - prec + 1` which is the minimum exponent value for subnormal results. When underflow occurs, the exponent is set to `Etiny`.

</div>

<div class="method">

Etop()

Returns a value equal to `Emax - prec + 1`.

</div>

The usual approach to working with decimals is to create `Decimal` instances and then apply arithmetic operations which take place within the current context for the active thread. An alternative approach is to use context methods for calculating within a specific context. The methods are similar to those for the `Decimal` class and are only briefly recounted here.

<div class="method">

abs(x, /)

Returns the absolute value of *x*.

</div>

<div class="method">

add(x, y, /)

Return the sum of *x* and *y*.

</div>

<div class="method">

canonical(x, /)

Returns the same Decimal object *x*.

</div>

<div class="method">

compare(x, y, /)

Compares *x* and *y* numerically.

</div>

<div class="method">

compare_signal(x, y, /)

Compares the values of the two operands numerically.

</div>

<div class="method">

compare_total(x, y, /)

Compares two operands using their abstract representation.

</div>

<div class="method">

compare_total_mag(x, y, /)

Compares two operands using their abstract representation, ignoring sign.

</div>

<div class="method">

copy_abs(x, /)

Returns a copy of *x* with the sign set to 0.

</div>

<div class="method">

copy_negate(x, /)

Returns a copy of *x* with the sign inverted.

</div>

<div class="method">

copy_sign(x, y, /)

Copies the sign from *y* to *x*.

</div>

<div class="method">

divide(x, y, /)

Return *x* divided by *y*.

</div>

<div class="method">

divide_int(x, y, /)

Return *x* divided by *y*, truncated to an integer.

</div>

<div class="method">

divmod(x, y, /)

Divides two numbers and returns the integer part of the result.

</div>

<div class="method">

exp(x, /)

Returns `e ** x`.

</div>

<div class="method">

fma(x, y, z, /)

Returns *x* multiplied by *y*, plus *z*.

</div>

<div class="method">

is_canonical(x, /)

Returns `True` if *x* is canonical; otherwise returns `False`.

</div>

<div class="method">

is_finite(x, /)

Returns `True` if *x* is finite; otherwise returns `False`.

</div>

<div class="method">

is_infinite(x, /)

Returns `True` if *x* is infinite; otherwise returns `False`.

</div>

<div class="method">

is_nan(x, /)

Returns `True` if *x* is a qNaN or sNaN; otherwise returns `False`.

</div>

<div class="method">

is_normal(x, /)

Returns `True` if *x* is a normal number; otherwise returns `False`.

</div>

<div class="method">

is_qnan(x, /)

Returns `True` if *x* is a quiet NaN; otherwise returns `False`.

</div>

<div class="method">

is_signed(x, /)

Returns `True` if *x* is negative; otherwise returns `False`.

</div>

<div class="method">

is_snan(x, /)

Returns `True` if *x* is a signaling NaN; otherwise returns `False`.

</div>

<div class="method">

is_subnormal(x, /)

Returns `True` if *x* is subnormal; otherwise returns `False`.

</div>

<div class="method">

is_zero(x, /)

Returns `True` if *x* is a zero; otherwise returns `False`.

</div>

<div class="method">

ln(x, /)

Returns the natural (base e) logarithm of *x*.

</div>

<div class="method">

log10(x, /)

Returns the base 10 logarithm of *x*.

</div>

<div class="method">

logb(x, /)

Returns the exponent of the magnitude of the operand's MSD.

</div>

<div class="method">

logical_and(x, y, /)

Applies the logical operation *and* between each operand's digits.

</div>

<div class="method">

logical_invert(x, /)

Invert all the digits in *x*.

</div>

<div class="method">

logical_or(x, y, /)

Applies the logical operation *or* between each operand's digits.

</div>

<div class="method">

logical_xor(x, y, /)

Applies the logical operation *xor* between each operand's digits.

</div>

<div class="method">

max(x, y, /)

Compares two values numerically and returns the maximum.

</div>

<div class="method">

max_mag(x, y, /)

Compares the values numerically with their sign ignored.

</div>

<div class="method">

min(x, y, /)

Compares two values numerically and returns the minimum.

</div>

<div class="method">

min_mag(x, y, /)

Compares the values numerically with their sign ignored.

</div>

<div class="method">

minus(x, /)

Minus corresponds to the unary prefix minus operator in Python.

</div>

<div class="method">

multiply(x, y, /)

Return the product of *x* and *y*.

</div>

<div class="method">

next_minus(x, /)

Returns the largest representable number smaller than *x*.

</div>

<div class="method">

next_plus(x, /)

Returns the smallest representable number larger than *x*.

</div>

<div class="method">

next_toward(x, y, /)

Returns the number closest to *x*, in direction towards *y*.

</div>

<div class="method">

normalize(x, /)

Reduces *x* to its simplest form.

</div>

<div class="method">

number_class(x, /)

Returns an indication of the class of *x*.

</div>

<div class="method">

plus(x, /)

Plus corresponds to the unary prefix plus operator in Python. This operation applies the context precision and rounding, so it is *not* an identity operation.

</div>

<div class="method">

power(x, y, modulo=None)

Return `x` to the power of `y`, reduced modulo `modulo` if given.

With two arguments, compute `x**y`. If `x` is negative then `y` must be integral. The result will be inexact unless `y` is integral and the result is finite and can be expressed exactly in 'precision' digits. The rounding mode of the context is used. Results are always correctly rounded in the Python version.

`Decimal(0) ** Decimal(0)` results in `InvalidOperation`, and if `InvalidOperation` is not trapped, then results in `Decimal('NaN')`.

<div class="versionchanged">

3.3 The C module computes `power` in terms of the correctly rounded `exp` and `ln` functions. The result is well-defined but only "almost always correctly rounded".

</div>

With three arguments, compute `(x**y) % modulo`. For the three argument form, the following restrictions on the arguments hold:

- all three arguments must be integral
- `y` must be nonnegative
- at least one of `x` or `y` must be nonzero
- `modulo` must be nonzero and have at most 'precision' digits

The value resulting from `Context.power(x, y, modulo)` is equal to the value that would be obtained by computing `(x**y) % modulo` with unbounded precision, but is computed more efficiently. The exponent of the result is zero, regardless of the exponents of `x`, `y` and `modulo`. The result is always exact.

</div>

<div class="method">

quantize(x, y, /)

Returns a value equal to *x* (rounded), having the exponent of *y*.

</div>

<div class="method">

radix()

Just returns 10, as this is Decimal, :)

</div>

<div class="method">

remainder(x, y, /)

Returns the remainder from integer division.

The sign of the result, if non-zero, is the same as that of the original dividend.

</div>

<div class="method">

remainder_near(x, y, /)

Returns `x - y * n`, where *n* is the integer nearest the exact value of `x / y` (if the result is 0 then its sign will be the sign of *x*).

</div>

<div class="method">

rotate(x, y, /)

Returns a rotated copy of *x*, *y* times.

</div>

<div class="method">

same_quantum(x, y, /)

Returns `True` if the two operands have the same exponent.

</div>

<div class="method">

scaleb (x, y, /)

Returns the first operand after adding the second value its exp.

</div>

<div class="method">

shift(x, y, /)

Returns a shifted copy of *x*, *y* times.

</div>

<div class="method">

sqrt(x, /)

Square root of a non-negative number to context precision.

</div>

<div class="method">

subtract(x, y, /)

Return the difference between *x* and *y*.

</div>

<div class="method">

to_eng_string(x, /)

Convert to a string, using engineering notation if an exponent is needed.

Engineering notation has an exponent which is a multiple of 3. This can leave up to 3 digits to the left of the decimal place and may require the addition of either one or two trailing zeros.

</div>

<div class="method">

to_integral_exact(x, /)

Rounds to an integer.

</div>

<div class="method">

to_sci_string(x, /)

Converts a number to a string using scientific notation.

</div>

</div>

## Constants

<div class="data">

SPEC_VERSION

The highest version of the General Decimal Arithmetic Specification that this implementation complies with. See <https://speleotrove.com/decimal/decarith.html> for the specification.

<div class="versionadded">

3.15

</div>

</div>

The following constants are only relevant for the C module. They are also included in the pure Python version for compatibility.

<table style="width:98%;">
<colgroup>
<col style="width: 38%" />
<col style="width: 24%" />
<col style="width: 35%" />
</colgroup>
<thead>
<tr>
<th></th>
<th><blockquote>
<p>32-bit</p>
</blockquote></th>
<th><blockquote>
<p>64-bit</p>
</blockquote></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>MAX_PREC</p>
</div></td>
<td><blockquote>
<p><code>425000000</code></p>
</blockquote></td>
<td><blockquote>
<p><code>999999999999999999</code></p>
</blockquote></td>
</tr>
<tr>
<td><div class="data">
<p>MAX_EMAX</p>
</div></td>
<td><blockquote>
<p><code>425000000</code></p>
</blockquote></td>
<td><blockquote>
<p><code>999999999999999999</code></p>
</blockquote></td>
</tr>
<tr>
<td><div class="data">
<p>MIN_EMIN</p>
</div></td>
<td><blockquote>
<p><code>-425000000</code></p>
</blockquote></td>
<td><blockquote>
<p><code>-999999999999999999</code></p>
</blockquote></td>
</tr>
<tr>
<td><div class="data">
<p>MIN_ETINY</p>
</div></td>
<td><blockquote>
<p><code>-849999999</code></p>
</blockquote></td>
<td><blockquote>
<p><code>-1999999999999999997</code></p>
</blockquote></td>
</tr>
<tr>
<td><div class="data">
<p>IEEE_CONTEXT_MAX_BITS</p>
</div></td>
<td><blockquote>
<p><code>256</code></p>
</blockquote></td>
<td><blockquote>
<p><code>512</code></p>
</blockquote></td>
</tr>
</tbody>
</table>

<div class="data">

HAVE_THREADS

The value is `True`. Deprecated, because Python now always has threads.

<div class="deprecated">

3.9

</div>

</div>

<div class="data">

HAVE_CONTEXTVAR

The default value is `True`. If Python is `configured using
the --without-decimal-contextvar option <--without-decimal-contextvar>`, the C version uses a thread-local rather than a coroutine-local context and the value is `False`. This is slightly faster in some nested context scenarios.

<div class="versionadded">

3.8.3

</div>

</div>

The following constants are only available in the C module.

<div class="data">

LIBMPDEC_VERSION

The version string of the libmpdec library that was used for building the module. This may be different from the libmpdec library actually used at runtime, which is available as `libmpdec_version`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

libmpdec_version

The version string of the libmpdec library actually loaded by the interpreter.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

LIBMPDEC_VERSION_INFO

A named tuple containing the three components of the libmpdec library version that was used for building the module: *major*, *minor*, and *micro*. All values are integers. The components can also be accessed by name, so `decimal.LIBMPDEC_VERSION_INFO[0]` is equivalent to `decimal.LIBMPDEC_VERSION_INFO.major` and so on. This may be different from the libmpdec library actually used at runtime, which is available as `libmpdec_version_info`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

libmpdec_version_info

A named tuple containing the version of the libmpdec library actually loaded by the interpreter, with the same fields as `LIBMPDEC_VERSION_INFO`.

<div class="versionadded">

3.16

</div>

</div>

## Rounding modes

<div class="data">

ROUND_CEILING

Round towards `Infinity`.

</div>

<div class="data">

ROUND_DOWN

Round towards zero.

</div>

<div class="data">

ROUND_FLOOR

Round towards `-Infinity`.

</div>

<div class="data">

ROUND_HALF_DOWN

Round to nearest with ties going towards zero.

</div>

<div class="data">

ROUND_HALF_EVEN

Round to nearest with ties going to nearest even integer.

</div>

<div class="data">

ROUND_HALF_UP

Round to nearest with ties going away from zero.

</div>

<div class="data">

ROUND_UP

Round away from zero.

</div>

<div class="data">

ROUND_05UP

Round away from zero if last digit after rounding towards zero would have been 0 or 5; otherwise round towards zero.

</div>

## Signals

Signals represent conditions that arise during computation. Each corresponds to one context flag and one context trap enabler.

The context flag is set whenever the condition is encountered. After the computation, flags may be checked for informational purposes (for instance, to determine whether a computation was exact). After checking the flags, be sure to clear all flags before starting the next computation.

If the context's trap enabler is set for the signal, then the condition causes a Python exception to be raised. For example, if the `DivisionByZero` trap is set, then a `DivisionByZero` exception is raised upon encountering the condition.

<div class="Clamped">

Altered an exponent to fit representation constraints.

Typically, clamping occurs when an exponent falls outside the context's `~Context.Emin` and `~Context.Emax` limits. If possible, the exponent is reduced to fit by adding zeros to the coefficient.

</div>

<div class="DecimalException">

Base class for other signals and a subclass of `ArithmeticError`.

</div>

<div class="DivisionByZero">

Signals the division of a non-infinite number by zero.

Can occur with division, modulo division, or when raising a number to a negative power. If this signal is not trapped, returns `Infinity` or `-Infinity` with the sign determined by the inputs to the calculation.

</div>

<div class="Inexact">

Indicates that rounding occurred and the result is not exact.

Signals when non-zero digits were discarded during rounding. The rounded result is returned. The signal flag or trap is used to detect when results are inexact.

</div>

<div class="InvalidOperation">

An invalid operation was performed.

Indicates that an operation was requested that does not make sense. If not trapped, returns `NaN`. Possible causes include:

    Infinity - Infinity
    0 * Infinity
    Infinity / Infinity
    x % 0
    Infinity % x
    sqrt(-x) and x > 0
    0 ** 0
    x ** (non-integer)
    x ** Infinity

</div>

<div class="Overflow">

Numerical overflow.

Indicates the exponent is larger than `Context.Emax` after rounding has occurred. If not trapped, the result depends on the rounding mode, either pulling inward to the largest representable finite number or rounding outward to `Infinity`. In either case, `Inexact` and `Rounded` are also signaled.

</div>

<div class="Rounded">

Rounding occurred though possibly no information was lost.

Signaled whenever rounding discards digits; even if those digits are zero (such as rounding `5.00` to `5.0`). If not trapped, returns the result unchanged. This signal is used to detect loss of significant digits.

</div>

<div class="Subnormal">

Exponent was lower than `~Context.Emin` prior to rounding.

Occurs when an operation result is subnormal (the exponent is too small). If not trapped, returns the result unchanged.

</div>

<div class="Underflow">

Numerical underflow with result rounded to zero.

Occurs when a subnormal result is pushed to zero by rounding. `Inexact` and `Subnormal` are also signaled.

</div>

<div class="FloatOperation">

Enable stricter semantics for mixing floats and Decimals.

If the signal is not trapped (default), mixing floats and Decimals is permitted in the `~decimal.Decimal` constructor, `~decimal.Context.create_decimal` and all comparison operators. Both conversion and comparisons are exact. Any occurrence of a mixed operation is silently recorded by setting `FloatOperation` in the context flags. Explicit conversions with `~decimal.Decimal.from_float` or `~decimal.Context.create_decimal_from_float` do not set the flag.

Otherwise (the signal is trapped), only equality comparisons and explicit conversions are silent. All other mixed operations raise `FloatOperation`.

</div>

The following table summarizes the hierarchy of signals:

    exceptions.ArithmeticError(exceptions.Exception)
        DecimalException
            Clamped
            DivisionByZero(DecimalException, exceptions.ZeroDivisionError)
            Inexact
                Overflow(Inexact, Rounded)
                Underflow(Inexact, Rounded, Subnormal)
            InvalidOperation
            Rounded
            Subnormal
            FloatOperation(DecimalException, exceptions.TypeError)

## Floating-point notes

### Mitigating round-off error with increased precision

The use of decimal floating point eliminates decimal representation error (making it possible to represent `0.1` exactly); however, some operations can still incur round-off error when non-zero digits exceed the fixed precision.

The effects of round-off error can be amplified by the addition or subtraction of nearly offsetting quantities resulting in loss of significance. Knuth provides two instructive examples where rounded floating-point arithmetic with insufficient precision causes the breakdown of the associative and distributive properties of addition:

<div class="doctest">

newcontext

\# Examples from Seminumerical Algorithms, Section 4.2.2. \>\>\> from decimal import Decimal, getcontext \>\>\> getcontext().prec = 8

\>\>\> u, v, w = Decimal(11111113), Decimal(-11111111), Decimal('7.51111111') \>\>\> (u + v) + w Decimal('9.5111111') \>\>\> u + (v + w) Decimal('10')

\>\>\> u, v, w = Decimal(20000), Decimal(-6), Decimal('6.0000003') \>\>\> (u\*v) + (u\*w) Decimal('0.01') \>\>\> u \* (v+w) Decimal('0.0060000')

</div>

The `!decimal` module makes it possible to restore the identities by expanding the precision sufficiently to avoid loss of significance:

<div class="doctest">

newcontext

\>\>\> getcontext().prec = 20 \>\>\> u, v, w = Decimal(11111113), Decimal(-11111111), Decimal('7.51111111') \>\>\> (u + v) + w Decimal('9.51111111') \>\>\> u + (v + w) Decimal('9.51111111') \>\>\> \>\>\> u, v, w = Decimal(20000), Decimal(-6), Decimal('6.0000003') \>\>\> (u\*v) + (u\*w) Decimal('0.0060000') \>\>\> u \* (v+w) Decimal('0.0060000')

</div>

### Special values

The number system for the `!decimal` module provides special values including `NaN`, `sNaN`, `-Infinity`, `Infinity`, and two zeros, `+0` and `-0`.

Infinities can be constructed directly with: `Decimal('Infinity')`. Also, they can arise from dividing by zero when the `DivisionByZero` signal is not trapped. Likewise, when the `Overflow` signal is not trapped, infinity can result from rounding beyond the limits of the largest representable number.

The infinities are signed (affine) and can be used in arithmetic operations where they get treated as very large, indeterminate numbers. For instance, adding a constant to infinity gives another infinite result.

Some operations are indeterminate and return `NaN`, or if the `InvalidOperation` signal is trapped, raise an exception. For example, `0/0` returns `NaN` which means "not a number". This variety of `NaN` is quiet and, once created, will flow through other computations always resulting in another `NaN`. This behavior can be useful for a series of computations that occasionally have missing inputs --- it allows the calculation to proceed while flagging specific results as invalid.

A variant is `sNaN` which signals rather than remaining quiet after every operation. This is a useful return value when an invalid result needs to interrupt a calculation for special handling.

The behavior of Python's comparison operators can be a little surprising where a `NaN` is involved. A test for equality where one of the operands is a quiet or signaling `NaN` always returns `False` (even when doing `Decimal('NaN')==Decimal('NaN')`), while a test for inequality always returns `True`. An attempt to compare two Decimals using any of the `<`, `<=`, `>` or `>=` operators will raise the `InvalidOperation` signal if either operand is a `NaN`, and return `False` if this signal is not trapped. Note that the General Decimal Arithmetic specification does not specify the behavior of direct comparisons; these rules for comparisons involving a `NaN` were taken from the IEEE 854 standard (see Table 3 in section 5.7). To ensure strict standards-compliance, use the `~Decimal.compare` and `~Decimal.compare_signal` methods instead.

The signed zeros can result from calculations that underflow. They keep the sign that would have resulted if the calculation had been carried out to greater precision. Since their magnitude is zero, both positive and negative zeros are treated as equal and their sign is informational.

In addition to the two signed zeros which are distinct yet equal, there are various representations of zero with differing precisions yet equivalent in value. This takes a bit of getting used to. For an eye accustomed to normalized floating-point representations, it is not immediately obvious that the following calculation returns a value equal to zero:

> \>\>\> 1 / Decimal('Infinity') Decimal('0E-1000026')

## Working with threads

The `getcontext` function accesses a different `Context` object for each thread. Having separate thread contexts means that threads may make changes (such as `getcontext().prec=10`) without interfering with other threads.

Likewise, the `setcontext` function automatically assigns its target to the current thread.

If `setcontext` has not been called before `getcontext`, then `getcontext` will automatically create a new context for use in the current thread. New context objects have default values set from the `decimal.DefaultContext` object.

The `sys.flags.thread_inherit_context` flag affects the context for new threads. If the flag is false, new threads will start with an empty context. In this case, `getcontext` will create a new context object when called and use the default values from *DefaultContext*. If the flag is true, new threads will start with a copy of context from the caller of `threading.Thread.start`.

To control the defaults so that each thread will use the same values throughout the application, directly modify the *DefaultContext* object. This should be done *before* any threads are started so that there won't be a race condition between threads calling `getcontext`. For example:

    # Set applicationwide defaults for all threads about to be launched
    DefaultContext.prec = 12
    DefaultContext.rounding = ROUND_DOWN
    DefaultContext.traps = ExtendedContext.traps.copy()
    DefaultContext.traps[InvalidOperation] = 1
    setcontext(DefaultContext)

    # Afterwards, the threads can be started
    t1.start()
    t2.start()
    t3.start()
     . . .

## Recipes

Here are a few recipes that serve as utility functions and that demonstrate ways to work with the `Decimal` class:

    def moneyfmt(value, places=2, curr='', sep=',', dp='.',
                 pos='', neg='-', trailneg=''):
        """Convert Decimal to a money formatted string.

        places:  required number of places after the decimal point
        curr:    optional currency symbol before the sign (may be blank)
        sep:     optional grouping separator (comma, period, space, or blank)
        dp:      decimal point indicator (comma or period)
                 only specify as blank when places is zero
        pos:     optional sign for positive numbers: '+', space or blank
        neg:     optional sign for negative numbers: '-', '(', space or blank
        trailneg:optional trailing minus indicator:  '-', ')', space or blank

        >>> d = Decimal('-1234567.8901')
        >>> moneyfmt(d, curr='$')
        '-$1,234,567.89'
        >>> moneyfmt(d, places=0, sep='.', dp='', neg='', trailneg='-')
        '1.234.568-'
        >>> moneyfmt(d, curr='$', neg='(', trailneg=')')
        '($1,234,567.89)'
        >>> moneyfmt(Decimal(123456789), sep=' ')
        '123 456 789.00'
        >>> moneyfmt(Decimal('-0.02'), neg='<', trailneg='>')
        '<0.02>'

        """
        q = Decimal(10) ** -places      # 2 places --> '0.01'
        sign, digits, exp = value.quantize(q).as_tuple()
        result = []
        digits = list(map(str, digits))
        build, next = result.append, digits.pop
        if sign:
            build(trailneg)
        for i in range(places):
            build(next() if digits else '0')
        if places:
            build(dp)
        if not digits:
            build('0')
        i = 0
        while digits:
            build(next())
            i += 1
            if i == 3 and digits:
                i = 0
                build(sep)
        build(curr)
        build(neg if sign else pos)
        return ''.join(reversed(result))

    def pi():
        """Compute Pi to the current precision.

        >>> print(pi())
        3.141592653589793238462643383

        """
        getcontext().prec += 2  # extra digits for intermediate steps
        three = Decimal(3)      # substitute "three=3.0" for regular floats
        lasts, t, s, n, na, d, da = 0, three, 3, 1, 0, 0, 24
        while s != lasts:
            lasts = s
            n, na = n+na, na+8
            d, da = d+da, da+32
            t = (t * n) / d
            s += t
        getcontext().prec -= 2
        return +s               # unary plus applies the new precision

    def exp(x):
        """Return e raised to the power of x.  Result type matches input type.

        >>> print(exp(Decimal(1)))
        2.718281828459045235360287471
        >>> print(exp(Decimal(2)))
        7.389056098930650227230427461
        >>> print(exp(2.0))
        7.38905609893
        >>> print(exp(2+0j))
        (7.38905609893+0j)

        """
        getcontext().prec += 2
        i, lasts, s, fact, num = 0, 0, 1, 1, 1
        while s != lasts:
            lasts = s
            i += 1
            fact *= i
            num *= x
            s += num / fact
        getcontext().prec -= 2
        return +s

    def cos(x):
        """Return the cosine of x as measured in radians.

        The Taylor series approximation works best for a small value of x.
        For larger values, first compute x = x % (2 * pi).

        >>> print(cos(Decimal('0.5')))
        0.8775825618903727161162815826
        >>> print(cos(0.5))
        0.87758256189
        >>> print(cos(0.5+0j))
        (0.87758256189+0j)

        """
        getcontext().prec += 2
        i, lasts, s, fact, num, sign = 0, 0, 1, 1, 1, 1
        while s != lasts:
            lasts = s
            i += 2
            fact *= i * (i-1)
            num *= x * x
            sign *= -1
            s += num / fact * sign
        getcontext().prec -= 2
        return +s

    def sin(x):
        """Return the sine of x as measured in radians.

        The Taylor series approximation works best for a small value of x.
        For larger values, first compute x = x % (2 * pi).

        >>> print(sin(Decimal('0.5')))
        0.4794255386042030002732879352
        >>> print(sin(0.5))
        0.479425538604
        >>> print(sin(0.5+0j))
        (0.479425538604+0j)

        """
        getcontext().prec += 2
        i, lasts, s, fact, num, sign = 1, 0, x, 1, x, 1
        while s != lasts:
            lasts = s
            i += 2
            fact *= i * (i-1)
            num *= x * x
            sign *= -1
            s += num / fact * sign
        getcontext().prec -= 2
        return +s

## Decimal FAQ

Q: It is cumbersome to type `decimal.Decimal('1234.5')`. Is there a way to minimize typing when using the interactive interpreter?

A: Some users abbreviate the constructor to just a single letter:

> \>\>\> D = decimal.Decimal \>\>\> D('1.23') + D('3.45') Decimal('4.68')

Q: In a fixed-point application with two decimal places, some inputs have many places and need to be rounded. Others are not supposed to have excess digits and need to be validated. What methods should be used?

A: The `~Decimal.quantize` method rounds to a fixed number of decimal places. If the `Inexact` trap is set, it is also useful for validation:

> \>\>\> TWOPLACES = Decimal(10) \*\* -2 \# same as Decimal('0.01')
>
> \>\>\> \# Round to two places \>\>\> Decimal('3.214').quantize(TWOPLACES) Decimal('3.21')
>
> \>\>\> \# Validate that a number does not exceed two places \>\>\> Decimal('3.21').quantize(TWOPLACES, context=Context(traps=\[Inexact\])) Decimal('3.21')
>
> \>\>\> Decimal('3.214').quantize(TWOPLACES, context=Context(traps=\[Inexact\])) Traceback (most recent call last): ... Inexact: None

Q: Once I have valid two place inputs, how do I maintain that invariant throughout an application?

A: Some operations like addition, subtraction, and multiplication by an integer will automatically preserve fixed point. Others operations, like division and non-integer multiplication, will change the number of decimal places and need to be followed-up with a `~Decimal.quantize` step:

> \>\>\> a = Decimal('102.72') \# Initial fixed-point values \>\>\> b = Decimal('3.17') \>\>\> a + b \# Addition preserves fixed-point Decimal('105.89') \>\>\> a - b Decimal('99.55') \>\>\> a \* 42 \# So does integer multiplication Decimal('4314.24') \>\>\> (a \* b).quantize(TWOPLACES) \# Must quantize non-integer multiplication Decimal('325.62') \>\>\> (b / a).quantize(TWOPLACES) \# And quantize division Decimal('0.03')

In developing fixed-point applications, it is convenient to define functions to handle the `~Decimal.quantize` step:

> \>\>\> def mul(x, y, fp=TWOPLACES): ... return (x \* y).quantize(fp) ... \>\>\> def div(x, y, fp=TWOPLACES): ... return (x / y).quantize(fp)
>
> \>\>\> mul(a, b) \# Automatically preserve fixed-point Decimal('325.62') \>\>\> div(b, a) Decimal('0.03')

Q: There are many ways to express the same value. The numbers `200`, `200.000`, `2E2`, and `.02E+4` all have the same value at various precisions. Is there a way to transform them to a single recognizable canonical value?

A: The `~Decimal.normalize` method maps all equivalent values to a single representative:

> \>\>\> values = map(Decimal, '200 200.000 2E2 .02E+4'.split()) \>\>\> \[v.normalize() for v in values\] \[Decimal('2E+2'), Decimal('2E+2'), Decimal('2E+2'), Decimal('2E+2')\]

Q: When does rounding occur in a computation?

A: It occurs *after* the computation. The philosophy of the decimal specification is that numbers are considered exact and are created independent of the current context. They can even have greater precision than current context. Computations process with those exact inputs and then rounding (or other context operations) is applied to the *result* of the computation:

    >>> getcontext().prec = 5
    >>> pi = Decimal('3.1415926535')   # More than 5 digits
    >>> pi                             # All digits are retained
    Decimal('3.1415926535')
    >>> pi + 0                         # Rounded after an addition
    Decimal('3.1416')
    >>> pi - Decimal('0.00005')        # Subtract unrounded numbers, then round
    Decimal('3.1415')
    >>> pi + 0 - Decimal('0.00005')    # Intermediate values are rounded
    Decimal('3.1416')

Q: Some decimal values always print with exponential notation. Is there a way to get a non-exponential representation?

A: For some values, exponential notation is the only way to express the number of significant places in the coefficient. For example, expressing `5.0E+3` as `5000` keeps the value constant but cannot show the original's two-place significance.

If an application does not care about tracking significance, it is easy to remove the exponent and trailing zeroes, losing significance, but keeping the value unchanged:

> \>\>\> def remove_exponent(d): ... return d.quantize(Decimal(1)) if d == d.to_integral() else d.normalize()
>
> \>\>\> remove_exponent(Decimal('5E+3')) Decimal('5000')

Q: Is there a way to convert a regular float to a `Decimal`?

A: Yes, any binary floating-point number can be exactly expressed as a Decimal though an exact conversion may take more precision than intuition would suggest:

<div class="doctest">

\>\>\> Decimal(math.pi) Decimal('3.141592653589793115997963468544185161590576171875')

</div>

Q: Within a complex calculation, how can I make sure that I haven't gotten a spurious result because of insufficient precision or rounding anomalies.

A: The decimal module makes it easy to test results. A best practice is to re-run calculations using greater precision and with various rounding modes. Widely differing results indicate insufficient precision, rounding mode issues, ill-conditioned inputs, or a numerically unstable algorithm.

Q: I noticed that context precision is applied to the results of operations but not to the inputs. Is there anything to watch out for when mixing values of different precisions?

A: Yes. The principle is that all values are considered to be exact and so is the arithmetic on those values. Only the results are rounded. The advantage for inputs is that "what you type is what you get". A disadvantage is that the results can look odd if you forget that the inputs haven't been rounded:

<div class="doctest">

newcontext

\>\>\> getcontext().prec = 3 \>\>\> Decimal('3.104') + Decimal('2.104') Decimal('5.21') \>\>\> Decimal('3.104') + Decimal('0.000') + Decimal('2.104') Decimal('5.20')

</div>

The solution is either to increase precision or to force rounding of inputs using the unary plus operation:

<div class="doctest">

newcontext

\>\>\> getcontext().prec = 3 \>\>\> +Decimal('1.23456789') \# unary plus triggers rounding Decimal('1.23')

</div>

Alternatively, inputs can be rounded upon creation using the `Context.create_decimal` method:

> \>\>\> Context(prec=5, rounding=ROUND_DOWN).create_decimal('1.2345678') Decimal('1.2345')

Q: Is the CPython implementation fast for large numbers?

A: Yes. In the CPython and PyPy3 implementations, the C/CFFI versions of the decimal module integrate the high speed [libmpdec](https://www.bytereef.org/mpdecimal/doc/libmpdec/index.html) library for arbitrary precision correctly rounded decimal floating-point arithmetic[^1]. `libmpdec` uses [Karatsuba multiplication](https://en.wikipedia.org/wiki/Karatsuba_algorithm) for medium-sized numbers and the [Number Theoretic Transform](https://en.wikipedia.org/wiki/Discrete_Fourier_transform_(general)#Number-theoretic_transform) for very large numbers.

The context must be adapted for exact arbitrary precision arithmetic. `~Context.Emin` and `~Context.Emax` should always be set to the maximum values, `~Context.clamp` should always be 0 (the default). Setting `~Context.prec` requires some care.

The easiest approach for trying out bignum arithmetic is to use the maximum value for `~Context.prec` as well[^2]:

    >>> setcontext(Context(prec=MAX_PREC, Emax=MAX_EMAX, Emin=MIN_EMIN))
    >>> x = Decimal(2) ** 256
    >>> x / 128
    Decimal('904625697166532776746648320380374280103671755200316906558262375061821325312')

For inexact results, `MAX_PREC` is far too large on 64-bit platforms and the available memory will be insufficient:

    >>> Decimal(1) / 3
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    MemoryError

On systems with overallocation (e.g. Linux), a more sophisticated approach is to adjust `~Context.prec` to the amount of available RAM. Suppose that you have 8GB of RAM and expect 10 simultaneous operands using a maximum of 500MB each:

    >>> import sys
    >>>
    >>> # Maximum number of digits for a single operand using 500MB in 8-byte words
    >>> # with 19 digits per word (4-byte and 9 digits for the 32-bit build):
    >>> maxdigits = 19 * ((500 * 1024**2) // 8)
    >>>
    >>> # Check that this works:
    >>> c = Context(prec=maxdigits, Emax=MAX_EMAX, Emin=MIN_EMIN)
    >>> c.traps[Inexact] = True
    >>> setcontext(c)
    >>>
    >>> # Fill the available precision with nines:
    >>> x = Decimal(0).logical_invert() * 9
    >>> sys.getsizeof(x)
    524288112
    >>> x + 2
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      decimal.Inexact: [<class 'decimal.Inexact'>]

In general (and especially on systems without overallocation), it is recommended to estimate even tighter bounds and set the `Inexact` trap if all calculations are expected to be exact.

[^1]: .. versionadded:: 3.3

[^2]: .. versionchanged:: 3.9 This approach now works for all exact results except for non-integer powers.

# Development Tools

The modules described in this chapter help you write software. For example, the `pydoc` module takes a module and generates documentation based on the module's contents. The `doctest` and `unittest` modules contains frameworks for writing unit tests that automatically exercise code and verify that the expected output is produced.

The list of modules described in this chapter is:

<div class="toctree">

typing.rst pydoc.rst devmode.rst doctest.rst unittest.rst unittest.mock.rst unittest.mock-examples.rst test.rst

</div>

# Python Development Mode

<div class="versionadded">

3.7

</div>

The Python Development Mode introduces additional runtime checks that are too expensive to be enabled by default. It should not be more verbose than the default if the code is correct; new warnings are only emitted when an issue is detected.

It can be enabled using the `-X dev <-X>` command line option or by setting the `PYTHONDEVMODE` environment variable to `1`.

See also `Python debug build <debug-build>`.

## Effects of the Python Development Mode

Enabling the Python Development Mode is similar to the following command, but with additional effects described below:

    PYTHONMALLOC=debug PYTHONASYNCIODEBUG=1 python -W default -X faulthandler

Effects of the Python Development Mode:

- Add `default` `warning filter <describing-warning-filters>`. The following warnings are shown:

  - `DeprecationWarning`
  - `ImportWarning`
  - `PendingDeprecationWarning`
  - `ResourceWarning`

  Normally, the above warnings are filtered by the default `warning
  filters <describing-warning-filters>`.

  It behaves as if the `-W default <-W>` command line option is used.

  Use the `-W error <-W>` command line option or set the `PYTHONWARNINGS` environment variable to `error` to treat warnings as errors.

- Install debug hooks on memory allocators to check for:

  - Buffer underflow
  - Buffer overflow
  - Memory allocator API violation
  - Unsafe usage of the GIL

  See the `PyMem_SetupDebugHooks` C function.

  It behaves as if the `PYTHONMALLOC` environment variable is set to `debug`.

  To enable the Python Development Mode without installing debug hooks on memory allocators, set the `PYTHONMALLOC` environment variable to `default`.

- Call `faulthandler.enable` at Python startup to install handlers for the `~signal.SIGSEGV`, `~signal.SIGFPE`, `~signal.SIGABRT`, `~signal.SIGBUS` and `~signal.SIGILL` signals to dump the Python traceback on a crash.

  It behaves as if the `-X faulthandler <-X>` command line option is used or if the `PYTHONFAULTHANDLER` environment variable is set to `1`.

- Enable `asyncio debug mode <asyncio-debug-mode>`. For example, `asyncio` checks for coroutines that were not awaited and logs them.

  It behaves as if the `PYTHONASYNCIODEBUG` environment variable is set to `1`.

- Check the *encoding* and *errors* arguments for string encoding and decoding operations. Examples: `open`, `str.encode` and `bytes.decode`.

  By default, for best performance, the *errors* argument is only checked at the first encoding/decoding error and the *encoding* argument is sometimes ignored for empty strings.

- The `io.IOBase` destructor logs `close()` exceptions.

- Set the `~sys.flags.dev_mode` attribute of `sys.flags` to `True`.

The Python Development Mode does not enable the `tracemalloc` module by default, because the overhead cost (to performance and memory) would be too large. Enabling the `tracemalloc` module provides additional information on the origin of some errors. For example, `ResourceWarning` logs the traceback where the resource was allocated, and a buffer overflow error logs the traceback where the memory block was allocated.

The Python Development Mode does not prevent the `-O` command line option from removing `assert` statements nor from setting `__debug__` to `False`.

The Python Development Mode can only be enabled at the Python startup. Its value can be read from `sys.flags.dev_mode <sys.flags>`.

<div class="versionchanged">

3.8 The `io.IOBase` destructor now logs `close()` exceptions.

</div>

<div class="versionchanged">

3.9 The *encoding* and *errors* arguments are now checked for string encoding and decoding operations.

</div>

## ResourceWarning Example

Example of a script counting the number of lines of the text file specified in the command line:

    import sys

    def main():
        fp = open(sys.argv[1])
        nlines = len(fp.readlines())
        print(nlines)
        # The file is closed implicitly

    if __name__ == "__main__":
        main()

The script does not close the file explicitly. By default, Python does not emit any warning. Example using README.txt, which has 269 lines:

```
$ python script.py README.txt
269
```

Enabling the Python Development Mode displays a `ResourceWarning` warning:

```
$ python -X dev script.py README.txt
269
script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>
  main()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
```

In addition, enabling `tracemalloc` shows the line where the file was opened:

```
$ python -X dev -X tracemalloc=5 script.py README.rst
269
script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>
  main()
Object allocated at (most recent call last):
  File "script.py", lineno 10
    main()
  File "script.py", lineno 4
    fp = open(sys.argv[1])
```

The fix is to close explicitly the file. Example using a context manager:

    def main():
        # Close the file explicitly when exiting the with block
        with open(sys.argv[1]) as fp:
            nlines = len(fp.readlines())
        print(nlines)

Not closing a resource explicitly can leave a resource open for way longer than expected; it can cause severe issues upon exiting Python. It is bad in CPython, but it is even worse in PyPy. Closing resources explicitly makes an application more deterministic and more reliable.

## Bad file descriptor error example

Script displaying the first line of itself:

    import os

    def main():
        fp = open(__file__)
        firstline = fp.readline()
        print(firstline.rstrip())
        os.close(fp.fileno())
        # The file is closed implicitly

    main()

By default, Python does not emit any warning:

```
$ python script.py
import os
```

The Python Development Mode shows a `ResourceWarning` and logs a "Bad file descriptor" error when finalizing the file object:

```
$ python -X dev script.py
import os
script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>
  main()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Exception ignored in: <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>
Traceback (most recent call last):
  File "script.py", line 10, in <module>
    main()
OSError: [Errno 9] Bad file descriptor
```

`os.close(fp.fileno())` closes the file descriptor. When the file object finalizer tries to close the file descriptor again, it fails with the `Bad file descriptor` error. A file descriptor must be closed only once. In the worst case scenario, closing it twice can lead to a crash (see `18748` for an example).

The fix is to remove the `os.close(fp.fileno())` line, or open the file with `closefd=False`.

# `!difflib` --- Helpers for computing deltas

<div class="module" data-synopsis="Helpers for computing differences between objects.">

difflib

</div>

**Source code:** `Lib/difflib.py`

<div class="testsetup">

import sys from difflib import \*

</div>

------------------------------------------------------------------------

This module provides classes and functions for comparing sequences. Most of them compare sequences of text lines (for example lists of strings, or `file objects <file object>`) and produce `diffs` -- reports on the differences. Diffs can be produced in various formats, including HTML and context and unified diffs -- formats produced by tools like `diff <diff(1)>` and `git diff <git-diff(1)>`.

Comparisons are done using a matching algorithm implemented in `SequenceMatcher` -- a flexible class for comparing pairs of sequences of any type, not just text, so long as the sequence elements are `hashable`.

## Junk heuristic

`!difflib` uses a `junk` heuristic: some items are deemed to be `junk`, and ignored when searching for similarities. Ideally, these are uninteresting or common items, such as blank lines or whitespace.

This heuristic can speed the algorithm up (because it reduces the number of possible combinations) and it can produce results that are more understandable for humans (typically breaking on whitespace). But it can also cause pathological cases:

- Inappropriately chosen junk items can cause an unexpectedly **large** (but still correct) result.
- The default heuristic is **asymmetric**: only the second sequence is inspected when determining what is considered junk, so comparing A to B can give different results than comparing B to A and reversing the result.

By default, if the second input sequence is at least 200 items long, items that account for more than 1% it are considered *junk*.

Depending on your data, you should consider turning this heuristic off (setting `~difflib.SequenceMatcher`'s *autojunk* argument to `False`) or tuning it (using the *isjunk* argument, perhaps to one of the `predefined functions <difflib-isjunk-functions>`).

## The `!difflib` algorithm

The algorithm used in `SequenceMatcher` predates, and is a little fancier than, an algorithm published in the late 1980s by Ratcliff and Obershelp under the hyperbolic name "gestalt pattern matching." The idea is to find the longest contiguous subsequence common to both inputs, then recursively handle the pieces of the sequences to the left and to the right of the matching subsequence.

<div class="seealso">

[Pattern Matching: The Gestalt Approach](https://jacobfilipp.com/DrDobbs/articles/DDJ/1988/8807/8807c/8807c.htm)  
Discussion of a similar algorithm by John W. Ratcliff and D. E. Metzener. This was published in Dr. Dobb's Journal in July, 1988.

</div>

As an extension to the Ratcliff and Obershelp algorithm, `!difflib` searches for the longest *junk-free* contiguous subsequence. See the `difflib-junk` section for details.

<div class="impl-detail">

Timing

The basic Ratcliff-Obershelp algorithm is cubic time in the worst case and quadratic time in the expected case. `difflib`'s algorithm is quadratic time for the worst case and has expected-case behavior dependent in a complicated way on how many elements the sequences have in common; best case time is linear.

</div>

## Diff generation

<div id="differ-objects">

<div class="Differ">

This is a class for comparing sequences of lines of text, and producing human-readable differences or deltas. Differ uses `SequenceMatcher` both to compare sequences of lines, and to compare sequences of characters within similar (near-matching) lines.

Each line of a `Differ` delta begins with a two-letter code:

| Code   | Meaning                                   |
|--------|-------------------------------------------|
| `'- '` | line unique to sequence 1                 |
| `'+ '` | line unique to sequence 2                 |
| `'  '` | line common to both sequences             |
| `'? '` | line not present in either input sequence |

Lines beginning with '`?`' attempt to guide the eye to intraline differences, and were not present in either input sequence. These lines can be confusing if the sequences contain whitespace characters, such as spaces, tabs or line breaks.

Note that `Differ`-generated deltas make no claim to be **minimal** diffs. To the contrary, minimal diffs are often counter-intuitive for humans, because they synch up anywhere possible, sometimes at accidental matches 100 pages apart. Restricting synch points to contiguous matches preserves some notion of locality, at the occasional cost of producing a longer diff.

The `Differ` class has this constructor:

<div class="method">

\_\_init\_\_(linejunk=None, charjunk=None, autojunk=True)

Optional keyword parameters *linejunk* and *charjunk* are for filter functions (or `None`):

*linejunk*: A function that accepts a single string argument, and returns true if the string is junk. The default is `None`, meaning that no line is considered junk.

*charjunk*: A function that accepts a single character argument (a string of length 1), and returns true if the character is junk. The default is `None`, meaning that no character is considered junk.

These junk-filtering functions speed up matching to find differences and do not cause any differing lines or characters to be ignored. Read the description of the `~SequenceMatcher.find_longest_match` method's *isjunk* parameter for an explanation.

Setting the optional *autojunk* argument to `False` will turn `automatic junk heuristic <difflib-junk>` off.

<div class="versionchanged">

3.16 Added keyword-only *autojunk* parameter.

</div>

</div>

`Differ` objects are used (deltas generated) via a single method:

<div class="method">

Differ.compare(a, b)

Compare two sequences of lines, and generate the delta (a sequence of lines).

Each sequence must contain individual single-line strings ending with newlines. Such sequences can be obtained from the `~io.IOBase.readlines` method of file-like objects. The generated delta also consists of newline-terminated strings, ready to be printed as-is via the `~io.IOBase.writelines` method of a file-like object.

</div>

</div>

</div>

<div class="HtmlDiff">

This class can be used to create an HTML table (or a complete HTML file containing the table) showing a side by side, line by line comparison of text with inter-line and intra-line change highlights. The table can be generated in either full or contextual difference mode.

> [!WARNING]
> The trailing newlines get stripped before the diff, so the result can be incomplete. See `71896` for details.

The constructor for this class is:

<div class="method">

\_\_init\_\_(tabsize=8, wrapcolumn=None, linejunk=None, charjunk=IS_CHARACTER_JUNK, \*, autojunk=True)

Initializes instance of `HtmlDiff`.

*tabsize* is an optional keyword argument to specify tab stop spacing and defaults to `8`.

*wrapcolumn* is an optional keyword to specify column number where lines are broken and wrapped, defaults to `None` where lines are not wrapped.

*linejunk* and *charjunk* are optional keyword arguments passed into `ndiff` (used by `HtmlDiff` to generate the side by side HTML differences). See `ndiff` documentation for argument default values and descriptions.

Setting the optional *autojunk* argument to `False` will turn `automatic junk heuristic <difflib-junk>` off.

<div class="versionchanged">

3.16 Added keyword-only *autojunk* parameter.

</div>

</div>

The following methods are public:

<div class="method">

make_file(fromlines, tolines, fromdesc='', todesc='', context=False, numlines=5, \*, charset='utf-8')

Compares *fromlines* and *tolines* (lists of strings) and returns a string which is a complete HTML file containing a table showing line by line differences with inter-line and intra-line changes highlighted.

*fromdesc* and *todesc* are optional keyword arguments to specify from/to file column header strings (both default to an empty string).

*context* and *numlines* are both optional keyword arguments. Set *context* to `True` when contextual differences are to be shown, else the default is `False` to show the full files. *numlines* defaults to `5`. When *context* is `True` *numlines* controls the number of context lines which surround the difference highlights. When *context* is `False` *numlines* controls the number of lines which are shown before a difference highlight when using the "next" hyperlinks (setting to zero would cause the "next" hyperlinks to place the next difference highlight at the top of the browser without any leading context).

> [!NOTE]
> *fromdesc* and *todesc* are interpreted as unescaped HTML and should be properly escaped while receiving input from untrusted sources.

<div class="versionchanged">

3.5 *charset* keyword-only argument was added. The default charset of HTML document changed from `'ISO-8859-1'` to `'utf-8'`.

</div>

</div>

<div class="method">

make_table(fromlines, tolines, fromdesc='', todesc='', context=False, numlines=5)

Compares *fromlines* and *tolines* (lists of strings) and returns a string which is a complete HTML table showing line by line differences with inter-line and intra-line changes highlighted.

The arguments for this method are the same as those for the `make_file` method.

</div>

</div>

<div class="function">

context_diff(a, b, fromfile='', tofile='', fromfiledate='', tofiledate='', n=3, lineterm='n', \*, autojunk=True)

Compare *a* and *b* (lists of strings); return a delta (a `generator` generating the delta lines) in context diff format.

Context diffs are a compact way of showing just the lines that have changed plus a few lines of context. The changes are shown in a before/after style. The number of context lines is set by *n* which defaults to three.

By default, the diff control lines (those with `***` or `---`) are created with a trailing newline. This is helpful so that inputs created from `io.IOBase.readlines` result in diffs that are suitable for use with `io.IOBase.writelines` since both the inputs and outputs have trailing newlines.

For inputs that do not have trailing newlines, set the *lineterm* argument to `""` so that the output will be uniformly newline free.

The context diff format normally has a header for filenames and modification times. Any or all of these may be specified using strings for *fromfile*, *tofile*, *fromfiledate*, and *tofiledate*. The modification times are normally expressed in the ISO 8601 format. If not specified, the strings default to blanks.

> \>\>\> import sys \>\>\> from difflib import \* \>\>\> s1 = \['baconn', 'eggsn', 'hamn', 'guidon'\] \>\>\> s2 = \['pythonn', 'eggyn', 'hamstern', 'guidon'\] \>\>\> sys.stdout.writelines(context_diff(s1, s2, fromfile='before.py', ... tofile='after.py')) **\* before.py --- after.py**\*\*\*\*\*\*\*\*\*\*\*\*\* **\* 1,4**\*\* ! bacon ! eggs ! ham guido --- 1,4 ---- ! python ! eggy ! hamster guido

See `difflib-interface` for a more detailed example.

Setting the optional *autojunk* argument to `False` will turn `automatic junk heuristic <difflib-junk>` off.

<div class="versionchanged">

3.16 Added keyword-only *autojunk* parameter.

</div>

</div>

<div class="function">

get_close_matches(word, possibilities, n=3, cutoff=0.6, \*, autojunk=True)

Return a list of the best "good enough" matches. *word* is a sequence for which close matches are desired (typically a string), and *possibilities* is a list of sequences against which to match *word* (typically a list of strings).

Optional argument *n* (default `3`) is the maximum number of close matches to return; *n* must be greater than `0`.

Optional argument *cutoff* (default `0.6`) is a float in the range \[0, 1\]. Possibilities that don't score at least that similar to *word* are ignored.

Setting the optional *autojunk* argument to `False` will turn `automatic junk heuristic <difflib-junk>` off.

The best (no more than *n*) matches among the possibilities are returned in a list, sorted by similarity score, most similar first.

> \>\>\> get_close_matches('appel', \['ape', 'apple', 'peach', 'puppy'\]) \['apple', 'ape'\] \>\>\> import keyword \>\>\> get_close_matches('wheel', keyword.kwlist) \['while'\] \>\>\> get_close_matches('pineapple', keyword.kwlist) \[\] \>\>\> get_close_matches('accept', keyword.kwlist) \['except'\]

<div class="versionchanged">

3.16 Added keyword-only *autojunk* parameter.

</div>

</div>

<div class="function">

ndiff(a, b, linejunk=None, charjunk=IS_CHARACTER_JUNK, \*, autojunk=True)

Compare *a* and *b* (lists of strings); return a `Differ`-style delta (a `generator` generating the delta lines).

Optional keyword parameters *linejunk* and *charjunk* are filtering functions (or `None`):

*linejunk*: A function that accepts a single string argument, and returns true if the string is junk, or false if not. The default is `None`. There is also a module-level function `IS_LINE_JUNK`, which filters out lines without visible characters, except for at most one hash character (`'#'`) -- however the underlying `SequenceMatcher` class does a dynamic analysis of which lines are so frequent as to constitute noise, and this usually works better than using this function.

*charjunk*: A function that accepts a character (a string of length 1), and returns if the character is junk, or false if not. The default is module-level function `IS_CHARACTER_JUNK`, which filters out whitespace characters (a blank or tab; it's a bad idea to include newline in this!).

Setting the optional *autojunk* argument to `False` will turn `automatic junk heuristic <difflib-junk>` off.

Example:

> \>\>\> diff = ndiff('onentwonthreen'.splitlines(keepends=True), ... 'orentreenemun'.splitlines(keepends=True)) \>\>\> print(''.join(diff), end="") - one ? ^ + ore ? ^ - two - three ? - + tree + emu

<div class="versionchanged">

3.16 Added keyword-only *autojunk* parameter.

</div>

</div>

<div class="function">

restore(sequence, which)

Return one of the two sequences that generated a delta.

Given a *sequence* produced by `Differ.compare` or `ndiff`, extract lines originating from file 1 or 2 (parameter *which*), stripping off line prefixes.

Example:

> \>\>\> diff = ndiff('onentwonthreen'.splitlines(keepends=True), ... 'orentreenemun'.splitlines(keepends=True)) \>\>\> diff = list(diff) \# materialize the generated delta into a list \>\>\> print(''.join(restore(diff, 1)), end="") one two three \>\>\> print(''.join(restore(diff, 2)), end="") ore tree emu

</div>

<div class="function">

unified_diff(a, b, fromfile='', tofile='', fromfiledate='', tofiledate='', n=3, lineterm='n', \*, autojunk=True, color=False)

Compare *a* and *b* (lists of strings); return a delta (a `generator` generating the delta lines) in unified diff format.

Unified diffs are a compact way of showing just the lines that have changed plus a few lines of context. The changes are shown in an inline style (instead of separate before/after blocks). The number of context lines is set by *n* which defaults to three.

By default, the diff control lines (those with `---`, `+++`, or `@@`) are created with a trailing newline. This is helpful so that inputs created from `io.IOBase.readlines` result in diffs that are suitable for use with `io.IOBase.writelines` since both the inputs and outputs have trailing newlines.

For inputs that do not have trailing newlines, set the *lineterm* argument to `""` so that the output will be uniformly newline free.

Set *color* to `True` to enable output in color, similar to `git diff --color`. Even if enabled, it can be `controlled using environment variables <using-on-controlling-color>`.

The unified diff format normally has a header for filenames and modification times. Any or all of these may be specified using strings for *fromfile*, *tofile*, *fromfiledate*, and *tofiledate*. The modification times are normally expressed in the ISO 8601 format. If not specified, the strings default to blanks.

> \>\>\> s1 = \['baconn', 'eggsn', 'hamn', 'guidon'\] \>\>\> s2 = \['pythonn', 'eggyn', 'hamstern', 'guidon'\] \>\>\> sys.stdout.writelines(unified_diff(s1, s2, fromfile='before.py', tofile='after.py')) --- before.py +++ after.py @@ -1,4 +1,4 @@ -bacon -eggs -ham +python +eggy +hamster guido

See `difflib-interface` for a more detailed example.

<div class="versionchanged">

3.15 Added the *color* parameter.

</div>

Setting the optional *autojunk* argument to `False` will turn `automatic junk heuristic <difflib-junk>` off.

<div class="versionchanged">

3.16 Added keyword-only *autojunk* parameter.

</div>

</div>

<div class="function">

diff_bytes(dfunc, a, b, fromfile=b'', tofile=b'', fromfiledate=b'', tofiledate=b'', n=3, lineterm=b'n')

Compare *a* and *b* (lists of bytes objects) using *dfunc*; yield a sequence of delta lines (also bytes) in the format returned by *dfunc*. *dfunc* must be a callable, typically either `unified_diff` or `context_diff`.

Allows you to compare data with unknown or inconsistent encoding. All inputs except *n* must be bytes objects, not str. Works by losslessly converting all inputs (except *n*) to str, and calling `dfunc(a, b, fromfile, tofile, fromfiledate, tofiledate, n, lineterm)`. The output of *dfunc* is then converted back to bytes, so the delta lines that you receive have the same unknown/inconsistent encodings as *a* and *b*.

<div class="versionadded">

3.5

</div>

</div>

## Junk definition functions

<div class="function">

IS_LINE_JUNK(line)

Return `True` for ignorable lines. The line *line* is ignorable if *line* is blank or contains a single `'#'`, otherwise it is not ignorable. Used as a default for parameter *linejunk* in `ndiff` in older versions.

</div>

<div class="function">

IS_CHARACTER_JUNK(ch)

Return `True` for ignorable characters. The character *ch* is ignorable if *ch* is a space or tab, otherwise it is not ignorable. Used as a default for parameter *charjunk* in `ndiff`.

</div>

## SequenceMatcher objects

<div class="SequenceMatcher(isjunk=None, a='', b='', autojunk=True)">

Optional argument *isjunk* must be `None` (the default) or a one-argument function that takes a sequence element and returns true if and only if the element is "junk" and should be ignored. Passing `None` for *isjunk* is equivalent to passing `lambda x: False`; in other words, no elements are ignored. For example, pass:

    lambda x: x in " \t"

if you're comparing lines as sequences of characters, and don't want to synch up on blanks or hard tabs.

The optional arguments *a* and *b* are sequences to be compared; both default to empty strings. The elements of both sequences must be `hashable`.

The optional argument *autojunk* can be used to disable the automatic junk heuristic.

<div class="versionchanged">

3.2 Added the *autojunk* parameter.

</div>

SequenceMatcher objects get three data attributes: *bjunk* is the set of elements of *b* for which *isjunk* is `True`; *bpopular* is the set of non-junk elements considered popular by the heuristic (if it is not disabled); *b2j* is a dict mapping the remaining elements of *b* to a list of positions where they occur. All three are reset whenever *b* is reset with `set_seqs` or `set_seq2`.

<div class="versionadded">

3.2 The *bjunk* and *bpopular* attributes.

</div>

`SequenceMatcher` objects have the following methods:

<div class="method">

set_seqs(a, b)

Set the two sequences to be compared.

</div>

`SequenceMatcher` computes and caches detailed information about the second sequence, so if you want to compare one sequence against many sequences, use `set_seq2` to set the commonly used sequence once and call `set_seq1` repeatedly, once for each of the other sequences.

<div class="method">

set_seq1(a)

Set the first sequence to be compared. The second sequence to be compared is not changed.

</div>

<div class="method">

set_seq2(b)

Set the second sequence to be compared. The first sequence to be compared is not changed.

</div>

<div class="method">

find_longest_match(alo=0, ahi=None, blo=0, bhi=None)

Find longest matching block in `a[alo:ahi]` and `b[blo:bhi]`.

If *isjunk* was omitted or `None`, `find_longest_match` returns `(i, j, k)` such that `a[i:i+k]` is equal to `b[j:j+k]`, where `alo <= i <= i+k <= ahi` and `blo <= j <= j+k <= bhi`. For all `(i', j', k')` meeting those conditions, the additional conditions `k >= k'`, `i <= i'`, and if `i == i'`, `j <= j'` are also met. In other words, of all maximal matching blocks, return one that starts earliest in *a*, and of all those maximal matching blocks that start earliest in *a*, return the one that starts earliest in *b*.

> \>\>\> s = SequenceMatcher(None, " abcd", "abcd abcd") \>\>\> s.find_longest_match(0, 5, 0, 9) Match(a=0, b=4, size=5)

If *isjunk* was provided, first the longest matching block is determined as above, but with the additional restriction that no junk element appears in the block. Then that block is extended as far as possible by matching (only) junk elements on both sides. So the resulting block never matches on junk except as identical junk happens to be adjacent to an interesting match.

Here's the same example as before, but considering blanks to be junk. That prevents `' abcd'` from matching the `' abcd'` at the tail end of the second sequence directly. Instead only the `'abcd'` can match, and matches the leftmost `'abcd'` in the second sequence:

> \>\>\> s = SequenceMatcher(lambda x: x==" ", " abcd", "abcd abcd") \>\>\> s.find_longest_match(0, 5, 0, 9) Match(a=1, b=0, size=4)

If no blocks match, this returns `(alo, blo, 0)`.

This method returns a `named tuple` `Match(a, b, size)`.

<div class="versionchanged">

3.9 Added default arguments.

</div>

</div>

<div class="method">

get_matching_blocks()

Return list of triples describing non-overlapping matching subsequences. Each triple is of the form `(i, j, n)`, and means that `a[i:i+n] == b[j:j+n]`. The triples are monotonically increasing in *i* and *j*.

The last triple is a dummy, and has the value `(len(a), len(b), 0)`. It is the only triple with `n == 0`. If `(i, j, n)` and `(i', j', n')` are adjacent triples in the list, and the second is not the last triple in the list, then `i+n < i'` or `j+n < j'`; in other words, adjacent triples always describe non-adjacent equal blocks.

<div class="doctest">

\>\>\> s = SequenceMatcher(None, "abxcd", "abcd") \>\>\> s.get_matching_blocks() \[Match(a=0, b=0, size=2), Match(a=3, b=2, size=2), Match(a=5, b=4, size=0)\]

</div>

</div>

<div class="method">

get_opcodes()

Return list of 5-tuples describing how to turn *a* into *b*. Each tuple is of the form `(tag, i1, i2, j1, j2)`. The first tuple has `i1 == j1 == 0`, and remaining tuples have *i1* equal to the *i2* from the preceding tuple, and, likewise, *j1* equal to the previous *j2*.

The *tag* values are strings, with these meanings:

| Value | Meaning |
|----|----|
| `'replace'` | `a[i1:i2]` should be replaced by `b[j1:j2]`. |
| `'delete'` | `a[i1:i2]` should be deleted. Note that `j1 == j2` in this case. |
| `'insert'` | `b[j1:j2]` should be inserted at `a[i1:i1]`. Note that `i1 == i2` in this case. |
| `'equal'` | `a[i1:i2] == b[j1:j2]` (the sub-sequences are equal). |

For example:

    >>> a = "qabxcd"
    >>> b = "abycdf"
    >>> s = SequenceMatcher(None, a, b)
    >>> for tag, i1, i2, j1, j2 in s.get_opcodes():
    ...     print('{:7}   a[{}:{}] --> b[{}:{}] {!r:>8} --> {!r}'.format(
    ...         tag, i1, i2, j1, j2, a[i1:i2], b[j1:j2]))
    delete    a[0:1] --> b[0:0]      'q' --> ''
    equal     a[1:3] --> b[0:2]     'ab' --> 'ab'
    replace   a[3:4] --> b[2:3]      'x' --> 'y'
    equal     a[4:6] --> b[3:5]     'cd' --> 'cd'
    insert    a[6:6] --> b[5:6]       '' --> 'f'

</div>

<div class="method">

get_grouped_opcodes(n=3)

Return a `generator` of groups with up to *n* lines of context.

Starting with the groups returned by `get_opcodes`, this method splits out smaller change clusters and eliminates intervening ranges which have no changes.

The groups are returned in the same format as `get_opcodes`.

</div>

<div class="method">

ratio()

Return a measure of the sequences' similarity as a float in the range \[0, 1\].

Where T is the total number of elements in both sequences, and M is the number of matches, this is 2.0\*M / T. Note that this is `1.0` if the sequences are identical, and `0.0` if they have nothing in common.

This is expensive to compute if `get_matching_blocks` or `get_opcodes` hasn't already been called, in which case you may want to try `quick_ratio` or `real_quick_ratio` first to get an upper bound.

</div>

<div class="method">

quick_ratio()

Return an upper bound on `ratio` relatively quickly.

</div>

<div class="method">

real_quick_ratio()

Return an upper bound on `ratio` very quickly.

</div>

</div>

The three methods that return the ratio of matching to total characters can give different results due to differing levels of approximation, although `~SequenceMatcher.quick_ratio` and `~SequenceMatcher.real_quick_ratio` are always at least as large as `~SequenceMatcher.ratio`:

> \>\>\> s = SequenceMatcher(None, "abcd", "bcde") \>\>\> s.ratio() 0.75 \>\>\> s.quick_ratio() 0.75 \>\>\> s.real_quick_ratio() 1.0

## Examples

### SequenceMatcher examples

This example compares two strings, considering blanks to be "junk":

> \>\>\> s = SequenceMatcher(lambda x: x == " ", ... "private Thread currentThread;", ... "private volatile Thread currentThread;")

`~SequenceMatcher.ratio` returns a float in \[0, 1\], measuring the similarity of the sequences. As a rule of thumb, a `~SequenceMatcher.ratio` value over 0.6 means the sequences are close matches:

> \>\>\> print(round(s.ratio(), 3)) 0.866

If you're only interested in where the sequences match, `~SequenceMatcher.get_matching_blocks` is handy:

> \>\>\> for block in s.get_matching_blocks(): ... print("a\[%d\] and b\[%d\] match for %d elements" % block) a\[0\] and b\[0\] match for 8 elements a\[8\] and b\[17\] match for 21 elements a\[29\] and b\[38\] match for 0 elements

Note that the last tuple returned by `~SequenceMatcher.get_matching_blocks` is always a dummy, `(len(a), len(b), 0)`, and this is the only case in which the last tuple element (number of elements matched) is `0`.

If you want to know how to change the first sequence into the second, use `~SequenceMatcher.get_opcodes`:

> \>\>\> for opcode in s.get_opcodes(): ... print("%6s a\[%d:%d\] b\[%d:%d\]" % opcode) equal a\[0:8\] b\[0:8\] insert a\[8:8\] b\[8:17\] equal a\[8:29\] b\[17:38\]

<div class="seealso">

- The `get_close_matches` function in this module which shows how simple code building on `SequenceMatcher` can be used to do useful work.
- [Simple version control recipe](https://code.activestate.com/recipes/576729-simple-version-control/) for a small application built with `SequenceMatcher`.

</div>

### Differ example

This example compares two texts. First we set up the texts, sequences of individual single-line strings ending with newlines (such sequences can also be obtained from the `~io.IOBase.readlines` method of file-like objects):

> \>\>\> text1 = ''' 1. Beautiful is better than ugly. ... 2. Explicit is better than implicit. ... 3. Simple is better than complex. ... 4. Complex is better than complicated. ... '''.splitlines(keepends=True) \>\>\> len(text1) 4 \>\>\> text1\[0\]\[-1\] 'n' \>\>\> text2 = ''' 1. Beautiful is better than ugly. ... 3. Simple is better than complex. ... 4. Complicated is better than complex. ... 5. Flat is better than nested. ... '''.splitlines(keepends=True)

Next we instantiate a Differ object:

> \>\>\> d = Differ()

Note that when instantiating a `Differ` object we may pass functions to filter out line and character "junk." See the `Differ` constructor for details.

Finally, we compare the two:

> \>\>\> result = list(d.compare(text1, text2))

`result` is a list of strings, so let's pretty-print it:

    >>> from pprint import pprint
    >>> pprint(result)
    ['    1. Beautiful is better than ugly.\n',
     '-   2. Explicit is better than implicit.\n',
     '-   3. Simple is better than complex.\n',
     '+   3.   Simple is better than complex.\n',
     '?     ++\n',
     '-   4. Complex is better than complicated.\n',
     '?            ^                     ---- ^\n',
     '+   4. Complicated is better than complex.\n',
     '?           ++++ ^                      ^\n',
     '+   5. Flat is better than nested.\n']

As a single multi-line string it looks like this:

    >>> import sys
    >>> sys.stdout.writelines(result)
        1. Beautiful is better than ugly.
    -   2. Explicit is better than implicit.
    -   3. Simple is better than complex.
    +   3.   Simple is better than complex.
    ?     ++
    -   4. Complex is better than complicated.
    ?            ^                     ---- ^
    +   4. Complicated is better than complex.
    ?           ++++ ^                      ^
    +   5. Flat is better than nested.

### A command-line interface to difflib

This example shows how to use difflib to create a `diff`-like utility.

<div class="literalinclude">

../includes/diff.py

</div>

### ndiff example

This example shows how to use `difflib.ndiff`.

<div class="literalinclude">

../includes/ndiff.py

</div>

# `!dis` --- Disassembler for Python bytecode

<div class="module" data-synopsis="Disassembler for Python bytecode.">

dis

</div>

**Source code:** `Lib/dis.py`

<div class="testsetup">

import dis def myfunc(alist): return len(alist)

</div>

------------------------------------------------------------------------

The `!dis` module supports the analysis of CPython `bytecode` by disassembling it. The CPython bytecode which this module takes as an input is defined in the file `Include/opcode.h` and used by the compiler and the interpreter.

<div class="impl-detail">

Bytecode is an implementation detail of the CPython interpreter. No guarantees are made that bytecode will not be added, removed, or changed between versions of Python. Use of this module should not be considered to work across Python VMs or Python releases.

<div class="versionchanged">

3.6 Use 2 bytes for each instruction. Previously the number of bytes varied by instruction.

</div>

<div class="versionchanged">

3.10 The argument of jump, exception handling and loop instructions is now the instruction offset rather than the byte offset.

</div>

<div class="versionchanged">

3.11 Some instructions are accompanied by one or more inline cache entries, which take the form of `CACHE` instructions. These instructions are hidden by default, but can be shown by passing `show_caches=True` to any `!dis` utility. Furthermore, the interpreter now adapts the bytecode to specialize it for different runtime conditions. The adaptive bytecode can be shown by passing `adaptive=True`.

</div>

<div class="versionchanged">

3.12 The argument of a jump is the offset of the target instruction relative to the instruction that appears immediately after the jump instruction's `CACHE` entries.

As a consequence, the presence of the `CACHE` instructions is transparent for forward jumps but needs to be taken into account when reasoning about backward jumps.

</div>

<div class="versionchanged">

3.13 The output shows logical labels rather than instruction offsets for jump targets and exception handlers. The `-O` command line option and the `show_offsets` argument were added.

</div>

<div class="versionchanged">

3.14 The `-P <dis --show-positions>` command-line option and the `show_positions` argument were added.

The `-S <dis --specialized>` command-line option is added.

</div>

</div>

Example: Given the function `!myfunc`:

    def myfunc(alist):
        return len(alist)

the following command can be used to display the disassembly of `!myfunc`:

<div class="doctest">

\>\>\> dis.dis(myfunc)  
2 RESUME 0

\<BLANKLINE\>  
3 LOAD_GLOBAL 1 (len + NULL)  
LOAD_FAST_BORROW 0 (alist) CALL 1 RETURN_VALUE

</div>

(The "2" is a line number).

## Command-line interface

The `!dis` module can be invoked as a script from the command line:

``` sh
python -m dis [-h] [-C] [-O] [-P] [-S] [infile]
```

The following options are accepted:

<div class="program">

dis

</div>

<div class="option">

`-h, --help`  

Display usage and exit.

</div>

<div class="option">

`-C, --show-caches`  

Show inline caches.

<div class="versionadded">

3.13

</div>

</div>

<div class="option">

`-O, --show-offsets`  

Show offsets of instructions.

<div class="versionadded">

3.13

</div>

</div>

<div class="option">

`-P, --show-positions`  

Show positions of instructions in the source code.

<div class="versionadded">

3.14

</div>

</div>

<div class="option">

`-S, --specialized`  

Show specialized bytecode.

<div class="versionadded">

3.14

</div>

</div>

If `infile` is specified, its disassembled code will be written to stdout. Otherwise, disassembly is performed on compiled source code received from stdin.

## Bytecode analysis

<div class="versionadded">

3.4

</div>

The bytecode analysis API allows pieces of Python code to be wrapped in a `Bytecode` object that provides easy access to details of the compiled code.

<div class="Bytecode(x, *, first_line=None, current_offset=None,\ show_caches=False, adaptive=False, show_offsets=False,\ show_positions=False, show_jit=False)">

Analyse the bytecode corresponding to a function, generator, asynchronous generator, coroutine, method, string of source code, or a code object (as returned by `compile`).

This is a convenience wrapper around many of the functions listed below, most notably `get_instructions`, as iterating over a `Bytecode` instance yields the bytecode operations as `Instruction` instances.

If *first_line* is not `None`, it indicates the line number that should be reported for the first source line in the disassembled code. Otherwise, the source line information (if any) is taken directly from the disassembled code object.

If *current_offset* is not `None`, it refers to an instruction offset in the disassembled code. Setting this means `.dis` will display a "current instruction" marker against the specified opcode.

If *show_caches* is `True`, `.dis` will display inline cache entries used by the interpreter to specialize the bytecode.

If *adaptive* is `True`, `.dis` will display specialized bytecode that may be different from the original bytecode.

If *show_offsets* is `True`, `.dis` will include instruction offsets in the output.

If *show_positions* is `True`, `.dis` will include instruction source code positions in the output.

If *show_jit* is `True`, `.dis` will show `ENTER_EXECUTOR` instructions, which mark JIT entry points and are hidden by default.

<div class="classmethod">

from_traceback(tb, \*, show_caches=False)

Construct a `Bytecode` instance from the given traceback, setting *current_offset* to the instruction responsible for the exception.

</div>

<div class="data">

codeobj

The compiled code object.

</div>

<div class="data">

first_line

The first source line of the code object (if available)

</div>

<div class="method">

dis()

Return a formatted view of the bytecode operations (the same as printed by `dis.dis`, but returned as a multi-line string).

</div>

<div class="method">

info()

Return a formatted multi-line string with detailed information about the code object, like `code_info`.

</div>

<div class="versionchanged">

3.7 This can now handle coroutine and asynchronous generator objects.

</div>

<div class="versionchanged">

3.11 Added the *show_caches* and *adaptive* parameters.

</div>

<div class="versionchanged">

3.13 Added the *show_offsets* parameter

</div>

<div class="versionchanged">

3.14 Added the *show_positions* parameter.

</div>

<div class="versionchanged">

3.16 Added the *show_jit* parameter.

</div>

</div>

Example:

<div class="doctest">

\>\>\> bytecode = dis.Bytecode(myfunc) \>\>\> for instr in bytecode: ... print(instr.opname) ... RESUME LOAD_GLOBAL LOAD_FAST_BORROW CALL RETURN_VALUE

</div>

## Analysis functions

The `!dis` module also defines the following analysis functions that convert the input directly to the desired output. They can be useful if only a single operation is being performed, so the intermediate analysis object isn't useful:

<div class="function">

code_info(x)

Return a formatted multi-line string with detailed code object information for the supplied function, generator, asynchronous generator, coroutine, method, source code string or code object.

Note that the exact contents of code info strings are highly implementation dependent and they may change arbitrarily across Python VMs or Python releases.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.7 This can now handle coroutine and asynchronous generator objects.

</div>

</div>

<div class="function">

show_code(x, \*, file=None)

Print detailed code object information for the supplied function, method, source code string or code object to *file* (or `sys.stdout` if *file* is not specified).

This is a convenient shorthand for `print(code_info(x), file=file)`, intended for interactive exploration at the interpreter prompt.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.4 Added *file* parameter.

</div>

</div>

<div class="function">

dis(x=None, \*, file=None, depth=None, show_caches=False,adaptive=False, show_offsets=False, show_positions=False,show_jit=False)

Disassemble the *x* object. *x* can denote either a module, a class, a method, a function, a generator, an asynchronous generator, a coroutine, a code object, a string of source code or a byte sequence of raw bytecode. For a module, it disassembles all functions. For a class, it disassembles all methods (including class and static methods). For a code object or sequence of raw bytecode, it prints one line per bytecode instruction. It also recursively disassembles nested code objects. These can include generator expressions, nested functions, the bodies of nested classes, and the code objects used for `annotation scopes <annotation-scopes>`. Strings are first compiled to code objects with the `compile` built-in function before being disassembled. If no object is provided, this function disassembles the last traceback.

The disassembly is written as text to the supplied *file* argument if provided and to `sys.stdout` otherwise.

The maximal depth of recursion is limited by *depth* unless it is `None`. `depth=0` means no recursion.

If *show_caches* is `True`, this function will display inline cache entries used by the interpreter to specialize the bytecode.

If *adaptive* is `True`, this function will display specialized bytecode that may be different from the original bytecode.

If *show_jit* is `True`, this function will show `ENTER_EXECUTOR` instructions, which mark JIT entry points and are hidden by default.

<div class="versionchanged">

3.4 Added *file* parameter.

</div>

<div class="versionchanged">

3.7 Implemented recursive disassembling and added *depth* parameter.

</div>

<div class="versionchanged">

3.7 This can now handle coroutine and asynchronous generator objects.

</div>

<div class="versionchanged">

3.11 Added the *show_caches* and *adaptive* parameters.

</div>

<div class="versionchanged">

3.13 Added the *show_offsets* parameter.

</div>

<div class="versionchanged">

3.14 Added the *show_positions* parameter.

</div>

<div class="versionchanged">

3.16 Added the *show_jit* parameter.

</div>

</div>

<div class="function">

distb(tb=None, \*, file=None, show_caches=False, adaptive=False,show_offsets=False, show_positions=False, show_jit=False)

Disassemble the top-of-stack function of a traceback, using the last traceback if none was passed. The instruction causing the exception is indicated.

The disassembly is written as text to the supplied *file* argument if provided and to `sys.stdout` otherwise.

<div class="versionchanged">

3.4 Added *file* parameter.

</div>

<div class="versionchanged">

3.11 Added the *show_caches* and *adaptive* parameters.

</div>

<div class="versionchanged">

3.13 Added the *show_offsets* parameter.

</div>

<div class="versionchanged">

3.14 Added the *show_positions* parameter.

</div>

<div class="versionchanged">

3.16 Added the *show_jit* parameter.

</div>

</div>

<div class="function">

disassemble(code, lasti=-1, *, file=None, show_caches=False,adaptive=False, show_offsets=False, show_positions=False,show_jit=False) disco(code, lasti=-1,*, file=None, show_caches=False, adaptive=False,show_offsets=False, show_positions=False, show_jit=False)

Disassemble a code object, indicating the last instruction if *lasti* was provided. The output is divided in the following columns:

1.  the source code location of the instruction. Complete location information is shown if *show_positions* is true. Otherwise (the default) only the line number is displayed.
2.  the current instruction, indicated as `-->`,
3.  a labelled instruction, indicated with `>>`,
4.  the address of the instruction,
5.  the operation code name,
6.  operation parameters, and
7.  interpretation of the parameters in parentheses.

The parameter interpretation recognizes local and global variable names, constant values, branch targets, and compare operators.

The disassembly is written as text to the supplied *file* argument if provided and to `sys.stdout` otherwise.

<div class="versionchanged">

3.4 Added *file* parameter.

</div>

<div class="versionchanged">

3.11 Added the *show_caches* and *adaptive* parameters.

</div>

<div class="versionchanged">

3.13 Added the *show_offsets* parameter.

</div>

<div class="versionchanged">

3.14 Added the *show_positions* parameter.

</div>

<div class="versionchanged">

3.16 Added the *show_jit* parameter.

</div>

</div>

<div class="function">

get_instructions(x, \*, first_line=None, show_caches=False, adaptive=False, show_jit=False)

Return an iterator over the instructions in the supplied function, method, source code string or code object.

The iterator generates a series of `Instruction` named tuples giving the details of each operation in the supplied code.

If *first_line* is not `None`, it indicates the line number that should be reported for the first source line in the disassembled code. Otherwise, the source line information (if any) is taken directly from the disassembled code object.

The *adaptive* parameter works as it does in `~dis.dis`.

The *show_jit* parameter works as it does in `~dis.dis`.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.11 Added the *show_caches* and *adaptive* parameters.

</div>

<div class="versionchanged">

3.13 The *show_caches* parameter is deprecated and has no effect. The iterator generates the `Instruction` instances with the *cache_info* field populated (regardless of the value of *show_caches*) and it no longer generates separate items for the cache entries.

</div>

<div class="versionchanged">

3.16 Added the *show_jit* parameter.

</div>

</div>

<div class="function">

findlinestarts(code)

This generator function uses the `~codeobject.co_lines` method of the `code object <code-objects>` *code* to find the offsets which are starts of lines in the source code. They are generated as `(offset, lineno)` pairs.

<div class="versionchanged">

3.6 Line numbers can be decreasing. Before, they were always increasing.

</div>

<div class="versionchanged">

3.10 The `626` `~codeobject.co_lines` method is used instead of the `~codeobject.co_firstlineno` and `!codeobject.co_lnotab` attributes of the `code object <code-objects>`.

</div>

<div class="versionchanged">

3.13 Line numbers can be `None` for bytecode that does not map to source lines.

</div>

</div>

<div class="function">

findlabels(code)

Detect all offsets in the raw compiled bytecode string *code* which are jump targets, and return a list of these offsets.

</div>

<div class="function">

stack_effect(opcode, oparg=None, \*, jump=None)

Compute the stack effect of *opcode* with argument *oparg*.

If the code has a jump target and *jump* is `True`, `~stack_effect` will return the stack effect of jumping. If *jump* is `False`, it will return the stack effect of not jumping. And if *jump* is `None` (default), it will return the maximal stack effect of both cases.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.8 Added *jump* parameter.

</div>

<div class="versionchanged">

3.13 If `oparg` is omitted (or `None`), the stack effect is now returned for `oparg=0`. Previously this was an error for opcodes that use their arg. It is also no longer an error to pass an integer `oparg` when the `opcode` does not use it; the `oparg` in this case is ignored.

</div>

</div>

## Python Bytecode Instructions

The `get_instructions` function and `Bytecode` class provide details of bytecode instructions as `Instruction` instances:

<div class="Instruction">

Details for a bytecode operation

<div class="data">

opcode

numeric code for operation, corresponding to the opcode values listed below and the bytecode values in the `opcode_collections`.

</div>

<div class="data">

opname

human readable name for operation

</div>

<div class="data">

baseopcode

numeric code for the base operation if operation is specialized; otherwise equal to `opcode`

</div>

<div class="data">

baseopname

human readable name for the base operation if operation is specialized; otherwise equal to `opname`

</div>

<div class="data">

arg

numeric argument to operation (if any), otherwise `None`

</div>

<div class="data">

oparg

alias for `arg`

</div>

<div class="data">

argval

resolved arg value (if any), otherwise `None`

</div>

<div class="data">

argrepr

human readable description of operation argument (if any), otherwise an empty string.

</div>

<div class="data">

offset

start index of operation within bytecode sequence

</div>

<div class="data">

start_offset

start index of operation within bytecode sequence, including prefixed `EXTENDED_ARG` operations if present; otherwise equal to `offset`

</div>

<div class="data">

cache_offset

start index of the cache entries following the operation

</div>

<div class="data">

end_offset

end index of the cache entries following the operation

</div>

<div class="data">

starts_line

`True` if this opcode starts a source line, otherwise `False`

</div>

<div class="data">

line_number

source line number associated with this opcode (if any), otherwise `None`

</div>

<div class="data">

is_jump_target

`True` if other code jumps to here, otherwise `False`

</div>

<div class="data">

jump_target

bytecode index of the jump target if this is a jump operation, otherwise `None`

</div>

<div class="data">

positions

`dis.Positions` object holding the start and end locations that are covered by this instruction.

</div>

<div class="data">

cache_info

Information about the cache entries of this instruction, as triplets of the form `(name, size, data)`, where the `name` and `size` describe the cache format and data is the contents of the cache. `cache_info` is `None` if the instruction does not have caches.

</div>

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.11

Field `positions` is added.

</div>

<div class="versionchanged">

3.13

Changed field `starts_line`.

Added fields `start_offset`, `cache_offset`, `end_offset`, `baseopname`, `baseopcode`, `jump_target`, `oparg`, `line_number` and `cache_info`.

</div>

</div>

<div class="Positions">

In case the information is not available, some fields might be `None`.

<div class="data">

lineno

</div>

<div class="data">

end_lineno

</div>

<div class="data">

col_offset

</div>

<div class="data">

end_col_offset

</div>

<div class="versionadded">

3.11

</div>

</div>

The Python compiler currently generates the following bytecode instructions.

**General instructions**

In the following, We will refer to the interpreter stack as `STACK` and describe operations on it as if it was a Python list. The top of the stack corresponds to `STACK[-1]` in this language.

<div class="opcode">

NOP

Do nothing code. Used as a placeholder by the bytecode optimizer, and to generate line tracing events.

</div>

<div class="opcode">

NOT_TAKEN

Do nothing code. Used by the interpreter to record `BRANCH_LEFT` and `BRANCH_RIGHT` events for `sys.monitoring`.

<div class="versionadded">

3.14

</div>

</div>

<div class="opcode">

POP_ITER

Removes the iterator from the top of the stack.

<div class="versionadded">

3.14

</div>

</div>

<div class="opcode">

POP_TOP

Removes the top-of-stack item:

    STACK.pop()

</div>

<div class="opcode">

END_FOR

Removes the top-of-stack item. Equivalent to `POP_TOP`. Used to clean up at the end of loops, hence the name.

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

END_SEND

Implements `del STACK[-2]`. Used to clean up when a generator exits.

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

COPY (i)

Push the i-th item to the top of the stack without removing it from its original location:

    assert i > 0
    STACK.append(STACK[-i])

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

SWAP (i)

Swap the top of the stack with the i-th element:

    STACK[-i], STACK[-1] = STACK[-1], STACK[-i]

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

CACHE

Rather than being an actual instruction, this opcode is used to mark extra space for the interpreter to cache useful data directly in the bytecode itself. It is automatically hidden by all `dis` utilities, but can be viewed with `show_caches=True`.

Logically, this space is part of the preceding instruction. Many opcodes expect to be followed by an exact number of caches, and will instruct the interpreter to skip over them at runtime.

Populated caches can look like arbitrary instructions, so great care should be taken when reading or modifying raw, adaptive bytecode containing quickened data.

<div class="versionadded">

3.11

</div>

</div>

**Unary operations**

Unary operations take the top of the stack, apply the operation, and push the result back on the stack.

<div class="opcode">

UNARY_NEGATIVE

Implements `STACK[-1] = -STACK[-1]`.

</div>

<div class="opcode">

UNARY_NOT

Implements `STACK[-1] = not STACK[-1]`.

<div class="versionchanged">

3.13 This instruction now requires an exact `bool` operand.

</div>

</div>

<div class="opcode">

UNARY_INVERT

Implements `STACK[-1] = ~STACK[-1]`.

</div>

<div class="opcode">

GET_ITER

Implements `STACK[-1] = iter(STACK[-1])`.

</div>

<div class="opcode">

GET_YIELD_FROM_ITER

If `STACK[-1]` is a `generator iterator` or `coroutine` object it is left as is. Otherwise, implements `STACK[-1] = iter(STACK[-1])`.

<div class="versionadded">

3.5

</div>

</div>

<div class="opcode">

TO_BOOL

Implements `STACK[-1] = bool(STACK[-1])`.

<div class="versionadded">

3.13

</div>

</div>

**Binary and in-place operations**

Binary operations remove the top two items from the stack (`STACK[-1]` and `STACK[-2]`). They perform the operation, then put the result back on the stack.

In-place operations are like binary operations, but the operation is done in-place when `STACK[-2]` supports it, and the resulting `STACK[-1]` may be (but does not have to be) the original `STACK[-2]`.

<div class="opcode">

BINARY_OP (op)

Implements the binary and in-place operators (depending on the value of *op*):

    rhs = STACK.pop()
    lhs = STACK.pop()
    STACK.append(lhs op rhs)

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.14 With oparg :`NB_SUBSCR`, implements binary subscript (replaces opcode `BINARY_SUBSCR`)

</div>

</div>

<div class="opcode">

STORE_SUBSCR

Implements:

    key = STACK.pop()
    container = STACK.pop()
    value = STACK.pop()
    container[key] = value

</div>

<div class="opcode">

DELETE_SUBSCR

Implements:

    key = STACK.pop()
    container = STACK.pop()
    del container[key]

</div>

<div class="opcode">

BINARY_SLICE

Implements:

    end = STACK.pop()
    start = STACK.pop()
    container = STACK.pop()
    STACK.append(container[start:end])

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

STORE_SLICE

Implements:

    end = STACK.pop()
    start = STACK.pop()
    container = STACK.pop()
    value = STACK.pop()
    container[start:end] = value

<div class="versionadded">

3.12

</div>

</div>

**Coroutine opcodes**

<div class="opcode">

GET_AWAITABLE (where)

Implements `STACK[-1] = get_awaitable(STACK[-1])`, where `get_awaitable(o)` returns `o` if `o` is a coroutine object or a generator object with the `~inspect.CO_ITERABLE_COROUTINE` flag, or resolves `o.__await__`.

> If the `where` operand is nonzero, it indicates where the instruction occurs:
>
> - `1`: After a call to `__aenter__`
> - `2`: After a call to `__aexit__`

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.11 Previously, this instruction did not have an oparg.

</div>

</div>

<div class="opcode">

GET_AITER

Implements `STACK[-1] = STACK[-1].__aiter__()`.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.7 Returning awaitable objects from `__aiter__` is no longer supported.

</div>

</div>

<div class="opcode">

GET_ANEXT

Implement `STACK.append(get_awaitable(STACK[-1].__anext__()))` to the stack. See `GET_AWAITABLE` for details about `get_awaitable`.

<div class="versionadded">

3.5

</div>

</div>

<div class="opcode">

END_ASYNC_FOR

Terminates an `async for` loop. Handles an exception raised when awaiting a next item. The stack contains the async iterable in `STACK[-2]` and the raised exception in `STACK[-1]`. Both are popped. If the exception is not `StopAsyncIteration`, it is re-raised.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.11 Exception representation on the stack now consist of one, not three, items.

</div>

</div>

<div class="opcode">

CLEANUP_THROW

Handles an exception raised during a `~generator.throw` or `~generator.close` call through the current frame. If `STACK[-1]` is an instance of `StopIteration`, pop three values from the stack and push its `value` member. Otherwise, re-raise `STACK[-1]`.

<div class="versionadded">

3.12

</div>

</div>

**Miscellaneous opcodes**

<div class="opcode">

SET_ADD (i)

Implements:

    item = STACK.pop()
    set.add(STACK[-i], item)

Used to implement set comprehensions.

</div>

<div class="opcode">

LIST_APPEND (i)

Implements:

    item = STACK.pop()
    list.append(STACK[-i], item)

Used to implement list comprehensions.

</div>

<div class="opcode">

MAP_ADD (i)

Implements:

    value = STACK.pop()
    key = STACK.pop()
    dict.__setitem__(STACK[-i], key, value)

Used to implement dict comprehensions.

<div class="versionadded">

3.1

</div>

<div class="versionchanged">

3.8 Map value is `STACK[-1]` and map key is `STACK[-2]`. Before, those were reversed.

</div>

</div>

For all of the `SET_ADD`, `LIST_APPEND` and `MAP_ADD` instructions, while the added value or key/value pair is popped off, the container object remains on the stack so that it is available for further iterations of the loop.

<div class="opcode">

RETURN_VALUE

Returns with `STACK[-1]` to the caller of the function.

</div>

<div class="opcode">

YIELD_VALUE

Yields `STACK.pop()` from a `generator`.

<div class="versionchanged">

3.11 oparg set to be the stack depth.

</div>

<div class="versionchanged">

3.12 oparg set to be the exception block depth, for efficient closing of generators.

</div>

<div class="versionchanged">

3.13 oparg is `1` if this instruction is part of a yield-from or await, and `0` otherwise.

</div>

</div>

<div class="opcode">

SETUP_ANNOTATIONS

Checks whether `__annotations__` is defined in `locals()`, if not it is set up to an empty `dict`. This opcode is only emitted if a class or module body contains `variable annotations <variable annotation>` statically.

<div class="versionadded">

3.6

</div>

</div>

<div class="opcode">

POP_EXCEPT

Pops a value from the stack, which is used to restore the exception state.

<div class="versionchanged">

3.11 Exception representation on the stack now consist of one, not three, items.

</div>

</div>

<div class="opcode">

RERAISE

Re-raises the exception currently on top of the stack. If oparg is non-zero, pops an additional value from the stack which is used to set `~frame.f_lasti` of the current frame.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.11 Exception representation on the stack now consist of one, not three, items.

</div>

</div>

<div class="opcode">

PUSH_EXC_INFO

Pops a value from the stack. Pushes the current exception to the top of the stack. Pushes the value originally popped back to the stack. Used in exception handlers.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

CHECK_EXC_MATCH

Performs exception matching for `except`. Tests whether the `STACK[-2]` is an exception matching `STACK[-1]`. Pops `STACK[-1]` and pushes the boolean result of the test.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

CHECK_EG_MATCH

Performs exception matching for `except*`. Applies `split(STACK[-1])` on the exception group representing `STACK[-2]`.

In case of a match, pops two items from the stack and pushes the non-matching subgroup (`None` in case of full match) followed by the matching subgroup. When there is no match, pops one item (the match type) and pushes `None`.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

WITH_EXCEPT_START

Calls the function in position 4 on the stack with arguments (type, val, tb) representing the exception at the top of the stack. Used to implement the call `context_manager.__exit__(*exc_info())` when an exception has occurred in a `with` statement.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.11 The `__exit__` function is in position 4 of the stack rather than 7. Exception representation on the stack now consist of one, not three, items.

</div>

</div>

<div class="opcode">

LOAD_COMMON_CONSTANT

Pushes a common constant onto the stack. The interpreter contains a hardcoded list of constants supported by this instruction. Used by the `assert` statement to load `AssertionError`.

<div class="versionadded">

3.14

</div>

</div>

<div class="opcode">

LOAD_BUILD_CLASS

Pushes `!builtins.__build_class__` onto the stack. It is later called to construct a class.

</div>

<div class="opcode">

GET_LEN

Perform `STACK.append(len(STACK[-1]))`. Used in `match` statements where comparison with structure of pattern is needed.

<div class="versionadded">

3.10

</div>

</div>

<div class="opcode">

MATCH_MAPPING

If `STACK[-1]` is an instance of `collections.abc.Mapping` (or, more technically: if it has the `Py_TPFLAGS_MAPPING` flag set in its `~PyTypeObject.tp_flags`), push `True` onto the stack. Otherwise, push `False`.

<div class="versionadded">

3.10

</div>

</div>

<div class="opcode">

MATCH_SEQUENCE

If `STACK[-1]` is an instance of `collections.abc.Sequence` and is *not* an instance of `str`/`bytes`/`bytearray` (or, more technically: if it has the `Py_TPFLAGS_SEQUENCE` flag set in its `~PyTypeObject.tp_flags`), push `True` onto the stack. Otherwise, push `False`.

<div class="versionadded">

3.10

</div>

</div>

<div class="opcode">

MATCH_KEYS

`STACK[-1]` is a tuple of mapping keys, and `STACK[-2]` is the match subject. If `STACK[-2]` contains all of the keys in `STACK[-1]`, push a `tuple` containing the corresponding values. Otherwise, push `None`.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.11 Previously, this instruction also pushed a boolean value indicating success (`True`) or failure (`False`).

</div>

</div>

<div class="opcode">

STORE_NAME (namei)

Implements `name = STACK.pop()`. *namei* is the index of *name* in the attribute `~codeobject.co_names` of the `code object <code-objects>`. The compiler tries to use `STORE_FAST` or `STORE_GLOBAL` if possible.

</div>

<div class="opcode">

UNPACK_SEQUENCE (count)

Unpacks `STACK[-1]` into *count* individual values, which are put onto the stack right-to-left. Require there to be exactly *count* values.:

    assert(len(STACK[-1]) == count)
    STACK.extend(STACK.pop()[:-count-1:-1])

</div>

<div class="opcode">

UNPACK_EX (counts)

Implements assignment with a starred target: Unpacks an iterable in `STACK[-1]` into individual values, where the total number of values can be smaller than the number of items in the iterable: one of the new values will be a list of all leftover items.

The number of values before and after the list value is limited to 255.

The number of values before the list value is encoded in the argument of the opcode. The number of values after the list if any is encoded using an `EXTENDED_ARG`. As a consequence, the argument can be seen as a two bytes values where the low byte of *counts* is the number of values before the list value, the high byte of *counts* the number of values after it.

The extracted values are put onto the stack right-to-left, i.e. `a, *b, c = d` will be stored after execution as `STACK.extend((a, b, c))`.

</div>

<div class="opcode">

STORE_ATTR (namei)

Implements:

    obj = STACK.pop()
    value = STACK.pop()
    obj.name = value

where *namei* is the index of name in `~codeobject.co_names` of the `code object <code-objects>`.

</div>

<div class="opcode">

STORE_GLOBAL (namei)

Works as `STORE_NAME`, but stores the name as a global.

</div>

<div class="opcode">

LOAD_CONST (consti)

Pushes `co_consts[consti]` onto the stack.

</div>

<div class="opcode">

LOAD_SMALL_INT (i)

Pushes the integer `i` onto the stack. `i` must be in `range(256)`

<div class="versionadded">

3.14

</div>

</div>

<div class="opcode">

LOAD_NAME (namei)

Pushes the value associated with `co_names[namei]` onto the stack. The name is looked up within the locals, then the globals, then the builtins.

</div>

<div class="opcode">

LOAD_LOCALS

Pushes a reference to the locals dictionary onto the stack. This is used to prepare namespace dictionaries for `LOAD_FROM_DICT_OR_DEREF` and `LOAD_FROM_DICT_OR_GLOBALS`.

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

LOAD_FROM_DICT_OR_GLOBALS (i)

Pops a mapping off the stack and looks up the value for `co_names[namei]`. If the name is not found there, looks it up in the globals and then the builtins, similar to `LOAD_GLOBAL`. This is used for loading global variables in `annotation scopes <annotation-scopes>` within class bodies.

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

BUILD_TEMPLATE

Constructs a new `~string.templatelib.Template` instance from a tuple of strings and a tuple of interpolations and pushes the resulting object onto the stack:

    interpolations = STACK.pop()
    strings = STACK.pop()
    STACK.append(_build_template(strings, interpolations))

<div class="versionadded">

3.14

</div>

</div>

<div class="opcode">

BUILD_INTERPOLATION (format)

Constructs a new `~string.templatelib.Interpolation` instance from a value and its source expression and pushes the resulting object onto the stack.

If no conversion or format specification is present, `format` is set to `2`.

If the low bit of `format` is set, it indicates that the interpolation contains a format specification.

If `format >> 2` is non-zero, it indicates that the interpolation contains a conversion. The value of `format >> 2` is the conversion type (`0` for no conversion, `1` for `!s`, `2` for `!r`, and `3` for `!a`):

    conversion = format >> 2
    if format & 1:
        format_spec = STACK.pop()
    else:
        format_spec = None
    expression = STACK.pop()
    value = STACK.pop()
    STACK.append(_build_interpolation(value, expression, conversion, format_spec))

<div class="versionadded">

3.14

</div>

</div>

<div class="opcode">

BUILD_TUPLE (count)

Creates a tuple consuming *count* items from the stack, and pushes the resulting tuple onto the stack:

    if count == 0:
        value = ()
    else:
        value = tuple(STACK[-count:])
        STACK = STACK[:-count]

    STACK.append(value)

</div>

<div class="opcode">

BUILD_LIST (count)

Works as `BUILD_TUPLE`, but creates a list.

</div>

<div class="opcode">

BUILD_SET (count)

Works as `BUILD_TUPLE`, but creates a set.

</div>

<div class="opcode">

BUILD_MAP (count)

Pushes a new dictionary object onto the stack. Pops `2 * count` items so that the dictionary holds *count* entries: `{..., STACK[-4]: STACK[-3], STACK[-2]: STACK[-1]}`.

<div class="versionchanged">

3.5 The dictionary is created from stack items instead of creating an empty dictionary pre-sized to hold *count* items.

</div>

</div>

<div class="opcode">

BUILD_STRING (count)

Concatenates *count* strings from the stack and pushes the resulting string onto the stack.

<div class="versionadded">

3.6

</div>

</div>

<div class="opcode">

LIST_EXTEND (i)

Implements:

    seq = STACK.pop()
    list.extend(STACK[-i], seq)

Used to build lists.

<div class="versionadded">

3.9

</div>

</div>

<div class="opcode">

SET_UPDATE (i)

Implements:

    seq = STACK.pop()
    set.update(STACK[-i], seq)

Used to build sets.

<div class="versionadded">

3.9

</div>

</div>

<div class="opcode">

DICT_UPDATE (i)

Implements:

    map = STACK.pop()
    dict.update(STACK[-i], map)

Used to build dicts.

<div class="versionadded">

3.9

</div>

</div>

<div class="opcode">

DICT_MERGE (i)

Like `DICT_UPDATE` but raises an exception for duplicate keys.

<div class="versionadded">

3.9

</div>

</div>

<div class="opcode">

LOAD_ATTR (namei)

If the low bit of `namei` is not set, this replaces `STACK[-1]` with `getattr(STACK[-1], co_names[namei>>1])`.

If the low bit of `namei` is set, this will attempt to load a method named `co_names[namei>>1]` from the `STACK[-1]` object. `STACK[-1]` is popped. This bytecode distinguishes two cases: if `STACK[-1]` has a method with the correct name, the bytecode pushes the unbound method and `STACK[-1]`. `STACK[-1]` will be used as the first argument (`self`) by `CALL` or `CALL_KW` when calling the unbound method. Otherwise, `NULL` and the object returned by the attribute lookup are pushed.

<div class="versionchanged">

3.12 If the low bit of `namei` is set, then a `NULL` or `self` is pushed to the stack before the attribute or unbound method respectively.

</div>

</div>

<div class="opcode">

LOAD_SUPER_ATTR (namei)

This opcode implements `super`, both in its zero-argument and two-argument forms (e.g. `super().method()`, `super().attr` and `super(cls, self).method()`, `super(cls, self).attr`).

It pops three values from the stack (from top of stack down):

- `self`: the first argument to the current method
- `cls`: the class within which the current method was defined
- the global `super`

With respect to its argument, it works similarly to `LOAD_ATTR`, except that `namei` is shifted left by 2 bits instead of 1.

The low bit of `namei` signals to attempt a method load, as with `LOAD_ATTR`, which results in pushing `NULL` and the loaded method. When it is unset a single value is pushed to the stack.

The second-low bit of `namei`, if set, means that this was a two-argument call to `super` (unset means zero-argument).

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

COMPARE_OP (opname)

Performs a Boolean operation. The operation name can be found in `cmp_op[opname >> 5]`. If the fifth-lowest bit of `opname` is set (`opname & 16`), the result should be coerced to `bool`.

<div class="versionchanged">

3.13 The fifth-lowest bit of the oparg now indicates a forced conversion to `bool`.

</div>

</div>

<div class="opcode">

IS_OP (invert)

Performs `is` comparison, or `is not` if `invert` is 1.

<div class="versionadded">

3.9

</div>

</div>

<div class="opcode">

CONTAINS_OP (invert)

Performs `in` comparison, or `not in` if `invert` is 1.

<div class="versionadded">

3.9

</div>

</div>

<div class="opcode">

IMPORT_NAME (namei)

Imports the module `co_names[namei]`. `STACK[-1]` and `STACK[-2]` are popped and provide the *fromlist* and *level* arguments of `__import__`. The module object is pushed onto the stack. The current namespace is not affected: for a proper import statement, a subsequent `STORE_FAST` instruction modifies the namespace.

</div>

<div class="opcode">

IMPORT_FROM (namei)

Loads the attribute `co_names[namei]` from the module found in `STACK[-1]`. The resulting object is pushed onto the stack, to be subsequently stored by a `STORE_FAST` instruction.

</div>

<div class="opcode">

JUMP_FORWARD (delta)

Increments bytecode counter by *delta*.

</div>

<div class="opcode">

JUMP_BACKWARD (delta)

Decrements bytecode counter by *delta*. Checks for interrupts.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

JUMP_BACKWARD_NO_INTERRUPT (delta)

Decrements bytecode counter by *delta*. Does not check for interrupts.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

POP_JUMP_IF_TRUE (delta)

If `STACK[-1]` is true, increments the bytecode counter by *delta*. `STACK[-1]` is popped.

<div class="versionchanged">

3.11 The oparg is now a relative delta rather than an absolute target. This opcode is a pseudo-instruction, replaced in final bytecode by the directed versions (forward/backward).

</div>

<div class="versionchanged">

3.12 This is no longer a pseudo-instruction.

</div>

<div class="versionchanged">

3.13 This instruction now requires an exact `bool` operand.

</div>

</div>

<div class="opcode">

POP_JUMP_IF_FALSE (delta)

If `STACK[-1]` is false, increments the bytecode counter by *delta*. `STACK[-1]` is popped.

<div class="versionchanged">

3.11 The oparg is now a relative delta rather than an absolute target. This opcode is a pseudo-instruction, replaced in final bytecode by the directed versions (forward/backward).

</div>

<div class="versionchanged">

3.12 This is no longer a pseudo-instruction.

</div>

<div class="versionchanged">

3.13 This instruction now requires an exact `bool` operand.

</div>

</div>

<div class="opcode">

POP_JUMP_IF_NOT_NONE (delta)

If `STACK[-1]` is not `None`, increments the bytecode counter by *delta*. `STACK[-1]` is popped.

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.12 This is no longer a pseudo-instruction.

</div>

</div>

<div class="opcode">

POP_JUMP_IF_NONE (delta)

If `STACK[-1]` is `None`, increments the bytecode counter by *delta*. `STACK[-1]` is popped.

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.12 This is no longer a pseudo-instruction.

</div>

</div>

<div class="opcode">

FOR_ITER (delta)

`STACK[-1]` is an `iterator`. Call its `~iterator.__next__` method. If this yields a new value, push it on the stack (leaving the iterator below it). If the iterator indicates it is `exhausted` then the byte code counter is incremented by *delta*.

<div class="versionchanged">

3.12 Up until 3.11 the iterator was popped when it was exhausted.

</div>

</div>

<div class="opcode">

LOAD_GLOBAL (namei)

Loads the global named `co_names[namei>>1]` onto the stack. If the low bit of `namei` is set, a `NULL` is pushed onto the stack after the global variable.

<div class="versionchanged">

3.11 The low bit of `namei` indicates whether a `NULL` is pushed.

</div>

<div class="versionchanged">

3.13 The `NULL` is pushed after the global variable, rather than before it.

</div>

</div>

<div class="opcode">

LOAD_FAST (var_num)

Pushes a reference to the local `co_varnames[var_num]` onto the stack.

<div class="versionchanged">

3.12 This opcode is now only used in situations where the local variable is guaranteed to be initialized. It cannot raise `UnboundLocalError`.

</div>

</div>

<div class="opcode">

LOAD_FAST_BORROW (var_num)

Pushes a borrowed reference to the local `co_varnames[var_num]` onto the stack.

<div class="versionadded">

3.14

</div>

</div>

<div class="opcode">

LOAD_FAST_LOAD_FAST (var_nums)

Pushes references to `co_varnames[var_nums >> 4]` and `co_varnames[var_nums & 15]` onto the stack.

<div class="versionadded">

3.13

</div>

</div>

<div class="opcode">

LOAD_FAST_BORROW_LOAD_FAST_BORROW (var_nums)

Pushes borrowed references to `co_varnames[var_nums >> 4]` and `co_varnames[var_nums & 15]` onto the stack.

<div class="versionadded">

3.14

</div>

</div>

<div class="opcode">

LOAD_FAST_CHECK (var_num)

Pushes a reference to the local `co_varnames[var_num]` onto the stack, raising an `UnboundLocalError` if the local variable has not been initialized.

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

LOAD_FAST_AND_CLEAR (var_num)

Pushes a reference to the local `co_varnames[var_num]` onto the stack (or pushes `NULL` onto the stack if the local variable has not been initialized) and sets `co_varnames[var_num]` to `NULL`.

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

STORE_FAST (var_num)

Stores `STACK.pop()` into the local `co_varnames[var_num]`.

</div>

<div class="opcode">

STORE_FAST_STORE_FAST (var_nums)

Stores `STACK[-1]` into `co_varnames[var_nums >> 4]` and `STACK[-2]` into `co_varnames[var_nums & 15]`.

<div class="versionadded">

3.13

</div>

</div>

<div class="opcode">

STORE_FAST_LOAD_FAST (var_nums)

Stores `STACK.pop()` into the local `co_varnames[var_nums >> 4]` and pushes a reference to the local `co_varnames[var_nums & 15]` onto the stack.

<div class="versionadded">

3.13

</div>

</div>

<div class="opcode">

DELETE_FAST (var_num)

Deletes local `co_varnames[var_num]`.

</div>

<div class="opcode">

MAKE_CELL (i)

Creates a new cell in slot `i`. If that slot is nonempty then that value is stored into the new cell.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

LOAD_DEREF (i)

Loads the cell contained in slot `i` of the "fast locals" storage. Pushes a reference to the object the cell contains on the stack.

<div class="versionchanged">

3.11 `i` is no longer offset by the length of `~codeobject.co_varnames`.

</div>

</div>

<div class="opcode">

LOAD_FROM_DICT_OR_DEREF (i)

Pops a mapping off the stack and looks up the name associated with slot `i` of the "fast locals" storage in this mapping. If the name is not found there, loads it from the cell contained in slot `i`, similar to `LOAD_DEREF`. This is used for loading `closure variables <closure variable>` in class bodies (which previously used `!LOAD_CLASSDEREF`) and in `annotation scopes <annotation-scopes>` within class bodies.

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

STORE_DEREF (i)

Stores `STACK.pop()` into the cell contained in slot `i` of the "fast locals" storage.

<div class="versionchanged">

3.11 `i` is no longer offset by the length of `~codeobject.co_varnames`.

</div>

</div>

<div class="opcode">

DELETE_DEREF (i)

Empties the cell contained in slot `i` of the "fast locals" storage. Used by the `del` statement.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.11 `i` is no longer offset by the length of `~codeobject.co_varnames`.

</div>

</div>

<div class="opcode">

COPY_FREE_VARS (n)

Copies the `n` `free (closure) variables <closure variable>` from the closure into the frame. Removes the need for special code on the caller's side when calling closures.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

RAISE_VARARGS (argc)

Raises an exception using one of the 3 forms of the `raise` statement, depending on the value of *argc*:

- 0: `raise` (re-raise previous exception)
- 1: `raise STACK[-1]` (raise exception instance or type at `STACK[-1]`)
- 2: `raise STACK[-2] from STACK[-1]` (raise exception instance or type at `STACK[-2]` with `__cause__` set to `STACK[-1]`)

</div>

<div class="opcode">

CALL (argc)

Calls a callable object with the number of arguments specified by `argc`. On the stack are (in ascending order):

- The callable
- `self` or `NULL`
- The remaining positional arguments

`argc` is the total of the positional arguments, excluding `self`.

`CALL` pops all arguments and the callable object off the stack, calls the callable object with those arguments, and pushes the return value returned by the callable object.

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.13 The callable now always appears at the same position on the stack.

</div>

<div class="versionchanged">

3.13 Calls with keyword arguments are now handled by `CALL_KW`.

</div>

</div>

<div class="opcode">

CALL_KW (argc)

Calls a callable object with the number of arguments specified by `argc`, including one or more named arguments. On the stack are (in ascending order):

- The callable
- `self` or `NULL`
- The remaining positional arguments
- The named arguments
- A `tuple` of keyword argument names

`argc` is the total of the positional and named arguments, excluding `self`. The length of the tuple of keyword argument names is the number of named arguments.

`CALL_KW` pops all arguments, the keyword names, and the callable object off the stack, calls the callable object with those arguments, and pushes the return value returned by the callable object.

<div class="versionadded">

3.13

</div>

</div>

<div class="opcode">

CALL_FUNCTION_EX (flags)

Calls a callable object with variable set of positional and keyword arguments. If the lowest bit of *flags* is set, the top of the stack contains a mapping object containing additional keyword arguments. Before the callable is called, the mapping object and iterable object are each "unpacked" and their contents passed in as keyword and positional arguments respectively. `CALL_FUNCTION_EX` pops all arguments and the callable object off the stack, calls the callable object with those arguments, and pushes the return value returned by the callable object.

<div class="versionadded">

3.6

</div>

</div>

<div class="opcode">

PUSH_NULL

Pushes a `NULL` to the stack. Used in the call sequence to match the `NULL` pushed by `LOAD_ATTR` for non-method calls.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

MAKE_FUNCTION

Pushes a new function object on the stack built from the code object at `STACK[-1]`.

<div class="versionchanged">

3.10 Flag value `0x04` is a tuple of strings instead of dictionary

</div>

<div class="versionchanged">

3.11 Qualified name at `STACK[-1]` was removed.

</div>

<div class="versionchanged">

3.13 Extra function attributes on the stack, signaled by oparg flags, were removed. They now use `SET_FUNCTION_ATTRIBUTE`.

</div>

</div>

<div class="opcode">

SET_FUNCTION_ATTRIBUTE (flag)

Sets an attribute on a function object. Expects the function at `STACK[-1]` and the attribute value to set at `STACK[-2]`; consumes both and leaves the function at `STACK[-1]`. The flag determines which attribute to set:

- `0x01` a tuple of default values for positional-only and positional-or-keyword parameters in positional order
- `0x02` a dictionary of keyword-only parameters' default values
- `0x04` a tuple of strings containing parameters' annotations
- `0x08` a tuple containing cells for free variables, making a closure
- `0x10` the `annotate function` for the function object

<div class="versionadded">

3.13

</div>

<div class="versionchanged">

3.14 Added `0x10` to indicate the annotate function for the function object.

</div>

</div>

<div class="opcode">

BUILD_SLICE (argc)

<div class="index">

pair: built-in function; slice

</div>

Pushes a slice object on the stack. *argc* must be 2 or 3. If it is 2, implements:

    end = STACK.pop()
    start = STACK.pop()
    STACK.append(slice(start, end))

if it is 3, implements:

    step = STACK.pop()
    end = STACK.pop()
    start = STACK.pop()
    STACK.append(slice(start, end, step))

See the `slice` built-in function for more information.

</div>

<div class="opcode">

EXTENDED_ARG (ext)

Prefixes any opcode which has an argument too big to fit into the default one byte. *ext* holds an additional byte which act as higher bits in the argument. For each opcode, at most three prefixal `EXTENDED_ARG` are allowed, forming an argument from two-byte to four-byte.

</div>

<div class="opcode">

CONVERT_VALUE (oparg)

Convert value to a string, depending on `oparg`:

    value = STACK.pop()
    result = func(value)
    STACK.append(result)

- `oparg == 1`: call `str` on *value*
- `oparg == 2`: call `repr` on *value*
- `oparg == 3`: call `ascii` on *value*

Used for implementing formatted string literals (f-strings).

<div class="versionadded">

3.13

</div>

</div>

<div class="opcode">

FORMAT_SIMPLE

Formats the value on top of stack:

    value = STACK.pop()
    result = value.__format__("")
    STACK.append(result)

Used for implementing formatted string literals (f-strings).

<div class="versionadded">

3.13

</div>

</div>

<div class="opcode">

FORMAT_WITH_SPEC

Formats the given value with the given format spec:

    spec = STACK.pop()
    value = STACK.pop()
    result = value.__format__(spec)
    STACK.append(result)

Used for implementing formatted string literals (f-strings).

<div class="versionadded">

3.13

</div>

</div>

<div class="opcode">

MATCH_CLASS (count)

`STACK[-1]` is a tuple of keyword attribute names, `STACK[-2]` is the class being matched against, and `STACK[-3]` is the match subject. *count* is the number of positional sub-patterns.

Pop `STACK[-1]`, `STACK[-2]`, and `STACK[-3]`. If `STACK[-3]` is an instance of `STACK[-2]` and has the positional and keyword attributes required by *count* and `STACK[-1]`, push a tuple of extracted attributes. Otherwise, push `None`.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.11 Previously, this instruction also pushed a boolean value indicating success (`True`) or failure (`False`).

</div>

</div>

<div class="opcode">

RESUME (context)

A no-op. Performs internal tracing, debugging and optimization checks.

The `context` operand consists of two parts. The lowest two bits indicate where the `RESUME` occurs:

- `0` The start of a function, which is neither a generator, coroutine nor an async generator
- `1` After a `yield` expression
- `2` After a `yield from` expression
- `3` After an `await` expression

The next bit is `1` if the RESUME is at except-depth `1`, and `0` otherwise.

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.13 The oparg value changed to include information about except-depth

</div>

</div>

<div class="opcode">

RETURN_GENERATOR

Create a generator, coroutine, or async generator from the current frame. Used as first opcode of in code object for the above mentioned callables. Clear the current frame and return the newly created generator.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

SEND (delta)

Equivalent to `STACK[-1] = STACK[-2].send(STACK[-1])`. Used in `yield from` and `await` statements.

If the call raises `StopIteration`, pop the top value from the stack, push the exception's `value` attribute, and increment the bytecode counter by *delta*.

<div class="versionadded">

3.11

</div>

</div>

<div class="opcode">

HAVE_ARGUMENT

This is not really an opcode. It identifies the dividing line between opcodes in the range \[0,255\] which don't use their argument and those that do (`< HAVE_ARGUMENT` and `>= HAVE_ARGUMENT`, respectively).

If your application uses pseudo instructions or specialized instructions, use the `hasarg` collection instead.

<div class="versionchanged">

3.6 Now every instruction has an argument, but opcodes `< HAVE_ARGUMENT` ignore it. Before, only opcodes `>= HAVE_ARGUMENT` had an argument.

</div>

<div class="versionchanged">

3.12 Pseudo instructions were added to the `!dis` module, and for them it is not true that comparison with `HAVE_ARGUMENT` indicates whether they use their arg.

</div>

<div class="deprecated">

3.13 Use `hasarg` instead.

</div>

</div>

<div class="opcode">

CALL_INTRINSIC_1

Calls an intrinsic function with one argument. Passes `STACK[-1]` as the argument and sets `STACK[-1]` to the result. Used to implement functionality that is not performance critical.

The operand determines which intrinsic function is called:

| Operand | Description |
|----|----|
| `INTRINSIC_1_INVALID` | Not valid |
| `INTRINSIC_PRINT` | Prints the argument to standard out. Used in the REPL. |
| `INTRINSIC_IMPORT_STAR` | Performs `import *` for the named module. |
| `INTRINSIC_STOPITERATION_ERROR` | Extracts the return value from a `StopIteration` exception. |
| `INTRINSIC_ASYNC_GEN_WRAP` | Wraps an async generator value |
| `INTRINSIC_UNARY_POSITIVE` | Performs the unary `+` operation |
| `INTRINSIC_LIST_TO_TUPLE` | Converts a list to a tuple |
| `INTRINSIC_TYPEVAR` | Creates a `typing.TypeVar` |
| `INTRINSIC_PARAMSPEC` | Creates a `typing.ParamSpec` |
| `INTRINSIC_TYPEVARTUPLE` | Creates a `typing.TypeVarTuple` |
| `INTRINSIC_SUBSCRIPT_GENERIC` | Returns `typing.Generic` subscripted with the argument |
| `INTRINSIC_TYPEALIAS` | Creates a `typing.TypeAliasType`; used in the `type` statement. The argument is a tuple of the type alias's name, type parameters, and value. |

<div class="versionadded">

3.12

</div>

</div>

<div class="opcode">

CALL_INTRINSIC_2

Calls an intrinsic function with two arguments. Used to implement functionality that is not performance critical:

    arg2 = STACK.pop()
    arg1 = STACK.pop()
    result = intrinsic2(arg1, arg2)
    STACK.append(result)

The operand determines which intrinsic function is called:

| Operand | Description |
|----|----|
| `INTRINSIC_2_INVALID` | Not valid |
| `INTRINSIC_PREP_RERAISE_STAR` | Calculates the `ExceptionGroup` to raise from a `try-except*`. |
| `INTRINSIC_TYPEVAR_WITH_BOUND` | Creates a `typing.TypeVar` with a bound. |
| `INTRINSIC_TYPEVAR_WITH_CONSTRAINTS` | Creates a `typing.TypeVar` with constraints. |
| `INTRINSIC_SET_FUNCTION_TYPE_PARAMS` | Sets the `__type_params__` attribute of a function. |
| `INTRINSIC_ADD_CONDITIONAL_ANNOTATION` | Adds an annotation index to the `__conditional_annotations__` set. |
| `INTRINSIC_MATCH_CLASS_ISINSTANCE` | Do `isinstance` checks for `Class patterns <class-patterns>`. |

<div class="versionadded">

3.12

</div>

<div class="versionchanged">

3.16 Added `INTRINSIC_MATCH_CLASS_ISINSTANCE`.

</div>

</div>

<div class="opcode">

LOAD_SPECIAL

Performs special method lookup on `STACK[-1]`. If `type(STACK[-1]).__xxx__` is a method, leave `type(STACK[-1]).__xxx__; STACK[-1]` on the stack. If `type(STACK[-1]).__xxx__` is not a method, leave `STACK[-1].__xxx__; NULL` on the stack.

<div class="versionadded">

3.14

</div>

</div>

**Pseudo-instructions**

These opcodes do not appear in Python bytecode. They are used by the compiler but are replaced by real opcodes or removed before bytecode is generated.

<div class="opcode">

SETUP_FINALLY (target)

Set up an exception handler for the following code block. If an exception occurs, the value stack level is restored to its current state and control is transferred to the exception handler at `target`.

</div>

<div class="opcode">

SETUP_CLEANUP (target)

Like `SETUP_FINALLY`, but in case of an exception also pushes the last instruction (`lasti`) to the stack so that `RERAISE` can restore it. If an exception occurs, the value stack level and the last instruction on the frame are restored to their current state, and control is transferred to the exception handler at `target`.

</div>

<div class="opcode">

SETUP_WITH (target)

Like `SETUP_CLEANUP`, but in case of an exception one more item is popped from the stack before control is transferred to the exception handler at `target`.

This variant is used in `with` and `async with` constructs, which push the return value of the context manager's `~object.__enter__` or `~object.__aenter__` to the stack.

</div>

<div class="opcode">

POP_BLOCK

Marks the end of the code block associated with the last `SETUP_FINALLY`, `SETUP_CLEANUP` or `SETUP_WITH`.

</div>

<div class="opcode">

JUMP JUMP_NO_INTERRUPT

Undirected relative jump instructions which are replaced by their directed (forward/backward) counterparts by the assembler.

</div>

<div class="opcode">

JUMP_IF_TRUE JUMP_IF_FALSE

Conditional jumps which do not impact the stack. Replaced by the sequence `COPY 1`, `TO_BOOL`, `POP_JUMP_IF_TRUE/FALSE`.

</div>

<div class="opcode">

LOAD_CLOSURE (i)

Pushes a reference to the cell contained in slot `i` of the "fast locals" storage.

Note that `LOAD_CLOSURE` is replaced with `LOAD_FAST` in the assembler.

<div class="versionchanged">

3.13 This opcode is now a pseudo-instruction.

</div>

</div>

## Opcode collections

These collections are provided for automatic introspection of bytecode instructions:

<div class="versionchanged">

3.12 The collections now contain pseudo instructions and instrumented instructions as well. These are opcodes with values `>= MIN_PSEUDO_OPCODE` and `>= MIN_INSTRUMENTED_OPCODE`.

</div>

<div class="data">

opname

Sequence of operation names, indexable using the bytecode.

</div>

<div class="data">

opmap

Dictionary mapping operation names to bytecodes.

</div>

<div class="data">

cmp_op

Sequence of all compare operation names.

</div>

<div class="data">

hasarg

Sequence of bytecodes that use their argument.

<div class="versionadded">

3.12

</div>

</div>

<div class="data">

hasconst

Sequence of bytecodes that access a constant.

</div>

<div class="data">

hasfree

Sequence of bytecodes that access a `free (closure) variable <closure variable>`. 'free' in this context refers to names in the current scope that are referenced by inner scopes or names in outer scopes that are referenced from this scope. It does *not* include references to global or builtin scopes.

</div>

<div class="data">

hasname

Sequence of bytecodes that access an attribute by name.

</div>

<div class="data">

hasjump

Sequence of bytecodes that have a jump target. All jumps are relative.

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

haslocal

Sequence of bytecodes that access a local variable.

</div>

<div class="data">

hascompare

Sequence of bytecodes of Boolean operations.

</div>

<div class="data">

hasexc

Sequence of bytecodes that set an exception handler.

<div class="versionadded">

3.12

</div>

</div>

<div class="data">

hasjrel

Sequence of bytecodes that have a relative jump target.

<div class="deprecated">

3.13 All jumps are now relative. Use `hasjump`.

</div>

</div>

<div class="data">

hasjabs

Sequence of bytecodes that have an absolute jump target.

<div class="deprecated">

3.13 All jumps are now relative. This list is empty.

</div>

</div>

# Software Packaging and Distribution

These libraries help you with publishing and installing Python software. While these modules are designed to work in conjunction with the [Python Package Index](https://pypi.org), they can also be used with a local index server, or without any index server at all.

<div class="toctree">

ensurepip.rst venv.rst zipapp.rst

</div>

# `!distutils` --- Building and installing Python modules

<div class="module" data-synopsis="Removed in 3.12." data-deprecated="">

distutils

</div>

<div class="deprecated-removed">

3.10 3.12

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.12 <whatsnew312-removed-distutils>` after being deprecated in Python 3.10. The removal was decided in `632`, which has [migration advice](https://peps.python.org/pep-0632/#migration-advice).

The last version of Python that provided the `!distutils` module was [Python 3.11](https://docs.python.org/3.11/library/distutils.html).

# `!doctest` --- Test interactive Python examples

<div class="module" data-synopsis="Test pieces of code within docstrings.">

doctest

</div>

**Source code:** `Lib/doctest.py`

------------------------------------------------------------------------

The `!doctest` module searches for pieces of text that look like interactive Python sessions, and then executes those sessions to verify that they work exactly as shown. There are several common ways to use doctest:

- To check that a module's docstrings are up-to-date by verifying that all interactive examples still work as documented.
- To perform regression testing by verifying that interactive examples from a test file or a test object work as expected.
- To write tutorial documentation for a package, liberally illustrated with input-output examples. Depending on whether the examples or the expository text are emphasized, this has the flavor of "literate testing" or "executable documentation".

Here's a complete but small example module:

    """
    This is the "example" module.

    The example module supplies one function, factorial().  For example,

    >>> factorial(5)
    120
    """

    def factorial(n):
        """Return the factorial of n, an exact integer >= 0.

        >>> [factorial(n) for n in range(6)]
        [1, 1, 2, 6, 24, 120]
        >>> factorial(30)
        265252859812191058636308480000000
        >>> factorial(-1)
        Traceback (most recent call last):
            ...
        ValueError: n must be >= 0

        Factorials of floats are OK, but the float must be an exact integer:
        >>> factorial(30.1)
        Traceback (most recent call last):
            ...
        ValueError: n must be exact integer
        >>> factorial(30.0)
        265252859812191058636308480000000

        It must also not be ridiculously large:
        >>> factorial(1e100)
        Traceback (most recent call last):
            ...
        OverflowError: n too large
        """

        import math
        if not n >= 0:
            raise ValueError("n must be >= 0")
        if math.floor(n) != n:
            raise ValueError("n must be exact integer")
        if n+1 == n:  # catch a value like 1e300
            raise OverflowError("n too large")
        result = 1
        factor = 2
        while factor <= n:
            result *= factor
            factor += 1
        return result


    if __name__ == "__main__":
        import doctest
        doctest.testmod()

If you run `example.py` directly from the command line, `!doctest` works its magic:

```
$ python example.py
$
```

There's no output! That's normal, and it means all the examples worked. Pass `-v` to the script, and `!doctest` prints a detailed log of what it's trying, and prints a summary at the end:

```
$ python example.py -v
Trying:
    factorial(5)
Expecting:
    120
ok
Trying:
    [factorial(n) for n in range(6)]
Expecting:
    [1, 1, 2, 6, 24, 120]
ok
```

And so on, eventually ending with:

```
Trying:
    factorial(1e100)
Expecting:
    Traceback (most recent call last):
        ...
    OverflowError: n too large
ok
2 items passed all tests:
   1 test in __main__
   6 tests in __main__.factorial
7 tests in 2 items.
7 passed.
Test passed.
$
```

That's all you need to know to start making productive use of `!doctest`! Jump in. The following sections provide full details. Note that there are many examples of doctests in the standard Python test suite and libraries. Especially useful examples can be found in the standard test file `Lib/test/test_doctest/test_doctest.py`.

<div class="versionadded">

3.13 Output is colorized by default and can be `controlled using environment variables <using-on-controlling-color>`.

</div>

## Simple Usage: Checking Examples in Docstrings

The simplest way to start using doctest (but not necessarily the way you'll continue to do it) is to end each module `!M` with:

    if __name__ == "__main__":
        import doctest
        doctest.testmod()

`!doctest` then examines docstrings in module `!M`.

Running the module as a script causes the examples in the docstrings to get executed and verified:

    python M.py

This won't display anything unless an example fails, in which case the failing example(s) and the cause(s) of the failure(s) are printed to stdout, and the final line of output is `***Test Failed*** N failures.`, where *N* is the number of examples that failed.

Run it with the `-v` switch instead:

    python M.py -v

and a detailed report of all examples tried is printed to standard output, along with assorted summaries at the end.

You can force verbose mode by passing `verbose=True` to `testmod`, or prohibit it by passing `verbose=False`. In either of those cases, `sys.argv` is not examined by `testmod` (so passing `-v` or not has no effect).

There is also a command line shortcut for running `testmod`, see section `doctest-cli`.

For more information on `testmod`, see section `doctest-basic-api`.

## Simple Usage: Checking Examples in a Text File

Another simple application of doctest is testing interactive examples in a text file. This can be done with the `testfile` function:

    import doctest
    doctest.testfile("example.txt")

That short script executes and verifies any interactive Python examples contained in the file `example.txt`. The file content is treated as if it were a single giant docstring; the file doesn't need to contain a Python program! For example, perhaps `example.txt` contains this:

```
The ``example`` module
======================

Using ``factorial``
-------------------

This is an example text file in reStructuredText format.  First import
``factorial`` from the ``example`` module:

    >>> from example import factorial

Now use it:

    >>> factorial(6)
    120
```

Running `doctest.testfile("example.txt")` then finds the error in this documentation:

    File "./example.txt", line 14, in example.txt
    Failed example:
        factorial(6)
    Expected:
        120
    Got:
        720

As with `testmod`, `testfile` won't display anything unless an example fails. If an example does fail, then the failing example(s) and the cause(s) of the failure(s) are printed to stdout, using the same format as `!testmod`.

By default, `testfile` looks for files in the calling module's directory. See section `doctest-basic-api` for a description of the optional arguments that can be used to tell it to look for files in other locations.

Like `testmod`, `testfile`'s verbosity can be set with the `-v` command-line switch or with the optional keyword argument *verbose*.

There is also a command line shortcut for running `testfile`, see section `doctest-cli`.

For more information on `testfile`, see section `doctest-basic-api`.

## Command-line Usage

The `!doctest` module can be invoked as a script from the command line:

``` bash
python -m doctest [-v] [-o OPTION] [-f] file [file ...]
```

<div class="program">

doctest

</div>

<div class="option">

`-v, --verbose`  

Detailed report of all examples tried is printed to standard output, along with assorted summaries at the end:

    python -m doctest -v example.py

This will import `example.py` as a standalone module and run `testmod` on it. Note that this may not work correctly if the file is part of a package and imports other submodules from that package.

If the file name does not end with `.py`, `!doctest` infers that it must be run with `testfile` instead:

    python -m doctest -v example.txt

</div>

<div class="option">

`-o, --option <option>`  

Option flags control various aspects of doctest's behavior, see section `doctest-options`.

<div class="versionadded">

3.4

</div>

</div>

<div class="option">

`-f, --fail-fast`  

This is shorthand for `-o FAIL_FAST`.

<div class="versionadded">

3.4

</div>

</div>

## How It Works

This section examines in detail how doctest works: which docstrings it looks at, how it finds interactive examples, what execution context it uses, how it handles exceptions, and how option flags can be used to control its behavior. This is the information that you need to know to write doctest examples; for information about actually running doctest on these examples, see the following sections.

### Which Docstrings Are Examined?

The module docstring, and all function, class and method docstrings are searched. Objects imported into the module are not searched.

<div class="currentmodule">

None

</div>

<div class="attribute" data-no-typesetting="">

module.\_\_test\_\_

</div>

<div class="currentmodule">

doctest

</div>

In addition, there are cases when you want tests to be part of a module but not part of the help text, which requires that the tests not be included in the docstring. Doctest looks for a module-level variable called `__test__` and uses it to locate other tests. If `M.__test__` exists, it must be a dict, and each entry maps a (string) name to a function object, class object, or string. Function and class object docstrings found from `M.__test__` are searched, and strings are treated as if they were docstrings. In output, a key `K` in `M.__test__` appears with name `M.__test__.K`.

For example, place this block of code at the top of `example.py`:

``` python
__test__ = {
    'numbers': """
>>> factorial(6)
720

>>> [factorial(n) for n in range(6)]
[1, 1, 2, 6, 24, 120]
"""
}
```

The value of `example.__test__["numbers"]` will be treated as a docstring and all the tests inside it will be run. It is important to note that the value can be mapped to a function, class object, or module; if so, `!doctest` searches them recursively for docstrings, which are then scanned for tests.

Any classes found are recursively searched similarly, to test docstrings in their contained methods and nested classes.

> [!NOTE]
> `doctest` can only automatically discover classes and functions that are defined at the module level or inside other classes.
>
> Since nested classes and functions only exist when an outer function is called, they cannot be discovered. Define them outside to make them visible.

### How are Docstring Examples Recognized?

In most cases a copy-and-paste of an interactive console session works fine, but doctest isn't trying to do an exact emulation of any specific Python shell.

    >>> # comments are ignored
    >>> x = 12
    >>> x
    12
    >>> if x == 13:
    ...     print("yes")
    ... else:
    ...     print("no")
    ...     print("NO")
    ...     print("NO!!!")
    ...
    no
    NO
    NO!!!
    >>>

<div class="index">

single: \>\>\>; interpreter prompt single: ...; interpreter prompt

</div>

Any expected output must immediately follow the final `'>>> '` or `'... '` line containing the code, and the expected output (if any) extends to the next `'>>> '` or all-whitespace line.

The fine print:

- Expected output cannot contain an all-whitespace line, since such a line is taken to signal the end of expected output. If expected output does contain a blank line, put `<BLANKLINE>` in your doctest example each place a blank line is expected.

- All hard tab characters are expanded to spaces, using 8-column tab stops. Tabs in output generated by the tested code are not modified. Because any hard tabs in the sample output *are* expanded, this means that if the code output includes hard tabs, the only way the doctest can pass is if the `NORMALIZE_WHITESPACE` option or `directive <doctest-directives>` is in effect. Alternatively, the test can be rewritten to capture the output and compare it to an expected value as part of the test. This handling of tabs in the source was arrived at through trial and error, and has proven to be the least error prone way of handling them. It is possible to use a different algorithm for handling tabs by writing a custom `DocTestParser` class.

- Output to stdout is captured, but not output to stderr (exception tracebacks are captured via a different means).

- If you continue a line via backslashing in an interactive session, or for any other reason use a backslash, you should use a raw docstring, which will preserve your backslashes exactly as you type them:

      >>> def f(x):
      ...     r'''Backslashes in a raw docstring: m\n'''
      ...
      >>> print(f.__doc__)
      Backslashes in a raw docstring: m\n

  Otherwise, the backslash will be interpreted as part of the string. For example, the `\n` above would be interpreted as a newline character. Alternatively, you can double each backslash in the doctest version (and not use a raw string):

      >>> def f(x):
      ...     '''Backslashes in a raw docstring: m\\n'''
      ...
      >>> print(f.__doc__)
      Backslashes in a raw docstring: m\n

- The starting column doesn't matter:

      >>> assert "Easy!"
            >>> import math
                >>> math.floor(1.9)
                1

  and as many leading whitespace characters are stripped from the expected output as appeared in the initial `'>>> '` line that started the example.

### What's the Execution Context?

By default, each time `!doctest` finds a docstring to test, it uses a *shallow copy* of `!M`'s globals, so that running tests doesn't change the module's real globals, and so that one test in `!M` can't leave behind crumbs that accidentally allow another test to work. This means examples can freely use any names defined at top-level in `!M`, and names defined earlier in the docstring being run. Examples cannot see names defined in other docstrings.

You can force use of your own dict as the execution context by passing `globs=your_dict` to `testmod` or `testfile` instead.

### What About Exceptions?

No problem, provided that the traceback is the only output produced by the example: just paste in the traceback.[^1] Since tracebacks contain details that are likely to change rapidly (for example, exact file paths and line numbers), this is one case where doctest works hard to be flexible in what it accepts.

Simple example:

    >>> [1, 2, 3].remove(42)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: list.remove(x): x not in list

That doctest succeeds if `ValueError` is raised, with the `list.remove(x): x not in list` detail as shown.

The expected output for an exception must start with a traceback header, which may be either of the following two lines, indented the same as the first line of the example:

    Traceback (most recent call last):
    Traceback (innermost last):

The traceback header is followed by an optional traceback stack, whose contents are ignored by doctest. The traceback stack is typically omitted, or copied verbatim from an interactive session.

The traceback stack is followed by the most interesting part: the line(s) containing the exception type and detail. This is usually the last line of a traceback, but can extend across multiple lines if the exception has a multi-line detail:

    >>> raise ValueError('multi\n    line\ndetail')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: multi
        line
    detail

The last three lines (starting with `ValueError`) are compared against the exception's type and detail, and the rest are ignored.

Best practice is to omit the traceback stack, unless it adds significant documentation value to the example. So the last example is probably better as:

    >>> raise ValueError('multi\n    line\ndetail')
    Traceback (most recent call last):
        ...
    ValueError: multi
        line
    detail

Note that tracebacks are treated very specially. In particular, in the rewritten example, the use of `...` is independent of doctest's `ELLIPSIS` option. The ellipsis in that example could be left out, or could just as well be three (or three hundred) commas or digits, or an indented transcript of a Monty Python skit.

Some details you should read once, but won't need to remember:

- Doctest can't guess whether your expected output came from an exception traceback or from ordinary printing. So, e.g., an example that expects `ValueError: 42 is prime` will pass whether `ValueError` is actually raised or if the example merely prints that traceback text. In practice, ordinary output rarely begins with a traceback header line, so this doesn't create real problems.
- Each line of the traceback stack (if present) must be indented further than the first line of the example, *or* start with a non-alphanumeric character. The first line following the traceback header indented the same and starting with an alphanumeric is taken to be the start of the exception detail. Of course this does the right thing for genuine tracebacks.
- When the `IGNORE_EXCEPTION_DETAIL` doctest option is specified, everything following the leftmost colon and any module information in the exception name is ignored.
- The interactive shell omits the traceback header line for some `SyntaxError`s. But doctest uses the traceback header line to distinguish exceptions from non-exceptions. So in the rare case where you need to test a `!SyntaxError` that omits the traceback header, you will need to manually add the traceback header line to your test example.

<div class="index">

single: ^ (caret); marker

</div>

- For some exceptions, Python displays the position of the error using `^` markers and tildes:

      >>> 1 + None
        File "<stdin>", line 1
          1 + None
          ~~^~~~~~
      TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

  Since the lines showing the position of the error come before the exception type and detail, they are not checked by doctest. For example, the following test would pass, even though it puts the `^` marker in the wrong location:

      >>> 1 + None
        File "<stdin>", line 1
          1 + None
          ^~~~~~~~
      TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

### Option Flags<span id="option-flags-and-directives"></span>

A number of option flags control various aspects of doctest's behavior. Symbolic names for the flags are supplied as module constants, which can be `bitwise ORed <bitwise>` together and passed to various functions. The names can also be used in `doctest directives <doctest-directives>`, and may be passed to the doctest command line interface via the `-o` option.

The first group of options define test semantics, controlling aspects of how doctest decides whether actual output matches an example's expected output:

<div class="data">

DONT_ACCEPT_TRUE_FOR_1

By default, if an expected output block contains just `1`, an actual output block containing just `1` or just `True` is considered to be a match, and similarly for `0` versus `False`. When `DONT_ACCEPT_TRUE_FOR_1` is specified, neither substitution is allowed. The default behavior caters to that Python changed the return type of many functions from integer to boolean; doctests expecting "little integer" output still work in these cases. This option will probably go away, but not for several years.

</div>

<div class="index">

single: \<BLANKLINE\>

</div>

<div class="data">

DONT_ACCEPT_BLANKLINE

By default, if an expected output block contains a line containing only the string `<BLANKLINE>`, then that line will match a blank line in the actual output. Because a genuinely blank line delimits the expected output, this is the only way to communicate that a blank line is expected. When `DONT_ACCEPT_BLANKLINE` is specified, this substitution is not allowed.

</div>

<div class="data">

NORMALIZE_WHITESPACE

When specified, all sequences of whitespace (blanks and newlines) are treated as equal. Any sequence of whitespace within the expected output will match any sequence of whitespace within the actual output. By default, whitespace must match exactly. `NORMALIZE_WHITESPACE` is especially useful when a line of expected output is very long, and you want to wrap it across multiple lines in your source.

</div>

<div class="index">

single: ...; in doctests

</div>

<div class="data">

ELLIPSIS

When specified, an ellipsis marker (`...`) in the expected output can match any substring in the actual output. This includes substrings that span line boundaries, and empty substrings, so it's best to keep usage of this simple. Complicated uses can lead to the same kinds of "oops, it matched too much!" surprises that `.*` is prone to in regular expressions.

</div>

<div class="data">

IGNORE_EXCEPTION_DETAIL

When specified, doctests expecting exceptions pass so long as an exception of the expected type is raised, even if the details (message and fully qualified exception name) don't match.

For example, an example expecting `ValueError: 42` will pass if the actual exception raised is `ValueError: 3*14`, but will fail if, say, a `TypeError` is raised instead. It will also ignore any fully qualified name included before the exception class, which can vary between implementations and versions of Python and the code/libraries in use. Hence, all three of these variations will work with the flag specified:

```
>>> raise Exception('message')
Traceback (most recent call last):
Exception: message

>>> raise Exception('message')
Traceback (most recent call last):
builtins.Exception: message

>>> raise Exception('message')
Traceback (most recent call last):
__main__.Exception: message
```

Note that `ELLIPSIS` can also be used to ignore the details of the exception message, but such a test may still fail based on whether the module name is present or matches exactly.

<div class="versionchanged">

3.2 `IGNORE_EXCEPTION_DETAIL` now also ignores any information relating to the module containing the exception under test.

</div>

</div>

<div class="data">

SKIP

When specified, do not run the example at all. This can be useful in contexts where doctest examples serve as both documentation and test cases, and an example should be included for documentation purposes, but should not be checked. E.g., the example's output might be random; or the example might depend on resources which would be unavailable to the test driver.

The SKIP flag can also be used for temporarily "commenting out" examples.

</div>

<div class="data">

COMPARISON_FLAGS

A bitmask or'ing together all the comparison flags above.

</div>

The second group of options controls how test failures are reported:

<div class="data">

REPORT_UDIFF

When specified, failures that involve multi-line expected and actual outputs are displayed using a unified diff.

</div>

<div class="data">

REPORT_CDIFF

When specified, failures that involve multi-line expected and actual outputs will be displayed using a context diff.

</div>

<div class="data">

REPORT_NDIFF

When specified, differences are computed by `difflib.Differ`, using the same algorithm as the popular `ndiff.py` utility. This is the only method that marks differences within lines as well as across lines. For example, if a line of expected output contains digit `1` where actual output contains letter `l`, a line is inserted with a caret marking the mismatching column positions.

</div>

<div class="data">

REPORT_ONLY_FIRST_FAILURE

When specified, display the first failing example in each doctest, but suppress output for all remaining examples. This will prevent doctest from reporting correct examples that break because of earlier failures; but it might also hide incorrect examples that fail independently of the first failure. When `REPORT_ONLY_FIRST_FAILURE` is specified, the remaining examples are still run, and still count towards the total number of failures reported; only the output is suppressed.

</div>

<div class="data">

FAIL_FAST

When specified, exit after the first failing example and don't attempt to run the remaining examples. Thus, the number of failures reported will be at most 1. This flag may be useful during debugging, since examples after the first failure won't even produce debugging output.

</div>

<div class="data">

REPORTING_FLAGS

A bitmask or'ing together all the reporting flags above.

</div>

There is also a way to register new option flag names, though this isn't useful unless you intend to extend `!doctest` internals via subclassing:

<div class="function">

register_optionflag(name)

Create a new option flag with a given name, and return the new flag's integer value. `register_optionflag` can be used when subclassing `OutputChecker` or `DocTestRunner` to create new options that are supported by your subclasses. `register_optionflag` should always be called using the following idiom:

    MY_FLAG = register_optionflag('MY_FLAG')

</div>

<div class="index">

single: \# (hash); in doctests single: + (plus); in doctests single: - (minus); in doctests

</div>

### Directives

Doctest directives may be used to modify the `option flags
<doctest-options>` for an individual example. Doctest directives are special Python comments following an example's source code:

<div class="productionlist">

doctest directive: "#" "doctest:" <span class="title-ref">directive_options</span> directive_options: <span class="title-ref">directive_option</span> ("," <span class="title-ref">directive_option</span>)\* directive_option: <span class="title-ref">on_or_off</span> <span class="title-ref">directive_option_name</span> on_or_off: "+" \| "-" directive_option_name: "DONT_ACCEPT_BLANKLINE" \| "NORMALIZE_WHITESPACE" \| ...

</div>

Whitespace is not allowed between the `+` or `-` and the directive option name. The directive option name can be any of the option flag names explained above.

An example's doctest directives modify doctest's behavior for that single example. Use `+` to enable the named behavior, or `-` to disable it.

For example, this test passes:

<div class="doctest" data-no-trim-doctest-flags="">

\>\>\> print(list(range(20))) \# doctest: +NORMALIZE_WHITESPACE \[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19\]

</div>

Without the directive it would fail, both because the actual output doesn't have two blanks before the single-digit list elements, and because the actual output is on a single line. This test also passes, and also requires a directive to do so:

<div class="doctest" data-no-trim-doctest-flags="">

\>\>\> print(list(range(20))) \# doctest: +ELLIPSIS \[0, 1, ..., 18, 19\]

</div>

Multiple directives can be used on a single physical line, separated by commas:

<div class="doctest" data-no-trim-doctest-flags="">

\>\>\> print(list(range(20))) \# doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE \[0, 1, ..., 18, 19\]

</div>

If multiple directive comments are used for a single example, then they are combined:

<div class="doctest" data-no-trim-doctest-flags="">

\>\>\> print(list(range(20))) \# doctest: +ELLIPSIS ... \# doctest: +NORMALIZE_WHITESPACE \[0, 1, ..., 18, 19\]

</div>

As the previous example shows, you can add `...` lines to your example containing only directives. This can be useful when an example is too long for a directive to comfortably fit on the same line:

<div class="doctest" data-no-trim-doctest-flags="">

\>\>\> print(list(range(5)) + list(range(10, 20)) + list(range(30, 40))) ... \# doctest: +ELLIPSIS \[0, ..., 4, 10, ..., 19, 30, ..., 39\]

</div>

Note that since all options are disabled by default, and directives apply only to the example they appear in, enabling options (via `+` in a directive) is usually the only meaningful choice. However, option flags can also be passed to functions that run doctests, establishing different defaults. In such cases, disabling an option via `-` in a directive can be useful.

### Warnings

`!doctest` is serious about requiring exact matches in expected output. If even a single character doesn't match, the test fails. This will probably surprise you a few times, as you learn exactly what Python does and doesn't guarantee about output. For example, when printing a set, Python doesn't guarantee that the element is printed in any particular order, so a test like :

    >>> foo()
    {"spam", "eggs"}

is vulnerable! One workaround is to do :

    >>> foo() == {"spam", "eggs"}
    True

instead. Another is to do :

    >>> d = sorted(foo())
    >>> d
    ['eggs', 'spam']

There are others, but you get the idea.

Another bad idea is to print things that embed an object address, like

<div class="doctest">

\>\>\> id(1.0) \# certain to fail some of the time \# doctest: +SKIP 7948648 \>\>\> class C: pass \>\>\> C() \# the default repr() for instances embeds an address \# doctest: +SKIP \<C object at 0x00AC18F0\>

</div>

The `ELLIPSIS` directive gives a nice approach for the last example:

<div class="doctest" data-no-trim-doctest-flags="">

\>\>\> C() \# doctest: +ELLIPSIS \<C object at 0x...\>

</div>

Floating-point numbers are also subject to small output variations across platforms, because Python defers to the platform C library for some floating-point calculations, and C libraries vary widely in quality here. :

    >>> 1000**0.1  # risky
    1.9952623149688797
    >>> round(1000**0.1, 9) # safer
    1.995262315
    >>> print(f'{1000**0.1:.4f}') # much safer
    1.9953

Numbers of the form `I/2.**J` are safe across all platforms, and I often contrive doctest examples to produce numbers of that form:

    >>> 3./4  # utterly safe
    0.75

Simple fractions are also easier for people to understand, and that makes for better documentation.

## Basic API

The functions `testmod` and `testfile` provide a simple interface to doctest that should be sufficient for most basic uses. For a less formal introduction to these two functions, see sections `doctest-simple-testmod` and `doctest-simple-testfile`.

<div class="function">

testfile(filename, module_relative=True, name=None, package=None, globs=None, verbose=None, report=True, optionflags=0, extraglobs=None, raise_on_error=False, parser=DocTestParser(), encoding=None)

All arguments except *filename* are optional, and should be specified in keyword form.

Test examples in the file named *filename*. Return `(failure_count, test_count)`.

Optional argument *module_relative* specifies how the filename should be interpreted:

- If *module_relative* is `True` (the default), then *filename* specifies an OS-independent module-relative path. By default, this path is relative to the calling module's directory; but if the *package* argument is specified, then it is relative to that package. To ensure OS-independence, *filename* should use `/` characters to separate path segments, and may not be an absolute path (i.e., it may not begin with `/`).
- If *module_relative* is `False`, then *filename* specifies an OS-specific path. The path may be absolute or relative; relative paths are resolved with respect to the current working directory.

Optional argument *name* gives the name of the test; by default, or if `None`, `os.path.basename(filename)` is used.

Optional argument *package* is a Python package or the name of a Python package whose directory should be used as the base directory for a module-relative filename. If no package is specified, then the calling module's directory is used as the base directory for module-relative filenames. It is an error to specify *package* if *module_relative* is `False`.

Optional argument *globs* gives a dict to be used as the globals when executing examples. A new shallow copy of this dict is created for the doctest, so its examples start with a clean slate. By default, or if `None`, a new empty dict is used.

Optional argument *extraglobs* gives a dict merged into the globals used to execute examples. This works like `dict.update`: if *globs* and *extraglobs* have a common key, the associated value in *extraglobs* appears in the combined dict. By default, or if `None`, no extra globals are used. This is an advanced feature that allows parameterization of doctests. For example, a doctest can be written for a base class, using a generic name for the class, then reused to test any number of subclasses by passing an *extraglobs* dict mapping the generic name to the subclass to be tested.

Optional argument *verbose* prints lots of stuff if true, and prints only failures if false; by default, or if `None`, it's true if and only if `'-v'` is in `sys.argv`.

Optional argument *report* prints a summary at the end when true, else prints nothing at the end. In verbose mode, the summary is detailed, else the summary is very brief (in fact, empty if all tests passed).

Optional argument *optionflags* (default value `0`) takes the `bitwise OR <bitwise>` of option flags. See section `doctest-options`.

Optional argument *raise_on_error* defaults to false. If true, an exception is raised upon the first failure or unexpected exception in an example. This allows failures to be post-mortem debugged. Default behavior is to continue running examples.

Optional argument *parser* specifies a `DocTestParser` (or subclass) that should be used to extract tests from the files. It defaults to a normal parser (i.e., `DocTestParser()`).

Optional argument *encoding* specifies an encoding that should be used to convert the file to unicode.

</div>

<div class="function">

testmod(m=None, name=None, globs=None, verbose=None, report=True, optionflags=0, extraglobs=None, raise_on_error=False, exclude_empty=False)

All arguments are optional, and all except for *m* should be specified in keyword form.

Test examples in docstrings in functions and classes reachable from module *m* (or module `__main__` if *m* is not supplied or is `None`), starting with `m.__doc__`.

Also test examples reachable from dict `m.__test__`, if it exists. `m.__test__` maps names (strings) to functions, classes and strings; function and class docstrings are searched for examples; strings are searched directly, as if they were docstrings.

Only docstrings attached to objects belonging to module *m* are searched.

Return `(failure_count, test_count)`.

Optional argument *name* gives the name of the module; by default, or if `None`, `m.__name__` is used.

Optional argument *exclude_empty* defaults to false. If true, objects for which no doctests are found are excluded from consideration. The default is a backward compatibility hack, so that code still using `doctest.master.summarize <DocTestRunner.summarize>` in conjunction with `testmod` continues to get output for objects with no tests. The *exclude_empty* argument to the newer `DocTestFinder` constructor defaults to true.

Optional arguments *extraglobs*, *verbose*, *report*, *optionflags*, *raise_on_error*, and *globs* are the same as for function `testfile` above, except that *globs* defaults to `m.__dict__`.

</div>

<div class="function">

run_docstring_examples(f, globs, verbose=False, name="NoName", compileflags=None, optionflags=0)

Test examples associated with object *f*; for example, *f* may be a string, a module, a function, or a class object.

A shallow copy of dictionary argument *globs* is used for the execution context.

Optional argument *name* is used in failure messages, and defaults to `"NoName"`.

If optional argument *verbose* is true, output is generated even if there are no failures. By default, output is generated only in case of an example failure.

Optional argument *compileflags* gives the set of flags that should be used by the Python compiler when running the examples. By default, or if `None`, flags are deduced corresponding to the set of future features found in *globs*.

Optional argument *optionflags* works as for function `testfile` above.

</div>

## Unittest API

As your collection of doctest'ed modules grows, you'll want a way to run all their doctests systematically. `!doctest` provides two functions that can be used to create `unittest` test suites from modules and text files containing doctests. To integrate with `unittest` test discovery, include a `load_tests <load_tests-protocol>` function in your test module:

    import unittest
    import doctest
    import my_module_with_doctests

    def load_tests(loader, tests, ignore):
        tests.addTests(doctest.DocTestSuite(my_module_with_doctests))
        return tests

There are two main functions for creating `unittest.TestSuite` instances from text files and modules with doctests:

<div class="function">

DocFileSuite(\*paths, module_relative=True, package=None, setUp=None, tearDown=None, globs=None, optionflags=0, parser=DocTestParser(), encoding=None)

Convert doctest tests from one or more text files to a `unittest.TestSuite`.

The returned `unittest.TestSuite` is to be run by the unittest framework and runs the interactive examples in each file. Each file is run as a separate unit test, and each example in a file is run as a `subtest <subtests>`. If any example in a file fails, then the synthesized unit test fails. The traceback for failure or error contains the name of the file containing the test and a (sometimes approximate) line number. If all the examples in a file are skipped, then the synthesized unit test is also marked as skipped.

Pass one or more paths (as strings) to text files to be examined.

Options may be provided as keyword arguments:

Optional argument *module_relative* specifies how the filenames in *paths* should be interpreted:

- If *module_relative* is `True` (the default), then each filename in *paths* specifies an OS-independent module-relative path. By default, this path is relative to the calling module's directory; but if the *package* argument is specified, then it is relative to that package. To ensure OS-independence, each filename should use `/` characters to separate path segments, and may not be an absolute path (i.e., it may not begin with `/`).
- If *module_relative* is `False`, then each filename in *paths* specifies an OS-specific path. The path may be absolute or relative; relative paths are resolved with respect to the current working directory.

Optional argument *package* is a Python package or the name of a Python package whose directory should be used as the base directory for module-relative filenames in *paths*. If no package is specified, then the calling module's directory is used as the base directory for module-relative filenames. It is an error to specify *package* if *module_relative* is `False`.

Optional argument *setUp* specifies a set-up function for the test suite. This is called before running the tests in each file. The *setUp* function will be passed a `DocTest` object. The *setUp* function can access the test globals as the `~DocTest.globs` attribute of the test passed.

Optional argument *tearDown* specifies a tear-down function for the test suite. This is called after running the tests in each file. The *tearDown* function will be passed a `DocTest` object. The *tearDown* function can access the test globals as the `~DocTest.globs` attribute of the test passed.

Optional argument *globs* is a dictionary containing the initial global variables for the tests. A new copy of this dictionary is created for each test. By default, *globs* is a new empty dictionary.

Optional argument *optionflags* specifies the default doctest options for the tests, created by or-ing together individual option flags. See section `doctest-options`. See function `set_unittest_reportflags` below for a better way to set reporting options.

Optional argument *parser* specifies a `DocTestParser` (or subclass) that should be used to extract tests from the files. It defaults to a normal parser (i.e., `DocTestParser()`).

Optional argument *encoding* specifies an encoding that should be used to convert the file to unicode.

The global `__file__` is added to the globals provided to doctests loaded from a text file using `DocFileSuite`.

<div class="versionchanged">

3.15 Run each example as a `subtest <subtests>`.

</div>

</div>

<div class="function">

DocTestSuite(module=None, globs=None, extraglobs=None, test_finder=None, setUp=None, tearDown=None, optionflags=0, checker=None)

Convert doctest tests for a module to a `unittest.TestSuite`.

The returned `unittest.TestSuite` is to be run by the unittest framework and runs each doctest in the module. Each docstring is run as a separate unit test, and each example in a docstring is run as a `subtest <subtests>`. If any of the doctests fail, then the synthesized unit test fails. The traceback for failure or error contains the name of the file containing the test and a (sometimes approximate) line number. If all the examples in a docstring are skipped, then the synthesized unit test is also marked as skipped.

Optional argument *module* provides the module to be tested. It can be a module object or a (possibly dotted) module name. If not specified, the module calling this function is used.

Optional argument *globs* is a dictionary containing the initial global variables for the tests. A new copy of this dictionary is created for each test. By default, *globs* is the module's `~module.__dict__`.

Optional argument *extraglobs* specifies an extra set of global variables, which is merged into *globs*. By default, no extra globals are used.

Optional argument *test_finder* is the `DocTestFinder` object (or a drop-in replacement) that is used to extract doctests from the module.

Optional arguments *setUp*, *tearDown*, and *optionflags* are the same as for function `DocFileSuite` above, but they are called for each docstring.

This function uses the same search technique as `testmod`.

<div class="versionchanged">

3.5 `DocTestSuite` returns an empty `unittest.TestSuite` if *module* contains no docstrings instead of raising `ValueError`.

</div>

<div class="versionchanged">

3.15 Run each example as a `subtest <subtests>`.

</div>

<div class="versionchanged">

3.16 Report every example, as in verbose mode, if the test runner reports more than the test names, i.e. its `~unittest.TestResult.verbosity` is 3 or higher (for example with `python -m unittest -vv`).

</div>

</div>

Under the covers, `DocTestSuite` creates a `unittest.TestSuite` out of `!doctest.DocTestCase` instances, and `!DocTestCase` is a subclass of `unittest.TestCase`. `!DocTestCase` isn't documented here (it's an internal detail), but studying its code can answer questions about the exact details of `unittest` integration.

Similarly, `DocFileSuite` creates a `unittest.TestSuite` out of `!doctest.DocFileCase` instances, and `!DocFileCase` is a subclass of `!DocTestCase`.

So both ways of creating a `unittest.TestSuite` run instances of `!DocTestCase`. This is important for a subtle reason: when you run `!doctest` functions yourself, you can control the `!doctest` options in use directly, by passing option flags to `!doctest` functions. However, if you're writing a `unittest` framework, `!unittest` ultimately controls when and how tests get run. The framework author typically wants to control `!doctest` reporting options (perhaps, e.g., specified by command line options), but there's no way to pass options through `!unittest` to `!doctest` test runners.

For this reason, `!doctest` also supports a notion of `!doctest` reporting flags specific to `unittest` support, via this function:

<div class="function">

set_unittest_reportflags(flags)

Set the `!doctest` reporting flags to use.

Argument *flags* takes the `bitwise OR <bitwise>` of option flags. See section `doctest-options`. Only "reporting flags" can be used.

This is a module-global setting, and affects all future doctests run by module `unittest`: the `!runTest` method of `!DocTestCase` looks at the option flags specified for the test case when the `!DocTestCase` instance was constructed. If no reporting flags were specified (which is the typical and expected case), `!doctest`'s `!unittest` reporting flags are `bitwise ORed <bitwise>` into the option flags, and the option flags so augmented are passed to the `DocTestRunner` instance created to run the doctest. If any reporting flags were specified when the `!DocTestCase` instance was constructed, `!doctest`'s `!unittest` reporting flags are ignored.

The value of the `unittest` reporting flags in effect before the function was called is returned by the function.

</div>

## Advanced API

The basic API is a simple wrapper that's intended to make doctest easy to use. It is fairly flexible, and should meet most users' needs; however, if you require more fine-grained control over testing, or wish to extend doctest's capabilities, then you should use the advanced API.

The advanced API revolves around two container classes, which are used to store the interactive examples extracted from doctest cases:

- `Example`: A single Python `statement`, paired with its expected output.
- `DocTest`: A collection of `Example`s, typically extracted from a single docstring or text file.

Additional processing classes are defined to find, parse, and run, and check doctest examples:

- `DocTestFinder`: Finds all docstrings in a given module, and uses a `DocTestParser` to create a `DocTest` from every docstring that contains interactive examples.
- `DocTestParser`: Creates a `DocTest` object from a string (such as an object's docstring).
- `DocTestRunner`: Executes the examples in a `DocTest`, and uses an `OutputChecker` to verify their output.
- `OutputChecker`: Compares the actual output from a doctest example with the expected output, and decides whether they match.

The relationships among these processing classes are summarized in the following diagram:

    list of:
    +------+                   +---------+
    |module| --DocTestFinder-> | DocTest | --DocTestRunner-> results
    +------+    |        ^     +---------+     |       ^    (printed)
    |        |     | Example |     |       |
    v        |     |   ...   |     v       |
    DocTestParser   | Example |   OutputChecker
    +---------+

### DocTest Objects

<div class="DocTest(examples, globs, name, filename, lineno, docstring)">

A collection of doctest examples that should be run in a single namespace. The constructor arguments are used to initialize the attributes of the same names.

`DocTest` defines the following attributes. They are initialized by the constructor, and should not be modified directly.

<div class="attribute">

examples

A list of `Example` objects encoding the individual interactive Python examples that should be run by this test.

</div>

<div class="attribute">

globs

The namespace (aka globals) that the examples should be run in. This is a dictionary mapping names to values. Any changes to the namespace made by the examples (such as binding new variables) will be reflected in `globs` after the test is run.

</div>

<div class="attribute">

name

A string name identifying the `DocTest`. Typically, this is the name of the object or file that the test was extracted from.

</div>

<div class="attribute">

filename

The name of the file that this `DocTest` was extracted from; or `None` if the filename is unknown, or if the `!DocTest` was not extracted from a file.

</div>

<div class="attribute">

lineno

The line number within `filename` where this `DocTest` begins, or `None` if the line number is unavailable. This line number is zero-based with respect to the beginning of the file.

</div>

<div class="attribute">

docstring

The string that the test was extracted from, or `None` if the string is unavailable, or if the test was not extracted from a string.

</div>

</div>

### Example Objects

<div class="Example(source, want, exc_msg=None, lineno=0, indent=0, options=None)">

A single interactive example, consisting of a Python statement and its expected output. The constructor arguments are used to initialize the attributes of the same names.

`Example` defines the following attributes. They are initialized by the constructor, and should not be modified directly.

<div class="attribute">

source

A string containing the example's source code. This source code consists of a single Python statement, and always ends with a newline; the constructor adds a newline when necessary.

</div>

<div class="attribute">

want

The expected output from running the example's source code (either from stdout, or a traceback in case of exception). `want` ends with a newline unless no output is expected, in which case it's an empty string. The constructor adds a newline when necessary.

</div>

<div class="attribute">

exc_msg

The exception message generated by the example, if the example is expected to generate an exception; or `None` if it is not expected to generate an exception. This exception message is compared against the return value of `traceback.format_exception_only`. `exc_msg` ends with a newline unless it's `None`. The constructor adds a newline if needed.

</div>

<div class="attribute">

lineno

The line number within the string containing this example where the example begins. This line number is zero-based with respect to the beginning of the containing string.

</div>

<div class="attribute">

indent

The example's indentation in the containing string, i.e., the number of space characters that precede the example's first prompt.

</div>

<div class="attribute">

options

A dictionary mapping from option flags to `True` or `False`, which is used to override default options for this example. Any option flags not contained in this dictionary are left at their default value (as specified by the `DocTestRunner`'s `optionflags <doctest-options>`). By default, no options are set.

</div>

</div>

### DocTestFinder objects

<div class="DocTestFinder(verbose=False, parser=DocTestParser(), recurse=True, exclude_empty=True)">

A processing class used to extract the `DocTest`s that are relevant to a given object, from its docstring and the docstrings of its contained objects. `DocTest`s can be extracted from modules, classes, functions, methods, staticmethods, classmethods, and properties.

The optional argument *verbose* can be used to display the objects searched by the finder. It defaults to `False` (no output).

The optional argument *parser* specifies the `DocTestParser` object (or a drop-in replacement) that is used to extract doctests from docstrings.

If the optional argument *recurse* is false, then `DocTestFinder.find` will only examine the given object, and not any contained objects.

If the optional argument *exclude_empty* is false, then `DocTestFinder.find` will include tests for objects with empty docstrings.

`DocTestFinder` defines the following method:

<div class="method">

find(obj\[, name\]\[, module\]\[, globs\]\[, extraglobs\])

Return a list of the `DocTest`s that are defined by *obj*'s docstring, or by any of its contained objects' docstrings.

The optional argument *name* specifies the object's name; this name will be used to construct names for the returned `DocTest`s. If *name* is not specified, then `obj.__name__` is used.

The optional parameter *module* is the module that contains the given object. If the module is not specified or is `None`, then the test finder will attempt to automatically determine the correct module. The object's module is used:

- As a default namespace, if *globs* is not specified.
- To prevent the DocTestFinder from extracting DocTests from objects that are imported from other modules. (Contained objects with modules other than *module* are ignored.)
- To find the name of the file containing the object.
- To help find the line number of the object within its file.

If *module* is `False`, no attempt to find the module will be made. This is obscure, of use mostly in testing doctest itself: if *module* is `False`, or is `None` but cannot be found automatically, then all objects are considered to belong to the (non-existent) module, so all contained objects will (recursively) be searched for doctests.

The globals for each `DocTest` is formed by combining *globs* and *extraglobs* (bindings in *extraglobs* override bindings in *globs*). A new shallow copy of the globals dictionary is created for each `!DocTest`. If *globs* is not specified, then it defaults to the module's `~module.__dict__`, if specified, or `{}` otherwise. If *extraglobs* is not specified, then it defaults to `{}`.

</div>

</div>

### DocTestParser objects

<div class="DocTestParser()">

A processing class used to extract interactive examples from a string, and use them to create a `DocTest` object.

`DocTestParser` defines the following methods:

<div class="method">

get_doctest(string, globs, name, filename, lineno)

Extract all doctest examples from the given string, and collect them into a `DocTest` object.

*globs*, *name*, *filename*, and *lineno* are attributes for the new `!DocTest` object. See the documentation for `DocTest` for more information.

</div>

<div class="method">

get_examples(string, name='\<string\>')

Extract all doctest examples from the given string, and return them as a list of `Example` objects. Line numbers are 0-based. The optional argument *name* is a name identifying this string, and is only used for error messages.

</div>

<div class="method">

parse(string, name='\<string\>')

Divide the given string into examples and intervening text, and return them as a list of alternating `Example`s and strings. Line numbers for the `!Example`s are 0-based. The optional argument *name* is a name identifying this string, and is only used for error messages.

</div>

</div>

### TestResults objects

<div class="TestResults(failed, attempted)">

<div class="attribute">

failed

Number of failed tests.

</div>

<div class="attribute">

attempted

Number of attempted tests.

</div>

<div class="attribute">

skipped

Number of skipped tests.

<div class="versionadded">

3.13

</div>

</div>

</div>

### DocTestRunner objects

<div class="DocTestRunner(checker=None, verbose=None, optionflags=0)">

A processing class used to execute and verify the interactive examples in a `DocTest`.

The comparison between expected outputs and actual outputs is done by an `OutputChecker`. This comparison may be customized with a number of option flags; see section `doctest-options` for more information. If the option flags are insufficient, then the comparison may also be customized by passing a subclass of `!OutputChecker` to the constructor.

The test runner's display output can be controlled in two ways. First, an output function can be passed to `run`; this function will be called with strings that should be displayed. It defaults to `sys.stdout.write`. If capturing the output is not sufficient, then the display output can be also customized by subclassing DocTestRunner, and overriding the methods `report_skip`, `report_start`, `report_success`, `report_unexpected_exception`, and `report_failure`.

The optional keyword argument *checker* specifies the `OutputChecker` object (or drop-in replacement) that should be used to compare the expected outputs to the actual outputs of doctest examples.

The optional keyword argument *verbose* controls the `DocTestRunner`'s verbosity. If *verbose* is `True`, then information is printed about each example, as it is run. If *verbose* is `False`, then only failures are printed. If *verbose* is unspecified, or `None`, then verbose output is used iff the command-line switch `-v` is used.

The optional keyword argument *optionflags* can be used to control how the test runner compares expected output to actual output, and how it displays failures. For more information, see section `doctest-options`.

The test runner accumulates statistics. The aggregated number of attempted, failed and skipped examples is also available via the `tries`, `failures` and `skips` attributes. The `run` and `summarize` methods return a `TestResults` instance.

`DocTestRunner` defines the following methods:

<div class="method">

report_skip(out, test, example)

Report that the given example was skipped. This method is provided to allow subclasses of `DocTestRunner` to customize their output; it should not be called directly.

*example* is the example about to be processed. *test* is the test containing *example*. *out* is the output function that was passed to `DocTestRunner.run`.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

report_start(out, test, example)

Report that the test runner is about to process the given example. This method is provided to allow subclasses of `DocTestRunner` to customize their output; it should not be called directly.

*example* is the example about to be processed. *test* is the test containing *example*. *out* is the output function that was passed to `DocTestRunner.run`.

</div>

<div class="method">

report_success(out, test, example, got)

Report that the given example ran successfully. This method is provided to allow subclasses of `DocTestRunner` to customize their output; it should not be called directly.

*example* is the example about to be processed. *got* is the actual output from the example. *test* is the test containing *example*. *out* is the output function that was passed to `DocTestRunner.run`.

</div>

<div class="method">

report_failure(out, test, example, got)

Report that the given example failed. This method is provided to allow subclasses of `DocTestRunner` to customize their output; it should not be called directly.

*example* is the example about to be processed. *got* is the actual output from the example. *test* is the test containing *example*. *out* is the output function that was passed to `DocTestRunner.run`.

</div>

<div class="method">

report_unexpected_exception(out, test, example, exc_info)

Report that the given example raised an unexpected exception. This method is provided to allow subclasses of `DocTestRunner` to customize their output; it should not be called directly.

*example* is the example about to be processed. *exc_info* is a tuple containing information about the unexpected exception (as returned by `sys.exc_info`). *test* is the test containing *example*. *out* is the output function that was passed to `DocTestRunner.run`.

</div>

<div class="method">

run(test, compileflags=None, out=None, clear_globs=True)

Run the examples in *test* (a `DocTest` object), and display the results using the writer function *out*. Return a `TestResults` instance.

The examples are run in the namespace `test.globs`. If *clear_globs* is true (the default), then this namespace will be cleared after the test runs, to help with garbage collection. If you would like to examine the namespace after the test completes, then use *clear_globs=False*.

*compileflags* gives the set of flags that should be used by the Python compiler when running the examples. If not specified, then it will default to the set of future-import flags that apply to *globs*.

The output of each example is checked using the `DocTestRunner`'s output checker, and the results are formatted by the `!DocTestRunner.report_\*` methods.

</div>

<div class="method">

summarize(verbose=None)

Print a summary of all the test cases that have been run by this DocTestRunner, and return a `TestResults` instance.

The optional *verbose* argument controls how detailed the summary is. If the verbosity is not specified, then the `DocTestRunner`'s verbosity is used.

</div>

`DocTestParser` has the following attributes:

<div class="attribute">

tries

Number of attempted examples.

</div>

<div class="attribute">

failures

Number of failed examples.

</div>

<div class="attribute">

skips

Number of skipped examples.

<div class="versionadded">

3.13

</div>

</div>

</div>

### OutputChecker objects

<div class="OutputChecker()">

A class used to check the whether the actual output from a doctest example matches the expected output. `OutputChecker` defines two methods: `check_output`, which compares a given pair of outputs, and returns `True` if they match; and `output_difference`, which returns a string describing the differences between two outputs.

`OutputChecker` defines the following methods:

<div class="method">

check_output(want, got, optionflags)

Return `True` iff the actual output from an example (*got*) matches the expected output (*want*). These strings are always considered to match if they are identical; but depending on what option flags the test runner is using, several non-exact match types are also possible. See section `doctest-options` for more information about option flags.

</div>

<div class="method">

output_difference(example, got, optionflags)

Return a string describing the differences between the expected output for a given example (*example*) and the actual output (*got*). *optionflags* is the set of option flags used to compare *want* and *got*.

</div>

</div>

## Debugging

Doctest provides several mechanisms for debugging doctest examples:

- Several functions convert doctests to executable Python programs, which can be run under the Python debugger, `pdb`.

- The `DebugRunner` class is a subclass of `DocTestRunner` that raises an exception for the first failing example, containing information about that example. This information can be used to perform post-mortem debugging on the example.

- The `unittest` cases generated by `DocTestSuite` support the `debug` method defined by `unittest.TestCase`.

- You can add a call to `pdb.set_trace` in a doctest example, and you'll drop into the Python debugger when that line is executed. Then you can inspect current values of variables, and so on. For example, suppose `a.py` contains just this module docstring:

      """
      >>> def f(x):
      ...     g(x*2)
      >>> def g(x):
      ...     print(x+3)
      ...     import pdb; pdb.set_trace()
      >>> f(3)
      9
      """

  Then an interactive Python session may look like this:

      >>> import a, doctest
      >>> doctest.testmod(a)
      --Return--
      > <doctest a[1]>(3)g()->None
      -> import pdb; pdb.set_trace()
      (Pdb) list
        1     def g(x):
        2         print(x+3)
        3  ->     import pdb; pdb.set_trace()
      [EOF]
      (Pdb) p x
      6
      (Pdb) step
      --Return--
      > <doctest a[0]>(2)f()->None
      -> g(x*2)
      (Pdb) list
        1     def f(x):
        2  ->     g(x*2)
      [EOF]
      (Pdb) p x
      3
      (Pdb) step
      --Return--
      > <doctest a[2]>(1)?()->None
      -> f(3)
      (Pdb) cont
      (0, 3)
      >>>

Functions that convert doctests to Python code, and possibly run the synthesized code under the debugger:

<div class="function">

script_from_examples(s)

Convert text with examples to a script.

Argument *s* is a string containing doctest examples. The string is converted to a Python script, where doctest examples in *s* are converted to regular code, and everything else is converted to Python comments. The generated script is returned as a string. For example, :

    import doctest
    print(doctest.script_from_examples(r"""
        Set x and y to 1 and 2.
        >>> x, y = 1, 2

        Print their sum:
        >>> print(x+y)
        3
    """))

displays:

    # Set x and y to 1 and 2.
    x, y = 1, 2
    #
    # Print their sum:
    print(x+y)
    # Expected:
    ## 3

This function is used internally by other functions (see below), but can also be useful when you want to transform an interactive Python session into a Python script.

</div>

<div class="function">

testsource(module, name)

Convert the doctest for an object to a script.

Argument *module* is a module object, or dotted name of a module, containing the object whose doctests are of interest. Argument *name* is the name (within the module) of the object with the doctests of interest. The result is a string, containing the object's docstring converted to a Python script, as described for `script_from_examples` above. For example, if module `a.py` contains a top-level function `!f`, then :

    import a, doctest
    print(doctest.testsource(a, "a.f"))

prints a script version of function `!f`'s docstring, with doctests converted to code, and the rest placed in comments.

</div>

<div class="function">

debug(module, name, pm=False)

Debug the doctests for an object.

The *module* and *name* arguments are the same as for function `testsource` above. The synthesized Python script for the named object's docstring is written to a temporary file, and then that file is run under the control of the Python debugger, `pdb`.

A shallow copy of `module.__dict__` is used for both local and global execution context.

Optional argument *pm* controls whether post-mortem debugging is used. If *pm* has a true value, the script file is run directly, and the debugger gets involved only if the script terminates via raising an unhandled exception. If it does, then post-mortem debugging is invoked, via `pdb.post_mortem`, passing the traceback object from the unhandled exception. If *pm* is not specified, or is false, the script is run under the debugger from the start, via passing an appropriate `exec` call to `pdb.run`.

</div>

<div class="function">

debug_src(src, pm=False, globs=None)

Debug the doctests in a string.

This is like function `debug` above, except that a string containing doctest examples is specified directly, via the *src* argument.

Optional argument *pm* has the same meaning as in function `debug` above.

Optional argument *globs* gives a dictionary to use as both local and global execution context. If not specified, or `None`, an empty dictionary is used. If specified, a shallow copy of the dictionary is used.

</div>

The `DebugRunner` class, and the special exceptions it may raise, are of most interest to testing framework authors, and will only be sketched here. See the source code, and especially `DebugRunner`'s docstring (which is a doctest!) for more details:

<div class="DebugRunner(checker=None, verbose=None, optionflags=0)">

A subclass of `DocTestRunner` that raises an exception as soon as a failure is encountered. If an unexpected exception occurs, an `UnexpectedException` exception is raised, containing the test, the example, and the original exception. If the output doesn't match, then a `DocTestFailure` exception is raised, containing the test, the example, and the actual output.

For information about the constructor parameters and methods, see the documentation for `DocTestRunner` in section `doctest-advanced-api`.

</div>

There are two exceptions that may be raised by `DebugRunner` instances:

<div class="exception">

DocTestFailure(test, example, got)

An exception raised by `DocTestRunner` to signal that a doctest example's actual output did not match its expected output. The constructor arguments are used to initialize the attributes of the same names.

</div>

`DocTestFailure` defines the following attributes:

<div class="attribute">

DocTestFailure.test

The `DocTest` object that was being run when the example failed.

</div>

<div class="attribute">

DocTestFailure.example

The `Example` that failed.

</div>

<div class="attribute">

DocTestFailure.got

The example's actual output.

</div>

<div class="exception">

UnexpectedException(test, example, exc_info)

An exception raised by `DocTestRunner` to signal that a doctest example raised an unexpected exception. The constructor arguments are used to initialize the attributes of the same names.

</div>

`UnexpectedException` defines the following attributes:

<div class="attribute">

UnexpectedException.test

The `DocTest` object that was being run when the example failed.

</div>

<div class="attribute">

UnexpectedException.example

The `Example` that failed.

</div>

<div class="attribute">

UnexpectedException.exc_info

A tuple containing information about the unexpected exception, as returned by `sys.exc_info`.

</div>

## Soapbox

As mentioned in the introduction, `!doctest` has grown to have three primary uses:

1.  Checking examples in docstrings.
2.  Regression testing.
3.  Executable documentation / literate testing.

These uses have different requirements, and it is important to distinguish them. In particular, filling your docstrings with obscure test cases makes for bad documentation.

When writing a docstring, choose docstring examples with care. There's an art to this that needs to be learned---it may not be natural at first. Examples should add genuine value to the documentation. A good example can often be worth many words. If done with care, the examples will be invaluable for your users, and will pay back the time it takes to collect them many times over as the years go by and things change. I'm still amazed at how often one of my `!doctest` examples stops working after a "harmless" change.

Doctest also makes an excellent tool for regression testing, especially if you don't skimp on explanatory text. By interleaving prose and examples, it becomes much easier to keep track of what's actually being tested, and why. When a test fails, good prose can make it much easier to figure out what the problem is, and how it should be fixed. It's true that you could write extensive comments in code-based testing, but few programmers do. Many have found that using doctest approaches instead leads to much clearer tests. Perhaps this is simply because doctest makes writing prose a little easier than writing code, while writing comments in code is a little harder. I think it goes deeper than just that: the natural attitude when writing a doctest-based test is that you want to explain the fine points of your software, and illustrate them with examples. This in turn naturally leads to test files that start with the simplest features, and logically progress to complications and edge cases. A coherent narrative is the result, instead of a collection of isolated functions that test isolated bits of functionality seemingly at random. It's a different attitude, and produces different results, blurring the distinction between testing and explaining.

Regression testing is best confined to dedicated objects or files. There are several options for organizing tests:

- Write text files containing test cases as interactive examples, and test the files using `testfile` or `DocFileSuite`. This is recommended, although is easiest to do for new projects, designed from the start to use doctest.
- Define functions named `_regrtest_topic` that consist of single docstrings, containing test cases for the named topics. These functions can be included in the same file as the module, or separated out into a separate test file.
- Define a `~module.__test__` dictionary mapping from regression test topics to docstrings containing test cases.

When you have placed your tests in a module, the module can itself be the test runner. When a test fails, you can arrange for your test runner to re-run only the failing doctest while you debug the problem. Here is a minimal example of such a test runner:

    if __name__ == '__main__':
        import doctest
        flags = doctest.REPORT_NDIFF|doctest.FAIL_FAST
        if len(sys.argv) > 1:
            name = sys.argv[1]
            if name in globals():
                obj = globals()[name]
            else:
                obj = __test__[name]
            doctest.run_docstring_examples(obj, globals(), name=name,
                                           optionflags=flags)
        else:
            fail, total = doctest.testmod(optionflags=flags)
            print(f"{fail} failures out of {total} tests")

**Footnotes**

[^1]: Examples containing both expected output and an exception are not supported. Trying to guess where one ends and the other begins is too error-prone, and that also makes for a confusing test.

# `!email.charset`: Representing character sets

<div class="module" data-synopsis="Character Sets">

email.charset

</div>

**Source code:** `Lib/email/charset.py`

------------------------------------------------------------------------

This module is part of the legacy (`Compat32`) email API. In the new API only the aliases table is used.

The remaining text in this section is the original documentation of the module.

This module provides a class `Charset` for representing character sets and character set conversions in email messages, as well as a character set registry and several convenience methods for manipulating this registry. Instances of `Charset` are used in several other modules within the `email` package.

Import this class from the `!email.charset` module.

<div class="Charset(input_charset=DEFAULT_CHARSET)">

Map character sets to their email properties.

This class provides information about the requirements imposed on email for a specific character set. It also provides convenience routines for converting between character sets, given the availability of the applicable codecs. Given a character set, it will do its best to provide information on how to use that character set in an email message in an RFC-compliant way.

Certain character sets must be encoded with quoted-printable or base64 when used in email headers or bodies. Certain character sets must be converted outright, and are not allowed in email.

Optional *input_charset* is as described below; it is always coerced to lower case. After being alias normalized it is also used as a lookup into the registry of character sets to find out the header encoding, body encoding, and output conversion codec to be used for the character set. For example, if *input_charset* is `iso-8859-1`, then headers and bodies will be encoded using quoted-printable and no output conversion codec is necessary. If *input_charset* is `euc-jp`, then headers will be encoded with base64, bodies will not be encoded, but output text will be converted from the `euc-jp` character set to the `iso-2022-jp` character set.

`Charset` instances have the following data attributes:

<div class="attribute">

input_charset

The initial character set specified. Common aliases are converted to their *official* email names (e.g. `latin_1` is converted to `iso-8859-1`). Defaults to 7-bit `us-ascii`.

</div>

<div class="attribute">

header_encoding

If the character set must be encoded before it can be used in an email header, this attribute will be set to `charset.QP` (for quoted-printable), `charset.BASE64` (for base64 encoding), or `charset.SHORTEST` for the shortest of QP or BASE64 encoding. Otherwise, it will be `None`.

</div>

<div class="attribute">

body_encoding

Same as *header_encoding*, but describes the encoding for the mail message's body, which indeed may be different than the header encoding. `charset.SHORTEST` is not allowed for *body_encoding*.

</div>

<div class="attribute">

output_charset

Some character sets must be converted before they can be used in email headers or bodies. If the *input_charset* is one of them, this attribute will contain the name of the character set output will be converted to. Otherwise, it will be `None`.

</div>

<div class="attribute">

input_codec

The name of the Python codec used to convert the *input_charset* to Unicode. If no conversion codec is necessary, this attribute will be `None`.

</div>

<div class="attribute">

output_codec

The name of the Python codec used to convert Unicode to the *output_charset*. If no conversion codec is necessary, this attribute will have the same value as the *input_codec*.

</div>

`Charset` instances also have the following methods:

<div class="method">

get_body_encoding()

Return the content transfer encoding used for body encoding.

This is either the string `quoted-printable` or `base64` depending on the encoding used, or it is a function, in which case you should call the function with a single argument, the Message object being encoded. The function should then set the `Content-Transfer-Encoding` header itself to whatever is appropriate.

Returns the string `quoted-printable` if *body_encoding* is `QP`, returns the string `base64` if *body_encoding* is `BASE64`, and returns the string `7bit` otherwise.

</div>

<div class="method">

get_output_charset()

Return the output character set.

This is the *output_charset* attribute if that is not `None`, otherwise it is *input_charset*.

</div>

<div class="method">

header_encode(string)

Header-encode the string *string*.

The type of encoding (base64 or quoted-printable) will be based on the *header_encoding* attribute.

</div>

<div class="method">

header_encode_lines(string, maxlengths)

Header-encode a *string* by converting it first to bytes.

This is similar to `header_encode` except that the string is fit into maximum line lengths as given by the argument *maxlengths*, which must be an iterator: each element returned from this iterator will provide the next maximum line length.

</div>

<div class="method">

body_encode(string)

Body-encode the string *string*.

The type of encoding (base64 or quoted-printable) will be based on the *body_encoding* attribute.

</div>

The `Charset` class also provides a number of methods to support standard operations and built-in functions.

<div class="method">

\_\_str\_\_()

Returns *input_charset* as a string coerced to lower case. `!__repr__` is an alias for `!__str__`.

</div>

<div class="method">

\_\_eq\_\_(other)

This method allows you to compare two `Charset` instances for equality.

</div>

<div class="method">

\_\_ne\_\_(other)

This method allows you to compare two `Charset` instances for inequality.

</div>

</div>

The `!email.charset` module also provides the following functions for adding new entries to the global character set, alias, and codec registries:

<div class="function">

add_charset(charset, header_enc=None, body_enc=None, output_charset=None)

Add character properties to the global registry.

*charset* is the input character set, and must be the canonical name of a character set.

Optional *header_enc* and *body_enc* is either `charset.QP` for quoted-printable, `charset.BASE64` for base64 encoding, `charset.SHORTEST` for the shortest of quoted-printable or base64 encoding, or `None` for no encoding. `SHORTEST` is only valid for *header_enc*. The default is `None` for no encoding.

Optional *output_charset* is the character set that the output should be in. Conversions will proceed from input charset, to Unicode, to the output charset when the method `Charset.body_encode` is called. The default is to output in the same character set as the input.

Both *input_charset* and *output_charset* must have Unicode codec entries in the module's character set-to-codec mapping; use `add_codec` to add codecs the module does not know about. See the `codecs` module's documentation for more information.

The global character set registry is kept in the module global dictionary `CHARSETS`.

</div>

<div class="function">

add_alias(alias, canonical)

Add a character set alias. *alias* is the alias name, e.g. `latin-1`. *canonical* is the character set's canonical name, e.g. `iso-8859-1`.

The global charset alias registry is kept in the module global dictionary `ALIASES`.

</div>

<div class="function">

add_codec(charset, codecname)

Add a codec that map characters in the given character set to and from Unicode.

*charset* is the canonical name of a character set. *codecname* is the name of a Python codec, as appropriate for the second argument to the `str`'s `~str.encode` method.

</div>

# `email.message.Message`: Representing an email message using the `~email.policy.compat32` API

<div class="module" data-synopsis="The base class representing email messages in a fashion
backward compatible with Python 3.2
:noindex:
:no-index:">

email.message

</div>

The `Message` class is very similar to the `~email.message.EmailMessage` class, without the methods added by that class, and with the default behavior of certain other methods being slightly different. We also document here some methods that, while supported by the `~email.message.EmailMessage` class, are not recommended unless you are dealing with legacy code.

The philosophy and structure of the two classes is otherwise the same.

This document describes the behavior under the default (for `Message`) policy `~email.policy.Compat32`. If you are going to use another policy, you should be using the `~email.message.EmailMessage` class instead.

An email message consists of *headers* and a *payload*. Headers must be `5322` style names and values, where the field name and value are separated by a colon. The colon is not part of either the field name or the field value. The payload may be a simple text message, or a binary object, or a structured sequence of sub-messages each with their own set of headers and their own payload. The latter type of payload is indicated by the message having a MIME type such as `multipart/\*` or `message/rfc822`.

The conceptual model provided by a `Message` object is that of an ordered dictionary of headers with additional methods for accessing both specialized information from the headers, for accessing the payload, for generating a serialized version of the message, and for recursively walking over the object tree. Note that duplicate headers are supported but special methods must be used to access them.

The `Message` pseudo-dictionary is indexed by the header names, which must be ASCII values. The values of the dictionary are strings that are supposed to contain only ASCII characters; there is some special handling for non-ASCII input, but it doesn't always produce the correct results. Headers are stored and returned in case-preserving form, but field names are matched case-insensitively. There may also be a single envelope header, also known as the *Unix-From* header or the `From_` header. The *payload* is either a string or bytes, in the case of simple message objects, or a list of `Message` objects, for MIME container documents (e.g. `multipart/\*` and `message/rfc822`).

Here are the methods of the `Message` class:

<div class="Message(policy=compat32)">

If *policy* is specified (it must be an instance of a `~email.policy` class) use the rules it specifies to update and serialize the representation of the message. If *policy* is not set, use the `compat32
<email.policy.Compat32>` policy, which maintains backward compatibility with the Python 3.2 version of the email package. For more information see the `~email.policy` documentation.

<div class="versionchanged">

3.3 The *policy* keyword argument was added.

</div>

<div class="method">

as_string(unixfrom=False, maxheaderlen=0, policy=None)

Return the entire message flattened as a string. When optional *unixfrom* is true, the envelope header is included in the returned string. *unixfrom* defaults to `False`. For backward compatibility reasons, *maxheaderlen* defaults to `0`, so if you want a different value you must override it explicitly (the value specified for *max_line_length* in the policy will be ignored by this method). The *policy* argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified *policy* will be passed to the `Generator`.

Flattening the message may trigger changes to the `Message` if defaults need to be filled in to complete the transformation to a string (for example, MIME boundaries may be generated or modified).

Note that this method is provided as a convenience and may not always format the message the way you want. For example, by default it does not do the mangling of lines that begin with `From` that is required by the Unix mbox format. For more flexibility, instantiate a `~email.generator.Generator` instance and use its `~email.generator.Generator.flatten` method directly. For example:

    from io import StringIO
    from email.generator import Generator
    fp = StringIO()
    g = Generator(fp, mangle_from_=True, maxheaderlen=60)
    g.flatten(msg)
    text = fp.getvalue()

If the message object contains binary data that is not encoded according to RFC standards, the non-compliant data will be replaced by Unicode "unknown character" code points. (See also `.as_bytes` and `~email.generator.BytesGenerator`.)

<div class="versionchanged">

3.4 the *policy* keyword argument was added.

</div>

</div>

<div class="method">

\_\_str\_\_()

Equivalent to `.as_string`. Allows `str(msg)` to produce a string containing the formatted message.

</div>

<div class="method">

as_bytes(unixfrom=False, policy=None)

Return the entire message flattened as a bytes object. When optional *unixfrom* is true, the envelope header is included in the returned string. *unixfrom* defaults to `False`. The *policy* argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified *policy* will be passed to the `BytesGenerator`.

Flattening the message may trigger changes to the `Message` if defaults need to be filled in to complete the transformation to a string (for example, MIME boundaries may be generated or modified).

Note that this method is provided as a convenience and may not always format the message the way you want. For example, by default it does not do the mangling of lines that begin with `From` that is required by the Unix mbox format. For more flexibility, instantiate a `~email.generator.BytesGenerator` instance and use its `~email.generator.BytesGenerator.flatten` method directly. For example:

    from io import BytesIO
    from email.generator import BytesGenerator
    fp = BytesIO()
    g = BytesGenerator(fp, mangle_from_=True, maxheaderlen=60)
    g.flatten(msg)
    text = fp.getvalue()

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

\_\_bytes\_\_()

Equivalent to `.as_bytes`. Allows `bytes(msg)` to produce a bytes object containing the formatted message.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

is_multipart()

Return `True` if the message's payload is a list of sub-`Message` objects, otherwise return `False`. When `is_multipart` returns `False`, the payload should be a string object (which might be a CTE encoded binary payload). (Note that `is_multipart` returning `True` does not necessarily mean that "msg.get_content_maintype() == 'multipart'" will return the `True`. For example, `is_multipart` will return `True` when the `Message` is of type `message/rfc822`.)

</div>

<div class="method">

set_unixfrom(unixfrom)

Set the message's envelope header to *unixfrom*, which should be a string.

</div>

<div class="method">

get_unixfrom()

Return the message's envelope header. Defaults to `None` if the envelope header was never set.

</div>

<div class="method">

attach(payload)

Add the given *payload* to the current payload, which must be `None` or a list of `Message` objects before the call. After the call, the payload will always be a list of `Message` objects. If you want to set the payload to a scalar object (e.g. a string), use `set_payload` instead.

This is a legacy method. On the `~email.message.EmailMessage` class its functionality is replaced by `~email.message.EmailMessage.set_content` and the related `make` and `add` methods.

</div>

<div class="method">

get_payload(i=None, decode=False)

Return the current payload, which will be a list of `Message` objects when `is_multipart` is `True`, or a string when `is_multipart` is `False`. If the payload is a list and you mutate the list object, you modify the message's payload in place.

With optional argument *i*, `get_payload` will return the *i*-th element of the payload, counting from zero, if `is_multipart` is `True`. An `IndexError` will be raised if *i* is less than 0 or greater than or equal to the number of items in the payload. If the payload is a string (i.e. `is_multipart` is `False`) and *i* is given, a `TypeError` is raised.

Optional *decode* is a flag indicating whether the payload should be decoded or not, according to the `Content-Transfer-Encoding` header. When `True` and the message is not a multipart, the payload will be decoded if this header's value is `quoted-printable` or `base64`. If some other encoding is used, or `Content-Transfer-Encoding` header is missing, the payload is returned as-is (undecoded). In all cases the returned value is binary data. If the message is a multipart and the *decode* flag is `True`, then `None` is returned. If the payload is base64 and it was not perfectly formed (missing padding, characters outside the base64 alphabet), then an appropriate defect will be added to the message's defect property (`~email.errors.InvalidBase64PaddingDefect` or `~email.errors.InvalidBase64CharactersDefect`, respectively).

> [!NOTE]
> A `quoted-printable` payload is decoded without stripping trailing whitespace, contrary to `2045` but matching common mail clients. Use `binascii.a2b_qp` with `strip_ws=True` (or `email.quoprimime.decode`) for RFC-compliant decoding.

When *decode* is `False` (the default) the body is returned as a string without decoding the `Content-Transfer-Encoding`. However, for a `Content-Transfer-Encoding` of 8bit, an attempt is made to decode the original bytes using the `charset` specified by the `Content-Type` header, using the `replace` error handler. If no `charset` is specified, or if the `charset` given is not recognized by the email package, the body is decoded using the default ASCII charset.

This is a legacy method. On the `~email.message.EmailMessage` class its functionality is replaced by `~email.message.EmailMessage.get_content` and `~email.message.EmailMessage.iter_parts`.

</div>

<div class="method">

set_payload(payload, charset=None)

Set the entire message object's payload to *payload*. It is the client's responsibility to ensure the payload invariants. Optional *charset* sets the message's default character set; see `set_charset` for details.

This is a legacy method. On the `~email.message.EmailMessage` class its functionality is replaced by `~email.message.EmailMessage.set_content`.

</div>

<div class="method">

set_charset(charset)

Set the character set of the payload to *charset*, which can either be a `~email.charset.Charset` instance (see `email.charset`), a string naming a character set, or `None`. If it is a string, it will be converted to a `~email.charset.Charset` instance. If *charset* is `None`, the `charset` parameter will be removed from the `Content-Type` header (the message will not be otherwise modified). Anything else will generate a `TypeError`.

If there is no existing `MIME-Version` header one will be added. If there is no existing `Content-Type` header, one will be added with a value of `text/plain`. Whether the `Content-Type` header already exists or not, its `charset` parameter will be set to *charset.output_charset*. If *charset.input_charset* and *charset.output_charset* differ, the payload will be re-encoded to the *output_charset*. If there is no existing `Content-Transfer-Encoding` header, then the payload will be transfer-encoded, if needed, using the specified `~email.charset.Charset`, and a header with the appropriate value will be added. If a `Content-Transfer-Encoding` header already exists, the payload is assumed to already be correctly encoded using that `Content-Transfer-Encoding` and is not modified.

This is a legacy method. On the `~email.message.EmailMessage` class its functionality is replaced by the *charset* parameter of the `email.message.EmailMessage.set_content` method.

</div>

<div class="method">

get_charset()

Return the `~email.charset.Charset` instance associated with the message's payload.

This is a legacy method. On the `~email.message.EmailMessage` class it always returns `None`.

</div>

The following methods implement a mapping-like interface for accessing the message's `2822` headers. Note that there are some semantic differences between these methods and a normal mapping (i.e. dictionary) interface. For example, in a dictionary there are no duplicate keys, but here there may be duplicate message headers. Also, in dictionaries there is no guaranteed order to the keys returned by `keys`, but in a `Message` object, headers are always returned in the order they appeared in the original message, or were added to the message later. Any header deleted and then re-added are always appended to the end of the header list.

These semantic differences are intentional and are biased toward maximal convenience.

Note that in all cases, any envelope header present in the message is not included in the mapping interface.

In a model generated from bytes, any header values that (in contravention of the RFCs) contain non-ASCII bytes will, when retrieved through this interface, be represented as `~email.header.Header` objects with a charset of `unknown-8bit`.

<div class="method">

\_\_len\_\_()

Return the total number of headers, including duplicates.

</div>

<div class="method">

\_\_contains\_\_(name)

Return `True` if the message object has a field named *name*. Matching is done case-insensitively and *name* should not include the trailing colon. Used for the `in` operator, e.g.:

    if 'message-id' in myMessage:
       print('Message-ID:', myMessage['message-id'])

</div>

<div class="method">

\_\_getitem\_\_(name)

Return the value of the named header field. *name* should not include the colon field separator. If the header is missing, `None` is returned; a `KeyError` is never raised.

Note that if the named field appears more than once in the message's headers, exactly which of those field values will be returned is undefined. Use the `get_all` method to get the values of all the extant named headers.

</div>

<div class="method">

\_\_setitem\_\_(name, val)

Add a header to the message with field name *name* and value *val*. The field is appended to the end of the message's existing fields.

Note that this does *not* overwrite or delete any existing header with the same name. If you want to ensure that the new header is the only one present in the message with field name *name*, delete the field first, e.g.:

    del msg['subject']
    msg['subject'] = 'Python roolz!'

</div>

<div class="method">

\_\_delitem\_\_(name)

Delete all occurrences of the field with name *name* from the message's headers. No exception is raised if the named field isn't present in the headers.

</div>

<div class="method">

keys()

Return a list of all the message's header field names.

</div>

<div class="method">

values()

Return a list of all the message's field values.

</div>

<div class="method">

items()

Return a list of 2-tuples containing all the message's field headers and values.

</div>

<div class="method">

get(name, failobj=None)

Return the value of the named header field. This is identical to `~object.__getitem__` except that optional *failobj* is returned if the named header is missing (defaults to `None`).

</div>

Here are some additional useful methods:

<div class="method">

get_all(name, failobj=None)

Return a list of all the values for the field named *name*. If there are no such named headers in the message, *failobj* is returned (defaults to `None`).

</div>

<div class="method">

add_header(<span id="name">name</span>, <span id="value">value</span>, \*\*\_params)

Extended header setting. This method is similar to `__setitem__` except that additional header parameters can be provided as keyword arguments. *\_name* is the header field to add and *\_value* is the *primary* value for the header.

For each item in the keyword argument dictionary *\_params*, the key is taken as the parameter name, with underscores converted to dashes (since dashes are illegal in Python identifiers). Normally, the parameter will be added as `key="value"` unless the value is `None`, in which case only the key will be added. If the value contains non-ASCII characters, it can be specified as a three tuple in the format `(CHARSET, LANGUAGE, VALUE)`, where `CHARSET` is a string naming the charset to be used to encode the value, `LANGUAGE` can usually be set to `None` or the empty string (see `2231` for other possibilities), and `VALUE` is the string value containing non-ASCII code points. If a three tuple is not passed and the value contains non-ASCII characters, it is automatically encoded in `2231` format using a `CHARSET` of `utf-8` and a `LANGUAGE` of `None`.

Here's an example:

    msg.add_header('Content-Disposition', 'attachment', filename='bud.gif')

This will add a header that looks like :

    Content-Disposition: attachment; filename="bud.gif"

An example with non-ASCII characters:

    msg.add_header('Content-Disposition', 'attachment',
                   filename=('iso-8859-1', '', 'Fußballer.ppt'))

Which produces :

    Content-Disposition: attachment; filename*="iso-8859-1''Fu%DFballer.ppt"

</div>

<div class="method">

replace_header(<span id="name">name</span>, <span id="value">value</span>)

Replace a header. Replace the first header found in the message that matches *\_name*, retaining header order and field name case. If no matching header was found, a `KeyError` is raised.

</div>

<div class="method">

get_content_type()

Return the message's content type. The returned string is coerced to lower case of the form `maintype/subtype`. If there was no `Content-Type` header in the message the default type as given by `get_default_type` will be returned. Since according to `2045`, messages always have a default type, `get_content_type` will always return a value.

`2045` defines a message's default type to be `text/plain` unless it appears inside a `multipart/digest` container, in which case it would be `message/rfc822`. If the `Content-Type` header has an invalid type specification, `2045` mandates that the default type be `text/plain`.

</div>

<div class="method">

get_content_maintype()

Return the message's main content type. This is the `maintype` part of the string returned by `get_content_type`.

</div>

<div class="method">

get_content_subtype()

Return the message's sub-content type. This is the `subtype` part of the string returned by `get_content_type`.

</div>

<div class="method">

get_default_type()

Return the default content type. Most messages have a default content type of `text/plain`, except for messages that are subparts of `multipart/digest` containers. Such subparts have a default content type of `message/rfc822`.

</div>

<div class="method">

set_default_type(ctype)

Set the default content type. *ctype* should either be `text/plain` or `message/rfc822`, although this is not enforced. The default content type is not stored in the `Content-Type` header.

</div>

<div class="method">

get_params(failobj=None, header='content-type', unquote=True)

Return the message's `Content-Type` parameters, as a list. The elements of the returned list are 2-tuples of key/value pairs, as split on the `'='` sign. The left hand side of the `'='` is the key, while the right hand side is the value. If there is no `'='` sign in the parameter the value is the empty string, otherwise the value is as described in `get_param` and is unquoted if optional *unquote* is `True` (the default).

Optional *failobj* is the object to return if there is no `Content-Type` header. Optional *header* is the header to search instead of `Content-Type`.

This is a legacy method. On the `~email.message.EmailMessage` class its functionality is replaced by the *params* property of the individual header objects returned by the header access methods.

</div>

<div class="method">

get_param(param, failobj=None, header='content-type', unquote=True)

Return the value of the `Content-Type` header's parameter *param* as a string. If the message has no `Content-Type` header or if there is no such parameter, then *failobj* is returned (defaults to `None`).

Optional *header* if given, specifies the message header to use instead of `Content-Type`.

Parameter keys are always compared case insensitively. The return value can either be a string, or a 3-tuple if the parameter was `2231` encoded. When it's a 3-tuple, the elements of the value are of the form `(CHARSET, LANGUAGE, VALUE)`. Note that both `CHARSET` and `LANGUAGE` can be `None`, in which case you should consider `VALUE` to be encoded in the `us-ascii` charset. You can usually ignore `LANGUAGE`.

If your application doesn't care whether the parameter was encoded as in `2231`, you can collapse the parameter value by calling `email.utils.collapse_rfc2231_value`, passing in the return value from `get_param`. This will return a suitably decoded Unicode string when the value is a tuple, or the original string unquoted if it isn't. For example:

    rawparam = msg.get_param('foo')
    param = email.utils.collapse_rfc2231_value(rawparam)

In any case, the parameter value (either the returned string, or the `VALUE` item in the 3-tuple) is always unquoted, unless *unquote* is set to `False`.

This is a legacy method. On the `~email.message.EmailMessage` class its functionality is replaced by the *params* property of the individual header objects returned by the header access methods.

</div>

<div class="method">

set_param(param, value, header='Content-Type', requote=True, charset=None, language='', replace=False)

Set a parameter in the `Content-Type` header. If the parameter already exists in the header, its value will be replaced with *value*. If the `Content-Type` header as not yet been defined for this message, it will be set to `text/plain` and the new parameter value will be appended as per `2045`.

Optional *header* specifies an alternative header to `Content-Type`, and all parameters will be quoted as necessary unless optional *requote* is `False` (the default is `True`).

If optional *charset* is specified, the parameter will be encoded according to `2231`. Optional *language* specifies the RFC 2231 language, defaulting to the empty string. Both *charset* and *language* should be strings.

If *replace* is `False` (the default) the header is moved to the end of the list of headers. If *replace* is `True`, the header will be updated in place.

<div class="versionchanged">

3.4 `replace` keyword was added.

</div>

</div>

<div class="method">

del_param(param, header='content-type', requote=True)

Remove the given parameter completely from the `Content-Type` header. The header will be re-written in place without the parameter or its value. All values will be quoted as necessary unless *requote* is `False` (the default is `True`). Optional *header* specifies an alternative to `Content-Type`.

</div>

<div class="method">

set_type(type, header='Content-Type', requote=True)

Set the main type and subtype for the `Content-Type` header. *type* must be a string in the form `maintype/subtype`, otherwise a `ValueError` is raised.

This method replaces the `Content-Type` header, keeping all the parameters in place. If *requote* is `False`, this leaves the existing header's quoting as is, otherwise the parameters will be quoted (the default).

An alternative header can be specified in the *header* argument. When the `Content-Type` header is set a `MIME-Version` header is also added.

This is a legacy method. On the `~email.message.EmailMessage` class its functionality is replaced by the `make_` and `add_` methods.

</div>

<div class="method">

get_filename(failobj=None)

Return the value of the `filename` parameter of the `Content-Disposition` header of the message. If the header does not have a `filename` parameter, this method falls back to looking for the `name` parameter on the `Content-Type` header. If neither is found, or the header is missing, then *failobj* is returned. The returned string will always be unquoted as per `email.utils.unquote`.

</div>

<div class="method">

get_boundary(failobj=None)

Return the value of the `boundary` parameter of the `Content-Type` header of the message, or *failobj* if either the header is missing, or has no `boundary` parameter. The returned string will always be unquoted as per `email.utils.unquote`.

</div>

<div class="method">

set_boundary(boundary)

Set the `boundary` parameter of the `Content-Type` header to *boundary*. `set_boundary` will always quote *boundary* if necessary. A `~email.errors.HeaderParseError` is raised if the message object has no `Content-Type` header.

Note that using this method is subtly different than deleting the old `Content-Type` header and adding a new one with the new boundary via `add_header`, because `set_boundary` preserves the order of the `Content-Type` header in the list of headers. However, it does *not* preserve any continuation lines which may have been present in the original `Content-Type` header.

</div>

<div class="method">

get_content_charset(failobj=None)

Return the `charset` parameter of the `Content-Type` header, coerced to lower case. If there is no `Content-Type` header, or if that header has no `charset` parameter, *failobj* is returned.

Note that this method differs from `get_charset` which returns the `~email.charset.Charset` instance for the default encoding of the message body.

</div>

<div class="method">

get_charsets(failobj=None)

Return a list containing the character set names in the message. If the message is a `multipart`, then the list will contain one element for each subpart in the payload, otherwise, it will be a list of length 1.

Each item in the list will be a string which is the value of the `charset` parameter in the `Content-Type` header for the represented subpart. However, if the subpart has no `Content-Type` header, no `charset` parameter, or is not of the `text` main MIME type, then that item in the returned list will be *failobj*.

</div>

<div class="method">

get_content_disposition()

Return the lowercased value (without parameters) of the message's `Content-Disposition` header if it has one, or `None`. The possible values for this method are *inline*, *attachment* or `None` if the message follows `2183`.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

walk()

The `walk` method is an all-purpose generator which can be used to iterate over all the parts and subparts of a message object tree, in depth-first traversal order. You will typically use `walk` as the iterator in a `for` loop; each iteration returns the next subpart.

Here's an example that prints the MIME type of every part of a multipart message structure:

<div class="testsetup">

import email from email import message_from_binary_file from os.path import join, dirname lib_dir = dirname(dirname(email.\_\_file\_\_)) file_path = join(lib_dir, 'test/test_email/data/msg_16.txt') with open(file_path, 'rb') as f: msg = message_from_binary_file(f) from email.iterators import <span id="structure">structure</span>

</div>

<div class="doctest">

\>\>\> for part in msg.walk(): ... print(part.get_content_type()) multipart/report text/plain message/delivery-status text/plain text/plain message/rfc822 text/plain

</div>

`walk` iterates over the subparts of any part where `is_multipart` returns `True`, even though `msg.get_content_maintype() == 'multipart'` may return `False`. We can see this in our example by making use of the `_structure` debug helper function:

<div class="doctest">

\>\>\> for part in msg.walk(): ... print(part.get_content_maintype() == 'multipart', ... part.is_multipart()) True True False False False True False False False False False True False False \>\>\> <span id="structure">structure</span>(msg) multipart/report text/plain message/delivery-status text/plain text/plain message/rfc822 text/plain

</div>

Here the `message` parts are not `multiparts`, but they do contain subparts. `is_multipart()` returns `True` and `walk` descends into the subparts.

</div>

`Message` objects can also optionally contain two instance attributes, which can be used when generating the plain text of a MIME message.

<div class="attribute">

preamble

The format of a MIME document allows for some text between the blank line following the headers, and the first multipart boundary string. Normally, this text is never visible in a MIME-aware mail reader because it falls outside the standard MIME armor. However, when viewing the raw text of the message, or when viewing the message in a non-MIME aware reader, this text can become visible.

The *preamble* attribute contains this leading extra-armor text for MIME documents. When the `~email.parser.Parser` discovers some text after the headers but before the first boundary string, it assigns this text to the message's *preamble* attribute. When the `~email.generator.Generator` is writing out the plain text representation of a MIME message, and it finds the message has a *preamble* attribute, it will write this text in the area between the headers and the first boundary. See `email.parser` and `email.generator` for details.

Note that if the message object has no preamble, the *preamble* attribute will be `None`.

</div>

<div class="attribute">

epilogue

The *epilogue* attribute acts the same way as the *preamble* attribute, except that it contains text that appears between the last boundary and the end of the message.

You do not need to set the epilogue to the empty string in order for the `~email.generator.Generator` to print a newline at the end of the file.

</div>

<div class="attribute">

defects

The *defects* attribute contains a list of all the problems found when parsing this message. See `email.errors` for a detailed description of the possible parsing defects.

</div>

</div>

# `!email.contentmanager`: Managing MIME Content

<div class="module" data-synopsis="Storing and Retrieving Content from MIME Parts">

email.contentmanager

</div>

**Source code:** `Lib/email/contentmanager.py`

------------------------------------------------------------------------

<div class="versionadded">

3.6[^1]

</div>

<div class="ContentManager()">

Base class for content managers. Provides the standard registry mechanisms to register converters between MIME content and other representations, as well as the `get_content` and `set_content` dispatch methods.

<div class="method">

get_content(msg, *args,*\*kw)

Look up a handler function based on the `mimetype` of *msg* (see next paragraph), call it, passing through all arguments, and return the result of the call. The expectation is that the handler will extract the payload from *msg* and return an object that encodes information about the extracted data.

To find the handler, look for the following keys in the registry, stopping with the first one found:

- the string representing the full MIME type (`maintype/subtype`)
- the string representing the `maintype`
- the empty string

If none of these keys produce a handler, raise a `KeyError` for the full MIME type.

</div>

<div class="method">

set_content(msg, obj, *args,*\*kw)

If the `maintype` is `multipart`, raise a `TypeError`; otherwise look up a handler function based on the type of *obj* (see next paragraph), call `~email.message.EmailMessage.clear_content` on the *msg*, and call the handler function, passing through all arguments. The expectation is that the handler will transform and store *obj* into *msg*, possibly making other changes to *msg* as well, such as adding various MIME headers to encode information needed to interpret the stored data.

To find the handler, obtain the type of *obj* (`typ = type(obj)`), and look for the following keys in the registry, stopping with the first one found:

- the type itself (`typ`)
- the type's fully qualified name (`typ.__module__ + '.' + typ.__qualname__`).
- the type's `qualname <type.__qualname__>` (`typ.__qualname__`)
- the type's `name <type.__name__>` (`typ.__name__`).

If none of the above match, repeat all of the checks above for each of the types in the `MRO` (`typ.__mro__ <type.__mro__>`). Finally, if no other key yields a handler, check for a handler for the key `None`. If there is no handler for `None`, raise a `KeyError` for the fully qualified name of the type.

Also add a `MIME-Version` header if one is not present (see also `.MIMEPart`).

</div>

<div class="method">

add_get_handler(key, handler)

Record the function *handler* as the handler for *key*. For the possible values of *key*, see `get_content`.

</div>

<div class="method">

add_set_handler(typekey, handler)

Record *handler* as the function to call when an object of a type matching *typekey* is passed to `set_content`. For the possible values of *typekey*, see `set_content`.

</div>

</div>

## Content Manager Instances

Currently the email package provides only one concrete content manager, `raw_data_manager`, although more may be added in the future. `raw_data_manager` is the `~email.policy.EmailPolicy.content_manager` provided by `~email.policy.EmailPolicy` and its derivatives.

<div class="data">

raw_data_manager

This content manager provides only a minimum interface beyond that provided by `~email.message.Message` itself: it deals only with text, raw bytes, and `~email.message.Message` objects. Nevertheless, it provides significant advantages compared to the base API: `get_content` on a text part will return a string without the application needing to manually decode it, `set_content` provides a rich set of options for controlling the headers added to a part and controlling the content transfer encoding, and it enables the use of the various `add_` methods, thereby simplifying the creation of multipart messages.

<div class="method">

get_content(msg, errors='replace')

Return the payload of the part as either a string (for `text` parts), an `~email.message.EmailMessage` object (for `message/rfc822` parts), or a `bytes` object (for all other non-multipart types). Raise a `KeyError` if called on a `multipart`. If the part is a `text` part and *errors* is specified, use it as the error handler when decoding the payload to a string. The default error handler is `replace`.

</div>

<div class="method">

set_content(msg, \<'str'\>, subtype="plain", charset='utf-8', cte=None, disposition=None, filename=None, cid=None, params=None, headers=None) set_content(msg, \<'bytes'\>, maintype, subtype, cte="base64", disposition=None, filename=None, cid=None, params=None, headers=None) set_content(msg, \<'EmailMessage'\>, cte=None, disposition=None, filename=None, cid=None, params=None, headers=None)

Add headers and payload to *msg*:

Add a `Content-Type` header with a `maintype/subtype` value.

- For `str`, set the MIME `maintype` to `text`, and set the subtype to *subtype* if it is specified, or `plain` if it is not.
- For `bytes`, use the specified *maintype* and *subtype*, or raise a `TypeError` if they are not specified.
- For `~email.message.EmailMessage` objects, set the maintype to `message`, and set the subtype to *subtype* if it is specified or `rfc822` if it is not. If *subtype* is `partial`, raise an error (`bytes` objects must be used to construct `message/partial` parts).

If *charset* is provided (which is valid only for `str`), encode the string to bytes using the specified character set. The default is `utf-8`. If the specified *charset* is a known alias for a standard MIME charset name, use the standard charset instead.

If *cte* is set, encode the payload using the specified content transfer encoding, and set the `Content-Transfer-Encoding` header to that value. Possible values for *cte* are `quoted-printable`, `base64`, `7bit`, `8bit`, and `binary`. If the input cannot be encoded in the specified encoding (for example, specifying a *cte* of `7bit` for an input that contains non-ASCII values), raise a `ValueError`.

- For `str` objects, if *cte* is not set use heuristics to determine the most compact encoding. Prior to encoding, `str.splitlines` is used to normalize all line boundaries, ensuring that each line of the payload is terminated by the current policy's `~email.policy.Policy.linesep` property (even if the original string did not end with one).
- For `bytes` objects, *cte* is taken to be base64 if not set, and the aforementioned newline translation is not performed.
- For `~email.message.EmailMessage`, per `2046`, raise an error if a *cte* of `quoted-printable` or `base64` is requested for *subtype* `rfc822`, and for any *cte* other than `7bit` for *subtype* `external-body`. For `message/rfc822`, use `8bit` if *cte* is not specified. For all other values of *subtype*, use `7bit`.

> [!NOTE]
> A *cte* of `binary` does not actually work correctly yet. The `EmailMessage` object as modified by `set_content` is correct, but `~email.generator.BytesGenerator` does not serialize it correctly.

If *disposition* is set, use it as the value of the `Content-Disposition` header. If not specified, and *filename* is specified, add the header with the value `attachment`. If *disposition* is not specified and *filename* is also not specified, do not add the header. The only valid values for *disposition* are `attachment` and `inline`.

If *filename* is specified, use it as the value of the `filename` parameter of the `Content-Disposition` header.

If *cid* is specified, add a `Content-ID` header with *cid* as its value.

If *params* is specified, iterate its `items` method and use the resulting `(key, value)` pairs to set additional parameters on the `Content-Type` header.

If *headers* is specified and is a list of strings of the form `headername: headervalue` or a list of `header` objects (distinguished from strings by having a `name` attribute), add the headers to *msg*.

</div>

</div>

**Footnotes**

[^1]: Originally added in 3.4 as a `provisional module <provisional
    package>`

# `!email.encoders`: Encoders

<div class="module" data-synopsis="Encoders for email message payloads.">

email.encoders

</div>

**Source code:** `Lib/email/encoders.py`

------------------------------------------------------------------------

This module is part of the legacy (`Compat32`) email API. In the new API the functionality is provided by the *cte* parameter of the `~email.message.EmailMessage.set_content` method.

This module is deprecated in Python 3. The functions provided here should not be called explicitly since the `~email.mime.text.MIMEText` class sets the content type and CTE header using the *\_subtype* and *\_charset* values passed during the instantiation of that class.

The remaining text in this section is the original documentation of the module.

When creating `~email.message.Message` objects from scratch, you often need to encode the payloads for transport through compliant mail servers. This is especially true for `image/\*` and `text/\*` type messages containing binary data.

The `email` package provides some convenient encoders in its `!encoders` module. These encoders are actually used by the `~email.mime.audio.MIMEAudio` and `~email.mime.image.MIMEImage` class constructors to provide default encodings. All encoder functions take exactly one argument, the message object to encode. They usually extract the payload, encode it, and reset the payload to this newly encoded value. They should also set the `Content-Transfer-Encoding` header as appropriate.

Note that these functions are not meaningful for a multipart message. They must be applied to individual subparts instead, and will raise a `TypeError` if passed a message whose type is multipart.

Here are the encoding functions provided:

<div class="function">

encode_quopri(msg)

Encodes the payload into quoted-printable form and sets the `Content-Transfer-Encoding` header to `quoted-printable`[^1]. This is a good encoding to use when most of your payload is normal printable data, but contains a few unprintable characters.

</div>

<div class="function">

encode_base64(msg)

Encodes the payload into base64 form and sets the `Content-Transfer-Encoding` header to `base64`. This is a good encoding to use when most of your payload is unprintable data since it is a more compact form than quoted-printable. The drawback of base64 encoding is that it renders the text non-human readable.

</div>

<div class="function">

encode_7or8bit(msg)

This doesn't actually modify the message's payload, but it does set the `Content-Transfer-Encoding` header to either `7bit` or `8bit` as appropriate, based on the payload data.

</div>

<div class="function">

encode_noop(msg)

This does nothing; it doesn't even set the `Content-Transfer-Encoding` header.

</div>

**Footnotes**

[^1]: Note that encoding with `encode_quopri` also encodes all tabs and space characters in the data.

# `!email.errors`: Exception and Defect classes

<div class="module" data-synopsis="The exception classes used by the email package.">

email.errors

</div>

**Source code:** `Lib/email/errors.py`

------------------------------------------------------------------------

The following exception classes are defined in the `!email.errors` module:

<div class="exception">

MessageError()

This is the base class for all exceptions that the `email` package can raise. It is derived from the standard `Exception` class and defines no additional methods.

</div>

<div class="exception">

MessageParseError()

This is the base class for exceptions raised by the `~email.parser.Parser` class. It is derived from `MessageError`. This class is also used internally by the parser used by `~email.headerregistry`.

</div>

<div class="exception">

HeaderParseError()

Raised under some error conditions when parsing the `5322` headers of a message, this class is derived from `MessageParseError`. The `~email.message.EmailMessage.set_boundary` method will raise this error if the content type is unknown when the method is called. `~email.header.Header` may raise this error for certain base64 decoding errors, and when an attempt is made to create a header that appears to contain an embedded header (that is, there is what is supposed to be a continuation line that has no leading whitespace and looks like a header).

</div>

<div class="exception">

BoundaryError()

Deprecated and no longer used.

</div>

<div class="exception">

MultipartConversionError()

Raised if the `~email.message.Message.attach` method is called on an instance of a class derived from `~email.mime.nonmultipart.MIMENonMultipart` (e.g. `~email.mime.image.MIMEImage`). `MultipartConversionError` multiply inherits from `MessageError` and the built-in `TypeError`.

</div>

<div class="exception">

HeaderWriteError()

Raised when an error occurs when the `~email.generator` outputs headers.

</div>

<div class="exception">

MessageDefect()

This is the base class for all defects found when parsing email messages. It is derived from `ValueError`.

</div>

<div class="exception">

HeaderDefect()

This is the base class for all defects found when parsing email headers. It is derived from `MessageDefect`.

</div>

Here is the list of the defects that the `~email.parser.FeedParser` can find while parsing messages. Note that the defects are added to the message where the problem was found, so for example, if a message nested inside a `multipart/alternative` had a malformed header, that nested message object would have a defect, but the containing messages would not.

All defect classes are subclassed from `email.errors.MessageDefect`.

<div class="exception">

NoBoundaryInMultipartDefect

A message claimed to be a multipart, but had no `boundary` parameter.

</div>

<div class="exception">

StartBoundaryNotFoundDefect

The start boundary claimed in the `Content-Type` header was never found.

</div>

<div class="exception">

CloseBoundaryNotFoundDefect

A start boundary was found, but no corresponding close boundary was ever found.

<div class="versionadded">

3.3

</div>

</div>

<div class="exception">

FirstHeaderLineIsContinuationDefect

The message had a continuation line as its first header line.

</div>

<div class="exception">

MisplacedEnvelopeHeaderDefect

A "Unix From" header was found in the middle of a header block.

</div>

<div class="exception">

MissingHeaderBodySeparatorDefect

A line was found while parsing headers that had no leading white space but contained no ':'. Parsing continues assuming that the line represents the first line of the body.

<div class="versionadded">

3.3

</div>

</div>

<div class="exception">

MalformedHeaderDefect

A header was found that was missing a colon, or was otherwise malformed.

<div class="deprecated">

3.3 This defect has not been used for several Python versions.

</div>

</div>

<div class="exception">

MultipartInvariantViolationDefect

A message claimed to be a `multipart`, but no subparts were found. Note that when a message has this defect, its `~email.message.Message.is_multipart` method may return `False` even though its content type claims to be `multipart`.

</div>

<div class="exception">

InvalidBase64PaddingDefect

When decoding a block of base64 encoded bytes, the padding was not correct. Enough padding is added to perform the decode, but the resulting decoded bytes may be invalid.

</div>

<div class="exception">

InvalidBase64CharactersDefect

When decoding a block of base64 encoded bytes, characters outside the base64 alphabet were encountered. The characters are ignored, but the resulting decoded bytes may be invalid.

</div>

<div class="exception">

InvalidBase64LengthDefect

When decoding a block of base64 encoded bytes, the number of non-padding base64 characters was invalid (1 more than a multiple of 4). The encoded block was kept as-is.

</div>

<div class="exception">

InvalidDateDefect

When decoding an invalid or unparsable date field. The original value is kept as-is.

</div>

# `email`: Examples

Here are a few examples of how to use the `email` package to read, write, and send simple email messages, as well as more complex MIME messages.

First, let's see how to create and send a simple text message (both the text content and the addresses may contain Unicode characters):

<div class="literalinclude">

../includes/email-simple.py

</div>

Parsing `822` headers can easily be done by the using the classes from the `~email.parser` module:

<div class="literalinclude">

../includes/email-headers.py

</div>

Here's an example of how to send a MIME message containing a bunch of family pictures that may be residing in a directory:

<div class="literalinclude">

../includes/email-mime.py

</div>

Here's an example of how to send the entire contents of a directory as an email message:[^1]

<div class="literalinclude">

../includes/email-dir.py

</div>

Here's an example of how to unpack a MIME message like the one above, into a directory of files:

<div class="literalinclude">

../includes/email-unpack.py

</div>

Here's an example of how to create an HTML message with an alternative plain text version. To make things a bit more interesting, we include a related image in the html part, and we save a copy of what we are going to send to disk, as well as sending it.

<div class="literalinclude">

../includes/email-alternative.py

</div>

If we were sent the message from the last example, here is one way we could process it:

<div class="literalinclude">

../includes/email-read-alternative.py

</div>

Up to the prompt, the output from the above is:

```
To: Penelope Pussycat <penelope@example.com>, Fabrette Pussycat <fabrette@example.com>
From: Pepé Le Pew <pepe@example.com>
Subject: Pourquoi pas des asperges pour ce midi ?

Salut!

Cette recette [1] sera sûrement un très bon repas.
```

**Footnotes**

[^1]: Thanks to Matthew Dixon Cowles for the original inspiration and examples.

# `!email.generator`: Generating MIME documents

<div class="module" data-synopsis="Generate flat text email messages from a message structure.">

email.generator

</div>

**Source code:** `Lib/email/generator.py`

------------------------------------------------------------------------

One of the most common tasks is to generate the flat (serialized) version of the email message represented by a message object structure. You will need to do this if you want to send your message via `smtplib.SMTP.sendmail`, or print the message on the console. Taking a message object structure and producing a serialized representation is the job of the generator classes.

As with the `email.parser` module, you aren't limited to the functionality of the bundled generator; you could write one from scratch yourself. However the bundled generator knows how to generate most email in a standards-compliant way, should handle MIME and non-MIME email messages just fine, and is designed so that the bytes-oriented parsing and generation operations are inverses, assuming the same non-transforming `~email.policy` is used for both. That is, parsing the serialized byte stream via the `~email.parser.BytesParser` class and then regenerating the serialized byte stream using `BytesGenerator` should produce output identical to the input[^1]. (On the other hand, using the generator on an `~email.message.EmailMessage` constructed by program may result in changes to the `~email.message.EmailMessage` object as defaults are filled in.)

The `Generator` class can be used to flatten a message into a text (as opposed to binary) serialized representation, but since Unicode cannot represent binary data directly, the message is of necessity transformed into something that contains only ASCII characters, using the standard email RFC Content Transfer Encoding techniques for encoding email messages for transport over channels that are not "8 bit clean".

To accommodate reproducible processing of SMIME-signed messages `Generator` disables header folding for message parts of type `multipart/signed` and all subparts.

<div class="BytesGenerator(outfp, mangle_from_=None, maxheaderlen=None, *, \ policy=None)">

Return a `BytesGenerator` object that will write any message provided to the `flatten` method, or any surrogateescape encoded text provided to the `write` method, to the `file-like object` *outfp*. *outfp* must support a `write` method that accepts binary data.

If optional *mangle_from\_* is `True`, put a `>` character in front of any line in the body that starts with the exact string `"From "`, that is `From` followed by a space at the beginning of a line. *mangle_from\_* defaults to the value of the `~email.policy.Policy.mangle_from_` setting of the *policy* (which is `True` for the `~email.policy.compat32` policy and `False` for all others). *mangle_from\_* is intended for use when messages are stored in Unix mbox format (see `mailbox` and [WHY THE CONTENT-LENGTH FORMAT IS BAD](https://www.jwz.org/doc/content-length.html)).

If *maxheaderlen* is not `None`, refold any header lines that are longer than *maxheaderlen*, or if `0`, do not rewrap any headers. If *maxheaderlen* is `None` (the default), wrap headers and other message lines according to the *policy* settings.

If *policy* is specified, use that policy to control message generation. If *policy* is `None` (the default), use the policy associated with the `~email.message.Message` or `~email.message.EmailMessage` object passed to `flatten` to control the message generation. See `email.policy` for details on what *policy* controls.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Added the *policy* keyword.

</div>

<div class="versionchanged">

3.6 The default behavior of the *mangle_from\_* and *maxheaderlen* parameters is to follow the policy.

</div>

<div class="method">

flatten(msg, unixfrom=False, linesep=None)

Print the textual representation of the message object structure rooted at *msg* to the output file specified when the `BytesGenerator` instance was created.

If the `~email.policy` option `~email.policy.Policy.cte_type` is `8bit` (the default), copy any headers in the original parsed message that have not been modified to the output with any bytes with the high bit set reproduced as in the original, and preserve the non-ASCII `Content-Transfer-Encoding` of any body parts that have them. If `cte_type` is `7bit`, convert the bytes with the high bit set as needed using an ASCII-compatible `Content-Transfer-Encoding`. That is, transform parts with non-ASCII `Content-Transfer-Encoding` (`Content-Transfer-Encoding: 8bit`) to an ASCII compatible `Content-Transfer-Encoding`, and encode RFC-invalid non-ASCII bytes in headers using the MIME `unknown-8bit` character set, thus rendering them RFC-compliant.

If *unixfrom* is `True`, print the envelope header delimiter used by the Unix mailbox format (see `mailbox`) before the first of the `5322` headers of the root message object. If the root object has no envelope header, craft a standard one. The default is `False`. Note that for subparts, no envelope header is ever printed.

If *linesep* is not `None`, use it as the separator character between all the lines of the flattened message. If *linesep* is `None` (the default), use the value specified in the *policy*.

</div>

<div class="method">

clone(fp)

Return an independent clone of this `BytesGenerator` instance with the exact same option settings, and *fp* as the new *outfp*.

</div>

<div class="method">

write(s)

Encode *s* using the `ASCII` codec and the `surrogateescape` error handler, and pass it to the *write* method of the *outfp* passed to the `BytesGenerator`'s constructor.

</div>

</div>

As a convenience, `~email.message.EmailMessage` provides the methods `~email.message.EmailMessage.as_bytes` and `bytes(aMessage)` (a.k.a. `~email.message.EmailMessage.__bytes__`), which simplify the generation of a serialized binary representation of a message object. For more detail, see `email.message`.

Because strings cannot represent binary data, the `Generator` class must convert any binary data in any message it flattens to an ASCII compatible format, by converting them to an ASCII compatible `Content-Transfer-Encoding`. Using the terminology of the email RFCs, you can think of this as `Generator` serializing to an I/O stream that is not "8 bit clean". In other words, most applications will want to be using `BytesGenerator`, and not `Generator`.

<div class="Generator(outfp, mangle_from_=None, maxheaderlen=None, *, \ policy=None)">

Return a `Generator` object that will write any message provided to the `flatten` method, or any text provided to the `write` method, to the `file-like object` *outfp*. *outfp* must support a `write` method that accepts string data.

If optional *mangle_from\_* is `True`, put a `>` character in front of any line in the body that starts with the exact string `"From "`, that is `From` followed by a space at the beginning of a line. *mangle_from\_* defaults to the value of the `~email.policy.Policy.mangle_from_` setting of the *policy* (which is `True` for the `~email.policy.compat32` policy and `False` for all others). *mangle_from\_* is intended for use when messages are stored in Unix mbox format (see `mailbox` and [WHY THE CONTENT-LENGTH FORMAT IS BAD](https://www.jwz.org/doc/content-length.html)).

If *maxheaderlen* is not `None`, refold any header lines that are longer than *maxheaderlen*, or if `0`, do not rewrap any headers. If *maxheaderlen* is `None` (the default), wrap headers and other message lines according to the *policy* settings.

If *policy* is specified, use that policy to control message generation. If *policy* is `None` (the default), use the policy associated with the `~email.message.Message` or `~email.message.EmailMessage` object passed to `flatten` to control the message generation. See `email.policy` for details on what *policy* controls.

<div class="versionchanged">

3.3 Added the *policy* keyword.

</div>

<div class="versionchanged">

3.6 The default behavior of the *mangle_from\_* and *maxheaderlen* parameters is to follow the policy.

</div>

<div class="method">

flatten(msg, unixfrom=False, linesep=None)

Print the textual representation of the message object structure rooted at *msg* to the output file specified when the `Generator` instance was created.

If the `~email.policy` option `~email.policy.Policy.cte_type` is `8bit`, generate the message as if the option were set to `7bit`. (This is required because strings cannot represent non-ASCII bytes.) Convert any bytes with the high bit set as needed using an ASCII-compatible `Content-Transfer-Encoding`. That is, transform parts with non-ASCII `Content-Transfer-Encoding` (`Content-Transfer-Encoding: 8bit`) to an ASCII compatible `Content-Transfer-Encoding`, and encode RFC-invalid non-ASCII bytes in headers using the MIME `unknown-8bit` character set, thus rendering them RFC-compliant.

If *unixfrom* is `True`, print the envelope header delimiter used by the Unix mailbox format (see `mailbox`) before the first of the `5322` headers of the root message object. If the root object has no envelope header, craft a standard one. The default is `False`. Note that for subparts, no envelope header is ever printed.

If *linesep* is not `None`, use it as the separator character between all the lines of the flattened message. If *linesep* is `None` (the default), use the value specified in the *policy*.

<div class="versionchanged">

3.2 Added support for re-encoding `8bit` message bodies, and the *linesep* argument.

</div>

</div>

<div class="method">

clone(fp)

Return an independent clone of this `Generator` instance with the exact same options, and *fp* as the new *outfp*.

</div>

<div class="method">

write(s)

Write *s* to the *write* method of the *outfp* passed to the `Generator`'s constructor. This provides just enough file-like API for `Generator` instances to be used in the `print` function.

</div>

</div>

As a convenience, `~email.message.EmailMessage` provides the methods `~email.message.EmailMessage.as_string` and `str(aMessage)` (a.k.a. `~email.message.EmailMessage.__str__`), which simplify the generation of a formatted string representation of a message object. For more detail, see `email.message`.

The `!email.generator` module also provides a derived class, `DecodedGenerator`, which is like the `Generator` base class, except that non-`text` parts are not serialized, but are instead represented in the output stream by a string derived from a template filled in with information about the part.

<div class="DecodedGenerator(outfp, mangle_from_=None, maxheaderlen=None, \ fmt=None, *, policy=None)">

Act like `Generator`, except that for any subpart of the message passed to `Generator.flatten`, if the subpart is of main type `text`, print the decoded payload of the subpart, and if the main type is not `text`, instead of printing it fill in the string *fmt* using information from the part and print the resulting filled-in string.

To fill in *fmt*, execute `fmt % part_info`, where `part_info` is a dictionary composed of the following keys and values:

- `type` -- Full MIME type of the non-`text` part
- `maintype` -- Main MIME type of the non-`text` part
- `subtype` -- Sub-MIME type of the non-`text` part
- `filename` -- Filename of the non-`text` part
- `description` -- Description associated with the non-`text` part
- `encoding` -- Content transfer encoding of the non-`text` part

If *fmt* is `None`, use the following default *fmt*:

> "\[Non-text (%(type)s) part of message omitted, filename %(filename)s\]"

Optional *mangle_from\_* and *maxheaderlen* are as with the `Generator` base class.

</div>

**Footnotes**

[^1]: This statement assumes that you use the appropriate setting for `unixfrom`, and that there are no `email.policy` settings calling for automatic adjustments (for example, `~email.policy.EmailPolicy.refold_source` must be `none`, which is *not* the default). It is also not 100% true, since if the message does not conform to the RFC standards occasionally information about the exact original text is lost during parsing error recovery. It is a goal to fix these latter edge cases when possible.

# `!email.header`: Internationalized headers

<div class="module" data-synopsis="Representing non-ASCII headers">

email.header

</div>

**Source code:** `Lib/email/header.py`

------------------------------------------------------------------------

This module is part of the legacy (`Compat32`) email API. In the current API encoding and decoding of headers is handled transparently by the dictionary-like API of the `~email.message.EmailMessage` class. In addition to uses in legacy code, this module can be useful in applications that need to completely control the character sets used when encoding headers.

The remaining text in this section is the original documentation of the module.

`2822` is the base standard that describes the format of email messages. It derives from the older `822` standard which came into widespread use at a time when most email was composed of ASCII characters only. `2822` is a specification written assuming email contains only 7-bit ASCII characters.

Of course, as email has been deployed worldwide, it has become internationalized, such that language specific character sets can now be used in email messages. The base standard still requires email messages to be transferred using only 7-bit ASCII characters, so a slew of RFCs have been written describing how to encode email containing non-ASCII characters into `2822`-compliant format. These RFCs include `2045`, `2046`, `2047`, and `2231`. The `email` package supports these standards in its `!email.header` and `email.charset` modules.

If you want to include non-ASCII characters in your email headers, say in the `Subject` or `To` fields, you should use the `Header` class and assign the field in the `~email.message.Message` object to an instance of `Header` instead of using a string for the header value. Import the `Header` class from the `!email.header` module. For example:

    >>> from email.message import Message
    >>> from email.header import Header
    >>> msg = Message()
    >>> h = Header('p\xf6stal', 'iso-8859-1')
    >>> msg['Subject'] = h
    >>> msg.as_string()
    'Subject: =?iso-8859-1?q?p=F6stal?=\n\n'

Notice here how we wanted the `Subject` field to contain a non-ASCII character? We did this by creating a `Header` instance and passing in the character set to use when encoding it. When the subsequent `~email.message.Message` instance was flattened, the `Subject` field was properly `2047` encoded. MIME-aware mail readers would show this header using the embedded ISO-8859-1 character.

Here is the `Header` class description:

<div class="Header(s=None, charset=None, maxlinelen=None, header_name=None, continuation_ws=' ', errors='strict')">

Create a MIME-compliant header that can contain strings in different character sets.

Optional *s* is the initial header value. If `None` (the default), the initial header value is not set. You can later append to the header with `append` method calls. *s* may be an instance of `bytes` or `str`, but see the `append` documentation for semantics.

Optional *charset* serves two purposes: it has the same meaning as the *charset* argument to the `append` method. It also sets the default character set for all subsequent `append` calls that omit the *charset* argument. If *charset* is not provided in the constructor (the default), the `us-ascii` character set is used both as *s*'s initial charset and as the default for subsequent `append` calls.

The maximum line length can be specified explicitly via *maxlinelen*. For splitting the first line to a shorter value (to account for the field header which isn't included in *s*, e.g. `Subject`) pass in the name of the field in *header_name*. The default *maxlinelen* is 78, and the default value for *header_name* is `None`, meaning it is not taken into account for the first line of a long, split header.

Optional *continuation_ws* must be `2822`-compliant folding whitespace, and is usually either a space or a hard tab character. This character will be prepended to continuation lines. *continuation_ws* defaults to a single space character.

Optional *errors* is passed straight through to the `append` method.

<div class="method">

append(s, charset=None, errors='strict')

Append the string *s* to the MIME header.

Optional *charset*, if given, should be a `~email.charset.Charset` instance (see `email.charset`) or the name of a character set, which will be converted to a `~email.charset.Charset` instance. A value of `None` (the default) means that the *charset* given in the constructor is used.

*s* may be an instance of `bytes` or `str`. If it is an instance of `bytes`, then *charset* is the encoding of that byte string, and a `UnicodeError` will be raised if the string cannot be decoded with that character set.

If *s* is an instance of `str`, then *charset* is a hint specifying the character set of the characters in the string.

In either case, when producing an `2822`-compliant header using `2047` rules, the string will be encoded using the output codec of the charset. If the string cannot be encoded using the output codec, a UnicodeError will be raised.

Optional *errors* is passed as the errors argument to the decode call if *s* is a byte string.

</div>

<div class="method">

encode(splitchars=';, t', maxlinelen=None, linesep='n')

Encode a message header into an RFC-compliant format, possibly wrapping long lines and encapsulating non-ASCII parts in base64 or quoted-printable encodings.

Optional *splitchars* is a string containing characters which should be given extra weight by the splitting algorithm during normal header wrapping. This is in very rough support of [RFC 2822](http://www.faqs.org/rfcs/rfc2822.html "RFC 2822")'s 'higher level syntactic breaks': split points preceded by a splitchar are preferred during line splitting, with the characters preferred in the order in which they appear in the string. Space and tab may be included in the string to indicate whether preference should be given to one over the other as a split point when other split chars do not appear in the line being split. Splitchars does not affect [RFC 2047](http://www.faqs.org/rfcs/rfc2047.html "RFC 2047") encoded lines.

*maxlinelen*, if given, overrides the instance's value for the maximum line length.

*linesep* specifies the characters used to separate the lines of the folded header. It defaults to the most useful value for Python application code (`\n`), but `\r\n` can be specified in order to produce headers with RFC-compliant line separators.

<div class="versionchanged">

3.2 Added the *linesep* argument.

</div>

</div>

The `Header` class also provides a number of methods to support standard operators and built-in functions.

<div class="method">

\_\_str\_\_()

Returns an approximation of the `Header` as a string, using an unlimited line length. All pieces are decoded using the specified encoding and joined together appropriately. Any pieces with a charset of `'unknown-8bit'` are decoded as ASCII using the `'replace'` error handler.

<div class="versionchanged">

3.2 Added handling for the `'unknown-8bit'` charset.

</div>

</div>

<div class="method">

\_\_eq\_\_(other)

This method allows you to compare two `Header` instances for equality.

</div>

<div class="method">

\_\_ne\_\_(other)

This method allows you to compare two `Header` instances for inequality.

</div>

</div>

The `!email.header` module also provides the following convenient functions.

<div class="function">

decode_header(header)

Decode a message header value without converting the character set. The header value is in *header*.

For historical reasons, this function may return either:

1.  A list of pairs containing each of the decoded parts of the header, `(decoded_bytes, charset)`, where *decoded_bytes* is always an instance of `bytes`, and *charset* is either:

    > - A lower case string containing the name of the character set specified.
    > - `None` for non-encoded parts of the header.

2.  A list of length 1 containing a pair `(string, None)`, where *string* is always an instance of `str`.

An `email.errors.HeaderParseError` may be raised when certain decoding errors occur (e.g. a base64 decoding exception).

Here are examples:

> \>\>\> from email.header import decode_header \>\>\> decode_header('=?iso-8859-1?q?p=F6stal?=') \[(b'pxf6stal', 'iso-8859-1')\] \>\>\> decode_header('unencoded_string') \[('unencoded_string', None)\] \>\>\> decode_header('bar =?utf-8?B?ZsOzbw==?=') \[(b'bar ', None), (b'fxc3xb3o', 'utf-8')\]

> [!NOTE]
> This function exists for backwards compatibility only. For new code, we recommend using `email.headerregistry.HeaderRegistry`.

</div>

<div class="function">

make_header(decoded_seq, maxlinelen=None, header_name=None, continuation_ws=' ')

Create a `Header` instance from a sequence of pairs as returned by `decode_header`.

`decode_header` takes a header value string and returns a sequence of pairs of the format `(decoded_string, charset)` where *charset* is the name of the character set.

This function takes one of those sequence of pairs and returns a `Header` instance. Optional *maxlinelen*, *header_name*, and *continuation_ws* are as in the `Header` constructor.

> [!NOTE]
> This function exists for backwards compatibility only, and is not recommended for use in new code.

</div>

# `!email.headerregistry`: Custom Header Objects

<div class="module" data-synopsis="Automatic Parsing of headers based on the field name">

email.headerregistry

</div>

**Source code:** `Lib/email/headerregistry.py`

------------------------------------------------------------------------

<div class="versionadded">

3.6[^1]

</div>

Headers are represented by customized subclasses of `str`. The particular class used to represent a given header is determined by the `~email.policy.EmailPolicy.header_factory` of the `~email.policy` in effect when the headers are created. This section documents the particular `header_factory` implemented by the email package for handling [RFC 5322](http://www.faqs.org/rfcs/rfc5322.html "RFC 5322") compliant email messages, which not only provides customized header objects for various header types, but also provides an extension mechanism for applications to add their own custom header types.

When using any of the policy objects derived from `~email.policy.EmailPolicy`, all headers are produced by `.HeaderRegistry` and have `.BaseHeader` as their last base class. Each header class has an additional base class that is determined by the type of the header. For example, many headers have the class `.UnstructuredHeader` as their other base class. The specialized second class for a header is determined by the name of the header, using a lookup table stored in the `.HeaderRegistry`. All of this is managed transparently for the typical application program, but interfaces are provided for modifying the default behavior for use by more complex applications.

The sections below first document the header base classes and their attributes, followed by the API for modifying the behavior of `.HeaderRegistry`, and finally the support classes used to represent the data parsed from structured headers.

<div class="BaseHeader(name, value)">

*name* and *value* are passed to `BaseHeader` from the `~email.policy.EmailPolicy.header_factory` call. The string value of any header object is the *value* fully decoded to a string.

This base class defines the following read-only properties:

<div class="attribute">

name

The name of the header (the portion of the field before the ':'). This is exactly the value passed in the `~email.policy.EmailPolicy.header_factory` call for *name*; that is, case is preserved.

</div>

<div class="attribute">

defects

A tuple of `~email.errors.HeaderDefect` instances reporting any RFC compliance problems found during parsing. The email package tries to be complete about detecting compliance issues. See the `~email.errors` module for a discussion of the types of defects that may be reported.

</div>

<div class="attribute">

max_count

The maximum number of headers of this type that can have the same `name`. A value of `None` means unlimited. The `BaseHeader` value for this attribute is `None`; it is expected that specialized header classes will override this value as needed.

</div>

`BaseHeader` also provides the following method, which is called by the email library code and should not in general be called by application programs:

<div class="method">

fold(\*, policy)

Return a string containing `~email.policy.Policy.linesep` characters as required to correctly fold the header according to *policy*. A `~email.policy.Policy.cte_type` of `8bit` will be treated as if it were `7bit`, since headers may not contain arbitrary binary data. If `~email.policy.EmailPolicy.utf8` is `False`, non-ASCII data will be `2047` encoded.

</div>

`BaseHeader` by itself cannot be used to create a header object. It defines a protocol that each specialized header cooperates with in order to produce the header object. Specifically, `BaseHeader` requires that the specialized class provide a `classmethod` named `parse`. This method is called as follows:

    parse(string, kwds)

`kwds` is a dictionary containing one pre-initialized key, `defects`. `defects` is an empty list. The parse method should append any detected defects to this list. On return, the `kwds` dictionary *must* contain values for at least the keys `decoded`, `defects` and `parse_tree`. `decoded` should be the string value for the header (that is, the header value fully decoded to a string). `parse_tree` is set to the parse tree obtained from parsing the header. The parse method should assume that *string* may contain content-transfer-encoded parts, but should correctly handle all valid Unicode characters as well so that it can parse un-encoded header values.

`BaseHeader`'s `__new__` then creates the header instance, and calls its `init` method. The specialized class only needs to provide an `init` method if it wishes to set additional attributes beyond those provided by `BaseHeader` itself. Such an `init` method should look like this:

    def init(self, /, *args, **kw):
        self._myattr = kw.pop('myattr')
        super().init(*args, **kw)

That is, anything extra that the specialized class puts in to the `kwds` dictionary should be removed and handled, and the remaining contents of `kw` (and `args`) passed to the `BaseHeader` `init` method.

</div>

<div class="UnstructuredHeader">

An "unstructured" header is the default type of header in `5322`. Any header that does not have a specified syntax is treated as unstructured. The classic example of an unstructured header is the `Subject` header.

In `5322`, an unstructured header is a run of arbitrary text in the ASCII character set. `2047`, however, has an `5322` compatible mechanism for encoding non-ASCII text as ASCII characters within a header value. When a *value* containing encoded words is passed to the constructor, the `UnstructuredHeader` parser converts such encoded words into a string, following the `2047` rules for unstructured text. The parser uses heuristics to attempt to decode certain non-compliant encoded words. Defects are registered in such cases, as well as defects for issues such as invalid characters within the encoded words or the non-encoded text.

This header type provides no additional attributes.

</div>

<div class="DateHeader">

`5322` specifies a very specific format for dates within email headers. The `DateHeader` parser recognizes that date format, as well as recognizing a number of variant forms that are sometimes found "in the wild".

This header type provides the following additional attributes:

<div class="attribute">

datetime

If the header value can be recognized as a valid date of one form or another, this attribute will contain a `~datetime.datetime` instance representing that date. If the timezone of the input date is specified as `-0000` (indicating it is in UTC but contains no information about the source timezone), then `.datetime` will be a naive `~datetime.datetime`. If a specific timezone offset is found (including `+0000`), then `.datetime` will contain an aware `datetime` that uses `datetime.timezone` to record the timezone offset.

</div>

The `decoded` value of the header is determined by formatting the `datetime` according to the `5322` rules; that is, it is set to:

    email.utils.format_datetime(self.datetime)

When creating a `DateHeader`, *value* may be `~datetime.datetime` instance. This means, for example, that the following code is valid and does what one would expect:

    msg['Date'] = datetime(2011, 7, 15, 21)

Because this is a naive `datetime` it will be interpreted as a UTC timestamp, and the resulting value will have a timezone of `-0000`. Much more useful is to use the `~email.utils.localtime` function from the `~email.utils` module:

    msg['Date'] = utils.localtime()

This example sets the date header to the current time and date using the current timezone offset.

</div>

<div class="AddressHeader">

Address headers are one of the most complex structured header types. The `AddressHeader` class provides a generic interface to any address header.

This header type provides the following additional attributes:

<div class="attribute">

groups

A tuple of `.Group` objects encoding the addresses and groups found in the header value. Addresses that are not part of a group are represented in this list as single-address `Groups` whose `~.Group.display_name` is `None`.

</div>

<div class="attribute">

addresses

A tuple of `.Address` objects encoding all of the individual addresses from the header value. If the header value contains any groups, the individual addresses from the group are included in the list at the point where the group occurs in the value (that is, the list of addresses is "flattened" into a one dimensional list).

</div>

The `decoded` value of the header will have all encoded words decoded to a string. `~encodings.idna` encoded domain names are also decoded to a string. The `decoded` value is set by `joining <meth-str-join>` the `str` value of the elements of the `groups` attribute with `', '`.

A list of `.Address` and `.Group` objects in any combination may be used to set the value of an address header. `Group` objects whose `display_name` is `None` will be interpreted as single addresses, which allows an address list to be copied with groups intact by using the list obtained from the `groups` attribute of the source header.

</div>

<div class="SingleAddressHeader">

A subclass of `.AddressHeader` that adds one additional attribute:

<div class="attribute">

address

The single address encoded by the header value. If the header value actually contains more than one address (which would be a violation of the RFC under the default `~email.policy`), accessing this attribute will result in a `ValueError`.

</div>

</div>

Many of the above classes also have a `Unique` variant (for example, `UniqueUnstructuredHeader`). The only difference is that in the `Unique` variant, `~.BaseHeader.max_count` is set to 1.

<div class="MIMEVersionHeader">

There is really only one valid value for the `MIME-Version` header, and that is `1.0`. For future proofing, this header class supports other valid version numbers. If a version number has a valid value per `2045`, then the header object will have non-`None` values for the following attributes:

<div class="attribute">

version

The version number as a string, with any whitespace and/or comments removed.

</div>

<div class="attribute">

major

The major version number as an integer

</div>

<div class="attribute">

minor

The minor version number as an integer

</div>

</div>

<div class="ParameterizedMIMEHeader">

MIME headers all start with the prefix 'Content-'. Each specific header has a certain value, described under the class for that header. Some can also take a list of supplemental parameters, which have a common format. This class serves as a base for all the MIME headers that take parameters.

<div class="attribute">

params

A dictionary mapping parameter names to parameter values.

<div class="versionchanged">

3.15 It is now a `frozendict` instead of a `types.MappingProxyType`.

</div>

</div>

</div>

<div class="ContentTypeHeader">

A `ParameterizedMIMEHeader` class that handles the `Content-Type` header.

<div class="attribute">

content_type

The content type string, in the form `maintype/subtype`.

</div>

<div class="attribute">

maintype

</div>

<div class="attribute">

subtype

</div>

</div>

<div class="ContentDispositionHeader">

A `ParameterizedMIMEHeader` class that handles the `Content-Disposition` header.

<div class="attribute">

content_disposition

`inline` and `attachment` are the only valid values in common use.

</div>

</div>

<div class="ContentTransferEncodingHeader">

Handles the `Content-Transfer-Encoding` header.

<div class="attribute">

cte

Valid values are `7bit`, `8bit`, `base64`, and `quoted-printable`. See `2045` for more information.

</div>

</div>

<div class="HeaderRegistry(base_class=BaseHeader, \ default_class=UnstructuredHeader, \ use_default_map=True)">

This is the factory used by `~email.policy.EmailPolicy` by default. `HeaderRegistry` builds the class used to create a header instance dynamically, using *base_class* and a specialized class retrieved from a registry that it holds. When a given header name does not appear in the registry, the class specified by *default_class* is used as the specialized class. When *use_default_map* is `True` (the default), the standard mapping of header names to classes is copied in to the registry during initialization. *base_class* is always the last class in the generated class's `~type.__bases__` list.

The default mappings are:

> subject  
> UniqueUnstructuredHeader
>
> date  
> UniqueDateHeader
>
> resent-date  
> DateHeader
>
> orig-date  
> UniqueDateHeader
>
> sender  
> UniqueSingleAddressHeader
>
> resent-sender  
> SingleAddressHeader
>
> to  
> UniqueAddressHeader
>
> resent-to  
> AddressHeader
>
> cc  
> UniqueAddressHeader
>
> resent-cc  
> AddressHeader
>
> bcc  
> UniqueAddressHeader
>
> resent-bcc  
> AddressHeader
>
> from  
> UniqueAddressHeader
>
> resent-from  
> AddressHeader
>
> reply-to  
> UniqueAddressHeader
>
> mime-version  
> MIMEVersionHeader
>
> content-type  
> ContentTypeHeader
>
> content-disposition  
> ContentDispositionHeader
>
> content-transfer-encoding  
> ContentTransferEncodingHeader
>
> message-id  
> MessageIDHeader

`HeaderRegistry` has the following methods:

<div class="method">

map_to_type(self, name, cls)

*name* is the name of the header to be mapped. It will be converted to lower case in the registry. *cls* is the specialized class to be used, along with *base_class*, to create the class used to instantiate headers that match *name*.

</div>

<div class="method">

\_\_getitem\_\_(name)

Construct and return a class to handle creating a *name* header.

</div>

<div class="method">

\_\_call\_\_(name, value)

Retrieves the specialized header associated with *name* from the registry (using *default_class* if *name* does not appear in the registry) and composes it with *base_class* to produce a class, calls the constructed class's constructor, passing it the same argument list, and finally returns the class instance created thereby.

</div>

</div>

The following classes are the classes used to represent data parsed from structured headers and can, in general, be used by an application program to construct structured values to assign to specific headers.

<div class="Address(display_name='', username='', domain='', addr_spec=None)">

The class used to represent an email address. The general form of an address is:

    [display_name] <username@domain>

or:

    username@domain

where each part must conform to specific syntax rules spelled out in `5322`.

As a convenience *addr_spec* can be specified instead of *username* and *domain*, in which case *username* and *domain* will be parsed from the *addr_spec*. An *addr_spec* must be a properly RFC quoted string; if it is not `Address` will raise an error. Unicode characters are allowed and will be property encoded when serialized. However, per the RFCs, Unicode is *not* allowed in the username portion of the address.

<div class="attribute">

display_name

The display name portion of the address, if any, with all quoting removed. If the address does not have a display name, this attribute will be an empty string.

</div>

<div class="attribute">

username

The `username` portion of the address, with all quoting removed.

</div>

<div class="attribute">

domain

The `domain` portion of the address.

</div>

<div class="attribute">

addr_spec

The `username@domain` portion of the address, correctly quoted for use as a bare address (the second form shown above). This attribute is not mutable.

</div>

<div class="method">

\_\_str\_\_()

The `str` value of the object is the address quoted according to `5322` rules, but with no Content Transfer Encoding of any non-ASCII characters.

</div>

To support SMTP (`5321`), `Address` handles one special case: if `username` and `domain` are both the empty string (or `None`), then the string value of the `Address` is `<>`.

</div>

<div class="Group(display_name=None, addresses=None)">

The class used to represent an address group. The general form of an address group is:

    display_name: [address-list];

As a convenience for processing lists of addresses that consist of a mixture of groups and single addresses, a `Group` may also be used to represent single addresses that are not part of a group by setting *display_name* to `None` and providing a list of the single address as *addresses*.

<div class="attribute">

display_name

The `display_name` of the group. If it is `None` and there is exactly one `Address` in `addresses`, then the `Group` represents a single address that is not in a group.

</div>

<div class="attribute">

addresses

A possibly empty tuple of `.Address` objects representing the addresses in the group.

</div>

<div class="method">

\_\_str\_\_()

The `str` value of a `Group` is formatted according to `5322`, but with no Content Transfer Encoding of any non-ASCII characters. If `display_name` is none and there is a single `Address` in the `addresses` list, the `str` value will be the same as the `str` of that single `Address`.

</div>

</div>

**Footnotes**

[^1]: Originally added in 3.3 as a `provisional module <provisional
    package>`

# `!email.iterators`: Iterators

<div class="module" data-synopsis="Iterate over a  message object tree.">

email.iterators

</div>

**Source code:** `Lib/email/iterators.py`

------------------------------------------------------------------------

Iterating over a message object tree is fairly easy with the `Message.walk <email.message.Message.walk>` method. The `!email.iterators` module provides some useful higher level iterations over message object trees.

<div class="function">

body_line_iterator(msg, decode=False)

This iterates over all the payloads in all the subparts of *msg*, returning the string payloads line-by-line. It skips over all the subpart headers, and it skips over any subpart with a payload that isn't a Python string. This is somewhat equivalent to reading the flat text representation of the message from a file using `~io.TextIOBase.readline`, skipping over all the intervening headers.

Optional *decode* is passed through to `Message.get_payload
<email.message.Message.get_payload>`.

</div>

<div class="function">

typed_subpart_iterator(msg, maintype='text', subtype=None)

This iterates over all the subparts of *msg*, returning only those subparts that match the MIME type specified by *maintype* and *subtype*.

Note that *subtype* is optional; if omitted, then subpart MIME type matching is done only with the main type. *maintype* is optional too; it defaults to `text`.

Thus, by default `typed_subpart_iterator` returns each subpart that has a MIME type of `text/\*`.

</div>

The following function has been added as a useful debugging tool. It should *not* be considered part of the supported public interface for the package.

<div class="function">

<span id="structure">structure</span>(msg, fp=None, level=0, include_default=False)

Prints an indented representation of the content types of the message object structure. For example:

<div class="testsetup">

import email from email.iterators import <span id="structure">structure</span> somefile = open('../Lib/test/test_email/data/msg_02.txt')

</div>

<div class="doctest">

\>\>\> msg = email.message_from_file(somefile) \>\>\> <span id="structure">structure</span>(msg) multipart/mixed text/plain text/plain multipart/digest message/rfc822 text/plain message/rfc822 text/plain message/rfc822 text/plain message/rfc822 text/plain message/rfc822 text/plain text/plain

</div>

<div class="testcleanup">

somefile.close()

</div>

Optional *fp* is a file-like object to print the output to. It must be suitable for Python's `print` function. *level* is used internally. *include_default*, if true, prints the default type as well.

</div>

# `!email` --- An email and MIME handling package

<div class="module" data-synopsis="Package supporting the parsing, manipulating, and generating
email messages.">

email

</div>

**Source code:** `Lib/email/__init__.py`

------------------------------------------------------------------------

The `!email` package is a library for managing email messages. It is specifically *not* designed to do any sending of email messages to SMTP (`2821`), NNTP, or other servers; those are functions of modules such as `smtplib`. The `!email` package attempts to be as RFC-compliant as possible, supporting `5322` and `6532`, as well as such MIME-related RFCs as `2045`, `2046`, `2047`, `2183`, and `2231`.

The overall structure of the email package can be divided into three major components, plus a fourth component that controls the behavior of the other components.

The central component of the package is an "object model" that represents email messages. An application interacts with the package primarily through the object model interface defined in the `~email.message` sub-module. The application can use this API to ask questions about an existing email, to construct a new email, or to add or remove email subcomponents that themselves use the same object model interface. That is, following the nature of email messages and their MIME subcomponents, the email object model is a tree structure of objects that all provide the `~email.message.EmailMessage` API.

The other two major components of the package are the `~email.parser` and the `~email.generator`. The parser takes the serialized version of an email message (a stream of bytes) and converts it into a tree of `~email.message.EmailMessage` objects. The generator takes an `~email.message.EmailMessage` and turns it back into a serialized byte stream. (The parser and generator also handle streams of text characters, but this usage is discouraged as it is too easy to end up with messages that are not valid in one way or another.)

The control component is the `~email.policy` module. Every `~email.message.EmailMessage`, every `~email.generator`, and every `~email.parser` has an associated `~email.policy` object that controls its behavior. Usually an application only needs to specify the policy when an `~email.message.EmailMessage` is created, either by directly instantiating an `~email.message.EmailMessage` to create a new email, or by parsing an input stream using a `~email.parser`. But the policy can be changed when the message is serialized using a `~email.generator`. This allows, for example, a generic email message to be parsed from disk, but to serialize it using standard SMTP settings when sending it to an email server.

The email package does its best to hide the details of the various governing RFCs from the application. Conceptually the application should be able to treat the email message as a structured tree of Unicode text and binary attachments, without having to worry about how these are represented when serialized. In practice, however, it is often necessary to be aware of at least some of the rules governing MIME messages and their structure, specifically the names and nature of the MIME "content types" and how they identify multipart documents. For the most part this knowledge should only be required for more complex applications, and even then it should only be the high level structure in question, and not the details of how those structures are represented. Since MIME content types are used widely in modern internet software (not just email), this will be a familiar concept to many programmers.

The following sections describe the functionality of the `!email` package. We start with the `~email.message` object model, which is the primary interface an application will use, and follow that with the `~email.parser` and `~email.generator` components. Then we cover the `~email.policy` controls, which completes the treatment of the main components of the library.

The next three sections cover the exceptions the package may raise and the defects (non-compliance with the RFCs) that the `~email.parser` may detect. Then we cover the `~email.headerregistry` and the `~email.contentmanager` sub-components, which provide tools for doing more detailed manipulation of headers and payloads, respectively. Both of these components contain features relevant to consuming and producing non-trivial messages, but also document their extensibility APIs, which will be of interest to advanced applications.

Following those is a set of examples of using the fundamental parts of the APIs covered in the preceding sections.

The foregoing represent the modern (Unicode friendly) API of the email package. The remaining sections, starting with the `~email.message.Message` class, cover the legacy `~email.policy.compat32` API that deals much more directly with the details of how email messages are represented. The `~email.policy.compat32` API does *not* hide the details of the RFCs from the application, but for applications that need to operate at that level, they can be useful tools. This documentation is also relevant for applications that are still using the `~email.policy.compat32` API for backward compatibility reasons.

<div class="versionchanged">

3.6 Docs reorganized and rewritten to promote the new `~email.message.EmailMessage`/`~email.policy.EmailPolicy` API.

</div>

Contents of the `!email` package documentation:

<div class="toctree">

email.message.rst email.parser.rst email.generator.rst email.policy.rst

email.errors.rst email.headerregistry.rst email.contentmanager.rst

email.examples.rst

</div>

Legacy API:

<div class="toctree">

email.compat32-message.rst email.mime.rst email.header.rst email.charset.rst email.encoders.rst email.utils.rst email.iterators.rst

</div>

<div class="seealso">

Module `smtplib`  
SMTP (Simple Mail Transport Protocol) client

Module `poplib`  
POP (Post Office Protocol) client

Module `imaplib`  
IMAP (Internet Message Access Protocol) client

Module `mailbox`  
Tools for creating, reading, and managing collections of messages on disk using a variety standard formats.

</div>

# `!email.message`: Representing an email message

<div class="module" data-synopsis="The base class representing email messages.">

email.message

</div>

**Source code:** `Lib/email/message.py`

------------------------------------------------------------------------

<div class="versionadded">

3.6[^1]

</div>

The central class in the `email` package is the `EmailMessage` class, imported from the `!email.message` module. It is the base class for the `email` object model. `EmailMessage` provides the core functionality for setting and querying header fields, for accessing message bodies, and for creating or modifying structured messages.

An email message consists of *headers* and a *payload* (which is also referred to as the *content*). Headers are `5322` or `6532` style field names and values, where the field name and value are separated by a colon. The colon is not part of either the field name or the field value. The payload may be a simple text message, or a binary object, or a structured sequence of sub-messages each with their own set of headers and their own payload. The latter type of payload is indicated by the message having a MIME type such as `multipart/\*` or `message/rfc822`.

The conceptual model provided by an `EmailMessage` object is that of an ordered dictionary of headers coupled with a *payload* that represents the `5322` body of the message, which might be a list of sub-`EmailMessage` objects. In addition to the normal dictionary methods for accessing the header names and values, there are methods for accessing specialized information from the headers (for example the MIME content type), for operating on the payload, for generating a serialized version of the message, and for recursively walking over the object tree.

The `EmailMessage` dictionary-like interface is indexed by the header names, which must be ASCII values. The values of the dictionary are strings with some extra methods. Headers are stored and returned in case-preserving form, but field names are matched case-insensitively. The keys are ordered, but unlike a real dict, there can be duplicates. Additional methods are provided for working with headers that have duplicate keys.

The *payload* is either a string or bytes object, in the case of simple message objects, or a list of `EmailMessage` objects, for MIME container documents such as `multipart/\*` and `message/rfc822` message objects.

<div class="EmailMessage(policy=default)">

If *policy* is specified use the rules it specifies to update and serialize the representation of the message. If *policy* is not set, use the `~email.policy.default` policy, which follows the rules of the email RFCs except for line endings (instead of the RFC mandated `\r\n`, it uses the Python standard `\n` line endings). For more information see the `~email.policy` documentation.[^2]

<div class="method">

as_string(unixfrom=False, maxheaderlen=None, policy=None)

Return the entire message flattened as a string. When optional *unixfrom* is true, the envelope header is included in the returned string. *unixfrom* defaults to `False`. For backward compatibility with the base `~email.message.Message` class *maxheaderlen* is accepted, but defaults to `None`, which means that by default the line length is controlled by the `~email.policy.Policy.max_line_length` of the policy. The *policy* argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified *policy* will be passed to the `~email.generator.Generator`.

Flattening the message may trigger changes to the `EmailMessage` if defaults need to be filled in to complete the transformation to a string (for example, MIME boundaries may be generated or modified).

Note that this method is provided as a convenience and may not be the most useful way to serialize messages in your application, especially if you are dealing with multiple messages. See `email.generator.Generator` for a more flexible API for serializing messages. Note also that this method is restricted to producing messages serialized as "7 bit clean" when `~email.policy.EmailPolicy.utf8` is `False`, which is the default.

<div class="versionchanged">

3.6 the default behavior when *maxheaderlen* is not specified was changed from defaulting to 0 to defaulting to the value of *max_line_length* from the policy.

</div>

</div>

<div class="method">

\_\_str\_\_()

Equivalent to `as_string(policy=self.policy.clone(utf8=True))`. Allows `str(msg)` to produce a string containing the serialized message in a readable format.

<div class="versionchanged">

3.4 the method was changed to use `utf8=True`, thus producing an `6531`-like message representation, instead of being a direct alias for `as_string`.

</div>

</div>

<div class="method">

as_bytes(unixfrom=False, policy=None)

Return the entire message flattened as a bytes object. When optional *unixfrom* is true, the envelope header is included in the returned string. *unixfrom* defaults to `False`. The *policy* argument may be used to override the default policy obtained from the message instance. This can be used to control some of the formatting produced by the method, since the specified *policy* will be passed to the `~email.generator.BytesGenerator`.

Flattening the message may trigger changes to the `EmailMessage` if defaults need to be filled in to complete the transformation to a string (for example, MIME boundaries may be generated or modified).

Note that this method is provided as a convenience and may not be the most useful way to serialize messages in your application, especially if you are dealing with multiple messages. See `email.generator.BytesGenerator` for a more flexible API for serializing messages.

</div>

<div class="method">

\_\_bytes\_\_()

Equivalent to `.as_bytes`. Allows `bytes(msg)` to produce a bytes object containing the serialized message.

</div>

<div class="method">

is_multipart()

Return `True` if the message's payload is a list of sub-`EmailMessage` objects, otherwise return `False`. When `is_multipart` returns `False`, the payload should be a string object (which might be a CTE encoded binary payload). Note that `is_multipart` returning `True` does not necessarily mean that "msg.get_content_maintype() == 'multipart'" will return the `True`. For example, `is_multipart` will return `True` when the `EmailMessage` is of type `message/rfc822`.

</div>

<div class="method">

set_unixfrom(unixfrom)

Set the message's envelope header to *unixfrom*, which should be a string. (See `~mailbox.mboxMessage` for a brief description of this header.)

</div>

<div class="method">

get_unixfrom()

Return the message's envelope header. Defaults to `None` if the envelope header was never set.

</div>

The following methods implement the mapping-like interface for accessing the message's headers. Note that there are some semantic differences between these methods and a normal mapping (i.e. dictionary) interface. For example, in a dictionary there are no duplicate keys, but here there may be duplicate message headers. Also, in dictionaries there is no guaranteed order to the keys returned by `keys`, but in an `EmailMessage` object, headers are always returned in the order they appeared in the original message, or in which they were added to the message later. Any header deleted and then re-added is always appended to the end of the header list.

These semantic differences are intentional and are biased toward convenience in the most common use cases.

Note that in all cases, any envelope header present in the message is not included in the mapping interface.

<div class="method">

\_\_len\_\_()

Return the total number of headers, including duplicates.

</div>

<div class="method">

\_\_contains\_\_(name)

Return `True` if the message object has a field named *name*. Matching is done without regard to case and *name* does not include the trailing colon. Used for the `in` operator. For example:

    if 'message-id' in myMessage:
       print('Message-ID:', myMessage['message-id'])

</div>

<div class="method">

\_\_getitem\_\_(name)

Return the value of the named header field. *name* does not include the colon field separator. If the header is missing, `None` is returned; a `KeyError` is never raised.

Note that if the named field appears more than once in the message's headers, exactly which of those field values will be returned is undefined. Use the `get_all` method to get the values of all the extant headers named *name*.

Using the standard (non-`compat32`) policies, the returned value is an instance of a subclass of `email.headerregistry.BaseHeader`.

</div>

<div class="method">

\_\_setitem\_\_(name, val)

Add a header to the message with field name *name* and value *val*. The field is appended to the end of the message's existing headers.

Note that this does *not* overwrite or delete any existing header with the same name. If you want to ensure that the new header is the only one present in the message with field name *name*, delete the field first, e.g.:

    del msg['subject']
    msg['subject'] = 'Python roolz!'

If the `policy <email.policy>` defines certain headers to be unique (as the standard policies do), this method may raise a `ValueError` when an attempt is made to assign a value to such a header when one already exists. This behavior is intentional for consistency's sake, but do not depend on it as we may choose to make such assignments do an automatic deletion of the existing header in the future.

</div>

<div class="method">

\_\_delitem\_\_(name)

Delete all occurrences of the field with name *name* from the message's headers. No exception is raised if the named field isn't present in the headers.

</div>

<div class="method">

keys()

Return a list of all the message's header field names.

</div>

<div class="method">

values()

Return a list of all the message's field values.

</div>

<div class="method">

items()

Return a list of 2-tuples containing all the message's field headers and values.

</div>

<div class="method">

get(name, failobj=None)

Return the value of the named header field. This is identical to `~object.__getitem__` except that optional *failobj* is returned if the named header is missing (*failobj* defaults to `None`).

</div>

Here are some additional useful header related methods:

<div class="method">

get_all(name, failobj=None)

Return a list of all the values for the field named *name*. If there are no such named headers in the message, *failobj* is returned (defaults to `None`).

</div>

<div class="method">

add_header(<span id="name">name</span>, <span id="value">value</span>, \*\*\_params)

Extended header setting. This method is similar to `__setitem__` except that additional header parameters can be provided as keyword arguments. *\_name* is the header field to add and *\_value* is the *primary* value for the header.

For each item in the keyword argument dictionary *\_params*, the key is taken as the parameter name, with underscores converted to dashes (since dashes are illegal in Python identifiers). Normally, the parameter will be added as `key="value"` unless the value is `None`, in which case only the key will be added.

If the value contains non-ASCII characters, the charset and language may be explicitly controlled by specifying the value as a three tuple in the format `(CHARSET, LANGUAGE, VALUE)`, where `CHARSET` is a string naming the charset to be used to encode the value, `LANGUAGE` can usually be set to `None` or the empty string (see `2231` for other possibilities), and `VALUE` is the string value containing non-ASCII code points. If a three tuple is not passed and the value contains non-ASCII characters, it is automatically encoded in `2231` format using a `CHARSET` of `utf-8` and a `LANGUAGE` of `None`.

Here is an example:

    msg.add_header('Content-Disposition', 'attachment', filename='bud.gif')

This will add a header that looks like :

    Content-Disposition: attachment; filename="bud.gif"

An example of the extended interface with non-ASCII characters:

    msg.add_header('Content-Disposition', 'attachment',
                   filename=('iso-8859-1', '', 'Fußballer.ppt'))

</div>

<div class="method">

replace_header(<span id="name">name</span>, <span id="value">value</span>)

Replace a header. Replace the first header found in the message that matches *\_name*, retaining header order and field name case of the original header. If no matching header is found, raise a `KeyError`.

</div>

<div class="method">

get_content_type()

Return the message's content type, coerced to lower case of the form `maintype/subtype`. If there is no `Content-Type` header in the message return the value returned by `get_default_type`. If the `Content-Type` header is invalid, return `text/plain`.

(According to `2045`, messages always have a default type, `get_content_type` will always return a value. `2045` defines a message's default type to be `text/plain` unless it appears inside a `multipart/digest` container, in which case it would be `message/rfc822`. If the `Content-Type` header has an invalid type specification, `2045` mandates that the default type be `text/plain`.)

</div>

<div class="method">

get_content_maintype()

Return the message's main content type. This is the `maintype` part of the string returned by `get_content_type`.

</div>

<div class="method">

get_content_subtype()

Return the message's sub-content type. This is the `subtype` part of the string returned by `get_content_type`.

</div>

<div class="method">

get_default_type()

Return the default content type. Most messages have a default content type of `text/plain`, except for messages that are subparts of `multipart/digest` containers. Such subparts have a default content type of `message/rfc822`.

</div>

<div class="method">

set_default_type(ctype)

Set the default content type. *ctype* should either be `text/plain` or `message/rfc822`, although this is not enforced. The default content type is not stored in the `Content-Type` header, so it only affects the return value of the `get_content_type` methods when no `Content-Type` header is present in the message.

</div>

<div class="method">

set_param(param, value, header='Content-Type', requote=True, charset=None, language='', replace=False)

Set a parameter in the `Content-Type` header. If the parameter already exists in the header, replace its value with *value*. When *header* is `Content-Type` (the default) and the header does not yet exist in the message, add it, set its value to `text/plain`, and append the new parameter value. Optional *header* specifies an alternative header to `Content-Type`.

If the value contains non-ASCII characters, the charset and language may be explicitly specified using the optional *charset* and *language* parameters. Optional *language* specifies the `2231` language, defaulting to the empty string. Both *charset* and *language* should be strings. The default is to use the `utf8` *charset* and `None` for the *language*.

If *replace* is `False` (the default) the header is moved to the end of the list of headers. If *replace* is `True`, the header will be updated in place.

Use of the *requote* parameter with `EmailMessage` objects is deprecated.

Note that existing parameter values of headers may be accessed through the `~email.headerregistry.ParameterizedMIMEHeader.params` attribute of the header value (for example, `msg['Content-Type'].params['charset']`).

<div class="versionchanged">

3.4 `replace` keyword was added.

</div>

</div>

<div class="method">

del_param(param, header='content-type', requote=True)

Remove the given parameter completely from the `Content-Type` header. The header will be re-written in place without the parameter or its value. Optional *header* specifies an alternative to `Content-Type`.

Use of the *requote* parameter with `EmailMessage` objects is deprecated.

</div>

<div class="method">

get_filename(failobj=None)

Return the value of the `filename` parameter of the `Content-Disposition` header of the message. If the header does not have a `filename` parameter, this method falls back to looking for the `name` parameter on the `Content-Type` header. If neither is found, or the header is missing, then *failobj* is returned. The returned string will always be unquoted as per `email.utils.unquote`.

</div>

<div class="method">

get_boundary(failobj=None)

Return the value of the `boundary` parameter of the `Content-Type` header of the message, or *failobj* if either the header is missing, or has no `boundary` parameter. The returned string will always be unquoted as per `email.utils.unquote`.

</div>

<div class="method">

set_boundary(boundary)

Set the `boundary` parameter of the `Content-Type` header to *boundary*. `set_boundary` will always quote *boundary* if necessary. A `~email.errors.HeaderParseError` is raised if the message object has no `Content-Type` header.

Note that using this method is subtly different from deleting the old `Content-Type` header and adding a new one with the new boundary via `add_header`, because `set_boundary` preserves the order of the `Content-Type` header in the list of headers.

</div>

<div class="method">

get_content_charset(failobj=None)

Return the `charset` parameter of the `Content-Type` header, coerced to lower case. If there is no `Content-Type` header, or if that header has no `charset` parameter, *failobj* is returned.

</div>

<div class="method">

get_charsets(failobj=None)

Return a list containing the character set names in the message. If the message is a `multipart`, then the list will contain one element for each subpart in the payload, otherwise, it will be a list of length 1.

Each item in the list will be a string which is the value of the `charset` parameter in the `Content-Type` header for the represented subpart. If the subpart has no `Content-Type` header, no `charset` parameter, or is not of the `text` main MIME type, then that item in the returned list will be *failobj*.

</div>

<div class="method">

is_attachment

Return `True` if there is a `Content-Disposition` header and its (case insensitive) value is `attachment`, `False` otherwise.

<div class="versionchanged">

3.4.2 is_attachment is now a method instead of a property, for consistency with `~email.message.Message.is_multipart`.

</div>

</div>

<div class="method">

get_content_disposition()

Return the lowercased value (without parameters) of the message's `Content-Disposition` header if it has one, or `None`. The possible values for this method are *inline*, *attachment* or `None` if the message follows `2183`.

<div class="versionadded">

3.5

</div>

</div>

The following methods relate to interrogating and manipulating the content (payload) of the message.

<div class="method">

walk()

The `walk` method is an all-purpose generator which can be used to iterate over all the parts and subparts of a message object tree, in depth-first traversal order. You will typically use `walk` as the iterator in a `for` loop; each iteration returns the next subpart.

Here's an example that prints the MIME type of every part of a multipart message structure:

<div class="testsetup">

from email import message_from_binary_file with open('../Lib/test/test_email/data/msg_16.txt', 'rb') as f: msg = message_from_binary_file(f)

</div>

<div class="doctest">

\>\>\> for part in msg.walk(): ... print(part.get_content_type()) multipart/report text/plain message/delivery-status text/plain text/plain message/rfc822 text/plain

</div>

`walk` iterates over the subparts of any part where `is_multipart` returns `True`, even though `msg.get_content_maintype() == 'multipart'` may return `False`. We can see this in our example by making use of the `_structure` debug helper function:

<div class="doctest">

\>\>\> from email.iterators import <span id="structure">structure</span> \>\>\> for part in msg.walk(): ... print(part.get_content_maintype() == 'multipart', ... part.is_multipart()) True True False False False True False False False False False True False False \>\>\> <span id="structure">structure</span>(msg) multipart/report text/plain message/delivery-status text/plain text/plain message/rfc822 text/plain

</div>

Here the `message` parts are not `multiparts`, but they do contain subparts. `is_multipart()` returns `True` and `walk` descends into the subparts.

</div>

<div class="method">

get_body(preferencelist=('related', 'html', 'plain'))

Return the MIME part that is the best candidate to be the "body" of the message.

*preferencelist* must be a sequence of strings from the set `related`, `html`, and `plain`, and indicates the order of preference for the content type of the part returned.

Start looking for candidate matches with the object on which the `get_body` method is called.

If `related` is not included in *preferencelist*, consider the root part (or subpart of the root part) of any related encountered as a candidate if the (sub-)part matches a preference.

When encountering a `multipart/related`, check the `start` parameter and if a part with a matching `Content-ID` is found, consider only it when looking for candidate matches. Otherwise consider only the first (default root) part of the `multipart/related`.

If a part has a `Content-Disposition` header, only consider the part a candidate match if the value of the header is `inline`.

If none of the candidates matches any of the preferences in *preferencelist*, return `None`.

Notes: (1) For most applications the only *preferencelist* combinations that really make sense are `('plain',)`, `('html', 'plain')`, and the default `('related', 'html', 'plain')`. (2) Because matching starts with the object on which `get_body` is called, calling `get_body` on a `multipart/related` will return the object itself unless *preferencelist* has a non-default value. (3) Messages (or message parts) that do not specify a `Content-Type` or whose `Content-Type` header is invalid will be treated as if they are of type `text/plain`, which may occasionally cause `get_body` to return unexpected results.

</div>

<div class="method">

iter_attachments()

Return an iterator over all of the immediate sub-parts of the message that are not candidate "body" parts. That is, skip the first occurrence of each of `text/plain`, `text/html`, `multipart/related`, or `multipart/alternative` (unless they are explicitly marked as attachments via `Content-Disposition: attachment`), and return all remaining parts. When applied directly to a `multipart/related`, return an iterator over the all the related parts except the root part (ie: the part pointed to by the `start` parameter, or the first part if there is no `start` parameter or the `start` parameter doesn't match the `Content-ID` of any of the parts). When applied directly to a `multipart/alternative` or a non-`multipart`, return an empty iterator.

</div>

<div class="method">

iter_parts()

Return an iterator over all of the immediate sub-parts of the message, which will be empty for a non-`multipart`. (See also `~email.message.EmailMessage.walk`.)

</div>

<div class="method">

get_content(*args, content_manager=None,*\*kw)

Call the `~email.contentmanager.ContentManager.get_content` method of the *content_manager*, passing self as the message object, and passing along any other arguments or keywords as additional arguments. If *content_manager* is not specified, use the `content_manager` specified by the current `~email.policy`.

</div>

<div class="method">

set_content(*args, content_manager=None,*\*kw)

Call the `~email.contentmanager.ContentManager.set_content` method of the *content_manager*, passing self as the message object, and passing along any other arguments or keywords as additional arguments. If *content_manager* is not specified, use the `content_manager` specified by the current `~email.policy`.

</div>

<div class="method">

make_related(boundary=None)

Convert a non-`multipart` message into a `multipart/related` message, moving any existing `Content-` headers and payload into a (new) first part of the `multipart`. If *boundary* is specified, use it as the boundary string in the multipart, otherwise leave the boundary to be automatically created when it is needed (for example, when the message is serialized).

</div>

<div class="method">

make_alternative(boundary=None)

Convert a non-`multipart` or a `multipart/related` into a `multipart/alternative`, moving any existing `Content-` headers and payload into a (new) first part of the `multipart`. If *boundary* is specified, use it as the boundary string in the multipart, otherwise leave the boundary to be automatically created when it is needed (for example, when the message is serialized).

</div>

<div class="method">

make_mixed(boundary=None)

Convert a non-`multipart`, a `multipart/related`, or a `multipart/alternative` into a `multipart/mixed`, moving any existing `Content-` headers and payload into a (new) first part of the `multipart`. If *boundary* is specified, use it as the boundary string in the multipart, otherwise leave the boundary to be automatically created when it is needed (for example, when the message is serialized).

</div>

<div class="method">

add_related(*args, content_manager=None,*\*kw)

If the message is a `multipart/related`, create a new message object, pass all of the arguments to its `set_content` method, and `~email.message.Message.attach` it to the `multipart`. If the message is a non-`multipart`, call `make_related` and then proceed as above. If the message is any other type of `multipart`, raise a `TypeError`. If *content_manager* is not specified, use the `content_manager` specified by the current `~email.policy`. If the added part has no `Content-Disposition` header, add one with the value `inline`.

</div>

<div class="method">

add_alternative(*args, content_manager=None,*\*kw)

If the message is a `multipart/alternative`, create a new message object, pass all of the arguments to its `set_content` method, and `~email.message.Message.attach` it to the `multipart`. If the message is a non-`multipart` or `multipart/related`, call `make_alternative` and then proceed as above. If the message is any other type of `multipart`, raise a `TypeError`. If *content_manager* is not specified, use the `content_manager` specified by the current `~email.policy`.

</div>

<div class="method">

add_attachment(*args, content_manager=None,*\*kw)

If the message is a `multipart/mixed`, create a new message object, pass all of the arguments to its `set_content` method, and `~email.message.Message.attach` it to the `multipart`. If the message is a non-`multipart`, `multipart/related`, or `multipart/alternative`, call `make_mixed` and then proceed as above. If *content_manager* is not specified, use the `content_manager` specified by the current `~email.policy`. If the added part has no `Content-Disposition` header, add one with the value `attachment`. This method can be used both for explicit attachments (`Content-Disposition: attachment`) and `inline` attachments (`Content-Disposition: inline`), by passing appropriate options to the `content_manager`.

</div>

<div class="method">

clear()

Remove the payload and all of the headers.

</div>

<div class="method">

clear_content()

Remove the payload and all of the `!Content-` headers, leaving all other headers intact and in their original order.

</div>

`EmailMessage` objects have the following instance attributes:

<div class="attribute">

preamble

The format of a MIME document allows for some text between the blank line following the headers, and the first multipart boundary string. Normally, this text is never visible in a MIME-aware mail reader because it falls outside the standard MIME armor. However, when viewing the raw text of the message, or when viewing the message in a non-MIME aware reader, this text can become visible.

The *preamble* attribute contains this leading extra-armor text for MIME documents. When the `~email.parser.Parser` discovers some text after the headers but before the first boundary string, it assigns this text to the message's *preamble* attribute. When the `~email.generator.Generator` is writing out the plain text representation of a MIME message, and it finds the message has a *preamble* attribute, it will write this text in the area between the headers and the first boundary. See `email.parser` and `email.generator` for details.

Note that if the message object has no preamble, the *preamble* attribute will be `None`.

</div>

<div class="attribute">

epilogue

The *epilogue* attribute acts the same way as the *preamble* attribute, except that it contains text that appears between the last boundary and the end of the message. As with the `~EmailMessage.preamble`, if there is no epilog text this attribute will be `None`.

</div>

<div class="attribute">

defects

The *defects* attribute contains a list of all the problems found when parsing this message. See `email.errors` for a detailed description of the possible parsing defects.

</div>

</div>

<div class="MIMEPart(policy=default)">

This class represents a subpart of a MIME message. It is identical to `EmailMessage`, except that no `MIME-Version` headers are added when `~EmailMessage.set_content` is called, since sub-parts do not need their own `MIME-Version` headers.

</div>

**Footnotes**

[^1]: Originally added in 3.4 as a `provisional module <provisional
    package>`. Docs for legacy message class moved to `compat32_message`.

[^2]: The `EmailMessage` class requires a policy that provides a `content_manager` attribute for content management methods like `set_content()` and `get_content()` to work. The legacy `~email.policy.compat32` policy does not support these methods and should not be used with `EmailMessage`.

# `!email.mime`: Creating email and MIME objects from scratch

<div class="module" data-synopsis="Build MIME messages.">

email.mime

</div>

**Source code:** `Lib/email/mime/`

------------------------------------------------------------------------

This module is part of the legacy (`Compat32`) email API. Its functionality is partially replaced by the `~email.contentmanager` in the new API, but in certain applications these classes may still be useful, even in non-legacy code.

Ordinarily, you get a message object structure by passing a file or some text to a parser, which parses the text and returns the root message object. However you can also build a complete message structure from scratch, or even individual `~email.message.Message` objects by hand. In fact, you can also take an existing structure and add new `~email.message.Message` objects, move them around, etc. This makes a very convenient interface for slicing-and-dicing MIME messages.

You can create a new object structure by creating `~email.message.Message` instances, adding attachments and all the appropriate headers manually. For MIME messages though, the `email` package provides some convenient subclasses to make things easier.

Here are the classes:

<div class="module">

email.mime.base

</div>

<div class="MIMEBase(_maintype, _subtype, *, policy=compat32, **_params)">

Module: `email.mime.base`

This is the base class for all the MIME-specific subclasses of `~email.message.Message`. Ordinarily you won't create instances specifically of `MIMEBase`, although you could. `MIMEBase` is provided primarily as a convenient base class for more specific MIME-aware subclasses.

*\_maintype* is the `Content-Type` major type (e.g. `text` or `image`), and *\_subtype* is the `Content-Type` minor type (e.g. `plain` or `gif`). *\_params* is a parameter key/value dictionary and is passed directly to `Message.add_header
<email.message.Message.add_header>`.

If *policy* is specified, (defaults to the `compat32 <email.policy.Compat32>` policy) it will be passed to `~email.message.Message`.

The `MIMEBase` class always adds a `Content-Type` header (based on *\_maintype*, *\_subtype*, and *\_params*), and a `MIME-Version` header (always set to `1.0`).

<div class="versionchanged">

3.6 Added *policy* keyword-only parameter.

</div>

</div>

<div class="module">

email.mime.nonmultipart

</div>

<div class="MIMENonMultipart()">

Module: `email.mime.nonmultipart`

A subclass of `~email.mime.base.MIMEBase`, this is an intermediate base class for MIME messages that are not `multipart`. The primary purpose of this class is to prevent the use of the `~email.message.Message.attach` method, which only makes sense for `multipart` messages. If `~email.message.Message.attach` is called, a `~email.errors.MultipartConversionError` exception is raised.

</div>

<div class="module">

email.mime.multipart

</div>

<div class="MIMEMultipart(_subtype='mixed', boundary=None, _subparts=None, \ *, policy=compat32, **_params)">

Module: `email.mime.multipart`

A subclass of `~email.mime.base.MIMEBase`, this is an intermediate base class for MIME messages that are `multipart`. Optional *\_subtype* defaults to `mixed`, but can be used to specify the subtype of the message. A `Content-Type` header of `multipart/_subtype` will be added to the message object. A `MIME-Version` header will also be added.

Optional *boundary* is the multipart boundary string. When `None` (the default), the boundary is calculated when needed (for example, when the message is serialized).

*\_subparts* is a sequence of initial subparts for the payload. It must be possible to convert this sequence to a list. You can always attach new subparts to the message by using the `Message.attach
<email.message.Message.attach>` method.

Optional *policy* argument defaults to `compat32 <email.policy.Compat32>`.

Additional parameters for the `Content-Type` header are taken from the keyword arguments, or passed into the *\_params* argument, which is a keyword dictionary.

<div class="versionchanged">

3.6 Added *policy* keyword-only parameter.

</div>

</div>

<div class="module">

email.mime.application

</div>

<div class="MIMEApplication(_data, _subtype='octet-stream', \ _encoder=email.encoders.encode_base64, \ *, policy=compat32, **_params)">

Module: `email.mime.application`

A subclass of `~email.mime.nonmultipart.MIMENonMultipart`, the `MIMEApplication` class is used to represent MIME message objects of major type `application`. *\_data* contains the bytes for the raw application data. Optional *\_subtype* specifies the MIME subtype and defaults to `octet-stream`.

Optional *\_encoder* is a callable (i.e. function) which will perform the actual encoding of the data for transport. This callable takes one argument, which is the `MIMEApplication` instance. It should use `~email.message.Message.get_payload` and `~email.message.Message.set_payload` to change the payload to encoded form. It should also add any `Content-Transfer-Encoding` or other headers to the message object as necessary. The default encoding is base64. See the `email.encoders` module for a list of the built-in encoders.

Optional *policy* argument defaults to `compat32 <email.policy.Compat32>`.

*\_params* are passed straight through to the base class constructor.

<div class="versionchanged">

3.6 Added *policy* keyword-only parameter.

</div>

</div>

<div class="module">

email.mime.audio

</div>

<div class="MIMEAudio(_audiodata, _subtype=None, \ _encoder=email.encoders.encode_base64, \ *, policy=compat32, **_params)">

Module: `email.mime.audio`

A subclass of `~email.mime.nonmultipart.MIMENonMultipart`, the `MIMEAudio` class is used to create MIME message objects of major type `audio`. *\_audiodata* contains the bytes for the raw audio data. If this data can be decoded as au, wav, aiff, or aifc, then the subtype will be automatically included in the `Content-Type` header. Otherwise you can explicitly specify the audio subtype via the *\_subtype* argument. If the minor type could not be guessed and *\_subtype* was not given, then `TypeError` is raised.

Optional *\_encoder* is a callable (i.e. function) which will perform the actual encoding of the audio data for transport. This callable takes one argument, which is the `MIMEAudio` instance. It should use `~email.message.Message.get_payload` and `~email.message.Message.set_payload` to change the payload to encoded form. It should also add any `Content-Transfer-Encoding` or other headers to the message object as necessary. The default encoding is base64. See the `email.encoders` module for a list of the built-in encoders.

Optional *policy* argument defaults to `compat32 <email.policy.Compat32>`.

*\_params* are passed straight through to the base class constructor.

<div class="versionchanged">

3.6 Added *policy* keyword-only parameter.

</div>

</div>

<div class="module">

email.mime.image

</div>

<div class="MIMEImage(_imagedata, _subtype=None, \ _encoder=email.encoders.encode_base64, \ *, policy=compat32, **_params)">

Module: `email.mime.image`

A subclass of `~email.mime.nonmultipart.MIMENonMultipart`, the `MIMEImage` class is used to create MIME message objects of major type `image`. *\_imagedata* contains the bytes for the raw image data. If this data type can be detected (jpeg, png, gif, tiff, rgb, pbm, pgm, ppm, rast, xbm, bmp, webp, and exr attempted), then the subtype will be automatically included in the `Content-Type` header. Otherwise you can explicitly specify the image subtype via the *\_subtype* argument. If the minor type could not be guessed and *\_subtype* was not given, then `TypeError` is raised.

Optional *\_encoder* is a callable (i.e. function) which will perform the actual encoding of the image data for transport. This callable takes one argument, which is the `MIMEImage` instance. It should use `~email.message.Message.get_payload` and `~email.message.Message.set_payload` to change the payload to encoded form. It should also add any `Content-Transfer-Encoding` or other headers to the message object as necessary. The default encoding is base64. See the `email.encoders` module for a list of the built-in encoders.

Optional *policy* argument defaults to `compat32 <email.policy.Compat32>`.

*\_params* are passed straight through to the `~email.mime.base.MIMEBase` constructor.

<div class="versionchanged">

3.6 Added *policy* keyword-only parameter.

</div>

</div>

<div class="module">

email.mime.message

</div>

<div class="MIMEMessage(_msg, _subtype='rfc822', *, policy=compat32)">

Module: `email.mime.message`

A subclass of `~email.mime.nonmultipart.MIMENonMultipart`, the `MIMEMessage` class is used to create MIME objects of main type `message`. *\_msg* is used as the payload, and must be an instance of class `~email.message.Message` (or a subclass thereof), otherwise a `TypeError` is raised.

Optional *\_subtype* sets the subtype of the message; it defaults to `rfc822`.

Optional *policy* argument defaults to `compat32 <email.policy.Compat32>`.

<div class="versionchanged">

3.6 Added *policy* keyword-only parameter.

</div>

</div>

<div class="module">

email.mime.text

</div>

<div class="MIMEText(_text, _subtype='plain', _charset=None, *, policy=compat32)">

Module: `email.mime.text`

A subclass of `~email.mime.nonmultipart.MIMENonMultipart`, the `MIMEText` class is used to create MIME objects of major type `text`. *\_text* is the string for the payload. *\_subtype* is the minor type and defaults to `plain`. *\_charset* is the character set of the text and is passed as an argument to the `~email.mime.nonmultipart.MIMENonMultipart` constructor; it defaults to `us-ascii` if the string contains only `ascii` code points, and `utf-8` otherwise. The *\_charset* parameter accepts either a string or a `~email.charset.Charset` instance.

Unless the *\_charset* argument is explicitly set to `None`, the MIMEText object created will have both a `Content-Type` header with a `charset` parameter, and a `Content-Transfer-Encoding` header. This means that a subsequent `set_payload` call will not result in an encoded payload, even if a charset is passed in the `set_payload` command. You can "reset" this behavior by deleting the `Content-Transfer-Encoding` header, after which a `set_payload` call will automatically encode the new payload (and add a new `Content-Transfer-Encoding` header).

Optional *policy* argument defaults to `compat32 <email.policy.Compat32>`.

<div class="versionchanged">

3.5 *\_charset* also accepts `~email.charset.Charset` instances.

</div>

<div class="versionchanged">

3.6 Added *policy* keyword-only parameter.

</div>

</div>

# `!email.parser`: Parsing email messages

<div class="module" data-synopsis="Parse flat text email messages to produce a message object structure.">

email.parser

</div>

**Source code:** `Lib/email/parser.py`

------------------------------------------------------------------------

Message object structures can be created in one of two ways: they can be created from whole cloth by creating an `~email.message.EmailMessage` object, adding headers using the dictionary interface, and adding payload(s) using `~email.message.EmailMessage.set_content` and related methods, or they can be created by parsing a serialized representation of the email message.

The `email` package provides a standard parser that understands most email document structures, including MIME documents. You can pass the parser a bytes, string or file object, and the parser will return to you the root `~email.message.EmailMessage` instance of the object structure. For simple, non-MIME messages the payload of this root object will likely be a string containing the text of the message. For MIME messages, the root object will return `True` from its `~email.message.EmailMessage.is_multipart` method, and the subparts can be accessed via the payload manipulation methods, such as `~email.message.EmailMessage.get_body`, `~email.message.EmailMessage.iter_parts`, and `~email.message.EmailMessage.walk`.

There are actually two parser interfaces available for use, the `Parser` API and the incremental `FeedParser` API. The `Parser` API is most useful if you have the entire text of the message in memory, or if the entire message lives in a file on the file system. `FeedParser` is more appropriate when you are reading the message from a stream which might block waiting for more input (such as reading an email message from a socket). The `FeedParser` can consume and parse the message incrementally, and only returns the root object when you close the parser.

Note that the parser can be extended in limited ways, and of course you can implement your own parser completely from scratch. All of the logic that connects the `email` package's bundled parser and the `~email.message.EmailMessage` class is embodied in the `~email.policy.Policy` class, so a custom parser can create message object trees any way it finds necessary by implementing custom versions of the appropriate `!Policy` methods.

## FeedParser API

The `BytesFeedParser`, imported from the `!email.feedparser` module, provides an API that is conducive to incremental parsing of email messages, such as would be necessary when reading the text of an email message from a source that can block (such as a socket). The `BytesFeedParser` can of course be used to parse an email message fully contained in a `bytes-like
object`, string, or file, but the `BytesParser` API may be more convenient for such use cases. The semantics and results of the two parser APIs are identical.

The `BytesFeedParser`'s API is simple; you create an instance, feed it a bunch of bytes until there's no more to feed it, then close the parser to retrieve the root message object. The `BytesFeedParser` is extremely accurate when parsing standards-compliant messages, and it does a very good job of parsing non-compliant messages, providing information about how a message was deemed broken. It will populate a message object's `~email.message.EmailMessage.defects` attribute with a list of any problems it found in a message. See the `email.errors` module for the list of defects that it can find.

Here is the API for the `BytesFeedParser`:

<div class="BytesFeedParser(_factory=None, *, policy=policy.compat32)">

Create a `BytesFeedParser` instance. Optional *\_factory* is a no-argument callable; if not specified use the `~email.policy.Policy.message_factory` from the *policy*. Call *\_factory* whenever a new message object is needed.

If *policy* is specified use the rules it specifies to update the representation of the message. If *policy* is not set, use the `compat32 <email.policy.Compat32>` policy, which maintains backward compatibility with the Python 3.2 version of the email package and provides `~email.message.Message` as the default factory. All other policies provide `~email.message.EmailMessage` as the default *\_factory*. For more information on what else *policy* controls, see the `~email.policy` documentation.

Note: **The policy keyword should always be specified**. The default will change to `email.policy.default` in a future version of Python.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Added the *policy* keyword.

</div>

<div class="versionchanged">

3.6 *\_factory* defaults to the policy `message_factory`.

</div>

<div class="method">

feed(data)

Feed the parser some more data. *data* should be a `bytes-like
object` containing one or more lines. The lines can be partial and the parser will stitch such partial lines together properly. The lines can have any of the three common line endings: carriage return, newline, or carriage return and newline (they can even be mixed).

</div>

<div class="method">

close()

Complete the parsing of all previously fed data and return the root message object. It is undefined what happens if `~feed` is called after this method has been called.

</div>

</div>

<div class="FeedParser(_factory=None, *, policy=policy.compat32)">

Works like `BytesFeedParser` except that the input to the `~BytesFeedParser.feed` method must be a string. This is of limited utility, since the only way for such a message to be valid is for it to contain only ASCII text or, if `~email.policy.EmailPolicy.utf8` is `True`, no binary attachments.

<div class="versionchanged">

3.3 Added the *policy* keyword.

</div>

</div>

## Parser API

The `BytesParser` class, imported from the `!email.parser` module, provides an API that can be used to parse a message when the complete contents of the message are available in a `bytes-like object` or file. The `!email.parser` module also provides `Parser` for parsing strings, and header-only parsers, `BytesHeaderParser` and `HeaderParser`, which can be used if you're only interested in the headers of the message. `BytesHeaderParser` and `HeaderParser` can be much faster in these situations, since they do not attempt to parse the message body, instead setting the payload to the raw body.

<div class="BytesParser(_class=None, *, policy=policy.compat32)">

Create a `BytesParser` instance. The *\_class* and *policy* arguments have the same meaning and semantics as the *\_factory* and *policy* arguments of `BytesFeedParser`.

Note: **The policy keyword should always be specified**. The default will change to `email.policy.default` in a future version of Python.

<div class="versionchanged">

3.3 Removed the *strict* argument that was deprecated in 2.4. Added the *policy* keyword.

</div>

<div class="versionchanged">

3.6 *\_class* defaults to the policy `message_factory`.

</div>

<div class="method">

parse(fp, headersonly=False)

Read all the data from the binary file-like object *fp*, parse the resulting bytes, and return the message object. *fp* must support both the `~io.IOBase.readline` and the `~io.BufferedIOBase.read` methods.

The bytes contained in *fp* must be formatted as a block of `5322` (or, if `~email.policy.EmailPolicy.utf8` is `True`, `6532`) style headers and header continuation lines, optionally preceded by an envelope header. The header block is terminated either by the end of the data or by a blank line. Following the header block is the body of the message (which may contain MIME-encoded subparts, including subparts with a `Content-Transfer-Encoding` of `8bit`).

Optional *headersonly* is a flag specifying whether to stop parsing after reading the headers or not. The default is `False`, meaning it parses the entire contents of the file.

</div>

<div class="method">

parsebytes(bytes, headersonly=False)

Similar to the `parse` method, except it takes a `bytes-like
object` instead of a file-like object. Calling this method on a `bytes-like object` is equivalent to wrapping *bytes* in a `~io.BytesIO` instance first and calling `parse`.

Optional *headersonly* is as with the `parse` method.

</div>

<div class="versionadded">

3.2

</div>

</div>

<div class="BytesHeaderParser(_class=None, *, policy=policy.compat32)">

Exactly like `BytesParser`, except that *headersonly* defaults to `True`.

<div class="versionadded">

3.3

</div>

</div>

<div class="Parser(_class=None, *, policy=policy.compat32)">

This class is parallel to `BytesParser`, but handles string input.

<div class="versionchanged">

3.3 Removed the *strict* argument. Added the *policy* keyword.

</div>

<div class="versionchanged">

3.6 *\_class* defaults to the policy `message_factory`.

</div>

<div class="method">

parse(fp, headersonly=False)

Read all the data from the text-mode file-like object *fp*, parse the resulting text, and return the root message object. *fp* must support both the `~io.TextIOBase.readline` and the `~io.TextIOBase.read` methods on file-like objects.

Other than the text mode requirement, this method operates like `BytesParser.parse`.

</div>

<div class="method">

parsestr(text, headersonly=False)

Similar to the `parse` method, except it takes a string object instead of a file-like object. Calling this method on a string is equivalent to wrapping *text* in a `~io.StringIO` instance first and calling `parse`.

Optional *headersonly* is as with the `parse` method.

</div>

</div>

<div class="HeaderParser(_class=None, *, policy=policy.compat32)">

Exactly like `Parser`, except that *headersonly* defaults to `True`.

</div>

Since creating a message object structure from a string or a file object is such a common task, four functions are provided as a convenience. They are available in the top-level `email` package namespace.

<div class="currentmodule">

email

</div>

<div class="function">

message_from_bytes(s, <span id="class">class</span>=None, \*, policy=policy.compat32)

Return a message object structure from a `bytes-like object`. This is equivalent to `BytesParser().parsebytes(s)`. Optional *\_class* and *policy* are interpreted as with the `~email.parser.BytesParser` class constructor.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Removed the *strict* argument. Added the *policy* keyword.

</div>

</div>

<div class="function">

message_from_binary_file(fp, <span id="class">class</span>=None, \*, policy=policy.compat32)

Return a message object structure tree from an open binary `file
object`. This is equivalent to `BytesParser().parse(fp)`. *\_class* and *policy* are interpreted as with the `~email.parser.BytesParser` class constructor.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Removed the *strict* argument. Added the *policy* keyword.

</div>

</div>

<div class="function">

message_from_string(s, <span id="class">class</span>=None, \*, policy=policy.compat32)

Return a message object structure from a string. This is equivalent to `Parser().parsestr(s)`. *\_class* and *policy* are interpreted as with the `~email.parser.Parser` class constructor.

<div class="versionchanged">

3.3 Removed the *strict* argument. Added the *policy* keyword.

</div>

</div>

<div class="function">

message_from_file(fp, <span id="class">class</span>=None, \*, policy=policy.compat32)

Return a message object structure tree from an open `file object`. This is equivalent to `Parser().parse(fp)`. *\_class* and *policy* are interpreted as with the `~email.parser.Parser` class constructor.

<div class="versionchanged">

3.3 Removed the *strict* argument. Added the *policy* keyword.

</div>

<div class="versionchanged">

3.6 *\_class* defaults to the policy `message_factory`.

</div>

</div>

Here's an example of how you might use `message_from_bytes` at an interactive Python prompt:

    >>> import email
    >>> msg = email.message_from_bytes(myBytes)  # doctest: +SKIP

## Additional notes

Here are some notes on the parsing semantics:

- Most non-`multipart` type messages are parsed as a single message object with a string payload. These objects will return `False` for `~email.message.EmailMessage.is_multipart`, and `~email.message.EmailMessage.iter_parts` will yield an empty list.
- All `multipart` type messages will be parsed as a container message object with a list of sub-message objects for their payload. The outer container message will return `True` for `~email.message.EmailMessage.is_multipart`, and `~email.message.EmailMessage.iter_parts` will yield a list of subparts.
- Most messages with a content type of `message/\*` (such as `message/delivery-status` and `message/rfc822`) will also be parsed as container object containing a list payload of length 1. Their `~email.message.EmailMessage.is_multipart` method will return `True`. The single element yielded by `~email.message.EmailMessage.iter_parts` will be a sub-message object.
- Some non-standards-compliant messages may not be internally consistent about their `multipart`-edness. Such messages may have a `Content-Type` header of type `multipart`, but their `~email.message.EmailMessage.is_multipart` method may return `False`. If such messages were parsed with the `~email.parser.FeedParser`, they will have an instance of the `~email.errors.MultipartInvariantViolationDefect` class in their *defects* attribute list. See `email.errors` for details.

# `!email.policy`: Policy Objects

<div class="module" data-synopsis="Controlling the parsing and generating of messages">

email.policy

</div>

<div class="versionadded">

3.3

</div>

**Source code:** `Lib/email/policy.py`

------------------------------------------------------------------------

The `email` package's prime focus is the handling of email messages as described by the various email and MIME RFCs. However, the general format of email messages (a block of header fields each consisting of a name followed by a colon followed by a value, the whole block followed by a blank line and an arbitrary 'body'), is a format that has found utility outside of the realm of email. Some of these uses conform fairly closely to the main email RFCs, some do not. Even when working with email, there are times when it is desirable to break strict compliance with the RFCs, such as generating emails that interoperate with email servers that do not themselves follow the standards, or that implement extensions you want to use in ways that violate the standards.

Policy objects give the email package the flexibility to handle all these disparate use cases.

A `Policy` object encapsulates a set of attributes and methods that control the behavior of various components of the email package during use. `Policy` instances can be passed to various classes and methods in the email package to alter the default behavior. The settable values and their defaults are described below.

There is a default policy used by all classes in the email package. For all of the `~email.parser` classes and the related convenience functions, and for the `~email.message.Message` class, this is the `Compat32` policy, via its corresponding pre-defined instance `compat32`. This policy provides for complete backward compatibility (in some cases, including bug compatibility) with the pre-Python3.3 version of the email package.

This default value for the *policy* keyword to `~email.message.EmailMessage` is the `EmailPolicy` policy, via its pre-defined instance `~default`.

When a `~email.message.Message` or `~email.message.EmailMessage` object is created, it acquires a policy. If the message is created by a `~email.parser`, a policy passed to the parser will be the policy used by the message it creates. If the message is created by the program, then the policy can be specified when it is created. When a message is passed to a `~email.generator`, the generator uses the policy from the message by default, but you can also pass a specific policy to the generator that will override the one stored on the message object.

The default value for the *policy* keyword for the `email.parser` classes and the parser convenience functions **will be changing** in a future version of Python. Therefore you should **always specify explicitly which policy you want to use** when calling any of the classes and functions described in the `~email.parser` module.

The first part of this documentation covers the features of `Policy`, an `abstract base class` that defines the features that are common to all policy objects, including `compat32`. This includes certain hook methods that are called internally by the email package, which a custom policy could override to obtain different behavior. The second part describes the concrete classes `EmailPolicy` and `Compat32`, which implement the hooks that provide the standard behavior and the backward compatible behavior and features, respectively.

`Policy` instances are immutable, but they can be cloned, accepting the same keyword arguments as the class constructor and returning a new `Policy` instance that is a copy of the original but with the specified attributes values changed.

As an example, the following code could be used to read an email message from a file on disk and pass it to the system `sendmail` program on a Unix system:

<div class="testsetup">

from unittest import mock mocker = mock.patch('subprocess.Popen') m = mocker.start() proc = mock.MagicMock() m.return_value = proc proc.stdin.close.return_value = None mymsg = open('mymsg.txt', 'w') mymsg.write('To: <abc@xyz.com>nn') mymsg.flush()

</div>

<div class="doctest">

\>\>\> from email import message_from_binary_file \>\>\> from email.generator import BytesGenerator \>\>\> from email import policy \>\>\> from subprocess import Popen, PIPE \>\>\> with open('mymsg.txt', 'rb') as f: ... msg = message_from_binary_file(f, policy=policy.default) ... \>\>\> p = Popen(\['sendmail', msg\['To'\].addresses\[0\]\], stdin=PIPE) \>\>\> g = BytesGenerator(p.stdin, policy=msg.policy.clone(linesep='rn')) \>\>\> g.flatten(msg) \>\>\> p.stdin.close() \>\>\> rc = p.wait()

</div>

<div class="testcleanup">

mymsg.close() mocker.stop() import os os.remove('mymsg.txt')

</div>

Here we are telling `~email.generator.BytesGenerator` to use the RFC correct line separator characters when creating the binary string to feed into `sendmail's` `stdin`, where the default policy would use `\n` line separators.

Some email package methods accept a *policy* keyword argument, allowing the policy to be overridden for that method. For example, the following code uses the `~email.message.Message.as_bytes` method of the *msg* object from the previous example and writes the message to a file using the native line separators for the platform on which it is running:

    >>> import os
    >>> with open('converted.txt', 'wb') as f:
    ...     f.write(msg.as_bytes(policy=msg.policy.clone(linesep=os.linesep)))
    17

Policy objects can also be combined using the addition operator, producing a policy object whose settings are a combination of the non-default values of the summed objects:

    >>> compat_SMTP = policy.compat32.clone(linesep='\r\n')
    >>> compat_strict = policy.compat32.clone(raise_on_defect=True)
    >>> compat_strict_SMTP = compat_SMTP + compat_strict

This operation is not commutative; that is, the order in which the objects are added matters. To illustrate:

    >>> policy100 = policy.compat32.clone(max_line_length=100)
    >>> policy80 = policy.compat32.clone(max_line_length=80)
    >>> apolicy = policy100 + policy80
    >>> apolicy.max_line_length
    80
    >>> apolicy = policy80 + policy100
    >>> apolicy.max_line_length
    100

<div class="Policy(**kw)">

This is the `abstract base class` for all policy classes. It provides default implementations for a couple of trivial methods, as well as the implementation of the immutability property, the `clone` method, and the constructor semantics.

The constructor of a policy class can be passed various keyword arguments. The arguments that may be specified are any non-method properties on this class, plus any additional non-method properties on the concrete class. A value specified in the constructor will override the default value for the corresponding attribute.

This class defines the following properties, and thus values for the following may be passed in the constructor of any policy class:

<div class="attribute">

max_line_length

The maximum length of any line in the serialized output, not counting the end of line character(s). Default is 78, per `5322`. A value of `0` or `None` indicates that no line wrapping should be done at all.

</div>

<div class="attribute">

linesep

The string to be used to terminate lines in serialized output. The default is `\n` because that's the internal end-of-line discipline used by Python, though `\r\n` is required by the RFCs.

</div>

<div class="attribute">

cte_type

Controls the type of Content Transfer Encodings that may be or are required to be used. The possible values are:

<div class="tabularcolumns">

\|l\|L\|

</div>

|  |  |
|----|----|
| `7bit` | all data must be "7 bit clean" (ASCII-only). This means that where necessary data will be encoded using either quoted-printable or base64 encoding. |
| `8bit` | data is not constrained to be 7 bit clean. Data in headers is still required to be ASCII-only and so will be encoded (see `fold_binary` and `~EmailPolicy.utf8` below for exceptions), but body parts may use the `8bit` CTE. |

A `cte_type` value of `8bit` only works with `BytesGenerator`, not `Generator`, because strings cannot contain binary data. If a `Generator` is operating under a policy that specifies `cte_type=8bit`, it will act as if `cte_type` is `7bit`.

</div>

<div class="attribute">

raise_on_defect

If `True`, any defects encountered will be raised as errors. If `False` (the default), defects will be passed to the `register_defect` method.

</div>

<div class="attribute">

[mangle_from]()

If `True`, lines starting with *"From "* in the body are escaped by putting a `>` in front of them. This parameter is used when the message is being serialized by a generator. Default: `False`.

<div class="versionadded">

3.5

</div>

</div>

<div class="attribute">

message_factory

A factory function for constructing a new empty message object. Used by the parser when building messages. Defaults to `None`, in which case `~email.message.Message` is used.

<div class="versionadded">

3.6

</div>

</div>

<div class="attribute">

verify_generated_headers

If `True` (the default), the generator will raise `~email.errors.HeaderWriteError` instead of writing a header that is improperly folded or delimited, such that it would be parsed as multiple headers or joined with adjacent data. Such headers can be generated by custom header classes or bugs in the `email` module.

As it's a security feature, this defaults to `True` even in the `~email.policy.Compat32` policy. For backwards compatible, but unsafe, behavior, it must be set to `False` explicitly.

<div class="versionadded">

3.13

</div>

</div>

The following `Policy` method is intended to be called by code using the email library to create policy instances with custom settings:

<div class="method">

clone(\*\*kw)

Return a new `Policy` instance whose attributes have the same values as the current instance, except where those attributes are given new values by the keyword arguments.

</div>

The remaining `Policy` methods are called by the email package code, and are not intended to be called by an application using the email package. A custom policy must implement all of these methods.

<div class="method">

handle_defect(obj, defect)

Handle a *defect* found on *obj*. When the email package calls this method, *defect* will always be a subclass of `~email.errors.MessageDefect`.

The default implementation checks the `raise_on_defect` flag. If it is `True`, *defect* is raised as an exception. If it is `False` (the default), *obj* and *defect* are passed to `register_defect`.

</div>

<div class="method">

register_defect(obj, defect)

Register a *defect* on *obj*. In the email package, *defect* will always be a subclass of `~email.errors.MessageDefect`.

The default implementation calls the `append` method of the `defects` attribute of *obj*. When the email package calls `handle_defect`, *obj* will normally have a `defects` attribute that has an `append` method. Custom object types used with the email package (for example, custom `Message` objects) should also provide such an attribute, otherwise defects in parsed messages will raise unexpected errors.

</div>

<div class="method">

header_max_count(name)

Return the maximum allowed number of headers named *name*.

Called when a header is added to an `~email.message.EmailMessage` or `~email.message.Message` object. If the returned value is not `0` or `None`, and there are already a number of headers with the name *name* greater than or equal to the value returned, a `ValueError` is raised.

Because the default behavior of `Message.__setitem__` is to append the value to the list of headers, it is easy to create duplicate headers without realizing it. This method allows certain headers to be limited in the number of instances of that header that may be added to a `Message` programmatically. (The limit is not observed by the parser, which will faithfully produce as many headers as exist in the message being parsed.)

The default implementation returns `None` for all header names.

</div>

<div class="method">

header_source_parse(sourcelines)

The email package calls this method with a list of strings, each string ending with the line separation characters found in the source being parsed. The first line includes the field header name and separator. All whitespace in the source is preserved. The method should return the `(name, value)` tuple that is to be stored in the `Message` to represent the parsed header.

If an implementation wishes to retain compatibility with the existing email package policies, *name* should be the case preserved name (all characters up to the '`:`' separator), while *value* should be the unfolded value (all line separator characters removed, but whitespace kept intact), stripped of leading whitespace.

*sourcelines* may contain surrogateescaped binary data.

There is no default implementation

</div>

<div class="method">

header_store_parse(name, value)

The email package calls this method with the name and value provided by the application program when the application program is modifying a `Message` programmatically (as opposed to a `Message` created by a parser). The method should return the `(name, value)` tuple that is to be stored in the `Message` to represent the header.

If an implementation wishes to retain compatibility with the existing email package policies, the *name* and *value* should be strings or string subclasses that do not change the content of the passed in arguments.

There is no default implementation

</div>

<div class="method">

header_fetch_parse(name, value)

The email package calls this method with the *name* and *value* currently stored in the `Message` when that header is requested by the application program, and whatever the method returns is what is passed back to the application as the value of the header being retrieved. Note that there may be more than one header with the same name stored in the `Message`; the method is passed the specific name and value of the header destined to be returned to the application.

*value* may contain surrogateescaped binary data. There should be no surrogateescaped binary data in the value returned by the method.

There is no default implementation

</div>

<div class="method">

fold(name, value)

The email package calls this method with the *name* and *value* currently stored in the `Message` for a given header. The method should return a string that represents that header "folded" correctly (according to the policy settings) by composing the *name* with the *value* and inserting `linesep` characters at the appropriate places. See `5322` for a discussion of the rules for folding email headers.

*value* may contain surrogateescaped binary data. There should be no surrogateescaped binary data in the string returned by the method.

</div>

<div class="method">

fold_binary(name, value)

The same as `fold`, except that the returned value should be a bytes object rather than a string.

*value* may contain surrogateescaped binary data. These could be converted back into binary data in the returned bytes object.

</div>

</div>

<div class="EmailPolicy(**kw)">

This concrete `Policy` provides behavior that is intended to be fully compliant with the current email RFCs. These include (but are not limited to) `5322`, `2047`, and the current MIME RFCs.

This policy adds new header parsing and folding algorithms. Instead of simple strings, headers are `str` subclasses with attributes that depend on the type of the field. The parsing and folding algorithm fully implement `2047` and `5322`.

The default value for the `~email.policy.Policy.message_factory` attribute is `~email.message.EmailMessage`.

In addition to the settable attributes listed above that apply to all policies, this policy adds the following additional attributes:

<div class="versionadded">

3.6[^1]

</div>

<div class="attribute">

utf8

If `False`, follow `5322`, supporting non-ASCII characters in headers by encoding them as `2047` "encoded words". If `True`, follow `6532` and use `utf-8` encoding for headers. Messages formatted in this way may be passed to SMTP servers that support the `SMTPUTF8` extension (`6531`).

When `False`, the generator will raise `~email.errors.HeaderWriteError` if any header includes non-ASCII characters in a context where `2047` does not permit encoded words. This particularly applies to mailboxes ("addr-spec") with non-ASCII characters, which can be created via `~email.headerregistry.Address`. To use a mailbox with a non-ASCII domain name with `utf8=False`, first encode the domain using the third-party `idna` or `uts46` module or with `encodings.idna`. It is not possible to use a non-ASCII username ("local-part") in a mailbox when `utf8=False`.

<div class="versionchanged">

3.15 Can trigger the raising of `~email.errors.HeaderWriteError`. (Earlier versions incorrectly applied `2047` in certain contexts, mostly notably in addr-specs.)

</div>

</div>

<div class="attribute">

refold_source

If the value for a header in the `Message` object originated from a `~email.parser` (as opposed to being set by a program), this attribute indicates whether or not a generator should refold that value when transforming the message back into serialized form. The possible values are:

|  |  |
|----|----|
| `none` | all source values use original folding |
| `long` | source values that have any line that is longer than `max_line_length` will be refolded |
| `all` | all values are refolded. |

The default is `long`.

</div>

<div class="attribute">

header_factory

A callable that takes two arguments, `name` and `value`, where `name` is a header field name and `value` is an unfolded header field value, and returns a string subclass that represents that header. A default `header_factory` (see `~email.headerregistry`) is provided that supports custom parsing for the various address and date [RFC 5322](http://www.faqs.org/rfcs/rfc5322.html "RFC 5322") header field types, and the major MIME header field stypes. Support for additional custom parsing will be added in the future.

</div>

<div class="attribute">

content_manager

An object with at least two methods: get_content and set_content. When the `~email.message.EmailMessage.get_content` or `~email.message.EmailMessage.set_content` method of an `~email.message.EmailMessage` object is called, it calls the corresponding method of this object, passing it the message object as its first argument, and any arguments or keywords that were passed to it as additional arguments. By default `content_manager` is set to `~email.contentmanager.raw_data_manager`.

<div class="versionadded">

3.4

</div>

</div>

The class provides the following concrete implementations of the abstract methods of `Policy`:

<div class="method">

header_max_count(name)

Returns the value of the `~email.headerregistry.BaseHeader.max_count` attribute of the specialized class used to represent the header with the given name.

</div>

<div class="method">

header_source_parse(sourcelines)

The name is parsed as everything up to the '`:`' and returned unmodified. The value is determined by stripping leading whitespace off the remainder of the first line, joining all subsequent lines together, and stripping any trailing carriage return or linefeed characters.

</div>

<div class="method">

header_store_parse(name, value)

The name is returned unchanged. If the input value has a `name` attribute and it matches *name* ignoring case, the value is returned unchanged. Otherwise the *name* and *value* are passed to `header_factory`, and the resulting header object is returned as the value. In this case a `ValueError` is raised if the input value contains CR or LF characters.

</div>

<div class="method">

header_fetch_parse(name, value)

If the value has a `name` attribute, it is returned to unmodified. Otherwise the *name*, and the *value* with any CR or LF characters removed, are passed to the `header_factory`, and the resulting header object is returned. Any surrogateescaped bytes get turned into the Unicode unknown-character glyph.

</div>

<div class="method">

fold(name, value)

Header folding is controlled by the `refold_source` policy setting. A value is considered to be a 'source value' if and only if it does not have a `name` attribute (having a `name` attribute means it is a header object of some sort). If a source value needs to be refolded according to the policy, it is converted into a header object by passing the *name* and the *value* with any CR and LF characters removed to the `header_factory`. Folding of a header object is done by calling its `fold` method with the current policy.

Source values are split into lines using `~str.splitlines`. If the value is not to be refolded, the lines are rejoined using the `linesep` from the policy and returned. The exception is lines containing non-ascii binary data. In that case the value is refolded regardless of the `refold_source` setting, which causes the binary data to be CTE encoded using the `unknown-8bit` charset.

</div>

<div class="method">

fold_binary(name, value)

The same as `fold` if `~Policy.cte_type` is `7bit`, except that the returned value is bytes.

If `~Policy.cte_type` is `8bit`, non-ASCII binary data is converted back into bytes. Headers with binary data are not refolded, regardless of the `refold_source` setting, since there is no way to know whether the binary data consists of single byte characters or multibyte characters.

</div>

</div>

The following instances of `EmailPolicy` provide defaults suitable for specific application domains. Note that in the future the behavior of these instances (in particular the `HTTP` instance) may be adjusted to conform even more closely to the RFCs relevant to their domains.

<div class="data">

default

An instance of `EmailPolicy` with all defaults unchanged. This policy uses the standard Python `\n` line endings rather than the RFC-correct `\r\n`.

</div>

<div class="data">

SMTP

Suitable for serializing messages in conformance with the email RFCs. Like `default`, but with `linesep` set to `\r\n`, which is RFC compliant.

</div>

<div class="data">

SMTPUTF8

The same as `SMTP` except that `~EmailPolicy.utf8` is `True`. Useful for serializing messages to a message store without using encoded words in the headers. Should only be used for SMTP transmission if the sender or recipient addresses have non-ASCII characters (the `smtplib.SMTP.send_message` method handles this automatically).

</div>

<div class="data">

HTTP

Suitable for serializing headers with for use in HTTP traffic. Like `SMTP` except that `max_line_length` is set to `None` (unlimited).

</div>

<div class="data">

strict

Convenience instance. The same as `default` except that `raise_on_defect` is set to `True`. This allows any policy to be made strict by writing:

    somepolicy + policy.strict

</div>

With all of these `EmailPolicies <.EmailPolicy>`, the effective API of the email package is changed from the Python 3.2 API in the following ways:

- Setting a header on a `~email.message.Message` results in that header being parsed and a header object created.
- Fetching a header value from a `~email.message.Message` results in that header being parsed and a header object created and returned.
- Any header object, or any header that is refolded due to the policy settings, is folded using an algorithm that fully implements the RFC folding algorithms, including knowing where encoded words are required and allowed.

From the application view, this means that any header obtained through the `~email.message.EmailMessage` is a header object with extra attributes, whose string value is the fully decoded value of the header. Likewise, a header may be assigned a new value, or a new header created, using a string, and the policy will take care of converting the string into the correct RFC encoded form.

The header objects and their attributes are described in `~email.headerregistry`.

<div class="Compat32(**kw)">

This concrete `Policy` is the backward compatibility policy. It replicates the behavior of the email package in Python 3.2. The `!policy` module also defines an instance of this class, `compat32`, that is used as the default policy. Thus the default behavior of the email package is to maintain compatibility with Python 3.2.

The following attributes have values that are different from the `Policy` default:

<div class="attribute">

[mangle_from]()

The default is `True`.

</div>

The class provides the following concrete implementations of the abstract methods of `Policy`:

<div class="method">

header_source_parse(sourcelines)

The name is parsed as everything up to the '`:`' and returned unmodified. The value is determined by stripping leading whitespace off the remainder of the first line, joining all subsequent lines together, and stripping any trailing carriage return or linefeed characters.

</div>

<div class="method">

header_store_parse(name, value)

The name and value are returned unmodified.

</div>

<div class="method">

header_fetch_parse(name, value)

If the value contains binary data, it is converted into a `~email.header.Header` object using the `unknown-8bit` charset. Otherwise it is returned unmodified.

</div>

<div class="method">

fold(name, value)

Headers are folded using the `~email.header.Header` folding algorithm, which preserves existing line breaks in the value, and wraps each resulting line to the `max_line_length`. Non-ASCII binary data are CTE encoded using the `unknown-8bit` charset.

</div>

<div class="method">

fold_binary(name, value)

Headers are folded using the `~email.header.Header` folding algorithm, which preserves existing line breaks in the value, and wraps each resulting line to the `max_line_length`. If `cte_type` is `7bit`, non-ascii binary data is CTE encoded using the `unknown-8bit` charset. Otherwise the original source header is used, with its existing line breaks and any (RFC invalid) binary data it may contain.

</div>

</div>

<div class="data">

compat32

An instance of `Compat32`, providing backward compatibility with the behavior of the email package in Python 3.2.

> [!NOTE]
> The `compat32` policy should not be used as a policy for `~email.message.EmailMessage` objects, and should only be used to serialize messages that were created using the `compat32` policy.

</div>

**Footnotes**

[^1]: Originally added in 3.3 as a `provisional feature <provisional
    package>`.

# `!email.utils`: Miscellaneous utilities

<div class="module" data-synopsis="Miscellaneous email package utilities.">

email.utils

</div>

**Source code:** `Lib/email/utils.py`

------------------------------------------------------------------------

There are a couple of useful utilities provided in the `!email.utils` module:

<div class="function">

localtime(dt=None)

Return local time as an aware datetime object. If called without arguments, return the current time. Otherwise the *dt* argument should be a `~datetime.datetime` instance, and it is converted to the local time zone according to the system time zone database. If *dt* is naive (that is, `dt.tzinfo` is `None`), it is assumed to be in local time.

<div class="versionadded">

3.3

</div>

<div class="deprecated-removed">

3.12 3.14 The *isdst* parameter.

</div>

</div>

<div class="function">

make_msgid(idstring=None, domain=None)

Returns a string suitable for an `2822`-compliant `Message-ID` header. Optional *idstring*, if given, is a string used to strengthen the uniqueness of the message id. Optional *domain*, if given, provides the portion of the msgid after the '@'. The default is the local hostname. It is normally not necessary to override this default, but may be useful in certain cases, such as constructing a distributed system that uses a consistent domain name across multiple hosts.

<div class="versionchanged">

3.2 Added the *domain* keyword.

</div>

</div>

The remaining functions are part of the legacy (`Compat32`) email API. There is no need to directly use these with the new API, since the parsing and formatting they provide is done automatically by the header parsing machinery of the new API.

<div class="function">

quote(str)

Return a new string with backslashes in *str* replaced by two backslashes, and double quotes replaced by backslash-double quote.

</div>

<div class="function">

unquote(str)

Return a new string which is an *unquoted* version of *str*. If *str* ends and begins with double quotes, they are stripped off. Likewise if *str* ends and begins with angle brackets, they are stripped off.

</div>

<div class="function">

parseaddr(address, \*, strict=True)

Parse address -- which should be the value of some address-containing field such as `To` or `Cc` -- into its constituent *realname* and *email address* parts. Returns a tuple of that information, unless the parse fails, in which case a 2-tuple of `('', '')` is returned.

If *strict* is true, use a strict parser which rejects malformed inputs.

<div class="versionchanged">

3.13 Add *strict* optional parameter and reject malformed inputs by default.

</div>

</div>

<div class="function">

formataddr(pair, charset='utf-8', \*, strict=True)

The inverse of `parseaddr`, this takes a 2-tuple of the form `(realname, email_address)` and returns the string value suitable for a `To` or `Cc` header. If the first element of *pair* is false, then the second element is returned unmodified.

Optional *charset* is the character set that will be used in the `2047` encoding of the `realname` if the `realname` contains non-ASCII characters. Can be an instance of `str` or a `~email.charset.Charset`. Defaults to `utf-8`.

If *strict* is true (the default), raise `ValueError` for inputs that contain CR or LF, which are not allowed in an email address. Set *strict* to `False` to allow non-strict inputs.

<div class="versionchanged">

3.3 Added the *charset* option.

</div>

<div class="versionchanged">

3.16 Added the *strict* parameter.

</div>

</div>

<div class="function">

getaddresses(fieldvalues, \*, strict=True)

This function returns a list of 2-tuples of the form returned by `parseaddr`. *fieldvalues* is a sequence of header field values as might be returned by `Message.get_all() <email.message.Message.get_all>`.

If *strict* is true, use a strict parser which rejects malformed inputs.

Here's a simple example that gets all the recipients of a message:

    from email.utils import getaddresses

    tos = msg.get_all('to', [])
    ccs = msg.get_all('cc', [])
    resent_tos = msg.get_all('resent-to', [])
    resent_ccs = msg.get_all('resent-cc', [])
    all_recipients = getaddresses(tos + ccs + resent_tos + resent_ccs)

<div class="versionchanged">

3.13 Add *strict* optional parameter and reject malformed inputs by default.

</div>

</div>

<div class="function">

parsedate(date)

Attempts to parse a date according to the rules in `2822#section-3.3`. However, some mailers don't follow that format as specified, so `parsedate` tries to guess correctly in such cases. *date* is a string containing an `2822` date, such as `"Mon, 20 Nov 1995 19:12:08 -0500"`. If it succeeds in parsing the date, `parsedate` returns a 9-tuple that can be passed directly to `time.mktime`; otherwise `None` will be returned. Note that indexes 6, 7, and 8 of the result tuple are not usable.

</div>

<div class="function">

parsedate_tz(date)

Performs the same function as `parsedate`, but returns either `None` or a 10-tuple; the first 9 elements make up a tuple that can be passed directly to `time.mktime`, and the tenth is the offset of the date's timezone from UTC (which is the official term for Greenwich Mean Time)[^1]. If the input string has no timezone, the last element of the tuple returned is `0`, which represents UTC. Note that indexes 6, 7, and 8 of the result tuple are not usable.

</div>

<div class="function">

parsedate_to_datetime(date)

The inverse of `format_datetime`. Performs the same function as `parsedate`, but on success returns a `~datetime.datetime`; otherwise `ValueError` is raised if *date* contains an invalid value such as an hour greater than 23 or a timezone offset not between -24 and 24 hours. If the input date has a timezone of `-0000`, the `datetime` will be a naive `datetime`, and if the date conforms to the RFCs it will represent a time in UTC but with no indication of the actual source timezone of the message the date comes from. If the input date has any other valid timezone offset, the `datetime` will be an aware `datetime` whose corresponding `~datetime.datetime.tzinfo` attribute is a `~datetime.timezone` object.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

mktime_tz(tuple)

Turn a 10-tuple as returned by `parsedate_tz` into a UTC timestamp (seconds since the Epoch). If the timezone item in the tuple is `None`, assume local time.

</div>

<div class="function">

formatdate(timeval=None, localtime=False, usegmt=False)

Returns a date string as per `2822`, for example:

    Fri, 09 Nov 2001 01:08:47 -0000

Optional *timeval*, if given, is a floating-point time value as accepted by `time.gmtime` and `time.localtime`, otherwise the current time is used.

Optional *localtime* is a flag that when `True`, interprets *timeval*, and returns a date relative to the local timezone instead of UTC, properly taking daylight saving time into account. The default is `False` meaning UTC is used.

Optional *usegmt* is a flag that when `True`, outputs a date string with the timezone as an ASCII string `GMT`, rather than a numeric `-0000`. This is needed for some protocols (such as HTTP). This only applies when *localtime* is `False`. The default is `False`.

</div>

<div class="function">

format_datetime(dt, usegmt=False)

Like `formatdate`, but the input is a `~datetime.datetime` instance. If it is a naive datetime, it is assumed to be "UTC with no information about the source timezone", and the conventional `-0000` is used for the timezone. If it is an aware `datetime`, then the numeric timezone offset is used. If it is an aware timezone with offset zero, then *usegmt* may be set to `True`, in which case the string `GMT` is used instead of the numeric timezone offset. This provides a way to generate standards-conformant HTTP date headers.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

decode_rfc2231(s)

Decode the string *s* according to `2231`.

</div>

<div class="function">

encode_rfc2231(s, charset=None, language=None)

Encode the string *s* according to `2231`. Optional *charset* and *language*, if given, are the character set name and language name to use. If neither is given, *s* is returned as-is. If *charset* is given but *language* is not, the string is encoded using the empty string for *language*.

</div>

<div class="function">

collapse_rfc2231_value(value, errors='replace', fallback_charset='us-ascii')

When a header parameter is encoded in `2231` format, `Message.get_param() <email.message.Message.get_param>` may return a 3-tuple containing the character set, language, and value. `collapse_rfc2231_value` turns this into a string. Optional *errors* is passed to the *errors* argument of `str`'s `~str.encode` method; it defaults to `'replace'`. Optional *fallback_charset* specifies the character set to use if the one in the `2231` header is not known by Python; it defaults to `'us-ascii'`.

For convenience, if the *value* passed to `collapse_rfc2231_value` is not a tuple, it should be a string and it is returned unquoted.

</div>

<div class="function">

decode_params(params)

Decode parameters list according to `2231`. *params* is a sequence of 2-tuples containing elements of the form `(content-type, string-value)`.

</div>

**Footnotes**

[^1]: Note that the sign of the timezone offset is the opposite of the sign of the `time.timezone` variable for the same timezone; the latter variable follows the POSIX standard while this module follows `2822`.

# `!ensurepip` --- Bootstrapping the `pip` installer

<div class="module" data-synopsis="Bootstrapping the &quot;pip&quot; installer into an existing Python
installation or virtual environment.">

ensurepip

</div>

<div class="versionadded">

3.4

</div>

**Source code:** `Lib/ensurepip`

------------------------------------------------------------------------

The `!ensurepip` package provides support for bootstrapping the `pip` installer into an existing Python installation or virtual environment. This bootstrapping approach reflects the fact that `pip` is an independent project with its own release cycle, and the latest available stable version is bundled with maintenance and feature releases of the CPython reference interpreter.

In most cases, end users of Python shouldn't need to invoke this module directly (as `pip` should be bootstrapped by default), but it may be needed if installing `pip` was skipped when installing Python (or when creating a virtual environment) or after explicitly uninstalling `pip`.

> [!NOTE]
> This module *does not* access the internet. All of the components needed to bootstrap `pip` are included as internal parts of the package.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

<div class="seealso">

`installing-index`  
The end user guide for installing Python packages

`453`: Explicit bootstrapping of pip in Python installations  
The original rationale and specification for this module.

</div>

<div class="availability">

not Android, not iOS, not WASI.

This module is not supported on `mobile platforms <mobile-availability>` or `WebAssembly platforms <wasm-availability>`.

</div>

## Command-line interface

<div class="program">

ensurepip

</div>

The command line interface is invoked using the interpreter's `-m` switch.

The simplest possible invocation is:

    python -m ensurepip

This invocation will install `pip` if it is not already installed, but otherwise does nothing. To ensure the installed version of `pip` is at least as recent as the one available in `ensurepip`, pass the `--upgrade` option:

    python -m ensurepip --upgrade

By default, `pip` is installed into the current virtual environment (if one is active) or into the system site packages (if there is no active virtual environment). The installation location can be controlled through two additional command line options:

<div class="option">

`--root <dir>`  

Installs `pip` relative to the given root directory rather than the root of the currently active virtual environment (if any) or the default root for the current Python installation.

</div>

<div class="option">

`--user`  

Installs `pip` into the user site packages directory rather than globally for the current Python installation (this option is not permitted inside an active virtual environment).

</div>

By default, the scripts `pipX` and `pipX.Y` will be installed (where X.Y stands for the version of Python used to invoke `ensurepip`). The scripts installed can be controlled through two additional command line options:

<div class="option">

`--altinstall`  

If an alternate installation is requested, the `pipX` script will *not* be installed.

</div>

<div class="option">

`--default-pip`  

If a "default pip" installation is requested, the `pip` script will be installed in addition to the two regular scripts.

</div>

Providing both of the script selection options will trigger an exception.

## Module API

`!ensurepip` exposes two functions for programmatic use:

<div class="function">

version()

Returns a string specifying the available version of pip that will be installed when bootstrapping an environment.

</div>

<div class="function">

bootstrap(root=None, upgrade=False, user=False, altinstall=False, default_pip=False, verbosity=0)

Bootstraps `pip` into the current or designated environment.

*root* specifies an alternative root directory to install relative to. If *root* is `None`, then installation uses the default install location for the current environment.

*upgrade* indicates whether or not to upgrade an existing installation of an earlier version of `pip` to the available version.

*user* indicates whether to use the user scheme rather than installing globally.

By default, the scripts `pipX` and `pipX.Y` will be installed (where X.Y stands for the current version of Python).

If *altinstall* is set, then `pipX` will *not* be installed.

If *default_pip* is set, then `pip` will be installed in addition to the two regular scripts.

Setting both *altinstall* and *default_pip* will trigger `ValueError`.

*verbosity* controls the level of output to `sys.stdout` from the bootstrapping operation.

<div class="audit-event">

ensurepip.bootstrap root ensurepip.bootstrap

</div>

> [!NOTE]
> The bootstrapping process has side effects on both `sys.path` and `os.environ`. Invoking the command line interface in a subprocess instead allows these side effects to be avoided.

> [!NOTE]
> The bootstrapping process may install additional modules required by `pip`, but other software should not assume those dependencies will always be present by default (as the dependencies may be removed in a future version of `pip`).

</div>

# `!enum` --- Support for enumerations

<div class="module" data-synopsis="Implementation of an enumeration class.">

enum

</div>

<div class="versionadded">

3.4

</div>

**Source code:** `Lib/enum.py`

<div class="sidebar">

**Important**

This page contains the API reference information. For tutorial information and discussion of more advanced topics, see

- `Basic Tutorial <enum-basic-tutorial>`
- `Advanced Tutorial <enum-advanced-tutorial>`
- `Enum Cookbook <enum-cookbook>`

</div>

------------------------------------------------------------------------

An enumeration:

- is a set of symbolic names (members) bound to unique values
- can be iterated over to return its canonical (i.e. non-alias) members in definition order
- uses *call* syntax to return members by value
- uses *index* syntax to return members by name

Enumerations are created either by using `class` syntax, or by using function-call syntax:

    >>> from enum import Enum

    >>> # class syntax
    >>> class Color(Enum):
    ...     RED = 1
    ...     GREEN = 2
    ...     BLUE = 3

    >>> # functional syntax
    >>> Color = Enum('Color', [('RED', 1), ('GREEN', 2), ('BLUE', 3)])

Even though we can use `class` syntax to create Enums, Enums are not normal Python classes. See `How are Enums different? <enum-class-differences>` for more details.

> [!NOTE]
> Nomenclature
>
> - The class `!Color` is an *enumeration* (or *enum*)
> - The attributes `!Color.RED`, `!Color.GREEN`, etc., are *enumeration members* (or *members*) and are functionally constants.
> - The enum members have *names* and *values* (the name of `!Color.RED` is `RED`, the value of `!Color.BLUE` is `3`, etc.)

------------------------------------------------------------------------

## Module contents

> `EnumType`
>
> > The `type` for Enum and its subclasses.
>
> `Enum`
>
> > Base class for creating enumerated constants.
>
> `IntEnum`
>
> > Base class for creating enumerated constants that are also subclasses of `int`. ([Notes](#notes))
>
> `StrEnum`
>
> > Base class for creating enumerated constants that are also subclasses of `str`. ([Notes](#notes))
>
> `Flag`
>
> > Base class for creating enumerated constants that can be combined using the bitwise operations without losing their `Flag` membership.
>
> `IntFlag`
>
> > Base class for creating enumerated constants that can be combined using the bitwise operators without losing their `IntFlag` membership. `IntFlag` members are also subclasses of `int`. ([Notes](#notes))
>
> `ReprEnum`
>
> > Used by `IntEnum`, `StrEnum`, and `IntFlag` to keep the `str() <str>` of the mixed-in type.
>
> `EnumCheck`
>
> > An enumeration with the values `CONTINUOUS`, `NAMED_FLAGS`, and `UNIQUE`, for use with `verify` to ensure various constraints are met by a given enumeration.
>
> `FlagBoundary`
>
> > An enumeration with the values `STRICT`, `CONFORM`, `EJECT`, and `KEEP` which allows for more fine-grained control over how invalid values are dealt with in an enumeration.
>
> `EnumDict`
>
> > A subclass of `dict` for use when subclassing `EnumType`.
>
> `auto`
>
> > Instances are replaced with an appropriate value for Enum members. `StrEnum` defaults to the lower-cased version of the member name, while other Enums default to 1 and increase from there.
>
> `~enum.property`
>
> > Allows `Enum` members to have attributes without conflicting with member names. The `value` and `name` attributes are implemented this way.
>
> `unique`
>
> > Enum class decorator that ensures only one name is bound to any one value.
>
> `verify`
>
> > Enum class decorator that checks user-selectable constraints on an enumeration.
>
> `member`
>
> > Make `obj` a member. Can be used as a decorator.
>
> `nonmember`
>
> > Do not make `obj` a member. Can be used as a decorator.
>
> `global_enum`
>
> > Modify the `str() <str>` and `repr` of an enum to show its members as belonging to the module instead of its class, and export the enum members to the global namespace.
>
> `show_flag_values`
>
> > Return a list of all power-of-two integers contained in a flag.
>
> `enum.bin`
>
> > Like built-in `bin`, except negative values are represented in two's complement, and the leading bit always indicates sign (`0` implies positive, `1` implies negative).

<div class="versionadded">

3.6 `Flag`, `IntFlag`, `auto`

</div>

<div class="versionadded">

3.11 `StrEnum`, `EnumCheck`, `ReprEnum`, `FlagBoundary`, `property`, `member`, `nonmember`, `global_enum`, `show_flag_values`

</div>

<div class="versionadded">

3.13 `EnumDict`

</div>

------------------------------------------------------------------------

## Data types

<div class="EnumType">

*EnumType* is the `metaclass` for *enum* enumerations. It is possible to subclass *EnumType* -- see `Subclassing EnumType <enumtype-examples>` for details.

`EnumType` is responsible for setting the correct `!__repr__`, `!__str__`, `!__format__`, and `!__reduce__` methods on the final *enum*, as well as creating the enum members, properly handling duplicates, providing iteration over the enum class, etc.

<div class="versionadded">

3.11

Before 3.11 `EnumType` was called `EnumMeta`, which is still available as an alias.

</div>

<div class="method">

EnumType.\_\_call\_\_(cls, value, names=None, \*, module=None, qualname=None, type=None, start=1, boundary=None)

This method is called in two different ways:

- to look up an existing member:

  > cls  
  > The enum class being called.
  >
  > value  
  > The value to lookup.

- to use the `cls` enum to create a new enum (only if the existing enum does not have any members):

  > cls  
  > The enum class being called.
  >
  > value  
  > The name of the new Enum to create.
  >
  > names  
  > The names/values of the members for the new Enum.
  >
  > module  
  > The name of the module the new Enum is created in.
  >
  > qualname  
  > The actual location in the module where this Enum can be found.
  >
  > type  
  > A mix-in type for the new Enum.
  >
  > start  
  > The first integer value for the Enum (used by `auto`).
  >
  > boundary  
  > How to handle out-of-range values from bit operations (`Flag` only).

</div>

<div class="method">

EnumType.\_\_contains\_\_(cls, member)

Returns `True` if member belongs to the `cls`:

    >>> some_var = Color.RED
    >>> some_var in Color
    True
    >>> Color.RED.value in Color
    True

<div class="versionchanged">

3.12

Before Python 3.12, a `TypeError` is raised if a non-Enum-member is used in a containment check.

</div>

</div>

<div class="method">

EnumType.\_\_dir\_\_(cls)

Returns `['__class__', '__doc__', '__members__', '__module__']` and the names of the members in *cls*:

    >>> dir(Color)
    ['BLUE', 'GREEN', 'RED', '__class__', '__contains__', '__doc__', '__getitem__', '__init_subclass__', '__iter__', '__len__', '__members__', '__module__', '__name__', '__qualname__', '_generate_next_value_', '_missing_']

</div>

<div class="method">

EnumType.\_\_getitem\_\_(cls, name)

Returns the Enum member in *cls* matching *name*, or raises a `KeyError`:

    >>> Color['BLUE']
    <Color.BLUE: 3>

</div>

<div class="method">

EnumType.\_\_iter\_\_(cls)

Returns each member in *cls* in definition order:

    >>> list(Color)
    [<Color.RED: 1>, <Color.GREEN: 2>, <Color.BLUE: 3>]

</div>

<div class="method">

EnumType.\_\_len\_\_(cls)

Returns the number of members in *cls*:

    >>> len(Color)
    3

</div>

<div class="attribute">

EnumType.\_\_members\_\_

Returns a mapping of every enum name to its member, including aliases

</div>

<div class="method">

EnumType.\_\_reversed\_\_(cls)

Returns each member in *cls* in reverse definition order:

    >>> list(reversed(Color))
    [<Color.BLUE: 3>, <Color.GREEN: 2>, <Color.RED: 1>]

</div>

</div>

<div class="Enum">

*Enum* is the base class for all *enum* enumerations.

<div class="attribute">

Enum.name

The name used to define the `Enum` member:

    >>> Color.BLUE.name
    'BLUE'

</div>

<div class="attribute">

Enum.value

The value given to the `Enum` member:

    >>> Color.RED.value
    1

Value of the member, can be set in `~Enum.__new__`.

> [!NOTE]
> Enum member values
>
> Member values can be anything: `int`, `str`, etc. If the exact value is unimportant you may use `auto` instances and an appropriate value will be chosen for you. See `auto` for the details.
>
> While mutable/unhashable values, such as `dict`, `list` or a mutable `~dataclasses.dataclass`, can be used, they will have a quadratic performance impact during creation relative to the total number of mutable/unhashable values in the enum.

</div>

<div class="attribute">

Enum.\_name\_

Name of the member.

</div>

<div class="attribute">

Enum.\_value\_

Value of the member, can be set in `~Enum.__new__`.

</div>

<div class="attribute">

Enum.\_order\_

No longer used, kept for backward compatibility. (class attribute, removed during class creation).

The `~Enum._order_` attribute can be provided to help keep Python 2 / Python 3 code in sync. It will be checked against the actual order of the enumeration and raise an error if the two do not match:

    >>> class Color(Enum):
    ...     _order_ = 'RED GREEN BLUE'
    ...     RED = 1
    ...     BLUE = 3
    ...     GREEN = 2
    ...
    Traceback (most recent call last):
    ...
    TypeError: member order does not match _order_:
       ['RED', 'BLUE', 'GREEN']
       ['RED', 'GREEN', 'BLUE']

> [!NOTE]
> In Python 2 code the `~Enum._order_` attribute is necessary as definition order is lost before it can be recorded.

<div class="versionadded">

3.6

</div>

</div>

<div class="attribute">

Enum.\_ignore\_

`_ignore_` is only used during creation and is removed from the enumeration once creation is complete.

`_ignore_` is a list of names that will not become members, and whose names will also be removed from the completed enumeration. See `TimePeriod <enum-time-period>` for an example.

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

Enum.\_\_dir\_\_(self)

Returns `['__class__', '__doc__', '__module__', 'name', 'value']` and any public methods defined on *self.\_\_class\_\_*:

    >>> from enum import Enum
    >>> import datetime as dt
    >>> class Weekday(Enum):
    ...     MONDAY = 1
    ...     TUESDAY = 2
    ...     WEDNESDAY = 3
    ...     THURSDAY = 4
    ...     FRIDAY = 5
    ...     SATURDAY = 6
    ...     SUNDAY = 7
    ...     @classmethod
    ...     def today(cls):
    ...         print(f'today is {cls(dt.date.today().isoweekday()).name}')
    ...
    >>> dir(Weekday.SATURDAY)
    ['__class__', '__doc__', '__eq__', '__hash__', '__module__', '_add_alias_', '_add_value_alias_', '_generate_next_value_', '_missing_', 'name', 'today', 'value']

</div>

<div class="method">

Enum.\_generate_next_value\_(name, start, count, last_values)

name  
The name of the member being defined (e.g. 'RED').

start  
The start value for the Enum; the default is 1.

count  
The number of members currently defined, not including this one.

last_values  
A list of the previous values.

A *staticmethod* that is used to determine the next value returned by `auto`.

> [!NOTE]

For standard `Enum` classes the next value chosen is the highest value seen incremented by one.

For `Flag` classes the next value chosen will be the next highest power-of-two.

This method may be overridden, e.g.:

    >>> from enum import auto, Enum
    >>> class PowersOfThree(Enum):

... @staticmethod ... def <span id="generate_next_value">generate_next_value</span>\_(name, start, count, last_values): ... return 3 \*\* (count + 1) ... FIRST = auto() ... SECOND = auto() ... \>\>\> PowersOfThree.SECOND.value 9

<div class="versionadded">

3.6

</div>

<div class="versionchanged">

3.13

</div>

Prior versions would use the last seen value instead of the highest value.

</div>

<div class="method">

Enum.\_\_init\_\_(self, *args,*\*kwds)

By default, does nothing. If multiple values are given in the member assignment, those values become separate arguments to `__init__`; e.g.

> \>\>\> from enum import Enum \>\>\> class Weekday(Enum): ... MONDAY = 1, 'Mon'

`Weekday.__init__()` would be called as `Weekday.__init__(self, 1, 'Mon')`

</div>

<div class="method">

Enum.\_\_init_subclass\_\_(cls, \*\*kwds)

A *classmethod* that is used to further configure subsequent subclasses. By default, does nothing.

</div>

<div class="method">

Enum.\_missing\_(cls, value)

A *classmethod* for looking up values not found in *cls*. By default it does nothing, but can be overridden to implement custom search behavior:

    >>> from enum import auto, StrEnum
    >>> class Build(StrEnum):
    ...     DEBUG = auto()
    ...     OPTIMIZED = auto()
    ...     @classmethod
    ...     def _missing_(cls, value):
    ...         value = value.lower()
    ...         for member in cls:
    ...             if member.value == value:
    ...                 return member
    ...         return None
    ...
    >>> Build.DEBUG.value
    'debug'
    >>> Build('deBUG')
    <Build.DEBUG: 'debug'>

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

Enum.\_\_new\_\_(cls, *args,*\*kwds)

By default, doesn't exist. If specified, either in the enum class definition or in a mixin class (such as `int`), all values given in the member assignment will be passed; e.g.

> \>\>\> from enum import Enum \>\>\> class MyIntEnum(int, Enum): ... TWENTYSIX = '1a', 16

results in the call `int('1a', 16)` and a value of `26` for the member.

> [!NOTE]
> When writing a custom `__new__`, do not use `super().__new__` -- call the appropriate `__new__` instead.

</div>

<div class="method">

Enum.\_\_repr\_\_(self)

Returns the string used for *repr()* calls. By default, returns the *Enum* name, member name, and value, but can be overridden:

    >>> from enum import auto, Enum
    >>> class OtherStyle(Enum):
    ...     ALTERNATE = auto()
    ...     OTHER = auto()
    ...     SOMETHING_ELSE = auto()
    ...     def __repr__(self):
    ...         cls_name = self.__class__.__name__
    ...         return f'{cls_name}.{self.name}'
    ...
    >>> OtherStyle.ALTERNATE, str(OtherStyle.ALTERNATE), f"{OtherStyle.ALTERNATE}"
    (OtherStyle.ALTERNATE, 'OtherStyle.ALTERNATE', 'OtherStyle.ALTERNATE')

</div>

<div class="method">

Enum.\_\_str\_\_(self)

Returns the string used for *str()* calls. By default, returns the *Enum* name and member name, but can be overridden:

    >>> from enum import auto, Enum
    >>> class OtherStyle(Enum):
    ...     ALTERNATE = auto()
    ...     OTHER = auto()
    ...     SOMETHING_ELSE = auto()
    ...     def __str__(self):
    ...         return f'{self.name}'
    ...
    >>> OtherStyle.ALTERNATE, str(OtherStyle.ALTERNATE), f"{OtherStyle.ALTERNATE}"
    (<OtherStyle.ALTERNATE: 1>, 'ALTERNATE', 'ALTERNATE')

</div>

<div class="method">

Enum.\_\_format\_\_(self)

Returns the string used for *format()* and *f-string* calls. By default, returns `__str__` return value, but can be overridden:

    >>> from enum import auto, Enum
    >>> class OtherStyle(Enum):
    ...     ALTERNATE = auto()
    ...     OTHER = auto()
    ...     SOMETHING_ELSE = auto()
    ...     def __format__(self, spec):
    ...         return f'{self.name}'
    ...
    >>> OtherStyle.ALTERNATE, str(OtherStyle.ALTERNATE), f"{OtherStyle.ALTERNATE}"
    (<OtherStyle.ALTERNATE: 1>, 'OtherStyle.ALTERNATE', 'ALTERNATE')

</div>

> [!NOTE]
> Using `auto` with `Enum` results in integers of increasing value, starting with `1`.

<div class="versionchanged">

3.12 Added `enum-dataclass-support`.

</div>

<div class="method">

Enum.\_add_alias\_

Adds a new name as an alias to an existing member:

    >>> Color.RED._add_alias_("ERROR")
    >>> Color.ERROR
    <Color.RED: 1>

Raises a `NameError` if the name is already assigned to a different member.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

Enum.\_add_value_alias\_

Adds a new value as an alias to an existing member:

    >>> Color.RED._add_value_alias_(42)
    >>> Color(42)
    <Color.RED: 1>

Raises a `ValueError` if the value is already linked with a different member.\
See `multi-value-enum` for an example.

<div class="versionadded">

3.13

</div>

</div>

</div>

<div class="IntEnum">

*IntEnum* is the same as `Enum`, but its members are also integers and can be used anywhere that an integer can be used. If any integer operation is performed with an *IntEnum* member, the resulting value loses its enumeration status.

> \>\>\> from enum import IntEnum \>\>\> class Number(IntEnum): ... ONE = 1 ... TWO = 2 ... THREE = 3 ... \>\>\> Number.THREE \<Number.THREE: 3\> \>\>\> Number.ONE + Number.TWO 3 \>\>\> Number.THREE + 5 8 \>\>\> Number.THREE == 3 True

> [!NOTE]
> Using `auto` with `IntEnum` results in integers of increasing value, starting with `1`.

<div class="versionchanged">

3.11 `~object.__str__` is now `!int.__str__` to better support the *replacement of existing constants* use-case. `~object.__format__` was already `!int.__format__` for that same reason.

</div>

</div>

<div class="StrEnum">

*StrEnum* is the same as `Enum`, but its members are also strings and can be used in most of the same places that a string can be used. The result of any string operation performed on or with a *StrEnum* member is not part of the enumeration.

\>\>\> from enum import StrEnum, auto \>\>\> class Color(StrEnum): ... RED = 'r' ... GREEN = 'g' ... BLUE = 'b' ... UNKNOWN = auto() ... \>\>\> Color.RED \<Color.RED: 'r'\> \>\>\> Color.UNKNOWN \<Color.UNKNOWN: 'unknown'\> \>\>\> str(Color.UNKNOWN) 'unknown'

> [!NOTE]
> There are places in the stdlib that check for an exact `str` instead of a `str` subclass (i.e. `type(unknown) == str` instead of `isinstance(unknown, str)`), and in those locations you will need to use `str(MyStrEnum.MY_MEMBER)`.

> [!NOTE]
> Using `auto` with `StrEnum` results in the lower-cased member name as the value.

> [!NOTE]
> `~object.__str__` is `!str.__str__` to better support the *replacement of existing constants* use-case. `~object.__format__` is likewise `!str.__format__` for that same reason.

<div class="versionadded">

3.11

</div>

</div>

<div class="Flag">

`Flag` is the same as `Enum`, but its members support the bitwise operators `&` (*AND*), `|` (*OR*), `^` (*XOR*), and `~` (*INVERT*); the results of those operations are (aliases of) members of the enumeration.

<div class="method">

\_\_contains\_\_(self, value)

Returns *True* if value is in self:

    >>> from enum import Flag, auto
    >>> class Color(Flag):
    ...     RED = auto()
    ...     GREEN = auto()
    ...     BLUE = auto()
    ...
    >>> purple = Color.RED | Color.BLUE
    >>> white = Color.RED | Color.GREEN | Color.BLUE
    >>> Color.GREEN in purple
    False
    >>> Color.GREEN in white
    True
    >>> purple in white
    True
    >>> white in purple
    False

</div>

<div class="method">

\_\_iter\_\_(self)

Returns all contained non-alias members:

    >>> list(Color.RED)
    [<Color.RED: 1>]
    >>> list(purple)
    [<Color.RED: 1>, <Color.BLUE: 4>]

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

\_\_len\_\_(self)

Returns number of members in flag:

    >>> len(Color.GREEN)
    1
    >>> len(white)
    3

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

\_\_bool\_\_(self)

Returns *True* if any members in flag, *False* otherwise:

    >>> bool(Color.GREEN)
    True
    >>> bool(white)
    True
    >>> black = Color(0)
    >>> bool(black)
    False

</div>

<div class="method">

\_\_or\_\_(self, other)

Returns current flag binary or'ed with other:

    >>> Color.RED | Color.GREEN
    <Color.RED|GREEN: 3>

</div>

<div class="method">

\_\_and\_\_(self, other)

Returns current flag binary and'ed with other:

    >>> purple & white
    <Color.RED|BLUE: 5>
    >>> purple & Color.GREEN
    <Color: 0>

</div>

<div class="method">

\_\_xor\_\_(self, other)

Returns current flag binary xor'ed with other:

    >>> purple ^ white
    <Color.GREEN: 2>
    >>> purple ^ Color.GREEN
    <Color.RED|GREEN|BLUE: 7>

</div>

<div class="method">

\_\_invert\_\_(self)

Returns all the flags in *type(self)* that are not in *self*:

    >>> ~white
    <Color: 0>
    >>> ~purple
    <Color.GREEN: 2>
    >>> ~Color.RED
    <Color.GREEN|BLUE: 6>

</div>

<div class="method">

<span id="numeric_repr">numeric_repr</span>\_

Function used to format any remaining unnamed numeric values. Default is the value's repr; common choices are `hex` and `oct`.

</div>

> [!NOTE]
> Using `auto` with `Flag` results in integers that are powers of two, starting with `1`.

<div class="versionchanged">

3.11 The *repr()* of zero-valued flags has changed. It is now:

\>\>\> Color(0) \# doctest: +SKIP \<Color: 0\>

</div>

</div>

<div class="IntFlag">

`IntFlag` is the same as `Flag`, but its members are also integers and can be used anywhere that an integer can be used.

> \>\>\> from enum import IntFlag, auto \>\>\> class Color(IntFlag): ... RED = auto() ... GREEN = auto() ... BLUE = auto() ... \>\>\> Color.RED & 2 \<Color: 0\> \>\>\> Color.RED \| 2 \<Color.RED\|GREEN: 3\>

If any integer operation is performed with an *IntFlag* member, the result is not an *IntFlag*:

    >>> Color.RED + 2
    3

If a `Flag` operation is performed with an *IntFlag* member and:

- the result is a valid *IntFlag*: an *IntFlag* is returned
- the result is not a valid *IntFlag*: the result depends on the `FlagBoundary` setting

The `repr` of unnamed zero-valued flags has changed. It is now:

    >>> Color(0)
    <Color: 0>

> [!NOTE]
> Using `auto` with `IntFlag` results in integers that are powers of two, starting with `1`.

<div class="versionchanged">

3.11

`~object.__str__` is now `!int.__str__` to better support the *replacement of existing constants* use-case. `~object.__format__` was already `!int.__format__` for that same reason.

Inversion of an `!IntFlag` now returns a positive value that is the union of all flags not in the given flag, rather than a negative value. This matches the existing `Flag` behavior.

</div>

</div>

<div class="ReprEnum">

`!ReprEnum` uses the `repr() <Enum.__repr__>` of `Enum`, but the `str() <str>` of the mixed-in data type:

- `!int.__str__` for `IntEnum` and `IntFlag`
- `!str.__str__` for `StrEnum`

Inherit from `!ReprEnum` to keep the `str() <str>` / `format` of the mixed-in data type instead of using the `Enum`-default `str() <Enum.__str__>`.

<div class="versionadded">

3.11

</div>

</div>

<div class="EnumCheck">

*EnumCheck* contains the options used by the `verify` decorator to ensure various constraints; failed constraints result in a `ValueError`.

<div class="attribute">

UNIQUE

Ensure that each value has only one name:

    >>> from enum import Enum, verify, UNIQUE
    >>> @verify(UNIQUE)
    ... class Color(Enum):
    ...     RED = 1
    ...     GREEN = 2
    ...     BLUE = 3
    ...     CRIMSON = 1
    Traceback (most recent call last):
    ...
    ValueError: aliases found in <enum 'Color'>: CRIMSON -> RED

</div>

<div class="attribute">

CONTINUOUS

Ensure that there are no missing values between the lowest-valued member and the highest-valued member:

    >>> from enum import Enum, verify, CONTINUOUS
    >>> @verify(CONTINUOUS)
    ... class Color(Enum):
    ...     RED = 1
    ...     GREEN = 2
    ...     BLUE = 5
    Traceback (most recent call last):
    ...
    ValueError: invalid enum 'Color': missing values 3, 4

</div>

<div class="attribute">

NAMED_FLAGS

Ensure that any flag groups/masks contain only named flags -- useful when values are specified instead of being generated by `auto`:

    >>> from enum import Flag, verify, NAMED_FLAGS
    >>> @verify(NAMED_FLAGS)
    ... class Color(Flag):
    ...     RED = 1
    ...     GREEN = 2
    ...     BLUE = 4
    ...     WHITE = 15
    ...     NEON = 31
    Traceback (most recent call last):
    ...
    ValueError: invalid Flag 'Color': aliases WHITE and NEON are missing combined values of 0x18 [use enum.show_flag_values(value) for details]

</div>

> [!NOTE]
> CONTINUOUS and NAMED_FLAGS are designed to work with integer-valued members.

<div class="versionadded">

3.11

</div>

</div>

<div class="FlagBoundary">

`FlagBoundary` controls how out-of-range values are handled in `Flag` and its subclasses.

<div class="attribute">

STRICT

Out-of-range values cause a `ValueError` to be raised. This is the default for `Flag`:

    >>> from enum import Flag, STRICT, auto
    >>> class StrictFlag(Flag, boundary=STRICT):
    ...     RED = auto()
    ...     GREEN = auto()
    ...     BLUE = auto()
    ...
    >>> StrictFlag(2**2 + 2**4)
    Traceback (most recent call last):
    ...
    ValueError: <flag 'StrictFlag'> invalid value 20
        given 0b0 10100
      allowed 0b0 00111

</div>

<div class="attribute">

CONFORM

Out-of-range values have invalid values removed, leaving a valid `Flag` value:

    >>> from enum import Flag, CONFORM, auto
    >>> class ConformFlag(Flag, boundary=CONFORM):
    ...     RED = auto()
    ...     GREEN = auto()
    ...     BLUE = auto()
    ...
    >>> ConformFlag(2**2 + 2**4)
    <ConformFlag.BLUE: 4>

</div>

<div class="attribute">

EJECT

Out-of-range values lose their `Flag` membership and revert to `int`.

> \>\>\> from enum import Flag, EJECT, auto \>\>\> class EjectFlag(Flag, boundary=EJECT): ... RED = auto() ... GREEN = auto() ... BLUE = auto() ... \>\>\> EjectFlag(2\*\*2 + 2\*\*4) 20

</div>

<div class="attribute">

KEEP

Out-of-range values are kept, and the `Flag` membership is kept. This is the default for `IntFlag`:

    >>> from enum import Flag, KEEP, auto
    >>> class KeepFlag(Flag, boundary=KEEP):
    ...     RED = auto()
    ...     GREEN = auto()
    ...     BLUE = auto()
    ...
    >>> KeepFlag(2**2 + 2**4)
    <KeepFlag.BLUE|16: 20>

</div>

<div class="versionadded">

3.11

</div>

</div>

<div class="EnumDict">

*EnumDict* is a subclass of `dict` that is used as the namespace for defining enum classes (see `prepare`). It is exposed to allow subclasses of `EnumType` with advanced behavior like having multiple values per member. It should be called with the name of the enum class being created, otherwise private names and internal classes will not be handled correctly.

Note that only the `~collections.abc.MutableMapping` interface (`~object.__setitem__` and `~dict.update`) is overridden. It may be possible to bypass the checks using other `!dict` operations like `|= <object.__ior__>`.

<div class="attribute">

EnumDict.member_names

A list of member names.

</div>

<div class="versionadded">

3.13

</div>

</div>

------------------------------------------------------------------------

### Supported `__dunder__` names

`~EnumType.__members__` is a read-only ordered mapping of `member_name`:`member` items. It is only available on the class.

`~Enum.__new__`, if specified, must create and return the enum members; it is also a very good idea to set the member's `~Enum._value_` appropriately. Once all the members are created it is no longer used.

### Supported `_sunder_` names

- `~Enum._name_` -- name of the member
- `~Enum._value_` -- value of the member; can be set in `__new__`
- `~Enum._missing_` -- a lookup function used when a value is not found; may be overridden
- `~Enum._ignore_` -- a list of names, either as a `list` or a `str`, that will not be transformed into members, and will be removed from the final class
- `~Enum._order_` -- no longer used, kept for backward compatibility (class attribute, removed during class creation)
- `~Enum._generate_next_value_` -- used to get an appropriate value for an enum member; may be overridden
- `~Enum._add_alias_` -- adds a new name as an alias to an existing member.
- `~Enum._add_value_alias_` -- adds a new value as an alias to an existing member.
- While `_sunder_` names are generally reserved for the further development of the `Enum` class and can not be used, some are explicitly allowed:
  - `_repr_*` (e.g. `_repr_html_`), as used in [IPython's rich display](https://ipython.readthedocs.io/en/stable/config/integrating.html#rich-display)

<div class="versionadded">

3.6 `_missing_`, `_order_`, `_generate_next_value_`

</div>

<div class="versionadded">

3.7 `_ignore_`

</div>

<div class="versionadded">

3.13 `_add_alias_`, `_add_value_alias_`, `_repr_*`

</div>

------------------------------------------------------------------------

## Utilities and decorators

<div class="auto">

*auto* can be used in place of a value. If used, the *Enum* machinery will call an `Enum`'s `~Enum._generate_next_value_` to get an appropriate value. For `Enum` and `IntEnum` that appropriate value will be the highest value seen plus one; for `Flag` and `IntFlag` it will be the first power-of-two greater than the highest value seen; for `StrEnum` it will be the lower-cased version of the member's name. Care must be taken if mixing *auto()* with manually specified values.

*auto* instances are only resolved when at the top level of an assignment, either by itself or as part of a tuple:

- `FIRST = auto()` will work (auto() is replaced with `1`);
- `SECOND = auto(), -2` will work (auto is replaced with `2`, so `2, -2` is used to create the `SECOND` enum member;
- `THIRD = [auto(), -3]` will *not* work (`[<auto instance>, -3]` is used to create the `THIRD` enum member)

<div class="versionchanged">

3.11.1

In prior versions, `auto()` had to be the only thing on the assignment line to work properly.

</div>

`_generate_next_value_` can be overridden to customize the values used by *auto*.

> [!NOTE]
> In version 3.13 the default `_generate_next_value_` will always return the highest member value incremented by 1, and will fail if any member is an incompatible type.

</div>

<div class="decorator">

property

A decorator similar to the built-in `property`, but specifically for enumerations. It allows member attributes to have the same names as members themselves.

> [!NOTE]
> The *property* and the member must be defined in separate classes; for example, the *value* and *name* attributes are defined in the *Enum* class, and *Enum* subclasses can define members with the names `value` and `name`.

<div class="versionadded">

3.11

</div>

</div>

<div class="decorator">

unique

A `class` decorator specifically for enumerations. It searches an enumeration's `~EnumType.__members__`, gathering any aliases it finds; if any are found `ValueError` is raised with the details:

    >>> from enum import Enum, unique
    >>> @unique
    ... class Mistake(Enum):
    ...     ONE = 1
    ...     TWO = 2
    ...     THREE = 3
    ...     FOUR = 3
    ...
    Traceback (most recent call last):
    ...
    ValueError: duplicate values found in <enum 'Mistake'>: FOUR -> THREE

</div>

<div class="decorator">

verify

A `class` decorator specifically for enumerations. Members from `EnumCheck` are used to specify which constraints should be checked on the decorated enumeration.

<div class="versionadded">

3.11

</div>

</div>

<div class="decorator">

member

A decorator for use in enums: its target will become a member.

<div class="versionadded">

3.11

</div>

</div>

<div class="decorator">

nonmember

A decorator for use in enums: its target will not become a member.

<div class="versionadded">

3.11

</div>

</div>

<div class="decorator">

global_enum

A decorator to change the `str() <str>` and `repr` of an enum to show its members as belonging to the module instead of its class. Should only be used when the enum members are exported to the module global namespace (see `re.RegexFlag` for an example).

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

show_flag_values(value)

Return a list of all power-of-two integers contained in a flag *value*.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

bin(num, max_bits=None)

Like built-in `bin`, except negative values are represented in two's complement, and the leading bit always indicates sign (`0` implies positive, `1` implies negative).

> \>\>\> import enum \>\>\> enum.bin(10) '0b0 1010' \>\>\> enum.bin(~10) \# ~10 is -11 '0b1 0101'

<div class="versionadded">

3.11

</div>

</div>

------------------------------------------------------------------------

## Notes

`IntEnum`, `StrEnum`, and `IntFlag`

> These three enum types are designed to be drop-in replacements for existing integer- and string-based values; as such, they have extra limitations:
>
> - `__str__` uses the value and not the name of the enum member
> - `__format__`, because it uses `__str__`, will also use the value of the enum member instead of its name
>
> If you do not need/want those limitations, you can either create your own base class by mixing in the `int` or `str` type yourself:
>
>     >>> from enum import Enum
>     >>> class MyIntEnum(int, Enum):
>     ...     pass
>
> or you can reassign the appropriate `str`, etc., in your enum:
>
>     >>> from enum import Enum, IntEnum
>     >>> class MyIntEnum(IntEnum):
>     ...     __str__ = Enum.__str__

# `!errno` --- Standard errno system symbols

<div class="module" data-synopsis="Standard errno system symbols.">

errno

</div>

------------------------------------------------------------------------

This module makes available standard `errno` system symbols. The value of each symbol is the corresponding integer value. The names and descriptions are borrowed from `linux/include/errno.h`, which should be all-inclusive.

<div class="data">

errorcode

Dictionary providing a mapping from the errno value to the string name in the underlying system. For instance, `errno.errorcode[errno.EPERM]` maps to `'EPERM'`.

</div>

To translate a numeric error code to an error message, use `os.strerror`.

Of the following list, symbols that are not used on the current platform are not defined by the module. The specific list of defined symbols is available as `errno.errorcode.keys()`. Symbols available can include:

<div class="data">

EPERM

Operation not permitted. This error is mapped to the exception `PermissionError`.

</div>

<div class="data">

ENOENT

No such file or directory. This error is mapped to the exception `FileNotFoundError`.

</div>

<div class="data">

ESRCH

No such process. This error is mapped to the exception `ProcessLookupError`.

</div>

<div class="data">

EINTR

Interrupted system call. This error is mapped to the exception `InterruptedError`.

</div>

<div class="data">

EIO

I/O error

</div>

<div class="data">

ENXIO

No such device or address

</div>

<div class="data">

E2BIG

Arg list too long

</div>

<div class="data">

ENOEXEC

Exec format error

</div>

<div class="data">

EBADF

Bad file number

</div>

<div class="data">

ECHILD

No child processes. This error is mapped to the exception `ChildProcessError`.

</div>

<div class="data">

EAGAIN

Try again. This error is mapped to the exception `BlockingIOError`.

</div>

<div class="data">

ENOMEM

Out of memory

</div>

<div class="data">

EACCES

Permission denied. This error is mapped to the exception `PermissionError`.

</div>

<div class="data">

EFAULT

Bad address

</div>

<div class="data">

ENOTBLK

Block device required

</div>

<div class="data">

EBUSY

Device or resource busy

</div>

<div class="data">

EEXIST

File exists. This error is mapped to the exception `FileExistsError`.

</div>

<div class="data">

EXDEV

Cross-device link

</div>

<div class="data">

ENODEV

No such device

</div>

<div class="data">

ENOTDIR

Not a directory. This error is mapped to the exception `NotADirectoryError`.

</div>

<div class="data">

EISDIR

Is a directory. This error is mapped to the exception `IsADirectoryError`.

</div>

<div class="data">

EINVAL

Invalid argument

</div>

<div class="data">

ENFILE

File table overflow

</div>

<div class="data">

EMFILE

Too many open files

</div>

<div class="data">

ENOTTY

Not a typewriter

</div>

<div class="data">

ETXTBSY

Text file busy

</div>

<div class="data">

EFBIG

File too large

</div>

<div class="data">

ENOSPC

No space left on device

</div>

<div class="data">

ESPIPE

Illegal seek

</div>

<div class="data">

EROFS

Read-only file system

</div>

<div class="data">

EMLINK

Too many links

</div>

<div class="data">

EPIPE

Broken pipe. This error is mapped to the exception `BrokenPipeError`.

</div>

<div class="data">

EDOM

Math argument out of domain of func

</div>

<div class="data">

ERANGE

Math result not representable

</div>

<div class="data">

EDEADLK

Resource deadlock would occur

</div>

<div class="data">

ENAMETOOLONG

File name too long

</div>

<div class="data">

ENOLCK

No record locks available

</div>

<div class="data">

ENOSYS

Function not implemented

</div>

<div class="data">

ENOTEMPTY

Directory not empty

</div>

<div class="data">

ELOOP

Too many symbolic links encountered

</div>

<div class="data">

EWOULDBLOCK

Operation would block. This error is mapped to the exception `BlockingIOError`.

</div>

<div class="data">

ENOMSG

No message of desired type

</div>

<div class="data">

EIDRM

Identifier removed

</div>

<div class="data">

ECHRNG

Channel number out of range

</div>

<div class="data">

EL2NSYNC

Level 2 not synchronized

</div>

<div class="data">

EL3HLT

Level 3 halted

</div>

<div class="data">

EL3RST

Level 3 reset

</div>

<div class="data">

ELNRNG

Link number out of range

</div>

<div class="data">

EUNATCH

Protocol driver not attached

</div>

<div class="data">

ENOCSI

No CSI structure available

</div>

<div class="data">

EL2HLT

Level 2 halted

</div>

<div class="data">

EBADE

Invalid exchange

</div>

<div class="data">

EBADR

Invalid request descriptor

</div>

<div class="data">

EXFULL

Exchange full

</div>

<div class="data">

ENOANO

No anode

</div>

<div class="data">

EBADRQC

Invalid request code

</div>

<div class="data">

EBADSLT

Invalid slot

</div>

<div class="data">

EDEADLOCK

File locking deadlock error

</div>

<div class="data">

EBFONT

Bad font file format

</div>

<div class="data">

ENOSTR

Device not a stream

</div>

<div class="data">

ENODATA

No data available

</div>

<div class="data">

ETIME

Timer expired

</div>

<div class="data">

ENOSR

Out of streams resources

</div>

<div class="data">

ENONET

Machine is not on the network

</div>

<div class="data">

ENOPKG

Package not installed

</div>

<div class="data">

EREMOTE

Object is remote

</div>

<div class="data">

ENOLINK

Link has been severed

</div>

<div class="data">

EADV

Advertise error

</div>

<div class="data">

ESRMNT

Srmount error

</div>

<div class="data">

ECOMM

Communication error on send

</div>

<div class="data">

EPROTO

Protocol error

</div>

<div class="data">

EMULTIHOP

Multihop attempted

</div>

<div class="data">

EDOTDOT

RFS specific error

</div>

<div class="data">

EBADMSG

Not a data message

</div>

<div class="data">

EOVERFLOW

Value too large for defined data type

</div>

<div class="data">

ENOTUNIQ

Name not unique on network

</div>

<div class="data">

EBADFD

File descriptor in bad state

</div>

<div class="data">

EREMCHG

Remote address changed

</div>

<div class="data">

ELIBACC

Can not access a needed shared library

</div>

<div class="data">

ELIBBAD

Accessing a corrupted shared library

</div>

<div class="data">

ELIBSCN

.lib section in a.out corrupted

</div>

<div class="data">

ELIBMAX

Attempting to link in too many shared libraries

</div>

<div class="data">

ELIBEXEC

Cannot exec a shared library directly

</div>

<div class="data">

EILSEQ

Illegal byte sequence

</div>

<div class="data">

ERESTART

Interrupted system call should be restarted

</div>

<div class="data">

ESTRPIPE

Streams pipe error

</div>

<div class="data">

EUSERS

Too many users

</div>

<div class="data">

ENOTSOCK

Socket operation on non-socket

</div>

<div class="data">

EDESTADDRREQ

Destination address required

</div>

<div class="data">

EMSGSIZE

Message too long

</div>

<div class="data">

EPROTOTYPE

Protocol wrong type for socket

</div>

<div class="data">

ENOPROTOOPT

Protocol not available

</div>

<div class="data">

EPROTONOSUPPORT

Protocol not supported

</div>

<div class="data">

ESOCKTNOSUPPORT

Socket type not supported

</div>

<div class="data">

EOPNOTSUPP

Operation not supported on transport endpoint

</div>

<div class="data">

ENOTSUP

Operation not supported

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EPFNOSUPPORT

Protocol family not supported

</div>

<div class="data">

EAFNOSUPPORT

Address family not supported by protocol

</div>

<div class="data">

EADDRINUSE

Address already in use

</div>

<div class="data">

EADDRNOTAVAIL

Cannot assign requested address

</div>

<div class="data">

ENETDOWN

Network is down

</div>

<div class="data">

ENETUNREACH

Network is unreachable

</div>

<div class="data">

ENETRESET

Network dropped connection because of reset

</div>

<div class="data">

ECONNABORTED

Software caused connection abort. This error is mapped to the exception `ConnectionAbortedError`.

</div>

<div class="data">

ECONNRESET

Connection reset by peer. This error is mapped to the exception `ConnectionResetError`.

</div>

<div class="data">

ENOBUFS

No buffer space available

</div>

<div class="data">

EISCONN

Transport endpoint is already connected

</div>

<div class="data">

ENOTCONN

Transport endpoint is not connected

</div>

<div class="data">

ESHUTDOWN

Cannot send after transport endpoint shutdown. This error is mapped to the exception `BrokenPipeError`.

</div>

<div class="data">

ETOOMANYREFS

Too many references: cannot splice

</div>

<div class="data">

ETIMEDOUT

Connection timed out. This error is mapped to the exception `TimeoutError`.

</div>

<div class="data">

ECONNREFUSED

Connection refused. This error is mapped to the exception `ConnectionRefusedError`.

</div>

<div class="data">

EHOSTDOWN

Host is down

</div>

<div class="data">

EHOSTUNREACH

No route to host

</div>

<div class="data">

EHWPOISON

Memory page has hardware error.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

EALREADY

Operation already in progress. This error is mapped to the exception `BlockingIOError`.

</div>

<div class="data">

EINPROGRESS

Operation now in progress. This error is mapped to the exception `BlockingIOError`.

</div>

<div class="data">

ESTALE

Stale NFS file handle

</div>

<div class="data">

EUCLEAN

Structure needs cleaning

</div>

<div class="data">

ENOTNAM

Not a XENIX named type file

</div>

<div class="data">

ENAVAIL

No XENIX semaphores available

</div>

<div class="data">

EISNAM

Is a named type file

</div>

<div class="data">

EREMOTEIO

Remote I/O error

</div>

<div class="data">

EDQUOT

Quota exceeded

</div>

<div class="data">

EQFULL

Interface output queue is full

<div class="versionadded">

3.11

</div>

</div>

<div class="data">

ENOMEDIUM

No medium found

</div>

<div class="data">

EMEDIUMTYPE

Wrong medium type

</div>

<div class="data">

ENOKEY

Required key not available

</div>

<div class="data">

EKEYEXPIRED

Key has expired

</div>

<div class="data">

EKEYREVOKED

Key has been revoked

</div>

<div class="data">

EKEYREJECTED

Key was rejected by service

</div>

<div class="data">

ERFKILL

Operation not possible due to RF-kill

</div>

<div class="data">

ELOCKUNMAPPED

Locked lock was unmapped

</div>

<div class="data">

ENOTACTIVE

Facility is not active

</div>

<div class="data">

EAUTH

Authentication error

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EBADARCH

Bad CPU type in executable

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EBADEXEC

Bad executable (or shared library)

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EBADMACHO

Malformed Mach-o file

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EDEVERR

Device error

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EFTYPE

Inappropriate file type or format

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ENEEDAUTH

Need authenticator

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ENOATTR

Attribute not found

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ENOPOLICY

Policy not found

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EPROCLIM

Too many processes

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EPROCUNAVAIL

Bad procedure for program

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EPROGMISMATCH

Program version wrong

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EPROGUNAVAIL

RPC prog. not avail

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EPWROFF

Device power is off

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EBADRPC

RPC struct is bad

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ERPCMISMATCH

RPC version wrong

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ESHLIBVERS

Shared library version mismatch

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ENOTCAPABLE

Capabilities insufficient. This error is mapped to the exception `PermissionError`.

<div class="availability">

WASI, FreeBSD

</div>

<div class="versionadded">

3.11.1

</div>

</div>

<div class="data">

ECANCELED

Operation canceled

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EOWNERDEAD

Owner died

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ENOTRECOVERABLE

State not recoverable

<div class="versionadded">

3.2

</div>

</div>

# `!faulthandler` --- Dump the Python traceback

<div class="module" data-synopsis="Dump the Python traceback.">

faulthandler

</div>

<div class="versionadded">

3.3

</div>

------------------------------------------------------------------------

This module contains functions to dump Python tracebacks explicitly, on a fault, after a timeout, or on a user signal. Call `faulthandler.enable` to install fault handlers for the `~signal.SIGSEGV`, `~signal.SIGFPE`, `~signal.SIGABRT`, `~signal.SIGBUS`, and `~signal.SIGILL` signals. You can also enable them at startup by setting the `PYTHONFAULTHANDLER` environment variable or by using the `-X` `faulthandler` command line option.

The fault handler is compatible with system fault handlers like Apport or the Windows fault handler. The module uses an alternative stack for signal handlers if the `!sigaltstack` function is available. This allows it to dump the traceback even on a stack overflow.

The fault handler is called on catastrophic cases and therefore can only use signal-safe functions (e.g. it cannot allocate memory on the heap). Because of this limitation traceback dumping is minimal compared to normal Python tracebacks:

- Only ASCII is supported. The `backslashreplace` error handler is used on encoding.
- Each string is limited to 500 characters.
- Only the filename, the function name and the line number are displayed. (no source code)
- It is limited to 100 frames per thread, and 100 threads (configurable via *max_threads*).
- The order is reversed: the most recent call is shown first.

By default, the Python traceback is written to `sys.stderr`. To see tracebacks, applications must be run in the terminal. A log file can alternatively be passed to `faulthandler.enable`.

The module is implemented in C, so tracebacks can be dumped on a crash or when Python is deadlocked.

The `Python Development Mode <devmode>` calls `faulthandler.enable` at Python startup.

<div class="seealso">

Module `pdb`  
Interactive source code debugger for Python programs.

Module `traceback`  
Standard interface to extract, format and print stack traces of Python programs.

</div>

## Dumping the traceback

<div class="function">

dump_traceback(file=sys.stderr, all_threads=True, \*, max_threads=100)

Dump the tracebacks of all threads into *file*. If *all_threads* is `False`, dump only the current thread. *max_threads* caps the number of threads dumped.

<div class="seealso">

`traceback.print_tb`, which can be used to print a traceback object.

</div>

<div class="versionchanged">

3.5 Added support for passing file descriptor to this function.

</div>

<div class="versionchanged">

3.15 Added the *max_threads* keyword argument.

</div>

</div>

## Dumping the C stack

<div class="versionadded">

3.14

</div>

<div class="function">

dump_c_stack(file=sys.stderr)

Dump the C stack trace of the current thread into *file*.

If the Python build does not support it or the operating system does not provide a stack trace, then this prints an error in place of a dumped C stack.

</div>

### C Stack Compatibility

If the system does not support the C-level `backtrace(3)` or `dladdr1(3)`, then C stack dumps will not work. An error will be printed instead of the stack.

Additionally, some compilers do not support `CPython's <CPython>` implementation of C stack dumps. As a result, a different error may be printed instead of the stack, even if the operating system supports dumping stacks.

> [!NOTE]
> Dumping C stacks can be arbitrarily slow, depending on the DWARF level of the binaries in the call stack.

## Fault handler state

<div class="function">

enable(file=sys.stderr, all_threads=True, c_stack=True, \*, max_threads=100)

Enable the fault handler: install handlers for the `~signal.SIGSEGV`, `~signal.SIGFPE`, `~signal.SIGABRT`, `~signal.SIGBUS` and `~signal.SIGILL` signals to dump the Python traceback. If *all_threads* is `True`, produce tracebacks for every running thread. Otherwise, dump only the current thread.

The *file* must be kept open until the fault handler is disabled: see `issue with file descriptors <faulthandler-fd>`.

If *c_stack* is `True`, then the C stack trace is printed after the Python traceback, unless the system does not support it. See `dump_c_stack` for more information on compatibility.

*max_threads* caps the number of threads dumped when a fatal signal fires.

<div class="versionchanged">

3.5 Added support for passing file descriptor to this function.

</div>

<div class="versionchanged">

3.6 On Windows, a handler for Windows exception is also installed.

</div>

<div class="versionchanged">

3.10 The dump now mentions if a garbage collector collection is running if *all_threads* is true.

</div>

<div class="versionchanged">

3.14 Only the current thread is dumped if the `GIL` is disabled to prevent the risk of data races.

</div>

<div class="versionchanged">

3.14 The dump now displays the C stack trace if *c_stack* is true.

</div>

<div class="versionchanged">

3.15 Added the *max_threads* keyword argument.

</div>

</div>

<div class="function">

disable()

Disable the fault handler: uninstall the signal handlers installed by `enable`.

</div>

<div class="function">

is_enabled()

Check if the fault handler is enabled.

</div>

## Dumping the tracebacks after a timeout

<div class="function">

dump_traceback_later(timeout, repeat=False, file=sys.stderr, exit=False, \*, max_threads=100)

Dump the tracebacks of all threads, after a timeout of *timeout* seconds, or every *timeout* seconds if *repeat* is `True`. If *exit* is `True`, call `!_exit` with status=1 after dumping the tracebacks. (Note `!_exit` exits the process immediately, which means it doesn't do any cleanup like flushing file buffers.) If the function is called twice, the new call replaces previous parameters and resets the timeout. The timer has a sub-second resolution. *max_threads* caps the number of threads dumped.

The *file* must be kept open until the traceback is dumped or `cancel_dump_traceback_later` is called: see `issue with file
descriptors <faulthandler-fd>`.

This function is implemented using a watchdog thread.

<div class="versionchanged">

3.5 Added support for passing file descriptor to this function.

</div>

<div class="versionchanged">

3.7 This function is now always available.

</div>

<div class="versionchanged">

3.15 Added the *max_threads* keyword argument.

</div>

</div>

<div class="function">

cancel_dump_traceback_later()

Cancel the last call to `dump_traceback_later`.

</div>

## Dumping the traceback on a user signal

<div class="function">

register(signum, file=sys.stderr, all_threads=True, chain=False, \*, max_threads=100)

Register a user signal: install a handler for the *signum* signal to dump the traceback of all threads, or of the current thread if *all_threads* is `False`, into *file*. Call the previous handler if chain is `True`. *max_threads* caps the number of threads dumped.

The *file* must be kept open until the signal is unregistered by `unregister`: see `issue with file descriptors <faulthandler-fd>`.

Not available on Windows.

<div class="versionchanged">

3.5 Added support for passing file descriptor to this function.

</div>

<div class="versionchanged">

3.15 Added the *max_threads* keyword argument.

</div>

</div>

<div class="function">

unregister(signum)

Unregister a user signal: uninstall the handler of the *signum* signal installed by `register`. Return `True` if the signal was registered, `False` otherwise.

Not available on Windows.

</div>

## Issue with file descriptors

`enable`, `dump_traceback_later` and `register` keep the file descriptor of their *file* argument. If the file is closed and its file descriptor is reused by a new file, or if `os.dup2` is used to replace the file descriptor, the traceback will be written into a different file. Call these functions again each time that the file is replaced.

## Example

Example of a segmentation fault on Linux with and without enabling the fault handler:

```
$ python -c "import ctypes; ctypes.string_at(0)"
Segmentation fault

$ python -q -X faulthandler
>>> import ctypes
>>> ctypes.string_at(0)
Fatal Python error: Segmentation fault

Current thread 0x00007fb899f39700 (most recent call first):
  File "/opt/python/Lib/ctypes/__init__.py", line 486 in string_at
  File "<stdin>", line 1 in <module>

Current thread's C stack trace (most recent call first):
  Binary file "/opt/python/python", at _Py_DumpStack+0x42 [0x5b27f7d7147e]
  Binary file "/opt/python/python", at +0x32dcbd [0x5b27f7d85cbd]
  Binary file "/opt/python/python", at +0x32df8a [0x5b27f7d85f8a]
  Binary file "/usr/lib/libc.so.6", at +0x3def0 [0x77b73226bef0]
  Binary file "/usr/lib/libc.so.6", at +0x17ef9c [0x77b7323acf9c]
  Binary file "/opt/python/build/lib.linux-x86_64-3.15/_ctypes.cpython-315d-x86_64-linux-gnu.so", at +0xcdf6 [0x77b7315dddf6]
  Binary file "/usr/lib/libffi.so.8", at +0x7976 [0x77b73158f976]
  Binary file "/usr/lib/libffi.so.8", at +0x413c [0x77b73158c13c]
  Binary file "/usr/lib/libffi.so.8", at ffi_call+0x12e [0x77b73158ef0e]
  Binary file "/opt/python/build/lib.linux-x86_64-3.15/_ctypes.cpython-315d-x86_64-linux-gnu.so", at +0x15a33 [0x77b7315e6a33]
  Binary file "/opt/python/build/lib.linux-x86_64-3.15/_ctypes.cpython-315d-x86_64-linux-gnu.so", at +0x164fa [0x77b7315e74fa]
  Binary file "/opt/python/build/lib.linux-x86_64-3.15/_ctypes.cpython-315d-x86_64-linux-gnu.so", at +0xc624 [0x77b7315dd624]
  Binary file "/opt/python/python", at _PyObject_MakeTpCall+0xce [0x5b27f7b73883]
  Binary file "/opt/python/python", at +0x11bab6 [0x5b27f7b73ab6]
  Binary file "/opt/python/python", at PyObject_Vectorcall+0x23 [0x5b27f7b73b04]
  Binary file "/opt/python/python", at _PyEval_EvalFrameDefault+0x490c [0x5b27f7cbb302]
  Binary file "/opt/python/python", at +0x2818e6 [0x5b27f7cd98e6]
  Binary file "/opt/python/python", at +0x281aab [0x5b27f7cd9aab]
  Binary file "/opt/python/python", at PyEval_EvalCode+0xc5 [0x5b27f7cd9ba3]
  Binary file "/opt/python/python", at +0x255957 [0x5b27f7cad957]
  Binary file "/opt/python/python", at +0x255ab4 [0x5b27f7cadab4]
  Binary file "/opt/python/python", at _PyEval_EvalFrameDefault+0x6c3e [0x5b27f7cbd634]
  Binary file "/opt/python/python", at +0x2818e6 [0x5b27f7cd98e6]
  Binary file "/opt/python/python", at +0x281aab [0x5b27f7cd9aab]
  Binary file "/opt/python/python", at +0x11b6e1 [0x5b27f7b736e1]
  Binary file "/opt/python/python", at +0x11d348 [0x5b27f7b75348]
  Binary file "/opt/python/python", at +0x11d626 [0x5b27f7b75626]
  Binary file "/opt/python/python", at PyObject_Call+0x20 [0x5b27f7b7565e]
  Binary file "/opt/python/python", at +0x32a67a [0x5b27f7d8267a]
  Binary file "/opt/python/python", at +0x32a7f8 [0x5b27f7d827f8]
  Binary file "/opt/python/python", at +0x32ac1b [0x5b27f7d82c1b]
  Binary file "/opt/python/python", at Py_RunMain+0x31 [0x5b27f7d82ebe]
  <truncated rest of calls>
Segmentation fault
```

# `!fcntl` --- The `fcntl` and `ioctl` system calls

<div class="module" data-synopsis="The fcntl() and ioctl() system calls.">

fcntl

</div>

<div class="index">

pair: UNIX; file control pair: UNIX; I/O control

</div>

------------------------------------------------------------------------

This module performs file and I/O control on file descriptors. It is an interface to the `fcntl` and `ioctl` Unix routines. See the `fcntl(2)` and `ioctl(2)` Unix manual pages for full details.

<div class="availability">

Unix, not WASI.

</div>

All functions in this module take a file descriptor *fd* as their first argument. This can be an integer file descriptor, such as returned by `sys.stdin.fileno()`, or an `io.IOBase` object, such as `sys.stdin` itself, which provides a `~io.IOBase.fileno` that returns a genuine file descriptor.

<div class="versionchanged">

3.3 Operations in this module used to raise an `IOError` where they now raise an `OSError`.

</div>

<div class="versionchanged">

3.8 The `!fcntl` module now contains `F_ADD_SEALS`, `F_GET_SEALS`, and `F_SEAL_*` constants for sealing of `os.memfd_create` file descriptors.

</div>

<div class="versionchanged">

3.9 On macOS, the `!fcntl` module exposes the `F_GETPATH` constant, which obtains the path of a file from a file descriptor. On Linux(\>=3.15), the `!fcntl` module exposes the `F_OFD_GETLK`, `F_OFD_SETLK` and `F_OFD_SETLKW` constants, which are used when working with open file description locks.

</div>

<div class="versionchanged">

3.10 On Linux \>= 2.6.11, the `!fcntl` module exposes the `F_GETPIPE_SZ` and `F_SETPIPE_SZ` constants, which allow to check and modify a pipe's size respectively.

</div>

<div class="versionchanged">

3.11 On FreeBSD, the `!fcntl` module exposes the `F_DUP2FD` and `F_DUP2FD_CLOEXEC` constants, which allow to duplicate a file descriptor, the latter setting `FD_CLOEXEC` flag in addition.

</div>

<div class="versionchanged">

3.12 On Linux \>= 4.5, the `!fcntl` module exposes the `FICLONE` and `FICLONERANGE` constants, which allow to share some data of one file with another file by reflinking on some filesystems (e.g., btrfs, OCFS2, and XFS). This behavior is commonly referred to as "copy-on-write".

</div>

<div class="versionchanged">

3.13 On Linux \>= 2.6.32, the `!fcntl` module exposes the `F_GETOWN_EX`, `F_SETOWN_EX`, `F_OWNER_TID`, `F_OWNER_PID`, `F_OWNER_PGRP` constants, which allow to direct I/O availability signals to a specific thread, process, or process group. On Linux \>= 4.13, the `!fcntl` module exposes the `F_GET_RW_HINT`, `F_SET_RW_HINT`, `F_GET_FILE_RW_HINT`, `F_SET_FILE_RW_HINT`, and `RWH_WRITE_LIFE_*` constants, which allow to inform the kernel about the relative expected lifetime of writes on a given inode or via a particular open file description. On Linux \>= 5.1 and NetBSD, the `!fcntl` module exposes the `F_SEAL_FUTURE_WRITE` constant for use with `F_ADD_SEALS` and `F_GET_SEALS` operations. On FreeBSD, the `!fcntl` module exposes the `F_READAHEAD`, `F_ISUNIONSTACK`, and `F_KINFO` constants. On macOS and FreeBSD, the `!fcntl` module exposes the `F_RDAHEAD` constant. On NetBSD and AIX, the `!fcntl` module exposes the `F_CLOSEM` constant. On NetBSD, the `!fcntl` module exposes the `F_MAXFD` constant. On macOS and NetBSD, the `!fcntl` module exposes the `F_GETNOSIGPIPE` and `F_SETNOSIGPIPE` constant.

</div>

<div class="versionchanged">

3.14 On Linux \>= 6.1, the `!fcntl` module exposes the `F_DUPFD_QUERY` to query a file descriptor pointing to the same file.

</div>

The module defines the following functions:

<div class="function">

fcntl(fd, cmd, arg=0, /)

Perform the operation *cmd* on file descriptor *fd* (file objects providing a `~io.IOBase.fileno` method are accepted as well). The values used for *cmd* are operating system dependent, and are available as constants in the `!fcntl` module, using the same names as used in the relevant C header files. The argument *arg* can either be an integer value, a `bytes-like object`, or a string. The type and size of *arg* must match the type and size of the argument of the operation as specified in the relevant C documentation.

When *arg* is an integer, the function returns the integer return value of the C `fcntl` call.

When the argument is bytes-like object, it represents a binary structure, for example, created by `struct.pack`. A string value is encoded to binary using the UTF-8 encoding. The binary data is copied to a buffer whose address is passed to the C `fcntl` call. The return value after a successful call is the contents of the buffer, converted to a `bytes` object. The length of the returned object will be the same as the length of the *arg* argument.

If the `fcntl` call fails, an `OSError` is raised.

> [!NOTE]
> If the type or size of *arg* does not match the type or size of the operation's argument (for example, if an integer is passed when a pointer is expected, or the information returned in the buffer by the operating system is larger than the size of *arg*), this is most likely to result in a segmentation violation or a more subtle data corruption.

<div class="audit-event">

fcntl.fcntl fd,cmd,arg fcntl.fcntl

</div>

<div class="versionchanged">

3.14 Add support of arbitrary `bytes-like objects <bytes-like object>`, not only `bytes`.

</div>

<div class="versionchanged">

3.15 The size of bytes-like objects is no longer limited to 1024 bytes.

</div>

</div>

<div class="function">

ioctl(fd, request, arg=0, mutate_flag=True, /)

This function is identical to the `~fcntl.fcntl` function, except that the argument handling is even more complicated.

The *request* parameter is limited to values that can fit in 32-bits or 64-bits, depending on the platform. Additional constants of interest for use as the *request* argument can be found in the `termios` module, under the same names as used in the relevant C header files.

The parameter *arg* can be an integer, a `bytes-like object`, or a string. The type and size of *arg* must match the type and size of the argument of the operation as specified in the relevant C documentation.

If *arg* does not support the read-write buffer interface or the *mutate_flag* is false, behavior is as for the `~fcntl.fcntl` function.

If *arg* supports the read-write buffer interface (like `bytearray`) and *mutate_flag* is true (the default), then the buffer is (in effect) passed to the underlying `!ioctl` system call, the latter's return code is passed back to the calling Python, and the buffer's new contents reflect the action of the `ioctl`. This is a slight simplification, because if the supplied buffer is less than 1024 bytes long it is first copied into a static buffer 1024 bytes long which is then passed to `ioctl` and copied back into the supplied buffer.

If the `ioctl` call fails, an `OSError` exception is raised.

> [!NOTE]
> If the type or size of *arg* does not match the type or size of the operation's argument (for example, if an integer is passed when a pointer is expected, or the information returned in the buffer by the operating system is larger than the size of *arg*), this is most likely to result in a segmentation violation or a more subtle data corruption.

An example:

    >>> import array, fcntl, struct, termios, os
    >>> os.getpgrp()
    13341
    >>> struct.unpack('h', fcntl.ioctl(0, termios.TIOCGPGRP, "  "))[0]
    13341
    >>> buf = array.array('h', [0])
    >>> fcntl.ioctl(0, termios.TIOCGPGRP, buf, 1)
    0
    >>> buf
    array('h', [13341])

<div class="audit-event">

fcntl.ioctl fd,request,arg fcntl.ioctl

</div>

<div class="versionchanged">

3.14 The GIL is always released during a system call. System calls failing with EINTR are automatically retried.

</div>

<div class="versionchanged">

3.15 The size of not mutated bytes-like objects is no longer limited to 1024 bytes.

</div>

</div>

<div class="function">

flock(fd, operation, /)

Perform the lock operation *operation* on file descriptor *fd* (file objects providing a `~io.IOBase.fileno` method are accepted as well). See the Unix manual `flock(2)` for details. (On some systems, this function is emulated using `fcntl`.)

If the `flock` call fails, an `OSError` exception is raised.

<div class="audit-event">

fcntl.flock fd,operation fcntl.flock

</div>

</div>

<div class="function">

lockf(fd, cmd, len=0, start=0, whence=0, /)

This is essentially a wrapper around the `~fcntl.fcntl` locking calls. *fd* is the file descriptor (file objects providing a `~io.IOBase.fileno` method are accepted as well) of the file to lock or unlock, and *cmd* is one of the following values:

<div class="data">

LOCK_UN

Release an existing lock.

</div>

<div class="data">

LOCK_SH

Acquire a shared lock.

</div>

<div class="data">

LOCK_EX

Acquire an exclusive lock.

</div>

<div class="data">

LOCK_NB

Bitwise OR with any of the other three `LOCK_*` constants to make the request non-blocking.

</div>

If `!LOCK_NB` is used and the lock cannot be acquired, an `OSError` will be raised and the exception will have an *errno* attribute set to `~errno.EACCES` or `~errno.EAGAIN` (depending on the operating system; for portability, check for both values). On at least some systems, `!LOCK_EX` can only be used if the file descriptor refers to a file opened for writing.

*len* is the number of bytes to lock, *start* is the byte offset at which the lock starts, relative to *whence*, and *whence* is as with `io.IOBase.seek`, specifically:

- `0` -- relative to the start of the file (`os.SEEK_SET`)
- `1` -- relative to the current buffer position (`os.SEEK_CUR`)
- `2` -- relative to the end of the file (`os.SEEK_END`)

The default for *start* is 0, which means to start at the beginning of the file. The default for *len* is 0 which means to lock to the end of the file. The default for *whence* is also 0.

<div class="audit-event">

fcntl.lockf fd,cmd,len,start,whence fcntl.lockf

</div>

</div>

Examples (all on a SVR4 compliant system):

    import struct, fcntl, os

    f = open(...)
    rv = fcntl.fcntl(f, fcntl.F_SETFL, os.O_NDELAY)

    lockdata = struct.pack('hhllhh', fcntl.F_WRLCK, 0, 0, 0, 0, 0)
    rv = fcntl.fcntl(f, fcntl.F_SETLKW, lockdata)

Note that in the first example the return value variable *rv* will hold an integer value; in the second example it will hold a `bytes` object. The structure lay-out for the *lockdata* variable is system dependent --- therefore using the `flock` call may be better.

<div class="seealso">

Module `os`  
If the locking flags `~os.O_SHLOCK` and `~os.O_EXLOCK` are present in the `os` module (on BSD only), the `os.open` function provides an alternative to the `lockf` and `flock` functions.

</div>

# `!filecmp` --- File and Directory Comparisons

<div class="module" data-synopsis="Compare files efficiently.">

filecmp

</div>

**Source code:** `Lib/filecmp.py`

------------------------------------------------------------------------

The `!filecmp` module defines functions to compare files and directories, with various optional time/correctness trade-offs. For comparing files, see also the `difflib` module.

The `!filecmp` module defines the following functions:

<div class="function">

cmp(f1, f2, shallow=True)

Compare the files named *f1* and *f2*, returning `True` if they seem equal, `False` otherwise.

If *shallow* is true and the `os.stat` signatures (file type, size, and modification time) of both files are identical, the files are taken to be equal.

Otherwise, the files are treated as different if their sizes or contents differ.

Note that no external programs are called from this function, giving it portability and efficiency.

This function uses a cache for past comparisons and the results, with cache entries invalidated if the `os.stat` information for the file changes. The entire cache may be cleared using `clear_cache`.

</div>

<div class="function">

cmpfiles(a, b, common, shallow=True)

Compare the files in the two directories *a* and *b* whose names are given by *common*.

Returns three lists of file names: *match*, *mismatch*, *errors*. *match* contains the list of files that match, *mismatch* contains the names of those that don't, and *errors* lists the names of files which could not be compared. Files are listed in *errors* if they don't exist in one of the directories, the user lacks permission to read them or if the comparison could not be done for some other reason.

The *shallow* parameter has the same meaning and default value as for `filecmp.cmp`.

For example, `cmpfiles('a', 'b', ['c', 'd/e'])` will compare `a/c` with `b/c` and `a/d/e` with `b/d/e`. `'c'` and `'d/e'` will each be in one of the three returned lists.

</div>

<div class="function">

clear_cache()

Clear the filecmp cache. This may be useful if a file is compared so quickly after it is modified that it is within the mtime resolution of the underlying filesystem.

<div class="versionadded">

3.4

</div>

</div>

## The `dircmp` class

<div class="dircmp(a, b, ignore=None, hide=None, *, shallow=True)">

Construct a new directory comparison object, to compare the directories *a* and *b*. *ignore* is a list of names to ignore, and defaults to `filecmp.DEFAULT_IGNORES`. *hide* is a list of names to hide, and defaults to `[os.curdir, os.pardir]`.

The `dircmp` class compares files by doing *shallow* comparisons as described for `filecmp.cmp` by default using the *shallow* parameter.

<div class="versionchanged">

3.13

Added the *shallow* parameter.

</div>

The `dircmp` class provides the following methods:

<div class="method">

report()

Print (to `sys.stdout`) a comparison between *a* and *b*.

</div>

<div class="method">

report_partial_closure()

Print a comparison between *a* and *b* and common immediate subdirectories.

</div>

<div class="method">

report_full_closure()

Print a comparison between *a* and *b* and common subdirectories (recursively).

</div>

The `dircmp` class offers a number of interesting attributes that may be used to get various bits of information about the directory trees being compared.

Note that via `~object.__getattr__` hooks, all attributes are computed lazily, so there is no speed penalty if only those attributes which are lightweight to compute are used.

<div class="attribute">

left

The directory *a*.

</div>

<div class="attribute">

right

The directory *b*.

</div>

<div class="attribute">

left_list

Files and subdirectories in *a*, filtered by *hide* and *ignore*.

</div>

<div class="attribute">

right_list

Files and subdirectories in *b*, filtered by *hide* and *ignore*.

</div>

<div class="attribute">

common

Files and subdirectories in both *a* and *b*.

</div>

<div class="attribute">

left_only

Files and subdirectories only in *a*.

</div>

<div class="attribute">

right_only

Files and subdirectories only in *b*.

</div>

<div class="attribute">

common_dirs

Subdirectories in both *a* and *b*.

</div>

<div class="attribute">

common_files

Files in both *a* and *b*.

</div>

<div class="attribute">

common_funny

Names in both *a* and *b*, such that the type differs between the directories, or names for which `os.stat` reports an error.

</div>

<div class="attribute">

same_files

Files which are identical in both *a* and *b*, using the class's file comparison operator.

</div>

<div class="attribute">

diff_files

Files which are in both *a* and *b*, whose contents differ according to the class's file comparison operator.

</div>

<div class="attribute">

funny_files

Files which are in both *a* and *b*, but could not be compared.

</div>

<div class="attribute">

subdirs

A dictionary mapping names in `common_dirs` to `dircmp` instances (or MyDirCmp instances if this instance is of type MyDirCmp, a subclass of `dircmp`).

<div class="versionchanged">

3.10 Previously entries were always `dircmp` instances. Now entries are the same type as *self*, if *self* is a subclass of `dircmp`.

</div>

</div>

</div>

<div class="data">

DEFAULT_IGNORES

<div class="versionadded">

3.4

</div>

List of directories ignored by `dircmp` by default.

</div>

Here is a simplified example of using the `subdirs` attribute to search recursively through two directories to show common different files:

    >>> from filecmp import dircmp
    >>> def print_diff_files(dcmp):
    ...     for name in dcmp.diff_files:
    ...         print("diff_file %s found in %s and %s" % (name, dcmp.left,
    ...               dcmp.right))
    ...     for sub_dcmp in dcmp.subdirs.values():
    ...         print_diff_files(sub_dcmp)
    ...
    >>> dcmp = dircmp('dir1', 'dir2') # doctest: +SKIP
    >>> print_diff_files(dcmp) # doctest: +SKIP

# File Formats

The modules described in this chapter parse various miscellaneous file formats that aren't markup languages and are not related to e-mail.

<div class="toctree">

csv.rst configparser.rst tomllib.rst netrc.rst plistlib.rst

</div>

# `!fileinput` --- Iterate over lines from multiple input streams

<div class="module" data-synopsis="Loop over standard input or a list of files.">

fileinput

</div>

**Source code:** `Lib/fileinput.py`

------------------------------------------------------------------------

This module implements a helper class and functions to quickly write a loop over standard input or a list of files. If you just want to read or write one file see `open`.

The typical use is:

    import fileinput
    for line in fileinput.input(encoding="utf-8"):
        process(line)

This iterates over the lines of all files listed in `sys.argv[1:]`, defaulting to `sys.stdin` if the list is empty. If a filename is `'-'`, it is also replaced by `sys.stdin` and the optional arguments *mode* and *openhook* are ignored. To specify an alternative list of filenames, pass it as the first argument to `.input`. A single file name is also allowed.

All files are opened in text mode by default, but you can override this by specifying the *mode* parameter in the call to `.input` or `FileInput`. If an I/O error occurs during opening or reading a file, `OSError` is raised.

<div class="versionchanged">

3.3 `IOError` used to be raised; it is now an alias of `OSError`.

</div>

If `sys.stdin` is used more than once, the second and further use will return no lines, except perhaps for interactive use, or if it has been explicitly reset (e.g. using `sys.stdin.seek(0)`).

Empty files are opened and immediately closed; the only time their presence in the list of filenames is noticeable at all is when the last file opened is empty.

Lines are returned with any newlines intact, which means that the last line in a file may not have one.

You can control how files are opened by providing an opening hook via the *openhook* parameter to `fileinput.input` or `FileInput`. The hook must be a function that takes two arguments, *filename* and *mode*, and returns an accordingly opened file-like object. If *encoding* and/or *errors* are specified, they will be passed to the hook as additional keyword arguments. This module provides a `hook_compressed` to support compressed files.

The following function is the primary interface of this module:

<div class="function">

input(files=None, inplace=False, backup='', \*, mode='r', openhook=None, encoding=None, errors=None)

Create an instance of the `FileInput` class. The instance will be used as global state for the functions of this module, and is also returned to use during iteration. The parameters to this function will be passed along to the constructor of the `FileInput` class.

The `FileInput` instance can be used as a context manager in the `with` statement. In this example, *input* is closed after the `!with` statement is exited, even if an exception occurs:

    with fileinput.input(files=('spam.txt', 'eggs.txt'), encoding="utf-8") as f:
        for line in f:
            process(line)

<div class="versionchanged">

3.2 Can be used as a context manager.

</div>

<div class="versionchanged">

3.8 The keyword parameters *mode* and *openhook* are now keyword-only.

</div>

<div class="versionchanged">

3.10 The keyword-only parameter *encoding* and *errors* are added.

</div>

</div>

The following functions use the global state created by `fileinput.input`; if there is no active state, `RuntimeError` is raised.

<div class="function">

filename()

Return the name of the file currently being read. Before the first line has been read, returns `None`.

</div>

<div class="function">

fileno()

Return the integer "file descriptor" for the current file. When no file is opened (before the first line and between files), returns `-1`.

</div>

<div class="function">

lineno()

Return the cumulative line number of the line that has just been read. Before the first line has been read, returns `0`. After the last line of the last file has been read, returns the line number of that line.

</div>

<div class="function">

filelineno()

Return the line number in the current file. Before the first line has been read, returns `0`. After the last line of the last file has been read, returns the line number of that line within the file.

</div>

<div class="function">

isfirstline()

Return `True` if the line just read is the first line of its file, otherwise return `False`.

</div>

<div class="function">

isstdin()

Return `True` if the last line was read from `sys.stdin`, otherwise return `False`.

</div>

<div class="function">

nextfile()

Close the current file so that the next iteration will read the first line from the next file (if any); lines not read from the file will not count towards the cumulative line count. The filename is not changed until after the first line of the next file has been read. Before the first line has been read, this function has no effect; it cannot be used to skip the first file. After the last line of the last file has been read, this function has no effect.

</div>

<div class="function">

close()

Close the sequence.

</div>

The class which implements the sequence behavior provided by the module is available for subclassing as well:

<div class="FileInput(files=None, inplace=False, backup='', *, mode='r', openhook=None, encoding=None, errors=None)">

Class `FileInput` is the implementation; its methods `filename`, `fileno`, `lineno`, `filelineno`, `isfirstline`, `isstdin`, `nextfile` and `close` correspond to the functions of the same name in the module. In addition it is `iterable` and has a `~io.TextIOBase.readline` method which returns the next input line. The sequence must be accessed in strictly sequential order; random access and `~io.TextIOBase.readline` cannot be mixed.

With *mode* you can specify which file mode will be passed to `open`. It must be one of `'r'` and `'rb'`.

The *openhook*, when given, must be a function that takes two arguments, *filename* and *mode*, and returns an accordingly opened file-like object. You cannot use *inplace* and *openhook* together.

You can specify *encoding* and *errors* that is passed to `open` or *openhook*.

A `FileInput` instance can be used as a context manager in the `with` statement. In this example, *input* is closed after the `!with` statement is exited, even if an exception occurs:

    with FileInput(files=('spam.txt', 'eggs.txt')) as input:
        process(input)

<div class="versionchanged">

3.2 Can be used as a context manager.

</div>

<div class="versionchanged">

3.8 The keyword parameter *mode* and *openhook* are now keyword-only.

</div>

<div class="versionchanged">

3.10 The keyword-only parameter *encoding* and *errors* are added.

</div>

<div class="versionchanged">

3.11 The `'rU'` and `'U'` modes and the `!__getitem__` method have been removed.

</div>

</div>

**Optional in-place filtering:** if the keyword argument `inplace=True` is passed to `fileinput.input` or to the `FileInput` constructor, the file is moved to a backup file and standard output is directed to the input file (if a file of the same name as the backup file already exists, it will be replaced silently). This makes it possible to write a filter that rewrites its input file in place. If the *backup* parameter is given (typically as `backup='.<some extension>'`), it specifies the extension for the backup file, and the backup file remains around; by default, the extension is `'.bak'` and it is deleted when the output file is closed. In-place filtering is disabled when standard input is read.

The output file is created exclusively: if something else creates a file or symbolic link with the name of the input file after it has been moved to the backup file, `FileExistsError` is raised. If the output file cannot be created, the backup file is kept. In-place filtering of files in a directory that other, untrusted users can modify is not supported.

<div class="versionchanged">

next The output file is now created exclusively, and the backup file is no longer deleted if the output file cannot be created.

</div>

The two following opening hooks are provided by this module:

<div class="function">

hook_compressed(filename, mode, \*, encoding=None, errors=None)

Transparently opens files compressed with gzip and bzip2 (recognized by the extensions `'.gz'` and `'.bz2'`) using the `gzip` and `bz2` modules. If the filename extension is not `'.gz'` or `'.bz2'`, the file is opened normally (ie, using `open` without any decompression).

The *encoding* and *errors* values are passed to `io.TextIOWrapper` for compressed files and open for normal files.

Usage example: `fi = fileinput.FileInput(openhook=fileinput.hook_compressed, encoding="utf-8")`

<div class="versionchanged">

3.10 The keyword-only parameter *encoding* and *errors* are added.

</div>

</div>

<div class="function">

hook_encoded(encoding, errors=None)

Returns a hook which opens each file with `open`, using the given *encoding* and *errors* to read the file.

Usage example: `fi = fileinput.FileInput(openhook=fileinput.hook_encoded("utf-8", "surrogateescape"))`

<div class="versionchanged">

3.6 Added the optional *errors* parameter.

</div>

<div class="deprecated">

3.10 This function is deprecated since `fileinput.input` and `FileInput` now have *encoding* and *errors* parameters.

</div>

</div>

# File and Directory Access

The modules described in this chapter deal with disk files and directories. For example, there are modules for reading the properties of files, manipulating paths in a portable way, and creating temporary files. The full list of modules in this chapter is:

<div class="toctree">

pathlib.rst os.path.rst stat.rst filecmp.rst tempfile.rst glob.rst fnmatch.rst linecache.rst shutil.rst

</div>

<div class="seealso">

Module `os`  
Operating system interfaces, including functions to work with files at a lower level than Python `file objects <file object>`.

Module `io`  
Python's built-in I/O library, including both abstract classes and some concrete classes such as file I/O.

Built-in function `open`  
The standard way to open files for reading and writing with Python.

</div>

# `!fnmatch` --- Unix filename pattern matching

<div class="module" data-synopsis="Unix shell style filename pattern matching.">

fnmatch

</div>

**Source code:** `Lib/fnmatch.py`

<div class="index">

single: filenames; wildcard expansion

</div>

<div class="index">

pair: module; re

</div>

------------------------------------------------------------------------

This module provides support for Unix shell-style wildcards, which are *not* the same as regular expressions (which are documented in the `re` module). The special characters used in shell-style wildcards are:

<div class="index">

single: \* (asterisk); in glob-style wildcards single: ? (question mark); in glob-style wildcards single: \[\] (square brackets); in glob-style wildcards single: ! (exclamation); in glob-style wildcards single: - (minus); in glob-style wildcards

</div>

| Pattern  | Meaning                            |
|----------|------------------------------------|
| `*`      | matches everything                 |
| `?`      | matches any single character       |
| `[seq]`  | matches any character in *seq*     |
| `[!seq]` | matches any character not in *seq* |

For a literal match, wrap the meta-characters in brackets. For example, `'[?]'` matches the character `'?'`.

<div class="index">

pair: module; glob

</div>

Note that the filename separator (`'/'` on Unix) is *not* special to this module. See module `glob` for pathname expansion (`glob` uses `.filter` to match pathname segments). Similarly, filenames starting with a period are not special for this module, and are matched by the `*` and `?` patterns.

Unless stated otherwise, "filename string" and "pattern string" either refer to `str` or `ISO-8859-1` encoded `bytes` objects. Note that the functions documented below do not allow to mix a `!bytes` pattern with a `!str` filename, and vice-versa.

Finally, note that `functools.lru_cache` with a *maxsize* of 32768 is used to cache the (typed) compiled regex patterns in the following functions: `~fnmatch.fnmatch`, `fnmatchcase`, `.filter`, `.filterfalse`.

<div class="function">

fnmatch(name, pat)

Test whether the filename string *name* matches the pattern string *pat*, returning `True` or `False`. Both parameters are case-normalized using `os.path.normcase`. `fnmatchcase` can be used to perform a case-sensitive comparison, regardless of whether that's standard for the operating system.

This example will print all file names in the current directory with the extension `.txt`:

    import fnmatch
    import os

    for file in os.listdir('.'):
        if fnmatch.fnmatch(file, '*.txt'):
            print(file)

</div>

<div class="function">

fnmatchcase(name, pat)

Test whether the filename string *name* matches the pattern string *pat*, returning `True` or `False`; the comparison is case-sensitive and does not apply `os.path.normcase`.

</div>

<div class="function">

filter(names, pat)

Construct a list from those elements of the `iterable` of filename strings *names* that match the pattern string *pat*. It is the same as `[n for n in names if fnmatch(n, pat)]`, but implemented more efficiently.

</div>

<div class="function">

filterfalse(names, pat)

Construct a list from those elements of the `iterable` of filename strings *names* that do not match the pattern string *pat*. It is the same as `[n for n in names if not fnmatch(n, pat)]`, but implemented more efficiently.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

translate(pat)

Return the shell-style pattern *pat* converted to a regular expression for using with `re.prefixmatch`. The pattern is expected to be a `str`.

Example:

> \>\>\> import fnmatch, re \>\>\> \>\>\> regex = fnmatch.translate('\*.txt') \>\>\> regex '(?s:.\*\\.txt)\z' \>\>\> reobj = re.compile(regex) \>\>\> reobj.prefixmatch('foobar.txt') \<re.Match object; span=(0, 10), match='foobar.txt'\>

</div>

<div class="seealso">

Module `glob`  
Unix shell-style path expansion.

</div>

# `!fractions` --- Rational numbers

<div class="module" data-synopsis="Rational numbers.">

fractions

</div>

**Source code:** `Lib/fractions.py`

------------------------------------------------------------------------

The `!fractions` module provides support for rational number arithmetic.

A Fraction instance can be constructed from a pair of rational numbers, from a single number, or from a string.

<div class="index">

single: as_integer_ratio()

</div>

<div class="Fraction(numerator=0, denominator=1) Fraction(number) Fraction(string)">

The first version requires that *numerator* and *denominator* are instances of `numbers.Rational` and returns a new `Fraction` instance with a value equal to `numerator/denominator`. If *denominator* is zero, it raises a `ZeroDivisionError`.

The second version requires that *number* is an instance of `numbers.Rational` or has the `!as_integer_ratio` method (this includes `float` and `decimal.Decimal`). It returns a `Fraction` instance with exactly the same value. Assumed, that the `!as_integer_ratio` method returns a pair of coprime integers and last one is positive. Note that due to the usual issues with binary point (see `tut-fp-issues`), the argument to `Fraction(1.1)` is not exactly equal to 11/10, and so `Fraction(1.1)` does *not* return `Fraction(11, 10)` as one might expect. (But see the documentation for the `limit_denominator` method below.)

The last version of the constructor expects a string. The usual form for this instance is:

    [sign] numerator ['/' denominator]

where the optional `sign` may be either '+' or '-' and `numerator` and `denominator` (if present) are strings of decimal digits (underscores may be used to delimit digits as with integral literals in code). In addition, any string that represents a finite value and is accepted by the `float` constructor is also accepted by the `Fraction` constructor. In either form the input string may also have leading and/or trailing whitespace. Here are some examples:

    >>> from fractions import Fraction
    >>> Fraction(16, -10)
    Fraction(-8, 5)
    >>> Fraction(123)
    Fraction(123, 1)
    >>> Fraction()
    Fraction(0, 1)
    >>> Fraction('3/7')
    Fraction(3, 7)
    >>> Fraction(' -3/7 ')
    Fraction(-3, 7)
    >>> Fraction('1.414213 \t\n')
    Fraction(1414213, 1000000)
    >>> Fraction('-.125')
    Fraction(-1, 8)
    >>> Fraction('7e-6')
    Fraction(7, 1000000)
    >>> Fraction(2.25)
    Fraction(9, 4)
    >>> Fraction(1.1)
    Fraction(2476979795053773, 2251799813685248)
    >>> from decimal import Decimal
    >>> Fraction(Decimal('1.1'))
    Fraction(11, 10)

The `Fraction` class inherits from the abstract base class `numbers.Rational`, and implements all of the methods and operations from that class. `Fraction` instances are `hashable`, and should be treated as immutable. In addition, `Fraction` has the following properties and methods:

<div class="versionchanged">

3.2 The `Fraction` constructor now accepts `float` and `decimal.Decimal` instances.

</div>

<div class="versionchanged">

3.9 The `math.gcd` function is now used to normalize the *numerator* and *denominator*. `math.gcd` always returns an `int` type. Previously, the GCD type depended on *numerator* and *denominator*.

</div>

<div class="versionchanged">

3.11 Underscores are now permitted when creating a `Fraction` instance from a string, following [PEP 515](http://www.python.org/dev/peps/pep-0515/ "PEP 515") rules.

</div>

<div class="versionchanged">

3.11 `Fraction` implements `__int__` now to satisfy `typing.SupportsInt` instance checks.

</div>

<div class="versionchanged">

3.12 Space is allowed around the slash for string inputs: `Fraction('2 / 3')`.

</div>

<div class="versionchanged">

3.12 `Fraction` instances now support float-style formatting, with presentation types `"e"`, `"E"`, `"f"`, `"F"`, `"g"`, `"G"` and `"%"`.

</div>

<div class="versionchanged">

3.13 Formatting of `Fraction` instances without a presentation type now supports fill, alignment, sign handling, minimum width and grouping.

</div>

<div class="versionchanged">

3.14 The `Fraction` constructor now accepts any objects with the `!as_integer_ratio` method.

</div>

<div class="attribute">

numerator

Numerator of the Fraction in lowest term.

</div>

<div class="attribute">

denominator

Denominator of the Fraction in lowest terms. Guaranteed to be positive.

</div>

<div class="method">

as_integer_ratio()

Return a tuple of two integers, whose ratio is equal to the original Fraction. The ratio is in lowest terms and has a positive denominator.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

is_integer()

Return `True` if the Fraction is an integer.

<div class="versionadded">

3.12

</div>

</div>

<div class="classmethod">

from_float(f)

Alternative constructor which only accepts instances of `float` or `numbers.Integral`. Beware that `Fraction.from_float(0.3)` is not the same value as `Fraction(3, 10)`.

> [!NOTE]
> From Python 3.2 onwards, you can also construct a `Fraction` instance directly from a `float`.

</div>

<div class="classmethod">

from_decimal(dec)

Alternative constructor which only accepts instances of `decimal.Decimal` or `numbers.Integral`.

> [!NOTE]
> From Python 3.2 onwards, you can also construct a `Fraction` instance directly from a `decimal.Decimal` instance.

</div>

<div class="classmethod">

from_number(number)

Alternative constructor which only accepts instances of `numbers.Integral`, `numbers.Rational`, `float` or `decimal.Decimal`, and objects with the `!as_integer_ratio` method, but not strings.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

limit_denominator(max_denominator=1000000)

Finds and returns the closest `Fraction` to `self` that has denominator at most max_denominator. This method is useful for finding rational approximations to a given floating-point number:

> \>\>\> from fractions import Fraction \>\>\> Fraction('3.1415926535897932').limit_denominator(1000) Fraction(355, 113)

or for recovering a rational number that's represented as a float:

> \>\>\> from math import pi, cos \>\>\> Fraction(cos(pi/3)) Fraction(4503599627370497, 9007199254740992) \>\>\> Fraction(cos(pi/3)).limit_denominator() Fraction(1, 2) \>\>\> Fraction(1.1).limit_denominator() Fraction(11, 10)

</div>

<div class="method">

\_\_floor\_\_()

Returns the greatest `int` `<= self`. This method can also be accessed through the `math.floor` function:

> \>\>\> from math import floor \>\>\> floor(Fraction(355, 113)) 3

</div>

<div class="method">

\_\_ceil\_\_()

Returns the least `int` `>= self`. This method can also be accessed through the `math.ceil` function.

</div>

<div class="method">

\_\_round\_\_() \_\_round\_\_(ndigits)

The first version returns the nearest `int` to `self`, rounding half to even. The second version rounds `self` to the nearest multiple of `Fraction(1, 10**ndigits)` (logically, if `ndigits` is negative), again rounding half toward even. This method can also be accessed through the `round` function.

</div>

<div class="method">

\_\_format\_\_(format_spec, /)

Provides support for formatting of `Fraction` instances via the `str.format` method, the `format` built-in function, or `Formatted string literals <f-strings>`.

If the `format_spec` format specification string does not end with one of the presentation types `'e'`, `'E'`, `'f'`, `'F'`, `'g'`, `'G'` or `'%'` then formatting follows the general rules for fill, alignment, sign handling, minimum width, and grouping as described in the `format specification mini-language <formatspec>`. The "alternate form" flag `'#'` is supported: if present, it forces the output string to always include an explicit denominator, even when the value being formatted is an exact integer. The zero-fill flag `'0'` is not supported.

If the `format_spec` format specification string ends with one of the presentation types `'e'`, `'E'`, `'f'`, `'F'`, `'g'`, `'G'` or `'%'` then formatting follows the rules outlined for the `float` type in the `formatspec` section.

Here are some examples:

    >>> from fractions import Fraction
    >>> format(Fraction(103993, 33102), '_')
    '103_993/33_102'
    >>> format(Fraction(1, 7), '.^+10')
    '...+1/7...'
    >>> format(Fraction(3, 1), '')
    '3'
    >>> format(Fraction(3, 1), '#')
    '3/1'
    >>> format(Fraction(1, 7), '.40g')
    '0.1428571428571428571428571428571428571429'
    >>> format(Fraction('1234567.855'), '_.2f')
    '1_234_567.86'
    >>> f"{Fraction(355, 113):*>20.6e}"
    '********3.141593e+00'
    >>> old_price, new_price = 499, 672
    >>> "{:.2%} price increase".format(Fraction(new_price, old_price) - 1)
    '34.67% price increase'

</div>

</div>

<div class="seealso">

Module `numbers`  
The abstract base classes making up the numeric tower.

</div>

# frameworks

orphan  

# Program frameworks

This chapter is no longer maintained, and the modules it contained have been moved to their respective topical documentation.

- `cmd` — `Command Line Interface Libraries <./cmdlinelibs>`
- `shlex` — `Unix Specific Services <./unix>`
- `turtle` — `Graphical User Interfaces with Tk <./tk>`

# `!ftplib` --- FTP protocol client

<div class="module" data-synopsis="FTP protocol client (requires sockets).">

ftplib

</div>

**Source code:** `Lib/ftplib.py`

<div class="index">

pair: FTP; protocol single: FTP; ftplib (standard module)

</div>

------------------------------------------------------------------------

This module defines the class `FTP` and a few related items. The `FTP` class implements the client side of the FTP protocol. You can use this to write Python programs that perform a variety of automated FTP jobs, such as mirroring other FTP servers. It is also used by the module `urllib.request` to handle URLs that use FTP. For more information on FTP (File Transfer Protocol), see internet `959`.

The default encoding is UTF-8, following `2640`.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

Here's a sample session using the `!ftplib` module:

    >>> from ftplib import FTP
    >>> ftp = FTP('ftp.us.debian.org')  # connect to host, default port
    >>> ftp.login()                     # user anonymous, passwd anonymous@
    '230 Login successful.'
    >>> ftp.cwd('debian')               # change into "debian" directory
    '250 Directory successfully changed.'
    >>> ftp.retrlines('LIST')           # list directory contents
    -rw-rw-r--    1 1176     1176         1063 Jun 15 10:18 README
    ...
    drwxr-sr-x    5 1176     1176         4096 Dec 19  2000 pool
    drwxr-sr-x    4 1176     1176         4096 Nov 17  2008 project
    drwxr-xr-x    3 1176     1176         4096 Oct 10  2012 tools
    '226 Directory send OK.'
    >>> with open('README', 'wb') as fp:
    ...     ftp.retrbinary('RETR README', fp.write)
    '226 Transfer complete.'
    >>> ftp.quit()
    '221 Goodbye.'

## Reference

### FTP objects

<div class="FTP(host='', user='', passwd='', acct='', timeout=None, \ source_address=None, *, encoding='utf-8')">

Return a new instance of the `FTP` class.

param str host  
The hostname to connect to. If given, `connect(host)` is implicitly called by the constructor.

param str user  
The username to log in with (default: `'anonymous'`). If given, `login(host, passwd, acct)` is implicitly called by the constructor.

param str passwd  
The password to use when logging in. If not given, and if *passwd* is the empty string or `"-"`, a password will be automatically generated.

param str acct  
Account information to be used for the `ACCT` FTP command. Few systems implement this. See [RFC-959](https://datatracker.ietf.org/doc/html/rfc959.html) for more details.

param timeout  
A timeout in seconds for blocking operations like `connect` (default: the global default timeout setting).

type timeout  
float \| None

param source_address  
A 2-tuple `(host, port)` for the socket to bind to as its source address before connecting.

type source_address  
tuple \| None

param str encoding  
The encoding for directories and filenames (default: `'utf-8'`).

The `FTP` class supports the `with` statement, e.g.:

> \>\>\> from ftplib import FTP \>\>\> with FTP("ftp1.at.proftpd.org") as ftp: ... ftp.login() ... ftp.dir() ... \# doctest: +SKIP '230 Anonymous login ok, restrictions apply.' dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 . dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 .. dr-xr-xr-x 5 ftp ftp 4096 May 6 10:43 CentOS dr-xr-xr-x 3 ftp ftp 18 Jul 10 2008 Fedora \>\>\>

<div class="versionchanged">

3.2 Support for the `with` statement was added.

</div>

<div class="versionchanged">

3.3 *source_address* parameter was added.

</div>

<div class="versionchanged">

3.9 If the *timeout* parameter is set to be zero, it will raise a `ValueError` to prevent the creation of a non-blocking socket. The *encoding* parameter was added, and the default was changed from Latin-1 to UTF-8 to follow `2640`.

</div>

Several `!FTP` methods are available in two flavors: one for handling text files and another for binary files. The methods are named for the command which is used followed by `lines` for the text version or `binary` for the binary version.

`FTP` instances have the following methods:

<div class="method">

FTP.set_debuglevel(level)

Set the instance's debugging level as an `int`. This controls the amount of debugging output printed. The debug levels are:

- `0` (default): No debug output.
- `1`: Produce a moderate amount of debug output, generally a single line per request.
- `2` or higher: Produce the maximum amount of debugging output, logging each line sent and received on the control connection.

</div>

<div class="method">

FTP.connect(host='', port=0, timeout=None, source_address=None)

Connect to the given host and port. This function should be called only once for each instance; it should not be called if a *host* argument was given when the `FTP` instance was created. All other `!FTP` methods can only be called after a connection has successfully been made.

param str host  
The host to connect to.

param int port  
The TCP port to connect to (default: `21`, as specified by the FTP protocol specification). It is rarely needed to specify a different port number.

param timeout  
A timeout in seconds for the connection attempt (default: the global default timeout setting).

type timeout  
float \| None

param source_address  
A 2-tuple `(host, port)` for the socket to bind to as its source address before connecting.

type source_address  
tuple \| None

<div class="audit-event">

ftplib.connect self,host,port ftplib.FTP.connect

</div>

<div class="versionchanged">

3.3 *source_address* parameter was added.

</div>

</div>

<div class="method">

FTP.getwelcome()

Return the welcome message sent by the server in reply to the initial connection. (This message sometimes contains disclaimers or help information that may be relevant to the user.)

</div>

<div class="method">

FTP.login(user='anonymous', passwd='', acct='')

Log on to the connected FTP server. This function should be called only once for each instance, after a connection has been established; it should not be called if the *host* and *user* arguments were given when the `FTP` instance was created. Most FTP commands are only allowed after the client has logged in.

param str user  
The username to log in with (default: `'anonymous'`).

param str passwd  
The password to use when logging in. If not given, and if *passwd* is the empty string or `"-"`, a password will be automatically generated.

param str acct  
Account information to be used for the `ACCT` FTP command. Few systems implement this. See [RFC-959](https://datatracker.ietf.org/doc/html/rfc959.html) for more details.

</div>

<div class="method">

FTP.abort()

Abort a file transfer that is in progress. Using this does not always work, but it's worth a try.

</div>

<div class="method">

FTP.sendcmd(cmd)

Send a simple command string to the server and return the response string.

<div class="audit-event">

ftplib.sendcmd self,cmd ftplib.FTP.sendcmd

</div>

</div>

<div class="method">

FTP.voidcmd(cmd)

Send a simple command string to the server and handle the response. Return the response string if the response code corresponds to success (codes in the range 200--299). Raise `error_reply` otherwise.

<div class="audit-event">

ftplib.sendcmd self,cmd ftplib.FTP.voidcmd

</div>

</div>

<div class="method">

FTP.retrbinary(cmd, callback, blocksize=8192, rest=None)

Retrieve a file in binary transfer mode.

param str cmd  
An appropriate `RETR` command: `"RETR {filename}"`.

param callback  
A single parameter callable that is called for each block of data received, with its single argument being the data as `bytes`.

type callback  
`callable`

param int blocksize  
The maximum chunk size to read on the low-level `~socket.socket` object created to do the actual transfer. This also corresponds to the largest size of data that will be passed to *callback*. Defaults to `8192`.

param int rest  
A `REST` command to be sent to the server. See the documentation for the *rest* parameter of the `transfercmd` method.

</div>

<div class="method">

FTP.retrlines(cmd, callback=None)

Retrieve a file or directory listing in the encoding specified by the *encoding* parameter at initialization. *cmd* should be an appropriate `RETR` command (see `retrbinary`) or a command such as `LIST` or `NLST` (usually just the string `'LIST'`). `LIST` retrieves a list of files and information about those files. `NLST` retrieves a list of file names. The *callback* function is called for each line with a string argument containing the line with the trailing CRLF stripped. The default *callback* prints the line to `sys.stdout`.

</div>

<div class="method">

FTP.set_pasv(val)

Enable "passive" mode if *val* is true, otherwise disable passive mode. Passive mode is on by default.

</div>

<div class="method">

FTP.storbinary(cmd, fp, blocksize=8192, callback=None, rest=None)

Store a file in binary transfer mode.

param str cmd  
An appropriate `STOR` command: `"STOR {filename}"`.

param fp  
A file object (opened in binary mode) which is read until EOF, using its `~io.RawIOBase.read` method in blocks of size *blocksize* to provide the data to be stored.

type fp  
`file object`

param int blocksize  
The read block size. Defaults to `8192`.

param callback  
A single parameter callable that is called for each block of data sent, with its single argument being the data as `bytes`.

type callback  
`callable`

param int rest  
A `REST` command to be sent to the server. See the documentation for the *rest* parameter of the `transfercmd` method.

<div class="versionchanged">

3.2 The *rest* parameter was added.

</div>

</div>

<div class="method">

FTP.storlines(cmd, fp, callback=None)

Store a file in line mode. *cmd* should be an appropriate `STOR` command (see `storbinary`). Lines are read until EOF from the `file object` *fp* (opened in binary mode) using its `~io.IOBase.readline` method to provide the data to be stored. *callback* is an optional single parameter callable that is called on each line after it is sent.

</div>

<div class="method">

FTP.transfercmd(cmd, rest=None)

Initiate a transfer over the data connection. If the transfer is active, send an `EPRT` or `PORT` command and the transfer command specified by *cmd*, and accept the connection. If the server is passive, send an `EPSV` or `PASV` command, connect to it, and start the transfer command. Either way, return the socket for the connection.

If optional *rest* is given, a `REST` command is sent to the server, passing *rest* as an argument. *rest* is usually a byte offset into the requested file, telling the server to restart sending the file's bytes at the requested offset, skipping over the initial bytes. Note however that the `transfercmd` method converts *rest* to a string with the *encoding* parameter specified at initialization, but no check is performed on the string's contents. If the server does not recognize the `REST` command, an `error_reply` exception will be raised. If this happens, simply call `transfercmd` without a *rest* argument.

</div>

<div class="method">

FTP.ntransfercmd(cmd, rest=None)

Like `transfercmd`, but returns a tuple of the data connection and the expected size of the data. If the expected size could not be computed, `None` will be returned as the expected size. *cmd* and *rest* means the same thing as in `transfercmd`.

</div>

<div class="method">

FTP.mlsd(path="", facts=\[\])

List a directory in a standardized format by using `MLSD` command (`3659`). If *path* is omitted the current directory is assumed. *facts* is a list of strings representing the type of information desired (e.g. `["type", "size", "perm"]`). Return a generator object yielding a tuple of two elements for every file found in path. First element is the file name, the second one is a dictionary containing facts about the file name. Content of this dictionary might be limited by the *facts* argument but server is not guaranteed to return all requested facts.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

FTP.nlst(argument\[, ...\])

Return a list of file names as returned by the `NLST` command. The optional *argument* is a directory to list (default is the current server directory). Multiple arguments can be used to pass non-standard options to the `NLST` command.

> [!NOTE]
> If your server supports the command, `mlsd` offers a better API.

</div>

<div class="method">

FTP.dir(argument\[, ...\])

Produce a directory listing as returned by the `LIST` command, printing it to standard output. The optional *argument* is a directory to list (default is the current server directory). Multiple arguments can be used to pass non-standard options to the `LIST` command. If the last argument is a function, it is used as a *callback* function as for `retrlines`; the default prints to `sys.stdout`. This method returns `None`.

> [!NOTE]
> If your server supports the command, `mlsd` offers a better API.

</div>

<div class="method">

FTP.rename(fromname, toname)

Rename file *fromname* on the server to *toname*.

</div>

<div class="method">

FTP.delete(filename)

Remove the file named *filename* from the server. If successful, returns the text of the response, otherwise raises `error_perm` on permission errors or `error_reply` on other errors.

</div>

<div class="method">

FTP.cwd(pathname)

Set the current directory on the server.

</div>

<div class="method">

FTP.mkd(pathname)

Create a new directory on the server.

</div>

<div class="method">

FTP.pwd()

Return the pathname of the current directory on the server.

</div>

<div class="method">

FTP.rmd(dirname)

Remove the directory named *dirname* on the server.

</div>

<div class="method">

FTP.size(filename)

Request the size of the file named *filename* on the server. On success, the size of the file is returned as an integer, otherwise `None` is returned. Note that the `SIZE` command is not standardized, but is supported by many common server implementations.

</div>

<div class="method">

FTP.quit()

Send a `QUIT` command to the server and close the connection. This is the "polite" way to close a connection, but it may raise an exception if the server responds with an error to the `QUIT` command. This implies a call to the `close` method which renders the `FTP` instance useless for subsequent calls (see below).

</div>

<div class="method">

FTP.close()

Close the connection unilaterally. This should not be applied to an already closed connection such as after a successful call to `~FTP.quit`. After this call the `FTP` instance should not be used any more (after a call to `close` or `~FTP.quit` you cannot reopen the connection by issuing another `login` method).

</div>

</div>

### FTP_TLS objects

<div class="FTP_TLS(host='', user='', passwd='', acct='', *, context=None, \ timeout=None, source_address=None, encoding='utf-8')">

An `FTP` subclass which adds TLS support to FTP as described in `4217`. Connect to port 21 implicitly securing the FTP control connection before authenticating.

> [!NOTE]
> The user must explicitly secure the data connection by calling the `prot_p` method.

param str host  
The hostname to connect to. If given, `connect(host)` is implicitly called by the constructor.

param str user  
The username to log in with (default: `'anonymous'`). If given, `login(host, passwd, acct)` is implicitly called by the constructor.

param str passwd  
The password to use when logging in. If not given, and if *passwd* is the empty string or `"-"`, a password will be automatically generated.

param str acct  
Account information to be used for the `ACCT` FTP command. Few systems implement this. See [RFC-959](https://datatracker.ietf.org/doc/html/rfc959.html) for more details.

param context  
An SSL context object which allows bundling SSL configuration options, certificates and private keys into a single, potentially long-lived, structure. Please read `ssl-security` for best practices.

type context  
`ssl.SSLContext`

param timeout  
A timeout in seconds for blocking operations like `~FTP.connect` (default: the global default timeout setting).

type timeout  
float \| None

param source_address  
A 2-tuple `(host, port)` for the socket to bind to as its source address before connecting.

type source_address  
tuple \| None

param str encoding  
The encoding for directories and filenames (default: `'utf-8'`).

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Added the *source_address* parameter.

</div>

<div class="versionchanged">

3.4 The class now supports hostname check with `ssl.SSLContext.check_hostname` and *Server Name Indication* (see `ssl.HAS_SNI`).

</div>

<div class="versionchanged">

3.9 If the *timeout* parameter is set to be zero, it will raise a `ValueError` to prevent the creation of a non-blocking socket. The *encoding* parameter was added, and the default was changed from Latin-1 to UTF-8 to follow `2640`.

</div>

<div class="versionchanged">

3.12 The deprecated *keyfile* and *certfile* parameters have been removed.

</div>

Here's a sample session using the `FTP_TLS` class:

    >>> ftps = FTP_TLS('ftp.pureftpd.org')
    >>> ftps.login()
    '230 Anonymous user logged in'
    >>> ftps.prot_p()
    '200 Data protection level set to "private"'
    >>> ftps.nlst()
    ['6jack', 'OpenBSD', 'antilink', 'blogbench', 'bsdcam', 'clockspeed', 'djbdns-jedi', 'docs', 'eaccelerator-jedi', 'favicon.ico', 'francotone', 'fugu', 'ignore', 'libpuzzle', 'metalog', 'minidentd', 'misc', 'mysql-udf-global-user-variables', 'php-jenkins-hash', 'php-skein-hash', 'php-webdav', 'phpaudit', 'phpbench', 'pincaster', 'ping', 'posto', 'pub', 'public', 'public_keys', 'pure-ftpd', 'qscan', 'qtc', 'sharedance', 'skycache', 'sound', 'tmp', 'ucarp']

`!FTP_TLS` class inherits from `FTP`, defining these additional methods and attributes:

<div class="method">

FTP_TLS.auth()

Set up a secure control connection by using TLS.

<div class="versionchanged">

3.4 The method now supports hostname check with `ssl.SSLContext.check_hostname` and *Server Name Indication* (see `ssl.HAS_SNI`).

</div>

</div>

<div class="method">

FTP_TLS.ccc()

Revert control channel back to plaintext. This can be useful to take advantage of firewalls that know how to handle NAT with non-secure FTP without opening fixed ports.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

FTP_TLS.prot_p()

Set up secure data connection by using TLS.

</div>

<div class="method">

FTP_TLS.prot_c()

Set up clear text data connection.

</div>

</div>

### Module variables

<div class="exception">

error_reply

Exception raised when an unexpected reply is received from the server.

</div>

<div class="exception">

error_temp

Exception raised when an error code signifying a temporary error (response codes in the range 400--499) is received.

</div>

<div class="exception">

error_perm

Exception raised when an error code signifying a permanent error (response codes in the range 500--599) is received.

</div>

<div class="exception">

error_proto

Exception raised when a reply is received from the server that does not fit the response specifications of the File Transfer Protocol, i.e. begin with a digit in the range 1--5.

</div>

<div class="data">

all_errors

The set of all exceptions (as a tuple) that methods of `FTP` instances may raise as a result of problems with the FTP connection (as opposed to programming errors made by the caller). This set includes the four exceptions listed above as well as `OSError` and `EOFError`.

</div>

<div class="seealso">

Module `netrc`  
Parser for the `.netrc` file format. The file `.netrc` is typically used by FTP clients to load user authentication information before prompting the user.

</div>

# Functional Programming Modules

The modules described in this chapter provide functions and classes that support a functional programming style, and general operations on callables.

The following modules are documented in this chapter:

<div class="toctree">

itertools.rst functools.rst operator.rst

</div>

# `!functools` --- Higher-order functions and operations on callable objects

<div class="module" data-synopsis="Higher-order functions and operations on callable objects.">

functools

</div>

**Source code:** `Lib/functools.py`

<div class="testsetup">

default

import functools from functools import \*

</div>

------------------------------------------------------------------------

The `!functools` module is for higher-order functions: functions that act on or return other functions. In general, any callable object can be treated as a function for the purposes of this module.

The `!functools` module defines the following functions:

<div class="decorator">

cache(user_function)

Simple lightweight unbounded function cache. Sometimes called ["memoize"](https://en.wikipedia.org/wiki/Memoization).

Returns the same as `lru_cache(maxsize=None)`, creating a thin wrapper around a dictionary lookup for the function arguments. Because it never needs to evict old values, this is smaller and faster than `lru_cache` with a size limit.

For example:

    @cache
    def factorial(n):
        return n * factorial(n-1) if n else 1

    >>> factorial(10)   # no previously cached result, makes 11 recursive calls
    3628800
    >>> factorial(5)    # no new calls, just returns the cached result
    120
    >>> factorial(12)   # two new recursive calls, factorial(10) is cached
    479001600

The cache is threadsafe so that the wrapped function can be used in multiple threads. This means that the underlying data structure will remain coherent during concurrent updates.

It is possible for the wrapped function to be called more than once if another thread makes an additional call before the initial call has been completed and cached.

Call-once behavior is not guaranteed because locks are not held during the function call. Potentially another call with the same arguments could occur while the first call is still running.

<div class="versionadded">

3.9

</div>

</div>

<div class="decorator">

cached_property(func)

Transform a method of a class into a property whose value is computed once and then cached as a normal attribute for the life of the instance. Similar to `property`, with the addition of caching. Useful for expensive computed properties of instances that are otherwise effectively immutable.

Example:

    class DataSet:

        def __init__(self, sequence_of_numbers):
            self._data = tuple(sequence_of_numbers)

        @cached_property
        def stdev(self):
            return statistics.stdev(self._data)

The mechanics of `cached_property` are somewhat different from `property`. A regular property blocks attribute writes unless a setter is defined. In contrast, a *cached_property* allows writes.

The *cached_property* decorator only runs on lookups and only when an attribute of the same name doesn't exist. When it does run, the *cached_property* writes to the attribute with the same name. Subsequent attribute reads and writes take precedence over the *cached_property* method and it works like a normal attribute.

The cached value can be cleared by deleting the attribute. This allows the *cached_property* method to run again.

The *cached_property* does not prevent a possible race condition in multi-threaded usage. The getter function could run more than once on the same instance, with the latest run setting the cached value. If the cached property is idempotent or otherwise not harmful to run more than once on an instance, this is fine. If synchronization is needed, implement the necessary locking inside the decorated getter function or around the cached property access.

Note, this decorator interferes with the operation of `412` key-sharing dictionaries. This means that instance dictionaries can take more space than usual.

Also, this decorator requires that the `__dict__` attribute on each instance be a mutable mapping. This means it will not work with some types, such as metaclasses (since the `__dict__` attributes on type instances are read-only proxies for the class namespace), and those that specify `__slots__` without including `__dict__` as one of the defined slots (as such classes don't provide a `__dict__` attribute at all).

If a mutable mapping is not available or if space-efficient key sharing is desired, an effect similar to `cached_property` can also be achieved by stacking `property` on top of `lru_cache`. See `faq-cache-method-calls` for more details on how this differs from `cached_property`.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.12 Prior to Python 3.12, `!cached_property` included an undocumented lock to ensure that in multi-threaded usage the getter function was guaranteed to run only once per instance. However, the lock was per-property, not per-instance, which could result in unacceptably high lock contention. In Python 3.12+ this locking is removed.

</div>

</div>

<div class="function">

cmp_to_key(func)

Transform an old-style comparison function to a `key function`. Used with tools that accept key functions (such as `sorted`, `min`, `max`, `heapq.nlargest`, `heapq.nsmallest`, `itertools.groupby`). This function is primarily used as a transition tool for programs being converted from Python 2 which supported the use of comparison functions.

A comparison function is any callable that accepts two arguments, compares them, and returns a negative number for less-than, zero for equality, or a positive number for greater-than. A key function is a callable that accepts one argument and returns another value to be used as the sort key.

Example:

    sorted(iterable, key=cmp_to_key(locale.strcoll))  # locale-aware sort order

For sorting examples and a brief sorting tutorial, see `sortinghowto`.

<div class="versionadded">

3.2

</div>

</div>

<div class="decorator">

lru_cache(user_function) lru_cache(maxsize=128, typed=False)

Decorator to wrap a function with a memoizing callable that saves up to the *maxsize* most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.

The cache is threadsafe so that the wrapped function can be used in multiple threads. This means that the underlying data structure will remain coherent during concurrent updates.

It is possible for the wrapped function to be called more than once if another thread makes an additional call before the initial call has been completed and cached.

Since a dictionary is used to cache results, the positional and keyword arguments to the function must be `hashable`.

Distinct argument patterns may be considered to be distinct calls with separate cache entries. For example, `f(a=1, b=2)` and `f(b=2, a=1)` differ in their keyword argument order and may have two separate cache entries.

If *user_function* is specified, it must be a callable. This allows the *lru_cache* decorator to be applied directly to a user function, leaving the *maxsize* at its default value of 128:

    @lru_cache
    def count_vowels(word):
        return sum(word.count(vowel) for vowel in 'AEIOUaeiou')

If *maxsize* is set to `None`, the LRU feature is disabled and the cache can grow without bound.

If *typed* is set to true, function arguments of different types will be cached separately. If *typed* is false, the implementation will usually regard them as equivalent calls and only cache a single result. (Some types such as *str* and *int* may be cached separately even when *typed* is false.)

Note, type specificity applies only to the function's immediate arguments rather than their contents. The scalar arguments, `Decimal(42)` and `Fraction(42)` are treated as distinct calls with distinct results. In contrast, the tuple arguments `('answer', Decimal(42))` and `('answer', Fraction(42))` are treated as equivalent.

The wrapped function is instrumented with a `!cache_parameters` function that returns a new `dict` showing the values for *maxsize* and *typed*. This is for information purposes only. Mutating the values has no effect.

<div class="method" data-no-typesetting="">

lru_cache.cache_info()

</div>

To help measure the effectiveness of the cache and tune the *maxsize* parameter, the wrapped function is instrumented with a `!cache_info` function that returns a `named tuple` showing *hits*, *misses*, *maxsize* and *currsize*.

<div class="method" data-no-typesetting="">

lru_cache.cache_clear()

</div>

The decorator also provides a `!cache_clear` function for clearing or invalidating the cache.

The original underlying function is accessible through the `__wrapped__` attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

The cache keeps references to the arguments and return values until they age out of the cache or until the cache is cleared.

If a method is cached, the `self` instance argument is included in the cache. See `faq-cache-method-calls`

An [LRU (least recently used) cache](https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_Recently_Used_(LRU)) works best when the most recent calls are the best predictors of upcoming calls (for example, the most popular articles on a news server tend to change each day). The cache's size limit assures that the cache does not grow without bound on long-running processes such as web servers.

In general, the LRU cache should only be used when you want to reuse previously computed values. Accordingly, it doesn't make sense to cache functions with side-effects, functions that need to create distinct mutable objects on each call (such as generators and async functions), or impure functions such as time() or random().

Example of an LRU cache for static web content:

    @lru_cache(maxsize=32)
    def get_pep(num):
        'Retrieve text of a Python Enhancement Proposal'
        resource = f'https://peps.python.org/pep-{num:04d}'
        try:
            with urllib.request.urlopen(resource) as s:
                return s.read()
        except urllib.error.HTTPError:
            return 'Not Found'

    >>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991:
    ...     pep = get_pep(n)
    ...     print(n, len(pep))

    >>> get_pep.cache_info()
    CacheInfo(hits=3, misses=8, maxsize=32, currsize=8)

Example of efficiently computing [Fibonacci numbers](https://en.wikipedia.org/wiki/Fibonacci_number) using a cache to implement a [dynamic programming](https://en.wikipedia.org/wiki/Dynamic_programming) technique:

    @lru_cache(maxsize=None)
    def fib(n):
        if n < 2:
            return n
        return fib(n-1) + fib(n-2)

    >>> [fib(n) for n in range(16)]
    [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610]

    >>> fib.cache_info()
    CacheInfo(hits=28, misses=16, maxsize=None, currsize=16)

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Added the *typed* option.

</div>

<div class="versionchanged">

3.8 Added the *user_function* option.

</div>

<div class="versionchanged">

3.9 Added the function `!cache_parameters`

</div>

</div>

<div class="decorator">

total_ordering

Given a class defining one or more rich comparison ordering methods, this class decorator supplies the rest. This simplifies the effort involved in specifying all of the possible rich comparison operations:

The class must define one of `~object.__lt__`, `~object.__le__`, `~object.__gt__`, or `~object.__ge__`. In addition, the class should supply an `~object.__eq__` method.

For example:

    @total_ordering
    class Student:
        def _is_valid_operand(self, other):
            return (hasattr(other, "lastname") and
                    hasattr(other, "firstname"))
        def __eq__(self, other):
            if not self._is_valid_operand(other):
                return NotImplemented
            return ((self.lastname.lower(), self.firstname.lower()) ==
                    (other.lastname.lower(), other.firstname.lower()))
        def __lt__(self, other):
            if not self._is_valid_operand(other):
                return NotImplemented
            return ((self.lastname.lower(), self.firstname.lower()) <
                    (other.lastname.lower(), other.firstname.lower()))

> [!NOTE]
> While this decorator makes it easy to create well behaved totally ordered types, it *does* come at the cost of slower execution and more complex stack traces for the derived comparison methods. If performance benchmarking indicates this is a bottleneck for a given application, implementing all six rich comparison methods instead is likely to provide an easy speed boost.

> [!NOTE]
> This decorator makes no attempt to override methods that have been declared in the class *or its superclasses*. Meaning that if a superclass defines a comparison operator, *total_ordering* will not implement it again, even if the original method is abstract.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.4 Returning `NotImplemented` from the underlying comparison function for unrecognised types is now supported.

</div>

</div>

<div class="data">

Placeholder

A singleton object used as a sentinel to reserve a place for positional arguments when calling `partial` and `partialmethod`.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

partial(func, /, *args,*\*keywords)

Return a new `partial object<partial-objects>` which when called will behave like *func* called with the positional arguments *args* and keyword arguments *keywords*. If more arguments are supplied to the call, they are appended to *args*. If additional keyword arguments are supplied, they extend and override *keywords*. Roughly equivalent to:

    def partial(func, /, *args, **keywords):
        def newfunc(*more_args, **more_keywords):
            return func(*args, *more_args, **(keywords | more_keywords))
        newfunc.func = func
        newfunc.args = args
        newfunc.keywords = keywords
        return newfunc

The `!partial` function is used for partial function application which "freezes" some portion of a function's arguments and/or keywords resulting in a new object with a simplified signature. For example, `partial` can be used to create a callable that behaves like the `int` function where the *base* argument defaults to `2`:

<div class="doctest">

\>\>\> basetwo = partial(int, base=2) \>\>\> basetwo.\_\_doc\_\_ = 'Convert base 2 string to an int.' \>\>\> basetwo('10010') 18

</div>

If `Placeholder` sentinels are present in *args*, they will be filled first when `!partial` is called. This makes it possible to pre-fill any positional argument with a call to `!partial`; without `!Placeholder`, only the chosen number of leading positional arguments can be pre-filled.

If any `!Placeholder` sentinels are present, all must be filled at call time:

<div class="doctest">

\>\>\> say_to_world = partial(print, Placeholder, Placeholder, "world!") \>\>\> say_to_world('Hello', 'dear') Hello dear world!

</div>

Calling `say_to_world('Hello')` raises a `TypeError`, because only one positional argument is provided, but there are two placeholders that must be filled in.

If `!partial` is applied to an existing `partial object <partial-objects>`, `!Placeholder` sentinels of the input object are filled in with new positional arguments. A placeholder can be retained by inserting a new `!Placeholder` sentinel to the place held by a previous `!Placeholder`:

<div class="doctest">

\>\>\> from functools import partial, Placeholder as \_ \>\>\> remove = partial(str.replace, \_, \_, '') \>\>\> message = 'Hello, dear dear world!' \>\>\> remove(message, ' dear') 'Hello, world!' \>\>\> remove_dear = partial(remove, \_, ' dear') \>\>\> remove_dear(message) 'Hello, world!' \>\>\> remove_first_dear = partial(remove_dear, \_, 1) \>\>\> remove_first_dear(message) 'Hello, dear world!'

</div>

`!Placeholder` cannot be passed to `!partial` as a keyword argument.

<div class="versionchanged">

3.14 Added support for `Placeholder` in positional arguments.

</div>

</div>

<div class="partialmethod(func, /, *args, **keywords)">

Return a new `partialmethod` descriptor which behaves like `partial` except that it is designed to be used as a method definition rather than being directly callable.

*func* must be a `descriptor` or a callable (objects which are both, like normal functions, are handled as descriptors).

When *func* is a descriptor (such as a normal Python function, `classmethod`, `staticmethod`, `~abc.abstractmethod` or another instance of `partialmethod`), calls to `__get__` are delegated to the underlying descriptor, and an appropriate `partial object<partial-objects>` returned as the result.

When *func* is a non-descriptor callable, an appropriate bound method is created dynamically. This behaves like a normal Python function when used as a method: the *self* argument will be inserted as the first positional argument, even before the *args* and *keywords* supplied to the `partialmethod` constructor.

Example:

    >>> class Cell:
    ...     def __init__(self):
    ...         self._alive = False
    ...     @property
    ...     def alive(self):
    ...         return self._alive
    ...     def set_state(self, state):
    ...         self._alive = bool(state)
    ...     set_alive = partialmethod(set_state, True)
    ...     set_dead = partialmethod(set_state, False)
    ...
    >>> c = Cell()
    >>> c.alive
    False
    >>> c.set_alive()
    >>> c.alive
    True

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

reduce(function, iterable, /\[, initial\])

Apply *function* of two arguments cumulatively to the items of *iterable*, from left to right, so as to reduce the iterable to a single value. For example, `reduce(lambda x, y: x+y, [1, 2, 3, 4, 5])` calculates `((((1+2)+3)+4)+5)`. The left argument, *x*, is the accumulated value and the right argument, *y*, is the update value from the *iterable*. If the optional *initial* is present, it is placed before the items of the iterable in the calculation, and serves as a default when the iterable is empty. If *initial* is not given and *iterable* contains only one item, the first item is returned.

Roughly equivalent to:

    initial_missing = sentinel('initial_missing')

    def reduce(function, iterable, /, initial=initial_missing):
        it = iter(iterable)
        if initial is initial_missing:
            value = next(it)
        else:
            value = initial
        for element in it:
            value = function(value, element)
        return value

See `itertools.accumulate` for an iterator that yields all intermediate values.

<div class="versionchanged">

3.14 *initial* is now supported as a keyword argument.

</div>

</div>

<div class="decorator">

singledispatch

Transform a function into a `single-dispatch <single
dispatch>` `generic function`.

To define a generic function, decorate it with the `@singledispatch` decorator. When defining a function using `@singledispatch`, note that the dispatch happens on the type of the first argument:

    >>> from functools import singledispatch
    >>> @singledispatch
    ... def fun(arg, verbose=False):
    ...     if verbose:
    ...         print("Let me just say,", end=" ")
    ...     print(arg)

<div class="method" data-no-typesetting="">

singledispatch.register()

</div>

To add overloaded implementations to the function, use the `!register` attribute of the generic function, which can be used as a decorator. Pass the dispatch type to the decorator:

    >>> @fun.register(int)
    ... def _(arg, verbose=False):
    ...     if verbose:
    ...         print("Strength in numbers, eh?", end=" ")
    ...     print(arg)
    ...
    >>> @fun.register(list)
    ... def _(arg, verbose=False):
    ...     if verbose:
    ...         print("Enumerate this:")
    ...     for i, elem in enumerate(arg):
    ...         print(i, elem)

As a shortcut, for functions annotated with types, the decorator can infer the dispatch type without an explicit type argument:

    >>> @fun.register
    ... def _(arg: int, verbose=False):
    ...     if verbose:
    ...         print("Strength in numbers, eh?", end=" ")
    ...     print(arg)
    ...

This shortcut uses the first annotation, even if it belongs to another parameter or the return value. Annotate the dispatch parameter, or pass the dispatch type explicitly if its annotation is not first.

`typing.Union` can also be used with this shortcut:

    >>> @fun.register
    ... def _(arg: int | float, verbose=False):
    ...     if verbose:
    ...         print("Strength in numbers, eh?", end=" ")
    ...     print(arg)
    ...
    >>> from typing import Union
    >>> @fun.register
    ... def _(arg: Union[list, set], verbose=False):
    ...     if verbose:
    ...         print("Enumerate this:")
    ...     for i, elem in enumerate(arg):
    ...         print(i, elem)
    ...

For code that dispatches on a collections type (e.g., `list`), but wants to typehint the items of the collection (e.g., `list[int]`), the dispatch type should be passed explicitly to the decorator itself with the typehint going into the function definition:

    >>> @fun.register(list)
    ... def _(arg: list[int], verbose=False):
    ...     if verbose:
    ...         print("Enumerate this:")
    ...     for i, elem in enumerate(arg):
    ...         print(i, elem)

> [!NOTE]
> At runtime the function will dispatch on an instance of a list regardless of the type contained within the list i.e. `[1,2,3]` will be dispatched the same as `["foo", "bar", "baz"]`. The annotation provided in this example is for static type checkers only and has no runtime impact.

To enable registering `lambdas<lambda>` and pre-existing functions, the `~singledispatch.register` attribute can also be used in a functional form:

    >>> def nothing(arg, verbose=False):
    ...     print("Nothing.")
    ...
    >>> fun.register(type(None), nothing)

The `~singledispatch.register` attribute returns the undecorated function. This enables decorator stacking, `pickling<pickle>`, and the creation of unit tests for each variant independently:

    >>> @fun.register(float)
    ... @fun.register(Decimal)
    ... def fun_num(arg, verbose=False):
    ...     if verbose:
    ...         print("Half of your number:", end=" ")
    ...     print(arg / 2)
    ...
    >>> fun_num is fun
    False

When called, the generic function dispatches on the type of the first argument:

    >>> fun("Hello, world.")
    Hello, world.
    >>> fun("test.", verbose=True)
    Let me just say, test.
    >>> fun(42, verbose=True)
    Strength in numbers, eh? 42
    >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
    Enumerate this:
    0 spam
    1 spam
    2 eggs
    3 spam
    >>> fun(None)
    Nothing.
    >>> fun(1.23)
    0.615

Where there is no registered implementation for a specific type, its method resolution order is used to find a more generic implementation. The original function decorated with `@singledispatch` is registered for the base `object` type, which means it is used if no better implementation is found.

If an implementation is registered to an `abstract base class`, virtual subclasses of the base class will be dispatched to that implementation:

    >>> from collections.abc import Mapping
    >>> @fun.register
    ... def _(arg: Mapping, verbose=False):
    ...     if verbose:
    ...         print("Keys & Values")
    ...     for key, value in arg.items():
    ...         print(key, "=>", value)
    ...
    >>> fun({"a": "b"})
    a => b

To check which implementation the generic function will choose for a given type, use the `dispatch()` attribute:

    >>> fun.dispatch(float)
    <function fun_num at 0x1035a2840>
    >>> fun.dispatch(dict)    # note: default implementation
    <function fun at 0x103fe0000>

To access all registered implementations, use the read-only `registry` attribute:

    >>> fun.registry.keys()
    dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,
              <class 'decimal.Decimal'>, <class 'list'>,
              <class 'float'>])
    >>> fun.registry[float]
    <function fun_num at 0x1035a2840>
    >>> fun.registry[object]
    <function fun at 0x103fe0000>

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.7 The `~singledispatch.register` attribute now supports using type annotations.

</div>

<div class="versionchanged">

3.11 The `~singledispatch.register` attribute now supports `typing.Union` as a type annotation.

</div>

<div class="deprecated">

3.16 Using a return annotation to infer the dispatch type is deprecated and will raise `TypeError` in Python 3.21. Annotate the dispatch parameter or pass the dispatch type explicitly to `!register` instead.

</div>

</div>

<div class="singledispatchmethod(func)">

Transform a method into a `single-dispatch <single
dispatch>` `generic function`.

To define a generic method, decorate it with the `@singledispatchmethod` decorator. When defining a method using `@singledispatchmethod`, note that the dispatch happens on the type of the first non-*self* or non-*cls* argument:

    class Negator:
        @singledispatchmethod
        def neg(self, arg):
            raise NotImplementedError("Cannot negate a")

        @neg.register
        def _(self, arg: int):
            return -arg

        @neg.register
        def _(self, arg: bool):
            return not arg

When *self* or *cls* is annotated, pass the dispatch type explicitly rather than relying on annotation inference:

    from typing import Self

    class Negator:
        @singledispatchmethod
        def neg(self, arg):
            raise NotImplementedError("Cannot negate a")

        @neg.register(int)
        def _(self: Self, arg: int):
            return -arg

`@singledispatchmethod` supports nesting with other decorators such as `classmethod`. Note that to allow for `dispatcher.register`, `singledispatchmethod` must be the *outer most* decorator. Here is the `Negator` class with the `neg` methods bound to the class, rather than an instance of the class:

    class Negator:
        @singledispatchmethod
        @classmethod
        def neg(cls, arg):
            raise NotImplementedError("Cannot negate a")

        @neg.register
        @classmethod
        def _(cls, arg: int):
            return -arg

        @neg.register
        @classmethod
        def _(cls, arg: bool):
            return not arg

The same pattern can be used for other similar decorators: `staticmethod`, `~abc.abstractmethod`, and others.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.15 Added support of non-`descriptor` callables.

</div>

</div>

<div class="function">

update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)

Update a *wrapper* function to look like the *wrapped* function. The optional arguments are tuples to specify which attributes of the original function are assigned directly to the matching attributes on the wrapper function and which attributes of the wrapper function are updated with the corresponding attributes from the original function. The default values for these arguments are the module level constants `WRAPPER_ASSIGNMENTS` (which assigns to the wrapper function's `~function.__module__`, `~function.__name__`, `~function.__qualname__`, `~function.__annotations__`, `~function.__type_params__`, and `~function.__doc__`, the documentation string) and `WRAPPER_UPDATES` (which updates the wrapper function's `~function.__dict__`, i.e. the instance dictionary).

To allow access to the original function for introspection and other purposes (e.g. bypassing a caching decorator such as `lru_cache`), this function automatically adds a `__wrapped__` attribute to the wrapper that refers to the function being wrapped.

The main intended use for this function is in `decorator` functions which wrap the decorated function and return the wrapper. If the wrapper function is not updated, the metadata of the returned function will reflect the wrapper definition rather than the original function definition, which is typically less than helpful.

`update_wrapper` may be used with callables other than functions. Any attributes named in *assigned* or *updated* that are missing from the object being wrapped are ignored (i.e. this function will not attempt to set them on the wrapper function). `AttributeError` is still raised if the wrapper function itself is missing any attributes named in *updated*.

<div class="versionchanged">

3.2 The `__wrapped__` attribute is now automatically added. The `~function.__annotations__` attribute is now copied by default. Missing attributes no longer trigger an `AttributeError`.

</div>

<div class="versionchanged">

3.4 The `__wrapped__` attribute now always refers to the wrapped function, even if that function defined a `__wrapped__` attribute. (see `17482`)

</div>

<div class="versionchanged">

3.12 The `~function.__type_params__` attribute is now copied by default.

</div>

</div>

<div class="decorator">

wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)

This is a convenience function for invoking `update_wrapper` as a function decorator when defining a wrapper function. It is equivalent to `partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated)`. For example:

    >>> from functools import wraps
    >>> def my_decorator(f):
    ...     @wraps(f)
    ...     def wrapper(*args, **kwds):
    ...         print('Calling decorated function')
    ...         return f(*args, **kwds)
    ...     return wrapper
    ...
    >>> @my_decorator
    ... def example():
    ...     """Docstring"""
    ...     print('Called example function')
    ...
    >>> example()
    Calling decorated function
    Called example function
    >>> example.__name__
    'example'
    >>> example.__doc__
    'Docstring'

Without the use of this decorator factory, the name of the example function would have been `'wrapper'`, and the docstring of the original `!example` would have been lost.

</div>

## `partial` Objects

`partial` objects are callable objects created by `partial`. They have three read-only attributes:

<div class="attribute">

partial.func

A callable object or function. Calls to the `partial` object will be forwarded to `func` with new arguments and keywords.

</div>

<div class="attribute">

partial.args

The leftmost positional arguments that will be prepended to the positional arguments provided to a `partial` object call.

</div>

<div class="attribute">

partial.keywords

The keyword arguments that will be supplied when the `partial` object is called.

</div>

`partial` objects are like `function objects <user-defined-funcs>` in that they are callable, weak referenceable, and can have attributes. There are some important differences. For instance, the `~definition.__name__` and `~definition.__doc__` attributes are not created automatically.

# `!gc` --- Garbage Collector interface

<div class="module" data-synopsis="Interface to the cycle-detecting garbage collector.">

gc

</div>

------------------------------------------------------------------------

This module provides an interface to the optional garbage collector. It provides the ability to disable the collector, tune the collection frequency, and set debugging options. It also provides access to unreachable objects that the collector found but cannot free. Since the collector supplements the reference counting already used in Python, you can disable the collector if you are sure your program does not create reference cycles. Automatic collection can be disabled by calling `gc.disable()`. To debug a leaking program call `gc.set_debug(gc.DEBUG_LEAK)`. Notice that this includes `gc.DEBUG_SAVEALL`, causing garbage-collected objects to be saved in gc.garbage for inspection.

The `!gc` module provides the following functions:

<div class="function">

enable()

Enable automatic garbage collection.

</div>

<div class="function">

disable()

Disable automatic garbage collection.

</div>

<div class="function">

isenabled()

Return `True` if automatic collection is enabled.

</div>

<div class="function">

collect(generation=2)

With no arguments, run a full collection. The optional argument *generation* may be an integer specifying which generation to collect (from 0 to 2). A `ValueError` is raised if the generation number is invalid. The sum of collected objects and uncollectable objects is returned.

The free lists maintained for a number of built-in types are cleared whenever a full collection or collection of the highest generation (2) is run. Not all items in some free lists may be freed due to the particular implementation, in particular `float`.

The effect of calling `gc.collect()` while the interpreter is already performing a collection is undefined.

<div class="versionchanged">

3.14 `generation=1` performs an increment of collection.

</div>

<div class="versionchanged">

3.14.5 `generation=1` performs collection of the middle generation.

</div>

</div>

<div class="function">

set_debug(flags)

Set the garbage collection debugging flags. Debugging information will be written to `sys.stderr`. See below for a list of debugging flags which can be combined using bit operations to control debugging.

</div>

<div class="function">

get_debug()

Return the debugging flags currently set.

</div>

<div class="function">

get_objects(generation=None)

Returns a list of all objects tracked by the collector, excluding the list returned. If *generation* is not `None`, return only the objects tracked by the collector that are in that generation.

> [!WARNING]
> Care must be taken when using objects returned by `get_objects` because some of them could still be under construction and hence in a temporarily invalid state. Avoid using `get_objects` for any purpose other than debugging.

<div class="versionchanged">

3.8 New *generation* parameter.

</div>

<div class="versionchanged">

3.14 Generation 1 is removed

</div>

<div class="versionchanged">

3.14.5 Generation 1 is reintroduced to maintain GC behavior from 3.13.

</div>

<div class="audit-event">

gc.get_objects generation gc.get_objects

</div>

</div>

<div class="function">

get_stats()

Return a list of three per-generation dictionaries containing collection statistics since interpreter start. The number of keys may change in the future, but currently each dictionary will contain the following items:

- `collections` is the number of times this generation was collected;
- `collected` is the total number of objects collected inside this generation;
- `uncollectable` is the total number of objects which were found to be uncollectable (and were therefore moved to the `garbage` list) inside this generation;
- `candidates` is the total number of objects in this generation which were considered for collection and traversed;
- `duration` is the total time in seconds spent in collections for this generation.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.15 Add `duration` and `candidates`.

</div>

</div>

<div class="function">

set_threshold(threshold0, \[threshold1, \[threshold2\]\])

Set the garbage collection thresholds (the collection frequency). Setting *threshold0* to zero disables collection.

The GC classifies objects into three generations depending on how many collection sweeps they have survived. New objects are placed in the youngest generation (generation `0`). If an object survives a collection it is moved into the next older generation. Since generation `2` is the oldest generation, objects in that generation remain there after a collection. In order to decide when to run, the collector keeps track of the number object allocations and deallocations since the last collection. When the number of allocations minus the number of deallocations exceeds *threshold0*, collection starts. Initially only generation `0` is examined. If generation `0` has been examined more than *threshold1* times since generation `1` has been examined, then generation `1` is examined as well. With the third generation, things are a bit more complicated, see [Collecting the oldest generation](https://github.com/python/cpython/blob/ff0ef0a54bef26fc507fbf9b7a6009eb7d3f17f5/InternalDocs/garbage_collector.md#collecting-the-oldest-generation) for more information.

See [Garbage collector design](https://github.com/python/cpython/blob/3.15/InternalDocs/garbage_collector.md) for more information.

<div class="versionchanged">

3.14 *threshold2* is ignored

</div>

<div class="versionchanged">

3.14.5 *threshold2* is restored to match Python 3.13 behavior.

</div>

</div>

<div class="function">

get_count()

Return the current collection counts as a tuple of `(count0, count1, count2)`.

</div>

<div class="function">

get_threshold()

Return the current collection thresholds as a tuple of `(threshold0, threshold1, threshold2)`.

</div>

<div class="function">

get_referrers(\*objs)

Return the list of objects that directly refer to any of objs. This function will only locate those containers which support garbage collection; extension types which do refer to other objects but do not support garbage collection will not be found.

Note that objects which have already been dereferenced, but which live in cycles and have not yet been collected by the garbage collector can be listed among the resulting referrers. To get only currently live objects, call `collect` before calling `get_referrers`.

> [!WARNING]
> Care must be taken when using objects returned by `get_referrers` because some of them could still be under construction and hence in a temporarily invalid state. Avoid using `get_referrers` for any purpose other than debugging.

<div class="audit-event">

gc.get_referrers objs gc.get_referrers

</div>

</div>

<div class="function">

get_referents(\*objs)

Return a list of objects directly referred to by any of the arguments. The referents returned are those objects visited by the arguments' C-level `~PyTypeObject.tp_traverse` methods (if any), and may not be all objects actually directly reachable. `~PyTypeObject.tp_traverse` methods are supported only by objects that support garbage collection, and are only required to visit objects that may be involved in a cycle. So, for example, if an integer is directly reachable from an argument, that integer object may or may not appear in the result list.

<div class="audit-event">

gc.get_referents objs gc.get_referents

</div>

</div>

<div class="function">

is_tracked(obj)

Returns `True` if the object is currently tracked by the garbage collector, `False` otherwise. As a general rule, instances of atomic types aren't tracked and instances of non-atomic types (containers, user-defined objects...) are. However, some type-specific optimizations can be present in order to suppress the garbage collector footprint of simple instances (e.g. dicts containing only atomic keys and values):

    >>> gc.is_tracked(0)
    False
    >>> gc.is_tracked("a")
    False
    >>> gc.is_tracked([])
    True
    >>> gc.is_tracked({})
    False
    >>> gc.is_tracked({"a": 1})
    True

<div class="versionadded">

3.1

</div>

</div>

<div class="function">

is_finalized(obj)

Returns `True` if the given object has been finalized by the garbage collector, `False` otherwise. :

    >>> x = None
    >>> class Lazarus:
    ...     def __del__(self):
    ...         global x
    ...         x = self
    ...
    >>> lazarus = Lazarus()
    >>> gc.is_finalized(lazarus)
    False
    >>> del lazarus
    >>> gc.is_finalized(x)
    True

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

freeze()

Freeze all the objects tracked by the garbage collector; move them to a permanent generation and ignore them in all the future collections.

If a process will `fork()` without `exec()`, avoiding unnecessary copy-on-write in child processes will maximize memory sharing and reduce overall memory usage. This requires both avoiding creation of freed "holes" in memory pages in the parent process and ensuring that GC collections in child processes won't touch the `gc_refs` counter of long-lived objects originating in the parent process. To accomplish both, call `gc.disable()` early in the parent process, `gc.freeze()` right before `fork()`, and `gc.enable()` early in child processes.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

unfreeze()

Unfreeze the objects in the permanent generation, put them back into the oldest generation.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

get_freeze_count()

Return the number of objects in the permanent generation.

<div class="versionadded">

3.7

</div>

</div>

The following variables are provided for read-only access (you can mutate the values but should not rebind them):

<div class="data">

garbage

A list of objects which the collector found to be unreachable but could not be freed (uncollectable objects). Starting with Python 3.4, this list should be empty most of the time, except when using instances of C extension types with a non-`NULL` `tp_del` slot.

If `DEBUG_SAVEALL` is set, then all unreachable objects will be added to this list rather than freed.

<div class="versionchanged">

3.2 If this list is non-empty at `interpreter shutdown`, a `ResourceWarning` is emitted, which is silent by default. If `DEBUG_UNCOLLECTABLE` is set, in addition all uncollectable objects are printed.

</div>

<div class="versionchanged">

3.4 Following `442`, objects with a `~object.__del__` method don't end up in `gc.garbage` anymore.

</div>

</div>

<div class="data">

callbacks

A list of callbacks that will be invoked by the garbage collector before and after collection. The callbacks will be called with two arguments, *phase* and *info*.

*phase* can be one of two values:

> "start": The garbage collection is about to start.
>
> "stop": The garbage collection has finished.

*info* is a dict providing more information for the callback. The following keys are currently defined:

> "generation": The oldest generation being collected.
>
> "collected": When *phase* is "stop", the number of objects successfully collected.
>
> "uncollectable": When *phase* is "stop", the number of objects that could not be collected and were put in `garbage`.
>
> "candidates": When *phase* is "stop", the total number of objects in this generation which were considered for collection and traversed.
>
> "duration": When *phase* is "stop", the time in seconds spent in the collection.

Applications can add their own callbacks to this list. The primary use cases are:

> Gathering statistics about garbage collection, such as how often various generations are collected, and how long the collection takes.
>
> Allowing applications to identify and clear their own uncollectable types when they appear in `garbage`.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.15 Add "duration" and "candidates".

</div>

</div>

The following constants are provided for use with `set_debug`:

<div class="data">

DEBUG_STATS

Print statistics during collection. This information can be useful when tuning the collection frequency.

</div>

<div class="data">

DEBUG_COLLECTABLE

Print information on collectable objects found.

</div>

<div class="data">

DEBUG_UNCOLLECTABLE

Print information of uncollectable objects found (objects which are not reachable but cannot be freed by the collector). These objects will be added to the `garbage` list.

<div class="versionchanged">

3.2 Also print the contents of the `garbage` list at `interpreter shutdown`, if it isn't empty.

</div>

</div>

<div class="data">

DEBUG_SAVEALL

When set, all unreachable objects found will be appended to *garbage* rather than being freed. This can be useful for debugging a leaking program.

</div>

<div class="data">

DEBUG_LEAK

The debugging flags necessary for the collector to print information about a leaking program (equal to `DEBUG_COLLECTABLE | DEBUG_UNCOLLECTABLE | DEBUG_SAVEALL`).

</div>

# `!getopt` --- C-style parser for command line options

<div class="module" data-synopsis="Portable parser for command line options; support both short and
long option names.">

getopt

</div>

**Source code:** `Lib/getopt.py`

> [!NOTE]
> This module is considered feature complete. A more declarative and extensible alternative to this API is provided in the `optparse` module. Further functional enhancements for command line parameter processing are provided either as third party modules on PyPI, or else as features in the `argparse` module.

------------------------------------------------------------------------

This module helps scripts to parse the command line arguments in `sys.argv`. It supports the same conventions as the Unix `!getopt` function (including the special meanings of arguments of the form '`-`' and '`--`'). Long options similar to those supported by GNU software may be used as well via an optional third argument.

Users who are unfamiliar with the Unix `!getopt` function should consider using the `argparse` module instead. Users who are familiar with the Unix `!getopt` function, but would like to get equivalent behavior while writing less code and getting better help and error messages should consider using the `optparse` module. See `choosing-an-argument-parser` for additional details.

This module provides two functions and an exception:

<div class="function">

getopt(args, shortopts, longopts=\[\])

Parses command line options and parameter list. *args* is the argument list to be parsed, without the leading reference to the running program. Typically, this means `sys.argv[1:]`. *shortopts* is the string of option letters that the script wants to recognize, with options that require an argument followed by a colon (`':'`) and options that accept an optional argument followed by two colons (`'::'`); i.e., the same format that Unix `!getopt` uses.

> [!NOTE]
> Unlike GNU `!getopt`, after a non-option argument, all further arguments are considered also non-options. This is similar to the way non-GNU Unix systems work.

*longopts*, if specified, must be a list of strings with the names of the long options which should be supported. The leading `'--'` characters should not be included in the option name. Long options which require an argument should be followed by an equal sign (`'='`). Long options which accept an optional argument should be followed by an equal sign and question mark (`'=?'`). To accept only long options, *shortopts* should be an empty string. Long options on the command line can be recognized so long as they provide a prefix of the option name that matches exactly one of the accepted options. For example, if *longopts* is `['foo', 'frob']`, the option `--fo` will match as `--foo`, but `--f` will not match uniquely, so `GetoptError` will be raised.

If *longopts* is a string it gets treated as a list of a single element.

The return value consists of two elements: the first is a list of `(option, value)` pairs; the second is the list of program arguments left after the option list was stripped (this is a trailing slice of *args*). Each option-and-value pair returned has the option as its first element, prefixed with a hyphen for short options (e.g., `'-x'`) or two hyphens for long options (e.g., `'--long-option'`), and the option argument as its second element, or an empty string if the option has no argument. The options occur in the list in the same order in which they were found, thus allowing multiple occurrences. Long and short options may be mixed.

<div class="versionchanged">

3.14 Optional arguments are supported.

</div>

</div>

<div class="function">

gnu_getopt(args, shortopts, longopts=\[\])

This function works like `~getopt.getopt`, except that GNU style scanning mode is used by default. This means that option and non-option arguments may be intermixed. The `~getopt.getopt` function stops processing options as soon as a non-option argument is encountered.

If the first character of the option string is `'+'`, or if the environment variable `!POSIXLY_CORRECT` is set, then option processing stops as soon as a non-option argument is encountered.

If the first character of the option string is `'-'`, non-option arguments that are followed by options are added to the list of option-and-value pairs as a pair that has `None` as its first element and the list of non-option arguments as its second element. The second element of the `!gnu_getopt` result is a list of program arguments after the last option.

<div class="versionchanged">

3.14 Support for returning intermixed options and non-option arguments in order.

</div>

</div>

<div class="exception">

GetoptError

This is raised when an unrecognized option is found in the argument list or when an option requiring an argument is given none. The argument to the exception is a string indicating the cause of the error. For long options, an argument given to an option which does not require one will also cause this exception to be raised. The attributes `!msg` and `!opt` give the error message and related option; if there is no specific option to which the exception relates, `!opt` is an empty string.

</div>

<div class="exception">

error

Alias for `GetoptError`; for backward compatibility.

</div>

An example using only Unix style options:

<div class="doctest">

\>\>\> import getopt \>\>\> args = '-a -b -cfoo -d bar a1 a2'.split() \>\>\> args \['-a', '-b', '-cfoo', '-d', 'bar', 'a1', 'a2'\] \>\>\> optlist, args = getopt.getopt(args, 'abc:d:') \>\>\> optlist \[('-a', ''), ('-b', ''), ('-c', 'foo'), ('-d', 'bar')\] \>\>\> args \['a1', 'a2'\]

</div>

Using long option names is equally easy:

<div class="doctest">

\>\>\> s = '--condition=foo --testing --output-file abc.def -x a1 a2' \>\>\> args = s.split() \>\>\> args \['--condition=foo', '--testing', '--output-file', 'abc.def', '-x', 'a1', 'a2'\] \>\>\> optlist, args = getopt.getopt(args, 'x', \[ ... 'condition=', 'output-file=', 'testing'\]) \>\>\> optlist \[('--condition', 'foo'), ('--testing', ''), ('--output-file', 'abc.def'), ('-x', '')\] \>\>\> args \['a1', 'a2'\]

</div>

Optional arguments should be specified explicitly:

<div class="doctest">

\>\>\> s = '-Con -C --color=off --color a1 a2' \>\>\> args = s.split() \>\>\> args \['-Con', '-C', '--color=off', '--color', 'a1', 'a2'\] \>\>\> optlist, args = getopt.getopt(args, 'C::', \['color=?'\]) \>\>\> optlist \[('-C', 'on'), ('-C', ''), ('--color', 'off'), ('--color', '')\] \>\>\> args \['a1', 'a2'\]

</div>

The order of options and non-option arguments can be preserved:

<div class="doctest">

\>\>\> s = 'a1 -x a2 a3 a4 --long a5 a6' \>\>\> args = s.split() \>\>\> args \['a1', '-x', 'a2', 'a3', 'a4', '--long', 'a5', 'a6'\] \>\>\> optlist, args = getopt.gnu_getopt(args, '-x:', \['long='\]) \>\>\> optlist \[(None, \['a1'\]), ('-x', 'a2'), (None, \['a3', 'a4'\]), ('--long', 'a5')\] \>\>\> args \['a6'\]

</div>

In a script, typical usage is something like this:

<div class="testcode">

import getopt, sys

def main():  
try:  
opts, args = getopt.getopt(sys.argv\[1:\], "ho:v", \["help", "output="\])

except getopt.GetoptError as err:  
\# print help information and exit: print(err) \# will print something like "option -a not recognized" usage() sys.exit(2)

output = None verbose = False for o, a in opts: if o == "-v": verbose = True elif o in ("-h", "--help"): usage() sys.exit() elif o in ("-o", "--output"): output = a else: assert False, "unhandled option" process(args, output=output, verbose=verbose)

if \_\_name\_\_ == "\_\_main\_\_":  
main()

</div>

Note that an equivalent command line interface could be produced with less code and more informative help and error messages by using the `optparse` module:

<div class="testcode">

import optparse

if \_\_name\_\_ == '\_\_main\_\_':  
parser = optparse.OptionParser() parser.add_option('-o', '--output') parser.add_option('-v', dest='verbose', action='store_true') opts, args = parser.parse_args() process(args, output=opts.output, verbose=opts.verbose)

</div>

A roughly equivalent command line interface for this case can also be produced by using the `argparse` module:

<div class="testcode">

import argparse

if \_\_name\_\_ == '\_\_main\_\_':  
parser = argparse.ArgumentParser() parser.add_argument('-o', '--output') parser.add_argument('-v', dest='verbose', action='store_true') parser.add_argument('rest', nargs='\*') args = parser.parse_args() process(args.rest, output=args.output, verbose=args.verbose)

</div>

See `choosing-an-argument-parser` for details on how the `argparse` version of this code differs in behaviour from the `optparse` (and `getopt`) version.

<div class="seealso">

Module `optparse`  
Declarative command line option parsing.

Module `argparse`  
More opinionated command line option and argument parsing library.

</div>

# `!getpass` --- Portable password input

<div class="module" data-synopsis="Portable reading of passwords and retrieval of the userid.">

getpass

</div>

**Source code:** `Lib/getpass.py`

------------------------------------------------------------------------

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

The `!getpass` module provides two functions:

<div class="function">

getpass(prompt='Password: ', stream=None, \*, echo_char=None)

Prompt the user for a password without echoing. The user is prompted using the string *prompt*, which defaults to `'Password: '`. On Unix, the prompt is written to the file-like object *stream* using the replace error handler if needed. *stream* defaults to the controlling terminal (`/dev/tty`) or if that is unavailable to `sys.stderr` (this argument is ignored on Windows).

The *echo_char* argument controls how user input is displayed while typing. If *echo_char* is `None` (default), input remains hidden. Otherwise, *echo_char* must be a single printable ASCII character and each typed character is replaced by it. For example, `echo_char='*'` will display asterisks instead of the actual input.

If echo-free input is unavailable, `~getpass.getpass` falls back to printing a warning message to *stream* and reading from `sys.stdin` and issuing a `GetPassWarning`.

> [!NOTE]
> If you call `~getpass.getpass` from within IDLE, the input may be done in the terminal you launched IDLE from rather than the idle window itself.

> [!NOTE]
> On Unix systems, when *echo_char* is set, the terminal will be configured to operate in `noncanonical mode <termios(3)#Canonical_and_noncanonical_mode>`. Common terminal control characters are supported:
>
> - `Ctrl+A` - Move cursor to beginning of line
> - `Ctrl+E` - Move cursor to end of line
> - `Ctrl+K` - Kill (delete) from cursor to end of line
> - `Ctrl+U` - Kill (delete) entire line
> - `Ctrl+W` - Erase previous word
> - `Ctrl+V` - Insert next character literally (quote)
> - `Backspace`/`DEL` - Delete character before cursor
>
> These shortcuts work by reading the terminal's configured control character mappings from termios settings.

<div class="versionchanged">

3.14 Added the *echo_char* parameter for keyboard feedback.

</div>

<div class="versionchanged">

3.15 When using non-empty *echo_char* on Unix, keyboard shortcuts (including cursor movement and line editing) are now properly handled using the terminal's control character configuration.

</div>

</div>

<div class="exception">

GetPassWarning

A `UserWarning` subclass issued when password input may be echoed.

</div>

<div class="function">

getuser()

Return the "login name" of the user.

This function checks the environment variables `LOGNAME`, `USER`, `!LNAME` and `USERNAME`, in order, and returns the value of the first one which is set to a non-empty string. If none are set, the login name from the password database is returned on systems which support the `pwd` module, otherwise, an `OSError` is raised.

In general, this function should be preferred over `os.getlogin`.

<div class="versionchanged">

3.13 Previously, various exceptions beyond just `OSError` were raised.

</div>

</div>

# `!gettext` --- Multilingual internationalization services

<div class="module" data-synopsis="Multilingual internationalization services.">

gettext

</div>

**Source code:** `Lib/gettext.py`

------------------------------------------------------------------------

The `!gettext` module provides internationalization (I18N) and localization (L10N) services for your Python modules and applications. It supports both the GNU `gettext` message catalog API and a higher level, class-based API that may be more appropriate for Python files. The interface described below allows you to write your module and application messages in one natural language, and provide a catalog of translated messages for running under different natural languages.

Some hints on localizing your Python modules and applications are also given.

## GNU `gettext` API

The `!gettext` module defines the following API, which is very similar to the GNU `gettext` API. If you use this API you will affect the translation of your entire application globally. Often this is what you want if your application is monolingual, with the choice of language dependent on the locale of your user. If you are localizing a Python module, or if your application needs to switch languages on the fly, you probably want to use the class-based API instead.

<div class="function">

bindtextdomain(domain, localedir=None)

Bind the *domain* to the locale directory *localedir*. More concretely, `!gettext` will look for binary `.mo` files for the given domain using the path (on Unix): `{localedir}/{language}/LC_MESSAGES/{domain}.mo`, where *language* is searched for in the environment variables `LANGUAGE`, `LC_ALL`, `LC_MESSAGES`, and `LANG` respectively.

If *localedir* is omitted or `None`, then the current binding for *domain* is returned.[^1]

</div>

<div class="function">

textdomain(domain=None)

Change or query the current global domain. If *domain* is `None`, then the current global domain is returned, otherwise the global domain is set to *domain*, which is returned.

</div>

<div class="index">

single: \_ (underscore); gettext

</div>

<div class="function">

gettext(message, /)

Return the localized translation of *message*, based on the current global domain, language, and locale directory. This function is usually aliased as `!_` in the local namespace (see examples below).

</div>

<div class="function">

dgettext(domain, message, /)

Like `.gettext`, but look the message up in the specified *domain*.

</div>

<div class="function">

ngettext(singular, plural, n, /)

Like `.gettext`, but consider plural forms. If a translation is found, apply the plural formula to *n*, and return the resulting message (some languages have more than two plural forms). If no translation is found, return *singular* if *n* is 1; return *plural* otherwise.

The Plural formula is taken from the catalog header. It is a C or Python expression that has a free variable *n*; the expression evaluates to the index of the plural in the catalog. See [the GNU gettext documentation](https://www.gnu.org/software/gettext/manual/gettext.html) for the precise syntax to be used in `.po` files and the formulas for a variety of languages.

</div>

<div class="function">

dngettext(domain, singular, plural, n, /)

Like `ngettext`, but look the message up in the specified *domain*.

</div>

<div class="function">

pgettext(context, message, /)

</div>

<div class="function">

dpgettext(domain, context, message, /)

</div>

<div class="function">

npgettext(context, singular, plural, n, /)

</div>

<div class="function">

dnpgettext(domain, context, singular, plural, n, /)

Similar to the corresponding functions without the `p` in the prefix (that is, `~gettext.gettext`, `dgettext`, `ngettext`, `dngettext`), but the translation is restricted to the given message *context*.

<div class="versionadded">

3.8

</div>

</div>

Note that GNU `gettext` also defines a `!dcgettext` method, but this was deemed not useful and so it is currently unimplemented.

Here's an example of typical usage for this API:

    import gettext
    gettext.bindtextdomain('myapplication', '/path/to/my/language/directory')
    gettext.textdomain('myapplication')
    _ = gettext.gettext
    # ...
    print(_('This is a translatable string.'))

## Class-based API

The class-based API of the `!gettext` module gives you more flexibility and greater convenience than the GNU `gettext` API. It is the recommended way of localizing your Python applications and modules. `!gettext` defines a `GNUTranslations` class which implements the parsing of GNU `.mo` format files, and has methods for returning strings. Instances of this class can also install themselves in the built-in namespace as the function `!_`.

<div class="function">

find(domain, localedir=None, languages=None, all=False)

This function implements the standard `.mo` file search algorithm. It takes a *domain*, identical to what `textdomain` takes. Optional *localedir* is as in `bindtextdomain`. Optional *languages* is a list of strings, where each string is a language code.

If *localedir* is not given, then the default system locale directory is used. [^2] If *languages* is not given, then the following environment variables are searched: `LANGUAGE`, `LC_ALL`, `LC_MESSAGES`, and `LANG`. The first one returning a non-empty value is used for the *languages* variable. The environment variables should contain a colon separated list of languages, which will be split on the colon to produce the expected list of language code strings.

`find` then expands and normalizes the languages, and then iterates through them, searching for an existing file built of these components:

`{localedir}/{language}/LC_MESSAGES/{domain}.mo`

The first such file name that exists is returned by `find`. If no such file is found, then `None` is returned. If *all* is given, it returns a list of all file names, in the order in which they appear in the languages list or the environment variables.

</div>

<div class="function">

translation(domain, localedir=None, languages=None, [class]()=None, fallback=False)

Return a `*Translations` instance based on the *domain*, *localedir*, and *languages*, which are first passed to `find` to get a list of the associated `.mo` file paths. Instances with identical `.mo` file names are cached. The actual class instantiated is *class\_* if provided, otherwise `GNUTranslations`. The class's constructor must take a single `file object` argument.

If multiple files are found, later files are used as fallbacks for earlier ones. To allow setting the fallback, `copy.copy` is used to clone each translation object from the cache; the actual instance data is still shared with the cache.

If no `.mo` file is found, this function raises `OSError` if *fallback* is false (which is the default), and returns a `NullTranslations` instance if *fallback* is true.

<div class="versionchanged">

3.3 `IOError` used to be raised, it is now an alias of `OSError`.

</div>

<div class="versionchanged">

3.11 *codeset* parameter is removed.

</div>

</div>

<div class="function">

install(domain, localedir=None, \*, names=None)

This installs the function `!_` in Python's builtins namespace, based on *domain* and *localedir* which are passed to the function `translation`.

For the *names* parameter, please see the description of the translation object's `~NullTranslations.install` method.

As seen below, you usually mark the strings in your application that are candidates for translation, by wrapping them in a call to the `!_` function, like this:

    print(_('This string will be translated.'))

For convenience, you want the `!_` function to be installed in Python's builtins namespace, so it is easily accessible in all modules of your application.

<div class="versionchanged">

3.11 *names* is now a keyword-only parameter.

</div>

</div>

### The `NullTranslations` class

Translation classes are what actually implement the translation of original source file message strings to translated message strings. The base class used by all translation classes is `NullTranslations`; this provides the basic interface you can use to write your own specialized translation classes. Here are the methods of `!NullTranslations`:

<div class="NullTranslations(fp=None)">

Takes an optional `file object` *fp*, which is ignored by the base class. Initializes "protected" instance variables *\_info* and *\_charset* which are set by derived classes, as well as *\_fallback*, which is set through `add_fallback`. It then calls `self._parse(fp)` if *fp* is not `None`.

<div class="method">

<span id="parse">parse</span>(fp)

No-op in the base class, this method takes file object *fp*, and reads the data from the file, initializing its message catalog. If you have an unsupported message catalog file format, you should override this method to parse your format.

</div>

<div class="method">

add_fallback(fallback)

Add *fallback* as the fallback object for the current translation object. A translation object should consult the fallback if it cannot provide a translation for a given message.

</div>

<div class="method">

gettext(message, /)

If a fallback has been set, forward `!gettext` to the fallback. Otherwise, return *message*. Overridden in derived classes.

</div>

<div class="method">

ngettext(singular, plural, n, /)

If a fallback has been set, forward `!ngettext` to the fallback. Otherwise, return *singular* if *n* is 1; return *plural* otherwise. Overridden in derived classes.

</div>

<div class="method">

pgettext(context, message, /)

If a fallback has been set, forward `pgettext` to the fallback. Otherwise, return the translated message. Overridden in derived classes.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

npgettext(context, singular, plural, n, /)

If a fallback has been set, forward `npgettext` to the fallback. Otherwise, return the translated message. Overridden in derived classes.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

info()

Return a dictionary containing the metadata found in the message catalog file.

</div>

<div class="method">

charset()

Return the encoding of the message catalog file.

</div>

<div class="method">

install(names=None)

This method installs `.gettext` into the built-in namespace, binding it to `_`.

If the *names* parameter is given, it must be a sequence containing the names of functions you want to install in the builtins namespace in addition to `!_`. Supported names are `'gettext'`, `'ngettext'`, `'pgettext'`, and `'npgettext'`.

Note that this is only one way, albeit the most convenient way, to make the `!_` function available to your application. Because it affects the entire application globally, and specifically the built-in namespace, localized modules should never install `!_`. Instead, they should use this code to make `!_` available to their module:

    import gettext
    t = gettext.translation('mymodule', ...)
    _ = t.gettext

This puts `!_` only in the module's global namespace and so only affects calls within this module.

<div class="versionchanged">

3.8 Added `'pgettext'` and `'npgettext'`.

</div>

</div>

</div>

### The `GNUTranslations` class

The `!gettext` module provides one additional class derived from `NullTranslations`: `GNUTranslations`. This class overrides `!_parse` to enable reading GNU `gettext` format `.mo` files in both big-endian and little-endian format.

`GNUTranslations` parses optional metadata out of the translation catalog. It is convention with GNU `gettext` to include metadata as the translation for the empty string. This metadata is in `822`-style `key: value` pairs, and should contain the `Project-Id-Version` key. If the key `Content-Type` is found, then the `charset` property is used to initialize the "protected" `!_charset` instance variable, defaulting to `None` if not found. If the charset encoding is specified, then all message ids and message strings read from the catalog are converted to Unicode using this encoding, else ASCII is assumed.

Since message ids are read as Unicode strings too, all `*gettext()` methods will assume message ids as Unicode strings, not byte strings.

The entire set of key/value pairs are placed into a dictionary and set as the "protected" `!_info` instance variable.

If the `.mo` file's magic number is invalid, the major version number is unexpected, or if other problems occur while reading the file, instantiating a `GNUTranslations` class can raise `OSError`.

<div class="GNUTranslations">

The following methods are overridden from the base class implementation:

<div class="method">

gettext(message, /)

Look up the *message* id in the catalog and return the corresponding message string, as a Unicode string. If there is no entry in the catalog for the *message* id, and a fallback has been set, the look up is forwarded to the fallback's `~NullTranslations.gettext` method. Otherwise, the *message* id is returned.

</div>

<div class="method">

ngettext(singular, plural, n, /)

Do a plural-forms lookup of a message id. *singular* is used as the message id for purposes of lookup in the catalog, while *n* is used to determine which plural form to use. The returned message string is a Unicode string.

If the message id is not found in the catalog, and a fallback is specified, the request is forwarded to the fallback's `~NullTranslations.ngettext` method. Otherwise, when *n* is 1 *singular* is returned, and *plural* is returned in all other cases.

Here is an example:

    n = len(os.listdir('.'))
    cat = GNUTranslations(somefile)
    message = cat.ngettext(
        'There is %(num)d file in this directory',
        'There are %(num)d files in this directory',
        n) % {'num': n}

</div>

<div class="method">

pgettext(context, message, /)

Look up the *context* and *message* id in the catalog and return the corresponding message string, as a Unicode string. If there is no entry in the catalog for the *message* id and *context*, and a fallback has been set, the look up is forwarded to the fallback's `pgettext` method. Otherwise, the *message* id is returned.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

npgettext(context, singular, plural, n, /)

Do a plural-forms lookup of a message id. *singular* is used as the message id for purposes of lookup in the catalog, while *n* is used to determine which plural form to use.

If the message id for *context* is not found in the catalog, and a fallback is specified, the request is forwarded to the fallback's `npgettext` method. Otherwise, when *n* is 1 *singular* is returned, and *plural* is returned in all other cases.

<div class="versionadded">

3.8

</div>

</div>

</div>

### Solaris message catalog support

The Solaris operating system defines its own binary `.mo` file format, but since no documentation can be found on this format, it is not supported at this time.

### The Catalog constructor

<div class="index">

single: GNOME

</div>

GNOME uses a version of the `!gettext` module by James Henstridge, but this version has a slightly different API. Its documented usage was:

    import gettext
    cat = gettext.Catalog(domain, localedir)
    _ = cat.gettext
    print(_('hello world'))

For compatibility with this older module, the function `!Catalog` is an alias for the `translation` function described above.

One difference between this module and Henstridge's: his catalog objects supported access through a mapping API, but this appears to be unused and so is not currently supported.

## Internationalizing your programs and modules

Internationalization (I18N) refers to the operation by which a program is made aware of multiple languages. Localization (L10N) refers to the adaptation of your program, once internationalized, to the local language and cultural habits. In order to provide multilingual messages for your Python programs, you need to take the following steps:

1.  prepare your program or module by specially marking translatable strings
2.  run a suite of tools over your marked files to generate raw messages catalogs
3.  create language-specific translations of the message catalogs
4.  use the `!gettext` module so that message strings are properly translated

In order to prepare your code for I18N, you need to look at all the strings in your files. Any string that needs to be translated should be marked by wrapping it in `_('...')` --- that is, a call to the function `_ <gettext.gettext>`. For example:

    filename = 'mylog.txt'
    message = _('writing a log message')
    with open(filename, 'w') as fp:
        fp.write(message)

In this example, the string `'writing a log message'` is marked as a candidate for translation, while the strings `'mylog.txt'` and `'w'` are not.

There are a few tools to extract the strings meant for translation. The original GNU `gettext` only supported C or C++ source code but its extended version `xgettext` scans code written in a number of languages, including Python, to find strings marked as translatable. [Babel](https://babel.pocoo.org/) is a Python internationalization library that includes a `pybabel` script to extract and compile message catalogs. François Pinard's program called `xpot` does a similar job and is available as part of his [po-utils package](https://github.com/pinard/po-utils).

(Python also includes pure-Python versions of these programs, called `pygettext.py` and `msgfmt.py`; some Python distributions will install them for you. `pygettext.py` is similar to `xgettext`, but only understands Python source code and cannot handle other programming languages such as C or C++. `pygettext.py` supports a command-line interface similar to `xgettext`; for details on its use, run `pygettext.py --help`. `msgfmt.py` is binary compatible with GNU `msgfmt`. With these two programs, you may not need the GNU `gettext` package to internationalize your Python applications.)

`xgettext`, `pygettext`, and similar tools generate `.po` files that are message catalogs. They are structured human-readable files that contain every marked string in the source code, along with a placeholder for the translated versions of these strings.

Copies of these `.po` files are then handed over to the individual human translators who write translations for every supported natural language. They send back the completed language-specific versions as a `<language-name>.po` file that's compiled into a machine-readable `.mo` binary catalog file using the `msgfmt` program. The `.mo` files are used by the `!gettext` module for the actual translation processing at run-time.

How you use the `!gettext` module in your code depends on whether you are internationalizing a single module or your entire application. The next two sections will discuss each case.

### Localizing your module

If you are localizing your module, you must take care not to make global changes, e.g. to the built-in namespace. You should not use the GNU `gettext` API but instead the class-based API.

Let's say your module is called "spam" and the module's various natural language translation `.mo` files reside in `/usr/share/locale` in GNU `gettext` format. Here's what you would put at the top of your module:

    import gettext
    t = gettext.translation('spam', '/usr/share/locale')
    _ = t.gettext

### Localizing your application

If you are localizing your application, you can install the `!_` function globally into the built-in namespace, usually in the main driver file of your application. This will let all your application-specific files just use `_('...')` without having to explicitly install it in each file.

In the simple case then, you need only add the following bit of code to the main driver file of your application:

    import gettext
    gettext.install('myapplication')

If you need to set the locale directory, you can pass it into the `install` function:

    import gettext
    gettext.install('myapplication', '/usr/share/locale')

### Changing languages on the fly

If your program needs to support many languages at the same time, you may want to create multiple translation instances and then switch between them explicitly, like so:

    import gettext

    lang1 = gettext.translation('myapplication', languages=['en'])
    lang2 = gettext.translation('myapplication', languages=['fr'])
    lang3 = gettext.translation('myapplication', languages=['de'])

    # start by using language1
    lang1.install()

    # ... time goes by, user selects language 2
    lang2.install()

    # ... more time goes by, user selects language 3
    lang3.install()

### Deferred translations

In most coding situations, strings are translated where they are coded. Occasionally however, you need to mark strings for translation, but defer actual translation until later. A classic example is:

    animals = ['mollusk',
               'albatross',
               'rat',
               'penguin',
               'python', ]
    # ...
    for a in animals:
        print(a)

Here, you want to mark the strings in the `animals` list as being translatable, but you don't actually want to translate them until they are printed.

Here is one way you can handle this situation:

    def _(message): return message

    animals = [_('mollusk'),
               _('albatross'),
               _('rat'),
               _('penguin'),
               _('python'), ]

    del _

    # ...
    for a in animals:
        print(_(a))

This works because the dummy definition of `!_` simply returns the string unchanged. And this dummy definition will temporarily override any definition of `!_` in the built-in namespace (until the `del` command). Take care, though if you have a previous definition of `!_` in the local namespace.

Note that the second use of `!_` will not identify "a" as being translatable to the `gettext` program, because the parameter is not a string literal.

Another way to handle this is with the following example:

    def N_(message): return message

    animals = [N_('mollusk'),
               N_('albatross'),
               N_('rat'),
               N_('penguin'),
               N_('python'), ]

    # ...
    for a in animals:
        print(_(a))

In this case, you are marking translatable strings with the function `!N_`, which won't conflict with any definition of `!_`. However, you will need to teach your message extraction program to look for translatable strings marked with `!N_`. `xgettext`, `pygettext`, `pybabel extract`, and `xpot` all support this through the use of the `!-k` command-line switch. The choice of `!N_` here is totally arbitrary; it could have just as easily been `!MarkThisStringForTranslation`.

## Acknowledgements

The following people contributed code, feedback, design suggestions, previous implementations, and valuable experience to the creation of this module:

- Peter Funk
- James Henstridge
- Juan David Ibáñez Palomar
- Marc-André Lemburg
- Martin von Löwis
- François Pinard
- Barry Warsaw
- Gustavo Niemeyer

**Footnotes**

[^1]: The default locale directory is system dependent; for example, on Red Hat Linux it is `/usr/share/locale`, but on Solaris it is `/usr/lib/locale`. The `!gettext` module does not try to support these system dependent defaults; instead its default is `{sys.base_prefix}/share/locale` (see `sys.base_prefix`). For this reason, it is always best to call `bindtextdomain` with an explicit absolute path at the start of your application.

[^2]: See the footnote for `bindtextdomain` above.

# `!glob` --- Unix style pathname pattern expansion

<div class="module" data-synopsis="Unix shell style pathname pattern expansion.">

glob

</div>

**Source code:** `Lib/glob.py`

<div class="index">

single: filenames; pathname expansion

</div>

------------------------------------------------------------------------

<div class="index">

single: \* (asterisk); in glob-style wildcards single: ? (question mark); in glob-style wildcards single: \[\] (square brackets); in glob-style wildcards single: ! (exclamation); in glob-style wildcards single: - (minus); in glob-style wildcards single: . (dot); in glob-style wildcards

</div>

The `!glob` module finds pathnames using pattern matching rules similar to the Unix shell. No tilde expansion is done, but `*`, `?`, and character ranges expressed with `[]` will be correctly matched. This is done by using the `os.scandir` and `fnmatch.fnmatch` functions in concert, and not by actually invoking a subshell.

> [!NOTE]
> The pathnames are returned in no particular order. If you need a specific order, sort the results.

By default, files beginning with a dot (`.`) can only be matched by patterns that also start with a dot, unlike `fnmatch.fnmatch` or `pathlib.Path.glob`. For tilde and shell variable expansion, use `os.path.expanduser` and `os.path.expandvars`.

For a literal match, wrap the meta-characters in brackets. For example, `'[?]'` matches the character `'?'`.

The `!glob` module defines the following functions:

<div class="function">

glob(pathname, \*, root_dir=None, dir_fd=None, recursive=False, include_hidden=False)

Return a possibly empty list of path names that match *pathname*, which must be a string containing a path specification. *pathname* can be either absolute (like `/usr/src/Python-1.5/Makefile`) or relative (like `../../Tools/\*/\*.gif`), and can contain shell-style wildcards. Broken symlinks are included in the results (as in the shell). Whether or not the results are sorted depends on the file system. If a file that satisfies conditions is removed or added during the call of this function, whether a path name for that file will be included is unspecified.

If *root_dir* is not `None`, it should be a `path-like object` specifying the root directory for searching. It has the same effect on `!glob` as changing the current directory before calling it. If *pathname* is relative, the result will contain paths relative to *root_dir*.

This function can support `paths relative to directory descriptors
<dir_fd>` with the *dir_fd* parameter.

<div class="index">

single: \*\*; in glob-style wildcards

</div>

If *recursive* is true, the pattern "`**`" will match any files and zero or more directories, subdirectories and symbolic links to directories. If the pattern is followed by an `os.sep` or `os.altsep` then files will not match.

If *include_hidden* is true, wildcards can match path segments that begin with a dot (`.`).

<div class="audit-event">

glob.glob pathname,recursive glob.glob

</div>

<div class="audit-event">

glob.glob/2 pathname,recursive,root_dir,dir_fd glob.glob

</div>

> [!NOTE]
> Using the "`**`" pattern in large directory trees may consume an inordinate amount of time.

> [!NOTE]
> This function may return duplicate path names if *pathname* contains multiple "`**`" patterns and *recursive* is true.

> [!NOTE]
> Any `OSError` exceptions raised from scanning the filesystem are suppressed. This includes `PermissionError` when accessing directories without read permission.

<div class="versionchanged">

3.5 Support for recursive globs using "`**`".

</div>

<div class="versionchanged">

3.10 Added the *root_dir* and *dir_fd* parameters.

</div>

<div class="versionchanged">

3.11 Added the *include_hidden* parameter.

</div>

</div>

<div class="function">

iglob(pathname, \*, root_dir=None, dir_fd=None, recursive=False, include_hidden=False)

Return an `iterator` which yields the same values as `~glob.glob` without actually storing them all simultaneously.

<div class="audit-event">

glob.glob pathname,recursive glob.iglob

</div>

<div class="audit-event">

glob.glob/2 pathname,recursive,root_dir,dir_fd glob.iglob

</div>

> [!NOTE]
> This function may return duplicate path names if *pathname* contains multiple "`**`" patterns and *recursive* is true.

> [!NOTE]
> Any `OSError` exceptions raised from scanning the filesystem are suppressed. This includes `PermissionError` when accessing directories without read permission.

<div class="versionchanged">

3.5 Support for recursive globs using "`**`".

</div>

<div class="versionchanged">

3.10 Added the *root_dir* and *dir_fd* parameters.

</div>

<div class="versionchanged">

3.11 Added the *include_hidden* parameter.

</div>

</div>

<div class="function">

escape(pathname)

Escape all special characters (`'?'`, `'*'` and `'['`). This is useful if you want to match an arbitrary literal string that may have special characters in it. Special characters in drive/UNC sharepoints are not escaped, for example on Windows `escape('//?/c:/Quo vadis?.txt')` returns `'//?/c:/Quo vadis[?].txt'`.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

translate(pathname, \*, recursive=False, include_hidden=False, seps=None)

Convert the given path specification to a regular expression for use with `re.prefixmatch`. The path specification can contain shell-style wildcards.

For example:

> \>\>\> import glob, re \>\>\> \>\>\> regex = glob.translate('\*\*/\*.txt', recursive=True, include_hidden=True) \>\>\> regex '(?s:(?:.+/)?\[^/\]\*\\.txt)\z' \>\>\> reobj = re.compile(regex) \>\>\> reobj.prefixmatch('foo/bar/baz.txt') \<re.Match object; span=(0, 15), match='foo/bar/baz.txt'\>

Path separators and segments are meaningful to this function, unlike `fnmatch.translate`. By default wildcards do not match path separators, and `*` pattern segments match precisely one path segment.

If *recursive* is true, the pattern segment "`**`" will match any number of path segments.

If *include_hidden* is true, wildcards can match path segments that start with a dot (`.`).

A sequence of path separators may be supplied to the *seps* argument. If not given, `os.sep` and `~os.altsep` (if available) are used.

<div class="seealso">

`pathlib.PurePath.full_match` and `pathlib.Path.glob` methods, which call this function to implement pattern matching and globbing.

</div>

<div class="versionadded">

3.13

</div>

</div>

## Examples

Consider a directory containing the following files: `1.gif`, `2.txt`, `card.gif` and a subdirectory `sub` which contains only the file `3.txt`. `~glob.glob` will produce the following results. Notice how any leading components of the path are preserved. :

    >>> import glob
    >>> glob.glob('./[0-9].*')
    ['./1.gif', './2.txt']
    >>> glob.glob('*.gif')
    ['1.gif', 'card.gif']
    >>> glob.glob('?.gif')
    ['1.gif']
    >>> glob.glob('**/*.txt', recursive=True)
    ['2.txt', 'sub/3.txt']
    >>> glob.glob('./**/', recursive=True)
    ['./', './sub/']

If the directory contains files starting with `.` they won't be matched by default. For example, consider a directory containing `card.gif` and `.card.gif`:

    >>> import glob
    >>> glob.glob('*.gif')
    ['card.gif']
    >>> glob.glob('.c*')
    ['.card.gif']

<div class="seealso">

The `fnmatch` module offers shell-style filename (not path) expansion.

</div>

<div class="seealso">

The `pathlib` module offers high-level path objects.

</div>

# `!graphlib` --- Functionality to operate with graph-like structures

<div class="module" data-synopsis="Functionality to operate with graph-like structures">

graphlib

</div>

**Source code:** `Lib/graphlib.py`

<div class="testsetup">

default

import graphlib from graphlib import \*

</div>

------------------------------------------------------------------------

<div class="TopologicalSorter(graph=None)">

Provides functionality to topologically sort a graph of `hashable` nodes.

A topological order is a linear ordering of the vertices in a graph such that for every directed edge u -\> v from vertex u to vertex v, vertex u comes before vertex v in the ordering. For instance, the vertices of the graph may represent tasks to be performed, and the edges may represent constraints that one task must be performed before another; in this example, a topological ordering is just a valid sequence for the tasks. A complete topological ordering is possible if and only if the graph has no directed cycles, that is, if it is a directed acyclic graph.

If the optional *graph* argument is provided it must be a dictionary representing a directed acyclic graph where the keys are nodes and the values are iterables of all predecessors of that node in the graph (the nodes that have edges that point to the value in the key). Additional nodes can be added to the graph using the `~TopologicalSorter.add` method.

In the general case, the steps required to perform the sorting of a given graph are as follows:

- Create an instance of the `TopologicalSorter` with an optional initial graph.
- Add additional nodes to the graph.
- Call `~TopologicalSorter.prepare` on the graph.
- While `~TopologicalSorter.is_active` is `True`, iterate over the nodes returned by `~TopologicalSorter.get_ready` and process them. Call `~TopologicalSorter.done` on each node as it finishes processing.

In case just an immediate sorting of the nodes in the graph is required and no parallelism is involved, the convenience method `TopologicalSorter.static_order` can be used directly:

<div class="doctest">

\>\>\> graph = {"D": {"B", "C"}, "C": {"A"}, "B": {"A"}} \>\>\> ts = TopologicalSorter(graph) \>\>\> tuple(ts.static_order()) ('A', 'C', 'B', 'D')

</div>

The class is designed to easily support parallel processing of the nodes as they become ready. For instance:

    topological_sorter = TopologicalSorter()

    # Add nodes to 'topological_sorter'...

    topological_sorter.prepare()
    while topological_sorter.is_active():
        for node in topological_sorter.get_ready():
            # Worker threads or processes take nodes to work on off the
            # 'task_queue' queue.
            task_queue.put(node)

        # When the work for a node is done, workers put the node in
        # 'finalized_tasks_queue' so we can get more nodes to work on.
        # The definition of 'is_active()' guarantees that, at this point, at
        # least one node has been placed on 'task_queue' that hasn't yet
        # been passed to 'done()', so this blocking 'get()' must (eventually)
        # succeed.  After calling 'done()', we loop back to call 'get_ready()'
        # again, so put newly freed nodes on 'task_queue' as soon as
        # logically possible.
        node = finalized_tasks_queue.get()
        topological_sorter.done(node)

<div class="method">

add(node, \*predecessors)

Add a new node and its predecessors to the graph. Both the *node* and all elements in *predecessors* must be `hashable`.

If called multiple times with the same node argument, the set of dependencies will be the union of all dependencies passed in.

It is possible to add a node with no dependencies (*predecessors* is not provided) or to provide a dependency twice. If a node that has not been provided before is included among *predecessors* it will be automatically added to the graph with no predecessors of its own.

Raises `ValueError` if called after `~TopologicalSorter.prepare`.

</div>

<div class="method">

prepare()

Mark the graph as finished and check for cycles in the graph. If any cycle is detected, `CycleError` will be raised, but `~TopologicalSorter.get_ready` can still be used to obtain as many nodes as possible until cycles block more progress. After a call to this function, the graph cannot be modified, and therefore no more nodes can be added using `~TopologicalSorter.add`.

A `ValueError` will be raised if the sort has been started by `~.static_order` or `~.get_ready`.

<div class="versionchanged">

3.14

`prepare()` can now be called more than once as long as the sort has not started. Previously this raised `ValueError`.

</div>

</div>

<div class="method">

is_active()

Returns `True` if more progress can be made and `False` otherwise. Progress can be made if cycles do not block the resolution and either there are still nodes ready that haven't yet been returned by `TopologicalSorter.get_ready` or the number of nodes marked `TopologicalSorter.done` is less than the number that have been returned by `TopologicalSorter.get_ready`.

The `~object.__bool__` method of this class defers to this function, so instead of:

    if ts.is_active():
        ...

it is possible to simply do:

    if ts:
        ...

Raises `ValueError` if called without calling `~TopologicalSorter.prepare` previously.

</div>

<div class="method">

done(\*nodes)

Marks a set of nodes returned by `TopologicalSorter.get_ready` as processed, unblocking any successor of each node in *nodes* for being returned in the future by a call to `TopologicalSorter.get_ready`.

Raises `ValueError` if any node in *nodes* has already been marked as processed by a previous call to this method or if a node was not added to the graph by using `TopologicalSorter.add`, if called without calling `~TopologicalSorter.prepare` or if node has not yet been returned by `~TopologicalSorter.get_ready`.

</div>

<div class="method">

get_ready()

Returns a `tuple` with all the nodes that are ready. Initially it returns all nodes with no predecessors, and once those are marked as processed by calling `TopologicalSorter.done`, further calls will return all new nodes that have all their predecessors already processed. Once no more progress can be made, empty tuples are returned.

Raises `ValueError` if called without calling `~TopologicalSorter.prepare` previously.

</div>

<div class="method">

static_order()

Returns an iterator object which will iterate over nodes in a topological order. When using this method, `~TopologicalSorter.prepare` and `~TopologicalSorter.done` should not be called. This method is equivalent to:

    def static_order(self):
        self.prepare()
        while self.is_active():
            node_group = self.get_ready()
            yield from node_group
            self.done(*node_group)

The particular order that is returned may depend on the specific order in which the items were inserted in the graph. For example:

<div class="doctest">

\>\>\> ts = TopologicalSorter() \>\>\> ts.add(3, 2, 1) \>\>\> ts.add(1, 0) \>\>\> print(\[\*ts.static_order()\]) \[2, 0, 1, 3\]

\>\>\> ts2 = TopologicalSorter() \>\>\> ts2.add(1, 0) \>\>\> ts2.add(3, 2, 1) \>\>\> print(\[\*ts2.static_order()\]) \[0, 2, 1, 3\]

</div>

This is due to the fact that "0" and "2" are in the same level in the graph (they would have been returned in the same call to `~TopologicalSorter.get_ready`) and the order between them is determined by the order of insertion.

If any cycle is detected, `CycleError` will be raised.

</div>

<div class="versionadded">

3.9

</div>

</div>

## Exceptions

The `!graphlib` module defines the following exception classes:

<div class="exception">

CycleError

Subclass of `ValueError` raised by `TopologicalSorter.prepare` if cycles exist in the working graph. If multiple cycles exist, only one undefined choice among them will be reported and included in the exception.

The detected cycle can be accessed via the second element in the `~BaseException.args` attribute of the exception instance and consists in a list of nodes, such that each node is, in the graph, an immediate predecessor of the next node in the list. In the reported list, the first and the last node will be the same, to make it clear that it is cyclic.

</div>

# `!grp` --- The group database

<div class="module" data-synopsis="The group database (getgrnam() and friends).">

grp

</div>

------------------------------------------------------------------------

This module provides access to the Unix group database. It is available on all Unix versions.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

Group database entries are reported as a tuple-like object, whose attributes correspond to the members of the `group` structure (Attribute field below, see `<grp.h>`):

| Index | Attribute | Meaning                                     |
|-------|-----------|---------------------------------------------|
| 0     | gr_name   | the name of the group                       |
| 1     | gr_passwd | the (encrypted) group password; often empty |
| 2     | gr_gid    | the numerical group ID                      |
| 3     | gr_mem    | all the group member's user names           |

The gid is an integer, name and password are strings, and the member list is a list of strings. (Note that most users are not explicitly listed as members of the group they are in according to the password database. Check both databases to get complete membership information. Also note that a `gr_name` that starts with a `+` or `-` is likely to be a YP/NIS reference and may not be accessible via `getgrnam` or `getgrgid`.)

It defines the following items:

<div class="function">

getgrgid(id)

Return the group database entry for the given numeric group ID. `KeyError` is raised if the entry asked for cannot be found.

<div class="versionchanged">

3.10 `TypeError` is raised for non-integer arguments like floats or strings.

</div>

</div>

<div class="function">

getgrnam(name)

Return the group database entry for the given group name. `KeyError` is raised if the entry asked for cannot be found.

</div>

<div class="function">

getgrall()

Return a list of all available group entries, in arbitrary order.

</div>

<div class="seealso">

Module `pwd`  
An interface to the user database, similar to this.

</div>

# `!gzip` --- Support for `gzip` files

<div class="module" data-synopsis="Interfaces for gzip compression and decompression using file objects.">

gzip

</div>

**Source code:** `Lib/gzip.py`

------------------------------------------------------------------------

This module provides a simple interface to compress and decompress files just like the GNU programs `gzip` and `gunzip` would.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

The data compression is provided by the `zlib` module.

The `!gzip` module provides the `GzipFile` class, as well as the `.open`, `compress` and `decompress` convenience functions. The `GzipFile` class reads and writes `gzip`-format files, automatically compressing or decompressing the data so that it looks like an ordinary `file object`.

Note that additional file formats which can be decompressed by the `gzip` and `gunzip` programs, such as those produced by `compress` and `pack`, are not supported by this module.

The module defines the following items:

<div class="function">

open(filename, mode='rb', compresslevel=6, encoding=None, errors=None, newline=None, \*, mtime=None)

Open a gzip-compressed file in binary or text mode, returning a `file
object`.

The *filename* argument can be an actual filename (a `str` or `bytes` object), or an existing file object to read from or write to.

The *mode* argument can be any of `'r'`, `'rb'`, `'a'`, `'ab'`, `'w'`, `'wb'`, `'x'` or `'xb'` for binary mode, or `'rt'`, `'at'`, `'wt'`, or `'xt'` for text mode. The default is `'rb'`.

The *compresslevel* argument is an integer from 0 to 9, as for the `GzipFile` constructor.

The keyword-only argument *mtime* represents a Unix timestamp.

For binary mode, this function is equivalent to the `GzipFile` constructor: `GzipFile(filename, mode, compresslevel, mtime=mtime)`. In this case, the *encoding*, *errors* and *newline* arguments must not be provided.

For text mode, a `GzipFile` object is created, and wrapped in an `io.TextIOWrapper` instance with the specified encoding, error handling behavior, and line ending(s).

<div class="versionchanged">

3.3 Added support for *filename* being a file object, support for text mode, and the *encoding*, *errors* and *newline* arguments.

</div>

<div class="versionchanged">

3.4 Added support for the `'x'`, `'xb'` and `'xt'` modes.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.15 The default compression level was reduced to 6 (down from 9). It is the default level used by most compression tools and a better tradeoff between speed and performance.

</div>

<div class="versionchanged">

3.16 Added keyword-only argument *mtime* which is passed to the class constructor of `~gzip.GzipFile`.

</div>

</div>

<div class="exception">

BadGzipFile

An exception raised for invalid gzip files. It inherits from `OSError`. `EOFError` and `zlib.error` can also be raised for invalid gzip files.

<div class="versionadded">

3.8

</div>

</div>

<div class="GzipFile(filename=None, mode=None, compresslevel=6, fileobj=None, mtime=None)">

Constructor for the `GzipFile` class, which simulates most of the methods of a `file object`, with the exception of the `~io.IOBase.truncate` method. At least one of *fileobj* and *filename* must be given a non-trivial value.

The new class instance is based on *fileobj*, which can be a regular file, an `io.BytesIO` object, or any other object which simulates a file. It defaults to `None`, in which case *filename* is opened to provide a file object.

When *fileobj* is not `None`, the *filename* argument is only used to be included in the `gzip` file header, which may include the original filename of the uncompressed file. It defaults to the filename of *fileobj*, if discernible; otherwise, it defaults to the empty string, and in this case the original filename is not included in the header.

The *mode* argument can be any of `'r'`, `'rb'`, `'a'`, `'ab'`, `'w'`, `'wb'`, `'x'`, or `'xb'`, depending on whether the file will be read or written. The default is the mode of *fileobj* if discernible; otherwise, the default is `'rb'`. In future Python releases the mode of *fileobj* will not be used. It is better to always specify *mode* for writing.

Note that the file is always opened in binary mode. To open a compressed file in text mode, use `.open` (or wrap your `GzipFile` with an `io.TextIOWrapper`).

The *compresslevel* argument is an integer from `0` to `9` controlling the level of compression; `1` is fastest and produces the least compression, and `9` is slowest and produces the most compression. `0` is no compression. The default is `6`.

The optional *mtime* argument is the timestamp requested by gzip. The time is in Unix format, i.e., seconds since 00:00:00 UTC, January 1, 1970. Set *mtime* to `0` to generate a compressed stream that does not depend on creation time. If *mtime* is omitted or `None`, the current time is used; however, if the current time is outside the range 00:00:00 UTC, January 1, 1970 through 06:28:15 UTC, February 7, 2106, or explicitly passed *mtime* argument is outside the range `0` to `2**32-1`, then the value `0` is used instead.

See below for the `mtime` attribute that is set when decompressing.

Calling a `GzipFile` object's `!close` method does not close *fileobj*, since you might wish to append more material after the compressed data. This also allows you to pass an `io.BytesIO` object opened for writing as *fileobj*, and retrieve the resulting memory buffer using the `io.BytesIO` object's `~io.BytesIO.getvalue` method.

`GzipFile` supports the `io.BufferedIOBase` interface, including iteration and the `with` statement. Only the `~io.IOBase.truncate` method isn't implemented.

`GzipFile` also provides the following method and attribute:

<div class="method">

peek(n)

Read *n* uncompressed bytes without advancing the file position. The number of bytes returned may be more or less than requested.

> [!NOTE]
> While calling `peek` does not change the file position of the `GzipFile`, it may change the position of the underlying file object (e.g. if the `GzipFile` was constructed with the *fileobj* parameter).

<div class="versionadded">

3.2

</div>

</div>

<div class="attribute">

mode

`'rb'` for reading and `'wb'` for writing.

<div class="versionchanged">

3.13 In previous versions it was an integer `1` or `2`.

</div>

</div>

<div class="attribute">

mtime

When decompressing, this attribute is set to the last timestamp in the most recently read header. It is an integer, holding the number of seconds since the Unix epoch (00:00:00 UTC, January 1, 1970). The initial value before reading any headers is `None`.

</div>

<div class="attribute">

name

The path to the gzip file on disk, as a `str` or `bytes`. Equivalent to the output of `os.fspath` on the original input path, with no other normalization, resolution or expansion.

</div>

<div class="versionchanged">

3.1 Support for the `with` statement was added, along with the *mtime* constructor argument and `mtime` attribute.

</div>

<div class="versionchanged">

3.2 Support for zero-padded and unseekable files was added.

</div>

<div class="versionchanged">

3.3 The `io.BufferedIOBase.read1` method is now implemented.

</div>

<div class="versionchanged">

3.4 Added support for the `'x'` and `'xb'` modes.

</div>

<div class="versionchanged">

3.5 Added support for writing arbitrary `bytes-like objects <bytes-like object>`. The `~io.BufferedIOBase.read` method now accepts an argument of `None`.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="deprecated">

3.9 Opening `GzipFile` for writing without specifying the *mode* argument is deprecated.

</div>

<div class="versionchanged">

3.12 Remove the `filename` attribute, use the `~GzipFile.name` attribute instead.

</div>

<div class="versionchanged">

3.15 The default compression level was reduced to 6 (down from 9). It is the default level used by most compression tools and a better tradeoff between speed and performance.

</div>

</div>

<div class="function">

compress(data, compresslevel=6, \*, mtime=0)

Compress the *data*, returning a `bytes` object containing the compressed data. *compresslevel* and *mtime* have the same meaning as in the `GzipFile` constructor above, but *mtime* defaults to 0 for reproducible output.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.8 Added the *mtime* parameter for reproducible output.

</div>

<div class="versionchanged">

3.11 Speed is improved by compressing all data at once instead of in a streamed fashion. Calls with *mtime* set to `0` are delegated to `zlib.compress` for better speed. In this situation the output may contain a gzip header "OS" byte value other than 255 "unknown" as supplied by the underlying zlib implementation.

</div>

<div class="versionchanged">

3.13 The gzip header OS byte is guaranteed to be set to 255 when this function is used as was the case in 3.10 and earlier.

</div>

<div class="versionchanged">

3.14 The *mtime* parameter now defaults to 0 for reproducible output. For the previous behaviour of using the current time, pass `None` to *mtime*.

</div>

<div class="versionchanged">

3.15 The default compression level was reduced to 6 (down from 9). It is the default level used by most compression tools and a better tradeoff between speed and performance.

</div>

</div>

<div class="function">

decompress(data)

Decompress the *data*, returning a `bytes` object containing the uncompressed data. This function is capable of decompressing multi-member gzip data (multiple gzip blocks concatenated together). When the data is certain to contain only one member the `zlib.decompress` function with *wbits* set to 31 is faster.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.11 Speed is improved by decompressing members at once in memory instead of in a streamed fashion.

</div>

</div>

## Examples of usage

Example of how to read a compressed file:

    import gzip
    with gzip.open('/home/joe/file.txt.gz', 'rb') as f:
        file_content = f.read()

Example of how to create a compressed GZIP file:

    import gzip
    content = b"Lots of content here"
    with gzip.open('/home/joe/file.txt.gz', 'wb') as f:
        f.write(content)

Example of how to GZIP compress an existing file:

    import gzip
    import shutil
    with open('/home/joe/file.txt', 'rb') as f_in:
        with gzip.open('/home/joe/file.txt.gz', 'wb') as f_out:
            shutil.copyfileobj(f_in, f_out)

Example of how to GZIP compress a binary string:

    import gzip
    s_in = b"Lots of content here"
    s_out = gzip.compress(s_in)

<div class="seealso">

Module `zlib`  
The basic data compression module needed to support the `gzip` file format.

In case gzip (de)compression is a bottleneck, the [python-isal](https://github.com/pycompression/python-isal) package speeds up (de)compression with a mostly compatible API.

</div>

<div class="program">

gzip

</div>

## Command-line interface

The `!gzip` module provides a simple command line interface to compress or decompress files.

Once executed the `!gzip` module keeps the input file(s).

<div class="versionchanged">

3.8

Add a new command line interface with a usage. By default, when you will execute the CLI, the default compression level is 6.

</div>

### Command-line options

<div class="option">

file

If *file* is not specified, read from `sys.stdin`.

</div>

<div class="option">

`--fast`  

Indicates the fastest compression method (less compression).

</div>

<div class="option">

`--best`  

Indicates the slowest compression method (best compression).

</div>

<div class="option">

`-d, --decompress`  

Decompress the given file.

</div>

<div class="option">

`-h, --help`  

Show the help message.

</div>

# `!hashlib` --- Secure hashes and message digests

<div class="module" data-synopsis="Secure hash and message digest algorithms.">

hashlib

</div>

**Source code:** `Lib/hashlib.py`

<div class="index">

single: message digest, MD5 single: secure hash algorithm, SHA1, SHA2, SHA224, SHA256, SHA384, SHA512, SHA3, Shake, Blake2

</div>

<div class="testsetup">

import hashlib

</div>

------------------------------------------------------------------------

This module implements a common interface to many different hash algorithms. Included are the FIPS secure hash algorithms SHA224, SHA256, SHA384, SHA512, (defined in [the FIPS 180-4 standard](https://csrc.nist.gov/pubs/fips/180-4/upd1/final)), the SHA-3 series (defined in [the FIPS 202 standard](https://csrc.nist.gov/pubs/fips/202/final)) as well as the legacy algorithms SHA1 ([formerly part of FIPS](https://csrc.nist.gov/news/2023/decision-to-revise-fips-180-4)) and the MD5 algorithm (defined in internet `1321`).

> [!NOTE]
> If you want the adler32 or crc32 hash functions, they are available in the `zlib` module.

## Hash algorithms

There is one constructor method named for each type of `hash`. All return a hash object with the same simple interface. For example: use `sha256` to create a SHA-256 hash object. You can now feed this object with `bytes-like objects <bytes-like object>` (normally `bytes`) using the `update<hash.update>` method. At any point you can ask it for the `digest` of the concatenation of the data fed to it so far using the `digest()<hash.digest>` or `hexdigest()<hash.hexdigest>` methods.

To allow multithreading, the Python `GIL` is released while computing a hash supplied more than 2047 bytes of data at once in its constructor or `.update<hash.update>` method.

<div class="index">

single: OpenSSL; (use in module hashlib)

</div>

Constructors for hash algorithms that are always present in this module are `md5`, `sha1`, `sha224`, `sha256`, `sha384`, `sha512`, `sha3_224`, `sha3_256`, `sha3_384`, `sha3_512`, `shake_128`, `shake_256`, `blake2b`, and `blake2s`. These correspond to `algorithms_guaranteed`.

Any of these may nonetheless be missing or blocked in unusual environments, such as a rare "FIPS compliant" build of Python or when OpenSSL's "FIPS mode" is configured to exclude some algorithms from its default provider. Calling the constructor of an algorithm that is unavailable raises `ValueError`.

Additional algorithms may also be available if your Python distribution's `!hashlib` was linked against a build of OpenSSL that provides others. Others *are not guaranteed available* on all installations and will only be accessible by name via `new`. See `algorithms_available`.

> [!WARNING]
> Some algorithms have known hash collision weaknesses (including MD5 and SHA1). Refer to [Attacks on cryptographic hash algorithms](https://en.wikipedia.org/wiki/Cryptographic_hash_function#Attacks_on_cryptographic_hash_algorithms) and the [hashlib-seealso](#hashlib-seealso) section at the end of this document.

<div class="versionadded">

3.6 SHA3 (Keccak) and SHAKE constructors `sha3_224`, `sha3_256`, `sha3_384`, `sha3_512`, `shake_128`, `shake_256` were added. `blake2b` and `blake2s` were added.

</div>

<div id="hashlib-usedforsecurity">

<div class="versionchanged">

3.9 All hashlib constructors take a keyword-only argument *usedforsecurity* with default value `True`. A false value allows the use of insecure and blocked hashing algorithms in restricted environments. `False` indicates that the hashing algorithm is not used in a security context, e.g. as a non-cryptographic one-way compression function.

</div>

</div>

<div class="versionchanged">

3.9 Hashlib now uses SHA3 and SHAKE from OpenSSL if it provides it.

</div>

<div class="versionchanged">

3.12 For any of the MD5, SHA1, SHA2, or SHA3 algorithms that the linked OpenSSL does not provide we fall back to a verified implementation from the [HACL\\\* project]().

</div>

<div class="deprecated-removed">

3.15 3.19 The undocumented `string` keyword parameter in `!_hashlib.new` and hash-named constructors such as `!_md5.md5` is deprecated. Prefer passing the initial data as a positional argument for maximum backwards compatibility.

</div>

## Usage

To obtain the digest of the byte string `b"Nobody inspects the spammish repetition"`:

    >>> import hashlib
    >>> m = hashlib.sha256()
    >>> m.update(b"Nobody inspects")
    >>> m.update(b" the spammish repetition")
    >>> m.digest()
    b'\x03\x1e\xdd}Ae\x15\x93\xc5\xfe\\\x00o\xa5u+7\xfd\xdf\xf7\xbcN\x84:\xa6\xaf\x0c\x95\x0fK\x94\x06'
    >>> m.hexdigest()
    '031edd7d41651593c5fe5c006fa5752b37fddff7bc4e843aa6af0c950f4b9406'

More condensed:

> \>\>\> hashlib.sha256(b"Nobody inspects the spammish repetition").hexdigest() '031edd7d41651593c5fe5c006fa5752b37fddff7bc4e843aa6af0c950f4b9406'

## Constructors

<div class="function">

new(name\[, data\], \*, usedforsecurity=True)

Is a generic constructor that takes the string *name* of the desired algorithm as its first parameter. It also exists to allow access to the above listed hashes as well as any other algorithms that your OpenSSL library may offer.

</div>

Using `new` with an algorithm name:

> \>\>\> h = hashlib.new('sha256') \>\>\> h.update(b"Nobody inspects the spammish repetition") \>\>\> h.hexdigest() '031edd7d41651593c5fe5c006fa5752b37fddff7bc4e843aa6af0c950f4b9406'

<div class="function">

md5(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

sha1(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

sha224(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

sha256(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

sha384(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

sha512(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

sha3_224(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

sha3_256(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

sha3_384(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

sha3_512(\[, data\], \*, usedforsecurity=True)

</div>

Named constructors such as these are faster than passing an algorithm name to `new`.

## Attributes

Hashlib provides the following constant module attributes:

<div class="data">

algorithms_guaranteed

A set containing the names of the hash algorithms guaranteed to be supported by this module on all platforms. Note that 'md5' is in this list despite some upstream vendors offering an odd "FIPS compliant" Python build that excludes it.

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

algorithms_available

A set containing the names of the hash algorithms that are available in the running Python interpreter. These names will be recognized when passed to `new`. `algorithms_guaranteed` will always be a subset. The same algorithm may appear multiple times in this set under different names (thanks to OpenSSL).

<div class="versionadded">

3.2

</div>

</div>

## Hash Objects

The following values are provided as constant attributes of the hash objects returned by the constructors:

<div class="data">

hash.digest_size

The size of the resulting hash in bytes.

</div>

<div class="data">

hash.block_size

The internal block size of the hash algorithm in bytes.

</div>

A hash object has the following attributes:

<div class="attribute">

hash.name

The canonical name of this hash, always lowercase and always suitable as a parameter to `new` to create another hash of this type.

<div class="versionchanged">

3.4 The name attribute has been present in CPython since its inception, but until Python 3.4 was not formally specified, so may not exist on some platforms.

</div>

</div>

A hash object has the following methods:

<div class="method">

hash.update(data)

Update the hash object with the `bytes-like object`. Repeated calls are equivalent to a single call with the concatenation of all the arguments: `m.update(a); m.update(b)` is equivalent to `m.update(a+b)`.

</div>

<div class="method">

hash.digest()

Return the digest of the data passed to the `update` method so far. This is a bytes object of size `digest_size` which may contain bytes in the whole range from 0 to 255.

</div>

<div class="method">

hash.hexdigest()

Like `digest` except the digest is returned as a string object of double length, containing only hexadecimal digits. This may be used to exchange the value safely in email or other non-binary environments.

</div>

<div class="method">

hash.copy()

Return a copy ("clone") of the hash object. This can be used to efficiently compute the digests of data sharing a common initial substring.

</div>

## SHAKE variable length digests

<div class="function">

shake_128(\[, data\], \*, usedforsecurity=True)

</div>

<div class="function">

shake_256(\[, data\], \*, usedforsecurity=True)

</div>

The `shake_128` and `shake_256` algorithms provide variable length digests with length_in_bits//2 up to 128 or 256 bits of security. As such, their digest methods require a length. Maximum length is not limited by the SHAKE algorithm.

<div class="method">

shake.digest(length)

Return the digest of the data passed to the `~hash.update` method so far. This is a bytes object of size *length* which may contain bytes in the whole range from 0 to 255.

</div>

<div class="method">

shake.hexdigest(length)

Like `digest` except the digest is returned as a string object of double length, containing only hexadecimal digits. This may be used to exchange the value in email or other non-binary environments.

</div>

Example use:

> \>\>\> h = hashlib.shake_256(b'Nobody inspects the spammish repetition') \>\>\> h.hexdigest(20) '44709d6fcb83d92a76dcb0b668c98e1b1d3dafe7'

## File hashing

The hashlib module provides a helper function for efficient hashing of a file or file-like object.

<div class="function">

file_digest(fileobj, digest, /)

Return a digest object that has been updated with contents of file object.

*fileobj* must be a file-like object opened for reading in binary mode. It accepts file objects from builtin `open`, `~io.BytesIO` instances, SocketIO objects from `socket.socket.makefile`, and similar. *fileobj* must be opened in blocking mode, otherwise a `BlockingIOError` may be raised.

The function may bypass Python's I/O and use the file descriptor from `~io.IOBase.fileno` directly. *fileobj* must be assumed to be in an unknown state after this function returns or raises. It is up to the caller to close *fileobj*.

*digest* must either be a hash algorithm name as a *str*, a hash constructor, or a callable that returns a hash object.

Example:

> \>\>\> import io, hashlib, hmac \>\>\> with open("library/hashlib.rst", "rb") as f: ... digest = hashlib.file_digest(f, "sha256") ... \>\>\> digest.hexdigest() \# doctest: +ELLIPSIS '...'
>
> \>\>\> buf = io.BytesIO(b"somedata") \>\>\> mac1 = hmac.HMAC(b"key", digestmod=hashlib.sha512) \>\>\> digest = hashlib.file_digest(buf, lambda: mac1)
>
> \>\>\> digest is mac1 True \>\>\> mac2 = hmac.HMAC(b"key", b"somedata", digestmod=hashlib.sha512) \>\>\> mac1.digest() == mac2.digest() True

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.14 Now raises a `BlockingIOError` if the file is opened in non-blocking mode. Previously, spurious null bytes were added to the digest.

</div>

</div>

## Key derivation

Key derivation and key stretching algorithms are designed for secure password hashing. Naive algorithms such as `sha1(password)` are not resistant against brute-force attacks. A good password hashing function must be tunable, slow, and include a [salt](https://en.wikipedia.org/wiki/Salt_%28cryptography%29).

<div class="function">

pbkdf2_hmac(hash_name, password, salt, iterations, dklen=None)

The function provides PKCS#5 password-based key derivation function 2. It uses HMAC as pseudorandom function.

The string *hash_name* is the desired name of the hash digest algorithm for HMAC, e.g. 'sha1' or 'sha256'. *password* and *salt* are interpreted as buffers of bytes. Applications and libraries should limit *password* to a sensible length (e.g. 1024). *salt* should be about 16 or more bytes from a proper source, e.g. `os.urandom`.

The number of *iterations* should be chosen based on the hash algorithm and computing power. As of 2022, hundreds of thousands of iterations of SHA-256 are suggested. For rationale as to why and how to choose what is best for your application, read *Appendix A.2.2* of [NIST-SP-800-132](https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-132.pdf). The answers on the [stackexchange pbkdf2 iterations question](https://security.stackexchange.com/questions/3959/recommended-of-iterations-when-using-pbkdf2-sha256/) explain in detail.

*dklen* is the length of the derived key in bytes. If *dklen* is `None` then the digest size of the hash algorithm *hash_name* is used, e.g. 64 for SHA-512.

\>\>\> from hashlib import pbkdf2_hmac \>\>\> our_app_iters = 500_000 \# Application specific, read above. \>\>\> dk = pbkdf2_hmac('sha256', b'password', b'bad salt' \* 2, our_app_iters) \>\>\> dk.hex() '15530bba69924174860db778f2c6f8104d3aaf9d26241840c8c4a641c8d000a9'

Function only available when Python is compiled with OpenSSL.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.12 Function now only available when Python is built with OpenSSL. The slow pure Python implementation has been removed.

</div>

</div>

<div class="function">

scrypt(password, \*, salt, n, r, p, maxmem=0, dklen=64)

The function provides scrypt password-based key derivation function as defined in `7914`.

*password* and *salt* must be `bytes-like objects
<bytes-like object>`. Applications and libraries should limit *password* to a sensible length (e.g. 1024). *salt* should be about 16 or more bytes from a proper source, e.g. `os.urandom`.

*n* is the CPU/Memory cost factor, *r* the block size, *p* parallelization factor and *maxmem* limits memory (OpenSSL 1.1.0 defaults to 32 MiB). *dklen* is the length of the derived key in bytes.

<div class="versionadded">

3.6

</div>

</div>

## BLAKE2

<div class="index">

single: blake2b, blake2s

</div>

[BLAKE2](https://www.blake2.net) is a cryptographic hash function defined in `7693` that comes in two flavors:

- **BLAKE2b**, optimized for 64-bit platforms and produces digests of any size between 1 and 64 bytes,
- **BLAKE2s**, optimized for 8- to 32-bit platforms and produces digests of any size between 1 and 32 bytes.

BLAKE2 supports **keyed mode** (a faster and simpler replacement for [HMAC](https://en.wikipedia.org/wiki/Hash-based_message_authentication_code)), **salted hashing**, **personalization**, and **tree hashing**.

Hash objects from this module follow the API of standard library's `!hashlib` objects.

### Creating hash objects

New hash objects are created by calling constructor functions:

<div class="function">

blake2b(data=b'', \*, digest_size=64, key=b'', salt=b'', person=b'', fanout=1, depth=1, leaf_size=0, node_offset=0, node_depth=0, inner_size=0, last_node=False, usedforsecurity=True)

</div>

<div class="function">

blake2s(data=b'', \*, digest_size=32, key=b'', salt=b'', person=b'', fanout=1, depth=1, leaf_size=0, node_offset=0, node_depth=0, inner_size=0, last_node=False, usedforsecurity=True)

</div>

These functions return the corresponding hash objects for calculating BLAKE2b or BLAKE2s. They optionally take these general parameters:

- *data*: initial chunk of data to hash, which must be `bytes-like object`. It can be passed only as positional argument.
- *digest_size*: size of output digest in bytes.
- *key*: key for keyed hashing (up to 64 bytes for BLAKE2b, up to 32 bytes for BLAKE2s).
- *salt*: salt for randomized hashing (up to 16 bytes for BLAKE2b, up to 8 bytes for BLAKE2s).
- *person*: personalization string (up to 16 bytes for BLAKE2b, up to 8 bytes for BLAKE2s).

The following table shows limits for general parameters (in bytes):

| Hash    | digest_size | len(key) | len(salt) | len(person) |
|---------|-------------|----------|-----------|-------------|
| BLAKE2b | 64          | 64       | 16        | 16          |
| BLAKE2s | 32          | 32       | 8         | 8           |

> [!NOTE]
> BLAKE2 specification defines constant lengths for salt and personalization parameters, however, for convenience, this implementation accepts byte strings of any size up to the specified length. If the length of the parameter is less than specified, it is padded with zeros, thus, for example, `b'salt'` and `b'salt\x00'` is the same value. (This is not the case for *key*.)

These sizes are available as module [constants](#constants) described below.

Constructor functions also accept the following tree hashing parameters:

- *fanout*: fanout (0 to 255, 0 if unlimited, 1 in sequential mode).
- *depth*: maximal depth of tree (1 to 255, 255 if unlimited, 1 in sequential mode).
- *leaf_size*: maximal byte length of leaf (0 to `2**32-1`, 0 if unlimited or in sequential mode).
- *node_offset*: node offset (0 to `2**64-1` for BLAKE2b, 0 to `2**48-1` for BLAKE2s, 0 for the first, leftmost, leaf, or in sequential mode).
- *node_depth*: node depth (0 to 255, 0 for leaves, or in sequential mode).
- *inner_size*: inner digest size (0 to 64 for BLAKE2b, 0 to 32 for BLAKE2s, 0 in sequential mode).
- *last_node*: boolean indicating whether the processed node is the last one (`False` for sequential mode).

<figure class="invert-in-dark-mode">
<img src="hashlib-blake2-tree.png" class="invert-in-dark-mode" alt="Explanation of tree mode parameters." />
</figure>

See section 2.10 in [BLAKE2 specification](https://www.blake2.net/blake2_20130129.pdf) for comprehensive review of tree hashing.

### Constants

<div class="data">

blake2b.SALT_SIZE

</div>

<div class="data">

blake2s.SALT_SIZE

</div>

Salt length (maximum length accepted by constructors).

<div class="data">

blake2b.PERSON_SIZE

</div>

<div class="data">

blake2s.PERSON_SIZE

</div>

Personalization string length (maximum length accepted by constructors).

<div class="data">

blake2b.MAX_KEY_SIZE

</div>

<div class="data">

blake2s.MAX_KEY_SIZE

</div>

Maximum key size.

<div class="data">

blake2b.MAX_DIGEST_SIZE

</div>

<div class="data">

blake2s.MAX_DIGEST_SIZE

</div>

Maximum digest size that the hash function can output.

### Examples

#### Simple hashing

To calculate hash of some data, you should first construct a hash object by calling the appropriate constructor function (`blake2b` or `blake2s`), then update it with the data by calling `~hash.update` on the object, and, finally, get the digest out of the object by calling `~hash.digest` (or `~hash.hexdigest` for hex-encoded string).

> \>\>\> from hashlib import blake2b \>\>\> h = blake2b() \>\>\> h.update(b'Hello world') \>\>\> h.hexdigest() '6ff843ba685842aa82031d3f53c48b66326df7639a63d128974c5c14f31a0f33343a8c65551134ed1ae0f2b0dd2bb495dc81039e3eeb0aa1bb0388bbeac29183'

As a shortcut, you can pass the first chunk of data to update directly to the constructor as the positional argument:

> \>\>\> from hashlib import blake2b \>\>\> blake2b(b'Hello world').hexdigest() '6ff843ba685842aa82031d3f53c48b66326df7639a63d128974c5c14f31a0f33343a8c65551134ed1ae0f2b0dd2bb495dc81039e3eeb0aa1bb0388bbeac29183'

You can call `hash.update` as many times as you need to iteratively update the hash:

> \>\>\> from hashlib import blake2b \>\>\> items = \[b'Hello', b' ', b'world'\] \>\>\> h = blake2b() \>\>\> for item in items: ... h.update(item) ... \>\>\> h.hexdigest() '6ff843ba685842aa82031d3f53c48b66326df7639a63d128974c5c14f31a0f33343a8c65551134ed1ae0f2b0dd2bb495dc81039e3eeb0aa1bb0388bbeac29183'

#### Using different digest sizes

BLAKE2 has configurable size of digests up to 64 bytes for BLAKE2b and up to 32 bytes for BLAKE2s. For example, to replace SHA-1 with BLAKE2b without changing the size of output, we can tell BLAKE2b to produce 20-byte digests:

> \>\>\> from hashlib import blake2b \>\>\> h = blake2b(digest_size=20) \>\>\> h.update(b'Replacing SHA1 with the more secure function') \>\>\> h.hexdigest() 'd24f26cf8de66472d58d4e1b1774b4c9158b1f4c' \>\>\> h.digest_size 20 \>\>\> len(h.digest()) 20

Hash objects with different digest sizes have completely different outputs (shorter hashes are *not* prefixes of longer hashes); BLAKE2b and BLAKE2s produce different outputs even if the output length is the same:

> \>\>\> from hashlib import blake2b, blake2s \>\>\> blake2b(digest_size=10).hexdigest() '6fa1d8fcfd719046d762' \>\>\> blake2b(digest_size=11).hexdigest() 'eb6ec15daf9546254f0809' \>\>\> blake2s(digest_size=10).hexdigest() '1bf21a98c78a1c376ae9' \>\>\> blake2s(digest_size=11).hexdigest() '567004bf96e4a25773ebf4'

#### Keyed hashing

Keyed hashing can be used for authentication as a faster and simpler replacement for [Hash-based message authentication code](https://en.wikipedia.org/wiki/HMAC) (HMAC). BLAKE2 can be securely used in prefix-MAC mode thanks to the indifferentiability property inherited from BLAKE.

This example shows how to get a (hex-encoded) 128-bit authentication code for message `b'message data'` with key `b'pseudorandom key'`:

    >>> from hashlib import blake2b
    >>> h = blake2b(key=b'pseudorandom key', digest_size=16)
    >>> h.update(b'message data')
    >>> h.hexdigest()
    '3d363ff7401e02026f4a4687d4863ced'

As a practical example, a web application can symmetrically sign cookies sent to users and later verify them to make sure they weren't tampered with:

    >>> from hashlib import blake2b
    >>> from hmac import compare_digest
    >>>
    >>> SECRET_KEY = b'pseudorandomly generated server secret key'
    >>> AUTH_SIZE = 16
    >>>
    >>> def sign(cookie):
    ...     h = blake2b(digest_size=AUTH_SIZE, key=SECRET_KEY)
    ...     h.update(cookie)
    ...     return h.hexdigest().encode('utf-8')
    >>>
    >>> def verify(cookie, sig):
    ...     good_sig = sign(cookie)
    ...     return compare_digest(good_sig, sig)
    >>>
    >>> cookie = b'user-alice'
    >>> sig = sign(cookie)
    >>> print("{0},{1}".format(cookie.decode('utf-8'), sig))
    user-alice,b'43b3c982cf697e0c5ab22172d1ca7421'
    >>> verify(cookie, sig)
    True
    >>> verify(b'user-bob', sig)
    False
    >>> verify(cookie, b'0102030405060708090a0b0c0d0e0f00')
    False

Even though there's a native keyed hashing mode, BLAKE2 can, of course, be used in HMAC construction with `hmac` module:

    >>> import hmac, hashlib
    >>> m = hmac.new(b'secret key', digestmod=hashlib.blake2s)
    >>> m.update(b'message')
    >>> m.hexdigest()
    'e3c8102868d28b5ff85fc35dda07329970d1a01e273c37481326fe0c861c8142'

#### Randomized hashing

By setting *salt* parameter users can introduce randomization to the hash function. Randomized hashing is useful for protecting against collision attacks on the hash function used in digital signatures.

> Randomized hashing is designed for situations where one party, the message preparer, generates all or part of a message to be signed by a second party, the message signer. If the message preparer is able to find cryptographic hash function collisions (i.e., two messages producing the same hash value), then they might prepare meaningful versions of the message that would produce the same hash value and digital signature, but with different results (e.g., transferring \$1,000,000 to an account, rather than \$10). Cryptographic hash functions have been designed with collision resistance as a major goal, but the current concentration on attacking cryptographic hash functions may result in a given cryptographic hash function providing less collision resistance than expected. Randomized hashing offers the signer additional protection by reducing the likelihood that a preparer can generate two or more messages that ultimately yield the same hash value during the digital signature generation process --- even if it is practical to find collisions for the hash function. However, the use of randomized hashing may reduce the amount of security provided by a digital signature when all portions of the message are prepared by the signer.
>
> ([NIST SP-800-106 "Randomized Hashing for Digital Signatures"](https://csrc.nist.gov/pubs/sp/800/106/final))

In BLAKE2 the salt is processed as a one-time input to the hash function during initialization, rather than as an input to each compression function.

> [!WARNING]
> *Salted hashing* (or just hashing) with BLAKE2 or any other general-purpose cryptographic hash function, such as SHA-256, is not suitable for hashing passwords. See [BLAKE2 FAQ](https://www.blake2.net/#qa) for more information.

> \>\>\> import os \>\>\> from hashlib import blake2b \>\>\> msg = b'some message' \>\>\> \# Calculate the first hash with a random salt. \>\>\> salt1 = os.urandom(blake2b.SALT_SIZE) \>\>\> h1 = blake2b(salt=salt1) \>\>\> h1.update(msg) \>\>\> \# Calculate the second hash with a different random salt. \>\>\> salt2 = os.urandom(blake2b.SALT_SIZE) \>\>\> h2 = blake2b(salt=salt2) \>\>\> h2.update(msg) \>\>\> \# The digests are different. \>\>\> h1.digest() != h2.digest() True

#### Personalization

Sometimes it is useful to force hash function to produce different digests for the same input for different purposes. Quoting the authors of the Skein hash function:

> We recommend that all application designers seriously consider doing this; we have seen many protocols where a hash that is computed in one part of the protocol can be used in an entirely different part because two hash computations were done on similar or related data, and the attacker can force the application to make the hash inputs the same. Personalizing each hash function used in the protocol summarily stops this type of attack.
>
> ([The Skein Hash Function Family](https://www.schneier.com/wp-content/uploads/2016/02/skein.pdf), p. 21)

BLAKE2 can be personalized by passing bytes to the *person* argument:

    >>> from hashlib import blake2b
    >>> FILES_HASH_PERSON = b'MyApp Files Hash'
    >>> BLOCK_HASH_PERSON = b'MyApp Block Hash'
    >>> h = blake2b(digest_size=32, person=FILES_HASH_PERSON)
    >>> h.update(b'the same content')
    >>> h.hexdigest()
    '20d9cd024d4fb086aae819a1432dd2466de12947831b75c5a30cf2676095d3b4'
    >>> h = blake2b(digest_size=32, person=BLOCK_HASH_PERSON)
    >>> h.update(b'the same content')
    >>> h.hexdigest()
    'cf68fb5761b9c44e7878bfb2c4c9aea52264a80b75005e65619778de59f383a3'

Personalization together with the keyed mode can also be used to derive different keys from a single one.

> \>\>\> from hashlib import blake2s \>\>\> from base64 import b64decode, b64encode \>\>\> orig_key = b64decode(b'Rm5EPJai72qcK3RGBpW3vPNfZy5OZothY+kHY6h21KM=') \>\>\> enc_key = blake2s(key=orig_key, person=b'kEncrypt').digest() \>\>\> mac_key = blake2s(key=orig_key, person=b'kMAC').digest() \>\>\> print(b64encode(enc_key).decode('utf-8')) rbPb15S/Z9t+agffno5wuhB77VbRi6F9Iv2qIxU7WHw= \>\>\> print(b64encode(mac_key).decode('utf-8')) G9GtHFE1YluXY1zWPlYk1e/nWfu0WSEb0KRcjhDeP/o=

#### Tree mode

Here's an example of hashing a minimal tree with two leaf nodes:

    10
    /  \
    00  01

This example uses 64-byte internal digests, and returns the 32-byte final digest:

    >>> from hashlib import blake2b
    >>>
    >>> FANOUT = 2
    >>> DEPTH = 2
    >>> LEAF_SIZE = 4096
    >>> INNER_SIZE = 64
    >>>
    >>> buf = bytearray(6000)
    >>>
    >>> # Left leaf
    ... h00 = blake2b(buf[0:LEAF_SIZE], fanout=FANOUT, depth=DEPTH,
    ...               leaf_size=LEAF_SIZE, inner_size=INNER_SIZE,
    ...               node_offset=0, node_depth=0, last_node=False)
    >>> # Right leaf
    ... h01 = blake2b(buf[LEAF_SIZE:], fanout=FANOUT, depth=DEPTH,
    ...               leaf_size=LEAF_SIZE, inner_size=INNER_SIZE,
    ...               node_offset=1, node_depth=0, last_node=True)
    >>> # Root node
    ... h10 = blake2b(digest_size=32, fanout=FANOUT, depth=DEPTH,
    ...               leaf_size=LEAF_SIZE, inner_size=INNER_SIZE,
    ...               node_offset=0, node_depth=1, last_node=True)
    >>> h10.update(h00.digest())
    >>> h10.update(h01.digest())
    >>> h10.hexdigest()
    '3ad2a9b37c6070e374c7a8c508fe20ca86b6ed54e286e93a0318e95e881db5aa'

### Credits

[BLAKE2](https://www.blake2.net) was designed by *Jean-Philippe Aumasson*, *Samuel Neves*, *Zooko Wilcox-O'Hearn*, and *Christian Winnerlein* based on [SHA-3](https://en.wikipedia.org/wiki/Secure_Hash_Algorithms) finalist [BLAKE](https://web.archive.org/web/20200918190133/https://131002.net/blake/) created by *Jean-Philippe Aumasson*, *Luca Henzen*, *Willi Meier*, and *Raphael C.-W. Phan*.

It uses core algorithm from [ChaCha](https://cr.yp.to/chacha.html) cipher designed by *Daniel J. Bernstein*.

The stdlib implementation is based on [pyblake2](https://pythonhosted.org/pyblake2/) module. It was written by *Dmitry Chestnykh* based on C implementation written by *Samuel Neves*. The documentation was copied from [pyblake2](https://pythonhosted.org/pyblake2/) and written by *Dmitry Chestnykh*.

The C code was partly rewritten for Python by *Christian Heimes*.

The following public domain dedication applies for both C hash function implementation, extension code, and this documentation:

> To the extent possible under law, the author(s) have dedicated all copyright and related and neighboring rights to this software to the public domain worldwide. This software is distributed without any warranty.
>
> You should have received a copy of the CC0 Public Domain Dedication along with this software. If not, see <https://creativecommons.org/publicdomain/zero/1.0/>.

The following people have helped with development or contributed their changes to the project and the public domain according to the Creative Commons Public Domain Dedication 1.0 Universal:

- *Alexandr Sokolovskiy*

<div id="hashlib-seealso">

<div class="seealso">

Module `hmac`  
A module to generate message authentication codes using hashes.

Module `base64`  
Another way to encode binary hashes for non-binary environments.

<https://nvlpubs.nist.gov/nistpubs/fips/nist.fips.180-4.pdf>  
The FIPS 180-4 publication on Secure Hash Algorithms.

<https://csrc.nist.gov/pubs/fips/202/final>  
The FIPS 202 publication on the SHA-3 Standard.

<https://www.blake2.net/>  
Official BLAKE2 website.

<https://en.wikipedia.org/wiki/Cryptographic_hash_function>  
Wikipedia article with information on which algorithms have known issues and what that means regarding their use.

<https://www.ietf.org/rfc/rfc8018.txt>  
PKCS \#5: Password-Based Cryptography Specification Version 2.1

<https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-132.pdf>  
NIST Recommendation for Password-Based Key Derivation.

</div>

</div>

# `!heapq` --- Heap queue algorithm

<div class="module" data-synopsis="Heap queue algorithm (a.k.a. priority queue).">

heapq

</div>

**Source code:** `Lib/heapq.py`

------------------------------------------------------------------------

This module provides an implementation of the heap queue algorithm, also known as the priority queue algorithm.

Min-heaps are binary trees for which every parent node has a value less than or equal to any of its children. We refer to this condition as the heap invariant.

For min-heaps, this implementation uses lists for which `heap[k] <= heap[2*k+1]` and `heap[k] <= heap[2*k+2]` for all *k* for which the compared elements exist. Elements are counted from zero. The interesting property of a min-heap is that its smallest element is always the root, `heap[0]`.

Max-heaps satisfy the reverse invariant: every parent node has a value *greater* than any of its children. These are implemented as lists for which `maxheap[2*k+1] <= maxheap[k]` and `maxheap[2*k+2] <= maxheap[k]` for all *k* for which the compared elements exist. The root, `maxheap[0]`, contains the *largest* element; `heap.sort(reverse=True)` maintains the max-heap invariant.

The `!heapq` API differs from textbook heap algorithms in two aspects: (a) We use zero-based indexing. This makes the relationship between the index for a node and the indexes for its children slightly less obvious, but is more suitable since Python uses zero-based indexing. (b) Textbooks often focus on max-heaps, due to their suitability for in-place sorting. Our implementation favors min-heaps as they better correspond to Python `lists <list>`.

These two aspects make it possible to view the heap as a regular Python list without surprises: `heap[0]` is the smallest item, and `heap.sort()` maintains the heap invariant!

Like `list.sort`, this implementation uses only the `<` operator for comparisons, for both min-heaps and max-heaps.

In the API below, and in this documentation, the unqualified term *heap* generally refers to a min-heap. The API for max-heaps is named using a `_max` suffix.

To create a heap, use a list initialized as `[]`, or transform an existing list into a min-heap or max-heap using the `heapify` or `heapify_max` functions, respectively.

The following functions are provided for min-heaps:

<div class="function">

heapify(x)

Transform list *x* into a min-heap, in-place, in linear time.

</div>

<div class="function">

heappush(heap, item)

Push the value *item* onto the *heap*, maintaining the min-heap invariant.

</div>

<div class="function">

heappop(heap)

Pop and return the smallest item from the *heap*, maintaining the min-heap invariant. If the heap is empty, `IndexError` is raised. To access the smallest item without popping it, use `heap[0]`.

</div>

<div class="function">

heappushpop(heap, item)

Push *item* on the heap, then pop and return the smallest item from the *heap*. The combined action runs more efficiently than `heappush` followed by a separate call to `heappop`.

</div>

<div class="function">

heapreplace(heap, item)

Pop and return the smallest item from the *heap*, and also push the new *item*. The heap size doesn't change. If the heap is empty, `IndexError` is raised.

This one step operation is more efficient than a `heappop` followed by `heappush` and can be more appropriate when using a fixed-size heap. The pop/push combination always returns an element from the heap and replaces it with *item*.

The value returned may be larger than the *item* added. If that isn't desired, consider using `heappushpop` instead. Its push/pop combination returns the smaller of the two values, leaving the larger value on the heap.

</div>

For max-heaps, the following functions are provided:

<div class="function">

heapify_max(x)

Transform list *x* into a max-heap, in-place, in linear time.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

heappush_max(heap, item)

Push the value *item* onto the max-heap *heap*, maintaining the max-heap invariant.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

heappop_max(heap)

Pop and return the largest item from the max-heap *heap*, maintaining the max-heap invariant. If the max-heap is empty, `IndexError` is raised. To access the largest item without popping it, use `maxheap[0]`.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

heappushpop_max(heap, item)

Push *item* on the max-heap *heap*, then pop and return the largest item from *heap*. The combined action runs more efficiently than `heappush_max` followed by a separate call to `heappop_max`.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

heapreplace_max(heap, item)

Pop and return the largest item from the max-heap *heap* and also push the new *item*. The max-heap size doesn't change. If the max-heap is empty, `IndexError` is raised.

The value returned may be smaller than the *item* added. Refer to the analogous function `heapreplace` for detailed usage notes.

<div class="versionadded">

3.14

</div>

</div>

The module also offers three general purpose functions based on heaps.

<div class="function">

merge(\*iterables, key=None, reverse=False)

Merge multiple sorted inputs into a single sorted output (for example, merge timestamped entries from multiple log files). Returns an `iterator` over the sorted values.

Similar to `sorted(itertools.chain(*iterables))` but returns an iterable, does not pull the data into memory all at once, and assumes that each of the input streams is already sorted (smallest to largest).

Has two optional arguments which must be specified as keyword arguments.

*key* specifies a `key function` of one argument that is used to extract a comparison key from each input element. The default value is `None` (compare the elements directly).

*reverse* is a boolean value. If set to `True`, then the input elements are merged as if each comparison were reversed. To achieve behavior similar to `sorted(itertools.chain(*iterables), reverse=True)`, all iterables must be sorted from largest to smallest.

<div class="versionchanged">

3.5 Added the optional *key* and *reverse* parameters.

</div>

</div>

<div class="function">

nlargest(n, iterable, key=None)

Return a list with the *n* largest elements from the dataset defined by *iterable*. *key*, if provided, specifies a function of one argument that is used to extract a comparison key from each element in *iterable* (for example, `key=str.lower`). Equivalent to: `sorted(iterable, key=key, reverse=True)[:n]`.

</div>

<div class="function">

nsmallest(n, iterable, key=None)

Return a list with the *n* smallest elements from the dataset defined by *iterable*. *key*, if provided, specifies a function of one argument that is used to extract a comparison key from each element in *iterable* (for example, `key=str.lower`). Equivalent to: `sorted(iterable, key=key)[:n]`.

</div>

The latter two functions perform best for smaller values of *n*. For larger values, it is more efficient to use the `sorted` function. Also, when `n==1`, it is more efficient to use the built-in `min` and `max` functions. If repeated usage of these functions is required, consider turning the iterable into an actual heap.

## Basic Examples

A [heapsort](https://en.wikipedia.org/wiki/Heapsort) can be implemented by pushing all values onto a heap and then popping off the smallest values one at a time:

    >>> def heapsort(iterable):
    ...     h = []
    ...     for value in iterable:
    ...         heappush(h, value)
    ...     return [heappop(h) for i in range(len(h))]
    ...
    >>> heapsort([1, 3, 5, 7, 9, 2, 4, 6, 8, 0])
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

This is similar to `sorted(iterable)`, but unlike `sorted`, this implementation is not stable.

Heap elements can be tuples. This is useful for assigning comparison values (such as task priorities) alongside the main record being tracked:

    >>> h = []
    >>> heappush(h, (5, 'write code'))
    >>> heappush(h, (7, 'release product'))
    >>> heappush(h, (1, 'write spec'))
    >>> heappush(h, (3, 'create tests'))
    >>> heappop(h)
    (1, 'write spec')

## Other Applications

[Medians](https://en.wikipedia.org/wiki/Median) are a measure of central tendency for a set of numbers. In distributions skewed by outliers, the median provides a more stable estimate than an average (arithmetic mean). A running median is an [online algorithm](https://en.wikipedia.org/wiki/Online_algorithm) that updates continuously as new data arrives.

A running median can be efficiently implemented by balancing two heaps, a max-heap for values at or below the midpoint and a min-heap for values above the midpoint. When the two heaps have the same size, the new median is the average of the tops of the two heaps; otherwise, the median is at the top of the larger heap:

    def running_median(iterable):
        "Yields the cumulative median of values seen so far."

        lo = []  # max-heap
        hi = []  # min-heap (same size as or one smaller than lo)

        for x in iterable:
            if len(lo) == len(hi):
                heappush_max(lo, heappushpop(hi, x))
                yield lo[0]
            else:
                heappush(hi, heappushpop_max(lo, x))
                yield (lo[0] + hi[0]) / 2

For example:

    >>> list(running_median([5.0, 9.0, 4.0, 12.0, 8.0, 9.0]))
    [5.0, 7.0, 5.0, 7.0, 8.0, 8.5]

## Priority Queue Implementation Notes

A [priority queue](https://en.wikipedia.org/wiki/Priority_queue) is common use for a heap, and it presents several implementation challenges:

- Sort stability: how do you get two tasks with equal priorities to be returned in the order they were originally added?
- Tuple comparison breaks for (priority, task) pairs if the priorities are equal and the tasks do not have a default comparison order.
- If the priority of a task changes, how do you move it to a new position in the heap?
- Or if a pending task needs to be deleted, how do you find it and remove it from the queue?

A solution to the first two challenges is to store entries as 3-element list including the priority, an entry count, and the task. The entry count serves as a tie-breaker so that two tasks with the same priority are returned in the order they were added. And since no two entry counts are the same, the tuple comparison will never attempt to directly compare two tasks.

Another solution to the problem of non-comparable tasks is to create a wrapper class that ignores the task item and only compares the priority field:

    from dataclasses import dataclass, field
    from typing import Any

    @dataclass(order=True)
    class PrioritizedItem:
        priority: int
        item: Any=field(compare=False)

The remaining challenges revolve around finding a pending task and making changes to its priority or removing it entirely. Finding a task can be done with a dictionary pointing to an entry in the queue.

Removing the entry or changing its priority is more difficult because it would break the heap structure invariants. So, a possible solution is to mark the entry as removed and add a new entry with the revised priority:

    pq = []                         # list of entries arranged in a heap
    entry_finder = {}               # mapping of tasks to entries
    REMOVED = '<removed-task>'      # placeholder for a removed task
    counter = itertools.count()     # unique sequence count

    def add_task(task, priority=0):
        'Add a new task or update the priority of an existing task'
        if task in entry_finder:
            remove_task(task)
        count = next(counter)
        entry = [priority, count, task]
        entry_finder[task] = entry
        heappush(pq, entry)

    def remove_task(task):
        'Mark an existing task as REMOVED.  Raise KeyError if not found.'
        entry = entry_finder.pop(task)
        entry[-1] = REMOVED

    def pop_task():
        'Remove and return the lowest priority task. Raise KeyError if empty.'
        while pq:
            priority, count, task = heappop(pq)
            if task is not REMOVED:
                del entry_finder[task]
                return task
        raise KeyError('pop from an empty priority queue')

## Theory

Heaps are arrays for which `a[k] <= a[2*k+1]` and `a[k] <= a[2*k+2]` for all *k*, counting elements from 0. For the sake of comparison, non-existing elements are considered to be infinite. The interesting property of a heap is that `a[0]` is always its smallest element.

The strange invariant above is meant to be an efficient memory representation for a tournament. The numbers below are *k*, not `a[k]`:

<figure class="invert-in-dark-modealign-center">
<img src="heapq-binary-tree.svg" class="invert-in-dark-mode" alt="Example (min-heap) binary tree." />
</figure>

In the tree above, each cell *k* is topping `2*k+1` and `2*k+2`. In a usual binary tournament we see in sports, each cell is the winner over the two cells it tops, and we can trace the winner down the tree to see all opponents s/he had. However, in many computer applications of such tournaments, we do not need to trace the history of a winner. To be more memory efficient, when a winner is promoted, we try to replace it by something else at a lower level, and the rule becomes that a cell and the two cells it tops contain three different items, but the top cell "wins" over the two topped cells.

If this heap invariant is protected at all time, index 0 is clearly the overall winner. The simplest algorithmic way to remove it and find the "next" winner is to move some loser (let's say cell 30 in the diagram above) into the 0 position, and then percolate this new 0 down the tree, exchanging values, until the invariant is re-established. This is clearly logarithmic on the total number of items in the tree. By iterating over all items, you get an *O*(*n* log *n*) sort.

A nice feature of this sort is that you can efficiently insert new items while the sort is going on, provided that the inserted items are not "better" than the last 0'th element you extracted. This is especially useful in simulation contexts, where the tree holds all incoming events, and the "win" condition means the smallest scheduled time. When an event schedules other events for execution, they are scheduled into the future, so they can easily go into the heap. So, a heap is a good structure for implementing schedulers (this is what I used for my MIDI sequencer :-).

Various structures for implementing schedulers have been extensively studied, and heaps are good for this, as they are reasonably speedy, the speed is almost constant, and the worst case is not much different than the average case. However, there are other representations which are more efficient overall, yet the worst cases might be terrible.

Heaps are also very useful in big disk sorts. You most probably all know that a big sort implies producing "runs" (which are pre-sorted sequences, whose size is usually related to the amount of CPU memory), followed by a merging passes for these runs, which merging is often very cleverly organised[^1]. It is very important that the initial sort produces the longest runs possible. Tournaments are a good way to achieve that. If, using all the memory available to hold a tournament, you replace and percolate items that happen to fit the current run, you'll produce runs which are twice the size of the memory for random input, and much better for input fuzzily ordered.

Moreover, if you output the 0'th item on disk and get an input which may not fit in the current tournament (because the value "wins" over the last output value), it cannot fit in the heap, so the size of the heap decreases. The freed memory could be cleverly reused immediately for progressively building a second heap, which grows at exactly the same rate the first heap is melting. When the first heap completely vanishes, you switch heaps and start a new run. Clever and quite effective!

In a word, heaps are useful memory structures to know. I use them in a few applications, and I think it is good to keep a 'heap' module around. :-)

**Footnotes**

[^1]: The disk balancing algorithms which are current, nowadays, are more annoying than clever, and this is a consequence of the seeking capabilities of the disks. On devices which cannot seek, like big tape drives, the story was quite different, and one had to be very clever to ensure (far in advance) that each tape movement will be the most effective possible (that is, will best participate at "progressing" the merge). Some tapes were even able to read backwards, and this was also used to avoid the rewinding time. Believe me, real good tape sorts were quite spectacular to watch! From all times, sorting has always been a Great Art! :-)

# `!hmac` --- Keyed-Hashing for Message Authentication

<div class="module" data-synopsis="Keyed-Hashing for Message Authentication (HMAC) implementation">

hmac

</div>

**Source code:** `Lib/hmac.py`

------------------------------------------------------------------------

This module implements the HMAC algorithm as described by `2104`. The interface allows to use any hash function with a *fixed* digest size. In particular, extendable output functions such as SHAKE-128 or SHAKE-256 cannot be used with HMAC.

<div class="function">

new(key, msg=None, digestmod)

Return a new hmac object. *key* is a bytes or bytearray object giving the secret key. If *msg* is present, the method call `update(msg)` is made. *digestmod* is the digest name, digest constructor or module for the HMAC object to use. It may be any name suitable to `hashlib.new`. Despite its argument position, it is required.

<div class="versionchanged">

3.4 Parameter *key* can be a bytes or bytearray object. Parameter *msg* can be of any type supported by `hashlib`. Parameter *digestmod* can be the name of a hash algorithm.

</div>

<div class="versionchanged">

3.8 The *digestmod* argument is now required. Pass it as a keyword argument to avoid awkwardness when you do not have an initial *msg*.

</div>

</div>

<div class="function">

digest(key, msg, digest)

Return digest of *msg* for given secret *key* and *digest*. The function is equivalent to `HMAC(key, msg, digest).digest()`, but uses an optimized C or inline implementation, which is faster for messages that fit into memory. The parameters *key*, *msg*, and *digest* have the same meaning as in `~hmac.new`.

CPython implementation detail, the optimized C implementation is only used when *digest* is a string and name of a digest algorithm, which is supported by OpenSSL.

<div class="versionadded">

3.7

</div>

</div>

<div class="HMAC">

An HMAC object has the following methods:

</div>

<div class="method">

HMAC.update(msg)

Update the hmac object with *msg*. Repeated calls are equivalent to a single call with the concatenation of all the arguments: `m.update(a); m.update(b)` is equivalent to `m.update(a + b)`.

<div class="versionchanged">

3.4 Parameter *msg* can be of any type supported by `hashlib`.

</div>

</div>

<div class="method">

HMAC.digest()

Return the digest of the bytes passed to the `update` method so far. This bytes object will be the same length as the *digest_size* of the digest given to the constructor. It may contain non-ASCII bytes, including NUL bytes.

> [!WARNING]
> When comparing the output of `digest` to an externally supplied digest during a verification routine, it is recommended to use the `compare_digest` function instead of the `==` operator to reduce the vulnerability to timing attacks.

</div>

<div class="method">

HMAC.hexdigest()

Like `digest` except the digest is returned as a string twice the length containing only hexadecimal digits. This may be used to exchange the value safely in email or other non-binary environments.

> [!WARNING]
> When comparing the output of `hexdigest` to an externally supplied digest during a verification routine, it is recommended to use the `compare_digest` function instead of the `==` operator to reduce the vulnerability to timing attacks.

</div>

<div class="method">

HMAC.copy()

Return a copy ("clone") of the hmac object. This can be used to efficiently compute the digests of strings that share a common initial substring.

</div>

A hash object has the following attributes:

<div class="attribute">

HMAC.digest_size

The size of the resulting HMAC digest in bytes.

</div>

<div class="attribute">

HMAC.block_size

The internal block size of the hash algorithm in bytes.

<div class="versionadded">

3.4

</div>

</div>

<div class="attribute">

HMAC.name

The canonical name of this HMAC, always lowercase, e.g. `hmac-md5`.

<div class="versionadded">

3.4

</div>

</div>

<div class="versionchanged">

3.10 Removed the undocumented attributes `HMAC.digest_cons`, `HMAC.inner`, and `HMAC.outer`.

</div>

This module also provides the following helper function:

<div class="function">

compare_digest(a, b)

Return `a == b`. This function uses an approach designed to prevent timing analysis by avoiding content-based short circuiting behaviour, making it appropriate for cryptography. *a* and *b* must both be of the same type: either `str` (ASCII only, as e.g. returned by `HMAC.hexdigest`), or a `bytes-like object`.

> [!NOTE]
> If *a* and *b* are of different lengths, or if an error occurs, a timing attack could theoretically reveal information about the types and lengths of *a* and *b*—but not their values.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.10

The function uses OpenSSL's `CRYPTO_memcmp()` internally when available.

</div>

</div>

<div class="seealso">

Module `hashlib`  
The Python module providing secure hash functions.

</div>

# `!html.entities` --- Definitions of HTML general entities

<div class="module" data-synopsis="Definitions of HTML general entities.">

html.entities

</div>

**Source code:** `Lib/html/entities.py`

------------------------------------------------------------------------

This module defines four dictionaries, `html5`, `name2codepoint`, `codepoint2name`, and `entitydefs`.

<div class="data">

html5

A dictionary that maps HTML5 named character references[^1] to the equivalent Unicode character(s), e.g. `html5['gt;'] == '>'`. Note that the trailing semicolon is included in the name (e.g. `'gt;'`), however some of the names are accepted by the standard even without the semicolon: in this case the name is present with and without the `';'`. See also `html.unescape`.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

entitydefs

A dictionary mapping XHTML 1.0 entity definitions to their replacement text in ISO Latin-1.

</div>

<div class="data">

name2codepoint

A dictionary that maps HTML4 entity names to the Unicode code points.

</div>

<div class="data">

codepoint2name

A dictionary that maps Unicode code points to HTML4 entity names.

</div>

**Footnotes**

[^1]: See <https://html.spec.whatwg.org/multipage/named-characters.html#named-character-references>

# `!html` --- HyperText Markup Language support

<div class="module" data-synopsis="Helpers for manipulating HTML.">

html

</div>

**Source code:** `Lib/html/__init__.py`

------------------------------------------------------------------------

This module defines utilities to manipulate HTML.

<div class="function">

escape(s, quote=True)

Convert the characters `&`, `<` and `>` in string *s* to HTML-safe sequences. Use this if you need to display text that might contain such characters in HTML. If the optional flag *quote* is true (the default), the characters (`"`) and (`'`) are also translated; this helps for inclusion in an HTML attribute value delimited by quotes, as in `<a href="...">`. If *quote* is set to false, the characters (`"`) and (`'`) are not translated.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

unescape(s)

Convert all named and numeric character references (e.g. `&gt;`, `&#62;`, `&#x3e;`) in the string *s* to the corresponding Unicode characters. This function uses the rules defined by the HTML 5 standard for both valid and invalid character references, and the `list of
HTML 5 named character references <html.entities.html5>`.

<div class="versionadded">

3.4

</div>

</div>

------------------------------------------------------------------------

Submodules in the `html` package are:

- `html.parser` -- HTML/XHTML parser with lenient parsing mode
- `html.entities` -- HTML entity definitions

# `!html.parser` --- Simple HTML and XHTML parser

<div class="module" data-synopsis="A simple parser that can handle HTML and XHTML.">

html.parser

</div>

**Source code:** `Lib/html/parser.py`

<div class="index">

single: HTML single: XHTML

</div>

------------------------------------------------------------------------

This module defines a class `HTMLParser` which serves as the basis for parsing text files formatted in HTML (HyperText Mark-up Language) and XHTML.

<div class="HTMLParser(*, convert_charrefs=True, scripting=False)">

Create a parser instance able to parse invalid markup.

If *convert_charrefs* is true (the default), all character references (except the ones in elements like `script` and `style`) are automatically converted to the corresponding Unicode characters.

If *scripting* is false (the default), the content of the `noscript` element is parsed normally; if it's true, it's returned as is without being parsed.

An `.HTMLParser` instance is fed HTML data and calls handler methods when start tags, end tags, text, comments, and other markup elements are encountered. The user should subclass `.HTMLParser` and override its methods to implement the desired behavior.

This parser does not check that end tags match start tags or call the end-tag handler for elements which are closed implicitly by closing an outer element.

<div class="versionchanged">

3.4 *convert_charrefs* keyword argument added.

</div>

<div class="versionchanged">

3.5 The default value for argument *convert_charrefs* is now `True`.

</div>

<div class="versionchanged">

3.14.1 Added the *scripting* parameter.

</div>

</div>

## Example HTML Parser Application

As a basic example, below is a simple HTML parser that uses the `HTMLParser` class to print out start tags, end tags, and data as they are encountered:

<div class="testcode">

from html.parser import HTMLParser

class MyHTMLParser(HTMLParser):  
def handle_starttag(self, tag, attrs):  
print("Encountered a start tag:", tag)

def handle_endtag(self, tag):  
print("Encountered an end tag :", tag)

def handle_data(self, data):  
print("Encountered some data :", data)

parser = MyHTMLParser() parser.feed('\<html\>\<head\>\<title\>Test\</title\>\</head\>' '\<body\>\<h1\>Parse me!\</h1\>\</body\>\</html\>')

</div>

The output will then be:

<div class="testoutput">

Encountered a start tag: html Encountered a start tag: head Encountered a start tag: title Encountered some data : Test Encountered an end tag : title Encountered an end tag : head Encountered a start tag: body Encountered a start tag: h1 Encountered some data : Parse me! Encountered an end tag : h1 Encountered an end tag : body Encountered an end tag : html

</div>

## `.HTMLParser` Methods

`HTMLParser` instances have the following methods:

<div class="method">

HTMLParser.feed(data)

Feed some text to the parser. It is processed insofar as it consists of complete elements; incomplete data is buffered until more data is fed or `close` is called. *data* must be `str`.

</div>

<div class="method">

HTMLParser.close()

Force processing of all buffered data as if it were followed by an end-of-file mark. This method may be redefined by a derived class to define additional processing at the end of the input, but the redefined version should always call the `HTMLParser` base class method `close`.

</div>

<div class="method">

HTMLParser.reset()

Reset the instance. Loses all unprocessed data. This is called implicitly at instantiation time.

</div>

<div class="method">

HTMLParser.getpos()

Return current line number and offset.

</div>

<div class="method">

HTMLParser.get_starttag_text()

Return the text of the most recently opened start tag. This should not normally be needed for structured processing, but may be useful in dealing with HTML "as deployed" or for re-generating input with minimal changes (whitespace between attributes can be preserved, etc.).

</div>

The following methods are called when data or markup elements are encountered and they are meant to be overridden in a subclass. The base class implementations do nothing (except for `~HTMLParser.handle_startendtag`):

<div class="method">

HTMLParser.handle_starttag(tag, attrs)

This method is called to handle the start tag of an element (e.g. `<div id="main">`).

The *tag* argument is the name of the tag converted to lower case. The *attrs* argument is a list of `(name, value)` pairs containing the attributes found inside the tag's `<>` brackets. The *name* will be translated to lower case, and quotes in the *value* have been removed, and character and entity references have been replaced. For empty attributes, *value* is `None`.

For instance, for the tag `<A HREF="https://www.cwi.nl/">`, this method would be called as `handle_starttag('a', [('href', 'https://www.cwi.nl/')])`.

All entity references from `html.entities` are replaced in the attribute values.

</div>

<div class="method">

HTMLParser.handle_endtag(tag)

This method is called to handle the end tag of an element (e.g. `</div>`).

The *tag* argument is the name of the tag converted to lower case.

</div>

<div class="method">

HTMLParser.handle_startendtag(tag, attrs)

Similar to `handle_starttag`, but called when the parser encounters an XHTML-style empty tag (`<img ... />`). This method may be overridden by subclasses which require this particular lexical information; the default implementation simply calls `handle_starttag` and `handle_endtag`.

</div>

<div class="method">

HTMLParser.handle_data(data)

This method is called to process arbitrary data (e.g. text nodes and the content of elements like `script` and `style`).

</div>

<div class="method">

HTMLParser.handle_entityref(name)

This method is called to process a named character reference of the form `&name;` (e.g. `&gt;`), where *name* is a general entity reference (e.g. `'gt'`). This method is only called if *convert_charrefs* is false.

</div>

<div class="method">

HTMLParser.handle_charref(name)

This method is called to process decimal and hexadecimal numeric character references of the form `&#{NNN};` and `&#x{NNN};`. For example, the decimal equivalent for `&gt;` is `&#62;`, whereas the hexadecimal is `&#x3E;`; in this case the method will receive `'62'` or `'x3E'`. This method is only called if *convert_charrefs* is false.

</div>

<div class="method">

HTMLParser.handle_comment(data)

This method is called when a comment is encountered (e.g. `<!--comment-->`).

For example, the comment `<!-- comment -->` will cause this method to be called with the argument `' comment '`.

The content of Internet Explorer conditional comments (condcoms) will also be sent to this method, so, for `<!--[if IE 9]>IE9-specific content<![endif]-->`, this method will receive `'[if IE 9]>IE9-specific content<![endif]'`.

</div>

<div class="method">

HTMLParser.handle_decl(decl)

This method is called to handle an HTML doctype declaration (e.g. `<!DOCTYPE html>`).

The *decl* parameter will be the entire contents of the declaration inside the `<!...>` markup (e.g. `'DOCTYPE html'`).

</div>

<div class="method">

HTMLParser.handle_pi(data)

Method called when a processing instruction is encountered. The *data* parameter will contain the entire processing instruction. For example, for the processing instruction `<?proc color='red'>`, this method would be called as `handle_pi("proc color='red'")`. It is intended to be overridden by a derived class; the base class implementation does nothing.

> [!NOTE]
> The `HTMLParser` class uses the SGML syntactic rules for processing instructions. An XHTML processing instruction using the trailing `'?'` will cause the `'?'` to be included in *data*.

</div>

<div class="method">

HTMLParser.unknown_decl(data)

This method is called when an unrecognized declaration is read by the parser.

The *data* parameter will be the entire contents of the declaration inside the `<![...]>` markup. It is sometimes useful to be overridden by a derived class. The base class implementation does nothing.

</div>

## Examples

The following class implements a parser that will be used to illustrate more examples:

<div class="testcode">

from html.parser import HTMLParser from html.entities import name2codepoint

class MyHTMLParser(HTMLParser):  
def handle_starttag(self, tag, attrs):  
print("Start tag:", tag) for attr in attrs: print(" attr:", attr)

def handle_endtag(self, tag):  
print("End tag :", tag)

def handle_data(self, data):  
print("Data :", data)

def handle_comment(self, data):  
print("Comment :", data)

def handle_entityref(self, name):  
c = chr(name2codepoint\[name\]) print("Named ent:", c)

def handle_charref(self, name):  
if name.startswith('x'):  
c = chr(int(name\[1:\], 16))

else:  
c = chr(int(name))

print("Num ent :", c)

def handle_decl(self, data):  
print("Decl :", data)

parser = MyHTMLParser()

</div>

Parsing a doctype:

<div class="doctest">

\>\>\> parser.feed('\<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" ' ... '"http://www.w3.org/TR/html4/strict.dtd"\>') Decl : DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"

</div>

Parsing an element with a few attributes and a title:

<div class="doctest">

\>\>\> parser.feed('\<img src="python-logo.png" alt="The Python logo"\>') Start tag: img attr: ('src', 'python-logo.png') attr: ('alt', 'The Python logo') \>\>\> \>\>\> parser.feed('\<h1\>Python\</h1\>') Start tag: h1 Data : Python End tag : h1

</div>

The content of elements like `script` and `style` is returned as is, without further parsing:

<div class="doctest">

\>\>\> parser.feed('\<style type="text/css"\>#python { color: green }\</style\>') Start tag: style attr: ('type', 'text/css') Data : \#python { color: green } End tag : style

\>\>\> parser.feed('\<script type="text/javascript"\>' ... 'alert("\<strong\>hello! \&#9786;\</strong\>");\</script\>') Start tag: script attr: ('type', 'text/javascript') Data : alert("\<strong\>hello! \&#9786;\</strong\>"); End tag : script

</div>

Attribute names are converted to lowercase, quotes from attribute values removed, and `None` is returned as *value* for empty attributes (such as `checked`):

<div class="doctest">

\>\>\> parser.feed("\<input TYPE='checkbox' checked required='' disabled=disabled\>") Start tag: input attr: ('type', 'checkbox') attr: ('checked', None) attr: ('required', '') attr: ('disabled', 'disabled')

</div>

Parsing comments:

<div class="doctest">

\>\>\> parser.feed('\<!--a comment--\>' ... '\<!--\[if IE 9\]\>IE-specific content\<\![endif\]--\>') Comment : a comment Comment : \[if IE 9\]\>IE-specific content\<\![endif\]

</div>

Parsing named and numeric character references and converting them to the correct char (note: these 3 references are all equivalent to `'>'`):

<div class="doctest">

\>\>\> parser = MyHTMLParser() \>\>\> parser.feed('\&gt;\&#62;\&#x3E;') Data : \>\>\>

\>\>\> parser = MyHTMLParser(convert_charrefs=False) \>\>\> parser.feed('\&gt;\&#62;\&#x3E;') Named ent: \> Num ent : \> Num ent : \>

</div>

Feeding incomplete chunks to `~HTMLParser.feed` works, but `~HTMLParser.handle_data` might be called more than once if *convert_charrefs* is false:

<div class="doctest">

\>\>\> for chunk in \['\<sp', 'an\>buff', 'ered', ' text\</s', 'pan\>'\]: ... parser.feed(chunk) ... Start tag: span Data : buff Data : ered Data : text End tag : span

</div>

Parsing invalid HTML (e.g. unquoted attributes) also works:

<div class="doctest">

\>\>\> parser.feed('\<p\>\<a class=link href=#main\>tag soup\</p \>\</a\>') Start tag: p Start tag: a attr: ('class', 'link') attr: ('href', '#main') Data : tag soup End tag : p End tag : a

</div>

# `!http.client` --- HTTP protocol client

<div class="module" data-synopsis="HTTP and HTTPS protocol client (requires sockets).">

http.client

</div>

**Source code:** `Lib/http/client.py`

<div class="index">

pair: HTTP; protocol single: HTTP; http.client (standard module)

</div>

<div class="index">

pair: module; urllib.request

</div>

------------------------------------------------------------------------

This module defines classes that implement the client side of the HTTP and HTTPS protocols. It is normally not used directly --- the module `urllib.request` uses it to handle URLs that use HTTP and HTTPS.

<div class="seealso">

The [Requests package](https://requests.readthedocs.io/en/latest/) is recommended for a higher-level HTTP client interface.

</div>

> [!NOTE]
> HTTPS support is only available if Python was compiled with SSL support (through the `ssl` module).

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

The module provides the following classes:

<div class="HTTPConnection(host, port=None[, timeout], source_address=None, \ blocksize=8192, max_response_headers=None)">

An `HTTPConnection` instance represents one transaction with an HTTP server. It should be instantiated by passing it a host and optional port number. If no port number is passed, the port is extracted from the host string if it has the form `host:port`, else the default HTTP port (80) is used. If the optional *timeout* parameter is given, blocking operations (like connection attempts) will timeout after that many seconds (if it is not given, the global default timeout setting is used). The optional *source_address* parameter may be a tuple of a (host, port) to use as the source address the HTTP connection is made from. The optional *blocksize* parameter sets the buffer size in bytes for sending a file-like message body. The optional *max_response_headers* parameter sets the maximum number of allowed response headers to help prevent denial-of-service attacks, otherwise the default value (100) is used.

For example, the following calls all create instances that connect to the server at the same host and port:

    >>> h1 = http.client.HTTPConnection('www.python.org')
    >>> h2 = http.client.HTTPConnection('www.python.org:80')
    >>> h3 = http.client.HTTPConnection('www.python.org', 80)
    >>> h4 = http.client.HTTPConnection('www.python.org', 80, timeout=10)

<div class="versionchanged">

3.2 *source_address* was added.

</div>

<div class="versionchanged">

3.4 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are no longer supported.

</div>

<div class="versionchanged">

3.7 *blocksize* parameter was added.

</div>

<div class="versionchanged">

3.15 *max_response_headers* parameter was added.

</div>

</div>

<div class="HTTPSConnection(host, port=None, *[, timeout], \ source_address=None, context=None, \ blocksize=8192, max_response_headers=None)">

A subclass of `HTTPConnection` that uses SSL for communication with secure servers. Default port is `443`. If *context* is specified, it must be a `ssl.SSLContext` instance describing the various SSL options.

Please read `ssl-security` for more information on best practices.

<div class="versionchanged">

3.2 *source_address*, *context* and *check_hostname* were added.

</div>

<div class="versionchanged">

3.2 This class now supports HTTPS virtual hosts if possible (that is, if `ssl.HAS_SNI` is true).

</div>

<div class="versionchanged">

3.4 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are no longer supported.

</div>

<div class="versionchanged">

3.4.3 This class now performs all the necessary certificate and hostname checks by default. To revert to the previous, unverified, behavior `!ssl._create_unverified_context` can be passed to the *context* parameter.

</div>

<div class="versionchanged">

3.8 This class now enables TLS 1.3 `ssl.SSLContext.post_handshake_auth` for the default *context* or when *cert_file* is passed with a custom *context*.

</div>

<div class="versionchanged">

3.10 This class now sends an ALPN extension with protocol indicator `http/1.1` when no *context* is given. Custom *context* should set ALPN protocols with `~ssl.SSLContext.set_alpn_protocols`.

</div>

<div class="versionchanged">

3.12 The deprecated *key_file*, *cert_file* and *check_hostname* parameters have been removed.

</div>

<div class="versionchanged">

3.15 *max_response_headers* parameter was added.

</div>

</div>

<div class="HTTPResponse(sock, debuglevel=0, method=None, url=None)">

Class whose instances are returned upon successful connection. Not instantiated directly by user.

<div class="versionchanged">

3.4 The *strict* parameter was removed. HTTP 0.9 style "Simple Responses" are no longer supported.

</div>

</div>

This module provides the following function:

<div class="function">

parse_headers(fp)

Parse the headers from a file pointer *fp* representing a HTTP request/response. The file has to be a `~io.BufferedIOBase` reader (i.e. not text) and must provide a valid `5322` style header.

This function returns an instance of `http.client.HTTPMessage` that holds the header fields, but no payload (the same as `HTTPResponse.msg` and `http.server.BaseHTTPRequestHandler.headers`). After returning, the file pointer *fp* is ready to read the HTTP body.

> [!NOTE]
> `parse_headers` does not parse the start-line of a HTTP message; it only parses the `Name: value` lines. The file has to be ready to read these field lines, so the first line should already be consumed before calling the function.

</div>

The following exceptions are raised as appropriate:

<div class="exception">

HTTPException

The base class of the other exceptions in this module. It is a subclass of `Exception`.

</div>

<div class="exception">

NotConnected

A subclass of `HTTPException`.

</div>

<div class="exception">

InvalidURL

A subclass of `HTTPException`, raised if a port is given and is either non-numeric or empty.

</div>

<div class="exception">

UnknownProtocol

A subclass of `HTTPException`.

</div>

<div class="exception">

UnknownTransferEncoding

A subclass of `HTTPException`.

</div>

<div class="exception">

UnimplementedFileMode

A subclass of `HTTPException`.

</div>

<div class="exception">

IncompleteRead

A subclass of `HTTPException`.

</div>

<div class="exception">

ImproperConnectionState

A subclass of `HTTPException`.

</div>

<div class="exception">

CannotSendRequest

A subclass of `ImproperConnectionState`.

</div>

<div class="exception">

CannotSendHeader

A subclass of `ImproperConnectionState`.

</div>

<div class="exception">

ResponseNotReady

A subclass of `ImproperConnectionState`.

</div>

<div class="exception">

BadStatusLine

A subclass of `HTTPException`. Raised if a server responds with a HTTP status code that we don't understand.

</div>

<div class="exception">

LineTooLong

A subclass of `HTTPException`. Raised if an excessively long line is received in the HTTP protocol from the server.

</div>

<div class="exception">

RemoteDisconnected

A subclass of `ConnectionResetError` and `BadStatusLine`. Raised by `HTTPConnection.getresponse` when the attempt to read the response results in no data read from the connection, indicating that the remote end has closed the connection.

<div class="versionadded">

3.5 Previously, `BadStatusLine``('')` was raised.

</div>

</div>

The constants defined in this module are:

<div class="data">

HTTP_PORT

The default port for the HTTP protocol (always `80`).

</div>

<div class="data">

HTTPS_PORT

The default port for the HTTPS protocol (always `443`).

</div>

<div class="data">

responses

This dictionary maps the HTTP 1.1 status codes to the W3C names.

Example: `http.client.responses[http.client.NOT_FOUND]` is `'Not Found'`.

</div>

See `http-status-codes` for a list of HTTP status codes that are available in this module as constants.

## HTTPConnection Objects

`HTTPConnection` instances have the following methods:

<div class="method">

HTTPConnection.request(method, url, body=None, headers={}, \*, encode_chunked=False)

This will send a request to the server using the HTTP request method *method* and the request URI *url*. The provided *url* must be an absolute path to conform with `RFC 2616 §5.1.2 <2616#section-5.1.2>` (unless connecting to an HTTP proxy server or using the `OPTIONS` or `CONNECT` methods).

If *body* is specified, the specified data is sent after the headers are finished. It may be a `str`, a `bytes-like object`, an open `file object`, or an iterable of `bytes`. If *body* is a string, it is encoded as ISO-8859-1, the default for HTTP. If it is a bytes-like object, the bytes are sent as is. If it is a `file
object`, the contents of the file is sent; this file object should support at least the `read()` method. If the file object is an instance of `io.TextIOBase`, the data returned by the `read()` method will be encoded as ISO-8859-1, otherwise the data returned by `read()` is sent as is. If *body* is an iterable, the elements of the iterable are sent as is until the iterable is `exhausted`.

The *headers* argument should be a mapping of extra HTTP headers to send with the request. A `Host header <2616#section-14.23>` must be provided to conform with `RFC 2616 §5.1.2 <2616#section-5.1.2>` (unless connecting to an HTTP proxy server or using the `OPTIONS` or `CONNECT` methods).

If *headers* contains neither Content-Length nor Transfer-Encoding, but there is a request body, one of those header fields will be added automatically. If *body* is `None`, the Content-Length header is set to `0` for methods that expect a body (`PUT`, `POST`, and `PATCH`). If *body* is a string or a bytes-like object that is not also a `file <file object>`, the Content-Length header is set to its length. Any other type of *body* (files and iterables in general) will be chunk-encoded, and the Transfer-Encoding header will automatically be set instead of Content-Length.

The *encode_chunked* argument is only relevant if Transfer-Encoding is specified in *headers*. If *encode_chunked* is `False`, the HTTPConnection object assumes that all encoding is handled by the calling code. If it is `True`, the body will be chunk-encoded.

For example, to perform a `GET` request to `https://docs.python.org/3/`:

    >>> import http.client
    >>> host = "docs.python.org"
    >>> conn = http.client.HTTPSConnection(host)
    >>> conn.request("GET", "/3/", headers={"Host": host})
    >>> response = conn.getresponse()
    >>> print(response.status, response.reason)
    200 OK

> [!NOTE]
> Chunked transfer encoding has been added to the HTTP protocol version 1.1. Unless the HTTP server is known to handle HTTP 1.1, the caller must either specify the Content-Length, or must pass a `str` or bytes-like object that is not also a file as the body representation.

> [!NOTE]
> Note that you must have read the whole response or call `close` if `getresponse` raised an non-`ConnectionError` exception before you can send a new request to the server.

<div class="versionchanged">

3.2 *body* can now be an iterable.

</div>

<div class="versionchanged">

3.6 If neither Content-Length nor Transfer-Encoding are set in *headers*, file and iterable *body* objects are now chunk-encoded. The *encode_chunked* argument was added. No attempt is made to determine the Content-Length for file objects.

</div>

</div>

<div class="method">

HTTPConnection.getresponse()

Should be called after a request is sent to get the response from the server. Returns an `HTTPResponse` instance.

<div class="versionchanged">

3.5 If a `ConnectionError` or subclass is raised, the `HTTPConnection` object will be ready to reconnect when a new request is sent.

Note that this does not apply to `OSError`s raised by the underlying socket. Instead the caller is responsible to call `close` on the existing connection.

</div>

</div>

<div class="method">

HTTPConnection.set_debuglevel(level)

Set the debugging level. The default debug level is `0`, meaning no debugging output is printed. Any value greater than `0` will cause all currently defined debug output to be printed to stdout. The `debuglevel` is passed to any new `HTTPResponse` objects that are created.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

HTTPConnection.set_tunnel(host, port=None, headers=None)

Set the host and the port for HTTP Connect Tunnelling. This allows running the connection through a proxy server.

The *host* and *port* arguments specify the endpoint of the tunneled connection (i.e. the address included in the CONNECT request, *not* the address of the proxy server).

The *headers* argument should be a mapping of extra HTTP headers to send with the CONNECT request. Non-string header values are converted to strings via `str`.

As HTTP/1.1 is used for HTTP CONNECT tunnelling request, [as per the RFC](https://datatracker.ietf.org/doc/html/rfc7231#section-4.3.6), a HTTP `Host:` header must be provided, matching the authority-form of the request target provided as the destination for the CONNECT request. If a HTTP `Host:` header is not provided via the headers argument, one is generated and transmitted automatically.

For example, to tunnel through a HTTPS proxy server running locally on port 8080, we would pass the address of the proxy to the `HTTPSConnection` constructor, and the address of the host that we eventually want to reach to the `~HTTPConnection.set_tunnel` method:

    >>> import http.client
    >>> conn = http.client.HTTPSConnection("localhost", 8080)
    >>> conn.set_tunnel("www.python.org")
    >>> conn.request("HEAD","/index.html")

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.12 HTTP CONNECT tunnelling requests use protocol HTTP/1.1, upgraded from protocol HTTP/1.0. `Host:` HTTP headers are mandatory for HTTP/1.1, so one will be automatically generated and transmitted if not provided in the headers argument.

</div>

</div>

<div class="method">

HTTPConnection.get_proxy_response_headers()

Returns a dictionary with the headers of the response received from the proxy server to the CONNECT request.

If the CONNECT request was not sent, the method returns `None`.

<div class="versionadded">

3.12

</div>

</div>

<div class="method">

HTTPConnection.connect()

Connect to the server specified when the object was created. By default, this is called automatically when making a request if the client does not already have a connection.

<div class="audit-event">

http.client.connect self,host,port http.client.HTTPConnection.connect

</div>

</div>

<div class="method">

HTTPConnection.close()

Close the connection to the server.

</div>

<div class="attribute">

HTTPConnection.blocksize

Buffer size in bytes for sending a file-like message body.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

HTTPConnection.max_response_headers

The maximum number of allowed response headers to help prevent denial-of-service attacks. By default, the maximum number of allowed headers is set to 100. The same limit applies to the trailer section of a chunked response.

<div class="versionadded">

3.15

</div>

</div>

As an alternative to using the `~HTTPConnection.request` method described above, you can also send your request step by step, by using the four functions below.

<div class="method">

HTTPConnection.putrequest(method, url, skip_host=False, skip_accept_encoding=False)

This should be the first call after the connection to the server has been made. It sends a line to the server consisting of the *method* string, the *url* string, and the HTTP version (`HTTP/1.1`). To disable automatic sending of `Host:` or `Accept-Encoding:` headers (for example to accept additional content encodings), specify *skip_host* or *skip_accept_encoding* with non-False values.

</div>

<div class="method">

HTTPConnection.putheader(header, argument\[, ...\])

Send an `822`-style header to the server. It sends a line to the server consisting of the header, a colon and a space, and the first argument. If more arguments are given, continuation lines are sent, each consisting of a tab and an argument.

</div>

<div class="method">

HTTPConnection.endheaders(message_body=None, \*, encode_chunked=False)

Send a blank line to the server, signalling the end of the headers. The optional *message_body* argument can be used to pass a message body associated with the request.

If *encode_chunked* is `True`, the result of each iteration of *message_body* will be chunk-encoded as specified in `7230`, Section 3.3.1. How the data is encoded is dependent on the type of *message_body*. If *message_body* implements the `buffer interface
<bufferobjects>` the encoding will result in a single chunk. If *message_body* is a `collections.abc.Iterable`, each iteration of *message_body* will result in a chunk. If *message_body* is a `file object`, each call to `.read()` will result in a chunk. The method automatically signals the end of the chunk-encoded data immediately after *message_body*.

> [!NOTE]
> Due to the chunked encoding specification, empty chunks yielded by an iterator body will be ignored by the chunk-encoder. This is to avoid premature termination of the read of the request by the target server due to malformed encoding.

<div class="versionchanged">

3.6 Added chunked encoding support and the *encode_chunked* parameter.

</div>

</div>

<div class="method">

HTTPConnection.send(data)

Send data to the server. This should be used directly only after the `endheaders` method has been called and before `getresponse` is called.

<div class="audit-event">

http.client.send self,data http.client.HTTPConnection.send

</div>

</div>

## HTTPResponse Objects

An `HTTPResponse` instance wraps the HTTP response from the server. It provides access to the request headers and the entity body. The response is an iterable object and can be used in a with statement.

<div class="versionchanged">

3.5 The `io.BufferedIOBase` interface is now implemented and all of its reader operations are supported.

</div>

<div class="method">

HTTPResponse.read(\[amt\])

Reads and returns the response body, or up to the next *amt* bytes.

</div>

<div class="method">

HTTPResponse.readinto(b)

Reads up to the next len(b) bytes of the response body into the buffer *b*. Returns the number of bytes read.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

HTTPResponse.getheader(name, default=None)

Return the value of the header *name*, or *default* if there is no header matching *name*. If there is more than one header with the name *name*, return all of the values joined by ', '. If *default* is any iterable other than a single string, its elements are similarly returned joined by commas.

</div>

<div class="method">

HTTPResponse.getheaders()

Return a list of (header, value) tuples.

</div>

<div class="method">

HTTPResponse.fileno()

Return the `fileno` of the underlying socket.

</div>

<div class="attribute">

HTTPResponse.msg

A `http.client.HTTPMessage` instance containing the response headers. `http.client.HTTPMessage` is a subclass of `email.message.Message`.

</div>

<div class="attribute">

HTTPResponse.version

HTTP protocol version used by server. 10 for HTTP/1.0, 11 for HTTP/1.1.

</div>

<div class="attribute">

HTTPResponse.url

URL of the resource retrieved, commonly used to determine if a redirect was followed.

</div>

<div class="attribute">

HTTPResponse.headers

Headers of the response in the form of an `email.message.EmailMessage` instance.

</div>

<div class="attribute">

HTTPResponse.status

Status code returned by server.

</div>

<div class="attribute">

HTTPResponse.reason

Reason phrase returned by server.

</div>

<div class="attribute">

HTTPResponse.debuglevel

A debugging hook. If `debuglevel` is greater than zero, messages will be printed to stdout as the response is read and parsed.

</div>

<div class="attribute">

HTTPResponse.closed

Is `True` if the stream is closed.

</div>

<div class="method">

HTTPResponse.geturl()

<div class="deprecated">

3.9 Deprecated in favor of `~HTTPResponse.url`.

</div>

</div>

<div class="method">

HTTPResponse.info()

<div class="deprecated">

3.9 Deprecated in favor of `~HTTPResponse.headers`.

</div>

</div>

<div class="method">

HTTPResponse.getcode()

<div class="deprecated">

3.9 Deprecated in favor of `~HTTPResponse.status`.

</div>

</div>

## Examples

Here is an example session that uses the `GET` method:

    >>> import http.client
    >>> conn = http.client.HTTPSConnection("www.python.org")
    >>> conn.request("GET", "/")
    >>> r1 = conn.getresponse()
    >>> print(r1.status, r1.reason)
    200 OK
    >>> data1 = r1.read()  # This will return entire content.
    >>> # The following example demonstrates reading data in chunks.
    >>> conn.request("GET", "/")
    >>> r1 = conn.getresponse()
    >>> while chunk := r1.read(200):
    ...     print(repr(chunk))
    b'<!doctype html>\n<!--[if"...
    ...
    >>> # Example of an invalid request
    >>> conn = http.client.HTTPSConnection("docs.python.org")
    >>> conn.request("GET", "/parrot.spam")
    >>> r2 = conn.getresponse()
    >>> print(r2.status, r2.reason)
    404 Not Found
    >>> data2 = r2.read()
    >>> conn.close()

Here is an example session that uses the `HEAD` method. Note that the `HEAD` method never returns any data. :

    >>> import http.client
    >>> conn = http.client.HTTPSConnection("www.python.org")
    >>> conn.request("HEAD", "/")
    >>> res = conn.getresponse()
    >>> print(res.status, res.reason)
    200 OK
    >>> data = res.read()
    >>> print(len(data))
    0
    >>> data == b''
    True

Here is an example session that uses the `POST` method:

    >>> import http.client, urllib.parse
    >>> params = urllib.parse.urlencode({'@number': 12524, '@type': 'issue', '@action': 'show'})
    >>> headers = {"Content-type": "application/x-www-form-urlencoded",
    ...            "Accept": "text/plain"}
    >>> conn = http.client.HTTPConnection("bugs.python.org")
    >>> conn.request("POST", "", params, headers)
    >>> response = conn.getresponse()
    >>> print(response.status, response.reason)
    302 Found
    >>> data = response.read()
    >>> data
    b'Redirecting to <a href="https://bugs.python.org/issue12524">https://bugs.python.org/issue12524</a>'
    >>> conn.close()

Client side HTTP `PUT` requests are very similar to `POST` requests. The difference lies only on the server side where HTTP servers will allow resources to be created via `PUT` requests. It should be noted that custom HTTP methods are also handled in `urllib.request.Request` by setting the appropriate method attribute. Here is an example session that uses the `PUT` method:

    >>> # This creates an HTTP request
    >>> # with the content of BODY as the enclosed representation
    >>> # for the resource http://localhost:8080/file
    ...
    >>> import http.client
    >>> BODY = "***filecontents***"
    >>> conn = http.client.HTTPConnection("localhost", 8080)
    >>> conn.request("PUT", "/file", BODY)
    >>> response = conn.getresponse()
    >>> print(response.status, response.reason)
    200, OK

## HTTPMessage Objects

<div class="HTTPMessage(email.message.Message)">

An `http.client.HTTPMessage` instance holds the headers from an HTTP response. It is implemented using the `email.message.Message` class.

</div>

# `!http.cookiejar` --- Cookie handling for HTTP clients

<div class="module" data-synopsis="Classes for automatic handling of HTTP cookies.">

http.cookiejar

</div>

**Source code:** `Lib/http/cookiejar.py`

------------------------------------------------------------------------

The `!http.cookiejar` module defines classes for automatic handling of HTTP cookies. It is useful for accessing websites that require small pieces of data -- `cookies` -- to be set on the client machine by an HTTP response from a web server, and then returned to the server in later HTTP requests.

Both the regular Netscape cookie protocol and the protocol defined by `2965` are handled. RFC 2965 handling is switched off by default. `2109` cookies are parsed as Netscape cookies and subsequently treated either as Netscape or RFC 2965 cookies according to the 'policy' in effect. Note that the great majority of cookies on the internet are Netscape cookies. `!http.cookiejar` attempts to follow the de-facto Netscape cookie protocol (which differs substantially from that set out in the original Netscape specification), including taking note of the `max-age` and `port` cookie-attributes introduced with RFC 2965.

> [!NOTE]
> The various named parameters found in `Set-Cookie` and `Set-Cookie2` headers (for example, `domain` and `expires`) are conventionally referred to as `attributes`. To distinguish them from Python attributes, the documentation for this module uses the term `cookie-attribute` instead.

The module defines the following exception:

<div class="exception">

LoadError

Instances of `FileCookieJar` raise this exception on failure to load cookies from a file. `LoadError` is a subclass of `OSError`.

<div class="versionchanged">

3.3 `LoadError` used to be a subtype of `IOError`, which is now an alias of `OSError`.

</div>

</div>

The following classes are provided:

<div class="CookieJar(policy=None)">

*policy* is an object implementing the `CookiePolicy` interface.

The `CookieJar` class stores HTTP cookies. It extracts cookies from HTTP requests, and returns them in HTTP responses. `CookieJar` instances automatically expire contained cookies when necessary. Subclasses are also responsible for storing and retrieving cookies from a file or database.

</div>

<div class="FileCookieJar(filename=None, delayload=None, policy=None)">

*policy* is an object implementing the `CookiePolicy` interface. For the other arguments, see the documentation for the corresponding attributes.

A `CookieJar` which can load cookies from, and perhaps save cookies to, a file on disk. Cookies are **NOT** loaded from the named file until either the `load` or `revert` method is called. Subclasses of this class are documented in section `file-cookie-jar-classes`.

This should not be initialized directly – use its subclasses below instead.

<div class="versionchanged">

3.8

The filename parameter supports a `path-like object`.

</div>

</div>

<div class="CookiePolicy()">

This class is responsible for deciding whether each cookie should be accepted from / returned to the server.

</div>

<div class="DefaultCookiePolicy( blocked_domains=None, allowed_domains=None, netscape=True, rfc2965=False, rfc2109_as_netscape=None, hide_cookie2=False, strict_domain=False, strict_rfc2965_unverifiable=True, strict_ns_unverifiable=False, strict_ns_domain=DefaultCookiePolicy.DomainLiberal, strict_ns_set_initial_dollar=False, strict_ns_set_path=False, secure_protocols=(&quot;https&quot;, &quot;wss&quot;) )">

Constructor arguments should be passed as keyword arguments only. *blocked_domains* is a sequence of domain names that we never accept cookies from, nor return cookies to. *allowed_domains* if not `None`, this is a sequence of the only domains for which we accept and return cookies. *secure_protocols* is a sequence of protocols for which secure cookies can be added to. By default *https* and *wss* (secure websocket) are considered secure protocols. For all other arguments, see the documentation for `CookiePolicy` and `DefaultCookiePolicy` objects.

`DefaultCookiePolicy` implements the standard accept / reject rules for Netscape and `2965` cookies. By default, `2109` cookies (that is, cookies received in a `Set-Cookie` header with a version cookie-attribute of 1) are treated according to the RFC 2965 rules. However, if RFC 2965 handling is turned off or `rfc2109_as_netscape` is `True`, RFC 2109 cookies are 'downgraded' by the `CookieJar` instance to Netscape cookies, by setting the `~Cookie.version` attribute of the `Cookie` instance to 0. `DefaultCookiePolicy` also provides some parameters to allow some fine-tuning of policy.

</div>

<div class="Cookie()">

This class represents Netscape, `2109` and `2965` cookies. It is not expected that users of `!http.cookiejar` construct their own `Cookie` instances. Instead, if necessary, call `~CookieJar.make_cookies` on a `CookieJar` instance.

</div>

<div class="seealso">

Module `urllib.request`  
URL opening with automatic cookie handling.

Module `http.cookies`  
HTTP cookie classes, principally useful for server-side code. The `!http.cookiejar` and `http.cookies` modules do not depend on each other.

<https://curl.se/rfc/cookie_spec.html>  
The specification of the original Netscape cookie protocol. Though this is still the dominant protocol, the 'Netscape cookie protocol' implemented by all the major browsers (and `!http.cookiejar`) only bears a passing resemblance to the one sketched out in `cookie_spec.html`.

`2109` - HTTP State Management Mechanism  
Obsoleted by `2965`. Uses `Set-Cookie` with version=1.

`2965` - HTTP State Management Mechanism  
The Netscape protocol with the bugs fixed. Uses `Set-Cookie2` in place of `Set-Cookie`. Not widely used.

<https://kristol.org/cookie/errata.html>  
Unfinished errata to `2965`.

`2964` - Use of HTTP State Management

</div>

## CookieJar and FileCookieJar objects

`CookieJar` objects support the `iterator` protocol for iterating over contained `Cookie` objects.

`CookieJar` has the following methods:

<div class="method">

CookieJar.add_cookie_header(request)

Add correct `Cookie` header to *request*.

If policy allows (that is, the `~CookiePolicy.rfc2965` and `~CookiePolicy.hide_cookie2` attributes of the `CookieJar`'s `CookiePolicy` instance are true and false respectively), the `Cookie2` header is also added when appropriate.

The *request* object (usually a `urllib.request.Request` instance) must support the methods `~urllib.request.Request.get_full_url`, `~urllib.request.Request.has_header`, `~urllib.request.Request.get_header`, `~urllib.request.Request.header_items`, `~urllib.request.Request.add_unredirected_header` and the attributes `~urllib.request.Request.host`, `~urllib.request.Request.type`, `~urllib.request.Request.unverifiable` and `~urllib.request.Request.origin_req_host` as documented by `urllib.request`.

<div class="versionchanged">

3.3

*request* object needs `~urllib.request.Request.origin_req_host` attribute. Dependency on a deprecated method `!get_origin_req_host` has been removed.

</div>

</div>

<div class="method">

CookieJar.extract_cookies(response, request)

Extract cookies from HTTP *response* and store them in the `CookieJar`, where allowed by policy.

The `CookieJar` will look for allowable `Set-Cookie` and `Set-Cookie2` headers in the *response* argument, and store cookies as appropriate (subject to the `CookiePolicy.set_ok` method's approval).

The *response* object (usually the result of a call to `urllib.request.urlopen`, or similar) should support an `~http.client.HTTPResponse.info` method, which returns an `email.message.Message` instance.

The *request* object (usually a `urllib.request.Request` instance) must support the method `~urllib.request.Request.get_full_url` and the attributes `~urllib.request.Request.host`, `~urllib.request.Request.unverifiable` and `~urllib.request.Request.origin_req_host`, as documented by `urllib.request`. The request is used to set default values for cookie-attributes as well as for checking that the cookie is allowed to be set.

<div class="versionchanged">

3.3

*request* object needs `~urllib.request.Request.origin_req_host` attribute. Dependency on a deprecated method `!get_origin_req_host` has been removed.

</div>

</div>

<div class="method">

CookieJar.set_policy(policy)

Set the `CookiePolicy` instance to be used.

</div>

<div class="method">

CookieJar.make_cookies(response, request)

Return sequence of `Cookie` objects extracted from *response* object.

See the documentation for `extract_cookies` for the interfaces required of the *response* and *request* arguments.

</div>

<div class="method">

CookieJar.set_cookie_if_ok(cookie, request)

Set a `Cookie` if policy says it's OK to do so.

</div>

<div class="method">

CookieJar.set_cookie(cookie)

Set a `Cookie`, without checking with policy to see whether or not it should be set.

</div>

<div class="method">

CookieJar.clear(\[domain\[, path\[, name\]\]\])

Clear some cookies.

If invoked without arguments, clear all cookies. If given a single argument, only cookies belonging to that *domain* will be removed. If given two arguments, cookies belonging to the specified *domain* and URL *path* are removed. If given three arguments, then the cookie with the specified *domain*, *path* and *name* is removed.

Raises `KeyError` if no matching cookie exists.

</div>

<div class="method">

CookieJar.clear_session_cookies()

Discard all session cookies.

Discards all contained cookies that have a true `~Cookie.discard` attribute (usually because they had either no `max-age` or `expires` cookie-attribute, or an explicit `discard` cookie-attribute). For interactive browsers, the end of a session usually corresponds to closing the browser window.

Note that the `~FileCookieJar.save` method won't save session cookies anyway, unless you ask otherwise by passing a true *ignore_discard* argument.

</div>

`FileCookieJar` implements the following additional methods:

<div class="method">

FileCookieJar.save(filename=None, ignore_discard=False, ignore_expires=False)

Save cookies to a file.

This base class raises `NotImplementedError`. Subclasses may leave this method unimplemented.

*filename* is the name of file in which to save cookies. If *filename* is not specified, `self.filename <FileCookieJar.filename>` is used (whose default is the value passed to the constructor, if any); if `self.filename <FileCookieJar.filename>` is `None`, `ValueError` is raised.

*ignore_discard*: save even cookies set to be discarded. *ignore_expires*: save even cookies that have expired

The file is overwritten if it already exists, thus wiping all the cookies it contains. Saved cookies can be restored later using the `load` or `revert` methods.

</div>

<div class="method">

FileCookieJar.load(filename=None, ignore_discard=False, ignore_expires=False)

Load cookies from a file.

Old cookies are kept unless overwritten by newly loaded ones.

Arguments are as for `save`.

The named file must be in the format understood by the class, or `LoadError` will be raised. Also, `OSError` may be raised, for example if the file does not exist.

<div class="versionchanged">

3.3 `IOError` used to be raised, it is now an alias of `OSError`.

</div>

</div>

<div class="method">

FileCookieJar.revert(filename=None, ignore_discard=False, ignore_expires=False)

Clear all cookies and reload cookies from a saved file.

`revert` can raise the same exceptions as `load`. If there is a failure, the object's state will not be altered.

</div>

`FileCookieJar` instances have the following public attributes:

<div class="attribute">

FileCookieJar.filename

Filename of default file in which to keep cookies. This attribute may be assigned to.

</div>

<div class="attribute">

FileCookieJar.delayload

If true, load cookies lazily from disk. This attribute should not be assigned to. This is only a hint, since this only affects performance, not behaviour (unless the cookies on disk are changing). A `CookieJar` object may ignore it. None of the `FileCookieJar` classes included in the standard library lazily loads cookies.

</div>

## FileCookieJar subclasses and co-operation with web browsers

The following `CookieJar` subclasses are provided for reading and writing.

<div class="MozillaCookieJar(filename=None, delayload=None, policy=None)">

A `FileCookieJar` that can load from and save cookies to disk in the Mozilla `cookies.txt` file format (which is also used by curl and the Lynx and Netscape browsers).

> [!NOTE]
> This loses information about `2965` cookies, and also about newer or non-standard cookie-attributes such as `port`.

> [!WARNING]
> Back up your cookies before saving if you have cookies whose loss / corruption would be inconvenient (there are some subtleties which may lead to slight changes in the file over a load / save round-trip).

Also note that cookies saved while Mozilla is running will get clobbered by Mozilla.

</div>

<div class="LWPCookieJar(filename=None, delayload=None, policy=None)">

A `FileCookieJar` that can load from and save cookies to disk in format compatible with the libwww-perl library's `Set-Cookie3` file format. This is convenient if you want to store cookies in a human-readable file.

<div class="versionchanged">

3.8

The filename parameter supports a `path-like object`.

</div>

</div>

## CookiePolicy objects

Objects implementing the `CookiePolicy` interface have the following methods:

<div class="method">

CookiePolicy.set_ok(cookie, request)

Return boolean value indicating whether cookie should be accepted from server.

*cookie* is a `Cookie` instance. *request* is an object implementing the interface defined by the documentation for `CookieJar.extract_cookies`.

</div>

<div class="method">

CookiePolicy.return_ok(cookie, request)

Return boolean value indicating whether cookie should be returned to server.

*cookie* is a `Cookie` instance. *request* is an object implementing the interface defined by the documentation for `CookieJar.add_cookie_header`.

</div>

<div class="method">

CookiePolicy.domain_return_ok(domain, request)

Return `False` if cookies should not be returned, given cookie domain.

This method is an optimization. It removes the need for checking every cookie with a particular domain (which might involve reading many files). Returning true from `domain_return_ok` and `path_return_ok` leaves all the work to `return_ok`.

If `domain_return_ok` returns true for the cookie domain, `path_return_ok` is called for the cookie path. Otherwise, `path_return_ok` and `return_ok` are never called for that cookie domain. If `path_return_ok` returns true, `return_ok` is called with the `Cookie` object itself for a full check. Otherwise, `return_ok` is never called for that cookie path.

Note that `domain_return_ok` is called for every *cookie* domain, not just for the *request* domain. For example, the function might be called with both `".example.com"` and `"www.example.com"` if the request domain is `"www.example.com"`. The same goes for `path_return_ok`.

The *request* argument is as documented for `return_ok`.

</div>

<div class="method">

CookiePolicy.path_return_ok(path, request)

Return `False` if cookies should not be returned, given cookie path.

See the documentation for `domain_return_ok`.

</div>

In addition to implementing the methods above, implementations of the `CookiePolicy` interface must also supply the following attributes, indicating which protocols should be used, and how. All of these attributes may be assigned to.

<div class="attribute">

CookiePolicy.netscape

Implement Netscape protocol.

</div>

<div class="attribute">

CookiePolicy.rfc2965

Implement `2965` protocol.

</div>

<div class="attribute">

CookiePolicy.hide_cookie2

Don't add `Cookie2` header to requests (the presence of this header indicates to the server that we understand `2965` cookies).

</div>

The most useful way to define a `CookiePolicy` class is by subclassing from `DefaultCookiePolicy` and overriding some or all of the methods above. `CookiePolicy` itself may be used as a 'null policy' to allow setting and receiving any and all cookies (this is unlikely to be useful).

## DefaultCookiePolicy objects

Implements the standard rules for accepting and returning cookies.

Both `2965` and Netscape cookies are covered. RFC 2965 handling is switched off by default.

The easiest way to provide your own policy is to override this class and call its methods in your overridden implementations before adding your own additional checks:

    import http.cookiejar
    class MyCookiePolicy(http.cookiejar.DefaultCookiePolicy):
        def set_ok(self, cookie, request):
            if not http.cookiejar.DefaultCookiePolicy.set_ok(self, cookie, request):
                return False
            if i_dont_want_to_store_this_cookie(cookie):
                return False
            return True

In addition to the features required to implement the `CookiePolicy` interface, this class allows you to block and allow domains from setting and receiving cookies. There are also some strictness switches that allow you to tighten up the rather loose Netscape protocol rules a little bit (at the cost of blocking some benign cookies).

A domain blocklist and allowlist is provided (both off by default). Only domains not in the blocklist and present in the allowlist (if the allowlist is active) participate in cookie setting and returning. Use the *blocked_domains* constructor argument, and `~DefaultCookiePolicy.blocked_domains` and `~DefaultCookiePolicy.set_blocked_domains` methods (and the corresponding argument and methods for *allowed_domains*). If you set an allowlist, you can turn it off again by setting it to `None`.

Domains in block or allow lists that do not start with a dot must equal the cookie domain to be matched. For example, `"example.com"` matches a blocklist entry of `"example.com"`, but `"www.example.com"` does not. Domains that do start with a dot are matched by more specific domains too. For example, both `"www.example.com"` and `"www.coyote.example.com"` match `".example.com"` (but `"example.com"` itself does not). IP addresses are an exception, and must match exactly. For example, if blocked_domains contains `"192.168.1.2"` and `".168.1.2"`, 192.168.1.2 is blocked, but 193.168.1.2 is not.

`DefaultCookiePolicy` implements the following additional methods:

<div class="method">

DefaultCookiePolicy.blocked_domains()

Return the sequence of blocked domains (as a tuple).

</div>

<div class="method">

DefaultCookiePolicy.set_blocked_domains(blocked_domains)

Set the sequence of blocked domains.

</div>

<div class="method">

DefaultCookiePolicy.is_blocked(domain)

Return `True` if *domain* is on the blocklist for setting or receiving cookies.

</div>

<div class="method">

DefaultCookiePolicy.allowed_domains()

Return `None`, or the sequence of allowed domains (as a tuple).

</div>

<div class="method">

DefaultCookiePolicy.set_allowed_domains(allowed_domains)

Set the sequence of allowed domains, or `None`.

</div>

<div class="method">

DefaultCookiePolicy.is_not_allowed(domain)

Return `True` if *domain* is not on the allowlist for setting or receiving cookies.

</div>

`DefaultCookiePolicy` instances have the following attributes, which are all initialised from the constructor arguments of the same name, and which may all be assigned to.

<div class="attribute">

DefaultCookiePolicy.rfc2109_as_netscape

If true, request that the `CookieJar` instance downgrade `2109` cookies (that is, cookies received in a `Set-Cookie` header with a version cookie-attribute of 1) to Netscape cookies by setting the version attribute of the `Cookie` instance to 0. The default value is `None`, in which case RFC 2109 cookies are downgraded if and only if `2965` handling is turned off. Therefore, RFC 2109 cookies are downgraded by default.

</div>

General strictness switches:

<div class="attribute">

DefaultCookiePolicy.strict_domain

Don't allow sites to set two-component domains with country-code top-level domains like `.co.uk`, `.gov.uk`, `.co.nz`.etc. This is far from perfect and isn't guaranteed to work!

</div>

`2965` protocol strictness switches:

<div class="attribute">

DefaultCookiePolicy.strict_rfc2965_unverifiable

Follow `2965` rules on unverifiable transactions (usually, an unverifiable transaction is one resulting from a redirect or a request for an image hosted on another site). If this is false, cookies are *never* blocked on the basis of verifiability

</div>

Netscape protocol strictness switches:

<div class="attribute">

DefaultCookiePolicy.strict_ns_unverifiable

Apply `2965` rules on unverifiable transactions even to Netscape cookies.

</div>

<div class="attribute">

DefaultCookiePolicy.strict_ns_domain

Flags indicating how strict to be with domain-matching rules for Netscape cookies. See below for acceptable values.

</div>

<div class="attribute">

DefaultCookiePolicy.strict_ns_set_initial_dollar

Ignore cookies in Set-Cookie: headers that have names starting with `'$'`.

</div>

<div class="attribute">

DefaultCookiePolicy.strict_ns_set_path

Don't allow setting cookies whose path doesn't path-match request URI.

</div>

`~DefaultCookiePolicy.strict_ns_domain` is a collection of flags. Its value is constructed by or-ing together (for example, `DomainStrictNoDots|DomainStrictNonDomain` means both flags are set).

<div class="attribute">

DefaultCookiePolicy.DomainStrictNoDots

When setting cookies, the 'host prefix' must not contain a dot (for example, `www.foo.bar.com` can't set a cookie for `.bar.com`, because `www.foo` contains a dot).

</div>

<div class="attribute">

DefaultCookiePolicy.DomainStrictNonDomain

Cookies that did not explicitly specify a `domain` cookie-attribute can only be returned to a domain equal to the domain that set the cookie (for example, `spam.example.com` won't be returned cookies from `example.com` that had no `domain` cookie-attribute).

</div>

<div class="attribute">

DefaultCookiePolicy.DomainRFC2965Match

When setting cookies, require a full `2965` domain-match.

</div>

The following attributes are provided for convenience, and are the most useful combinations of the above flags:

<div class="attribute">

DefaultCookiePolicy.DomainLiberal

Equivalent to 0 (that is, all of the above Netscape domain strictness flags switched off).

</div>

<div class="attribute">

DefaultCookiePolicy.DomainStrict

Equivalent to `DomainStrictNoDots|DomainStrictNonDomain`.

</div>

## Cookie objects

`Cookie` instances have Python attributes roughly corresponding to the standard cookie-attributes specified in the various cookie standards. The correspondence is not one-to-one, because there are complicated rules for assigning default values, because the `max-age` and `expires` cookie-attributes contain equivalent information, and because `2109` cookies may be 'downgraded' by `!http.cookiejar` from version 1 to version 0 (Netscape) cookies.

Assignment to these attributes should not be necessary other than in rare circumstances in a `CookiePolicy` method. The class does not enforce internal consistency, so you should know what you're doing if you do that.

<div class="attribute">

Cookie.version

Integer or `None`. Netscape cookies have `version` 0. `2965` and `2109` cookies have a `version` cookie-attribute of 1. However, note that `!http.cookiejar` may 'downgrade' RFC 2109 cookies to Netscape cookies, in which case `version` is 0.

</div>

<div class="attribute">

Cookie.name

Cookie name (a string).

</div>

<div class="attribute">

Cookie.value

Cookie value (a string), or `None`.

</div>

<div class="attribute">

Cookie.port

String representing a port or a set of ports (for example, '80', or '80,8080'), or `None`.

</div>

<div class="attribute">

Cookie.domain

Cookie domain (a string).

</div>

<div class="attribute">

Cookie.path

Cookie path (a string, for example, `'/acme/rocket_launchers'`).

</div>

<div class="attribute">

Cookie.secure

`True` if cookie should only be returned over a secure connection.

</div>

<div class="attribute">

Cookie.expires

Integer expiry date in seconds since epoch, or `None`. See also the `is_expired` method.

</div>

<div class="attribute">

Cookie.discard

`True` if this is a session cookie.

</div>

<div class="attribute">

Cookie.comment

String comment from the server explaining the function of this cookie, or `None`.

</div>

<div class="attribute">

Cookie.comment_url

URL linking to a comment from the server explaining the function of this cookie, or `None`.

</div>

<div class="attribute">

Cookie.rfc2109

`True` if this cookie was received as an `2109` cookie (that is, the cookie arrived in a `Set-Cookie` header, and the value of the Version cookie-attribute in that header was 1). This attribute is provided because `!http.cookiejar` may 'downgrade' RFC 2109 cookies to Netscape cookies, in which case `version` is 0.

</div>

<div class="attribute">

Cookie.port_specified

`True` if a port or set of ports was explicitly specified by the server (in the `Set-Cookie` / `Set-Cookie2` header).

</div>

<div class="attribute">

Cookie.domain_specified

`True` if a domain was explicitly specified by the server.

</div>

<div class="attribute">

Cookie.domain_initial_dot

`True` if the domain explicitly specified by the server began with a dot (`'.'`).

</div>

Cookies may have additional non-standard cookie-attributes. These may be accessed using the following methods:

<div class="method">

Cookie.has_nonstandard_attr(name)

Return `True` if cookie has the named cookie-attribute.

</div>

<div class="method">

Cookie.get_nonstandard_attr(name, default=None)

If cookie has the named cookie-attribute, return its value. Otherwise, return *default*.

</div>

<div class="method">

Cookie.set_nonstandard_attr(name, value)

Set the value of the named cookie-attribute.

</div>

The `Cookie` class also defines the following method:

<div class="method">

Cookie.is_expired(now=None)

`True` if cookie has passed the time at which the server requested it should expire. If *now* is given (in seconds since the epoch), return whether the cookie has expired at the specified time.

</div>

## Examples

The first example shows the most common usage of `!http.cookiejar`:

    import http.cookiejar, urllib.request
    cj = http.cookiejar.CookieJar()
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    r = opener.open("http://example.com/")

This example illustrates how to open a URL using your Netscape, Mozilla, or Lynx cookies (assumes Unix/Netscape convention for location of the cookies file):

    import os, http.cookiejar, urllib.request
    cj = http.cookiejar.MozillaCookieJar()
    cj.load(os.path.join(os.path.expanduser("~"), ".netscape", "cookies.txt"))
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    r = opener.open("http://example.com/")

The next example illustrates the use of `DefaultCookiePolicy`. Turn on `2965` cookies, be more strict about domains when setting and returning Netscape cookies, and block some domains from setting cookies or having them returned:

    import urllib.request
    from http.cookiejar import CookieJar, DefaultCookiePolicy
    policy = DefaultCookiePolicy(
        rfc2965=True, strict_ns_domain=DefaultCookiePolicy.DomainStrict,
        blocked_domains=["ads.net", ".ads.net"])
    cj = CookieJar(policy)
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    r = opener.open("http://example.com/")

# `!http.cookies` --- HTTP state management

<div class="module" data-synopsis="Support for HTTP state management (cookies).">

http.cookies

</div>

**Source code:** `Lib/http/cookies.py`

------------------------------------------------------------------------

The `!http.cookies` module defines classes for abstracting the concept of cookies, an HTTP state management mechanism. It supports both simple string-only cookies, and provides an abstraction for having any serializable data-type as cookie value.

The module formerly strictly applied the parsing rules described in the `2109` and `2068` specifications. It has since been discovered that MSIE 3.0x didn't follow the character rules outlined in those specs; many current-day browsers and servers have also relaxed parsing rules when it comes to cookie handling. As a result, this module now uses parsing rules that are a bit less strict than they once were.

The character set, `string.ascii_letters`, `string.digits` and `` !#$%&'*+-.^_`|~: `` denote the set of valid characters allowed by this module in a cookie name (as `~Morsel.key`).

<div class="versionchanged">

3.3 Allowed ':' as a valid cookie name character.

</div>

> [!NOTE]
> On encountering an invalid cookie, `CookieError` is raised, so if your cookie data comes from a browser you should always prepare for invalid data and catch `CookieError` on parsing.

<div class="exception">

CookieError

Exception failing because of `2109` invalidity: incorrect attributes, incorrect `Set-Cookie` header, etc.

</div>

<div class="BaseCookie([input])">

This class is a dictionary-like object whose keys are strings and whose values are `Morsel` instances. Note that upon setting a key to a value, the value is first converted to a `Morsel` containing the key and the value.

If *input* is given, it is passed to the `load` method.

</div>

<div class="SimpleCookie([input])">

This class derives from `BaseCookie` and overrides `~BaseCookie.value_decode` and `~BaseCookie.value_encode`. `!SimpleCookie` supports strings as cookie values. When setting the value, `!SimpleCookie` calls the builtin `str` to convert the value to a string. Values received from HTTP are kept as strings.

</div>

<div class="seealso">

Module `http.cookiejar`  
HTTP cookie handling for web *clients*. The `http.cookiejar` and `!http.cookies` modules do not depend on each other.

`2109` - HTTP State Management Mechanism  
This is the state management specification implemented by this module.

</div>

## Cookie Objects

<div class="method">

BaseCookie.value_decode(val)

Return a tuple `(real_value, coded_value)` from a string representation. `real_value` can be any type. This method does no decoding in `BaseCookie` --- it exists so it can be overridden.

</div>

<div class="method">

BaseCookie.value_encode(val)

Return a tuple `(real_value, coded_value)`. *val* can be any type, but `coded_value` will always be converted to a string. This method does no encoding in `BaseCookie` --- it exists so it can be overridden.

In general, it should be the case that `value_encode` and `value_decode` are inverses on the range of *value_decode*.

</div>

<div class="method">

BaseCookie.output(attrs=None, header='Set-Cookie:', sep='rn')

Return a string representation suitable to be sent as HTTP headers. *attrs* and *header* are sent to each `Morsel`'s `~Morsel.output` method. *sep* is used to join the headers together, and is by default the combination `'\r\n'` (CRLF).

</div>

<div class="method">

BaseCookie.js_output(attrs=None)

Return an embeddable JavaScript snippet, which, if run on a browser which supports JavaScript, will act the same as if the HTTP headers was sent.

The meaning for *attrs* is the same as in `output`.

<div class="deprecated-removed">

3.15 3.19 This method generates a JavaScript snippet to set cookies in the browser, which is no longer considered a standard or recommended approach. Use `~http.cookies.BaseCookie.output` instead to generate HTTP headers.

</div>

</div>

<div class="method">

BaseCookie.load(rawdata)

If *rawdata* is a string, parse it as an `HTTP_COOKIE` and add the values found there as `Morsel`s. If it is a dictionary, it is equivalent to:

    for k, v in rawdata.items():
        cookie[k] = v

</div>

## Morsel Objects

<div class="Morsel">

Abstract a key/value pair, which has some `2109` attributes.

Morsels are dictionary-like objects, whose set of keys is constant --- the valid `2109` attributes, which are:

> <div class="attribute">
>
> expires path comment domain max-age secure version httponly samesite partitioned
>
> </div>

The attribute `httponly` specifies that the cookie is only transferred in HTTP requests, and is not accessible through JavaScript. This is intended to mitigate some forms of cross-site scripting.

The attribute `samesite` controls when the browser sends the cookie with cross-site requests. This helps to mitigate CSRF attacks. Valid values are "Strict" (only sent with same-site requests), "Lax" (sent with same-site requests and top-level navigations), and "None" (sent with same-site and cross-site requests). When using "None", the "secure" attribute must also be set, as required by modern browsers.

The attribute `partitioned` indicates to user agents that these cross-site cookies *should* only be available in the same top-level context that the cookie was first set in. For this to be accepted by the user agent, you **must** also set `Secure`.

In addition, it is recommended to use the `__Host` prefix when setting partitioned cookies to make them bound to the hostname and not the registrable domain. Read [CHIPS (Cookies Having Independent Partitioned State)](https://github.com/privacycg/CHIPS/blob/main/README.md) for full details and examples.

The keys are case-insensitive and their default value is `''`.

<div class="versionchanged">

3.5 `!__eq__` now takes `~Morsel.key` and `~Morsel.value` into account.

</div>

<div class="versionchanged">

3.7 Attributes `~Morsel.key`, `~Morsel.value` and `~Morsel.coded_value` are read-only. Use `~Morsel.set` for setting them.

</div>

<div class="versionchanged">

3.8 Added support for the `samesite` attribute.

</div>

<div class="versionchanged">

3.14 Added support for the `partitioned` attribute.

</div>

</div>

<div class="attribute">

Morsel.value

The value of the cookie.

</div>

<div class="attribute">

Morsel.coded_value

The encoded value of the cookie --- this is what should be sent.

</div>

<div class="attribute">

Morsel.key

The name of the cookie.

</div>

<div class="method">

Morsel.set(key, value, coded_value)

Set the *key*, *value* and *coded_value* attributes.

</div>

<div class="method">

Morsel.isReservedKey(K)

Whether *K* is a member of the set of keys of a `Morsel`.

</div>

<div class="method">

Morsel.output(attrs=None, header='Set-Cookie:')

Return a string representation of the Morsel, suitable to be sent as an HTTP header. By default, all the attributes are included, unless *attrs* is given, in which case it should be a list of attributes to use. *header* is by default `"Set-Cookie:"`.

</div>

<div class="method">

Morsel.js_output(attrs=None)

Return an embeddable JavaScript snippet, which, if run on a browser which supports JavaScript, will act the same as if the HTTP header was sent.

The meaning for *attrs* is the same as in `output`.

<div class="deprecated-removed">

3.15 3.19 This method generates a JavaScript snippet to set cookies in the browser, which is no longer considered a standard or recommended approach. Use `~http.cookies.Morsel.output` instead to generate HTTP headers.

</div>

</div>

<div class="method">

Morsel.OutputString(attrs=None)

Return a string representing the Morsel, without any surrounding HTTP or JavaScript.

The meaning for *attrs* is the same as in `output`.

</div>

<div class="method">

Morsel.update(values)

Update the values in the Morsel dictionary with the values in the dictionary *values*. Raise an error if any of the keys in the *values* dict is not a valid `2109` attribute.

<div class="versionchanged">

3.5 an error is raised for invalid keys.

</div>

</div>

<div class="method">

Morsel.copy(value)

Return a shallow copy of the Morsel object.

<div class="versionchanged">

3.5 return a Morsel object instead of a dict.

</div>

</div>

<div class="method">

Morsel.setdefault(key, value=None)

Raise an error if key is not a valid `2109` attribute, otherwise behave the same as `dict.setdefault`.

</div>

## Example

The following example demonstrates how to use the `!http.cookies` module.

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> from http import cookies \>\>\> C = cookies.SimpleCookie() \>\>\> C\["fig"\] = "newton" \>\>\> C\["sugar"\] = "wafer" \>\>\> print(C) \# generate HTTP headers Set-Cookie: fig=newton Set-Cookie: sugar=wafer \>\>\> print(C.output()) \# same thing Set-Cookie: fig=newton Set-Cookie: sugar=wafer \>\>\> C = cookies.SimpleCookie() \>\>\> C\["rocky"\] = "road" \>\>\> C\["rocky"\]\["path"\] = "/cookie" \>\>\> print(C.output(header="Cookie:")) Cookie: rocky=road; Path=/cookie \>\>\> print(C.output(attrs=\[\], header="Cookie:")) Cookie: rocky=road \>\>\> C = cookies.SimpleCookie() \>\>\> C.load("chips=ahoy; vienna=finger") \# load from a string (HTTP header) \>\>\> print(C) Set-Cookie: chips=ahoy Set-Cookie: vienna=finger \>\>\> C = cookies.SimpleCookie() \>\>\> C.load('keebler="E=everybody; L=\\"Loves\\"; fudge=;";') \>\>\> print(C) Set-Cookie: keebler="E=everybody; L="Loves"; fudge=;" \>\>\> C = cookies.SimpleCookie() \>\>\> C\["oreo"\] = "doublestuff" \>\>\> C\["oreo"\]\["path"\] = "/" \>\>\> print(C) Set-Cookie: oreo=doublestuff; Path=/ \>\>\> C = cookies.SimpleCookie() \>\>\> C\["twix"\] = "none for you" \>\>\> C\["twix"\].value 'none for you' \>\>\> C = cookies.SimpleCookie() \>\>\> C\["number"\] = 7 \# equivalent to C\["number"\] = str(7) \>\>\> C\["string"\] = "seven" \>\>\> C\["number"\].value '7' \>\>\> C\["string"\].value 'seven' \>\>\> print(C) Set-Cookie: number=7 Set-Cookie: string=seven

</div>

# `!http` --- HTTP modules

<div class="module" data-synopsis="HTTP status codes and messages">

http

</div>

**Source code:** `Lib/http/__init__.py`

<div class="index">

pair: HTTP; protocol single: HTTP; http (standard module)

</div>

------------------------------------------------------------------------

`!http` is a package that collects several modules for working with the HyperText Transfer Protocol:

- `http.client` is a low-level HTTP protocol client; for high-level URL opening use `urllib.request`
- `http.server` contains basic HTTP server classes based on `socketserver`
- `http.cookies` has utilities for implementing state management with cookies
- `http.cookiejar` provides persistence of cookies

The `!http` module also defines the following enums that help you work with http related code:

<div class="HTTPStatus">

<div class="versionadded">

3.5

</div>

A subclass of `enum.IntEnum` that defines a set of HTTP status codes, reason phrases and long descriptions written in English.

Usage:

    >>> from http import HTTPStatus
    >>> HTTPStatus.OK
    HTTPStatus.OK
    >>> HTTPStatus.OK == 200
    True
    >>> HTTPStatus.OK.value
    200
    >>> HTTPStatus.OK.phrase
    'OK'
    >>> HTTPStatus.OK.description
    'Request fulfilled, document follows'
    >>> list(HTTPStatus)
    [HTTPStatus.CONTINUE, HTTPStatus.SWITCHING_PROTOCOLS, ...]

</div>

## HTTP status codes

Supported, [IANA-registered status codes](https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml) available in `http.HTTPStatus` are:

| Code | Enum Name | Details |
|----|----|----|
| `100` | `CONTINUE` | HTTP Semantics `9110`, Section 15.2.1 |
| `101` | `SWITCHING_PROTOCOLS` | HTTP Semantics `9110`, Section 15.2.2 |
| `102` | `PROCESSING` | WebDAV `2518`, Section 10.1 |
| `103` | `EARLY_HINTS` | An HTTP Status Code for Indicating Hints `8297` |
| `200` | `OK` | HTTP Semantics `9110`, Section 15.3.1 |
| `201` | `CREATED` | HTTP Semantics `9110`, Section 15.3.2 |
| `202` | `ACCEPTED` | HTTP Semantics `9110`, Section 15.3.3 |
| `203` | `NON_AUTHORITATIVE_INFORMATION` | HTTP Semantics `9110`, Section 15.3.4 |
| `204` | `NO_CONTENT` | HTTP Semantics `9110`, Section 15.3.5 |
| `205` | `RESET_CONTENT` | HTTP Semantics `9110`, Section 15.3.6 |
| `206` | `PARTIAL_CONTENT` | HTTP Semantics `9110`, Section 15.3.7 |
| `207` | `MULTI_STATUS` | WebDAV `4918`, Section 11.1 |
| `208` | `ALREADY_REPORTED` | WebDAV Binding Extensions `5842`, Section 7.1 (Experimental) |
| `226` | `IM_USED` | Delta Encoding in HTTP `3229`, Section 10.4.1 |
| `300` | `MULTIPLE_CHOICES` | HTTP Semantics `9110`, Section 15.4.1 |
| `301` | `MOVED_PERMANENTLY` | HTTP Semantics `9110`, Section 15.4.2 |
| `302` | `FOUND` | HTTP Semantics `9110`, Section 15.4.3 |
| `303` | `SEE_OTHER` | HTTP Semantics `9110`, Section 15.4.4 |
| `304` | `NOT_MODIFIED` | HTTP Semantics `9110`, Section 15.4.5 |
| `305` | `USE_PROXY` | HTTP Semantics `9110`, Section 15.4.6 |
| `307` | `TEMPORARY_REDIRECT` | HTTP Semantics `9110`, Section 15.4.8 |
| `308` | `PERMANENT_REDIRECT` | HTTP Semantics `9110`, Section 15.4.9 |
| `400` | `BAD_REQUEST` | HTTP Semantics `9110`, Section 15.5.1 |
| `401` | `UNAUTHORIZED` | HTTP Semantics `9110`, Section 15.5.2 |
| `402` | `PAYMENT_REQUIRED` | HTTP Semantics `9110`, Section 15.5.3 |
| `403` | `FORBIDDEN` | HTTP Semantics `9110`, Section 15.5.4 |
| `404` | `NOT_FOUND` | HTTP Semantics `9110`, Section 15.5.5 |
| `405` | `METHOD_NOT_ALLOWED` | HTTP Semantics `9110`, Section 15.5.6 |
| `406` | `NOT_ACCEPTABLE` | HTTP Semantics `9110`, Section 15.5.7 |
| `407` | `PROXY_AUTHENTICATION_REQUIRED` | HTTP Semantics `9110`, Section 15.5.8 |
| `408` | `REQUEST_TIMEOUT` | HTTP Semantics `9110`, Section 15.5.9 |
| `409` | `CONFLICT` | HTTP Semantics `9110`, Section 15.5.10 |
| `410` | `GONE` | HTTP Semantics `9110`, Section 15.5.11 |
| `411` | `LENGTH_REQUIRED` | HTTP Semantics `9110`, Section 15.5.12 |
| `412` | `PRECONDITION_FAILED` | HTTP Semantics `9110`, Section 15.5.13 |
| `413` | `CONTENT_TOO_LARGE` | HTTP Semantics `9110`, Section 15.5.14 |
| `414` | `URI_TOO_LONG` | HTTP Semantics `9110`, Section 15.5.15 |
| `415` | `UNSUPPORTED_MEDIA_TYPE` | HTTP Semantics `9110`, Section 15.5.16 |
| `416` | `RANGE_NOT_SATISFIABLE` | HTTP Semantics `9110`, Section 15.5.17 |
| `417` | `EXPECTATION_FAILED` | HTTP Semantics `9110`, Section 15.5.18 |
| `418` | `IM_A_TEAPOT` | HTCPCP/1.0 `2324`, Section 2.3.2 |
| `421` | `MISDIRECTED_REQUEST` | HTTP Semantics `9110`, Section 15.5.20 |
| `422` | `UNPROCESSABLE_CONTENT` | HTTP Semantics `9110`, Section 15.5.21 |
| `423` | `LOCKED` | WebDAV `4918`, Section 11.3 |
| `424` | `FAILED_DEPENDENCY` | WebDAV `4918`, Section 11.4 |
| `425` | `TOO_EARLY` | Using Early Data in HTTP `8470` |
| `426` | `UPGRADE_REQUIRED` | HTTP Semantics `9110`, Section 15.5.22 |
| `428` | `PRECONDITION_REQUIRED` | Additional HTTP Status Codes `6585` |
| `429` | `TOO_MANY_REQUESTS` | Additional HTTP Status Codes `6585` |
| `431` | `REQUEST_HEADER_FIELDS_TOO_LARGE` | Additional HTTP Status Codes `6585` |
| `451` | `UNAVAILABLE_FOR_LEGAL_REASONS` | An HTTP Status Code to Report Legal Obstacles `7725` |
| `500` | `INTERNAL_SERVER_ERROR` | HTTP Semantics `9110`, Section 15.6.1 |
| `501` | `NOT_IMPLEMENTED` | HTTP Semantics `9110`, Section 15.6.2 |
| `502` | `BAD_GATEWAY` | HTTP Semantics `9110`, Section 15.6.3 |
| `503` | `SERVICE_UNAVAILABLE` | HTTP Semantics `9110`, Section 15.6.4 |
| `504` | `GATEWAY_TIMEOUT` | HTTP Semantics `9110`, Section 15.6.5 |
| `505` | `HTTP_VERSION_NOT_SUPPORTED` | HTTP Semantics `9110`, Section 15.6.6 |
| `506` | `VARIANT_ALSO_NEGOTIATES` | Transparent Content Negotiation in HTTP `2295`, Section 8.1 (Experimental) |
| `507` | `INSUFFICIENT_STORAGE` | WebDAV `4918`, Section 11.5 |
| `508` | `LOOP_DETECTED` | WebDAV Binding Extensions `5842`, Section 7.2 (Experimental) |
| `510` | `NOT_EXTENDED` | An HTTP Extension Framework `2774`, Section 7 (Experimental) |
| `511` | `NETWORK_AUTHENTICATION_REQUIRED` | Additional HTTP Status Codes `6585`, Section 6 |

In order to preserve backwards compatibility, enum values are also present in the `http.client` module in the form of constants. The enum name is equal to the constant name (i.e. `http.HTTPStatus.OK` is also available as `http.client.OK`).

<div class="versionchanged">

3.7 Added `421 MISDIRECTED_REQUEST` status code.

</div>

<div class="versionadded">

3.8 Added `451 UNAVAILABLE_FOR_LEGAL_REASONS` status code.

</div>

<div class="versionadded">

3.9 Added `103 EARLY_HINTS`, `418 IM_A_TEAPOT` and `425 TOO_EARLY` status codes.

</div>

<div class="versionchanged">

3.13 Implemented RFC9110 naming for status constants. Old constant names are preserved for backwards compatibility: `413 REQUEST_ENTITY_TOO_LARGE`, `414 REQUEST_URI_TOO_LONG`, `416 REQUESTED_RANGE_NOT_SATISFIABLE` and `422 UNPROCESSABLE_ENTITY`.

</div>

## HTTP status category

<div class="versionadded">

3.12

</div>

The enum values have several properties to indicate the HTTP status category:

| Property | Indicates that | Details |
|----|----|----|
| `is_informational` | `100 <= status <= 199` | HTTP Semantics `9110`, Section 15 |
| `is_success` | `200 <= status <= 299` | HTTP Semantics `9110`, Section 15 |
| `is_redirection` | `300 <= status <= 399` | HTTP Semantics `9110`, Section 15 |
| `is_client_error` | `400 <= status <= 499` | HTTP Semantics `9110`, Section 15 |
| `is_server_error` | `500 <= status <= 599` | HTTP Semantics `9110`, Section 15 |

> Usage:
>
>     >>> from http import HTTPStatus
>     >>> HTTPStatus.OK.is_success
>     True
>     >>> HTTPStatus.OK.is_client_error
>     False

<div class="HTTPMethod">

<div class="versionadded">

3.11

</div>

A subclass of `enum.StrEnum` that defines a set of HTTP methods and descriptions written in English.

Usage:

    >>> from http import HTTPMethod
    >>>
    >>> HTTPMethod.GET
    <HTTPMethod.GET>
    >>> HTTPMethod.GET == 'GET'
    True
    >>> HTTPMethod.GET.value
    'GET'
    >>> HTTPMethod.GET.description
    'Retrieve the target.'
    >>> list(HTTPMethod)
    [<HTTPMethod.CONNECT>,
     <HTTPMethod.DELETE>,
     <HTTPMethod.GET>,
     <HTTPMethod.HEAD>,
     <HTTPMethod.OPTIONS>,
     <HTTPMethod.PATCH>,
     <HTTPMethod.POST>,
     <HTTPMethod.PUT>,
     <HTTPMethod.TRACE>]

</div>

## HTTP methods

Supported, [IANA-registered methods](https://www.iana.org/assignments/http-methods/http-methods.xhtml) available in `http.HTTPMethod` are:

| Method    | Enum Name | Details                              |
|-----------|-----------|--------------------------------------|
| `GET`     | `GET`     | HTTP Semantics `9110`, Section 9.3.1 |
| `HEAD`    | `HEAD`    | HTTP Semantics `9110`, Section 9.3.2 |
| `POST`    | `POST`    | HTTP Semantics `9110`, Section 9.3.3 |
| `PUT`     | `PUT`     | HTTP Semantics `9110`, Section 9.3.4 |
| `DELETE`  | `DELETE`  | HTTP Semantics `9110`, Section 9.3.5 |
| `CONNECT` | `CONNECT` | HTTP Semantics `9110`, Section 9.3.6 |
| `OPTIONS` | `OPTIONS` | HTTP Semantics `9110`, Section 9.3.7 |
| `TRACE`   | `TRACE`   | HTTP Semantics `9110`, Section 9.3.8 |
| `PATCH`   | `PATCH`   | HTTP/1.1 `5789`                      |

# `!http.server` --- HTTP servers

<div class="module" data-synopsis="HTTP server and request handlers.">

http.server

</div>

**Source code:** `Lib/http/server.py`

<div class="index">

pair: WWW; server pair: HTTP; protocol single: URL single: httpd

</div>

------------------------------------------------------------------------

This module defines classes for implementing HTTP servers.

> [!WARNING]
> `!http.server` is not recommended for production. It only implements `basic security checks <http.server-security>`.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

One class, `HTTPServer`, is a `socketserver.TCPServer` subclass. It creates and listens at the HTTP socket, dispatching the requests to a handler. Code to create and run the server looks like this:

    def run(server_class=HTTPServer, handler_class=BaseHTTPRequestHandler):
        server_address = ('', 8000)
        httpd = server_class(server_address, handler_class)
        httpd.serve_forever()

<div class="HTTPServer(server_address, RequestHandlerClass)">

This class builds on the `~socketserver.TCPServer` class by storing the server address as instance variables named `server_name` and `server_port`. The server is accessible by the handler, typically through the handler's `~socketserver.BaseRequestHandler.server` instance variable.

<div class="attribute">

server_name

The HTTP server's fully qualified domain name.

</div>

<div class="attribute">

server_port

The HTTP server's port number obtained from *server_address*.

</div>

</div>

<div class="ThreadingHTTPServer(server_address, RequestHandlerClass)">

This class is identical to HTTPServer but uses threads to handle requests by using the `~socketserver.ThreadingMixIn`. This is useful to handle web browsers pre-opening sockets, on which `HTTPServer` would wait indefinitely.

<div class="versionadded">

3.7

</div>

</div>

<div class="HTTPSServer(server_address, RequestHandlerClass,\ bind_and_activate=True, *, certfile, keyfile=None,\ password=None, alpn_protocols=None)">

Subclass of `HTTPServer` with a wrapped socket using the `ssl` module. If the `ssl` module is not available, instantiating a `!HTTPSServer` object fails with a `RuntimeError`.

The *certfile* argument is the path to the SSL certificate chain file, and the *keyfile* is the path to the file containing the private key.

A *password* can be specified for files protected and wrapped with PKCS#8, but beware that this could possibly expose hardcoded passwords in clear.

<div class="seealso">

See `ssl.SSLContext.load_cert_chain` for additional information on the accepted values for *certfile*, *keyfile* and *password*.

</div>

When specified, the *alpn_protocols* argument must be a sequence of strings specifying the "Application-Layer Protocol Negotiation" (ALPN) protocols supported by the server. ALPN allows the server and the client to negotiate the application protocol during the TLS handshake.

By default, it is set to `["http/1.1"]`, meaning the server supports HTTP/1.1.

<div class="versionadded">

3.14

</div>

</div>

<div class="ThreadingHTTPSServer(server_address, RequestHandlerClass,\ bind_and_activate=True, *, certfile, keyfile=None,\ password=None, alpn_protocols=None)">

This class is identical to `HTTPSServer` but uses threads to handle requests by inheriting from `~socketserver.ThreadingMixIn`. This is analogous to `ThreadingHTTPServer` only using `HTTPSServer`.

<div class="versionadded">

3.14

</div>

</div>

The `HTTPServer`, `ThreadingHTTPServer`, `HTTPSServer` and `ThreadingHTTPSServer` must be given a *RequestHandlerClass* on instantiation, of which this module provides three different variants:

<div class="BaseHTTPRequestHandler(request, client_address, server)">

This class is used to handle the HTTP requests that arrive at the server. By itself, it cannot respond to any actual HTTP requests; it must be subclassed to handle each request method (for example, `'GET'` or `'POST'`). `BaseHTTPRequestHandler` provides a number of class and instance variables, and methods for use by subclasses.

The handler will parse the request and the headers, then call a method specific to the request type. The method name is constructed from the request. For example, for the request method `SPAM`, the `!do_SPAM` method will be called with no arguments. All of the relevant information is stored in instance variables of the handler. Subclasses should not need to override or extend the `!__init__` method.

`BaseHTTPRequestHandler` has the following instance variables:

<div class="attribute">

client_address

Contains a tuple of the form `(host, port)` referring to the client's address.

</div>

<div class="attribute">

server

Contains the server instance.

</div>

<div class="attribute">

close_connection

Boolean that should be set before `handle_one_request` returns, indicating if another request may be expected, or if the connection should be shut down.

</div>

<div class="attribute">

requestline

Contains the string representation of the HTTP request line. The terminating CRLF is stripped. This attribute should be set by `handle_one_request`. If no valid request line was processed, it should be set to the empty string.

</div>

<div class="attribute">

command

Contains the command (request type). For example, `'GET'`.

</div>

<div class="attribute">

path

Contains the request path. If the query component of the URL is present, then `path` includes the query. Using the terminology of `3986`, `path` here includes `hier-part` and the `query`.

</div>

<div class="attribute">

request_version

Contains the version string from the request. For example, `'HTTP/1.0'`.

</div>

<div class="attribute">

headers

Holds an instance of the class specified by the `MessageClass` class variable. This instance parses and manages the headers in the HTTP request. The `~http.client.parse_headers` function from `http.client` is used to parse the headers and it requires that the HTTP request provide a valid `5322` style header.

</div>

<div class="attribute">

rfile

An `io.BufferedIOBase` input stream, ready to read from the start of the optional input data.

</div>

<div class="attribute">

wfile

Contains the output stream for writing a response back to the client. Proper adherence to the HTTP protocol must be used when writing to this stream in order to achieve successful interoperation with HTTP clients.

<div class="versionchanged">

3.6 This is an `io.BufferedIOBase` stream.

</div>

</div>

`BaseHTTPRequestHandler` has the following attributes:

<div class="attribute">

server_version

Specifies the server software version. You may want to override this. The format is multiple whitespace-separated strings, where each string is of the form name\[/version\]. For example, `'BaseHTTP/0.2'`.

</div>

<div class="attribute">

sys_version

Contains the Python system version, in a form usable by the `version_string` method and the `server_version` class variable. For example, `'Python/1.4'`.

</div>

<div class="attribute">

error_message_format

Specifies a format string that should be used by `send_error` method for building an error response to the client. The string is filled by default with variables from `responses` based on the status code passed to `send_error`.

</div>

<div class="attribute">

error_content_type

Specifies the Content-Type HTTP header of error responses sent to the client. The default value is `'text/html'`.

</div>

<div class="attribute">

protocol_version

Specifies the HTTP version to which the server is conformant. It is sent in responses to let the client know the server's communication capabilities for future requests. If set to `'HTTP/1.1'`, the server will permit HTTP persistent connections; however, your server *must* then include an accurate `Content-Length` header (using `send_header`) in all of its responses to clients. For backwards compatibility, the setting defaults to `'HTTP/1.0'`.

</div>

<div class="attribute">

MessageClass

Specifies an `email.message.Message`-like class to parse HTTP headers. Typically, this is not overridden, and it defaults to `http.client.HTTPMessage`.

</div>

<div class="attribute">

responses

This attribute contains a mapping of error code integers to two-element tuples containing a short and long message. For example, `{code: (shortmessage, longmessage)}`. The *shortmessage* is usually used as the *message* key in an error response, and *longmessage* as the *explain* key. It is used by `send_response_only` and `send_error` methods.

</div>

A `BaseHTTPRequestHandler` instance has the following methods:

<div class="method">

handle()

Calls `handle_one_request` once (or, if persistent connections are enabled, multiple times) to handle incoming HTTP requests. You should never need to override it; instead, implement appropriate `!do_\*` methods.

</div>

<div class="method">

handle_one_request()

This method will parse and dispatch the request to the appropriate `!do_\*` method. You should never need to override it.

</div>

<div class="method">

handle_expect_100()

When an HTTP/1.1 conformant server receives an `Expect: 100-continue` request header it responds with a `100 Continue` followed by `200 OK` headers. This method can be overridden to raise an error if the server does not want the client to continue. For example, the server can choose to send `417 Expectation Failed` as a response header and `return False`.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

send_error(code, message=None, explain=None)

Sends and logs a complete error reply to the client. The numeric *code* specifies the HTTP error code, with *message* as an optional, short, human readable description of the error. The *explain* argument can be used to provide more detailed information about the error; it will be formatted using the `error_message_format` attribute and emitted, after a complete set of headers, as the response body. The `responses` attribute holds the default values for *message* and *explain* that will be used if no value is provided; for unknown codes the default value for both is the string `???`. The body will be empty if the method is HEAD or the response code is one of the following: `1{xx}`, `204 No Content`, `205 Reset Content`, `304 Not Modified`.

<div class="versionchanged">

3.4 The error response includes a Content-Length header. Added the *explain* argument.

</div>

</div>

<div class="method">

send_response(code, message=None)

Adds a response header to the headers buffer and logs the accepted request. The HTTP response line is written to the internal buffer, followed by *Server* and *Date* headers. The values for these two headers are picked up from the `version_string` and `date_time_string` methods, respectively. If the server does not intend to send any other headers using the `send_header` method, then `send_response` should be followed by an `end_headers` call.

<div class="versionchanged">

3.3 Headers are stored to an internal buffer and `end_headers` needs to be called explicitly.

</div>

</div>

<div class="method">

send_header(keyword, value)

Adds the HTTP header to an internal buffer which will be written to the output stream when either `end_headers` or `flush_headers` is invoked. *keyword* should specify the header keyword, with *value* specifying its value. Note that, after the send_header calls are done, `end_headers` MUST BE called in order to complete the operation.

This method does not reject input containing CRLF sequences.

<div class="versionchanged">

3.2 Headers are stored in an internal buffer.

</div>

</div>

<div class="method">

send_response_only(code, message=None)

Sends the response header only, used for the purposes when `100 Continue` response is sent by the server to the client. The headers are not buffered and sent directly the output stream. If the *message* is not specified, the HTTP message corresponding the response *code* is sent.

This method does not reject *message* containing CRLF sequences.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

end_headers()

Adds a blank line (indicating the end of the HTTP headers in the response) to the headers buffer and calls `flush_headers`.

<div class="versionchanged">

3.2 The buffered headers are written to the output stream.

</div>

</div>

<div class="method">

flush_headers()

Finally send the headers to the output stream and flush the internal headers buffer.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

log_request(code='-', size='-')

Logs an accepted (successful) request. *code* should specify the numeric HTTP code associated with the response. If a size of the response is available, then it should be passed as the *size* parameter.

</div>

<div class="method">

log_error(...)

Logs an error when a request cannot be fulfilled. By default, it passes the message to `log_message`, so it takes the same arguments (*format* and additional values).

</div>

<div class="method">

log_message(format, ...)

Logs an arbitrary message to `sys.stderr`. This is typically overridden to create custom error logging mechanisms. The *format* argument is a standard printf-style format string, where the additional arguments to `log_message` are applied as inputs to the formatting. The client IP address and current date and time are prefixed to every message logged.

</div>

<div class="method">

version_string()

Returns the server software's version string. This is a combination of the `server_version` and `sys_version` attributes.

</div>

<div class="method">

date_time_string(timestamp=None)

Returns the date and time given by *timestamp* (which must be `None` or in the format returned by `time.time`), formatted for a message header. If *timestamp* is omitted, it uses the current date and time.

The result looks like `'Sun, 06 Nov 1994 08:49:37 GMT'`.

</div>

<div class="method">

log_date_time_string()

Returns the current date and time, formatted for logging.

</div>

<div class="method">

address_string()

Returns the client address.

<div class="versionchanged">

3.3 Previously, a name lookup was performed. To avoid name resolution delays, it now always returns the IP address.

</div>

</div>

</div>

<div class="SimpleHTTPRequestHandler(request, client_address, server, \ *, directory=None, extra_response_headers=None)">

This class serves files from the directory *directory* and below, or the current directory if *directory* is not provided, directly mapping the directory structure to HTTP requests.

<div class="versionchanged">

3.7 Added the *directory* parameter.

</div>

<div class="versionchanged">

3.9 The *directory* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.15 Added *extra_response_headers* parameter.

</div>

A lot of the work, such as parsing the request, is done by the base class `BaseHTTPRequestHandler`. This class implements the `do_GET` and `do_HEAD` functions.

The following are defined as class-level attributes of `SimpleHTTPRequestHandler`:

<div class="attribute">

server_version

This will be `"SimpleHTTP/" + __version__`, where `__version__` is defined at the module level.

</div>

<div class="attribute">

default_content_type

Specifies the Content-Type header value sent when the MIME type cannot be guessed from the file extension of the requested URL. By default, it is set to `'application/octet-stream'`.

<div class="versionadded">

3.15

</div>

</div>

<div class="attribute">

index_pages

Specifies the filenames that are treated as directory index pages.

Defaults to `("index.html", "index.htm")`.

<div class="versionadded">

3.12

</div>

</div>

<div class="attribute">

extensions_map

A dictionary mapping suffixes into MIME types, contains custom overrides for the default system mappings. The mapping is used case-insensitively, and so should contain only lower-cased keys.

<div class="versionchanged">

3.9 This dictionary is no longer filled with the default system mappings, but only contains overrides.

</div>

</div>

<div class="attribute">

extra_response_headers

A sequence of `(name, value)` pairs containing user-defined extra HTTP response headers to add to each successful HTTP status 200 response. These headers are not included in other status code responses.

Headers that the server sends automatically such as `Content-Type` will not be overwritten by `!extra_response_headers`.

</div>

The `SimpleHTTPRequestHandler` class defines the following methods:

<div class="method">

do_HEAD()

This method serves the `'HEAD'` request type: it sends the headers it would send for the equivalent `GET` request. See the `do_GET` method for a more complete explanation of the possible headers.

</div>

<div class="method">

do_GET()

The request is mapped to a local file by interpreting the request as a path relative to the current working directory.

If the request was mapped to a directory, the directory is checked for an index page as specified by `index_pages`. If found, the file's contents are returned; otherwise a directory listing is generated by calling the `list_directory` method. This method uses `os.listdir` to scan the directory, and returns a `404` error response if the `~os.listdir` fails.

If the request was mapped to a file, it is opened. Any `OSError` exception in opening the requested file is mapped to a `404`, `'File not found'` error. If there was an `'If-Modified-Since'` header in the request, and the file was not modified after this time, a `304`, `'Not Modified'` response is sent. Otherwise, the content type is guessed by calling the `guess_type` method, which in turn uses the *extensions_map* variable, and the file contents are returned.

A `'Content-type:'` header with the guessed content type is output, followed by a `'Content-Length:'` header with the file's size and a `'Last-Modified:'` header with the file's modification time.

The instance attribute `extra_response_headers` is a sequence of `(name, value)` pairs containing user-defined extra response headers.

Then follows a blank line signifying the end of the headers, and then the contents of the file are output.

For example usage, see the implementation of the `test` function in `Lib/http/server.py`.

<div class="versionchanged">

3.7 Support of the `'If-Modified-Since'` header.

</div>

</div>

<div class="method">

list_directory(path)

Helper to list the contents of *path* when no index page is present.

This returns either a `file-like object` (which must be closed by the caller) or `None` to indicate an error, in which case the caller has nothing further to do. In either case, the headers are sent.

</div>

<div class="method">

guess_type(path)

Guess the type of the file at the given *path*.

This returns a string of the form `type/subtype`, usable for a MIME Content-type header.

The default implementation looks the file's extension up in `extensions_map`, falling back to `mimetypes.guess_file_type` and then to `default_content_type`.

<div class="versionchanged">

3.13 Add `mimetypes.guess_file_type` as a fallback.

</div>

</div>

</div>

The `SimpleHTTPRequestHandler` class can be used to create a very basic webserver serving files relative to the current directory as follows:

    import http.server
    import socketserver

    PORT = 8000

    Handler = http.server.SimpleHTTPRequestHandler

    with socketserver.TCPServer(("", PORT), Handler) as httpd:
        print("serving at port", PORT)
        httpd.serve_forever()

`SimpleHTTPRequestHandler` can also be subclassed to enhance behavior, such as using different index file names by overriding the class attribute `~SimpleHTTPRequestHandler.index_pages`.

## Command-line interface

`!http.server` can also be invoked directly using the `-m` switch of the interpreter. The following example illustrates how to serve files relative to the current directory:

``` bash
python -m http.server [OPTIONS] [port]
```

The following options are accepted:

<div class="program">

http.server

</div>

<div class="option">

port

The server listens to port 8000 by default. The default can be overridden by passing the desired port number as an argument:

``` bash
python -m http.server 9000
```

</div>

<div class="option">

`-b, --bind <address>`  

Specifies a specific address to which it should bind. Both IPv4 and IPv6 addresses are supported. By default, the server binds itself to all interfaces. For example, the following command causes the server to bind to localhost only:

``` bash
python -m http.server --bind 127.0.0.1
```

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.8 Support IPv6 in the `--bind` option.

</div>

</div>

<div class="option">

`-d, --directory <dir>`  

Specifies a directory to which it should serve the files. By default, the server uses the current directory. For example, the following command uses a specific directory:

``` bash
python -m http.server --directory /tmp/
```

<div class="versionadded">

3.7

</div>

</div>

<div class="option">

`-p, --protocol <version>`  

Specifies the HTTP version to which the server is conformant. By default, the server is conformant to HTTP/1.0. For example, the following command runs an HTTP/1.1 conformant server:

``` bash
python -m http.server --protocol HTTP/1.1
```

<div class="versionadded">

3.11

</div>

</div>

<div class="option">

`--content-type <content_type>`  

Specifies the default Content-Type HTTP header used when the MIME type cannot be guessed from the URL's file extension. By default, the server uses `'application/octet-stream'`:

``` bash
python -m http.server --content-type text/html
```

<div class="versionadded">

3.15

</div>

</div>

<div class="option">

`--tls-cert`  

Specifies a TLS certificate chain for HTTPS connections:

``` bash
python -m http.server --tls-cert fullchain.pem
```

<div class="versionadded">

3.14

</div>

</div>

<div class="option">

`--tls-key`  

Specifies a private key file for HTTPS connections.

This option requires `--tls-cert` to be specified.

<div class="versionadded">

3.14

</div>

</div>

<div class="option">

`--tls-password-file`  

Specifies the password file for password-protected private keys:

``` bash
python -m http.server \
       --tls-cert cert.pem \
       --tls-key key.pem \
       --tls-password-file password.txt
```

This option requires `--tls-cert` to be specified.

<div class="versionadded">

3.14

</div>

</div>

<div class="option">

-H, --header \<header\> \<value\>

Specify an additional HTTP Response Header to send on successful HTTP 200 responses. Can be used multiple times to send additional custom response headers. Headers that are sent automatically by the server (for instance Content-Type) will not be overwritten by the server.

<div class="versionadded">

3.15

</div>

</div>

## Security considerations

<div class="index">

pair: http.server; security

</div>

`SimpleHTTPRequestHandler` will follow symbolic links when handling requests which makes it possible for files outside of the specified directory to be served.

Methods `BaseHTTPRequestHandler.send_header` and `BaseHTTPRequestHandler.send_response_only` assume sanitized input and do not perform input validation such as checking for the presence of CRLF sequences. Untrusted input may result in HTTP header injection attacks.

Earlier versions of Python did not scrub control characters from the log messages emitted to stderr from `python -m http.server` or the default `BaseHTTPRequestHandler` `.log_message` implementation. This could allow remote clients connecting to your server to send nefarious control codes to your terminal.

<div class="versionchanged">

3.12 Control characters are scrubbed in stderr logs.

</div>

# Internationalization

The modules described in this chapter help you write software that is independent of language and locale by providing mechanisms for selecting a language to be used in program messages or by tailoring output to match local conventions.

The list of modules described in this chapter is:

<div class="toctree">

gettext.rst locale.rst

</div>

# IDLE --- Python editor and shell

**Source code:** `Lib/idlelib/`

<div class="index">

single: IDLE single: Python Editor single: Integrated Development Environment

</div>

------------------------------------------------------------------------

IDLE is Python's Integrated Development and Learning Environment.

IDLE has the following features:

- cross-platform: works mostly the same on Windows, Unix, and macOS
- Python shell window (interactive interpreter) with colorizing of code input, output, and error messages
- multi-window text editor with multiple undo, Python colorizing, smart indent, call tips, auto completion, and other features
- search within any window, replace within editor windows, and search through multiple files (grep)
- debugger with persistent breakpoints, stepping, and viewing of global and local namespaces
- configuration, browsers, and other dialogs

The IDLE application is implemented in the `idlelib` package.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

## Menus

IDLE has two main window types, the Shell window and the Editor window. It is possible to have multiple editor windows simultaneously. On Windows and Linux, each has its own top menu. Each menu documented below indicates which window type it is associated with.

Output windows, such as used for Edit =\> Find in Files, are a subtype of editor window. They currently have the same top menu but a different default title and context menu.

On macOS, there is one application menu. It dynamically changes according to the window currently selected. It has an IDLE menu, and some entries described below are moved around to conform to Apple guidelines.

### File menu (Shell and Editor)

New File  
Create a new file editing window.

Open...  
Open an existing file with an Open dialog.

Open Module...  
Open an existing module (searches sys.path).

Recent Files  
Open a list of recent files. Click one to open it.

<div class="index">

single: Module browser single: Path browser

</div>

Module Browser  
Show functions, classes, and methods in the current Editor file in a tree structure. In the shell, open a module first.

Path Browser  
Show sys.path directories, modules, functions, classes and methods in a tree structure.

Save  
Save the current window to the associated file, if there is one. Windows that have been changed since being opened or last saved have a \* before and after the window title. If there is no associated file, do Save As instead.

Save As...  
Save the current window with a Save As dialog. The file saved becomes the new associated file for the window. (If your file manager is set to hide extensions, the current extension will be omitted in the file name box. If the new filename has no '.', '.py' and '.txt' will be added for Python and text files, except that on macOS Aqua,'.py' is added for all files.)

Save Copy As...  
Save the current window to different file without changing the associated file. (See Save As note above about filename extensions.)

Print Window  
Print the current window to the default printer.

Close Window  
Close the current window (if an unsaved editor, ask to save; if an unsaved Shell, ask to quit execution). Calling `exit()` or `close()` in the Shell window also closes Shell. If this is the only window, also exit IDLE.

Exit IDLE  
Close all windows and quit IDLE (ask to save unsaved edit windows).

### Edit menu (Shell and Editor)

Undo  
Undo the last change to the current window. A maximum of 1000 changes may be undone.

Redo  
Redo the last undone change to the current window.

Select All  
Select the entire contents of the current window.

Cut  
Copy selection into the system-wide clipboard; then delete the selection.

Copy  
Copy selection into the system-wide clipboard.

Paste  
Insert contents of the system-wide clipboard into the current window.

The clipboard functions are also available in context menus.

Find...  
Open a search dialog with many options

Find Again  
Repeat the last search, if there is one.

Find Selection  
Search for the currently selected string, if there is one.

Find in Files...  
Open a file search dialog. Put results in a new output window.

Replace...  
Open a search-and-replace dialog.

Go to Line  
Move the cursor to the beginning of the line requested and make that line visible. A request past the end of the file goes to the end. Clear any selection and update the line and column status.

Show Completions  
Open a scrollable list allowing selection of existing names. See `Completions <completions>` in the Editing and Navigation section below.

Expand Word  
Expand a prefix you have typed to match a full word in the same window; repeat to get a different expansion.

Show Call Tip  
After an unclosed parenthesis for a function, open a small window with function parameter hints. See `Calltips <calltips>` in the Editing and Navigation section below.

Show Surrounding Parens  
Highlight the surrounding parenthesis.

### Format menu (Editor window only)

Format Paragraph  
Rewrap the text block containing the text insert cursor. Avoid code lines. See `Format block<format-block>` in the Editing and Navigation section below.

Indent Region  
Shift selected lines right by the indent width (default 4 spaces).

Dedent Region  
Shift selected lines left by the indent width (default 4 spaces).

Comment Out Region  
Insert \## in front of selected lines.

Uncomment Region  
Remove leading \# or \## from selected lines.

Tabify Region  
Turn *leading* stretches of spaces into tabs. (Note: We recommend using 4 space blocks to indent Python code.)

Untabify Region  
Turn *all* tabs into the correct number of spaces.

Toggle Tabs  
Open a dialog to switch between indenting with spaces and tabs.

New Indent Width  
Open a dialog to change indent width. The accepted default by the Python community is 4 spaces.

Strip Trailing Whitespace  
Remove trailing space and other whitespace characters after the last non-whitespace character of a line by applying `str.rstrip` to each line, including lines within multiline strings. Except for Shell windows, remove extra newlines at the end of the file.

<div class="index">

single: Run script

</div>

### Run menu (Editor window only)

<div id="run-module">

Run Module  
Do `Check Module <check-module>`. If no error, restart the shell to clean the environment, then execute the module. Output is displayed in the Shell window. Note that output requires use of `print` or `write`. When execution is complete, the Shell retains focus and displays a prompt. At this point, one may interactively explore the result of execution. This is similar to executing a file with `python -i file` at a command line.

</div>

<div id="run-custom">

Run... Customized  
Same as `Run Module <run-module>`, but run the module with customized settings. *Command Line Arguments* extend `sys.argv` as if passed on a command line. The module can be run in the Shell without restarting.

</div>

<div id="check-module">

Check Module  
Check the syntax of the module currently open in the Editor window. If the module has not been saved IDLE will either prompt the user to save or autosave, as selected in the General tab of the Idle Settings dialog. If there is a syntax error, the approximate location is indicated in the Editor window.

</div>

<div id="python-shell">

Python Shell  
Open or wake up the Python Shell window.

</div>

### Shell menu (Shell window only)

View Last Restart  
Scroll the shell window to the last Shell restart.

Restart Shell  
Restart the shell to clean the environment and reset display and exception handling.

Previous History  
Cycle through earlier commands in history which match the current entry.

Next History  
Cycle through later commands in history which match the current entry.

Interrupt Execution  
Stop a running program.

### Debug menu (Shell window only)

Go to File/Line  
Look on the current line. with the cursor, and the line above for a filename and line number. If found, open the file if not already open, and show the line. Use this to view source lines referenced in an exception traceback and lines found by Find in Files. Also available in the context menu of the Shell window and Output windows.

<div class="index">

single: debugger single: stack viewer

</div>

Debugger (toggle)  
When activated, code entered in the Shell or run from an Editor will run under the debugger. In the Editor, breakpoints can be set with the context menu. This feature is still incomplete and somewhat experimental.

Stack Viewer  
Show the stack traceback of the last exception in a tree widget, with access to locals and globals.

Auto-open Stack Viewer  
Toggle automatically opening the stack viewer on an unhandled exception.

### Options menu (Shell and Editor)

Configure IDLE  
Open a configuration dialog and change preferences for the following: fonts, indentation, keybindings, text color themes, startup windows and size, additional help sources, and extensions. On macOS, open the configuration dialog by selecting Preferences in the application menu. For more details, see `Setting preferences <preferences>` under Help and preferences.

Most configuration options apply to all windows or all future windows. The option items below only apply to the active window.

Show/Hide Code Context (Editor Window only)  
Open a pane at the top of the edit window which shows the block context of the code which has scrolled above the top of the window. See `Code Context <code-context>` in the Editing and Navigation section below.

Show/Hide Line Numbers (Editor Window only)  
Open a column to the left of the edit window which shows the number of each line of text. The default is off, which may be changed in the preferences (see `Setting preferences <preferences>`).

Zoom/Restore Height  
Toggles the window between normal size and maximum height. The initial size defaults to 40 lines by 80 chars unless changed on the General tab of the Configure IDLE dialog. The maximum height for a screen is determined by momentarily maximizing a window the first time one is zoomed on the screen. Changing screen settings may invalidate the saved height. This toggle has no effect when a window is maximized.

### Window menu (Shell and Editor)

Lists the names of all open windows; select one to bring it to the foreground (deiconifying it if necessary).

### Help menu (Shell and Editor)

About IDLE  
Display version, copyright, license, credits, and more.

IDLE Help  
Display this IDLE document, detailing the menu options, basic editing and navigation, and other tips.

Python Docs  
Access local Python documentation, if installed, or start a web browser and open docs.python.org showing the latest Python documentation.

Turtle Demo  
Run the turtledemo module with example Python code and turtle drawings.

Additional help sources may be added here with the Configure IDLE dialog under the General tab. See the `Help sources <help-sources>` subsection below for more on Help menu choices.

<div class="index">

single: Cut single: Copy single: Paste single: Set Breakpoint single: Clear Breakpoint single: breakpoints

</div>

### Context menus

Open a context menu by right-clicking in a window (Control-click on macOS). Context menus have the standard clipboard functions also on the Edit menu.

Cut  
Copy selection into the system-wide clipboard; then delete the selection.

Copy  
Copy selection into the system-wide clipboard.

Paste  
Insert contents of the system-wide clipboard into the current window.

Editor windows also have breakpoint functions. Lines with a breakpoint set are specially marked. Breakpoints only have an effect when running under the debugger. Breakpoints for a file are saved in the user's `.idlerc` directory.

Set Breakpoint  
Set a breakpoint on the current line.

Clear Breakpoint  
Clear the breakpoint on that line.

Shell and Output windows also have the following.

Go to file/line  
Same as in Debug menu.

The Shell window also has an output squeezing facility explained in the *Python Shell window* subsection below.

Squeeze  
If the cursor is over an output line, squeeze all the output between the code above and the prompt below down to a 'Squeezed text' label.

## Editing and Navigation

### Editor windows

IDLE may open editor windows when it starts, depending on settings and how you start IDLE. Thereafter, use the File menu. There can be only one open editor window for a given file.

The title bar contains the name of the file, the full path, and the version of Python and IDLE running the window. The status bar contains the line number ('Ln') and column number ('Col'). Line numbers start with 1; column numbers with 0.

IDLE assumes that files with a known .py\* extension contain Python code and that other files do not. Run Python code with the Run menu.

### Key bindings

The IDLE insertion cursor is a thin vertical bar between character positions. When characters are entered, the insertion cursor and everything to its right moves right one character and the new character is entered in the new space.

Several non-character keys move the cursor and possibly delete characters. Deletion does not puts text on the clipboard, but IDLE has an undo list. Wherever this doc discusses keys, 'C' refers to the `Control` key on Windows and Unix and the `Command` key on macOS. (And all such discussions assume that the keys have not been re-bound to something else.)

- Arrow keys move the cursor one character or line.
- `C-LeftArrow` and `C-RightArrow` moves left or right one word.
- `Home` and `End` go to the beginning or end of the line.
- `Page Up` and `Page Down` go up or down one screen.
- `C-Home` and `C-End` go to beginning or end of the file.
- `Backspace` and `Del` (or `C-d`) delete the previous or next character.
- `C-Backspace` and `C-Del` delete one word left or right.
- `C-k` deletes ('kills') everything to the right.

Standard keybindings (like `C-c` to copy and `C-v` to paste) may work. Keybindings are selected in the Configure IDLE dialog.

### Automatic indentation

After a block-opening statement, the next line is indented by 4 spaces (in the Python Shell window by one tab). After certain keywords (break, return etc.) the next line is dedented. In leading indentation, `Backspace` deletes up to 4 spaces if they are there. `Tab` inserts spaces (in the Python Shell window one tab), number depends on Indent width. Currently, tabs are restricted to four spaces due to Tcl/Tk limitations.

See also the indent/dedent region commands on the `Format menu <format-menu>`.

### Search and Replace

Any selection becomes a search target. However, only selections within a line work because searches are only performed within lines with the terminal newline removed. If `[x] Regular expression` is checked, the target is interpreted according to the Python re module.

### Completions

Completions are supplied, when requested and available, for module names, attributes of classes or functions, or filenames. Each request method displays a completion box with existing names. (See tab completions below for an exception.) For any box, change the name being completed and the item highlighted in the box by typing and deleting characters; by hitting `Up`, `Down`, `PageUp`, `PageDown`, `Home`, and `End` keys; and by a single click within the box. Close the box with `Escape`, `Enter`, and double `Tab` keys or clicks outside the box. A double click within the box selects and closes.

One way to open a box is to type a key character and wait for a predefined interval. This defaults to 2 seconds; customize it in the settings dialog. (To prevent auto popups, set the delay to a large number of milliseconds, such as 100000000.) For imported module names or class or function attributes, type '.'. For filenames in the root directory, type `os.sep` or `os.altsep` immediately after an opening quote. (On Windows, one can specify a drive first.) Move into subdirectories by typing a directory name and a separator.

Instead of waiting, or after a box is closed, open a completion box immediately with Show Completions on the Edit menu. The default hot key is `C-space`. If one types a prefix for the desired name before opening the box, the first match or near miss is made visible. The result is the same as if one enters a prefix after the box is displayed. Show Completions after a quote completes filenames in the current directory instead of a root directory.

Hitting `Tab` after a prefix usually has the same effect as Show Completions. (With no prefix, it indents.) However, if there is only one match to the prefix, that match is immediately added to the editor text without opening a box.

Invoking 'Show Completions', or hitting `Tab` after a prefix, outside of a string and without a preceding '.' opens a box with keywords, builtin names, and available module-level names.

When editing code in an editor (as oppose to Shell), increase the available module-level names by running your code and not restarting the Shell thereafter. This is especially useful after adding imports at the top of a file. This also increases possible attribute completions.

Completion boxes initially exclude names beginning with '\_' or, for modules, not included in '\_\_all\_\_'. The hidden names can be accessed by typing '\_' after '.', either before or after the box is opened.

### Calltips

A calltip is shown automatically when one types `(` after the name of an *accessible* function. A function name expression may include dots and subscripts. A calltip remains until it is clicked, the cursor is moved out of the argument area, or `)` is typed. Whenever the cursor is in the argument part of a definition, select Edit and "Show Call Tip" on the menu or enter its shortcut to display a calltip.

The calltip consists of the function's signature and docstring up to the latter's first blank line or the fifth non-blank line. (Some builtin functions lack an accessible signature.) A '/' or '\*' in the signature indicates that the preceding or following arguments are passed by position or name (keyword) only. Details are subject to change.

In Shell, the accessible functions depends on what modules have been imported into the user process, including those imported by Idle itself, and which definitions have been run, all since the last restart.

For example, restart the Shell and enter `itertools.count(`. A calltip appears because Idle imports itertools into the user process for its own use. (This could change.) Enter `turtle.write(` and nothing appears. Idle does not itself import turtle. The menu entry and shortcut also do nothing. Enter `import turtle`. Thereafter, `turtle.write(` will display a calltip.

In an editor, import statements have no effect until one runs the file. One might want to run a file after writing import statements, after adding function definitions, or after opening an existing file.

### Format block

Reformat Paragraph rewraps a block ('paragraph') of contiguous equally indented non-blank comments, a similar block of text within a multiline string, or a selected subset of either. If needed, add a blank line to separate string from code. Partial lines in a selection expand to complete lines. The resulting lines have the same indent as before but have maximum total length of N columns (characters). Change the default N of 72 on the Window tab of IDLE Settings.

### Code Context

Within an editor window containing Python code, code context can be toggled in order to show or hide a pane at the top of the window. When shown, this pane freezes the opening lines for block code, such as those beginning with `class`, `def`, or `if` keywords, that would have otherwise scrolled out of view. The size of the pane will be expanded and contracted as needed to show the all current levels of context, up to the maximum number of lines defined in the Configure IDLE dialog (which defaults to 15). If there are no current context lines and the feature is toggled on, a single blank line will display. Clicking on a line in the context pane will move that line to the top of the editor.

The text and background colors for the context pane can be configured under the Highlights tab in the Configure IDLE dialog.

### Shell window

In IDLE's Shell, enter, edit, and recall complete statements. (Most consoles and terminals only work with a single physical line at a time).

Submit a single-line statement for execution by hitting `Return` with the cursor anywhere on the line. If a line is extended with Backslash (`\\`), the cursor must be on the last physical line. Submit a multi-line compound statement by entering a blank line after the statement.

When one pastes code into Shell, it is not compiled and possibly executed until one hits `Return`, as specified above. One may edit pasted code first. If one pastes more than one statement into Shell, the result will be a `SyntaxError` when multiple statements are compiled as if they were one.

Lines containing `RESTART` mean that the user execution process has been re-started. This occurs when the user execution process has crashed, when one requests a restart on the Shell menu, or when one runs code in an editor window.

The editing features described in previous subsections work when entering code interactively. IDLE's Shell window also responds to the following:

- `C-c` attempts to interrupt statement execution (but may fail).
- `C-d` closes Shell if typed at a `>>>` prompt.
- `Alt-p` and `Alt-n` (`C-p` and `C-n` on macOS) retrieve to the current prompt the previous or next previously entered statement that matches anything already typed.
- `Return` while the cursor is on any previous statement appends the latter to anything already typed at the prompt.

### Text colors

Idle defaults to black on white text, but colors text with special meanings. For the shell, these are shell output, shell error, user output, and user error. For Python code, at the shell prompt or in an editor, these are keywords, builtin class and function names, names following `class` and `def`, strings, and comments. For any text window, these are the cursor (when present), found text (when possible), and selected text.

IDLE also highlights the `soft keywords <soft-keywords>` `match`, `case <match>`, and `_ <wildcard-patterns>` in pattern-matching statements. However, this highlighting is not perfect and will be incorrect in some rare cases, including some `_`-s in `case` patterns.

Text coloring is done in the background, so uncolorized text is occasionally visible. To change the color scheme, use the Configure IDLE dialog Highlighting tab. The marking of debugger breakpoint lines in the editor and text in popups and dialogs is not user-configurable.

## Startup and Code Execution

Upon startup with the `-s` option, IDLE will execute the file referenced by the environment variables `IDLESTARTUP` or `PYTHONSTARTUP`. IDLE first checks for `IDLESTARTUP`; if `IDLESTARTUP` is present the file referenced is run. If `IDLESTARTUP` is not present, IDLE checks for `PYTHONSTARTUP`. Files referenced by these environment variables are convenient places to store functions that are used frequently from the IDLE shell, or for executing import statements to import common modules.

In addition, `Tk` also loads a startup file if it is present. Note that the Tk file is loaded unconditionally. This additional file is `.Idle.py` and is looked for in the user's home directory. Statements in this file will be executed in the Tk namespace, so this file is not useful for importing functions to be used from IDLE's Python shell.

### Command-line usage

<div class="program">

idle

</div>

IDLE can be invoked from the command line with various options. The general syntax is:

``` bash
python -m idlelib [options] [file ...]
```

The following options are available:

<div class="option">

`-c <command>`  

Run the specified Python command in the shell window. For example, pass `-c "print('Hello, World!')"`. On Windows, the outer quotes must be double quotes as shown.

</div>

<div class="option">

`-d`  

Enable the debugger and open the shell window.

</div>

<div class="option">

`-e`  

Open an editor window.

</div>

<div class="option">

`-h`  

Print a help message with legal combinations of options and exit.

</div>

<div class="option">

`-i`  

Open a shell window.

</div>

<div class="option">

`-r <file>`  

Run the specified file in the shell window.

</div>

<div class="option">

`-s`  

Run the startup file (as defined by the environment variables `IDLESTARTUP` or `PYTHONSTARTUP`) before opening the shell window.

</div>

<div class="option">

`-t <title>`  

Set the title of the shell window.

</div>

<div class="option">

- 

Read and execute standard input in the shell window. This option must be the last one before any arguments.

</div>

If arguments are provided:

- If `-`, `-c`, or `-r` is used, all arguments are placed in `sys.argv[1:]`, and `sys.argv[0]` is set to `''`, `'-c'`, or `'-r'` respectively. No editor window is opened, even if that is the default set in the *Options* dialog.
- Otherwise, arguments are treated as files to be opened for editing, and `sys.argv` reflects the arguments passed to IDLE itself.

### Startup failure

IDLE uses a socket to communicate between the IDLE GUI process and the user code execution process. A connection must be established whenever the Shell starts or restarts. (The latter is indicated by a divider line that says 'RESTART'). If the user process fails to connect to the GUI process, it usually displays a `Tk` error box with a 'cannot connect' message that directs the user here. It then exits.

One specific connection failure on Unix systems results from misconfigured masquerading rules somewhere in a system's network setup. When IDLE is started from a terminal, one will see a message starting with `** Invalid host:`. The valid value is `127.0.0.1 (idlelib.rpc.LOCALHOST)`. One can diagnose with `tcpconnect -irv 127.0.0.1 6543` in one terminal window and `tcplisten <same args>` in another.

A common cause of failure is a user-written file with the same name as a standard library module, such as *random.py* and *tkinter.py*. When such a file is located in the same directory as a file that is about to be run, IDLE cannot import the stdlib file. The current fix is to rename the user file.

Though less common than in the past, an antivirus or firewall program may stop the connection. If the program cannot be taught to allow the connection, then it must be turned off for IDLE to work. It is safe to allow this internal connection because no data is visible on external ports. A similar problem is a network mis-configuration that blocks connections.

Python installation issues occasionally stop IDLE: multiple versions can clash, or a single installation might need admin access. If one undo the clash, or cannot or does not want to run as admin, it might be easiest to completely remove Python and start over.

A zombie pythonw.exe process could be a problem. On Windows, use Task Manager to check for one and stop it if there is. Sometimes a restart initiated by a program crash or Keyboard Interrupt (control-C) may fail to connect. Dismissing the error box or using Restart Shell on the Shell menu may fix a temporary problem.

When IDLE first starts, it attempts to read user configuration files in `~/.idlerc/` (~ is one's home directory). If there is a problem, an error message should be displayed. Leaving aside random disk glitches, this can be prevented by never editing the files by hand. Instead, use the configuration dialog, under Options. Once there is an error in a user configuration file, the best solution may be to delete it and start over with the settings dialog.

If IDLE quits with no message, and it was not started from a console, try starting it from a console or terminal (`python -m idlelib`) and see if this results in an error message.

On Unix-based systems with tcl/tk older than `8.6.11` (see `About IDLE`) certain characters of certain fonts can cause a tk failure with a message to the terminal. This can happen either if one starts IDLE to edit a file with such a character or later when entering such a character. If one cannot upgrade tcl/tk, then re-configure IDLE to use a font that works better.

### Running user code

With rare exceptions, the result of executing Python code with IDLE is intended to be the same as executing the same code by the default method, directly with Python in a text-mode system console or terminal window. However, the different interface and operation occasionally affect visible results. For instance, `sys.modules` starts with more entries, and `threading.active_count()` returns 2 instead of 1.

By default, IDLE runs user code in a separate OS process rather than in the user interface process that runs the shell and editor. In the execution process, it replaces `sys.stdin`, `sys.stdout`, and `sys.stderr` with objects that get input from and send output to the Shell window. The original values stored in `sys.__stdin__`, `sys.__stdout__`, and `sys.__stderr__` are not touched, but may be `None`.

Sending print output from one process to a text widget in another is slower than printing to a system terminal in the same process. This has the most effect when printing multiple arguments, as the string for each argument, each separator, the newline are sent separately. For development, this is usually not a problem, but if one wants to print faster in IDLE, format and join together everything one wants displayed together and then print a single string. Both format strings and `str.join` can help combine fields and lines.

IDLE's standard stream replacements are not inherited by subprocesses created in the execution process, whether directly by user code or by modules such as multiprocessing. If such subprocess use `input` from sys.stdin or `print` or `write` to sys.stdout or sys.stderr, IDLE should be started in a command line window. (On Windows, use `python` or `py` rather than `pythonw` or `pyw`.) The secondary subprocess will then be attached to that window for input and output.

If `sys` is reset by user code, such as with `importlib.reload(sys)`, IDLE's changes are lost and input from the keyboard and output to the screen will not work correctly.

When Shell has the focus, it controls the keyboard and screen. This is normally transparent, but functions that directly access the keyboard and screen will not work. These include system-specific functions that determine whether a key has been pressed and if so, which.

The IDLE code running in the execution process adds frames to the call stack that would not be there otherwise. IDLE wraps `sys.getrecursionlimit` and `sys.setrecursionlimit` to reduce the effect of the additional stack frames.

When user code raises SystemExit either directly or by calling sys.exit, IDLE returns to a Shell prompt instead of exiting.

### User output in Shell

When a program outputs text, the result is determined by the corresponding output device. When IDLE executes user code, `sys.stdout` and `sys.stderr` are connected to the display area of IDLE's Shell. Some of its features are inherited from the underlying Tk Text widget. Others are programmed additions. Where it matters, Shell is designed for development rather than production runs.

For instance, Shell never throws away output. A program that sends unlimited output to Shell will eventually fill memory, resulting in a memory error. In contrast, some system text windows only keep the last n lines of output. A Windows console, for instance, keeps a user-settable 1 to 9999 lines, with 300 the default.

A Tk Text widget, and hence IDLE's Shell, displays characters (codepoints) in the BMP (Basic Multilingual Plane) subset of Unicode. Which characters are displayed with a proper glyph and which with a replacement box depends on the operating system and installed fonts. Tab characters cause the following text to begin after the next tab stop. (They occur every 8 'characters'). Newline characters cause following text to appear on a new line. Other control characters are ignored or displayed as a space, box, or something else, depending on the operating system and font. (Moving the text cursor through such output with arrow keys may exhibit some surprising spacing behavior.) :

    >>> s = 'a\tb\a<\x02><\r>\bc\nd'  # Enter 22 chars.
    >>> len(s)
    14
    >>> s  # Display repr(s)
    'a\tb\x07<\x02><\r>\x08c\nd'
    >>> print(s, end='')  # Display s as is.
    # Result varies by OS and font.  Try it.

The `repr` function is used for interactive echo of expression values. It returns an altered version of the input string in which control codes, some BMP codepoints, and all non-BMP codepoints are replaced with escape codes. As demonstrated above, it allows one to identify the characters in a string, regardless of how they are displayed.

Normal and error output are generally kept separate (on separate lines) from code input and each other. They each get different highlight colors.

For SyntaxError tracebacks, the normal '^' marking where the error was detected is replaced by coloring the text with an error highlight. When code run from a file causes other exceptions, one may right click on a traceback line to jump to the corresponding line in an IDLE editor. The file will be opened if necessary.

Shell has a special facility for squeezing output lines down to a 'Squeezed text' label. This is done automatically for output over N lines (N = 50 by default). N can be changed in the PyShell section of the General page of the Settings dialog. Output with fewer lines can be squeezed by right clicking on the output. This can be useful lines long enough to slow down scrolling.

Squeezed output is expanded in place by double-clicking the label. It can also be sent to the clipboard or a separate view window by right-clicking the label.

### Developing tkinter applications

IDLE is intentionally different from standard Python in order to facilitate development of tkinter programs. Enter `import tkinter as tk; root = tk.Tk()` in standard Python and nothing appears. Enter the same in IDLE and a tk window appears. In standard Python, one must also enter `root.update()` to see the window. IDLE does the equivalent in the background, about 20 times a second, which is about every 50 milliseconds. Next enter `b = tk.Button(root, text='button'); b.pack()`. Again, nothing visibly changes in standard Python until one enters `root.update()`.

Most tkinter programs run `root.mainloop()`, which usually does not return until the tk app is destroyed. If the program is run with `python -i` or from an IDLE editor, a `>>>` shell prompt does not appear until `mainloop()` returns, at which time there is nothing left to interact with.

When running a tkinter program from an IDLE editor, one can comment out the mainloop call. One then gets a shell prompt immediately and can interact with the live application. One just has to remember to re-enable the mainloop call when running in standard Python.

### Running without a subprocess

By default, IDLE executes user code in a separate subprocess via a socket, which uses the internal loopback interface. This connection is not externally visible and no data is sent to or received from the internet. If firewall software complains anyway, you can ignore it.

If the attempt to make the socket connection fails, Idle will notify you. Such failures are sometimes transient, but if persistent, the problem may be either a firewall blocking the connection or misconfiguration of a particular system. Until the problem is fixed, one can run Idle with the -n command line switch.

If IDLE is started with the -n command line switch it will run in a single process and will not create the subprocess which runs the RPC Python execution server. This can be useful if Python cannot create the subprocess or the RPC socket interface on your platform. However, in this mode user code is not isolated from IDLE itself. Also, the environment is not restarted when Run/Run Module (F5) is selected. If your code has been modified, you must reload() the affected modules and re-import any specific items (e.g. from foo import baz) if the changes are to take effect. For these reasons, it is preferable to run IDLE with the default subprocess if at all possible.

<div class="deprecated">

3.4

</div>

## Help and Preferences

### Help sources

Help menu entry "IDLE Help" displays a formatted html version of the IDLE chapter of the Library Reference. The result, in a read-only tkinter text window, is close to what one sees in a web browser. Navigate through the text with a mousewheel, the scrollbar, or up and down arrow keys held down. Or click the TOC (Table of Contents) button and select a section header in the opened box.

Help menu entry "Python Docs" opens the extensive sources of help, including tutorials, available at `docs.python.org/x.y`, where 'x.y' is the currently running Python version. If your system has an off-line copy of the docs (this may be an installation option), that will be opened instead.

Selected URLs can be added or removed from the help menu at any time using the General tab of the Configure IDLE dialog.

### Setting preferences

The font preferences, highlighting, keys, and general preferences can be changed via Configure IDLE on the Option menu. Non-default user settings are saved in a `.idlerc` directory in the user's home directory. Problems caused by bad user configuration files are solved by editing or deleting one or more of the files in `.idlerc`.

On the Font tab, see the text sample for the effect of font face and size on multiple characters in multiple languages. Edit the sample to add other characters of personal interest. Use the sample to select monospaced fonts. If particular characters have problems in Shell or an editor, add them to the top of the sample and try changing first size and then font.

On the Highlights and Keys tab, select a built-in or custom color theme and key set. To use a newer built-in color theme or key set with older IDLEs, save it as a new custom theme or key set and it well be accessible to older IDLEs.

### IDLE on macOS

Under System Preferences: Dock, one can set "Prefer tabs when opening documents" to "Always". This setting is not compatible with the tk/tkinter GUI framework used by IDLE, and it breaks a few IDLE features.

### Extensions

IDLE contains an extension facility. Preferences for extensions can be changed with the Extensions tab of the preferences dialog. See the beginning of config-extensions.def in the idlelib directory for further information. The only current default extension is zzdummy, an example also used for testing.

## idlelib --- implementation of IDLE application

<div class="module" data-synopsis="Implementation package for the IDLE shell/editor.">

idlelib

</div>

**Source code:** `Lib/idlelib`

------------------------------------------------------------------------

The Lib/idlelib package implements the IDLE application. See the rest of this page for how to use IDLE.

The files in idlelib are described in idlelib/README.txt. Access it either in idlelib or click Help =\> About IDLE on the IDLE menu. This file also maps IDLE menu items to the code that implements the item. Except for files listed under 'Startup', the idlelib code is 'private' in sense that feature changes can be backported (see `434`).

# `!imaplib` --- IMAP4 protocol client

<div class="module" data-synopsis="IMAP4 protocol client (requires sockets).">

imaplib

</div>

**Source code:** `Lib/imaplib.py`

<div class="index">

pair: IMAP4; protocol pair: IMAP4_SSL; protocol pair: IMAP4_stream; protocol

</div>

------------------------------------------------------------------------

This module defines three classes, `IMAP4`, `IMAP4_SSL` and `IMAP4_stream`, which encapsulate a connection to an IMAP4 server and implement a large subset of the IMAP4rev1 client protocol as defined in `3501`. It is backward compatible with IMAP4 (`1730`) servers, but note that the `STATUS` command is not supported in IMAP4.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

Three classes are provided by the `!imaplib` module, `IMAP4` is the base class:

<div class="IMAP4(host='', port=IMAP4_PORT, timeout=None)">

This class implements the actual IMAP4 protocol. The connection is created and protocol version (IMAP4 or IMAP4rev1) is determined when the instance is initialized. If *host* is not specified, `''` (the local host) is used. If *port* is omitted, the standard IMAP4 port (143) is used. The optional *timeout* parameter specifies a timeout in seconds for the connection attempt. If timeout is not given or is `None`, the global default socket timeout is used.

The `IMAP4` class supports the `with` statement. When used like this, the IMAP4 `LOGOUT` command is issued automatically when the `!with` statement exits. E.g.:

    >>> from imaplib import IMAP4
    >>> with IMAP4("domain.org") as M:
    ...     M.noop()
    ...
    ('OK', [b'Nothing Accomplished. d25if65hy903weo.87'])

<div class="versionchanged">

3.5 Support for the `with` statement was added.

</div>

<div class="versionchanged">

3.9 The optional *timeout* parameter was added.

</div>

</div>

Three exceptions are defined as attributes of the `IMAP4` class:

<div class="exception">

IMAP4.error

Exception raised on any errors. The reason for the exception is passed to the constructor as a string.

</div>

<div class="exception">

IMAP4.abort

IMAP4 server errors cause this exception to be raised. This is a sub-class of `IMAP4.error`. Note that closing the instance and instantiating a new one will usually allow recovery from this exception.

</div>

<div class="exception">

IMAP4.readonly

This exception is raised when a writable mailbox has its status changed by the server. This is a sub-class of `IMAP4.error`. Some other client now has write permission, and the mailbox will need to be re-opened to re-obtain write permission.

</div>

There's also a subclass for secure connections:

<div class="IMAP4_SSL(host='', port=IMAP4_SSL_PORT, *, ssl_context=None, \ timeout=None)">

This is a subclass derived from `IMAP4` that connects over an SSL encrypted socket (to use this class you need a socket module that was compiled with SSL support). If *host* is not specified, `''` (the local host) is used. If *port* is omitted, the standard IMAP4-over-SSL port (993) is used. *ssl_context* is a `ssl.SSLContext` object which allows bundling SSL configuration options, certificates and private keys into a single (potentially long-lived) structure. Please read `ssl-security` for best practices.

> [!NOTE]
> With the default *ssl_context*, the connection is encrypted but the server certificate and hostname are not verified. To verify them, pass a context created by `ssl.create_default_context`.

The optional *timeout* parameter specifies a timeout in seconds for the connection attempt. If timeout is not given or is `None`, the global default socket timeout is used.

<div class="versionchanged">

3.3 *ssl_context* parameter was added.

</div>

<div class="versionchanged">

3.4 The class now supports hostname check with `ssl.SSLContext.check_hostname` and *Server Name Indication* (see `ssl.HAS_SNI`).

</div>

<div class="versionchanged">

3.9 The optional *timeout* parameter was added.

</div>

<div class="versionchanged">

3.12 The deprecated *keyfile* and *certfile* parameters have been removed.

</div>

</div>

The second subclass allows for connections created by a child process:

<div class="IMAP4_stream(command)">

This is a subclass derived from `IMAP4` that connects to the `stdin/stdout` file descriptors created by passing *command* to `subprocess.Popen()`.

</div>

The following utility functions are defined:

<div class="function">

Internaldate2tuple(resp)

Parse a `bytes-like object` containing an IMAP4 `INTERNALDATE` response and return the corresponding local time. The return value is a `time.struct_time` tuple or `None` if the input has wrong format.

</div>

<div class="function">

Int2AP(num)

Converts an integer into a bytes representation using characters from the set \[`A` .. `P`\].

</div>

<div class="function">

ParseFlags(resp)

Converts a `bytes-like object` containing an IMAP4 `FLAGS` response to a tuple of individual flags as `bytes`. The return value is an empty tuple if the input has wrong format.

</div>

<div class="function">

Time2Internaldate(date_time)

Convert *date_time* to an IMAP4 `INTERNALDATE` representation. The return value is a string in the form: `"DD-Mmm-YYYY HH:MM:SS +HHMM"` (including double-quotes). The *date_time* argument can be a number (int or float) representing seconds since epoch (as returned by `time.time`), a 9-tuple representing local time an instance of `time.struct_time` (as returned by `time.localtime`), an aware instance of `datetime.datetime`, or a double-quoted string. In the last case, it is assumed to already be in the correct format.

</div>

Note that IMAP4 message numbers change as the mailbox changes; in particular, after an `EXPUNGE` command performs deletions the remaining messages are renumbered. So it is highly advisable to use UIDs instead, with the UID command.

At the end of the module, there is a test section that contains a more extensive example of usage.

<div class="seealso">

Documents describing the protocol, sources for servers implementing it, by the University of Washington's IMAP Information Center can all be found at (**Source Code**) <https://github.com/uw-imap/imap> (**Not Maintained**).

</div>

## IMAP4 Objects

All IMAP4rev1 commands are represented by methods of the same name, either uppercase or lowercase.

All arguments to commands are converted to strings, except for `AUTHENTICATE`, and the last argument to `APPEND` which is passed as an IMAP4 literal. If necessary (the string contains IMAP4 protocol-sensitive characters and isn't enclosed with either parentheses or double quotes) each string is quoted. However, the *password* argument to the `LOGIN` command is always quoted. If you want to avoid having an argument string quoted (eg: the *flags* argument to `STORE`) then enclose the string in parentheses (eg: `r'(\Deleted)'`). Or you can quote the string yourself; an argument that is already enclosed in double quotes is left unchanged. In general, however, it is better to pass arguments unquoted and let the module quote them as needed.

Mailbox names are encoded as modified UTF-7 (`3501`, section 5.1.3), so a mailbox name containing non-ASCII characters can be passed as an ordinary `str`. A `str` that is already valid modified UTF-7 is left unchanged, so that a name obtained from `~IMAP4.list` (raw `bytes` decoded to text) round-trips; pass `bytes` to send the exact bytes with no encoding. When `UTF8=ACCEPT` is enabled (see `~IMAP4.enable`), mailbox names are sent as UTF-8 instead.

<div class="versionchanged">

3.16 Non-ASCII mailbox names are automatically encoded as modified UTF-7.

</div>

Most commands return a tuple: `(type, [data, ...])` where *type* is usually `'OK'` or `'NO'`, and *data* is either the text from the command response, or mandated results from the command. Each *data* is either a `bytes`, or a tuple. If a tuple, then the first part is the header of the response, and the second part contains the data (ie: 'literal' value).

The *message_set* options to the commands below can be a string specifying one or more messages to be acted upon. It may be a simple message number (`'1'`), a range of message numbers (`'2:4'`), or a group of non-contiguous ranges separated by commas (`'1:3,6:9'`). A range can contain an asterisk to indicate an infinite upper bound (`'3:*'`).

Alternatively it can be specified using integers and `range` objects. It may be a single message number or a sequence. The sequence items may be integers, `(start, stop)` tuples (where `None` or `'*'` stands for the last message), or `range` objects. For example, `[1, (3, 5), 8]` and `[range(1, 6), 8]` are both equivalent to `'1,3:5,8'`.

<div class="versionchanged">

3.16 Added support for the structured *message_set*.

</div>

Command arguments that are parenthesized lists of atoms --- such as the *flag_list* argument of `~IMAP4.store` and the *flags* argument of `~IMAP4.append`, the *names* argument of `~IMAP4.status`, the *sort_criteria* argument of `~IMAP4.sort`, or the *message_parts* argument of `~IMAP4.fetch` --- can be passed as a sequence of strings instead of a single preformatted string. For example, `[r'\Seen', r'\Answered']` is equivalent to `(\Seen \Answered)`.

<div class="versionchanged">

3.16 Added support for passing these arguments as a sequence.

</div>

<div id="imap4-params">

The value-bearing arguments of the search and fetch commands can be quoted by hand, but this is error prone. Instead, they may contain `?` placeholders that are substituted, and quoted as required, from a *params* keyword argument, in the manner of `sqlite3` parameter substitution:

    # SEARCH FROM me@example.com SUBJECT "trip report"
    M.search(None, 'FROM ? SUBJECT ?', params=['me@example.com', 'trip report'])

    # FETCH 1:5 (FLAGS BODY[HEADER.FIELDS (DATE FROM)])
    M.fetch('1:5', 'FLAGS BODY[HEADER.FIELDS ?]', params=[['DATE', 'FROM']])

</div>

The placeholders are:

- `?` --- an `astring`: a string (which will be quoted if necessary), an integer, or a list of integers and/or strings (which will be sent as a parenthesized list);
- `?f` --- a flag or a list of flags, sent verbatim without quoting;
- `?s` --- a *message_set* in the structured form described above.

`??` stands for a literal `?`.

Substitution is only performed when *params* is given; if no *params* are given, an argument containing a literal `?` is unchanged. The *params* keyword is accepted by `~IMAP4.search`, `~IMAP4.fetch`, `~IMAP4.sort`, `~IMAP4.thread` and `~IMAP4.uid`.

<div class="versionadded">

3.16 The *params* keyword argument.

</div>

An `IMAP4` instance has the following methods:

<div class="method">

IMAP4.append(mailbox, flags, date_time, message, \*, translate_line_endings=True)

Append *message* to named mailbox.

*flags* may be `None` or a string of IMAP flag tokens. Multiple flags are separated by spaces, for example `r'\Seen \Answered'`. If *flags* is not already enclosed in parentheses, parentheses are added automatically.

If *translate_line_endings* is true (the default), line endings in *message* are translated to CRLF. Pass `False` to send the message literal exactly as given, which is required to preserve messages that contain bare CR or LF. In that case *message* must already use CRLF line endings as required by `3501`; for example, serialize `email` messages using `email.policy.SMTP`.

<div class="versionchanged">

3.16 Added the *translate_line_endings* parameter.

</div>

</div>

<div class="method">

IMAP4.authenticate(mechanism, authobject)

Authenticate command --- requires response processing.

*mechanism* specifies which authentication mechanism is to be used - it should appear in the instance variable `capabilities` in the form `AUTH=mechanism`.

*authobject* must be a callable object:

    data = authobject(response)

It will be called to process server continuation responses; the *response* argument it is passed will be `bytes`. It should return `bytes` *data* that will be base64 encoded and sent to the server. It should return `None` if the client abort response `*` should be sent instead.

<div class="versionchanged">

3.5 string usernames and passwords are now encoded to `utf-8` instead of being limited to ASCII.

</div>

</div>

<div class="method">

IMAP4.check()

Checkpoint mailbox on server.

</div>

<div class="method">

IMAP4.close()

Close currently selected mailbox. Deleted messages are removed from writable mailbox. This is the recommended command before `LOGOUT`.

</div>

<div class="method">

IMAP4.copy(message_set, new_mailbox, \*, uid=False)

Copy *message_set* messages onto end of *new_mailbox*.

If *uid* is true, *message_set* is a set of UIDs and the `UID COPY` command is used instead of `COPY`.

<div class="versionchanged">

3.16 Added the *uid* parameter.

</div>

</div>

<div class="method">

IMAP4.create(mailbox)

Create new mailbox named *mailbox*.

</div>

<div class="method">

IMAP4.delete(mailbox)

Delete old mailbox named *mailbox*.

</div>

<div class="method">

IMAP4.deleteacl(mailbox, who)

Delete the ACLs (remove any rights) set for who on mailbox.

</div>

<div class="method">

IMAP4.enable(capability)

Enable *capability* (see `5161`). Most capabilities do not need to be enabled. Currently only the `UTF8=ACCEPT` capability is supported (see [RFC 6855](http://www.faqs.org/rfcs/rfc6855.html "RFC 6855")).

<div class="versionadded">

3.5 The `enable` method itself, and [RFC 6855](http://www.faqs.org/rfcs/rfc6855.html "RFC 6855") support.

</div>

</div>

<div class="method">

IMAP4.expunge(message_set=None, \*, uid=False)

Permanently remove deleted items from selected mailbox. Generates an `EXPUNGE` response for each deleted message. Returned data contains a list of `EXPUNGE` message numbers in order received.

If *uid* is true, the `UID EXPUNGE` command (`4315`) is used to remove only the messages that both are marked as deleted and have a UID in *message_set*. *message_set* is required in this case, and must be omitted otherwise.

<div class="versionchanged">

3.16 Added the *message_set* and *uid* parameters.

</div>

</div>

<div class="method">

IMAP4.fetch(message_set, message_parts, \*, uid=False, params=None)

Fetch (parts of) messages. *message_parts* should be a string of message part names enclosed within parentheses, eg: `"(UID BODY[TEXT])"`. Returned data are tuples of message part envelope and data.

If *uid* is true, *message_set* is a set of UIDs and the message numbers in the response are UIDs (`UID FETCH`).

If *params* is given, `?` placeholders in *message_parts* are substituted with the quoted parameters (see `the placeholders <imap4-params>`).

<div class="versionchanged">

3.16 Added the *params* and *uid* parameters.

</div>

</div>

<div class="method">

IMAP4.getacl(mailbox)

Get the `ACL`s for *mailbox*. The method is non-standard, but is supported by the `Cyrus` server.

</div>

<div class="method">

IMAP4.getannotation(mailbox, entry, attribute)

Retrieve the specified `ANNOTATION`s for *mailbox*. The method is non-standard, but is supported by the `Cyrus` server.

</div>

<div class="method">

IMAP4.getquota(root)

Get the `quota` *root*'s resource usage and limits. This method is part of the IMAP4 QUOTA extension defined in rfc2087.

</div>

<div class="method">

IMAP4.getquotaroot(mailbox)

Get the list of `quota` `roots` for the named *mailbox*. This method is part of the IMAP4 QUOTA extension defined in rfc2087.

</div>

<div class="method">

IMAP4.id(fields=None)

Send client identification information to the server and return the identification information sent back by the server (the `ID` command, defined in `2971`). *fields* is a mapping of field names to values (for example, `{'name': 'myclient', 'version': '1.0'}`); a value can be `None`. The server must support the `ID` capability.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

IMAP4.idle(duration=None)

Return an `!Idler`: an iterable context manager implementing the IMAP4 `IDLE` command as defined in `2177`.

The returned object sends the `IDLE` command when activated by the `with` statement, produces IMAP untagged responses via the `iterator` protocol, and sends `DONE` upon context exit.

All untagged responses that arrive after sending the `IDLE` command (including any that arrive before the server acknowledges the command) will be available via iteration. Any leftover responses (those not iterated in the `with` context) can be retrieved in the usual way after `IDLE` ends, using `IMAP4.response`.

Responses are represented as `(type, [data, ...])` tuples, as described in `IMAP4 Objects <imap4-objects>`.

The *duration* argument sets a maximum duration (in seconds) to keep idling, after which any ongoing iteration will stop. It can be an `int` or `float`, or `None` for no time limit. Callers wishing to avoid inactivity timeouts on servers that impose them should keep this at most 29 minutes (1740 seconds). Requires a socket connection; *duration* must be `None` on `IMAP4_stream` connections.

```
>>> with M.idle(duration=29 * 60) as idler:
...     for typ, data in idler:
...         print(typ, data)
...
EXISTS [b'1']
RECENT [b'1']
```

<div class="method">

Idler.burst(interval=0.1)

Yield a burst of responses no more than *interval* seconds apart (expressed as an `int` or `float`).

This `generator` is an alternative to iterating one response at a time, intended to aid in efficient batch processing. It retrieves the next response along with any immediately available subsequent responses. (For example, a rapid series of `EXPUNGE` responses after a bulk delete.)

Requires a socket connection; does not work on `IMAP4_stream` connections.

```
>>> with M.idle() as idler:
...     # get a response and any others following by < 0.1 seconds
...     batch = list(idler.burst())
...     print(f'processing {len(batch)} responses...')
...     print(batch)
...
processing 3 responses...
[('EXPUNGE', [b'2']), ('EXPUNGE', [b'1']), ('RECENT', [b'0'])]
```

> [!TIP]
> The `IDLE` context's maximum duration, as passed to `IMAP4.idle`, is respected when waiting for the first response in a burst. Therefore, an expired `!Idler` will cause this generator to return immediately without producing anything. Callers should consider this if using it in a loop.

</div>

> [!NOTE]
> The iterator returned by `IMAP4.idle` is usable only within a `with` statement. Before or after that context, unsolicited responses are collected internally whenever a command finishes, and can be retrieved with `IMAP4.response`.

> [!NOTE]
> The `!Idler` class name and structure are internal interfaces, subject to change. Calling code can rely on its context management, iteration, and public method to remain stable, but should not subclass, instantiate, compare, or otherwise directly reference the class.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

IMAP4.list(directory='', pattern='\*')

List mailbox names in *directory* matching *pattern*. *directory* defaults to the top-level mail folder, and *pattern* defaults to match anything. Returned data contains a list of `LIST` responses.

</div>

<div class="method">

IMAP4.login(user, password)

Identify the client using a plaintext password. The *password* will be quoted.

</div>

<div class="method">

IMAP4.login_cram_md5(user, password)

Force use of `CRAM-MD5` authentication when identifying the client to protect the password. It will only work if the server `CAPABILITY` response includes the phrase `AUTH=CRAM-MD5`.

<div class="versionchanged">

3.15 An `IMAP4.error` is raised if MD5 support is not available.

</div>

</div>

<div class="method">

IMAP4.login_plain(user, password)

Authenticate using the `PLAIN` SASL mechanism (`4616`).

This is a plaintext authentication mechanism that can be used instead of `login` when UTF-8 support is required (see `6855`). Since the credentials are only base64-encoded, not encrypted, this method should only be used over a TLS-protected connection, such as `IMAP4_SSL` or after `starttls`.

It will only work if the server supports the `PLAIN` mechanism, which it need not advertise as `AUTH=PLAIN` in its `CAPABILITY` response.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

IMAP4.logout()

Shutdown connection to server. Returns server `BYE` response.

<div class="versionchanged">

3.8 The method no longer ignores silently arbitrary exceptions.

</div>

</div>

<div class="method">

IMAP4.lsub(directory='', pattern='\*')

List subscribed mailbox names in directory matching pattern. *directory* defaults to the top level directory and *pattern* defaults to match any mailbox. Returned data are tuples of message part envelope and data.

</div>

<div class="method">

IMAP4.move(message_set, new_mailbox, \*, uid=False)

Move *message_set* messages onto end of *new_mailbox*.

The server must support the `MOVE` capability (`6851`).

If *uid* is true, *message_set* is a set of UIDs and the `UID MOVE` command is used instead of `MOVE`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

IMAP4.myrights(mailbox)

Show my ACLs for a mailbox (i.e. the rights that I have on mailbox).

</div>

<div class="method">

IMAP4.namespace()

Returns IMAP namespaces as defined in `2342`.

</div>

<div class="method">

IMAP4.noop()

Send `NOOP` to server.

</div>

<div class="method">

IMAP4.open(host, port, timeout=None)

Opens socket to *port* at *host*. The optional *timeout* parameter specifies a timeout in seconds for the connection attempt. If timeout is not given or is `None`, the global default socket timeout is used. Also note that if the *timeout* parameter is set to be zero, it will raise a `ValueError` to reject creating a non-blocking socket. This method is implicitly called by the `IMAP4` constructor. The connection objects established by this method will be used in the `IMAP4.read`, `IMAP4.readline`, `IMAP4.send`, and `IMAP4.shutdown` methods. You may override this method.

<div class="audit-event">

imaplib.open self,host,port imaplib.IMAP4.open

</div>

<div class="versionchanged">

3.9 The *timeout* parameter was added.

</div>

</div>

<div class="method">

IMAP4.partial(message_num, message_part, start, length)

Fetch truncated part of a message. Returned data is a tuple of message part envelope and data.

</div>

<div class="method">

IMAP4.proxyauth(user)

Assume authentication as *user*. Allows an authorised administrator to proxy into any user's mailbox.

</div>

<div class="method">

IMAP4.read(size)

Reads *size* bytes from the remote server. You may override this method.

</div>

<div class="method">

IMAP4.readline()

Reads one line from the remote server. You may override this method.

</div>

<div class="method">

IMAP4.recent()

Prompt server for an update. Returned data is `None` if no new messages, else value of `RECENT` response.

</div>

<div class="method">

IMAP4.rename(oldmailbox, newmailbox)

Rename mailbox named *oldmailbox* to *newmailbox*.

</div>

<div class="method">

IMAP4.response(code)

Return data for response *code* if received, or `None`. Returns the given code, instead of the usual type.

</div>

<div class="method">

IMAP4.search(charset, criterion\[, ...\], \*, uid=False, params=None)

Search mailbox for matching messages. *charset* may be `None`, in which case no `CHARSET` will be specified in the request to the server. The IMAP protocol requires that at least one criterion be specified; an exception will be raised when the server returns an error. *charset* must be `None` if the `UTF8=ACCEPT` capability was enabled using the `enable` command.

If *uid* is true, the message numbers in the response are UIDs (`UID SEARCH`).

A criterion passed as `str` is encoded to *charset* (which must name a codec known to Python); pass `bytes` to send a criterion that is already encoded, for example when *charset* is one that Python does not support. When *charset* is `None` (as it must be under `UTF8=ACCEPT`), the criterion is sent using the connection's encoding instead.

If *params* is given, `?` placeholders in the criteria are substituted with the quoted parameters (see `the placeholders <imap4-params>`).

Example:

    # M is a connected IMAP4 instance...
    typ, msgnums = M.search(None, 'FROM', '"John Smith"')

    # or:
    typ, msgnums = M.search(None, '(FROM "John Smith")')

    # or, letting the module quote the value (this is recommended):
    typ, msgnums = M.search(None, 'FROM ?', params=['John Smith'])

<div class="versionchanged">

3.16 Added the *params* and *uid* parameters. `str` search criteria are encoded to *charset*.

</div>

</div>

<div class="method">

IMAP4.select(mailbox='INBOX', readonly=False)

Select a mailbox. Returned data is the count of messages in *mailbox* (`EXISTS` response). The default *mailbox* is `'INBOX'`. If the *readonly* flag is set, modifications to the mailbox are not allowed.

</div>

<div class="method">

IMAP4.send(data)

Sends `data` to the remote server. You may override this method.

<div class="audit-event">

imaplib.send self,data imaplib.IMAP4.send

</div>

</div>

<div class="method">

IMAP4.setacl(mailbox, who, what)

Set an `ACL` for *mailbox*. The method is non-standard, but is supported by the `Cyrus` server.

</div>

<div class="method">

IMAP4.setannotation(mailbox, entry, attribute\[, ...\])

Set `ANNOTATION`s for *mailbox*. The method is non-standard, but is supported by the `Cyrus` server.

</div>

<div class="method">

IMAP4.setquota(root, limits)

Set the `quota` *root*'s resource *limits*. This method is part of the IMAP4 QUOTA extension defined in rfc2087.

</div>

<div class="method">

IMAP4.shutdown()

Close connection established in `open`. This method is implicitly called by `IMAP4.logout`. You may override this method.

</div>

<div class="method">

IMAP4.socket()

Returns socket instance used to connect to server.

</div>

<div class="method">

IMAP4.sort(sort_criteria, charset, search_criterion\[, ...\], \*, uid=False, params=None)

The `sort` command is a variant of `search` with sorting semantics for the results. Returned data contains a space separated list of matching message numbers.

Sort has two arguments before the *search_criterion* argument(s); a parenthesized list of *sort_criteria*, and the searching *charset*. Note that unlike `search`, the searching *charset* argument is mandatory. There is also a `uid sort` command which corresponds to `sort` the way that `uid search` corresponds to `search`. The `sort` command first searches the mailbox for messages that match the given searching criteria using the charset argument for the interpretation of strings in the searching criteria. It then returns the numbers of matching messages.

If *uid* is true, the message numbers in the response are UIDs (`UID SORT`).

As with `search`, a *search_criterion* passed as `str` is encoded to *charset*; pass `bytes` to send one already encoded.

If *params* is given, `?` placeholders in the search criteria are substituted with the quoted parameters (see `the placeholders <imap4-params>`).

This is an `IMAP4rev1` extension command.

<div class="versionchanged">

3.16 Added the *params* and *uid* parameters. `str` search criteria are encoded to *charset*.

</div>

</div>

<div class="method">

IMAP4.starttls(ssl_context=None)

Send a `STARTTLS` command. The *ssl_context* argument is optional and should be a `ssl.SSLContext` object. This will enable encryption on the IMAP connection. Please read `ssl-security` for best practices.

> [!NOTE]
> With the default *ssl_context*, the connection is encrypted but the server certificate and hostname are not verified. To verify them, pass a context created by `ssl.create_default_context`.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.4 The method now supports hostname check with `ssl.SSLContext.check_hostname` and *Server Name Indication* (see `ssl.HAS_SNI`).

</div>

</div>

<div class="method">

IMAP4.status(mailbox, names)

Request named status conditions for *mailbox*.

</div>

<div class="method">

IMAP4.store(message_set, command, flag_list, \*, uid=False)

Alters flag dispositions for messages in mailbox. *command* is specified by section 6.4.6 of `3501` as being one of "FLAGS", "+FLAGS", or "-FLAGS", optionally with a suffix of ".SILENT".

If *uid* is true, *message_set* is a set of UIDs and the `UID STORE` command is used instead of `STORE`.

For example, to set the delete flag on all messages:

    typ, data = M.search(None, 'ALL')
    for num in data[0].split():
       M.store(num, '+FLAGS', r'\Deleted')
    M.expunge()

> [!NOTE]
> Creating flags containing '\]' (for example: "\[test\]") violates `3501` (the IMAP protocol). However, imaplib has historically allowed creation of such flags, and popular IMAP servers, such as Gmail, accept and produce such flags. There are non-Python programs which also create such flags. Although it is an RFC violation and IMAP clients and servers are supposed to be strict, imaplib still continues to allow such flags to be created for backward compatibility reasons, and as of Python 3.6, handles them if they are sent from the server, since this improves real-world compatibility.

<div class="versionchanged">

3.16 Added the *uid* parameter.

</div>

</div>

<div class="method">

IMAP4.subscribe(mailbox)

Subscribe to new mailbox.

</div>

<div class="method">

IMAP4.thread(threading_algorithm, charset, search_criterion\[, ...\], \*, uid=False, params=None)

The `thread` command is a variant of `search` with threading semantics for the results. Returned data contains a space separated list of thread members.

Thread members consist of zero or more messages numbers, delimited by spaces, indicating successive parent and child.

Thread has two arguments before the *search_criterion* argument(s); a *threading_algorithm*, and the searching *charset*. Note that unlike `search`, the searching *charset* argument is mandatory. There is also a `uid thread` command which corresponds to `thread` the way that `uid search` corresponds to `search`. The `thread` command first searches the mailbox for messages that match the given searching criteria using the *charset* argument for the interpretation of strings in the searching criteria. It then returns the matching messages threaded according to the specified threading algorithm.

If *uid* is true, the message numbers in the response are UIDs (`UID THREAD`).

As with `search`, a *search_criterion* passed as `str` is encoded to *charset*; pass `bytes` to send one already encoded.

If *params* is given, `?` placeholders in the search criteria are substituted with the quoted parameters (see `the placeholders <imap4-params>`).

This is an `IMAP4rev1` extension command.

<div class="versionchanged">

3.16 Added the *params* and *uid* parameters. `str` search criteria are encoded to *charset*.

</div>

</div>

<div class="method">

IMAP4.uid(command, arg\[, ...\], \*, params=None)

Execute command args with messages identified by UID, rather than message number. Returns response appropriate to command. At least one argument must be supplied; if none are provided, the server will return an error and an exception will be raised.

If *params* is given, `?` placeholders in the `SEARCH`, `SORT` and `THREAD` criteria or in the `FETCH` parts are substituted with the quoted parameters (see `the placeholders <imap4-params>`).

<div class="versionchanged">

3.16 Added the *params* parameter.

</div>

</div>

<div class="method">

IMAP4.unsubscribe(mailbox)

Unsubscribe from old mailbox.

</div>

<div class="method">

IMAP4.unselect()

`imaplib.IMAP4.unselect` frees server's resources associated with the selected mailbox and returns the server to the authenticated state. This command performs the same actions as `imaplib.IMAP4.close`, except that no messages are permanently removed from the currently selected mailbox.

<div class="versionadded">

3.9

</div>

</div>

<div class="method">

IMAP4.xatom(name\[, ...\])

Allow simple extension commands notified by server in `CAPABILITY` response.

</div>

The following attributes are defined on instances of `IMAP4`:

<div class="attribute">

IMAP4.capabilities

A tuple of the capabilities advertised by the server, in upper case.

It is set when the connection is established, and refreshed after a successful `~IMAP4.login`, `~IMAP4.authenticate` or `~IMAP4.starttls`, because the server can advertise different capabilities in different connection states.

<div class="versionchanged">

3.14.7 Refreshed after `~IMAP4.login` and `~IMAP4.authenticate`.

</div>

</div>

<div class="attribute">

IMAP4.PROTOCOL_VERSION

The most recent supported protocol in the `CAPABILITY` response from the server.

</div>

<div class="attribute">

IMAP4.debug

Integer value to control debugging output. The initialize value is taken from the module variable `Debug`. Values greater than three trace each command.

</div>

<div class="attribute">

IMAP4.utf8_enabled

Boolean value that is normally `False`, but is set to `True` if an `enable` command is successfully issued for the `UTF8=ACCEPT` capability.

<div class="versionadded">

3.5

</div>

</div>

<div class="property">

IMAP4.file

Internal `~io.BufferedReader` associated with the underlying socket. This property is documented for legacy purposes but not part of the public interface. The caller is responsible to ensure that the current file is closed before changing it.

<div class="deprecated-removed">

3.15 3.19

</div>

</div>

## IMAP4 Example

Here is a minimal example (without error checking) that opens a mailbox and retrieves and prints all messages:

    import getpass, imaplib

    M = imaplib.IMAP4(host='example.org')
    M.login(getpass.getuser(), getpass.getpass())
    M.select()
    typ, data = M.search(None, 'ALL')
    for num in data[0].split():
        typ, data = M.fetch(num, '(RFC822)')
        print('Message %s\n%s\n' % (num, data[0][1]))
    M.close()
    M.logout()

> [!NOTE]
> A `FETCH` response may contain additional or unsolicited data (see `3501`, section 7.4.2), so production code should inspect the whole response rather than rely on `data[0][1]`.

# `!imghdr` --- Determine the type of an image

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

imghdr

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

Possible replacements are third-party libraries from PyPI: `filetype`, `puremagic`, or `python-magic`. These are not supported or maintained by the Python core team.

The last version of Python that provided the `!imghdr` module was [Python 3.12](https://docs.python.org/3.12/library/imghdr.html).

# `!imp` --- Access the import internals

<div class="module" data-synopsis="Removed in 3.12." data-deprecated="">

imp

</div>

<div class="deprecated-removed">

3.4 3.12

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.12 <whatsnew312-removed-imp>` after being deprecated in Python 3.4.

The `removal notice <whatsnew312-removed-imp>` includes guidance for migrating code from `!imp` to `importlib`.

The last version of Python that provided the `!imp` module was [Python 3.11](https://docs.python.org/3.11/library/imp.html).

# `!importlib` --- The implementation of `!import`

<div class="module" data-synopsis="The implementation of the import machinery.">

importlib

</div>

<div class="versionadded">

3.1

</div>

**Source code:** `Lib/importlib/__init__.py`

------------------------------------------------------------------------

## Introduction

The purpose of the `!importlib` package is three-fold.

One is to provide the implementation of the `import` statement (and thus, by extension, the `__import__` function) in Python source code. This provides an implementation of `!import` which is portable to any Python interpreter. This also provides an implementation which is easier to comprehend than one implemented in a programming language other than Python.

Two, the components to implement `import` are exposed in this package, making it easier for users to create their own custom objects (known generically as an `importer`) to participate in the import process.

Three, the package contains modules exposing additional functionality for managing aspects of Python packages:

- `importlib.metadata` presents access to metadata from third-party distributions.
- `importlib.resources` provides routines for accessing non-code "resources" from Python packages.

<div class="seealso">

`import`  
The language reference for the `import` statement.

[Packages specification](https://www.python.org/doc/essays/packages/)  
Original specification of packages. Some semantics have changed since the writing of this document (e.g. redirecting based on `None` in `sys.modules`).

The `.__import__` function  
The `import` statement is syntactic sugar for this function.

`sys-path-init`  
The initialization of `sys.path`.

`235`  
Import on Case-Insensitive Platforms

`263`  
Defining Python Source Code Encodings

`302`  
New Import Hooks

`328`  
Imports: Multi-Line and Absolute/Relative

`366`  
Main module explicit relative imports

`420`  
Implicit namespace packages

`451`  
A ModuleSpec Type for the Import System

`488`  
Elimination of PYO files

`489`  
Multi-phase extension module initialization

`552`  
Deterministic pycs

`3120`  
Using UTF-8 as the Default Source Encoding

`3147`  
PYC Repository Directories

</div>

## Functions

<div class="function">

\_\_import\_\_(name, globals=None, locals=None, fromlist=(), level=0)

An implementation of the built-in `__import__` function.

> [!NOTE]
> Programmatic importing of modules should use `import_module` instead of this function.

</div>

<div class="function">

import_module(name, package=None)

Import a module. The *name* argument specifies what module to import in absolute or relative terms (e.g. either `pkg.mod` or `..mod`). If the name is specified in relative terms, then the *package* argument must be set to the name of the package which is to act as the anchor for resolving the package name (e.g. `import_module('..mod', 'pkg.subpkg')` will import `pkg.mod`).

The `import_module` function acts as a simplifying wrapper around `importlib.__import__`. This means all semantics of the function are derived from `importlib.__import__`. The most important difference between these two functions is that `import_module` returns the specified package or module (e.g. `pkg.mod`), while `__import__` returns the top-level package or module (e.g. `pkg`).

If you are dynamically importing a module that was created since the interpreter began execution (e.g., created a Python source file), you may need to call `invalidate_caches` in order for the new module to be noticed by the import system.

If the module cannot be imported, `import_module` will raise `ImportError` or an appropriate subclass like `ModuleNotFoundError`.

<div class="versionchanged">

3.3 Parent packages are automatically imported.

</div>

</div>

<div class="function">

invalidate_caches()

Invalidate the internal caches of finders stored at `sys.meta_path`. If a finder implements `invalidate_caches()` then it will be called to perform the invalidation. This function should be called if any modules are created/installed while your program is running to guarantee all finders will notice the new module's existence.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.10 Namespace packages created/installed in a different `sys.path` location after the same namespace was already imported are noticed.

</div>

</div>

<div class="function">

reload(module)

Reload a previously imported *module*. The argument must be a module object, so it must have been successfully imported before. This is useful if you have edited the module source file using an external editor and want to try out the new version without leaving the Python interpreter. The return value is the module object (which can be different if re-importing causes a different object to be placed in `sys.modules`).

When `reload` is executed:

- Python module's code is recompiled and the module-level code re-executed, defining a new set of objects which are bound to names in the module's dictionary by reusing the `loader` which originally loaded the module. The `init` function of extension modules is not called a second time.
- As with all other objects in Python the old objects are only reclaimed after their reference counts drop to zero.
- The names in the module namespace are updated to point to any new or changed objects.
- Other references to the old objects (such as names external to the module) are not rebound to refer to the new objects and must be updated in each namespace where they occur if that is desired.

There are a number of other caveats:

When a module is reloaded, its dictionary (containing the module's global variables) is retained. Redefinitions of names will override the old definitions, so this is generally not a problem. If the new version of a module does not define a name that was defined by the old version, the old definition remains. This feature can be used to the module's advantage if it maintains a global table or cache of objects --- with a `try` statement it can test for the table's presence and skip its initialization if desired:

    try:
        cache
    except NameError:
        cache = {}

It is generally not very useful to reload built-in or dynamically loaded modules. Reloading `sys`, `__main__`, `builtins` and other key modules is not recommended. In many cases extension modules are not designed to be initialized more than once, and may fail in arbitrary ways when reloaded.

If a module imports objects from another module using `from` ... `import` ..., calling `reload` for the other module does not redefine the objects imported from it --- one way around this is to re-execute the `!from` statement, another is to use `!import` and qualified names (*module.name*) instead.

If a module instantiates instances of a class, reloading the module that defines the class does not affect the method definitions of the instances --- they continue to use the old class definition. The same is true for derived classes.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.7 `ModuleNotFoundError` is raised when the module being reloaded lacks a `~importlib.machinery.ModuleSpec`.

</div>

> [!WARNING]
> This function is not thread-safe. Calling it from multiple threads can result in unexpected behavior. It's recommended to use the `threading.Lock` or other synchronization primitives for thread-safe module reloading.

</div>

## `!importlib.abc` -- Abstract base classes related to import

<div class="module" data-synopsis="Abstract base classes related to import">

importlib.abc

</div>

**Source code:** `Lib/importlib/abc.py`

------------------------------------------------------------------------

The `!importlib.abc` module contains all of the core abstract base classes used by `import`. Some subclasses of the core abstract base classes are also provided to help in implementing the core ABCs.

ABC hierarchy:

    object
     +-- MetaPathFinder
     +-- PathEntryFinder
     +-- Loader
          +-- ResourceLoader --------+
          +-- InspectLoader          |
               +-- ExecutionLoader --+
                                     +-- FileLoader
                                     +-- SourceLoader

<div class="MetaPathFinder">

An abstract base class representing a `meta path finder`.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.10 No longer a subclass of `!Finder`.

</div>

<div class="method">

find_spec(fullname, path, target=None)

An abstract method for finding a `spec <module spec>` for the specified module. If this is a top-level import, *path* will be `None`. Otherwise, this is a search for a subpackage or module and *path* will be the value of `~module.__path__` from the parent package. If a spec cannot be found, `None` is returned. When passed in, `target` is a module object that the finder may use to make a more educated guess about what spec to return. `importlib.util.spec_from_loader` may be useful for implementing concrete `MetaPathFinders`.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

invalidate_caches()

An optional method which, when called, should invalidate any internal cache used by the finder. Used by `importlib.invalidate_caches` when invalidating the caches of all finders on `sys.meta_path`.

<div class="versionchanged">

3.4 Returns `None` when called instead of `NotImplemented`.

</div>

</div>

<div class="method">

discover(parent=None)

An optional method which searches for possible specs with given *parent* module spec. If *parent* is *None*, `MetaPathFinder.discover` will search for top-level modules.

Returns an iterable of possible specs.

Raises `ValueError` if *parent* is not a package module.

> [!WARNING]
> This method can potentially yield a very large number of objects, and it may carry out IO operations when computing these values.
>
> Because of this, it will generally be desirable to compute the result values on-the-fly, as they are needed. As such, the returned object is only guaranteed to be an `iterable <collections.abc.Iterable>`, instead of a `list` or other `collection <collections.abc.Collection>` type.

<div class="versionadded">

3.15

</div>

</div>

</div>

<div class="PathEntryFinder">

An abstract base class representing a `path entry finder`. Though it bears some similarities to `MetaPathFinder`, `PathEntryFinder` is meant for use only within the path-based import subsystem provided by `importlib.machinery.PathFinder`.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.10 No longer a subclass of `!Finder`.

</div>

<div class="method">

find_spec(fullname, target=None)

An abstract method for finding a `spec <module spec>` for the specified module. The finder will search for the module only within the `path entry` to which it is assigned. If a spec cannot be found, `None` is returned. When passed in, `target` is a module object that the finder may use to make a more educated guess about what spec to return. `importlib.util.spec_from_loader` may be useful for implementing concrete `PathEntryFinders`.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

invalidate_caches()

An optional method which, when called, should invalidate any internal cache used by the finder. Used by `importlib.machinery.PathFinder.invalidate_caches` when invalidating the caches of all cached finders.

</div>

<div class="method">

discover(parent=None)

An optional method which searches for possible specs with given *parent* module spec. If *parent* is *None*, `PathEntryFinder.discover` will search for top-level modules.

Returns an iterable of possible specs.

Raises `ValueError` if *parent* is not a package module.

> [!WARNING]
> This method can potentially yield a very large number of objects, and it may carry out IO operations when computing these values.
>
> Because of this, it will generally be desirable to compute the result values on-the-fly, as they are needed. As such, the returned object is only guaranteed to be an `iterable <collections.abc.Iterable>`, instead of a `list` or other `collection <collections.abc.Collection>` type.

<div class="versionadded">

3.15

</div>

</div>

</div>

<div class="Loader">

An abstract base class for a `loader`. See `302` for the exact definition for a loader.

Loaders that wish to support resource reading should implement a `!get_resource_reader` method as specified by `importlib.resources.abc.ResourceReader`.

<div class="versionchanged">

3.7 Introduced the optional `!get_resource_reader` method.

</div>

<div class="versionchanged">

3.15 Removed the `load_module()` method.

</div>

<div class="method">

create_module(spec)

A method that returns the module object to use when importing a module. This method may return `None`, indicating that default module creation semantics should take place.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.6 This method is no longer optional when `exec_module` is defined.

</div>

</div>

<div class="method">

exec_module(module)

An abstract method that executes the module in its own namespace when a module is imported or reloaded. The module should already be initialized when `exec_module` is called. When this method exists, `create_module` must be defined.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.6 `create_module` must also be defined.

</div>

</div>

</div>

<div class="ResourceLoader">

*Superseded by TraversableResources*

> An abstract base class for a `loader` which implements the optional `302` protocol for loading arbitrary resources from the storage back-end.
>
> <div class="deprecated">
>
> 3.7 This ABC is deprecated in favour of supporting resource loading through `importlib.resources.abc.TraversableResources`. This class exists for backwards compatibility only with other ABCs in this module.
>
> </div>
>
> <div class="method" data-abstractmethod="">
>
> get_data(path)
>
> An abstract method to return the bytes for the data located at *path*. Loaders that have a file-like storage back-end that allows storing arbitrary data can implement this abstract method to give direct access to the data stored. `OSError` is to be raised if the *path* cannot be found. The *path* is expected to be constructed using a module's `~module.__file__` attribute or an item from a package's `~module.__path__`.
>
> <div class="versionchanged">
>
> 3.4 Raises `OSError` instead of `NotImplementedError`.
>
> </div>
>
> </div>

</div>

<div class="InspectLoader">

An abstract base class for a `loader` which implements the optional `302` protocol for loaders that inspect modules.

<div class="method">

get_code(fullname)

Return the code object for a module, or `None` if the module does not have a code object (as would be the case, for example, for a built-in module). Raise an `ImportError` if loader cannot find the requested module.

> [!NOTE]
> While the method has a default implementation, it is suggested that it be overridden if possible for performance.

<div class="index">

single: universal newlines; importlib.abc.InspectLoader.get_source method

</div>

<div class="versionchanged">

3.4 No longer abstract and a concrete implementation is provided.

</div>

</div>

<div class="method" data-abstractmethod="">

get_source(fullname)

An abstract method to return the source of a module. It is returned as a text string using `universal newlines`, translating all recognized line separators into `'\n'` characters. Returns `None` if no source is available (e.g. a built-in module). Raises `ImportError` if the loader cannot find the module specified.

<div class="versionchanged">

3.4 Raises `ImportError` instead of `NotImplementedError`.

</div>

</div>

<div class="method">

is_package(fullname)

An optional method to return a true value if the module is a package, a false value otherwise. `ImportError` is raised if the `loader` cannot find the module.

<div class="versionchanged">

3.4 Raises `ImportError` instead of `NotImplementedError`.

</div>

</div>

<div class="staticmethod">

source_to_code(data, path='\<string\>', fullname=None)

Create a code object from Python source.

The *data* argument can be whatever the `compile` function supports (i.e. string or bytes). The *path* argument should be the "path" to where the source code originated from, which can be an abstract concept (e.g. location in a zip file).

With the subsequent code object one can execute it in a module by running `exec(code, module.__dict__)`.

The optional argument *fullname* specifies the module name. It is needed to unambiguous `filter <warning-filter>` syntax warnings by module name.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.5 Made the method static.

</div>

<div class="versionadded">

3.15 Added the *fullname* parameter.

</div>

</div>

<div class="method">

exec_module(module)

Implementation of `Loader.exec_module`.

<div class="versionadded">

3.4

</div>

</div>

</div>

<div class="ExecutionLoader">

An abstract base class which inherits from `InspectLoader` that, when implemented, helps a module to be executed as a script. The ABC represents an optional `302` protocol.

<div class="method" data-abstractmethod="">

get_filename(fullname)

An abstract method that is to return the value of `~module.__file__` for the specified module. If no path is available, `ImportError` is raised.

If source code is available, then the method should return the path to the source file, regardless of whether a bytecode was used to load the module.

<div class="versionchanged">

3.4 Raises `ImportError` instead of `NotImplementedError`.

</div>

</div>

</div>

<div class="FileLoader(fullname, path)">

An abstract base class which inherits from `ResourceLoader` and `ExecutionLoader`, providing concrete implementations of `ResourceLoader.get_data` and `ExecutionLoader.get_filename`.

The *fullname* argument is a fully resolved name of the module the loader is to handle. The *path* argument is the path to the file for the module.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.15 Removed the `load_module()` method.

</div>

<div class="attribute">

name

The name of the module the loader can handle.

</div>

<div class="attribute">

path

Path to the file of the module.

</div>

<div class="method" data-abstractmethod="">

get_filename(fullname)

Returns `path`.

</div>

<div class="method" data-abstractmethod="">

get_data(path)

Reads *path* as a binary file and returns the bytes from it.

</div>

</div>

<div class="SourceLoader">

An abstract base class for implementing source (and optionally bytecode) file loading. The class inherits from both `ResourceLoader` and `ExecutionLoader`, requiring the implementation of:

- `ResourceLoader.get_data`

- `ExecutionLoader.get_filename`  
  Should only return the path to the source file; sourceless loading is not supported.

The abstract methods defined by this class are to add optional bytecode file support. Not implementing these optional methods (or causing them to raise `NotImplementedError`) causes the loader to only work with source code. Implementing the methods allows the loader to work with source *and* bytecode files; it does not allow for *sourceless* loading where only bytecode is provided. Bytecode files are an optimization to speed up loading by removing the parsing step of Python's compiler, and so no bytecode-specific API is exposed.

<div class="versionchanged">

3.15 Removed the `load_module()` method.

</div>

<div class="method">

path_stats(path)

Optional abstract method which returns a `dict` containing metadata about the specified path. Supported dictionary keys are:

- `'mtime'` (mandatory): an integer or floating-point number representing the modification time of the source code;
- `'size'` (optional): the size in bytes of the source code.

Any other keys in the dictionary are ignored, to allow for future extensions. If the path cannot be handled, `OSError` is raised.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.4 Raise `OSError` instead of `NotImplementedError`.

</div>

</div>

<div class="method">

path_mtime(path)

Optional abstract method which returns the modification time for the specified path.

<div class="deprecated">

3.3 This method is deprecated in favour of `path_stats`. You don't have to implement it, but it is still available for compatibility purposes. Raise `OSError` if the path cannot be handled.

</div>

<div class="versionchanged">

3.4 Raise `OSError` instead of `NotImplementedError`.

</div>

</div>

<div class="method">

set_data(path, data)

Optional abstract method which writes the specified bytes to a file path. Any intermediate directories which do not exist are to be created automatically.

When writing to the path fails because the path is read-only (`errno.EACCES`/`PermissionError`), do not propagate the exception.

<div class="versionchanged">

3.4 No longer raises `NotImplementedError` when called.

</div>

</div>

<div class="method">

get_code(fullname)

Concrete implementation of `InspectLoader.get_code`.

</div>

<div class="method">

exec_module(module)

Concrete implementation of `Loader.exec_module`.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

get_source(fullname)

Concrete implementation of `InspectLoader.get_source`.

</div>

<div class="method">

is_package(fullname)

Concrete implementation of `InspectLoader.is_package`. A module is determined to be a package if its file path (as provided by `ExecutionLoader.get_filename`) is a file named `__init__` when the file extension is removed **and** the module name itself does not end in `__init__`.

</div>

</div>

## `!importlib.machinery` -- Importers and path hooks

<div class="module" data-synopsis="Importers and path hooks">

importlib.machinery

</div>

**Source code:** `Lib/importlib/machinery.py`

------------------------------------------------------------------------

This module contains the various objects that help `import` find and load modules.

<div class="data">

SOURCE_SUFFIXES

A list of strings representing the recognized file suffixes for source modules.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

DEBUG_BYTECODE_SUFFIXES

A list of strings representing the file suffixes for non-optimized bytecode modules.

<div class="versionadded">

3.3

</div>

<div class="deprecated">

3.5 Use `BYTECODE_SUFFIXES` instead.

</div>

</div>

<div class="data">

OPTIMIZED_BYTECODE_SUFFIXES

A list of strings representing the file suffixes for optimized bytecode modules.

<div class="versionadded">

3.3

</div>

<div class="deprecated">

3.5 Use `BYTECODE_SUFFIXES` instead.

</div>

</div>

<div class="data">

BYTECODE_SUFFIXES

A list of strings representing the recognized file suffixes for bytecode modules (including the leading dot).

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.5 The value is no longer dependent on `__debug__`.

</div>

</div>

<div class="data">

EXTENSION_SUFFIXES

A list of strings representing the recognized file suffixes for extension modules.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

all_suffixes()

Returns a combined list of strings representing all file suffixes for modules recognized by the standard import machinery. This is a helper for code which simply needs to know if a filesystem path potentially refers to a module without needing any details on the kind of module (for example, `inspect.getmodulename`).

<div class="versionadded">

3.3

</div>

</div>

<div class="BuiltinImporter">

An `importer` for built-in modules. All known built-in modules are listed in `sys.builtin_module_names`. This class implements the `importlib.abc.MetaPathFinder` and `importlib.abc.InspectLoader` ABCs.

Only class methods are defined by this class to alleviate the need for instantiation.

<div class="versionchanged">

3.5 As part of `489`, the builtin importer now implements `Loader.create_module <importlib.abc.Loader.create_module>` and `Loader.exec_module <importlib.abc.Loader.exec_module>`

</div>

</div>

<div class="FrozenImporter">

An `importer` for frozen modules. This class implements the `importlib.abc.MetaPathFinder` and `importlib.abc.InspectLoader` ABCs.

Only class methods are defined by this class to alleviate the need for instantiation.

<div class="versionchanged">

3.4 Gained `~importlib.abc.Loader.create_module` and `~importlib.abc.Loader.exec_module` methods.

</div>

</div>

<div class="WindowsRegistryFinder">

`Finder <finder>` for modules declared in the Windows registry. This class implements the `importlib.abc.MetaPathFinder` ABC.

Only class methods are defined by this class to alleviate the need for instantiation.

<div class="versionadded">

3.3

</div>

<div class="deprecated">

3.6 Use `site` configuration instead. Future versions of Python may not enable this finder by default.

</div>

</div>

<div class="PathFinder">

A `Finder <finder>` for `sys.path` and package `__path__` attributes. This class implements the `importlib.abc.MetaPathFinder` ABC.

Only class methods are defined by this class to alleviate the need for instantiation.

<div class="classmethod">

find_spec(fullname, path=None, target=None)

Class method that attempts to find a `spec <module spec>` for the module specified by *fullname* on `sys.path` or, if defined, on *path*. For each path entry that is searched, `sys.path_importer_cache` is checked. If a non-false object is found then it is used as the `path entry finder` to look for the module being searched for. If no entry is found in `sys.path_importer_cache`, then `sys.path_hooks` is searched for a finder for the path entry and, if found, is stored in `sys.path_importer_cache` along with being queried about the module. If no finder is ever found then `None` is both stored in the cache and returned.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.5 If the current working directory -- represented by an empty string -- is no longer valid then `None` is returned but no value is cached in `sys.path_importer_cache`.

</div>

</div>

<div class="classmethod">

invalidate_caches()

Calls `importlib.abc.PathEntryFinder.invalidate_caches` on all finders stored in `sys.path_importer_cache` that define the method. Otherwise entries in `sys.path_importer_cache` set to `None` are deleted.

<div class="versionchanged">

3.7 Entries of `None` in `sys.path_importer_cache` are deleted.

</div>

</div>

<div class="versionchanged">

3.4 Calls objects in `sys.path_hooks` with the current working directory for `''` (i.e. the empty string).

</div>

</div>

<div class="FileFinder(path, *loader_details)">

A concrete implementation of `importlib.abc.PathEntryFinder` which caches results from the file system.

The *path* argument is the directory for which the finder is in charge of searching.

The *loader_details* argument is a variable number of 2-item tuples each containing a loader and a sequence of file suffixes the loader recognizes. The loaders are expected to be callables which accept two arguments of the module's name and the path to the file found.

The finder will cache the directory contents as necessary, making stat calls for each module search to verify the cache is not outdated. Because cache staleness relies upon the granularity of the operating system's state information of the file system, there is a potential race condition of searching for a module, creating a new file, and then searching for the module the new file represents. If the operations happen fast enough to fit within the granularity of stat calls, then the module search will fail. To prevent this from happening, when you create a module dynamically, make sure to call `importlib.invalidate_caches`.

<div class="versionadded">

3.3

</div>

<div class="attribute">

path

The path the finder will search in.

</div>

<div class="method">

find_spec(fullname, target=None)

Attempt to find the spec to handle *fullname* within `path`.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

invalidate_caches()

Clear out the internal cache.

</div>

<div class="classmethod">

path_hook(\*loader_details)

A class method which returns a closure for use on `sys.path_hooks`. An instance of `FileFinder` is returned by the closure using the path argument given to the closure directly and *loader_details* indirectly.

If the argument to the closure is not an existing directory, `ImportError` is raised.

</div>

</div>

<div class="NamespacePath(name, path, path_finder)">

Represents a `namespace package`'s path (`module.__path__`).

When its `__path__` value is accessed it will be recomputed if necessary. This keeps it in-sync with the global state (`sys.modules`).

The *name* argument is the name of the namespace module.

The *path* argument is the initial path value.

The *path_finder* argument is the callable used to recompute the path value. The callable has the same signature as `importlib.abc.MetaPathFinder.find_spec`.

When the parent's `module.__path__` attribute is updated, the path value is recomputed.

If the parent module is missing from `sys.modules`, then `ModuleNotFoundError` will be raised.

For top-level modules, the parent module's path is `sys.path`.

> [!NOTE]
> `PathFinder.invalidate_caches` invalidates `NamespacePath`, forcing the path value to be recomputed next time it is accessed.

<div class="versionadded">

3.15

</div>

</div>

<div class="SourceFileLoader(fullname, path)">

A concrete implementation of `importlib.abc.SourceLoader` by subclassing `importlib.abc.FileLoader` and providing some concrete implementations of other methods.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.15 Removed the `load_module()` method.

</div>

<div class="attribute">

name

The name of the module that this loader will handle.

</div>

<div class="attribute">

path

The path to the source file.

</div>

<div class="method">

is_package(fullname)

Return `True` if `path` appears to be for a package.

</div>

<div class="method">

path_stats(path)

Concrete implementation of `importlib.abc.SourceLoader.path_stats`.

</div>

<div class="method">

set_data(path, data)

Concrete implementation of `importlib.abc.SourceLoader.set_data`.

</div>

</div>

<div class="SourcelessFileLoader(fullname, path)">

A concrete implementation of `importlib.abc.FileLoader` which can import bytecode files (i.e. no source code files exist).

Please note that direct use of bytecode files (and thus not source code files) inhibits your modules from being usable by all Python implementations or new versions of Python which change the bytecode format.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.15 Removed the `load_module()` method.

</div>

<div class="attribute">

name

The name of the module the loader will handle.

</div>

<div class="attribute">

path

The path to the bytecode file.

</div>

<div class="method">

is_package(fullname)

Determines if the module is a package based on `path`.

</div>

<div class="method">

get_code(fullname)

Returns the code object for `name` created from `path`.

</div>

<div class="method">

get_source(fullname)

Returns `None` as bytecode files have no source when this loader is used.

</div>

</div>

<div class="ExtensionFileLoader(fullname, path)">

A concrete implementation of `importlib.abc.ExecutionLoader` for extension modules.

The *fullname* argument specifies the name of the module the loader is to support. The *path* argument is the path to the extension module's file.

Note that, by default, importing an extension module will fail in subinterpreters if it doesn't implement multi-phase init (see `489`), even if it would otherwise import successfully.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.12 Multi-phase init is now required for use in subinterpreters.

</div>

<div class="attribute">

name

Name of the module the loader supports.

</div>

<div class="attribute">

path

Path to the extension module.

</div>

<div class="method">

create_module(spec)

Creates the module object from the given specification in accordance with `489`.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

exec_module(module)

Initializes the given module object in accordance with `489`.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

is_package(fullname)

Returns `True` if the file path points to a package's `__init__` module based on `EXTENSION_SUFFIXES`.

</div>

<div class="method">

get_code(fullname)

Returns `None` as extension modules lack a code object.

</div>

<div class="method">

get_source(fullname)

Returns `None` as extension modules do not have source code.

</div>

<div class="method">

get_filename(fullname)

Returns `path`.

<div class="versionadded">

3.4

</div>

</div>

</div>

<div class="NamespaceLoader(name, path, path_finder)">

A concrete implementation of `importlib.abc.InspectLoader` for namespace packages. This is an alias for a private class and is only made public for introspecting the `__loader__` attribute on namespace packages:

    >>> from importlib.machinery import NamespaceLoader
    >>> import my_namespace
    >>> isinstance(my_namespace.__loader__, NamespaceLoader)
    True
    >>> import importlib.abc
    >>> isinstance(my_namespace.__loader__, importlib.abc.Loader)
    True

<div class="versionadded">

3.11

</div>

</div>

<div class="ModuleSpec(name, loader, *, origin=None, loader_state=None, is_package=None)">

A specification for a module's import-system-related state. This is typically exposed as the module's `~module.__spec__` attribute. Many of these attributes are also available directly on a module: for example, `module.__spec__.origin == module.__file__`. Note, however, that while the *values* are usually equivalent, they can differ since there is no synchronization between the two objects. For example, it is possible to update the module's `~module.__file__` at runtime and this will not be automatically reflected in the module's `__spec__.origin <ModuleSpec.origin>`, and vice versa.

<div class="versionadded">

3.4

</div>

<div class="attribute">

name

The module's fully qualified name (see `module.__name__`). The `finder` should always set this attribute to a non-empty string.

</div>

<div class="attribute">

loader

The `loader` used to load the module (see `module.__loader__`). The `finder` should always set this attribute.

</div>

<div class="attribute">

origin

The location the `loader` should use to load the module (see `module.__file__`). For example, for modules loaded from a `.py` file this is the filename. The `finder` should always set this attribute to a meaningful value for the `loader` to use. In the uncommon case that there is not one (like for namespace packages), it should be set to `None`.

</div>

<div class="attribute">

submodule_search_locations

A (possibly empty) `sequence` of strings enumerating the locations in which a package's submodules will be found (see `module.__path__`). Most of the time there will only be a single directory in this list.

The `finder` should set this attribute to a sequence, even an empty one, to indicate to the import system that the module is a package. It should be set to `None` for non-package modules. It is set automatically later to a special object for namespace packages.

</div>

<div class="attribute">

loader_state

The `finder` may set this attribute to an object containing additional, module-specific data to use when loading the module. Otherwise it should be set to `None`.

</div>

<div class="attribute">

cached

The filename of a compiled version of the module's code. The `finder` should always set this attribute but it may be `None` for modules that do not need compiled code stored.

</div>

<div class="attribute">

parent

(Read-only) The fully qualified name of the package the module is in (or the empty string for a top-level module). See `module.__package__`. If the module is a package then this is the same as `name`.

</div>

<div class="attribute">

has_location

`True` if the spec's `origin` refers to a loadable location, `False` otherwise. This value impacts how `!origin` is interpreted and how the module's `~module.__file__` is populated.

</div>

</div>

<div class="AppleFrameworkLoader(name, path)">

A specialization of `importlib.machinery.ExtensionFileLoader` that is able to load extension modules in Framework format.

For compatibility with the iOS App Store, *all* binary modules in an iOS app must be dynamic libraries, contained in a framework with appropriate metadata, stored in the `Frameworks` folder of the packaged app. There can be only a single binary per framework, and there can be no executable binary material outside the Frameworks folder.

To accommodate this requirement, when running on iOS, extension module binaries are *not* packaged as `.so` files on `sys.path`, but as individual standalone frameworks. To discover those frameworks, this loader is registered against the `.fwork` file extension, with a `.fwork` file acting as a placeholder in the original location of the binary on `sys.path`. The `.fwork` file contains the path of the actual binary in the `Frameworks` folder, relative to the app bundle. To allow for resolving a framework-packaged binary back to the original location, the framework is expected to contain a `.origin` file that contains the location of the `.fwork` file, relative to the app bundle.

For example, consider the case of an import `from foo.bar import _whiz`, where `_whiz` is implemented with the binary module `sources/foo/bar/_whiz.abi3.so`, with `sources` being the location registered on `sys.path`, relative to the application bundle. This module *must* be distributed as `Frameworks/foo.bar._whiz.framework/foo.bar._whiz` (creating the framework name from the full import path of the module), with an `Info.plist` file in the `.framework` directory identifying the binary as a framework. The `foo.bar._whiz` module would be represented in the original location with a `sources/foo/bar/_whiz.abi3.fwork` marker file, containing the path `Frameworks/foo.bar._whiz/foo.bar._whiz`. The framework would also contain `Frameworks/foo.bar._whiz.framework/foo.bar._whiz.origin`, containing the path to the `.fwork` file.

When a module is loaded with this loader, the `__file__` for the module will report as the location of the `.fwork` file. This allows code to use the `__file__` of a module as an anchor for file system traversal. However, the spec origin will reference the location of the *actual* binary in the `.framework` folder.

The Xcode project building the app is responsible for converting any `.so` files from wherever they exist in the `PYTHONPATH` into frameworks in the `Frameworks` folder (including stripping extensions from the module file, the addition of framework metadata, and signing the resulting framework), and creating the `.fwork` and `.origin` files. This will usually be done with a build step in the Xcode project; see the iOS documentation for details on how to construct this build step.

<div class="versionadded">

3.13

</div>

<div class="availability">

iOS.

</div>

<div class="attribute">

name

Name of the module the loader supports.

</div>

<div class="attribute">

path

Path to the `.fwork` file for the extension module.

</div>

</div>

## `!importlib.util` -- Utility code for importers

<div class="module" data-synopsis="Utility code for importers">

importlib.util

</div>

**Source code:** `Lib/importlib/util.py`

------------------------------------------------------------------------

This module contains the various objects that help in the construction of an `importer`.

<div class="data">

MAGIC_NUMBER

The bytes which represent the bytecode version number. If you need help with loading/writing bytecode then consider `importlib.abc.SourceLoader`.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

cache_from_source(path, \*, optimization=None)

Return the `3147`/`488` path to the byte-compiled file associated with the source *path*. For example, if *path* is `/foo/bar/baz.py` the return value would be `/foo/bar/__pycache__/baz.cpython-32.pyc` for Python 3.2. The `cpython-32` string comes from the current magic tag (see `sys.implementation.cache_tag <sys.implementation>`; if it is not defined then `NotImplementedError` will be raised).

The *optimization* parameter is used to specify the optimization level of the bytecode file. An empty string represents no optimization, so `/foo/bar/baz.py` with an *optimization* of `''` will result in a bytecode path of `/foo/bar/__pycache__/baz.cpython-32.pyc`. `None` causes the interpreter's optimization level to be used. Any other value's string representation is used, so `/foo/bar/baz.py` with an *optimization* of `2` will lead to the bytecode path of `/foo/bar/__pycache__/baz.cpython-32.opt-2.pyc`. The string representation of *optimization* can only be alphanumeric, else `ValueError` is raised.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.5 The *optimization* parameter was added and the *debug_override* parameter was deprecated.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.15 The *debug_override* parameter was removed.

</div>

</div>

<div class="function">

source_from_cache(path)

Given the *path* to a `3147` file name, return the associated source code file path. For example, if *path* is `/foo/bar/__pycache__/baz.cpython-32.pyc` the returned path would be `/foo/bar/baz.py`. *path* need not exist, however if it does not conform to `3147` or `488` format, a `ValueError` is raised. If `sys.implementation.cache_tag <sys.implementation>` is not defined, `NotImplementedError` is raised.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

decode_source(source_bytes)

Decode the given bytes representing source code and return it as a string with universal newlines (as required by `importlib.abc.InspectLoader.get_source`).

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

resolve_name(name, package)

Resolve a relative module name to an absolute one.

If **name** has no leading dots, then **name** is simply returned. This allows for usage such as `importlib.util.resolve_name('sys', __spec__.parent)` without doing a check to see if the **package** argument is needed.

`ImportError` is raised if **name** is a relative module name but **package** is a false value (e.g. `None` or the empty string). `ImportError` is also raised if a relative name would escape its containing package (e.g. requesting `..bacon` from within the `spam` package).

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.9 To improve consistency with import statements, raise `ImportError` instead of `ValueError` for invalid relative import attempts.

</div>

</div>

<div class="function">

find_spec(name, package=None)

Find the `spec <module spec>` for a module, optionally relative to the specified **package** name. If the module is in `sys.modules`, then `sys.modules[name].__spec__` is returned (unless the spec would be `None` or is not set, in which case `ValueError` is raised). Otherwise a search using `sys.meta_path` is done. `None` is returned if no spec is found.

If **name** is for a submodule (contains a dot), the parent module is automatically imported.

**name** and **package** work the same as for `importlib.import_module`.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.7 Raises `ModuleNotFoundError` instead of `AttributeError` if **package** is in fact not a package (i.e. lacks a `~module.__path__` attribute).

</div>

</div>

<div class="function">

module_from_spec(spec)

Create a new module based on **spec** and `spec.loader.create_module <importlib.abc.Loader.create_module>`.

If `spec.loader.create_module <importlib.abc.Loader.create_module>` does not return `None`, then any pre-existing attributes will not be reset. Also, no `AttributeError` will be raised if triggered while accessing **spec** or setting an attribute on the module.

This function is preferred over using `types.ModuleType` to create a new module as **spec** is used to set as many import-controlled attributes on the module as possible.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

spec_from_loader(name, loader, \*, origin=None, is_package=None)

A factory function for creating a `~importlib.machinery.ModuleSpec` instance based on a loader. The parameters have the same meaning as they do for ModuleSpec. The function uses available `loader` APIs, such as `InspectLoader.is_package
<importlib.abc.InspectLoader.is_package>`, to fill in any missing information on the spec.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

spec_from_file_location(name, location, \*, loader=None, submodule_search_locations=None)

A factory function for creating a `~importlib.machinery.ModuleSpec` instance based on the path to a file. Missing information will be filled in on the spec by making use of loader APIs and by the implication that the module will be file-based.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

source_hash(source_bytes)

Return the hash of *source_bytes* as bytes. A hash-based `.pyc` file embeds the `source_hash` of the corresponding source file's contents in its header.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

<span id="incompatible_extension_module_restrictions">incompatible_extension_module_restrictions</span>(\*, disable_check)

A context manager that can temporarily skip the compatibility check for extension modules. By default the check is enabled and will fail when a single-phase init module is imported in a subinterpreter. It will also fail for a multi-phase init module that doesn't explicitly support a per-interpreter GIL, when imported in an interpreter with its own GIL.

Note that this function is meant to accommodate an unusual case; one which is likely to eventually go away. There's is a pretty good chance this is not what you were looking for.

You can get the same effect as this function by implementing the basic interface of multi-phase init (`489`) and lying about support for multiple interpreters (or per-interpreter GIL).

> [!WARNING]
> Using this function to disable the check can lead to unexpected behavior and even crashes. It should only be used during extension module development.

<div class="versionadded">

3.12

</div>

</div>

<div class="LazyLoader(loader)">

A class which postpones the execution of the loader of a module until the module has an attribute accessed.

This class **only** works with loaders that define `~importlib.abc.Loader.exec_module` as control over what module type is used for the module is required. For those same reasons, the loader's `~importlib.abc.Loader.create_module` method must return `None` or a type for which its `__class__` attribute can be mutated along with not using `slots <__slots__>`. Finally, modules which substitute the object placed into `sys.modules` will not work as there is no way to properly replace the module references throughout the interpreter safely; `ValueError` is raised if such a substitution is detected.

> [!NOTE]
> For projects where startup time is critical, this class allows for potentially minimizing the cost of loading a module if it is never used. For projects where startup time is not essential then use of this class is **heavily** discouraged due to error messages created during loading being postponed and thus occurring out of context.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.6 Began calling `~importlib.abc.Loader.create_module`, removing the compatibility warning for `importlib.machinery.BuiltinImporter` and `importlib.machinery.ExtensionFileLoader`.

</div>

<div class="classmethod">

factory(loader)

A class method which returns a callable that creates a lazy loader. This is meant to be used in situations where the loader is passed by class instead of by instance. :

    suffixes = importlib.machinery.SOURCE_SUFFIXES
    loader = importlib.machinery.SourceFileLoader
    lazy_loader = importlib.util.LazyLoader.factory(loader)
    finder = importlib.machinery.FileFinder(path, (lazy_loader, suffixes))

</div>

</div>

## Examples

### Importing programmatically

To programmatically import a module, use `importlib.import_module`. :

    import importlib

    itertools = importlib.import_module('itertools')

### Checking if a module can be imported

If you need to find out if a module can be imported without actually doing the import, then you should use `importlib.util.find_spec`.

Note that if `name` is a submodule (contains a dot), `importlib.util.find_spec` will import the parent module. :

    import importlib.util
    import sys

    # For illustrative purposes.
    name = 'itertools'

    if name in sys.modules:
        print(f"{name!r} already in sys.modules")
    elif (spec := importlib.util.find_spec(name)) is not None:
        # If you chose to perform the actual import ...
        module = importlib.util.module_from_spec(spec)
        sys.modules[name] = module
        spec.loader.exec_module(module)
        print(f"{name!r} has been imported")
    else:
        print(f"can't find the {name!r} module")

### Importing a source file directly

This recipe should be used with caution: it is an approximation of an import statement where the file path is specified directly, rather than `sys.path` being searched. Alternatives should first be considered first, such as modifying `sys.path` when a proper module is required, or using `runpy.run_path` when the global namespace resulting from running a Python file is appropriate.

To import a Python source file directly from a path, use the following recipe:

    import importlib.util
    import sys


    def import_from_path(module_name, file_path):
        spec = importlib.util.spec_from_file_location(module_name, file_path)
        module = importlib.util.module_from_spec(spec)
        sys.modules[module_name] = module
        spec.loader.exec_module(module)
        return module


    # For illustrative purposes only (use of `json` is arbitrary).
    import json
    file_path = json.__file__
    module_name = json.__name__

    # Similar outcome as `import json`.
    json = import_from_path(module_name, file_path)

### Implementing lazy imports

The example below shows how to implement lazy imports:

    >>> import importlib.util
    >>> import sys
    >>> def lazy_import(name):
    ...     spec = importlib.util.find_spec(name)
    ...     loader = importlib.util.LazyLoader(spec.loader)
    ...     spec.loader = loader
    ...     module = importlib.util.module_from_spec(spec)
    ...     sys.modules[name] = module
    ...     loader.exec_module(module)
    ...     return module
    ...
    >>> lazy_typing = lazy_import("typing")
    >>> #lazy_typing is a real module object,
    >>> #but it is not loaded in memory yet.
    >>> lazy_typing.TYPE_CHECKING
    False

### Setting up an importer

For deep customizations of import, you typically want to implement an `importer`. This means managing both the `finder` and `loader` side of things. For finders there are two flavours to choose from depending on your needs: a `meta path finder` or a `path entry finder`. The former is what you would put on `sys.meta_path` while the latter is what you create using a `path entry hook` on `sys.path_hooks` which works with `sys.path` entries to potentially create a finder. This example will show you how to register your own importers so that import will use them (for creating an importer for yourself, read the documentation for the appropriate classes defined within this package):

    import importlib.machinery
    import sys

    # For illustrative purposes only.
    SpamMetaPathFinder = importlib.machinery.PathFinder
    SpamPathEntryFinder = importlib.machinery.FileFinder
    loader_details = (importlib.machinery.SourceFileLoader,
                      importlib.machinery.SOURCE_SUFFIXES)

    # Setting up a meta path finder.
    # Make sure to put the finder in the proper location in the list in terms of
    # priority.
    sys.meta_path.append(SpamMetaPathFinder)

    # Setting up a path entry finder.
    # Make sure to put the path hook in the proper location in the list in terms
    # of priority.
    sys.path_hooks.append(SpamPathEntryFinder.path_hook(loader_details))

### Approximating `importlib.import_module`

Import itself is implemented in Python code, making it possible to expose most of the import machinery through importlib. The following helps illustrate the various APIs that importlib exposes by providing an approximate implementation of `importlib.import_module`:

    import importlib.util
    import sys

    def import_module(name, package=None):
        """An approximate implementation of import."""
        absolute_name = importlib.util.resolve_name(name, package)
        try:
            return sys.modules[absolute_name]
        except KeyError:
            pass

        path = None
        if '.' in absolute_name:
            parent_name, _, child_name = absolute_name.rpartition('.')
            parent_module = import_module(parent_name)
            path = parent_module.__spec__.submodule_search_locations
        for finder in sys.meta_path:
            spec = finder.find_spec(absolute_name, path)
            if spec is not None:
                break
        else:
            msg = f'No module named {absolute_name!r}'
            raise ModuleNotFoundError(msg, name=absolute_name)
        module = importlib.util.module_from_spec(spec)
        sys.modules[absolute_name] = module
        spec.loader.exec_module(module)
        if path is not None:
            setattr(parent_module, child_name, module)
        return module

# `!importlib.metadata` -- Accessing package metadata

<div class="module" data-synopsis="Accessing package metadata">

importlib.metadata

</div>

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.10 `importlib.metadata` is no longer provisional.

</div>

**Source code:** `Lib/importlib/metadata/__init__.py`

`importlib.metadata` is a library that provides access to the metadata of an installed [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package), such as its entry points or its top-level names ([Import Package](https://packaging.python.org/en/latest/glossary/#term-Import-Package)s, modules, if any). Built in part on Python's import system, this library provides the entry point and metadata APIs that were previously exposed by the now-removed `pkg_resources` package. Along with `importlib.resources`, it supersedes `pkg_resources`.

`importlib.metadata` operates on third-party *distribution packages* installed into Python's `site-packages` directory via tools such as `pip`. Specifically, it works with distributions with discoverable `dist-info` or `egg-info` directories, and metadata defined by the [Core metadata specifications](https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata).

> [!IMPORTANT]
> These are *not* necessarily equivalent to or correspond 1:1 with the top-level *import package* names that can be imported inside Python code. One *distribution package* can contain multiple *import packages* (and single modules), and one top-level *import package* may map to multiple *distribution packages* if it is a namespace package. You can use `packages_distributions() <package-distributions>` to get a mapping between them.

By default, distribution metadata can live on the file system or in zip archives on `sys.path`. Through an extension mechanism, the metadata can live almost anywhere.

<div class="seealso">

<https://importlib-metadata.readthedocs.io/>  
The documentation for `importlib_metadata`, which supplies a backport of `importlib.metadata`. This includes an [API reference](https://importlib-metadata.readthedocs.io/en/latest/api.html) for this module's classes and functions, as well as a [migration guide](https://importlib-metadata.readthedocs.io/en/latest/migration.html) for existing users of `pkg_resources`.

</div>

## Overview

Let's say you wanted to get the version string for a [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package) you've installed using `pip`. We start by creating a virtual environment and installing something into it:

```
$ python -m venv example
$ source example/bin/activate
(example) $ python -m pip install wheel
```

You can get the version string for `wheel` by running the following:

```
(example) $ python
>>> from importlib.metadata import version  # doctest: +SKIP
>>> version('wheel')  # doctest: +SKIP
'0.32.3'
```

You can also get a collection of entry points selectable by properties of the EntryPoint (typically 'group' or 'name'), such as `console_scripts`, `distutils.commands` and others. Each group contains a collection of `EntryPoint <entry-points>` objects.

You can get the `metadata for a distribution <metadata>`:

    >>> from importlib.metadata import metadata  # doctest: +SKIP
    >>> list(metadata('wheel'))  # doctest: +SKIP
    ['Metadata-Version', 'Name', 'Version', 'Summary', 'Home-page', 'Author', 'Author-email', 'Maintainer', 'Maintainer-email', 'License', 'Project-URL', 'Project-URL', 'Project-URL', 'Keywords', 'Platform', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Requires-Python', 'Provides-Extra', 'Requires-Dist', 'Requires-Dist']

You can also get a `distribution's version number <version>`, list its `constituent files <files>`, and get a list of the distribution's `requirements`.

<div class="exception">

PackageNotFoundError

Subclass of `ModuleNotFoundError` raised by several functions in this module when queried for a distribution package which is not installed in the current Python environment.

</div>

<div class="exception">

MetadataNotFound

Subclass of `FileNotFoundError` raised when attempting to load metadata from a distribution folder that is empty or otherwise does not contain a metadata file.

</div>

## Functional API

This package provides the following functionality via its public API.

### Entry points

<div class="function">

entry_points(\*\*select_params)

Returns a `EntryPoints` instance describing entry points for the current environment. Any given keyword parameters are passed to the `!select` method for comparison to the attributes of the individual entry point definitions.

Note: to query for entry points based on `!EntryPoint.dist` attribute, use `Distribution.entry_points` instead (as different `Distribution` instances do not currently compare equal, even if they have the same attributes)

</div>

<div class="EntryPoints">

Details of a collection of installed entry points.

Also provides a `.groups` attribute that reports all identified entry point groups, and a `.names` attribute that reports all identified entry point names.

</div>

<div class="EntryPoint">

Details of an installed entry point.

Each `!EntryPoint` instance has `.name`, `.group`, and `.value` attributes and a `.load()` method to resolve the value. There are also `.module`, `.attr`, and `.extras` attributes for getting the components of the `.value` attribute, and `.dist` for obtaining information regarding the distribution package that provides the entry point.

</div>

Query all entry points:

    >>> eps = entry_points()  # doctest: +SKIP

The `!entry_points` function returns a `!EntryPoints` object, a collection of all `!EntryPoint` objects with `names` and `groups` attributes for convenience:

    >>> sorted(eps.groups)  # doctest: +SKIP
    ['console_scripts', 'distutils.commands', 'distutils.setup_keywords', 'egg_info.writers', 'setuptools.installation']

`!EntryPoints` has a `!select` method to select entry points matching specific properties. Select entry points in the `console_scripts` group:

    >>> scripts = eps.select(group='console_scripts')  # doctest: +SKIP

Equivalently, since `!entry_points` passes keyword arguments through to select:

    >>> scripts = entry_points(group='console_scripts')  # doctest: +SKIP

Pick out a specific script named "wheel" (found in the wheel project):

    >>> 'wheel' in scripts.names  # doctest: +SKIP
    True
    >>> wheel = scripts['wheel']  # doctest: +SKIP

Equivalently, query for that entry point during selection:

    >>> (wheel,) = entry_points(group='console_scripts', name='wheel')  # doctest: +SKIP
    >>> (wheel,) = entry_points().select(group='console_scripts', name='wheel')  # doctest: +SKIP

Inspect the resolved entry point:

    >>> wheel  # doctest: +SKIP
    EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')
    >>> wheel.module  # doctest: +SKIP
    'wheel.cli'
    >>> wheel.attr  # doctest: +SKIP
    'main'
    >>> wheel.extras  # doctest: +SKIP
    []
    >>> main = wheel.load()  # doctest: +SKIP
    >>> main  # doctest: +SKIP
    <function main at 0x103528488>

The `group` and `name` are arbitrary values defined by the package author and usually a client will wish to resolve all entry points for a particular group. Read [the setuptools docs](https://setuptools.pypa.io/en/latest/userguide/entry_point.html) for more information on entry points, their definition, and usage.

<div class="versionchanged">

3.12 The "selectable" entry points were introduced in `importlib_metadata` 3.6 and Python 3.10. Prior to those changes, `entry_points` accepted no parameters and always returned a dictionary of entry points, keyed by group. With `importlib_metadata` 5.0 and Python 3.12, `entry_points` always returns an `EntryPoints` object. See `backports.entry_points_selectable` for compatibility options.

</div>

<div class="versionchanged">

3.13 `EntryPoint` objects no longer present a tuple-like interface (`~object.__getitem__`).

</div>

### Distribution metadata

<div class="function">

metadata(distribution_name)

Return the distribution metadata corresponding to the named distribution package as a `PackageMetadata` instance.

Raises `PackageNotFoundError` if the named distribution package is not installed in the current Python environment.

Raises `MetadataNotFound` if a distribution package is present but no METADATA file is present.

</div>

<div class="PackageMetadata">

A concrete implementation of the [PackageMetadata protocol](https://importlib-metadata.readthedocs.io/en/latest/api.html#importlib_metadata.PackageMetadata).

In addition to providing the defined protocol methods and attributes, subscripting the instance is equivalent to calling the `!get` method.

</div>

Every [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package) includes some metadata, which you can extract using the `!metadata` function:

    >>> wheel_metadata = metadata('wheel')  # doctest: +SKIP

The keys of the returned data structure name the metadata keywords, and the values are returned unparsed from the distribution metadata:

    >>> wheel_metadata['Requires-Python']  # doctest: +SKIP
    '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'

`PackageMetadata` also presents a `!json` attribute that returns all the metadata in a JSON-compatible form per [PEP 566](http://www.python.org/dev/peps/pep-0566/ "PEP 566"):

    >>> wheel_metadata.json['requires_python']
    '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'

The full set of available metadata is not described here. See the PyPA [Core metadata specification](https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata) for additional details.

<div class="versionchanged">

3.15 Previously and incidentally, if a METADATA file was missing from a distribution, an empty `PackageMetadata` would be returned, indistinguishable from an empty METADATA file. Now, a missing METADATA file triggers a `MetadataNotFound` exception.

</div>

<div class="versionchanged">

3.10 The `Description` is now included in the metadata when presented through the payload. Line continuation characters have been removed.

The `json` attribute was added.

</div>

### Distribution versions

<div class="function">

version(distribution_name)

Return the installed distribution package [version](https://packaging.python.org/en/latest/specifications/core-metadata/#version) for the named distribution package.

Raises `PackageNotFoundError` if the named distribution package is not installed in the current Python environment.

</div>

The `!version` function is the quickest way to get a [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package)'s version number, as a string:

    >>> version('wheel')  # doctest: +SKIP
    '0.32.3'

### Distribution files

<div class="function">

files(distribution_name)

Return the full set of files contained within the named distribution package as `PackagePath` instances.

Raises `PackageNotFoundError` if the named distribution package is not installed in the current Python environment.

Returns `None` if the distribution is found but the installation database records reporting the files associated with the distribution package are missing.

</div>

<div class="PackagePath">

A `pathlib.PurePath` derived object with additional `dist`, `size`, and `hash` properties corresponding to the distribution package's installation metadata for that file, also:

<div class="method">

locate()

If possible, return the concrete `SimplePath` allowing to access data, or raise a `NotImplementedError` otherwise.

</div>

</div>

<div class="SimplePath">

A protocol representing a minimal subset of `pathlib.Path` that allows to check if it `exists()`, to traverse using `joinpath()` and `parent`, and to retrieve data using `read_text()` and `read_bytes()`.

</div>

The `!files` function takes a [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package) name and returns all of the files installed by this distribution. For example:

    >>> util = [p for p in files('wheel') if 'util.py' in str(p)][0]  # doctest: +SKIP
    >>> util  # doctest: +SKIP
    PackagePath('wheel/util.py')
    >>> util.size  # doctest: +SKIP
    859
    >>> util.dist  # doctest: +SKIP
    <importlib.metadata._hooks.PathDistribution object at 0x101e0cef0>
    >>> util.hash  # doctest: +SKIP
    <FileHash mode: sha256 value: bYkw5oMccfazVCoYQwKkkemoVyMAFoR34mmKBx8R1NI>

Once you have the file, you can also read its contents:

    >>> print(util.read_text())  # doctest: +SKIP
    import base64
    import sys
    ...
    def as_bytes(s):
        if isinstance(s, text_type):
            return s.encode('utf-8')
        return s

You can also use the `!locate` method to get the absolute path to the file:

    >>> util.locate()  # doctest: +SKIP
    PosixPath('/home/gustav/example/lib/site-packages/wheel/util.py')

In the case where the metadata file listing files (`RECORD` or `SOURCES.txt`) is missing, `!files` will return `None`. The caller may wish to wrap calls to `!files` in [always_iterable](https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.always_iterable) or otherwise guard against this condition if the target distribution is not known to have the metadata present.

### Distribution requirements

<div class="function">

requires(distribution_name)

Return the declared dependency specifiers for the named distribution package.

Raises `PackageNotFoundError` if the named distribution package is not installed in the current Python environment.

</div>

To get the full set of requirements for a [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package), use the `!requires` function:

    >>> requires('wheel')  # doctest: +SKIP
    ["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"]

### Mapping import to distribution packages<span id="package-distributions"></span>

<div class="function">

packages_distributions()

Return a mapping from the top level module and import package names found via `sys.meta_path` to the names of the distribution packages (if any) that provide the corresponding files.

To allow for namespace packages (which may have members provided by multiple distribution packages), each top level import name maps to a list of distribution names rather than mapping directly to a single name.

</div>

A convenience method to resolve the [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package) name (or names, in the case of a namespace package) that provide each importable top-level Python module or [Import Package](https://packaging.python.org/en/latest/glossary/#term-Import-Package):

    >>> packages_distributions()
    {'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'], 'jaraco': ['jaraco.classes', 'jaraco.functools'], ...}

Some editable installs, [do not supply top-level names](https://github.com/pypa/packaging-problems/issues/609), and thus this function is not reliable with such installs.

<div class="versionadded">

3.10

</div>

## Distributions

While the module level API described above is the most common and convenient usage, all that information is accessible from the `Distribution` class. `!Distribution` is an abstract object that represents the metadata for a Python [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package). Get the concrete `!Distribution` subclass instance for an installed distribution package by calling the `distribution` function:

    >>> from importlib.metadata import distribution  # doctest: +SKIP
    >>> dist = distribution('wheel')  # doctest: +SKIP
    >>> type(dist)  # doctest: +SKIP
    <class 'importlib.metadata.PathDistribution'>

<div class="function">

distribution(distribution_name)

Return a `Distribution` instance describing the named distribution package.

Raises `PackageNotFoundError` if the named distribution package is not installed in the current Python environment.

</div>

Thus, an alternative way to get e.g. the version number is through the `Distribution.version` attribute:

    >>> dist.version  # doctest: +SKIP
    '0.32.3'

The same applies for `entry_points` and `files`.

<div class="Distribution">

Details of an installed distribution package.

Note: different `!Distribution` instances do not currently compare equal, even if they relate to the same installed distribution and accordingly have the same attributes.

<div class="staticmethod">

at(path)

</div>

<div class="classmethod">

from_name(name)

Return a `!Distribution` instance at the given path or with the given name.

</div>

<div class="classmethod">

discover(*, context=None,*\*kwargs)

Returns an iterable of `!Distribution` instances for all packages (see [distribution-discovery](#distribution-discovery)).

The optional argument *context* is a `DistributionFinder.Context` instance, used to modify the search for distributions. Alternatively, *kwargs* may contain keyword arguments for constructing a new `!DistributionFinder.Context`.

</div>

<div class="attribute" type="PackageMetadata">

metadata

Raises `MetadataNotFound` if the METADATA file is not present in the distribution.

There are all kinds of additional metadata available on `!Distribution` instances as a `PackageMetadata` instance:

    >>> dist.metadata['Requires-Python']  # doctest: +SKIP
    '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
    >>> dist.metadata['License']  # doctest: +SKIP
    'MIT'

The full set of available metadata is not described here. See the PyPA [Core metadata specification](https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata) for additional details.

</div>

<div class="attribute" type="str">

name

</div>

<div class="attribute" type="list[str]">

requires

</div>

<div class="attribute" type="str">

version

A few metadata fields are also available as shortcut properties.

<div class="versionadded">

3.10

The `name` shortcut was added.

</div>

</div>

<div class="attribute">

origin

For editable packages, an `origin` property may present `610` metadata (for non-editable packages, `origin` is `None`):

    >>> dist.origin.url
    'file:///path/to/wheel-0.32.3.editable-py3-none-any.whl'

The `origin` object follows the [Direct URL Data Structure](https://packaging.python.org/en/latest/specifications/direct-url-data-structure/).

<div class="versionadded">

3.13

</div>

</div>

<div class="attribute" type="EntryPoints">

entry_points

The entry points provided by this distribution package.

</div>

<div class="attribute" type="list[PackagePath] | None">

files

All files contained in this distribution package. Like `files`, this returns `None` if there are no records.

</div>

The following two abstract methods need to be implemented when [implementing-custom-providers](#implementing-custom-providers):

<div class="method">

locate_file(path)

Like `!PackagePath.locate`, return a `SimplePath` for the given path. Takes a `os.PathLike` or a `str`.

</div>

<div class="method">

read_text(filename)

A shortcut for `distribution.locate_file(filename).read_text()`.

</div>

</div>

## Distribution Discovery

By default, this package provides built-in support for discovery of metadata for file system and zip file [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package)s. This metadata finder search defaults to `sys.path`, but varies slightly in how it interprets those values from how other import machinery does. In particular:

- `importlib.metadata` does not honor `bytes` objects on `sys.path`.
- `importlib.metadata` will incidentally honor `pathlib.Path` objects on `sys.path` even though such values will be ignored for imports.

<div class="DistributionFinder">

A `~importlib.abc.MetaPathFinder` subclass capable of discovering installed distributions.

Custom providers should implement this interface in order to supply metadata.

> <div class="Context(**kwargs)">
>
> A `!Context` gives a custom provider a means to solicit additional details from the callers of distribution discovery functions like `distributions` or `Distribution.discover` beyond `!.name` and `!.path` when searching for distributions.
>
> For example, a provider could expose suites of packages in either a "public" or "private" `realm`. A caller of distribution discovery functions may wish to query only for distributions in a particular realm and could call `distributions(realm="private")` to signal to the custom provider to only include distributions from that realm.
>
> Each `!DistributionFinder` must expect any parameters and should attempt to honor the canonical parameters defined below when appropriate.
>
> See the section on `implementing-custom-providers` for more details.
>
> <div class="attribute">
>
> name
>
> Specific name for which a distribution finder should match.
>
> A `!.name` of `None` matches all distributions.
>
> </div>
>
> <div class="attribute">
>
> path
>
> A property providing the sequence of directory paths that a distribution finder should search.
>
> Typically refers to Python installed package paths such as "site-packages" directories and defaults to `sys.path`.
>
> </div>
>
> </div>

</div>

<div class="function">

distributions(\*\*kwargs)

Returns an iterable of `Distribution` instances for all packages.

The *kwargs* argument may contain either a keyword argument `context`, a `DistributionFinder.Context` instance, or pass keyword arguments for constructing a new `!DistributionFinder.Context`. The `!DistributionFinder.Context` is used to modify the search for distributions.

</div>

## Implementing Custom Providers

`importlib.metadata` address two API surfaces, one for *consumers* and another for *providers*. Most users are consumers, consuming metadata provided by the packages. There are other use-cases, however, where users wish to expose metadata through some other mechanism, such as alongside a custom importer. Such a use case calls for a *custom provider*.

Because [Distribution Package](https://packaging.python.org/en/latest/glossary/#term-Distribution-Package) metadata is not available through `sys.path` searches, or package loaders directly, the metadata for a distribution is found through import system `finders <finders-and-loaders>`. To find a distribution package's metadata, `importlib.metadata` queries the list of `meta path finders <meta path finder>` on `sys.meta_path`.

The implementation has hooks integrated into the `PathFinder`, serving metadata for distribution packages found on the file system.

The abstract class `importlib.abc.MetaPathFinder` defines the interface expected of finders by Python's import system. `importlib.metadata` extends this protocol by looking for an optional `find_distributions` callable on the finders from `sys.meta_path` and presents this extended interface as the `DistributionFinder` abstract base class, which defines this abstract method:

    @abc.abstractmethod
    def find_distributions(context=DistributionFinder.Context()) -> Iterable[Distribution]:
        """Return an iterable of all Distribution instances capable of
        loading the metadata for packages for the indicated ``context``.
        """

The `DistributionFinder.Context` object provides `~DistributionFinder.Context.path` and `~DistributionFinder.Context.name` properties indicating the path to search and name to match and may supply other relevant context sought by the consumer.

In practice, to support finding distribution package metadata in locations other than the file system, subclass `!Distribution` and implement the abstract methods. Then from a custom finder, return instances of this derived `!Distribution` in the `find_distributions()` method.

### Example

Imagine a custom finder that loads Python modules from a database:

    class DatabaseImporter(importlib.abc.MetaPathFinder):
        def __init__(self, db):
            self.db = db

        def find_spec(self, fullname, target=None) -> ModuleSpec:
            return self.db.spec_from_name(fullname)

    sys.meta_path.append(DatabaseImporter(connect_db(...)))

That importer now presumably provides importable modules from a database, but it provides no metadata or entry points. For this custom importer to provide metadata, it would also need to implement `DistributionFinder`:

    from importlib.metadata import DistributionFinder

    class DatabaseImporter(DistributionFinder):
        ...

        def find_distributions(self, context=DistributionFinder.Context()):
            query = dict(name=context.name) if context.name else {}
            for dist_record in self.db.query_distributions(query):
                yield DatabaseDistribution(dist_record)

In this way, `query_distributions` would return records for each distribution served by the database matching the query. For example, if `requests-1.0` is in the database, `find_distributions` would yield a `DatabaseDistribution` for `Context(name='requests')` or `Context(name=None)`.

For the sake of simplicity, this example ignores `context.path`. The `path` attribute defaults to `sys.path` and is the set of import paths to be considered in the search. A `DatabaseImporter` could potentially function without any concern for a search path. Assuming the importer does no partitioning, the "path" would be irrelevant. In order to illustrate the purpose of `path`, the example would need to illustrate a more complex `DatabaseImporter` whose behavior varied depending on `sys.path`/`PYTHONPATH`. In that case, the `find_distributions` should honor the `context.path` and only yield `Distribution`s pertinent to that path.

`DatabaseDistribution`, then, would look something like:

    class DatabaseDistribution(importlib.metadata.Distribution):
        def __init__(self, record):
            self.record = record

        def read_text(self, filename):
            """
            Read a file like "METADATA" for the current distribution.
            """
            if filename == "METADATA":
                return f"""Name: {self.record.name}
    Version: {self.record.version}
    """
            if filename == "entry_points.txt":
                return "\n".join(
                  f"""[{ep.group}]\n{ep.name}={ep.value}"""
                  for ep in self.record.entry_points)

        def locate_file(self, path):
            raise RuntimeError("This distribution has no file system")

This basic implementation should provide metadata and entry points for packages served by the `DatabaseImporter`, assuming that the `record` supplies suitable `.name`, `.version`, and `.entry_points` attributes.

The `DatabaseDistribution` may also provide other metadata files, like `RECORD` (required for `!Distribution.files`) or override the implementation of `!Distribution.files`. See the source for more inspiration.

# `!importlib.resources.abc` -- Abstract base classes for resources

<div class="module" data-synopsis="Abstract base classes for resources">

importlib.resources.abc

</div>

**Source code:** `Lib/importlib/resources/abc.py`

------------------------------------------------------------------------

<div class="versionadded">

3.11

</div>

<div class="ResourceReader">

*Superseded by TraversableResources*

An `abstract base class` to provide the ability to read *resources*.

From the perspective of this ABC, a *resource* is a binary artifact that is shipped within a package. Typically this is something like a data file that lives next to the `__init__.py` file of the package. The purpose of this class is to help abstract out the accessing of such data files so that it does not matter if the package and its data file(s) are stored e.g. in a zip file versus on the file system.

For any of methods of this class, a *resource* argument is expected to be a `path-like object` which represents conceptually just a file name. This means that no subdirectory paths should be included in the *resource* argument. This is because the location of the package the reader is for, acts as the "directory". Hence the metaphor for directories and file names is packages and resources, respectively. This is also why instances of this class are expected to directly correlate to a specific package (instead of potentially representing multiple packages or a module).

Loaders that wish to support resource reading are expected to provide a method called `get_resource_reader(fullname)` which returns an object implementing this ABC's interface. If the module specified by fullname is not a package, this method should return `None`. An object compatible with this ABC should only be returned when the specified module is a package.

<div class="deprecated">

3.12 Use `importlib.resources.abc.TraversableResources` instead.

</div>

<div class="method" data-abstractmethod="">

open_resource(resource)

Returns an opened, `file-like object` for binary reading of the *resource*.

If the resource cannot be found, `FileNotFoundError` is raised.

</div>

<div class="method" data-abstractmethod="">

resource_path(resource)

Returns the file system path to the *resource*.

If the resource does not concretely exist on the file system, raise `FileNotFoundError`.

</div>

<div class="method" data-abstractmethod="">

is_resource(path)

Returns `True` if the named *path* is considered a resource. `FileNotFoundError` is raised if *path* does not exist.

<div class="versionchanged">

3.10 The argument *name* was renamed to *path*.

</div>

</div>

<div class="method" data-abstractmethod="">

contents()

Returns an `iterable` of strings over the contents of the package. Do note that it is not required that all names returned by the iterator be actual resources, e.g. it is acceptable to return names for which `is_resource` would be false.

Allowing non-resource names to be returned is to allow for situations where how a package and its resources are stored are known a priori and the non-resource names would be useful. For instance, returning subdirectory names is allowed so that when it is known that the package and resources are stored on the file system then those subdirectory names can be used directly.

The abstract method returns an iterable of no items.

</div>

</div>

<div class="Traversable">

An object with a subset of `pathlib.Path` methods suitable for traversing directories and opening files.

For a representation of the object on the file-system, use `importlib.resources.as_file`.

<div class="attribute">

name

Abstract. The base name of this object without any parent references.

</div>

<div class="method" data-abstractmethod="">

iterdir()

Yield Traversable objects in self.

</div>

<div class="method" data-abstractmethod="">

is_dir()

Return `True` if self is a directory.

</div>

<div class="method" data-abstractmethod="">

is_file()

Return `True` if self is a file.

</div>

<div class="method" data-abstractmethod="">

joinpath(\*pathsegments)

Traverse directories according to *pathsegments* and return the result as `!Traversable`.

Each *pathsegments* argument may contain multiple names separated by forward slashes (`/`, `posixpath.sep` ). For example, the following are equivalent:

    files.joinpath('subdir', 'subsuddir', 'file.txt')
    files.joinpath('subdir/subsuddir/file.txt')

Note that some `!Traversable` implementations might not be updated to the latest version of the protocol. For compatibility with such implementations, provide a single argument without path separators to each call to `joinpath`. For example:

    files.joinpath('subdir').joinpath('subsubdir').joinpath('file.txt')

<div class="versionchanged">

3.11

`joinpath` accepts multiple *pathsegments*, and these segments may contain forward slashes as path separators. Previously, only a single *child* argument was accepted.

</div>

</div>

<div class="method" data-abstractmethod="">

\_\_truediv\_\_(child)

Return Traversable child in self. Equivalent to `joinpath(child)`.

</div>

<div class="method" data-abstractmethod="">

open(mode='r', *args,*\*kwargs)

*mode* may be 'r' or 'rb' to open as text or binary. Return a handle suitable for reading (same as `pathlib.Path.open`).

When opening as text, accepts encoding parameters such as those accepted by `io.TextIOWrapper`.

</div>

<div class="method">

read_bytes()

Read contents of self as bytes.

</div>

<div class="method">

read_text(encoding=None)

Read contents of self as text.

</div>

</div>

<div class="TraversableResources">

An abstract base class for resource readers capable of serving the `importlib.resources.files` interface. Subclasses `ResourceReader` and provides concrete implementations of the `!ResourceReader`'s abstract methods. Therefore, any loader supplying `!TraversableResources` also supplies `!ResourceReader`.

Loaders that wish to support resource reading are expected to implement this interface.

<div class="method" data-abstractmethod="">

files()

Returns a `importlib.resources.abc.Traversable` object for the loaded package.

</div>

</div>

# `!importlib.resources` -- Package resource reading, opening and access

<div class="module" data-synopsis="Package resource reading, opening, and access">

importlib.resources

</div>

**Source code:** `Lib/importlib/resources/__init__.py`

------------------------------------------------------------------------

<div class="versionadded">

3.7

</div>

This module leverages Python's import system to provide access to *resources* within *packages*.

"Resources" are file-like resources associated with a module or package in Python. The resources may be contained directly in a package, within a subdirectory contained in that package, or adjacent to modules outside a package. Resources may be text or binary. As a result, a package's Python module sources (.py), compilation artifacts (pycache), and installation artifacts (like `reserved filenames <os.path.isreserved>` in directories) are technically de-facto resources of that package. In practice, however, resources are primarily those non-Python artifacts exposed specifically by the package author.

Resources can be opened or read in either binary or text mode.

Resources are roughly akin to files inside directories, though it's important to keep in mind that this is just a metaphor. Resources and packages **do not** have to exist as physical files and directories on the file system: for example, a package and its resources can be imported from a zip file using `zipimport`.

> [!WARNING]
> `importlib.resources` follows the same security model as the built-in `open` function. Passing untrusted inputs to the functions in this module is unsafe.

> [!NOTE]
> The standalone backport of this module provides more information on [using importlib.resources](https://importlib-resources.readthedocs.io/en/latest/using.html) and [migrating from pkg_resources to importlib.resources](https://importlib-resources.readthedocs.io/en/latest/migration.html).

`Loaders <importlib.abc.Loader>` that wish to support resource reading should implement a `get_resource_reader(fullname)` method as specified by `importlib.resources.abc.ResourceReader`.

<div class="Anchor">

Represents an anchor for resources, either a `module object
<types.ModuleType>` or a module name as a string. Defined as `Union[str, ModuleType]`.

</div>

<div class="function">

files(anchor: Optional\[Anchor\] = None)

Returns a `~importlib.resources.abc.Traversable` object representing the resource container (think directory) and its resources (think files). A Traversable may contain other containers (think subdirectories).

*anchor* is an optional `Anchor`. If the anchor is a package, resources are resolved from that package. If a module, resources are resolved adjacent to that module (in the same package or the package root). If the anchor is omitted, the caller's module is used.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.12 *package* parameter was renamed to *anchor*. *package* was still accepted, but deprecated.

</div>

<div class="versionchanged">

3.15 *package* parameter was fully removed. *anchor* can now be a non-package module and if omitted will default to the caller's module. *package* is no longer accepted since Python 3.15. Consider passing the anchor positionally or using `importlib_resources >= 5.10` for a compatible interface on older Pythons.

</div>

</div>

<div class="function">

as_file(traversable)

Given a `~importlib.resources.abc.Traversable` object representing a file or directory, typically from `importlib.resources.files`, return a context manager for use in a `with` statement. The context manager provides a `pathlib.Path` object.

Exiting the context manager cleans up any temporary file or directory created when the resource was extracted from e.g. a zip file.

Use `as_file` when the Traversable methods (`read_text`, etc) are insufficient and an actual file or directory on the file system is required.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.12 Added support for *traversable* representing a directory.

</div>

</div>

## Functional API

A set of simplified, backwards-compatible helpers is available. These allow common operations in a single function call.

For all the following functions:

- *anchor* is an `~importlib.resources.Anchor`, as in `~importlib.resources.files`. Unlike in `files`, it may not be omitted.

- *path_names* are components of a resource's path name, relative to the anchor. For example, to get the text of resource named `info.txt`, use:

      importlib.resources.read_text(my_module, "info.txt")

  Like `Traversable.joinpath <importlib.resources.abc.Traversable>`, The individual components should use forward slashes (`/`) as path separators. For example, the following are equivalent:

      importlib.resources.read_binary(my_module, "pics/painting.png")
      importlib.resources.read_binary(my_module, "pics", "painting.png")

  For backward compatibility reasons, functions that read text require an explicit *encoding* argument if multiple *path_names* are given. For example, to get the text of `info/chapter1.txt`, use:

      importlib.resources.read_text(my_module, "info", "chapter1.txt",
                                    encoding='utf-8')

<div class="function">

open_binary(anchor, \*path_names)

Open the named resource for binary reading.

See `the introduction <importlib_resources_functional>` for details on *anchor* and *path_names*.

This function returns a `~typing.BinaryIO` object, that is, a binary stream open for reading.

This function is roughly equivalent to:

    files(anchor).joinpath(*path_names).open('rb')

<div class="versionchanged">

3.13 Multiple *path_names* are accepted.

</div>

</div>

<div class="function">

open_text(anchor, \*path_names, encoding='utf-8', errors='strict')

Open the named resource for text reading. By default, the contents are read as strict UTF-8.

See `the introduction <importlib_resources_functional>` for details on *anchor* and *path_names*. *encoding* and *errors* have the same meaning as in built-in `open`.

For backward compatibility reasons, the *encoding* argument must be given explicitly if there are multiple *path_names*. This limitation is scheduled to be removed in Python 3.15.

This function returns a `~typing.TextIO` object, that is, a text stream open for reading.

This function is roughly equivalent to:

    files(anchor).joinpath(*path_names).open('r', encoding=encoding)

<div class="versionchanged">

3.13 Multiple *path_names* are accepted. *encoding* and *errors* must be given as keyword arguments.

</div>

</div>

<div class="function">

read_binary(anchor, \*path_names)

Read and return the contents of the named resource as `bytes`.

See `the introduction <importlib_resources_functional>` for details on *anchor* and *path_names*.

This function is roughly equivalent to:

    files(anchor).joinpath(*path_names).read_bytes()

<div class="versionchanged">

3.13 Multiple *path_names* are accepted.

</div>

</div>

<div class="function">

read_text(anchor, \*path_names, encoding='utf-8', errors='strict')

Read and return the contents of the named resource as `str`. By default, the contents are read as strict UTF-8.

See `the introduction <importlib_resources_functional>` for details on *anchor* and *path_names*. *encoding* and *errors* have the same meaning as in built-in `open`.

For backward compatibility reasons, the *encoding* argument must be given explicitly if there are multiple *path_names*. This limitation is scheduled to be removed in Python 3.15.

This function is roughly equivalent to:

    files(anchor).joinpath(*path_names).read_text(encoding=encoding)

<div class="versionchanged">

3.13 Multiple *path_names* are accepted. *encoding* and *errors* must be given as keyword arguments.

</div>

</div>

<div class="function">

path(anchor, \*path_names)

Provides the path to the *resource* as an actual file system path. This function returns a context manager for use in a `with` statement. The context manager provides a `pathlib.Path` object.

Exiting the context manager cleans up any temporary files created, e.g. when the resource needs to be extracted from a zip file.

For example, the `~pathlib.Path.stat` method requires an actual file system path; it can be used like this:

    with importlib.resources.path(anchor, "resource.txt") as fspath:
        result = fspath.stat()

See `the introduction <importlib_resources_functional>` for details on *anchor* and *path_names*.

This function is roughly equivalent to:

    as_file(files(anchor).joinpath(*path_names))

<div class="versionchanged">

3.13 Multiple *path_names* are accepted.

</div>

</div>

<div class="function">

is_resource(anchor, \*path_names)

Return `True` if the named resource exists, otherwise `False`. This function does not consider directories to be resources.

See `the introduction <importlib_resources_functional>` for details on *anchor* and *path_names*.

This function is roughly equivalent to:

    files(anchor).joinpath(*path_names).is_file()

<div class="versionchanged">

3.13 Multiple *path_names* are accepted.

</div>

</div>

<div class="function">

contents(anchor, \*path_names)

Return an iterable over the named items within the package or path. The iterable returns names of resources (e.g. files) and non-resources (e.g. directories) as `str`. The iterable does not recurse into subdirectories.

See `the introduction <importlib_resources_functional>` for details on *anchor* and *path_names*.

This function is roughly equivalent to:

    for resource in files(anchor).joinpath(*path_names).iterdir():
        yield resource.name

<div class="deprecated">

3.11 Prefer `iterdir()` as above, which offers more control over the results and richer functionality.

</div>

</div>

# The Python standard library

This library reference manual describes the standard library distributed with Python. It also describes some of the optional components that are commonly included in Python distributions.

Elsewhere, `reference-index` describes the exact syntax and semantics of the Python language, and `builtins-index` describes the built-in functions.

Python's standard library is extensive, offering a wide range of facilities as indicated by the long table of contents listed below. The library contains built-in modules (written in C) that provide access to system functionality such as file I/O that would otherwise be inaccessible to Python programmers, as well as modules written in Python that provide standardized solutions for many problems that occur in everyday programming. Some of these modules are explicitly designed to encourage and enhance the portability of Python programs by abstracting away platform-specifics into platform-neutral APIs.

The Python installers for the Windows platform usually include the entire standard library and often also include many additional components. For Unix-like operating systems Python is normally provided as a collection of packages, so it may be necessary to use the packaging tools provided with the operating system to obtain some or all of the optional components.

In addition to the standard library, there is an active collection of hundreds of thousands of components (from individual programs and modules to packages and entire application development frameworks), available from the [Python Package Index](https://pypi.org).

<div class="toctree" data-maxdepth="2">

intro.rst

text.rst binary.rst datatypes.rst numeric.rst functional.rst filesys.rst persistence.rst archiving.rst fileformats.rst crypto.rst allos.rst cmdlinelibs.rst concurrency.rst ipc.rst netdata.rst markup.rst internet.rst mm.rst i18n.rst tk.rst development.rst debug.rst distribution.rst python.rst custominterp.rst modules.rst language.rst windows.rst unix.rst cmdline.rst superseded.rst removed.rst security_warnings.rst

</div>

# `!inspect` --- Inspect live objects

<div class="testsetup">

- 

import inspect from inspect import \*

</div>

<div class="module" data-synopsis="Extract information and source code from live objects.">

inspect

</div>

**Source code:** `Lib/inspect.py`

------------------------------------------------------------------------

The `!inspect` module provides several useful functions to help get information about live objects such as modules, classes, methods, functions, tracebacks, frame objects, and code objects. For example, it can help you examine the contents of a class, retrieve the source code of a method, extract and format the argument list for a function, or get all the information you need to display a detailed traceback.

There are four main kinds of services provided by this module: type checking, getting source code, inspecting classes and functions, and examining the interpreter stack.

## Types and members

The `getmembers` function retrieves the members of an object such as a class or module. The functions whose names begin with "is" are mainly provided as convenient choices for the second argument to `getmembers`. They also help you determine when you can expect to find the following special attributes (see `import-mod-attrs` for module attributes):

| Type | Attribute | Description |
|----|----|----|
| class | \_\_doc\_\_ | documentation string |
|  | \_\_name\_\_ | name with which this class was defined |
|  | \_\_qualname\_\_ | qualified name |
|  | \_\_module\_\_ | name of module in which this class was defined |
|  | \_\_type_params\_\_ | A tuple containing the `type parameters
<type-params>` of a generic class |
| method | \_\_doc\_\_ | documentation string |
|  | \_\_name\_\_ | name with which this method was defined |
|  | \_\_qualname\_\_ | qualified name |
|  | \_\_func\_\_ | function object containing implementation of method |
|  | \_\_self\_\_ | instance to which this method is bound, or `None` |
|  | \_\_module\_\_ | name of module in which this method was defined |
| function | \_\_doc\_\_ | documentation string |
|  | \_\_name\_\_ | name with which this function was defined |
|  | \_\_qualname\_\_ | qualified name |
|  | \_\_code\_\_ | code object containing compiled function `bytecode` |
|  | \_\_defaults\_\_ | tuple of any default values for positional or keyword parameters |
|  | \_\_kwdefaults\_\_ | mapping of any default values for keyword-only parameters |
|  | \_\_globals\_\_ | global namespace in which this function was defined |
|  | \_\_builtins\_\_ | builtins namespace |
|  | \_\_annotations\_\_ | mapping of parameters names to annotations; `"return"` key is reserved for return annotations. |
|  | \_\_type_params\_\_ | A tuple containing the `type parameters
<type-params>` of a generic function |
|  | \_\_module\_\_ | name of module in which this function was defined |
| traceback | tb_frame | frame object at this level |
|  | tb_lasti | index of last attempted instruction in bytecode |
|  | tb_lineno | current line number in Python source code |
|  | tb_next | next inner traceback object (called by this level) |
| frame | f_back | next outer frame object (this frame's caller) |
|  | f_builtins | builtins namespace seen by this frame |
|  | f_code | code object being executed in this frame |
|  | f_globals | global namespace seen by this frame |
|  | f_lasti | index of last attempted instruction in bytecode |
|  | f_lineno | current line number in Python source code |
|  | f_locals | local namespace seen by this frame |
|  | f_generator | returns the generator or coroutine object that owns this frame, or `None` if the frame is of a regular function |
|  | f_trace | tracing function for this frame, or `None` |
|  | f_trace_lines | indicate whether a tracing event is triggered for each source source line |
|  | f_trace_opcodes | indicate whether per-opcode events are requested |
|  | clear() | used to clear all references to local variables |
| code | co_argcount | number of arguments (not including keyword only arguments, \* or \*\* args) |
|  | co_code | string of raw compiled bytecode |
|  | co_cellvars | tuple of names of cell variables (referenced by containing scopes) |
|  | co_consts | tuple of constants used in the bytecode |
|  | co_filename | name of file in which this code object was created |
|  | co_firstlineno | number of first line in Python source code |
|  | co_flags | bitmap of `CO_*` flags, read more `here
<inspect-module-co-flags>` |
|  | co_freevars | tuple of names of free variables (referenced via a function's closure) |
|  | co_posonlyargcount | number of positional only arguments |
|  | co_kwonlyargcount | number of keyword only arguments (not including \*\* arg) |
|  | co_name | name with which this code object was defined |
|  | co_qualname | fully qualified name with which this code object was defined |
|  | co_names | tuple of names other than arguments and function locals |
|  | co_nlocals | number of local variables |
|  | co_stacksize | virtual machine stack space required |
|  | co_varnames | tuple of names of arguments and local variables |
|  | co_lines() | returns an iterator that yields successive bytecode ranges |
|  | co_positions() | returns an iterator of source code positions for each bytecode instruction |
|  | replace() | returns a copy of the code object with new values |
| generator | \_\_name\_\_ | name |
|  | \_\_qualname\_\_ | qualified name |
|  | gi_frame | frame |
|  | gi_running | is the generator running? |
|  | gi_suspended | is the generator suspended? |
|  | gi_code | code |
|  | gi_yieldfrom | object being iterated by `yield from`, or `None` |
|  | gi_state | state of the generator, one of `GEN_CREATED`, `GEN_RUNNING`, `GEN_SUSPENDED`, or `GEN_CLOSED` |
| async generator | \_\_name\_\_ | name |
|  | \_\_qualname\_\_ | qualified name |
|  | ag_await | object being awaited on, or `None` |
|  | ag_frame | frame |
|  | ag_running | is the generator running? |
|  | ag_suspended | is the generator suspended? |
|  | ag_code | code |
|  | ag_state | state of the async generator, one of `AGEN_CREATED`, `AGEN_RUNNING`, `AGEN_SUSPENDED`, or `AGEN_CLOSED` |
| coroutine | \_\_name\_\_ | name |
|  | \_\_qualname\_\_ | qualified name |
|  | cr_await | object being awaited on, or `None` |
|  | cr_frame | frame |
|  | cr_running | is the coroutine running? |
|  | cr_suspended | is the coroutine suspended? |
|  | cr_code | code |
|  | cr_origin | where coroutine was created, or `None`. See `sys.set_coroutine_origin_tracking_depth` |
|  | cr_state | state of the coroutine, one of `CORO_CREATED`, `CORO_RUNNING`, `CORO_SUSPENDED`, or `CORO_CLOSED` |
| builtin | \_\_doc\_\_ | documentation string |
|  | \_\_name\_\_ | original name of this function or method |
|  | \_\_qualname\_\_ | qualified name |
|  | \_\_self\_\_ | instance to which a method is bound, or module object a function was defined in, or `None` |

<div class="versionchanged">

3.5

Add `__qualname__` and `gi_yieldfrom` attributes to generators.

The `__name__` attribute of generators is now set from the function name, instead of the code name, and it can now be modified.

</div>

<div class="versionchanged">

3.7

Add `cr_origin` attribute to coroutines.

</div>

<div class="versionchanged">

3.10

Add `__builtins__` attribute to functions.

</div>

<div class="versionchanged">

3.11

Add `gi_suspended` attribute to generators.

</div>

<div class="versionchanged">

3.11

Add `cr_suspended` attribute to coroutines.

</div>

<div class="versionchanged">

3.12

Add `ag_suspended` attribute to async generators.

</div>

<div class="versionchanged">

3.14

Add `f_generator` attribute to frames.

</div>

<div class="versionchanged">

3.15

Add `gi_state` attribute to generators, `cr_state` attribute to coroutines, and `ag_state` attribute to async generators.

</div>

<div class="function">

getmembers(object\[, predicate\])

Return all the members of an object in a list of `(name, value)` pairs sorted by name. If the optional *predicate* argument—which will be called with the `value` object of each member—is supplied, only members for which the predicate returns a true value are included.

> [!NOTE]
> `getmembers` will only return class attributes defined in the metaclass when the argument is a class and those attributes have been listed in the metaclass' custom `~object.__dir__`.

</div>

<div class="function">

getmembers_static(object\[, predicate\])

Return all the members of an object in a list of `(name, value)` pairs sorted by name without triggering dynamic lookup via the descriptor protocol, \_\_getattr\_\_ or \_\_getattribute\_\_. Optionally, only return members that satisfy a given predicate.

> [!NOTE]
> `getmembers_static` may not be able to retrieve all members that getmembers can fetch (like dynamically created attributes) and may find members that getmembers can't (like descriptors that raise AttributeError). It can also return descriptor objects instead of instance members in some cases.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

getmodulename(path)

Return the name of the module named by the file *path*, without including the names of enclosing packages. The file extension is checked against all of the entries in `importlib.machinery.all_suffixes`. If it matches, the final path component is returned with the extension removed. Otherwise, `None` is returned.

Note that this function *only* returns a meaningful name for actual Python modules - paths that potentially refer to Python packages will still return `None`.

<div class="versionchanged">

3.3 The function is based directly on `importlib`.

</div>

</div>

<div class="function">

ismodule(object)

Return `True` if the object is a module.

</div>

<div class="function">

isclass(object)

Return `True` if the object is a class, whether built-in or created in Python code.

This function returns `False` for `generic aliases <types-genericalias>` of classes, such as `list[int]`.

</div>

<div class="function">

ismethod(object)

Return `True` if the object is a bound method written in Python.

> [!NOTE]
> For example, given this class:
>
>     >>> class Greeter:
>     ...     def say_hello(self):
>     ...         print('hello!')
>
> A bound method (also known as an *instance method*) is created when accessing `say_hello` (a `function` defined in the `Greeter` namespace) through an instance of the `Greeter` class:
>
>     >>> instance = Greeter()
>
>     >>> instance.say_hello
>     <bound method Greeter.say_hello of <__main__.Greeter object ...>>
>     >>> ismethod(instance.say_hello)
>     True
>     >>> isfunction(instance.say_hello)
>     False
>
> Accessing `say_hello` through the `Greeter` class will return the function itself. For this function, `ismethod` will return `False`, but `isfunction` will return `True`:
>
>     >>> Greeter.say_hello
>     <function Greeter.say_hello at 0x7f7503854a90>
>     >>> ismethod(Greeter.say_hello)
>     False
>     >>> isfunction(Greeter.say_hello)
>     True
>
> See `typesmethods` for details.

</div>

<div class="function">

isfunction(object)

Return `True` if the object is a Python function, which includes functions created by a `lambda` expression.

See the note for `~inspect.ismethod` for an example.

</div>

<div class="function">

ispackage(object)

Return `True` if the object is a `package`.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

isgeneratorfunction(object)

Return `True` if the object is a Python generator function.

It also returns `True` for bound methods created from Python generator functions (see `typesmethods` for more information).

<div class="versionchanged">

3.8 Functions wrapped in `functools.partial` now return `True` if the wrapped function is a Python generator function.

</div>

<div class="versionchanged">

3.10.6 `Duck-typed <duck-typing>` function-like objects now return `True` if their code object has the `CO_GENERATOR` flag.

</div>

<div class="versionchanged">

3.13 Functions wrapped in `functools.partialmethod` now return `True` if the wrapped function is a Python generator function.

</div>

</div>

<div class="function">

isgenerator(object)

Return `True` if the object is a generator.

</div>

<div class="function">

iscoroutinefunction(object)

Return `True` if the object is a `coroutine function` (a function defined with an `async def` syntax), a `functools.partial` wrapping a `coroutine function`, or a sync function marked with `markcoroutinefunction`.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.8 Functions wrapped in `functools.partial` now return `True` if the wrapped function is a `coroutine function`.

</div>

<div class="versionchanged">

3.10.6 `Duck-typed <duck-typing>` function-like objects now return `True` if their code object has the `CO_COROUTINE` flag.

</div>

<div class="versionchanged">

3.12 Sync functions marked with `markcoroutinefunction` now return `True`.

</div>

<div class="versionchanged">

3.13 Functions wrapped in `functools.partialmethod` now return `True` if the wrapped function is a `coroutine function`.

</div>

</div>

<div class="function">

markcoroutinefunction(func)

Decorator to mark a callable as a `coroutine function` if it would not otherwise be detected by `iscoroutinefunction`.

This may be of use for sync functions that return a `coroutine`, if the function is passed to an API that requires `iscoroutinefunction`.

When possible, using an `async def` function is preferred. Also acceptable is calling the function and testing the return with `iscoroutine`.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

iscoroutine(object)

Return `True` if the object is a `coroutine` created by an `async def` function.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

isawaitable(object)

Return `True` if the object can be used in `await` expression.

Can also be used to distinguish generator-based coroutines from regular generators:

<div class="testcode">

import types

def gen():  
yield

@types.coroutine def gen_coro(): yield

assert not isawaitable(gen()) assert isawaitable(gen_coro())

</div>

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

isasyncgenfunction(object)

Return `True` if the object is an `asynchronous generator` function, for example:

<div class="doctest">

\>\>\> async def agen(): ... yield 1 ... \>\>\> inspect.isasyncgenfunction(agen) True

</div>

<div class="versionadded">

3.6

</div>

<div class="versionchanged">

3.8 Functions wrapped in `functools.partial` now return `True` if the wrapped function is an `asynchronous generator` function.

</div>

<div class="versionchanged">

3.10.6 `Duck-typed <duck-typing>` function-like objects now return `True` if their code object has the `CO_ASYNC_GENERATOR` flag.

</div>

<div class="versionchanged">

3.13 Functions wrapped in `functools.partialmethod` now return `True` if the wrapped function is a `asynchronous generator` function.

</div>

</div>

<div class="function">

isasyncgen(object)

Return `True` if the object is an `asynchronous generator iterator` created by an `asynchronous generator` function.

<div class="versionadded">

3.6

</div>

</div>

<div class="function">

istraceback(object)

Return `True` if the object is a traceback.

</div>

<div class="function">

isframe(object)

Return `True` if the object is a frame.

</div>

<div class="function">

iscode(object)

Return `True` if the object is a code.

</div>

<div class="function">

isbuiltin(object)

Return `True` if the object is a built-in function or a bound built-in method.

</div>

<div class="function">

ismethodwrapper(object)

Return `True` if the type of object is a `~types.MethodWrapperType`.

These are instances of `~types.MethodWrapperType`, such as `~object.__str__`, `~object.__eq__` and `~object.__repr__`.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

isroutine(object)

Return `True` if the object is a user-defined or built-in function or method.

</div>

<div class="function">

isabstract(object)

Return `True` if the object is an abstract base class.

</div>

<div class="function">

ismethoddescriptor(object)

Return `True` if the object is a method descriptor, but not if `isclass`, `ismethod` or `isfunction` is true.

This, for example, is true of `int.__add__`. An object passing this test has a `~object.__get__` method, but not a `~object.__set__` method or a `~object.__delete__` method. Beyond that, the set of attributes varies. A `~definition.__name__` attribute is usually sensible, and `~definition.__doc__` often is.

Method descriptors that also pass any of the other tests (`!isclass`, `!ismethod` or `!isfunction`) make this function return `False`, simply because those other tests promise more -- you can, for example, count on having the `~method.__func__` attribute when an object passes `ismethod`.

<div class="versionchanged">

3.13 This function no longer incorrectly reports objects with `~object.__get__` and `~object.__delete__`, but not `~object.__set__`, as being method descriptors (such objects are data descriptors, not method descriptors).

</div>

</div>

<div class="function">

isdatadescriptor(object)

Return `True` if the object is a data descriptor, but not if `isclass`, `ismethod` or `isfunction` is true.

Data descriptors always have a `~object.__set__` method and/or a `~object.__delete__` method. Optionally, they may also have a `~object.__get__` method.

Examples of data descriptors are `properties <property>`, getsets and member descriptors. Note that for the latter two (defined only in C extension modules), more specific tests are available: `isgetsetdescriptor` and `ismemberdescriptor`, respectively.

While data descriptors may also have `~definition.__name__` and `!__doc__` attributes (as properties, getsets and member descriptors do), this is not necessarily the case in general.

<div class="versionchanged">

3.8 This function now reports objects with only a `~object.__set__` method as being data descriptors (the presence of `~object.__get__` is no longer required for that). Moreover, objects with `~object.__delete__`, but not `~object.__set__`, are now properly recognized as data descriptors as well, which was not the case previously.

</div>

</div>

<div class="function">

isgetsetdescriptor(object)

Return `True` if the object is a getset descriptor.

<div class="impl-detail">

getsets are attributes defined in extension modules via `PyGetSetDef` structures. For Python implementations without such types, this method will always return `False`.

</div>

</div>

<div class="function">

ismemberdescriptor(object)

Return `True` if the object is a member descriptor.

<div class="impl-detail">

Member descriptors are attributes defined in extension modules via `PyMemberDef` structures. For Python implementations without such types, this method will always return `False`.

</div>

</div>

## Retrieving source code

<div class="function">

getdoc(object, \*, inherit_class_doc=True, fallback_to_class_doc=True, dedent=True)

Get the documentation string for an object, cleaned up with `cleandoc` (with the same meaning of *dedent*). If the documentation string for an object is not provided:

- if the object is a class and *inherit_class_doc* is true (by default), retrieve the documentation string from the inheritance hierarchy;
- if the object is a method, a property or a descriptor, retrieve the documentation string from the inheritance hierarchy;
- otherwise, if *fallback_to_class_doc* is true (by default), retrieve the documentation string from the class of the object.

Return `None` if the documentation string is invalid or missing.

<div class="versionchanged">

3.5 Documentation strings are now inherited if not overridden.

</div>

<div class="versionchanged">

3.15 Added parameters *inherit_class_doc* and *fallback_to_class_doc*.

Documentation strings on `~functools.cached_property` objects are now inherited if not overridden.

</div>

<div class="versionchanged">

3.16 Added the *dedent* parameter.

</div>

</div>

<div class="function">

getcomments(object)

Return in a single string any lines of comments immediately preceding the object's source code (for a class, function, or method), or at the top of the Python source file (if the object is a module). If the object's source code is unavailable, return `None`. This could happen if the object has been defined in C or the interactive shell.

</div>

<div class="function">

getfile(object)

Return the name of the (text or binary) file in which an object was defined. An `OSError` is raised if the source code cannot be retrieved. This will fail with a `TypeError` if the object is a built-in module, class, or function.

</div>

<div class="function">

getmodule(object)

Try to guess which module an object was defined in. Return `None` if the module cannot be determined.

</div>

<div class="function">

getsourcefile(object)

Return the name of the Python source file in which an object was defined or `None` if no way can be identified to get the source. An `OSError` is raised if the source code cannot be retrieved. This will fail with a `TypeError` if the object is a built-in module, class, or function.

</div>

<div class="function">

getsourcelines(object)

Return a list of source lines and starting line number for an object. The argument may be a module, class, method, function, traceback, frame, or code object. The source code is returned as a list of the lines corresponding to the object and the line number indicates where in the original source file the first line of code was found. An `OSError` is raised if the source code cannot be retrieved. A `TypeError` is raised if the object is a built-in module, class, or function.

<div class="versionchanged">

3.3 `OSError` is raised instead of `IOError`, now an alias of the former.

</div>

</div>

<div class="function">

getsource(object)

Return the text of the source code for an object. The argument may be a module, class, method, function, traceback, frame, or code object. The source code is returned as a single string. An `OSError` is raised if the source code cannot be retrieved. A `TypeError` is raised if the object is a built-in module, class, or function.

<div class="versionchanged">

3.3 `OSError` is raised instead of `IOError`, now an alias of the former.

</div>

</div>

<div class="function">

cleandoc(doc, \*, dedent=True)

Clean up indentation from docstrings that are indented to line up with blocks of code.

All leading whitespace is removed from the first line. Any leading whitespace that can be uniformly removed from the second line onwards is removed, unless *dedent* is false. Empty lines at the beginning and end are subsequently removed. Also, all tabs are expanded to spaces.

Since Python 3.13 the compiler removes the indentation of docstrings, so *dedent* only affects documentation strings which are not written as docstrings in the source code, like those generated by Argument Clinic, where the indentation is meaningful.

<div class="versionchanged">

3.16 Added the *dedent* parameter.

</div>

</div>

## Introspecting callables with the Signature object

<div class="versionadded">

3.3

</div>

The `Signature` object represents the call signature of a callable object and its return annotation. To retrieve a `!Signature` object, use the `!signature` function.

<div class="function">

signature(callable, \*, follow_wrapped=True, globals=None, locals=None, eval_str=False, annotation_format=Format.VALUE)

Return a `Signature` object for the given *callable*:

<div class="doctest">

\>\>\> from inspect import signature \>\>\> def foo(a, *, b:int,*\*kwargs): ... pass

\>\>\> sig = signature(foo)

\>\>\> str(sig) '(a, *, b: int,*\*kwargs)'

\>\>\> str(sig.parameters\['b'\]) 'b: int'

\>\>\> sig.parameters\['b'\].annotation \<class 'int'\>

</div>

Accepts a wide range of Python callables, from plain functions and classes to `functools.partial` objects.

If some of the annotations are strings (e.g., because `from __future__ import annotations` was used), `signature` will attempt to automatically un-stringize the annotations using `annotationlib.get_annotations`. The *globals*, *locals*, and *eval_str* parameters are passed into `!annotationlib.get_annotations` when resolving the annotations; see the documentation for `!annotationlib.get_annotations` for instructions on how to use these parameters. A member of the `annotationlib.Format` enum can be passed to the *annotation_format* parameter to control the format of the returned annotations. For example, use `annotation_format=annotationlib.Format.STRING` to return annotations in string format.

Raises `ValueError` if no signature can be provided, and `TypeError` if that type of object is not supported. Also, if the annotations are stringized, and *eval_str* is not false, the `eval()` call(s) to un-stringize the annotations in `annotationlib.get_annotations` could potentially raise any kind of exception.

A slash (/) in the signature of a function denotes that the parameters prior to it are positional-only. For more info, see `the FAQ entry on positional-only parameters <faq-positional-only-arguments>`.

<div class="versionchanged">

3.5 The *follow_wrapped* parameter was added. Pass `False` to get a signature of *callable* specifically (`callable.__wrapped__` will not be used to unwrap decorated callables.)

</div>

<div class="versionchanged">

3.10 The *globals*, *locals*, and *eval_str* parameters were added.

</div>

<div class="versionchanged">

3.14 The *annotation_format* parameter was added.

</div>

> [!NOTE]
> Some callables may not be introspectable in certain implementations of Python. For example, in CPython, some built-in functions defined in C provide no metadata about their arguments.

<div class="impl-detail">

If the passed object has a `!__signature__` attribute, we may use it to create the signature. The exact semantics are an implementation detail and are subject to unannounced changes. Consult the source code for current semantics.

</div>

</div>

<div class="Signature(parameters=None, *, return_annotation=Signature.empty)">

A `!Signature` object represents the call signature of a function and its return annotation. For each parameter accepted by the function it stores a `Parameter` object in its `parameters` collection.

The optional *parameters* argument is a sequence of `Parameter` objects, which is validated to check that there are no parameters with duplicate names, and that the parameters are in the right order, i.e. positional-only first, then positional-or-keyword, and that parameters with defaults follow parameters without defaults.

The optional *return_annotation* argument can be an arbitrary Python object. It represents the "return" annotation of the callable.

`!Signature` objects are *immutable*. Use `Signature.replace` or `copy.replace` to make a modified copy.

<div class="versionchanged">

3.5 `!Signature` objects are now picklable and `hashable`.

</div>

<div class="attribute">

Signature.empty

A special class-level marker to specify absence of a return annotation.

</div>

<div class="attribute">

Signature.parameters

An ordered mapping of parameters' names to the corresponding `Parameter` objects. Parameters appear in strict definition order, including keyword-only parameters.

<div class="versionchanged">

3.7 Python only explicitly guaranteed that it preserved the declaration order of keyword-only parameters as of version 3.7, although in practice this order had always been preserved in Python 3.

</div>

</div>

<div class="attribute">

Signature.return_annotation

The "return" annotation for the callable. If the callable has no "return" annotation, this attribute is set to `Signature.empty`.

</div>

<div class="method">

Signature.bind(*args,*\*kwargs)

Create a mapping from positional and keyword arguments to parameters. Returns `BoundArguments` if `*args` and `**kwargs` match the signature, or raises a `TypeError`.

</div>

<div class="method">

Signature.bind_partial(*args,*\*kwargs)

Works the same way as `Signature.bind`, but allows the omission of some required arguments (mimics `functools.partial` behavior.) Returns `BoundArguments`, or raises a `TypeError` if the passed arguments do not match the signature.

</div>

<div class="method">

Signature.replace(\*\[, parameters\]\[, return_annotation\])

Create a new `Signature` instance based on the instance `replace` was invoked on. It is possible to pass different *parameters* and/or *return_annotation* to override the corresponding properties of the base signature. To remove `return_annotation` from the copied `!Signature`, pass in `Signature.empty`.

<div class="doctest">

\>\>\> def test(a, b): ... pass ... \>\>\> sig = signature(test) \>\>\> new_sig = sig.replace(return_annotation="new return anno") \>\>\> str(new_sig) "(a, b) -\> 'new return anno'"

</div>

`Signature` objects are also supported by the generic function `copy.replace`.

</div>

<div class="method">

format(\*, max_width=None, quote_annotation_strings=True)

Create a string representation of the `Signature` object.

If *max_width* is passed, the method will attempt to fit the signature into lines of at most *max_width* characters. If the signature is longer than *max_width*, all parameters will be on separate lines.

If *quote_annotation_strings* is False, `annotations <annotation>` in the signature are displayed without opening and closing quotation marks if they are strings. This is useful if the signature was created with the `~annotationlib.Format.STRING` format or if `from __future__ import annotations` was used.

<div class="versionadded">

3.13

</div>

<div class="versionchanged">

3.14 The *unquote_annotations* parameter was added.

</div>

</div>

<div class="classmethod">

Signature.from_callable(obj, \*, follow_wrapped=True, globals=None, locals=None, eval_str=False)

Return a `Signature` (or its subclass) object for a given callable *obj*.

This method simplifies subclassing of `Signature`:

<div class="testcode">

class MySignature(Signature):  
pass

sig = MySignature.from_callable(sum) assert isinstance(sig, MySignature)

</div>

Its behavior is otherwise identical to that of `signature`.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.10 The *globals*, *locals*, and *eval_str* parameters were added.

</div>

</div>

</div>

<div class="Parameter(name, kind, *, default=Parameter.empty, annotation=Parameter.empty)">

`!Parameter` objects are *immutable*. Instead of modifying a `!Parameter` object, you can use `Parameter.replace` or `copy.replace` to create a modified copy.

<div class="versionchanged">

3.5 Parameter objects are now picklable and `hashable`.

</div>

<div class="attribute">

Parameter.empty

A special class-level marker to specify absence of default values and annotations.

</div>

<div class="attribute">

Parameter.name

The name of the parameter as a string. The name must be a valid Python identifier.

<div class="impl-detail">

CPython generates implicit parameter names of the form `.0` on the code objects used to implement comprehensions and generator expressions.

<div class="versionchanged">

3.6 These parameter names are now exposed by this module as names like `implicit0`.

</div>

</div>

</div>

<div class="attribute">

Parameter.default

The default value for the parameter. If the parameter has no default value, this attribute is set to `Parameter.empty`.

</div>

<div class="attribute">

Parameter.annotation

The annotation for the parameter. If the parameter has no annotation, this attribute is set to `Parameter.empty`.

</div>

<div class="attribute">

Parameter.kind

Describes how argument values are bound to the parameter. The possible values are accessible via `Parameter` (like `Parameter.KEYWORD_ONLY`), and support comparison and ordering, in the following order:

<div class="tabularcolumns">

\|l\|L\|

</div>

<table>
<colgroup>
<col style="width: 34%" />
<col style="width: 65%" />
</colgroup>
<thead>
<tr>
<th><blockquote>
<p>Name</p>
</blockquote></th>
<th><p>Meaning</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p><em>POSITIONAL_ONLY</em></p></td>
<td><p>Value must be supplied as a positional argument. Positional only parameters are those which appear before a <code>/</code> entry (if present) in a Python function definition.</p></td>
</tr>
<tr>
<td><p><em>POSITIONAL_OR_KEYWORD</em></p></td>
<td><p>Value may be supplied as either a keyword or positional argument (this is the standard binding behaviour for functions implemented in Python.)</p></td>
</tr>
<tr>
<td><p><em>VAR_POSITIONAL</em></p></td>
<td><p>A tuple of positional arguments that aren't bound to any other parameter. This corresponds to a <code>*args</code> parameter in a Python function definition.</p></td>
</tr>
<tr>
<td><p><em>KEYWORD_ONLY</em></p></td>
<td><p>Value must be supplied as a keyword argument. Keyword only parameters are those which appear after a <code>*</code> or <code>*args</code> entry in a Python function definition.</p></td>
</tr>
<tr>
<td><p><em>VAR_KEYWORD</em></p></td>
<td><p>A dict of keyword arguments that aren't bound to any other parameter. This corresponds to a <code>**kwargs</code> parameter in a Python function definition.</p></td>
</tr>
</tbody>
</table>

Example: print all keyword-only arguments without default values:

<div class="doctest">

\>\>\> def foo(a, b, \*, c, d=10): ... pass

\>\>\> sig = signature(foo) \>\>\> for param in sig.parameters.values(): ... if (param.kind == param.KEYWORD_ONLY and ... param.default is param.empty): ... print('Parameter:', param) Parameter: c

</div>

</div>

<div class="attribute">

Parameter.kind.description

Describes an enum value of `Parameter.kind`.

<div class="versionadded">

3.8

</div>

Example: print all descriptions of arguments:

<div class="doctest">

\>\>\> def foo(a, b, \*, c, d=10): ... pass

\>\>\> sig = signature(foo) \>\>\> for param in sig.parameters.values(): ... print(param.kind.description) positional or keyword positional or keyword keyword-only keyword-only

</div>

</div>

<div class="method">

Parameter.replace(\*\[, name\]\[, kind\]\[, default\]\[, annotation\])

Create a new `Parameter` instance based on the instance replaced was invoked on. To override a `!Parameter` attribute, pass the corresponding argument. To remove a default value or/and an annotation from a `!Parameter`, pass `Parameter.empty`.

<div class="doctest">

\>\>\> from inspect import Parameter \>\>\> param = Parameter('foo', Parameter.KEYWORD_ONLY, default=42) \>\>\> str(param) 'foo=42'

\>\>\> str(param.replace()) \# Will create a shallow copy of 'param' 'foo=42'

\>\>\> str(param.replace(default=Parameter.empty, annotation='spam')) "foo: 'spam'"

</div>

`Parameter` objects are also supported by the generic function `copy.replace`.

</div>

<div class="versionchanged">

3.4 In Python 3.3 `Parameter` objects were allowed to have `name` set to `None` if their `kind` was set to `POSITIONAL_ONLY`. This is no longer permitted.

</div>

</div>

<div class="BoundArguments">

Result of a `Signature.bind` or `Signature.bind_partial` call. Holds the mapping of arguments to the function's parameters.

<div class="attribute">

BoundArguments.arguments

A mutable mapping of parameters' names to arguments' values. Contains only explicitly bound arguments. Changes in `arguments` will reflect in `args` and `kwargs`.

Should be used in conjunction with `Signature.parameters` for any argument processing purposes.

> [!NOTE]
> Arguments for which `Signature.bind` or `Signature.bind_partial` relied on a default value are skipped. However, if needed, use `BoundArguments.apply_defaults` to add them.

<div class="versionchanged">

3.9 `arguments` is now of type `dict`. Formerly, it was of type `collections.OrderedDict`.

</div>

</div>

<div class="attribute">

BoundArguments.args

A tuple of positional arguments values. Dynamically computed from the `arguments` attribute.

</div>

<div class="attribute">

BoundArguments.kwargs

A dict of keyword arguments values. Dynamically computed from the `arguments` attribute. Arguments that can be passed positionally are included in `args` instead.

</div>

<div class="attribute">

BoundArguments.signature

A reference to the parent `Signature` object.

</div>

<div class="method">

BoundArguments.apply_defaults()

Set default values for missing arguments.

For variable-positional arguments (`*args`) the default is an empty tuple.

For variable-keyword arguments (`**kwargs`) the default is an empty dict.

<div class="doctest">

\>\>\> def foo(a, b='ham', \*args): pass \>\>\> ba = inspect.signature(foo).bind('spam') \>\>\> ba.apply_defaults() \>\>\> ba.arguments {'a': 'spam', 'b': 'ham', 'args': ()}

</div>

<div class="versionadded">

3.5

</div>

</div>

The `args` and `kwargs` properties can be used to invoke functions:

<div class="testcode">

def test(a, \*, b):  
...

sig = signature(test) ba = sig.bind(10, b=20) test(*ba.args,*\*ba.kwargs)

</div>

</div>

<div class="seealso">

`362` - Function Signature Object.  
The detailed specification, implementation details and examples.

</div>

## Classes and functions

<div class="function">

getclasstree(classes, unique=False)

Arrange the given list of classes into a hierarchy of nested lists. Where a nested list appears, it contains classes derived from the class whose entry immediately precedes the list. Each entry is a 2-tuple containing a class and a tuple of its base classes. If the *unique* argument is true, exactly one entry appears in the returned structure for each class in the given list. Otherwise, classes using multiple inheritance and their descendants will appear multiple times.

</div>

<div class="function">

getfullargspec(func, \*, annotation_format=Format.VALUE)

Get the names and default values of a Python function's parameters. A `named tuple` is returned:

`FullArgSpec(args, varargs, varkw, defaults, kwonlyargs, kwonlydefaults, annotations)`

*args* is a list of the positional parameter names. *varargs* is the name of the `*` parameter or `None` if arbitrary positional arguments are not accepted. *varkw* is the name of the `**` parameter or `None` if arbitrary keyword arguments are not accepted. *defaults* is an *n*-tuple of default argument values corresponding to the last *n* positional parameters, or `None` if there are no such defaults defined. *kwonlyargs* is a list of keyword-only parameter names in declaration order. *kwonlydefaults* is a dictionary mapping parameter names from *kwonlyargs* to the default values used if no argument is supplied. *annotations* is a dictionary mapping parameter names to annotations. The special key `"return"` is used to report the function return value annotation (if any).

Note that `signature` and `Signature Object <inspect-signature-object>` provide the recommended API for callable introspection, and support additional behaviours (like positional-only arguments) that are sometimes encountered in extension module APIs. This function is retained primarily for use in code that needs to maintain compatibility with the Python 2 `inspect` module API.

A member of the `annotationlib.Format` enum can be passed to the *annotation_format* parameter to control the format of the returned annotations. For example, use `annotation_format=annotationlib.Format.STRING` to return annotations in string format. Note that with the default `VALUE` format, creation of some argspecs may raise an exception.

<div class="versionchanged">

3.4 This function is now based on `signature`, but still ignores `__wrapped__` attributes and includes the already bound first parameter in the signature output for bound methods.

</div>

<div class="versionchanged">

3.6 This method was previously documented as deprecated in favour of `signature` in Python 3.5, but that decision has been reversed in order to restore a clearly supported standard interface for single-source Python 2/3 code migrating away from the legacy `!getargspec` API.

</div>

<div class="versionchanged">

3.7 Python only explicitly guaranteed that it preserved the declaration order of keyword-only parameters as of version 3.7, although in practice this order had always been preserved in Python 3.

</div>

<div class="versionchanged">

3.15 The *annotation_format* parameter was added.

</div>

</div>

<div class="function">

getargvalues(frame)

Get information about arguments passed into a particular frame. A `named tuple` `ArgInfo(args, varargs, keywords, locals)` is returned. *args* is a list of the argument names. *varargs* and *keywords* are the names of the `*` and `**` arguments or `None`. *locals* is the locals dictionary of the given frame.

> [!NOTE]
> This function was inadvertently marked as deprecated in Python 3.5.

</div>

<div class="function">

formatargvalues(args\[, varargs, varkw, locals, formatarg, formatvarargs, formatvarkw, formatvalue\])

Format a pretty argument spec from the four values returned by `getargvalues`. The format\* arguments are the corresponding optional formatting functions that are called to turn names and values into strings.

Arguments whose names are absent from *locals*, including `*` and `**` arguments, are displayed with a value of `<deleted>`. A custom *formatvalue* function receives a placeholder object whose `repr` is `<deleted>` for these arguments.

> [!NOTE]
> This function was inadvertently marked as deprecated in Python 3.5.

</div>

<div class="function">

getmro(cls)

Return a tuple of class cls's base classes, including cls, in method resolution order. No class appears more than once in this tuple. Note that the method resolution order depends on cls's type. Unless a very peculiar user-defined metatype is in use, cls will be the first element of the tuple.

</div>

<div class="function">

getcallargs(func, /, *args,*\*kwds)

Bind the *args* and *kwds* to the argument names of the Python function or method *func*, as if it was called with them. For bound methods, bind also the first argument (typically named `self`) to the associated instance. A dict is returned, mapping the argument names (including the names of the `*` and `**` arguments, if any) to their values from *args* and *kwds*. In case of invoking *func* incorrectly, i.e. whenever `func(*args, **kwds)` would raise an exception because of incompatible signature, an exception of the same type and the same or similar message is raised. For example:

<div class="doctest">

\>\>\> from inspect import getcallargs \>\>\> def f(a, b=1, *pos,*\*named): ... pass ... \>\>\> getcallargs(f, 1, 2, 3) == {'a': 1, 'named': {}, 'b': 2, 'pos': (3,)} True \>\>\> getcallargs(f, a=2, x=4) == {'a': 2, 'named': {'x': 4}, 'b': 1, 'pos': ()} True \>\>\> getcallargs(f) Traceback (most recent call last): ... TypeError: f() missing 1 required positional argument: 'a'

</div>

<div class="versionadded">

3.2

</div>

<div class="deprecated">

3.5 Use `Signature.bind` and `Signature.bind_partial` instead.

</div>

</div>

<div class="function">

getclosurevars(func)

Get the mapping of external name references in a Python function or method *func* to their current values. A `named tuple` `ClosureVars(nonlocals, globals, builtins, unbound)` is returned. *nonlocals* maps referenced names to lexical closure variables, *globals* to the function's module globals and *builtins* to the builtins visible from the function body. *unbound* is the set of names referenced in the function that could not be resolved at all given the current module globals and builtins.

`TypeError` is raised if *func* is not a Python function or method.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

unwrap(func, \*, stop=None)

Get the object wrapped by *func*. It follows the chain of `__wrapped__` attributes returning the last object in the chain.

*stop* is an optional callback accepting an object in the wrapper chain as its sole argument that allows the unwrapping to be terminated early if the callback returns a true value. If the callback never returns a true value, the last object in the chain is returned as usual. For example, `signature` uses this to stop unwrapping if any object in the chain has a `__signature__` attribute defined.

`ValueError` is raised if a cycle is encountered.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

get_annotations(obj, \*, globals=None, locals=None, eval_str=False, format=annotationlib.Format.VALUE)

Compute the annotations dict for an object.

This is an alias for `annotationlib.get_annotations`; see the documentation of that function for more information.

> [!CAUTION]
> This function may execute arbitrary code contained in annotations. See `annotationlib-security` for more information.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.14 This function is now an alias for `annotationlib.get_annotations`. Calling it as `inspect.get_annotations` will continue to work.

</div>

</div>

## The interpreter stack

Some of the following functions return `FrameInfo` objects. For backwards compatibility these objects allow tuple-like operations on all attributes except `positions`. This behavior is considered deprecated and may be removed in the future.

<div class="FrameInfo">

<div class="attribute">

frame

The `frame object <frame-objects>` that the record corresponds to.

</div>

<div class="attribute">

filename

The file name associated with the code being executed by the frame this record corresponds to.

</div>

<div class="attribute">

lineno

The line number of the current line associated with the code being executed by the frame this record corresponds to.

</div>

<div class="attribute">

function

The function name that is being executed by the frame this record corresponds to.

</div>

<div class="attribute">

code_context

A list of lines of context from the source code that's being executed by the frame this record corresponds to.

</div>

<div class="attribute">

index

The index of the current line being executed in the `code_context` list.

</div>

<div class="attribute">

positions

A `dis.Positions` object containing the start line number, end line number, start column offset, and end column offset associated with the instruction being executed by the frame this record corresponds to.

</div>

<div class="versionchanged">

3.5 Return a `named tuple` instead of a `tuple`.

</div>

<div class="versionchanged">

3.11 `!FrameInfo` is now a class instance (that is backwards compatible with the previous `named tuple`).

</div>

</div>

<div class="Traceback">

<div class="attribute">

filename

The file name associated with the code being executed by the frame this traceback corresponds to.

</div>

<div class="attribute">

lineno

The line number of the current line associated with the code being executed by the frame this traceback corresponds to.

</div>

<div class="attribute">

function

The function name that is being executed by the frame this traceback corresponds to.

</div>

<div class="attribute">

code_context

A list of lines of context from the source code that's being executed by the frame this traceback corresponds to.

</div>

<div class="attribute">

index

The index of the current line being executed in the `code_context` list.

</div>

<div class="attribute">

positions

A `dis.Positions` object containing the start line number, end line number, start column offset, and end column offset associated with the instruction being executed by the frame this traceback corresponds to.

</div>

<div class="versionchanged">

3.11 `!Traceback` is now a class instance (that is backwards compatible with the previous `named tuple`).

</div>

</div>

> [!NOTE]
> Keeping references to frame objects, as found in the first element of the frame records these functions return, can cause your program to create reference cycles. Once a reference cycle has been created, the lifespan of all objects which can be accessed from the objects which form the cycle can become much longer even if Python's optional cycle detector is enabled. If such cycles must be created, it is important to ensure they are explicitly broken to avoid the delayed destruction of objects and increased memory consumption which occurs.
>
> Though the cycle detector will catch these, destruction of the frames (and local variables) can be made deterministic by removing the cycle in a `finally` clause. This is also important if the cycle detector was disabled when Python was compiled or using `gc.disable`. For example:
>
>     def handle_stackframe_without_leak():
>         frame = inspect.currentframe()
>         try:
>             # do something with the frame
>         finally:
>             del frame
>
> If you want to keep the frame around (for example to print a traceback later), you can also break reference cycles by using the `frame.clear` method.

The optional *context* argument supported by most of these functions specifies the number of lines of context to return, which are centered around the current line.

<div class="function">

getframeinfo(frame, context=1)

Get information about a frame or traceback object. A `Traceback` object is returned.

<div class="versionchanged">

3.11 A `Traceback` object is returned instead of a named tuple.

</div>

</div>

<div class="function">

getouterframes(frame, context=1)

Get a list of `FrameInfo` objects for a frame and all outer frames. These frames represent the calls that lead to the creation of *frame*. The first entry in the returned list represents *frame*; the last entry represents the outermost call on *frame*'s stack.

<div class="versionchanged">

3.5 A list of `named tuples <named tuple>` `FrameInfo(frame, filename, lineno, function, code_context, index)` is returned.

</div>

<div class="versionchanged">

3.11 A list of `FrameInfo` objects is returned.

</div>

</div>

<div class="function">

getinnerframes(traceback, context=1)

Get a list of `FrameInfo` objects for a traceback's frame and all inner frames. These frames represent calls made as a consequence of *frame*. The first entry in the list represents *traceback*; the last entry represents where the exception was raised.

<div class="versionchanged">

3.5 A list of `named tuples <named tuple>` `FrameInfo(frame, filename, lineno, function, code_context, index)` is returned.

</div>

<div class="versionchanged">

3.11 A list of `FrameInfo` objects is returned.

</div>

</div>

<div class="function">

currentframe()

Return the frame object for the caller's stack frame.

<div class="impl-detail">

This function relies on Python stack frame support in the interpreter, which isn't guaranteed to exist in all implementations of Python. If running in an implementation without Python stack frame support this function returns `None`.

</div>

</div>

<div class="function">

stack(context=1)

Return a list of `FrameInfo` objects for the caller's stack. The first entry in the returned list represents the caller; the last entry represents the outermost call on the stack.

<div class="versionchanged">

3.5 A list of `named tuples <named tuple>` `FrameInfo(frame, filename, lineno, function, code_context, index)` is returned.

</div>

<div class="versionchanged">

3.11 A list of `FrameInfo` objects is returned.

</div>

</div>

<div class="function">

trace(context=1)

Return a list of `FrameInfo` objects for the stack between the current frame and the frame in which an exception currently being handled was raised in. The first entry in the list represents the caller; the last entry represents where the exception was raised.

<div class="versionchanged">

3.5 A list of `named tuples <named tuple>` `FrameInfo(frame, filename, lineno, function, code_context, index)` is returned.

</div>

<div class="versionchanged">

3.11 A list of `FrameInfo` objects is returned.

</div>

</div>

## Fetching attributes statically

Both `getattr` and `hasattr` can trigger code execution when fetching or checking for the existence of attributes. Descriptors, like properties, will be invoked and `~object.__getattr__` and `~object.__getattribute__` may be called.

For cases where you want passive introspection, like documentation tools, this can be inconvenient. `getattr_static` has a similar signature as `getattr` but avoids executing code when it fetches attributes.

<div class="function">

getattr_static(obj, attr) getattr_static(obj, attr, default)

Retrieve attributes without triggering dynamic lookup via the descriptor protocol, `~object.__getattr__` or `~object.__getattribute__`.

Note: this function may not be able to retrieve all attributes that getattr can fetch (like dynamically created attributes) and may find attributes that getattr can't (like descriptors that raise AttributeError). It can also return descriptors objects instead of instance members.

If the instance `~object.__dict__` is shadowed by another member (for example a property) then this function will be unable to find instance members.

<div class="versionadded">

3.2

</div>

</div>

`getattr_static` does not resolve descriptors, for example slot descriptors or getset descriptors on objects implemented in C. The descriptor object is returned instead of the underlying attribute.

You can handle these with code like the following. Note that for arbitrary getset descriptors invoking these may trigger code execution:

    # example code for resolving the builtin descriptor types
    class _foo:
        __slots__ = ['foo']

    slot_descriptor = type(_foo.foo)
    getset_descriptor = type(type(open(__file__)).name)
    wrapper_descriptor = type(str.__dict__['__add__'])
    descriptor_types = (slot_descriptor, getset_descriptor, wrapper_descriptor)

    result = getattr_static(some_object, 'foo')
    if type(result) in descriptor_types:
        try:
            result = result.__get__()
        except AttributeError:
            # descriptors can raise AttributeError to
            # indicate there is no underlying value
            # in which case the descriptor itself will
            # have to do
            pass

## Current State of Generators, Coroutines, and Asynchronous Generators

When implementing coroutine schedulers and for other advanced uses of generators, it is useful to determine whether a generator is currently executing, is waiting to start or resume or execution, or has already terminated. `getgeneratorstate` allows the current state of a generator to be determined easily.

<div class="function">

getgeneratorstate(generator)

Get current state of a generator-iterator.

Possible states are:

- GEN_CREATED: Waiting to start execution.
- GEN_RUNNING: Currently being executed by the interpreter.
- GEN_SUSPENDED: Currently suspended at a yield expression.
- GEN_CLOSED: Execution has completed.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

getcoroutinestate(coroutine)

Get current state of a coroutine object. The function is intended to be used with coroutine objects created by `async def` functions, but will accept any coroutine-like object that has `cr_running` and `cr_frame` attributes.

Possible states are:

- CORO_CREATED: Waiting to start execution.
- CORO_RUNNING: Currently being executed by the interpreter.
- CORO_SUSPENDED: Currently suspended at an await expression.
- CORO_CLOSED: Execution has completed.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

getasyncgenstate(agen)

Get current state of an asynchronous generator object. The function is intended to be used with asynchronous iterator objects created by `async def` functions which use the `yield` statement, but will accept any asynchronous generator-like object that has `ag_running` and `ag_frame` attributes.

Possible states are:

- AGEN_CREATED: Waiting to start execution.
- AGEN_RUNNING: Currently being executed by the interpreter.
- AGEN_SUSPENDED: Currently suspended at a yield expression.
- AGEN_CLOSED: Execution has completed.

<div class="versionadded">

3.12

</div>

</div>

The current internal state of the generator can also be queried. This is mostly useful for testing purposes, to ensure that internal state is being updated as expected:

<div class="function">

getgeneratorlocals(generator)

Get the mapping of live local variables in *generator* to their current values. A dictionary is returned that maps from variable names to values. This is the equivalent of calling `locals` in the body of the generator, and all the same caveats apply.

If *generator* is a `generator` with no currently associated frame, then an empty dictionary is returned. `TypeError` is raised if *generator* is not a Python generator object.

<div class="impl-detail">

This function relies on the generator exposing a Python stack frame for introspection, which isn't guaranteed to be the case in all implementations of Python. In such cases, this function will always return an empty dictionary.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

getcoroutinelocals(coroutine)

This function is analogous to `~inspect.getgeneratorlocals`, but works for coroutine objects created by `async def` functions.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

getasyncgenlocals(agen)

This function is analogous to `~inspect.getgeneratorlocals`, but works for asynchronous generator objects created by `async def` functions which use the `yield` statement.

<div class="versionadded">

3.12

</div>

</div>

## Code Objects Bit Flags

Python code objects have a `~codeobject.co_flags` attribute, which is a bitmap of the following flags:

<div class="data">

CO_OPTIMIZED

The code object is optimized, using fast locals.

</div>

<div class="data">

CO_NEWLOCALS

If set, a new dict will be created for the frame's `~frame.f_locals` when the code object is executed.

</div>

<div class="data">

CO_VARARGS

The code object has a variable positional parameter (`*args`-like).

</div>

<div class="data">

CO_VARKEYWORDS

The code object has a variable keyword parameter (`**kwargs`-like).

</div>

<div class="data">

CO_NESTED

The flag is set when the code object is a nested function.

</div>

<div class="data">

CO_GENERATOR

The flag is set when the code object is a generator function, i.e. a generator object is returned when the code object is executed.

</div>

<div class="data">

CO_COROUTINE

The flag is set when the code object is a coroutine function. When the code object is executed it returns a coroutine object. See `492` for more details.

<div class="versionadded">

3.5

</div>

</div>

<div class="data">

CO_ITERABLE_COROUTINE

The flag is used to transform generators into generator-based coroutines. Generator objects with this flag can be used in `await` expression, and can `yield from` coroutine objects. See `492` for more details.

<div class="versionadded">

3.5

</div>

</div>

<div class="data">

CO_ASYNC_GENERATOR

The flag is set when the code object is an asynchronous generator function. When the code object is executed it returns an asynchronous generator object. See `525` for more details.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

CO_HAS_DOCSTRING

The flag is set when there is a docstring for the code object in the source code. If set, it will be the first item in `~codeobject.co_consts`.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

CO_METHOD

The flag is set when the code object is a function defined in class scope.

<div class="versionadded">

3.14

</div>

</div>

> [!NOTE]
> The flags are specific to CPython, and may not be defined in other Python implementations. Furthermore, the flags are an implementation detail, and can be removed or deprecated in future Python releases. It's recommended to use public APIs from the `!inspect` module for any introspection needs.

## Buffer flags

<div class="BufferFlags">

This is an `enum.IntFlag` that represents the flags that can be passed to the `~object.__buffer__` method of objects implementing the `buffer protocol <bufferobjects>`.

The meaning of the flags is explained at `buffer-request-types`.

<div class="attribute">

BufferFlags.SIMPLE

</div>

<div class="attribute">

BufferFlags.WRITABLE

</div>

<div class="attribute">

BufferFlags.FORMAT

</div>

<div class="attribute">

BufferFlags.ND

</div>

<div class="attribute">

BufferFlags.STRIDES

</div>

<div class="attribute">

BufferFlags.C_CONTIGUOUS

</div>

<div class="attribute">

BufferFlags.F_CONTIGUOUS

</div>

<div class="attribute">

BufferFlags.ANY_CONTIGUOUS

</div>

<div class="attribute">

BufferFlags.INDIRECT

</div>

<div class="attribute">

BufferFlags.CONTIG

</div>

<div class="attribute">

BufferFlags.CONTIG_RO

</div>

<div class="attribute">

BufferFlags.STRIDED

</div>

<div class="attribute">

BufferFlags.STRIDED_RO

</div>

<div class="attribute">

BufferFlags.RECORDS

</div>

<div class="attribute">

BufferFlags.RECORDS_RO

</div>

<div class="attribute">

BufferFlags.FULL

</div>

<div class="attribute">

BufferFlags.FULL_RO

</div>

<div class="attribute">

BufferFlags.READ

</div>

<div class="attribute">

BufferFlags.WRITE

</div>

<div class="versionadded">

3.12

</div>

</div>

## Command-line interface

The `!inspect` module also provides a basic introspection capability from the command line.

<div class="program">

inspect

</div>

By default, accepts the name of a module and prints the source of that module. A class or function within the module can be printed instead by appending a colon and the qualified name of the target object.

<div class="option">

`--details`  

Print information about the specified object rather than the source code

</div>

<div class="versionchanged">

3.15

The `--details` option now supports basic introspection for modules without available source code and indicates when modules are frozen. It also indicates when the given target reference is not the canonical name of the referenced object.

</div>

# Internet Protocols and Support

<div class="index">

single: WWW single: Internet single: World Wide Web

</div>

<div class="index">

pair: module; socket

</div>

The modules described in this chapter implement internet protocols and support for related technology. They are all implemented in Python. Most of these modules require the presence of the system-dependent module `socket`, which is currently supported on most popular platforms. Here is an overview:

<div class="toctree">

webbrowser.rst wsgiref.rst urllib.rst urllib.request.rst urllib.parse.rst urllib.error.rst urllib.robotparser.rst http.rst http.client.rst ftplib.rst poplib.rst imaplib.rst smtplib.rst uuid.rst socketserver.rst http.server.rst http.cookies.rst http.cookiejar.rst xmlrpc.rst xmlrpc.client.rst xmlrpc.server.rst ipaddress.rst

</div>

# Introduction

The Python standard library consists of a collection of modules. There are many ways to dissect this collection. Most modules are written in Python, but some are written in C. All can be imported into your program to add functionality. Some modules provide interfaces that are highly specific to Python, like printing a stack trace; some provide interfaces that are specific to particular operating systems, such as access to specific hardware; others provide interfaces that are specific to a particular application domain, like web development. Some modules are available in all versions and ports of Python; others are only available when the underlying system supports or requires them; yet others are available only when a particular configuration option was chosen at the time when Python was compiled and installed.

If you start reading this manual from the start, and skip to the next chapter when you get bored, you will get a reasonable overview of the available modules and application areas that are supported by the Python library. Of course, you don't *have* to read it like a novel --- you can also browse the table of contents (in front of the manual), or look for a specific function, module or term in the index (in the back). And finally, if you enjoy learning about random subjects, you choose a random page and read a section or two. Regardless of the order in which you read the sections of this manual, it helps to first read `built-in-funcs`, as the remainder of this section assumes familiarity with this material.

<div class="seealso">

The built-in functions and classes (which can be used without an `import` statement) are described in `builtins-index`.

</div>

## Notes on availability

- An "Availability: Unix" note means that this function is commonly found on Unix systems. It does not make any claims about its existence on a specific operating system.
- If not separately noted, all functions that claim "Availability: Unix" are supported on macOS, iOS and Android, all of which build on a Unix core.
- If an availability note contains both a minimum Kernel version and a minimum libc version, then both conditions must hold. For example a feature with note *Availability: Linux \>= 3.17 with glibc \>= 2.27* requires both Linux 3.17 or newer and glibc 2.27 or newer.

### WebAssembly platforms

The [WebAssembly](https://webassembly.org/) platforms `wasm32-emscripten` ([Emscripten](https://emscripten.org/)) and `wasm32-wasi` ([WASI](https://wasi.dev/)) provide a subset of POSIX APIs. WebAssembly runtimes and browsers are sandboxed and have limited access to the host and external resources. Any Python standard library module that uses processes, threading, networking, signals, or other forms of inter-process communication (IPC), is either not available or may not work as on other Unix-like systems. File I/O, file system, and Unix permission-related functions are restricted, too. Emscripten does not permit blocking I/O. Other blocking operations like `~time.sleep` block the browser event loop.

The properties and behavior of Python on WebAssembly platforms depend on the [Emscripten](https://emscripten.org/)-SDK or [WASI](https://wasi.dev/)-SDK version, WASM runtimes (browser, NodeJS, [wasmtime](https://wasmtime.dev/)), and Python build time flags. WebAssembly, Emscripten, and WASI are evolving standards; some features like networking may be supported in the future.

For Python in the browser, users should consider [Pyodide](https://pyodide.org/) or [PyScript](https://pyscript.net/). PyScript is built on top of Pyodide, which itself is built on top of CPython and Emscripten. Pyodide provides access to browsers' JavaScript and DOM APIs as well as limited networking capabilities with JavaScript's `XMLHttpRequest` and `Fetch` APIs.

- Process-related APIs are not available or always fail with an error. That includes APIs that spawn new processes (`~os.fork`, `~os.execve`), wait for processes (`~os.waitpid`), send signals (`~os.kill`), or otherwise interact with processes. The `subprocess` is importable but does not work.
- The `socket` module is available, but is limited and behaves differently from other platforms. On Emscripten, sockets are always non-blocking and require additional JavaScript code and helpers on the server to proxy TCP through WebSockets; see [Emscripten Networking](https://emscripten.org/docs/porting/networking.html) for more information. WASI snapshot preview 1 only permits sockets from an existing file descriptor.
- Some functions are stubs that either don't do anything and always return hardcoded values.
- Functions related to file descriptors, file permissions, file ownership, and links are limited and don't support some operations. For example, WASI does not permit symlinks with absolute file names.

### Mobile platforms<span id="mobile-availability"></span>

Android and iOS are, in most respects, POSIX operating systems. File I/O, socket handling, and threading all behave as they would on any POSIX operating system. However, there are several major differences:

- Mobile platforms can only use Python in "embedded" mode. There is no Python REPL, and no ability to use separate executables such as `python` or `pip`. To add Python code to your mobile app, you must use the `Python embedding API <embedding>`. For more details, see `using-android` and `using-ios`.
- Subprocesses:
  - On Android, creating subprocesses is possible but [officially unsupported](https://issuetracker.google.com/issues/128554619#comment4). In particular, Android does not support any part of the System V IPC API, so `multiprocessing` is not available.
  - An iOS app cannot use any form of subprocessing, multiprocessing, or inter-process communication. If an iOS app attempts to create a subprocess, the process creating the subprocess will either lock up, or crash. An iOS app has no visibility of other applications that are running, nor any ability to communicate with other running applications, outside of the iOS-specific APIs that exist for this purpose.
- Mobile apps have limited access to modify system resources (such as the system clock). These resources will often be *readable*, but attempts to modify those resources will usually fail.
- Console input and output:
  - On Android, the native `stdout` and `stderr` are not connected to anything, so Python installs its own streams which redirect messages to the system log. These can be seen under the tags `python.stdout` and `python.stderr` respectively.

  - iOS apps have a limited concept of console output. `stdout` and `stderr` *exist*, and content written to `stdout` and `stderr` will be visible in logs when running in Xcode, but this content *won't* be recorded in the system log. If a user who has installed your app provides their app logs as a diagnostic aid, they will not include any detail written to `stdout` or `stderr`.

  - Mobile apps have no usable `stdin` at all. While apps can display an on-screen keyboard, this is a software feature, not something that is attached to `stdin`.

    As a result, Python modules that involve console manipulation (such as `curses` and `readline`) are not available on mobile platforms.

# `!io` --- Core tools for working with streams

<div class="module" data-synopsis="Core tools for working with streams.">

io

</div>

**Source code:** `Lib/io.py`

------------------------------------------------------------------------

## Overview

<div class="index">

single: file object; io module

</div>

The `!io` module provides Python's main facilities for dealing with various types of I/O. There are three main types of I/O: *text I/O*, *binary I/O* and *raw I/O*. These are generic categories, and various backing stores can be used for each of them. A concrete object belonging to any of these categories is called a `file object`. Other common terms are *stream* and *file-like object*.

Independent of its category, each concrete stream object will also have various capabilities: it can be read-only, write-only, or read-write. It can also allow arbitrary random access (seeking forwards or backwards to any location), or only sequential access (for example in the case of a socket or pipe).

All streams are careful about the type of data you give to them. For example giving a `str` object to the `!write` method of a binary stream will raise a `TypeError`. So will giving a `bytes` object to the `!write` method of a text stream.

<div class="versionchanged">

3.3 Operations that used to raise `IOError` now raise `OSError`, since `IOError` is now an alias of `OSError`.

</div>

### Text I/O

Text I/O expects and produces `str` objects. This means that whenever the backing store is natively made of bytes (such as in the case of a file), encoding and decoding of data is made transparently as well as optional translation of platform-specific newline characters.

The easiest way to create a text stream is with `open`, optionally specifying an encoding:

    f = open("myfile.txt", "r", encoding="utf-8")

In-memory text streams are also available as `StringIO` objects:

    f = io.StringIO("some initial text data")

> [!NOTE]
> When working with a non-blocking stream, be aware that read operations on text I/O objects might raise a `BlockingIOError` if the stream cannot perform the operation immediately.

The text stream API is described in detail in the documentation of `TextIOBase`.

### Binary I/O

Binary I/O (also called *buffered I/O*) expects `bytes-like objects <bytes-like object>` and produces `bytes` objects. No encoding, decoding, or newline translation is performed. This category of streams can be used for all kinds of non-text data, and also when manual control over the handling of text data is desired.

The easiest way to create a binary stream is with `open` with `'b'` in the mode string:

    f = open("myfile.jpg", "rb")

In-memory binary streams are also available as `BytesIO` objects:

    f = io.BytesIO(b"some initial binary data: \x00\x01")

The binary stream API is described in detail in the docs of `BufferedIOBase`.

Other library modules may provide additional ways to create text or binary streams. See `socket.socket.makefile` for example.

### Raw I/O

Raw I/O (also called *unbuffered I/O*) is generally used as a low-level building-block for binary and text streams; it is rarely useful to directly manipulate a raw stream from user code. Nevertheless, you can create a raw stream by opening a file in binary mode with buffering disabled:

    f = open("myfile.jpg", "rb", buffering=0)

The raw stream API is described in detail in the docs of `RawIOBase`.

> [!WARNING]
> Raw I/O is a low-level interface and methods generally must have their return values checked and be explicitly retried to ensure an operation completes. For instance `~RawIOBase.write` returns the number of bytes written which may be less than the number of bytes provided (a partial write). High-level I/O objects like `binary-io` and `text-io` implement retry behavior.

## Text encoding

The default encoding of `TextIOWrapper` and `open` is UTF-8. If `Python UTF-8 Mode <utf8-mode>` is disabled, the default encoding is locale-specific (`locale.getencoding`).

<div class="seealso">

`utf8-mode`  
Python UTF-8 Mode ignores the locale encoding and forces the use of UTF-8.

`686`  
Python 3.15 made `utf8-mode` the default.

</div>

### Opt-in EncodingWarning

<div class="versionadded">

3.10 See `597` for more details.

</div>

To find where the default encoding is used, you can enable the `-X warn_default_encoding <-X>` command line option or set the `PYTHONWARNDEFAULTENCODING` environment variable, which will emit an `EncodingWarning` when the default encoding is used.

If you are providing an API that uses `open` or `TextIOWrapper` and passes `encoding=None` as a parameter, you can use `text_encoding` so that callers of the API will emit an `EncodingWarning` if they don't pass an `encoding`. However, please consider using UTF-8 by default (i.e. `encoding="utf-8"`) for new APIs.

## High-level module interface

<div class="data">

DEFAULT_BUFFER_SIZE

An int containing the default buffer size used by the module's buffered I/O classes. `open` uses the file's blksize (as obtained by `os.stat`) if possible.

</div>

<div class="function">

open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None)

This is an alias for the builtin `open` function.

<div class="audit-event">

open path,mode,flags io.open

This function raises an `auditing event <auditing>` `open` with arguments *path*, *mode* and *flags*. The *mode* and *flags* arguments may have been modified or inferred from the original call.

</div>

</div>

<div class="function">

open_code(path)

Opens the provided file with mode `'rb'`. This function should be used when the intent is to treat the contents as executable code.

*path* should be a `str` and an absolute path.

The behavior of this function may be overridden by an earlier call to the `PyFile_SetOpenCodeHook`. However, assuming that *path* is a `str` and an absolute path, `open_code(path)` should always behave the same as `open(path, 'rb')`. Overriding the behavior is intended for additional validation or preprocessing of the file.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

text_encoding(encoding, stacklevel=2, /)

This is a helper function for callables that use `open` or `TextIOWrapper` and have an `encoding=None` parameter.

This function returns *encoding* if it is not `None`. Otherwise, it returns `"locale"` or `"utf-8"` depending on `UTF-8 Mode <utf8-mode>`.

This function emits an `EncodingWarning` if `sys.flags.warn_default_encoding <sys.flags>` is true and *encoding* is `None`. *stacklevel* specifies where the warning is emitted. For example:

    def read_text(path, encoding=None):
        encoding = io.text_encoding(encoding)  # stacklevel=2
        with open(path, encoding) as f:
            return f.read()

In this example, an `EncodingWarning` is emitted for the caller of `read_text()`.

See `io-text-encoding` for more information.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.11 `text_encoding` returns "utf-8" when UTF-8 mode is enabled and *encoding* is `None`.

</div>

</div>

<div class="exception">

BlockingIOError

This is a compatibility alias for the builtin `BlockingIOError` exception.

</div>

<div class="exception">

UnsupportedOperation

An exception inheriting `OSError` and `ValueError` that is raised when an unsupported operation is called on a stream.

</div>

<div class="seealso">

`sys`  
contains the standard IO streams: `sys.stdin`, `sys.stdout`, and `sys.stderr`.

</div>

## Class hierarchy

The implementation of I/O streams is organized as a hierarchy of classes. First `abstract base classes <abstract base class>` (ABCs), which are used to specify the various categories of streams, then concrete classes providing the standard stream implementations.

> [!NOTE]
> The abstract base classes also provide default implementations of some methods in order to help implementation of concrete stream classes. For example, `BufferedIOBase` provides unoptimized implementations of `!readinto` and `!readline`.

At the top of the I/O hierarchy is the abstract base class `IOBase`. It defines the basic interface to a stream. Note, however, that there is no separation between reading and writing to streams; implementations are allowed to raise `UnsupportedOperation` if they do not support a given operation.

The `RawIOBase` ABC extends `IOBase`. It deals with the reading and writing of bytes to a stream. `FileIO` subclasses `RawIOBase` to provide an interface to files in the machine's file system.

The `BufferedIOBase` ABC extends `IOBase`. It deals with buffering on a raw binary stream (`RawIOBase`). Its subclasses, `BufferedWriter`, `BufferedReader`, and `BufferedRWPair` buffer raw binary streams that are writable, readable, and both readable and writable, respectively. `BufferedRandom` provides a buffered interface to seekable streams. Another `BufferedIOBase` subclass, `BytesIO`, is a stream of in-memory bytes.

The `TextIOBase` ABC extends `IOBase`. It deals with streams whose bytes represent text, and handles encoding and decoding to and from strings. `TextIOWrapper`, which extends `TextIOBase`, is a buffered text interface to a buffered raw stream (`BufferedIOBase`). Finally, `StringIO` is an in-memory stream for text.

Argument names are not part of the specification, and only the arguments of `open` are intended to be used as keyword arguments.

The following table summarizes the ABCs provided by the `!io` module:

<div class="tabularcolumns">

\|l\|l\|L\|L\|

</div>

| ABC | Inherits | Stub Methods | Mixin Methods and Properties |
|----|----|----|----|
| `IOBase` |  | `fileno`, `seek`, and `truncate` | `close`, `closed`, `__enter__`, `__exit__`, `flush`, `isatty`, `__iter__`, `__next__`, `readable`, `readline`, `readlines`, `seekable`, `tell`, `writable`, and `writelines` |
| `RawIOBase` | `IOBase` | `readinto` and `write` | Inherited `IOBase` methods, `read`, and `readall` |
| `BufferedIOBase` | `IOBase` | `detach`, `read`, `read1`, and `write` | Inherited `IOBase` methods, `readinto`, and `readinto1` |
| `TextIOBase` | `IOBase` | `detach`, `read`, `readline`, and `write` | Inherited `IOBase` methods, `encoding`, `errors`, and `newlines` |

### I/O base classes

<div class="IOBase">

The abstract base class for all I/O classes.

This class provides empty abstract implementations for many methods that derived classes can override selectively; the default implementations represent a file that cannot be read, written or seeked.

Even though `IOBase` does not declare `!read` or `!write` because their signatures will vary, implementations and clients should consider those methods part of the interface. Also, implementations may raise a `ValueError` (or `UnsupportedOperation`) when operations they do not support are called.

The basic type used for binary data read from or written to a file is `bytes`. Other `bytes-like objects <bytes-like object>` are accepted as method arguments too. Text I/O classes work with `str` data.

Note that calling any method (even inquiries) on a closed stream is undefined. Implementations may raise `ValueError` in this case.

`IOBase` (and its subclasses) supports the iterator protocol, meaning that an `IOBase` object can be iterated over yielding the lines in a stream. Lines are defined slightly differently depending on whether the stream is a binary stream (yielding bytes), or a text stream (yielding character strings). See `~IOBase.readline` below.

`IOBase` is also a context manager and therefore supports the `with` statement. In this example, *file* is closed after the `!with` statement's suite is finished---even if an exception occurs:

    with open('spam.txt', 'w') as file:
        file.write('Spam and eggs!')

`IOBase` provides these data attributes and methods:

<div class="method">

close()

Flush and close this stream. This method has no effect if the file is already closed. Once the file is closed, any operation on the file (e.g. reading or writing) will raise a `ValueError`.

As a convenience, it is allowed to call this method more than once; only the first call, however, will have an effect.

</div>

<div class="attribute">

closed

`True` if the stream is closed.

</div>

<div class="method">

fileno()

Return the underlying file descriptor (an integer) of the stream if it exists. An `OSError` is raised if the IO object does not use a file descriptor.

</div>

<div class="method">

flush()

Flush the write buffers of the stream if applicable. This does nothing for read-only and non-blocking streams.

</div>

<div class="method">

isatty()

Return `True` if the stream is interactive (i.e., connected to a terminal/tty device).

</div>

<div class="method">

readable()

Return `True` if the stream can be read from. If `False`, `!read` will raise `OSError`.

</div>

<div class="method">

readline(size=-1, /)

Read and return one line from the stream. If *size* is specified, at most *size* bytes will be read.

The line terminator is always `b'\n'` for binary files; for text files, the *newline* argument to `open` can be used to select the line terminator(s) recognized.

</div>

<div class="method">

readlines(hint=-1, /)

Read and return a list of lines from the stream. *hint* can be specified to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds *hint*.

*hint* values of `0` or less, as well as `None`, are treated as no hint.

Note that it's already possible to iterate on file objects using `for line in file: ...` without calling `!file.readlines`.

</div>

<div class="method">

seek(offset, whence=os.SEEK_SET, /)

Change the stream position to the given byte *offset*, interpreted relative to the position indicated by *whence*, and return the new absolute position. Values for *whence* are:

- `os.SEEK_SET` or `0` -- start of the stream (the default); *offset* should be zero or positive
- `os.SEEK_CUR` or `1` -- current stream position; *offset* may be negative
- `os.SEEK_END` or `2` -- end of the stream; *offset* is usually negative

<div class="versionadded">

3.1 The `!SEEK_*` constants.

</div>

<div class="versionadded">

3.3 Some operating systems could support additional values, like `os.SEEK_HOLE` or `os.SEEK_DATA`. The valid values for a file could depend on it being open in text or binary mode.

</div>

</div>

<div class="method">

seekable()

Return `True` if the stream supports random access. If `False`, `seek`, `tell` and `truncate` will raise `OSError`.

</div>

<div class="method">

tell()

Return the current stream position.

</div>

<div class="method">

truncate(size=None, /)

Resize the stream to the given *size* in bytes (or the current position if *size* is not specified). The current stream position isn't changed. This resizing can extend or reduce the current file size. In case of extension, the contents of the new file area depend on the platform (on most systems, additional bytes are zero-filled). The new file size is returned.

<div class="versionchanged">

3.5 Windows will now zero-fill files when extending.

</div>

</div>

<div class="method">

writable()

Return `True` if the stream supports writing. If `False`, `!write` and `truncate` will raise `OSError`.

</div>

<div class="method">

writelines(lines, /)

Write a list of lines to the stream. Line separators are not added, so it is usual for each of the lines provided to have a line separator at the end.

</div>

<div class="method">

\_\_del\_\_()

Prepare for object destruction. `IOBase` provides a default implementation of this method that calls the instance's `~IOBase.close` method.

</div>

</div>

<div class="RawIOBase">

Base class for raw binary streams. It inherits from `IOBase`.

Raw binary streams typically provide low-level access to an underlying OS device or API, and do not try to encapsulate it in high-level primitives (this functionality is done at a higher-level in buffered binary streams and text streams, described later in this page).

`RawIOBase` provides these methods in addition to those from `IOBase`:

<div class="method">

read(size=-1, /)

Read up to *size* bytes from the object and return them. As a convenience, if *size* is unspecified or -1, all bytes until EOF are returned.

Attempts to make only one system call but will retry if interrupted and the signal handler does not raise an exception (see `475` for the rationale). This means fewer than *size* bytes may be returned if the operating system call returns fewer than *size* bytes.

If 0 bytes are returned, and *size* was not 0, this indicates end of file. If the object is in non-blocking mode and no bytes are available, `None` is returned.

The default implementation defers to `readall` and `readinto`.

</div>

<div class="method">

readall()

Read and return all the bytes from the stream until EOF, using multiple calls to the stream if necessary.

If `0` bytes are returned this indicates end of file. If the object is in non-blocking mode and the underlying `read` returns `None` indicating no bytes are available, `None` is returned.

</div>

<div class="method">

readinto(b, /)

Read bytes into a pre-allocated, writable `bytes-like object` *b*, and return the number of bytes read. For example, *b* might be a `bytearray`.

If `0` is returned and `len(b)` is not `0`, this indicates end of file. If the object is in non-blocking mode and no bytes are available, `None` is returned.

</div>

<div class="method">

write(b, /)

Write the given `bytes-like object`, *b*, to the underlying raw stream, and return the number of bytes written. This can be less than the length of *b* in bytes, depending on specifics of the underlying raw stream, and especially if it is in non-blocking mode. `None` is returned if the raw stream is set not to block and no single byte could be readily written to it. The caller may release or mutate *b* after this method returns, so the implementation should only access *b* during the method call.

> [!WARNING]
> This function does not ensure all bytes are written or an exception is thrown. Callers may implement that behavior by checking the return value and, if it is less than the length of *b*, looping with additional write calls until all unwritten bytes are written. High-level I/O objects like `binary-io` and `text-io` implement retry behavior.

</div>

</div>

<div class="BufferedIOBase">

Base class for binary streams that support some kind of buffering. It inherits from `IOBase`.

The main difference with `RawIOBase` is that methods `read`, `readinto` and `write` will try (respectively) to read as much input as requested or to emit all provided data.

In addition, if the underlying raw stream is in non-blocking mode, when the system returns would block `write` will raise `BlockingIOError` with `BlockingIOError.characters_written` and `read` will return data read so far or `None` if no data is available.

Besides, the `read` method does not have a default implementation that defers to `readinto`.

A typical `BufferedIOBase` implementation should not inherit from a `RawIOBase` implementation, but wrap one, like `BufferedWriter` and `BufferedReader` do.

`BufferedIOBase` provides or overrides these data attributes and methods in addition to those from `IOBase`:

<div class="attribute">

raw

The underlying raw stream (a `RawIOBase` instance) that `BufferedIOBase` deals with. This is not part of the `BufferedIOBase` API and may not exist on some implementations.

</div>

<div class="method">

detach()

Separate the underlying raw stream from the buffer and return it.

After the raw stream has been detached, the buffer is in an unusable state.

Some buffers, like `BytesIO`, do not have the concept of a single raw stream to return from this method. They raise `UnsupportedOperation`.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

read(size=-1, /)

Read and return up to *size* bytes. If the argument is omitted, `None`, or negative read as much as possible.

Fewer bytes may be returned than requested. An empty `bytes` object is returned if the stream is already at EOF. More than one read may be made and calls may be retried if specific errors are encountered, see `os.read` and `475` for more details. Less than size bytes being returned does not imply that EOF is imminent.

When reading as much as possible the default implementation will use `raw.readall` if available (which should implement `RawIOBase.readall`), otherwise will read in a loop until read returns `None`, an empty `bytes`, or a non-retryable error. For most streams this is to EOF, but for non-blocking streams more data may become available.

> [!NOTE]
> When the underlying raw stream is non-blocking, implementations may either raise `BlockingIOError` or return `None` if no data is available. `!io` implementations return `None`.

</div>

<div class="method">

read1(size=-1, /)

Read and return up to *size* bytes, calling `~RawIOBase.readinto` which may retry if `~errno.EINTR` is encountered per `475`. If *size* is `-1` or not provided, the implementation will choose an arbitrary value for *size*.

> [!NOTE]
> When the underlying raw stream is non-blocking, implementations may either raise `BlockingIOError` or return `None` if no data is available. `!io` implementations return `None`.

</div>

<div class="method">

readinto(b, /)

Read bytes into a pre-allocated, writable `bytes-like object` *b* and return the number of bytes read. For example, *b* might be a `bytearray`.

Like `read`, multiple reads may be issued to the underlying raw stream, unless the latter is interactive.

A `BlockingIOError` is raised if the underlying raw stream is in non blocking-mode, and has no data available at the moment.

</div>

<div class="method">

readinto1(b, /)

Read bytes into a pre-allocated, writable `bytes-like object` *b*, using at most one call to the underlying raw stream's `~RawIOBase.read` (or `~RawIOBase.readinto`) method. Return the number of bytes read.

A `BlockingIOError` is raised if the underlying raw stream is in non blocking-mode, and has no data available at the moment.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

write(b, /)

Write the given `bytes-like object`, *b*, and return the number of bytes written (always equal to the length of *b* in bytes, since if the write fails an `OSError` will be raised). Depending on the actual implementation, these bytes may be readily written to the underlying stream, or held in a buffer for performance and latency reasons.

When in non-blocking mode, a `BlockingIOError` is raised if the data needed to be written to the raw stream but it couldn't accept all the data without blocking.

The caller may release or mutate *b* after this method returns, so the implementation should only access *b* during the method call.

</div>

</div>

### Raw file I/O

<div class="FileIO(name, mode='r', closefd=True, opener=None)">

A raw binary stream representing an OS-level file containing bytes data. It inherits from `RawIOBase` and implements its low-level access design. This means `~RawIOBase.write` does not guarantee all bytes are written and `~RawIOBase.read` may read less bytes than requested even when more bytes may be present in the underlying file. To get "write all" and "read at least" behavior, use `binary-io`.

The *name* can be one of two things:

- a character string or `bytes` object representing the path to the file which will be opened. In this case closefd must be `True` (the default) otherwise an error will be raised.
- an integer representing the number of an existing OS-level file descriptor to which the resulting `FileIO` object will give access. When the FileIO object is closed this fd will be closed as well, unless *closefd* is set to `False`.

The *mode* can be `'r'`, `'w'`, `'x'` or `'a'` for reading (default), writing, exclusive creation or appending. The file will be created if it doesn't exist when opened for writing or appending; it will be truncated when opened for writing. `FileExistsError` will be raised if it already exists when opened for creating. Opening a file for creating implies writing, so this mode behaves in a similar way to `'w'`. Add a `'+'` to the mode to allow simultaneous reading and writing.

A custom opener can be used by passing a callable as *opener*. The underlying file descriptor for the file object is then obtained by calling *opener* with (*name*, *flags*). *opener* must return an open file descriptor (passing `os.open` as *opener* results in functionality similar to passing `None`).

The newly created file is `non-inheritable <fd_inheritance>`.

See the `open` built-in function for examples on using the *opener* parameter.

> [!WARNING]
> `FileIO` is a low-level I/O object and members, such as `~RawIOBase.read` and `~RawIOBase.write`, need to have their return values checked explicitly in a retry loop to implement "write all" and "read at least" behavior. High-level I/O objects `binary-io` and `text-io` implement retry behavior.

<div class="versionchanged">

3.3 The *opener* parameter was added. The `'x'` mode was added.

</div>

<div class="versionchanged">

3.4 The file is now non-inheritable.

</div>

`FileIO` provides these data attributes in addition to those from `RawIOBase` and `IOBase`:

<div class="attribute">

mode

The mode as given in the constructor.

</div>

<div class="attribute">

name

The file name. This is the file descriptor of the file when no name is given in the constructor.

</div>

</div>

### Buffered streams

Buffered I/O streams provide a higher-level interface to an I/O device than raw I/O does.

<div class="BytesIO(initial_bytes=b'')">

A binary stream using an in-memory bytes buffer. It inherits from `BufferedIOBase`. The buffer is discarded when the `~IOBase.close` method is called.

The optional argument *initial_bytes* is a `bytes-like object` that contains initial data.

Methods may be used from multiple threads without external locking in `free-threaded builds <free-threaded build>`.

`BytesIO` provides or overrides these methods in addition to those from `BufferedIOBase` and `IOBase`:

<div class="method">

getbuffer()

Return a readable and writable view over the contents of the buffer without copying them. Also, mutating the view will transparently update the contents of the buffer:

    >>> b = io.BytesIO(b"abcdef")
    >>> view = b.getbuffer()
    >>> view[2:4] = b"56"
    >>> b.getvalue()
    b'ab56ef'

> [!NOTE]
> As long as the view exists, the `BytesIO` object cannot be resized. Closing it does not invalidate the view.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.16 The `BytesIO` object can now be closed while the view exists.

</div>

</div>

<div class="method">

getvalue()

Return `bytes` containing the entire contents of the buffer.

</div>

<div class="method">

peek(size=0, /)

Return a copy of the buffer from the current position onwards without advancing the position.

If *size* is less than one or omitted, at most `DEFAULT_BUFFER_SIZE` bytes are returned. Otherwise, at most *size* bytes are returned. Return an empty `bytes` object at EOF.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

read1(size=-1, /)

In `BytesIO`, this is the same as `~BufferedIOBase.read`.

<div class="versionchanged">

3.7 The *size* argument is now optional.

</div>

</div>

<div class="method">

readinto1(b, /)

In `BytesIO`, this is the same as `~BufferedIOBase.readinto`.

<div class="versionadded">

3.5

</div>

</div>

</div>

<div class="BufferedReader(raw, buffer_size=DEFAULT_BUFFER_SIZE)">

A buffered binary stream providing higher-level access to a readable, non seekable `RawIOBase` raw binary stream. It inherits from `BufferedIOBase`.

When reading data from this object, a larger amount of data may be requested from the underlying raw stream, and kept in an internal buffer. The buffered data can then be returned directly on subsequent reads.

The constructor creates a `BufferedReader` for the given readable *raw* stream and *buffer_size*. If *buffer_size* is omitted, `DEFAULT_BUFFER_SIZE` is used.

`BufferedReader` provides or overrides these methods in addition to those from `BufferedIOBase` and `IOBase`:

<div class="method">

peek(size=0, /)

Return bytes from the stream without advancing the position. The number of bytes returned may be less or more than requested. If the underlying raw stream is non-blocking and the operation would block, returns empty bytes.

</div>

<div class="method">

read(size=-1, /)

In `BufferedReader` this is the same as `io.BufferedIOBase.read`

</div>

<div class="method">

read1(size=-1, /)

In `BufferedReader` this is the same as `io.BufferedIOBase.read1`

<div class="versionchanged">

3.7 The *size* argument is now optional.

</div>

</div>

</div>

<div class="BufferedWriter(raw, buffer_size=DEFAULT_BUFFER_SIZE)">

A buffered binary stream providing higher-level access to a writeable, non seekable `RawIOBase` raw binary stream. It inherits from `BufferedIOBase`.

When writing to this object, data is normally placed into an internal buffer. The buffer will be written out to the underlying `RawIOBase` object under various conditions, including:

- when the buffer gets too small for all pending data;
- when `flush` is called;
- when a `~IOBase.seek` is requested (for `BufferedRandom` objects);
- when the `BufferedWriter` object is closed or destroyed.

The constructor creates a `BufferedWriter` for the given writeable *raw* stream. If the *buffer_size* is not given, it defaults to `DEFAULT_BUFFER_SIZE`.

`BufferedWriter` provides or overrides these methods in addition to those from `BufferedIOBase` and `IOBase`:

<div class="method">

flush()

Force bytes held in the buffer into the raw stream. A `BlockingIOError` should be raised if the raw stream blocks.

</div>

<div class="method">

write(b, /)

Write the `bytes-like object`, *b*, and return the number of bytes written. When in non-blocking mode, a `BlockingIOError` with `BlockingIOError.characters_written` set is raised if the buffer needs to be written out but the raw stream blocks.

</div>

</div>

<div class="BufferedRandom(raw, buffer_size=DEFAULT_BUFFER_SIZE)">

A buffered binary stream implementing `BufferedIOBase` interfaces providing higher-level access to a seekable `RawIOBase` raw binary stream.

The constructor creates a reader and writer for a seekable raw stream, given in the first argument. If the *buffer_size* is omitted it defaults to `DEFAULT_BUFFER_SIZE`.

`BufferedRandom` is capable of anything `BufferedReader` or `BufferedWriter` can do. In addition, `~IOBase.seek` and `~IOBase.tell` are guaranteed to be implemented.

</div>

<div class="BufferedRWPair(reader, writer, buffer_size=DEFAULT_BUFFER_SIZE, /)">

A buffered binary stream providing higher-level access to two non seekable `RawIOBase` raw binary streams---one readable, the other writeable. It inherits from `BufferedIOBase`.

*reader* and *writer* are `RawIOBase` objects that are readable and writeable respectively. If the *buffer_size* is omitted it defaults to `DEFAULT_BUFFER_SIZE`.

`BufferedRWPair` implements all of `BufferedIOBase`'s methods except for `~BufferedIOBase.detach`, which raises `UnsupportedOperation`.

> [!WARNING]
> `BufferedRWPair` does not attempt to synchronize accesses to its underlying raw streams. You should not pass it the same object as reader and writer; use `BufferedRandom` instead.

</div>

### Text I/O

<div class="TextIOBase">

Base class for text streams. This class provides a character and line based interface to stream I/O. It inherits from `IOBase`.

`TextIOBase` provides or overrides these data attributes and methods in addition to those from `IOBase`:

<div class="attribute">

encoding

The name of the encoding used to decode the stream's bytes into strings, and to encode strings into bytes.

</div>

<div class="attribute">

errors

The error setting of the decoder or encoder.

</div>

<div class="attribute">

newlines

A string, a tuple of strings, or `None`, indicating the newlines translated so far. Depending on the implementation and the initial constructor flags, this may not be available.

</div>

<div class="attribute">

buffer

The underlying binary buffer (a `BufferedIOBase` or `RawIOBase` instance) that `TextIOBase` deals with. This is not part of the `TextIOBase` API and may not exist in some implementations.

</div>

<div class="method">

detach()

Separate the underlying binary buffer from the `TextIOBase` and return it.

After the underlying buffer has been detached, the `TextIOBase` is in an unusable state.

Some `TextIOBase` implementations, like `StringIO`, may not have the concept of an underlying buffer and calling this method will raise `UnsupportedOperation`.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

read(size=-1, /)

Read and return at most *size* characters from the stream as a single `str`. If *size* is negative or `None`, reads until EOF.

</div>

<div class="method">

readline(size=-1, /)

Read until newline or EOF and return a single `str`. If the stream is already at EOF, an empty string is returned.

If *size* is specified, at most *size* characters will be read.

</div>

<div class="method">

seek(offset, whence=SEEK_SET, /)

Change the stream position to the given *offset*. Behaviour depends on the *whence* parameter. The default value for *whence* is `!SEEK_SET`.

- `!SEEK_SET` or `0`: seek from the start of the stream (the default); *offset* must either be a number returned by `TextIOBase.tell`, or zero. Any other *offset* value produces undefined behaviour.
- `!SEEK_CUR` or `1`: "seek" to the current position; *offset* must be zero, which is a no-operation (all other values are unsupported).
- `!SEEK_END` or `2`: seek to the end of the stream; *offset* must be zero (all other values are unsupported).

Return the new absolute position as an opaque number.

<div class="versionadded">

3.1 The `!SEEK_*` constants.

</div>

</div>

<div class="method">

tell()

Return the current stream position as an opaque number. The number does not usually represent a number of bytes in the underlying binary storage.

</div>

<div class="method">

write(s, /)

Write the string *s* to the stream and return the number of characters written.

</div>

</div>

<div class="TextIOWrapper(buffer, encoding=None, errors=None, newline=None, \ line_buffering=False, write_through=False)">

A buffered text stream providing higher-level access to a `BufferedIOBase` buffered binary stream. It inherits from `TextIOBase`.

*encoding* gives the name of the encoding that the stream will be decoded or encoded with. This defaults to UTF-8; if `UTF-8 Mode <utf8-mode>` is disabled, it defaults to `locale.getencoding`. `encoding="locale"` can be used to specify the current locale's encoding explicitly. See `io-text-encoding` for more information.

*errors* is an optional string that specifies how encoding and decoding errors are to be handled. Pass `'strict'` to raise a `ValueError` exception if there is an encoding error (the default of `None` has the same effect), or pass `'ignore'` to ignore errors. (Note that ignoring encoding errors can lead to data loss.) `'replace'` causes a replacement marker (such as `'?'`) to be inserted where there is malformed data. `'backslashreplace'` causes malformed data to be replaced by a backslashed escape sequence. When writing, `'xmlcharrefreplace'` (replace with the appropriate XML character reference) or `'namereplace'` (replace with `\N{...}` escape sequences) can be used. Any other error handling name that has been registered with `codecs.register_error` is also valid.

<div class="index">

single: universal newlines; io.TextIOWrapper class

</div>

*newline* controls how line endings are handled. It can be `None`, `''`, `'\n'`, `'\r'`, and `'\r\n'`. It works as follows:

- When reading input from the stream, if *newline* is `None`, `universal newlines` mode is enabled. Lines in the input can end in `'\n'`, `'\r'`, or `'\r\n'`, and these are translated into `'\n'` before being returned to the caller. If *newline* is `''`, universal newlines mode is enabled, but line endings are returned to the caller untranslated. If *newline* has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated.
- When writing output to the stream, if *newline* is `None`, any `'\n'` characters written are translated to the system default line separator, `os.linesep`. If *newline* is `''` or `'\n'`, no translation takes place. If *newline* is any of the other legal values, any `'\n'` characters written are translated to the given string.

If *line_buffering* is `True`, `~IOBase.flush` is implied when a call to write contains a newline character or a carriage return.

If *write_through* is `True`, calls to `~BufferedIOBase.write` are guaranteed not to be buffered: any data written on the `TextIOWrapper` object is immediately handled to its underlying binary *buffer*.

<div class="versionchanged">

3.3 The *write_through* argument has been added.

</div>

<div class="versionchanged">

3.3 The default *encoding* is now `locale.getpreferredencoding(False)` instead of `locale.getpreferredencoding()`. Don't change temporary the locale encoding using `locale.setlocale`, use the current locale encoding instead of the user preferred encoding.

</div>

<div class="versionchanged">

3.10 The *encoding* argument now supports the `"locale"` dummy encoding name.

</div>

> [!NOTE]
> When the underlying raw stream is non-blocking, a `BlockingIOError` may be raised if a read operation cannot be completed immediately.

`TextIOWrapper` provides these data attributes and methods in addition to those from `TextIOBase` and `IOBase`:

<div class="attribute">

line_buffering

Whether line buffering is enabled.

</div>

<div class="attribute">

write_through

Whether writes are passed immediately to the underlying binary buffer.

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

reconfigure(\*, encoding=None, errors=None, newline=None, line_buffering=None, write_through=None)

Reconfigure this text stream using new settings for *encoding*, *errors*, *newline*, *line_buffering* and *write_through*.

Parameters not specified keep current settings, except `errors='strict'` is used when *encoding* is specified but *errors* is not specified.

It is not possible to change the encoding or newline if some data has already been read from the stream. On the other hand, changing encoding after write is possible.

This method does an implicit stream flush before setting the new parameters.

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.11 The method supports `encoding="locale"` option.

</div>

</div>

<div class="method">

seek(cookie, whence=os.SEEK_SET, /)

Set the stream position. Return the new stream position as an `int`.

Four operations are supported, given by the following argument combinations:

- `seek(0, SEEK_SET)`: Rewind to the start of the stream.
- `seek(cookie, SEEK_SET)`: Restore a previous position; *cookie* **must be** a number returned by `tell`.
- `seek(0, SEEK_END)`: Fast-forward to the end of the stream.
- `seek(0, SEEK_CUR)`: Leave the current stream position unchanged.

Any other argument combinations are invalid, and may raise exceptions.

<div class="seealso">

`os.SEEK_SET`, `os.SEEK_CUR`, and `os.SEEK_END`.

</div>

</div>

<div class="method">

tell()

Return the stream position as an opaque number. The return value of `!tell` can be given as input to `seek`, to restore a previous stream position.

</div>

</div>

<div class="StringIO(initial_value='', newline='\n')">

A text stream using an in-memory text buffer. It inherits from `TextIOBase`.

The text buffer is discarded when the `~IOBase.close` method is called.

The initial value of the buffer can be set by providing *initial_value*. If newline translation is enabled, newlines will be encoded as if by `~TextIOBase.write`. The stream is positioned at the start of the buffer which emulates opening an existing file in a `w+` mode, making it ready for an immediate write from the beginning or for a write that would overwrite the initial value. To emulate opening a file in an `a+` mode ready for appending, use `f.seek(0, io.SEEK_END)` to reposition the stream at the end of the buffer.

The *newline* argument works like that of `TextIOWrapper`, except that when writing output to the stream, if *newline* is `None`, newlines are written as `\n` on all platforms.

`StringIO` provides this method in addition to those from `TextIOBase` and `IOBase`:

<div class="method">

getvalue()

Return a `str` containing the entire contents of the buffer. Newlines are decoded as if by `~TextIOBase.read`, although the stream position is not changed.

</div>

Example usage:

    import io

    output = io.StringIO()
    output.write('First line.\n')
    print('Second line.', file=output)

    # Retrieve file contents -- this will be
    # 'First line.\nSecond line.\n'
    contents = output.getvalue()

    # Close object and discard memory buffer --
    # .getvalue() will now raise an exception.
    output.close()

</div>

<div class="index">

single: universal newlines; io.IncrementalNewlineDecoder class

</div>

<div class="IncrementalNewlineDecoder">

A helper codec that decodes newlines for `universal newlines` mode. It inherits from `codecs.IncrementalDecoder`.

</div>

## Static typing

The following protocols can be used for annotating function and method arguments for simple stream reading or writing operations. They are decorated with `typing.runtime_checkable`.

<div class="Reader[T]">

Generic protocol for reading from a file or other input stream. `T` will usually be `str` or `bytes`, but can be any type that is read from the stream.

<div class="versionadded">

3.14

</div>

<div class="method">

read() read(size, /)

Read data from the input stream and return it. If *size* is specified, it should be an integer, and at most *size* items (bytes/characters) will be read.

</div>

For example:

    def read_it(reader: Reader[str]):
        data = reader.read(11)
        assert isinstance(data, str)

</div>

<div class="Writer[T]">

Generic protocol for writing to a file or other output stream. `T` will usually be `str` or `bytes`, but can be any type that can be written to the stream.

<div class="versionadded">

3.14

</div>

<div class="method">

write(data, /)

Write *data* to the output stream and return the number of items (bytes/characters) written.

</div>

For example:

    def write_binary(writer: Writer[bytes]):
        writer.write(b"Hello world!\n")

</div>

See `typing-io` for other I/O related protocols and classes that can be used for static type checking.

## Performance

This section discusses the performance of the provided concrete I/O implementations.

### Binary I/O

By reading and writing only large chunks of data even when the user asks for a single byte, buffered I/O hides any inefficiency in calling and executing the operating system's unbuffered I/O routines. The gain depends on the OS and the kind of I/O which is performed. For example, on some modern OSes such as Linux, unbuffered disk I/O can be as fast as buffered I/O. The bottom line, however, is that buffered I/O offers predictable performance regardless of the platform and the backing device. Therefore, it is almost always preferable to use buffered I/O rather than unbuffered I/O for binary data.

### Text I/O

Text I/O over a binary storage (such as a file) is significantly slower than binary I/O over the same storage, because it requires conversions between unicode and binary data using a character codec. This can become noticeable handling huge amounts of text data like large log files. Also, `~TextIOBase.tell` and `~TextIOBase.seek` are both quite slow due to the reconstruction algorithm used.

`StringIO`, however, is a native in-memory unicode container and will exhibit similar speed to `BytesIO`.

### Multi-threading

`FileIO` objects are thread-safe to the extent that the operating system calls (such as `read(2)` under Unix) they wrap are thread-safe too.

Binary buffered objects (instances of `BufferedReader`, `BufferedWriter`, `BufferedRandom` and `BufferedRWPair`) protect their internal structures using a lock; it is therefore safe to call them from multiple threads at once.

`TextIOWrapper` objects are not thread-safe.

### Reentrancy

Binary buffered objects (instances of `BufferedReader`, `BufferedWriter`, `BufferedRandom` and `BufferedRWPair`) are not reentrant. While reentrant calls will not happen in normal situations, they can arise from doing I/O in a `signal` handler. If a thread tries to re-enter a buffered object which it is already accessing, a `RuntimeError` is raised. Note this doesn't prohibit a different thread from entering the buffered object.

The above implicitly extends to text files, since the `open` function will wrap a buffered object inside a `TextIOWrapper`. This includes standard streams and therefore affects the built-in `print` function as well.

# `!ipaddress` --- IPv4/IPv6 manipulation library

<div class="module" data-synopsis="IPv4/IPv6 manipulation library.">

ipaddress

</div>

**Source code:** `Lib/ipaddress.py`

------------------------------------------------------------------------

`!ipaddress` provides the capabilities to create, manipulate and operate on IPv4 and IPv6 addresses and networks.

The functions and classes in this module make it straightforward to handle various tasks related to IP addresses, including checking whether or not two hosts are on the same subnet, iterating over all hosts in a particular subnet, checking whether or not a string represents a valid IP address or network definition, and so on.

This is the full module API reference—for an overview and introduction, see `ipaddress-howto`.

<div class="versionadded">

3.3

</div>

<div class="testsetup">

import ipaddress from ipaddress import ( ip_network, IPv4Address, IPv4Interface, IPv4Network, )

</div>

## Convenience factory functions

The `!ipaddress` module provides factory functions to conveniently create IP addresses, networks and interfaces:

<div class="function">

ip_address(address)

Return an `IPv4Address` or `IPv6Address` object depending on the IP address passed as argument. Either IPv4 or IPv6 addresses may be supplied; integers less than `2**32` will be considered to be IPv4 by default. A `ValueError` is raised if *address* does not represent a valid IPv4 or IPv6 address.

\>\>\> ipaddress.ip_address('192.168.0.1') IPv4Address('192.168.0.1') \>\>\> ipaddress.ip_address('2001:db8::') IPv6Address('2001:db8::')

</div>

<div class="function">

ip_network(address, strict=True)

Return an `IPv4Network` or `IPv6Network` object depending on the IP address passed as argument. *address* is a string or integer representing the IP network. Either IPv4 or IPv6 networks may be supplied; integers less than `2**32` will be considered to be IPv4 by default. *strict* is passed to `IPv4Network` or `IPv6Network` constructor. A `ValueError` is raised if *address* does not represent a valid IPv4 or IPv6 address, or if the network has host bits set.

\>\>\> ipaddress.ip_network('192.168.0.0/28') IPv4Network('192.168.0.0/28')

</div>

<div class="function">

ip_interface(address)

Return an `IPv4Interface` or `IPv6Interface` object depending on the IP address passed as argument. *address* is a string or integer representing the IP address. Either IPv4 or IPv6 addresses may be supplied; integers less than `2**32` will be considered to be IPv4 by default. A `ValueError` is raised if *address* does not represent a valid IPv4 or IPv6 address.

</div>

One downside of these convenience functions is that the need to handle both IPv4 and IPv6 formats means that error messages provide minimal information on the precise error, as the functions don't know whether the IPv4 or IPv6 format was intended. More detailed error reporting can be obtained by calling the appropriate version specific class constructors directly.

## IP Addresses

### Address objects

The `IPv4Address` and `IPv6Address` objects share a lot of common attributes. Some attributes that are only meaningful for IPv6 addresses are also implemented by `IPv4Address` objects, in order to make it easier to write code that handles both IP versions correctly. Address objects are `hashable`, so they can be used as keys in dictionaries.

<div class="IPv4Address(address)">

Construct an IPv4 address. An `AddressValueError` is raised if *address* is not a valid IPv4 address.

The following constitutes a valid IPv4 address:

1.  A string in decimal-dot notation, consisting of four decimal integers in the inclusive range 0--255, separated by dots (e.g. `192.168.0.1`). Each integer represents an octet (byte) in the address. Leading zeroes are not tolerated to prevent confusion with octal notation.
2.  An integer that fits into 32 bits.
3.  An integer packed into a `bytes` object of length 4 (most significant octet first).

\>\>\> ipaddress.IPv4Address('192.168.0.1') IPv4Address('192.168.0.1') \>\>\> ipaddress.IPv4Address(3232235521) IPv4Address('192.168.0.1') \>\>\> ipaddress.IPv4Address(b'xC0xA8x00x01') IPv4Address('192.168.0.1')

<div class="versionchanged">

3.8

Leading zeros are tolerated, even in ambiguous cases that look like octal notation.

</div>

<div class="versionchanged">

3.9.5

Leading zeros are no longer tolerated and are treated as an error. IPv4 address strings are now parsed as strict as glibc `~socket.inet_pton`.

</div>

<div class="attribute">

version

The appropriate version number: `4` for IPv4, `6` for IPv6.

<div class="versionchanged">

3.14

Made available on the class.

</div>

</div>

<div class="attribute">

max_prefixlen

The total number of bits in the address representation for this version: `32` for IPv4, `128` for IPv6.

The prefix defines the number of leading bits in an address that are compared to determine whether or not an address is part of a network.

<div class="versionchanged">

3.14

Made available on the class.

</div>

</div>

<div class="attribute">

compressed

</div>

<div class="attribute">

exploded

The string representation in dotted decimal notation. Leading zeroes are never included in the representation.

As IPv4 does not define a shorthand notation for addresses with octets set to zero, these two attributes are always the same as `str(addr)` for IPv4 addresses. Exposing these attributes makes it easier to write display code that can handle both IPv4 and IPv6 addresses.

</div>

<div class="attribute">

packed

The binary representation of this address - a `bytes` object of the appropriate length (most significant octet first). This is 4 bytes for IPv4 and 16 bytes for IPv6.

</div>

<div class="attribute">

reverse_pointer

The name of the reverse DNS PTR record for the IP address, e.g.:

    >>> ipaddress.ip_address("127.0.0.1").reverse_pointer
    '1.0.0.127.in-addr.arpa'
    >>> ipaddress.ip_address("2001:db8::1").reverse_pointer
    '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.b.d.0.1.0.0.2.ip6.arpa'

This is the name that could be used for performing a PTR lookup, not the resolved hostname itself.

<div class="versionadded">

3.5

</div>

</div>

<div class="attribute">

is_multicast

`True` if the address is reserved for multicast use. See [RFC 3171](http://www.faqs.org/rfcs/rfc3171.html "RFC 3171") (for IPv4) or [RFC 2373](http://www.faqs.org/rfcs/rfc2373.html "RFC 2373") (for IPv6).

</div>

<div class="attribute">

is_private

`True` if the address is defined as not globally reachable by [iana-ipv4-special-registry](https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml) (for IPv4) or [iana-ipv6-special-registry](https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml) (for IPv6) with the following exceptions:

- `is_private` is `False` for the shared address space (`100.64.0.0/10`)

- For IPv4-mapped IPv6-addresses the `is_private` value is determined by the semantics of the underlying IPv4 addresses and the following condition holds (see `IPv6Address.ipv4_mapped`):

      address.is_private == address.ipv4_mapped.is_private

`is_private` has value opposite to `is_global`, except for the shared address space (`100.64.0.0/10` range) where they are both `False`.

<div class="versionchanged">

3.13

Fixed some false positives and false negatives.

- `192.0.0.0/24` is considered private with the exception of `192.0.0.9/32` and `192.0.0.10/32` (previously: only the `192.0.0.0/29` sub-range was considered private).
- `64:ff9b:1::/48` is considered private.
- `2002::/16` is considered private.
- There are exceptions within `2001::/23` (otherwise considered private): `2001:1::1/128`, `2001:1::2/128`, `2001:3::/32`, `2001:4:112::/48`, `2001:20::/28`, `2001:30::/28`. The exceptions are not considered private.

</div>

</div>

<div class="attribute">

is_global

`True` if the address is defined as globally reachable by [iana-ipv4-special-registry](https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml) (for IPv4) or [iana-ipv6-special-registry](https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml) (for IPv6) with the following exception:

For IPv4-mapped IPv6-addresses the `is_private` value is determined by the semantics of the underlying IPv4 addresses and the following condition holds (see `IPv6Address.ipv4_mapped`):

    address.is_global == address.ipv4_mapped.is_global

`is_global` has value opposite to `is_private`, except for the shared address space (`100.64.0.0/10` range) where they are both `False`.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.13

Fixed some false positives and false negatives, see `is_private` for details.

</div>

</div>

<div class="attribute">

is_unspecified

`True` if the address is unspecified. See [RFC 5735](http://www.faqs.org/rfcs/rfc5735.html "RFC 5735") (for IPv4) or [RFC 2373](http://www.faqs.org/rfcs/rfc2373.html "RFC 2373") (for IPv6).

</div>

<div class="attribute">

is_reserved

`True` if the address is noted as reserved by the IETF. For IPv4, this is only `240.0.0.0/4`, the `Reserved` address block. For IPv6, this is all addresses [allocated](https://www.iana.org/assignments/ipv6-address-space/ipv6-address-space.xhtml) as `Reserved by IETF` for future use.

> [!NOTE]
> For IPv4, `is_reserved` is not related to the address block value of the `Reserved-by-Protocol` column in [iana-ipv4-special-registry](https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml).

> [!CAUTION]
> For IPv6, `fec0::/10` a former Site-Local scoped address prefix is currently excluded from that list (see `~IPv6Address.is_site_local` & `3879`).

</div>

<div class="attribute">

is_loopback

`True` if this is a loopback address. See [RFC 3330](http://www.faqs.org/rfcs/rfc3330.html "RFC 3330") (for IPv4) or [RFC 2373](http://www.faqs.org/rfcs/rfc2373.html "RFC 2373") (for IPv6).

</div>

<div class="attribute">

is_link_local

`True` if the address is reserved for link-local usage. See [RFC 3927](http://www.faqs.org/rfcs/rfc3927.html "RFC 3927").

</div>

<div class="attribute">

ipv6_mapped

`IPv6Address` object representing the IPv4-mapped IPv6 address. See [RFC 4291](http://www.faqs.org/rfcs/rfc4291.html "RFC 4291").

<div class="versionadded">

3.13

</div>

</div>

</div>

<div class="method">

IPv4Address.\_\_format\_\_(fmt)

Returns a string representation of the IP address, controlled by an explicit format string. *fmt* can be one of the following: `'s'`, the default option, equivalent to `str`, `'b'` for a zero-padded binary string, `'X'` or `'x'` for an uppercase or lowercase hexadecimal representation, or `'n'`, which is equivalent to `'b'` for IPv4 addresses and `'x'` for IPv6. For binary and hexadecimal representations, the form specifier `'#'` and the grouping option `'_'` are available. `__format__` is used by `format`, `str.format` and f-strings.

> \>\>\> format(ipaddress.IPv4Address('192.168.0.1')) '192.168.0.1' \>\>\> '{:#b}'.format(ipaddress.IPv4Address('192.168.0.1')) '0b11000000101010000000000000000001' \>\>\> f'{ipaddress.IPv6Address("2001:db8::1000"):s}' '2001:db8::1000' \>\>\> format(ipaddress.IPv6Address('2001:db8::1000'), '\_X') '2001_0DB8_0000_0000_0000_0000_0000_1000' \>\>\> '{:#\_n}'.format(ipaddress.IPv6Address('2001:db8::1000')) '0x2001_0db8_0000_0000_0000_0000_0000_1000'

<div class="versionadded">

3.9

</div>

</div>

<div class="IPv6Address(address)">

Construct an IPv6 address. An `AddressValueError` is raised if *address* is not a valid IPv6 address.

The following constitutes a valid IPv6 address:

1.  A string consisting of eight groups of four hexadecimal digits, each group representing 16 bits. The groups are separated by colons. This describes an *exploded* (longhand) notation. The string can also be *compressed* (shorthand notation) by various means. See [RFC 4291](http://www.faqs.org/rfcs/rfc4291.html "RFC 4291") for details. For example, `"0000:0000:0000:0000:0000:0abc:0007:0def"` can be compressed to `"::abc:7:def"`.

    Optionally, the string may also have a scope zone ID, expressed with a suffix `%scope_id`. If present, the scope ID must be non-empty, and may not contain `%`. See [RFC 4007](http://www.faqs.org/rfcs/rfc4007.html "RFC 4007") for details. For example, `fe80::1234%1` might identify address `fe80::1234` on the first link of the node.

2.  An integer that fits into 128 bits.

3.  An integer packed into a `bytes` object of length 16, big-endian.

\>\>\> ipaddress.IPv6Address('2001:db8::1000') IPv6Address('2001:db8::1000') \>\>\> ipaddress.IPv6Address('ff02::5678%1') IPv6Address('ff02::5678%1')

<div class="attribute">

compressed

</div>

The short form of the address representation, with leading zeroes in groups omitted and the longest sequence of groups consisting entirely of zeroes collapsed to a single empty group.

This is also the value returned by `str(addr)` for IPv6 addresses.

<div class="attribute">

exploded

</div>

The long form of the address representation, with all leading zeroes and groups consisting entirely of zeroes included.

For the following attributes and methods, see the corresponding documentation of the `IPv4Address` class:

<div class="attribute">

packed

</div>

<div class="attribute">

reverse_pointer

</div>

<div class="attribute">

version

</div>

<div class="attribute">

max_prefixlen

</div>

<div class="attribute">

is_multicast

</div>

<div class="attribute">

is_private

</div>

<div class="attribute">

is_global

<div class="versionadded">

3.4

</div>

</div>

<div class="attribute">

is_unspecified

</div>

<div class="attribute">

is_reserved

</div>

<div class="attribute">

is_loopback

</div>

<div class="attribute">

is_link_local

</div>

<div class="attribute">

is_site_local

`True` if the address is reserved for site-local usage. Note that the site-local address space has been deprecated by [RFC 3879](http://www.faqs.org/rfcs/rfc3879.html "RFC 3879"). Use `~IPv4Address.is_private` to test if this address is in the space of unique local addresses as defined by [RFC 4193](http://www.faqs.org/rfcs/rfc4193.html "RFC 4193").

</div>

<div class="attribute">

ipv4_mapped

For addresses that appear to be IPv4 mapped addresses in the range `::FFFF:0:0/96` as defined by [RFC 4291](http://www.faqs.org/rfcs/rfc4291.html "RFC 4291"), this property reports the embedded IPv4 address. For any other address, this property will be `None`.

</div>

<div class="attribute">

scope_id

For scoped addresses as defined by [RFC 4007](http://www.faqs.org/rfcs/rfc4007.html "RFC 4007"), this property identifies the particular zone of the address's scope that the address belongs to, as a string. When no scope zone is specified, this property will be `None`.

</div>

<div class="attribute">

sixtofour

For addresses that appear to be 6to4 addresses (starting with `2002::/16`) as defined by [RFC 3056](http://www.faqs.org/rfcs/rfc3056.html "RFC 3056"), this property will report the embedded IPv4 address. For any other address, this property will be `None`.

</div>

<div class="attribute">

teredo

For addresses that appear to be Teredo addresses (starting with `2001::/32`) as defined by [RFC 4380](http://www.faqs.org/rfcs/rfc4380.html "RFC 4380"), this property will report the embedded `(server, client)` IP address pair. For any other address, this property will be `None`.

</div>

</div>

<div class="method">

IPv6Address.\_\_format\_\_(fmt)

Refer to the corresponding method documentation in `IPv4Address`.

<div class="versionadded">

3.9

</div>

</div>

### Conversion to Strings and Integers

To interoperate with networking interfaces such as the socket module, addresses must be converted to strings or integers. This is handled using the `str` and `int` builtin functions:

    >>> str(ipaddress.IPv4Address('192.168.0.1'))
    '192.168.0.1'
    >>> int(ipaddress.IPv4Address('192.168.0.1'))
    3232235521
    >>> str(ipaddress.IPv6Address('::1'))
    '::1'
    >>> int(ipaddress.IPv6Address('::1'))
    1

Note that IPv6 scoped addresses are converted to integers without scope zone ID.

### Operators

Address objects support some operators. Unless stated otherwise, operators can only be applied between compatible objects (i.e. IPv4 with IPv4, IPv6 with IPv6).

#### Comparison operators

Address objects can be compared with the usual set of comparison operators. Same IPv6 addresses with different scope zone IDs are not equal. Some examples:

    >>> IPv4Address('127.0.0.2') > IPv4Address('127.0.0.1')
    True
    >>> IPv4Address('127.0.0.2') == IPv4Address('127.0.0.1')
    False
    >>> IPv4Address('127.0.0.2') != IPv4Address('127.0.0.1')
    True
    >>> IPv6Address('fe80::1234') == IPv6Address('fe80::1234%1')
    False
    >>> IPv6Address('fe80::1234%1') != IPv6Address('fe80::1234%2')
    True

#### Arithmetic operators

Integers can be added to or subtracted from address objects. Some examples:

    >>> IPv4Address('127.0.0.2') + 3
    IPv4Address('127.0.0.5')
    >>> IPv4Address('127.0.0.2') - 3
    IPv4Address('126.255.255.255')
    >>> IPv4Address('255.255.255.255') + 1
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ipaddress.AddressValueError: 4294967296 (>= 2**32) is not permitted as an IPv4 address

## IP Network definitions

The `IPv4Network` and `IPv6Network` objects provide a mechanism for defining and inspecting IP network definitions. A network definition consists of a *mask* and a *network address*, and as such defines a range of IP addresses that equal the network address when masked (binary AND) with the mask. For example, a network definition with the mask `255.255.255.0` and the network address `192.168.1.0` consists of IP addresses in the inclusive range `192.168.1.0` to `192.168.1.255`.

### Prefix, net mask and host mask

There are several equivalent ways to specify IP network masks. A *prefix* `/<nbits>` is a notation that denotes how many high-order bits are set in the network mask. A *net mask* is an IP address with some number of high-order bits set. Thus the prefix `/24` is equivalent to the net mask `255.255.255.0` in IPv4, or `ffff:ff00::` in IPv6. In addition, a *host mask* is the logical inverse of a *net mask*, and is sometimes used (for example in Cisco access control lists) to denote a network mask. The host mask equivalent to `/24` in IPv4 is `0.0.0.255`.

### Network objects

All attributes implemented by address objects are implemented by network objects as well. In addition, network objects implement additional attributes. All of these are common between `IPv4Network` and `IPv6Network`, so to avoid duplication they are only documented for `IPv4Network`. Network objects are `hashable`, so they can be used as keys in dictionaries.

<div class="IPv4Network(address, strict=True)">

Construct an IPv4 network definition. *address* can be one of the following:

1.  A string consisting of an IP address and an optional mask, separated by a slash (`/`). The IP address is the network address, and the mask can be either a single number, which means it's a *prefix*, or a string representation of an IPv4 address. If it's the latter, the mask is interpreted as a *net mask* if it starts with a non-zero field, or as a *host mask* if it starts with a zero field, with the single exception of an all-zero mask which is treated as a *net mask*. If no mask is provided, it's considered to be `/32`.

    For example, the following *address* specifications are equivalent: `192.168.1.0/24`, `192.168.1.0/255.255.255.0` and `192.168.1.0/0.0.0.255`.

2.  An integer that fits into 32 bits. This is equivalent to a single-address network, with the network address being *address* and the mask being `/32`.

3.  An integer packed into a `bytes` object of length 4, big-endian. The interpretation is similar to an integer *address*.

4.  A two-tuple of an address description and a netmask, where the address description is either a string, a 32-bit integer, a 4-byte packed integer, or an existing `IPv4Address` object; and the netmask is either an integer representing the prefix length (e.g. `24`) or a string representing the prefix mask (e.g. `255.255.255.0`).

An `AddressValueError` is raised if *address* is not a valid IPv4 address. A `NetmaskValueError` is raised if the mask is not valid for an IPv4 address.

If *strict* is `True` and host bits are set in the supplied address, then `ValueError` is raised. Otherwise, the host bits are masked out to determine the appropriate network address.

Unless stated otherwise, all network methods accepting other network/address objects will raise `TypeError` if the argument's IP version is incompatible to `self`.

<div class="versionchanged">

3.5

Added the two-tuple form for the *address* constructor parameter.

</div>

<div class="attribute">

version

</div>

<div class="attribute">

max_prefixlen

Refer to the corresponding attribute documentation in `IPv4Address`.

</div>

<div class="attribute">

is_multicast

</div>

<div class="attribute">

is_private

</div>

<div class="attribute">

is_unspecified

</div>

<div class="attribute">

is_reserved

</div>

<div class="attribute">

is_loopback

</div>

<div class="attribute">

is_link_local

These attributes are true for the network as a whole if they are true for both the network address and the broadcast address.

</div>

<div class="attribute">

network_address

The network address for the network. The network address and the prefix length together uniquely define a network.

</div>

<div class="attribute">

broadcast_address

The broadcast address for the network. Packets sent to the broadcast address should be received by every host on the network.

</div>

<div class="attribute">

hostmask

The host mask, as an `IPv4Address` object.

</div>

<div class="attribute">

netmask

The net mask, as an `IPv4Address` object.

</div>

<div class="attribute">

with_prefixlen

</div>

<div class="attribute">

compressed

</div>

<div class="attribute">

exploded

A string representation of the network, with the mask in prefix notation.

`with_prefixlen` and `compressed` are always the same as `str(network)`. `exploded` uses the exploded form the network address.

</div>

<div class="attribute">

with_netmask

A string representation of the network, with the mask in net mask notation.

</div>

<div class="attribute">

with_hostmask

A string representation of the network, with the mask in host mask notation.

</div>

<div class="attribute">

num_addresses

The total number of addresses in the network.

</div>

<div class="attribute">

prefixlen

Length of the network prefix, in bits.

</div>

<div class="method">

hosts()

Returns an iterator over the usable hosts in the network. The usable hosts are all the IP addresses that belong to the network, except the network address itself and the network broadcast address. For networks with a mask length of 31, the network address and network broadcast address are also included in the result. Networks with a mask of 32 will return a list containing the single host address.

> \>\>\> list(ip_network('192.0.2.0/29').hosts()) \#doctest: +NORMALIZE_WHITESPACE \[IPv4Address('192.0.2.1'), IPv4Address('192.0.2.2'), IPv4Address('192.0.2.3'), IPv4Address('192.0.2.4'), IPv4Address('192.0.2.5'), IPv4Address('192.0.2.6')\] \>\>\> list(ip_network('192.0.2.0/31').hosts()) \[IPv4Address('192.0.2.0'), IPv4Address('192.0.2.1')\] \>\>\> list(ip_network('192.0.2.1/32').hosts()) \[IPv4Address('192.0.2.1')\]

</div>

<div class="method">

overlaps(other)

`True` if this network is partly or wholly contained in *other* or *other* is wholly contained in this network.

</div>

<div class="method">

address_exclude(network)

Computes the network definitions resulting from removing the given *network* from this one. Returns an iterator of network objects. Raises `ValueError` if *network* is not completely contained in this network.

> \>\>\> n1 = ip_network('192.0.2.0/28') \>\>\> n2 = ip_network('192.0.2.1/32') \>\>\> list(n1.address_exclude(n2)) \#doctest: +NORMALIZE_WHITESPACE \[IPv4Network('192.0.2.8/29'), IPv4Network('192.0.2.4/30'), IPv4Network('192.0.2.2/31'), IPv4Network('192.0.2.0/32')\]

</div>

<div class="method">

subnets(prefixlen_diff=1, new_prefix=None)

The subnets that join to make the current network definition, depending on the argument values. *prefixlen_diff* is the amount our prefix length should be increased by. *new_prefix* is the desired new prefix of the subnets; it must be larger than our prefix. One and only one of *prefixlen_diff* and *new_prefix* must be set. Returns an iterator of network objects.

> \>\>\> list(ip_network('192.0.2.0/24').subnets()) \[IPv4Network('192.0.2.0/25'), IPv4Network('192.0.2.128/25')\] \>\>\> list(ip_network('192.0.2.0/24').subnets(prefixlen_diff=2)) \#doctest: +NORMALIZE_WHITESPACE \[IPv4Network('192.0.2.0/26'), IPv4Network('192.0.2.64/26'), IPv4Network('192.0.2.128/26'), IPv4Network('192.0.2.192/26')\] \>\>\> list(ip_network('192.0.2.0/24').subnets(new_prefix=26)) \#doctest: +NORMALIZE_WHITESPACE \[IPv4Network('192.0.2.0/26'), IPv4Network('192.0.2.64/26'), IPv4Network('192.0.2.128/26'), IPv4Network('192.0.2.192/26')\] \>\>\> list(ip_network('192.0.2.0/24').subnets(new_prefix=23)) Traceback (most recent call last): File "\<stdin\>", line 1, in \<module\> raise ValueError('new prefix must be longer') ValueError: new prefix must be longer \>\>\> list(ip_network('192.0.2.0/24').subnets(new_prefix=25)) \[IPv4Network('192.0.2.0/25'), IPv4Network('192.0.2.128/25')\]

</div>

<div class="method">

supernet(prefixlen_diff=1, new_prefix=None)

The supernet containing this network definition, depending on the argument values. *prefixlen_diff* is the amount our prefix length should be decreased by. *new_prefix* is the desired new prefix of the supernet; it must be smaller than our prefix. One and only one of *prefixlen_diff* and *new_prefix* must be set. Returns a single network object.

> \>\>\> ip_network('192.0.2.0/24').supernet() IPv4Network('192.0.2.0/23') \>\>\> ip_network('192.0.2.0/24').supernet(prefixlen_diff=2) IPv4Network('192.0.0.0/22') \>\>\> ip_network('192.0.2.0/24').supernet(new_prefix=20) IPv4Network('192.0.0.0/20')

</div>

<div class="method">

subnet_of(other)

Return `True` if this network is a subnet of *other*.

> \>\>\> a = ip_network('192.168.1.0/24') \>\>\> b = ip_network('192.168.1.128/30') \>\>\> b.subnet_of(a) True

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

supernet_of(other)

Return `True` if this network is a supernet of *other*.

> \>\>\> a = ip_network('192.168.1.0/24') \>\>\> b = ip_network('192.168.1.128/30') \>\>\> a.supernet_of(b) True

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

compare_networks(other)

Compare this network to *other*. In this comparison only the network addresses are considered; host bits aren't. Returns either `-1`, `0` or `1`.

> \>\>\> ip_network('192.0.2.1/32').compare_networks(ip_network('192.0.2.2/32')) -1 \>\>\> ip_network('192.0.2.1/32').compare_networks(ip_network('192.0.2.0/32')) 1 \>\>\> ip_network('192.0.2.1/32').compare_networks(ip_network('192.0.2.1/32')) 0

<div class="deprecated">

3.7 It uses the same ordering and comparison algorithm as "\<", "==", and "\>"

</div>

</div>

<div class="method">

next_network(next_prefix=None)

Finds the next closest network of prefix size *next_prefix*. If *next_prefix=None*, then the current network prefix will be used. Returns a single network object.

Raises `ValueError` if *next_prefix* is out of range or no further network of the requested size exists.

> \>\>\> IPv4Network('192.0.2.0/24').next_network() IPv4Network('192.0.3.0/24') \>\>\> IPv4Network('192.0.2.0/24').next_network(next_prefix=25) IPv4Network('192.0.3.0/25') \>\>\> IPv4Network('192.0.2.0/24').next_network(next_prefix=23) IPv4Network('192.0.4.0/23') \>\>\> IPv4Network('192.0.80.0/22').next_network(next_prefix=18) IPv4Network('192.0.128.0/18') \>\>\> IPv4Network('192.0.80.0/22').next_network(next_prefix=50) Traceback (most recent call last): File "\<stdin\>", line 1, in \<module\> raise ValueError( ValueError: next prefix must be between 1 and 32 \>\>\> IPv4Network('255.255.255.0/24').next_network() Traceback (most recent call last): File "\<stdin\>", line 1, in \<module\> raise ValueError( ValueError: out of address space, cannot make another /24 network

<div class="versionadded">

3.16

</div>

</div>

</div>

<div class="IPv6Network(address, strict=True)">

Construct an IPv6 network definition. *address* can be one of the following:

1.  A string consisting of an IP address and an optional prefix length, separated by a slash (`/`). The IP address is the network address, and the prefix length must be a single number, the *prefix*. If no prefix length is provided, it's considered to be `/128`.

    Note that currently expanded netmasks are not supported. That means `2001:db00::0/24` is a valid argument while `2001:db00::0/ffff:ff00::` is not.

2.  An integer that fits into 128 bits. This is equivalent to a single-address network, with the network address being *address* and the mask being `/128`.

3.  An integer packed into a `bytes` object of length 16, big-endian. The interpretation is similar to an integer *address*.

4.  A two-tuple of an address description and a netmask, where the address description is either a string, a 128-bit integer, a 16-byte packed integer, or an existing `IPv6Address` object; and the netmask is an integer representing the prefix length.

An `AddressValueError` is raised if *address* is not a valid IPv6 address. A `NetmaskValueError` is raised if the mask is not valid for an IPv6 address.

If *strict* is `True` and host bits are set in the supplied address, then `ValueError` is raised. Otherwise, the host bits are masked out to determine the appropriate network address.

<div class="versionchanged">

3.5

Added the two-tuple form for the *address* constructor parameter.

</div>

<div class="attribute">

version

</div>

<div class="attribute">

max_prefixlen

</div>

<div class="attribute">

is_multicast

</div>

<div class="attribute">

is_private

</div>

<div class="attribute">

is_unspecified

</div>

<div class="attribute">

is_reserved

</div>

<div class="attribute">

is_loopback

</div>

<div class="attribute">

is_link_local

</div>

<div class="attribute">

network_address

</div>

<div class="attribute">

broadcast_address

</div>

<div class="attribute">

hostmask

</div>

<div class="attribute">

netmask

</div>

<div class="attribute">

with_prefixlen

</div>

<div class="attribute">

compressed

</div>

<div class="attribute">

exploded

</div>

<div class="attribute">

with_netmask

</div>

<div class="attribute">

with_hostmask

</div>

<div class="attribute">

num_addresses

</div>

<div class="attribute">

prefixlen

</div>

<div class="method">

hosts()

Returns an iterator over the usable hosts in the network. The usable hosts are all the IP addresses that belong to the network, except the Subnet-Router anycast address. For networks with a mask length of 127, the Subnet-Router anycast address is also included in the result. Networks with a mask of 128 will return a list containing the single host address.

</div>

<div class="method">

overlaps(other)

</div>

<div class="method">

address_exclude(network)

</div>

<div class="method">

subnets(prefixlen_diff=1, new_prefix=None)

</div>

<div class="method">

supernet(prefixlen_diff=1, new_prefix=None)

</div>

<div class="method">

subnet_of(other)

</div>

<div class="method">

supernet_of(other)

</div>

<div class="method">

next_network(next_prefix=None)

</div>

<div class="method">

compare_networks(other)

Refer to the corresponding attribute documentation in `IPv4Network`.

</div>

<div class="attribute">

is_site_local

This attribute is true for the network as a whole if it is true for both the network address and the broadcast address.

</div>

</div>

### Operators

Network objects support some operators. Unless stated otherwise, operators can only be applied between compatible objects (i.e. IPv4 with IPv4, IPv6 with IPv6).

#### Logical operators

Network objects can be compared with the usual set of logical operators. Network objects are ordered first by network address, then by net mask.

#### Iteration

Network objects can be iterated to list all the addresses belonging to the network. For iteration, *all* hosts are returned, including unusable hosts (for usable hosts, use the `~IPv4Network.hosts` method). An example:

    >>> for addr in IPv4Network('192.0.2.0/28'):
    ...     addr
    ...
    IPv4Address('192.0.2.0')
    IPv4Address('192.0.2.1')
    IPv4Address('192.0.2.2')
    IPv4Address('192.0.2.3')
    IPv4Address('192.0.2.4')
    IPv4Address('192.0.2.5')
    IPv4Address('192.0.2.6')
    IPv4Address('192.0.2.7')
    IPv4Address('192.0.2.8')
    IPv4Address('192.0.2.9')
    IPv4Address('192.0.2.10')
    IPv4Address('192.0.2.11')
    IPv4Address('192.0.2.12')
    IPv4Address('192.0.2.13')
    IPv4Address('192.0.2.14')
    IPv4Address('192.0.2.15')

#### Networks as containers of addresses

Network objects can act as containers of addresses. Some examples:

    >>> IPv4Network('192.0.2.0/28')[0]
    IPv4Address('192.0.2.0')
    >>> IPv4Network('192.0.2.0/28')[15]
    IPv4Address('192.0.2.15')
    >>> IPv4Address('192.0.2.6') in IPv4Network('192.0.2.0/28')
    True
    >>> IPv4Address('192.0.3.6') in IPv4Network('192.0.2.0/28')
    False

## Interface objects

Interface objects are `hashable`, so they can be used as keys in dictionaries.

<div class="IPv4Interface(address)">

Construct an IPv4 interface. The meaning of *address* is as in the constructor of `IPv4Network`, except that arbitrary host addresses are always accepted.

`IPv4Interface` is a subclass of `IPv4Address`, so it inherits all the attributes from that class. In addition, the following attributes are available:

<div class="attribute">

ip

The address (`IPv4Address`) without network information.

> \>\>\> interface = IPv4Interface('192.0.2.5/24') \>\>\> interface.ip IPv4Address('192.0.2.5')

</div>

<div class="attribute">

network

The network (`IPv4Network`) this interface belongs to.

> \>\>\> interface = IPv4Interface('192.0.2.5/24') \>\>\> interface.network IPv4Network('192.0.2.0/24')

</div>

<div class="attribute">

with_prefixlen

A string representation of the interface with the mask in prefix notation.

> \>\>\> interface = IPv4Interface('192.0.2.5/24') \>\>\> interface.with_prefixlen '192.0.2.5/24'

</div>

<div class="attribute">

with_netmask

A string representation of the interface with the network as a net mask.

> \>\>\> interface = IPv4Interface('192.0.2.5/24') \>\>\> interface.with_netmask '192.0.2.5/255.255.255.0'

</div>

<div class="attribute">

with_hostmask

A string representation of the interface with the network as a host mask.

> \>\>\> interface = IPv4Interface('192.0.2.5/24') \>\>\> interface.with_hostmask '192.0.2.5/0.0.0.255'

</div>

</div>

<div class="IPv6Interface(address)">

Construct an IPv6 interface. The meaning of *address* is as in the constructor of `IPv6Network`, except that arbitrary host addresses are always accepted.

`IPv6Interface` is a subclass of `IPv6Address`, so it inherits all the attributes from that class. In addition, the following attributes are available:

<div class="attribute">

ip

</div>

<div class="attribute">

network

</div>

<div class="attribute">

with_prefixlen

</div>

<div class="attribute">

with_netmask

</div>

<div class="attribute">

with_hostmask

Refer to the corresponding attribute documentation in `IPv4Interface`.

</div>

</div>

### Operators

Interface objects support some operators. Unless stated otherwise, operators can only be applied between compatible objects (i.e. IPv4 with IPv4, IPv6 with IPv6).

#### Logical operators

Interface objects can be compared with the usual set of logical operators.

For equality comparison (`==` and `!=`), both the IP address and network must be the same for the objects to be equal. An interface will not compare equal to any address or network object.

For ordering (`<`, `>`, etc) the rules are different. Interface and address objects with the same IP version can be compared, and the address objects will always sort before the interface objects. Two interface objects are first compared by their networks and, if those are the same, then by their IP addresses.

## Other Module Level Functions

The module also provides the following module level functions:

<div class="function">

v4_int_to_packed(address)

Represent an address as 4 packed bytes in network (big-endian) order. *address* is an integer representation of an IPv4 IP address. A `ValueError` is raised if the integer is negative or too large to be an IPv4 IP address.

\>\>\> ipaddress.ip_address(3221225985) IPv4Address('192.0.2.1') \>\>\> ipaddress.v4_int_to_packed(3221225985) b'xc0x00x02x01'

</div>

<div class="function">

v6_int_to_packed(address)

Represent an address as 16 packed bytes in network (big-endian) order. *address* is an integer representation of an IPv6 IP address. A `ValueError` is raised if the integer is negative or too large to be an IPv6 IP address.

</div>

<div class="function">

summarize_address_range(first, last)

Return an iterator of the summarized network range given the first and last IP addresses. *first* is the first `IPv4Address` or `IPv6Address` in the range and *last* is the last `IPv4Address` or `IPv6Address` in the range. A `TypeError` is raised if *first* or *last* are not IP addresses or are not of the same version. A `ValueError` is raised if *last* is not greater than *first* or if *first* address version is not 4 or 6.

\>\>\> \[ipaddr for ipaddr in ipaddress.summarize_address_range( ... ipaddress.IPv4Address('192.0.2.0'), ... ipaddress.IPv4Address('192.0.2.130'))\] \[IPv4Network('192.0.2.0/25'), IPv4Network('192.0.2.128/31'), IPv4Network('192.0.2.130/32')\]

</div>

<div class="function">

collapse_addresses(addresses)

Return an iterator of the collapsed `IPv4Network` or `IPv6Network` objects. *addresses* is an `iterable` of `IPv4Network` or `IPv6Network` objects. A `TypeError` is raised if *addresses* contains mixed version objects.

\>\>\> \[ipaddr for ipaddr in ... ipaddress.collapse_addresses(\[ipaddress.IPv4Network('192.0.2.0/25'), ... ipaddress.IPv4Network('192.0.2.128/25')\])\] \[IPv4Network('192.0.2.0/24')\]

</div>

<div class="function">

get_mixed_type_key(obj)

Return a key suitable for sorting between networks and addresses. Address and Network objects are not sortable by default; they're fundamentally different, so the expression:

    IPv4Address('192.0.2.0') <= IPv4Network('192.0.2.0/24')

doesn't make sense. There are some times however, where you may wish to have `!ipaddress` sort these anyway. If you need to do this, you can use this function as the *key* argument to `sorted`.

*obj* is either a network or address object.

</div>

## Custom Exceptions

To support more specific error reporting from class constructors, the module defines the following exceptions:

<div class="exception">

AddressValueError(ValueError)

Any value error related to the address.

</div>

<div class="exception">

NetmaskValueError(ValueError)

Any value error related to the net mask.

</div>

# Networking and Interprocess Communication

The modules described in this chapter provide mechanisms for networking and inter-processes communication.

Some modules only work for two processes that are on the same machine, e.g. `signal` and `mmap`. Other modules support networking protocols that two or more processes can use to communicate across machines.

The list of modules described in this chapter is:

<div class="toctree" data-maxdepth="1">

asyncio.rst socket.rst ssl.rst select.rst selectors.rst signal.rst mmap.rst

</div>

# `!itertools` --- Functions creating iterators for efficient looping

<div class="module" data-synopsis="Functions creating iterators for efficient looping.">

itertools

</div>

<div class="testsetup">

from itertools import \* import collections import math import operator import random

</div>

------------------------------------------------------------------------

This module implements a number of `iterator` building blocks inspired by constructs from APL, Haskell, and SML. Each has been recast in a form suitable for Python.

The module standardizes a core set of fast, memory efficient tools that are useful by themselves or in combination. Together, they form an "iterator algebra" making it possible to construct specialized tools succinctly and efficiently in pure Python.

For instance, SML provides a tabulation tool: `tabulate(f)` which produces a sequence `f(0), f(1), ...`. The same effect can be achieved in Python by combining `map` and `count` to form `map(f, count())`.

**General iterators:**

| Iterator | Arguments | Results | Example |
|----|----|----|----|
| `accumulate` | p \[,func\] | p0, p0+p1, p0+p1+p2, ... | `accumulate([1,2,3,4,5]) → 1 3 6 10 15` |
| `batched` | p, n | (p0, p1, ..., p_n-1), ... | `batched('ABCDEFG', n=3) → ABC DEF G` |
| `chain` | p, q, ... | p0, p1, ... plast, q0, q1, ... | `chain('ABC', 'DEF') → A B C D E F` |
| `chain.from_iterable` | iterable | p0, p1, ... plast, q0, q1, ... | `chain.from_iterable(['ABC', 'DEF']) → A B C D E F` |
| `compress` | data, selectors | (d\[0\] if s\[0\]), (d\[1\] if s\[1\]), ... | `compress('ABCDEF', [1,0,1,0,1,1]) → A C E F` |
| `count` | \[start\[, step\]\] | start, start+step, start+2\*step, ... | `count(10) → 10 11 12 13 14 ...` |
| `cycle` | p | p0, p1, ... plast, p0, p1, ... | `cycle('ABCD') → A B C D A B C D ...` |
| `dropwhile` | predicate, seq | seq\[n\], seq\[n+1\], starting when predicate fails | `dropwhile(lambda x: x<5, [1,4,6,3,8]) → 6 3 8` |
| `filterfalse` | predicate, seq | elements of seq where predicate(elem) fails | `filterfalse(lambda x: x<5, [1,4,6,3,8]) → 6 8` |
| `groupby` | iterable\[, key\] | sub-iterators grouped by value of key(v) | `groupby(['A','B','DEF'], len) → (1, A B) (3, DEF)` |
| `islice` | seq, \[start,\] stop \[, step\] | elements from seq\[start:stop:step\] | `islice('ABCDEFG', 2, None) → C D E F G` |
| `pairwise` | iterable | (p\[0\], p\[1\]), (p\[1\], p\[2\]) | `pairwise('ABCDEFG') → AB BC CD DE EF FG` |
| `repeat` | elem \[,n\] | elem, elem, elem, ... endlessly or up to n times | `repeat(10, 3) → 10 10 10` |
| `starmap` | func, seq | func(\*seq\[0\]), func(\*seq\[1\]), ... | `starmap(pow, [(2,5), (3,2), (10,3)]) → 32 9 1000` |
| `takewhile` | predicate, seq | seq\[0\], seq\[1\], until predicate fails | `takewhile(lambda x: x<5, [1,4,6,3,8]) → 1 4` |
| `tee` | it, n | it1, it2, ... itn splits one iterator into n | `tee('ABC', 2) → A B C, A B C` |
| `zip_longest` | p, q, ... | (p\[0\], q\[0\]), (p\[1\], q\[1\]), ... | `zip_longest('ABCD', 'xy', fillvalue='-') → Ax By C- D-` |

**Combinatoric iterators:**

| Iterator | Arguments | Results |
|----|----|----|
| `product` | p, q, ... \[repeat=1\] | cartesian product, equivalent to a nested for-loop |
| `permutations` | p\[, r\] | r-length tuples, all possible orderings, no repeated elements |
| `combinations` | p, r | r-length tuples, in sorted order, no repeated elements |
| `combinations_with_replacement` | p, r | r-length tuples, in sorted order, with repeated elements |

| Examples | Results |
|----|----|
| `product('ABCD', repeat=2)` | `AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD` |
| `permutations('ABCD', 2)` | `AB AC AD BA BC BD CA CB CD DA DB DC` |
| `combinations('ABCD', 2)` | `AB AC AD BC BD CD` |
| `combinations_with_replacement('ABCD', 2)` | `AA AB AC AD BB BC BD CC CD DD` |

## Itertool Functions

The following functions all construct and return iterators. Some provide streams of infinite length, so they should only be accessed by functions or loops that truncate the stream.

<div class="function">

accumulate(iterable\[, function, \*, initial=None\])

Make an iterator that returns accumulated sums or accumulated results from other binary functions.

The *function* defaults to addition. The *function* should accept two arguments, an accumulated total and a value from the *iterable*.

If an *initial* value is provided, the accumulation will start with that value and the output will have one more element than the input iterable.

Roughly equivalent to:

    def accumulate(iterable, function=operator.add, *, initial=None):
        'Return running totals'
        # accumulate([1,2,3,4,5]) → 1 3 6 10 15
        # accumulate([1,2,3,4,5], initial=100) → 100 101 103 106 110 115
        # accumulate([1,2,3,4,5], operator.mul) → 1 2 6 24 120

        iterator = iter(iterable)
        total = initial
        if initial is None:
            try:
                total = next(iterator)
            except StopIteration:
                return

        yield total
        for element in iterator:
            total = function(total, element)
            yield total

To compute a running minimum, set *function* to `min`. For a running maximum, set *function* to `max`. Or for a running product, set *function* to `operator.mul`. To build an [amortization table](https://www.ramseysolutions.com/real-estate/amortization-schedule), accumulate the interest and apply payments:

<div class="doctest">

\>\>\> data = \[3, 4, 6, 2, 1, 9, 0, 7, 5, 8\] \>\>\> list(accumulate(data, max)) \# running maximum \[3, 4, 6, 6, 6, 9, 9, 9, 9, 9\] \>\>\> list(accumulate(data, operator.mul)) \# running product \[3, 12, 72, 144, 144, 1296, 0, 0, 0, 0\]

\# Amortize a 5% loan of 1000 with 10 annual payments of 90 \>\>\> update = lambda balance, payment: round(balance \* 1.05) - payment \>\>\> list(accumulate(repeat(90, 10), update, initial=1_000)) \[1000, 960, 918, 874, 828, 779, 728, 674, 618, 559, 497\]

</div>

See `functools.reduce` for a similar function that returns only the final accumulated value.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Added the optional *function* parameter.

</div>

<div class="versionchanged">

3.8 Added the optional *initial* parameter.

</div>

</div>

<div class="function">

batched(iterable, n, \*, strict=False)

Batch data from the *iterable* into tuples of length *n*. The last batch may be shorter than *n*.

If *strict* is true, will raise a `ValueError` if the final batch is shorter than *n*.

Loops over the input iterable and accumulates data into tuples up to size *n*. The input is consumed lazily, just enough to fill a batch. The result is yielded as soon as the batch is full or when the input iterable is `exhausted`:

<div class="doctest">

\>\>\> flattened_data = \['roses', 'red', 'violets', 'blue', 'sugar', 'sweet'\] \>\>\> unflattened = list(batched(flattened_data, 2)) \>\>\> unflattened \[('roses', 'red'), ('violets', 'blue'), ('sugar', 'sweet')\]

</div>

Roughly equivalent to:

    def batched(iterable, n, *, strict=False):
        # batched('ABCDEFG', 3) → ABC DEF G
        if n < 1:
            raise ValueError('n must be at least one')
        iterator = iter(iterable)
        while batch := tuple(islice(iterator, n)):
            if strict and len(batch) != n:
                raise ValueError('batched(): incomplete batch')
            yield batch

<div class="versionadded">

3.12

</div>

<div class="versionchanged">

3.13 Added the *strict* option.

</div>

</div>

<div class="function">

chain(\*iterables)

Make an iterator that returns elements from the first iterable until it is `exhausted`, then proceeds to the next iterable, until all of the iterables are exhausted. This combines multiple data sources into a single iterator. Roughly equivalent to:

    def chain(*iterables):
        # chain('ABC', 'DEF') → A B C D E F
        for iterable in iterables:
            yield from iterable

Note that `unpacking in comprehensions <unpacking-comprehensions>` provides similar functionality so that `list(chain(p, q))` could be written as `[*s for s in (p, q)]`.

</div>

<div class="classmethod">

chain.from_iterable(iterable)

Alternate constructor for `chain`. Gets chained inputs from a single iterable argument that is evaluated lazily. Roughly equivalent to:

    def from_iterable(iterables):
        # chain.from_iterable(['ABC', 'DEF']) → A B C D E F
        for iterable in iterables:
            yield from iterable

Note that `unpacking in comprehensions <unpacking-comprehensions>` provides similar functionality so that `list(chain.from_iterable(iterables))` could be written as `[*s for s in iterables]`.

</div>

<div class="function">

combinations(iterable, r)

Return *r* length subsequences of elements from the input *iterable*.

The output is a subsequence of `product` keeping only entries that are subsequences of the *iterable*. The length of the output is given by `math.comb` which computes `n! / r! / (n - r)!` when `0 ≤ r ≤ n` or zero when `r > n`.

The combination tuples are emitted in lexicographic order according to the order of the input *iterable*. If the input *iterable* is sorted, the output tuples will be produced in sorted order.

Elements are treated as unique based on their position, not on their value. If the input elements are unique, there will be no repeated values within each combination.

Roughly equivalent to:

    def combinations(iterable, r):
        # combinations('ABCD', 2) → AB AC AD BC BD CD
        # combinations(range(4), 3) → 012 013 023 123

        pool = tuple(iterable)
        n = len(pool)
        if r > n:
            return
        indices = list(range(r))

        yield tuple(pool[i] for i in indices)
        while True:
            for i in reversed(range(r)):
                if indices[i] != i + n - r:
                    break
            else:
                return
            indices[i] += 1
            for j in range(i+1, r):
                indices[j] = indices[j-1] + 1
            yield tuple(pool[i] for i in indices)

</div>

<div class="function">

combinations_with_replacement(iterable, r)

Return *r* length subsequences of elements from the input *iterable* allowing individual elements to be repeated more than once.

The output is a subsequence of `product` that keeps only entries that are subsequences (with possible repeated elements) of the *iterable*. The number of subsequence returned is `(n + r - 1)! / r! / (n - 1)!` when `n > 0`.

The combination tuples are emitted in lexicographic order according to the order of the input *iterable*. if the input *iterable* is sorted, the output tuples will be produced in sorted order.

Elements are treated as unique based on their position, not on their value. If the input elements are unique, the generated combinations will also be unique.

Roughly equivalent to:

    def combinations_with_replacement(iterable, r):
        # combinations_with_replacement('ABC', 2) → AA AB AC BB BC CC

        pool = tuple(iterable)
        n = len(pool)
        if not n and r:
            return
        indices = [0] * r

        yield tuple(pool[i] for i in indices)
        while True:
            for i in reversed(range(r)):
                if indices[i] != n - 1:
                    break
            else:
                return
            indices[i:] = [indices[i] + 1] * (r - i)
            yield tuple(pool[i] for i in indices)

<div class="versionadded">

3.1

</div>

</div>

<div class="function">

compress(data, selectors)

Make an iterator that returns elements from *data* where the corresponding element in *selectors* is true. Stops when either the *data* or *selectors* iterables have been `exhausted`. Roughly equivalent to:

    def compress(data, selectors):
        # compress('ABCDEF', [1,0,1,0,1,1]) → A C E F
        return (datum for datum, selector in zip(data, selectors) if selector)

<div class="versionadded">

3.1

</div>

</div>

<div class="function">

count(start=0, step=1)

Make an iterator that returns evenly spaced values beginning with *start*. Can be used with `map` to generate consecutive data points or with `zip` to add sequence numbers. Roughly equivalent to:

    def count(start=0, step=1):
        # count(10) → 10 11 12 13 14 ...
        # count(2.5, 0.5) → 2.5 3.0 3.5 ...
        n = start
        while True:
            yield n
            n += step

When counting with floating-point numbers, better accuracy can sometimes be achieved by substituting multiplicative code such as: `(start + step * i for i in count())`.

<div class="versionchanged">

3.1 Added *step* argument and allowed non-integer arguments.

</div>

</div>

<div class="function">

cycle(iterable)

Make an iterator returning elements from the *iterable* and saving a copy of each. When the iterable is `exhausted`, return elements from the saved copy. Repeats indefinitely. Roughly equivalent to:

    def cycle(iterable):
        # cycle('ABCD') → A B C D A B C D A B C D ...

        saved = []
        for element in iterable:
            yield element
            saved.append(element)

        while saved:
            for element in saved:
                yield element

This itertool may require significant auxiliary storage (depending on the length of the iterable).

</div>

<div class="function">

dropwhile(predicate, iterable)

Make an iterator that drops elements from the *iterable* while the *predicate* is true and afterwards returns every element. Roughly equivalent to:

    def dropwhile(predicate, iterable):
        # dropwhile(lambda x: x<5, [1,4,6,3,8]) → 6 3 8

        iterator = iter(iterable)
        for x in iterator:
            if not predicate(x):
                yield x
                break

        for x in iterator:
            yield x

Note this does not produce *any* output until the predicate first becomes false, so this itertool may have a lengthy start-up time.

</div>

<div class="function">

filterfalse(predicate, iterable)

Make an iterator that filters elements from the *iterable* returning only those for which the *predicate* returns a false value. If *predicate* is `None`, returns the items that are false. Roughly equivalent to:

    def filterfalse(predicate, iterable):
        # filterfalse(lambda x: x<5, [1,4,6,3,8]) → 6 8

        if predicate is None:
            predicate = bool

        for x in iterable:
            if not predicate(x):
                yield x

</div>

<div class="function">

groupby(iterable, key=None)

Make an iterator that returns consecutive keys and groups from the *iterable*. The *key* is a function computing a key value for each element. If not specified or is `None`, *key* defaults to an identity function and returns the element unchanged. Generally, the iterable needs to already be sorted on the same key function.

The operation of `groupby` is similar to the `uniq` filter in Unix. It generates a break or new group every time the value of the key function changes (which is why it is usually necessary to have sorted the data using the same key function). That behavior differs from SQL's GROUP BY which aggregates common elements regardless of their input order.

The returned group is itself an iterator that shares the underlying iterable with `groupby`. Because the source is shared, when the `groupby` object is advanced, the previous group is no longer visible. So, if that data is needed later, it should be stored as a list:

    groups = []
    uniquekeys = []
    data = sorted(data, key=keyfunc)
    for k, g in groupby(data, keyfunc):
        groups.append(list(g))      # Store group iterator as a list
        uniquekeys.append(k)

`groupby` is roughly equivalent to:

    def groupby(iterable, key=None):
        # [k for k, g in groupby('AAAABBBCCDAABBB')] → A B C D A B
        # [list(g) for k, g in groupby('AAAABBBCCD')] → AAAA BBB CC D

        keyfunc = (lambda x: x) if key is None else key
        iterator = iter(iterable)
        exhausted = False

        def _grouper(target_key):
            nonlocal curr_value, curr_key, exhausted
            yield curr_value
            for curr_value in iterator:
                curr_key = keyfunc(curr_value)
                if curr_key != target_key:
                    return
                yield curr_value
            exhausted = True

        try:
            curr_value = next(iterator)
        except StopIteration:
            return
        curr_key = keyfunc(curr_value)

        while not exhausted:
            target_key = curr_key
            curr_group = _grouper(target_key)
            yield curr_key, curr_group
            if curr_key == target_key:
                for _ in curr_group:
                    pass

</div>

<div class="function">

islice(iterable, stop) islice(iterable, start, stop\[, step\])

Make an iterator that returns selected elements from the iterable. Works like sequence slicing but does not support negative values for *start*, *stop*, or *step*.

If *start* is zero or `None`, iteration starts at zero. Otherwise, elements from the iterable are skipped until *start* is reached.

If *stop* is `None`, iteration continues until the input is `exhausted`, if at all. Otherwise, it stops at the specified position.

If *step* is `None`, the step defaults to one. Elements are returned consecutively unless *step* is set higher than one which results in items being skipped.

Roughly equivalent to:

    def islice(iterable, *args):
        # islice('ABCDEFG', 2) → A B
        # islice('ABCDEFG', 2, 4) → C D
        # islice('ABCDEFG', 2, None) → C D E F G
        # islice('ABCDEFG', 0, None, 2) → A C E G

        s = slice(*args)
        start = 0 if s.start is None else s.start
        stop = s.stop
        step = 1 if s.step is None else s.step
        if start < 0 or (stop is not None and stop < 0) or step <= 0:
            raise ValueError

        indices = count() if stop is None else range(max(start, stop))
        next_i = start
        for i, element in zip(indices, iterable):
            if i == next_i:
                yield element
                next_i += step

If the input is an iterator, then fully consuming the *islice* advances the input iterator by `max(start, stop)` steps regardless of the *step* value.

</div>

<div class="function">

pairwise(iterable)

Return successive overlapping pairs taken from the input *iterable*.

The number of 2-tuples in the output iterator will be one fewer than the number of inputs. It will be empty if the input iterable has fewer than two values.

Roughly equivalent to:

    def pairwise(iterable):
        # pairwise('ABCDEFG') → AB BC CD DE EF FG

        iterator = iter(iterable)
        a = next(iterator, None)

        for b in iterator:
            yield a, b
            a = b

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

permutations(iterable, r=None)

Return successive *r* length [permutations of elements](https://www.britannica.com/science/permutation) from the *iterable*.

If *r* is not specified or is `None`, then *r* defaults to the length of the *iterable* and all possible full-length permutations are generated.

The output is a subsequence of `product` where entries with repeated elements have been filtered out. The length of the output is given by `math.perm` which computes `n! / (n - r)!` when `0 ≤ r ≤ n` or zero when `r > n`.

The permutation tuples are emitted in lexicographic order according to the order of the input *iterable*. If the input *iterable* is sorted, the output tuples will be produced in sorted order.

Elements are treated as unique based on their position, not on their value. If the input elements are unique, there will be no repeated values within a permutation.

Roughly equivalent to:

    def permutations(iterable, r=None):
        # permutations('ABCD', 2) → AB AC AD BA BC BD CA CB CD DA DB DC
        # permutations(range(3)) → 012 021 102 120 201 210

        pool = tuple(iterable)
        n = len(pool)
        r = n if r is None else r
        if r > n:
            return

        indices = list(range(n))
        cycles = list(range(n, n-r, -1))
        yield tuple(pool[i] for i in indices[:r])

        while n:
            for i in reversed(range(r)):
                cycles[i] -= 1
                if cycles[i] == 0:
                    indices[i:] = indices[i+1:] + indices[i:i+1]
                    cycles[i] = n - i
                else:
                    j = cycles[i]
                    indices[i], indices[-j] = indices[-j], indices[i]
                    yield tuple(pool[i] for i in indices[:r])
                    break
            else:
                return

</div>

<div class="function">

product(\*iterables, repeat=1)

[Cartesian product](https://en.wikipedia.org/wiki/Cartesian_product) of the input iterables.

Roughly equivalent to nested for-loops in a generator expression. For example, `product(A, B)` returns the same as `((x,y) for x in A for y in B)`.

The nested loops cycle like an odometer with the rightmost element advancing on every iteration. This pattern creates a lexicographic ordering so that if the input's iterables are sorted, the product tuples are emitted in sorted order.

To compute the product of an iterable with itself, specify the number of repetitions with the optional *repeat* keyword argument. For example, `product(A, repeat=4)` means the same as `product(A, A, A, A)`.

This function is roughly equivalent to the following code, except that the actual implementation does not build up intermediate results in memory:

    def product(*iterables, repeat=1):
        # product('ABCD', 'xy') → Ax Ay Bx By Cx Cy Dx Dy
        # product(range(2), repeat=3) → 000 001 010 011 100 101 110 111

        if repeat < 0:
            raise ValueError('repeat argument cannot be negative')
        pools = [tuple(pool) for pool in iterables] * repeat

        result = [[]]
        for pool in pools:
            result = [x+[y] for x in result for y in pool]

        for prod in result:
            yield tuple(prod)

Before `product` runs, it completely consumes the input iterables, keeping pools of values in memory to generate the products. Accordingly, it is only useful with finite inputs.

</div>

<div class="function">

repeat(object\[, times\])

Make an iterator that returns *object* over and over again. Runs indefinitely unless the *times* argument is specified.

Roughly equivalent to:

    def repeat(object, times=None):
        # repeat(10, 3) → 10 10 10
        if times is None:
            while True:
                yield object
        else:
            for i in range(times):
                yield object

A common use for *repeat* is to supply a stream of constant values to *map* or *zip*:

<div class="doctest">

\>\>\> list(map(pow, range(10), repeat(2))) \[0, 1, 4, 9, 16, 25, 36, 49, 64, 81\]

</div>

</div>

<div class="function">

starmap(function, iterable)

Make an iterator that computes the *function* using arguments obtained from the *iterable*. Used instead of `map` when argument parameters have already been "pre-zipped" into tuples.

The difference between `map` and `starmap` parallels the distinction between `function(a,b)` and `function(*c)`. Roughly equivalent to:

    def starmap(function, iterable):
        # starmap(pow, [(2,5), (3,2), (10,3)]) → 32 9 1000
        for args in iterable:
            yield function(*args)

</div>

<div class="function">

takewhile(predicate, iterable)

Make an iterator that returns elements from the *iterable* as long as the *predicate* is true. Roughly equivalent to:

    def takewhile(predicate, iterable):
        # takewhile(lambda x: x<5, [1,4,6,3,8]) → 1 4
        for x in iterable:
            if not predicate(x):
                break
            yield x

Note, the element that first fails the predicate condition is consumed from the input iterator and there is no way to access it. This could be an issue if an application wants to further consume the input iterator after *takewhile* has been run to `exhaustion <exhausted>`. To work around this problem, consider using [more-itertools before_and_after()](https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.before_and_after) instead.

</div>

<div class="function">

tee(iterable, n=2)

Return *n* independent iterators from a single iterable.

Roughly equivalent to:

    def tee(iterable, n=2):
        if n < 0:
            raise ValueError
        if n == 0:
            return ()
        iterator = _tee(iterable)
        result = [iterator]
        for _ in range(n - 1):
            result.append(_tee(iterator))
        return tuple(result)

    class _tee:

        def __init__(self, iterable):
            it = iter(iterable)
            if isinstance(it, _tee):
                self.iterator = it.iterator
                self.link = it.link
            else:
                self.iterator = it
                self.link = [None, None]

        def __iter__(self):
            return self

        def __next__(self):
            link = self.link
            if link[1] is None:
                link[0] = next(self.iterator)
                link[1] = [None, None]
            value, self.link = link
            return value

When the input *iterable* is already a tee iterator object, all members of the return tuple are constructed as if they had been produced by the upstream `tee` call. This "flattening step" allows nested `tee` calls to share the same underlying data chain and to have a single update step rather than a chain of calls.

The flattening property makes tee iterators efficiently peekable:

<div class="testcode">

def lookahead(tee_iterator):  
"Return the next value without moving the input forward" \[forked_iterator\] = tee(tee_iterator, 1) return next(forked_iterator)

</div>

<div class="doctest">

\>\>\> iterator = iter('abcdef') \>\>\> \[iterator\] = tee(iterator, 1) \# Make the input peekable \>\>\> next(iterator) \# Move the iterator forward 'a' \>\>\> lookahead(iterator) \# Check next value 'b' \>\>\> next(iterator) \# Continue moving forward 'b'

</div>

`tee` iterators are not threadsafe. A `RuntimeError` may be raised when simultaneously using iterators returned by the same `tee` call, even if the original *iterable* is threadsafe.

This itertool may require significant auxiliary storage (depending on how much temporary data needs to be stored). In general, if one iterator uses most or all of the data before another iterator starts, it is faster to use `list` instead of `tee`.

</div>

<div class="function">

zip_longest(\*iterables, fillvalue=None)

Make an iterator that aggregates elements from each of the *iterables*.

If the iterables are of uneven length, missing values are filled-in with *fillvalue*. If not specified, *fillvalue* defaults to `None`.

Iteration continues until the longest iterable is `exhausted`.

Roughly equivalent to:

    def zip_longest(*iterables, fillvalue=None):
        # zip_longest('ABCD', 'xy', fillvalue='-') → Ax By C- D-

        iterators = list(map(iter, iterables))
        num_active = len(iterators)
        if not num_active:
            return

        while True:
            values = []
            for i, iterator in enumerate(iterators):
                try:
                    value = next(iterator)
                except StopIteration:
                    num_active -= 1
                    if not num_active:
                        return
                    iterators[i] = repeat(fillvalue)
                    value = fillvalue
                values.append(value)
            yield tuple(values)

If one of the iterables is potentially infinite, then the `zip_longest` function should be wrapped with something that limits the number of calls (for example `islice` or `takewhile`).

</div>

## Itertools Recipes

This section shows recipes for creating an extended toolset using the existing itertools as building blocks.

The primary purpose of the itertools recipes is educational. The recipes show various ways of thinking about individual tools — for example, that `chain.from_iterable` is related to the concept of flattening. The recipes also give ideas about ways that the tools can be combined — for example, how `starmap()` and `repeat()` can work together. The recipes also show patterns for using itertools with the `operator` and `collections` modules as well as with the built-in itertools such as `map()`, `filter()`, `reversed()`, and `enumerate()`.

A secondary purpose of the recipes is to serve as an incubator. The `accumulate()`, `compress()`, and `pairwise()` itertools started out as recipes. Currently, the `sliding_window()`, `derangements()`, and `sieve()` recipes are being tested to see whether they prove their worth.

Substantially all of these recipes and many, many others can be installed from the `more-itertools` project found on the Python Package Index:

    python -m pip install more-itertools

Many of the recipes offer the same high performance as the underlying toolset. Superior memory performance is kept by processing elements one at a time rather than bringing the whole iterable into memory all at once. Code volume is kept small by linking the tools together in a [functional style](https://www.cs.kent.ac.uk/people/staff/dat/miranda/whyfp90.pdf). High speed is retained by preferring "vectorized" building blocks over the use of for-loops and `generators <generator>` which incur interpreter overhead.

<div class="testcode">

from itertools import (accumulate, batched, chain, combinations, compress,  
count, cycle, filterfalse, groupby, islice, permutations, product, repeat, starmap, tee, zip_longest)

from collections import Counter, deque from contextlib import suppress from functools import reduce from heapq import heappush, heappushpop, heappush_max, heappushpop_max from math import comb, isqrt, prod, sumprod from operator import getitem, is_not, itemgetter, mul, neg, truediv

\# ==== Basic one liners ====

def take(n, iterable):  
"Return first n items of the iterable as a list." return list(islice(iterable, n))

def prepend(value, iterable):  
"Prepend a single value in front of an iterable." \# prepend(1, \[2, 3, 4\]) → 1 2 3 4 return chain(\[value\], iterable)

def repeatfunc(function, times=None, \*args):  
"Repeat calls to a function with specified arguments." if times is None: return starmap(function, repeat(args)) return starmap(function, repeat(args, times))

def flatten(list_of_lists):  
"Flatten one level of nesting." return chain.from_iterable(list_of_lists)

def ncycles(iterable, n):  
"Returns the sequence elements n times." return chain.from_iterable(repeat(tuple(iterable), n))

def loops(n):  
"Loop n times. Like range(n) but without creating integers." \# for \_ in loops(100): ... return repeat(None, n)

def tail(n, iterable):  
"Return an iterator over the last n items." \# tail(3, 'ABCDEFG') → E F G return iter(deque(iterable, maxlen=n))

def consume(iterator, n=None):  
"Advance the iterator n-steps ahead. If n is None, consume entirely." \# Use functions that consume iterators at C speed. if n is None: deque(iterator, maxlen=0) else: next(islice(iterator, n, n), None)

def nth(iterable, n, default=None):  
"Returns the nth item or a default value." return next(islice(iterable, n, None), default)

def quantify(iterable, predicate=bool):  
"Given a predicate that returns True or False, count the True results." return sum(map(predicate, iterable))

def first_true(iterable, default=False, predicate=None):  
"Returns the first true value or the *default* if there is no true value." \# first_true(\[a, b, c\], x) → a or b or c or x \# first_true(\[a, b\], x, f) → a if f(a) else b if f(b) else x return next(filter(predicate, iterable), default)

def all_equal(iterable, key=None):  
"Returns True if all the elements are equal to each other." \# all_equal('4٤௪౪໔', key=int) → True return len(take(2, groupby(iterable, key))) \<= 1

\# ==== Data pipelines ====

def unique_justseen(iterable, key=None):  
"Yield unique elements, preserving order. Remember only the element just seen." \# unique_justseen('AAAABBBCCDAABBB') → A B C D A B \# unique_justseen('ABBcCAD', str.casefold) → A B c A D if key is None: return map(itemgetter(0), groupby(iterable)) return map(next, map(itemgetter(1), groupby(iterable, key)))

def unique_everseen(iterable, key=None):  
"Yield unique elements, preserving order. Remember all elements ever seen." \# unique_everseen('AAAABBBCCDAABBB') → A B C D \# unique_everseen('ABBcCAD', str.casefold) → A B c D seen = set() if key is None: for element in filterfalse(seen.\_\_contains\_\_, iterable): seen.add(element) yield element else: for element in iterable: k = key(element) if k not in seen: seen.add(k) yield element

def unique(iterable, key=None, reverse=False):  
"Yield unique elements in sorted order. Supports unhashable inputs." \# unique(\[\[1, 2\], \[3, 4\], \[1, 2\]\]) → \[1, 2\] \[3, 4\] sequenced = sorted(iterable, key=key, reverse=reverse) return unique_justseen(sequenced, key=key)

def sliding_window(iterable, n):  
"Collect data into overlapping fixed-length chunks or blocks." \# sliding_window('ABCDEFG', 3) → ABC BCD CDE DEF EFG iterator = iter(iterable) window = deque(islice(iterator, n - 1), maxlen=n) for x in iterator: window.append(x) yield tuple(window)

def grouper(iterable, n, *, incomplete='fill', fillvalue=None): "Collect data into non-overlapping fixed-length chunks or blocks." \# grouper('ABCDEFG', 3, fillvalue='x') → ABC DEF Gxx \# grouper('ABCDEFG', 3, incomplete='strict') → ABC DEF ValueError \# grouper('ABCDEFG', 3, incomplete='ignore') → ABC DEF iterators = \[iter(iterable)\]* n  
match incomplete:  
case 'fill':  
return zip_longest(\*iterators, fillvalue=fillvalue)

case 'strict':  
return zip(\*iterators, strict=True)

case 'ignore':  
return zip(\*iterators)

case \_:  
raise ValueError('Expected fill, strict, or ignore')

def roundrobin(\*iterables):  
"Visit input iterables in a cycle until each is exhausted." \# roundrobin('ABC', 'D', 'EF') → A D E B F C \# Algorithm credited to George Sakkis iterators = map(iter, iterables) for num_active in range(len(iterables), 0, -1): iterators = cycle(islice(iterators, num_active)) yield from map(next, iterators)

def subslices(seq):  
"Return all contiguous non-empty subslices of a sequence." \# subslices('ABCD') → A AB ABC ABCD B BC BCD C CD D slices = starmap(slice, combinations(range(len(seq) + 1), 2)) return map(getitem, repeat(seq), slices)

def derangements(iterable, r=None):  
"Produce r length permutations without fixed points." \# derangements('ABCD') → BADC BCDA BDAC CADB CDAB CDBA DABC DCAB DCBA \# Algorithm credited to Stefan Pochmann seq = tuple(iterable) pos = tuple(range(len(seq))) have_moved = map(map, repeat(is_not), repeat(pos), permutations(pos, r=r)) valid_derangements = map(all, have_moved) return compress(permutations(seq, r=r), valid_derangements)

def iter_index(iterable, value, start=0, stop=None):  
"Return indices where a value occurs in a sequence or iterable." \# iter_index('AABCADEAF', 'A') → 0 1 4 7 seq_index = getattr(iterable, 'index', None) if seq_index is None: iterator = islice(iterable, start, stop) for i, element in enumerate(iterator, start): if element is value or element == value: yield i else: stop = len(iterable) if stop is None else stop i = start with suppress(ValueError): while True: yield (i := seq_index(value, i, stop)) i += 1

def iter_except(function, exception, first=None):  
"Convert a call-until-exception interface to an iterator interface." \# iter_except(d.popitem, KeyError) → non-blocking dictionary iterator with suppress(exception): if first is not None: yield first() while True: yield function()

\# ==== Mathematical operations ====

def multinomial(\*counts):  
"Number of distinct arrangements of a multiset." \# Counter('abracadabra').values() → 5 2 2 1 1 \# multinomial(5, 2, 2, 1, 1) → 83160 return prod(map(comb, accumulate(counts), counts))

def powerset(iterable):  
"Subsequences of the iterable from shortest to longest." \# powerset(\[1,2,3\]) → () (1,) (2,) (3,) (1,2) (1,3) (2,3) (1,2,3) s = list(iterable) return chain.from_iterable(combinations(s, r) for r in range(len(s)+1))

def sum_of_squares(iterable):  
"Add up the squares of the input values." \# sum_of_squares(\[10, 20, 30\]) → 1400 return sumprod(\*tee(iterable))

\# ==== Matrix operations ====

def reshape(matrix, columns):  
"Reshape a 2-D matrix to have a given number of columns." \# reshape(\[(0, 1), (2, 3), (4, 5)\], 3) → (0, 1, 2) (3, 4, 5) return batched(chain.from_iterable(matrix), columns, strict=True)

def transpose(matrix):  
"Swap the rows and columns of a 2-D matrix." \# transpose(\[(1, 2, 3), (11, 22, 33)\]) → (1, 11) (2, 22) (3, 33) return zip(\*matrix, strict=True)

def matmul(m1, m2):  
"Multiply two matrices." \# matmul(\[(7, 5), (3, 5)\], \[(2, 5), (7, 9)\]) → (49, 80) (41, 60) n = len(m2\[0\]) return batched(starmap(sumprod, product(m1, transpose(m2))), n)

\# ==== Polynomial arithmetic ====

def convolve(signal, kernel):  
"""Discrete linear convolution of two iterables. Equivalent to polynomial multiplication.

Convolutions are mathematically commutative; however, the inputs are evaluated differently. The signal is consumed lazily and can be infinite. The kernel is fully consumed before the calculations begin.

Article: <https://betterexplained.com/articles/intuitive-convolution/> Video: <https://www.youtube.com/watch?v=KuXjwB4LzSA> """ \# convolve(\[1, -1, -20\], \[1, -3\]) → 1 -4 -17 60 \# convolve(data, \[0.25, 0.25, 0.25, 0.25\]) → Moving average (blur) \# convolve(data, \[1/2, 0, -1/2\]) → 1st derivative estimate \# convolve(data, \[1, -2, 1\]) → 2nd derivative estimate kernel = tuple(kernel)\[::-1\] n = len(kernel) padded_signal = chain(repeat(0, n-1), signal, repeat(0, n-1)) windowed_signal = sliding_window(padded_signal, n) return map(sumprod, repeat(kernel), windowed_signal)

def polynomial_from_roots(roots):  
"""Compute a polynomial's coefficients from its roots.

> (x - 5) (x + 4) (x - 3) expands to: x³ -4x² -17x + 60

""" \# polynomial_from_roots(\[5, -4, 3\]) → \[1, -4, -17, 60\] factors = zip(repeat(1), map(neg, roots)) return list(reduce(convolve, factors, \[1\]))

def polynomial_eval(coefficients, x):  
"""Evaluate a polynomial at a specific value.

Computes with better numeric stability than Horner's method. """ \# Evaluate x³ -4x² -17x + 60 at x = 5 \# polynomial_eval(\[1, -4, -17, 60\], x=5) → 0 n = len(coefficients) if not n: return type(x)(0) powers = map(pow, repeat(x), reversed(range(n))) return sumprod(coefficients, powers)

def polynomial_derivative(coefficients):  
"""Compute the first derivative of a polynomial.

> f(x) = x³ -4x² -17x + 60 f'(x) = 3x² -8x -17

""" \# polynomial_derivative(\[1, -4, -17, 60\]) → \[3, -8, -17\] n = len(coefficients) powers = reversed(range(1, n)) return list(map(mul, coefficients, powers))

\# ==== Number theory ====

def sieve(n):  
"Primes less than n." \# sieve(30) → 2 3 5 7 11 13 17 19 23 29 if n \> 2: yield 2 data = bytearray((0, 1)) \* (n // 2) for p in iter_index(data, 1, start=3, stop=isqrt(n) + 1): data\[p\*p : n : p+p\] = bytes(len(range(p\*p, n, p+p))) yield from iter_index(data, 1, start=3)

def factor(n):  
"Prime factors of n." \# factor(99) → 3 3 11 \# factor(1_000_000_000_000_007) → 47 59 360620266859 \# factor(1_000_000_000_000_403) → 1000000000000403 for prime in sieve(isqrt(n) + 1): while not n % prime: yield prime n //= prime if n == 1: return if n \> 1: yield n

def is_prime(n):  
"Return True if n is prime." \# is_prime(1_000_000_000_000_403) → True return n \> 1 and next(factor(n)) == n

def totient(n):  
"Count of natural numbers up to n that are coprime to n." \# <https://mathworld.wolfram.com/TotientFunction.html> \# totient(12) → 4 because len(\[1, 5, 7, 11\]) == 4 for prime in set(factor(n)): n -= n // prime return n

\# ==== Running statistics ====

def running_mean(iterable):  
"Average of values seen so far." \# running_mean(\[37, 33, 38, 28\]) → 37 35 36 34 return map(truediv, accumulate(iterable), count(1))

def running_min(iterable):  
"Smallest of values seen so far." \# running_min(\[37, 33, 38, 28\]) → 37 33 33 28 return accumulate(iterable, func=min)

def running_max(iterable):  
"Largest of values seen so far." \# running_max(\[37, 33, 38, 28\]) → 37 37 38 38 return accumulate(iterable, func=max)

def running_median(iterable):  
"Median of values seen so far." \# running_median(\[37, 33, 38, 28\]) → 37 35 37 35 read = iter(iterable).\_\_next\_\_ lo = \[\] \# max-heap hi = \[\] \# min-heap the same size as or one smaller than lo with suppress(StopIteration): while True: heappush_max(lo, heappushpop(hi, read())) yield lo\[0\] heappush(hi, heappushpop_max(lo, read())) yield (lo\[0\] + hi\[0\]) / 2

def running_statistics(iterable):  
"Aggregate statistics for values seen so far." \# Generate tuples: (size, minimum, median, maximum, mean) t0, t1, t2, t3 = tee(iterable, 4) return zip( count(1), running_min(t0), running_median(t1), running_max(t2), running_mean(t3), )

</div>

<div class="doctest" data-hide="">

These examples no longer appear in the docs but are guaranteed to keep working.

\>\>\> amounts = \[120.15, 764.05, 823.14\] \>\>\> for checknum, amount in zip(count(1200), amounts): ... print('Check %d is for \$%.2f' % (checknum, amount)) ... Check 1200 is for \$120.15 Check 1201 is for \$764.05 Check 1202 is for \$823.14

\>\>\> import operator \>\>\> for cube in map(operator.pow, range(1,4), repeat(3)): ... print(cube) ... 1 8 27

\>\>\> reportlines = \['EuroPython', 'Roster', '', 'alex', '', 'laura', '', 'martin', '', 'walter', '', 'samuele'\] \>\>\> for name in islice(reportlines, 3, None, 2): ... print(name.title()) ... Alex Laura Martin Walter Samuele

\>\>\> from operator import itemgetter \>\>\> d = dict(a=1, b=2, c=1, d=2, e=1, f=2, g=3) \>\>\> di = sorted(sorted(d.items()), key=itemgetter(1)) \>\>\> for k, g in groupby(di, itemgetter(1)): ... print(k, list(map(itemgetter(0), g))) ... 1 \['a', 'c', 'e'\] 2 \['b', 'd', 'f'\] 3 \['g'\]

\# Find runs of consecutive numbers using groupby. The key to the solution \# is differencing with a range so that consecutive numbers all appear in \# same group. \>\>\> data = \[ 1, 4,5,6, 10, 15,16,17,18, 22, 25,26,27,28\] \>\>\> for k, g in groupby(enumerate(data), lambda t:t\[0\]-t\[1\]): ... print(list(map(operator.itemgetter(1), g))) ... \[1\] \[4, 5, 6\] \[10\] \[15, 16, 17, 18\] \[22\] \[25, 26, 27, 28\]

Now, we test all of the itertool recipes

\>\>\> take(10, count()) \[0, 1, 2, 3, 4, 5, 6, 7, 8, 9\] \>\>\> \# Verify that the input is consumed lazily \>\>\> it = iter('abcdef') \>\>\> take(3, it) \['a', 'b', 'c'\] \>\>\> list(it) \['d', 'e', 'f'\]

\>\>\> list(prepend(1, \[2, 3, 4\])) \[1, 2, 3, 4\]

\>\>\> list(enumerate('abc')) \[(0, 'a'), (1, 'b'), (2, 'c')\]

\>\>\> for \_ in loops(5): ... print('hi') ... hi hi hi hi hi

\>\>\> list(tail(3, 'ABCDEFG')) \['E', 'F', 'G'\] \>\>\> \# Verify the input is consumed greedily \>\>\> input_iterator = iter('ABCDEFG') \>\>\> output_iterator = tail(3, input_iterator) \>\>\> list(input_iterator) \[\]

\>\>\> it = iter(range(10)) \>\>\> consume(it, 3) \>\>\> \# Verify the input is consumed lazily \>\>\> next(it) 3 \>\>\> \# Verify the input is consumed completely \>\>\> consume(it) \>\>\> next(it, 'Done') 'Done'

\>\>\> nth('abcde', 3) 'd' \>\>\> nth('abcde', 9) is None True \>\>\> \# Verify that the input is consumed lazily \>\>\> it = iter('abcde') \>\>\> nth(it, 2) 'c' \>\>\> list(it) \['d', 'e'\]

\>\>\> \[all_equal(s) for s in ('', 'A', 'AAAA', 'AAAB', 'AAABA')\] \[True, True, True, False, False\] \>\>\> \[all_equal(s, key=str.casefold) for s in ('', 'A', 'AaAa', 'AAAB', 'AAABA')\] \[True, True, True, False, False\] \>\>\> \# Verify that the input is consumed lazily and that only \>\>\> \# one element of a second equivalence class is used to disprove \>\>\> \# the assertion that all elements are equal. \>\>\> it = iter('aaabbbccc') \>\>\> all_equal(it) False \>\>\> ''.join(it) 'bbccc'

\>\>\> quantify(range(99), lambda x: x%2==0) 50 \>\>\> quantify(\[True, False, False, True, True\]) 3 \>\>\> quantify(range(12), predicate=lambda x: x%2==1) 6

\>\>\> a = \[\[1, 2, 3\], \[4, 5, 6\]\] \>\>\> list(flatten(a)) \[1, 2, 3, 4, 5, 6\]

\>\>\> list(ncycles('abc', 3)) \['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c'\] \>\>\> \# Verify greedy consumption of input iterator \>\>\> input_iterator = iter('abc') \>\>\> output_iterator = ncycles(input_iterator, 3) \>\>\> list(input_iterator) \[\]

\>\>\> sum_of_squares(\[10, 20, 30\]) 1400

\>\>\> list(reshape(\[(0, 1), (2, 3), (4, 5)\], 3)) \[(0, 1, 2), (3, 4, 5)\] \>\>\> M = \[(0, 1, 2, 3), (4, 5, 6, 7), (8, 9, 10, 11)\] \>\>\> list(reshape(M, 1)) \[(0,), (1,), (2,), (3,), (4,), (5,), (6,), (7,), (8,), (9,), (10,), (11,)\] \>\>\> list(reshape(M, 2)) \[(0, 1), (2, 3), (4, 5), (6, 7), (8, 9), (10, 11)\] \>\>\> list(reshape(M, 3)) \[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, 10, 11)\] \>\>\> list(reshape(M, 4)) \[(0, 1, 2, 3), (4, 5, 6, 7), (8, 9, 10, 11)\] \>\>\> list(reshape(M, 5)) Traceback (most recent call last): ... ValueError: batched(): incomplete batch \>\>\> list(reshape(M, 6)) \[(0, 1, 2, 3, 4, 5), (6, 7, 8, 9, 10, 11)\] \>\>\> list(reshape(M, 12)) \[(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)\]

\>\>\> list(transpose(\[(1, 2, 3), (11, 22, 33)\])) \[(1, 11), (2, 22), (3, 33)\] \>\>\> \# Verify that the inputs are consumed lazily \>\>\> input1 = iter(\[1, 2, 3\]) \>\>\> input2 = iter(\[11, 22, 33\]) \>\>\> output_iterator = transpose(\[input1, input2\]) \>\>\> next(output_iterator) (1, 11) \>\>\> list(zip(input1, input2)) \[(2, 22), (3, 33)\]

\>\>\> list(matmul(\[(7, 5), (3, 5)\], \[\[2, 5\], \[7, 9\]\])) \[(49, 80), (41, 60)\] \>\>\> list(matmul(\[\[2, 5\], \[7, 9\], \[3, 4\]\], \[\[7, 11, 5, 4, 9\], \[3, 5, 2, 6, 3\]\])) \[(29, 47, 20, 38, 33), (76, 122, 53, 82, 90), (33, 53, 23, 36, 39)\]

\>\>\> list(convolve(\[1, -1, -20\], \[1, -3\])) == \[1, -4, -17, 60\] True \>\>\> data = \[20, 40, 24, 32, 20, 28, 16\] \>\>\> list(convolve(data, \[0.25, 0.25, 0.25, 0.25\])) \[5.0, 15.0, 21.0, 29.0, 29.0, 26.0, 24.0, 16.0, 11.0, 4.0\] \>\>\> list(convolve(data, \[1, -1\])) \[20, 20, -16, 8, -12, 8, -12, -16\] \>\>\> list(convolve(data, \[1, -2, 1\])) \[20, 0, -36, 24, -20, 20, -20, -4, 16\] \>\>\> \# Verify signal is consumed lazily and the kernel greedily \>\>\> signal_iterator = iter(\[10, 20, 30, 40, 50\]) \>\>\> kernel_iterator = iter(\[1, 2, 3\]) \>\>\> output_iterator = convolve(signal_iterator, kernel_iterator) \>\>\> list(kernel_iterator) \[\] \>\>\> next(output_iterator) 10 \>\>\> next(output_iterator) 40 \>\>\> list(signal_iterator) \[30, 40, 50\]

\>\>\> from fractions import Fraction \>\>\> from decimal import Decimal \>\>\> polynomial_eval(\[1, -4, -17, 60\], x=5) 0 \>\>\> x = 5; x\*\*3 - 4\*x\*\*2 -17\*x + 60 0 \>\>\> polynomial_eval(\[1, -4, -17, 60\], x=2.5) 8.125 \>\>\> x = 2.5; x\*\*3 - 4\*x\*\*2 -17\*x + 60 8.125 \>\>\> polynomial_eval(\[1, -4, -17, 60\], x=Fraction(2, 3)) Fraction(1274, 27) \>\>\> x = Fraction(2, 3); x\*\*3 - 4\*x\*\*2 -17\*x + 60 Fraction(1274, 27) \>\>\> polynomial_eval(\[1, -4, -17, 60\], x=Decimal('1.75')) Decimal('23.359375') \>\>\> x = Decimal('1.75'); x\*\*3 - 4\*x\*\*2 -17\*x + 60 Decimal('23.359375') \>\>\> polynomial_eval(\[\], 2) 0 \>\>\> polynomial_eval(\[\], 2.5) 0.0 \>\>\> polynomial_eval(\[\], Fraction(2, 3)) Fraction(0, 1) \>\>\> polynomial_eval(\[\], Decimal('1.75')) Decimal('0') \>\>\> polynomial_eval(\[11\], 7) == 11 True \>\>\> polynomial_eval(\[11, 2\], 7) == 11 \* 7 + 2 True

\>\>\> polynomial_from_roots(\[5, -4, 3\]) \[1, -4, -17, 60\] \>\>\> factored = lambda x: (x - 5) \* (x + 4) \* (x - 3) \>\>\> expanded = lambda x: x\*\*3 -4\*x\*\*2 -17\*x + 60 \>\>\> all(factored(x) == expanded(x) for x in range(-10, 11)) True

\>\>\> polynomial_derivative(\[1, -4, -17, 60\]) \[3, -8, -17\]

\>\>\> list(iter_index('AABCADEAF', 'A')) \[0, 1, 4, 7\] \>\>\> list(iter_index('AABCADEAF', 'B')) \[2\] \>\>\> list(iter_index('AABCADEAF', 'X')) \[\] \>\>\> list(iter_index('', 'X')) \[\] \>\>\> list(iter_index('AABCADEAF', 'A', 1)) \[1, 4, 7\] \>\>\> list(iter_index(iter('AABCADEAF'), 'A', 1)) \[1, 4, 7\] \>\>\> list(iter_index('AABCADEAF', 'A', 2)) \[4, 7\] \>\>\> list(iter_index(iter('AABCADEAF'), 'A', 2)) \[4, 7\] \>\>\> list(iter_index('AABCADEAF', 'A', 10)) \[\] \>\>\> list(iter_index(iter('AABCADEAF'), 'A', 10)) \[\] \>\>\> list(iter_index('AABCADEAF', 'A', 1, 7)) \[1, 4\] \>\>\> list(iter_index(iter('AABCADEAF'), 'A', 1, 7)) \[1, 4\] \>\>\> \# Verify that ValueErrors not swallowed (gh-107208) \>\>\> def assert_no_value(iterable, forbidden_value): ... for item in iterable: ... if item == forbidden_value: ... raise ValueError ... yield item ... \>\>\> list(iter_index(assert_no_value('AABCADEAF', 'B'), 'A')) Traceback (most recent call last): ... ValueError \>\>\> \# Verify that both paths can find identical NaN values \>\>\> x = float('NaN') \>\>\> y = float('NaN') \>\>\> list(iter_index(\[0, x, x, y, 0\], x)) \[1, 2\] \>\>\> list(iter_index(iter(\[0, x, x, y, 0\]), x)) \[1, 2\] \>\>\> \# Test list input. Lists do not support None for the stop argument \>\>\> list(iter_index(list('AABCADEAF'), 'A')) \[0, 1, 4, 7\] \>\>\> \# Verify that input is consumed lazily \>\>\> input_iterator = iter('AABCADEAF') \>\>\> output_iterator = iter_index(input_iterator, 'A') \>\>\> next(output_iterator) 0 \>\>\> next(output_iterator) 1 \>\>\> next(output_iterator) 4 \>\>\> ''.join(input_iterator) 'DEAF'

\>\>\> \# Verify that the target value can be a sequence. \>\>\> seq = \[\[10, 20\], \[30, 40\], 30, 40, \[30, 40\], 50\] \>\>\> target = \[30, 40\] \>\>\> list(iter_index(seq, target)) \[1, 4\]

\>\>\> \# Verify faithfulness to type specific index() method behaviors. \>\>\> \# For example, bytes and str perform continuous-subsequence searches \>\>\> \# that do not match the general behavior specified \>\>\> \# in collections.abc.Sequence.index(). \>\>\> seq = 'abracadabra' \>\>\> target = 'ab' \>\>\> list(iter_index(seq, target)) \[0, 7\]

\>\>\> list(sieve(30)) \[2, 3, 5, 7, 11, 13, 17, 19, 23, 29\] \>\>\> small_primes = \[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97\] \>\>\> all(list(sieve(n)) == \[p for p in small_primes if p \< n\] for n in range(101)) True \>\>\> len(list(sieve(100))) 25 \>\>\> len(list(sieve(1_000))) 168 \>\>\> len(list(sieve(10_000))) 1229 \>\>\> len(list(sieve(100_000))) 9592 \>\>\> len(list(sieve(1_000_000))) 78498 \>\>\> carmichael = {561, 1105, 1729, 2465, 2821, 6601, 8911} \# <https://oeis.org/A002997> \>\>\> set(sieve(10_000)).isdisjoint(carmichael) True

\>\>\> small_primes = \[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97\] \>\>\> list(filter(is_prime, range(-100, 100))) == small_primes True \>\>\> carmichael = {561, 1105, 1729, 2465, 2821, 6601, 8911} \# <https://oeis.org/A002997> \>\>\> any(map(is_prime, carmichael)) False \>\>\> \# <https://www.wolframalpha.com/input?i=is+128884753939+prime> \>\>\> is_prime(128_884_753_939) \# large prime True \>\>\> is_prime(999953 \* 999983) \# large semiprime False \>\>\> is_prime(1_000_000_000_000_007) \# factor() example False \>\>\> is_prime(1_000_000_000_000_403) \# factor() example True

\>\>\> list(factor(99)) \# Code example 1 \[3, 3, 11\] \>\>\> list(factor(1_000_000_000_000_007)) \# Code example 2 \[47, 59, 360620266859\] \>\>\> list(factor(1_000_000_000_000_403)) \# Code example 3 \[1000000000000403\] \>\>\> list(factor(0)) \[\] \>\>\> list(factor(1)) \[\] \>\>\> list(factor(2)) \[2\] \>\>\> list(factor(3)) \[3\] \>\>\> list(factor(4)) \[2, 2\] \>\>\> list(factor(5)) \[5\] \>\>\> list(factor(6)) \[2, 3\] \>\>\> list(factor(7)) \[7\] \>\>\> list(factor(8)) \[2, 2, 2\] \>\>\> list(factor(9)) \[3, 3\] \>\>\> list(factor(10)) \[2, 5\] \>\>\> list(factor(128_884_753_939)) \# large prime \[128884753939\] \>\>\> list(factor(999953 \* 999983)) \# large semiprime \[999953, 999983\] \>\>\> list(factor(6 \*\* 20)) == \[2\] \* 20 + \[3\] \* 20 \# large power True \>\>\> list(factor(909_909_090_909)) \# large multiterm composite \[3, 3, 7, 13, 13, 751, 113797\] \>\>\> math.prod(\[3, 3, 7, 13, 13, 751, 113797\]) 909909090909 \>\>\> all(math.prod(factor(n)) == n for n in range(1, 2_000)) True \>\>\> all(set(factor(n)) \<= set(sieve(n+1)) for n in range(2_000)) True \>\>\> all(list(factor(n)) == sorted(factor(n)) for n in range(2_000)) True

\>\>\> totient(0) \# <https://www.wolframalpha.com/input?i=totient+0> 0 \>\>\> first_totients = \[1, 1, 2, 2, 4, 2, 6, 4, 6, 4, 10, 4, 12, 6, 8, 8, 16, 6, ... 18, 8, 12, 10, 22, 8, 20, 12, 18, 12, 28, 8, 30, 16, 20, 16, 24, 12, 36, 18, ... 24, 16, 40, 12, 42, 20, 24, 22, 46, 16, 42, 20, 32, 24, 52, 18, 40, 24, 36, ... 28, 58, 16, 60, 30, 36, 32, 48, 20, 66, 32, 44\] \# <https://oeis.org/A000010> ... \>\>\> list(map(totient, range(1, 70))) == first_totients True \>\>\> reference_totient = lambda n: sum(math.gcd(t, n) == 1 for t in range(1, n+1)) \>\>\> all(totient(n) == reference_totient(n) for n in range(1000)) True \>\>\> totient(128_884_753_939) == 128_884_753_938 \# large prime True \>\>\> totient(999953 \* 999983) == 999952 \* 999982 \# large semiprime True \>\>\> totient(6 \*\* 20) == 1 \* 2\*\*19 \* 2 \* 3\*\*19 \# repeated primes True

\>\>\> list(flatten(\[('a', 'b'), (), ('c', 'd', 'e'), ('f',), ('g', 'h', 'i')\])) \['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'\]

\>\>\> list(repeatfunc(pow, 5, 2, 3)) \[8, 8, 8, 8, 8\] \>\>\> take(5, map(int, repeatfunc(random.random))) \[0, 0, 0, 0, 0\] \>\>\> random.seed(85753098575309) \>\>\> list(repeatfunc(random.random, 3)) \[0.16370491282496968, 0.45889608687313455, 0.3747076837820118\] \>\>\> list(repeatfunc(chr, 3, 65)) \['A', 'A', 'A'\] \>\>\> list(repeatfunc(pow, 3, 2, 5)) \[32, 32, 32\]

\>\>\> list(grouper('abcdefg', 3, fillvalue='x')) \[('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'x', 'x')\]

\>\>\> it = grouper('abcdefg', 3, incomplete='strict') \>\>\> next(it) ('a', 'b', 'c') \>\>\> next(it) ('d', 'e', 'f') \>\>\> next(it) Traceback (most recent call last): ... ValueError: zip() argument 2 is shorter than argument 1

\>\>\> list(grouper('abcdefg', n=3, incomplete='ignore')) \[('a', 'b', 'c'), ('d', 'e', 'f')\]

\>\>\> list(sliding_window('ABCDEFG', 1)) \[('A',), ('B',), ('C',), ('D',), ('E',), ('F',), ('G',)\] \>\>\> list(sliding_window('ABCDEFG', 2)) \[('A', 'B'), ('B', 'C'), ('C', 'D'), ('D', 'E'), ('E', 'F'), ('F', 'G')\] \>\>\> list(sliding_window('ABCDEFG', 3)) \[('A', 'B', 'C'), ('B', 'C', 'D'), ('C', 'D', 'E'), ('D', 'E', 'F'), ('E', 'F', 'G')\] \>\>\> list(sliding_window('ABCDEFG', 4)) \[('A', 'B', 'C', 'D'), ('B', 'C', 'D', 'E'), ('C', 'D', 'E', 'F'), ('D', 'E', 'F', 'G')\] \>\>\> list(sliding_window('ABCDEFG', 5)) \[('A', 'B', 'C', 'D', 'E'), ('B', 'C', 'D', 'E', 'F'), ('C', 'D', 'E', 'F', 'G')\] \>\>\> list(sliding_window('ABCDEFG', 6)) \[('A', 'B', 'C', 'D', 'E', 'F'), ('B', 'C', 'D', 'E', 'F', 'G')\] \>\>\> list(sliding_window('ABCDEFG', 7)) \[('A', 'B', 'C', 'D', 'E', 'F', 'G')\] \>\>\> list(sliding_window('ABCDEFG', 8)) \[\] \>\>\> try: ... list(sliding_window('ABCDEFG', -1)) ... except ValueError: ... 'zero or negative n not supported' ... 'zero or negative n not supported' \>\>\> try: ... list(sliding_window('ABCDEFG', 0)) ... except ValueError: ... 'zero or negative n not supported' ... 'zero or negative n not supported'

\>\>\> list(roundrobin('abc', 'd', 'ef')) \['a', 'd', 'e', 'b', 'f', 'c'\] \>\>\> ranges = \[range(5, 1000), range(4, 3000), range(0), range(3, 2000), range(2, 5000), range(1, 3500)\] \>\>\> collections.Counter(roundrobin(*ranges)) == collections.Counter(chain(*ranges)) True \>\>\> \# Verify that the inputs are consumed lazily \>\>\> input_iterators = list(map(iter, \['abcd', 'ef', '', 'ghijk', 'l', 'mnopqr'\])) \>\>\> output_iterator = roundrobin(*input_iterators) \>\>\> ''.join(islice(output_iterator, 10)) 'aeglmbfhnc' \>\>\> ''.join(chain(*input_iterators)) 'dijkopqr'

\>\>\> list(subslices('ABCD')) \['A', 'AB', 'ABC', 'ABCD', 'B', 'BC', 'BCD', 'C', 'CD', 'D'\]

\>\>\> ' '.join(map(''.join, derangements('ABCD'))) 'BADC BCDA BDAC CADB CDAB CDBA DABC DCAB DCBA' \>\>\> ' '.join(map(''.join, derangements('ABCD', 3))) 'BAD BCA BCD BDA CAB CAD CDA CDB DAB DCA DCB' \>\>\> ' '.join(map(''.join, derangements('ABCD', 2))) 'BA BC BD CA CD DA DC' \>\>\> ' '.join(map(''.join, derangements('ABCD', 1))) 'B C D' \>\>\> ' '.join(map(''.join, derangements('ABCD', 0))) '' \>\>\> \# Compare number of derangements to <https://oeis.org/A000166> \>\>\> \[len(list(derangements(range(n)))) for n in range(10)\] \[1, 0, 1, 2, 9, 44, 265, 1854, 14833, 133496\] \>\>\> \# Verify that identical objects are treated as unique by position \>\>\> identical = 'X' \>\>\> distinct = 'x' \>\>\> seq1 = ('A', identical, 'B', identical) \>\>\> result1 = ' '.join(map(''.join, derangements(seq1))) \>\>\> result1 'XAXB XBXA XXAB BAXX BXAX BXXA XAXB XBAX XBXA' \>\>\> seq2 = ('A', identical, 'B', distinct) \>\>\> result2 = ' '.join(map(''.join, derangements(seq2))) \>\>\> result2 'XAxB XBxA XxAB BAxX BxAX BxXA xAXB xBAX xBXA' \>\>\> result1 == result2 False \>\>\> result1.casefold() == result2.casefold() True

\>\>\> list(powerset(\[1,2,3\])) \[(), (1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)\] \>\>\> all(len(list(powerset(range(n)))) == 2\*\*n for n in range(18)) True \>\>\> list(powerset('abcde')) == sorted(sorted(set(powerset('abcde'))), key=len) True

\>\>\> list(unique_everseen('AAAABBBCCDAABBB')) \['A', 'B', 'C', 'D'\] \>\>\> list(unique_everseen('ABBCcAD', str.casefold)) \['A', 'B', 'C', 'D'\] \>\>\> list(unique_everseen('ABBcCAD', str.casefold)) \['A', 'B', 'c', 'D'\] \>\>\> \# Verify that the input is consumed lazily \>\>\> input_iterator = iter('AAAABBBCCDAABBB') \>\>\> output_iterator = unique_everseen(input_iterator) \>\>\> next(output_iterator) 'A' \>\>\> ''.join(input_iterator) 'AAABBBCCDAABBB'

\>\>\> list(unique_justseen('AAAABBBCCDAABBB')) \['A', 'B', 'C', 'D', 'A', 'B'\] \>\>\> list(unique_justseen('ABBCcAD', str.casefold)) \['A', 'B', 'C', 'A', 'D'\] \>\>\> list(unique_justseen('ABBcCAD', str.casefold)) \['A', 'B', 'c', 'A', 'D'\] \>\>\> \# Verify that the input is consumed lazily \>\>\> input_iterator = iter('AAAABBBCCDAABBB') \>\>\> output_iterator = unique_justseen(input_iterator) \>\>\> next(output_iterator) 'A' \>\>\> ''.join(input_iterator) 'AAABBBCCDAABBB'

\>\>\> list(unique(\[\[1, 2\], \[3, 4\], \[1, 2\]\])) \[\[1, 2\], \[3, 4\]\] \>\>\> list(unique('ABBcCAD', str.casefold)) \['A', 'B', 'c', 'D'\] \>\>\> list(unique('ABBcCAD', str.casefold, reverse=True)) \['D', 'c', 'B', 'A'\]

\>\>\> d = dict(a=1, b=2, c=3) \>\>\> it = iter_except(d.popitem, KeyError) \>\>\> d\['d'\] = 4 \>\>\> next(it) ('d', 4) \>\>\> next(it) ('c', 3) \>\>\> next(it) ('b', 2) \>\>\> d\['e'\] = 5 \>\>\> next(it) ('e', 5) \>\>\> next(it) ('a', 1) \>\>\> next(it, 'empty') 'empty'

\>\>\> first_true('ABC0DEF1', '9', str.isdigit) '0' \>\>\> \# Verify that inputs are consumed lazily \>\>\> it = iter('ABC0DEF1') \>\>\> first_true(it, predicate=str.isdigit) '0' \>\>\> ''.join(it) 'DEF1'

\>\>\> multinomial(5, 2, 2, 1, 1) 83160 \>\>\> word = 'coffee' \>\>\> multinomial(\*Counter(word).values()) == len(set(permutations(word))) True

\>\>\> list(running_mean(\[8.5, 9.5, 7.5, 6.5\])) \[8.5, 9.0, 8.5, 8.0\] \>\>\> list(running_mean(\[37, 33, 38, 28\])) \[37.0, 35.0, 36.0, 34.0\]

\>\>\> list(running_min(\[37, 33, 38, 28\])) \[37, 33, 33, 28\]

\>\>\> list(running_max(\[37, 33, 38, 28\])) \[37, 37, 38, 38\]

\>\>\> list(running_median(\[37, 33, 38, 28\])) \[37, 35.0, 37, 35.0\]

\>\>\> list(running_statistics(\[37, 33, 38, 28\])) \[(1, 37, 37, 37, 37.0), (2, 33, 35.0, 37, 35.0), (3, 33, 37, 38, 36.0), (4, 28, 35.0, 38, 34.0)\]

</div>

<div class="testcode" data-hide="">

\# Old recipes and their tests which are guaranteed to continue to work.

def tabulate(function, start=0):  
"Return function(0), function(1), ..." return map(function, count(start))

def old_sumprod_recipe(vec1, vec2):  
"Compute a sum of products." return sum(starmap(operator.mul, zip(vec1, vec2, strict=True)))

def dotproduct(vec1, vec2):  
return sum(map(operator.mul, vec1, vec2))

def pad_none(iterable):  
"""Returns the sequence elements and then returns None indefinitely.

Useful for emulating the behavior of the built-in map() function. """ return chain(iterable, repeat(None))

def triplewise(iterable):  
"Return overlapping triplets from an iterable" \# triplewise('ABCDEFG') → ABC BCD CDE DEF EFG for (a, \_), (b, c) in pairwise(pairwise(iterable)): yield a, b, c

def nth_combination(iterable, r, index):  
"Equivalent to list(combinations(iterable, r))\[index\]" pool = tuple(iterable) n = len(pool) c = math.comb(n, r) if index \< 0: index += c if index \< 0 or index \>= c: raise IndexError result = \[\] while r: c, n, r = c\*r//n, n-1, r-1 while index \>= c: index -= c c, n = c\*(n-r)//n, n-1 result.append(pool\[-1-n\]) return tuple(result)

def before_and_after(predicate, it):  
""" Variant of takewhile() that allows complete  
access to the remainder of the iterator.

\>\>\> it = iter('ABCdEfGhI') \>\>\> all_upper, remainder = before_and_after(str.isupper, it) \>\>\> ''.join(all_upper) 'ABC' \>\>\> ''.join(remainder) \# takewhile() would lose the 'd' 'dEfGhI'

Note that the true iterator must be fully consumed before the remainder iterator can generate valid results.

""" it = iter(it) transition = \[\]

def true_iterator():  
for elem in it:  
if predicate(elem):  
yield elem

else:  
transition.append(elem) return

return true_iterator(), chain(transition, it)

def partition(predicate, iterable):  
"""Partition entries into false entries and true entries.

If *predicate* is slow, consider wrapping it with functools.lru_cache(). """ \# partition(is_odd, range(10)) → 0 2 4 6 8 and 1 3 5 7 9 t1, t2 = tee(iterable) return filterfalse(predicate, t1), filter(predicate, t2)

</div>

<div class="doctest" data-hide="">

\>\>\> list(islice(tabulate(lambda x: 2\*x), 4)) \[0, 2, 4, 6\]

\>\>\> dotproduct(\[1,2,3\], \[4,5,6\]) 32

\>\>\> old_sumprod_recipe(\[1,2,3\], \[4,5,6\]) 32

\>\>\> list(islice(pad_none('abc'), 0, 6)) \['a', 'b', 'c', None, None, None\]

\>\>\> list(triplewise('ABCDEFG')) \[('A', 'B', 'C'), ('B', 'C', 'D'), ('C', 'D', 'E'), ('D', 'E', 'F'), ('E', 'F', 'G')\]

\>\>\> population = 'ABCDEFGH' \>\>\> for r in range(len(population) + 1): ... seq = list(combinations(population, r)) ... for i in range(len(seq)): ... assert nth_combination(population, r, i) == seq\[i\] ... for i in range(-len(seq), 0): ... assert nth_combination(population, r, i) == seq\[i\] ... \>\>\> iterable = 'abcde' \>\>\> r = 3 \>\>\> combos = list(combinations(iterable, r)) \>\>\> all(nth_combination(iterable, r, i) == comb for i, comb in enumerate(combos)) True

\>\>\> it = iter('ABCdEfGhI') \>\>\> all_upper, remainder = before_and_after(str.isupper, it) \>\>\> ''.join(all_upper) 'ABC' \>\>\> ''.join(remainder) 'dEfGhI'

\>\>\> def is_odd(x): ... return x % 2 == 1 ... \>\>\> evens, odds = partition(is_odd, range(10)) \>\>\> list(evens) \[0, 2, 4, 6, 8\] \>\>\> list(odds) \[1, 3, 5, 7, 9\] \>\>\> \# Verify that the input is consumed lazily \>\>\> input_iterator = iter(range(10)) \>\>\> evens, odds = partition(is_odd, input_iterator) \>\>\> next(odds) 1 \>\>\> next(odds) 3 \>\>\> next(evens) 0 \>\>\> list(input_iterator) \[4, 5, 6, 7, 8, 9\]

</div>

# `!json` --- JSON encoder and decoder

<div class="module" data-synopsis="Encode and decode the JSON format.">

json

</div>

**Source code:** `Lib/json/__init__.py`

------------------------------------------------------------------------

[JSON (JavaScript Object Notation)](https://json.org), specified by `7159` (which obsoletes `4627`) and by [ECMA-404](https://ecma-international.org/publications-and-standards/standards/ecma-404/), is a lightweight data interchange format inspired by [JavaScript](https://en.wikipedia.org/wiki/JavaScript) object literal syntax (although it is not a strict subset of JavaScript[^1] ).

> [!NOTE]
> The term "object" in the context of JSON processing in Python can be ambiguous. All values in Python are objects. In JSON, an object refers to any data wrapped in curly braces, similar to a Python dictionary.

> [!WARNING]
> Be cautious when parsing JSON data from untrusted sources. A malicious JSON string may cause the decoder to consume considerable CPU and memory resources. Limiting the size of data to be parsed is recommended.

This module exposes an API familiar to users of the standard library `marshal` and `pickle` modules.

Encoding basic Python object hierarchies:

    >>> import json
    >>> json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])
    '["foo", {"bar": ["baz", null, 1.0, 2]}]'
    >>> print(json.dumps("\"foo\bar"))
    "\"foo\bar"
    >>> print(json.dumps('\u1234'))
    "\u1234"
    >>> print(json.dumps('\\'))
    "\\"
    >>> print(json.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True))
    {"a": 0, "b": 0, "c": 0}
    >>> from io import StringIO
    >>> io = StringIO()
    >>> json.dump(['streaming API'], io)
    >>> io.getvalue()
    '["streaming API"]'

Compact encoding:

    >>> import json
    >>> json.dumps([1, 2, 3, {'4': 5, '6': 7}], separators=(',', ':'))
    '[1,2,3,{"4":5,"6":7}]'

Pretty printing:

    >>> import json
    >>> print(json.dumps({'6': 7, '4': 5}, sort_keys=True, indent=4))
    {
        "4": 5,
        "6": 7
    }

Customizing JSON object encoding:

    >>> import json
    >>> def custom_json(obj):
    ...     if isinstance(obj, complex):
    ...         return {'__complex__': True, 'real': obj.real, 'imag': obj.imag}
    ...     raise TypeError(f'Cannot serialize object of {type(obj)}')
    ...
    >>> json.dumps(1 + 2j, default=custom_json)
    '{"__complex__": true, "real": 1.0, "imag": 2.0}'

Decoding JSON:

    >>> import json
    >>> json.loads('["foo", {"bar":["baz", null, 1.0, 2]}]')
    ['foo', {'bar': ['baz', None, 1.0, 2]}]
    >>> json.loads('"\\"foo\\bar"')
    '"foo\x08ar'
    >>> from io import StringIO
    >>> io = StringIO('["streaming API"]')
    >>> json.load(io)
    ['streaming API']

Customizing JSON object decoding:

    >>> import json
    >>> def as_complex(dct):
    ...     if '__complex__' in dct:
    ...         return complex(dct['real'], dct['imag'])
    ...     return dct
    ...
    >>> json.loads('{"__complex__": true, "real": 1, "imag": 2}',
    ...     object_hook=as_complex)
    (1+2j)
    >>> import decimal
    >>> json.loads('1.1', parse_float=decimal.Decimal)
    Decimal('1.1')
    >>> json.loads('{"name": "point", "coords": [[1, 2], [3, 4]]}',
    ...     array_hook=tuple)  # doctest: +SKIP
    {'name': 'point', 'coords': ((1, 2), (3, 4))}
    >>> json.loads('{"name": "point", "coords": [1, 2]}',
    ...     object_pairs_hook=frozendict, array_hook=tuple)  # doctest: +SKIP
    frozendict({'name': 'point', 'coords': (1, 2)})

Extending `JSONEncoder`:

    >>> import json
    >>> class ComplexEncoder(json.JSONEncoder):
    ...     def default(self, obj):
    ...         if isinstance(obj, complex):
    ...             return [obj.real, obj.imag]
    ...         # Let the base class default method raise the TypeError
    ...         return super().default(obj)
    ...
    >>> json.dumps(2 + 1j, cls=ComplexEncoder)
    '[2.0, 1.0]'
    >>> ComplexEncoder().encode(2 + 1j)
    '[2.0, 1.0]'
    >>> list(ComplexEncoder().iterencode(2 + 1j))
    ['[2.0', ', 1.0', ']']

Using `!json` from the shell to validate and pretty-print:

```
$ echo '{"json":"obj"}' | python -m json
{
    "json": "obj"
}
$ echo '{1.2:3.4}' | python -m json
Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
```

See `json-commandline` for detailed documentation.

> [!NOTE]
> JSON is a subset of [YAML](https://yaml.org/) 1.2. The JSON produced by this module's default settings (in particular, the default *separators* value) is also a subset of YAML 1.0 and 1.1. This module can thus also be used as a YAML serializer.

> [!NOTE]
> This module's encoders and decoders preserve input and output order by default. Order is only lost if the underlying containers are unordered.

## Basic Usage

<div class="function">

dump(obj, fp, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, default=None, sort_keys=False,*\*kw)

Serialize *obj* as a JSON formatted stream to *fp* (a `.write()`-supporting `file-like object`) using this `Python-to-JSON conversion table
<py-to-json-table>`.

> [!NOTE]
> Unlike `pickle` and `marshal`, JSON is not a framed protocol, so trying to serialize multiple objects with repeated calls to `dump` using the same *fp* will result in an invalid JSON file.

param object obj  
The Python object to be serialized.

param fp  
The file-like object *obj* will be serialized to. The `!json` module always produces `str` objects, not `bytes` objects, therefore `fp.write()` must support `str` input.

type fp  
`file-like object`

param bool skipkeys  
If `True`, keys that are not of a basic type (`str`, `int`, `float`, `bool`, `None`) will be skipped instead of raising a `TypeError`. Default `False`.

param bool ensure_ascii  
If `True` (the default), the output is guaranteed to have all incoming non-ASCII and non-printable characters escaped. If `False`, all characters will be outputted as-is, except for the characters that must be escaped: quotation mark, reverse solidus, and the control characters U+0000 through U+001F.

param bool check_circular  
If `False`, the circular reference check for container types is skipped and a circular reference will result in a `RecursionError` (or worse). Default `True`.

param bool allow_nan  
If `False`, serialization of out-of-range `float` values (`nan`, `inf`, `-inf`) will result in a `ValueError`, in strict compliance with the JSON specification. If `True` (the default), their JavaScript equivalents (`NaN`, `Infinity`, `-Infinity`) are used.

param cls  
If set, a custom JSON encoder with the `~JSONEncoder.default` method overridden, for serializing into custom datatypes. If `None` (the default), `!JSONEncoder` is used.

type cls  
a `JSONEncoder` subclass

param indent  
If a positive integer or string, JSON array elements and object members will be pretty-printed with that indent level. A positive integer indents that many spaces per level; a string (such as `"\t"`) is used to indent each level. If zero, negative, or `""` (the empty string), only newlines are inserted. If `None` (the default), no newlines are inserted.

type indent  
int \| str \| None

param separators  
A two-tuple: `(item_separator, key_separator)`. If `None` (the default), *separators* defaults to `(', ', ': ')` if *indent* is `None`, and `(',', ': ')` otherwise. For the most compact JSON, specify `(',', ':')` to eliminate whitespace.

type separators  
tuple \| None

param default  
A function that is called for objects that can't otherwise be serialized. It should return a JSON encodable version of the object or raise a `TypeError`. If `None` (the default), `!TypeError` is raised.

type default  
`callable` \| None

param bool sort_keys  
If `True`, dictionaries will be outputted sorted by key. Default `False`.

> [!NOTE]
> Keys in key/value pairs of JSON are always of the type `str`. When a dictionary is converted into JSON, all the keys of the dictionary are converted to strings. As a result of this, if a dictionary is converted into JSON and then back into a dictionary, the dictionary may not equal the original one. That is, `loads(dumps(x)) != x` if x has non-string keys. *sort_keys* sorts the keys before they are converted to strings, so numeric keys are sorted by value, not by their string representation.

<div class="versionchanged">

3.2 Allow strings for *indent* in addition to integers.

</div>

<div class="versionchanged">

3.4 Use `(',', ': ')` as default if *indent* is not `None`.

</div>

<div class="versionchanged">

3.6 All optional parameters are now `keyword-only <keyword-only_parameter>`.

</div>

</div>

<div class="function">

dumps(obj, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, default=None, sort_keys=False,*\*kw)

Serialize *obj* to a JSON formatted `str` using this `conversion
table <py-to-json-table>`. The arguments have the same meaning as in `dump`.

</div>

<div class="function">

load(fp, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, array_hook=None,*\*kw)

Deserialize *fp* to a Python object using the `JSON-to-Python conversion table <json-to-py-table>`.

param fp  
A `.read()`-supporting `text file` or `binary file` containing the JSON document to be deserialized.

type fp  
`file-like object`

param cls  
If set, a custom JSON decoder. Additional keyword arguments to `!load` will be passed to the constructor of *cls*. If `None` (the default), `!JSONDecoder` is used.

type cls  
a `JSONDecoder` subclass

param object_hook  
If set, a function that is called with the result of any JSON object literal decoded (a `dict`). The return value of this function will be used instead of the `dict`. This feature can be used to implement custom decoders, for example [JSON-RPC](https://www.jsonrpc.org) class hinting. Default `None`.

type object_hook  
`callable` \| None

param object_pairs_hook  
If set, a function that is called with the result of any JSON object literal decoded with an ordered list of pairs. The return value of this function will be used instead of the `dict`. This feature can be used to implement custom decoders. If *object_hook* is also set, *object_pairs_hook* takes priority. Default `None`.

type object_pairs_hook  
`callable` \| None

param array_hook  
If set, a function that is called with the result of any JSON array literal decoded as a Python list. The return value of this function will be used instead of the `list`. This feature can be used to implement custom decoders. Default `None`.

type array_hook  
`callable` \| None

param parse_float  
If set, a function that is called with the string of every JSON float to be decoded. If `None` (the default), it is equivalent to `float(num_str)`. This can be used to parse JSON floats into custom datatypes, for example `decimal.Decimal`.

type parse_float  
`callable` \| None

param parse_int  
If set, a function that is called with the string of every JSON int to be decoded. If `None` (the default), it is equivalent to `int(num_str)`. This can be used to parse JSON integers into custom datatypes, for example `float`.

type parse_int  
`callable` \| None

param parse_constant  
If set, a function that is called with one of the following strings: `'-Infinity'`, `'Infinity'`, or `'NaN'`. This can be used to raise an exception if invalid JSON numbers are encountered. Default `None`.

type parse_constant  
`callable` \| None

raises JSONDecodeError  
When the data being deserialized is not a valid JSON document.

raises UnicodeDecodeError  
When the data being deserialized does not contain UTF-8, UTF-16 or UTF-32 encoded data.

<div class="versionchanged">

3.1

- Added the optional *object_pairs_hook* parameter.
- *parse_constant* doesn't get called on 'null', 'true', 'false' anymore.

</div>

<div class="versionchanged">

3.6

- All optional parameters are now `keyword-only <keyword-only_parameter>`.
- *fp* can now be a `binary file`. The input encoding should be UTF-8, UTF-16 or UTF-32.

</div>

<div class="versionchanged">

3.11 The default *parse_int* of `int` now limits the maximum length of the integer string via the interpreter's `integer string
conversion length limitation <int_max_str_digits>` to help avoid denial of service attacks.

</div>

<div class="versionchanged">

3.15 Added the optional *array_hook* parameter.

</div>

</div>

<div class="function">

loads(s, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, array_hook=None,*\*kw)

Identical to `load`, but instead of a file-like object, deserialize *s* (a `str`, `bytes` or `bytearray` instance containing a JSON document) to a Python object using this `conversion table <json-to-py-table>`.

<div class="versionchanged">

3.6 *s* can now be of type `bytes` or `bytearray`. The input encoding should be UTF-8, UTF-16 or UTF-32.

</div>

<div class="versionchanged">

3.9 The keyword argument *encoding* has been removed.

</div>

</div>

## Encoders and Decoders

<div class="JSONDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None, array_hook=None)">

Simple JSON decoder.

Performs the following translations in decoding by default:

<div id="json-to-py-table">

| JSON          | Python |
|---------------|--------|
| object        | dict   |
| array         | list   |
| string        | str    |
| number (int)  | int    |
| number (real) | float  |
| true          | True   |
| false         | False  |
| null          | None   |

</div>

It also understands `NaN`, `Infinity`, and `-Infinity` as their corresponding `float` values, which is outside the JSON spec.

*object_hook* is an optional function that will be called with the result of every JSON object decoded and its return value will be used in place of the given `dict`. This can be used to provide custom deserializations (e.g. to support [JSON-RPC](https://www.jsonrpc.org) class hinting).

*object_pairs_hook* is an optional function that will be called with the result of every JSON object decoded with an ordered list of pairs. The return value of *object_pairs_hook* will be used instead of the `dict`. This feature can be used to implement custom decoders. If *object_hook* is also defined, the *object_pairs_hook* takes priority.

<div class="versionchanged">

3.1 Added support for *object_pairs_hook*.

</div>

*array_hook* is an optional function that will be called with the result of every JSON array decoded as a list. The return value of *array_hook* will be used instead of the `list`. This feature can be used to implement custom decoders.

<div class="versionchanged">

3.15 Added support for *array_hook*.

</div>

*parse_float* is an optional function that will be called with the string of every JSON float to be decoded. By default, this is equivalent to `float(num_str)`. This can be used to use another datatype or parser for JSON floats (e.g. `decimal.Decimal`).

*parse_int* is an optional function that will be called with the string of every JSON int to be decoded. By default, this is equivalent to `int(num_str)`. This can be used to use another datatype or parser for JSON integers (e.g. `float`).

*parse_constant* is an optional function that will be called with one of the following strings: `'-Infinity'`, `'Infinity'`, `'NaN'`. This can be used to raise an exception if invalid JSON numbers are encountered.

If *strict* is false (`True` is the default), then control characters will be allowed inside strings. Control characters in this context are those with character codes in the 0--31 range, including `'\t'` (tab), `'\n'`, `'\r'` and `'\0'`.

If the data being deserialized is not a valid JSON document, a `JSONDecodeError` will be raised.

<div class="versionchanged">

3.6 All parameters are now `keyword-only <keyword-only_parameter>`.

</div>

<div class="method">

decode(s)

Return the Python representation of *s* (a `str` instance containing a JSON document).

`JSONDecodeError` will be raised if the given JSON document is not valid.

</div>

<div class="method">

raw_decode(s)

Decode a JSON document from *s* (a `str` beginning with a JSON document) and return a 2-tuple of the Python representation and the index in *s* where the document ended.

This can be used to decode a JSON document from a string that may have extraneous data at the end.

</div>

</div>

<div class="JSONEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)">

Extensible JSON encoder for Python data structures.

Supports the following objects and types by default:

<div id="py-to-json-table">

| Python                                 | JSON   |
|----------------------------------------|--------|
| dict, frozendict                       | object |
| list, tuple                            | array  |
| str                                    | string |
| int, float, int- & float-derived Enums | number |
| True                                   | true   |
| False                                  | false  |
| None                                   | null   |

</div>

<div class="versionchanged">

3.4 Added support for int- and float-derived Enum classes.

</div>

<div class="versionchanged">

3.15 Added support for `frozendict`.

</div>

To extend this to recognize other objects, subclass and implement a `~JSONEncoder.default` method with another method that returns a serializable object for `o` if possible, otherwise it should call the superclass implementation (to raise `TypeError`).

If *skipkeys* is false (the default), a `TypeError` will be raised when trying to encode keys that are not `str`, `int`, `float`, `bool` or `None`. If *skipkeys* is true, such items are simply skipped.

If *ensure_ascii* is true (the default), the output is guaranteed to have all incoming non-ASCII and non-printable characters escaped. If *ensure_ascii* is false, all characters will be output as-is, except for the characters that must be escaped: quotation mark, reverse solidus, and the control characters U+0000 through U+001F.

If *check_circular* is true (the default), then lists, dicts, and custom encoded objects will be checked for circular references during encoding to prevent an infinite recursion (which would cause a `RecursionError`). Otherwise, no such check takes place.

If *allow_nan* is true (the default), then `NaN`, `Infinity`, and `-Infinity` will be encoded as such. This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders. Otherwise, it will be a `ValueError` to encode such floats.

If *sort_keys* is true (default: `False`), then the output of dictionaries will be sorted by key; this is useful for regression tests to ensure that JSON serializations can be compared on a day-to-day basis.

If *indent* is a non-negative integer or string, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0, negative, or `""` will only insert newlines. `None` (the default) selects the most compact representation. Using a positive integer indent indents that many spaces per level. If *indent* is a string (such as `"\t"`), that string is used to indent each level.

<div class="versionchanged">

3.2 Allow strings for *indent* in addition to integers.

</div>

If specified, *separators* should be an `(item_separator, key_separator)` tuple. The default is `(', ', ': ')` if *indent* is `None` and `(',', ': ')` otherwise. To get the most compact JSON representation, you should specify `(',', ':')` to eliminate whitespace.

<div class="versionchanged">

3.4 Use `(',', ': ')` as default if *indent* is not `None`.

</div>

If specified, *default* should be a function that gets called for objects that can't otherwise be serialized. It should return a JSON encodable version of the object or raise a `TypeError`. If not specified, `TypeError` is raised.

<div class="versionchanged">

3.6 All parameters are now `keyword-only <keyword-only_parameter>`.

</div>

<div class="method">

default(o)

Implement this method in a subclass such that it returns a serializable object for *o*, or calls the base implementation (to raise a `TypeError`).

For example, to support arbitrary iterators, you could implement `~JSONEncoder.default` like this:

    def default(self, o):
       try:
           iterable = iter(o)
       except TypeError:
           pass
       else:
           return list(iterable)
       # Let the base class default method raise the TypeError
       return super().default(o)

</div>

<div class="method">

encode(o)

Return a JSON string representation of a Python data structure, *o*. For example:

    >>> json.JSONEncoder().encode({"foo": ["bar", "baz"]})
    '{"foo": ["bar", "baz"]}'

</div>

<div class="method">

iterencode(o)

Encode the given object, *o*, and yield each string representation as available. For example:

    for chunk in json.JSONEncoder().iterencode(bigobject):
        mysocket.write(chunk)

</div>

</div>

## Exceptions

<div class="exception">

JSONDecodeError(msg, doc, pos)

Subclass of `ValueError` with the following additional attributes:

<div class="attribute">

msg

The unformatted error message.

</div>

<div class="attribute">

doc

The JSON document being parsed.

</div>

<div class="attribute">

pos

The start index of *doc* where parsing failed.

</div>

<div class="attribute">

lineno

The line corresponding to *pos*.

</div>

<div class="attribute">

colno

The column corresponding to *pos*.

</div>

<div class="versionadded">

3.5

</div>

</div>

## Standard Compliance and Interoperability

The JSON format is specified by `7159` and by [ECMA-404](https://ecma-international.org/publications-and-standards/standards/ecma-404/). This section details this module's level of compliance with the RFC. For simplicity, `JSONEncoder` and `JSONDecoder` subclasses, and parameters other than those explicitly mentioned, are not considered.

This module does not comply with the RFC in a strict fashion, implementing some extensions that are valid JavaScript but not valid JSON. In particular:

- Infinite and NaN number values are accepted and output;
- Repeated names within an object are accepted, and only the value of the last name-value pair is used.

Since the RFC permits RFC-compliant parsers to accept input texts that are not RFC-compliant, this module's deserializer is technically RFC-compliant under default settings.

### Character Encodings

The RFC requires that JSON be represented using either UTF-8, UTF-16, or UTF-32, with UTF-8 being the recommended default for maximum interoperability.

As permitted, though not required, by the RFC, this module's serializer sets *ensure_ascii=True* by default, thus escaping the output so that the resulting strings only contain printable ASCII characters.

Other than the *ensure_ascii* parameter, this module is defined strictly in terms of conversion between Python objects and `Unicode strings <str>`, and thus does not otherwise directly address the issue of character encodings.

The RFC prohibits adding a byte order mark (BOM) to the start of a JSON text, and this module's serializer does not add a BOM to its output. The RFC permits, but does not require, JSON deserializers to ignore an initial BOM in their input. This module's deserializer raises a `ValueError` when an initial BOM is present.

The RFC does not explicitly forbid JSON strings which contain byte sequences that don't correspond to valid Unicode characters (e.g. unpaired UTF-16 surrogates), but it does note that they may cause interoperability problems. By default, this module accepts and outputs (when present in the original `str`) code points for such sequences.

### Infinite and NaN Number Values

The RFC does not permit the representation of infinite or NaN number values. Despite that, by default, this module accepts and outputs `Infinity`, `-Infinity`, and `NaN` as if they were valid JSON number literal values:

    >>> # Neither of these calls raises an exception, but the results are not valid JSON
    >>> json.dumps(float('-inf'))
    '-Infinity'
    >>> json.dumps(float('nan'))
    'NaN'
    >>> # Same when deserializing
    >>> json.loads('-Infinity')
    -inf
    >>> json.loads('NaN')
    nan

In the serializer, the *allow_nan* parameter can be used to alter this behavior. In the deserializer, the *parse_constant* parameter can be used to alter this behavior.

### Repeated Names Within an Object

The RFC specifies that the names within a JSON object should be unique, but does not mandate how repeated names in JSON objects should be handled. By default, this module does not raise an exception; instead, it ignores all but the last name-value pair for a given name:

    >>> weird_json = '{"x": 1, "x": 2, "x": 3}'
    >>> json.loads(weird_json)
    {'x': 3}

The *object_pairs_hook* parameter can be used to alter this behavior.

### Top-level Non-Object, Non-Array Values

The old version of JSON specified by the obsolete `4627` required that the top-level value of a JSON text must be either a JSON object or array (Python `dict` or `list`), and could not be a JSON null, boolean, number, or string value. `7159` removed that restriction, and this module does not and has never implemented that restriction in either its serializer or its deserializer.

Regardless, for maximum interoperability, you may wish to voluntarily adhere to the restriction yourself.

### Implementation Limitations

Some JSON deserializer implementations may set limits on:

- the size of accepted JSON texts
- the maximum level of nesting of JSON objects and arrays
- the range and precision of JSON numbers
- the content and maximum length of JSON strings

This module does not impose any such limits beyond those of the relevant Python datatypes themselves or the Python interpreter itself.

When serializing to JSON, beware any such limitations in applications that may consume your JSON. In particular, it is common for JSON numbers to be deserialized into IEEE 754 double precision numbers and thus subject to that representation's range and precision limitations. This is especially relevant when serializing Python `int` values of extremely large magnitude, or when serializing instances of "exotic" numerical types such as `decimal.Decimal`.

## Command-line interface

<div class="module" data-synopsis="A command-line interface to validate and pretty-print JSON.">

json.tool

</div>

**Source code:** `Lib/json/tool.py`

------------------------------------------------------------------------

The `!json` module can be invoked as a script via `python -m json` to validate and pretty-print JSON objects. The `!json.tool` submodule implements this interface.

If the optional `infile` and `outfile` arguments are not specified, `sys.stdin` and `sys.stdout` will be used respectively:

```
$ echo '{"json": "obj"}' | python -m json
{
    "json": "obj"
}
$ echo '{1.2:3.4}' | python -m json
Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
```

<div class="versionchanged">

3.5 The output is now in the same order as the input. Use the `--sort-keys` option to sort the output of dictionaries alphabetically by key.

</div>

<div class="versionchanged">

3.14 The `!json` module may now be directly executed as `python -m json`. For backwards compatibility, invoking the CLI as `python -m json.tool` remains supported.

</div>

### Command-line options

<div class="option">

infile

The JSON file to be validated or pretty-printed:

```
$ python -m json mp_films.json
[
    {
        "title": "And Now for Something Completely Different",
        "year": 1971
    },
    {
        "title": "Monty Python and the Holy Grail",
        "year": 1975
    }
]
```

If *infile* is not specified, read from `sys.stdin`.

</div>

<div class="option">

outfile

Write the output of the *infile* to the given *outfile*. Otherwise, write it to `sys.stdout`.

</div>

<div class="option">

`--sort-keys`  

Sort the output of dictionaries alphabetically by key.

<div class="versionadded">

3.5

</div>

</div>

<div class="option">

`--no-ensure-ascii`  

Disable escaping of non-ascii characters, see `json.dumps` for more information.

<div class="versionadded">

3.9

</div>

</div>

<div class="option">

`--json-lines`  

Parse every input line as separate JSON object.

<div class="versionadded">

3.8

</div>

</div>

<div class="option">

`--indent, --tab, --no-indent, --compact`  

Mutually exclusive options for whitespace control.

<div class="versionadded">

3.9

</div>

</div>

<div class="option">

`-h, --help`  

Show the help message.

</div>

**Footnotes**

[^1]: As noted in [the errata for RFC 7159](https://www.rfc-editor.org/errata_search.php?rfc=7159), JSON permits literal U+2028 (LINE SEPARATOR) and U+2029 (PARAGRAPH SEPARATOR) characters in strings, whereas JavaScript (as of ECMAScript Edition 5.1) does not.

# `!keyword` --- Testing for Python keywords

<div class="module" data-synopsis="Test whether a string is a keyword in Python.">

keyword

</div>

**Source code:** `Lib/keyword.py`

------------------------------------------------------------------------

This module allows a Python program to determine if a string is a `keyword <keywords>` or `soft keyword <soft-keywords>`.

<div class="function">

iskeyword(s)

Return `True` if *s* is a Python `keyword <keywords>`.

</div>

<div class="data">

kwlist

Sequence containing all the `keywords <keywords>` defined for the interpreter. If any keywords are defined to only be active when particular `__future__` statements are in effect, these will be included as well.

</div>

<div class="function">

issoftkeyword(s)

Return `True` if *s* is a Python `soft keyword <soft-keywords>`.

<div class="versionadded">

3.9

</div>

</div>

<div class="data">

softkwlist

Sequence containing all the `soft keywords <soft-keywords>` defined for the interpreter. If any soft keywords are defined to only be active when particular `__future__` statements are in effect, these will be included as well.

<div class="versionadded">

3.9

</div>

</div>

# Python Language Services

Python provides a number of modules to assist in working with the Python language. These modules support tokenizing, parsing, syntax analysis, bytecode disassembly, and various other facilities.

These modules include:

<div class="toctree">

ast.rst symtable.rst token.rst keyword.rst tokenize.rst tabnanny.rst pyclbr.rst py_compile.rst compileall.rst dis.rst pickletools.rst

</div>

# `!linecache` --- Random access to text lines

<div class="module" data-synopsis="Provides random access to individual lines from text files.">

linecache

</div>

**Source code:** `Lib/linecache.py`

------------------------------------------------------------------------

The `!linecache` module allows one to get any line from a Python source file, while attempting to optimize internally, using a cache, the common case where many lines are read from a single file. This is used by the `traceback` module to retrieve source lines for inclusion in the formatted traceback.

The `tokenize.open` function is used to open files. This function uses `tokenize.detect_encoding` to get the encoding of the file; in the absence of an encoding token, the file encoding defaults to UTF-8.

The `!linecache` module defines the following functions:

<div class="function">

getline(filename, lineno, module_globals=None)

Get line *lineno* from file named *filename*. This function will never raise an exception --- it will return `''` on errors (the terminating newline character will be included for lines that are found).

<div class="index">

triple: module; search; path

</div>

If *filename* indicates a frozen module (starting with `'<frozen '`), the function will attempt to get the real file name from `module_globals['__file__']` if *module_globals* is not `None`.

If a file named *filename* is not found, the function first checks for a `302` `__loader__` in *module_globals*. If there is such a loader and it defines a `get_source` method, then that determines the source lines (if `get_source()` returns `None`, then `''` is returned). Finally, if *filename* is a relative filename, it is looked up relative to the entries in the module search path, `sys.path`.

<div class="versionchanged">

3.14

Support *filename* of frozen modules.

</div>

</div>

<div class="function">

clearcache()

Clear the cache. Use this function if you no longer need lines from files previously read using `getline`.

</div>

<div class="function">

checkcache(filename=None)

Check the cache for validity. Use this function if files in the cache may have changed on disk, and you require the updated version. If *filename* is omitted, it will check all the entries in the cache.

</div>

<div class="function">

lazycache(filename, module_globals)

Capture enough detail about a non-file-based module to permit getting its lines later via `getline` even if *module_globals* is `None` in the later call. This avoids doing I/O until a line is actually needed, without having to carry the module globals around indefinitely.

<div class="versionadded">

3.5

</div>

</div>

Example:

    >>> import linecache
    >>> linecache.getline(linecache.__file__, 8)
    'import sys\n'

# `!locale` --- Internationalization services

<div class="module" data-synopsis="Internationalization services.">

locale

</div>

**Source code:** `Lib/locale.py`

------------------------------------------------------------------------

The `!locale` module opens access to the POSIX locale database and functionality. The POSIX locale mechanism allows programmers to deal with certain cultural issues in an application, without requiring the programmer to know all the specifics of each country where the software is executed.

<div class="index">

pair: module; <span id="locale">locale</span>

</div>

The `!locale` module is implemented on top of the `!_locale` module, which in turn uses an ANSI C locale implementation if available.

The `!locale` module defines the following exception and functions:

<div class="exception">

Error

Exception raised when the locale passed to `setlocale` is not recognized.

</div>

<div class="function">

setlocale(category, locale=None)

If *locale* is given and not `None`, `setlocale` modifies the locale setting for the *category*. The available categories are listed in the data description below. *locale* may be a `string <locale_name>`, or a pair, language code and encoding. An empty string specifies the user's default settings. If the modification of the locale fails, the exception `Error` is raised. If successful, the new locale setting is returned.

If *locale* is a pair, it is converted to a locale name using the locale aliasing engine. The language code has the same format as a `locale name <locale_name>`, but without encoding. The language code and encoding can be `None`.

If *locale* is omitted or `None`, the current setting for *category* is returned.

Example:

    >>> import locale
    >>> loc = locale.setlocale(locale.LC_ALL)  # get current locale
    # use German locale; name and availability varies with platform
    >>> locale.setlocale(locale.LC_ALL, 'de_DE.UTF-8')
    >>> locale.strcoll('f\xe4n', 'foo')  # compare a string containing an umlaut
    >>> locale.setlocale(locale.LC_ALL, '')   # use user's preferred locale
    >>> locale.setlocale(locale.LC_ALL, 'C')  # use default (C) locale
    >>> locale.setlocale(locale.LC_ALL, loc)  # restore saved locale

`setlocale` is not thread-safe on most systems. Applications typically start with a call of:

    import locale
    locale.setlocale(locale.LC_ALL, '')

This sets the locale for all categories to the user's default setting (typically specified in the `LANG` environment variable). If the locale is not changed thereafter, using multithreading should not cause problems.

<div class="versionchanged">

3.15 Support language codes with `@`-modifiers.

</div>

</div>

<div class="function">

localeconv()

Returns the database of the local conventions as a dictionary. This dictionary has the following strings as keys:

<div class="tabularcolumns">

\|l\|l\|L\|

</div>

| Category | Key | Meaning |
|----|----|----|
| `LC_NUMERIC` | `'decimal_point'` | Decimal point character. |
|  | `'grouping'` | Sequence of numbers specifying which relative positions the `'thousands_sep'` is expected. If the sequence is terminated with `CHAR_MAX`, no further grouping is performed. If the sequence terminates with a `0`, the last group size is repeatedly used. |
|  | `'thousands_sep'` | Character used between groups. |
| `LC_MONETARY` | `'int_curr_symbol'` | International currency symbol. |
|  | `'currency_symbol'` | Local currency symbol. |
|  | `'p_cs_precedes/n_cs_precedes'` | Whether the currency symbol precedes the value (for positive resp. negative values). |
|  | `'p_sep_by_space/n_sep_by_space'` | Whether the currency symbol is separated from the value by a space (for positive resp. negative values). |
|  | `'mon_decimal_point'` | Decimal point used for monetary values. |
|  | `'frac_digits'` | Number of fractional digits used in local formatting of monetary values. |
|  | `'int_frac_digits'` | Number of fractional digits used in international formatting of monetary values. |
|  | `'mon_thousands_sep'` | Group separator used for monetary values. |
|  | `'mon_grouping'` | Equivalent to `'grouping'`, used for monetary values. |
|  | `'positive_sign'` | Symbol used to annotate a positive monetary value. |
|  | `'negative_sign'` | Symbol used to annotate a negative monetary value. |
|  | `'p_sign_posn/n_sign_posn'` | The position of the sign (for positive resp. negative values), see below. |

All numeric values can be set to `CHAR_MAX` to indicate that there is no value specified in this locale.

The possible values for `'p_sign_posn'` and `'n_sign_posn'` are given below.

| Value      | Explanation                                            |
|------------|--------------------------------------------------------|
| `0`        | Currency and value are surrounded by parentheses.      |
| `1`        | The sign should precede the value and currency symbol. |
| `2`        | The sign should follow the value and currency symbol.  |
| `3`        | The sign should immediately precede the value.         |
| `4`        | The sign should immediately follow the value.          |
| `CHAR_MAX` | Nothing is specified in this locale.                   |

The function temporarily sets the `LC_CTYPE` locale to the `LC_NUMERIC` locale or the `LC_MONETARY` locale if locales are different and numeric or monetary strings are non-ASCII. This temporary change affects other threads.

<div class="versionchanged">

3.7 The function now temporarily sets the `LC_CTYPE` locale to the `LC_NUMERIC` locale in some cases.

</div>

</div>

<div class="function">

nl_langinfo(option)

Return some locale-specific information as a string. This function is not available on all systems, and the set of possible options might also vary across platforms. The possible argument values are numbers, for which symbolic constants are available in the locale module.

The `nl_langinfo` function accepts one of the following keys. Most descriptions are taken from the corresponding description in the GNU C library.

<div class="data">

CODESET

Get a string with the name of the character encoding used in the selected locale.

</div>

<div class="data">

D_T_FMT

Get a string that can be used as a format string for `time.strftime` to represent date and time in a locale-specific way.

</div>

<div class="data">

D_FMT

Get a string that can be used as a format string for `time.strftime` to represent a date in a locale-specific way.

</div>

<div class="data">

T_FMT

Get a string that can be used as a format string for `time.strftime` to represent a time in a locale-specific way.

</div>

<div class="data">

T_FMT_AMPM

Get a format string for `time.strftime` to represent time in the am/pm format.

</div>

<div class="data">

DAY_1 DAY_2 DAY_3 DAY_4 DAY_5 DAY_6 DAY_7

Get the name of the n-th day of the week.

> [!NOTE]
> This follows the US convention of `DAY_1` being Sunday, not the international convention (ISO 8601) that Monday is the first day of the week.

</div>

<div class="data">

ABDAY_1 ABDAY_2 ABDAY_3 ABDAY_4 ABDAY_5 ABDAY_6 ABDAY_7

Get the abbreviated name of the n-th day of the week.

</div>

<div class="data">

MON_1 MON_2 MON_3 MON_4 MON_5 MON_6 MON_7 MON_8 MON_9 MON_10 MON_11 MON_12

Get the name of the n-th month.

</div>

<div class="data">

ABMON_1 ABMON_2 ABMON_3 ABMON_4 ABMON_5 ABMON_6 ABMON_7 ABMON_8 ABMON_9 ABMON_10 ABMON_11 ABMON_12

Get the abbreviated name of the n-th month.

</div>

<div class="data">

RADIXCHAR

Get the radix character (decimal dot, decimal comma, etc.).

</div>

<div class="data">

THOUSEP

Get the separator character for thousands (groups of three digits).

</div>

<div class="data">

YESEXPR

Get a regular expression that can be used with the regex function to recognize a positive response to a yes/no question.

</div>

<div class="data">

NOEXPR

Get a regular expression that can be used with the `regex(3)` function to recognize a negative response to a yes/no question.

> [!NOTE]
> The regular expressions for `YESEXPR` and `NOEXPR` use syntax suitable for the `regex` function from the C library, which might differ from the syntax used in `re`.

</div>

<div class="data">

CRNCYSTR

Get the currency symbol, preceded by "-" if the symbol should appear before the value, "+" if the symbol should appear after the value, or "." if the symbol should replace the radix character.

</div>

<div class="data">

ERA

Get a string which describes how years are counted and displayed for each era in a locale.

Most locales do not define this value. An example of a locale which does define this value is the Japanese one. In Japan, the traditional representation of dates includes the name of the era corresponding to the then-emperor's reign.

Normally it should not be necessary to use this value directly. Specifying the `E` modifier in their format strings causes the `time.strftime` function to use this information. The format of the returned string is specified in *The Open Group Base Specifications Issue 8*, paragraph [7.3.5.2 LC_TIME C-Language Access](https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap07.html#tag_07_03_05_02).

</div>

<div class="data">

ERA_D_T_FMT

Get a format string for `time.strftime` to represent date and time in a locale-specific era-based way.

</div>

<div class="data">

ERA_D_FMT

Get a format string for `time.strftime` to represent a date in a locale-specific era-based way.

</div>

<div class="data">

ERA_T_FMT

Get a format string for `time.strftime` to represent a time in a locale-specific era-based way.

</div>

<div class="data">

ALT_DIGITS

Get a string consisting of up to 100 semicolon-separated symbols used to represent the values 0 to 99 in a locale-specific way. In most locales this is an empty string.

</div>

The function temporarily sets the `LC_CTYPE` locale to the locale of the category that determines the requested value (`LC_TIME`, `LC_NUMERIC`, `LC_MONETARY` or `LC_MESSAGES`) if locales are different and the resulting string is non-ASCII. This temporary change affects other threads.

<div class="versionchanged">

3.14 The function now temporarily sets the `LC_CTYPE` locale in some cases.

</div>

<div class="versionchanged">

3.16 On glibc, the `LC_TIME` items (except `ERA`) are now decoded independently of the `LC_CTYPE` encoding.

</div>

</div>

<div class="function">

getdefaultlocale(\[envvars\])

Tries to determine the default locale settings and returns them as a tuple of the form `(language code, encoding)`.

According to POSIX, a program which has not called `setlocale(LC_ALL, '')` runs using the portable `'C'` locale. Calling `setlocale(LC_ALL, '')` lets it use the default locale as defined by the `LANG` variable. Since we do not want to interfere with the current locale setting we thus emulate the behavior in the way described above.

To maintain compatibility with other platforms, not only the `LANG` variable is tested, but a list of variables given as envvars parameter. The first found to be defined will be used. *envvars* defaults to the search path used in GNU gettext; it must always contain the variable name `'LANG'`. The GNU gettext search path contains `'LC_ALL'`, `'LC_CTYPE'`, `'LANG'` and `'LANGUAGE'`, in that order.

The language code has the same format as a `locale name <locale_name>`, but without encoding and `@`-modifier. The language code and encoding may be `None` if their values cannot be determined. The "C" locale is represented as `(None, None)`.

</div>

<div class="function">

getlocale(category=LC_CTYPE)

Returns the current setting for the given locale category as a tuple containing the language code and encoding. *category* may be one of the `!LC_\*` values except `LC_ALL`. It defaults to `LC_CTYPE`.

The language code has the same format as a `locale name <locale_name>`, but without encoding. The language code and encoding may be `None` if their values cannot be determined. The "C" locale is represented as `(None, None)`.

<div class="versionchanged">

3.15 `@`-modifier are no longer silently removed, but included in the language code.

</div>

</div>

<div class="function">

getpreferredencoding(do_setlocale=True)

Return the `locale encoding` used for text data, according to user preferences. User preferences are expressed differently on different systems, and might not be available programmatically on some systems, so this function only returns a guess.

On some systems, it is necessary to invoke `setlocale` to obtain the user preferences, so this function is not thread-safe. If invoking setlocale is not necessary or desired, *do_setlocale* should be set to `False`.

On Android or if the `Python UTF-8 Mode <utf8-mode>` is enabled, always return `'utf-8'`, the `locale encoding` and the *do_setlocale* argument are ignored.

The `Python preinitialization <c-preinit>` configures the LC_CTYPE locale. See also the `filesystem encoding and error handler`.

<div class="versionchanged">

3.7 The function now always returns `"utf-8"` on Android or if the `Python UTF-8 Mode <utf8-mode>` is enabled.

</div>

</div>

<div class="function">

getencoding()

Get the current `locale encoding`:

- On Android and VxWorks, return `"utf-8"`.
- On Unix, return the encoding of the current `LC_CTYPE` locale. Return `"utf-8"` if `nl_langinfo(CODESET)` returns an empty string: for example, if the current LC_CTYPE locale is not supported.
- On Windows, return the ANSI code page.

The `Python preinitialization <c-preinit>` configures the LC_CTYPE locale. See also the `filesystem encoding and error handler`.

This function is similar to `getpreferredencoding(False) <getpreferredencoding>` except this function ignores the `Python UTF-8 Mode <utf8-mode>`.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

normalize(localename)

Returns a normalized locale code for the given locale name. The returned locale code is formatted for use with `setlocale`. If normalization fails, the original name is returned unchanged.

If the given encoding is not known, the function defaults to the default encoding for the locale code just like `setlocale`.

</div>

<div class="function">

strcoll(string1, string2)

Compares two strings according to the current `LC_COLLATE` setting. As any other compare function, returns a negative, or a positive value, or `0`, depending on whether *string1* collates before or after *string2* or is equal to it.

</div>

<div class="function">

strxfrm(string)

Transforms a string to one that can be used in locale-aware comparisons. For example, `strxfrm(s1) < strxfrm(s2)` is equivalent to `strcoll(s1, s2) < 0`. This function can be used when the same string is compared repeatedly, e.g. when collating a sequence of strings.

</div>

<div class="function">

format_string(format, val, grouping=False, monetary=False)

Formats a number *val* according to the current `LC_NUMERIC` setting. The format follows the conventions of the `%` operator. For floating-point values, the decimal point is modified if appropriate. If *grouping* is `True`, also takes the grouping into account.

If *monetary* is true, the conversion uses monetary thousands separator and grouping strings.

Processes formatting specifiers as in `format % val`, but takes the current locale settings into account.

<div class="versionchanged">

3.7 The *monetary* keyword parameter was added.

</div>

</div>

<div class="function">

currency(val, symbol=True, grouping=False, international=False)

Formats a number *val* according to the current `LC_MONETARY` settings.

The returned string includes the currency symbol if *symbol* is true, which is the default. If *grouping* is `True` (which is not the default), grouping is done with the value. If *international* is `True` (which is not the default), the international currency symbol is used.

> [!NOTE]
> This function will not work with the 'C' locale, so you have to set a locale via `setlocale` first.

</div>

<div class="function">

str(float)

Formats a floating-point number using the same format as the built-in function `str(float)`, but takes the decimal point into account.

</div>

<div class="function">

delocalize(string)

Converts a string into a normalized number string, following the `LC_NUMERIC` settings.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

localize(string, grouping=False, monetary=False)

Converts a normalized number string into a formatted string following the `LC_NUMERIC` settings.

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

atof(string, func=float)

Converts a string to a number, following the `LC_NUMERIC` settings, by calling *func* on the result of calling `delocalize` on *string*.

</div>

<div class="function">

atoi(string)

Converts a string to an integer, following the `LC_NUMERIC` conventions.

</div>

<div class="data">

LC_CTYPE

Locale category for the character type functions. Most importantly, this category defines the text encoding, i.e. how bytes are interpreted as Unicode codepoints. See `538` and `540` for how this variable might be automatically coerced to `C.UTF-8` to avoid issues created by invalid settings in containers or incompatible settings passed over remote SSH connections.

Python doesn't internally use locale-dependent character transformation functions from `ctype.h`. Instead, `pyctype.h` provides locale-independent equivalents like `Py_TOLOWER`.

</div>

<div class="data">

LC_COLLATE

Locale category for sorting strings. The functions `strcoll` and `strxfrm` of the `!locale` module are affected.

</div>

<div class="data">

LC_TIME

Locale category for the formatting of time. The function `time.strftime` follows these conventions.

</div>

<div class="data">

LC_MONETARY

Locale category for formatting of monetary values. The available options are available from the `localeconv` function.

</div>

<div class="data">

LC_MESSAGES

Locale category for message display. Python currently does not support application specific locale-aware messages. Messages displayed by the operating system, like those returned by `os.strerror` might be affected by this category.

This value may not be available on operating systems not conforming to the POSIX standard, most notably Windows.

</div>

<div class="data">

LC_NUMERIC

Locale category for formatting numbers. The functions `format_string`, `atoi`, `atof` and `.str` of the `!locale` module are affected by that category. All other numeric formatting operations are not affected.

</div>

<div class="data">

LC_ALL

Combination of all locale settings. If this flag is used when the locale is changed, setting the locale for all categories is attempted. If that fails for any category, no category is changed at all. When the locale is retrieved using this flag, a string indicating the setting for all categories is returned. This string can be later used to restore the settings.

</div>

<div class="data">

CHAR_MAX

This is a symbolic constant used for different values returned by `localeconv`.

</div>

## Background, details, hints, tips and caveats

The C standard defines the locale as a program-wide property that may be relatively expensive to change. On top of that, some implementations are broken in such a way that frequent locale changes may cause core dumps. This makes the locale somewhat painful to use correctly.

Initially, when a program is started, the locale is the `C` locale, no matter what the user's preferred locale is. There is one exception: the `LC_CTYPE` category is changed at startup to set the current locale encoding to the user's preferred locale encoding. The program must explicitly say that it wants the user's preferred locale settings for other categories by calling `setlocale(LC_ALL, '')`.

It is generally a bad idea to call `setlocale` in some library routine, since as a side effect it affects the entire program. Saving and restoring it is almost as bad: it is expensive and affects other threads that happen to run before the settings have been restored.

If, when coding a module for general use, you need a locale independent version of an operation that is affected by the locale (such as certain formats used with `time.strftime`), you will have to find a way to do it without using the standard library routine. Even better is convincing yourself that using locale settings is okay. Only as a last resort should you document that your module is not compatible with non-`C` locale settings.

The only way to perform numeric operations according to the locale is to use the special functions defined by this module: `atof`, `atoi`, `format_string`, `.str`.

There is no way to perform case conversions and character classifications according to the locale. For (Unicode) text strings these are done according to the character value only, while for byte strings, the conversions and classifications are done according to the ASCII value of the byte, and bytes whose high bit is set (i.e., non-ASCII bytes) are never converted or considered part of a character class such as letter or whitespace.

## Locale names

The format of the locale name is platform dependent, and the set of supported locales can depend on the system configuration.

On Posix platforms, it usually has the format[^1]:

<div class="productionlist">

locale_name : language \["\_" territory\] \["." charset\] \["@" modifier\]

</div>

where *language* is a two- or three-letter language code from [ISO 639](https://www.iso.org/iso-639-language-code), *territory* is a two-letter country or region code from [ISO 3166](https://www.iso.org/iso-3166-country-codes.html), *charset* is a locale encoding, and *modifier* is a script name, a language subtag, a sort order identifier, or other locale modifier (for example, "latin", "valencia", "stroke" and "euro").

On Windows, several formats are supported.[^2][^3] A subset of [IETF BCP 47](https://www.rfc-editor.org/info/bcp47) tags:

<div class="productionlist">

locale_name : language \["-" script\] \["-" territory\] \["." charset\] : language \["-" script\] "-" territory "-" modifier

</div>

where *language* and *territory* have the same meaning as in Posix, *script* is a four-letter script code from [ISO 15924](https://www.unicode.org/iso15924/), and *modifier* is a language subtag, a sort order identifier or custom modifier (for example, "valencia", "stroke" or "x-python"). Both hyphen (`'-'`) and underscore (`'_'`) separators are supported. Only UTF-8 encoding is allowed for BCP 47 tags.

Windows also supports locale names in the format:

<div class="productionlist">

locale_name : language \["\_" territory\] \["." charset\]

</div>

where *language* and *territory* are full names, such as "English" and "United States", and *charset* is either a code page number (for example, "1252") or UTF-8. Only the underscore separator is supported in this format.

The "C" locale is supported on all platforms.

## For extension writers and programs that embed Python

Extension modules should never call `setlocale`, except to find out what the current locale is. But since the return value can only be used portably to restore it, that is not very useful (except perhaps to find out whether or not the locale is `C`).

When Python code uses the `!locale` module to change the locale, this also affects the embedding application. If the embedding application doesn't want this to happen, it should remove the `!_locale` extension module (which does all the work) from the table of built-in modules in the `config.c` file, and make sure that the `!_locale` module is not accessible as a shared library.

## Access to message catalogs

<div class="function">

gettext(msg)

</div>

<div class="function">

dgettext(domain, msg)

</div>

<div class="function">

dcgettext(domain, msg, category)

</div>

<div class="function">

textdomain(domain)

</div>

<div class="function">

bindtextdomain(domain, dir)

</div>

<div class="function">

bind_textdomain_codeset(domain, codeset)

</div>

The locale module exposes the C library's gettext interface on systems that provide this interface. It consists of the functions `~locale.gettext`, `dgettext`, `dcgettext`, `textdomain`, `bindtextdomain`, and `bind_textdomain_codeset`. These are similar to the same functions in the `gettext` module, but use the C library's binary format for message catalogs, and the C library's search algorithms for locating message catalogs.

Python applications should normally find no need to invoke these functions, and should use `gettext` instead. A known exception to this rule are applications that link with additional C libraries which internally invoke C functions `gettext` or `dcgettext`. For these applications, it may be necessary to bind the text domain, so that the libraries can properly locate their message catalogs.

[^1]: [IEEE Std 1003.1-2024; 8.2 Internationalization Variables](https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap08.html#tag_08_02)

[^2]: [UCRT Locale names, Languages, and Country/Region strings](https://learn.microsoft.com/en-us/cpp/c-runtime-library/locale-names-languages-and-country-region-strings)

[^3]: [Locale Names](https://learn.microsoft.com/en-us/windows/win32/intl/locale-names)

# `!logging.config` --- Logging configuration

<div class="module" data-synopsis="Configuration of the logging module.">

logging.config

</div>

**Source code:** `Lib/logging/config.py`

<div class="sidebar">

**Important**

This page contains only reference information. For tutorials, please see

- `Basic Tutorial <logging-basic-tutorial>`
- `Advanced Tutorial <logging-advanced-tutorial>`
- `Logging Cookbook <logging-cookbook>`

</div>

------------------------------------------------------------------------

This section describes the API for configuring the logging module.

## Configuration functions

The following functions configure the logging module. They are located in the `!logging.config` module. Their use is optional --- you can configure the logging module using these functions or by making calls to the main API (defined in `logging` itself) and defining handlers which are declared either in `logging` or `logging.handlers`.

<div class="function">

dictConfig(config)

Takes the logging configuration from a dictionary. The contents of this dictionary are described in `logging-config-dictschema` below.

If an error is encountered during configuration, this function will raise a `ValueError`, `TypeError`, `AttributeError` or `ImportError` with a suitably descriptive message. The following is a (possibly incomplete) list of conditions which will raise an error:

- A `level` which is not a string or which is a string not corresponding to an actual logging level.
- A `propagate` value which is not a boolean.
- An id which does not have a corresponding destination.
- A non-existent handler id found during an incremental call.
- An invalid logger name.
- Inability to resolve to an internal or external object.

Parsing is performed by the `DictConfigurator` class, whose constructor is passed the dictionary used for configuration, and has a `configure` method. The `!logging.config` module has a callable attribute `dictConfigClass` which is initially set to `DictConfigurator`. You can replace the value of `dictConfigClass` with a suitable implementation of your own.

`dictConfig` calls `dictConfigClass` passing the specified dictionary, and then calls the `configure` method on the returned object to put the configuration into effect:

    def dictConfig(config):
        dictConfigClass(config).configure()

For example, a subclass of `DictConfigurator` could call `DictConfigurator.__init__()` in its own `__init__`, then set up custom prefixes which would be usable in the subsequent `configure` call. `dictConfigClass` would be bound to this new subclass, and then `dictConfig` could be called exactly as in the default, uncustomized state.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

fileConfig(fname, defaults=None, disable_existing_loggers=True, encoding=None)

Reads the logging configuration from a `configparser`-format file. The format of the file should be as described in `logging-config-fileformat`. This function can be called several times from an application, allowing an end user to select from various pre-canned configurations (if the developer provides a mechanism to present the choices and load the chosen configuration).

It will raise `FileNotFoundError` if the file doesn't exist and `RuntimeError` if the file is invalid or empty.

param fname  
A filename, or a file-like object, or an instance derived from `~configparser.RawConfigParser`. If a `!RawConfigParser`-derived instance is passed, it is used as is. Otherwise, a `~configparser.ConfigParser` is instantiated, and the configuration read by it from the object passed in `fname`. If that has a `readline` method, it is assumed to be a file-like object and read using `~configparser.ConfigParser.read_file`; otherwise, it is assumed to be a filename and passed to `~configparser.ConfigParser.read`.

param defaults  
Defaults to be passed to the `!ConfigParser` can be specified in this argument.

param disable_existing_loggers  
If specified as `False`, loggers which exist when this call is made are left enabled. The default is `True` because this enables old behaviour in a backward-compatible way. This behaviour is to disable any existing non-root loggers unless they or their ancestors are explicitly named in the logging configuration.

param encoding  
The encoding used to open file when *fname* is filename.

<div class="versionchanged">

3.4 An instance of a subclass of `~configparser.RawConfigParser` is now accepted as a value for `fname`. This facilitates:

- Use of a configuration file where logging configuration is just part of the overall application configuration.
- Use of a configuration read from a file, and then modified by the using application (e.g. based on command-line parameters or other aspects of the runtime environment) before being passed to `fileConfig`.

</div>

<div class="versionchanged">

3.10 Added the *encoding* parameter.

</div>

<div class="versionchanged">

3.12 An exception will be thrown if the provided file doesn't exist or is invalid or empty.

</div>

</div>

<div class="function">

listen(port=DEFAULT_LOGGING_CONFIG_PORT, verify=None)

Starts up a socket server on the specified port, and listens for new configurations. If no port is specified, the module's default `DEFAULT_LOGGING_CONFIG_PORT` is used. Logging configurations will be sent as a file suitable for processing by `dictConfig` or `fileConfig`. Returns a `~threading.Thread` instance on which you can call `~threading.Thread.start` to start the server, and which you can `~threading.Thread.join` when appropriate. To stop the server, call `stopListening`.

The `verify` argument, if specified, should be a callable which should verify whether bytes received across the socket are valid and should be processed. This could be done by encrypting and/or signing what is sent across the socket, such that the `verify` callable can perform signature verification and/or decryption. The `verify` callable is called with a single argument - the bytes received across the socket - and should return the bytes to be processed, or `None` to indicate that the bytes should be discarded. The returned bytes could be the same as the passed in bytes (e.g. when only verification is done), or they could be completely different (perhaps if decryption were performed).

To send a configuration to the socket, read in the configuration file and send it to the socket as a sequence of bytes preceded by a four-byte length string packed in binary using `struct.pack('>L', n)`.

<div id="logging-eval-security">

> [!NOTE]
> Because portions of the configuration are passed through `eval`, use of this function may open its users to a security risk. While the function only binds to a socket on `localhost`, and so does not accept connections from remote machines, there are scenarios where untrusted code could be run under the account of the process which calls `listen`. Specifically, if the process calling `listen` runs on a multi-user machine where users cannot trust each other, then a malicious user could arrange to run essentially arbitrary code in a victim user's process, simply by connecting to the victim's `listen` socket and sending a configuration which runs whatever code the attacker wants to have executed in the victim's process. This is especially easy to do if the default port is used, but not hard even if a different port is used. To avoid the risk of this happening, use the `verify` argument to `listen` to prevent unrecognised configurations from being applied.

</div>

<div class="versionchanged">

3.4 The `verify` argument was added.

</div>

> [!NOTE]
> If you want to send configurations to the listener which don't disable existing loggers, you will need to use a JSON format for the configuration, which will use `dictConfig` for configuration. This method allows you to specify `disable_existing_loggers` as `False` in the configuration you send.

</div>

<div class="function">

stopListening()

Stops the listening server which was created with a call to `listen`. This is typically called before calling `join` on the return value from `listen`.

</div>

## Security considerations

The logging configuration functionality tries to offer convenience, and in part this is done by offering the ability to convert text in configuration files into Python objects used in logging configuration - for example, as described in `logging-config-dict-userdef`. However, these same mechanisms (importing callables from user-defined modules and calling them with parameters from the configuration) could be used to invoke any code you like, and for this reason you should treat configuration files from untrusted sources with *extreme caution* and satisfy yourself that nothing bad can happen if you load them, before actually loading them.

## Configuration dictionary schema

Describing a logging configuration requires listing the various objects to create and the connections between them; for example, you may create a handler named 'console' and then say that the logger named 'startup' will send its messages to the 'console' handler. These objects aren't limited to those provided by the `logging` module because you might write your own formatter or handler class. The parameters to these classes may also need to include external objects such as `sys.stderr`. The syntax for describing these objects and connections is defined in `logging-config-dict-connections` below.

### Dictionary Schema Details

The dictionary passed to `dictConfig` must contain the following keys:

- *version* - to be set to an integer value representing the schema version. The only valid value at present is 1, but having this key allows the schema to evolve while still preserving backwards compatibility.

All other keys are optional, but if present they will be interpreted as described below. In all cases below where a 'configuring dict' is mentioned, it will be checked for the special `'()'` key to see if a custom instantiation is required. If so, the mechanism described in `logging-config-dict-userdef` below is used to create an instance; otherwise, the context is used to determine what to instantiate.

<div id="logging-config-dictschema-formatters">

- *formatters* - the corresponding value will be a dict in which each key is a formatter id and each value is a dict describing how to configure the corresponding `~logging.Formatter` instance.

  The configuring dict is searched for the following optional keys which correspond to the arguments passed to create a `~logging.Formatter` object:

  - `format`
  - `datefmt`
  - `style`
  - `validate` (since version \>=3.8)
  - `defaults` (since version \>=3.12)

  An optional `class` key indicates the name of the formatter's class (as a dotted module and class name). The instantiation arguments are as for `~logging.Formatter`, thus this key is most useful for instantiating a customised subclass of `~logging.Formatter`. For example, the alternative class might present exception tracebacks in an expanded or condensed format. If your formatter requires different or extra configuration keys, you should use `logging-config-dict-userdef`.

- *filters* - the corresponding value will be a dict in which each key is a filter id and each value is a dict describing how to configure the corresponding Filter instance.

  The configuring dict is searched for the key `name` (defaulting to the empty string) and this is used to construct a `logging.Filter` instance.

- *handlers* - the corresponding value will be a dict in which each key is a handler id and each value is a dict describing how to configure the corresponding Handler instance.

  The configuring dict is searched for the following keys:

  - `class` (mandatory). This is the fully qualified name of the handler class.

  - `level` (optional). The level of the handler.

  - `formatter` (optional). The id of the formatter for this handler.

  - `filters` (optional). A list of ids of the filters for this handler.

    <div class="versionchanged">

    3.11 `filters` can take filter instances in addition to ids.

    </div>

  All *other* keys are passed through as keyword arguments to the handler's constructor. For example, given the snippet:

  ``` yaml
  handlers:
    console:
      class : logging.StreamHandler
      formatter: brief
      level   : INFO
      filters: [allow_foo]
      stream  : ext://sys.stdout
    file:
      class : logging.handlers.RotatingFileHandler
      formatter: precise
      filename: logconfig.log
      maxBytes: 1024
      backupCount: 3
  ```

  the handler with id `console` is instantiated as a `logging.StreamHandler`, using `sys.stdout` as the underlying stream. The handler with id `file` is instantiated as a `logging.handlers.RotatingFileHandler` with the keyword arguments `filename='logconfig.log', maxBytes=1024, backupCount=3`.

- *loggers* - the corresponding value will be a dict in which each key is a logger name and each value is a dict describing how to configure the corresponding Logger instance.

  The configuring dict is searched for the following keys:

  - `level` (optional). The level of the logger.

  - `propagate` (optional). The propagation setting of the logger.

  - `filters` (optional). A list of ids of the filters for this logger.

    <div class="versionchanged">

    3.11 `filters` can take filter instances in addition to ids.

    </div>

  - `handlers` (optional). A list of ids of the handlers for this logger.

  The specified loggers will be configured according to the level, propagation, filters and handlers specified.

- *root* - this will be the configuration for the root logger. Processing of the configuration will be as for any logger, except that the `propagate` setting will not be applicable.

- *incremental* - whether the configuration is to be interpreted as incremental to the existing configuration. This value defaults to `False`, which means that the specified configuration replaces the existing configuration with the same semantics as used by the existing `fileConfig` API.

  If the specified value is `True`, the configuration is processed as described in the section on `logging-config-dict-incremental`.

- *disable_existing_loggers* - whether any existing non-root loggers are to be disabled. This setting mirrors the parameter of the same name in `fileConfig`. If absent, this parameter defaults to `True`. This value is ignored if *incremental* is `True`.

</div>

### Incremental Configuration

It is difficult to provide complete flexibility for incremental configuration. For example, because objects such as filters and formatters are anonymous, once a configuration is set up, it is not possible to refer to such anonymous objects when augmenting a configuration.

Furthermore, there is not a compelling case for arbitrarily altering the object graph of loggers, handlers, filters, formatters at run-time, once a configuration is set up; the verbosity of loggers and handlers can be controlled just by setting levels (and, in the case of loggers, propagation flags). Changing the object graph arbitrarily in a safe way is problematic in a multi-threaded environment; while not impossible, the benefits are not worth the complexity it adds to the implementation.

Thus, when the `incremental` key of a configuration dict is present and is `True`, the system will completely ignore any `formatters` and `filters` entries, and process only the `level` settings in the `handlers` entries, and the `level` and `propagate` settings in the `loggers` and `root` entries.

Using a value in the configuration dict lets configurations to be sent over the wire as pickled dicts to a socket listener. Thus, the logging verbosity of a long-running application can be altered over time with no need to stop and restart the application.

### Object connections

The schema describes a set of logging objects - loggers, handlers, formatters, filters - which are connected to each other in an object graph. Thus, the schema needs to represent connections between the objects. For example, say that, once configured, a particular logger has attached to it a particular handler. For the purposes of this discussion, we can say that the logger represents the source, and the handler the destination, of a connection between the two. Of course in the configured objects this is represented by the logger holding a reference to the handler. In the configuration dict, this is done by giving each destination object an id which identifies it unambiguously, and then using the id in the source object's configuration to indicate that a connection exists between the source and the destination object with that id.

So, for example, consider the following YAML snippet:

``` yaml
formatters:
  brief:
    # configuration for formatter with id 'brief' goes here
  precise:
    # configuration for formatter with id 'precise' goes here
handlers:
  h1: #This is an id
   # configuration of handler with id 'h1' goes here
   formatter: brief
  h2: #This is another id
   # configuration of handler with id 'h2' goes here
   formatter: precise
loggers:
  foo.bar.baz:
    # other configuration for logger 'foo.bar.baz'
    handlers: [h1, h2]
```

(Note: YAML used here because it's a little more readable than the equivalent Python source form for the dictionary.)

The ids for loggers are the logger names which would be used programmatically to obtain a reference to those loggers, e.g. `foo.bar.baz`. The ids for Formatters and Filters can be any string value (such as `brief`, `precise` above) and they are transient, in that they are only meaningful for processing the configuration dictionary and used to determine connections between objects, and are not persisted anywhere when the configuration call is complete.

The above snippet indicates that logger named `foo.bar.baz` should have two handlers attached to it, which are described by the handler ids `h1` and `h2`. The formatter for `h1` is that described by id `brief`, and the formatter for `h2` is that described by id `precise`.

### User-defined objects

The schema supports user-defined objects for handlers, filters and formatters. (Loggers do not need to have different types for different instances, so there is no support in this configuration schema for user-defined logger classes.)

Objects to be configured are described by dictionaries which detail their configuration. In some places, the logging system will be able to infer from the context how an object is to be instantiated, but when a user-defined object is to be instantiated, the system will not know how to do this. In order to provide complete flexibility for user-defined object instantiation, the user needs to provide a 'factory' - a callable which is called with a configuration dictionary and which returns the instantiated object. This is signalled by an absolute import path to the factory being made available under the special key `'()'`. Here's a concrete example:

``` yaml
formatters:
  brief:
    format: '%(message)s'
  default:
    format: '%(asctime)s %(levelname)-8s %(name)-15s %(message)s'
    datefmt: '%Y-%m-%d %H:%M:%S'
  custom:
      (): my.package.customFormatterFactory
      bar: baz
      spam: 99.9
      answer: 42
```

The above YAML snippet defines three formatters. The first, with id `brief`, is a standard `logging.Formatter` instance with the specified format string. The second, with id `default`, has a longer format and also defines the time format explicitly, and will result in a `logging.Formatter` initialized with those two format strings. Shown in Python source form, the `brief` and `default` formatters have configuration sub-dictionaries:

    {
      'format' : '%(message)s'
    }

and:

    {
      'format' : '%(asctime)s %(levelname)-8s %(name)-15s %(message)s',
      'datefmt' : '%Y-%m-%d %H:%M:%S'
    }

respectively, and as these dictionaries do not contain the special key `'()'`, the instantiation is inferred from the context: as a result, standard `logging.Formatter` instances are created. The configuration sub-dictionary for the third formatter, with id `custom`, is:

    {
      '()' : 'my.package.customFormatterFactory',
      'bar' : 'baz',
      'spam' : 99.9,
      'answer' : 42
    }

and this contains the special key `'()'`, which means that user-defined instantiation is wanted. In this case, the specified factory callable will be used. If it is an actual callable it will be used directly - otherwise, if you specify a string (as in the example) the actual callable will be located using normal import mechanisms. The callable will be called with the **remaining** items in the configuration sub-dictionary as keyword arguments. In the above example, the formatter with id `custom` will be assumed to be returned by the call:

    my.package.customFormatterFactory(bar='baz', spam=99.9, answer=42)

> [!WARNING]
> The values for keys such as `bar`, `spam` and `answer` in the above example should not be configuration dictionaries or references such as `cfg://foo` or `ext://bar`, because they will not be processed by the configuration machinery, but passed to the callable as-is.

The key `'()'` has been used as the special key because it is not a valid keyword parameter name, and so will not clash with the names of the keyword arguments used in the call. The `'()'` also serves as a mnemonic that the corresponding value is a callable.

<div class="versionchanged">

3.11 The `filters` member of `handlers` and `loggers` can take filter instances in addition to ids.

</div>

You can also specify a special key `'.'` whose value is a mapping of attribute names to values. If found, the specified attributes will be set on the user-defined object before it is returned. Thus, with the following configuration:

    {
      '()' : 'my.package.customFormatterFactory',
      'bar' : 'baz',
      'spam' : 99.9,
      'answer' : 42,
      '.' : {
        'foo': 'bar',
        'baz': 'bozz'
      }
    }

the returned formatter will have attribute `foo` set to `'bar'` and attribute `baz` set to `'bozz'`.

> [!WARNING]
> The values for attributes such as `foo` and `baz` in the above example should not be configuration dictionaries or references such as `cfg://foo` or `ext://bar`, because they will not be processed by the configuration machinery, but set as attribute values as-is.

### Handler configuration order

Handlers are configured in alphabetical order of their keys, and a configured handler replaces the configuration dictionary in (a working copy of) the `handlers` dictionary in the schema. If you use a construct such as `cfg://handlers.foo`, then initially `handlers['foo']` points to the configuration dictionary for the handler named `foo`, and later (once that handler has been configured) it points to the configured handler instance. Thus, `cfg://handlers.foo` could resolve to either a dictionary or a handler instance. In general, it is wise to name handlers in a way such that dependent handlers are configured *after* any handlers they depend on; that allows something like `cfg://handlers.foo` to be used in configuring a handler that depends on handler `foo`. If that dependent handler were named `bar`, problems would result, because the configuration of `bar` would be attempted before that of `foo`, and `foo` would not yet have been configured. However, if the dependent handler were named `foobar`, it would be configured after `foo`, with the result that `cfg://handlers.foo` would resolve to configured handler `foo`, and not its configuration dictionary.

### Access to external objects

There are times where a configuration needs to refer to objects external to the configuration, for example `sys.stderr`. If the configuration dict is constructed using Python code, this is straightforward, but a problem arises when the configuration is provided via a text file (e.g. JSON, YAML). In a text file, there is no standard way to distinguish `sys.stderr` from the literal string `'sys.stderr'`. To facilitate this distinction, the configuration system looks for certain special prefixes in string values and treat them specially. For example, if the literal string `'ext://sys.stderr'` is provided as a value in the configuration, then the `ext://` will be stripped off and the remainder of the value processed using normal import mechanisms.

The handling of such prefixes is done in a way analogous to protocol handling: there is a generic mechanism to look for prefixes which match the regular expression `^(?P<prefix>[a-z]+)://(?P<suffix>.*)$` whereby, if the `prefix` is recognised, the `suffix` is processed in a prefix-dependent manner and the result of the processing replaces the string value. If the prefix is not recognised, then the string value will be left as-is.

### Access to internal objects

As well as external objects, there is sometimes also a need to refer to objects in the configuration. This will be done implicitly by the configuration system for things that it knows about. For example, the string value `'DEBUG'` for a `level` in a logger or handler will automatically be converted to the value `logging.DEBUG`, and the `handlers`, `filters` and `formatter` entries will take an object id and resolve to the appropriate destination object.

However, a more generic mechanism is needed for user-defined objects which are not known to the `logging` module. For example, consider `logging.handlers.MemoryHandler`, which takes a `target` argument which is another handler to delegate to. Since the system already knows about this class, then in the configuration, the given `target` just needs to be the object id of the relevant target handler, and the system will resolve to the handler from the id. If, however, a user defines a `my.package.MyHandler` which has an `alternate` handler, the configuration system would not know that the `alternate` referred to a handler. To cater for this, a generic resolution system allows the user to specify:

``` yaml
handlers:
  file:
    # configuration of file handler goes here

  custom:
    (): my.package.MyHandler
    alternate: cfg://handlers.file
```

The literal string `'cfg://handlers.file'` will be resolved in an analogous way to strings with the `ext://` prefix, but looking in the configuration itself rather than the import namespace. The mechanism allows access by dot or by index, in a similar way to that provided by `str.format`. Thus, given the following snippet:

``` yaml
handlers:
  email:
    class: logging.handlers.SMTPHandler
    mailhost: localhost
    fromaddr: my_app@domain.tld
    toaddrs:
      - support_team@domain.tld
      - dev_team@domain.tld
    subject: Houston, we have a problem.
```

in the configuration, the string `'cfg://handlers'` would resolve to the dict with key `handlers`, the string `'cfg://handlers.email` would resolve to the dict with key `email` in the `handlers` dict, and so on. The string `'cfg://handlers.email.toaddrs[1]` would resolve to `'dev_team@domain.tld'` and the string `'cfg://handlers.email.toaddrs[0]'` would resolve to the value `'support_team@domain.tld'`. The `subject` value could be accessed using either `'cfg://handlers.email.subject'` or, equivalently, `'cfg://handlers.email[subject]'`. The latter form only needs to be used if the key contains spaces or non-alphanumeric characters. Please note that the characters `[` and `]` are not allowed in the keys. If an index value consists only of decimal digits, access will be attempted using the corresponding integer value, falling back to the string value if needed.

Given a string `cfg://handlers.myhandler.mykey.123`, this will resolve to `config_dict['handlers']['myhandler']['mykey']['123']`. If the string is specified as `cfg://handlers.myhandler.mykey[123]`, the system will attempt to retrieve the value from `config_dict['handlers']['myhandler']['mykey'][123]`, and fall back to `config_dict['handlers']['myhandler']['mykey']['123']` if that fails.

### Import resolution and custom importers

Import resolution, by default, uses the builtin `__import__` function to do its importing. You may want to replace this with your own importing mechanism: if so, you can replace the `importer` attribute of the `DictConfigurator` or its superclass, the `BaseConfigurator` class. However, you need to be careful because of the way functions are accessed from classes via descriptors. If you are using a Python callable to do your imports, and you want to define it at class level rather than instance level, you need to wrap it with `staticmethod`. For example:

    from importlib import import_module
    from logging.config import BaseConfigurator

    BaseConfigurator.importer = staticmethod(import_module)

You don't need to wrap with `staticmethod` if you're setting the import callable on a configurator *instance*.

### Configuring QueueHandler and QueueListener

If you want to configure a `~logging.handlers.QueueHandler`, noting that this is normally used in conjunction with a `~logging.handlers.QueueListener`, you can configure both together. After the configuration, the `QueueListener` instance will be available as the `~logging.handlers.QueueHandler.listener` attribute of the created handler, and that in turn will be available to you using `~logging.getHandlerByName` and passing the name you have used for the `QueueHandler` in your configuration. The dictionary schema for configuring the pair is shown in the example YAML snippet below.

``` yaml
handlers:
  qhand:
    class: logging.handlers.QueueHandler
    queue: my.module.queue_factory
    listener: my.package.CustomListener
    handlers:
      - hand_name_1
      - hand_name_2
      ...
```

The `queue` and `listener` keys are optional.

If the `queue` key is present, the corresponding value can be one of the following:

- An object implementing the `Queue.put_nowait <queue.Queue.put_nowait>` and `Queue.get <queue.Queue.get>` public API. For instance, this may be an actual instance of `queue.Queue` or a subclass thereof, or a proxy obtained by `multiprocessing.managers.SyncManager.Queue`.

  This is of course only possible if you are constructing or modifying the configuration dictionary in code.

- A string that resolves to a callable which, when called with no arguments, returns the queue instance to use. That callable could be a `queue.Queue` subclass or a function which returns a suitable queue instance, such as `my.module.queue_factory()`.

- A dict with a `'()'` key which is constructed in the usual way as discussed in `logging-config-dict-userdef`. The result of this construction should be a `queue.Queue` instance.

If the `queue` key is absent, a standard unbounded `queue.Queue` instance is created and used.

If the `listener` key is present, the corresponding value can be one of the following:

- A subclass of `logging.handlers.QueueListener`. This is of course only possible if you are constructing or modifying the configuration dictionary in code.
- A string which resolves to a class which is a subclass of `QueueListener`, such as `'my.package.CustomListener'`.
- A dict with a `'()'` key which is constructed in the usual way as discussed in `logging-config-dict-userdef`. The result of this construction should be a callable with the same signature as the `QueueListener` initializer.

If the `listener` key is absent, `logging.handlers.QueueListener` is used.

The values under the `handlers` key are the names of other handlers in the configuration (not shown in the above snippet) which will be passed to the queue listener.

Any custom queue handler and listener classes will need to be defined with the same initialization signatures as `~logging.handlers.QueueHandler` and `~logging.handlers.QueueListener`.

<div class="versionadded">

3.12

</div>

## Configuration file format

The configuration file format understood by `fileConfig` is based on `configparser` functionality. The file must contain sections called `[loggers]`, `[handlers]` and `[formatters]` which identify by name the entities of each type which are defined in the file. For each such entity, there is a separate section which identifies how that entity is configured. Thus, for a logger named `log01` in the `[loggers]` section, the relevant configuration details are held in a section `[logger_log01]`. Similarly, a handler called `hand01` in the `[handlers]` section will have its configuration held in a section called `[handler_hand01]`, while a formatter called `form01` in the `[formatters]` section will have its configuration specified in a section called `[formatter_form01]`. The root logger configuration must be specified in a section called `[logger_root]`.

> [!NOTE]
> The `fileConfig` API is older than the `dictConfig` API and does not provide functionality to cover certain aspects of logging. For example, you cannot configure `~logging.Filter` objects, which provide for filtering of messages beyond simple integer levels, using `fileConfig`. If you need to have instances of `~logging.Filter` in your logging configuration, you will need to use `dictConfig`. Note that future enhancements to configuration functionality will be added to `dictConfig`, so it's worth considering transitioning to this newer API when it's convenient to do so.

Examples of these sections in the file are given below.

``` ini
[loggers]
keys=root,log02,log03,log04,log05,log06,log07

[handlers]
keys=hand01,hand02,hand03,hand04,hand05,hand06,hand07,hand08,hand09

[formatters]
keys=form01,form02,form03,form04,form05,form06,form07,form08,form09
```

The root logger must specify a level and a list of handlers. An example of a root logger section is given below.

``` ini
[logger_root]
level=NOTSET
handlers=hand01
```

The `level` entry can be one of `DEBUG, INFO, WARNING, ERROR, CRITICAL` or `NOTSET`. For the root logger only, `NOTSET` means that all messages will be logged. Level values are `evaluated <func-eval>` in the context of the `logging` package's namespace.

The `handlers` entry is a comma-separated list of handler names, which must appear in the `[handlers]` section. These names must appear in the `[handlers]` section and have corresponding sections in the configuration file.

For loggers other than the root logger, some additional information is required. This is illustrated by the following example.

``` ini
[logger_parser]
level=DEBUG
handlers=hand01
propagate=1
qualname=compiler.parser
```

The `level` and `handlers` entries are interpreted as for the root logger, except that if a non-root logger's level is specified as `NOTSET`, the system consults loggers higher up the hierarchy to determine the effective level of the logger. The `propagate` entry is set to 1 to indicate that messages must propagate to handlers higher up the logger hierarchy from this logger, or 0 to indicate that messages are **not** propagated to handlers up the hierarchy. The `qualname` entry is the hierarchical channel name of the logger, that is to say the name used by the application to get the logger.

Sections which specify handler configuration are exemplified by the following.

``` ini
[handler_hand01]
class=StreamHandler
level=NOTSET
formatter=form01
args=(sys.stdout,)
```

The `class` entry indicates the handler's class (as determined by `eval` in the `logging` package's namespace). The `level` is interpreted as for loggers, and `NOTSET` is taken to mean 'log everything'.

The `formatter` entry indicates the key name of the formatter for this handler. If blank, a default formatter (`logging._defaultFormatter`) is used. If a name is specified, it must appear in the `[formatters]` section and have a corresponding section in the configuration file.

The `args` entry, when `evaluated <func-eval>` in the context of the `logging` package's namespace, is the list of arguments to the constructor for the handler class. Refer to the constructors for the relevant handlers, or to the examples below, to see how typical entries are constructed. If not provided, it defaults to `()`.

The optional `kwargs` entry, when `evaluated <func-eval>` in the context of the `logging` package's namespace, is the keyword argument dict to the constructor for the handler class. If not provided, it defaults to `{}`.

``` ini
[handler_hand02]
class=FileHandler
level=DEBUG
formatter=form02
args=('python.log', 'w')

[handler_hand03]
class=handlers.SocketHandler
level=INFO
formatter=form03
args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT)

[handler_hand04]
class=handlers.DatagramHandler
level=WARN
formatter=form04
args=('localhost', handlers.DEFAULT_UDP_LOGGING_PORT)

[handler_hand05]
class=handlers.SysLogHandler
level=ERROR
formatter=form05
args=(('localhost', handlers.SYSLOG_UDP_PORT), handlers.SysLogHandler.LOG_USER)

[handler_hand06]
class=handlers.NTEventLogHandler
level=CRITICAL
formatter=form06
args=('Python Application', '', 'Application')

[handler_hand07]
class=handlers.SMTPHandler
level=WARN
formatter=form07
args=('localhost', 'from@abc', ['user1@abc', 'user2@xyz'], 'Logger Subject')
kwargs={'timeout': 10.0}

[handler_hand08]
class=handlers.MemoryHandler
level=NOTSET
formatter=form08
target=
args=(10, ERROR)

[handler_hand09]
class=handlers.HTTPHandler
level=NOTSET
formatter=form09
args=('localhost:9022', '/log', 'GET')
kwargs={'secure': True}
```

Sections which specify formatter configuration are typified by the following.

``` ini
[formatter_form01]
format=F1 %(asctime)s %(levelname)s %(message)s %(customfield)s
datefmt=
style=%
validate=True
defaults={'customfield': 'defaultvalue'}
class=logging.Formatter
```

The arguments for the formatter configuration are the same as the keys in the dictionary schema `formatters section
<logging-config-dictschema-formatters>`.

The `defaults` entry, when `evaluated <func-eval>` in the context of the `logging` package's namespace, is a dictionary of default values for custom formatting fields. If not provided, it defaults to `None`.

> [!NOTE]
> Due to the use of `eval` as described above, there are potential security risks which result from using the `listen` to send and receive configurations via sockets. The risks are limited to where multiple users with no mutual trust run code on the same machine; see the `listen` documentation for more information.

<div class="seealso">

Module `logging`  
API reference for the logging module.

Module `logging.handlers`  
Useful handlers included with the logging module.

</div>

# `!logging.handlers` --- Logging handlers

<div class="module" data-synopsis="Handlers for the logging module.">

logging.handlers

</div>

**Source code:** `Lib/logging/handlers.py`

<div class="sidebar">

**Important**

This page contains only reference information. For tutorials, please see

- `Basic Tutorial <logging-basic-tutorial>`
- `Advanced Tutorial <logging-advanced-tutorial>`
- `Logging Cookbook <logging-cookbook>`

</div>

------------------------------------------------------------------------

<div class="currentmodule">

logging

</div>

The following useful handlers are provided in the package. Note that three of the handlers (`StreamHandler`, `FileHandler` and `NullHandler`) are actually defined in the `logging` module itself, but have been documented here along with the other handlers.

## StreamHandler

The `StreamHandler` class, located in the core `logging` package, sends logging output to streams such as *sys.stdout*, *sys.stderr* or any file-like object (or, more precisely, any object which supports `write` and `flush` methods).

<div class="StreamHandler(stream=None)">

Returns a new instance of the `StreamHandler` class. If *stream* is specified, the instance will use it for logging output; otherwise, *sys.stderr* will be used.

<div class="method">

emit(record)

If a formatter is specified, it is used to format the record. The record is then written to the stream followed by `terminator`. If exception information is present, it is formatted using `traceback.print_exception` and appended to the stream.

</div>

<div class="method">

flush()

Flushes the stream by calling its `flush` method. Note that the `close` method is inherited from `~logging.Handler` and so does no output, so an explicit `flush` call may be needed at times.

</div>

<div class="method">

setStream(stream)

Sets the instance's stream to the specified value, if it is different. The old stream is flushed before the new stream is set.

param stream  
The stream that the handler should use.

return  
the old stream, if the stream was changed, or `None` if it wasn't.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

terminator

String used as the terminator when writing a formatted record to a stream. Default value is `'\n'`.

If you don't want a newline termination, you can set the handler instance's `terminator` attribute to the empty string.

In earlier versions, the terminator was hardcoded as `'\n'`.

<div class="versionadded">

3.2

</div>

</div>

</div>

## FileHandler

The `FileHandler` class, located in the core `logging` package, sends logging output to a disk file. It inherits the output functionality from `StreamHandler`.

<div class="FileHandler(filename, mode='a', encoding=None, delay=False, errors=None)">

Returns a new instance of the `FileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, `'a'` is used. If *encoding* is not `None`, it is used to open the file with that encoding. If *delay* is true, then file opening is deferred until the first call to `emit`. By default, the file grows indefinitely. If *errors* is specified, it's used to determine how encoding errors are handled.

<div class="versionchanged">

3.6 As well as string values, `~pathlib.Path` objects are also accepted for the *filename* argument.

</div>

<div class="versionchanged">

3.9 The *errors* parameter was added.

</div>

<div class="method">

close()

Closes the file.

</div>

<div class="method">

emit(record)

Outputs the record to the file.

Note that if the file was closed due to logging shutdown at exit and the file mode is 'w', the record will not be emitted (see `42378`).

</div>

</div>

## NullHandler

<div class="versionadded">

3.1

</div>

The `NullHandler` class, located in the core `logging` package, does not do any formatting or output. It is essentially a 'no-op' handler for use by library developers.

<div class="NullHandler()">

Returns a new instance of the `NullHandler` class.

<div class="method">

emit(record)

This method does nothing.

</div>

<div class="method">

handle(record)

This method does nothing.

</div>

<div class="method">

createLock()

This method returns `None` for the lock, since there is no underlying I/O to which access needs to be serialized.

</div>

</div>

See `library-config` for more information on how to use `NullHandler`.

## WatchedFileHandler

<div class="currentmodule">

logging.handlers

</div>

The `WatchedFileHandler` class, located in the `!logging.handlers` module, is a `FileHandler` which watches the file it is logging to. If the file changes, it is closed and reopened using the file name.

A file change can happen because of usage of programs such as *newsyslog* and *logrotate* which perform log file rotation. This handler, intended for use under Unix/Linux, watches the file to see if it has changed since the last emit. (A file is deemed to have changed if its device or inode have changed.) If the file has changed, the old file stream is closed, and the file opened to get a new stream.

This handler is not appropriate for use under Windows, because under Windows open log files cannot be moved or renamed - logging opens the files with exclusive locks - and so there is no need for such a handler. Furthermore, *ST_INO* is not supported under Windows; `~os.stat` always returns zero for this value.

<div class="WatchedFileHandler(filename, mode='a', encoding=None, delay=False, errors=None)">

Returns a new instance of the `WatchedFileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, `'a'` is used. If *encoding* is not `None`, it is used to open the file with that encoding. If *delay* is true, then file opening is deferred until the first call to `emit`. By default, the file grows indefinitely. If *errors* is provided, it determines how encoding errors are handled.

<div class="versionchanged">

3.6 As well as string values, `~pathlib.Path` objects are also accepted for the *filename* argument.

</div>

<div class="versionchanged">

3.9 The *errors* parameter was added.

</div>

<div class="method">

reopenIfNeeded()

Checks to see if the file has changed. If it has, the existing stream is flushed and closed and the file opened again, typically as a precursor to outputting the record to the file.

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

emit(record)

Outputs the record to the file, but first calls `reopenIfNeeded` to reopen the file if it has changed.

</div>

</div>

## BaseRotatingHandler

The `BaseRotatingHandler` class, located in the `!logging.handlers` module, is the base class for the rotating file handlers, `RotatingFileHandler` and `TimedRotatingFileHandler`. You should not need to instantiate this class, but it has attributes and methods you may need to override.

<div class="BaseRotatingHandler(filename, mode, encoding=None, delay=False, errors=None)">

The parameters are as for `FileHandler`. The attributes are:

<div class="attribute">

namer

If this attribute is set to a callable, the `rotation_filename` method delegates to this callable. The parameters passed to the callable are those passed to `rotation_filename`.

> [!NOTE]
> The namer function is called quite a few times during rollover, so it should be as simple and as fast as possible. It should also return the same output every time for a given input, otherwise the rollover behaviour may not work as expected.
>
> It's also worth noting that care should be taken when using a namer to preserve certain attributes in the filename which are used during rotation. For example, `RotatingFileHandler` expects to have a set of log files whose names contain successive integers, so that rotation works as expected, and `TimedRotatingFileHandler` deletes old log files (based on the `backupCount` parameter passed to the handler's initializer) by determining the oldest files to delete. For this to happen, the filenames should be sortable using the date/time portion of the filename, and a namer needs to respect this. (If a namer is wanted that doesn't respect this scheme, it will need to be used in a subclass of `TimedRotatingFileHandler` which overrides the `~TimedRotatingFileHandler.getFilesToDelete` method to fit in with the custom naming scheme.)

<div class="versionadded">

3.3

</div>

</div>

<div class="attribute">

BaseRotatingHandler.rotator

If this attribute is set to a callable, the `rotate` method delegates to this callable. The parameters passed to the callable are those passed to `rotate`.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

BaseRotatingHandler.rotation_filename(default_name)

Modify the filename of a log file when rotating.

This is provided so that a custom filename can be provided.

The default implementation calls the 'namer' attribute of the handler, if it's callable, passing the default name to it. If the attribute isn't callable (the default is `None`), the name is returned unchanged.

param default_name  
The default name for the log file.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

BaseRotatingHandler.rotate(source, dest)

When rotating, rotate the current log.

The default implementation calls the 'rotator' attribute of the handler, if it's callable, passing the source and dest arguments to it. If the attribute isn't callable (the default is `None`), the source is simply renamed to the destination.

param source  
The source filename. This is normally the base filename, e.g. 'test.log'.

param dest  
The destination filename. This is normally what the source is rotated to, e.g. 'test.log.1'.

<div class="versionadded">

3.3

</div>

</div>

</div>

The reason the attributes exist is to save you having to subclass - you can use the same callables for instances of `RotatingFileHandler` and `TimedRotatingFileHandler`. If either the namer or rotator callable raises an exception, this will be handled in the same way as any other exception during an `emit` call, i.e. via the `handleError` method of the handler.

If you need to make more significant changes to rotation processing, you can override the methods.

For an example, see `cookbook-rotator-namer`.

## RotatingFileHandler

The `RotatingFileHandler` class, located in the `!logging.handlers` module, supports rotation of disk log files.

<div class="RotatingFileHandler(filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=False, errors=None)">

Returns a new instance of the `RotatingFileHandler` class. The specified file is opened and used as the stream for logging. If *mode* is not specified, `'a'` is used. If *encoding* is not `None`, it is used to open the file with that encoding. If *delay* is true, then file opening is deferred until the first call to `emit`. By default, the file grows indefinitely. If *errors* is provided, it determines how encoding errors are handled.

You can use the *maxBytes* and *backupCount* values to allow the file to `rollover` at a predetermined size. When the size is about to be exceeded, the file is closed and a new file is silently opened for output. Rollover occurs whenever the current log file is nearly *maxBytes* in length; but if either of *maxBytes* or *backupCount* is zero, rollover never occurs, so you generally want to set *backupCount* to at least 1, and have a non-zero *maxBytes*. When *backupCount* is non-zero, the system will save old log files by appending the extensions '.1', '.2' etc., to the filename. For example, with a *backupCount* of 5 and a base file name of `app.log`, you would get `app.log`, `app.log.1`, `app.log.2`, up to `app.log.5`. The file being written to is always `app.log`. When this file is filled, it is closed and renamed to `app.log.1`, and if files `app.log.1`, `app.log.2`, etc. exist, then they are renamed to `app.log.2`, `app.log.3` etc. respectively.

<div class="versionchanged">

3.6 As well as string values, `~pathlib.Path` objects are also accepted for the *filename* argument.

</div>

<div class="versionchanged">

3.9 The *errors* parameter was added.

</div>

<div class="method">

doRollover()

Does a rollover, as described above.

</div>

<div class="method">

emit(record)

Outputs the record to the file, catering for rollover as described previously.

</div>

<div class="method">

shouldRollover(record)

See if the supplied record would cause the file to exceed the configured size limit.

</div>

</div>

## TimedRotatingFileHandler

The `TimedRotatingFileHandler` class, located in the `!logging.handlers` module, supports rotation of disk log files at certain timed intervals.

<div class="TimedRotatingFileHandler(filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, atTime=None, errors=None)">

Returns a new instance of the `TimedRotatingFileHandler` class. The specified file is opened and used as the stream for logging. On rotating it also sets the filename suffix. Rotating happens based on the product of *when* and *interval*.

You can use the *when* to specify the type of *interval*. The list of possible values is below. Note that they are not case sensitive.

| Value | Type of interval | If/how *atTime* is used |
|----|----|----|
| `'S'` | Seconds | Ignored |
| `'M'` | Minutes | Ignored |
| `'H'` | Hours | Ignored |
| `'D'` | Days | Ignored |
| `'W0'-'W6'` | Weekday (0=Monday) | Used to compute initial rollover time |
| `'midnight'` | Roll over at midnight, if *atTime* not specified, else at time *atTime* | Used to compute initial rollover time |

When using weekday-based rotation, specify 'W0' for Monday, 'W1' for Tuesday, and so on up to 'W6' for Sunday. In this case, the value passed for *interval* isn't used.

The system will save old log files by appending extensions to the filename. The extensions are date-and-time based, using the strftime format `%Y-%m-%d_%H-%M-%S` or a leading portion thereof, depending on the rollover interval.

When computing the next rollover time for the first time (when the handler is created), the creation time (if supported by the OS and file system) or the last modification of an existing log file, or else the current time, is used to compute when the next rotation will occur.

If the *utc* argument is true, times in UTC will be used; otherwise local time is used.

If *backupCount* is nonzero, at most *backupCount* files will be kept, and if more would be created when rollover occurs, the oldest one is deleted. The deletion logic uses the interval to determine which files to delete, so changing the interval may leave old files lying around.

If *delay* is true, then file opening is deferred until the first call to `emit`.

If *atTime* is not `None`, it must be a `datetime.time` instance which specifies the time of day when rollover occurs, for the cases where rollover is set to happen "at midnight" or "on a particular weekday". Note that in these cases, the *atTime* value is effectively used to compute the *initial* rollover, and subsequent rollovers would be calculated via the normal interval calculation.

If *errors* is specified, it's used to determine how encoding errors are handled.

> [!NOTE]
> Calculation of the initial rollover time is done when the handler is initialised. Calculation of subsequent rollover times is done only when rollover occurs, and rollover occurs only when emitting output. If this is not kept in mind, it might lead to some confusion. For example, if an interval of "every minute" is set, that does not mean you will always see log files with times (in the filename) separated by a minute; if, during application execution, logging output is generated more frequently than once a minute, *then* you can expect to see log files with times separated by a minute. If, on the other hand, logging messages are only output once every five minutes (say), then there will be gaps in the file times corresponding to the minutes where no output (and hence no rollover) occurred.

<div class="versionchanged">

3.4 *atTime* parameter was added.

</div>

<div class="versionchanged">

3.6 As well as string values, `~pathlib.Path` objects are also accepted for the *filename* argument.

</div>

<div class="versionchanged">

3.9 The *errors* parameter was added.

</div>

<div class="versionchanged">

3.16 Use the creation time instead of the last modification time, if supported by the OS and file system.

</div>

<div class="method">

doRollover()

Does a rollover, as described above.

</div>

<div class="method">

emit(record)

Outputs the record to the file, catering for rollover as described above.

</div>

<div class="method">

getFilesToDelete()

Returns a list of filenames which should be deleted as part of rollover. These are the absolute paths of the oldest backup log files written by the handler.

</div>

<div class="method">

shouldRollover(record)

See if enough time has passed for a rollover to occur and if it has, compute the next rollover time.

</div>

</div>

## SocketHandler

The `SocketHandler` class, located in the `!logging.handlers` module, sends logging output to a network socket. The base class uses a TCP socket.

<div class="SocketHandler(host, port)">

Returns a new instance of the `SocketHandler` class intended to communicate with a remote machine whose address is given by *host* and *port*.

<div class="versionchanged">

3.4 If `port` is specified as `None`, a Unix domain socket is created using the value in `host` - otherwise, a TCP socket is created.

</div>

<div class="method">

close()

Closes the socket.

</div>

<div class="method">

emit()

Pickles the record's attribute dictionary and writes it to the socket in binary format. If there is an error with the socket, silently drops the packet. If the connection was previously lost, re-establishes the connection. To unpickle the record at the receiving end into a `~logging.LogRecord`, use the `~logging.makeLogRecord` function.

</div>

<div class="method">

handleError()

Handles an error which has occurred during `emit`. The most likely cause is a lost connection. Closes the socket so that we can retry on the next event.

</div>

<div class="method">

makeSocket()

This is a factory method which allows subclasses to define the precise type of socket they want. The default implementation creates a TCP socket (`socket.SOCK_STREAM`).

</div>

<div class="method">

makePickle(record)

Pickles the record's attribute dictionary in binary format with a length prefix, and returns it ready for transmission across the socket. The details of this operation are equivalent to:

    data = pickle.dumps(record_attr_dict, 1)
    datalen = struct.pack('>L', len(data))
    return datalen + data

Note that pickles aren't completely secure. If you are concerned about security, you may want to override this method to implement a more secure mechanism. For example, you can sign pickles using HMAC and then verify them on the receiving end, or alternatively you can disable unpickling of global objects on the receiving end.

</div>

<div class="method">

send(packet)

Send a pickled byte-string *packet* to the socket. The format of the sent byte-string is as described in the documentation for `~SocketHandler.makePickle`.

This function allows for partial sends, which can happen when the network is busy.

</div>

<div class="method">

createSocket()

Tries to create a socket; on failure, uses an exponential back-off algorithm. On initial failure, the handler will drop the message it was trying to send. When subsequent messages are handled by the same instance, it will not try connecting until some time has passed. The default parameters are such that the initial delay is one second, and if after that delay the connection still can't be made, the handler will double the delay each time up to a maximum of 30 seconds.

This behaviour is controlled by the following handler attributes:

- `retryStart` (initial delay, defaulting to 1.0 seconds).
- `retryFactor` (multiplier, defaulting to 2.0).
- `retryMax` (maximum delay, defaulting to 30.0 seconds).

This means that if the remote listener starts up *after* the handler has been used, you could lose messages (since the handler won't even attempt a connection until the delay has elapsed, but just silently drop messages during the delay period).

</div>

</div>

## DatagramHandler

The `DatagramHandler` class, located in the `!logging.handlers` module, inherits from `SocketHandler` to support sending logging messages over UDP sockets.

<div class="DatagramHandler(host, port)">

Returns a new instance of the `DatagramHandler` class intended to communicate with a remote machine whose address is given by *host* and *port*.

> [!NOTE]
> As UDP is not a streaming protocol, there is no persistent connection between an instance of this handler and *host*. For this reason, when using a network socket, a DNS lookup might have to be made each time an event is logged, which can introduce some latency into the system. If this affects you, you can do a lookup yourself and initialize this handler using the looked-up IP address rather than the hostname.

<div class="versionchanged">

3.4 If `port` is specified as `None`, a Unix domain socket is created using the value in `host` - otherwise, a UDP socket is created.

</div>

<div class="method">

emit()

Pickles the record's attribute dictionary and writes it to the socket in binary format. If there is an error with the socket, silently drops the packet. To unpickle the record at the receiving end into a `~logging.LogRecord`, use the `~logging.makeLogRecord` function.

</div>

<div class="method">

makeSocket()

The factory method of `SocketHandler` is here overridden to create a UDP socket (`socket.SOCK_DGRAM`).

</div>

<div class="method">

send(s)

Send a pickled byte-string to a socket. The format of the sent byte-string is as described in the documentation for `SocketHandler.makePickle`.

</div>

</div>

## SysLogHandler

The `SysLogHandler` class, located in the `!logging.handlers` module, supports sending logging messages to a remote or local Unix syslog.

<div class="SysLogHandler(address=('localhost', SYSLOG_UDP_PORT), facility=LOG_USER, socktype=socket.SOCK_DGRAM, timeout=None)">

Returns a new instance of the `SysLogHandler` class intended to communicate with a remote Unix machine whose address is given by *address* in the form of a `(host, port)` tuple. If *address* is not specified, `('localhost', 514)` is used. The address is used to open a socket. An alternative to providing a `(host, port)` tuple is providing an address as a string or a `bytes` object, for example '/dev/log'. In this case, a Unix domain socket is used to send the message to the syslog. If *facility* is not specified, `LOG_USER` is used. The type of socket opened depends on the *socktype* argument, which defaults to `socket.SOCK_DGRAM` and thus opens a UDP socket. To open a TCP socket (for use with the newer syslog daemons such as rsyslog), specify a value of `socket.SOCK_STREAM`. If *timeout* is specified, it sets a timeout (in seconds) for the socket operations. This can help prevent the program from hanging indefinitely if the syslog server is unreachable. By default, *timeout* is `None`, meaning no timeout is applied.

Note that if your server is not listening on UDP port 514, `SysLogHandler` may appear not to work. In that case, check what address you should be using for a domain socket - it's system dependent. For example, on Linux it's usually '/dev/log' but on OS/X it's '/var/run/syslog'. You'll need to check your platform and use the appropriate address (you may need to do this check at runtime if your application needs to run on several platforms). On Windows, you pretty much have to use the UDP option.

> [!NOTE]
> On macOS 12.x (Monterey), Apple has changed the behaviour of their syslog daemon - it no longer listens on a domain socket. Therefore, you cannot expect `SysLogHandler` to work on this system.
>
> See `91070` for more information.

<div class="versionchanged">

3.2 *socktype* was added.

</div>

<div class="versionchanged">

3.14 *timeout* was added.

</div>

<div class="versionchanged">

3.16 *address* can now be a `bytes` object.

</div>

<div class="method">

close()

Closes the socket to the remote host.

</div>

<div class="method">

createSocket()

Tries to create a socket and, if it's not a datagram socket, connect it to the other end. This method is called during handler initialization, but it's not regarded as an error if the other end isn't listening at this point - the method will be called again when emitting an event, if there is no socket at that point.

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

emit(record)

The record is formatted, and then sent to the syslog server. If exception information is present, it is *not* sent to the server.

<div class="versionchanged">

3.2.1 (See: `12168`.) In earlier versions, the message sent to the syslog daemons was always terminated with a NUL byte, because early versions of these daemons expected a NUL terminated message - even though it's not in the relevant specification (`5424`). More recent versions of these daemons don't expect the NUL byte but strip it off if it's there, and even more recent daemons (which adhere more closely to RFC 5424) pass the NUL byte on as part of the message.

To enable easier handling of syslog messages in the face of all these differing daemon behaviours, the appending of the NUL byte has been made configurable, through the use of a class-level attribute, `append_nul`. This defaults to `True` (preserving the existing behaviour) but can be set to `False` on a `SysLogHandler` instance in order for that instance to *not* append the NUL terminator.

</div>

<div class="versionchanged">

3.3 (See: `12419`.) In earlier versions, there was no facility for an "ident" or "tag" prefix to identify the source of the message. This can now be specified using a class-level attribute, defaulting to `""` to preserve existing behaviour, but which can be overridden on a `SysLogHandler` instance in order for that instance to prepend the ident to every message handled. Note that the provided ident must be text, not bytes, and is prepended to the message exactly as is.

</div>

</div>

<div class="method">

encodePriority(facility, priority)

Encodes the facility and priority into an integer. You can pass in strings or integers - if strings are passed, internal mapping dictionaries are used to convert them to integers.

The symbolic `LOG_` values are defined in `SysLogHandler` and mirror the values defined in the `sys/syslog.h` header file.

**Priorities**

| Name (string)        | Symbolic value |
|----------------------|----------------|
| `alert`              | LOG_ALERT      |
| `crit` or `critical` | LOG_CRIT       |
| `debug`              | LOG_DEBUG      |
| `emerg` or `panic`   | LOG_EMERG      |
| `err` or `error`     | LOG_ERR        |
| `info`               | LOG_INFO       |
| `notice`             | LOG_NOTICE     |
| `warn` or `warning`  | LOG_WARNING    |

**Facilities**

| Name (string) | Symbolic value |
|---------------|----------------|
| `auth`        | LOG_AUTH       |
| `authpriv`    | LOG_AUTHPRIV   |
| `cron`        | LOG_CRON       |
| `daemon`      | LOG_DAEMON     |
| `ftp`         | LOG_FTP        |
| `kern`        | LOG_KERN       |
| `lpr`         | LOG_LPR        |
| `mail`        | LOG_MAIL       |
| `news`        | LOG_NEWS       |
| `syslog`      | LOG_SYSLOG     |
| `user`        | LOG_USER       |
| `uucp`        | LOG_UUCP       |
| `local0`      | LOG_LOCAL0     |
| `local1`      | LOG_LOCAL1     |
| `local2`      | LOG_LOCAL2     |
| `local3`      | LOG_LOCAL3     |
| `local4`      | LOG_LOCAL4     |
| `local5`      | LOG_LOCAL5     |
| `local6`      | LOG_LOCAL6     |
| `local7`      | LOG_LOCAL7     |

</div>

<div class="method">

mapPriority(levelname)

Maps a logging level name to a syslog priority name. You may need to override this if you are using custom levels, or if the default algorithm is not suitable for your needs. The default algorithm maps `DEBUG`, `INFO`, `WARNING`, `ERROR` and `CRITICAL` to the equivalent syslog names, and all other level names to 'warning'.

</div>

</div>

## NTEventLogHandler

The `NTEventLogHandler` class, located in the `!logging.handlers` module, supports sending logging messages to a local Windows NT, Windows 2000 or Windows XP event log. Before you can use it, you need Mark Hammond's Win32 extensions for Python installed.

<div class="NTEventLogHandler(appname, dllname=None, logtype='Application')">

Returns a new instance of the `NTEventLogHandler` class. The *appname* is used to define the application name as it appears in the event log. An appropriate registry entry is created using this name. The *dllname* should give the fully qualified pathname of a .dll or .exe which contains message definitions to hold in the log (if not specified, `'win32service.pyd'` is used - this is installed with the Win32 extensions and contains some basic placeholder message definitions. Note that use of these placeholders will make your event logs big, as the entire message source is held in the log. If you want slimmer logs, you have to pass in the name of your own .dll or .exe which contains the message definitions you want to use in the event log). The *logtype* is one of `'Application'`, `'System'` or `'Security'`, and defaults to `'Application'`.

<div class="method">

close()

At this point, you can remove the application name from the registry as a source of event log entries. However, if you do this, you will not be able to see the events as you intended in the Event Log Viewer - it needs to be able to access the registry to get the .dll name. The current version does not do this.

</div>

<div class="method">

emit(record)

Determines the message ID, event category and event type, and then logs the message in the NT event log.

</div>

<div class="method">

getEventCategory(record)

Returns the event category for the record. Override this if you want to specify your own categories. This version returns 0.

</div>

<div class="method">

getEventType(record)

Returns the event type for the record. Override this if you want to specify your own types. This version does a mapping using the handler's typemap attribute, which is set up in `__init__` to a dictionary which contains mappings for `DEBUG`, `INFO`, `WARNING`, `ERROR` and `CRITICAL`. If you are using your own levels, you will either need to override this method or place a suitable dictionary in the handler's *typemap* attribute.

</div>

<div class="method">

getMessageID(record)

Returns the message ID for the record. If you are using your own messages, you could do this by having the *msg* passed to the logger being an ID rather than a format string. Then, in here, you could use a dictionary lookup to get the message ID. This version returns 1, which is the base message ID in `win32service.pyd`.

</div>

</div>

## SMTPHandler

The `SMTPHandler` class, located in the `!logging.handlers` module, supports sending logging messages to an email address via SMTP.

<div class="SMTPHandler(mailhost, fromaddr, toaddrs, subject, credentials=None, secure=None, timeout=1.0)">

Returns a new instance of the `SMTPHandler` class. The instance is initialized with the from and to addresses and subject line of the email. The *toaddrs* should be a list of strings. To specify a non-standard SMTP port, use the (host, port) tuple format for the *mailhost* argument. If you use a string, the standard SMTP port is used. If your SMTP server requires authentication, you can specify a (username, password) tuple for the *credentials* argument.

To specify the use of a secure protocol (TLS), pass in a tuple to the *secure* argument. This will only be used when authentication credentials are supplied. The tuple should be either an empty tuple, or a single-value tuple with the name of a keyfile, or a 2-value tuple with the names of the keyfile and certificate file. (This tuple is passed to the `smtplib.SMTP.starttls` method.)

A timeout can be specified for communication with the SMTP server using the *timeout* argument.

<div class="versionchanged">

3.3 Added the *timeout* parameter.

</div>

<div class="method">

emit(record)

Formats the record and sends it to the specified addressees.

</div>

<div class="method">

getSubject(record)

If you want to specify a subject line which is record-dependent, override this method.

</div>

</div>

## MemoryHandler

The `MemoryHandler` class, located in the `!logging.handlers` module, supports buffering of logging records in memory, periodically flushing them to a `target` handler. Flushing occurs whenever the buffer is full, or when an event of a certain severity or greater is seen.

`MemoryHandler` is a subclass of the more general `BufferingHandler`, which is an abstract class. This buffers logging records in memory. Whenever each record is added to the buffer, a check is made by calling `shouldFlush` to see if the buffer should be flushed. If it should, then `flush` is expected to do the flushing.

<div class="BufferingHandler(capacity)">

Initializes the handler with a buffer of the specified capacity. Here, *capacity* means the number of logging records buffered.

<div class="method">

emit(record)

Append the record to the buffer. If `shouldFlush` returns true, call `flush` to process the buffer.

</div>

<div class="method">

flush()

For a `BufferingHandler` instance, flushing means that it sets the buffer to an empty list. This method can be overwritten to implement more useful flushing behavior.

</div>

<div class="method">

shouldFlush(record)

Return `True` if the buffer is up to capacity. This method can be overridden to implement custom flushing strategies.

</div>

</div>

<div class="MemoryHandler(capacity, flushLevel=ERROR, target=None, flushOnClose=True)">

Returns a new instance of the `MemoryHandler` class. The instance is initialized with a buffer size of *capacity* (number of records buffered). If *flushLevel* is not specified, `ERROR` is used. If no *target* is specified, the target will need to be set using `setTarget` before this handler does anything useful. If *flushOnClose* is specified as `False`, then the buffer is *not* flushed when the handler is closed. If not specified or specified as `True`, the previous behaviour of flushing the buffer will occur when the handler is closed.

<div class="versionchanged">

3.6 The *flushOnClose* parameter was added.

</div>

<div class="method">

close()

Calls `flush`, sets the target to `None` and clears the buffer.

</div>

<div class="method">

flush()

For a `MemoryHandler` instance, flushing means just sending the buffered records to the target, if there is one. The buffer is also cleared when buffered records are sent to the target. Override if you want different behavior.

</div>

<div class="method">

setTarget(target)

Sets the target handler for this handler.

</div>

<div class="method">

shouldFlush(record)

Checks for buffer full or a record at the *flushLevel* or higher.

</div>

</div>

## HTTPHandler

The `HTTPHandler` class, located in the `!logging.handlers` module, supports sending logging messages to a web server, using either `GET` or `POST` semantics.

<div class="HTTPHandler(host, url, method='GET', secure=False, credentials=None, context=None)">

Returns a new instance of the `HTTPHandler` class. The *host* can be of the form `host:port`, should you need to use a specific port number. If no *method* is specified, `GET` is used. If *secure* is true, a HTTPS connection will be used. The *context* parameter may be set to a `ssl.SSLContext` instance to configure the SSL settings used for the HTTPS connection. If *credentials* is specified, it should be a 2-tuple consisting of userid and password, which will be placed in a HTTP 'Authorization' header using Basic authentication. If you specify credentials, you should also specify secure=True so that your userid and password are not passed in cleartext across the wire.

<div class="versionchanged">

3.5 The *context* parameter was added.

</div>

<div class="method">

mapLogRecord(record)

Provides a dictionary, based on `record`, which is to be URL-encoded and sent to the web server. The default implementation just returns `record.__dict__`. This method can be overridden if e.g. only a subset of `~logging.LogRecord` is to be sent to the web server, or if more specific customization of what's sent to the server is required.

</div>

<div class="method">

emit(record)

Sends the record to the web server as a URL-encoded dictionary. The `mapLogRecord` method is used to convert the record to the dictionary to be sent.

</div>

> [!NOTE]
> Since preparing a record for sending it to a web server is not the same as a generic formatting operation, using `~logging.Handler.setFormatter` to specify a `~logging.Formatter` for a `HTTPHandler` has no effect. Instead of calling `~logging.Handler.format`, this handler calls `mapLogRecord` and then `urllib.parse.urlencode` to encode the dictionary in a form suitable for sending to a web server.

</div>

## QueueHandler

<div class="versionadded">

3.2

</div>

The `QueueHandler` class, located in the `!logging.handlers` module, supports sending logging messages to a queue, such as those implemented in the `queue` or `multiprocessing` modules.

Along with the `QueueListener` class, `QueueHandler` can be used to let handlers do their work on a separate thread from the one which does the logging. This is important in web applications and also other service applications where threads servicing clients need to respond as quickly as possible, while any potentially slow operations (such as sending an email via `SMTPHandler`) are done on a separate thread.

<div class="QueueHandler(queue)">

Returns a new instance of the `QueueHandler` class. The instance is initialized with the queue to send messages to. The *queue* can be any queue-like object; it's used as-is by the `enqueue` method, which needs to know how to send messages to it. The queue is not *required* to have the task tracking API, which means that you can use `~queue.SimpleQueue` instances for *queue*.

> [!NOTE]
> If you are using `multiprocessing`, you should avoid using `~queue.SimpleQueue` and instead use `multiprocessing.Queue`.

> [!WARNING]
> The `multiprocessing` module uses an internal logger created and accessed via `~multiprocessing.get_logger`. `multiprocessing.Queue` will log `DEBUG` level messages upon items being queued. If those log messages are processed by a `QueueHandler` using the same `multiprocessing.Queue` instance, it will cause a deadlock or infinite recursion.

<div class="method">

emit(record)

Enqueues the result of preparing the LogRecord. Should an exception occur (e.g. because a bounded queue has filled up), the `~logging.Handler.handleError` method is called to handle the error. This can result in the record silently being dropped (if `logging.raiseExceptions` is `False`) or a message printed to `sys.stderr` (if `logging.raiseExceptions` is `True`).

</div>

<div class="method">

prepare(record)

Prepares a record for queuing. The object returned by this method is enqueued.

The base implementation formats the record to merge the message, arguments, exception and stack information, if present. It also removes unpickleable items from the record in-place. Specifically, it overwrites the record's `msg` and `message` attributes with the merged message (obtained by calling the handler's `format` method), and sets the `args`, `exc_info` and `exc_text` attributes to `None`.

You might want to override this method if you want to convert the record to a dict or JSON string, or send a modified copy of the record while leaving the original intact.

> [!NOTE]
> The base implementation formats the message with arguments, sets the `message` and `msg` attributes to the formatted message and sets the `args` and `exc_text` attributes to `None` to allow pickling and to prevent further attempts at formatting. This means that a handler on the `QueueListener` side won't have the information to do custom formatting, e.g. of exceptions. You may wish to subclass `QueueHandler` and override this method to e.g. avoid setting `exc_text` to `None`. Note that the `message` / `msg` / `args` changes are related to ensuring the record is pickleable, and you might or might not be able to avoid doing that depending on whether your `args` are pickleable. (Note that you may have to consider not only your own code but also code in any libraries that you use.)

</div>

<div class="method">

enqueue(record)

Enqueues the record on the queue using `put_nowait()`; you may want to override this if you want to use blocking behaviour, or a timeout, or a customized queue implementation.

</div>

<div class="attribute">

listener

When created via configuration using `~logging.config.dictConfig`, this attribute will contain a `QueueListener` instance for use with this handler. Otherwise, it will be `None`.

<div class="versionadded">

3.12

</div>

</div>

</div>

## QueueListener

<div class="versionadded">

3.2

</div>

The `QueueListener` class, located in the `!logging.handlers` module, supports receiving logging messages from a queue, such as those implemented in the `queue` or `multiprocessing` modules. The messages are received from a queue in an internal thread and passed, on the same thread, to one or more handlers for processing. While `QueueListener` is not itself a handler, it is documented here because it works hand-in-hand with `QueueHandler`.

Along with the `QueueHandler` class, `QueueListener` can be used to let handlers do their work on a separate thread from the one which does the logging. This is important in web applications and also other service applications where threads servicing clients need to respond as quickly as possible, while any potentially slow operations (such as sending an email via `SMTPHandler`) are done on a separate thread.

<div class="QueueListener(queue, *handlers, respect_handler_level=False)">

Returns a new instance of the `QueueListener` class. The instance is initialized with the queue to send messages to and a list of handlers which will handle entries placed on the queue. The queue can be any queue-like object; it's passed as-is to the `dequeue` method, which needs to know how to get messages from it. The queue is not *required* to have the task tracking API (though it's used if available), which means that you can use `~queue.SimpleQueue` instances for *queue*.

> [!NOTE]
> If you are using `multiprocessing`, you should avoid using `~queue.SimpleQueue` and instead use `multiprocessing.Queue`.

If `respect_handler_level` is `True`, a handler's level is respected (compared with the level for the message) when deciding whether to pass messages to that handler; otherwise, the behaviour is as in previous Python versions - to always pass each message to each handler.

<div class="versionchanged">

3.5 The `respect_handler_level` argument was added.

</div>

<div class="versionchanged">

3.14 `QueueListener` can now be used as a context manager via `with`. When entering the context, the listener is started. When exiting the context, the listener is stopped. `~contextmanager.__enter__` returns the `QueueListener` object.

</div>

<div class="method">

dequeue(block)

Dequeues a record and return it, optionally blocking.

The base implementation uses `get()`. You may want to override this method if you want to use timeouts or work with custom queue implementations.

</div>

<div class="method">

prepare(record)

Prepare a record for handling.

This implementation just returns the passed-in record. You may want to override this method if you need to do any custom marshalling or manipulation of the record before passing it to the handlers.

</div>

<div class="method">

handle(record)

Handle a record.

This just loops through the handlers offering them the record to handle. The actual object passed to the handlers is that which is returned from `prepare`.

</div>

<div class="method">

start()

Starts the listener.

This starts up a background thread to monitor the queue for LogRecords to process.

<div class="versionchanged">

3.14 Raises `RuntimeError` if called and the listener is already running.

</div>

</div>

<div class="method">

stop()

Stops the listener.

This asks the thread to terminate, and then waits for it to do so. Note that if you don't call this before your application exits, there may be some records still left on the queue, which won't be processed.

</div>

<div class="method">

enqueue_sentinel()

Writes a sentinel to the queue to tell the listener to quit. This implementation uses `put_nowait()`. You may want to override this method if you want to use timeouts or work with custom queue implementations.

<div class="versionadded">

3.3

</div>

</div>

</div>

<div class="seealso">

Module `logging`  
API reference for the logging module.

Module `logging.config`  
Configuration API for the logging module.

</div>

# `!logging` --- Logging facility for Python

<div class="module" data-synopsis="Flexible event logging system for applications.">

logging

</div>

**Source code:** `Lib/logging/__init__.py`

<div class="index">

pair: Errors; logging

</div>

<div class="sidebar">

**Important**

This page contains the API reference information. For tutorial information and discussion of more advanced topics, see

- `Basic Tutorial <logging-basic-tutorial>`
- `Advanced Tutorial <logging-advanced-tutorial>`
- `Logging Cookbook <logging-cookbook>`

</div>

------------------------------------------------------------------------

This module defines functions and classes which implement a flexible event logging system for applications and libraries.

The key benefit of having the logging API provided by a standard library module is that all Python modules can participate in logging, so your application log can include your own messages integrated with messages from third-party modules.

Here's a simple example of idiomatic usage: :

    # myapp.py
    import logging
    import mylib
    logger = logging.getLogger(__name__)

    def main():
        logging.basicConfig(filename='myapp.log', level=logging.INFO)
        logger.info('Started')
        mylib.do_something()
        logger.info('Finished')

    if __name__ == '__main__':
        main()

    # mylib.py
    import logging
    logger = logging.getLogger(__name__)

    def do_something():
        logger.info('Doing something')

If you run *myapp.py*, you should see this in *myapp.log*:

```
INFO:__main__:Started
INFO:mylib:Doing something
INFO:__main__:Finished
```

The key feature of this idiomatic usage is that the majority of code is simply creating a module level logger with `getLogger(__name__)`, and using that logger to do any needed logging. This is concise, while allowing downstream code fine-grained control if needed. Logged messages to the module-level logger get forwarded to handlers of loggers in higher-level modules, all the way up to the highest-level logger known as the root logger; this approach is known as hierarchical logging.

For logging to be useful, it needs to be configured: setting the levels and destinations for each logger, potentially changing how specific modules log, often based on command-line arguments or application configuration. In most cases, like the one above, only the root logger needs to be so configured, since all the lower level loggers at module level eventually forward their messages to its handlers. `~logging.basicConfig` provides a quick way to configure the root logger that handles many use cases.

The module provides a lot of functionality and flexibility. If you are unfamiliar with logging, the best way to get to grips with it is to view the tutorials (**see the links above and on the right**).

The basic classes defined by the module, together with their attributes and methods, are listed in the sections below.

- Loggers expose the interface that application code directly uses.
- Handlers send the log records (created by loggers) to the appropriate destination.
- Filters provide a finer grained facility for determining which log records to output.
- Formatters specify the layout of log records in the final output.

## Logger Objects

Loggers have the following attributes and methods. Note that Loggers should *NEVER* be instantiated directly, but always through the module-level function `logging.getLogger(name)`. Multiple calls to `getLogger` with the same name will always return a reference to the same Logger object.

The `name` is potentially a period-separated hierarchical value, like `foo.bar.baz` (though it could also be just plain `foo`, for example). Loggers that are further down in the hierarchical list are children of loggers higher up in the list. For example, given a logger with a name of `foo`, loggers with names of `foo.bar`, `foo.bar.baz`, and `foo.bam` are all descendants of `foo`. In addition, all loggers are descendants of the root logger. The logger name hierarchy is analogous to the Python package hierarchy, and identical to it if you organise your loggers on a per-module basis using the recommended construction `logging.getLogger(__name__)`. That's because in a module, `__name__` is the module's name in the Python package namespace.

<div class="Logger">

<div class="attribute">

Logger.name

This is the logger's name, and is the value that was passed to `getLogger` to obtain the logger.

> [!NOTE]
> This attribute should be treated as read-only.

</div>

<div class="attribute">

Logger.level

The threshold of this logger, as set by the `setLevel` method.

> [!NOTE]
> Do not set this attribute directly - always use `setLevel`, which has checks for the level passed to it.

</div>

<div class="attribute">

Logger.parent

The parent logger of this logger. It may change based on later instantiation of loggers which are higher up in the namespace hierarchy.

> [!NOTE]
> This value should be treated as read-only.

</div>

<div class="attribute">

Logger.propagate

If this attribute evaluates to true, events logged to this logger will be passed to the handlers of higher level (ancestor) loggers, in addition to any handlers attached to this logger. Messages are passed directly to the ancestor loggers' handlers - neither the level nor filters of the ancestor loggers in question are considered.

If this evaluates to false, logging messages are not passed to the handlers of ancestor loggers.

Spelling it out with an example: If the propagate attribute of the logger named `A.B.C` evaluates to true, any event logged to `A.B.C` via a method call such as `logging.getLogger('A.B.C').error(...)` will \[subject to passing that logger's level and filter settings\] be passed in turn to any handlers attached to loggers named `A.B`, `A` and the root logger, after first being passed to any handlers attached to `A.B.C`. If any logger in the chain `A.B.C`, `A.B`, `A` has its `propagate` attribute set to false, then that is the last logger whose handlers are offered the event to handle, and propagation stops at that point.

The constructor sets this attribute to `True`.

> [!NOTE]
> If you attach a handler to a logger *and* one or more of its ancestors, it may emit the same record multiple times. In general, you should not need to attach a handler to more than one logger - if you just attach it to the appropriate logger which is highest in the logger hierarchy, then it will see all events logged by all descendant loggers, provided that their propagate setting is left set to `True`. A common scenario is to attach handlers only to the root logger, and to let propagation take care of the rest.

</div>

<div class="attribute">

Logger.handlers

The list of handlers directly attached to this logger instance.

> [!NOTE]
> This attribute should be treated as read-only; it is normally changed via the `addHandler` and `removeHandler` methods, which use locks to ensure thread-safe operation.

</div>

<div class="attribute">

Logger.disabled

This attribute disables handling of any events. It is set to `False` in the initializer, and only changed by logging configuration code.

> [!NOTE]
> This attribute should be treated as read-only.

</div>

<div class="method">

Logger.setLevel(level)

Sets the threshold for this logger to *level*. Logging messages which are less severe than *level* will be ignored; logging messages which have severity *level* or higher will be emitted by whichever handler or handlers service this logger, unless a handler's level has been set to a higher severity level than *level*.

When a logger is created, the level is set to `NOTSET` (which causes all messages to be processed when the logger is the root logger, or delegation to the parent when the logger is a non-root logger). Note that the root logger is created with level `WARNING`.

The term 'delegation to the parent' means that if a logger has a level of NOTSET, its chain of ancestor loggers is traversed until either an ancestor with a level other than NOTSET is found, or the root is reached.

If an ancestor is found with a level other than NOTSET, then that ancestor's level is treated as the effective level of the logger where the ancestor search began, and is used to determine how a logging event is handled.

If the root is reached, and it has a level of NOTSET, then all messages will be processed. Otherwise, the root's level will be used as the effective level.

See `levels` for a list of levels.

<div class="versionchanged">

3.2 The *level* parameter now accepts a string representation of the level such as 'INFO' as an alternative to the integer constants such as `INFO`. Note, however, that levels are internally stored as integers, and methods such as e.g. `getEffectiveLevel` and `isEnabledFor` will return/expect to be passed integers.

</div>

</div>

<div class="method">

Logger.isEnabledFor(level)

Indicates if a message of severity *level* would be processed by this logger. This method checks first the module-level level set by `logging.disable(level)` and then the logger's effective level as determined by `getEffectiveLevel`.

</div>

<div class="method">

Logger.getEffectiveLevel()

Indicates the effective level for this logger. If a value other than `NOTSET` has been set using `setLevel`, it is returned. Otherwise, the hierarchy is traversed towards the root until a value other than `NOTSET` is found, and that value is returned. The value returned is an integer, typically one of `logging.DEBUG`, `logging.INFO` etc.

</div>

<div class="method">

Logger.getChild(suffix)

Returns a logger which is a descendant to this logger, as determined by the suffix. Thus, `logging.getLogger('abc').getChild('def.ghi')` would return the same logger as would be returned by `logging.getLogger('abc.def.ghi')`. This is a convenience method, useful when the parent logger is named using e.g. `__name__` rather than a literal string.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

Logger.getChildren()

Returns a set of loggers which are immediate children of this logger. So for example `logging.getLogger().getChildren()` might return a set containing loggers named `foo` and `bar`, but a logger named `foo.bar` wouldn't be included in the set. Likewise, `logging.getLogger('foo').getChildren()` might return a set including a logger named `foo.bar`, but it wouldn't include one named `foo.bar.baz`.

<div class="versionadded">

3.12

</div>

</div>

<div class="method">

Logger.debug(msg, *args,*\*kwargs)

Logs a message with level `DEBUG` on this logger. The *msg* is the message format string, and the *args* are the arguments which are merged into *msg* using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.) No % formatting operation is performed on *msg* when no *args* are supplied.

There are four keyword arguments in *kwargs* which are inspected: *exc_info*, *stack_info*, *stacklevel* and *extra*.

If *exc_info* does not evaluate as false, it causes exception information to be added to the logging message. If an exception tuple (in the format returned by `sys.exc_info`) or an exception instance is provided, it is used; otherwise, `sys.exc_info` is called to get the exception information.

The second optional keyword argument is *stack_info*, which defaults to `False`. If true, stack information is added to the logging message, including the actual logging call. Note that this is not the same stack information as that displayed through specifying *exc_info*: The former is stack frames from the bottom of the stack up to the logging call in the current thread, whereas the latter is information about stack frames which have been unwound, following an exception, while searching for exception handlers.

You can specify *stack_info* independently of *exc_info*, e.g. to just show how you got to a certain point in your code, even when no exceptions were raised. The stack frames are printed following a header line which says:

```
Stack (most recent call last):
```

This mimics the `Traceback (most recent call last):` which is used when displaying exception frames.

The third optional keyword argument is *stacklevel*, which defaults to `1`. If greater than 1, the corresponding number of stack frames are skipped when computing the line number and function name set in the `LogRecord` created for the logging event. This can be used in logging helpers so that the function name, filename and line number recorded are not the information for the helper function/method, but rather its caller. The name of this parameter mirrors the equivalent one in the `warnings` module.

The fourth keyword argument is *extra* which can be used to pass a dictionary which is used to populate the `~object.__dict__` of the `LogRecord` created for the logging event with user-defined attributes. These custom attributes can then be used as you like. For example, they could be incorporated into logged messages. For example:

    FORMAT = '%(asctime)s %(clientip)-15s %(user)-8s %(message)s'
    logging.basicConfig(format=FORMAT)
    d = {'clientip': '192.168.0.1', 'user': 'fbloggs'}
    logger = logging.getLogger('tcpserver')
    logger.warning('Protocol problem: %s', 'connection reset', extra=d)

would print something like

```
2006-02-08 22:20:02,165 192.168.0.1 fbloggs  Protocol problem: connection reset
```

The keys in the dictionary passed in *extra* should not clash with the keys used by the logging system. (See the section on `logrecord-attributes` for more information on which keys are used by the logging system.)

If you choose to use these attributes in logged messages, you need to exercise some care. In the above example, for instance, the `Formatter` has been set up with a format string which expects 'clientip' and 'user' in the attribute dictionary of the `LogRecord`. If these are missing, the message will not be logged because a string formatting exception will occur. So in this case, you always need to pass the *extra* dictionary with these keys.

While this might be annoying, this feature is intended for use in specialized circumstances, such as multi-threaded servers where the same code executes in many contexts, and interesting conditions which arise are dependent on this context (such as remote client IP address and authenticated user name, in the above example). In such circumstances, it is likely that specialized `Formatter`s would be used with particular `Handler`s.

If no handler is attached to this logger (or any of its ancestors, taking into account the relevant `Logger.propagate` attributes), the message will be sent to the handler set on `lastResort`.

<div class="versionchanged">

3.2 The *stack_info* parameter was added.

</div>

<div class="versionchanged">

3.5 The *exc_info* parameter can now accept exception instances.

</div>

<div class="versionchanged">

3.8 The *stacklevel* parameter was added.

</div>

</div>

<div class="method">

Logger.info(msg, *args,*\*kwargs)

Logs a message with level `INFO` on this logger. The arguments are interpreted as for `debug`.

</div>

<div class="method">

Logger.warning(msg, *args,*\*kwargs)

Logs a message with level `WARNING` on this logger. The arguments are interpreted as for `debug`.

> [!NOTE]
> There is an obsolete method `warn` which is functionally identical to `warning`. As `warn` is deprecated, please do not use it - use `warning` instead.

</div>

<div class="method">

Logger.error(msg, *args,*\*kwargs)

Logs a message with level `ERROR` on this logger. The arguments are interpreted as for `debug`.

</div>

<div class="method">

Logger.critical(msg, *args,*\*kwargs)

Logs a message with level `CRITICAL` on this logger. The arguments are interpreted as for `debug`.

</div>

<div class="method">

Logger.log(level, msg, *args,*\*kwargs)

Logs a message with integer level *level* on this logger. The other arguments are interpreted as for `debug`.

</div>

<div class="method">

Logger.exception(msg, *args,*\*kwargs)

Logs a message with level `ERROR` on this logger. The arguments are interpreted as for `debug`. Exception info is added to the logging message. This method should only be called from an exception handler.

</div>

<div class="method">

Logger.addFilter(filter)

Adds the specified filter *filter* to this logger.

</div>

<div class="method">

Logger.removeFilter(filter)

Removes the specified filter *filter* from this logger.

</div>

<div class="method">

Logger.filter(record)

Apply this logger's filters to the record and return `True` if the record is to be processed. The filters are consulted in turn, until one of them returns a false value. If none of them return a false value, the record will be processed (passed to handlers). If one returns a false value, no further processing of the record occurs.

</div>

<div class="method">

Logger.addHandler(hdlr)

Adds the specified handler *hdlr* to this logger.

</div>

<div class="method">

Logger.removeHandler(hdlr)

Removes the specified handler *hdlr* from this logger.

</div>

<div class="method">

Logger.findCaller(stack_info=False, stacklevel=1)

Finds the caller's source filename and line number. Returns the filename, line number, function name and stack information as a 4-element tuple. The stack information is returned as `None` unless *stack_info* is `True`.

The *stacklevel* parameter is passed from code calling the `debug` and other APIs. If greater than 1, the excess is used to skip stack frames before determining the values to be returned. This will generally be useful when calling logging APIs from helper/wrapper code, so that the information in the event log refers not to the helper/wrapper code, but to the code that calls it.

</div>

<div class="method">

Logger.handle(record)

Handles a record by passing it to all handlers associated with this logger and its ancestors (until a false value of *propagate* is found). This method is used for unpickled records received from a socket, as well as those created locally. Logger-level filtering is applied using `~Logger.filter`.

</div>

<div class="method">

Logger.makeRecord(name, level, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None)

This is a factory method which can be overridden in subclasses to create specialized `LogRecord` instances.

</div>

<div class="method">

Logger.hasHandlers()

Checks to see if this logger has any handlers configured. This is done by looking for handlers in this logger and its parents in the logger hierarchy. Returns `True` if a handler was found, else `False`. The method stops searching up the hierarchy whenever a logger with the 'propagate' attribute set to false is found - that will be the last logger which is checked for the existence of handlers.

<div class="versionadded">

3.2

</div>

</div>

<div class="versionchanged">

3.7 Loggers can now be pickled and unpickled.

</div>

</div>

## Logging Levels

The numeric values of logging levels are given in the following table. These are primarily of interest if you want to define your own levels, and need them to have specific values relative to the predefined levels. If you define a level with the same numeric value, it overwrites the predefined value; the predefined name is lost.

| Level | Numeric value | What it means / When to use it |
|----|----|----|
|  | 0 | When set on a logger, indicates that ancestor loggers are to be consulted to determine the effective level. If that still resolves to `!NOTSET`, then all events are logged. When set on a handler, all events are handled. |
|  | 10 | Detailed information, typically only of interest to a developer trying to diagnose a problem. |
|  | 20 | Confirmation that things are working as expected. |
|  | 30 | An indication that something unexpected happened, or that a problem might occur in the near future (e.g. 'disk space low'). The software is still working as expected. |
|  | 40 | Due to a more serious problem, the software has not been able to perform some function. |
|  | 50 | A serious error, indicating that the program itself may be unable to continue running. |

## Handler Objects

Handlers have the following attributes and methods. Note that `Handler` is never instantiated directly; this class acts as a base for more useful subclasses. However, the `!__init__` method in subclasses needs to call `Handler.__init__`.

<div class="Handler">

<div class="method">

Handler.\_\_init\_\_(level=NOTSET)

Initializes the `Handler` instance by setting its level, setting the list of filters to the empty list and creating a lock (using `createLock`) for serializing access to an I/O mechanism.

</div>

<div class="method">

Handler.createLock()

Initializes a thread lock which can be used to serialize access to underlying I/O functionality which may not be threadsafe.

</div>

<div class="method">

Handler.acquire()

Acquires the thread lock created with `createLock`.

</div>

<div class="method">

Handler.release()

Releases the thread lock acquired with `acquire`.

</div>

<div class="method">

Handler.setLevel(level)

Sets the threshold for this handler to *level*. Logging messages which are less severe than *level* will be ignored. When a handler is created, the level is set to `NOTSET` (which causes all messages to be processed).

See `levels` for a list of levels.

<div class="versionchanged">

3.2 The *level* parameter now accepts a string representation of the level such as 'INFO' as an alternative to the integer constants such as `INFO`.

</div>

</div>

<div class="method">

Handler.setFormatter(fmt)

Sets the formatter for this handler to *fmt*. The *fmt* argument must be a `Formatter` instance or `None`.

</div>

<div class="method">

Handler.addFilter(filter)

Adds the specified filter *filter* to this handler.

</div>

<div class="method">

Handler.removeFilter(filter)

Removes the specified filter *filter* from this handler.

</div>

<div class="method">

Handler.filter(record)

Apply this handler's filters to the record and return `True` if the record is to be processed. The filters are consulted in turn, until one of them returns a false value. If none of them return a false value, the record will be emitted. If one returns a false value, the handler will not emit the record.

</div>

<div class="method">

Handler.flush()

Ensure all logging output has been flushed. This version does nothing and is intended to be implemented by subclasses.

</div>

<div class="method">

Handler.close()

Tidy up any resources used by the handler. This version does no output but removes the handler from an internal map of handlers, which is used for handler lookup by name.

Subclasses should ensure that this gets called from overridden `close` methods.

</div>

<div class="method">

Handler.handle(record)

Conditionally emits the specified logging record, depending on filters which may have been added to the handler. Wraps the actual emission of the record with acquisition/release of the I/O thread lock.

</div>

<div class="method">

Handler.handleError(record)

This method should be called from handlers when an exception is encountered during an `emit` call. If the module-level attribute `raiseExceptions` is `False`, exceptions get silently ignored. This is what is mostly wanted for a logging system - most users will not care about errors in the logging system, they are more interested in application errors. You could, however, replace this with a custom handler if you wish. The specified record is the one which was being processed when the exception occurred. (The default value of `raiseExceptions` is `True`, as that is more useful during development).

</div>

<div class="method">

Handler.format(record)

Do formatting for a record - if a formatter is set, use it. Otherwise, use the default formatter for the module.

</div>

<div class="method">

Handler.emit(record)

Do whatever it takes to actually log the specified logging record. This version is intended to be implemented by subclasses and so raises a `NotImplementedError`.

> [!WARNING]
> This method is called after a handler-level lock is acquired, which is released after this method returns. When you override this method, note that you should be careful when calling anything that invokes other parts of the logging API which might do locking, because that might result in a deadlock. Specifically:
>
> - Logging configuration APIs acquire the module-level lock, and then individual handler-level locks as those handlers are configured.
> - Many logging APIs lock the module-level lock. If such an API is called from this method, it could cause a deadlock if a configuration call is made on another thread, because that thread will try to acquire the module-level lock *before* the handler-level lock, whereas this thread tries to acquire the module-level lock *after* the handler-level lock (because in this method, the handler-level lock has already been acquired).

</div>

</div>

For a list of handlers included as standard, see `logging.handlers`.

## Formatter Objects

<div class="currentmodule">

logging

</div>

<div class="Formatter(fmt=None, datefmt=None, style='%', validate=True, *, defaults=None)">

Responsible for converting a `LogRecord` to an output string to be interpreted by a human or external system.

param fmt  
A format string in the given *style* for the logged output as a whole. The possible mapping keys are drawn from the `LogRecord` object's `logrecord-attributes`. If not specified, `'%(message)s'` is used, which is just the logged message.

type fmt  
str

param datefmt  
A format string for the date/time portion of the logged output. If not specified, the default described in `formatTime` is used.

type datefmt  
str

param style  
Can be one of `'%'`, `'{'` or `'$'` and determines how the format string will be merged with its data: using one of `old-string-formatting` (`%`), `str.format` (`{`) or `string.Template` (`$`). This only applies to *fmt* (e.g. `'%(message)s'` versus `'{message}'`), not to the actual log messages passed to the logging methods. However, there are `other ways <formatting-styles>` to use `{`- and `$`-formatting for log messages.

type style  
str

param validate  
If `True` (the default), incorrect or mismatched *fmt* and *style* will raise a `ValueError`; for example, `logging.Formatter('%(asctime)s - %(message)s', style='{')`.

type validate  
bool

param defaults  
A dictionary with default values to use in custom fields. For example, `logging.Formatter('%(ip)s %(message)s', defaults={"ip": None})`

type defaults  
dict\[str, typing.Any\]

<div class="versionchanged">

3.2 Added the *style* parameter.

</div>

<div class="versionchanged">

3.8 Added the *validate* parameter.

</div>

<div class="versionchanged">

3.10 Added the *defaults* parameter.

</div>

<div class="method">

format(record)

The record's attribute dictionary is used as the operand to a string formatting operation. Returns the resulting string. Before formatting the dictionary, a couple of preparatory steps are carried out. The *message* attribute of the record is computed using *msg* % *args*. If the formatting string contains `'(asctime)'`, `formatTime` is called to format the event time. If there is exception information, it is formatted using `formatException` and appended to the message. Note that the formatted exception information is cached in attribute *exc_text*. This is useful because the exception information can be pickled and sent across the wire, but you should be careful if you have more than one `Formatter` subclass which customizes the formatting of exception information. In this case, you will have to clear the cached value (by setting the *exc_text* attribute to `None`) after a formatter has done its formatting, so that the next formatter to handle the event doesn't use the cached value, but recalculates it afresh.

If stack information is available, it's appended after the exception information, using `formatStack` to transform it if necessary.

</div>

<div class="method">

formatTime(record, datefmt=None)

This method should be called from `format` by a formatter which wants to make use of a formatted time. This method can be overridden in formatters to provide for any specific requirement, but the basic behavior is as follows: if *datefmt* (a string) is specified, it is used with `time.strftime` to format the creation time of the record. Otherwise, the format '%Y-%m-%d %H:%M:%S,uuu' is used, where the uuu part is a millisecond value and the other letters are as per the `time.strftime` documentation. An example time in this format is `2003-01-23 00:29:50,411`. The resulting string is returned.

This function uses a user-configurable function to convert the creation time to a tuple. By default, `time.localtime` is used; to change this for a particular formatter instance, set the `converter` attribute to a function with the same signature as `time.localtime` or `time.gmtime`. To change it for all formatters, for example if you want all logging times to be shown in GMT, set the `converter` attribute in the `Formatter` class.

<div class="versionchanged">

3.3 Previously, the default format was hard-coded as in this example: `2010-09-06 22:38:15,292` where the part before the comma is handled by a strptime format string (`'%Y-%m-%d %H:%M:%S'`), and the part after the comma is a millisecond value. Because strptime does not have a format placeholder for milliseconds, the millisecond value is appended using another format string, `'%s,%03d'` --- and both of these format strings have been hardcoded into this method. With the change, these strings are defined as class-level attributes which can be overridden at the instance level when desired. The names of the attributes are `default_time_format` (for the strptime format string) and `default_msec_format` (for appending the millisecond value).

</div>

<div class="versionchanged">

3.9 The `default_msec_format` can be `None`.

</div>

</div>

<div class="method">

formatException(exc_info)

Formats the specified exception information (a standard exception tuple as returned by `sys.exc_info`) as a string. This default implementation just uses `traceback.print_exception`. The resulting string is returned.

</div>

<div class="method">

formatStack(stack_info)

Formats the specified stack information (a string as returned by `traceback.print_stack`, but with the last newline removed) as a string. This default implementation just returns the input value.

</div>

</div>

<div class="BufferingFormatter(linefmt=None)">

A base formatter class suitable for subclassing when you want to format a number of records. You can pass a `Formatter` instance which you want to use to format each line (that corresponds to a single record). If not specified, the default formatter (which just outputs the event message) is used as the line formatter.

<div class="method">

formatHeader(records)

Return a header for a list of *records*. The base implementation just returns the empty string. You will need to override this method if you want specific behaviour, e.g. to show the count of records, a title or a separator line.

</div>

<div class="method">

formatFooter(records)

Return a footer for a list of *records*. The base implementation just returns the empty string. You will need to override this method if you want specific behaviour, e.g. to show the count of records or a separator line.

</div>

<div class="method">

format(records)

Return formatted text for a list of *records*. The base implementation just returns the empty string if there are no records; otherwise, it returns the concatenation of the header, each record formatted with the line formatter, and the footer.

</div>

</div>

## Filter Objects

`Filters` can be used by `Handlers` and `Loggers` for more sophisticated filtering than is provided by levels. The base filter class only allows events which are below a certain point in the logger hierarchy. For example, a filter initialized with 'A.B' will allow events logged by loggers 'A.B', 'A.B.C', 'A.B.C.D', 'A.B.D' etc. but not 'A.BB', 'B.A.B' etc. If initialized with the empty string, all events are passed.

<div class="Filter(name='')">

Returns an instance of the `Filter` class. If *name* is specified, it names a logger which, together with its children, will have its events allowed through the filter. If *name* is the empty string, allows every event.

<div class="method">

filter(record)

Is the specified record to be logged? Returns false for no, true for yes. Filters can either modify log records in-place or return a completely different record instance which will replace the original log record in any future processing of the event.

</div>

</div>

Note that filters attached to handlers are consulted before an event is emitted by the handler, whereas filters attached to loggers are consulted whenever an event is logged (using `debug`, `info`, etc.), before sending an event to handlers. This means that events which have been generated by descendant loggers will not be filtered by a logger's filter setting, unless the filter has also been applied to those descendant loggers.

You don't actually need to subclass `Filter`: you can pass any instance which has a `filter` method with the same semantics.

<div class="versionchanged">

3.2 You don't need to create specialized `Filter` classes, or use other classes with a `filter` method: you can use a function (or other callable) as a filter. The filtering logic will check to see if the filter object has a `filter` attribute: if it does, it's assumed to be a `Filter` and its `~Filter.filter` method is called. Otherwise, it's assumed to be a callable and called with the record as the single parameter. The returned value should conform to that returned by `~Filter.filter`.

</div>

<div class="versionchanged">

3.12 You can now return a `LogRecord` instance from filters to replace the log record rather than modifying it in place. This allows filters attached to a `Handler` to modify the log record before it is emitted, without having side effects on other handlers.

</div>

Although filters are used primarily to filter records based on more sophisticated criteria than levels, they get to see every record which is processed by the handler or logger they're attached to: this can be useful if you want to do things like counting how many records were processed by a particular logger or handler, or adding, changing or removing attributes in the `LogRecord` being processed. Obviously changing the LogRecord needs to be done with some care, but it does allow the injection of contextual information into logs (see `filters-contextual`).

## LogRecord Objects

`LogRecord` instances are created automatically by the `Logger` every time something is logged, and can be created manually via `makeLogRecord` (for example, from a pickled event received over the wire).

<div class="LogRecord(name, level, pathname, lineno, msg, args, exc_info, func=None, sinfo=None)">

Contains all the information pertinent to the event being logged.

The primary information is passed in *msg* and *args*, which are combined using `msg % args` to create the `!message` attribute of the record.

param name  
The name of the logger used to log the event represented by this `!LogRecord`. Note that the logger name in the `!LogRecord` will always have this value, even though it may be emitted by a handler attached to a different (ancestor) logger.

type name  
str

param level  
The `numeric level <levels>` of the logging event (such as `10` for `DEBUG`, `20` for `INFO`, etc). Note that this is converted to *two* attributes of the LogRecord: `!levelno` for the numeric value and `!levelname` for the corresponding level name.

type level  
int

param pathname  
The full string path of the source file where the logging call was made.

type pathname  
str

param lineno  
The line number in the source file where the logging call was made.

type lineno  
int

param msg  
The event description message, which can be a %-format string with placeholders for variable data, or an arbitrary object (see `arbitrary-object-messages`).

type msg  
typing.Any

param args  
Variable data to merge into the *msg* argument to obtain the event description.

type args  
tuple \| dict\[str, typing.Any\]

param exc_info  
An exception tuple with the current exception information, as returned by `sys.exc_info`, or `None` if no exception information is available.

type exc_info  
tuple\[type\[BaseException\], BaseException, types.TracebackType\] \| None

param func  
The name of the function or method from which the logging call was invoked.

type func  
str \| None

param sinfo  
A text string representing stack information from the base of the stack in the current thread, up to the logging call.

type sinfo  
str \| None

<div class="method">

getMessage()

Returns the message for this `LogRecord` instance after merging any user-supplied arguments with the message. If the user-supplied message argument to the logging call is not a string, `str` is called on it to convert it to a string. This allows use of user-defined classes as messages, whose `__str__` method can return the actual format string to be used.

</div>

<div class="versionchanged">

3.2 The creation of a `LogRecord` has been made more configurable by providing a factory which is used to create the record. The factory can be set using `getLogRecordFactory` and `setLogRecordFactory` (see this for the factory's signature).

</div>

This functionality can be used to inject your own values into a `LogRecord` at creation time. You can use the following pattern:

    old_factory = logging.getLogRecordFactory()

    def record_factory(*args, **kwargs):
        record = old_factory(*args, **kwargs)
        record.custom_attribute = 0xdecafbad
        return record

    logging.setLogRecordFactory(record_factory)

With this pattern, multiple factories could be chained, and as long as they don't overwrite each other's attributes or unintentionally overwrite the standard attributes listed above, there should be no surprises.

</div>

## LogRecord attributes

The LogRecord has a number of attributes, most of which are derived from the parameters to the constructor. (Note that the names do not always correspond exactly between the LogRecord constructor parameters and the LogRecord attributes.) These attributes can be used to merge data from the record into the format string. The following table lists (in alphabetical order) the attribute names, their meanings and the corresponding placeholder in a %-style format string.

If you are using {}-formatting (`str.format`), you can use `{attrname}` as the placeholder in the format string. If you are using \$-formatting (`string.Template`), use the form `${attrname}`. In both cases, of course, replace `attrname` with the actual attribute name you want to use.

In the case of {}-formatting, you can specify formatting flags by placing them after the attribute name, separated from it with a colon. For example: a placeholder of `{msecs:03.0f}` would format a millisecond value of `4` as `004`. Refer to the `str.format` documentation for full details on the options available to you.

| Attribute name | Format | Description |
|----|----|----|
| args | You shouldn't need to format this yourself. | The tuple of arguments merged into `msg` to produce `message`, or a dict whose values are used for the merge (when there is only one argument, and it is a dictionary). |
| asctime | `%(asctime)s` | Human-readable time when the `LogRecord` was created. By default this is of the form '2003-07-08 16:49:45,896' (the numbers after the comma are millisecond portion of the time). |
| created | `%(created)f` | Time when the `LogRecord` was created (as returned by `time.time_ns` / 1e9). |
| exc_info | You shouldn't need to format this yourself. | Exception tuple (à la `sys.exc_info`) or, if no exception has occurred, `None`. |
| exc_text | You shouldn't need to format this yourself. | Exception information formatted as a string. This is set when `Formatter.format` is invoked, or `None` if no exception has occurred. |
| filename | `%(filename)s` | Filename portion of `pathname`. |
| funcName | `%(funcName)s` | Name of function containing the logging call. |
| levelname | `%(levelname)s` | Text logging level for the message (`'DEBUG'`, `'INFO'`, `'WARNING'`, `'ERROR'`, `'CRITICAL'`). |
| levelno | `%(levelno)s` | Numeric logging level for the message (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`). |
| lineno | `%(lineno)d` | Source line number where the logging call was issued (if available). |
| message | `%(message)s` | The logged message, computed as `msg % args`. This is set when `Formatter.format` is invoked. |
| module | `%(module)s` | Module (name portion of `filename`). |
| msecs | `%(msecs)d` | Millisecond portion of the time when the `LogRecord` was created. |
| msg | You shouldn't need to format this yourself. | The format string passed in the original logging call. Merged with `args` to produce `message`, or an arbitrary object (see `arbitrary-object-messages`). |
| name | `%(name)s` | Name of the logger used to log the call. |
| pathname | `%(pathname)s` | Full pathname of the source file where the logging call was issued (if available). |
| process | `%(process)d` | Process ID (if available). |
| processName | `%(processName)s` | Process name (if available). |
| relativeCreated | `%(relativeCreated)d` | Time in milliseconds when the LogRecord was created, relative to the time the logging module was loaded. |
| stack_info | You shouldn't need to format this yourself. | Stack frame information (where available) from the bottom of the stack in the current thread, up to and including the stack frame of the logging call which resulted in the creation of this record. |
| thread | `%(thread)d` | Thread ID (if available). |
| threadName | `%(threadName)s` | Thread name (if available). |
| taskName | `%(taskName)s` | `asyncio.Task` name (if available). |

<div class="versionchanged">

3.1 *processName* was added.

</div>

<div class="versionchanged">

3.12 *taskName* was added.

</div>

## LoggerAdapter Objects

`LoggerAdapter` instances are used to conveniently pass contextual information into logging calls. For a usage example, see the section on `adding contextual information to your logging output <context-info>`.

<div class="LoggerAdapter(logger, extra=None, merge_extra=False)">

Returns an instance of `LoggerAdapter` initialized with an underlying `Logger` instance, an optional dict-like object (*extra*), and an optional boolean (*merge_extra*) indicating whether or not the *extra* argument of individual log calls should be merged with the `LoggerAdapter` extra. The default behavior is to ignore the *extra* argument of individual log calls and only use the one of the `LoggerAdapter` instance

<div class="method">

process(msg, kwargs)

Modifies the message and/or keyword arguments passed to a logging call in order to insert contextual information. This implementation takes the object passed as *extra* to the constructor and adds it to *kwargs* using key 'extra'. The return value is a (*msg*, *kwargs*) tuple which has the (possibly modified) versions of the arguments passed in.

</div>

<div class="attribute">

manager

Delegates to the underlying `!manager` on *logger*.

</div>

<div class="attribute">

<span id="log">log</span>

Delegates to the underlying `!_log` method on *logger*.

</div>

In addition to the above, `LoggerAdapter` supports the following methods of `Logger`: `~Logger.debug`, `~Logger.info`, `~Logger.warning`, `~Logger.error`, `~Logger.exception`, `~Logger.critical`, `~Logger.log`, `~Logger.isEnabledFor`, `~Logger.getEffectiveLevel`, `~Logger.setLevel` and `~Logger.hasHandlers`. These methods have the same signatures as their counterparts in `Logger`, so you can use the two types of instances interchangeably.

<div class="versionchanged">

3.2

The `~Logger.isEnabledFor`, `~Logger.getEffectiveLevel`, `~Logger.setLevel` and `~Logger.hasHandlers` methods were added to `LoggerAdapter`. These methods delegate to the underlying logger.

</div>

<div class="versionchanged">

3.6

Attribute `!manager` and method `!_log` were added, which delegate to the underlying logger and allow adapters to be nested.

</div>

<div class="versionchanged">

3.10

The *extra* argument is now optional.

</div>

<div class="versionchanged">

3.13

The *merge_extra* parameter was added.

</div>

</div>

## Thread Safety

The logging module is intended to be thread-safe without any special work needing to be done by its clients. It achieves this through using threading locks; there is one lock to serialize access to the module's shared data, and each handler also creates a lock to serialize access to its underlying I/O.

If you are implementing asynchronous signal handlers using the `signal` module, you may not be able to use logging from within such handlers. This is because lock implementations in the `threading` module are not always re-entrant, and so cannot be invoked from such signal handlers.

## Module-Level Functions

In addition to the classes described above, there are a number of module-level functions.

<div class="function">

getLogger(name=None)

Return a logger with the specified name or, if name is `None`, return the root logger of the hierarchy. If specified, the name is typically a dot-separated hierarchical name like *'a'*, *'a.b'* or *'a.b.c.d'*. Choice of these names is entirely up to the developer who is using logging, though it is recommended that `__name__` be used unless you have a specific reason for not doing that, as mentioned in `logger`.

All calls to this function with a given name return the same logger instance. This means that logger instances never need to be passed between different parts of an application.

</div>

<div class="function">

getLoggerClass()

Return either the standard `Logger` class, or the last class passed to `setLoggerClass`. This function may be called from within a new class definition, to ensure that installing a customized `Logger` class will not undo customizations already applied by other code. For example:

    class MyLogger(logging.getLoggerClass()):
        # ... override behaviour here

</div>

<div class="function">

getLogRecordFactory()

Return a callable which is used to create a `LogRecord`.

<div class="versionadded">

3.2 This function has been provided, along with `setLogRecordFactory`, to allow developers more control over how the `LogRecord` representing a logging event is constructed.

</div>

See `setLogRecordFactory` for more information about the how the factory is called.

</div>

<div class="function">

debug(msg, *args,*\*kwargs)

This is a convenience function that calls `Logger.debug`, on the root logger. The handling of the arguments is in every way identical to what is described in that method.

The only difference is that if the root logger has no handlers, then `basicConfig` is called, prior to calling `debug` on the root logger.

For very short scripts or quick demonstrations of `logging` facilities, `debug` and the other module-level functions may be convenient. However, most programs will want to carefully and explicitly control the logging configuration, and should therefore prefer creating a module-level logger and calling `Logger.debug` (or other level-specific methods) on it, as described at the beginning of this documentation.

</div>

<div class="function">

info(msg, *args,*\*kwargs)

Logs a message with level `INFO` on the root logger. The arguments and behavior are otherwise the same as for `debug`.

</div>

<div class="function">

warning(msg, *args,*\*kwargs)

Logs a message with level `WARNING` on the root logger. The arguments and behavior are otherwise the same as for `debug`.

> [!NOTE]
> There is an obsolete function `warn` which is functionally identical to `warning`. As `warn` is deprecated, please do not use it - use `warning` instead.

</div>

<div class="function">

error(msg, *args,*\*kwargs)

Logs a message with level `ERROR` on the root logger. The arguments and behavior are otherwise the same as for `debug`.

</div>

<div class="function">

critical(msg, *args,*\*kwargs)

Logs a message with level `CRITICAL` on the root logger. The arguments and behavior are otherwise the same as for `debug`.

</div>

<div class="function">

exception(msg, *args,*\*kwargs)

Logs a message with level `ERROR` on the root logger. The arguments and behavior are otherwise the same as for `debug`. Exception info is added to the logging message. This function should only be called from an exception handler.

</div>

<div class="function">

log(level, msg, *args,*\*kwargs)

Logs a message with level *level* on the root logger. The arguments and behavior are otherwise the same as for `debug`.

</div>

<div class="function">

disable(level=CRITICAL)

Provides an overriding level *level* for all loggers which takes precedence over the logger's own level. When the need arises to temporarily throttle logging output down across the whole application, this function can be useful. Its effect is to disable all logging calls of severity *level* and below, so that if you call it with a value of INFO, then all INFO and DEBUG events would be discarded, whereas those of severity WARNING and above would be processed according to the logger's effective level. If `logging.disable(logging.NOTSET)` is called, it effectively removes this overriding level, so that logging output again depends on the effective levels of individual loggers.

Note that if you have defined any custom logging level higher than `CRITICAL` (this is not recommended), you won't be able to rely on the default value for the *level* parameter, but will have to explicitly supply a suitable value.

<div class="versionchanged">

3.7 The *level* parameter was defaulted to level `CRITICAL`. See `28524` for more information about this change.

</div>

</div>

<div class="function">

addLevelName(level, levelName)

Associates level *level* with text *levelName* in an internal dictionary, which is used to map numeric levels to a textual representation, for example when a `Formatter` formats a message. This function can also be used to define your own levels. The only constraints are that all levels used must be registered using this function, levels should be positive integers and they should increase in increasing order of severity.

> [!NOTE]
> If you are thinking of defining your own levels, please see the section on `custom-levels`.

</div>

<div class="function">

getLevelNamesMapping()

Returns a mapping from level names to their corresponding logging levels. For example, the string "CRITICAL" maps to `CRITICAL`. The returned mapping is copied from an internal mapping on each call to this function.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

getLevelName(level)

Returns the textual or numeric representation of logging level *level*.

If *level* is one of the predefined levels `CRITICAL`, `ERROR`, `WARNING`, `INFO` or `DEBUG` then you get the corresponding string. If you have associated levels with names using `addLevelName` then the name you have associated with *level* is returned. If a numeric value corresponding to one of the defined levels is passed in, the corresponding string representation is returned.

The *level* parameter also accepts a string representation of the level such as 'INFO'. In such cases, this functions returns the corresponding numeric value of the level.

If no matching numeric or string value is passed in, the string 'Level %s' % level is returned.

> [!NOTE]
> Levels are internally integers (as they need to be compared in the logging logic). This function is used to convert between an integer level and the level name displayed in the formatted log output by means of the `%(levelname)s` format specifier (see `logrecord-attributes`), and vice versa.

<div class="versionchanged">

3.4 In Python versions earlier than 3.4, this function could also be passed a text level, and would return the corresponding numeric value of the level. This undocumented behaviour was considered a mistake, and was removed in Python 3.4, but reinstated in 3.4.2 in order to retain backward compatibility.

</div>

</div>

<div class="function">

getHandlerByName(name)

Returns a handler with the specified *name*, or `None` if there is no handler with that name.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

getHandlerNames()

Returns an immutable set of all known handler names.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

makeLogRecord(attrdict)

Creates and returns a new `LogRecord` instance whose attributes are defined by *attrdict*. This function is useful for taking a pickled `LogRecord` attribute dictionary, sent over a socket, and reconstituting it as a `LogRecord` instance at the receiving end.

</div>

<div class="function">

basicConfig(\*\*kwargs)

Does basic configuration for the logging system by either creating a `StreamHandler` with a default `Formatter` or using the given *formatter* instance, and adding it to the root logger. The functions `debug`, `info`, `warning`, `error` and `critical` will call `basicConfig` automatically if no handlers are defined for the root logger.

This function does nothing if the root logger already has handlers configured, unless the keyword argument *force* is set to `True`.

> [!NOTE]
> This function should be called from the main thread before other threads are started. In versions of Python prior to 2.7.1 and 3.2, if this function is called from multiple threads, it is possible (in rare circumstances) that a handler will be added to the root logger more than once, leading to unexpected results such as messages being duplicated in the log.

The following keyword arguments are supported.

<div class="tabularcolumns">

\|l\|L\|

</div>

| Format | Description |
|----|----|
| *filename* | Specifies that a `FileHandler` be created, using the specified filename, rather than a `StreamHandler`. |
| *filemode* | If *filename* is specified, open the file in this `mode <filemodes>`. Defaults to `'a'`. |
| *format* | Use the specified format string for the handler. Defaults to attributes `levelname`, `name` and `message` separated by colons. |
| *datefmt* | Use the specified date/time format, as accepted by `time.strftime`. |
| *style* | If *format* is specified, use this style for the format string. One of `'%'`, `'{'` or `'$'` for `printf-style
<old-string-formatting>`, `str.format` or `string.Template` respectively. Defaults to `'%'`. |
| *level* | Set the root logger level to the specified `level <levels>`. |
| *stream* | Use the specified stream to initialize the `StreamHandler`. Note that this argument is incompatible with *filename* - if both are present, a `ValueError` is raised. |
| *handlers* | If specified, this should be an iterable of already created handlers to add to the root logger. Any handlers which don't already have a formatter set will be assigned the default formatter created in this function. Note that this argument is incompatible with *filename* or *stream* - if both are present, a `ValueError` is raised. |
| *force* | If this keyword argument is specified as true, any existing handlers attached to the root logger are removed and closed, before carrying out the configuration as specified by the other arguments. |
| *encoding* | If this keyword argument is specified along with *filename*, its value is used when the `FileHandler` is created, and thus used when opening the output file. |
| *errors* | If this keyword argument is specified along with *filename*, its value is used when the `FileHandler` is created, and thus used when opening the output file. If not specified, the value 'backslashreplace' is used. Note that if `None` is specified, it will be passed as such to `open`, which means that it will be treated the same as passing 'errors'. |
| *formatter* | If specified, set this formatter instance (see `formatter-objects`) for all involved handlers. If not specified, the default is to create and use an instance of `logging.Formatter` based on arguments *format*, *datefmt* and *style*. When *formatter* is specified together with any of the three arguments *format*, *datefmt* and *style*, a `ValueError` is raised to signal that these arguments would lose meaning otherwise. |

<div class="versionchanged">

3.2 The *style* argument was added.

</div>

<div class="versionchanged">

3.3 The *handlers* argument was added. Additional checks were added to catch situations where incompatible arguments are specified (e.g. *handlers* together with *stream* or *filename*, or *stream* together with *filename*).

</div>

<div class="versionchanged">

3.8 The *force* argument was added.

</div>

<div class="versionchanged">

3.9 The *encoding* and *errors* arguments were added.

</div>

<div class="versionchanged">

3.15 The *formatter* argument was added.

</div>

</div>

<div class="function">

shutdown()

Informs the logging system to perform an orderly shutdown by flushing and closing all handlers. This should be called at application exit and no further use of the logging system should be made after this call.

When the logging module is imported, it registers this function as an exit handler (see `atexit`), so normally there's no need to do that manually.

</div>

<div class="function">

setLoggerClass(klass)

Tells the logging system to use the class *klass* when instantiating a logger. The class should define `!__init__` such that only a name argument is required, and the `!__init__` should call `!Logger.__init__`. This function is typically called before any loggers are instantiated by applications which need to use custom logger behavior. After this call, as at any other time, do not instantiate loggers directly using the subclass: continue to use the `logging.getLogger` API to get your loggers.

</div>

<div class="function">

setLogRecordFactory(factory)

Set a callable which is used to create a `LogRecord`.

param factory  
The factory callable to be used to instantiate a log record.

<div class="versionadded">

3.2 This function has been provided, along with `getLogRecordFactory`, to allow developers more control over how the `LogRecord` representing a logging event is constructed.

</div>

The factory has the following signature:

`factory(name, level, fn, lno, msg, args, exc_info, func=None, sinfo=None, **kwargs)`

> name  
> The logger name.
>
> level  
> The logging level (numeric).
>
> fn  
> The full pathname of the file where the logging call was made.
>
> lno  
> The line number in the file where the logging call was made.
>
> msg  
> The logging message.
>
> args  
> The arguments for the logging message.
>
> exc_info  
> An exception tuple, or `None`.
>
> func  
> The name of the function or method which invoked the logging call.
>
> sinfo  
> A stack traceback such as is provided by `traceback.print_stack`, showing the call hierarchy.
>
> kwargs  
> Additional keyword arguments.

</div>

## Module-Level Attributes

<div class="data">

lastResort

A "handler of last resort" is available through this attribute. This is a `StreamHandler` writing to `sys.stderr` with a level of `WARNING`, and is used to handle logging events in the absence of any logging configuration. The end result is to just print the message to `sys.stderr`. This replaces the earlier error message saying that "no handlers could be found for logger XYZ". If you need the earlier behaviour for some reason, `lastResort` can be set to `None`.

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

raiseExceptions

Used to see if exceptions during handling should be propagated.

Default: `True`.

If `raiseExceptions` is `False`, exceptions get silently ignored. This is what is mostly wanted for a logging system - most users will not care about errors in the logging system, they are more interested in application errors.

</div>

## Integration with the warnings module

The `captureWarnings` function can be used to integrate `!logging` with the `warnings` module.

<div class="function">

captureWarnings(capture)

This function is used to turn the capture of warnings by logging on and off.

If *capture* is `True`, warnings issued by the `warnings` module will be redirected to the logging system. Specifically, a warning will be formatted using `warnings.formatwarning` and the resulting string logged to a logger named `'py.warnings'` with a severity of `WARNING`.

If *capture* is `False`, the redirection of warnings to the logging system will stop, and warnings will be redirected to their original destinations (i.e. those in effect before `captureWarnings(True)` was called).

</div>

<div class="seealso">

Module `logging.config`  
Configuration API for the logging module.

Module `logging.handlers`  
Useful handlers included with the logging module.

`282` - A Logging System  
The proposal which described this feature for inclusion in the Python standard library.

[Original Python logging package](https://old.red-dove.com/python_logging.html)  
This is the original source for the `!logging` package. The version of the package available from this site is suitable for use with Python 1.5.2, 2.1.x and 2.2.x, which do not include the `!logging` package in the standard library.

</div>

# `!lzma` --- Compression using the LZMA algorithm

<div class="module" data-synopsis="A Python wrapper for the liblzma compression library.">

lzma

</div>

<div class="versionadded">

3.3

</div>

**Source code:** `Lib/lzma.py`

------------------------------------------------------------------------

This module provides classes and convenience functions for compressing and decompressing data using the LZMA compression algorithm. Also included is a file interface supporting the `.xz` and legacy `.lzma` file formats used by the `xz` utility, as well as raw compressed streams.

The interface provided by this module is very similar to that of the `bz2` module. Note that `LZMAFile` and `bz2.BZ2File` are *not* thread-safe, so if you need to use a single `LZMAFile` instance from multiple threads, it is necessary to protect it with a lock.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

<div class="exception">

LZMAError

This exception is raised when an error occurs during compression or decompression, or while initializing the compressor/decompressor state.

</div>

## Reading and writing compressed files

<div class="function">

open(filename, mode="rb", \*, format=None, check=-1, preset=None, filters=None, encoding=None, errors=None, newline=None)

Open an LZMA-compressed file in binary or text mode, returning a `file
object`.

The *filename* argument can be either an actual file name (given as a `str`, `bytes` or `path-like <path-like object>` object), in which case the named file is opened, or it can be an existing file object to read from or write to.

The *mode* argument can be any of `"r"`, `"rb"`, `"w"`, `"wb"`, `"x"`, `"xb"`, `"a"` or `"ab"` for binary mode, or `"rt"`, `"wt"`, `"xt"`, or `"at"` for text mode. The default is `"rb"`.

When opening a file for reading, the *format* and *filters* arguments have the same meanings as for `LZMADecompressor`. In this case, the *check* and *preset* arguments should not be used.

When opening a file for writing, the *format*, *check*, *preset* and *filters* arguments have the same meanings as for `LZMACompressor`.

For binary mode, this function is equivalent to the `LZMAFile` constructor: `LZMAFile(filename, mode, ...)`. In this case, the *encoding*, *errors* and *newline* arguments must not be provided.

For text mode, a `LZMAFile` object is created, and wrapped in an `io.TextIOWrapper` instance with the specified encoding, error handling behavior, and line ending(s).

<div class="versionchanged">

3.4 Added support for the `"x"`, `"xb"` and `"xt"` modes.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="LZMAFile(filename=None, mode=&quot;r&quot;, *, format=None, check=-1, preset=None, filters=None)">

Open an LZMA-compressed file in binary mode.

An `LZMAFile` can wrap an already-open `file object`, or operate directly on a named file. The *filename* argument specifies either the file object to wrap, or the name of the file to open (as a `str`, `bytes` or `path-like <path-like object>` object). When wrapping an existing file object, the wrapped file will not be closed when the `LZMAFile` is closed.

The *mode* argument can be either `"r"` for reading (default), `"w"` for overwriting, `"x"` for exclusive creation, or `"a"` for appending. These can equivalently be given as `"rb"`, `"wb"`, `"xb"` and `"ab"` respectively.

If *filename* is a file object (rather than an actual file name), a mode of `"w"` does not truncate the file, and is instead equivalent to `"a"`.

When opening a file for reading, the input file may be the concatenation of multiple separate compressed streams. These are transparently decoded as a single logical stream.

When opening a file for reading, the *format* and *filters* arguments have the same meanings as for `LZMADecompressor`. In this case, the *check* and *preset* arguments should not be used.

When opening a file for writing, the *format*, *check*, *preset* and *filters* arguments have the same meanings as for `LZMACompressor`.

`LZMAFile` supports all the members specified by `io.BufferedIOBase`, except for `~io.BufferedIOBase.detach` and `~io.IOBase.truncate`. Iteration and the `with` statement are supported.

The following method and attributes are also provided:

<div class="method">

peek(size=-1)

Return buffered data without advancing the file position. At least one byte of data will be returned, unless EOF has been reached. The exact number of bytes returned is unspecified (the *size* argument is ignored).

> [!NOTE]
> While calling `peek` does not change the file position of the `LZMAFile`, it may change the position of the underlying file object (e.g. if the `LZMAFile` was constructed by passing a file object for *filename*).

</div>

<div class="attribute">

mode

`'rb'` for reading and `'wb'` for writing.

<div class="versionadded">

3.13

</div>

</div>

<div class="attribute">

name

The lzma file name. Equivalent to the `~io.FileIO.name` attribute of the underlying `file object`.

<div class="versionadded">

3.13

</div>

</div>

<div class="versionchanged">

3.4 Added support for the `"x"` and `"xb"` modes.

</div>

<div class="versionchanged">

3.5 The `~io.BufferedIOBase.read` method now accepts an argument of `None`.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

## Compressing and decompressing data in memory

<div class="LZMACompressor(format=FORMAT_XZ, check=-1, preset=None, filters=None)">

Create a compressor object, which can be used to compress data incrementally.

For a more convenient way of compressing a single chunk of data, see `compress`.

The *format* argument specifies what container format should be used. Possible values are `FORMAT_XZ` (the default), `FORMAT_ALONE` and `FORMAT_RAW`.

The *check* argument specifies the type of integrity check to include in the compressed data. This check is used when decompressing, to ensure that the data has not been corrupted. Possible values are `CHECK_NONE`, `CHECK_CRC32`, `CHECK_CRC64` (the default for `FORMAT_XZ`) and `CHECK_SHA256`.

If the specified check is not supported, an `LZMAError` is raised.

The compression settings can be specified either as a preset compression level (with the *preset* argument), or in detail as a custom filter chain (with the *filters* argument).

The *preset* argument (if provided) should be an integer between `0` and `9` (inclusive), optionally OR-ed with the constant `PRESET_EXTREME`. If neither *preset* nor *filters* are given, the default behavior is to use `PRESET_DEFAULT` (preset level `6`). Higher presets produce smaller output, but make the compression process slower.

> [!NOTE]
> In addition to being more CPU-intensive, compression with higher presets also requires much more memory (and produces output that needs more memory to decompress). With preset `9` for example, the overhead for an `LZMACompressor` object can be as high as 800 MiB. For this reason, it is generally best to stick with the default preset.

The *filters* argument (if provided) should be a filter chain specifier. See `filter-chain-specs` for details.

<div class="method">

compress(data)

Compress *data* (a `bytes` object), returning a `bytes` object containing compressed data for at least part of the input. Some of *data* may be buffered internally, for use in later calls to `compress` and `flush`. The returned data should be concatenated with the output of any previous calls to `compress`.

</div>

<div class="method">

flush()

Finish the compression process, returning a `bytes` object containing any data stored in the compressor's internal buffers.

The compressor cannot be used after this method has been called.

</div>

</div>

<div class="LZMADecompressor(format=FORMAT_AUTO, memlimit=None, filters=None)">

Create a decompressor object, which can be used to decompress data incrementally.

For a more convenient way of decompressing an entire compressed stream at once, see `decompress`.

The *format* argument specifies the container format that should be used. The default is `FORMAT_AUTO`, which can decompress both `.xz` and `.lzma` files. Other possible values are `FORMAT_XZ`, `FORMAT_ALONE`, and `FORMAT_RAW`.

The *memlimit* argument specifies a limit (in bytes) on the amount of memory that the decompressor can use. When this argument is used, decompression will fail with an `LZMAError` if it is not possible to decompress the input within the given memory limit.

The *filters* argument specifies the filter chain that was used to create the stream being decompressed. This argument is required if *format* is `FORMAT_RAW`, but should not be used for other formats. See `filter-chain-specs` for more information about filter chains.

> [!NOTE]
> This class does not transparently handle inputs containing multiple compressed streams, unlike `decompress` and `LZMAFile`. To decompress a multi-stream input with `LZMADecompressor`, you must create a new decompressor for each stream.

<div class="method">

decompress(data, max_length=-1)

Decompress *data* (a `bytes-like object`), returning uncompressed data as bytes. Some of *data* may be buffered internally, for use in later calls to `decompress`. The returned data should be concatenated with the output of any previous calls to `decompress`.

If *max_length* is nonnegative, returns at most *max_length* bytes of decompressed data. If this limit is reached and further output can be produced, the `~.needs_input` attribute will be set to `False`. In this case, the next call to `~.decompress` may provide *data* as `b''` to obtain more of the output.

If all of the input data was decompressed and returned (either because this was less than *max_length* bytes, or because *max_length* was negative), the `~.needs_input` attribute will be set to `True`.

Attempting to decompress data after the end of stream is reached raises an `EOFError`. Any data found after the end of the stream is ignored and saved in the `~.unused_data` attribute.

<div class="versionchanged">

3.5 Added the *max_length* parameter.

</div>

</div>

<div class="attribute">

check

The ID of the integrity check used by the input stream. This may be `CHECK_UNKNOWN` until enough of the input has been decoded to determine what integrity check it uses.

</div>

<div class="attribute">

eof

`True` if the end-of-stream marker has been reached.

</div>

<div class="attribute">

unused_data

Data found after the end of the compressed stream.

Before the end of the stream is reached, this will be `b""`.

</div>

<div class="attribute">

needs_input

`False` if the `.decompress` method can provide more decompressed data before requiring new uncompressed input.

<div class="versionadded">

3.5

</div>

</div>

</div>

<div class="function">

compress(data, format=FORMAT_XZ, check=-1, preset=None, filters=None)

Compress *data* (a `bytes` object), returning the compressed data as a `bytes` object.

See `LZMACompressor` above for a description of the *format*, *check*, *preset* and *filters* arguments.

</div>

<div class="function">

decompress(data, format=FORMAT_AUTO, memlimit=None, filters=None)

Decompress *data* (a `bytes` object), returning the uncompressed data as a `bytes` object.

If *data* is the concatenation of multiple distinct compressed streams, decompress all of these streams, and return the concatenation of the results.

See `LZMADecompressor` above for a description of the *format*, *memlimit* and *filters* arguments.

</div>

## Miscellaneous

<div class="function">

is_check_supported(check)

Return `True` if the given integrity check is supported on this system.

`CHECK_NONE` and `CHECK_CRC32` are always supported. `CHECK_CRC64` and `CHECK_SHA256` may be unavailable if you are using a version of `liblzma` that was compiled with a limited feature set.

</div>

Information about the version of the lzma library in use is available through the following constants:

<div class="data">

LZMA_VERSION

The version string of the lzma library that was used for building the module. This may be different from the lzma library actually used at runtime, which is available as `lzma_version`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

lzma_version

The version string of the lzma library actually loaded by the interpreter.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

LZMA_VERSION_INFO

A named tuple containing the four components of the lzma library version that was used for building the module: *major*, *minor*, *patch*, and *stability*. All values except *stability* are integers; *stability* is `'alpha'`, `'beta'`, or `'stable'`. The components can also be accessed by name, so `lzma.LZMA_VERSION_INFO[0]` is equivalent to `lzma.LZMA_VERSION_INFO.major` and so on. This may be different from the lzma library actually used at runtime, which is available as `lzma_version_info`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

lzma_version_info

A named tuple containing the version of the lzma library actually loaded by the interpreter, with the same fields as `LZMA_VERSION_INFO`.

<div class="versionadded">

3.16

</div>

</div>

## Specifying custom filter chains

A filter chain specifier is a sequence of dictionaries, where each dictionary contains the ID and options for a single filter. Each dictionary must contain the key `"id"`, and may contain additional keys to specify filter-dependent options. Valid filter IDs are as follows:

- Compression filters:
  - `FILTER_LZMA1` (for use with `FORMAT_ALONE`)
  - `FILTER_LZMA2` (for use with `FORMAT_XZ` and `FORMAT_RAW`)
- Delta filter:
  - `FILTER_DELTA`
- Branch-Call-Jump (BCJ) filters:
  - `!FILTER_X86`

  - `!FILTER_IA64`

  - `!FILTER_ARM`

  - `!FILTER_ARMTHUMB`

  - `!FILTER_POWERPC`

  - `!FILTER_SPARC`

    The above work on all lzma runtime library versions.

  - `!FILTER_ARM64`

    Only works if the lzma version is 5.4.0 or later.

    <div class="versionadded">

    3.16

    </div>

  - `!FILTER_RISCV`

    Only works if the lzma version is 5.6.0 or later.

    <div class="versionadded">

    3.16

    </div>

A filter chain can consist of up to 4 filters, and cannot be empty. The last filter in the chain must be a compression filter, and any other filters must be delta or BCJ filters.

Compression filters support the following options (specified as additional entries in the dictionary representing the filter):

- `preset`: A compression preset to use as a source of default values for options that are not specified explicitly.
- `dict_size`: Dictionary size in bytes. This should be between 4 KiB and 1.5 GiB (inclusive).
- `lc`: Number of literal context bits.
- `lp`: Number of literal position bits. The sum `lc + lp` must be at most 4.
- `pb`: Number of position bits; must be at most 4.
- `mode`: `MODE_FAST` or `MODE_NORMAL`.
- `nice_len`: What should be considered a "nice length" for a match. This should be 273 or less.
- `mf`: What match finder to use -- `MF_HC3`, `MF_HC4`, `MF_BT2`, `MF_BT3`, or `MF_BT4`.
- `depth`: Maximum search depth used by match finder. 0 (default) means to select automatically based on other filter options.

The delta filter stores the differences between bytes, producing more repetitive input for the compressor in certain circumstances. It supports one option, `dist`. This indicates the distance between bytes to be subtracted. The default is 1, i.e. take the differences between adjacent bytes.

The BCJ filters are intended to be applied to machine code. They convert relative branches, calls and jumps in the code to use absolute addressing, with the aim of increasing the redundancy that can be exploited by the compressor. These filters support one option, `start_offset`. This specifies the address that should be mapped to the beginning of the input data. The default is 0.

## Constants

The following module-level constants are provided for use as the *format*, *check*, *preset* and *filters* arguments of the classes and functions above.

Container formats:

<div class="data">

FORMAT_XZ

The `.xz` container format.

</div>

<div class="data">

FORMAT_ALONE

The legacy `.lzma` container format. This format is more limited than `.xz` -- it does not support integrity checks or multiple filters.

</div>

<div class="data">

FORMAT_RAW

A raw data stream, not using any container format. This format specifier does not support integrity checks, and requires that you always specify a custom filter chain (for both compression and decompression). Additionally, data compressed in this manner cannot be decompressed using `FORMAT_AUTO`.

</div>

<div class="data">

FORMAT_AUTO

Used for decompression only. The container format is detected automatically, so that both `.xz` and `.lzma` files can be decompressed.

</div>

Integrity checks:

<div class="data">

CHECK_NONE

No integrity check. This is the default (and the only acceptable value) for `FORMAT_ALONE` and `FORMAT_RAW`.

</div>

<div class="data">

CHECK_CRC32

A 32-bit Cyclic Redundancy Check.

</div>

<div class="data">

CHECK_CRC64

A 64-bit Cyclic Redundancy Check. This is the default for `FORMAT_XZ`.

</div>

<div class="data">

CHECK_SHA256

A 256-bit Secure Hash Algorithm.

</div>

<div class="data">

CHECK_UNKNOWN

The integrity check used by a stream could not yet be determined. This may be the value of the `LZMADecompressor.check` attribute until enough of the input has been decoded.

</div>

<div class="data">

CHECK_ID_MAX

The largest supported integrity-check ID.

</div>

Compression presets:

<div class="data">

PRESET_DEFAULT

The default compression preset, equivalent to preset level `6`.

</div>

<div class="data">

PRESET_EXTREME

A flag that may be bitwise OR-ed with a preset level (`0` to `9`) to select a slower but more thorough variant of that preset.

</div>

Filter IDs and options:

<div class="data">

FILTER_LZMA1 FILTER_LZMA2

The LZMA1 and LZMA2 compression filters. `FILTER_LZMA1` is for use with `FORMAT_ALONE`, while `FILTER_LZMA2` is for use with `FORMAT_XZ` and `FORMAT_RAW`.

</div>

<div class="data">

FILTER_DELTA

The delta filter.

</div>

<div class="data">

MODE_FAST MODE_NORMAL

Compression modes that may be used as the `mode` option of a filter specifier (see `filter-chain-specs`).

</div>

<div class="data">

MF_HC3 MF_HC4 MF_BT2 MF_BT3 MF_BT4

Match finders that may be used as the `mf` option of a filter specifier (see `filter-chain-specs`).

</div>

## Examples

Reading in a compressed file:

    import lzma
    with lzma.open("file.xz") as f:
        file_content = f.read()

Creating a compressed file:

    import lzma
    data = b"Insert Data Here"
    with lzma.open("file.xz", "w") as f:
        f.write(data)

Compressing data in memory:

    import lzma
    data_in = b"Insert Data Here"
    data_out = lzma.compress(data_in)

Incremental compression:

    import lzma
    lzc = lzma.LZMACompressor()
    out1 = lzc.compress(b"Some data\n")
    out2 = lzc.compress(b"Another piece of data\n")
    out3 = lzc.compress(b"Even more data\n")
    out4 = lzc.flush()
    # Concatenate all the partial results:
    result = b"".join([out1, out2, out3, out4])

Writing compressed data to an already-open file:

    import lzma
    with open("file.xz", "wb") as f:
        f.write(b"This data will not be compressed\n")
        with lzma.open(f, "w") as lzf:
            lzf.write(b"This *will* be compressed\n")
        f.write(b"Not compressed\n")

Creating a compressed file using a custom filter chain:

    import lzma
    my_filters = [
        {"id": lzma.FILTER_DELTA, "dist": 5},
        {"id": lzma.FILTER_LZMA2, "preset": 7 | lzma.PRESET_EXTREME},
    ]
    with lzma.open("file.xz", "w", filters=my_filters) as f:
        f.write(b"blah blah blah")

# `!mailbox` --- Manipulate mailboxes in various formats

<div class="module" data-synopsis="Manipulate mailboxes in various formats">

mailbox

</div>

**Source code:** `Lib/mailbox.py`

------------------------------------------------------------------------

This module defines two classes, `Mailbox` and `Message`, for accessing and manipulating on-disk mailboxes and the messages they contain. `!Mailbox` offers a dictionary-like mapping from keys to messages. `!Message` extends the `email.message` module's `~email.message.Message` class with format-specific state and behavior. Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF.

> [!WARNING]
> The `!mailbox` module is designed for use in trusted environments, where data from untrusted sources is never passed to the module's API calls and only trusted users have access to the filesystems accessible to the module.

<div class="seealso">

Module `email`  
Represent and manipulate messages.

</div>

## `!Mailbox` objects

<div class="Mailbox">

A mailbox, which may be inspected and modified.

The `!Mailbox` class defines an interface and is not intended to be instantiated. Instead, format-specific subclasses should inherit from `!Mailbox` and your code should instantiate a particular subclass.

The `!Mailbox` interface is dictionary-like, with small keys corresponding to messages. Keys are issued by the `!Mailbox` instance with which they will be used and are only meaningful to that `!Mailbox` instance. A key continues to identify a message even if the corresponding message is modified, such as by replacing it with another message.

Messages may be added to a `!Mailbox` instance using the set-like method `add` and removed using a `del` statement or the set-like methods `remove` and `discard`.

`!Mailbox` interface semantics differ from dictionary semantics in some noteworthy ways. Each time a message is requested, a new representation (typically a `Message` instance) is generated based upon the current state of the mailbox. Similarly, when a message is added to a `!Mailbox` instance, the provided message representation's contents are copied. In neither case is a reference to the message representation kept by the `!Mailbox` instance.

The default `!Mailbox` `iterator` iterates over message representations, not keys as the default `dictionary <dict>` iterator does. Moreover, modification of a mailbox during iteration is safe and well-defined. Messages added to the mailbox after an iterator is created will not be seen by the iterator. Messages removed from the mailbox before the iterator yields them will be silently skipped, though using a key from an iterator may result in a `KeyError` exception if the corresponding message is subsequently removed.

> [!WARNING]
> Be very cautious when modifying mailboxes that might be simultaneously changed by some other process. The safest mailbox format to use for such tasks is `Maildir`; try to avoid using single-file formats such as `mbox` for concurrent writing. If you're modifying a mailbox, you *must* lock it by calling the `lock` and `unlock` methods *before* reading any messages in the file or making any changes by adding or deleting a message. Failing to lock the mailbox runs the risk of losing messages or corrupting the entire mailbox.

The `!Mailbox` class supports the `with` statement. When used as a context manager, `!Mailbox` calls `lock` when the context is entered, returns the mailbox object as the context object, and at context end calls `close`, thereby releasing the lock.

<div class="versionchanged">

3.15 Support for the `with` statement was added.

</div>

`!Mailbox` instances have the following methods:

<div class="method">

add(message)

Add *message* to the mailbox and return the key that has been assigned to it.

Parameter *message* may be a `Message` instance, an `email.message.Message` instance, a string, a byte string, or a file-like object (which should be open in binary mode). If *message* is an instance of the appropriate format-specific `Message` subclass (e.g., if it's an `mboxMessage` instance and this is an `mbox` instance), its format-specific information is used. Otherwise, reasonable defaults for format-specific information are used.

<div class="versionchanged">

3.2 Support for binary input was added.

</div>

</div>

<div class="method">

remove(key) \_\_delitem\_\_(key) discard(key)

Delete the message corresponding to *key* from the mailbox.

If no such message exists, a `KeyError` exception is raised if the method was called as `remove` or `__delitem__` but no exception is raised if the method was called as `discard`. The behavior of `discard` may be preferred if the underlying mailbox format supports concurrent modification by other processes.

</div>

<div class="method">

\_\_setitem\_\_(key, message)

Replace the message corresponding to *key* with *message*. Raise a `KeyError` exception if no message already corresponds to *key*.

As with `add`, parameter *message* may be a `Message` instance, an `email.message.Message` instance, a string, a byte string, or a file-like object (which should be open in binary mode). If *message* is an instance of the appropriate format-specific `Message` subclass (e.g., if it's an `mboxMessage` instance and this is an `mbox` instance), its format-specific information is used. Otherwise, the format-specific information of the message that currently corresponds to *key* is left unchanged.

</div>

<div class="method">

iterkeys()

Return an `iterator` over all keys

</div>

<div class="method">

keys()

The same as `iterkeys`, except that a `list` is returned rather than an `iterator`

</div>

<div class="method">

itervalues() \_\_iter\_\_()

Return an `iterator` over representations of all messages. The messages are represented as instances of the appropriate format-specific `Message` subclass unless a custom message factory was specified when the `!Mailbox` instance was initialized.

> [!NOTE]
> The behavior of `__iter__` is unlike that of dictionaries, which iterate over keys.

</div>

<div class="method">

values()

The same as `itervalues`, except that a `list` is returned rather than an `iterator`

</div>

<div class="method">

iteritems()

Return an `iterator` over (*key*, *message*) pairs, where *key* is a key and *message* is a message representation. The messages are represented as instances of the appropriate format-specific `Message` subclass unless a custom message factory was specified when the `!Mailbox` instance was initialized.

</div>

<div class="method">

items()

The same as `iteritems`, except that a `list` of pairs is returned rather than an `iterator` of pairs.

</div>

<div class="method">

get(key, default=None) \_\_getitem\_\_(key)

Return a representation of the message corresponding to *key*. If no such message exists, *default* is returned if the method was called as `get` and a `KeyError` exception is raised if the method was called as `!__getitem__`. The message is represented as an instance of the appropriate format-specific `Message` subclass unless a custom message factory was specified when the `!Mailbox` instance was initialized.

</div>

<div class="method">

get_message(key)

Return a representation of the message corresponding to *key* as an instance of the appropriate format-specific `Message` subclass, or raise a `KeyError` exception if no such message exists.

</div>

<div class="method">

get_bytes(key)

Return a byte representation of the message corresponding to *key*, or raise a `KeyError` exception if no such message exists.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

get_string(key)

Return a string representation of the message corresponding to *key*, or raise a `KeyError` exception if no such message exists. The message is processed through `email.message.Message` to convert it to a 7bit clean representation.

</div>

<div class="method">

get_file(key)

Return a `file-like <file-like object>` representation of the message corresponding to *key*, or raise a `KeyError` exception if no such message exists. The file-like object behaves as if open in binary mode. This file should be closed once it is no longer needed.

<div class="versionchanged">

3.2 The file object really is a `binary file`; previously it was incorrectly returned in text mode. Also, the `file-like object` now supports the `context manager` protocol: you can use a `with` statement to automatically close it.

</div>

> [!NOTE]
> Unlike other representations of messages, `file-like <file-like object>` representations are not necessarily independent of the `!Mailbox` instance that created them or of the underlying mailbox. More specific documentation is provided by each subclass.

</div>

<div class="method">

\_\_contains\_\_(key)

Return `True` if *key* corresponds to a message, `False` otherwise.

</div>

<div class="method">

\_\_len\_\_()

Return a count of messages in the mailbox.

</div>

<div class="method">

clear()

Delete all messages from the mailbox.

</div>

<div class="method">

pop(key, default=None)

Return a representation of the message corresponding to *key* and delete the message. If no such message exists, return *default*. The message is represented as an instance of the appropriate format-specific `Message` subclass unless a custom message factory was specified when the `!Mailbox` instance was initialized.

</div>

<div class="method">

popitem()

Return an arbitrary (*key*, *message*) pair, where *key* is a key and *message* is a message representation, and delete the corresponding message. If the mailbox is empty, raise a `KeyError` exception. The message is represented as an instance of the appropriate format-specific `Message` subclass unless a custom message factory was specified when the `!Mailbox` instance was initialized.

</div>

<div class="method">

update(arg)

Parameter *arg* should be a *key*-to-*message* mapping or an iterable of (*key*, *message*) pairs. Updates the mailbox so that, for each given *key* and *message*, the message corresponding to *key* is set to *message* as if by using `__setitem__`. As with `__setitem__`, each *key* must already correspond to a message in the mailbox or else a `KeyError` exception will be raised, so in general it is incorrect for *arg* to be a `!Mailbox` instance.

> [!NOTE]
> Unlike with dictionaries, keyword arguments are not supported.

</div>

<div class="method">

flush()

Write any pending changes to the filesystem. For some `Mailbox` subclasses, changes are always written immediately and `!flush` does nothing, but you should still make a habit of calling this method.

</div>

<div class="method">

lock()

Acquire an exclusive advisory lock on the mailbox so that other processes know not to modify it. An `ExternalClashError` is raised if the lock is not available. The particular locking mechanisms used depend upon the mailbox format. You should *always* lock the mailbox before making any modifications to its contents.

</div>

<div class="method">

unlock()

Release the lock on the mailbox, if any.

</div>

<div class="method">

close()

Flush the mailbox, unlock it if necessary, and close any open files. For some `!Mailbox` subclasses, this method does nothing.

</div>

</div>

### `!Maildir` objects

<div class="Maildir(dirname, factory=None, create=True)">

A subclass of `Mailbox` for mailboxes in Maildir format. Parameter *factory* is a callable object that accepts a file-like message representation (which behaves as if opened in binary mode) and returns a custom representation. If *factory* is `None`, `MaildirMessage` is used as the default message representation. If *create* is `True`, the mailbox is created if it does not exist.

If *create* is `True` and the *dirname* path exists, it will be treated as an existing maildir without attempting to verify its directory layout.

It is for historical reasons that *dirname* is named as such rather than *path*.

Maildir is a directory-based mailbox format invented for the qmail mail transfer agent and now widely supported by other programs. Messages in a Maildir mailbox are stored in separate files within a common directory structure. This design allows Maildir mailboxes to be accessed and modified by multiple unrelated programs without data corruption, so file locking is unnecessary.

Maildir mailboxes contain three subdirectories, namely: `tmp`, `new`, and `cur`. Messages are created momentarily in the `tmp` subdirectory and then moved to the `new` subdirectory to finalize delivery. A mail user agent may subsequently move the message to the `cur` subdirectory and store information about the state of the message in a special "info" section appended to its file name.

Folders of the style introduced by the Courier mail transfer agent are also supported. Any subdirectory of the main mailbox is considered a folder if `'.'` is the first character in its name. Folder names are represented by `!Maildir` without the leading `'.'`. Each folder is itself a Maildir mailbox but should not contain other folders. Instead, a logical nesting is indicated using `'.'` to delimit levels, e.g., "Archived.2005.07".

<div class="attribute">

Maildir.colon

The Maildir specification requires the use of a colon (`':'`) in certain message file names. However, some operating systems do not permit this character in file names, If you wish to use a Maildir-like format on such an operating system, you should specify another character to use instead. The exclamation point (`'!'`) is a popular choice. For example:

    import mailbox
    mailbox.Maildir.colon = '!'

The `!colon` attribute may also be set on a per-instance basis.

</div>

<div class="versionchanged">

3.13 `Maildir` now ignores files with a leading dot.

</div>

`!Maildir` instances have all of the methods of `Mailbox` in addition to the following:

<div class="method">

list_folders()

Return a list of the names of all folders.

</div>

<div class="method">

get_folder(folder)

Return a `!Maildir` instance representing the folder whose name is *folder*. A `NoSuchMailboxError` exception is raised if the folder does not exist.

</div>

<div class="method">

add_folder(folder)

Create a folder whose name is *folder* and return a `!Maildir` instance representing it.

</div>

<div class="method">

remove_folder(folder)

Delete the folder whose name is *folder*. If the folder contains any messages, a `NotEmptyError` exception will be raised and the folder will not be deleted.

</div>

<div class="method">

clean()

Delete temporary files from the mailbox that have not been accessed in the last 36 hours. The Maildir specification says that mail-reading programs should do this occasionally.

</div>

<div class="method">

get_flags(key)

Return as a string the flags that are set on the message corresponding to *key*. This is the same as `get_message(key).get_flags()` but much faster, because it does not open the message file. Use this method when iterating over the keys to determine which messages are interesting to get.

If you do have a `MaildirMessage` object, use its `~MaildirMessage.get_flags` method instead, because changes made by the message's `~MaildirMessage.set_flags`, `~MaildirMessage.add_flag` and `~MaildirMessage.remove_flag` methods are not reflected here until the mailbox's `__setitem__` method is called.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

set_flags(key, flags)

On the message corresponding to *key*, set the flags specified by *flags* and unset all others. Calling `some_mailbox.set_flags(key, flags)` is similar to :

    one_message = some_mailbox.get_message(key)
    one_message.set_flags(flags)
    some_mailbox[key] = one_message

but faster, because it does not open the message file.

If you do have a `MaildirMessage` object, use its `~MaildirMessage.set_flags` method instead, because changes made with this mailbox method will not be visible to the message object's method, `~MaildirMessage.get_flags`.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

add_flag(key, flag)

On the message corresponding to *key*, set the flags specified by *flag* without changing other flags. To add more than one flag at a time, *flag* may be a string of more than one character.

Considerations for using this method versus the message object's `~MaildirMessage.add_flag` method are similar to those for `set_flags`; see the discussion there.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

remove_flag(key, flag)

On the message corresponding to *key*, unset the flags specified by *flag* without changing other flags. To remove more than one flag at a time, *flag* may be a string of more than one character.

Considerations for using this method versus the message object's `~MaildirMessage.remove_flag` method are similar to those for `set_flags`; see the discussion there.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

get_info(key)

Return a string containing the info for the message corresponding to *key*. This is the same as `get_message(key).get_info()` but much faster, because it does not open the message file. Use this method when iterating over the keys to determine which messages are interesting to get.

If you do have a `MaildirMessage` object, use its `~MaildirMessage.get_info` method instead, because changes made by the message's `~MaildirMessage.set_info` method are not reflected here until the mailbox's `__setitem__` method is called.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

set_info(key, info)

Set the info of the message corresponding to *key* to *info*. Calling `some_mailbox.set_info(key, info)` is similar to :

    one_message = some_mailbox.get_message(key)
    one_message.set_info(info)
    some_mailbox[key] = one_message

but faster, because it does not open the message file.

If you do have a `MaildirMessage` object, use its `~MaildirMessage.set_info` method instead, because changes made with this mailbox method will not be visible to the message object's method, `~MaildirMessage.get_info`.

<div class="versionadded">

3.13

</div>

</div>

Some `Mailbox` methods implemented by `!Maildir` deserve special remarks:

<div class="method">

add(message) \_\_setitem\_\_(key, message) update(arg)

> [!WARNING]
> These methods generate unique file names based upon the current process ID. When using multiple threads, undetected name clashes may occur and cause corruption of the mailbox unless threads are coordinated to avoid using these methods to manipulate the same mailbox simultaneously.

</div>

<div class="method">

flush()

All changes to Maildir mailboxes are immediately applied, so this method does nothing.

</div>

<div class="method">

lock() unlock()

Maildir mailboxes do not support (or require) locking, so these methods do nothing.

</div>

<div class="method">

close()

`!Maildir` instances do not keep any open files and the underlying mailboxes do not support locking, so this method does nothing.

</div>

<div class="method">

get_file(key)

Depending upon the host platform, it may not be possible to modify or remove the underlying message while the returned file remains open.

</div>

</div>

<div class="seealso">

[maildir man page from Courier](https://www.courier-mta.org/maildir.html)  
A specification of the format. Describes a common extension for supporting folders.

[Using maildir format](https://cr.yp.to/proto/maildir.html)  
Notes on Maildir by its inventor. Includes an updated name-creation scheme and details on "info" semantics.

</div>

### `!mbox` objects

<div class="mbox(path, factory=None, create=True)">

A subclass of `Mailbox` for mailboxes in mbox format. Parameter *factory* is a callable object that accepts a file-like message representation (which behaves as if opened in binary mode) and returns a custom representation. If *factory* is `None`, `mboxMessage` is used as the default message representation. If *create* is `True`, the mailbox is created if it does not exist.

The mbox format is the classic format for storing mail on Unix systems. All messages in an mbox mailbox are stored in a single file with the beginning of each message indicated by a line whose first five characters are "From ".

Several variations of the mbox format exist to address perceived shortcomings in the original. In the interest of compatibility, `!mbox` implements the original format, which is sometimes referred to as `mboxo`. This means that the `Content-Length` header, if present, is ignored and that any occurrences of "From " at the beginning of a line in a message body are transformed to "\>From " when storing the message, although occurrences of "\>From " are not transformed to "From " when reading the message.

Some `Mailbox` methods implemented by `!mbox` deserve special remarks:

<div class="method">

get_bytes(key, [from]()=False)

Note: This method has an extra parameter (*from\_*) compared with other classes. The first line of an mbox file entry is the Unix "From " line. If *from\_* is False, the first line of the file is dropped.

</div>

<div class="method">

get_file(key, [from]()=False)

Using the file after calling `~Mailbox.flush` or `~Mailbox.close` on the `!mbox` instance may yield unpredictable results or raise an exception.

Note: This method has an extra parameter (*from\_*) compared with other classes. The first line of an mbox file entry is the Unix "From " line. If *from\_* is False, the first line of the file is dropped.

</div>

<div class="method">

get_string(key, [from]()=False)

Note: This method has an extra parameter (*from\_*) compared with other classes. The first line of an mbox file entry is the Unix "From " line. If *from\_* is False, the first line of the file is dropped.

</div>

<div class="method">

lock() unlock()

Three locking mechanisms are used---dot locking and, if available, the `!flock` and `!lockf` system calls.

</div>

</div>

<div class="seealso">

[mbox man page from tin](http://www.tin.org/bin/man.cgi?section=5&topic=mbox)  
A specification of the format, with details on locking.

[Configuring Netscape Mail on Unix: Why The Content-Length Format is Bad](https://www.jwz.org/doc/content-length.html)  
An argument for using the original mbox format rather than a variation.

["mbox" is a family of several mutually incompatible mailbox formats](https://www.loc.gov/preservation/digital/formats/fdd/fdd000383.shtml)  
A history of mbox variations.

</div>

### `!MH` objects

<div class="MH(path, factory=None, create=True)">

A subclass of `Mailbox` for mailboxes in MH format. Parameter *factory* is a callable object that accepts a file-like message representation (which behaves as if opened in binary mode) and returns a custom representation. If *factory* is `None`, `MHMessage` is used as the default message representation. If *create* is `True`, the mailbox is created if it does not exist.

MH is a directory-based mailbox format invented for the MH Message Handling System, a mail user agent. Each message in an MH mailbox resides in its own file. An MH mailbox may contain other MH mailboxes (called `folders`) in addition to messages. Folders may be nested indefinitely. MH mailboxes also support `sequences`, which are named lists used to logically group messages without moving them to sub-folders. Sequences are defined in a file called `.mh_sequences` in each folder.

The `!MH` class manipulates MH mailboxes, but it does not attempt to emulate all of `mh`'s behaviors. In particular, it does not modify and is not affected by the `context` or `.mh_profile` files that are used by `mh` to store its state and configuration.

`!MH` instances have all of the methods of `Mailbox` in addition to the following:

<div class="versionchanged">

3.13

Supported folders that don't contain a `.mh_sequences` file.

</div>

<div class="method">

list_folders()

Return a list of the names of all folders.

</div>

<div class="method">

get_folder(folder)

Return an `!MH` instance representing the folder whose name is *folder*. A `NoSuchMailboxError` exception is raised if the folder does not exist.

</div>

<div class="method">

add_folder(folder)

Create a folder whose name is *folder* and return an `!MH` instance representing it.

</div>

<div class="method">

remove_folder(folder)

Delete the folder whose name is *folder*. If the folder contains any messages, a `NotEmptyError` exception will be raised and the folder will not be deleted.

</div>

<div class="method">

get_sequences()

Return a dictionary of sequence names mapped to key lists. If there are no sequences, the empty dictionary is returned.

</div>

<div class="method">

set_sequences(sequences)

Re-define the sequences that exist in the mailbox based upon *sequences*, a dictionary of names mapped to key lists, like returned by `get_sequences`.

</div>

<div class="method">

pack()

Rename messages in the mailbox as necessary to eliminate gaps in numbering. Entries in the sequences list are updated correspondingly.

> [!NOTE]
> Already-issued keys are invalidated by this operation and should not be subsequently used.

</div>

Some `Mailbox` methods implemented by `!MH` deserve special remarks:

<div class="method">

remove(key) \_\_delitem\_\_(key) discard(key)

These methods immediately delete the message. The MH convention of marking a message for deletion by prepending a comma to its name is not used.

</div>

<div class="method">

lock() unlock()

Three locking mechanisms are used---dot locking and, if available, the `!flock` and `!lockf` system calls. For MH mailboxes, locking the mailbox means locking the `.mh_sequences` file and, only for the duration of any operations that affect them, locking individual message files.

</div>

<div class="method">

get_file(key)

Depending upon the host platform, it may not be possible to remove the underlying message while the returned file remains open.

</div>

<div class="method">

flush()

All changes to MH mailboxes are immediately applied, so this method does nothing.

</div>

<div class="method">

close()

`!MH` instances do not keep any open files, so this method is equivalent to `unlock`.

</div>

</div>

<div class="seealso">

[nmh - Message Handling System](https://www.nongnu.org/nmh/)  
Home page of `nmh`, an updated version of the original `mh`.

[MH & nmh: Email for Users & Programmers](https://rand-mh.sourceforge.io/book/)  
A GPL-licensed book on `mh` and `nmh`, with some information on the mailbox format.

</div>

### `!Babyl` objects

<div class="Babyl(path, factory=None, create=True)">

A subclass of `Mailbox` for mailboxes in Babyl format. Parameter *factory* is a callable object that accepts a file-like message representation (which behaves as if opened in binary mode) and returns a custom representation. If *factory* is `None`, `BabylMessage` is used as the default message representation. If *create* is `True`, the mailbox is created if it does not exist.

Babyl is a single-file mailbox format used by the Rmail mail user agent included with Emacs. The beginning of a message is indicated by a line containing the two characters Control-Underscore (`'\037'`) and Control-L (`'\014'`). The end of a message is indicated by the start of the next message or, in the case of the last message, a line containing a Control-Underscore (`'\037'`) character.

Messages in a Babyl mailbox have two sets of headers, original headers and so-called visible headers. Visible headers are typically a subset of the original headers that have been reformatted or abridged to be more attractive. Each message in a Babyl mailbox also has an accompanying list of `labels`, or short strings that record extra information about the message, and a list of all user-defined labels found in the mailbox is kept in the Babyl options section.

`!Babyl` instances have all of the methods of `Mailbox` in addition to the following:

<div class="method">

get_labels()

Return a list of the names of all user-defined labels used in the mailbox.

> [!NOTE]
> The actual messages are inspected to determine which labels exist in the mailbox rather than consulting the list of labels in the Babyl options section, but the Babyl section is updated whenever the mailbox is modified.

</div>

Some `Mailbox` methods implemented by `!Babyl` deserve special remarks:

<div class="method">

get_file(key)

In Babyl mailboxes, the headers of a message are not stored contiguously with the body of the message. To generate a file-like representation, the headers and body are copied together into an `io.BytesIO` instance, which has an API identical to that of a file. As a result, the file-like object is truly independent of the underlying mailbox but does not save memory compared to a string representation.

</div>

<div class="method">

lock() unlock()

Three locking mechanisms are used---dot locking and, if available, the `!flock` and `!lockf` system calls.

</div>

</div>

<div class="seealso">

[Format of Version 5 Babyl Files](https://quimby.gnus.org/notes/BABYL)  
A specification of the Babyl format.

[Reading Mail with Rmail](https://www.gnu.org/software/emacs/manual/html_node/emacs/Rmail.html)  
The Rmail manual, with some information on Babyl semantics.

</div>

### `!MMDF` objects

<div class="MMDF(path, factory=None, create=True)">

A subclass of `Mailbox` for mailboxes in MMDF format. Parameter *factory* is a callable object that accepts a file-like message representation (which behaves as if opened in binary mode) and returns a custom representation. If *factory* is `None`, `MMDFMessage` is used as the default message representation. If *create* is `True`, the mailbox is created if it does not exist.

MMDF is a single-file mailbox format invented for the Multichannel Memorandum Distribution Facility, a mail transfer agent. Each message is in the same form as an mbox message but is bracketed before and after by lines containing four Control-A (`'\001'`) characters. As with the mbox format, the beginning of each message is indicated by a line whose first five characters are "From ", but additional occurrences of "From " are not transformed to "\>From " when storing messages because the extra message separator lines prevent mistaking such occurrences for the starts of subsequent messages.

Some `Mailbox` methods implemented by `!MMDF` deserve special remarks:

<div class="method">

get_bytes(key, [from]()=False)

Note: This method has an extra parameter (*from\_*) compared with other classes. The first line of an mbox file entry is the Unix "From " line. If *from\_* is False, the first line of the file is dropped.

</div>

<div class="method">

get_file(key, [from]()=False)

Using the file after calling `~Mailbox.flush` or `~Mailbox.close` on the `!MMDF` instance may yield unpredictable results or raise an exception.

Note: This method has an extra parameter (*from\_*) compared with other classes. The first line of an mbox file entry is the Unix "From " line. If *from\_* is False, the first line of the file is dropped.

</div>

<div class="method">

lock() unlock()

Three locking mechanisms are used---dot locking and, if available, the `!flock` and `!lockf` system calls.

</div>

</div>

<div class="seealso">

[mmdf man page from tin](http://www.tin.org/bin/man.cgi?section=5&topic=mmdf)  
A specification of MMDF format from the documentation of tin, a newsreader.

[MMDF](https://en.wikipedia.org/wiki/MMDF)  
A Wikipedia article describing the Multichannel Memorandum Distribution Facility.

</div>

## `!Message` objects

<div class="Message(message=None)">

A subclass of the `email.message` module's `~email.message.Message`. Subclasses of `!mailbox.Message` add mailbox-format-specific state and behavior.

If *message* is omitted, the new instance is created in a default, empty state. If *message* is an `email.message.Message` instance, its contents are copied; furthermore, any format-specific information is converted insofar as possible if *message* is a `!Message` instance. If *message* is a string, a byte string, or a file, it should contain an `5322`-compliant message, which is read and parsed. Files should be open in binary mode, but text mode files are accepted for backward compatibility.

The format-specific state and behaviors offered by subclasses vary, but in general it is only the properties that are not specific to a particular mailbox that are supported (although presumably the properties are specific to a particular mailbox format). For example, file offsets for single-file mailbox formats and file names for directory-based mailbox formats are not retained, because they are only applicable to the original mailbox. But state such as whether a message has been read by the user or marked as important is retained, because it applies to the message itself.

There is no requirement that `!Message` instances be used to represent messages retrieved using `Mailbox` instances. In some situations, the time and memory required to generate `!Message` representations might not be acceptable. For such situations, `!Mailbox` instances also offer string and file-like representations, and a custom message factory may be specified when a `!Mailbox` instance is initialized.

</div>

### `!MaildirMessage` objects

<div class="MaildirMessage(message=None)">

A message with Maildir-specific behaviors. Parameter *message* has the same meaning as with the `Message` constructor.

Typically, a mail user agent application moves all of the messages in the `new` subdirectory to the `cur` subdirectory after the first time the user opens and closes the mailbox, recording that the messages are old whether or not they've actually been read. Each message in `cur` has an "info" section added to its file name to store information about its state. (Some mail readers may also add an "info" section to messages in `new`.) The "info" section may take one of two forms: it may contain "2," followed by a list of standardized flags (e.g., "2,FR") or it may contain "1," followed by so-called experimental information. Standard flags for Maildir messages are as follows:

| Flag | Meaning | Explanation                    |
|------|---------|--------------------------------|
| D    | Draft   | Under composition              |
| F    | Flagged | Marked as important            |
| P    | Passed  | Forwarded, resent, or bounced  |
| R    | Replied | Replied to                     |
| S    | Seen    | Read                           |
| T    | Trashed | Marked for subsequent deletion |

`!MaildirMessage` instances offer the following methods:

<div class="method">

get_subdir()

Return either "new" (if the message should be stored in the `new` subdirectory) or "cur" (if the message should be stored in the `cur` subdirectory).

> [!NOTE]
> A message is typically moved from `new` to `cur` after its mailbox has been accessed, whether or not the message has been read. A message `msg` has been read if `"S" in msg.get_flags()` is `True`.

</div>

<div class="method">

set_subdir(subdir)

Set the subdirectory the message should be stored in. Parameter *subdir* must be either "new" or "cur".

</div>

<div class="method">

get_flags()

Return a string specifying the flags that are currently set. If the message complies with the standard Maildir format, the result is the concatenation in alphabetical order of zero or one occurrence of each of `'D'`, `'F'`, `'P'`, `'R'`, `'S'`, and `'T'`. The empty string is returned if no flags are set or if "info" contains experimental semantics.

</div>

<div class="method">

set_flags(flags)

Set the flags specified by *flags* and unset all others.

</div>

<div class="method">

add_flag(flag)

Set the flag(s) specified by *flag* without changing other flags. To add more than one flag at a time, *flag* may be a string of more than one character. The current "info" is overwritten whether or not it contains experimental information rather than flags.

</div>

<div class="method">

remove_flag(flag)

Unset the flag(s) specified by *flag* without changing other flags. To remove more than one flag at a time, *flag* may be a string of more than one character. If "info" contains experimental information rather than flags, the current "info" is not modified.

</div>

<div class="method">

get_date()

Return the delivery date of the message as a floating-point number representing seconds since the epoch.

</div>

<div class="method">

set_date(date)

Set the delivery date of the message to *date*, a floating-point number representing seconds since the epoch.

</div>

<div class="method">

get_info()

Return a string containing the "info" for a message. This is useful for accessing and modifying "info" that is experimental (i.e., not a list of flags).

</div>

<div class="method">

set_info(info)

Set "info" to *info*, which should be a string.

</div>

</div>

When a `!MaildirMessage` instance is created based upon an `mboxMessage` or `MMDFMessage` instance, the `Status` and `X-Status` headers are omitted and the following conversions take place:

| Resulting state    | `mboxMessage` or `MMDFMessage` state |
|--------------------|--------------------------------------|
| "cur" subdirectory | O flag                               |
| F flag             | F flag                               |
| R flag             | A flag                               |
| S flag             | R flag                               |
| T flag             | D flag                               |

When a `!MaildirMessage` instance is created based upon an `MHMessage` instance, the following conversions take place:

| Resulting state               | `MHMessage` state    |
|-------------------------------|----------------------|
| "cur" subdirectory            | "unseen" sequence    |
| "cur" subdirectory and S flag | no "unseen" sequence |
| F flag                        | "flagged" sequence   |
| R flag                        | "replied" sequence   |

When a `!MaildirMessage` instance is created based upon a `BabylMessage` instance, the following conversions take place:

| Resulting state               | `BabylMessage` state          |
|-------------------------------|-------------------------------|
| "cur" subdirectory            | "unseen" label                |
| "cur" subdirectory and S flag | no "unseen" label             |
| P flag                        | "forwarded" or "resent" label |
| R flag                        | "answered" label              |
| T flag                        | "deleted" label               |

### `!mboxMessage` objects

<div class="mboxMessage(message=None)">

A message with mbox-specific behaviors. Parameter *message* has the same meaning as with the `Message` constructor.

Messages in an mbox mailbox are stored together in a single file. The sender's envelope address and the time of delivery are typically stored in a line beginning with "From " that is used to indicate the start of a message, though there is considerable variation in the exact format of this data among mbox implementations. Flags that indicate the state of the message, such as whether it has been read or marked as important, are typically stored in `Status` and `X-Status` headers.

Conventional flags for mbox messages are as follows:

| Flag | Meaning  | Explanation                    |
|------|----------|--------------------------------|
| R    | Read     | Read                           |
| O    | Old      | Previously detected by MUA     |
| D    | Deleted  | Marked for subsequent deletion |
| F    | Flagged  | Marked as important            |
| A    | Answered | Replied to                     |

The "R" and "O" flags are stored in the `Status` header, and the "D", "F", and "A" flags are stored in the `X-Status` header. The flags and headers typically appear in the order mentioned.

`!mboxMessage` instances offer the following methods:

<div class="method">

get_from()

Return a string representing the "From " line that marks the start of the message in an mbox mailbox. The leading "From " and the trailing newline are excluded.

</div>

<div class="method">

set_from([from](), [time]()=None)

Set the "From " line to *from\_*, which should be specified without a leading "From " or trailing newline. For convenience, *time\_* may be specified and will be formatted appropriately and appended to *from\_*. If *time\_* is specified, it should be a `time.struct_time` instance, a tuple suitable for passing to `time.strftime`, or `True` (to use `time.gmtime`).

</div>

<div class="method">

get_flags()

Return a string specifying the flags that are currently set. If the message complies with the conventional format, the result is the concatenation in the following order of zero or one occurrence of each of `'R'`, `'O'`, `'D'`, `'F'`, and `'A'`.

</div>

<div class="method">

set_flags(flags)

Set the flags specified by *flags* and unset all others. Parameter *flags* should be the concatenation in any order of zero or more occurrences of each of `'R'`, `'O'`, `'D'`, `'F'`, and `'A'`.

</div>

<div class="method">

add_flag(flag)

Set the flag(s) specified by *flag* without changing other flags. To add more than one flag at a time, *flag* may be a string of more than one character.

</div>

<div class="method">

remove_flag(flag)

Unset the flag(s) specified by *flag* without changing other flags. To remove more than one flag at a time, *flag* may be a string of more than one character.

</div>

</div>

When an `!mboxMessage` instance is created based upon a `MaildirMessage` instance, a "From " line is generated based upon the `MaildirMessage` instance's delivery date, and the following conversions take place:

| Resulting state | `MaildirMessage` state |
|-----------------|------------------------|
| R flag          | S flag                 |
| O flag          | "cur" subdirectory     |
| D flag          | T flag                 |
| F flag          | F flag                 |
| A flag          | R flag                 |

When an `!mboxMessage` instance is created based upon an `MHMessage` instance, the following conversions take place:

| Resulting state   | `MHMessage` state    |
|-------------------|----------------------|
| R flag and O flag | no "unseen" sequence |
| O flag            | "unseen" sequence    |
| F flag            | "flagged" sequence   |
| A flag            | "replied" sequence   |

When an `!mboxMessage` instance is created based upon a `BabylMessage` instance, the following conversions take place:

| Resulting state   | `BabylMessage` state |
|-------------------|----------------------|
| R flag and O flag | no "unseen" label    |
| O flag            | "unseen" label       |
| D flag            | "deleted" label      |
| A flag            | "answered" label     |

When a `!mboxMessage` instance is created based upon an `MMDFMessage` instance, the "From " line is copied and all flags directly correspond:

| Resulting state | `MMDFMessage` state |
|-----------------|---------------------|
| R flag          | R flag              |
| O flag          | O flag              |
| D flag          | D flag              |
| F flag          | F flag              |
| A flag          | A flag              |

### `!MHMessage` objects

<div class="MHMessage(message=None)">

A message with MH-specific behaviors. Parameter *message* has the same meaning as with the `Message` constructor.

MH messages do not support marks or flags in the traditional sense, but they do support sequences, which are logical groupings of arbitrary messages. Some mail reading programs (although not the standard `mh` and `nmh`) use sequences in much the same way flags are used with other formats, as follows:

| Sequence | Explanation                              |
|----------|------------------------------------------|
| unseen   | Not read, but previously detected by MUA |
| replied  | Replied to                               |
| flagged  | Marked as important                      |

`!MHMessage` instances offer the following methods:

<div class="method">

get_sequences()

Return a list of the names of sequences that include this message.

</div>

<div class="method">

set_sequences(sequences)

Set the list of sequences that include this message.

</div>

<div class="method">

add_sequence(sequence)

Add *sequence* to the list of sequences that include this message.

</div>

<div class="method">

remove_sequence(sequence)

Remove *sequence* from the list of sequences that include this message.

</div>

</div>

When an `!MHMessage` instance is created based upon a `MaildirMessage` instance, the following conversions take place:

| Resulting state    | `MaildirMessage` state |
|--------------------|------------------------|
| "unseen" sequence  | no S flag              |
| "replied" sequence | R flag                 |
| "flagged" sequence | F flag                 |

When an `!MHMessage` instance is created based upon an `mboxMessage` or `MMDFMessage` instance, the `Status` and `X-Status` headers are omitted and the following conversions take place:

| Resulting state    | `mboxMessage` or `MMDFMessage` state |
|--------------------|--------------------------------------|
| "unseen" sequence  | no R flag                            |
| "replied" sequence | A flag                               |
| "flagged" sequence | F flag                               |

When an `!MHMessage` instance is created based upon a `BabylMessage` instance, the following conversions take place:

| Resulting state    | `BabylMessage` state |
|--------------------|----------------------|
| "unseen" sequence  | "unseen" label       |
| "replied" sequence | "answered" label     |

### `!BabylMessage` objects

<div class="BabylMessage(message=None)">

A message with Babyl-specific behaviors. Parameter *message* has the same meaning as with the `Message` constructor.

Certain message labels, called `attributes`, are defined by convention to have special meanings. The attributes are as follows:

| Label     | Explanation                              |
|-----------|------------------------------------------|
| unseen    | Not read, but previously detected by MUA |
| deleted   | Marked for subsequent deletion           |
| filed     | Copied to another file or mailbox        |
| answered  | Replied to                               |
| forwarded | Forwarded                                |
| edited    | Modified by the user                     |
| resent    | Resent                                   |

By default, Rmail displays only visible headers. The `!BabylMessage` class, though, uses the original headers because they are more complete. Visible headers may be accessed explicitly if desired.

`!BabylMessage` instances offer the following methods:

<div class="method">

get_labels()

Return a list of labels on the message.

</div>

<div class="method">

set_labels(labels)

Set the list of labels on the message to *labels*.

</div>

<div class="method">

add_label(label)

Add *label* to the list of labels on the message.

</div>

<div class="method">

remove_label(label)

Remove *label* from the list of labels on the message.

</div>

<div class="method">

get_visible()

Return a `Message` instance whose headers are the message's visible headers and whose body is empty.

</div>

<div class="method">

set_visible(visible)

Set the message's visible headers to be the same as the headers in *message*. Parameter *visible* should be a `Message` instance, an `email.message.Message` instance, a string, or a file-like object (which should be open in text mode).

</div>

<div class="method">

update_visible()

When a `!BabylMessage` instance's original headers are modified, the visible headers are not automatically modified to correspond. This method updates the visible headers as follows: each visible header with a corresponding original header is set to the value of the original header, each visible header without a corresponding original header is removed, and any of `Date`, `From`, `Reply-To`, `To`, `CC`, and `Subject` that are present in the original headers but not the visible headers are added to the visible headers.

</div>

</div>

When a `!BabylMessage` instance is created based upon a `MaildirMessage` instance, the following conversions take place:

| Resulting state   | `MaildirMessage` state |
|-------------------|------------------------|
| "unseen" label    | no S flag              |
| "deleted" label   | T flag                 |
| "answered" label  | R flag                 |
| "forwarded" label | P flag                 |

When a `!BabylMessage` instance is created based upon an `mboxMessage` or `MMDFMessage` instance, the `Status` and `X-Status` headers are omitted and the following conversions take place:

| Resulting state  | `mboxMessage` or `MMDFMessage` state |
|------------------|--------------------------------------|
| "unseen" label   | no R flag                            |
| "deleted" label  | D flag                               |
| "answered" label | A flag                               |

When a `!BabylMessage` instance is created based upon an `MHMessage` instance, the following conversions take place:

| Resulting state  | `MHMessage` state  |
|------------------|--------------------|
| "unseen" label   | "unseen" sequence  |
| "answered" label | "replied" sequence |

### `!MMDFMessage` objects

<div class="MMDFMessage(message=None)">

A message with MMDF-specific behaviors. Parameter *message* has the same meaning as with the `Message` constructor.

As with message in an mbox mailbox, MMDF messages are stored with the sender's address and the delivery date in an initial line beginning with "From ". Likewise, flags that indicate the state of the message are typically stored in `Status` and `X-Status` headers.

Conventional flags for MMDF messages are identical to those of mbox message and are as follows:

| Flag | Meaning  | Explanation                    |
|------|----------|--------------------------------|
| R    | Read     | Read                           |
| O    | Old      | Previously detected by MUA     |
| D    | Deleted  | Marked for subsequent deletion |
| F    | Flagged  | Marked as important            |
| A    | Answered | Replied to                     |

The "R" and "O" flags are stored in the `Status` header, and the "D", "F", and "A" flags are stored in the `X-Status` header. The flags and headers typically appear in the order mentioned.

`!MMDFMessage` instances offer the following methods, which are identical to those offered by `mboxMessage`:

<div class="method">

get_from()

Return a string representing the "From " line that marks the start of the message in an mbox mailbox. The leading "From " and the trailing newline are excluded.

</div>

<div class="method">

set_from([from](), [time]()=None)

Set the "From " line to *from\_*, which should be specified without a leading "From " or trailing newline. For convenience, *time\_* may be specified and will be formatted appropriately and appended to *from\_*. If *time\_* is specified, it should be a `time.struct_time` instance, a tuple suitable for passing to `time.strftime`, or `True` (to use `time.gmtime`).

</div>

<div class="method">

get_flags()

Return a string specifying the flags that are currently set. If the message complies with the conventional format, the result is the concatenation in the following order of zero or one occurrence of each of `'R'`, `'O'`, `'D'`, `'F'`, and `'A'`.

</div>

<div class="method">

set_flags(flags)

Set the flags specified by *flags* and unset all others. Parameter *flags* should be the concatenation in any order of zero or more occurrences of each of `'R'`, `'O'`, `'D'`, `'F'`, and `'A'`.

</div>

<div class="method">

add_flag(flag)

Set the flag(s) specified by *flag* without changing other flags. To add more than one flag at a time, *flag* may be a string of more than one character.

</div>

<div class="method">

remove_flag(flag)

Unset the flag(s) specified by *flag* without changing other flags. To remove more than one flag at a time, *flag* may be a string of more than one character.

</div>

</div>

When an `!MMDFMessage` instance is created based upon a `MaildirMessage` instance, a "From " line is generated based upon the `MaildirMessage` instance's delivery date, and the following conversions take place:

| Resulting state | `MaildirMessage` state |
|-----------------|------------------------|
| R flag          | S flag                 |
| O flag          | "cur" subdirectory     |
| D flag          | T flag                 |
| F flag          | F flag                 |
| A flag          | R flag                 |

When an `!MMDFMessage` instance is created based upon an `MHMessage` instance, the following conversions take place:

| Resulting state   | `MHMessage` state    |
|-------------------|----------------------|
| R flag and O flag | no "unseen" sequence |
| O flag            | "unseen" sequence    |
| F flag            | "flagged" sequence   |
| A flag            | "replied" sequence   |

When an `!MMDFMessage` instance is created based upon a `BabylMessage` instance, the following conversions take place:

| Resulting state   | `BabylMessage` state |
|-------------------|----------------------|
| R flag and O flag | no "unseen" label    |
| O flag            | "unseen" label       |
| D flag            | "deleted" label      |
| A flag            | "answered" label     |

When an `!MMDFMessage` instance is created based upon an `mboxMessage` instance, the "From " line is copied and all flags directly correspond:

| Resulting state | `mboxMessage` state |
|-----------------|---------------------|
| R flag          | R flag              |
| O flag          | O flag              |
| D flag          | D flag              |
| F flag          | F flag              |
| A flag          | A flag              |

## Exceptions

The following exception classes are defined in the `!mailbox` module:

<div class="exception">

Error()

The base class for all other module-specific exceptions.

</div>

<div class="exception">

NoSuchMailboxError()

Raised when a mailbox is expected but is not found, such as when instantiating a `Mailbox` subclass with a path that does not exist (and with the *create* parameter set to `False`), or when opening a folder that does not exist.

</div>

<div class="exception">

NotEmptyError()

Raised when a mailbox is not empty but is expected to be, such as when deleting a folder that contains messages.

</div>

<div class="exception">

ExternalClashError()

Raised when some mailbox-related condition beyond the control of the program causes it to be unable to proceed, such as when failing to acquire a lock that another program already holds, or when a uniquely generated file name already exists.

</div>

<div class="exception">

FormatError()

Raised when the data in a file cannot be parsed, such as when an `MH` instance attempts to read a corrupted `.mh_sequences` file.

</div>

## Examples

A simple example of printing the subjects of all messages in a mailbox that seem interesting:

    import mailbox
    for message in mailbox.mbox('~/mbox'):
        subject = message['subject']       # Could possibly be None.
        if subject and 'python' in subject.lower():
            print(subject)

To copy all mail from a Babyl mailbox to an MH mailbox, converting all of the format-specific information that can be converted:

    import mailbox
    destination = mailbox.MH('~/Mail')
    destination.lock()
    for message in mailbox.Babyl('~/RMAIL'):
        destination.add(mailbox.MHMessage(message))
    destination.flush()
    destination.unlock()

This example sorts mail from several mailing lists into different mailboxes, being careful to avoid mail corruption due to concurrent modification by other programs, mail loss due to interruption of the program, or premature termination due to malformed messages in the mailbox:

    import mailbox
    import email.errors

    list_names = ('python-list', 'python-dev', 'python-bugs')

    boxes = {name: mailbox.mbox('~/email/%s' % name) for name in list_names}
    inbox = mailbox.Maildir('~/Maildir', factory=None)

    for key in inbox.iterkeys():
        try:
            message = inbox[key]
        except email.errors.MessageParseError:
            continue                # The message is malformed. Just leave it.

        for name in list_names:
            list_id = message['list-id']
            if list_id and name in list_id:
                # Get mailbox to use
                box = boxes[name]

                # Write copy to disk before removing original.
                # If there's a crash, you might duplicate a message, but
                # that's better than losing a message completely.
                box.lock()
                box.add(message)
                box.flush()
                box.unlock()

                # Remove original message
                inbox.lock()
                inbox.discard(key)
                inbox.flush()
                inbox.unlock()
                break               # Found destination, so stop looking.

    for box in boxes.values():
        box.close()

# `!mailcap` --- Mailcap file handling

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

mailcap

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!mailcap` module was [Python 3.12](https://docs.python.org/3.12/library/mailcap.html).

# Structured Markup Processing Tools

Python supports a variety of modules to work with various forms of structured data markup. This includes modules to work with the Standard Generalized Markup Language (SGML) and the Hypertext Markup Language (HTML), and several interfaces for working with the Extensible Markup Language (XML).

<div class="toctree">

html.rst html.parser.rst html.entities.rst xml.rst xml.etree.elementtree.rst xml.dom.rst xml.dom.minidom.rst xml.dom.pulldom.rst xml.sax.rst xml.sax.handler.rst xml.sax.utils.rst xml.sax.reader.rst pyexpat.rst

</div>

# `!marshal` --- Internal Python object serialization

<div class="module" data-synopsis="Convert Python objects to streams of bytes and back (with different
constraints).">

marshal

</div>

------------------------------------------------------------------------

This module contains functions that can read and write Python values in a binary format. The format is specific to Python, but independent of machine architecture issues (e.g., you can write a Python value to a file on a PC, transport the file to a Mac, and read it back there). Details of the format are undocumented on purpose; it may change between Python versions (although it rarely does).[^1]

<div class="index">

pair: module; pickle pair: module; shelve

</div>

This is not a general "persistence" module. For general persistence and transfer of Python objects through RPC calls, see the modules `pickle` and `shelve`. The `!marshal` module exists mainly to support reading and writing the "pseudo-compiled" code for Python modules of `.pyc` files. Therefore, the Python maintainers reserve the right to modify the marshal format in backward incompatible ways should the need arise. The format of code objects is not compatible between Python versions, even if the version of the format is the same. De-serializing a code object in the incorrect Python version has undefined behavior. If you're serializing and de-serializing Python objects, use the `pickle` module instead -- the performance is comparable, version independence is guaranteed, and pickle supports a substantially wider range of objects than marshal.

> [!WARNING]
> The `!marshal` module is not intended to be secure against erroneous or maliciously constructed data. Never unmarshal data received from an untrusted or unauthenticated source.

There are functions that read/write files as well as functions operating on bytes-like objects.

<div class="index">

object; code, code object

</div>

Not all Python object types are supported; in general, only objects whose value is independent from a particular invocation of Python can be written and read by this module. The following types are supported:

- Numeric types: `int`, `bool`, `float`, `complex`.
- Strings (`str`) and `bytes`. `Bytes-like objects <bytes-like object>` like `bytearray` are marshalled as `!bytes`.
- Containers: `tuple`, `list`, `dict`, `frozendict` (since `version` 6), `set`, `frozenset`, and `slice` (since `version` 5). It should be understood that these are supported only if the values contained therein are themselves supported. Recursive containers are supported since `version` 3.
- The singletons `None`, `Ellipsis` and `StopIteration`.
- `code <types.CodeType>` objects, if *allow_code* is true. See note above about version dependence.

<div class="versionchanged">

3.4

- Added format version 3, which supports marshalling recursive lists, sets and dictionaries.
- Added format version 4, which supports efficient representations of short strings.

</div>

<div class="versionchanged">

3.14

Added format version 5, which allows marshalling slices.

</div>

<div class="versionchanged">

3.15

Added format version 6, which allows marshalling `frozendict`.

</div>

The module defines these functions:

<div class="function">

dump(value, file, version=version, /, \*, allow_code=True)

Write the value on the open file. The value must be a supported type. The file must be a writeable `binary file`.

If the value has (or contains an object that has) an unsupported type, a `ValueError` exception is raised --- but garbage data will also be written to the file. The object will not be properly read back by `load`. `Code objects <code-objects>` are only supported if *allow_code* is true.

The *version* argument indicates the data format that `dump` should use (see below).

<div class="audit-event">

marshal.dumps value,version marshal.dump

</div>

<div class="versionchanged">

3.13 Added the *allow_code* parameter.

</div>

</div>

<div class="function">

load(file, /, \*, allow_code=True)

Read one value from the open file and return it. If no valid value is read (e.g. because the data has a different Python version's incompatible marshal format), raise `EOFError`, `ValueError` or `TypeError`. `Code objects <code-objects>` are only supported if *allow_code* is true. The file must be a readable `binary file`.

<div class="audit-event">

marshal.load "" marshal.load

</div>

> [!NOTE]
> If an object containing an unsupported type was marshalled with `dump`, `load` will substitute `None` for the unmarshallable type.

<div class="versionchanged">

3.10

This call used to raise a `code.__new__` audit event for each code object. Now it raises a single `marshal.load` event for the entire load operation.

</div>

<div class="versionchanged">

3.13 Added the *allow_code* parameter.

</div>

</div>

<div class="function">

dumps(value, version=version, /, \*, allow_code=True)

Return the bytes object that would be written to a file by `dump(value, file)`. The value must be a supported type. Raise a `ValueError` exception if value has (or contains an object that has) an unsupported type. `Code objects <code-objects>` are only supported if *allow_code* is true.

The *version* argument indicates the data format that `dumps` should use (see below).

<div class="audit-event">

marshal.dumps value,version marshal.dump

</div>

<div class="versionchanged">

3.13 Added the *allow_code* parameter.

</div>

</div>

<div class="function">

loads(bytes, /, \*, allow_code=True)

Convert the `bytes-like object` to a value. If no valid value is found, raise `EOFError`, `ValueError` or `TypeError`. `Code objects <code-objects>` are only supported if *allow_code* is true. Extra bytes in the input are ignored.

<div class="audit-event">

marshal.loads bytes marshal.load

</div>

<div class="versionchanged">

3.10

This call used to raise a `code.__new__` audit event for each code object. Now it raises a single `marshal.loads` event for the entire load operation.

</div>

<div class="versionchanged">

3.13 Added the *allow_code* parameter.

</div>

</div>

In addition, the following constants are defined:

<div class="data">

version

Indicates the format that the module uses. Version 0 is the historical first version; subsequent versions add new features. Generally, a new version becomes the default when it is introduced.

| Version | Available since | New features                                |
|---------|-----------------|---------------------------------------------|
| 1       | Python 2.4      | Sharing interned strings                    |
| 2       | Python 2.5      | Binary representation of floats             |
| 3       | Python 3.4      | Support for object instancing and recursion |
| 4       | Python 3.4      | Efficient representation of short strings   |
| 5       | Python 3.14     | Support for `slice` objects                 |
| 6       | Python 3.15     | Support for `frozendict` objects            |

</div>

**Footnotes**

[^1]: The name of this module stems from a bit of terminology used by the designers of Modula-3 (amongst others), who use the term "marshalling" for shipping of data around in a self-contained form. Strictly speaking, "to marshal" means to convert some data from internal to external form (in an RPC buffer for instance) and "unmarshalling" for the reverse process.

# `!math.integer` --- integer-specific mathematics functions

<div class="module" data-synopsis="Integer-specific mathematics functions.">

math.integer

</div>

<div class="versionadded">

3.15

</div>

------------------------------------------------------------------------

This module provides access to the mathematical functions defined for integer arguments. These functions accept integers and objects that implement the `~object.__index__` method which is used to convert the object to an integer number.

The following functions are provided by this module. All return values are computed exactly and are integers.

<div class="function">

comb(n, k, /)

Return the number of ways to choose *k* items from *n* items without repetition and without order.

Evaluates to `n! / (k! * (n - k)!)` when `k <= n` and evaluates to zero when `k > n`.

Also called the binomial coefficient because it is equivalent to the coefficient of k-th term in polynomial expansion of `(1 + x)ⁿ`.

Raises `ValueError` if either of the arguments are negative.

</div>

<div class="function">

factorial(n, /)

Return factorial of the nonnegative integer *n*.

</div>

<div class="function">

gcd(\*integers)

Return the greatest common divisor of the specified integer arguments. If any of the arguments is nonzero, then the returned value is the largest positive integer that is a divisor of all arguments. If all arguments are zero, then the returned value is `0`. `gcd()` without arguments returns `0`.

</div>

<div class="function">

isqrt(n, /)

Return the integer square root of the nonnegative integer *n*. This is the floor of the exact square root of *n*, or equivalently the greatest integer *a* such that *a*²   ≤   *n*.

For some applications, it may be more convenient to have the least integer *a* such that *n*   ≤   *a*², or in other words the ceiling of the exact square root of *n*. For positive *n*, this can be computed using `a = 1 + isqrt(n - 1)`.

</div>

<div class="function">

lcm(\*integers)

Return the least common multiple of the specified integer arguments. If all arguments are nonzero, then the returned value is the smallest positive integer that is a multiple of all arguments. If any of the arguments is zero, then the returned value is `0`. `lcm()` without arguments returns `1`.

</div>

<div class="function">

perm(n, k=None, /)

Return the number of ways to choose *k* items from *n* items without repetition and with order.

Evaluates to `n! / (n - k)!` when `k <= n` and evaluates to zero when `k > n`.

If *k* is not specified or is `None`, then *k* defaults to *n* and the function returns `n!`.

Raises `ValueError` if either of the arguments are negative.

</div>

# `!math` --- Mathematical functions

<div class="module" data-synopsis="Mathematical functions (sin() etc.).">

math

</div>

<div class="testsetup">

from math import fsum

</div>

------------------------------------------------------------------------

This module provides access to common mathematical functions and constants, including those defined by the C standard.

These functions cannot be used with complex numbers; use the functions of the same name from the `cmath` module if you require support for complex numbers. The distinction between functions which support complex numbers and those which don't is made since most users do not want to learn quite as much mathematics as required to understand complex numbers. Receiving an exception instead of a complex result allows earlier detection of the unexpected complex number used as a parameter, so that the programmer can determine how and why it was generated in the first place.

The following functions are provided by this module. Except when explicitly noted otherwise, all return values are floats.

<table>
<tbody>
<tr>
<td colspan="2"><strong>Floating point arithmetic</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">ceil(x) &lt;ceil&gt;</code></td>
<td>Ceiling of <em>x</em>, the smallest integer greater than or equal to <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">fabs(x) &lt;fabs&gt;</code></td>
<td>Absolute value of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">floor(x)  &lt;floor&gt;</code></td>
<td>Floor of <em>x</em>, the largest integer less than or equal to <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">fma(x, y, z) &lt;fma&gt;</code></td>
<td>Fused multiply-add operation: <code>(x * y) + z</code></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">fmax(x, y) &lt;fmax&gt;</code></td>
<td>Maximum of two floating-point values</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">fmin(x, y) &lt;fmin&gt;</code></td>
<td>Minimum of two floating-point values</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">fmod(x, y) &lt;fmod&gt;</code></td>
<td>Remainder of division <code>x / y</code></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">modf(x) &lt;modf&gt;</code></td>
<td>Fractional and integer parts of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">remainder(x, y) &lt;remainder&gt;</code></td>
<td>Remainder of <em>x</em> with respect to <em>y</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">trunc(x) &lt;trunc&gt;</code></td>
<td>Integer part of <em>x</em></td>
</tr>
<tr>
<td colspan="2"><strong>Floating point manipulation functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">copysign(x, y) &lt;copysign&gt;</code></td>
<td>Magnitude (absolute value) of <em>x</em> with the sign of <em>y</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">frexp(x) &lt;frexp&gt;</code></td>
<td>Mantissa and exponent of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isclose(a, b, rel_tol, abs_tol) &lt;isclose&gt;</code></td>
<td>Check if the values <em>a</em> and <em>b</em> are close to each other</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isfinite(x) &lt;isfinite&gt;</code></td>
<td>Check if <em>x</em> is neither an infinity nor a NaN</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isnormal(x) &lt;isnormal&gt;</code></td>
<td>Check if <em>x</em> is a normal number</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">issubnormal(x) &lt;issubnormal&gt;</code></td>
<td>Check if <em>x</em> is a subnormal number</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isinf(x) &lt;isinf&gt;</code></td>
<td>Check if <em>x</em> is a positive or negative infinity</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isnan(x) &lt;isnan&gt;</code></td>
<td>Check if <em>x</em> is a NaN (not a number)</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">ldexp(x, i) &lt;ldexp&gt;</code></td>
<td><code>x * (2**i)</code>, inverse of function <code class="interpreted-text" role="func">frexp</code></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">nextafter(x, y, steps) &lt;nextafter&gt;</code></td>
<td>Floating-point value <em>steps</em> steps after <em>x</em> towards <em>y</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">signbit(x) &lt;signbit&gt;</code></td>
<td>Check if <em>x</em> is a negative number</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">ulp(x) &lt;ulp&gt;</code></td>
<td>Value of the least significant bit of <em>x</em></td>
</tr>
<tr>
<td colspan="2"><strong>Power, exponential and logarithmic functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">cbrt(x) &lt;cbrt&gt;</code></td>
<td>Cube root of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">exp(x) &lt;exp&gt;</code></td>
<td><em>e</em> raised to the power <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">exp2(x) &lt;exp2&gt;</code></td>
<td><em>2</em> raised to the power <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">expm1(x) &lt;expm1&gt;</code></td>
<td><em>e</em> raised to the power <em>x</em>, minus 1</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">log(x, base) &lt;log&gt;</code></td>
<td>Logarithm of <em>x</em> to the given base (<em>e</em> by default)</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">log1p(x) &lt;log1p&gt;</code></td>
<td>Natural logarithm of <em>1+x</em> (base <em>e</em>)</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">log2(x) &lt;log2&gt;</code></td>
<td>Base-2 logarithm of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">log10(x) &lt;log10&gt;</code></td>
<td>Base-10 logarithm of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">pow(x, y) &lt;math.pow&gt;</code></td>
<td><em>x</em> raised to the power <em>y</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">sqrt(x) &lt;sqrt&gt;</code></td>
<td>Square root of <em>x</em></td>
</tr>
<tr>
<td colspan="2"><strong>Summation and product functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">dist(p, q) &lt;dist&gt;</code></td>
<td>Euclidean distance between two points <em>p</em> and <em>q</em> given as an iterable of coordinates</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">fsum(iterable) &lt;fsum&gt;</code></td>
<td>Sum of values in the input <em>iterable</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">hypot(*coordinates) &lt;hypot&gt;</code></td>
<td>Euclidean norm of an iterable of coordinates</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">prod(iterable, start) &lt;prod&gt;</code></td>
<td>Product of elements in the input <em>iterable</em> with a <em>start</em> value</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">sumprod(p, q) &lt;sumprod&gt;</code></td>
<td>Sum of products from two iterables <em>p</em> and <em>q</em></td>
</tr>
<tr>
<td colspan="2"><strong>Angular conversion</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">degrees(x) &lt;degrees&gt;</code></td>
<td>Convert angle <em>x</em> from radians to degrees</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">radians(x) &lt;radians&gt;</code></td>
<td>Convert angle <em>x</em> from degrees to radians</td>
</tr>
<tr>
<td colspan="2"><strong>Trigonometric functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">acos(x) &lt;acos&gt;</code></td>
<td>Arc cosine of <em>x</em>, in radians</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">acospi(x) &lt;acospi&gt;</code></td>
<td>Arc cosine of <em>x</em>, in half-turns</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">asin(x) &lt;asin&gt;</code></td>
<td>Arc sine of <em>x</em>, in radians</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">asinpi(x) &lt;asinpi&gt;</code></td>
<td>Arc sine of <em>x</em>, in half-turns</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">atan(x) &lt;atan&gt;</code></td>
<td>Arc tangent of <em>x</em>, in radians</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">atanpi(x) &lt;atanpi&gt;</code></td>
<td>Arc tangent of <em>x</em>, in half-turns</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">atan2(y, x) &lt;atan2&gt;</code></td>
<td><code>atan(y / x)</code>, in radians</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">atan2pi(y, x) &lt;atan2pi&gt;</code></td>
<td><code>atanpi(y / x)</code>, in half-turns</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">cos(x) &lt;cos&gt;</code></td>
<td>Cosine of <em>x</em> radians</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">cospi(x) &lt;cospi&gt;</code></td>
<td>Cosine of <em>x⋅π</em> radians</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">sin(x) &lt;sin&gt;</code></td>
<td>Sine of <em>x</em> radians</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">sinpi(x) &lt;sinpi&gt;</code></td>
<td>Sine of <em>x⋅π</em> radians</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">tan(x) &lt;tan&gt;</code></td>
<td>Tangent of <em>x</em> radians</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">tanpi(x) &lt;tanpi&gt;</code></td>
<td>Tangent of <em>x⋅π</em> radians</td>
</tr>
<tr>
<td colspan="2"><strong>Hyperbolic functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">acosh(x) &lt;acosh&gt;</code></td>
<td>Inverse hyperbolic cosine of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">asinh(x) &lt;asinh&gt;</code></td>
<td>Inverse hyperbolic sine of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">atanh(x) &lt;atanh&gt;</code></td>
<td>Inverse hyperbolic tangent of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">cosh(x) &lt;cosh&gt;</code></td>
<td>Hyperbolic cosine of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">sinh(x) &lt;sinh&gt;</code></td>
<td>Hyperbolic sine of <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">tanh(x) &lt;tanh&gt;</code></td>
<td>Hyperbolic tangent of <em>x</em></td>
</tr>
<tr>
<td colspan="2"><strong>Special functions</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">erf(x) &lt;erf&gt;</code></td>
<td><a href="https://en.wikipedia.org/wiki/Error_function">Error function</a> at <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">erfc(x) &lt;erfc&gt;</code></td>
<td><a href="https://en.wikipedia.org/wiki/Error_function">Complementary error function</a> at <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">gamma(x) &lt;gamma&gt;</code></td>
<td><a href="https://en.wikipedia.org/wiki/Gamma_function">Gamma function</a> at <em>x</em></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">lgamma(x) &lt;lgamma&gt;</code></td>
<td>Natural logarithm of the absolute value of the <a href="https://en.wikipedia.org/wiki/Gamma_function">Gamma function</a> at <em>x</em></td>
</tr>
<tr>
<td colspan="2"><strong>Constants</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">pi</code></td>
<td><em>π</em> = 3.141592...</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">e</code></td>
<td><em>e</em> = 2.718281...</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">tau</code></td>
<td><em>τ</em> = 2<em>π</em> = 6.283185...</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">inf</code></td>
<td>Positive infinity</td>
</tr>
<tr>
<td><code class="interpreted-text" role="data">nan</code></td>
<td>"Not a number" (NaN)</td>
</tr>
</tbody>
</table>

## Floating point arithmetic

<div class="function">

ceil(x)

Return the ceiling of *x*, the smallest integer greater than or equal to *x*. If *x* is not a float, delegates to `x.__ceil__ <object.__ceil__>`, which should return an `~numbers.Integral` value.

</div>

<div class="function">

fabs(x)

Return the absolute value of *x*.

</div>

<div class="function">

floor(x)

Return the floor of *x*, the largest integer less than or equal to *x*. If *x* is not a float, delegates to `x.__floor__ <object.__floor__>`, which should return an `~numbers.Integral` value.

</div>

<div class="function">

fma(x, y, z)

Fused multiply-add operation. Return `(x * y) + z`, computed as though with infinite precision and range followed by a single round to the `float` format. This operation often provides better accuracy than the direct expression `(x * y) + z`.

This function follows the specification of the fusedMultiplyAdd operation described in the IEEE 754 standard. The standard leaves one case implementation-defined, namely the result of `fma(0, inf, nan)` and `fma(inf, 0, nan)`. In these cases, `math.fma` returns a NaN, and does not raise any exception.

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

fmax(x, y)

Get the larger of two floating-point values, treating NaNs as missing data.

When both operands are (signed) NaNs or zeroes, return `nan` and `0` respectively and the sign of the result is implementation-defined, that is, `!fmax` is not required to be sensitive to the sign of such operands (see Annex F of the C11 standard, §F.10.0.3 and §F.10.9.2).

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

fmin(x, y)

Get the smaller of two floating-point values, treating NaNs as missing data.

When both operands are (signed) NaNs or zeroes, return `nan` and `0` respectively and the sign of the result is implementation-defined, that is, `!fmin` is not required to be sensitive to the sign of such operands (see Annex F of the C11 standard, §F.10.0.3 and §F.10.9.3).

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

fmod(x, y)

Return the floating-point remainder of `x / y`, as defined by the platform C library function `fmod(x, y)`. Note that the Python expression `x % y` may not return the same result. The intent of the C standard is that `fmod(x, y)` be exactly (mathematically; to infinite precision) equal to `x - n*y` for some integer *n* such that the result has the same sign as *x* and magnitude less than `abs(y)`. Python's `x % y` returns a result with the sign of *y* instead, and may not be exactly computable for float arguments. For example, `fmod(-1e-100, 1e100)` is `-1e-100`, but the result of Python's `-1e-100 % 1e100` is `1e100-1e-100`, which cannot be represented exactly as a float, and rounds to the surprising `1e100`. For this reason, function `fmod` is generally preferred when working with floats, while Python's `x % y` is preferred when working with integers.

</div>

<div class="function">

modf(x)

Return the fractional and integer parts of *x*. Both results carry the sign of *x* and are floats.

Note that `modf` has a different call/return pattern than its C equivalents: it takes a single argument and return a pair of values, rather than returning its second return value through an 'output parameter' (there is no such thing in Python).

</div>

<div class="function">

remainder(x, y)

Return the IEEE 754-style remainder of *x* with respect to *y*. For finite *x* and finite nonzero *y*, this is the difference `x - n*y`, where `n` is the closest integer to the exact value of the quotient `x / y`. If `x / y` is exactly halfway between two consecutive integers, the nearest *even* integer is used for `n`. The remainder `r = remainder(x, y)` thus always satisfies `abs(r) <= 0.5 * abs(y)`.

Special cases follow IEEE 754: in particular, `remainder(x, math.inf)` is *x* for any finite *x*, and `remainder(x, 0)` and `remainder(math.inf, x)` raise `ValueError` for any non-NaN *x*. If the result of the remainder operation is zero, that zero will have the same sign as *x*.

On platforms using IEEE 754 binary floating point, the result of this operation is always exactly representable: no rounding error is introduced.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

trunc(x)

Return *x* with the fractional part removed, leaving the integer part. This rounds toward 0: `trunc()` is equivalent to `floor` for positive *x*, and equivalent to `ceil` for negative *x*. If *x* is not a float, delegates to `x.__trunc__
<object.__trunc__>`, which should return an `~numbers.Integral` value.

</div>

For the `ceil`, `floor`, and `modf` functions, note that *all* floating-point numbers of sufficiently large magnitude are exact integers. Python floats typically carry no more than 53 bits of precision (the same as the platform C double type), in which case any float *x* with `abs(x) >= 2**52` necessarily has no fractional bits.

## Floating point manipulation functions

<div class="function">

copysign(x, y)

Return a float with the magnitude (absolute value) of *x* but the sign of *y*. On platforms that support signed zeros, `copysign(1.0, -0.0)` returns *-1.0*.

</div>

<div class="function">

frexp(x)

Return the mantissa and exponent of *x* as the pair `(m, e)`. If *x* is a finite nonzero number, then *m* is a float with `0.5 <= abs(m) < 1.0` and an integer *e* is such that `x == m * 2**e` exactly. Else, return `(x, 0)`. This is used to "pick apart" the internal representation of a float in a portable way.

Note that `frexp` has a different call/return pattern than its C equivalents: it takes a single argument and return a pair of values, rather than returning its second return value through an 'output parameter' (there is no such thing in Python).

</div>

<div class="function">

isclose(a, b, \*, rel_tol=1e-09, abs_tol=0.0)

Return `True` if the values *a* and *b* are close to each other and `False` otherwise.

Whether or not two values are considered close is determined according to given absolute and relative tolerances. If no errors occur, the result will be: `abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)`.

*rel_tol* is the relative tolerance -- it is the maximum allowed difference between *a* and *b*, relative to the larger absolute value of *a* or *b*. For example, to set a tolerance of 5%, pass `rel_tol=0.05`. The default tolerance is `1e-09`, which assures that the two values are the same within about 9 decimal digits. *rel_tol* must be nonnegative and less than `1.0`.

*abs_tol* is the absolute tolerance; it defaults to `0.0` and it must be nonnegative. When comparing `x` to `0.0`, `isclose(x, 0)` is computed as `abs(x) <= rel_tol  * abs(x)`, which is `False` for any nonzero `x` and *rel_tol* less than `1.0`. So add an appropriate positive *abs_tol* argument to the call.

The IEEE 754 special values of `NaN`, `inf`, and `-inf` will be handled according to IEEE rules. Specifically, `NaN` is not considered close to any other value, including `NaN`. `inf` and `-inf` are only considered close to themselves.

<div class="versionadded">

3.5

</div>

<div class="seealso">

`485` -- A function for testing approximate equality

</div>

</div>

<div class="function">

isfinite(x)

Return `True` if *x* is neither an infinity nor a NaN, and `False` otherwise. (Note that `0.0` *is* considered finite.)

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

isnormal(x)

Return `True` if *x* is a normal number, that is a finite nonzero number that is not a subnormal (see `issubnormal`). Return `False` otherwise.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

issubnormal(x)

Return `True` if *x* is a subnormal number, that is a finite nonzero number with a magnitude smaller than `sys.float_info.min`. Return `False` otherwise.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

isinf(x)

Return `True` if *x* is a positive or negative infinity, and `False` otherwise.

</div>

<div class="function">

isnan(x)

Return `True` if *x* is a NaN (not a number), and `False` otherwise.

</div>

<div class="function">

ldexp(x, i)

Return `x * (2**i)`. This is essentially the inverse of function `frexp`.

</div>

<div class="function">

nextafter(x, y, steps=1)

Return the floating-point value *steps* steps after *x* towards *y*.

If *x* is equal to *y*, return *y*, unless *steps* is zero.

Examples:

- `math.nextafter(x, math.inf)` goes up: towards positive infinity.
- `math.nextafter(x, -math.inf)` goes down: towards minus infinity.
- `math.nextafter(x, 0.0)` goes towards zero.
- `math.nextafter(x, math.copysign(math.inf, x))` goes away from zero.

See also `math.ulp`.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.12 Added the *steps* argument.

</div>

</div>

<div class="function">

signbit(x)

Return `True` if the sign of *x* is negative and `False` otherwise.

This is useful to detect the sign bit of zeroes, infinities and NaNs.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

ulp(x)

Return the value of the least significant bit of the float *x*:

- If *x* is a NaN (not a number), return *x*.
- If *x* is negative, return `ulp(-x)`.
- If *x* is a positive infinity, return *x*.
- If *x* is equal to zero, return the smallest positive *denormalized* representable float (smaller than the minimum positive *normalized* float, `sys.float_info.min <sys.float_info>`).
- If *x* is equal to the largest positive representable float, return the value of the least significant bit of *x*, such that the first float smaller than *x* is `x - ulp(x)`.
- Otherwise (*x* is a positive finite number), return the value of the least significant bit of *x*, such that the first float bigger than *x* is `x + ulp(x)`.

ULP stands for "Unit in the Last Place".

See also `math.nextafter` and `sys.float_info.epsilon
<sys.float_info>`.

<div class="versionadded">

3.9

</div>

</div>

## Power, exponential and logarithmic functions

<div class="function">

cbrt(x)

Return the cube root of *x*.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

exp(x)

Return *e* raised to the power *x*, where *e* = 2.718281... is the base of natural logarithms. This is usually more accurate than `math.e ** x` or `pow(math.e, x)`.

</div>

<div class="function">

exp2(x)

Return *2* raised to the power *x*.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

expm1(x)

Return *e* raised to the power *x*, minus 1. Here *e* is the base of natural logarithms. For small floats *x*, the subtraction in `exp(x) - 1` can result in a [significant loss of precision](https://en.wikipedia.org/wiki/Loss_of_significance); the `expm1` function provides a way to compute this quantity to full precision:

> \>\>\> from math import exp, expm1 \>\>\> exp(1e-5) - 1 \# gives result accurate to 11 places 1.0000050000069649e-05 \>\>\> expm1(1e-5) \# result accurate to full precision 1.0000050000166668e-05

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

log(x\[, base\])

With one argument, return the natural logarithm of *x* (to base *e*).

With two arguments, return the logarithm of *x* to the given *base*, calculated as `log(x)/log(base)`.

</div>

<div class="function">

log1p(x)

Return the natural logarithm of *1+x* (base *e*). The result is calculated in a way which is accurate for *x* near zero.

</div>

<div class="function">

log2(x)

Return the base-2 logarithm of *x*. This is usually more accurate than `log(x, 2)`.

<div class="versionadded">

3.3

</div>

<div class="seealso">

`int.bit_length` returns the number of bits necessary to represent an integer in binary, excluding the sign and leading zeros.

</div>

</div>

<div class="function">

log10(x)

Return the base-10 logarithm of *x*. This is usually more accurate than `log(x, 10)`.

</div>

<div class="function">

pow(x, y)

Return *x* raised to the power *y*. Exceptional cases follow the IEEE 754 standard as far as possible. In particular, `pow(1.0, x)` and `pow(x, 0.0)` always return `1.0`, even when *x* is a zero or a NaN. If both *x* and *y* are finite, *x* is negative, and *y* is not an integer then `pow(x, y)` is undefined, and raises `ValueError`.

Unlike the built-in `**` operator, `math.pow` converts both its arguments to type `float`. Use `**` or the built-in `pow` function for computing exact integer powers.

<div class="versionchanged">

3.11 The special cases `pow(0.0, -inf)` and `pow(-0.0, -inf)` were changed to return `inf` instead of raising `ValueError`, for consistency with IEEE 754.

</div>

</div>

<div class="function">

sqrt(x)

Return the square root of *x*.

</div>

## Summation and product functions

<div class="function">

dist(p, q)

Return the Euclidean distance between two points *p* and *q*, each given as a sequence (or iterable) of coordinates. The two points must have the same dimension.

Roughly equivalent to:

    sqrt(sum((px - qx) ** 2.0 for px, qx in zip(p, q, strict=True)))

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

fsum(iterable)

Return an accurate floating-point sum of values in the iterable. Avoids loss of precision by tracking multiple intermediate partial sums.

The algorithm's accuracy depends on IEEE-754 arithmetic guarantees and the typical case where the rounding mode is half-even. On some non-Windows builds, the underlying C library uses extended precision addition and may occasionally double-round an intermediate sum causing it to be off in its least significant bit.

For further discussion and two alternative approaches, see the [ASPN cookbook recipes for accurate floating-point summation](https://code.activestate.com/recipes/393090-binary-floating-point-summation-accurate-to-full-p/).

</div>

<div class="function">

hypot(\*coordinates)

Return the Euclidean norm, `sqrt(sum(x**2 for x in coordinates))`. This is the length of the vector from the origin to the point given by the coordinates.

For a two dimensional point `(x, y)`, this is equivalent to computing the hypotenuse of a right triangle using the Pythagorean theorem, `sqrt(x*x + y*y)`.

<div class="versionchanged">

3.8 Added support for n-dimensional points. Formerly, only the two dimensional case was supported.

</div>

<div class="versionchanged">

3.10 Improved the algorithm's accuracy so that the maximum error is under 1 ulp (unit in the last place). More typically, the result is almost always correctly rounded to within 1/2 ulp.

</div>

</div>

<div class="function">

prod(iterable, \*, start=1)

Calculate the product of all the elements in the input *iterable*. The default *start* value for the product is `1`.

When the iterable is empty, return the start value. This function is intended specifically for use with numeric values and may reject non-numeric types.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

sumprod(p, q)

Return the sum of products of values from two iterables *p* and *q*.

Raises `ValueError` if the inputs do not have the same length.

Roughly equivalent to:

    sum(map(operator.mul, p, q, strict=True))

For float and mixed int/float inputs, the intermediate products and sums are computed with extended precision.

<div class="versionadded">

3.12

</div>

</div>

## Angular conversion

<div class="function">

degrees(x)

Convert angle *x* from radians to degrees.

</div>

<div class="function">

radians(x)

Convert angle *x* from degrees to radians.

</div>

## Trigonometric functions

<div class="function">

acos(x)

Return the arc cosine of *x*, in radians. The result is between `0` and `pi`.

</div>

<div class="function">

acospi(x)

Return the arc cosine of *x*, in half-turns. The result is between `0` and `1`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

asin(x)

Return the arc sine of *x*, in radians. The result is between `-pi/2` and `pi/2`.

</div>

<div class="function">

asinpi(x)

Return the arc sine of *x*, in half-turns. The result is between `-0.5` and `0.5`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

atan(x)

Return the arc tangent of *x*, in radians. The result is between `-pi/2` and `pi/2`.

</div>

<div class="function">

atanpi(x)

Return the arc tangent of *x*, in half-turns. The result is between `-0.5` and `0.5`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

atan2(y, x)

Return `atan(y / x)`, in radians. The result is between `-pi` and `pi`. The vector in the plane from the origin to point `(x, y)` makes this angle with the positive X axis. The point of `atan2` is that the signs of both inputs are known to it, so it can compute the correct quadrant for the angle. For example, `atan(1)` and `atan2(1, 1)` are both `pi/4`, but `atan2(-1, -1)` is `-3*pi/4`.

</div>

<div class="function">

atan2pi(y, x)

Return `atanpi(y / x)`, in half-turns. The result is between `-1` and `1`. The vector in the plane from the origin to point `(x, y)` makes this angle with the positive X axis. The point of `atan2pi` is that the signs of both inputs are known to it, so it can compute the correct quadrant for the angle. For example, `atanpi(1)` and `atan2pi(1, 1)` are both `0.25`, but `atan2pi(-1, -1)` is `-0.75`.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

cos(x)

Return the cosine of *x* radians.

</div>

<div class="function">

cospi(x)

Return the cosine of *x* half-turns (*x⋅π* radians).

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

sin(x)

Return the sine of *x* radians.

</div>

<div class="function">

sinpi(x)

Return the sine of *x* half-turns (*x⋅π* radians).

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

tan(x)

Return the tangent of *x* radians.

</div>

<div class="function">

tanpi(x)

Return the tangent of *x* half-turns (*x⋅π* radians).

<div class="versionadded">

3.16

</div>

</div>

## Hyperbolic functions

[Hyperbolic functions](https://en.wikipedia.org/wiki/Hyperbolic_functions) are analogs of trigonometric functions that are based on hyperbolas instead of circles.

<div class="function">

acosh(x)

Return the inverse hyperbolic cosine of *x*.

</div>

<div class="function">

asinh(x)

Return the inverse hyperbolic sine of *x*.

</div>

<div class="function">

atanh(x)

Return the inverse hyperbolic tangent of *x*.

</div>

<div class="function">

cosh(x)

Return the hyperbolic cosine of *x*.

</div>

<div class="function">

sinh(x)

Return the hyperbolic sine of *x*.

</div>

<div class="function">

tanh(x)

Return the hyperbolic tangent of *x*.

</div>

## Special functions

<div class="function">

erf(x)

Return the [error function](https://en.wikipedia.org/wiki/Error_function) at *x*.

The `erf` function can be used to compute traditional statistical functions such as the [cumulative standard normal distribution](https://en.wikipedia.org/wiki/Cumulative_distribution_function):

    def phi(x):
        'Cumulative distribution function for the standard normal distribution'
        return (1.0 + erf(x / sqrt(2.0))) / 2.0

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

erfc(x)

Return the complementary error function at *x*. The [complementary error function](https://en.wikipedia.org/wiki/Error_function) is defined as `1.0 - erf(x)`. It is used for large values of *x* where a subtraction from one would cause a [loss of significance](https://en.wikipedia.org/wiki/Loss_of_significance).

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

gamma(x)

Return the [Gamma function](https://en.wikipedia.org/wiki/Gamma_function) at *x*.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

lgamma(x)

Return the natural logarithm of the absolute value of the Gamma function at *x*.

<div class="versionadded">

3.2

</div>

</div>

## Number-theoretic functions

For backward compatibility, the `!math` module provides also aliases of the following functions from the `math.integer` module:

<table>
<tbody>
<tr>
<td><div class="function" data-no-typesetting="">
<p>comb(n, k)</p>
</div>
<p><code class="interpreted-text" role="func">comb(n, k) &lt;math.integer.comb&gt;</code></p></td>
<td><p>Number of ways to choose <em>k</em> items from <em>n</em> items without repetition and without order</p></td>
</tr>
<tr>
<td><div class="function" data-no-typesetting="">
<p>factorial(n)</p>
</div>
<p><code class="interpreted-text" role="func">factorial(n) &lt;math.integer.factorial&gt;</code></p></td>
<td><p><em>n</em> factorial</p></td>
</tr>
<tr>
<td><div class="function" data-no-typesetting="">
<p>gcd(*integers)</p>
</div>
<p><code class="interpreted-text" role="func">gcd(*integers) &lt;math.integer.gcd&gt;</code></p></td>
<td><p>Greatest common divisor of the integer arguments</p></td>
</tr>
<tr>
<td><div class="function" data-no-typesetting="">
<p>isqrt(n)</p>
</div>
<p><code class="interpreted-text" role="func">isqrt(n) &lt;math.integer.isqrt&gt;</code></p></td>
<td><p>Integer square root of a nonnegative integer <em>n</em></p></td>
</tr>
<tr>
<td><div class="function" data-no-typesetting="">
<p>lcm(*integers)</p>
</div>
<p><code class="interpreted-text" role="func">lcm(*integers) &lt;math.integer.lcm&gt;</code></p></td>
<td><p>Least common multiple of the integer arguments</p></td>
</tr>
<tr>
<td><div class="function" data-no-typesetting="">
<p>perm(n, k)</p>
</div>
<p><code class="interpreted-text" role="func">perm(n, k) &lt;math.integer.perm&gt;</code></p></td>
<td><p>Number of ways to choose <em>k</em> items from <em>n</em> items without repetition and with order</p></td>
</tr>
</tbody>
</table>

<div class="versionadded">

3.5 The `gcd` function.

</div>

<div class="versionadded">

3.8 The `comb`, `perm` and `isqrt` functions.

</div>

<div class="versionadded">

3.9 The `lcm` function.

</div>

<div class="versionchanged">

3.9 Added support for an arbitrary number of arguments in the `gcd` function. Formerly, only two arguments were supported.

</div>

<div class="versionchanged">

3.10 Floats with integral values (like `5.0`) are no longer accepted in the `factorial` function.

</div>

<div class="soft-deprecated">

3.15 Use the `math.integer` functions instead of these aliases.

</div>

## Constants

<div class="data">

pi

The mathematical constant *π* = 3.141592..., to available precision.

</div>

<div class="data">

e

The mathematical constant *e* = 2.718281..., to available precision.

</div>

<div class="data">

tau

The mathematical constant *τ* = 6.283185..., to available precision. Tau is a circle constant equal to 2*π*, the ratio of a circle's circumference to its radius. To learn more about Tau, check out Vi Hart's video [Pi is (still) Wrong](https://vimeo.com/147792667), and start celebrating [Tau day](https://tauday.com/) by eating twice as much pie!

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

inf

A floating-point positive infinity. (For negative infinity, use `-math.inf`.) Equivalent to the output of `float('inf')`.

<div class="versionadded">

3.5

</div>

</div>

<div class="data">

nan

A floating-point "not a number" (NaN) value. Equivalent to the output of `float('nan')`. Due to the requirements of the [IEEE-754 standard](https://en.wikipedia.org/wiki/IEEE_754), `math.nan` and `float('nan')` are not considered to equal to any other numeric value, including themselves. To check whether a number is a NaN, use the `isnan` function to test for NaNs instead of `is` or `==`. Example:

> \>\>\> import math \>\>\> math.nan == math.nan False \>\>\> float('nan') == float('nan') False \>\>\> math.isnan(math.nan) True \>\>\> math.isnan(float('nan')) True

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.11 It is now always available.

</div>

</div>

<div class="impl-detail">

The `!math` module consists mostly of thin wrappers around the platform C math library functions. Behavior in exceptional cases follows Annex F of the C99 standard where appropriate. The current implementation will raise `ValueError` for invalid operations like `sqrt(-1.0)` or `log(0.0)` (where C99 Annex F recommends signaling invalid operation or divide-by-zero), and `OverflowError` for results that overflow (for example, `exp(1000.0)`). A NaN will not be returned from any of the functions above unless one or more of the input arguments was a NaN; in that case, most functions will return a NaN, but (again following C99 Annex F) there are some exceptions to this rule, for example `pow(float('nan'), 0.0)` or `hypot(float('nan'), float('inf'))`.

Note that Python makes no effort to distinguish signaling NaNs from quiet NaNs, and behavior for signaling NaNs remains unspecified. Typical behavior is to treat all NaNs as though they were quiet.

</div>

<div class="seealso">

Module `cmath`  
Complex number versions of many of these functions.

Module `math.integer`  
Integer-specific mathematics functions.

</div>

# `!mimetypes` --- Map filenames to MIME types

<div class="module" data-synopsis="Mapping of filename extensions to MIME types.">

mimetypes

</div>

**Source code:** `Lib/mimetypes.py`

<div class="index">

pair: MIME; content type

</div>

------------------------------------------------------------------------

The `!mimetypes` module converts between a filename or URL and the MIME type associated with the filename extension. Conversions are provided from filename to MIME type and from MIME type to filename extension; encodings are not supported for the latter conversion.

The module provides one class and a number of convenience functions. The functions are the normal interface to this module, but some applications may be interested in the class as well.

The functions described below provide the primary interface for this module. If the module has not been initialized, they will call `init` if they rely on the information `init` sets up.

<div class="function">

guess_type(url, strict=True)

<div class="index">

pair: MIME; headers

</div>

Guess the type of a file based on its filename, path or URL, given by *url*. URL can be a string or a `path-like object`.

The return value is a tuple `(type, encoding)` where *type* is `None` if the type can't be guessed (missing or unknown suffix) or a string of the form `'type/subtype'`, usable for a MIME `content-type` header.

*encoding* is `None` for no encoding or the name of the program used to encode (e.g. `compress` or `gzip`). The encoding is suitable for use as a `Content-Encoding` header, **not** as a `Content-Transfer-Encoding` header. The mappings are table driven. Encoding suffixes are case-sensitive. Suffix mappings and type suffixes are first tried case-sensitively, then case-insensitively.

The optional *strict* argument is a flag specifying whether the list of known MIME types is limited to only the official types [registered with IANA](https://www.iana.org/assignments/media-types/media-types.xhtml). However, the behavior of this module also depends on the underlying operating system. Only file types recognized by the OS or explicitly registered with Python's internal database can be identified. When *strict* is `True` (the default), only the IANA types are supported; when *strict* is `False`, some additional non-standard but commonly used MIME types are also recognized.

<div class="versionchanged">

3.8 Added support for *url* being a `path-like object`.

</div>

<div class="soft-deprecated">

3.13 Passing a file path instead of URL. Use `guess_file_type` for this.

</div>

</div>

<div class="function">

guess_file_type(path, \*, strict=True)

<div class="index">

pair: MIME; headers

</div>

Guess the type of a file based on its path, given by *path*. Similar to the `guess_type` function, but accepts a path instead of URL. Path can be a string, a bytes object or a `path-like object`.

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

guess_all_extensions(type, strict=True)

Guess the extensions for a file based on its MIME type, given by *type*. The return value is a list of strings giving all possible filename extensions, including the leading dot (`'.'`). The extensions are not guaranteed to have been associated with any particular data stream, but would be mapped to the MIME type *type* by `guess_type` and `guess_file_type`.

The optional *strict* argument has the same meaning as with the `guess_type` function.

</div>

<div class="function">

guess_extension(type, strict=True)

Guess the extension for a file based on its MIME type, given by *type*. The return value is a string giving a filename extension, including the leading dot (`'.'`). The extension is not guaranteed to have been associated with any particular data stream, but would be mapped to the MIME type *type* by `guess_type` and `guess_file_type`. If no extension can be guessed for *type*, `None` is returned.

The optional *strict* argument has the same meaning as with the `guess_type` function.

</div>

Some additional functions and data items are available for controlling the behavior of the module.

<div class="function">

init(files=None)

Initialize the internal data structures. If given, *files* must be a sequence of file names which should be used to augment the default type map. If omitted, the file names to use are taken from `knownfiles`; on Windows, the current registry settings are loaded. Each file named in *files* or `knownfiles` takes precedence over those named before it. Calling `init` repeatedly is allowed.

Specifying an empty list for *files* will prevent the system defaults from being applied: only the well-known values will be present from a built-in list.

If *files* is `None` the internal data structure is completely rebuilt to its initial default value. This is a stable operation and will produce the same results when called multiple times.

<div class="versionchanged">

3.2 Previously, Windows registry settings were ignored.

</div>

</div>

<div class="function">

read_mime_types(file)

Load the type map given in the file named by *file*, if it exists. *file* must be a string specifying the name of the file to read. The type map is returned as a dictionary mapping file extensions, including the leading dot (`'.'`), to strings of the form `'type/subtype'`. If the file does not exist or cannot be read, `None` is returned.

</div>

<div class="function">

add_type(type, ext, strict=True)

Add a mapping from the MIME type *type* to the extension *ext*. When the extension is already known, the new type will replace the old one. When the type is already known the extension will be added to the list of known extensions. Valid extensions are empty or start with a `'.'`.

Registered lower-case extensions are matched case-insensitively.

When *strict* is `True` (the default), the mapping will be added to the official MIME types, otherwise to the non-standard ones.

<div class="deprecated">

3.14 *ext* values that do not start with `'.'` are deprecated.

</div>

<div class="versionchanged">

3.16 *ext* now must start with `'.'`. Otherwise `ValueError` is raised.

</div>

</div>

<div class="data">

inited

Flag indicating whether or not the global data structures have been initialized. This is set to `True` by `init`.

</div>

<div class="data">

knownfiles

<div class="index">

single: file; mime.types

</div>

List of type map file names commonly installed. These files are typically named `mime.types` and are installed in different locations by different packages.

</div>

<div class="data">

suffix_map

Dictionary mapping suffixes to suffixes. This is used to allow recognition of encoded files for which the encoding and the type are indicated by the same extension. For example, the `.tgz` extension is mapped to `.tar.gz` to allow the encoding and type to be recognized separately.

</div>

<div class="data">

encodings_map

Dictionary mapping filename extensions to encoding types.

</div>

<div class="data">

types_map

Dictionary mapping filename extensions to MIME types.

</div>

<div class="data">

common_types

Dictionary mapping filename extensions to non-standard, but commonly found MIME types.

</div>

An example usage of the module:

    >>> import mimetypes
    >>> mimetypes.init()
    >>> mimetypes.knownfiles
    ['/etc/mime.types', '/etc/httpd/mime.types', ... ]
    >>> mimetypes.suffix_map['.tgz']
    '.tar.gz'
    >>> mimetypes.encodings_map['.gz']
    'gzip'
    >>> mimetypes.types_map['.tgz']
    'application/x-tar-gz'

## MimeTypes objects

The `MimeTypes` class may be useful for applications which may want more than one MIME-type database; it provides an interface similar to the one of the `!mimetypes` module.

<div class="MimeTypes(filenames=(), strict=True)">

This class represents a MIME-types database. By default, it provides access to the same database as the rest of this module. The initial database is created from Python's built-in MIME type tables. It may be extended by loading additional `mime.types`-style files into the database using the `read` or `readfp` methods. The mapping dictionaries may also be cleared before loading additional data if the default data is not desired.

The optional *filenames* parameter can be used to cause additional files to be loaded "on top" of the default database.

<div class="attribute">

MimeTypes.suffix_map

Dictionary mapping suffixes to suffixes. This is used to allow recognition of encoded files for which the encoding and the type are indicated by the same extension. For example, the `.tgz` extension is mapped to `.tar.gz` to allow the encoding and type to be recognized separately. This is initialized with some predefined values.

</div>

<div class="attribute">

MimeTypes.encodings_map

Dictionary mapping filename extensions to encoding types. This is initialized with some predefined values.

</div>

<div class="attribute">

MimeTypes.types_map

Tuple containing two dictionaries, mapping filename extensions to MIME types: the first dictionary is for the non-standards types and the second one is for the standard types. They are initialized with some predefined values and MIME type information loaded from files specified by the *filenames* argument.

</div>

<div class="attribute">

MimeTypes.types_map_inv

Tuple containing two dictionaries, mapping MIME types to a list of filename extensions: the first dictionary is for the non-standards types and the second one is for the standard types. They are initialized with some predefined values and MIME type information loaded from files specified by the *filenames* argument.

</div>

<div class="method">

MimeTypes.guess_extension(type, strict=True)

Similar to the `guess_extension` function, using the tables stored as part of the object.

</div>

<div class="method">

MimeTypes.guess_type(url, strict=True)

Similar to the `guess_type` function, using the tables stored as part of the object.

</div>

<div class="method">

MimeTypes.guess_file_type(path, \*, strict=True)

Similar to the `guess_file_type` function, using the tables stored as part of the object.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

MimeTypes.guess_all_extensions(type, strict=True)

Similar to the `guess_all_extensions` function, using the tables stored as part of the object.

</div>

<div class="method">

MimeTypes.read(filename, strict=True)

Load MIME information from a file named *filename*. This uses `readfp` to parse the file.

If *strict* is `True`, information will be added to list of standard types, else to the list of non-standard types.

</div>

<div class="method">

MimeTypes.readfp(fp, strict=True)

Load MIME type information from an open file *fp*. The file must have the format of the standard `mime.types` files.

If *strict* is `True`, information will be added to the list of standard types, else to the list of non-standard types.

</div>

<div class="method">

MimeTypes.read_windows_registry(strict=True)

Load MIME type information from the Windows registry.

<div class="availability">

Windows.

</div>

If *strict* is `True`, information will be added to the list of standard types, else to the list of non-standard types.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

MimeTypes.add_type(type, ext, strict=True)

Add a mapping from the MIME type *type* to the extension *ext*. Valid extensions start with a '.' or are empty. When the extension is already known, the new type will replace the old one. When the type is already known the extension will be added to the list of known extensions.

Registered lower-case extensions are matched case-insensitively.

When *strict* is `True` (the default), the mapping will be added to the official MIME types, otherwise to the non-standard ones.

<div class="deprecated">

3.14 *ext* values that do not start with `'.'` are deprecated.

</div>

<div class="versionchanged">

3.16 *ext* now must start with `'.'`. Otherwise `ValueError` is raised.

</div>

</div>

</div>

## Command-line usage

The `!mimetypes` module can be executed as a script from the command line.

``` sh
python -m mimetypes [-h] [-e] [-l] type [type ...]
```

The following options are accepted:

<div class="program">

mimetypes

</div>

<div class="cmdoption">

`-h`  

`--help`  

Show the help message and exit.

</div>

<div class="cmdoption">

`-e`  

`--extension`  

Guess extension instead of type.

</div>

<div class="cmdoption">

`-l`  

`--lenient`  

Additionally search for some common, but non-standard types.

</div>

By default the script converts MIME types to file extensions. However, if `--extension` is specified, it converts file extensions to MIME types.

For each `type` entry, the script writes a line into the standard output stream. If an unknown type occurs, it writes an error message into the standard output stream and exits with the return code `1`.

## Command-line example

Here are some examples of typical usage of the `!mimetypes` command-line interface:

```
$ # get a MIME type by a file name
$ python -m mimetypes filename.png
type: image/png encoding: None

$ # get a MIME type by a URL
$ python -m mimetypes https://example.com/filename.txt
type: text/plain encoding: None

$ # get a complex MIME type
$ python -m mimetypes filename.tar.gz
type: application/x-tar encoding: gzip

$ # get a MIME type for a rare file extension
$ python -m mimetypes filename.pict
error: media type unknown for filename.pict

$ # now look in the extended database built into Python
$ python -m mimetypes --lenient filename.pict
type: image/pict encoding: None

$ # get a file extension by a MIME type
$ python -m mimetypes --extension text/javascript
.js

$ # get a file extension by a rare MIME type
$ python -m mimetypes --extension text/xul
error: unknown type text/xul

$ # now look in the extended database again
$ python -m mimetypes --extension --lenient text/xul
.xul

$ # try to feed an unknown file extension
$ python -m mimetypes filename.sh filename.nc filename.xxx filename.txt
type: application/x-sh encoding: None
type: application/x-netcdf encoding: None
error: media type unknown for filename.xxx
type: text/plain encoding: None

$ # try to feed an unknown MIME type
$ python -m mimetypes --extension audio/aac audio/opus audio/future audio/x-wav
.aac
.opus
error: unknown type audio/future
```

# Multimedia Services

The modules described in this chapter implement various algorithms or interfaces that are mainly useful for multimedia applications. They are available at the discretion of the installation. Here's an overview:

<div class="toctree">

wave.rst colorsys.rst

</div>

# `!mmap` --- Memory-mapped file support

<div class="module" data-synopsis="Interface to memory-mapped files for Unix and Windows.">

mmap

</div>

------------------------------------------------------------------------

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

Memory-mapped file objects behave like both `bytearray` and like `file objects <file object>`. You can use mmap objects in most places where `bytearray` are expected; for example, you can use the `re` module to search through a memory-mapped file. You can also change a single byte by doing `obj[index] = 97`, or change a subsequence by assigning to a slice: `obj[i1:i2] = b'...'`. You can also read and write data starting at the current file position, and `~mmap.mmap.seek` through the file to different positions.

A memory-mapped file is created by the `~mmap.mmap` constructor, which is different on Unix and on Windows. In either case you must provide a file descriptor for a file opened for update. If you wish to map an existing Python file object, use its `~io.IOBase.fileno` method to obtain the correct value for the *fileno* parameter. Otherwise, you can open the file using the `os.open` function, which returns a file descriptor directly (the file still needs to be closed when done).

> [!NOTE]
> If you want to create a memory-mapping for a writable, buffered file, you should `~io.IOBase.flush` the file first. This is necessary to ensure that local modifications to the buffers are actually available to the mapping.

For both the Unix and Windows versions of the constructor, *access* may be specified as an optional keyword parameter. *access* accepts one of four values: `ACCESS_READ`, `ACCESS_WRITE`, or `ACCESS_COPY` to specify read-only, write-through or copy-on-write memory respectively, or `ACCESS_DEFAULT` to defer to *prot*. *access* can be used on both Unix and Windows. If *access* is not specified, Windows mmap returns a write-through mapping. The initial memory values for all three access types are taken from the specified file. Assignment to an `ACCESS_READ` memory map raises a `TypeError` exception. Assignment to an `ACCESS_WRITE` memory map affects both memory and the underlying file. Assignment to an `ACCESS_COPY` memory map affects memory but does not update the underlying file.

<div class="versionchanged">

3.7 Added `ACCESS_DEFAULT` constant.

</div>

To map anonymous memory, -1 should be passed as the fileno along with the length.

<div class="mmap(fileno, length, tagname=None, \ access=ACCESS_DEFAULT, offset=0, *, trackfd=True)">

**(Windows version)** Maps *length* bytes from the file specified by the file descriptor *fileno*, and creates a mmap object. If *length* is larger than the current size of the file, the file is extended to contain *length* bytes. If *length* is `0`, the maximum length of the map is the current size of the file, except that if the file is empty Windows raises an exception (you cannot create an empty mapping on Windows).

*tagname*, if specified and not `None`, is a string giving a tag name for the mapping. Windows allows you to have many different mappings against the same file. If you specify the name of an existing tag, that tag is opened, otherwise a new tag of this name is created. If this parameter is omitted or `None`, the mapping is created without a name. Avoiding the use of the *tagname* parameter will assist in keeping your code portable between Unix and Windows.

*offset* may be specified as a non-negative integer offset. mmap references will be relative to the offset from the beginning of the file. *offset* defaults to 0. *offset* must be a multiple of the `ALLOCATIONGRANULARITY`.

If *trackfd* is `False`, the file handle corresponding to *fileno* will not be duplicated, and the resulting `!mmap` object will not be associated with the map's underlying file. This means that the `~mmap.mmap.size` and `~mmap.mmap.resize` methods will fail. This mode is useful to limit the number of open file handles. The original file can be renamed (but not deleted) after closing *fileno*.

<div class="versionchanged">

3.15 The *trackfd* parameter was added.

</div>

<div class="audit-event">

mmap.\_\_new\_\_ fileno,length,access,offset mmap.mmap

</div>

</div>

<div class="mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, \ access=ACCESS_DEFAULT, offset=0, *, trackfd=True)" data-noindex="">

**(Unix version)** Maps *length* bytes from the file specified by the file descriptor *fileno*, and returns a mmap object. If *length* is `0`, the maximum length of the map will be the current size of the file when `~mmap.mmap` is called.

*flags* specifies the nature of the mapping. `MAP_PRIVATE` creates a private copy-on-write mapping, so changes to the contents of the mmap object will be private to this process, and `MAP_SHARED` creates a mapping that's shared with all other processes mapping the same areas of the file. The default value is `MAP_SHARED`. Some systems have additional possible flags with the full list specified in `MAP_* constants <map-constants>`.

*prot*, if specified, gives the desired memory protection; the two most useful values are `PROT_READ` and `PROT_WRITE`, to specify that the pages may be read or written. *prot* defaults to `PROT_READ` \| `PROT_WRITE`.

*access* may be specified in lieu of *flags* and *prot* as an optional keyword parameter. It is an error to specify both *flags*, *prot* and *access*. See the description of *access* above for information on how to use this parameter.

*offset* may be specified as a non-negative integer offset. mmap references will be relative to the offset from the beginning of the file. *offset* defaults to 0. *offset* must be a multiple of `ALLOCATIONGRANULARITY` which is equal to `PAGESIZE` on Unix systems.

If *trackfd* is `False`, the file descriptor specified by *fileno* will not be duplicated, and the resulting `!mmap` object will not be associated with the map's underlying file. This means that the `~mmap.mmap.size` and `~mmap.mmap.resize` methods will fail. This mode is useful to limit the number of open file descriptors.

To ensure validity of the created memory mapping the file specified by the descriptor *fileno* is internally automatically synchronized with the physical backing store on macOS.

<div class="versionchanged">

3.13 The *trackfd* parameter was added.

</div>

This example shows a simple way of using `~mmap.mmap`:

    import mmap

    # write a simple example file
    with open("hello.txt", "wb") as f:
        f.write(b"Hello Python!\n")

    with open("hello.txt", "r+b") as f:
        # memory-map the file, size 0 means whole file
        mm = mmap.mmap(f.fileno(), 0)
        # read content via standard file methods
        print(mm.readline())  # prints b"Hello Python!\n"
        # read content via slice notation
        print(mm[:5])  # prints b"Hello"
        # update content using slice notation;
        # note that new content must have same size
        mm[6:] = b" world!\n"
        # ... and read again using standard file methods
        mm.seek(0)
        print(mm.readline())  # prints b"Hello  world!\n"
        # close the map
        mm.close()

`~mmap.mmap` can also be used as a context manager in a `with` statement:

    import mmap

    with mmap.mmap(-1, 13) as mm:
        mm.write(b"Hello world!")

<div class="versionadded">

3.2 Context manager support.

</div>

The next example demonstrates how to create an anonymous map and exchange data between the parent and child processes:

    import mmap
    import os

    mm = mmap.mmap(-1, 13)
    mm.write(b"Hello world!")

    pid = os.fork()

    if pid == 0:  # In a child process
        mm.seek(0)
        print(mm.readline())

        mm.close()

<div class="audit-event">

mmap.\_\_new\_\_ fileno,length,access,offset mmap.mmap

</div>

Memory-mapped file objects support the following methods:

<div class="method">

close()

Closes the mmap. Subsequent calls to other methods of the object will result in a ValueError exception being raised. This will not close the open file.

</div>

<div class="attribute">

closed

`True` if the file is closed.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

find(sub\[, start\[, end\]\])

Returns the lowest index in the object where the subsequence *sub* is found, such that *sub* is contained in the range \[*start*, *end*\]. Optional arguments *start* and *end* are interpreted as in slice notation. Returns `-1` on failure.

<div class="versionchanged">

3.5 Writable `bytes-like object` is now accepted.

</div>

</div>

<div class="method">

flush(\[offset\[, size\]\], \*, flags=MS_SYNC)

Flushes changes made to the in-memory copy of a file back to disk. Without use of this call there is no guarantee that changes are written back before the object is destroyed. If *offset* and *size* are specified, only changes to the given range of bytes will be flushed to disk; otherwise, the whole extent of the mapping is flushed. *offset* must be a multiple of the `PAGESIZE` or `ALLOCATIONGRANULARITY`.

The *flags* parameter specifies the synchronization behavior. *flags* must be one of the `MS_* constants <ms-constants>` available on the system.

On Windows, the *flags* parameter is ignored.

`None` is returned to indicate success. An exception is raised when the call failed.

<div class="versionchanged">

3.8 Previously, a nonzero value was returned on success; zero was returned on error under Windows. A zero value was returned on success; an exception was raised on error under Unix.

</div>

<div class="versionchanged">

3.15 Allow specifying *offset* without *size*. Previously, both *offset* and *size* parameters were required together. Now *offset* can be specified alone, and the flush operation will extend from *offset* to the end of the mmap.

</div>

<div class="versionchanged">

3.15 Added *flags* parameter to control synchronization behavior.

</div>

</div>

<div class="method">

madvise(option\[, start\[, length\]\])

Send advice *option* to the kernel about the memory region beginning at *start* and extending *length* bytes. *option* must be one of the `MADV_* constants <madvise-constants>` available on the system. If *start* and *length* are omitted, the entire mapping is spanned. On some systems (including Linux), *start* must be a multiple of the `PAGESIZE`.

Availability: Systems with the `madvise()` system call.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

move(dest, src, count)

Copy the *count* bytes starting at offset *src* to the destination index *dest*. If the mmap was created with `ACCESS_READ`, then calls to move will raise a `TypeError` exception.

</div>

<div class="method">

read(\[n\])

Return a `bytes` containing up to *n* bytes starting from the current file position. If the argument is omitted, `None` or negative, return all bytes from the current file position to the end of the mapping. The file position is updated to point after the bytes that were returned.

<div class="versionchanged">

3.3 Argument can be omitted or `None`.

</div>

</div>

<div class="method">

read_byte()

Returns a byte at the current file position as an integer, and advances the file position by 1.

</div>

<div class="method">

readline()

Returns a single line, starting at the current file position and up to the next newline. The file position is updated to point after the bytes that were returned.

</div>

<div class="method">

resize(newsize)

Resizes the map and the underlying file, if any.

Resizing a map created with *access* of `ACCESS_READ` or `ACCESS_COPY`, will raise a `TypeError` exception. Resizing a map created with *trackfd* set to `False`, will raise a `ValueError` exception.

**On Windows**: Resizing the map will raise an `OSError` if there are other maps against the same named file. Resizing an anonymous map (ie against the pagefile) will silently create a new map with the original data copied over up to the length of the new size.

Availability: Windows and systems with the `mremap()` system call.

<div class="versionchanged">

3.11 Correctly fails if attempting to resize when another map is held Allows resize against an anonymous map on Windows

</div>

</div>

<div class="method">

rfind(sub\[, start\[, end\]\])

Returns the highest index in the object where the subsequence *sub* is found, such that *sub* is contained in the range \[*start*, *end*\]. Optional arguments *start* and *end* are interpreted as in slice notation. Returns `-1` on failure.

<div class="versionchanged">

3.5 Writable `bytes-like object` is now accepted.

</div>

</div>

<div class="method">

seek(pos\[, whence\])

Set the file's current position. *whence* argument is optional and defaults to `os.SEEK_SET` or `0` (absolute file positioning); other values are `os.SEEK_CUR` or `1` (seek relative to the current position) and `os.SEEK_END` or `2` (seek relative to the file's end).

<div class="versionchanged">

3.13 Return the new absolute position instead of `None`.

</div>

</div>

<div class="method">

seekable()

Return whether the file supports seeking, and the return value is always `True`.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

set_name(name, /)

Annotate the memory mapping with the given *name* for easier identification in `/proc/<pid>/maps` if the kernel supports the feature and `-X dev <-X>` is passed to Python or if Python is built in `debug mode <debug-build>`. The length of *name* must not exceed 67 bytes including the `'\0'` terminator.

<div class="availability">

Linux \>= 5.17 (kernel built with `CONFIG_ANON_VMA_NAME` option)

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

size()

Return the length of the file, which can be larger than the size of the memory-mapped area. For an anonymous mapping, return its size.

<div class="versionchanged">

3.15 Anonymous mappings are now supported on Unix.

</div>

</div>

<div class="method">

tell()

Returns the current position of the file pointer.

</div>

<div class="method">

write(bytes)

Write the bytes in *bytes* into memory at the current position of the file pointer and return the number of bytes written (never less than `len(bytes)`, since if the write fails, a `ValueError` will be raised). The file position is updated to point after the bytes that were written. If the mmap was created with `ACCESS_READ`, then writing to it will raise a `TypeError` exception.

<div class="versionchanged">

3.5 Writable `bytes-like object` is now accepted.

</div>

<div class="versionchanged">

3.6 The number of bytes written is now returned.

</div>

</div>

<div class="method">

write_byte(byte)

Write the integer *byte* into memory at the current position of the file pointer; the file position is advanced by `1`. If the mmap was created with `ACCESS_READ`, then writing to it will raise a `TypeError` exception.

</div>

</div>

## [ACCESS]()\* constants

<div class="data">

ACCESS_DEFAULT ACCESS_READ ACCESS_WRITE ACCESS_COPY

These constants specify the access mode for a memory mapping. `ACCESS_READ` provides read-only access, `ACCESS_WRITE` provides write-through access, and `ACCESS_COPY` provides copy-on-write access. `ACCESS_DEFAULT` defers to the *prot* argument of `~mmap.mmap`.

</div>

## Page size constants

<div class="data">

PAGESIZE ALLOCATIONGRANULARITY

`PAGESIZE` is the system memory page size in bytes. `ALLOCATIONGRANULARITY` is the minimum alignment in bytes required for the *offset* argument of `~mmap.mmap`. On Unix, `ALLOCATIONGRANULARITY` is equal to `PAGESIZE`.

</div>

## [PROT]()\* constants

<div class="data">

PROT_READ PROT_WRITE PROT_EXEC

Memory protection flags accepted by the *prot* argument of `~mmap.mmap`. `PROT_READ` allows pages to be read, `PROT_WRITE` allows pages to be written, and `PROT_EXEC` allows pages to be executed. Values can be combined using the bitwise OR operator. Not every option is present on every system.

<div class="availability">

Unix.

</div>

</div>

## [MADV]()\* constants

<div class="data">

MADV_NORMAL MADV_RANDOM MADV_SEQUENTIAL MADV_WILLNEED MADV_DONTNEED MADV_REMOVE MADV_DONTFORK MADV_DOFORK MADV_HWPOISON MADV_MERGEABLE MADV_UNMERGEABLE MADV_SOFT_OFFLINE MADV_HUGEPAGE MADV_NOHUGEPAGE MADV_DONTDUMP MADV_DODUMP MADV_FREE MADV_NOSYNC MADV_AUTOSYNC MADV_NOCORE MADV_CORE MADV_PROTECT MADV_FREE_REUSABLE MADV_FREE_REUSE

These options can be passed to `mmap.madvise`. Not every option will be present on every system.

Availability: Systems with the madvise() system call.

<div class="versionadded">

3.8

</div>

</div>

## [MAP]()\* constants

<div class="data">

MAP_SHARED MAP_PRIVATE MAP_32BIT MAP_ALIGNED_SUPER MAP_ANON MAP_ANONYMOUS MAP_CONCEAL MAP_DENYWRITE MAP_EXECUTABLE MAP_HASSEMAPHORE MAP_JIT MAP_NOCACHE MAP_NOEXTEND MAP_NORESERVE MAP_POPULATE MAP_RESILIENT_CODESIGN MAP_RESILIENT_MEDIA MAP_STACK MAP_TPRO MAP_TRANSLATED_ALLOW_EXECUTE MAP_UNIX03

These are the various flags that can be passed to `mmap.mmap`. `MAP_ALIGNED_SUPER` is only available at FreeBSD and `MAP_CONCEAL` is only available at OpenBSD. Note that some options might not be present on some systems.

<div class="versionchanged">

3.10 Added `MAP_POPULATE` constant.

</div>

<div class="versionadded">

3.11 Added `MAP_STACK` constant.

</div>

<div class="versionadded">

3.12 Added `MAP_ALIGNED_SUPER` and `MAP_CONCEAL` constants.

</div>

<div class="versionadded">

3.13 Added `MAP_32BIT`, `MAP_HASSEMAPHORE`, `MAP_JIT`, `MAP_NOCACHE`, `MAP_NOEXTEND`, `MAP_NORESERVE`, `MAP_RESILIENT_CODESIGN`, `MAP_RESILIENT_MEDIA`, `MAP_TPRO`, `MAP_TRANSLATED_ALLOW_EXECUTE`, and `MAP_UNIX03` constants.

</div>

</div>

## [MS]()\* constants

<div class="data">

MS_SYNC MS_ASYNC MS_INVALIDATE

These flags control the synchronization behavior for `mmap.flush`:

- `MS_SYNC` - Synchronous flush: writes are scheduled and the call blocks until they are physically written to storage.
- `MS_ASYNC` - Asynchronous flush: writes are scheduled but the call returns immediately without waiting for completion.
- `MS_INVALIDATE` - Invalidate cached data: invalidates other mappings of the same file so they can see the changes.

<div class="versionadded">

3.15

</div>

</div>

# `!modulefinder` --- Find modules used by a script

<div class="module" data-synopsis="Find modules used by a script.">

modulefinder

</div>

**Source code:** `Lib/modulefinder.py`

------------------------------------------------------------------------

This module provides a `ModuleFinder` class that can be used to determine the set of modules imported by a script. `modulefinder.py` can also be run as a script, giving the filename of a Python script as its argument, after which a report of the imported modules will be printed.

<div class="function">

AddPackagePath(pkg_name, path)

Record that the package named *pkg_name* can be found in the specified *path*.

</div>

<div class="function">

ReplacePackage(oldname, newname)

Allows specifying that the module named *oldname* is in fact the package named *newname*.

</div>

<div class="ModuleFinder(path=None, debug=0, excludes=[], replace_paths=[])">

This class provides `run_script` and `report` methods to determine the set of modules imported by a script. *path* can be a list of directories to search for modules; if not specified, `sys.path` is used. *debug* sets the debugging level; higher values make the class print debugging messages about what it's doing. *excludes* is a list of module names to exclude from the analysis. *replace_paths* is a list of `(oldpath, newpath)` tuples that will be replaced in module paths.

<div class="method">

report()

Print a report to standard output that lists the modules imported by the script and their paths, as well as modules that are missing or seem to be missing.

</div>

<div class="method">

run_script(pathname)

Analyze the contents of the *pathname* file, which must contain Python code.

</div>

<div class="attribute">

modules

A dictionary mapping module names to modules. See `modulefinder-example`.

</div>

</div>

## Example usage of `ModuleFinder`

The script that is going to get analyzed later on (bacon.py):

    import re, itertools

    try:
        import baconhameggs
    except ImportError:
        pass

    try:
        import guido.python.ham
    except ImportError:
        pass

The script that will output the report of bacon.py:

    from modulefinder import ModuleFinder

    finder = ModuleFinder()
    finder.run_script('bacon.py')

    print('Loaded modules:')
    for name, mod in finder.modules.items():
        print('%s: ' % name, end='')
        print(','.join(list(mod.globalnames.keys())[:3]))

    print('-'*50)
    print('Modules not imported:')
    print('\n'.join(finder.badmodules.keys()))

Sample output (may vary depending on the architecture):

    Loaded modules:
    _types:
    copyreg:  _inverted_registry,_slotnames,__all__
    re._compiler:  isstring,_sre,_optimize_unicode
    _sre:
    re._constants:  REPEAT_ONE,makedict,AT_END_LINE
    sys:
    re:  __module__,finditer,_expand
    itertools:
    __main__:  re,itertools,baconhameggs
    re._parser:  _PATTERNENDERS,SRE_FLAG_UNICODE
    array:
    types:  __module__,IntType,TypeType
    ---------------------------------------------------
    Modules not imported:
    guido.python.ham
    baconhameggs

# Importing Modules

The modules described in this chapter provide new ways to import other Python modules and hooks for customizing the import process.

The full list of modules described in this chapter is:

<div class="toctree">

zipimport.rst pkgutil.rst modulefinder.rst runpy.rst importlib.rst importlib.resources.rst importlib.resources.abc.rst importlib.metadata.rst sys_path_init.rst

</div>

# `!msilib` --- Read and write Microsoft Installer files

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

msilib

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!msilib` module was [Python 3.12](https://docs.python.org/3.12/library/msilib.html).

# `!msvcrt` --- Useful routines from the MS VC++ runtime

<div class="module" data-synopsis="Miscellaneous useful routines from the MS VC++ runtime.">

msvcrt

</div>

**Source code:** `PC/msvcrtmodule.c`

------------------------------------------------------------------------

These functions provide access to some useful capabilities on Windows platforms. Some higher-level modules use these functions to build the Windows implementations of their services. For example, the `getpass` module uses this in the implementation of the `~getpass.getpass` function.

Further documentation on these functions can be found in the Platform API documentation.

The module implements both the normal and wide char variants of the console I/O api. The normal API deals only with ASCII characters and is of limited use for internationalized applications. The wide char API should be used where ever possible.

<div class="availability">

Windows.

</div>

<div class="versionchanged">

3.3 Operations in this module now raise `OSError` where `IOError` was raised.

</div>

## File Operations

<div class="function">

locking(fd, mode, nbytes)

Lock part of a file based on file descriptor *fd* from the C runtime. Raises `OSError` on failure. The locked region of the file extends from the current file position for *nbytes* bytes, and may continue beyond the end of the file. *mode* must be one of the `!LK_\*` constants listed below. Multiple regions in a file may be locked at the same time, but may not overlap. Adjacent regions are not merged; they must be unlocked individually.

<div class="audit-event">

msvcrt.locking fd,mode,nbytes msvcrt.locking

</div>

</div>

<div class="data">

LK_LOCK LK_RLCK

Locks the specified bytes. If the bytes cannot be locked, the program immediately tries again after 1 second. If, after 10 attempts, the bytes cannot be locked, `OSError` is raised.

</div>

<div class="data">

LK_NBLCK LK_NBRLCK

Locks the specified bytes. If the bytes cannot be locked, `OSError` is raised.

</div>

<div class="data">

LK_UNLCK

Unlocks the specified bytes, which must have been previously locked.

</div>

<div class="function">

setmode(fd, flags)

Set the line-end translation mode for the file descriptor *fd*. To set it to text mode, *flags* should be `os.O_TEXT`; for binary, it should be `os.O_BINARY`.

</div>

<div class="function">

open_osfhandle(handle, flags)

Create a C runtime file descriptor from the file handle *handle*. The *flags* parameter should be a bitwise OR of `os.O_APPEND`, `os.O_RDONLY`, `os.O_TEXT` and `os.O_NOINHERIT`. The returned file descriptor may be used as a parameter to `os.fdopen` to create a file object.

The file descriptor is inheritable by default. Pass `os.O_NOINHERIT` flag to make it non inheritable.

<div class="audit-event">

msvcrt.open_osfhandle handle,flags msvcrt.open_osfhandle

</div>

</div>

<div class="function">

get_osfhandle(fd)

Return the file handle for the file descriptor *fd*. Raises `OSError` if *fd* is not recognized.

<div class="audit-event">

msvcrt.get_osfhandle fd msvcrt.get_osfhandle

</div>

</div>

## Console I/O

<div class="function">

kbhit()

Returns a nonzero value if a keypress is waiting to be read. Otherwise, return 0.

</div>

<div class="function">

getch()

Read a keypress and return the resulting character as a byte string. Nothing is echoed to the console. This call will block if a keypress is not already available, but will not wait for `Enter` to be pressed. If the pressed key was a special function key, this will return `'\000'` or `'\xe0'`; the next call will return the keycode. The `Control-C` keypress cannot be read with this function.

</div>

<div class="function">

getwch()

Wide char variant of `getch`, returning a Unicode value.

</div>

<div class="function">

getche()

Similar to `getch`, but the keypress will be echoed if it represents a printable character.

</div>

<div class="function">

getwche()

Wide char variant of `getche`, returning a Unicode value.

</div>

<div class="function">

putch(char)

Print the byte string *char* to the console without buffering. Raises `OSError` on failure, for example when the process has no console attached.

<div class="versionchanged">

3.16 Failures are now reported by raising `OSError` instead of being silently ignored.

</div>

</div>

<div class="function">

putwch(unicode_char)

Wide char variant of `putch`, accepting a Unicode value.

<div class="versionchanged">

3.16 Failures are now reported by raising `OSError` instead of being silently ignored.

</div>

</div>

<div class="function">

ungetch(char)

Cause the byte string *char* to be "pushed back" into the console buffer; it will be the next character read by `getch` or `getche`.

</div>

<div class="function">

ungetwch(unicode_char)

Wide char variant of `ungetch`, accepting a Unicode value.

</div>

## Other Functions

<div class="function">

heapmin()

Force the `malloc` heap to clean itself up and return unused blocks to the operating system. On failure, this raises `OSError`.

</div>

<div class="function">

set_error_mode(mode)

Changes the location where the C runtime writes an error message for an error that might end the program. *mode* must be one of the `!OUT_\*` constants listed below or `REPORT_ERRMODE`. Returns the old setting or -1 if an error occurs. Only available in `debug build of Python <debug-build>`.

</div>

<div class="data">

OUT_TO_DEFAULT

Error sink is determined by the app's type. Only available in `debug build of Python <debug-build>`.

</div>

<div class="data">

OUT_TO_STDERR

Error sink is a standard error. Only available in `debug build of Python <debug-build>`.

</div>

<div class="data">

OUT_TO_MSGBOX

Error sink is a message box. Only available in `debug build of Python <debug-build>`.

</div>

<div class="data">

REPORT_ERRMODE

Report the current error mode value. Only available in `debug build of Python <debug-build>`.

</div>

<div class="function">

CrtSetReportMode(type, mode)

Specifies the destination or destinations for a specific report type generated by `!_CrtDbgReport` in the MS VC++ runtime. *type* must be one of the `!CRT_\*` constants listed below. *mode* must be one of the `!CRTDBG_\*` constants listed below. Only available in `debug build of Python <debug-build>`.

</div>

<div class="function">

CrtSetReportFile(type, file)

After you use `CrtSetReportMode` to specify `CRTDBG_MODE_FILE`, you can specify the file handle to receive the message text. *type* must be one of the `!CRT_\*` constants listed below. *file* should be the file handle your want specified. Only available in `debug build of Python <debug-build>`.

</div>

<div class="data">

CRT_WARN

Warnings, messages, and information that doesn't need immediate attention.

</div>

<div class="data">

CRT_ERROR

Errors, unrecoverable problems, and issues that require immediate attention.

</div>

<div class="data">

CRT_ASSERT

Assertion failures.

</div>

<div class="data">

CRTDBG_MODE_DEBUG

Writes the message to the debugger's output window.

</div>

<div class="data">

CRTDBG_MODE_FILE

Writes the message to a user-supplied file handle. `CrtSetReportFile` should be called to define the specific file or stream to use as the destination.

</div>

<div class="data">

CRTDBG_MODE_WNDW

Creates a message box to display the message along with the `Abort`, `Retry`, and `Ignore` buttons.

</div>

<div class="data">

CRTDBG_REPORT_MODE

Returns current *mode* for the specified *type*.

</div>

<div class="data">

CRT_ASSEMBLY_VERSION

The CRT Assembly version, from the `crtassem.h` header file.

</div>

<div class="data">

VC_ASSEMBLY_PUBLICKEYTOKEN

The VC Assembly public key token, from the `crtassem.h` header file.

</div>

<div class="data">

LIBRARIES_ASSEMBLY_NAME_PREFIX

The Libraries Assembly name prefix, from the `crtassem.h` header file.

</div>

# `!multiprocessing` --- Process-based parallelism

<div class="module" data-synopsis="Process-based parallelism.">

multiprocessing

</div>

**Source code:** `Lib/multiprocessing/`

------------------------------------------------------------------------

<div class="availability">

not Android, not iOS, not WASI.

This module is not supported on `mobile platforms <mobile-availability>` or `WebAssembly platforms <wasm-availability>`.

</div>

## Introduction

`!multiprocessing` is a package that supports spawning processes using an API similar to the `threading` module. The `!multiprocessing` package offers both local and remote concurrency, effectively side-stepping the `Global Interpreter Lock <global interpreter lock>` by using subprocesses instead of threads. Due to this, the `!multiprocessing` module allows the programmer to fully leverage multiple processors on a given machine. It runs on both POSIX and Windows.

The `!multiprocessing` module also introduces the `~multiprocessing.pool.Pool` object which offers a convenient means of parallelizing the execution of a function across multiple input values, distributing the input data across processes (data parallelism). The following example demonstrates the common practice of defining such functions in a module so that child processes can successfully import that module. This basic example of data parallelism using `~multiprocessing.pool.Pool`, :

    from multiprocessing import Pool

    def f(x):
        return x*x

    if __name__ == '__main__':
        with Pool(5) as p:
            print(p.map(f, [1, 2, 3]))

will print to standard output :

    [1, 4, 9]

The `!multiprocessing` module also introduces APIs which do not have analogs in the `threading` module, like the ability to `terminate
<Process.terminate>`, `interrupt <Process.interrupt>` or `kill
<Process.kill>` a running process.

<div class="seealso">

`concurrent.futures.ProcessPoolExecutor` offers a higher level interface to push tasks to a background process without blocking execution of the calling process. Compared to using the `~multiprocessing.pool.Pool` interface directly, the `concurrent.futures` API more readily allows the submission of work to the underlying process pool to be separated from waiting for the results.

</div>

### The `Process` class

In `!multiprocessing`, processes are spawned by creating a `Process` object and then calling its `~Process.start` method. `Process` follows the API of `threading.Thread`. A trivial example of a multiprocess program is :

    from multiprocessing import Process

    def f(name):
        print('hello', name)

    if __name__ == '__main__':
        p = Process(target=f, args=('bob',))
        p.start()
        p.join()

To show the individual process IDs involved, here is an expanded example:

    from multiprocessing import Process
    import os

    def info(title):
        print(title)
        print('module name:', __name__)
        print('parent process:', os.getppid())
        print('process id:', os.getpid())

    def f(name):
        info('function f')
        print('hello', name)

    if __name__ == '__main__':
        info('main line')
        p = Process(target=f, args=('bob',))
        p.start()
        p.join()

For an explanation of why the `if __name__ == '__main__'` part is necessary, see `multiprocessing-programming`.

The arguments to `Process` usually need to be picklable so they can be passed to the child process. If you tried typing the above example directly into a REPL it could lead to an `AttributeError` in the child process trying to locate the *f* function in the `__main__` module.

### Contexts and start methods

Depending on the platform, `!multiprocessing` supports three ways to start a process. These *start methods* are

> <div id="multiprocessing-start-method-spawn">
>
> *spawn*  
> The parent process starts a fresh Python interpreter process. The child process will only inherit those resources necessary to run the process object's `~Process.run` method. In particular, unnecessary file descriptors and handles from the parent process will not be inherited. Starting a process using this method is rather slow compared to using *fork* or *forkserver*.
>
> Available on POSIX and Windows platforms. The default on Windows and macOS.
>
> </div>
>
> <div id="multiprocessing-start-method-fork">
>
> *fork*  
> The parent process uses `os.fork` to fork the Python interpreter. The child process, when it begins, is effectively identical to the parent process. All resources of the parent are inherited by the child process. Note that safely forking a multithreaded process is problematic.
>
> Available on POSIX systems.
>
> <div class="versionchanged">
>
> 3.14 This is no longer the default start method on any platform. Code that requires *fork* must explicitly specify that via `get_context` or `set_start_method`.
>
> </div>
>
> <div class="versionchanged">
>
> 3.12 If Python is able to detect that your process has multiple threads, the `os.fork` function that this start method calls internally will raise a `DeprecationWarning`. Use a different start method. See the `os.fork` documentation for further explanation.
>
> </div>
>
> </div>
>
> <div id="multiprocessing-start-method-forkserver">
>
> *forkserver*  
> When the program starts and selects the *forkserver* start method, a server process is spawned. From then on, whenever a new process is needed, the parent process connects to the server and requests that it fork a new process. The fork server process is single threaded unless system libraries or preloaded imports spawn threads as a side-effect so it is generally safe for it to use `os.fork`. No unnecessary resources are inherited.
>
> Available on POSIX platforms which support passing file descriptors over Unix pipes such as Linux. The default on those, unless there is no writable temporary directory (see `tempfile.gettempdir`), which the fork server requires; *spawn* is the default in that case.
>
> <div class="versionchanged">
>
> 3.14 This became the default start method on POSIX platforms.
>
> </div>
>
> <div class="versionchanged">
>
> 3.16 *spawn* is now the default start method when there is no writable temporary directory, for example on a read-only file system. Previously *forkserver* remained the default there and starting a process failed with an `OSError`. The default is determined when it is first needed rather than when `!multiprocessing` is imported. See `155717`.
>
> </div>
>
> </div>

<div class="versionchanged">

3.4 *spawn* added on all POSIX platforms, and *forkserver* added for some POSIX platforms. Child processes no longer inherit all of the parents inheritable handles on Windows.

</div>

<div class="versionchanged">

3.8

On macOS, the *spawn* start method is now the default. The *fork* start method should be considered unsafe as it can lead to crashes of the subprocess as macOS system libraries may start threads. See `33725`.

</div>

<div class="versionchanged">

3.14

On POSIX platforms the default start method was changed from *fork* to *forkserver* to retain the performance but avoid common multithreaded process incompatibilities. See `84559`.

</div>

On POSIX using the *spawn* or *forkserver* start methods will also start a *resource tracker* process which tracks the unlinked named system resources (such as named semaphores or `~multiprocessing.shared_memory.SharedMemory` objects) created by processes of the program. When all processes have exited the resource tracker unlinks any remaining tracked object. Usually there should be none, but if a process was killed by a signal there may be some "leaked" resources. (Neither leaked semaphores nor shared memory segments will be automatically unlinked until the next reboot. This is problematic for both objects because the system allows only a limited number of named semaphores, and shared memory segments occupy some space in the main memory.)

To select a start method you use the `set_start_method` in the `if __name__ == '__main__'` clause of the main module. For example:

    import multiprocessing as mp

    def foo(q):
        q.put('hello')

    if __name__ == '__main__':
        mp.set_start_method('spawn')
        q = mp.Queue()
        p = mp.Process(target=foo, args=(q,))
        p.start()
        print(q.get())
        p.join()

`set_start_method` should not be used more than once in the program.

Alternatively, you can use `get_context` to obtain a context object. Context objects have the same API as the multiprocessing module, and allow one to use multiple start methods in the same program. :

    import multiprocessing as mp

    def foo(q):
        q.put('hello')

    if __name__ == '__main__':
        ctx = mp.get_context('spawn')
        q = ctx.Queue()
        p = ctx.Process(target=foo, args=(q,))
        p.start()
        print(q.get())
        p.join()

Note that objects related to one context may not be compatible with processes for a different context. In particular, locks created using the *fork* context cannot be passed to processes started using the *spawn* or *forkserver* start methods.

Libraries using `!multiprocessing` or `~concurrent.futures.ProcessPoolExecutor` should be designed to allow their users to provide their own multiprocessing context. Using a specific context of your own within a library can lead to incompatibilities with the rest of the library user's application. Always document if your library requires a specific start method.

> [!WARNING]
> The `'spawn'` and `'forkserver'` start methods generally cannot be used with "frozen" executables (i.e., binaries produced by packages like **PyInstaller** and **cx_Freeze**) on POSIX systems. The `'fork'` start method may work if code does not use threads.

### Exchanging objects between processes

`!multiprocessing` supports two types of communication channel between processes:

**Queues**

> The `Queue` class is a near clone of `queue.Queue`. For example:
>
>     from multiprocessing import Process, Queue
>
>     def f(q):
>         q.put([42, None, 'hello'])
>
>     if __name__ == '__main__':
>         q = Queue()
>         p = Process(target=f, args=(q,))
>         p.start()
>         print(q.get())    # prints "[42, None, 'hello']"
>         p.join()
>
> Queues are thread and process safe. Any object put into a `!multiprocessing` queue will be serialized.

**Pipes**

> The `Pipe` function returns a pair of connection objects connected by a pipe which by default is duplex (two-way). For example:
>
>     from multiprocessing import Process, Pipe
>
>     def f(conn):
>         conn.send([42, None, 'hello'])
>         conn.close()
>
>     if __name__ == '__main__':
>         parent_conn, child_conn = Pipe()
>         p = Process(target=f, args=(child_conn,))
>         p.start()
>         print(parent_conn.recv())   # prints "[42, None, 'hello']"
>         p.join()
>
> The two connection objects returned by `Pipe` represent the two ends of the pipe. Each connection object has `~Connection.send` and `~Connection.recv` methods (among others). Note that data in a pipe may become corrupted if two processes (or threads) try to read from or write to the *same* end of the pipe at the same time. Of course there is no risk of corruption from processes using different ends of the pipe at the same time.
>
> The `~Connection.send` method serializes the object and `~Connection.recv` re-creates the object.

### Synchronization between processes

`!multiprocessing` contains equivalents of all the synchronization primitives from `threading`. For instance one can use a lock to ensure that only one process prints to standard output at a time:

    from multiprocessing import Process, Lock

    def f(l, i):
        l.acquire()
        try:
            print('hello world', i)
        finally:
            l.release()

    if __name__ == '__main__':
        lock = Lock()

        for num in range(10):
            Process(target=f, args=(lock, num)).start()

Without using the lock output from the different processes is liable to get all mixed up.

### Sharing state between processes

As mentioned above, when doing concurrent programming it is usually best to avoid using shared state as far as possible. This is particularly true when using multiple processes.

However, if you really do need to use some shared data then `!multiprocessing` provides a couple of ways of doing so.

**Shared memory**

> Data can be stored in a shared memory map using `Value` or `Array`. For example, the following code :
>
>     from multiprocessing import Process, Value, Array
>
>     def f(n, a):
>         n.value = 3.1415927
>         for i in range(len(a)):
>             a[i] = -a[i]
>
>     if __name__ == '__main__':
>         num = Value('d', 0.0)
>         arr = Array('i', range(10))
>
>         p = Process(target=f, args=(num, arr))
>         p.start()
>         p.join()
>
>         print(num.value)
>         print(arr[:])
>
> will print :
>
>     3.1415927
>     [0, -1, -2, -3, -4, -5, -6, -7, -8, -9]
>
> The `'d'` and `'i'` arguments used when creating `num` and `arr` are typecodes of the kind used by the `array` module: `'d'` indicates a double precision float and `'i'` indicates a signed integer. These shared objects will be process and thread-safe.
>
> For more flexibility in using shared memory one can use the `multiprocessing.sharedctypes` module which supports the creation of arbitrary ctypes objects allocated from shared memory.

**Server process**

> A manager object returned by `Manager` controls a server process which holds Python objects and allows other processes to manipulate them using proxies.
>
> A manager returned by `Manager` will support types `list`, `dict`, `set`, `~managers.Namespace`, `Lock`, `RLock`, `Semaphore`, `BoundedSemaphore`, `Condition`, `Event`, `Barrier`, `Queue`, `Value` and `Array`. For example, :
>
>     from multiprocessing import Process, Manager
>
>     def f(d, l, s):
>         d[1] = '1'
>         d['2'] = 2
>         d[0.25] = None
>         l.reverse()
>         s.add('a')
>         s.add('b')
>
>     if __name__ == '__main__':
>         with Manager() as manager:
>             d = manager.dict()
>             l = manager.list(range(10))
>             s = manager.set()
>
>             p = Process(target=f, args=(d, l, s))
>             p.start()
>             p.join()
>
>             print(d)
>             print(l)
>             print(s)
>
> will print :
>
>     {0.25: None, 1: '1', '2': 2}
>     [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
>     {'a', 'b'}
>
> Server process managers are more flexible than using shared memory objects because they can be made to support arbitrary object types. Also, a single manager can be shared by processes on different computers over a network. They are, however, slower than using shared memory.

### Using a pool of workers

The `~multiprocessing.pool.Pool` class represents a pool of worker processes. It has methods which allows tasks to be offloaded to the worker processes in a few different ways.

For example:

    from multiprocessing import Pool, TimeoutError
    import time
    import os

    def f(x):
        return x*x

    if __name__ == '__main__':
        # start 4 worker processes
        with Pool(processes=4) as pool:

            # print "[0, 1, 4,..., 81]"
            print(pool.map(f, range(10)))

            # print same numbers in arbitrary order
            for i in pool.imap_unordered(f, range(10)):
                print(i)

            # evaluate "f(20)" asynchronously
            res = pool.apply_async(f, (20,))      # runs in *only* one process
            print(res.get(timeout=1))             # prints "400"

            # evaluate "os.getpid()" asynchronously
            res = pool.apply_async(os.getpid, ()) # runs in *only* one process
            print(res.get(timeout=1))             # prints the PID of that process

            # launching multiple evaluations asynchronously *may* use more processes
            multiple_results = [pool.apply_async(os.getpid, ()) for i in range(4)]
            print([res.get(timeout=1) for res in multiple_results])

            # make a single worker sleep for 10 seconds
            res = pool.apply_async(time.sleep, (10,))
            try:
                print(res.get(timeout=1))
            except TimeoutError:
                print("We lacked patience and got a multiprocessing.TimeoutError")

            print("For the moment, the pool remains available for more work")

        # exiting the 'with'-block has stopped the pool
        print("Now the pool is closed and no longer available")

Note that the methods of a pool should only ever be used by the process which created it.

> [!NOTE]
> Functionality within this package requires that the `__main__` module be importable by the children. This is covered in `multiprocessing-programming` however it is worth pointing out here. This means that some examples, such as the `multiprocessing.pool.Pool` examples will not work in the interactive interpreter. For example:
>
>     >>> from multiprocessing import Pool
>     >>> p = Pool(5)
>     >>> def f(x):
>     ...     return x*x
>     ...
>     >>> with p:
>     ...     p.map(f, [1,2,3])
>     Process PoolWorker-1:
>     Process PoolWorker-2:
>     Process PoolWorker-3:
>     Traceback (most recent call last):
>     Traceback (most recent call last):
>     Traceback (most recent call last):
>     AttributeError: Can't get attribute 'f' on <module '__main__' (<class '_frozen_importlib.BuiltinImporter'>)>
>     AttributeError: Can't get attribute 'f' on <module '__main__' (<class '_frozen_importlib.BuiltinImporter'>)>
>     AttributeError: Can't get attribute 'f' on <module '__main__' (<class '_frozen_importlib.BuiltinImporter'>)>
>
> (If you try this it will actually output three full tracebacks interleaved in a semi-random fashion, and then you may have to stop the parent process somehow.)

## Reference

The `!multiprocessing` package mostly replicates the API of the `threading` module.

### Global start method

Python supports several ways to create and initialize a process. The global start method sets the default mechanism for creating a process.

Several multiprocessing functions and methods that may also instantiate certain objects will implicitly set the global start method to the system's default, if it hasn’t been set already. The global start method can only be set once. If you need to change the start method from the system default, you must proactively set the global start method before calling functions or methods, or creating these objects.

### `Process` and exceptions

<div class="Process(group=None, target=None, name=None, args=(), kwargs={}, \ *, daemon=None)">

Process objects represent activity that is run in a separate process. The `Process` class has equivalents of all the methods of `threading.Thread`.

The constructor should always be called with keyword arguments. *group* should always be `None`; it exists solely for compatibility with `threading.Thread`. *target* is the callable object to be invoked by the `run` method. It defaults to `None`, meaning nothing is called. *name* is the process name (see `name` for more details). *args* is the argument tuple for the target invocation. *kwargs* is a dictionary of keyword arguments for the target invocation. If provided, the keyword-only *daemon* argument sets the process `daemon` flag to `True` or `False`. If `None` (the default), this flag will be inherited from the creating process.

By default, no arguments are passed to *target*. The *args* argument, which defaults to `()`, can be used to specify a list or tuple of the arguments to pass to *target*.

If a subclass overrides the constructor, it must make sure it invokes the base class constructor (`super().__init__()`) before doing anything else to the process.

> [!NOTE]
> In general, all arguments to `Process` must be picklable. This is frequently observed when trying to create a `Process` or use a `concurrent.futures.ProcessPoolExecutor` from a REPL with a locally defined *target* function.
>
> Passing a callable object defined in the current REPL session causes the child process to die via an uncaught `AttributeError` exception when starting as *target* must have been defined within an importable module in order to be loaded during unpickling.
>
> Example of this uncatchable error from the child:
>
>     >>> import multiprocessing as mp
>     >>> def knigit():
>     ...     print("Ni!")
>     ...
>     >>> process = mp.Process(target=knigit)
>     >>> process.start()
>     >>> Traceback (most recent call last):
>       File ".../multiprocessing/spawn.py", line ..., in spawn_main
>       File ".../multiprocessing/spawn.py", line ..., in _main
>     AttributeError: module '__main__' has no attribute 'knigit'
>     >>> process
>     <SpawnProcess name='SpawnProcess-1' pid=379473 parent=378707 stopped exitcode=1>
>
> See `multiprocessing-programming-spawn`. While this restriction is not true if using the `"fork"` start method, as of Python `3.14` that is no longer the default on any platform. See `multiprocessing-start-methods`. See also `132898`.

<div class="versionchanged">

3.3 Added the *daemon* parameter.

</div>

<div class="method">

run()

Method representing the process's activity.

You may override this method in a subclass. The standard `run` method invokes the callable object passed to the object's constructor as the target argument, if any, with sequential and keyword arguments taken from the *args* and *kwargs* arguments, respectively.

Using a list or tuple as the *args* argument passed to `Process` achieves the same effect.

Example:

    >>> from multiprocessing import Process
    >>> p = Process(target=print, args=[1])
    >>> p.run()
    1
    >>> p = Process(target=print, args=(1,))
    >>> p.run()
    1

</div>

<div class="method">

start()

Start the process's activity.

This must be called at most once per process object. It arranges for the object's `run` method to be invoked in a separate process.

</div>

<div class="method">

join(\[timeout\])

If the optional argument *timeout* is `None` (the default), the method blocks until the process whose `join` method is called terminates. If *timeout* is a positive number, it blocks at most *timeout* seconds. Note that the method returns `None` if its process terminates or if the method times out. Check the process's `exitcode` to determine if it terminated.

A process can be joined many times.

A process cannot join itself because this would cause a deadlock. It is an error to attempt to join a process before it has been started.

</div>

<div class="attribute">

name

The process's name. The name is a string used for identification purposes only. It has no semantics. Multiple processes may be given the same name.

The initial name is set by the constructor. If no explicit name is provided to the constructor, a name of the form 'Process-N<sub>1</sub>:N<sub>2</sub>:...:N<sub>k</sub>' is constructed, where each N<sub>k</sub> is the N-th child of its parent.

</div>

<div class="method">

is_alive

Return whether the process is alive.

Roughly, a process object is alive from the moment the `start` method returns until the child process terminates.

</div>

<div class="attribute">

daemon

The process's daemon flag, a Boolean value. This must be set before `start` is called.

The initial value is inherited from the creating process.

When a process exits, it attempts to terminate all of its daemonic child processes.

Note that a daemonic process is not allowed to create child processes. Otherwise a daemonic process would leave its children orphaned if it gets terminated when its parent process exits. Additionally, these are **not** Unix daemons or services, they are normal processes that will be terminated (and not joined) if non-daemonic processes have exited.

</div>

In addition to the `threading.Thread` API, `Process` objects also support the following attributes and methods:

<div class="attribute">

pid

Return the process ID. Before the process is spawned, this will be `None`.

</div>

<div class="attribute">

exitcode

The child's exit code. This will be `None` if the process has not yet terminated.

If the child's `run` method returned normally, the exit code will be 0. If it terminated via `sys.exit` with an integer argument *N*, the exit code will be *N*.

If the child terminated due to an exception not caught within `run`, the exit code will be 1. If it was terminated by signal *N*, the exit code will be the negative value *-N*.

</div>

<div class="attribute">

authkey

The process's authentication key (a byte string).

When `!multiprocessing` is initialized the main process is assigned a random string using `os.urandom`.

When a `Process` object is created, it will inherit the authentication key of its parent process, although this may be changed by setting `authkey` to another byte string.

See `multiprocessing-auth-keys`.

</div>

<div class="attribute">

sentinel

A numeric handle of a system object which will become "ready" when the process ends.

You can use this value if you want to wait on several events at once using `multiprocessing.connection.wait`. Otherwise calling `join` is simpler.

On Windows, this is an OS handle usable with the `WaitForSingleObject` and `WaitForMultipleObjects` family of API calls. On POSIX, this is a file descriptor usable with primitives from the `select` module.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

interrupt()

Terminate the process. Works on POSIX using the `~signal.SIGINT` signal. Behavior on Windows is undefined.

By default, this terminates the child process by raising `KeyboardInterrupt`. This behavior can be altered by setting the respective signal handler in the child process `signal.signal` for `~signal.SIGINT`.

Note: if the child process catches and discards `KeyboardInterrupt`, the process will not be terminated.

Note: the default behavior will also set `exitcode` to `1` as if an uncaught exception was raised in the child process. To have a different `exitcode` you may simply catch `KeyboardInterrupt` and call `exit(your_code)`.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

terminate()

Terminate the process. On POSIX this is done using the `~signal.SIGTERM` signal; on Windows `!TerminateProcess` is used. Note that exit handlers and finally clauses, etc., will not be executed.

Note that descendant processes of the process will *not* be terminated -- they will simply become orphaned.

> [!WARNING]
> If this method is used when the associated process is using a pipe or queue then the pipe or queue is liable to become corrupted and may become unusable by other process. Similarly, if the process has acquired a lock or semaphore etc. then terminating it is liable to cause other processes to deadlock.

</div>

<div class="method">

kill()

Same as `terminate` but using the `SIGKILL` signal on POSIX.

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

close()

Close the `Process` object, releasing all resources associated with it. `ValueError` is raised if the underlying process is still running. Once `close` returns successfully, most other methods and attributes of the `Process` object will raise `ValueError`.

<div class="versionadded">

3.7

</div>

</div>

Note that the `start`, `join`, `is_alive`, `terminate` and `exitcode` methods should only be called by the process that created the process object.

Example usage of some of the methods of `Process`:

<div class="doctest">

\>\>\> import multiprocessing, time, signal \>\>\> mp_context = multiprocessing.get_context('spawn') \>\>\> p = mp_context.Process(target=time.sleep, args=(1000,)) \>\>\> print(p, p.is_alive()) \<...Process ... initial\> False \>\>\> p.start() \>\>\> print(p, p.is_alive()) \<...Process ... started\> True \>\>\> p.terminate() \>\>\> time.sleep(0.1) \>\>\> print(p, p.is_alive()) \<...Process ... stopped exitcode=-SIGTERM\> False \>\>\> p.exitcode == -signal.SIGTERM True

</div>

</div>

<div class="exception">

ProcessError

The base class of all `!multiprocessing` exceptions.

</div>

<div class="exception">

BufferTooShort

Exception raised by `Connection.recv_bytes_into` when the supplied buffer object is too small for the message read.

If `e` is an instance of `BufferTooShort` then `e.args[0]` will give the message as a byte string.

</div>

<div class="exception">

AuthenticationError

Raised when there is an authentication error.

</div>

<div class="exception">

TimeoutError

Raised by methods with a timeout when the timeout expires.

</div>

### Pipes and Queues

When using multiple processes, one generally uses message passing for communication between processes and avoids having to use any synchronization primitives like locks.

For passing messages one can use `Pipe` (for a connection between two processes) or a queue (which allows multiple producers and consumers).

The `Queue`, `SimpleQueue` and `JoinableQueue` types are multi-producer, multi-consumer `FIFO (first-in, first-out)` queues modelled on the `queue.Queue` class in the standard library. They differ in that `Queue` lacks the `~queue.Queue.task_done` and `~queue.Queue.join` methods introduced into Python 2.5's `queue.Queue` class.

If you use `JoinableQueue` then you **must** call `JoinableQueue.task_done` for each task removed from the queue or else the semaphore used to count the number of unfinished tasks may eventually overflow, raising an exception.

One difference from other Python queue implementations, is that `!multiprocessing` queues serializes all objects that are put into them using `pickle`. The object returned by the get method is a re-created object that does not share memory with the original object.

Note that one can also create a shared queue by using a manager object -- see `multiprocessing-managers`.

> [!NOTE]
> `!multiprocessing` uses the usual `queue.Empty` and `queue.Full` exceptions to signal a timeout. They are not available in the `!multiprocessing` namespace so you need to import them from `queue`.

> [!NOTE]
> When an object is put on a queue, the object is pickled and a background thread later flushes the pickled data to an underlying pipe. This has some consequences which are a little surprising, but should not cause any practical difficulties -- if they really bother you then you can instead use a queue created with a `manager <multiprocessing-managers>`.
>
> 1)  After putting an object on an empty queue there may be an infinitesimal delay before the queue's `~Queue.empty` method returns `False` and `~Queue.get_nowait` can return without raising `queue.Empty`.
> 2)  If multiple processes are enqueuing objects, it is possible for the objects to be received at the other end out-of-order. However, objects enqueued by the same process will always be in the expected order with respect to each other.

> [!WARNING]
> If a process is killed using `Process.terminate` or `os.kill` while it is trying to use a `Queue`, then the data in the queue is likely to become corrupted. This may cause any other process to get an exception when it tries to use the queue later on.

> [!WARNING]
> As mentioned above, if a child process has put items on a queue (and it has not used `JoinableQueue.cancel_join_thread
> <multiprocessing.Queue.cancel_join_thread>`), then that process will not terminate until all buffered items have been flushed to the pipe.
>
> This means that if you try joining that process you may get a deadlock unless you are sure that all items which have been put on the queue have been consumed. Similarly, if the child process is non-daemonic then the parent process may hang on exit when it tries to join all its non-daemonic children.
>
> Note that a queue created using a manager does not have this issue. See `multiprocessing-programming`.

For an example of the usage of queues for interprocess communication see `multiprocessing-examples`.

<div class="function">

Pipe(duplex=True)

Returns a pair `(conn1, conn2)` of `~multiprocessing.connection.Connection` objects representing the ends of a pipe.

If *duplex* is `True` (the default) then the pipe is bidirectional. If *duplex* is `False` then the pipe is unidirectional: `conn1` can only be used for receiving messages and `conn2` can only be used for sending messages.

The `~multiprocessing.Connection.send` method serializes the object using `pickle` and the `~multiprocessing.Connection.recv` re-creates the object.

</div>

<div class="Queue([maxsize])">

Returns a process shared queue implemented using a pipe and a few locks/semaphores. When a process first puts an item on the queue a feeder thread is started which transfers objects from a buffer into the pipe.

Instantiating this class may set the global start method. See `global-start-method` for more details.

The usual `queue.Empty` and `queue.Full` exceptions from the standard library's `queue` module are raised to signal timeouts.

`Queue` implements all the methods of `queue.Queue` except for `~queue.Queue.task_done`, `~queue.Queue.join`, and `~queue.Queue.shutdown`.

<div class="method">

qsize()

Return the approximate size of the queue. Because of multithreading/multiprocessing semantics, this number is not reliable.

Note that this may raise `NotImplementedError` on platforms like macOS where `sem_getvalue()` is not implemented.

</div>

<div class="method">

empty()

Return `True` if the queue is empty, `False` otherwise. Because of multithreading/multiprocessing semantics, this is not reliable.

May raise an `OSError` on closed queues. (not guaranteed)

</div>

<div class="method">

full()

Return `True` if the queue is full, `False` otherwise. Because of multithreading/multiprocessing semantics, this is not reliable.

</div>

<div class="method">

put(obj\[, block\[, timeout\]\])

Put obj into the queue. If the optional argument *block* is `True` (the default) and *timeout* is `None` (the default), block if necessary until a free slot is available. If *timeout* is a positive number, it blocks at most *timeout* seconds and raises the `queue.Full` exception if no free slot was available within that time. Otherwise (*block* is `False`), put an item on the queue if a free slot is immediately available, else raise the `queue.Full` exception (*timeout* is ignored in that case).

<div class="versionchanged">

3.8 If the queue is closed, `ValueError` is raised instead of `AssertionError`.

</div>

</div>

<div class="method">

put_nowait(obj)

Equivalent to `put(obj, False)`.

</div>

<div class="method">

get(\[block\[, timeout\]\])

Remove and return an item from the queue. If optional args *block* is `True` (the default) and *timeout* is `None` (the default), block if necessary until an item is available. If *timeout* is a positive number, it blocks at most *timeout* seconds and raises the `queue.Empty` exception if no item was available within that time. Otherwise (block is `False`), return an item if one is immediately available, else raise the `queue.Empty` exception (*timeout* is ignored in that case).

<div class="versionchanged">

3.8 If the queue is closed, `ValueError` is raised instead of `OSError`.

</div>

</div>

<div class="method">

get_nowait()

Equivalent to `get(False)`.

</div>

`multiprocessing.Queue` has a few additional methods not found in `queue.Queue`. These methods are usually unnecessary for most code:

<div class="method">

close()

Close the queue: release internal resources.

A queue must not be used anymore after it is closed. For example, `~Queue.get`, `~Queue.put` and `~Queue.empty` methods must no longer be called.

The background thread will quit once it has flushed all buffered data to the pipe. This is called automatically when the queue is garbage collected.

</div>

<div class="method">

join_thread()

Join the background thread. This can only be used after `close` has been called. It blocks until the background thread exits, ensuring that all data in the buffer has been flushed to the pipe.

By default if a process is not the creator of the queue then on exit it will attempt to join the queue's background thread. The process can call `cancel_join_thread` to make `join_thread` do nothing.

</div>

<div class="method">

cancel_join_thread()

Prevent `join_thread` from blocking. In particular, this prevents the background thread from being joined automatically when the process exits -- see `join_thread`.

A better name for this method might be `allow_exit_without_flush()`. It is likely to cause enqueued data to be lost, and you almost certainly will not need to use it. It is really only there if you need the current process to exit immediately without waiting to flush enqueued data to the underlying pipe, and you don't care about lost data.

</div>

> [!NOTE]
> This class's functionality requires a functioning shared semaphore implementation on the host operating system. Without one, the functionality in this class will be disabled, and attempts to instantiate a `Queue` will result in an `ImportError`. See `3770` for additional information. The same holds true for any of the specialized queue types listed below.

</div>

<div class="SimpleQueue()">

It is a simplified `Queue` type, very close to a locked `Pipe`.

Instantiating this class may set the global start method. See `global-start-method` for more details.

<div class="method">

close()

Close the queue: release internal resources.

A queue must not be used anymore after it is closed. For example, `get`, `put` and `empty` methods must no longer be called.

<div class="versionadded">

3.9

</div>

</div>

<div class="method">

empty()

Return `True` if the queue is empty, `False` otherwise.

Always raises an `OSError` if the SimpleQueue is closed.

</div>

<div class="method">

get()

Remove and return an item from the queue.

</div>

<div class="method">

put(item)

Put *item* into the queue.

</div>

</div>

<div class="JoinableQueue([maxsize])">

`JoinableQueue`, a `Queue` subclass, is a queue which additionally has `task_done` and `join` methods.

Instantiating this class may set the global start method. See `global-start-method` for more details.

<div class="method">

task_done()

Indicate that a formerly enqueued task is complete. Used by queue consumers. For each `~Queue.get` used to fetch a task, a subsequent call to `task_done` tells the queue that the processing on the task is complete.

If a `~queue.Queue.join` is currently blocking, it will resume when all items have been processed (meaning that a `task_done` call was received for every item that had been `~Queue.put` into the queue).

Raises a `ValueError` if called more times than there were items placed in the queue.

</div>

<div class="method">

join()

Block until all items in the queue have been gotten and processed.

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer calls `task_done` to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, `~queue.Queue.join` unblocks.

</div>

</div>

### Miscellaneous

<div class="function">

active_children()

Return list of all live children of the current process.

Calling this has the side effect of "joining" any processes which have already finished.

</div>

<div class="function">

cpu_count()

Return the number of CPUs in the system.

This number is not equivalent to the number of CPUs the current process can use. The number of usable CPUs can be obtained with `os.process_cpu_count` (or `len(os.sched_getaffinity(0))`).

When the number of CPUs cannot be determined a `NotImplementedError` is raised.

<div class="seealso">

`os.cpu_count` `os.process_cpu_count`

</div>

<div class="versionchanged">

3.13

The return value can also be overridden using the `-X cpu_count <-X>` flag or `PYTHON_CPU_COUNT` as this is merely a wrapper around the `os` cpu count APIs.

</div>

</div>

<div class="function">

current_process()

Return the `Process` object corresponding to the current process.

An analogue of `threading.current_thread`.

</div>

<div class="function">

parent_process()

Return the `Process` object corresponding to the parent process of the `current_process`. For the main process, `parent_process` will be `None`.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

freeze_support()

Add support for when a program which uses `!multiprocessing` has been frozen to produce an executable. (Has been tested with **py2exe**, **PyInstaller** and **cx_Freeze**.)

One needs to call this function straight after the `if __name__ == '__main__'` line of the main module. For example:

    from multiprocessing import Process, freeze_support

    def f():
        print('hello world!')

    if __name__ == '__main__':
        freeze_support()
        Process(target=f).start()

If the `freeze_support()` line is omitted then trying to run the frozen executable will raise `RuntimeError`.

Calling `freeze_support()` has no effect when the start method is not *spawn*. In addition, if the module is being run normally by the Python interpreter (the program has not been frozen), then `freeze_support()` has no effect.

</div>

<div class="function">

get_all_start_methods()

Returns a list of the supported start methods, the first of which is the default. The possible start methods are `'fork'`, `'spawn'` and `'forkserver'`. Not all platforms support all methods. See `multiprocessing-start-methods`.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

get_context(method=None)

Return a context object which has the same attributes as the `!multiprocessing` module.

If *method* is `None` then the default context is returned. Note that if the global start method has not been set, this will set it to the system default See `global-start-method` for more details. Otherwise *method* should be `'fork'`, `'spawn'`, `'forkserver'`. `ValueError` is raised if the specified start method is not available. See `multiprocessing-start-methods`.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

get_start_method(allow_none=False)

Return the name of start method used for starting processes.

If the global start method is not set and *allow_none* is `False`, the global start method is set to the default, and its name is returned. See `global-start-method` for more details.

The return value can be `'fork'`, `'spawn'`, `'forkserver'` or `None`. See `multiprocessing-start-methods`.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.8

On macOS, the *spawn* start method is now the default. The *fork* start method should be considered unsafe as it can lead to crashes of the subprocess. See `33725`.

</div>

</div>

<div class="function">

set_executable(executable)

Set the path of the Python interpreter to use when starting a child process. (By default `sys.executable` is used). Embedders will probably need to do something like :

    set_executable(os.path.join(sys.exec_prefix, 'pythonw.exe'))

before they can create child processes.

<div class="versionchanged">

3.4 Now supported on POSIX when the `'spawn'` start method is used.

</div>

<div class="versionchanged">

3.11 Accepts a `path-like object`.

</div>

</div>

<div class="function">

set_forkserver_preload(module_names, \*, on_error='ignore')

Set a list of module names for the forkserver main process to attempt to import so that their already imported state is inherited by forked processes. This can be used as a performance enhancement to avoid repeated work in every process.

For this to work, it must be called before the forkserver process has been launched (before creating a `Pool` or starting a `Process`).

The *on_error* parameter controls how `ImportError` exceptions during module preloading are handled: `"ignore"` (default) silently ignores failures, `"warn"` causes the forkserver subprocess to emit an `ImportWarning` to stderr, and `"fail"` causes the forkserver subprocess to exit with the exception traceback on stderr, making subsequent process creation fail with `EOFError` or `ConnectionError`.

Only meaningful when using the `'forkserver'` start method. See `multiprocessing-start-methods`.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.15 Added the *on_error* parameter.

</div>

</div>

<div class="function">

set_start_method(method, force=False)

Set the method which should be used to start child processes. The *method* argument can be `'fork'`, `'spawn'` or `'forkserver'`. Raises `RuntimeError` if the start method has already been set and *force* is not `True`. If *method* is `None` and *force* is `True` then the start method is set to `None`. If *method* is `None` and *force* is `False` then the context is set to the default context.

Note that this should be called at most once, and it should be protected inside the `if __name__ == '__main__'` clause of the main module.

See `multiprocessing-start-methods`.

<div class="versionadded">

3.4

</div>

</div>

> [!NOTE]
> `!multiprocessing` contains no analogues of `threading.active_count`, `threading.enumerate`, `threading.settrace`, `threading.setprofile`, `threading.Timer`, or `threading.local`.

### Connection Objects

<div class="currentmodule">

multiprocessing.connection

</div>

Connection objects allow the sending and receiving of picklable objects or strings. They can be thought of as message oriented connected sockets.

Connection objects are usually created using `Pipe <multiprocessing.Pipe>` -- see also `multiprocessing-listeners-clients`.

<div class="Connection">

<div class="method">

send(obj)

Send an object to the other end of the connection which should be read using `recv`.

The object must be picklable. Very large pickles (approximately 32 MiB+, though it depends on the OS) may raise a `ValueError` exception.

</div>

<div class="method">

recv()

Return an object sent from the other end of the connection using `send`. Blocks until there is something to receive. Raises `EOFError` if there is nothing left to receive and the other end was closed.

</div>

<div class="method">

fileno()

Return the file descriptor or handle used by the connection.

</div>

<div class="method">

close()

Close the connection.

This is called automatically when the connection is garbage collected.

</div>

<div class="method">

poll(\[timeout\])

Return whether there is any data available to be read.

If *timeout* is not specified then it will return immediately. If *timeout* is a number then this specifies the maximum time in seconds to block. If *timeout* is `None` then an infinite timeout is used.

Note that multiple connection objects may be polled at once by using `multiprocessing.connection.wait`.

</div>

<div class="method">

send_bytes(buf\[, offset\[, size\]\])

Send byte data from a `bytes-like object` as a complete message.

If *offset* is given then data is read from that position in *buf*. If *size* is given then that many bytes will be read from *buf*. Very large buffers (approximately 32 MiB+, though it depends on the OS) may raise a `ValueError` exception

</div>

<div class="method">

recv_bytes(\[maxlength\])

Return a complete message of byte data sent from the other end of the connection as a string. Blocks until there is something to receive. Raises `EOFError` if there is nothing left to receive and the other end has closed.

If *maxlength* is specified and the message is longer than *maxlength* then `OSError` is raised and the connection will no longer be readable.

<div class="versionchanged">

3.3 This function used to raise `IOError`, which is now an alias of `OSError`.

</div>

</div>

<div class="method">

recv_bytes_into(buf\[, offset\])

Read into *buf* a complete message of byte data sent from the other end of the connection and return the number of bytes in the message. Blocks until there is something to receive. Raises `EOFError` if there is nothing left to receive and the other end was closed.

*buf* must be a writable `bytes-like object`. If *offset* is given then the message will be written into the buffer from that position. Offset must be a non-negative integer less than the length of *buf* (in bytes).

If the buffer is too short then a `BufferTooShort` exception is raised and the complete message is available as `e.args[0]` where `e` is the exception instance.

</div>

<div class="versionchanged">

3.3 Connection objects themselves can now be transferred between processes using `Connection.send` and `Connection.recv`.

Connection objects also now support the context management protocol -- see `typecontextmanager`. `~contextmanager.__enter__` returns the connection object, and `~contextmanager.__exit__` calls `close`.

</div>

</div>

For example:

<div class="doctest">

\>\>\> from multiprocessing import Pipe \>\>\> a, b = Pipe() \>\>\> a.send(\[1, 'hello', None\]) \>\>\> b.recv() \[1, 'hello', None\] \>\>\> b.send_bytes(b'thank you') \>\>\> a.recv_bytes() b'thank you' \>\>\> import array \>\>\> arr1 = array.array('i', range(5)) \>\>\> arr2 = array.array('i', \[0\] \* 10) \>\>\> a.send_bytes(arr1) \>\>\> count = b.recv_bytes_into(arr2) \>\>\> assert count == len(arr1) \* arr1.itemsize \>\>\> arr2 array('i', \[0, 1, 2, 3, 4, 0, 0, 0, 0, 0\])

</div>

<div id="multiprocessing-recv-pickle-security">

> [!WARNING]
> The `Connection.recv` method automatically unpickles the data it receives, which can be a security risk unless you can trust the process which sent the message.
>
> Therefore, unless the connection object was produced using `Pipe` you should only use the `~Connection.recv` and `~Connection.send` methods after performing some sort of authentication. See `multiprocessing-auth-keys`.

</div>

> [!WARNING]
> If a process is killed while it is trying to read or write to a pipe then the data in the pipe is likely to become corrupted, because it may become impossible to be sure where the message boundaries lie.

### Synchronization primitives

<div class="currentmodule">

multiprocessing

</div>

Generally synchronization primitives are not as necessary in a multiprocess program as they are in a multithreaded program. See the documentation for `threading` module.

Note that one can also create synchronization primitives by using a manager object -- see `multiprocessing-managers`.

<div class="Barrier(parties[, action[, timeout]])">

A barrier object: a clone of `threading.Barrier`.

Instantiating this class may set the global start method. See `global-start-method` for more details.

<div class="versionadded">

3.3

</div>

</div>

<div class="BoundedSemaphore([value])">

A bounded semaphore object: a close analog of `threading.BoundedSemaphore`.

Instantiating this class may set the global start method. See `global-start-method` for more details.

A solitary difference from its close analog exists: its `acquire` method's first argument is named *block*, as is consistent with `Lock.acquire`.

<div class="method">

locked()

Return a boolean indicating whether this object is locked right now.

<div class="versionadded">

3.14

</div>

</div>

> [!NOTE]
> On macOS, this is indistinguishable from `Semaphore` because `sem_getvalue()` is not implemented on that platform.

</div>

<div class="Condition([lock])">

A condition variable: an alias for `threading.Condition`.

If *lock* is specified then it should be a `Lock` or `RLock` object from `!multiprocessing`.

Instantiating this class may set the global start method. See `global-start-method` for more details.

<div class="versionchanged">

3.3 The `~threading.Condition.wait_for` method was added.

</div>

</div>

<div class="Event()">

A clone of `threading.Event`.

Instantiating this class may set the global start method. See `global-start-method` for more details.

</div>

<div class="Lock()">

A non-recursive lock object: a close analog of `threading.Lock`. Once a process or thread has acquired a lock, subsequent attempts to acquire it from any process or thread will block until it is released; any process or thread may release it. The concepts and behaviors of `threading.Lock` as it applies to threads are replicated here in `multiprocessing.Lock` as it applies to either processes or threads, except as noted.

Note that `Lock` is actually a factory function which returns an instance of `multiprocessing.synchronize.Lock` initialized with a default context.

Instantiating this class may set the global start method. See `global-start-method` for more details.

`Lock` supports the `context manager` protocol and thus may be used in `with` statements.

<div class="method">

acquire(block=True, timeout=None)

Acquire a lock, blocking or non-blocking.

With the *block* argument set to `True` (the default), the method call will block until the lock is in an unlocked state, then set it to locked and return `True`. Note that the name of this first argument differs from that in `threading.Lock.acquire`.

With the *block* argument set to `False`, the method call does not block. If the lock is currently in a locked state, return `False`; otherwise set the lock to a locked state and return `True`.

When invoked with a positive, floating-point value for *timeout*, block for at most the number of seconds specified by *timeout* as long as the lock can not be acquired. Invocations with a negative value for *timeout* are equivalent to a *timeout* of zero. Invocations with a *timeout* value of `None` (the default) set the timeout period to infinite. Note that the treatment of negative or `None` values for *timeout* differs from the implemented behavior in `threading.Lock.acquire`. The *timeout* argument has no practical implications if the *block* argument is set to `False` and is thus ignored. Returns `True` if the lock has been acquired or `False` if the timeout period has elapsed.

</div>

<div class="method">

release()

Release a lock. This can be called from any process or thread, not only the process or thread which originally acquired the lock.

Behavior is the same as in `threading.Lock.release` except that when invoked on an unlocked lock, a `ValueError` is raised.

</div>

<div class="method">

locked()

Return a boolean indicating whether this object is locked right now.

<div class="versionadded">

3.14

</div>

</div>

</div>

<div class="RLock()">

A recursive lock object: a close analog of `threading.RLock`. A recursive lock must be released by the process or thread that acquired it. Once a process or thread has acquired a recursive lock, the same process or thread may acquire it again without blocking; that process or thread must release it once for each time it has been acquired.

Note that `RLock` is actually a factory function which returns an instance of `multiprocessing.synchronize.RLock` initialized with a default context.

Instantiating this class may set the global start method. See `global-start-method` for more details.

`RLock` supports the `context manager` protocol and thus may be used in `with` statements.

<div class="method">

acquire(block=True, timeout=None)

Acquire a lock, blocking or non-blocking.

When invoked with the *block* argument set to `True`, block until the lock is in an unlocked state (not owned by any process or thread) unless the lock is already owned by the current process or thread. The current process or thread then takes ownership of the lock (if it does not already have ownership) and the recursion level inside the lock increments by one, resulting in a return value of `True`. Note that there are several differences in this first argument's behavior compared to the implementation of `threading.RLock.acquire`, starting with the name of the argument itself.

When invoked with the *block* argument set to `False`, do not block. If the lock has already been acquired (and thus is owned) by another process or thread, the current process or thread does not take ownership and the recursion level within the lock is not changed, resulting in a return value of `False`. If the lock is in an unlocked state, the current process or thread takes ownership and the recursion level is incremented, resulting in a return value of `True`.

Use and behaviors of the *timeout* argument are the same as in `Lock.acquire`. Note that some of these behaviors of *timeout* differ from the implemented behaviors in `threading.RLock.acquire`.

</div>

<div class="method">

release()

Release a lock, decrementing the recursion level. If after the decrement the recursion level is zero, reset the lock to unlocked (not owned by any process or thread) and if any other processes or threads are blocked waiting for the lock to become unlocked, allow exactly one of them to proceed. If after the decrement the recursion level is still nonzero, the lock remains locked and owned by the calling process or thread.

Only call this method when the calling process or thread owns the lock. An `AssertionError` is raised if this method is called by a process or thread other than the owner or if the lock is in an unlocked (unowned) state. Note that the type of exception raised in this situation differs from the implemented behavior in `threading.RLock.release`.

</div>

<div class="method">

locked()

Return a boolean indicating whether this object is locked right now.

<div class="versionadded">

3.14

</div>

</div>

</div>

<div class="Semaphore([value])">

A semaphore object: a close analog of `threading.Semaphore`.

Instantiating this class may set the global start method. See `global-start-method` for more details.

A solitary difference from its close analog exists: its `acquire` method's first argument is named *block*, as is consistent with `Lock.acquire`.

<div class="method">

get_value()

Return the current value of semaphore.

Note that this may raise `NotImplementedError` on platforms like macOS where `sem_getvalue()` is not implemented.

</div>

<div class="method">

locked()

Return a boolean indicating whether this object is locked right now.

<div class="versionadded">

3.14

</div>

</div>

</div>

> [!NOTE]
> On macOS, `sem_timedwait` is unsupported, so calling `acquire()` with a timeout will emulate that function's behavior using a sleeping loop.

> [!NOTE]
> Some of this package's functionality requires a functioning shared semaphore implementation on the host operating system. Without one, the `multiprocessing.synchronize` module will be disabled, and attempts to import it will result in an `ImportError`. See `3770` for additional information.

### Shared `ctypes` Objects

It is possible to create shared objects using shared memory which can be inherited by child processes.

<div class="function">

Value(typecode_or_type, \*args, lock=True)

Return a `ctypes` object allocated from shared memory. By default the return value is actually a synchronized wrapper for the object. The object itself can be accessed via the *value* attribute of a `Value`.

*typecode_or_type* determines the type of the returned object: it is either a ctypes type or a one character typecode of the kind used by the `array` module. *\*args* is passed on to the constructor for the type.

If *lock* is `True` (the default) then a new recursive lock object is created to synchronize access to the value. If *lock* is a `Lock` or `RLock` object then that will be used to synchronize access to the value. If *lock* is `False` then access to the returned object will not be automatically protected by a lock, so it will not necessarily be "process-safe".

Operations like `+=` which involve a read and write are not atomic. So if, for instance, you want to atomically increment a shared value it is insufficient to just do :

    counter.value += 1

Assuming the associated lock is recursive (which it is by default) you can instead do :

    with counter.get_lock():
        counter.value += 1

Note that *lock* is a keyword-only argument.

</div>

<div class="function">

Array(typecode_or_type, size_or_initializer, \*, lock=True)

Return a ctypes array allocated from shared memory. By default the return value is actually a synchronized wrapper for the array.

*typecode_or_type* determines the type of the elements of the returned array: it is either a `ctypes type <ctypes-fundamental-data-types>` or a one character typecode of the kind used by the `array` module with the exception of `'w'`, which is not supported. In addition, the `'c'` typecode is an alias for `ctypes.c_char`. If *size_or_initializer* is an integer, then it determines the length of the array, and the array will be initially zeroed. Otherwise, *size_or_initializer* is a sequence which is used to initialize the array and whose length determines the length of the array.

If *lock* is `True` (the default) then a new lock object is created to synchronize access to the value. If *lock* is a `Lock` or `RLock` object then that will be used to synchronize access to the value. If *lock* is `False` then access to the returned object will not be automatically protected by a lock, so it will not necessarily be "process-safe".

Note that *lock* is a keyword only argument.

Note that an array of `ctypes.c_char` has *value* and *raw* attributes which can both be used to store and retrieve byte strings. While *raw* allows interaction with a `bytes` object the full size of the array, reading *value* will terminate after a null byte, like most programming languages handle strings.

</div>

#### The `!multiprocessing.sharedctypes` module

<div class="module" data-synopsis="Allocate ctypes objects from shared memory.">

multiprocessing.sharedctypes

</div>

The `!multiprocessing.sharedctypes` module provides functions for allocating `ctypes` objects from shared memory which can be inherited by child processes.

> [!NOTE]
> Although it is possible to store a pointer in shared memory remember that this will refer to a location in the address space of a specific process. However, the pointer is quite likely to be invalid in the context of a second process and trying to dereference the pointer from the second process may cause a crash.

<div class="function">

RawArray(typecode_or_type, size_or_initializer)

Return a ctypes array allocated from shared memory.

*typecode_or_type* determines the type of the elements of the returned array: it is either a ctypes type or a one character typecode of the kind used by the `array` module. If *size_or_initializer* is an integer then it determines the length of the array, and the array will be initially zeroed. Otherwise *size_or_initializer* is a sequence which is used to initialize the array and whose length determines the length of the array.

Note that setting and getting an element is potentially non-atomic -- use `Array` instead to make sure that access is automatically synchronized using a lock.

</div>

<div class="function">

RawValue(typecode_or_type, \*args)

Return a ctypes object allocated from shared memory.

*typecode_or_type* determines the type of the returned object: it is either a ctypes type or a one character typecode of the kind used by the `array` module. *\*args* is passed on to the constructor for the type.

Note that setting and getting the value is potentially non-atomic -- use `Value` instead to make sure that access is automatically synchronized using a lock.

Note that an array of `ctypes.c_char` has `value` and `raw` attributes which allow one to use it to store and retrieve strings -- see documentation for `ctypes`.

</div>

<div class="function">

Array(typecode_or_type, size_or_initializer, \*, lock=True, ctx=None)

The same as `RawArray` except that depending on the value of *lock* a process-safe synchronization wrapper may be returned instead of a raw ctypes array.

If *lock* is `True` (the default) then a new lock object is created to synchronize access to the value. If *lock* is a `~multiprocessing.Lock` or `~multiprocessing.RLock` object then that will be used to synchronize access to the value. If *lock* is `False` then access to the returned object will not be automatically protected by a lock, so it will not necessarily be "process-safe".

*ctx* is a context object, or `None` (use the current context). If `None`, calling this may set the global start method. See `global-start-method` for more details.

Note that *lock* and *ctx* are keyword-only parameters.

</div>

<div class="function">

Value(typecode_or_type, \*args, lock=True, ctx=None)

The same as `RawValue` except that depending on the value of *lock* a process-safe synchronization wrapper may be returned instead of a raw ctypes object.

If *lock* is `True` (the default) then a new lock object is created to synchronize access to the value. If *lock* is a `~multiprocessing.Lock` or `~multiprocessing.RLock` object then that will be used to synchronize access to the value. If *lock* is `False` then access to the returned object will not be automatically protected by a lock, so it will not necessarily be "process-safe".

*ctx* is a context object, or `None` (use the current context). If `None`, calling this may set the global start method. See `global-start-method` for more details.

Note that *lock* and *ctx* are keyword-only parameters.

</div>

<div class="function">

copy(obj)

Return a ctypes object allocated from shared memory which is a copy of the ctypes object *obj*.

</div>

<div class="function">

synchronized(obj, lock=None, ctx=None)

Return a process-safe wrapper object for a ctypes object which uses *lock* to synchronize access. If *lock* is `None` (the default) then a `multiprocessing.RLock` object is created automatically.

*ctx* is a context object, or `None` (use the current context). If `None`, calling this may set the global start method. See `global-start-method` for more details.

A synchronized wrapper will have two methods in addition to those of the object it wraps: `get_obj` returns the wrapped object and `get_lock` returns the lock object used for synchronization.

Note that accessing the ctypes object through the wrapper can be a lot slower than accessing the raw ctypes object.

<div class="versionchanged">

3.5 Synchronized objects support the `context manager` protocol.

</div>

</div>

The table below compares the syntax for creating shared ctypes objects from shared memory with the normal ctypes syntax. (In the table `MyStruct` is some subclass of `ctypes.Structure`.)

| ctypes | sharedctypes using type | sharedctypes using typecode |
|----|----|----|
| c_double(2.4) | RawValue(c_double, 2.4) | RawValue('d', 2.4) |
| MyStruct(4, 6) | RawValue(MyStruct, 4, 6) |  |
| (c_short \* 7)() | RawArray(c_short, 7) | RawArray('h', 7) |
| (c_int \* 3)(9, 2, 8) | RawArray(c_int, (9, 2, 8)) | RawArray('i', (9, 2, 8)) |

Below is an example where a number of ctypes objects are modified by a child process:

    from multiprocessing import Process, Lock
    from multiprocessing.sharedctypes import Value, Array
    from ctypes import Structure, c_double

    class Point(Structure):
        _fields_ = [('x', c_double), ('y', c_double)]

    def modify(n, x, s, A):
        n.value **= 2
        x.value **= 2
        s.value = s.value.upper()
        for a in A:
            a.x **= 2
            a.y **= 2

    if __name__ == '__main__':
        lock = Lock()

        n = Value('i', 7)
        x = Value(c_double, 1.0/3.0, lock=False)
        s = Array('c', b'hello world', lock=lock)
        A = Array(Point, [(1.875,-6.25), (-5.75,2.0), (2.375,9.5)], lock=lock)

        p = Process(target=modify, args=(n, x, s, A))
        p.start()
        p.join()

        print(n.value)
        print(x.value)
        print(s.value)
        print([(a.x, a.y) for a in A])

The results printed are :

```
49
0.1111111111111111
HELLO WORLD
[(3.515625, 39.0625), (33.0625, 4.0), (5.640625, 90.25)]
```

### Managers

Managers provide a way to create data which can be shared between different processes, including sharing over a network between processes running on different machines. A manager object controls a server process which manages *shared objects*. Other processes can access the shared objects by using proxies.

<div class="function" data-module="">

multiprocessing.Manager()

Returns a started `~multiprocessing.managers.SyncManager` object which can be used for sharing objects between processes. The returned manager object corresponds to a spawned child process and has methods which will create shared objects and return corresponding proxies.

</div>

<div class="module" data-synopsis="Share data between process with shared objects.">

multiprocessing.managers

</div>

Manager processes will be shutdown as soon as they are garbage collected or their parent process exits. The manager classes are defined in the `multiprocessing.managers` module:

<div class="BaseManager(address=None, authkey=None, serializer='pickle', ctx=None, *, shutdown_timeout=1.0)">

Create a BaseManager object.

Once created one should call `start` or `get_server().serve_forever()` to ensure that the manager object refers to a started manager process.

*address* is the address on which the manager process listens for new connections. If *address* is `None` then an arbitrary one is chosen.

*authkey* is the authentication key which will be used to check the validity of incoming connections to the server process. If *authkey* is `None` then `current_process().authkey` is used. Otherwise *authkey* is used and it must be a byte string.

*serializer* must be `'pickle'` (use `pickle` serialization) or `'xmlrpclib'` (use `xmlrpc.client` serialization).

*ctx* is a context object, or `None` (use the current context). If `None`, calling this may set the global start method. See `global-start-method` for more details.

*shutdown_timeout* is a timeout in seconds used to wait until the process used by the manager completes in the `shutdown` method. If the shutdown times out, the process is terminated. If terminating the process also times out, the process is killed.

<div class="versionchanged">

3.11 Added the *shutdown_timeout* parameter.

</div>

<div class="method">

start(\[initializer\[, initargs\]\])

Start a subprocess to start the manager. If *initializer* is not `None` then the subprocess will call `initializer(*initargs)` when it starts.

</div>

<div class="method">

get_server()

Returns a `Server` object which represents the actual server under the control of the Manager. The `Server` object supports the `serve_forever` method:

```
>>> from multiprocessing.managers import BaseManager
>>> manager = BaseManager(address=('', 50000), authkey=b'abc')
>>> server = manager.get_server()
>>> server.serve_forever()
```

`Server` additionally has an `address` attribute.

</div>

<div class="method">

connect()

Connect a local manager object to a remote manager process:

```
>>> from multiprocessing.managers import BaseManager
>>> m = BaseManager(address=('127.0.0.1', 50000), authkey=b'abc')
>>> m.connect()
```

</div>

<div class="method">

shutdown()

Stop the process used by the manager. This is only available if `start` has been used to start the server process.

This can be called multiple times.

</div>

<div class="method">

register(typeid\[, callable\[, proxytype\[, exposed\[, method_to_typeid\[, create_method\]\]\]\]\])

A classmethod which can be used for registering a type or callable with the manager class.

*typeid* is a "type identifier" which is used to identify a particular type of shared object. This must be a string.

*callable* is a callable used for creating objects for this type identifier. If a manager instance will be connected to the server using the `connect` method, or if the *create_method* argument is `False` then this can be left as `None`.

*proxytype* is a subclass of `BaseProxy` which is used to create proxies for shared objects with this *typeid*. If `None` then a proxy class is created automatically.

*exposed* is used to specify a sequence of method names which proxies for this typeid should be allowed to access using `BaseProxy._callmethod`. (If *exposed* is `None` then `proxytype._exposed_` is used instead if it exists.) In the case where no exposed list is specified, all "public methods" of the shared object will be accessible. (Here a "public method" means any attribute which has a `~object.__call__` method and whose name does not begin with `'_'`.)

*method_to_typeid* is a mapping used to specify the return type of those exposed methods which should return a proxy. It maps method names to typeid strings. (If *method_to_typeid* is `None` then `proxytype._method_to_typeid_` is used instead if it exists.) If a method's name is not a key of this mapping or if the mapping is `None` then the object returned by the method will be copied by value.

*create_method* determines whether a method should be created with name *typeid* which can be used to tell the server process to create a new shared object and return a proxy for it. By default it is `True`.

</div>

`BaseManager` instances also have one read-only property:

<div class="attribute">

address

The address used by the manager.

</div>

<div class="versionchanged">

3.3 Manager objects support the context management protocol -- see `typecontextmanager`. `~contextmanager.__enter__` starts the server process (if it has not already started) and then returns the manager object. `~contextmanager.__exit__` calls `shutdown`.

In previous versions `~contextmanager.__enter__` did not start the manager's server process if it was not already started.

</div>

</div>

<div class="SyncManager">

A subclass of `BaseManager` which can be used for the synchronization of processes. Objects of this type are returned by `multiprocessing.Manager`.

Its methods create and return `multiprocessing-proxy_objects` for a number of commonly used data types to be synchronized across processes. This notably includes shared lists and dictionaries.

<div class="method">

Barrier(parties\[, action\[, timeout\]\])

Create a shared `threading.Barrier` object and return a proxy for it.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

BoundedSemaphore(\[value\])

Create a shared `threading.BoundedSemaphore` object and return a proxy for it.

</div>

<div class="method">

Condition(\[lock\])

Create a shared `threading.Condition` object and return a proxy for it.

If *lock* is supplied then it should be a proxy for a `threading.Lock` or `threading.RLock` object.

<div class="versionchanged">

3.3 The `~threading.Condition.wait_for` method was added.

</div>

</div>

<div class="method">

Event()

Create a shared `threading.Event` object and return a proxy for it.

</div>

<div class="method">

Lock()

Create a shared `threading.Lock` object and return a proxy for it.

</div>

<div class="method">

Namespace()

Create a shared `Namespace` object and return a proxy for it.

</div>

<div class="method">

Queue(\[maxsize\])

Create a shared `queue.Queue` object and return a proxy for it.

</div>

<div class="method">

RLock()

Create a shared `threading.RLock` object and return a proxy for it.

</div>

<div class="method">

Semaphore(\[value\])

Create a shared `threading.Semaphore` object and return a proxy for it.

</div>

<div class="method">

Array(typecode, sequence)

Create an array and return a proxy for it.

</div>

<div class="method">

Value(typecode, value)

Create an object with a writable `value` attribute and return a proxy for it.

</div>

<div class="method">

dict() dict(mapping) dict(sequence)

Create a shared `dict` object and return a proxy for it.

</div>

<div class="method">

list() list(sequence)

Create a shared `list` object and return a proxy for it.

</div>

<div class="method">

set() set(sequence) set(mapping)

Create a shared `set` object and return a proxy for it.

<div class="versionadded">

3.14 `set` support was added.

</div>

</div>

<div class="versionchanged">

3.6 Shared objects are capable of being nested. For example, a shared container object such as a shared list can contain other shared objects which will all be managed and synchronized by the `SyncManager`.

</div>

</div>

<div class="Namespace">

A type that can register with `SyncManager`.

A namespace object has no public methods, but does have writable attributes. Its representation shows the values of its attributes.

However, when using a proxy for a namespace object, an attribute beginning with `'_'` will be an attribute of the proxy and not an attribute of the referent:

<div class="doctest">

\>\>\> mp_context = multiprocessing.get_context('spawn') \>\>\> manager = mp_context.Manager() \>\>\> Global = manager.Namespace() \>\>\> Global.x = 10 \>\>\> Global.y = 'hello' \>\>\> Global.\_z = 12.3 \# this is an attribute of the proxy \>\>\> print(Global) Namespace(x=10, y='hello')

</div>

</div>

#### Customized managers

To create one's own manager, one creates a subclass of `BaseManager` and uses the `~BaseManager.register` classmethod to register new types or callables with the manager class. For example:

```
from multiprocessing.managers import BaseManager

class MathsClass:
    def add(self, x, y):
        return x + y
    def mul(self, x, y):
        return x * y

class MyManager(BaseManager):
    pass

MyManager.register('Maths', MathsClass)

if __name__ == '__main__':
    with MyManager() as manager:
        maths = manager.Maths()
        print(maths.add(4, 3))         # prints 7
        print(maths.mul(7, 8))         # prints 56
```

#### Using a remote manager

It is possible to run a manager server on one machine and have clients use it from other machines (assuming that the firewalls involved allow it).

Running the following commands creates a server for a single shared queue which remote clients can access:

```
>>> from multiprocessing.managers import BaseManager
>>> from queue import Queue
>>> queue = Queue()
>>> class QueueManager(BaseManager): pass
>>> QueueManager.register('get_queue', callable=lambda:queue)
>>> m = QueueManager(address=('', 50000), authkey=b'abracadabra')
>>> s = m.get_server()
>>> s.serve_forever()
```

One client can access the server as follows:

```
>>> from multiprocessing.managers import BaseManager
>>> class QueueManager(BaseManager): pass
>>> QueueManager.register('get_queue')
>>> m = QueueManager(address=('foo.bar.org', 50000), authkey=b'abracadabra')
>>> m.connect()
>>> queue = m.get_queue()
>>> queue.put('hello')
```

Another client can also use it:

```
>>> from multiprocessing.managers import BaseManager
>>> class QueueManager(BaseManager): pass
>>> QueueManager.register('get_queue')
>>> m = QueueManager(address=('foo.bar.org', 50000), authkey=b'abracadabra')
>>> m.connect()
>>> queue = m.get_queue()
>>> queue.get()
'hello'
```

Local processes can also access that queue, using the code from above on the client to access it remotely:

```
>>> from multiprocessing import Process, Queue
>>> from multiprocessing.managers import BaseManager
>>> class Worker(Process):
...     def __init__(self, q):
...         self.q = q
...         super().__init__()
...     def run(self):
...         self.q.put('local hello')
...
>>> queue = Queue()
>>> w = Worker(queue)
>>> w.start()
>>> class QueueManager(BaseManager): pass
...
>>> QueueManager.register('get_queue', callable=lambda: queue)
>>> m = QueueManager(address=('', 50000), authkey=b'abracadabra')
>>> s = m.get_server()
>>> s.serve_forever()
```

### Proxy Objects

A proxy is an object which *refers* to a shared object which lives (presumably) in a different process. The shared object is said to be the *referent* of the proxy. Multiple proxy objects may have the same referent.

A proxy object has methods which invoke corresponding methods of its referent (although not every method of the referent will necessarily be available through the proxy). In this way, a proxy can be used just like its referent can:

<div class="doctest">

\>\>\> mp_context = multiprocessing.get_context('spawn') \>\>\> manager = mp_context.Manager() \>\>\> l = manager.list(\[i\*i for i in range(10)\]) \>\>\> print(l) \[0, 1, 4, 9, 16, 25, 36, 49, 64, 81\] \>\>\> print(repr(l)) \<ListProxy object, typeid 'list' at 0x...\> \>\>\> l\[4\] 16 \>\>\> l\[2:5\] \[4, 9, 16\]

</div>

Notice that applying `str` to a proxy will return the representation of the referent, whereas applying `repr` will return the representation of the proxy.

An important feature of proxy objects is that they are picklable so they can be passed between processes. As such, a referent can contain `multiprocessing-proxy_objects`. This permits nesting of these managed lists, dicts, and other `multiprocessing-proxy_objects`:

<div class="doctest">

\>\>\> a = manager.list() \>\>\> b = manager.list() \>\>\> a.append(b) \# referent of a now contains referent of b \>\>\> print(a, b) \[\<ListProxy object, typeid 'list' at ...\>\] \[\] \>\>\> b.append('hello') \>\>\> print(a\[0\], b) \['hello'\] \['hello'\]

</div>

Similarly, dict and list proxies may be nested inside one another:

```
>>> l_outer = manager.list([ manager.dict() for i in range(2) ])
>>> d_first_inner = l_outer[0]
>>> d_first_inner['a'] = 1
>>> d_first_inner['b'] = 2
>>> l_outer[1]['c'] = 3
>>> l_outer[1]['z'] = 26
>>> print(l_outer[0])
{'a': 1, 'b': 2}
>>> print(l_outer[1])
{'c': 3, 'z': 26}
```

If standard (non-proxy) `list` or `dict` objects are contained in a referent, modifications to those mutable values will not be propagated through the manager because the proxy has no way of knowing when the values contained within are modified. However, storing a value in a container proxy (which triggers a `__setitem__` on the proxy object) does propagate through the manager and so to effectively modify such an item, one could re-assign the modified value to the container proxy:

```
# create a list proxy and append a mutable object (a dictionary)
lproxy = manager.list()
lproxy.append({})
# now mutate the dictionary
d = lproxy[0]
d['a'] = 1
d['b'] = 2
# at this point, the changes to d are not yet synced, but by
# updating the dictionary, the proxy is notified of the change
lproxy[0] = d
```

This approach is perhaps less convenient than employing nested `multiprocessing-proxy_objects` for most use cases but also demonstrates a level of control over the synchronization.

> [!NOTE]
> The proxy types in `!multiprocessing` do nothing to support comparisons by value. So, for instance, we have:
>
> <div class="doctest">
>
> \>\>\> manager.list(\[1,2,3\]) == \[1,2,3\] False
>
> </div>
>
> One should just use a copy of the referent instead when making comparisons.

<div class="BaseProxy">

Proxy objects are instances of subclasses of `BaseProxy`.

<div class="method">

<span id="callmethod">callmethod</span>(methodname\[, args\[, kwds\]\])

Call and return the result of a method of the proxy's referent.

If `proxy` is a proxy whose referent is `obj` then the expression :

```
proxy._callmethod(methodname, args, kwds)
```

will evaluate the expression :

```
getattr(obj, methodname)(*args, **kwds)
```

in the manager's process.

The returned value will be a copy of the result of the call or a proxy to a new shared object -- see documentation for the *method_to_typeid* argument of `BaseManager.register`.

If an exception is raised by the call, then is re-raised by `_callmethod`. If some other exception is raised in the manager's process then this is converted into a `RemoteError` exception and is raised by `_callmethod`.

Note in particular that an exception will be raised if *methodname* has not been *exposed*.

An example of the usage of `_callmethod`:

<div class="doctest">

\>\>\> l = manager.list(range(10)) \>\>\> l.\_callmethod('\_\_len\_\_') 10 \>\>\> l.\_callmethod('\_\_getitem\_\_', (slice(2, 7),)) \# equivalent to l\[2:7\] \[2, 3, 4, 5, 6\] \>\>\> l.\_callmethod('\_\_getitem\_\_', (20,)) \# equivalent to l\[20\] Traceback (most recent call last): ... IndexError: list index out of range

</div>

</div>

<div class="method">

<span id="getvalue">getvalue</span>()

Return a copy of the referent.

If the referent is unpicklable then this will raise an exception.

</div>

<div class="method">

\_\_repr\_\_

Return a representation of the proxy object.

</div>

<div class="method">

\_\_str\_\_

Return the representation of the referent.

</div>

</div>

#### Cleanup

A proxy object uses a weakref callback so that when it gets garbage collected it deregisters itself from the manager which owns its referent.

A shared object gets deleted from the manager process when there are no longer any proxies referring to it.

### Process Pools

<div class="module" data-synopsis="Create pools of processes.">

multiprocessing.pool

</div>

One can create a pool of processes which will carry out tasks submitted to it with the `Pool` class.

<div class="Pool([processes[, initializer[, initargs[, maxtasksperchild [, context]]]]])">

A process pool object which controls a pool of worker processes to which jobs can be submitted. It supports asynchronous results with timeouts and callbacks and has a parallel map implementation.

*processes* is the number of worker processes to use. If *processes* is `None` then the number returned by `os.process_cpu_count` is used.

If *initializer* is not `None` then each worker process will call `initializer(*initargs)` when it starts.

*maxtasksperchild* is the number of tasks a worker process can complete before it will exit and be replaced with a fresh worker process, to enable unused resources to be freed. The default *maxtasksperchild* is `None`, which means worker processes will live as long as the pool.

*context* can be used to specify the context used for starting the worker processes. Usually a pool is created using the function `multiprocessing.Pool` or the `Pool` method of a context object. In both cases *context* is set appropriately. If `None`, calling this function will have the side effect of setting the current global start method if it has not been set already. See the `get_context` function.

Note that the methods of the pool object should only be called by the process which created the pool.

> [!WARNING]
> `~multiprocessing.pool.Pool` objects have internal resources that need to be properly managed (like any other resource) by using the pool as a context manager or by calling `close` and `terminate` manually. Failure to do this can lead to the process hanging on finalization.
>
> Note that it is **not correct** to rely on the garbage collector to destroy the pool as CPython does not assure that the finalizer of the pool will be called (see `object.__del__` for more information).

<div class="versionchanged">

3.2 Added the *maxtasksperchild* parameter.

</div>

<div class="versionchanged">

3.4 Added the *context* parameter.

</div>

<div class="versionchanged">

3.13 *processes* uses `os.process_cpu_count` by default, instead of `os.cpu_count`.

</div>

> [!NOTE]
> Worker processes within a `Pool` typically live for the complete duration of the Pool's work queue. A frequent pattern found in other systems (such as Apache, mod_wsgi, etc) to free resources held by workers is to allow a worker within a pool to complete only a set amount of work before exiting, being cleaned up and a new process spawned to replace the old one. The *maxtasksperchild* argument to the `Pool` exposes this ability to the end user.

<div class="method">

apply(func\[, args\[, kwds\]\])

Call *func* with arguments *args* and keyword arguments *kwds*. It blocks until the result is ready. Given this blocks, `apply_async` is better suited for performing work in parallel. Additionally, *func* is only executed in one of the workers of the pool.

</div>

<div class="method">

apply_async(func\[, args\[, kwds\[, callback\[, error_callback\]\]\]\])

A variant of the `apply` method which returns a `~multiprocessing.pool.AsyncResult` object.

If *callback* is specified then it should be a callable which accepts a single argument. When the result becomes ready *callback* is applied to it, that is unless the call failed, in which case the *error_callback* is applied instead.

If *error_callback* is specified then it should be a callable which accepts a single argument. If the target function fails, then the *error_callback* is called with the exception instance.

Callbacks should complete immediately since otherwise the thread which handles the results will get blocked.

</div>

<div class="method">

map(func, iterable\[, chunksize\])

A parallel equivalent of the `map` built-in function (it supports only one *iterable* argument though, for multiple iterables see `starmap`). It blocks until the result is ready.

This method chops the iterable into a number of chunks which it submits to the process pool as separate tasks. The (approximate) size of these chunks can be specified by setting *chunksize* to a positive integer.

Note that it may cause high memory usage for very long iterables. Consider using `imap` or `imap_unordered` with explicit *chunksize* option for better efficiency.

</div>

<div class="method">

map_async(func, iterable\[, chunksize\[, callback\[, error_callback\]\]\])

A variant of the `.map` method which returns a `~multiprocessing.pool.AsyncResult` object.

If *callback* is specified then it should be a callable which accepts a single argument. When the result becomes ready *callback* is applied to it, that is unless the call failed, in which case the *error_callback* is applied instead.

If *error_callback* is specified then it should be a callable which accepts a single argument. If the target function fails, then the *error_callback* is called with the exception instance.

Callbacks should complete immediately since otherwise the thread which handles the results will get blocked.

</div>

<div class="method">

imap(func, iterable, chunksize=1, \*, buffersize=None)

An iterator-based version of `.map`.

The *chunksize* argument is the same as the one used by the `.map` method. For very long iterables using a large value for *chunksize* can make the job complete **much** faster than using the default value of `1`.

Also if *chunksize* is `1` then the `!next` method of the iterator returned by the `imap` method has an optional *timeout* parameter: `next(timeout)` will raise `multiprocessing.TimeoutError` if the result cannot be returned within *timeout* seconds.

The *iterable* is collected immediately rather than lazily, unless a *buffersize* is specified to limit the number of submitted tasks whose results have not yet been yielded. If the buffer is full, iteration over the *iterables* pauses until a result is yielded from the buffer. To fully utilize pool's capacity when using this feature, set *buffersize* at least to the number of processes in pool (to consume *iterable* as you go), or even higher (to prefetch the next `N=buffersize-processes` arguments).

<div class="versionchanged">

3.16 Added the *buffersize* parameter.

</div>

</div>

<div class="method">

imap_unordered(func, iterable, chunksize=1, \*, buffersize=None)

The same as `imap` except that the ordering of the results from the returned iterator should be considered arbitrary. (Only when there is only one worker process is the order guaranteed to be "correct".)

<div class="versionchanged">

3.16 Added the *buffersize* parameter.

</div>

</div>

<div class="method">

starmap(func, iterable\[, chunksize\])

Like `~multiprocessing.pool.Pool.map` except that the elements of the *iterable* are expected to be iterables that are unpacked as arguments.

Hence an *iterable* of `[(1,2), (3, 4)]` results in `[func(1,2), func(3,4)]`.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

starmap_async(func, iterable\[, chunksize\[, callback\[, error_callback\]\]\])

A combination of `starmap` and `map_async` that iterates over *iterable* of iterables and calls *func* with the iterables unpacked. Returns a result object.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

close()

Prevents any more tasks from being submitted to the pool. Once all the tasks have been completed the worker processes will exit.

</div>

<div class="method">

terminate()

Stops the worker processes immediately without completing outstanding work. When the pool object is garbage collected `terminate` will be called immediately.

</div>

<div class="method">

join()

Wait for the worker processes to exit. One must call `close` or `terminate` before using `join`.

</div>

<div class="versionchanged">

3.3 Pool objects now support the context management protocol -- see `typecontextmanager`. `~contextmanager.__enter__` returns the pool object, and `~contextmanager.__exit__` calls `terminate`.

</div>

</div>

<div class="AsyncResult">

The class of the result returned by `Pool.apply_async` and `Pool.map_async`.

<div class="method">

get(\[timeout\])

Return the result when it arrives. If *timeout* is not `None` and the result does not arrive within *timeout* seconds then `multiprocessing.TimeoutError` is raised. If the remote call raised an exception then that exception will be reraised by `get`.

</div>

<div class="method">

wait(\[timeout\])

Wait until the result is available or until *timeout* seconds pass.

</div>

<div class="method">

ready()

Return whether the call has completed.

</div>

<div class="method">

successful()

Return whether the call completed without raising an exception. Will raise `ValueError` if the result is not ready.

<div class="versionchanged">

3.7 If the result is not ready, `ValueError` is raised instead of `AssertionError`.

</div>

</div>

</div>

The following example demonstrates the use of a pool:

```
from multiprocessing import Pool
import time

def f(x):
    return x*x

if __name__ == '__main__':
    with Pool(processes=4) as pool:         # start 4 worker processes
        result = pool.apply_async(f, (10,)) # evaluate "f(10)" asynchronously in a single process
        print(result.get(timeout=1))        # prints "100" unless your computer is *very* slow

        print(pool.map(f, range(10)))       # prints "[0, 1, 4,..., 81]"

        it = pool.imap(f, range(10))
        print(next(it))                     # prints "0"
        print(next(it))                     # prints "1"
        print(it.next(timeout=1))           # prints "4" unless your computer is *very* slow

        result = pool.apply_async(time.sleep, (10,))
        print(result.get(timeout=1))        # raises multiprocessing.TimeoutError
```

### Listeners and Clients

<div class="module" data-synopsis="API for dealing with sockets.">

multiprocessing.connection

</div>

Usually message passing between processes is done using queues or by using `~Connection` objects returned by `~multiprocessing.Pipe`.

However, the `!multiprocessing.connection` module allows some extra flexibility. It basically gives a high level message oriented API for dealing with sockets or Windows named pipes. It also has support for *digest authentication* using the `hmac` module, and for polling multiple connections at the same time.

<div class="function">

deliver_challenge(connection, authkey)

Send a randomly generated message to the other end of the connection and wait for a reply.

If the reply matches the digest of the message using *authkey* as the key then a welcome message is sent to the other end of the connection. Otherwise `~multiprocessing.AuthenticationError` is raised.

</div>

<div class="function">

answer_challenge(connection, authkey)

Receive a message, calculate the digest of the message using *authkey* as the key, and then send the digest back.

If a welcome message is not received, then `~multiprocessing.AuthenticationError` is raised.

</div>

<div class="function">

Client(address\[, family\[, authkey\]\])

Attempt to set up a connection to the listener which is using address *address*, returning a `~Connection`.

The type of the connection is determined by *family* argument, but this can generally be omitted since it can usually be inferred from the format of *address*. (See `multiprocessing-address-formats`)

If *authkey* is given and not `None`, it should be a byte string and will be used as the secret key for an HMAC-based authentication challenge. No authentication is done if *authkey* is `None`. `~multiprocessing.AuthenticationError` is raised if authentication fails. See `multiprocessing-auth-keys`.

</div>

<div class="Listener([address[, family[, backlog[, authkey]]]])">

A wrapper for a bound socket or Windows named pipe which is 'listening' for connections.

*address* is the address to be used by the bound socket or named pipe of the listener object.

> [!NOTE]
> If an address of '0.0.0.0' is used, the address will not be a connectable end point on Windows. If you require a connectable end-point, you should use '127.0.0.1'.

*family* is the type of socket (or named pipe) to use. This can be one of the strings `'AF_INET'` (for a TCP socket), `'AF_UNIX'` (for a Unix domain socket) or `'AF_PIPE'` (for a Windows named pipe). Of these only the first is guaranteed to be available. If *family* is `None` then the family is inferred from the format of *address*. If *address* is also `None` then a default is chosen. This default is the family which is assumed to be the fastest available. See `multiprocessing-address-formats`. Note that if *family* is `'AF_UNIX'` and address is `None` then the socket will be created in a private temporary directory created using `tempfile.mkstemp`.

If the listener object uses a socket then *backlog* (1 by default) is passed to the `~socket.socket.listen` method of the socket once it has been bound.

If *authkey* is given and not `None`, it should be a byte string and will be used as the secret key for an HMAC-based authentication challenge. No authentication is done if *authkey* is `None`. `~multiprocessing.AuthenticationError` is raised if authentication fails. See `multiprocessing-auth-keys`.

<div class="method">

accept()

Accept a connection on the bound socket or named pipe of the listener object and return a `~Connection` object. If authentication is attempted and fails, then `~multiprocessing.AuthenticationError` is raised.

</div>

<div class="method">

close()

Close the bound socket or named pipe of the listener object. This is called automatically when the listener is garbage collected. However it is advisable to call it explicitly.

</div>

Listener objects have the following read-only properties:

<div class="attribute">

address

The address which is being used by the Listener object.

</div>

<div class="attribute">

last_accepted

The address from which the last accepted connection came. If this is unavailable then it is `None`.

</div>

<div class="versionchanged">

3.3 Listener objects now support the context management protocol -- see `typecontextmanager`. `~contextmanager.__enter__` returns the listener object, and `~contextmanager.__exit__` calls `close`.

</div>

</div>

<div class="function">

wait(object_list, timeout=None)

Wait till an object in *object_list* is ready. Returns the list of those objects in *object_list* which are ready. If *timeout* is a float then the call blocks for at most that many seconds. If *timeout* is `None` then it will block for an unlimited period. A negative timeout is equivalent to a zero timeout.

For both POSIX and Windows, an object can appear in *object_list* if it is

- a readable `~multiprocessing.connection.Connection` object;
- a connected and readable `socket.socket` object; or
- the `~multiprocessing.Process.sentinel` attribute of a `~multiprocessing.Process` object.

A connection or socket object is ready when there is data available to be read from it, or the other end has been closed.

**POSIX**: `wait(object_list, timeout)` almost equivalent `select.select(object_list, [], [], timeout)`. The difference is that, if `select.select` is interrupted by a signal, it can raise `OSError` with an error number of `EINTR`, whereas `wait` will not.

**Windows**: An item in *object_list* must either be an integer handle which is waitable (according to the definition used by the documentation of the Win32 function `WaitForMultipleObjects()`) or it can be an object with a `~io.IOBase.fileno` method which returns a socket handle or pipe handle. (Note that pipe handles and socket handles are **not** waitable handles.)

<div class="versionadded">

3.3

</div>

</div>

**Examples**

The following server code creates a listener which uses `'secret password'` as an authentication key. It then waits for a connection and sends some data to the client:

```
from multiprocessing.connection import Listener
from array import array

address = ('localhost', 6000)     # family is deduced to be 'AF_INET'

with Listener(address, authkey=b'secret password') as listener:
    with listener.accept() as conn:
        print('connection accepted from', listener.last_accepted)

        conn.send([2.25, None, 'junk', float])

        conn.send_bytes(b'hello')

        conn.send_bytes(array('i', [42, 1729]))
```

The following code connects to the server and receives some data from the server:

```
from multiprocessing.connection import Client
from array import array

address = ('localhost', 6000)

with Client(address, authkey=b'secret password') as conn:
    print(conn.recv())                  # => [2.25, None, 'junk', float]

    print(conn.recv_bytes())            # => 'hello'

    arr = array('i', [0, 0, 0, 0, 0])
    print(conn.recv_bytes_into(arr))    # => 8
    print(arr)                          # => array('i', [42, 1729, 0, 0, 0])
```

The following code uses `~multiprocessing.connection.wait` to wait for messages from multiple processes at once:

```
from multiprocessing import Process, Pipe, current_process
from multiprocessing.connection import wait

def foo(w):
    for i in range(10):
        w.send((i, current_process().name))
    w.close()

if __name__ == '__main__':
    readers = []

    for i in range(4):
        r, w = Pipe(duplex=False)
        readers.append(r)
        p = Process(target=foo, args=(w,))
        p.start()
        # We close the writable end of the pipe now to be sure that
        # p is the only process which owns a handle for it.  This
        # ensures that when p closes its handle for the writable end,
        # wait() will promptly report the readable end as being ready.
        w.close()

    while readers:
        for r in wait(readers):
            try:
                msg = r.recv()
            except EOFError:
                readers.remove(r)
            else:
                print(msg)
```

#### Address Formats

- An `'AF_INET'` address is a tuple of the form `(hostname, port)` where *hostname* is a string and *port* is an integer.
- An `'AF_UNIX'` address is a string representing a filename on the filesystem.
- An `'AF_PIPE'` address is a string of the form `r'\\\\\\.\\pipe\\\\{PipeName}'`. To use `Client` to connect to a named pipe on a remote computer called *ServerName* one should use an address of the form `r'\\\\\\\\{ServerName}\\pipe\\\\{PipeName}'` instead.

Note that any string beginning with two backslashes is assumed by default to be an `'AF_PIPE'` address rather than an `'AF_UNIX'` address.

### Authentication keys

When one uses `Connection.recv <Connection.recv>`, the data received is automatically unpickled. Unfortunately unpickling data from an untrusted source is a security risk. Therefore `Listener` and `Client` use the `hmac` module to provide digest authentication.

An authentication key is a byte string which can be thought of as a password: once a connection is established both ends will demand proof that the other knows the authentication key. (Demonstrating that both ends are using the same key does **not** involve sending the key over the connection.)

If authentication is requested but no authentication key is specified then the return value of `current_process().authkey` is used (see `~multiprocessing.Process`). This value will be automatically inherited by any `~multiprocessing.Process` object that the current process creates. This means that (by default) all processes of a multi-process program will share a single authentication key which can be used when setting up connections between themselves.

Suitable authentication keys can also be generated by using `os.urandom`.

This authentication protects `Listener` and `Client` connections, which are reachable by address. It is not applied to the anonymous pipes created by `~multiprocessing.Pipe` or used internally by `~multiprocessing.Queue`. `multiprocessing` treats all local processes running as the same user as trusted; on most operating systems such processes can access each other's pipe file descriptors regardless. Applications that require isolation between processes of the same user must arrange it at the operating-system level -- for example, by running workers under a different user account or in a sandbox.

### Logging

Some support for logging is available. Note, however, that the `logging` package does not use process shared locks so it is possible (depending on the handler type) for messages from different processes to get mixed up.

<div class="currentmodule">

multiprocessing

</div>

<div class="function">

get_logger()

Returns the logger used by `!multiprocessing`. If necessary, a new one will be created.

When first created the logger has level `logging.NOTSET` and no default handler. Messages sent to this logger will not by default propagate to the root logger.

Note that on Windows child processes will only inherit the level of the parent process's logger -- any other customization of the logger will not be inherited.

</div>

<div class="currentmodule">

multiprocessing

</div>

<div class="function">

log_to_stderr(level=None)

This function performs a call to `get_logger` but in addition to returning the logger created by get_logger, it adds a handler which sends output to `sys.stderr` using format `'[%(levelname)s/%(processName)s] %(message)s'`. You can modify `levelname` of the logger by passing a `level` argument.

</div>

Below is an example session with logging turned on:

```
>>> import multiprocessing, logging
>>> logger = multiprocessing.log_to_stderr()
>>> logger.setLevel(logging.INFO)
>>> logger.warning('doomed')
[WARNING/MainProcess] doomed
>>> m = multiprocessing.Manager()
[INFO/SyncManager-...] child process calling self.run()
[INFO/SyncManager-...] created temp directory /.../pymp-...
[INFO/SyncManager-...] manager serving at '/.../listener-...'
>>> del m
[INFO/MainProcess] sending shutdown message to manager
[INFO/SyncManager-...] manager exiting with exitcode 0
```

For a full table of logging levels, see the `logging` module.

### The `!multiprocessing.dummy` module

<div class="module" data-synopsis="Dumb wrapper around threading.">

multiprocessing.dummy

</div>

`!multiprocessing.dummy` replicates the API of `!multiprocessing` but is no more than a wrapper around the `threading` module.

<div class="currentmodule">

multiprocessing.pool

</div>

In particular, the `Pool` function provided by `!multiprocessing.dummy` returns an instance of `ThreadPool`, which is a subclass of `Pool` that supports all the same method calls but uses a pool of worker threads rather than worker processes.

<div class="ThreadPool([processes[, initializer[, initargs]]])">

A thread pool object which controls a pool of worker threads to which jobs can be submitted. `ThreadPool` instances are fully interface compatible with `Pool` instances, and their resources must also be properly managed, either by using the pool as a context manager or by calling `~multiprocessing.pool.Pool.close` and `~multiprocessing.pool.Pool.terminate` manually.

*processes* is the number of worker threads to use. If *processes* is `None` then the number returned by `os.process_cpu_count` is used.

If *initializer* is not `None` then each worker process will call `initializer(*initargs)` when it starts.

Unlike `Pool`, *maxtasksperchild* and *context* cannot be provided.

> [!NOTE]
> A `ThreadPool` shares the same interface as `Pool`, which is designed around a pool of processes and predates the introduction of the `concurrent.futures` module. As such, it inherits some operations that don't make sense for a pool backed by threads, and it has its own type for representing the status of asynchronous jobs, `AsyncResult`, that is not understood by any other libraries.
>
> Users should generally prefer to use `concurrent.futures.ThreadPoolExecutor`, which has a simpler interface that was designed around threads from the start, and which returns `concurrent.futures.Future` instances that are compatible with many other libraries, including `asyncio`.

</div>

## Programming guidelines

There are certain guidelines and idioms which should be adhered to when using `!multiprocessing`.

### All start methods

The following applies to all start methods.

Avoid shared state

> As far as possible one should try to avoid shifting large amounts of data between processes.
>
> It is probably best to stick to using queues or pipes for communication between processes rather than using the lower level synchronization primitives.

Picklability

> Ensure that the arguments to the methods of proxies are picklable.

Thread safety of proxies

> Do not use a proxy object from more than one thread unless you protect it with a lock.
>
> (There is never a problem with different processes using the *same* proxy.)

Joining zombie processes

> On POSIX when a process finishes but has not been joined it becomes a zombie. There should never be very many because each time a new process starts (or `~multiprocessing.active_children` is called) all completed processes which have not yet been joined will be joined. Also calling a finished process's `Process.is_alive <multiprocessing.Process.is_alive>` will join the process. Even so it is probably good practice to explicitly join all the processes that you start.

Better to inherit than pickle/unpickle

> When using the *spawn* or *forkserver* start methods many types from `!multiprocessing` need to be picklable so that child processes can use them. However, one should generally avoid sending shared objects to other processes using pipes or queues. Instead you should arrange the program so that a process which needs access to a shared resource created elsewhere can inherit it from an ancestor process.

Avoid terminating processes

> Using the `Process.terminate <multiprocessing.Process.terminate>` method to stop a process is liable to cause any shared resources (such as locks, semaphores, pipes and queues) currently being used by the process to become broken or unavailable to other processes.
>
> Therefore it is probably best to only consider using `Process.terminate <multiprocessing.Process.terminate>` on processes which never use any shared resources.

Joining processes that use queues

> Bear in mind that a process that has put items in a queue will wait before terminating until all the buffered items are fed by the "feeder" thread to the underlying pipe. (The child process can call the `Queue.cancel_join_thread <multiprocessing.Queue.cancel_join_thread>` method of the queue to avoid this behaviour.)
>
> This means that whenever you use a queue you need to make sure that all items which have been put on the queue will eventually be removed before the process is joined. Otherwise you cannot be sure that processes which have put items on the queue will terminate. Remember also that non-daemonic processes will be joined automatically.
>
> An example which will deadlock is the following:
>
> ```
> from multiprocessing import Process, Queue
>
> def f(q):
>     q.put('X' * 1000000)
>
> if __name__ == '__main__':
>     queue = Queue()
>     p = Process(target=f, args=(queue,))
>     p.start()
>     p.join()                    # this deadlocks
>     obj = queue.get()
> ```
>
> A fix here would be to swap the last two lines (or simply remove the `p.join()` line).

Explicitly pass resources to child processes

> On POSIX using the *fork* start method, a child process can make use of a shared resource created in a parent process using a global resource. However, it is better to pass the object as an argument to the constructor for the child process.
>
> Apart from making the code (potentially) compatible with Windows and the other start methods this also ensures that as long as the child process is still alive the object will not be garbage collected in the parent process. This might be important if some resource is freed when the object is garbage collected in the parent process.
>
> So for instance :
>
> ```
> from multiprocessing import Process, Lock
>
> def f():
>     ... do something using "lock" ...
>
> if __name__ == '__main__':
>     lock = Lock()
>     for i in range(10):
>         Process(target=f).start()
> ```
>
> should be rewritten as :
>
> ```
> from multiprocessing import Process, Lock
>
> def f(l):
>     ... do something using "l" ...
>
> if __name__ == '__main__':
>     lock = Lock()
>     for i in range(10):
>         Process(target=f, args=(lock,)).start()
> ```

Beware of replacing `sys.stdin` with a "file like object"

> `!multiprocessing` originally unconditionally called:
>
> ```
> os.close(sys.stdin.fileno())
> ```
>
> in the `multiprocessing.Process._bootstrap` method --- this resulted in issues with processes-in-processes. This has been changed to:
>
> ```
> sys.stdin.close()
> sys.stdin = open(os.open(os.devnull, os.O_RDONLY), closefd=False)
> ```
>
> Which solves the fundamental issue of processes colliding with each other resulting in a bad file descriptor error, but introduces a potential danger to applications which replace `sys.stdin` with a "file-like object" with output buffering. This danger is that if multiple processes call `~io.IOBase.close` on this file-like object, it could result in the same data being flushed to the object multiple times, resulting in corruption.
>
> If you write a file-like object and implement your own caching, you can make it fork-safe by storing the pid whenever you append to the cache, and discarding the cache when the pid changes. For example:
>
> ```
> @property
> def cache(self):
>     pid = os.getpid()
>     if pid != self._pid:
>         self._pid = pid
>         self._cache = []
>     return self._cache
> ```
>
> For more information, see `5155`, `5313` and `5331`

### The *spawn* and *forkserver* start methods<span id="multiprocessing-programming-spawn"></span>

There are a few extra restrictions which don't apply to the *fork* start method.

More picklability

> Ensure that all arguments to `~multiprocessing.Process` are picklable. Also, if you subclass `Process.__init__`, you must make sure that instances will be picklable when the `Process.start <multiprocessing.Process.start>` method is called.

Global variables

> Bear in mind that if code run in a child process tries to access a global variable, then the value it sees (if any) may not be the same as the value in the parent process at the time that `Process.start
> <multiprocessing.Process.start>` was called.
>
> However, global variables which are just module level constants cause no problems.

<div id="multiprocessing-safe-main-import">

Safe importing of main module

</div>

> Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such as starting a new process).
>
> For example, using the *spawn* or *forkserver* start method running the following module would fail with a `RuntimeError`:
>
> ```
> from multiprocessing import Process
>
> def foo():
>     print('hello')
>
> p = Process(target=foo)
> p.start()
> ```
>
> Instead one should protect the "entry point" of the program by using `if __name__ == '__main__':` as follows:
>
> ```
> from multiprocessing import Process, freeze_support, set_start_method
>
> def foo():
>     print('hello')
>
> if __name__ == '__main__':
>     freeze_support()
>     set_start_method('spawn')
>     p = Process(target=foo)
>     p.start()
> ```
>
> (The `freeze_support()` line can be omitted if the program will be run normally instead of frozen.)
>
> This allows the newly spawned Python interpreter to safely import the module and then run the module's `foo()` function.
>
> Similar restrictions apply if a pool or manager is created in the main module.

## Examples

Demonstration of how to create and use customized managers and proxies:

<div class="literalinclude" data-language="python3">

../includes/mp_newtype.py

</div>

Using `~multiprocessing.pool.Pool`:

<div class="literalinclude" data-language="python3">

../includes/mp_pool.py

</div>

An example showing how to use queues to feed tasks to a collection of worker processes and collect the results:

<div class="literalinclude">

../includes/mp_workers.py

</div>

# `!multiprocessing.shared_memory` --- Shared memory for direct access across processes

<div class="module" data-synopsis="Provides shared memory for direct access across processes.">

multiprocessing.shared_memory

</div>

**Source code:** `Lib/multiprocessing/shared_memory.py`

<div class="versionadded">

3.8

</div>

<div class="index">

single: Shared Memory single: POSIX Shared Memory single: Named Shared Memory

</div>

------------------------------------------------------------------------

This module provides a class, `SharedMemory`, for the allocation and management of shared memory to be accessed by one or more processes on a multicore or symmetric multiprocessor (SMP) machine. To assist with the life-cycle management of shared memory especially across distinct processes, a `~multiprocessing.managers.BaseManager` subclass, `~multiprocessing.managers.SharedMemoryManager`, is also provided in the `multiprocessing.managers` module.

In this module, shared memory refers to "POSIX style" shared memory blocks (though is not necessarily implemented explicitly as such) and does not refer to "distributed shared memory". This style of shared memory permits distinct processes to potentially read and write to a common (or shared) region of volatile memory. Processes are conventionally limited to only have access to their own process memory space but shared memory permits the sharing of data between processes, avoiding the need to instead send messages between processes containing that data. Sharing data directly via memory can provide significant performance benefits compared to sharing data via disk or socket or other communications requiring the serialization/deserialization and copying of data.

<div class="SharedMemory(name=None, create=False, size=0, *, track=True)">

Create an instance of the `!SharedMemory` class for either creating a new shared memory block or attaching to an existing shared memory block. Each shared memory block is assigned a unique name. In this way, one process can create a shared memory block with a particular name and a different process can attach to that same shared memory block using that same name.

As a resource for sharing data across processes, shared memory blocks may outlive the original process that created them. When one process no longer needs access to a shared memory block that might still be needed by other processes, the `close` method should be called. When a shared memory block is no longer needed by any process, the `unlink` method should be called to ensure proper cleanup.

param name  
The unique name for the requested shared memory, specified as a string. When creating a new shared memory block, if `None` (the default) is supplied for the name, a novel name will be generated.

type name  
str \| None

param bool create  
Control whether a new shared memory block is created (`True`) or an existing shared memory block is attached (`False`).

param int size  
The requested number of bytes when creating a new shared memory block. Because some platforms choose to allocate chunks of memory based upon that platform's memory page size, the exact size of the shared memory block may be larger or equal to the size requested. When attaching to an existing shared memory block, the *size* parameter is ignored.

param bool track  
When `True`, register the shared memory block with a resource tracker process on platforms where the OS does not do this automatically. The resource tracker ensures proper cleanup of the shared memory even if all other processes with access to the memory exit without doing so. Python processes created from a common ancestor using `multiprocessing` facilities share a single resource tracker process, and the lifetime of shared memory segments is handled automatically among these processes. Python processes created in any other way will receive their own resource tracker when accessing shared memory with *track* enabled. This will cause the shared memory to be deleted by the resource tracker of the first process that terminates. To avoid this issue, users of `subprocess` or standalone Python processes should set *track* to `False` when there is already another process in place that does the bookkeeping. *track* is ignored on Windows, which has its own tracking and automatically deletes shared memory when all handles to it have been closed.

<div class="versionchanged">

3.13 Added the *track* parameter.

</div>

<div class="method">

close()

Close the file descriptor/handle to the shared memory from this instance. `close` should be called once access to the shared memory block from this instance is no longer needed. Depending on operating system, the underlying memory may or may not be freed even if all handles to it have been closed. To ensure proper cleanup, use the `unlink` method.

</div>

<div class="method">

unlink()

Delete the underlying shared memory block. This should be called only once per shared memory block regardless of the number of handles to it, even in other processes. `unlink` and `close` can be called in any order, but trying to access data inside a shared memory block after `unlink` may result in memory access errors, depending on platform.

This method has no effect on Windows, where the only way to delete a shared memory block is to close all handles.

</div>

<div class="attribute">

buf

A memoryview of contents of the shared memory block.

</div>

<div class="attribute">

name

Read-only access to the unique name of the shared memory block.

</div>

<div class="attribute">

size

Read-only access to size in bytes of the shared memory block.

</div>

</div>

The following example demonstrates low-level use of `SharedMemory` instances:

    >>> from multiprocessing import shared_memory
    >>> shm_a = shared_memory.SharedMemory(create=True, size=10)
    >>> type(shm_a.buf)
    <class 'memoryview'>
    >>> buffer = shm_a.buf
    >>> len(buffer)
    10
    >>> buffer[:4] = bytearray([22, 33, 44, 55])  # Modify multiple at once
    >>> buffer[4] = 100                           # Modify single byte at a time
    >>> # Attach to an existing shared memory block
    >>> shm_b = shared_memory.SharedMemory(shm_a.name)
    >>> import array
    >>> array.array('b', shm_b.buf[:5])  # Copy the data into a new array.array
    array('b', [22, 33, 44, 55, 100])
    >>> shm_b.buf[:5] = b'howdy'  # Modify via shm_b using bytes
    >>> bytes(shm_a.buf[:5])      # Access via shm_a
    b'howdy'
    >>> shm_b.close()   # Close each SharedMemory instance
    >>> shm_a.close()
    >>> shm_a.unlink()  # Call unlink only once to release the shared memory

The following example demonstrates a practical use of the `SharedMemory` class with [NumPy arrays](https://numpy.org/), accessing the same `!numpy.ndarray` from two distinct Python shells:

<div class="doctest" data-options="+SKIP">

\>\>\> \# In the first Python interactive shell \>\>\> import numpy as np \>\>\> a = np.array(\[1, 1, 2, 3, 5, 8\]) \# Start with an existing NumPy array \>\>\> from multiprocessing import shared_memory \>\>\> shm = shared_memory.SharedMemory(create=True, size=a.nbytes) \>\>\> \# Now create a NumPy array backed by shared memory \>\>\> b = np.ndarray(a.shape, dtype=a.dtype, buffer=shm.buf) \>\>\> b\[:\] = a\[:\] \# Copy the original data into shared memory \>\>\> b array(\[1, 1, 2, 3, 5, 8\]) \>\>\> type(b) \<class 'numpy.ndarray'\> \>\>\> type(a) \<class 'numpy.ndarray'\> \>\>\> shm.name \# We did not specify a name so one was chosen for us 'psm_21467_46075'

\>\>\> \# In either the same shell or a new Python shell on the same machine \>\>\> import numpy as np \>\>\> from multiprocessing import shared_memory \>\>\> \# Attach to the existing shared memory block \>\>\> existing_shm = shared_memory.SharedMemory(name='psm_21467_46075') \>\>\> \# Note that a.shape is (6,) and a.dtype is np.int64 in this example \>\>\> c = np.ndarray((6,), dtype=np.int64, buffer=existing_shm.buf) \>\>\> c array(\[1, 1, 2, 3, 5, 8\]) \>\>\> c\[-1\] = 888 \>\>\> c array(\[ 1, 1, 2, 3, 5, 888\])

\>\>\> \# Back in the first Python interactive shell, b reflects this change \>\>\> b array(\[ 1, 1, 2, 3, 5, 888\])

\>\>\> \# Clean up from within the second Python shell \>\>\> del c \# Unnecessary; merely emphasizing the array is no longer used \>\>\> existing_shm.close()

\>\>\> \# Clean up from within the first Python shell \>\>\> del b \# Unnecessary; merely emphasizing the array is no longer used \>\>\> shm.close() \>\>\> shm.unlink() \# Free and release the shared memory block at the very end

</div>

<div class="SharedMemoryManager([address[, authkey]])" data-module="multiprocessing.managers">

A subclass of `multiprocessing.managers.BaseManager` which can be used for the management of shared memory blocks across processes.

A call to `~multiprocessing.managers.BaseManager.start` on a `!SharedMemoryManager` instance causes a new process to be started. This new process's sole purpose is to manage the life cycle of all shared memory blocks created through it. To trigger the release of all shared memory blocks managed by that process, call `~multiprocessing.managers.BaseManager.shutdown` on the instance. This triggers a `~multiprocessing.shared_memory.SharedMemory.unlink` call on all of the `~multiprocessing.shared_memory.SharedMemory` objects managed by that process and then stops the process itself. By creating `!SharedMemory` instances through a `!SharedMemoryManager`, we avoid the need to manually track and trigger the freeing of shared memory resources.

This class provides methods for creating and returning `~multiprocessing.shared_memory.SharedMemory` instances and for creating a list-like object (`~multiprocessing.shared_memory.ShareableList`) backed by shared memory.

Refer to `~multiprocessing.managers.BaseManager` for a description of the inherited *address* and *authkey* optional input arguments and how they may be used to connect to an existing `!SharedMemoryManager` service from other processes.

<div class="method">

SharedMemory(size)

Create and return a new `~multiprocessing.shared_memory.SharedMemory` object with the specified *size* in bytes.

</div>

<div class="method">

ShareableList(sequence)

Create and return a new `~multiprocessing.shared_memory.ShareableList` object, initialized by the values from the input *sequence*.

</div>

</div>

The following example demonstrates the basic mechanisms of a `~multiprocessing.managers.SharedMemoryManager`:

<div class="doctest" data-options="+SKIP">

\>\>\> from multiprocessing.managers import SharedMemoryManager \>\>\> smm = SharedMemoryManager() \>\>\> smm.start() \# Start the process that manages the shared memory blocks \>\>\> sl = smm.ShareableList(range(4)) \>\>\> sl ShareableList(\[0, 1, 2, 3\], name='psm_6572_7512') \>\>\> raw_shm = smm.SharedMemory(size=128) \>\>\> another_sl = smm.ShareableList('alpha') \>\>\> another_sl ShareableList(\['a', 'l', 'p', 'h', 'a'\], name='psm_6572_12221') \>\>\> smm.shutdown() \# Calls unlink() on sl, raw_shm, and another_sl

</div>

The following example depicts a potentially more convenient pattern for using `~multiprocessing.managers.SharedMemoryManager` objects via the `with` statement to ensure that all shared memory blocks are released after they are no longer needed:

<div class="doctest" data-options="+SKIP">

\>\>\> with SharedMemoryManager() as smm: ... sl = smm.ShareableList(range(2000)) ... \# Divide the work among two processes, storing partial results in sl ... p1 = Process(target=do_work, args=(sl, 0, 1000)) ... p2 = Process(target=do_work, args=(sl, 1000, 2000)) ... p1.start() ... p2.start() \# A multiprocessing.Pool might be more efficient ... p1.join() ... p2.join() \# Wait for all work to complete in both processes ... total_result = sum(sl) \# Consolidate the partial results now in sl

</div>

When using a `~multiprocessing.managers.SharedMemoryManager` in a `with` statement, the shared memory blocks created using that manager are all released when the `!with` statement's code block finishes execution.

<div class="ShareableList(sequence=None, *, name=None)">

Provide a mutable list-like object where all values stored within are stored in a shared memory block. This constrains storable values to the following built-in data types:

- `int` (signed 64-bit)
- `float`
- `bool`
- `str` (less than 10M bytes each when encoded as UTF-8)
- `bytes` (less than 10M bytes each)
- `None`

It also notably differs from the built-in `list` type in that these lists can not change their overall length (i.e. no `!append`, `!insert`, etc.) and do not support the dynamic creation of new `!ShareableList` instances via slicing.

*sequence* is used in populating a new `!ShareableList` full of values. Set to `None` to instead attach to an already existing `!ShareableList` by its unique shared memory name.

*name* is the unique name for the requested shared memory, as described in the definition for `SharedMemory`. When attaching to an existing `!ShareableList`, specify its shared memory block's unique name while leaving *sequence* set to `None`.

> [!NOTE]
> A known issue exists for `bytes` and `str` values. If they end with `\x00` nul bytes or characters, those may be *silently stripped* when fetching them by index from the `!ShareableList`. This `.rstrip(b'\x00')` behavior is considered a bug and may go away in the future. See `106939`.

For applications where rstripping of trailing nulls is a problem, work around it by always unconditionally appending an extra non-0 byte to the end of such values when storing and unconditionally removing it when fetching:

<div class="doctest">

\>\>\> from multiprocessing import shared_memory \>\>\> nul_bug_demo = shared_memory.ShareableList(\['?x00', b'x03x02x01x00x00x00'\]) \>\>\> nul_bug_demo\[0\] '?' \>\>\> nul_bug_demo\[1\] b'x03x02x01' \>\>\> nul_bug_demo.shm.unlink() \>\>\> padded = shared_memory.ShareableList(\['?x00x07', b'x03x02x01x00x00x00x07'\]) \>\>\> padded\[0\]\[:-1\] '?x00' \>\>\> padded\[1\]\[:-1\] b'x03x02x01x00x00x00' \>\>\> padded.shm.unlink()

</div>

<div class="method">

count(value)

Return the number of occurrences of *value*.

</div>

<div class="method">

index(value)

Return first index position of *value*. Raise `ValueError` if *value* is not present.

</div>

<div class="attribute">

format

Read-only attribute containing the `struct` packing format used by all currently stored values.

</div>

<div class="attribute">

shm

The `SharedMemory` instance where the values are stored.

</div>

</div>

The following example demonstrates basic use of a `ShareableList` instance:

> \>\>\> from multiprocessing import shared_memory \>\>\> a = shared_memory.ShareableList(\['howdy', b'HoWdY', -273.154, 100, None, True, 42\]) \>\>\> \[ type(entry) for entry in a \] \[\<class 'str'\>, \<class 'bytes'\>, \<class 'float'\>, \<class 'int'\>, \<class 'NoneType'\>, \<class 'bool'\>, \<class 'int'\>\] \>\>\> a\[2\] -273.154 \>\>\> a\[2\] = -78.5 \>\>\> a\[2\] -78.5 \>\>\> a\[2\] = 'dry ice' \# Changing data types is supported as well \>\>\> a\[2\] 'dry ice' \>\>\> a\[2\] = 'larger than previously allocated storage space' Traceback (most recent call last): ... ValueError: exceeds available storage for existing str \>\>\> a\[2\] 'dry ice' \>\>\> len(a) 7 \>\>\> a.index(42) 6 \>\>\> a.count(b'howdy') 0 \>\>\> a.count(b'HoWdY') 1 \>\>\> a.shm.close() \>\>\> a.shm.unlink() \>\>\> del a \# Use of a ShareableList after call to unlink() is unsupported

The following example depicts how one, two, or many processes may access the same `ShareableList` by supplying the name of the shared memory block behind it:

> \>\>\> b = shared_memory.ShareableList(range(5)) \# In a first process \>\>\> c = shared_memory.ShareableList(name=b.shm.name) \# In a second process \>\>\> c ShareableList(\[0, 1, 2, 3, 4\], name='...') \>\>\> c\[-1\] = -999 \>\>\> b\[-1\] -999 \>\>\> b.shm.close() \>\>\> c.shm.close() \>\>\> c.shm.unlink()

The following examples demonstrates that `ShareableList` (and underlying `SharedMemory`) objects can be pickled and unpickled if needed. Note, that it will still be the same shared object. This happens, because the deserialized object has the same unique name and is just attached to an existing object with the same name (if the object is still alive):

> \>\>\> import pickle \>\>\> from multiprocessing import shared_memory \>\>\> sl = shared_memory.ShareableList(range(10)) \>\>\> list(sl) \[0, 1, 2, 3, 4, 5, 6, 7, 8, 9\]
>
> \>\>\> deserialized_sl = pickle.loads(pickle.dumps(sl)) \>\>\> list(deserialized_sl) \[0, 1, 2, 3, 4, 5, 6, 7, 8, 9\]
>
> \>\>\> sl\[0\] = -1 \>\>\> deserialized_sl\[1\] = -2 \>\>\> list(sl) \[-1, -2, 2, 3, 4, 5, 6, 7, 8, 9\] \>\>\> list(deserialized_sl) \[-1, -2, 2, 3, 4, 5, 6, 7, 8, 9\]
>
> \>\>\> sl.shm.close() \>\>\> sl.shm.unlink()

# Internet Data Handling

This chapter describes modules which support handling data formats commonly used on the internet.

<div class="toctree">

email.rst json.rst mailbox.rst mimetypes.rst base64.rst binascii.rst quopri.rst

</div>

# `!netrc` --- netrc file processing

<div class="module" data-synopsis="Loading of .netrc files.">

netrc

</div>

**Source code:** `Lib/netrc.py`

------------------------------------------------------------------------

The `~netrc.netrc` class parses and encapsulates the netrc file format used by the Unix `ftp` program and other FTP clients.

<div class="netrc([file])">

A `~netrc.netrc` instance or subclass instance encapsulates data from a netrc file. The initialization argument, if present, specifies the file to parse. If no argument is given, the file `.netrc` in the user's home directory -- as determined by `os.path.expanduser` -- will be read. Otherwise, a `FileNotFoundError` exception will be raised. Parse errors will raise `NetrcParseError` with diagnostic information including the file name, line number, and terminating token.

If no argument is specified on a POSIX system, the presence of passwords in the `.netrc` file will raise a `NetrcParseError` if the file ownership or permissions are insecure (owned by a user other than the user running the process, or accessible for read or write by any other user). This implements security behavior equivalent to that of ftp and other programs that use `.netrc`. Such security checks are not available on platforms that do not support `os.getuid`.

<div class="versionchanged">

3.4 Added the POSIX permission check.

</div>

<div class="versionchanged">

3.7 `os.path.expanduser` is used to find the location of the `.netrc` file when *file* is not passed as argument.

</div>

<div class="versionchanged">

3.10 `~netrc.netrc` try UTF-8 encoding before using locale specific encoding. The entry in the netrc file no longer needs to contain all tokens. The missing tokens' value default to an empty string. All the tokens and their values now can contain arbitrary characters, like whitespace and non-ASCII characters. If the login name is anonymous, it won't trigger the security check.

</div>

</div>

<div class="exception">

NetrcParseError

Exception raised by the `~netrc.netrc` class when syntactical errors are encountered in source text. Instances of this exception provide three interesting attributes:

<div class="attribute">

msg

Textual explanation of the error.

</div>

<div class="attribute">

filename

The name of the source file.

</div>

<div class="attribute">

lineno

The line number on which the error was found.

</div>

</div>

## netrc Objects

A `~netrc.netrc` instance has the following methods:

<div class="method">

netrc.authenticators(host)

Return a 3-tuple `(login, account, password)` of authenticators for *host*. If the netrc file did not contain an entry for the given host, return the tuple associated with the 'default' entry. If neither matching host nor default entry is available, return `None`.

</div>

<div class="method">

netrc.\_\_repr\_\_()

Dump the class data as a string in the format of a netrc file. (This discards comments and may reorder the entries.)

</div>

Instances of `~netrc.netrc` have public instance variables:

<div class="attribute">

netrc.hosts

Dictionary mapping host names to `(login, account, password)` tuples. The 'default' entry, if any, is represented as a pseudo-host by that name.

</div>

<div class="attribute">

netrc.macros

Dictionary mapping macro names to string lists.

</div>

# `!nis` --- Interface to Sun’s NIS (Yellow Pages)

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

nis

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!nis` module was [Python 3.12](https://docs.python.org/3.12/library/nis.html).

# `!nntplib` --- NNTP protocol client

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

nntplib

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!nntplib` module was [Python 3.12](https://docs.python.org/3.12/library/nntplib.html).

# `!numbers` --- Numeric abstract base classes

<div class="module" data-synopsis="Numeric abstract base classes (Complex, Real, Integral, etc.).">

numbers

</div>

**Source code:** `Lib/numbers.py`

------------------------------------------------------------------------

The `!numbers` module (`3141`) defines a hierarchy of numeric `abstract base classes <abstract base class>` which progressively define more operations. None of the types defined in this module are intended to be instantiated.

<div class="Number">

The root of the numeric hierarchy. If you just want to check if an argument *x* is a number, without caring what kind, use `isinstance(x, Number)`.

</div>

## The numeric tower

<div class="Complex">

Subclasses of this type describe complex numbers and include the operations that work on the built-in `complex` type. These are: conversions to `complex` and `bool`, `.real`, `.imag`, `+`, `-`, `*`, `/`, `**`, `abs`, `conjugate`, `==`, and `!=`. All except `-` and `!=` are abstract.

<div class="attribute">

real

Abstract. Retrieves the real component of this number.

</div>

<div class="attribute">

imag

Abstract. Retrieves the imaginary component of this number.

</div>

<div class="method" data-abstractmethod="">

conjugate()

Abstract. Returns the complex conjugate. For example, `(1+3j).conjugate() == (1-3j)`.

</div>

</div>

<div class="Real">

To `Complex`, `!Real` adds the operations that work on real numbers.

In short, those are: a conversion to `float`, `math.trunc`, `round`, `math.floor`, `math.ceil`, `divmod`, `//`, `%`, `<`, `<=`, `>`, and `>=`.

Real also provides defaults for `complex`, `~Complex.real`, `~Complex.imag`, and `~Complex.conjugate`.

</div>

<div class="Rational">

Subtypes `Real` and adds `~Rational.numerator` and `~Rational.denominator` properties. It also provides a default for `float`.

The `~Rational.numerator` and `~Rational.denominator` values should be instances of `Integral` and should be in lowest terms with `~Rational.denominator` positive.

<div class="attribute">

numerator

Abstract. The numerator of this rational number.

</div>

<div class="attribute">

denominator

Abstract. The denominator of this rational number.

</div>

</div>

<div class="Integral">

Subtypes `Rational` and adds a conversion to `int`. Provides defaults for `float`, `~Rational.numerator`, and `~Rational.denominator`. Adds abstract methods for `pow` with modulus and bit-string operations: `<<`, `>>`, `&`, `^`, `|`, `~`.

</div>

## Notes for type implementers

Implementers should be careful to make equal numbers equal and hash them to the same values. This may be subtle if there are two different extensions of the real numbers. See also `numeric-hash`.

### Adding More Numeric ABCs

There are, of course, more possible ABCs for numbers, and this would be a poor hierarchy if it precluded the possibility of adding those. You can add `MyFoo` between `Complex` and `Real` with:

    class MyFoo(Complex): ...
    MyFoo.register(Real)

### Implementing the arithmetic operations

We want to implement the arithmetic operations so that mixed-mode operations either call an implementation whose author knew about the types of both arguments, or convert both to the nearest built in type and do the operation there. For subtypes of `Integral`, this means that `~object.__add__` and `~object.__radd__` should be defined as:

    class MyIntegral(Integral):

        def __add__(self, other):
            if isinstance(other, MyIntegral):
                return do_my_adding_stuff(self, other)
            elif isinstance(other, OtherTypeIKnowAbout):
                return do_my_other_adding_stuff(self, other)
            else:
                return NotImplemented

        def __radd__(self, other):
            if isinstance(other, MyIntegral):
                return do_my_adding_stuff(other, self)
            elif isinstance(other, OtherTypeIKnowAbout):
                return do_my_other_adding_stuff(other, self)
            elif isinstance(other, Integral):
                return int(other) + int(self)
            elif isinstance(other, Real):
                return float(other) + float(self)
            elif isinstance(other, Complex):
                return complex(other) + complex(self)
            else:
                return NotImplemented

There are 5 different cases for a mixed-type operation on subclasses of `Complex`. I'll refer to all of the above code that doesn't refer to `MyIntegral` and `OtherTypeIKnowAbout` as "boilerplate". `a` will be an instance of `A`, which is a subtype of `Complex` (`a : A <: Complex`), and `b : B <: Complex`. I'll consider `a + b`:

1.  If `A` defines an `~object.__add__` which accepts `b`, all is well.
2.  If `A` falls back to the boilerplate code, and it were to return a value from `~object.__add__`, we'd miss the possibility that `B` defines a more intelligent `~object.__radd__`, so the boilerplate should return `NotImplemented` from `!__add__`. (Or `A` may not implement `!__add__` at all.)
3.  Then `B`'s `~object.__radd__` gets a chance. If it accepts `a`, all is well.
4.  If it falls back to the boilerplate, there are no more possible methods to try, so this is where the default implementation should live.
5.  If `B <: A`, Python tries `B.__radd__` before `A.__add__`. This is ok, because it was implemented with knowledge of `A`, so it can handle those instances before delegating to `Complex`.

If `A <: Complex` and `B <: Real` without sharing any other knowledge, then the appropriate shared operation is the one involving the built in `complex`, and both `~object.__radd__` s land there, so `a+b == b+a`.

Because most of the operations on any given type will be very similar, it can be useful to define a helper function which generates the forward and reverse instances of any given operator. For example, `fractions.Fraction` uses:

    def _operator_fallbacks(monomorphic_operator, fallback_operator):
        def forward(a, b):
            if isinstance(b, (int, Fraction)):
                return monomorphic_operator(a, b)
            elif isinstance(b, float):
                return fallback_operator(float(a), b)
            elif isinstance(b, complex):
                return fallback_operator(complex(a), b)
            else:
                return NotImplemented
        forward.__name__ = '__' + fallback_operator.__name__ + '__'
        forward.__doc__ = monomorphic_operator.__doc__

        def reverse(b, a):
            if isinstance(a, Rational):
                # Includes ints.
                return monomorphic_operator(a, b)
            elif isinstance(a, Real):
                return fallback_operator(float(a), float(b))
            elif isinstance(a, Complex):
                return fallback_operator(complex(a), complex(b))
            else:
                return NotImplemented
        reverse.__name__ = '__r' + fallback_operator.__name__ + '__'
        reverse.__doc__ = monomorphic_operator.__doc__

        return forward, reverse

    def _add(a, b):
        """a + b"""
        return Fraction(a.numerator * b.denominator +
                        b.numerator * a.denominator,
                        a.denominator * b.denominator)

    __add__, __radd__ = _operator_fallbacks(_add, operator.add)

    # ...

# Numeric and Mathematical Modules

The modules described in this chapter provide numeric and math-related functions and data types. The `numbers` module defines an abstract hierarchy of numeric types. The `math` and `cmath` modules contain various mathematical functions for floating-point and complex numbers. The `decimal` module supports exact representations of decimal numbers, using arbitrary precision arithmetic.

The following modules are documented in this chapter:

<div class="toctree">

numbers.rst math.rst math.integer.rst cmath.rst decimal.rst fractions.rst random.rst statistics.rst

</div>

# `!operator` --- Standard operators as functions

<div class="module" data-synopsis="Functions corresponding to the standard operators.">

operator

</div>

**Source code:** `Lib/operator.py`

<div class="testsetup">

import operator from operator import itemgetter, iadd

</div>

------------------------------------------------------------------------

The `!operator` module exports a set of efficient functions corresponding to the intrinsic operators of Python. For example, `operator.add(x, y)` is equivalent to the expression `x+y`. Many function names are those used for special methods, without the double underscores. For backward compatibility, many of these have a variant with the double underscores kept. The variants without the double underscores are preferred for clarity.

The functions fall into categories that perform object comparisons, logical operations, mathematical operations and sequence operations.

The object comparison functions are useful for all objects, and are named after the rich comparison operators they support:

<div class="function">

lt(a, b) le(a, b) eq(a, b) ne(a, b) ge(a, b) gt(a, b) \_\_lt\_\_(a, b) \_\_le\_\_(a, b) \_\_eq\_\_(a, b) \_\_ne\_\_(a, b) \_\_ge\_\_(a, b) \_\_gt\_\_(a, b)

Perform "rich comparisons" between *a* and *b*. Specifically, `lt(a, b)` is equivalent to `a < b`, `le(a, b)` is equivalent to `a <= b`, `eq(a, b)` is equivalent to `a == b`, `ne(a, b)` is equivalent to `a != b`, `gt(a, b)` is equivalent to `a > b` and `ge(a, b)` is equivalent to `a >= b`. Note that these functions can return any value, which may or may not be interpretable as a Boolean value. See `comparisons` for more information about rich comparisons.

</div>

The logical operations are also generally applicable to all objects, and support truth tests, identity tests, and boolean operations:

<div class="function">

[not]()(obj) \_\_not\_\_(obj)

Return the outcome of `not` *obj*. (Note that there is no `!__not__` method for object instances; only the interpreter core defines this operation. The result is affected by the `~object.__bool__` and `~object.__len__` methods.)

</div>

<div class="function">

truth(obj)

Return `True` if *obj* is true, and `False` otherwise. This is equivalent to using the `bool` constructor.

</div>

<div class="function">

[is]()(a, b)

Return `a is b`. Tests object identity.

</div>

<div class="function">

is_not(a, b)

Return `a is not b`. Tests object identity.

</div>

<div class="function">

is_none(a)

Return `a is None`. Tests object identity.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

is_not_none(a)

Return `a is not None`. Tests object identity.

<div class="versionadded">

3.14

</div>

</div>

The mathematical and bitwise operations are the most numerous:

<div class="function">

abs(obj) \_\_abs\_\_(obj)

Return the absolute value of *obj*.

</div>

<div class="function">

add(a, b) \_\_add\_\_(a, b)

Return `a + b`, for *a* and *b* numbers.

</div>

<div class="function">

[and]()(a, b) \_\_and\_\_(a, b)

Return `a & b`.

</div>

<div class="function">

floordiv(a, b) \_\_floordiv\_\_(a, b)

Return `a // b`.

</div>

<div class="function">

index(a) \_\_index\_\_(a)

Return *a* converted to an integer. Equivalent to `a.__index__()`.

<div class="versionchanged">

3.10 The result always has exact type `int`. Previously, the result could have been an instance of a subclass of `int`.

</div>

</div>

<div class="function">

inv(obj) invert(obj) \_\_inv\_\_(obj) \_\_invert\_\_(obj)

Return `~obj`.

</div>

<div class="function">

lshift(a, b) \_\_lshift\_\_(a, b)

Return `a << b`.

</div>

<div class="function">

mod(a, b) \_\_mod\_\_(a, b)

Return `a % b`.

</div>

<div class="function">

mul(a, b) \_\_mul\_\_(a, b)

Return `a * b`.

</div>

<div class="function">

matmul(a, b) \_\_matmul\_\_(a, b)

Return `a @ b`.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

neg(obj) \_\_neg\_\_(obj)

Return *obj* negated (`-obj`).

</div>

<div class="function">

[or]()(a, b) \_\_or\_\_(a, b)

Return `a | b`.

</div>

<div class="function">

pos(obj) \_\_pos\_\_(obj)

Return `+obj`.

</div>

<div class="function">

pow(a, b) \_\_pow\_\_(a, b)

Return `a ** b`.

</div>

<div class="function">

rshift(a, b) \_\_rshift\_\_(a, b)

Return `a >> b`.

</div>

<div class="function">

sub(a, b) \_\_sub\_\_(a, b)

Return `a - b`.

</div>

<div class="function">

truediv(a, b) \_\_truediv\_\_(a, b)

Return `a / b` where 2/3 is .66 rather than 0. This is also known as "true" division.

</div>

<div class="function">

xor(a, b) \_\_xor\_\_(a, b)

Return `a ^ b`.

</div>

Operations which work with sequences (some of them with mappings too) include:

<div class="function">

concat(a, b) \_\_concat\_\_(a, b)

Return `a + b` for *a* and *b* sequences.

</div>

<div class="function">

contains(a, b) \_\_contains\_\_(a, b)

Return the outcome of the test `b in a`. Note the reversed operands.

</div>

<div class="function">

countOf(a, b)

Return the number of occurrences of *b* in *a*.

</div>

<div class="function">

delitem(a, b) \_\_delitem\_\_(a, b)

Remove the value of *a* at index *b*.

</div>

<div class="function">

getitem(a, b) \_\_getitem\_\_(a, b)

Return the value of *a* at index *b*.

</div>

<div class="function">

indexOf(a, b)

Return the index of the first of occurrence of *b* in *a*.

</div>

<div class="function">

setitem(a, b, c) \_\_setitem\_\_(a, b, c)

Set the value of *a* at index *b* to *c*.

</div>

<div class="function">

length_hint(obj, default=0)

Return an estimated length for the object *obj*. First try to return its actual length, then an estimate using `object.__length_hint__`, and finally return the default value.

<div class="versionadded">

3.4

</div>

</div>

The following operation works with callables:

<div class="function">

call(obj, /, *args,kwargs) \_\_call\_\_(obj, /,*args, \*\*kwargs)

Return `obj(*args, **kwargs)`.

<div class="versionadded">

3.11

</div>

</div>

The `!operator` module also defines tools for generalized attribute and item lookups. These are useful for making fast field extractors as arguments for `map`, `sorted`, `itertools.groupby`, or other functions that expect a function argument.

<div class="function">

attrgetter(attr) attrgetter(\*attrs)

Return a callable object that fetches *attr* from its operand. If more than one attribute is requested, returns a tuple of attributes. The attribute names can also contain dots. For example:

- After `f = attrgetter('name')`, the call `f(b)` returns `b.name`.
- After `f = attrgetter('name', 'date')`, the call `f(b)` returns `(b.name, b.date)`.
- After `f = attrgetter('name.first', 'name.last')`, the call `f(b)` returns `(b.name.first, b.name.last)`.

Equivalent to:

    def attrgetter(*items):
        if any(not isinstance(item, str) for item in items):
            raise TypeError('attribute name must be a string')
        if len(items) == 1:
            attr = items[0]
            def g(obj):
                return resolve_attr(obj, attr)
        else:
            def g(obj):
                return tuple(resolve_attr(obj, attr) for attr in items)
        return g

    def resolve_attr(obj, attr):
        for name in attr.split("."):
            obj = getattr(obj, name)
        return obj

</div>

<div class="function">

itemgetter(item) itemgetter(\*items)

Return a callable object that fetches *item* from its operand using the operand's `~object.__getitem__` method. If multiple items are specified, returns a tuple of lookup values. For example:

- After `f = itemgetter(2)`, the call `f(r)` returns `r[2]`.
- After `g = itemgetter(2, 5, 3)`, the call `g(r)` returns `(r[2], r[5], r[3])`.

Equivalent to:

    def itemgetter(*items):
        if len(items) == 1:
            item = items[0]
            def g(obj):
                return obj[item]
        else:
            def g(obj):
                return tuple(obj[item] for item in items)
        return g

The items can be any type accepted by the operand's `~object.__getitem__` method. Dictionaries accept any `hashable` value. Lists, tuples, and strings accept an index or a slice:

> \>\>\> itemgetter(1)('ABCDEFG') 'B' \>\>\> itemgetter(1, 3, 5)('ABCDEFG') ('B', 'D', 'F') \>\>\> itemgetter(slice(2, None))('ABCDEFG') 'CDEFG' \>\>\> soldier = dict(rank='captain', name='dotterbart') \>\>\> itemgetter('rank')(soldier) 'captain'

Example of using `itemgetter` to retrieve specific fields from a tuple record:

> \>\>\> inventory = \[('apple', 3), ('banana', 2), ('pear', 5), ('orange', 1)\] \>\>\> getcount = itemgetter(1) \>\>\> list(map(getcount, inventory)) \[3, 2, 5, 1\] \>\>\> sorted(inventory, key=getcount) \[('orange', 1), ('banana', 2), ('apple', 3), ('pear', 5)\]

</div>

<div class="function">

methodcaller(name, /, *args,*\*kwargs)

Return a callable object that calls the method *name* on its operand. If additional arguments and/or keyword arguments are given, they will be given to the method as well. For example:

- After `f = methodcaller('name')`, the call `f(b)` returns `b.name()`.
- After `f = methodcaller('name', 'foo', bar=1)`, the call `f(b)` returns `b.name('foo', bar=1)`.

Equivalent to:

    def methodcaller(name, /, *args, **kwargs):
        def caller(obj):
            return getattr(obj, name)(*args, **kwargs)
        return caller

</div>

## Mapping Operators to Functions

This table shows how abstract operations correspond to operator symbols in the Python syntax and the functions in the `!operator` module.

| Operation | Syntax | Function |
|----|----|----|
| Addition | `a + b` | `add(a, b)` |
| Concatenation | `seq1 + seq2` | `concat(seq1, seq2)` |
| Containment Test | `obj in seq` | `contains(seq, obj)` |
| Division | `a / b` | `truediv(a, b)` |
| Division | `a // b` | `floordiv(a, b)` |
| Bitwise And, or Intersection | `a & b` | `and_(a, b)` |
| Bitwise Exclusive Or, or Symmetric Difference | `a ^ b` | `xor(a, b)` |
| Bitwise Inversion, or Complement | `~ a` | `invert(a)` |
| Bitwise Or, or Union | `a | b` | `or_(a, b)` |
| Exponentiation | `a ** b` | `pow(a, b)` |
| Identity | `a is b` | `is_(a, b)` |
| Identity | `a is not b` | `is_not(a, b)` |
| Identity | `a is None` | `is_none(a)` |
| Identity | `a is not None` | `is_not_none(a)` |
| Indexed Assignment | `obj[k] = v` | `setitem(obj, k, v)` |
| Indexed Deletion | `del obj[k]` | `delitem(obj, k)` |
| Indexing | `obj[k]` | `getitem(obj, k)` |
| Left Shift | `a << b` | `lshift(a, b)` |
| Modulo | `a % b` | `mod(a, b)` |
| Multiplication | `a * b` | `mul(a, b)` |
| Matrix Multiplication | `a @ b` | `matmul(a, b)` |
| Negation (Arithmetic) | `- a` | `neg(a)` |
| Negation (Logical) | `not a` | `not_(a)` |
| Positive | `+ a` | `pos(a)` |
| Right Shift | `a >> b` | `rshift(a, b)` |
| Slice Assignment | `seq[i:j] = values` | `setitem(seq, slice(i, j), values)` |
| Slice Deletion | `del seq[i:j]` | `delitem(seq, slice(i, j))` |
| Slicing | `seq[i:j]` | `getitem(seq, slice(i, j))` |
| String Formatting | `s % obj` | `mod(s, obj)` |
| Subtraction | `a - b` | `sub(a, b)` |
| Truth Test | `obj` | `truth(obj)` |
| Ordering | `a < b` | `lt(a, b)` |
| Ordering | `a <= b` | `le(a, b)` |
| Equality | `a == b` | `eq(a, b)` |
| Difference | `a != b` | `ne(a, b)` |
| Ordering | `a >= b` | `ge(a, b)` |
| Ordering | `a > b` | `gt(a, b)` |

## In-place Operators

Many operations have an "in-place" version. Listed below are functions providing a more primitive access to in-place operators than the usual syntax does; for example, the `statement` `x += y` is equivalent to `x = operator.iadd(x, y)`. Another way to put it is to say that `z = operator.iadd(x, y)` is equivalent to the compound statement `z = x; z += y`.

In those examples, note that when an in-place method is called, the computation and assignment are performed in two separate steps. The in-place functions listed below only do the first step, calling the in-place method. The second step, assignment, is not handled.

For immutable targets such as strings, numbers, and tuples, the updated value is computed, but not assigned back to the input variable:

\>\>\> a = 'hello' \>\>\> iadd(a, ' world') 'hello world' \>\>\> a 'hello'

For mutable targets such as lists and dictionaries, the in-place method will perform the update, so no subsequent assignment is necessary:

\>\>\> s = \['h', 'e', 'l', 'l', 'o'\] \>\>\> iadd(s, \[' ', 'w', 'o', 'r', 'l', 'd'\]) \['h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd'\] \>\>\> s \['h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd'\]

<div class="function">

iadd(a, b) \_\_iadd\_\_(a, b)

`a = iadd(a, b)` is equivalent to `a += b`.

</div>

<div class="function">

iand(a, b) \_\_iand\_\_(a, b)

`a = iand(a, b)` is equivalent to `a &= b`.

</div>

<div class="function">

iconcat(a, b) \_\_iconcat\_\_(a, b)

`a = iconcat(a, b)` is equivalent to `a += b` for *a* and *b* sequences.

</div>

<div class="function">

ifloordiv(a, b) \_\_ifloordiv\_\_(a, b)

`a = ifloordiv(a, b)` is equivalent to `a //= b`.

</div>

<div class="function">

ilshift(a, b) \_\_ilshift\_\_(a, b)

`a = ilshift(a, b)` is equivalent to `a <<= b`.

</div>

<div class="function">

imod(a, b) \_\_imod\_\_(a, b)

`a = imod(a, b)` is equivalent to `a %= b`.

</div>

<div class="function">

imul(a, b) \_\_imul\_\_(a, b)

`a = imul(a, b)` is equivalent to `a *= b`.

</div>

<div class="function">

imatmul(a, b) \_\_imatmul\_\_(a, b)

`a = imatmul(a, b)` is equivalent to `a @= b`.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

ior(a, b) \_\_ior\_\_(a, b)

`a = ior(a, b)` is equivalent to `a |= b`.

</div>

<div class="function">

ipow(a, b) \_\_ipow\_\_(a, b)

`a = ipow(a, b)` is equivalent to `a **= b`.

</div>

<div class="function">

irshift(a, b) \_\_irshift\_\_(a, b)

`a = irshift(a, b)` is equivalent to `a >>= b`.

</div>

<div class="function">

isub(a, b) \_\_isub\_\_(a, b)

`a = isub(a, b)` is equivalent to `a -= b`.

</div>

<div class="function">

itruediv(a, b) \_\_itruediv\_\_(a, b)

`a = itruediv(a, b)` is equivalent to `a /= b`.

</div>

<div class="function">

ixor(a, b) \_\_ixor\_\_(a, b)

`a = ixor(a, b)` is equivalent to `a ^= b`.

</div>

# `!optparse` --- Parser for command line options

<div class="module" data-synopsis="Command-line option parsing library.">

optparse

</div>

**Source code:** `Lib/optparse.py`

------------------------------------------------------------------------

## Choosing an argument parsing library

The standard library includes three argument parsing libraries:

- `getopt`: a module that closely mirrors the procedural C `getopt` API.
- `!optparse`: a declarative replacement for `getopt` that provides equivalent functionality without requiring each application to implement its own procedural option parsing logic.
- `argparse`: a more opinionated alternative to `optparse` that provides more functionality by default, at the expense of reduced application flexibility in controlling exactly how arguments are processed.

In the absence of more specific argument parsing design constraints, `argparse` is the recommended choice for implementing command line applications, as it offers the highest level of baseline functionality with the least application level code.

`getopt` is retained almost entirely for backwards compatibility reasons. However, it also serves a niche use case as a tool for prototyping and testing command line argument handling in `getopt`-based C applications.

`!optparse` should be considered as an alternative to `argparse` in the following cases:

- an application is already using `!optparse` and doesn't want to risk the subtle behavioural changes that may arise when migrating to `argparse`
- the application requires additional control over the way options and positional parameters are interleaved on the command line (including the ability to disable the interleaving feature completely)
- the application requires additional control over the incremental parsing of command line elements (while `argparse` does support this, the exact way it works in practice is undesirable for some use cases)
- the application requires additional control over the handling of options which accept parameter values that may start with `-` (such as delegated options to be passed to invoked subprocesses)
- the application requires some other command line parameter processing behavior which `argparse` does not support, but which can be implemented in terms of the lower level interface offered by `optparse`

These considerations also mean that `!optparse` is likely to provide a better foundation for library authors writing third party command line argument processing libraries.

As a concrete example, consider the following two command line argument parsing configurations, the first using `optparse`, and the second using `argparse`:

<div class="testcode">

import optparse

if \_\_name\_\_ == '\_\_main\_\_':  
parser = optparse.OptionParser() parser.add_option('-o', '--output') parser.add_option('-v', dest='verbose', action='store_true') opts, args = parser.parse_args() process(args, output=opts.output, verbose=opts.verbose)

</div>

<div class="testcode">

import argparse

if \_\_name\_\_ == '\_\_main\_\_':  
parser = argparse.ArgumentParser() parser.add_argument('-o', '--output') parser.add_argument('-v', dest='verbose', action='store_true') parser.add_argument('rest', nargs='\*') args = parser.parse_args() process(args.rest, output=args.output, verbose=args.verbose)

</div>

The most obvious difference is that in the `optparse` version, the non-option arguments are processed separately by the application after the option processing is complete. In the `argparse` version, positional arguments are declared and processed in the same way as the named options.

However, the `argparse` version will also handle some parameter combination differently from the way the `optparse` version would handle them. For example (amongst other differences):

- supplying `-o -v` gives `output="-v"` and `verbose=False` when using `optparse`, but a usage error with `argparse` (complaining that no value has been supplied for `-o/--output`, since `-v` is interpreted as meaning the verbosity flag)
- similarly, supplying `-o --` gives `output="--"` and `args=()` when using `optparse`, but a usage error with `argparse` (also complaining that no value has been supplied for `-o/--output`, since `--` is interpreted as terminating the option processing and treating all remaining values as positional arguments)
- supplying `-o=foo` gives `output="=foo"` when using `optparse`, but gives `output="foo"` with `argparse` (since `=` is special cased as an alternative separator for option parameter values)

Whether these differing behaviors in the `argparse` version are considered desirable or a problem will depend on the specific command line application use case.

<div class="seealso">

`click` is a third party argument processing library (originally based on `optparse`), which allows command line applications to be developed as a set of decorated command implementation functions.

Other third party libraries, such as `typer` or `msgspec-click`, allow command line interfaces to be specified in ways that more effectively integrate with static checking of Python type annotations.

</div>

## Introduction

`!optparse` is a more convenient, flexible, and powerful library for parsing command-line options than the minimalist `getopt` module. `!optparse` uses a more declarative style of command-line parsing: you create an instance of `OptionParser`, populate it with options, and parse the command line. `!optparse` allows users to specify options in the conventional GNU/POSIX syntax, and additionally generates usage and help messages for you.

Here's an example of using `!optparse` in a simple script:

    from optparse import OptionParser
    ...
    parser = OptionParser()
    parser.add_option("-f", "--file", dest="filename",
                      help="write report to FILE", metavar="FILE")
    parser.add_option("-q", "--quiet",
                      action="store_false", dest="verbose", default=True,
                      help="don't print status messages to stdout")

    (options, args) = parser.parse_args()

With these few lines of code, users of your script can now do the "usual thing" on the command-line, for example:

    <yourscript> --file=outfile -q

As it parses the command line, `!optparse` sets attributes of the `options` object returned by `~OptionParser.parse_args` based on user-supplied command-line values. When `~OptionParser.parse_args` returns from parsing this command line, `options.filename` will be `"outfile"` and `options.verbose` will be `False`. `!optparse` supports both long and short options, allows short options to be merged together, and allows options to be associated with their arguments in a variety of ways. Thus, the following command lines are all equivalent to the above example:

    <yourscript> -f outfile --quiet
    <yourscript> --quiet --file outfile
    <yourscript> -q -foutfile
    <yourscript> -qfoutfile

Additionally, users can run one of the following :

    <yourscript> -h
    <yourscript> --help

and `!optparse` will print out a brief summary of your script's options:

```
Usage: <yourscript> [options]

Options:
  -h, --help            show this help message and exit
  -f FILE, --file=FILE  write report to FILE
  -q, --quiet           don't print status messages to stdout
```

where the value of *yourscript* is determined at runtime (normally from `sys.argv[0]`).

## Background

`!optparse` was explicitly designed to encourage the creation of programs with straightforward command-line interfaces that follow the conventions established by the `!getopt` family of functions available to C developers. To that end, it supports only the most common command-line syntax and semantics conventionally used under Unix. If you are unfamiliar with these conventions, reading this section will allow you to acquaint yourself with them.

### Terminology

argument  
a string entered on the command-line, and passed by the shell to `execl()` or `execv()`. In Python, arguments are elements of `sys.argv[1:]` (`sys.argv[0]` is the name of the program being executed). Unix shells also use the term "word".

It is occasionally desirable to substitute an argument list other than `sys.argv[1:]`, so you should read "argument" as "an element of `sys.argv[1:]`, or of some other list provided as a substitute for `sys.argv[1:]`".

option  
an argument used to supply extra information to guide or customize the execution of a program. There are many different syntaxes for options; the traditional Unix syntax is a hyphen ("-") followed by a single letter, e.g. `-x` or `-F`. Also, traditional Unix syntax allows multiple options to be merged into a single argument, e.g. `-x -F` is equivalent to `-xF`. The GNU project introduced `--` followed by a series of hyphen-separated words, e.g. `--file` or `--dry-run`. These are the only two option syntaxes provided by `!optparse`.

Some other option syntaxes that the world has seen include:

- a hyphen followed by a few letters, e.g. `-pf` (this is *not* the same as multiple options merged into a single argument)
- a hyphen followed by a whole word, e.g. `-file` (this is technically equivalent to the previous syntax, but they aren't usually seen in the same program)
- a plus sign followed by a single letter, or a few letters, or a word, e.g. `+f`, `+rgb`
- a slash followed by a letter, or a few letters, or a word, e.g. `/f`, `/file`

These option syntaxes are not supported by `!optparse`, and they never will be. This is deliberate: the first three are non-standard on any environment, and the last only makes sense if you're exclusively targeting Windows or certain legacy platforms (e.g. VMS, MS-DOS).

option argument  
an argument that follows an option, is closely associated with that option, and is consumed from the argument list when that option is. With `!optparse`, option arguments may either be in a separate argument from their option:

```
-f foo
--file foo
```

or included in the same argument:

```
-ffoo
--file=foo
```

Typically, a given option either takes an argument or it doesn't. Lots of people want an "optional option arguments" feature, meaning that some options will take an argument if they see it, and won't if they don't. This is somewhat controversial, because it makes parsing ambiguous: if `-a` takes an optional argument and `-b` is another option entirely, how do we interpret `-ab`? Because of this ambiguity, `!optparse` does not support this feature.

positional argument  
something leftover in the argument list after options have been parsed, i.e. after options and their arguments have been parsed and removed from the argument list.

required option  
an option that must be supplied on the command-line; note that the phrase "required option" is self-contradictory in English. `!optparse` doesn't prevent you from implementing required options, but doesn't give you much help at it either.

For example, consider this hypothetical command-line:

    prog -v --report report.txt foo bar

`-v` and `--report` are both options. Assuming that `--report` takes one argument, `report.txt` is an option argument. `foo` and `bar` are positional arguments.

### What are options for?

Options are used to provide extra information to tune or customize the execution of a program. In case it wasn't clear, options are usually *optional*. A program should be able to run just fine with no options whatsoever. (Pick a random program from the Unix or GNU toolsets. Can it run without any options at all and still make sense? The main exceptions are `find`, `tar`, and `dd`---all of which are mutant oddballs that have been rightly criticized for their non-standard syntax and confusing interfaces.)

Lots of people want their programs to have "required options". Think about it. If it's required, then it's *not optional*! If there is a piece of information that your program absolutely requires in order to run successfully, that's what positional arguments are for.

As an example of good command-line interface design, consider the humble `cp` utility, for copying files. It doesn't make much sense to try to copy files without supplying a destination and at least one source. Hence, `cp` fails if you run it with no arguments. However, it has a flexible, useful syntax that does not require any options at all:

    cp SOURCE DEST
    cp SOURCE ... DEST-DIR

You can get pretty far with just that. Most `cp` implementations provide a bunch of options to tweak exactly how the files are copied: you can preserve mode and modification time, avoid following symlinks, ask before clobbering existing files, etc. But none of this distracts from the core mission of `cp`, which is to copy either one file to another, or several files to another directory.

### What are positional arguments for?

Positional arguments are for those pieces of information that your program absolutely, positively requires to run.

A good user interface should have as few absolute requirements as possible. If your program requires 17 distinct pieces of information in order to run successfully, it doesn't much matter *how* you get that information from the user---most people will give up and walk away before they successfully run the program. This applies whether the user interface is a command-line, a configuration file, or a GUI: if you make that many demands on your users, most of them will simply give up.

In short, try to minimize the amount of information that users are absolutely required to supply---use sensible defaults whenever possible. Of course, you also want to make your programs reasonably flexible. That's what options are for. Again, it doesn't matter if they are entries in a config file, widgets in the "Preferences" dialog of a GUI, or command-line options---the more options you implement, the more flexible your program is, and the more complicated its implementation becomes. Too much flexibility has drawbacks as well, of course; too many options can overwhelm users and make your code much harder to maintain.

## Tutorial

While `!optparse` is quite flexible and powerful, it's also straightforward to use in most cases. This section covers the code patterns that are common to any `!optparse`-based program.

First, you need to import the OptionParser class; then, early in the main program, create an OptionParser instance:

    from optparse import OptionParser
    ...
    parser = OptionParser()

Then you can start defining options. The basic syntax is:

    parser.add_option(opt_str, ...,
                      attr=value, ...)

Each option has one or more option strings, such as `-f` or `--file`, and several option attributes that tell `!optparse` what to expect and what to do when it encounters that option on the command line.

Typically, each option will have one short option string and one long option string, e.g.:

    parser.add_option("-f", "--file", ...)

You're free to define as many short option strings and as many long option strings as you like (including zero), as long as there is at least one option string overall.

The option strings passed to `OptionParser.add_option` are effectively labels for the option defined by that call. For brevity, we will frequently refer to *encountering an option* on the command line; in reality, `!optparse` encounters *option strings* and looks up options from them.

Once all of your options are defined, instruct `!optparse` to parse your program's command line:

    (options, args) = parser.parse_args()

(If you like, you can pass a custom argument list to `~OptionParser.parse_args`, but that's rarely necessary: by default it uses `sys.argv[1:]`.)

`~OptionParser.parse_args` returns two values:

- `options`, an object containing values for all of your options---e.g. if `--file` takes a single string argument, then `options.file` will be the filename supplied by the user, or `None` if the user did not supply that option
- `args`, the list of positional arguments leftover after parsing options

This tutorial section only covers the four most important option attributes: `~Option.action`, `~Option.type`, `~Option.dest` (destination), and `~Option.help`. Of these, `~Option.action` is the most fundamental.

### Understanding option actions

Actions tell `!optparse` what to do when it encounters an option on the command line. There is a fixed set of actions hard-coded into `!optparse`; adding new actions is an advanced topic covered in section `optparse-extending-optparse`. Most actions tell `!optparse` to store a value in some variable---for example, take a string from the command line and store it in an attribute of `options`.

If you don't specify an option action, `!optparse` defaults to `store`.

### The store action

The most common option action is `store`, which tells `!optparse` to take the next argument (or the remainder of the current argument), ensure that it is of the correct type, and store it to your chosen destination.

For example:

    parser.add_option("-f", "--file",
                      action="store", type="string", dest="filename")

Now let's make up a fake command line and ask `!optparse` to parse it:

    args = ["-f", "foo.txt"]
    (options, args) = parser.parse_args(args)

When `!optparse` sees the option string `-f`, it consumes the next argument, `foo.txt`, and stores it in `options.filename`. So, after this call to `~OptionParser.parse_args`, `options.filename` is `"foo.txt"`.

Some other option types supported by `!optparse` are `int` and `float`. Here's an option that expects an integer argument:

    parser.add_option("-n", type="int", dest="num")

Note that this option has no long option string, which is perfectly acceptable. Also, there's no explicit action, since the default is `store`.

Let's parse another fake command-line. This time, we'll jam the option argument right up against the option: since `-n42` (one argument) is equivalent to `-n 42` (two arguments), the code :

    (options, args) = parser.parse_args(["-n42"])
    print(options.num)

will print `42`.

If you don't specify a type, `!optparse` assumes `string`. Combined with the fact that the default action is `store`, that means our first example can be a lot shorter:

    parser.add_option("-f", "--file", dest="filename")

If you don't supply a destination, `!optparse` figures out a sensible default from the option strings: if the first long option string is `--foo-bar`, then the default destination is `foo_bar`. If there are no long option strings, `!optparse` looks at the first short option string: the default destination for `-f` is `f`.

`!optparse` also includes the built-in `complex` type. Adding types is covered in section `optparse-extending-optparse`.

### Handling boolean (flag) options

Flag options---set a variable to true or false when a particular option is seen---are quite common. `!optparse` supports them with two separate actions, `store_true` and `store_false`. For example, you might have a `verbose` flag that is turned on with `-v` and off with `-q`:

    parser.add_option("-v", action="store_true", dest="verbose")
    parser.add_option("-q", action="store_false", dest="verbose")

Here we have two different options with the same destination, which is perfectly OK. (It just means you have to be a bit careful when setting default values---see below.)

When `!optparse` encounters `-v` on the command line, it sets `options.verbose` to `True`; when it encounters `-q`, `options.verbose` is set to `False`.

### Other actions

Some other actions supported by `!optparse` are:

`"store_const"`  
store a constant value, pre-set via `Option.const`

`"append"`  
append this option's argument to a list

`"count"`  
increment a counter by one

`"callback"`  
call a specified function

These are covered in section `optparse-reference-guide`, and section `optparse-option-callbacks`.

### Default values

All of the above examples involve setting some variable (the "destination") when certain command-line options are seen. What happens if those options are never seen? Since we didn't supply any defaults, they are all set to `None`. This is usually fine, but sometimes you want more control. `!optparse` lets you supply a default value for each destination, which is assigned before the command line is parsed.

First, consider the verbose/quiet example. If we want `!optparse` to set `verbose` to `True` unless `-q` is seen, then we can do this:

    parser.add_option("-v", action="store_true", dest="verbose", default=True)
    parser.add_option("-q", action="store_false", dest="verbose")

Since default values apply to the *destination* rather than to any particular option, and these two options happen to have the same destination, this is exactly equivalent:

    parser.add_option("-v", action="store_true", dest="verbose")
    parser.add_option("-q", action="store_false", dest="verbose", default=True)

Consider this:

    parser.add_option("-v", action="store_true", dest="verbose", default=False)
    parser.add_option("-q", action="store_false", dest="verbose", default=True)

Again, the default value for `verbose` will be `True`: the last default value supplied for any particular destination is the one that counts.

A clearer way to specify default values is the `set_defaults` method of OptionParser, which you can call at any time before calling `~OptionParser.parse_args`:

    parser.set_defaults(verbose=True)
    parser.add_option(...)
    (options, args) = parser.parse_args()

As before, the last value specified for a given option destination is the one that counts. For clarity, try to use one method or the other of setting default values, not both.

### Generating help

`!optparse`'s ability to generate help and usage text automatically is useful for creating user-friendly command-line interfaces. All you have to do is supply a `~Option.help` value for each option, and optionally a short usage message for your whole program. Here's an OptionParser populated with user-friendly (documented) options:

    usage = "usage: %prog [options] arg1 arg2"
    parser = OptionParser(usage=usage)
    parser.add_option("-v", "--verbose",
                      action="store_true", dest="verbose", default=True,
                      help="make lots of noise [default]")
    parser.add_option("-q", "--quiet",
                      action="store_false", dest="verbose",
                      help="be vewwy quiet (I'm hunting wabbits)")
    parser.add_option("-f", "--filename",
                      metavar="FILE", help="write output to FILE")
    parser.add_option("-m", "--mode",
                      default="intermediate",
                      help="interaction mode: novice, intermediate, "
                           "or expert [default: %default]")

If `!optparse` encounters either `-h` or `--help` on the command-line, or if you just call `parser.print_help`, it prints the following to standard output:

```
Usage: <yourscript> [options] arg1 arg2

Options:
  -h, --help            show this help message and exit
  -v, --verbose         make lots of noise [default]
  -q, --quiet           be vewwy quiet (I'm hunting wabbits)
  -f FILE, --filename=FILE
                        write output to FILE
  -m MODE, --mode=MODE  interaction mode: novice, intermediate, or
                        expert [default: intermediate]
```

(If the help output is triggered by a help option, `!optparse` exits after printing the help text.)

There's a lot going on here to help `!optparse` generate the best possible help message:

- the script defines its own usage message:

      usage = "usage: %prog [options] arg1 arg2"

  `!optparse` expands `%prog` in the usage string to the name of the current program, i.e. `os.path.basename(sys.argv[0])`. The expanded string is then printed before the detailed option help.

  If you don't supply a usage string, `!optparse` uses a bland but sensible default: `"Usage: %prog [options]"`, which is fine if your script doesn't take any positional arguments.

- every option defines a help string, and doesn't worry about line-wrapping---`!optparse` takes care of wrapping lines and making the help output look good.

- options that take a value indicate this fact in their automatically generated help message, e.g. for the "mode" option:

      -m MODE, --mode=MODE

  Here, "MODE" is called the meta-variable: it stands for the argument that the user is expected to supply to `-m`/`--mode`. By default, `!optparse` converts the destination variable name to uppercase and uses that for the meta-variable. Sometimes, that's not what you want---for example, the `--filename` option explicitly sets `metavar="FILE"`, resulting in this automatically generated option description:

      -f FILE, --filename=FILE

  This is important for more than just saving space, though: the manually written help text uses the meta-variable `FILE` to clue the user in that there's a connection between the semi-formal syntax `-f FILE` and the informal semantic description "write output to FILE". This is a simple but effective way to make your help text a lot clearer and more useful for end users.

- options that have a default value can include `%default` in the help string---`!optparse` will replace it with `str` of the option's default value. If an option has no default value (or the default value is `None`), `%default` expands to `none`.

#### Grouping Options

When dealing with many options, it is convenient to group these options for better help output. An `OptionParser` can contain several option groups, each of which can contain several options.

An option group is obtained using the class `OptionGroup`:

<div class="OptionGroup(parser, title, description=None)">

where

- parser is the `OptionParser` instance the group will be inserted in to
- title is the group title
- description, optional, is a long description of the group

</div>

`OptionGroup` inherits from `OptionContainer` (like `OptionParser`) and so the `add_option` method can be used to add an option to the group.

Once all the options are declared, using the `OptionParser` method `add_option_group` the group is added to the previously defined parser.

Continuing with the parser defined in the previous section, adding an `OptionGroup` to a parser is easy:

    group = OptionGroup(parser, "Dangerous Options",
                        "Caution: use these options at your own risk.  "
                        "It is believed that some of them bite.")
    group.add_option("-g", action="store_true", help="Group option.")
    parser.add_option_group(group)

This would result in the following help output:

```
Usage: <yourscript> [options] arg1 arg2

Options:
  -h, --help            show this help message and exit
  -v, --verbose         make lots of noise [default]
  -q, --quiet           be vewwy quiet (I'm hunting wabbits)
  -f FILE, --filename=FILE
                        write output to FILE
  -m MODE, --mode=MODE  interaction mode: novice, intermediate, or
                        expert [default: intermediate]

  Dangerous Options:
    Caution: use these options at your own risk.  It is believed that some
    of them bite.

    -g                  Group option.
```

A bit more complete example might involve using more than one group: still extending the previous example:

    group = OptionGroup(parser, "Dangerous Options",
                        "Caution: use these options at your own risk.  "
                        "It is believed that some of them bite.")
    group.add_option("-g", action="store_true", help="Group option.")
    parser.add_option_group(group)

    group = OptionGroup(parser, "Debug Options")
    group.add_option("-d", "--debug", action="store_true",
                     help="Print debug information")
    group.add_option("-s", "--sql", action="store_true",
                     help="Print all SQL statements executed")
    group.add_option("-e", action="store_true", help="Print every action done")
    parser.add_option_group(group)

that results in the following output:

```
Usage: <yourscript> [options] arg1 arg2

Options:
  -h, --help            show this help message and exit
  -v, --verbose         make lots of noise [default]
  -q, --quiet           be vewwy quiet (I'm hunting wabbits)
  -f FILE, --filename=FILE
                        write output to FILE
  -m MODE, --mode=MODE  interaction mode: novice, intermediate, or expert
                        [default: intermediate]

  Dangerous Options:
    Caution: use these options at your own risk.  It is believed that some
    of them bite.

    -g                  Group option.

  Debug Options:
    -d, --debug         Print debug information
    -s, --sql           Print all SQL statements executed
    -e                  Print every action done
```

Another interesting method, in particular when working programmatically with option groups is:

<div class="method">

OptionParser.get_option_group(opt_str)

Return the `OptionGroup` to which the short or long option string *opt_str* (e.g. `'-o'` or `'--option'`) belongs. If there's no such `OptionGroup`, return `None`.

</div>

### Printing a version string

Similar to the brief usage string, `!optparse` can also print a version string for your program. You have to supply the string as the `version` argument to OptionParser:

    parser = OptionParser(usage="%prog [-f] [-q]", version="%prog 1.0")

`%prog` is expanded just like it is in `usage`. Apart from that, `version` can contain anything you like. When you supply it, `!optparse` automatically adds a `--version` option to your parser. If it encounters this option on the command line, it expands your `version` string (by replacing `%prog`), prints it to stdout, and exits.

For example, if your script is called `/usr/bin/foo`:

```
$ /usr/bin/foo --version
foo 1.0
```

The following two methods can be used to print and get the `version` string:

<div class="method">

OptionParser.print_version(file=None)

Print the version message for the current program (`self.version`) to *file* (default stdout). As with `print_usage`, any occurrence of `%prog` in `self.version` is replaced with the name of the current program. Does nothing if `self.version` is empty or undefined.

</div>

<div class="method">

OptionParser.get_version()

Same as `print_version` but returns the version string instead of printing it.

</div>

### How `!optparse` handles errors

There are two broad classes of errors that `!optparse` has to worry about: programmer errors and user errors. Programmer errors are usually erroneous calls to `OptionParser.add_option`, e.g. invalid option strings, unknown option attributes, missing option attributes, etc. These are dealt with in the usual way: raise an exception (either `optparse.OptionError` or `TypeError`) and let the program crash.

Handling user errors is much more important, since they are guaranteed to happen no matter how stable your code is. `!optparse` can automatically detect some user errors, such as bad option arguments (passing `-n 4x` where `-n` takes an integer argument), missing arguments (`-n` at the end of the command line, where `-n` takes an argument of any type). Also, you can call `OptionParser.error` to signal an application-defined error condition:

    (options, args) = parser.parse_args()
    ...
    if options.a and options.b:
        parser.error("options -a and -b are mutually exclusive")

In either case, `!optparse` handles the error the same way: it prints the program's usage message and an error message to standard error and exits with error status 2.

Consider the first example above, where the user passes `4x` to an option that takes an integer:

```
$ /usr/bin/foo -n 4x
Usage: foo [options]

foo: error: option -n: invalid integer value: '4x'
```

Or, where the user fails to pass a value at all:

```
$ /usr/bin/foo -n
Usage: foo [options]

foo: error: -n option requires an argument
```

`!optparse`-generated error messages take care always to mention the option involved in the error; be sure to do the same when calling `OptionParser.error` from your application code.

If `!optparse`'s default error-handling behaviour does not suit your needs, you'll need to subclass OptionParser and override its `~OptionParser.exit` and/or `~OptionParser.error` methods.

### Putting it all together

Here's what `!optparse`-based scripts usually look like:

    from optparse import OptionParser
    ...
    def main():
        usage = "usage: %prog [options] arg"
        parser = OptionParser(usage)
        parser.add_option("-f", "--file", dest="filename",
                          help="read data from FILENAME")
        parser.add_option("-v", "--verbose",
                          action="store_true", dest="verbose")
        parser.add_option("-q", "--quiet",
                          action="store_false", dest="verbose")
        ...
        (options, args) = parser.parse_args()
        if len(args) != 1:
            parser.error("incorrect number of arguments")
        if options.verbose:
            print("reading %s..." % options.filename)
        ...

    if __name__ == "__main__":
        main()

## Reference Guide

### Creating the parser

The first step in using `!optparse` is to create an OptionParser instance.

<div class="OptionParser(...)">

The OptionParser constructor has no required arguments, but a number of optional keyword arguments. You should always pass them as keyword arguments, i.e. do not rely on the order in which the arguments are declared.

`usage` (default: `"%prog [options]"`)  
The usage summary to print when your program is run incorrectly or with a help option. When `!optparse` prints the usage string, it expands `%prog` to `os.path.basename(sys.argv[0])` (or to `prog` if you passed that keyword argument). To suppress a usage message, pass the special value `optparse.SUPPRESS_USAGE`.

`option_list` (default: `[]`)  
A list of Option objects to populate the parser with. The options in `option_list` are added after any options in `standard_option_list` (a class attribute that may be set by OptionParser subclasses), but before any version or help options. Deprecated; use `add_option` after creating the parser instead.

`option_class` (default: optparse.Option)  
Class to use when adding options to the parser in `add_option`.

`version` (default: `None`)  
A version string to print when the user supplies a version option. If you supply a true value for `version`, `!optparse` automatically adds a version option with the single option string `--version`. The substring `%prog` is expanded the same as for `usage`.

`conflict_handler` (default: `"error"`)  
Specifies what to do when options with conflicting option strings are added to the parser; see section `optparse-conflicts-between-options`.

`description` (default: `None`)  
A paragraph of text giving a brief overview of your program. `!optparse` reformats this paragraph to fit the current terminal width and prints it when the user requests help (after `usage`, but before the list of options).

`formatter` (default: a new `IndentedHelpFormatter`)  
An instance of optparse.HelpFormatter that will be used for printing help text. `!optparse` provides two concrete classes for this purpose: IndentedHelpFormatter and TitledHelpFormatter.

`add_help_option` (default: `True`)  
If true, `!optparse` will add a help option (with option strings `-h` and `--help`) to the parser.

`prog`  
The string to use when expanding `%prog` in `usage` and `version` instead of `os.path.basename(sys.argv[0])`.

`epilog` (default: `None`)  
A paragraph of help text to print after the option help.

</div>

### Populating the parser

There are several ways to populate the parser with options. The preferred way is by using `OptionParser.add_option`, as shown in section `optparse-tutorial`. `add_option` can be called in one of two ways:

- pass it an Option instance (as returned by `make_option`)
- pass it any combination of positional and keyword arguments that are acceptable to `make_option` (i.e., to the Option constructor), and it will create the Option instance for you

The other alternative is to pass a list of pre-constructed Option instances to the OptionParser constructor, as in:

    option_list = [
        make_option("-f", "--filename",
                    action="store", type="string", dest="filename"),
        make_option("-q", "--quiet",
                    action="store_false", dest="verbose"),
        ]
    parser = OptionParser(option_list=option_list)

(`make_option` is a factory function for creating Option instances; currently it is an alias for the Option constructor. A future version of `!optparse` may split Option into several classes, and `make_option` will pick the right class to instantiate. Do not instantiate Option directly.)

### Defining options

Each Option instance represents a set of synonymous command-line option strings, e.g. `-f` and `--file`. You can specify any number of short or long option strings, but you must specify at least one overall option string.

The canonical way to create an `Option` instance is with the `add_option` method of `OptionParser`.

<div class="method">

OptionParser.add_option(option) OptionParser.add_option(\*opt_str, attr=value, ...)

To define an option with only a short option string:

    parser.add_option("-f", attr=value, ...)

And to define an option with only a long option string:

    parser.add_option("--foo", attr=value, ...)

The keyword arguments define attributes of the new Option object. The most important option attribute is `~Option.action`, and it largely determines which other attributes are relevant or required. If you pass irrelevant option attributes, or fail to pass required ones, `!optparse` raises an `OptionError` exception explaining your mistake.

An option's *action* determines what `!optparse` does when it encounters this option on the command-line. The standard option actions hard-coded into `!optparse` are:

`"store"`  
store this option's argument (default)

`"store_const"`  
store a constant value, pre-set via `Option.const`

`"store_true"`  
store `True`

`"store_false"`  
store `False`

`"append"`  
append this option's argument to a list

`"append_const"`  
append a constant value to a list, pre-set via `Option.const`

`"count"`  
increment a counter by one

`"callback"`  
call a specified function

`"help"`  
print a usage message including all options and the documentation for them

(If you don't supply an action, the default is `"store"`. For this action, you may also supply `~Option.type` and `~Option.dest` option attributes; see `optparse-standard-option-actions`.)

</div>

As you can see, most actions involve storing or updating a value somewhere. `!optparse` always creates a special object for this, conventionally called `options`, which is an instance of `optparse.Values`.

<div class="Values">

An object holding parsed argument names and values as attributes. Normally created by calling when calling `OptionParser.parse_args`, and can be overridden by a custom subclass passed to the *values* argument of `OptionParser.parse_args` (as described in `optparse-parsing-arguments`).

`!Values` objects support `copy.replace`, which returns a copy of the object with the specified attributes replaced.

<div class="versionchanged">

3.16 Added support for `copy.replace`.

</div>

</div>

Option arguments (and various other values) are stored as attributes of this object, according to the `~Option.dest` (destination) option attribute.

For example, when you call :

    parser.parse_args()

one of the first things `!optparse` does is create the `options` object:

    options = Values()

If one of the options in this parser is defined with :

    parser.add_option("-f", "--file", action="store", type="string", dest="filename")

and the command-line being parsed includes any of the following:

    -ffoo
    -f foo
    --file=foo
    --file foo

then `!optparse`, on seeing this option, will do the equivalent of :

    options.filename = "foo"

The `~Option.type` and `~Option.dest` option attributes are almost as important as `~Option.action`, but `~Option.action` is the only one that makes sense for *all* options.

### Option attributes

<div class="Option">

A single command line argument, with various attributes passed by keyword to the constructor. Normally created with `OptionParser.add_option` rather than directly, and can be overridden by a custom class via the *option_class* argument to `OptionParser`.

</div>

The following option attributes may be passed as keyword arguments to `OptionParser.add_option`. If you pass an option attribute that is not relevant to a particular option, or fail to pass a required option attribute, `!optparse` raises `OptionError`.

<div class="attribute">

Option.action

(default: `"store"`)

Determines `!optparse`'s behaviour when this option is seen on the command line; the available options are documented `here
<optparse-standard-option-actions>`.

</div>

<div class="attribute">

Option.type

(default: `"string"`)

The argument type expected by this option (e.g., `"string"` or `"int"`); the available option types are documented `here
<optparse-standard-option-types>`.

</div>

<div class="attribute">

Option.dest

(default: derived from option strings)

If the option's action implies writing or modifying a value somewhere, this tells `!optparse` where to write it: `~Option.dest` names an attribute of the `options` object that `!optparse` builds as it parses the command line.

</div>

<div class="attribute">

Option.default

The value to use for this option's destination if the option is not seen on the command line. See also `OptionParser.set_defaults`.

</div>

<div class="attribute">

Option.nargs

(default: 1)

How many arguments of type `~Option.type` should be consumed when this option is seen. If \> 1, `!optparse` will store a tuple of values to `~Option.dest`.

</div>

<div class="attribute">

Option.const

For actions that store a constant value, the constant value to store.

</div>

<div class="attribute">

Option.choices

For options of type `"choice"`, the list of strings the user may choose from.

</div>

<div class="attribute">

Option.callback

For options with action `"callback"`, the callable to call when this option is seen. See section `optparse-option-callbacks` for detail on the arguments passed to the callable.

</div>

<div class="attribute">

Option.callback_args Option.callback_kwargs

Additional positional and keyword arguments to pass to `callback` after the four standard callback arguments.

</div>

<div class="attribute">

Option.help

Help text to print for this option when listing all available options after the user supplies a `~Option.help` option (such as `--help`). If no help text is supplied, the option will be listed without help text. To hide this option, use the special value `optparse.SUPPRESS_HELP`.

</div>

<div class="attribute">

Option.metavar

(default: derived from option strings)

Stand-in for the option argument(s) to use when printing help text. See section `optparse-tutorial` for an example.

</div>

### Standard option actions

The various option actions all have slightly different requirements and effects. Most actions have several relevant option attributes which you may specify to guide `!optparse`'s behaviour; a few have required attributes, which you must specify for any option using that action.

- `"store"` \[relevant: `~Option.type`, `~Option.dest`, `~Option.nargs`, `~Option.choices`\]

  The option must be followed by an argument, which is converted to a value according to `~Option.type` and stored in `~Option.dest`. If `~Option.nargs` \> 1, multiple arguments will be consumed from the command line; all will be converted according to `~Option.type` and stored to `~Option.dest` as a tuple. See the `optparse-standard-option-types` section.

  If `~Option.choices` is supplied (a list or tuple of strings), the type defaults to `"choice"`.

  If `~Option.type` is not supplied, it defaults to `"string"`.

  If `~Option.dest` is not supplied, `!optparse` derives a destination from the first long option string (e.g., `--foo-bar` implies `foo_bar`). If there are no long option strings, `!optparse` derives a destination from the first short option string (e.g., `-f` implies `f`).

  Example:

      parser.add_option("-f")
      parser.add_option("-p", type="float", nargs=3, dest="point")

  As it parses the command line :

      -f foo.txt -p 1 -3.5 4 -fbar.txt

  `!optparse` will set :

      options.f = "foo.txt"
      options.point = (1.0, -3.5, 4.0)
      options.f = "bar.txt"

- `"store_const"` \[required: `~Option.const`; relevant: `~Option.dest`\]

  The value `~Option.const` is stored in `~Option.dest`.

  Example:

      parser.add_option("-q", "--quiet",
                        action="store_const", const=0, dest="verbose")
      parser.add_option("-v", "--verbose",
                        action="store_const", const=1, dest="verbose")
      parser.add_option("--noisy",
                        action="store_const", const=2, dest="verbose")

  If `--noisy` is seen, `!optparse` will set :

      options.verbose = 2

- `"store_true"` \[relevant: `~Option.dest`\]

  A special case of `"store_const"` that stores `True` to `~Option.dest`.

- `"store_false"` \[relevant: `~Option.dest`\]

  Like `"store_true"`, but stores `False`.

  Example:

      parser.add_option("--clobber", action="store_true", dest="clobber")
      parser.add_option("--no-clobber", action="store_false", dest="clobber")

- `"append"` \[relevant: `~Option.type`, `~Option.dest`, `~Option.nargs`, `~Option.choices`\]

  The option must be followed by an argument, which is appended to the list in `~Option.dest`. If no default value for `~Option.dest` is supplied, an empty list is automatically created when `!optparse` first encounters this option on the command-line. If `~Option.nargs` \> 1, multiple arguments are consumed, and a tuple of length `~Option.nargs` is appended to `~Option.dest`.

  The defaults for `~Option.type` and `~Option.dest` are the same as for the `"store"` action.

  Example:

      parser.add_option("-t", "--tracks", action="append", type="int")

  If `-t3` is seen on the command-line, `!optparse` does the equivalent of:

      options.tracks = []
      options.tracks.append(int("3"))

  If, a little later on, `--tracks=4` is seen, it does:

      options.tracks.append(int("4"))

  The `append` action calls the `append` method on the current value of the option. This means that any default value specified must have an `append` method. It also means that if the default value is non-empty, the default elements will be present in the parsed value for the option, with any values from the command line appended after those default values:

      >>> parser.add_option("--files", action="append", default=['~/.mypkg/defaults'])
      >>> opts, args = parser.parse_args(['--files', 'overrides.mypkg'])
      >>> opts.files
      ['~/.mypkg/defaults', 'overrides.mypkg']

- `"append_const"` \[required: `~Option.const`; relevant: `~Option.dest`\]

  Like `"store_const"`, but the value `~Option.const` is appended to `~Option.dest`; as with `"append"`, `~Option.dest` defaults to `None`, and an empty list is automatically created the first time the option is encountered.

- `"count"` \[relevant: `~Option.dest`\]

  Increment the integer stored at `~Option.dest`. If no default value is supplied, `~Option.dest` is set to zero before being incremented the first time.

  Example:

      parser.add_option("-v", action="count", dest="verbosity")

  The first time `-v` is seen on the command line, `!optparse` does the equivalent of:

      options.verbosity = 0
      options.verbosity += 1

  Every subsequent occurrence of `-v` results in :

      options.verbosity += 1

- `"callback"` \[required: `~Option.callback`; relevant: `~Option.type`, `~Option.nargs`, `~Option.callback_args`, `~Option.callback_kwargs`\]

  Call the function specified by `~Option.callback`, which is called as :

      func(option, opt_str, value, parser, *args, **kwargs)

  See section `optparse-option-callbacks` for more detail.

- `"help"`

  Prints a complete help message for all the options in the current option parser. The help message is constructed from the `usage` string passed to OptionParser's constructor and the `~Option.help` string passed to every option.

  If no `~Option.help` string is supplied for an option, it will still be listed in the help message. To omit an option entirely, use the special value `optparse.SUPPRESS_HELP`.

  `!optparse` automatically adds a `~Option.help` option to all OptionParsers, so you do not normally need to create one.

  Example:

      from optparse import OptionParser, SUPPRESS_HELP

      # usually, a help option is added automatically, but that can
      # be suppressed using the add_help_option argument
      parser = OptionParser(add_help_option=False)

      parser.add_option("-h", "--help", action="help")
      parser.add_option("-v", action="store_true", dest="verbose",
                        help="Be moderately verbose")
      parser.add_option("--file", dest="filename",
                        help="Input file to read data from")
      parser.add_option("--secret", help=SUPPRESS_HELP)

  If `!optparse` sees either `-h` or `--help` on the command line, it will print something like the following help message to stdout (assuming `sys.argv[0]` is `"foo.py"`):

  ```
  Usage: foo.py [options]

  Options:
    -h, --help        Show this help message and exit
    -v                Be moderately verbose
    --file=FILENAME   Input file to read data from
  ```

  After printing the help message, `!optparse` terminates your process with `sys.exit(0)`.

- `"version"`

  Prints the version number supplied to the OptionParser to stdout and exits. The version number is actually formatted and printed by the `print_version()` method of OptionParser. Generally only relevant if the `version` argument is supplied to the OptionParser constructor. As with `~Option.help` options, you will rarely create `version` options, since `!optparse` automatically adds them when needed.

### Standard option types

`!optparse` has five built-in option types: `"string"`, `"int"`, `"choice"`, `"float"` and `"complex"`. If you need to add new option types, see section `optparse-extending-optparse`.

Arguments to string options are not checked or converted in any way: the text on the command line is stored in the destination (or passed to the callback) as-is.

Integer arguments (type `"int"`) are parsed as follows:

- if the number starts with `0x`, it is parsed as a hexadecimal number
- if the number starts with `0`, it is parsed as an octal number
- if the number starts with `0b`, it is parsed as a binary number
- otherwise, the number is parsed as a decimal number

The conversion is done by calling `int` with the appropriate base (2, 8, 10, or 16). If this fails, so will `!optparse`, although with a more useful error message.

`"float"` and `"complex"` option arguments are converted directly with `float` and `complex`, with similar error-handling.

`"choice"` options are a subtype of `"string"` options. The `~Option.choices` option attribute (a sequence of strings) defines the set of allowed option arguments. `optparse.check_choice` compares user-supplied option arguments against this master list and raises `OptionValueError` if an invalid string is given.

### Parsing arguments

The whole point of creating and populating an OptionParser is to call its `~OptionParser.parse_args` method.

<div class="method">

OptionParser.parse_args(args=None, values=None)

Parse the command-line options found in *args*.

The input parameters are

`args`  
the list of arguments to process (default: `sys.argv[1:]`)

`values`  
a `Values` object to store option arguments in (default: a new instance of `Values`) -- if you give an existing object, the option defaults will not be initialized on it

and the return value is a pair `(options, args)` where

`options`  
the same object that was passed in as *values*, or the `optparse.Values` instance created by `!optparse`

`args`  
the leftover positional arguments after all options have been processed

</div>

The most common usage is to supply neither keyword argument. If you supply `values`, it will be modified with repeated `setattr` calls (roughly one for every option argument stored to an option destination) and returned by `~OptionParser.parse_args`.

If `~OptionParser.parse_args` encounters any errors in the argument list, it calls the OptionParser's `error` method with an appropriate end-user error message. This ultimately terminates your process with an exit status of 2 (the traditional Unix exit status for command-line errors).

### Querying and manipulating your option parser

The default behavior of the option parser can be customized slightly, and you can also poke around your option parser and see what's there. OptionParser provides several methods to help you out:

<div class="method">

OptionParser.disable_interspersed_args()

Set parsing to stop on the first non-option. For example, if `-a` and `-b` are both simple options that take no arguments, `!optparse` normally accepts this syntax:

    prog -a arg1 -b arg2

and treats it as equivalent to :

    prog -a -b arg1 arg2

To disable this feature, call `disable_interspersed_args`. This restores traditional Unix syntax, where option parsing stops with the first non-option argument.

Use this if you have a command processor which runs another command which has options of its own and you want to make sure these options don't get confused. For example, each command might have a different set of options.

</div>

<div class="method">

OptionParser.enable_interspersed_args()

Set parsing to not stop on the first non-option, allowing interspersing switches with command arguments. This is the default behavior.

</div>

<div class="method">

OptionParser.get_option(opt_str)

Returns the Option instance with the option string *opt_str*, or `None` if no options have that option string.

</div>

<div class="method">

OptionParser.has_option(opt_str)

Return `True` if the OptionParser has an option with option string *opt_str* (e.g., `-q` or `--verbose`).

</div>

<div class="method">

OptionParser.remove_option(opt_str)

If the `OptionParser` has an option corresponding to *opt_str*, that option is removed. If that option provided any other option strings, all of those option strings become invalid. If *opt_str* does not occur in any option belonging to this `OptionParser`, raises `ValueError`.

</div>

### Conflicts between options

If you're not careful, it's easy to define options with conflicting option strings:

    parser.add_option("-n", "--dry-run", ...)
    ...
    parser.add_option("-n", "--noisy", ...)

(This is particularly true if you've defined your own OptionParser subclass with some standard options.)

Every time you add an option, `!optparse` checks for conflicts with existing options. If it finds any, it invokes the current conflict-handling mechanism. You can set the conflict-handling mechanism either in the constructor:

    parser = OptionParser(..., conflict_handler=handler)

or with a separate call:

    parser.set_conflict_handler(handler)

The available conflict handlers are:

> `"error"` (default)  
> assume option conflicts are a programming error and raise `OptionConflictError`
>
> `"resolve"`  
> resolve option conflicts intelligently (see below)

As an example, let's define an `OptionParser` that resolves conflicts intelligently and add conflicting options to it:

    parser = OptionParser(conflict_handler="resolve")
    parser.add_option("-n", "--dry-run", ..., help="do no harm")
    parser.add_option("-n", "--noisy", ..., help="be noisy")

At this point, `!optparse` detects that a previously added option is already using the `-n` option string. Since `conflict_handler` is `"resolve"`, it resolves the situation by removing `-n` from the earlier option's list of option strings. Now `--dry-run` is the only way for the user to activate that option. If the user asks for help, the help message will reflect that:

    Options:
      --dry-run     do no harm
      ...
      -n, --noisy   be noisy

It's possible to whittle away the option strings for a previously added option until there are none left, and the user has no way of invoking that option from the command-line. In that case, `!optparse` removes that option completely, so it doesn't show up in help text or anywhere else. Carrying on with our existing OptionParser:

    parser.add_option("--dry-run", ..., help="new dry-run option")

At this point, the original `-n`/`--dry-run` option is no longer accessible, so `!optparse` removes it, leaving this help text:

    Options:
      ...
      -n, --noisy   be noisy
      --dry-run     new dry-run option

### Cleanup

OptionParser instances have several cyclic references. This should not be a problem for Python's garbage collector, but you may wish to break the cyclic references explicitly by calling `~OptionParser.destroy` on your OptionParser once you are done with it. This is particularly useful in long-running applications where large object graphs are reachable from your OptionParser.

### Other methods

OptionParser supports several other public methods:

<div class="method">

OptionParser.set_usage(usage)

Set the usage string according to the rules described above for the `usage` constructor keyword argument. Passing `None` sets the default usage string; use `optparse.SUPPRESS_USAGE` to suppress a usage message.

</div>

<div class="method">

OptionParser.print_usage(file=None)

Print the usage message for the current program (`self.usage`) to *file* (default stdout). Any occurrence of the string `%prog` in `self.usage` is replaced with the name of the current program. Does nothing if `self.usage` is empty or not defined.

</div>

<div class="method">

OptionParser.get_usage()

Same as `print_usage` but returns the usage string instead of printing it.

</div>

<div class="method">

OptionParser.set_defaults(dest=value, ...)

Set default values for several option destinations at once. Using `set_defaults` is the preferred way to set default values for options, since multiple options can share the same destination. For example, if several "mode" options all set the same destination, any one of them can set the default, and the last one wins:

    parser.add_option("--advanced", action="store_const",
                      dest="mode", const="advanced",
                      default="novice")    # overridden below
    parser.add_option("--novice", action="store_const",
                      dest="mode", const="novice",
                      default="advanced")  # overrides above setting

To avoid this confusion, use `set_defaults`:

    parser.set_defaults(mode="advanced")
    parser.add_option("--advanced", action="store_const",
                      dest="mode", const="advanced")
    parser.add_option("--novice", action="store_const",
                      dest="mode", const="novice")

</div>

## Option Callbacks

When `!optparse`'s built-in actions and types aren't quite enough for your needs, you have two choices: extend `!optparse` or define a callback option. Extending `!optparse` is more general, but overkill for a lot of simple cases. Quite often a simple callback is all you need.

There are two steps to defining a callback option:

- define the option itself using the `"callback"` action
- write the callback; this is a function (or method) that takes at least four arguments, as described below

### Defining a callback option

As always, the easiest way to define a callback option is by using the `OptionParser.add_option` method. Apart from `~Option.action`, the only option attribute you must specify is `callback`, the function to call:

    parser.add_option("-c", action="callback", callback=my_callback)

`callback` is a function (or other callable object), so you must have already defined `my_callback()` when you create this callback option. In this simple case, `!optparse` doesn't even know if `-c` takes any arguments, which usually means that the option takes no arguments---the mere presence of `-c` on the command-line is all it needs to know. In some circumstances, though, you might want your callback to consume an arbitrary number of command-line arguments. This is where writing callbacks gets tricky; it's covered later in this section.

`!optparse` always passes four particular arguments to your callback, and it will only pass additional arguments if you specify them via `~Option.callback_args` and `~Option.callback_kwargs`. Thus, the minimal callback function signature is:

    def my_callback(option, opt, value, parser):

The four arguments to a callback are described below.

There are several other option attributes that you can supply when you define a callback option:

`~Option.type`  
has its usual meaning: as with the `"store"` or `"append"` actions, it instructs `!optparse` to consume one argument and convert it to `~Option.type`. Rather than storing the converted value(s) anywhere, though, `!optparse` passes it to your callback function.

`~Option.nargs`  
also has its usual meaning: if it is supplied and \> 1, `!optparse` will consume `~Option.nargs` arguments, each of which must be convertible to `~Option.type`. It then passes a tuple of converted values to your callback.

`~Option.callback_args`  
a tuple of extra positional arguments to pass to the callback

`~Option.callback_kwargs`  
a dictionary of extra keyword arguments to pass to the callback

### How callbacks are called

All callbacks are called as follows:

    func(option, opt_str, value, parser, *args, **kwargs)

where

`option`  
is the Option instance that's calling the callback

`opt_str`  
is the option string seen on the command-line that's triggering the callback. (If an abbreviated long option was used, `opt_str` will be the full, canonical option string---e.g. if the user puts `--foo` on the command-line as an abbreviation for `--foobar`, then `opt_str` will be `"--foobar"`.)

`value`  
is the argument to this option seen on the command-line. `!optparse` will only expect an argument if `~Option.type` is set; the type of `value` will be the type implied by the option's type. If `~Option.type` for this option is `None` (no argument expected), then `value` will be `None`. If `~Option.nargs` \> 1, `value` will be a tuple of values of the appropriate type.

`parser`  
is the OptionParser instance driving the whole thing, mainly useful because you can access some other interesting data through its instance attributes:

`parser.largs`  
the current list of leftover arguments, ie. arguments that have been consumed but are neither options nor option arguments. Feel free to modify `parser.largs`, e.g. by adding more arguments to it. (This list will become `args`, the second return value of `~OptionParser.parse_args`.)

`parser.rargs`  
the current list of remaining arguments, ie. with `opt_str` and `value` (if applicable) removed, and only the arguments following them still there. Feel free to modify `parser.rargs`, e.g. by consuming more arguments.

`parser.values`  
the object where option values are by default stored (an instance of optparse.OptionValues). This lets callbacks use the same mechanism as the rest of `!optparse` for storing option values; you don't need to mess around with globals or closures. You can also access or modify the value(s) of any options already encountered on the command-line.

`args`  
is a tuple of arbitrary positional arguments supplied via the `~Option.callback_args` option attribute.

`kwargs`  
is a dictionary of arbitrary keyword arguments supplied via `~Option.callback_kwargs`.

### Raising errors in a callback

The callback function should raise `OptionValueError` if there are any problems with the option or its argument(s). `!optparse` catches this and terminates the program, printing the error message you supply to stderr. Your message should be clear, concise, accurate, and mention the option at fault. Otherwise, the user will have a hard time figuring out what they did wrong.

### Callback example 1: trivial callback

Here's an example of a callback option that takes no arguments, and simply records that the option was seen:

    def record_foo_seen(option, opt_str, value, parser):
        parser.values.saw_foo = True

    parser.add_option("--foo", action="callback", callback=record_foo_seen)

Of course, you could do that with the `"store_true"` action.

### Callback example 2: check option order

Here's a slightly more interesting example: record the fact that `-a` is seen, but blow up if it comes after `-b` in the command-line. :

    def check_order(option, opt_str, value, parser):
        if parser.values.b:
            raise OptionValueError("can't use -a after -b")
        parser.values.a = 1
    ...
    parser.add_option("-a", action="callback", callback=check_order)
    parser.add_option("-b", action="store_true", dest="b")

### Callback example 3: check option order (generalized)

If you want to reuse this callback for several similar options (set a flag, but blow up if `-b` has already been seen), it needs a bit of work: the error message and the flag that it sets must be generalized. :

    def check_order(option, opt_str, value, parser):
        if parser.values.b:
            raise OptionValueError("can't use %s after -b" % opt_str)
        setattr(parser.values, option.dest, 1)
    ...
    parser.add_option("-a", action="callback", callback=check_order, dest='a')
    parser.add_option("-b", action="store_true", dest="b")
    parser.add_option("-c", action="callback", callback=check_order, dest='c')

### Callback example 4: check arbitrary condition

Of course, you could put any condition in there---you're not limited to checking the values of already-defined options. For example, if you have options that should not be called when the moon is full, all you have to do is this:

    def check_moon(option, opt_str, value, parser):
        if is_moon_full():
            raise OptionValueError("%s option invalid when moon is full"
                                   % opt_str)
        setattr(parser.values, option.dest, 1)
    ...
    parser.add_option("--foo",
                      action="callback", callback=check_moon, dest="foo")

(The definition of `is_moon_full()` is left as an exercise for the reader.)

### Callback example 5: fixed arguments

Things get slightly more interesting when you define callback options that take a fixed number of arguments. Specifying that a callback option takes arguments is similar to defining a `"store"` or `"append"` option: if you define `~Option.type`, then the option takes one argument that must be convertible to that type; if you further define `~Option.nargs`, then the option takes `~Option.nargs` arguments.

Here's an example that just emulates the standard `"store"` action:

    def store_value(option, opt_str, value, parser):
        setattr(parser.values, option.dest, value)
    ...
    parser.add_option("--foo",
                      action="callback", callback=store_value,
                      type="int", nargs=3, dest="foo")

Note that `!optparse` takes care of consuming 3 arguments and converting them to integers for you; all you have to do is store them. (Or whatever; obviously you don't need a callback for this example.)

### Callback example 6: variable arguments

Things get hairy when you want an option to take a variable number of arguments. For this case, you must write a callback, as `!optparse` doesn't provide any built-in capabilities for it. And you have to deal with certain intricacies of conventional Unix command-line parsing that `!optparse` normally handles for you. In particular, callbacks should implement the conventional rules for bare `--` and `-` arguments:

- either `--` or `-` can be option arguments
- bare `--` (if not the argument to some option): halt command-line processing and discard the `--`
- bare `-` (if not the argument to some option): halt command-line processing but keep the `-` (append it to `parser.largs`)

If you want an option that takes a variable number of arguments, there are several subtle, tricky issues to worry about. The exact implementation you choose will be based on which trade-offs you're willing to make for your application (which is why `!optparse` doesn't support this sort of thing directly).

Nevertheless, here's a stab at a callback for an option with variable arguments:

    def vararg_callback(option, opt_str, value, parser):
        assert value is None
        value = []

        def floatable(str):
            try:
                float(str)
                return True
            except ValueError:
                return False

        for arg in parser.rargs:
            # stop on --foo like options
            if arg[:2] == "--" and len(arg) > 2:
                break
            # stop on -a, but not on -3 or -3.0
            if arg[:1] == "-" and len(arg) > 1 and not floatable(arg):
                break
            value.append(arg)

        del parser.rargs[:len(value)]
        setattr(parser.values, option.dest, value)

    ...
    parser.add_option("-c", "--callback", dest="vararg_attr",
                      action="callback", callback=vararg_callback)

## Extending `!optparse`

Since the two major controlling factors in how `!optparse` interprets command-line options are the action and type of each option, the most likely direction of extension is to add new actions and new types.

### Adding new types

To add new types, you need to define your own subclass of `!optparse`'s `Option` class. This class has a couple of attributes that define `!optparse`'s types: `~Option.TYPES` and `~Option.TYPE_CHECKER`.

<div class="attribute">

Option.TYPES

A tuple of type names; in your subclass, simply define a new tuple `TYPES` that builds on the standard one.

</div>

<div class="attribute">

Option.TYPE_CHECKER

A dictionary mapping type names to type-checking functions. A type-checking function has the following signature:

    def check_mytype(option, opt, value)

where `option` is an `Option` instance, `opt` is an option string (e.g., `-f`), and `value` is the string from the command line that must be checked and converted to your desired type. `check_mytype()` should return an object of the hypothetical type `mytype`. The value returned by a type-checking function will wind up in the OptionValues instance returned by `OptionParser.parse_args`, or be passed to a callback as the `value` parameter.

Your type-checking function should raise `OptionValueError` if it encounters any problems. `OptionValueError` takes a single string argument, which is passed as-is to `OptionParser`'s `error` method, which in turn prepends the program name and the string `"error:"` and prints everything to stderr before terminating the process.

</div>

Here's a silly example that demonstrates adding a `"complex"` option type to parse Python-style complex numbers on the command line. (This is even sillier than it used to be, because `!optparse` 1.3 added built-in support for complex numbers, but never mind.)

First, the necessary imports:

    from copy import copy
    from optparse import Option, OptionValueError

You need to define your type-checker first, since it's referred to later (in the `~Option.TYPE_CHECKER` class attribute of your Option subclass):

    def check_complex(option, opt, value):
        try:
            return complex(value)
        except ValueError:
            raise OptionValueError(
                "option %s: invalid complex value: %r" % (opt, value))

Finally, the Option subclass:

    class MyOption (Option):
        TYPES = Option.TYPES + ("complex",)
        TYPE_CHECKER = copy(Option.TYPE_CHECKER)
        TYPE_CHECKER["complex"] = check_complex

(If we didn't make a `~copy.copy` of `Option.TYPE_CHECKER`, we would end up modifying the `~Option.TYPE_CHECKER` attribute of `!optparse`'s Option class. This being Python, nothing stops you from doing that except good manners and common sense.)

That's it! Now you can write a script that uses the new option type just like any other `!optparse`-based script, except you have to instruct your OptionParser to use MyOption instead of Option:

    parser = OptionParser(option_class=MyOption)
    parser.add_option("-c", type="complex")

Alternately, you can build your own option list and pass it to OptionParser; if you don't use `add_option` in the above way, you don't need to tell OptionParser which option class to use:

    option_list = [MyOption("-c", action="store", type="complex", dest="c")]
    parser = OptionParser(option_list=option_list)

### Adding new actions

Adding new actions is a bit trickier, because you have to understand that `!optparse` has a couple of classifications for actions:

"store" actions  
actions that result in `!optparse` storing a value to an attribute of the current OptionValues instance; these options require a `~Option.dest` attribute to be supplied to the Option constructor.

"typed" actions  
actions that take a value from the command line and expect it to be of a certain type; or rather, a string that can be converted to a certain type. These options require a `~Option.type` attribute to the Option constructor.

These are overlapping sets: some default "store" actions are `"store"`, `"store_const"`, `"append"`, and `"count"`, while the default "typed" actions are `"store"`, `"append"`, and `"callback"`.

When you add an action, you need to categorize it by listing it in at least one of the following class attributes of Option (all are lists of strings):

<div class="attribute">

Option.ACTIONS

All actions must be listed in ACTIONS.

</div>

<div class="attribute">

Option.STORE_ACTIONS

"store" actions are additionally listed here.

</div>

<div class="attribute">

Option.TYPED_ACTIONS

"typed" actions are additionally listed here.

</div>

<div class="attribute">

Option.ALWAYS_TYPED_ACTIONS

Actions that always take a type (i.e. whose options always take a value) are additionally listed here. The only effect of this is that `!optparse` assigns the default type, `"string"`, to options with no explicit type whose action is listed in `ALWAYS_TYPED_ACTIONS`.

</div>

In order to actually implement your new action, you must override Option's `take_action` method and add a case that recognizes your action.

For example, let's add an `"extend"` action. This is similar to the standard `"append"` action, but instead of taking a single value from the command-line and appending it to an existing list, `"extend"` will take multiple values in a single comma-delimited string, and extend an existing list with them. That is, if `--names` is an `"extend"` option of type `"string"`, the command line :

    --names=foo,bar --names blah --names ding,dong

would result in a list :

    ["foo", "bar", "blah", "ding", "dong"]

Again we define a subclass of Option:

    class MyOption(Option):

        ACTIONS = Option.ACTIONS + ("extend",)
        STORE_ACTIONS = Option.STORE_ACTIONS + ("extend",)
        TYPED_ACTIONS = Option.TYPED_ACTIONS + ("extend",)
        ALWAYS_TYPED_ACTIONS = Option.ALWAYS_TYPED_ACTIONS + ("extend",)

        def take_action(self, action, dest, opt, value, values, parser):
            if action == "extend":
                lvalue = value.split(",")
                values.ensure_value(dest, []).extend(lvalue)
            else:
                Option.take_action(
                    self, action, dest, opt, value, values, parser)

Features of note:

- `"extend"` both expects a value on the command-line and stores that value somewhere, so it goes in both `~Option.STORE_ACTIONS` and `~Option.TYPED_ACTIONS`.

- to ensure that `!optparse` assigns the default type of `"string"` to `"extend"` actions, we put the `"extend"` action in `~Option.ALWAYS_TYPED_ACTIONS` as well.

- `MyOption.take_action` implements just this one new action, and passes control back to `Option.take_action` for the standard `!optparse` actions.

- `values` is an instance of the optparse_parser.Values class, which provides the very useful `ensure_value` method. `ensure_value` is essentially `getattr` with a safety valve; it is called as :

      values.ensure_value(attr, value)

  If the `attr` attribute of `values` doesn't exist or is `None`, then ensure_value() first sets it to `value`, and then returns `value`. This is very handy for actions like `"extend"`, `"append"`, and `"count"`, all of which accumulate data in a variable and expect that variable to be of a certain type (a list for the first two, an integer for the latter). Using `ensure_value` means that scripts using your action don't have to worry about setting a default value for the option destinations in question; they can just leave the default as `None` and `ensure_value` will take care of getting it right when it's needed.

## Exceptions

<div class="exception">

OptionError

Raised if an `Option` instance is created with invalid or inconsistent arguments.

</div>

<div class="exception">

OptionConflictError

Raised if conflicting options are added to an `OptionParser`.

</div>

<div class="exception">

OptionValueError

Raised if an invalid option value is encountered on the command line.

</div>

<div class="exception">

BadOptionError

Raised if an invalid option is passed on the command line.

</div>

<div class="exception">

AmbiguousOptionError

Raised if an ambiguous option is passed on the command line.

</div>

# `!os` --- Miscellaneous operating system interfaces

<div class="module" data-synopsis="Miscellaneous operating system interfaces.">

os

</div>

**Source code:** `Lib/os.py`

------------------------------------------------------------------------

This module provides a portable way of using operating system dependent functionality. If you just want to read or write a file see `open`, if you want to manipulate paths, see the `os.path` module, and if you want to read all the lines in all the files on the command line see the `fileinput` module. For creating temporary files and directories see the `tempfile` module, and for high-level file and directory handling see the `shutil` module.

Notes on the availability of these functions:

- The design of all built-in operating system dependent modules of Python is such that as long as the same functionality is available, it uses the same interface; for example, the function `os.stat(path)` returns stat information about *path* in the same format (which happens to have originated with the POSIX interface).
- Extensions peculiar to a particular operating system are also available through the `!os` module, but using them is of course a threat to portability.
- All functions accepting path or file names accept both bytes and string objects, and result in an object of the same type, if a path or file name is returned.
- On VxWorks, os.popen, os.fork, os.execv and os.spawn\*p\* are not supported.
- On WebAssembly platforms, Android and iOS, large parts of the `!os` module are not available or behave differently. APIs related to processes (e.g. `~os.fork`, `~os.execve`) and resources (e.g. `~os.nice`) are not available. Others like `~os.getuid` and `~os.getpid` are emulated or stubs. WebAssembly platforms also lack support for signals (e.g. `~os.kill`, `~os.wait`).

> [!NOTE]
> All functions in this module raise `OSError` (or subclasses thereof) in the case of invalid or inaccessible file names and paths, or other arguments that have the correct type, but are not accepted by the operating system.

<div class="exception">

error

An alias for the built-in `OSError` exception.

</div>

<div class="data">

name

The name of the operating system dependent module imported. The following names have currently been registered: `'posix'`, `'nt'`, `'java'`.

<div class="seealso">

`sys.platform` has a finer granularity. `os.uname` gives system-dependent version information.

The `platform` module provides detailed checks for the system's identity.

</div>

</div>

## File Names, Command Line Arguments, and Environment Variables<span id="os-filenames"></span>

In Python, file names, command line arguments, and environment variables are represented using the string type. On some systems, decoding these strings to and from bytes is necessary before passing them to the operating system. Python uses the `filesystem encoding and error handler` to perform this conversion (see `sys.getfilesystemencoding`).

The `filesystem encoding and error handler` are configured at Python startup by the `PyConfig_Read` function: see `~PyConfig.filesystem_encoding` and `~PyConfig.filesystem_errors` members of `PyConfig`.

<div class="versionchanged">

3.1 On some systems, conversion using the file system encoding may fail. In this case, Python uses the `surrogateescape encoding error handler
<surrogateescape>`, which means that undecodable bytes are replaced by a Unicode character U+DC*xx* on decoding, and these are again translated to the original byte on encoding.

</div>

The `file system encoding <filesystem encoding and error handler>` must guarantee to successfully decode all bytes below 128. If the file system encoding fails to provide this guarantee, API functions can raise `UnicodeError`.

See also the `locale encoding`.

## Python UTF-8 Mode

<div class="versionadded">

3.7 See `540` for more details.

</div>

<div class="versionchanged">

3.15

Python UTF-8 mode is now enabled by default (`686`). It may be disabled by setting `PYTHONUTF8=0 <PYTHONUTF8>` as an environment variable or by using the `-X utf8=0 <-X>` command line option.

</div>

The Python UTF-8 Mode ignores the `locale encoding` and forces the usage of the UTF-8 encoding:

- Use UTF-8 as the `filesystem encoding <filesystem encoding and error
  handler>`.
- `sys.getfilesystemencoding` returns `'utf-8'`.
- `locale.getpreferredencoding` returns `'utf-8'` (the *do_setlocale* argument has no effect).
- `sys.stdin`, `sys.stdout`, and `sys.stderr` all use UTF-8 as their text encoding, with the `surrogateescape` `error handler <error-handlers>` being enabled for `sys.stdin` and `sys.stdout` (`sys.stderr` continues to use `backslashreplace` as it does in the default locale-aware mode)
- On Unix, `os.device_encoding` returns `'utf-8'` rather than the device encoding.

Note that the standard stream settings in UTF-8 mode can be overridden by `PYTHONIOENCODING` (just as they can be in the default locale-aware mode).

As a consequence of the changes in those lower level APIs, other higher level APIs also exhibit different default behaviours:

- Command line arguments, environment variables and filenames are decoded to text using the UTF-8 encoding.
- `os.fsdecode` and `os.fsencode` use the UTF-8 encoding.
- `open`, `io.open`, and `codecs.open` use the UTF-8 encoding by default. However, they still use the strict error handler by default so that attempting to open a binary file in text mode is likely to raise an exception rather than producing nonsense data.

The `Python UTF-8 Mode <utf8-mode>` is enabled by default. It can be disabled using the `-X utf8=0 <-X>` command line option or the `PYTHONUTF8=0 <PYTHONUTF8>` environment variable. The Python UTF-8 Mode can only be disabled at Python startup. Its value can be read from `sys.flags.utf8_mode <sys.flags>`.

If the UTF-8 mode is disabled, the interpreter defaults to using the current locale settings, *unless* the current locale is identified as a legacy ASCII-based locale (as described for `PYTHONCOERCECLOCALE`), and locale coercion is either disabled or fails. In such legacy locales, the interpreter will default to enabling UTF-8 mode unless explicitly instructed not to do so.

See also the `UTF-8 mode on Windows <win-utf8-mode>` and the `filesystem encoding and error handler`.

## Process Parameters

These functions and data items provide information and operate on the current process and user.

<div class="function">

ctermid()

Return the filename corresponding to the controlling terminal of the process.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

environ

A `mapping` object where keys and values are strings that represent the process environment. For example, `environ['HOME']` is the pathname of your home directory (on some platforms), and is equivalent to `getenv("HOME")` in C.

This mapping is captured the first time the `!os` module is imported, typically during Python startup as part of processing `site.py`. Changes to the environment made after this time are not reflected in `os.environ`, except for changes made by modifying `os.environ` directly.

This mapping may be used to modify the environment as well as query the environment. `putenv` will be called automatically when the mapping is modified.

On Unix, keys and values use `sys.getfilesystemencoding` and `'surrogateescape'` error handler. Use `environb` if you would like to use a different encoding.

On Windows, the keys are converted to uppercase. This also applies when getting, setting, or deleting an item. For example, `environ['monty'] = 'python'` maps the key `'MONTY'` to the value `'python'`.

> [!NOTE]
> Calling `putenv` directly does not change `os.environ`, so it's better to modify `os.environ`.

> [!NOTE]
> On some platforms, including FreeBSD and macOS, setting `environ` may cause memory leaks. Refer to the system documentation for `!putenv`.

You can delete items in this mapping to unset environment variables. `unsetenv` will be called automatically when an item is deleted from `os.environ`, and when one of the `~dict.pop` or `~dict.clear` methods is called.

If the `clearenv(3)` function is available, the `~dict.clear` method uses it and emits a single `os._clearenv` audit event. Otherwise, it emits an `os.unsetenv` event on each deleted variable.

<div class="audit-event">

os.unsetenv key os.unsetenv

</div>

<div class="audit-event">

os.\_clearenv "" os.\_clearenv

</div>

<div class="seealso">

The `os.reload_environ` function.

</div>

<div class="versionchanged">

3.9 Updated to support `584`'s merge (`|`) and update (`|=`) operators.

</div>

<div class="versionchanged">

3.15 The `~dict.clear` method can now emit an `os._clearenv` audit event.

</div>

</div>

<div class="data">

environb

Bytes version of `environ`: a `mapping` object where both keys and values are `bytes` objects representing the process environment. `environ` and `environb` are synchronized (modifying `environb` updates `environ`, and vice versa).

`environb` is only available if `supports_bytes_environ` is `True`.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.9 Updated to support `584`'s merge (`|`) and update (`|=`) operators.

</div>

</div>

<div class="function">

reload_environ()

The `os.environ` and `os.environb` mappings are a cache of environment variables at the time that Python started. As such, changes to the current process environment are not reflected if made outside Python, or by `os.putenv` or `os.unsetenv`. Use `!os.reload_environ` to update `os.environ` and `os.environb` with any such changes to the current process environment.

> [!WARNING]
> This function is not thread-safe. Calling it while the environment is being modified in another thread is an undefined behavior. Reading from `os.environ` or `os.environb`, or calling `os.getenv` while reloading, may return an empty result.

<div class="versionadded">

3.14

</div>

</div>

<div class="function" data-noindex="">

chdir(path) fchdir(fd) getcwd()

These functions are described in `os-file-dir`.

</div>

<div class="function">

fsencode(filename)

Encode `path-like <path-like object>` *filename* to the `filesystem encoding and error handler`; return `bytes` unchanged.

`fsdecode` is the reverse function.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.6 Support added to accept objects implementing the `os.PathLike` interface.

</div>

</div>

<div class="function">

fsdecode(filename)

Decode the `path-like <path-like object>` *filename* from the `filesystem encoding and error handler`; return `str` unchanged.

`fsencode` is the reverse function.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.6 Support added to accept objects implementing the `os.PathLike` interface.

</div>

</div>

<div class="function">

fspath(path)

Return the file system representation of the path.

If `str` or `bytes` is passed in, it is returned unchanged. Otherwise `~os.PathLike.__fspath__` is called and its value is returned as long as it is a `str` or `bytes` object. In all other cases, `TypeError` is raised.

<div class="versionadded">

3.6

</div>

</div>

<div class="PathLike">

An `abstract base class` for objects representing a file system path, e.g. `pathlib.PurePath`.

<div class="versionadded">

3.6

</div>

<div class="method" data-abstractmethod="">

\_\_fspath\_\_()

Return the file system path representation of the object.

The method should only return a `str` or `bytes` object, with the preference being for `str`.

</div>

</div>

<div class="function">

getenv(key, default=None)

Return the value of the environment variable *key* as a string if it exists, or *default* if it doesn't. *key* is a string. Note that since `getenv` uses `os.environ`, the mapping of `getenv` is similarly also captured on import, and the function may not reflect future environment changes.

On Unix, keys and values are decoded with `sys.getfilesystemencoding` and `'surrogateescape'` error handler. Use `os.getenvb` if you would like to use a different encoding.

<div class="availability">

Unix, Windows.

</div>

</div>

<div class="function">

getenvb(key, default=None)

Return the value of the environment variable *key* as bytes if it exists, or *default* if it doesn't. *key* must be bytes. Note that since `getenvb` uses `os.environb`, the mapping of `getenvb` is similarly also captured on import, and the function may not reflect future environment changes.

`getenvb` is only available if `supports_bytes_environ` is `True`.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

get_exec_path(env=None)

Returns the list of directories that will be searched for a named executable, similar to a shell, when launching a process. *env*, when specified, should be an environment variable dictionary to lookup the PATH in. By default, when *env* is `None`, `environ` is used.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

getegid()

Return the effective group id of the current process. This corresponds to the "set id" bit on the file being executed in the current process.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

geteuid()

<div class="index">

single: user; effective id

</div>

Return the current process's effective user id.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

getgid()

<div class="index">

single: process; group

</div>

Return the real group id of the current process.

<div class="availability">

Unix.

The function is a stub on WASI, see `wasm-availability` for more information.

</div>

</div>

<div class="function">

getgrouplist(user, group, /)

Return list of group ids that *user* belongs to. If *group* is not in the list, it is included; typically, *group* is specified as the group ID field from the password record for *user*, because that group ID will otherwise be potentially omitted.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

getgroups()

Return list of supplemental group ids associated with the current process.

<div class="availability">

Unix, not WASI.

</div>

> [!NOTE]
> On macOS, `getgroups` behavior differs somewhat from other Unix platforms. If the Python interpreter was built with a deployment target of `10.5` or earlier, `getgroups` returns the list of effective group ids associated with the current user process; this list is limited to a system-defined number of entries, typically 16, and may be modified by calls to `setgroups` if suitably privileged. If built with a deployment target greater than `10.5`, `getgroups` returns the current group access list for the user associated with the effective user id of the process; the group access list may change over the lifetime of the process, it is not affected by calls to `setgroups`, and its length is not limited to 16. The deployment target value can be obtained with `sysconfig.get_config_var('MACOSX_DEPLOYMENT_TARGET') <sysconfig.get_config_var>`.

</div>

<div class="function">

getlogin()

Return the name of the user logged in on the controlling terminal of the process. For most purposes, it is more useful to use `getpass.getuser` since the latter checks the environment variables `LOGNAME` or `USERNAME` to find out who the user is, and falls back to `pwd.getpwuid(os.getuid()).pw_name` to get the login name of the current real user id.

<div class="availability">

Unix, Windows, not WASI.

</div>

</div>

<div class="function">

getpgid(pid)

Return the process group id of the process with process id *pid*. If *pid* is 0, the process group id of the current process is returned.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

getpgrp()

<div class="index">

single: process; group

</div>

Return the id of the current process group.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

getpid()

<div class="index">

single: process; id

</div>

Return the current process id.

The function is a stub on WASI, see `wasm-availability` for more information.

</div>

<div class="function">

getppid()

<div class="index">

single: process; id of parent

</div>

Return the parent's process id. When the parent process has exited, on Unix the id returned is the one of the init process (1), on Windows it is still the same id, which may be already reused by another process.

<div class="availability">

Unix, Windows, not WASI.

</div>

<div class="versionchanged">

3.2 Added support for Windows.

</div>

</div>

<div class="function">

getpriority(which, who)

<div class="index">

single: process; scheduling priority

</div>

Get program scheduling priority. The value *which* is one of `PRIO_PROCESS`, `PRIO_PGRP`, or `PRIO_USER`, and *who* is interpreted relative to *which* (a process identifier for `PRIO_PROCESS`, process group identifier for `PRIO_PGRP`, and a user ID for `PRIO_USER`). A zero value for *who* denotes (respectively) the calling process, the process group of the calling process, or the real user ID of the calling process.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

PRIO_PROCESS PRIO_PGRP PRIO_USER

Parameters for the `getpriority` and `setpriority` functions.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

PRIO_DARWIN_THREAD PRIO_DARWIN_PROCESS PRIO_DARWIN_BG PRIO_DARWIN_NONUI

Parameters for the `getpriority` and `setpriority` functions.

<div class="availability">

macOS

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

getresuid()

Return a tuple (ruid, euid, suid) denoting the current process's real, effective, and saved user ids.

<div class="availability">

Unix, not WASI, not macOS, not iOS.

</div>

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

getresgid()

Return a tuple (rgid, egid, sgid) denoting the current process's real, effective, and saved group ids.

<div class="availability">

Unix, not WASI, not macOS, not iOS.

</div>

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

getuid()

<div class="index">

single: user; id

</div>

Return the current process's real user id.

<div class="availability">

Unix.

The function is a stub on WASI, see `wasm-availability` for more information.

</div>

</div>

<div class="function">

initgroups(username, gid, /)

Call the system `initgroups()` to initialize the group access list with all of the groups of which the specified username is a member, plus the specified group id.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

putenv(key, value, /)

<div class="index">

single: environment variables; setting

</div>

Set the environment variable named *key* to the string *value*. Such changes to the environment affect subprocesses started with `os.system`, `popen` or `fork` and `execv`.

Assignments to items in `os.environ` are automatically translated into corresponding calls to `putenv`; however, calls to `putenv` don't update `os.environ`, so it is actually preferable to assign to items of `os.environ`. This also applies to `getenv` and `getenvb`, which respectively use `os.environ` and `os.environb` in their implementations.

See also the `os.reload_environ` function.

> [!NOTE]
> On some platforms, including FreeBSD and macOS, setting `environ` may cause memory leaks. Refer to the system documentation for `!putenv`.

<div class="audit-event">

os.putenv key,value os.putenv

</div>

<div class="versionchanged">

3.9 The function is now always available.

</div>

</div>

<div class="function">

setegid(egid, /)

Set the current process's effective group id.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

seteuid(euid, /)

Set the current process's effective user id.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

setgid(gid, /)

Set the current process' group id.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

setgroups(groups, /)

Set the list of supplemental group ids associated with the current process to *groups*. *groups* must be a sequence, and each element must be an integer identifying a group. This operation is typically available only to the superuser.

<div class="availability">

Unix, not WASI.

</div>

> [!NOTE]
> On macOS, the length of *groups* may not exceed the system-defined maximum number of effective group ids, typically 16. See the documentation for `getgroups` for cases where it may not return the same group list set by calling setgroups().

</div>

<div class="function">

setns(fd, nstype=0)

Reassociate the current thread with a Linux namespace. See the `setns(2)` and `namespaces(7)` man pages for more details.

If *fd* refers to a `/proc/{pid}/ns/` link, `setns()` reassociates the calling thread with the namespace associated with that link, and *nstype* may be set to one of the `CLONE_NEW* constants <os-unshare-clone-flags>` to impose constraints on the operation (`0` means no constraints).

Since Linux 5.8, *fd* may refer to a PID file descriptor obtained from `~os.pidfd_open`. In this case, `setns()` reassociates the calling thread into one or more of the same namespaces as the thread referred to by *fd*. This is subject to any constraints imposed by *nstype*, which is a bit mask combining one or more of the `CLONE_NEW* constants <os-unshare-clone-flags>`, e.g. `setns(fd, os.CLONE_NEWUTS | os.CLONE_NEWPID)`. The caller's memberships in unspecified namespaces are left unchanged.

*fd* can be any object with a `~io.IOBase.fileno` method, or a raw file descriptor.

This example reassociates the thread with the `init` process's network namespace:

    fd = os.open("/proc/1/ns/net", os.O_RDONLY)
    os.setns(fd, os.CLONE_NEWNET)
    os.close(fd)

<div class="availability">

Linux \>= 3.0 with glibc \>= 2.14.

</div>

<div class="versionadded">

3.12

</div>

<div class="seealso">

The `~os.unshare` function.

</div>

</div>

<div class="function">

setpgrp()

Call the system call `!setpgrp` or `setpgrp(0, 0)` depending on which version is implemented (if any). See the Unix manual for the semantics.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

setpgid(pid, pgrp, /)

Call the system call `!setpgid` to set the process group id of the process with id *pid* to the process group with id *pgrp*. See the Unix manual for the semantics.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

setpriority(which, who, priority)

<div class="index">

single: process; scheduling priority

</div>

Set program scheduling priority. The value *which* is one of `PRIO_PROCESS`, `PRIO_PGRP`, or `PRIO_USER`, and *who* is interpreted relative to *which* (a process identifier for `PRIO_PROCESS`, process group identifier for `PRIO_PGRP`, and a user ID for `PRIO_USER`). A zero value for *who* denotes (respectively) the calling process, the process group of the calling process, or the real user ID of the calling process. *priority* is a value in the range -20 to 19. The default priority is 0; lower priorities cause more favorable scheduling.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

setregid(rgid, egid, /)

Set the current process's real and effective group ids.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

setresgid(rgid, egid, sgid, /)

Set the current process's real, effective, and saved group ids.

<div class="availability">

Unix, not WASI, not macOS, not iOS.

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

setresuid(ruid, euid, suid, /)

Set the current process's real, effective, and saved user ids.

<div class="availability">

Unix, not WASI, not macOS, not iOS.

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

setreuid(ruid, euid, /)

Set the current process's real and effective user ids.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

getsid(pid, /)

Call the system call `!getsid`. See the Unix manual for the semantics.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

setsid()

Call the system call `!setsid`. See the Unix manual for the semantics.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

setuid(uid, /)

<div class="index">

single: user; id, setting

</div>

Set the current process's user id.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

strerror(code, /)

Return the error message corresponding to the error code in *code*. On platforms where `!strerror` returns `NULL` when given an unknown error number, `ValueError` is raised.

</div>

<div class="data">

supports_bytes_environ

`True` if the native OS type of the environment is bytes (eg. `False` on Windows).

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

umask(mask, /)

Set the current numeric umask and return the previous umask.

The function is a stub on WASI, see `wasm-availability` for more information.

</div>

<div class="function">

uname()

<div class="index">

single: gethostname() (in module socket) single: gethostbyaddr() (in module socket)

</div>

Returns information identifying the current operating system. The return value is a `uname_result`.

On macOS, iOS and Android, this returns the *kernel* name and release (i.e., `'Darwin'` on macOS and iOS; `'Linux'` on Android). `platform.uname` can be used to get the user-facing operating system name and release on iOS and Android.

<div class="seealso">

`sys.platform` which has finer granularity.

The `platform` module provides detailed checks for the system's identity.

</div>

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.3 Return type changed from a tuple to a tuple-like object with named attributes.

</div>

</div>

<div class="uname_result">

Name and information about the system returned by `os.uname`. These attributes correspond to the members described in `uname(2)`.

For backwards compatibility, this object is also iterable, behaving like a five-tuple containing `~uname_result.sysname`, `~uname_result.nodename`, `~uname_result.release`, `~uname_result.version`, and `~uname_result.machine` in that order.

<div class="attribute">

sysname

Operating system name.

</div>

<div class="attribute">

nodename

Name of machine on network. Some systems truncate `~uname_result.nodename` to 8 characters or to the leading component; a better way to get the hostname is `socket.gethostname` or even `socket.gethostbyaddr(socket.gethostname())`.

</div>

<div class="attribute">

release

Operating system release.

</div>

<div class="attribute">

version

Operating system version.

</div>

<div class="attribute">

machine

Hardware identifier.

</div>

</div>

<div class="function">

unsetenv(key, /)

<div class="index">

single: environment variables; deleting

</div>

Unset (delete) the environment variable named *key*. Such changes to the environment affect subprocesses started with `os.system`, `popen` or `fork` and `execv`.

Deletion of items in `os.environ` is automatically translated into a corresponding call to `unsetenv`; however, calls to `unsetenv` don't update `os.environ`, so it is actually preferable to delete items of `os.environ`.

See also the `os.reload_environ` function.

<div class="audit-event">

os.unsetenv key os.unsetenv

</div>

<div class="versionchanged">

3.9 The function is now always available and is also available on Windows.

</div>

</div>

<div class="function">

unshare(flags)

Disassociate parts of the process execution context, and move them into a newly created namespace. See the `unshare(2)` man page for more details. The *flags* argument is a bit mask, combining zero or more of the `CLONE_* constants <os-unshare-clone-flags>`, that specifies which parts of the execution context should be unshared from their existing associations and moved to a new namespace. If the *flags* argument is `0`, no changes are made to the calling process's execution context.

<div class="availability">

Linux \>= 2.6.16.

</div>

<div class="versionadded">

3.12

</div>

<div class="seealso">

The `~os.setns` function.

</div>

</div>

<div id="os-unshare-clone-flags">

Flags to the `unshare` function, if the implementation supports them. See `unshare(2)` in the Linux manual for their exact effect and availability.

</div>

<div class="data">

CLONE_FILES CLONE_FS CLONE_NEWCGROUP CLONE_NEWIPC CLONE_NEWNET CLONE_NEWNS CLONE_NEWPID CLONE_NEWTIME CLONE_NEWUSER CLONE_NEWUTS CLONE_SIGHAND CLONE_SYSVSEM CLONE_THREAD CLONE_VM

</div>

## File Object Creation

These functions create new `file objects <file object>`. (See also `~os.open` for opening file descriptors.)

<div class="function">

fdopen(fd, *args,*\*kwargs)

Return an open file object connected to the file descriptor *fd*. This is an alias of the `open` built-in function and accepts the same arguments. The only difference is that the first argument of `fdopen` must always be an integer.

</div>

## File Descriptor Operations

These functions operate on I/O streams referenced using file descriptors.

File descriptors are small integers corresponding to a file that has been opened by the current process. For example, standard input is usually file descriptor 0, standard output is 1, and standard error is 2. Further files opened by a process will then be assigned 3, 4, 5, and so forth. The name "file descriptor" is slightly deceptive; on Unix platforms, sockets and pipes are also referenced by file descriptors.

The `~io.IOBase.fileno` method can be used to obtain the file descriptor associated with a `file object` when required. Note that using the file descriptor directly will bypass the file object methods, ignoring aspects such as internal buffering of data.

<div class="function">

close(fd)

Close file descriptor *fd*.

> [!NOTE]
> This function is intended for low-level I/O and must be applied to a file descriptor as returned by `os.open` or `pipe`. To close a "file object" returned by the built-in function `open` or by `popen` or `fdopen`, use its `~io.IOBase.close` method.

</div>

<div class="function">

closerange(fd_low, fd_high, /)

Close all file descriptors from *fd_low* (inclusive) to *fd_high* (exclusive), ignoring errors. Equivalent to (but much faster than):

    for fd in range(fd_low, fd_high):
        try:
            os.close(fd)
        except OSError:
            pass

</div>

<div class="function">

copy_file_range(src, dst, count, offset_src=None, offset_dst=None)

Copy *count* bytes from file descriptor *src*, starting from offset *offset_src*, to file descriptor *dst*, starting from offset *offset_dst*. If *offset_src* is `None`, then *src* is read from the current position; respectively for *offset_dst*.

In Linux kernel older than 5.3, the files pointed to by *src* and *dst* must reside in the same filesystem, otherwise an `OSError` is raised with `~OSError.errno` set to `errno.EXDEV`.

This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, some filesystems could implement extra optimizations, such as the use of reflinks (i.e., two or more inodes that share pointers to the same copy-on-write disk blocks; supported file systems include btrfs and XFS) and server-side copy (in the case of NFS).

The function copies bytes between two file descriptors. Text options, like the encoding and the line ending, are ignored.

The return value is the amount of bytes copied. This could be less than the amount requested.

> [!NOTE]
> On Linux, `os.copy_file_range` should not be used for copying a range of a pseudo file from a special filesystem like procfs and sysfs. It will always copy no bytes and return 0 as if the file was empty because of a known Linux kernel issue.

<div class="availability">

Linux \>= 4.5.

</div>

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.16 The function is now also available when Python is built against a libc that lacks `copy_file_range()`, such as glibc older than 2.27.

</div>

</div>

<div class="function">

device_encoding(fd)

Return a string describing the encoding of the device associated with *fd* if it is connected to a terminal; else return `None`.

On Unix, if the `Python UTF-8 Mode <utf8-mode>` is enabled, return `'UTF-8'` rather than the device encoding.

<div class="versionchanged">

3.10 On Unix, the function now implements the Python UTF-8 Mode.

</div>

</div>

<div class="function">

dup(fd, /)

Return a duplicate of file descriptor *fd*. The new file descriptor is `non-inheritable <fd_inheritance>`.

On Windows, when duplicating a standard stream (0: stdin, 1: stdout, 2: stderr), the new file descriptor is `inheritable
<fd_inheritance>`.

<div class="availability">

not WASI.

</div>

<div class="versionchanged">

3.4 The new file descriptor is now non-inheritable.

</div>

</div>

<div class="function">

dup2(fd, fd2, inheritable=True)

Duplicate file descriptor *fd* to *fd2*, closing the latter first if necessary. Return *fd2*. The new file descriptor is `inheritable
<fd_inheritance>` by default or non-inheritable if *inheritable* is `False`.

<div class="availability">

not WASI.

</div>

<div class="versionchanged">

3.4 Add the optional *inheritable* parameter.

</div>

<div class="versionchanged">

3.7 Return *fd2* on success. Previously, `None` was always returned.

</div>

</div>

<div class="function">

fchmod(fd, mode)

Change the mode of the file given by *fd* to the numeric *mode*. See the docs for `chmod` for possible values of *mode*. As of Python 3.3, this is equivalent to `os.chmod(fd, mode)`.

<div class="audit-event">

os.chmod path,mode,dir_fd os.fchmod

</div>

<div class="availability">

Unix, Windows.

The function is limited on WASI, see `wasm-availability` for more information.

</div>

<div class="versionchanged">

3.13 Added support on Windows.

</div>

</div>

<div class="function">

fchown(fd, uid, gid)

Change the owner and group id of the file given by *fd* to the numeric *uid* and *gid*. To leave one of the ids unchanged, set it to -1. See `chown`. As of Python 3.3, this is equivalent to `os.chown(fd, uid, gid)`.

<div class="audit-event">

os.chown path,uid,gid,dir_fd os.fchown

</div>

<div class="availability">

Unix.

The function is limited on WASI, see `wasm-availability` for more information.

</div>

</div>

<div class="function">

fdatasync(fd)

Force write of file with filedescriptor *fd* to disk. Does not force update of metadata.

<div class="availability">

Unix, not macOS, not iOS.

</div>

</div>

<div class="function">

fpathconf(fd, name, /)

Return system configuration information relevant to an open file. *name* specifies the configuration value to retrieve; it may be a string which is the name of a defined system value; these names are specified in a number of standards (POSIX.1, Unix 95, Unix 98, and others). Some platforms define additional names as well. The names known to the host operating system are given in the `pathconf_names` dictionary. For configuration variables not included in that mapping, passing an integer for *name* is also accepted.

If *name* is a string and is not known, `ValueError` is raised. If a specific value for *name* is not supported by the host system, even if it is included in `pathconf_names`, an `OSError` is raised with `errno.EINVAL` for the error number.

As of Python 3.3, this is equivalent to `os.pathconf(fd, name)`.

<div class="availability">

Unix.

</div>

</div>

<div class="function">

fstat(fd)

Get the status of the file descriptor *fd*. Return a `stat_result` object.

As of Python 3.3, this is equivalent to `os.stat(fd)`.

<div class="seealso">

The `.stat` function.

</div>

</div>

<div class="function">

fstatvfs(fd, /)

Return information about the filesystem containing the file associated with file descriptor *fd* in a `statvfs_result`, like `statvfs`. As of Python 3.3, this is equivalent to `os.statvfs(fd)`.

<div class="availability">

Unix.

</div>

</div>

<div class="function">

fsync(fd)

Force write of file with filedescriptor *fd* to disk. On Unix, this calls the native `!fsync` function; on Windows, the MS `!_commit` function.

If you're starting with a buffered Python `file object` *f*, first do `f.flush()`, and then do `os.fsync(f.fileno())`, to ensure that all internal buffers associated with *f* are written to disk.

<div class="availability">

Unix, Windows.

</div>

</div>

<div class="function">

ftruncate(fd, length, /)

Truncate the file corresponding to file descriptor *fd*, so that it is at most *length* bytes in size. As of Python 3.3, this is equivalent to `os.truncate(fd, length)`.

<div class="audit-event">

os.truncate fd,length os.ftruncate

</div>

<div class="availability">

Unix, Windows.

</div>

<div class="versionchanged">

3.5 Added support for Windows

</div>

</div>

<div class="function">

get_blocking(fd, /)

Get the blocking mode of the file descriptor: `False` if the `O_NONBLOCK` flag is set, `True` if the flag is cleared.

See also `set_blocking` and `socket.socket.setblocking`.

<div class="availability">

Unix, Windows.

The function is limited on WASI, see `wasm-availability` for more information.

On Windows, this function is limited to pipes.

</div>

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.12 Added support for pipes on Windows.

</div>

</div>

<div class="function">

grantpt(fd, /)

Grant access to the slave pseudo-terminal device associated with the master pseudo-terminal device to which the file descriptor *fd* refers. The file descriptor *fd* is not closed upon failure.

Calls the C standard library function `grantpt`.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

isatty(fd, /)

Return `True` if the file descriptor *fd* is open and connected to a tty(-like) device, else `False`.

</div>

<div class="function">

lockf(fd, cmd, len, /)

Apply, test or remove a POSIX lock on an open file descriptor. *fd* is an open file descriptor. *cmd* specifies the command to use - one of `F_LOCK`, `F_TLOCK`, `F_ULOCK` or `F_TEST`. *len* specifies the section of the file to lock.

<div class="audit-event">

os.lockf fd,cmd,len os.lockf

</div>

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

F_LOCK F_TLOCK F_ULOCK F_TEST

Flags that specify what action `lockf` will take.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

login_tty(fd, /)

Prepare the tty of which fd is a file descriptor for a new login session. Make the calling process a session leader; make the tty the controlling tty, the stdin, the stdout, and the stderr of the calling process; close fd.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

lseek(fd, pos, whence, /)

Set the current position of file descriptor *fd* to position *pos*, modified by *whence*, and return the new position in bytes relative to the start of the file. Valid values for *whence* are:

- `SEEK_SET` or `0` -- set *pos* relative to the beginning of the file
- `SEEK_CUR` or `1` -- set *pos* relative to the current file position
- `SEEK_END` or `2` -- set *pos* relative to the end of the file
- `SEEK_DATA` -- set *pos* to the next data location, relative to *pos*
- `SEEK_HOLE` -- set *pos* to the next data hole, relative to *pos*

<div class="versionchanged">

3.3

Add support for `!SEEK_HOLE` and `!SEEK_DATA`.

</div>

</div>

<div class="data">

SEEK_SET SEEK_CUR SEEK_END

Parameters to the `lseek` function and the `~io.IOBase.seek` method on `file-like objects <file object>`, for whence to adjust the file position indicator.

`SEEK_SET`  
Adjust the file position relative to the beginning of the file.

`SEEK_CUR`  
Adjust the file position relative to the current file position.

`SEEK_END`  
Adjust the file position relative to the end of the file.

Their values are 0, 1, and 2, respectively.

</div>

<div class="data">

SEEK_HOLE SEEK_DATA

Parameters to the `lseek` function and the `~io.IOBase.seek` method on `file-like objects <file object>`, for seeking file data and holes on sparsely allocated files.

`!SEEK_DATA`  
Adjust the file offset to the next location containing data, relative to the seek position.

`!SEEK_HOLE`  
Adjust the file offset to the next location containing a hole, relative to the seek position. A hole is defined as a sequence of zeros.

> [!NOTE]
> These operations only make sense for filesystems that support them.

<div class="availability">

Linux \>= 3.1, macOS, Unix

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

open(path, flags, mode=0o777, \*, dir_fd=None)

Open the file *path* and set various flags according to *flags* and possibly its mode according to *mode*. When computing *mode*, the current umask value is first masked out. Return the file descriptor for the newly opened file. The new file descriptor is `non-inheritable <fd_inheritance>`.

For a description of the flag and mode values, see the C run-time documentation; flag constants (like `O_RDONLY` and `O_WRONLY`) are defined in the `!os` module. In particular, on Windows adding `O_BINARY` is needed to open files in binary mode.

This function can support `paths relative to directory descriptors
<dir_fd>` with the *dir_fd* parameter.

<div class="audit-event">

open path,mode,flags os.open

</div>

<div class="versionchanged">

3.4 The new file descriptor is now non-inheritable.

</div>

> [!NOTE]
> This function is intended for low-level I/O. For normal usage, use the built-in function `open`, which returns a `file object` with `~io.BufferedIOBase.read` and `~io.BufferedIOBase.write` methods. To wrap a file descriptor in a file object, use `fdopen`.

<div class="versionchanged">

3.3 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the function now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

The following constants are options for the *flags* parameter to the `~os.open` function. They can be combined using the bitwise OR operator `|`. Some of them are not available on all platforms. For descriptions of their availability and use, consult the `open(2)` manual page on Unix or [the MSDN](https://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx) on Windows.

<div class="data">

O_RDONLY O_WRONLY O_RDWR O_APPEND O_CREAT O_EXCL O_TRUNC

The above constants are available on Unix and Windows.

</div>

<div class="data">

O_DSYNC O_RSYNC O_SYNC O_NDELAY O_NONBLOCK O_NOCTTY O_CLOEXEC

The above constants are only available on Unix.

<div class="versionchanged">

3.3 Add `O_CLOEXEC` constant.

</div>

</div>

<div class="data">

O_BINARY O_NOINHERIT O_SHORT_LIVED O_TEMPORARY O_RANDOM O_SEQUENTIAL O_TEXT

The above constants are only available on Windows.

</div>

<div class="data">

O_EVTONLY O_FSYNC O_SYMLINK O_NOFOLLOW_ANY

The above constants are only available on macOS.

<div class="versionchanged">

3.10 Add `O_EVTONLY`, `O_FSYNC`, `O_SYMLINK` and `O_NOFOLLOW_ANY` constants.

</div>

</div>

<div class="data">

O_ASYNC O_DIRECT O_DIRECTORY O_NOFOLLOW O_NOATIME O_PATH O_TMPFILE O_SHLOCK O_EXLOCK

The above constants are extensions and not present if they are not defined by the C library.

<div class="versionchanged">

3.4 Add `O_PATH` on systems that support it. Add `O_TMPFILE`, only available on Linux Kernel 3.11 or newer.

</div>

</div>

<div class="function">

openpty()

<div class="index">

pair: module; pty

</div>

Open a new pseudo-terminal pair. Return a pair of file descriptors `(master, slave)` for the pty and the tty, respectively. The new file descriptors are `non-inheritable <fd_inheritance>`. For a (slightly) more portable approach, use the `pty` module.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.4 The new file descriptors are now non-inheritable.

</div>

</div>

<div class="function">

pipe()

Create a pipe. Return a pair of file descriptors `(r, w)` usable for reading and writing, respectively. The new file descriptor is `non-inheritable <fd_inheritance>`.

<div class="availability">

Unix, Windows.

</div>

<div class="versionchanged">

3.4 The new file descriptors are now non-inheritable.

</div>

</div>

<div class="function">

pipe2(flags, /)

Create a pipe with *flags* set atomically. *flags* can be constructed by ORing together one or more of these values: `O_NONBLOCK`, `O_CLOEXEC`. Return a pair of file descriptors `(r, w)` usable for reading and writing, respectively.

<div class="availability">

Unix, macOS \>= 27.0, iOS \>= 27.0, not WASI.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

posix_fallocate(fd, offset, len, /)

Ensures that enough disk space is allocated for the file specified by *fd* starting from *offset* and continuing for *len* bytes.

<div class="availability">

Unix, not macOS, not iOS.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

posix_fadvise(fd, offset, len, advice, /)

Announces an intention to access data in a specific pattern thus allowing the kernel to make optimizations. The advice applies to the region of the file specified by *fd* starting at *offset* and continuing for *len* bytes. *advice* is one of `POSIX_FADV_NORMAL`, `POSIX_FADV_SEQUENTIAL`, `POSIX_FADV_RANDOM`, `POSIX_FADV_NOREUSE`, `POSIX_FADV_WILLNEED` or `POSIX_FADV_DONTNEED`.

<div class="availability">

Unix, not macOS, not iOS.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

POSIX_FADV_NORMAL POSIX_FADV_SEQUENTIAL POSIX_FADV_RANDOM POSIX_FADV_NOREUSE POSIX_FADV_WILLNEED POSIX_FADV_DONTNEED

Flags that can be used in *advice* in `posix_fadvise` that specify the access pattern that is likely to be used.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

pread(fd, n, offset, /)

Read at most *n* bytes from file descriptor *fd* at a position of *offset*, leaving the file offset unchanged.

Return a bytestring containing the bytes read. If the end of the file referred to by *fd* has been reached, an empty bytes object is returned.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

posix_openpt(oflag, /)

Open and return a file descriptor for a master pseudo-terminal device.

Calls the C standard library function `posix_openpt`. The *oflag* argument is used to set file status flags and file access modes as specified in the manual page of `posix_openpt` of your system.

The returned file descriptor is `non-inheritable <fd_inheritance>`. If the value `O_CLOEXEC` is available on the system, it is added to *oflag*.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

preadv(fd, buffers, offset, flags=0, /)

Read from a file descriptor *fd* at a position of *offset* into mutable `bytes-like objects <bytes-like object>` *buffers*, leaving the file offset unchanged. Transfer data into each buffer until it is full and then move on to the next buffer in the sequence to hold the rest of the data.

The flags argument contains a bitwise OR of zero or more of the following flags:

- `RWF_HIPRI`
- `RWF_NOWAIT`
- `RWF_DONTCACHE`

Return the total number of bytes actually read which can be less than the total capacity of all the objects.

The operating system may set a limit (`sysconf` value `'SC_IOV_MAX'`) on the number of buffers that can be used.

Combine the functionality of `os.readv` and `os.pread`.

<div class="availability">

Linux \>= 2.6.30, FreeBSD \>= 6.0, OpenBSD \>= 2.7, AIX \>= 7.1.

Using flags requires Linux \>= 4.6.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

RWF_NOWAIT

Do not wait for data which is not immediately available. If this flag is specified, the system call will return instantly if it would have to read data from the backing storage or wait for a lock.

If some data was successfully read, it will return the number of bytes read. If no bytes were read, it will return `-1` and set errno to `errno.EAGAIN`.

<div class="availability">

Linux \>= 4.14.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

RWF_HIPRI

High priority read/write. Allows block-based filesystems to use polling of the device, which provides lower latency, but may use additional resources.

Currently, on Linux, this feature is usable only on a file descriptor opened using the `O_DIRECT` flag.

<div class="availability">

Linux \>= 4.6.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

RWF_DONTCACHE

Use uncached buffered IO.

<div class="availability">

Linux \>= 6.14

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

RWF_ATOMIC

Write data atomically. Requires alignment to the device's atomic write unit.

<div class="availability">

Linux \>= 6.11

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

ptsname(fd, /)

Return the name of the slave pseudo-terminal device associated with the master pseudo-terminal device to which the file descriptor *fd* refers. The file descriptor *fd* is not closed upon failure.

Calls the reentrant C standard library function `ptsname_r` if it is available; otherwise, the C standard library function `ptsname`, which is not guaranteed to be thread-safe, is called.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

pwrite(fd, str, offset, /)

Write the bytestring in *str* to file descriptor *fd* at position of *offset*, leaving the file offset unchanged.

Return the number of bytes actually written.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

pwritev(fd, buffers, offset, flags=0, /)

Write the *buffers* contents to file descriptor *fd* at an offset *offset*, leaving the file offset unchanged. *buffers* must be a sequence of `bytes-like objects <bytes-like object>`. Buffers are processed in array order. Entire contents of the first buffer is written before proceeding to the second, and so on.

The flags argument contains a bitwise OR of zero or more of the following flags:

- `RWF_DSYNC`
- `RWF_SYNC`
- `RWF_APPEND`
- `RWF_DONTCACHE`
- `RWF_ATOMIC`
- `RWF_NOSIGNAL`

Return the total number of bytes actually written.

The operating system may set a limit (`sysconf` value `'SC_IOV_MAX'`) on the number of buffers that can be used.

Combine the functionality of `os.writev` and `os.pwrite`.

<div class="availability">

Linux \>= 2.6.30, FreeBSD \>= 6.0, OpenBSD \>= 2.7, AIX \>= 7.1.

Using flags requires Linux \>= 4.6.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

RWF_DSYNC

Provide a per-write equivalent of the `O_DSYNC` `os.open` flag. This flag effect applies only to the data range written by the system call.

<div class="availability">

Linux \>= 4.7.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

RWF_SYNC

Provide a per-write equivalent of the `O_SYNC` `os.open` flag. This flag effect applies only to the data range written by the system call.

<div class="availability">

Linux \>= 4.7.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

RWF_APPEND

Provide a per-write equivalent of the `O_APPEND` `os.open` flag. This flag is meaningful only for `os.pwritev`, and its effect applies only to the data range written by the system call. The *offset* argument does not affect the write operation; the data is always appended to the end of the file. However, if the *offset* argument is `-1`, the current file *offset* is updated.

<div class="availability">

Linux \>= 4.16.

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

RWF_NOSIGNAL

Prevent pipe and socket writes from raising `~signal.SIGPIPE`. This flag is meaningful only for `os.pwritev`.

<div class="availability">

Linux \>= 6.18.

</div>

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

read(fd, n, /)

Read at most *n* bytes from file descriptor *fd*.

Return a bytestring containing the bytes read. If the end of the file referred to by *fd* has been reached, an empty bytes object is returned.

> [!NOTE]
> This function is intended for low-level I/O and must be applied to a file descriptor as returned by `os.open` or `pipe`. To read a "file object" returned by the built-in function `open` or by `popen` or `fdopen`, or `sys.stdin`, use its `~io.TextIOBase.read` or `~io.IOBase.readline` methods.

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the function now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="function">

readinto(fd, buffer, /)

Read from a file descriptor *fd* into a mutable `buffer object <bufferobjects>` *buffer*.

The *buffer* should be mutable and `bytes-like <bytes-like object>`. On success, returns the number of bytes read. Less bytes may be read than the size of the buffer. The underlying system call will be retried when interrupted by a signal, unless the signal handler raises an exception. Other errors will not be retried and an error will be raised.

Returns 0 if *fd* is at end of file or if the provided *buffer* has length 0 (which can be used to check for errors without reading data). Never returns negative.

> [!NOTE]
> This function is intended for low-level I/O and must be applied to a file descriptor as returned by `os.open` or `os.pipe`. To read a "file object" returned by the built-in function `open`, or `sys.stdin`, use its member functions, for example `io.BufferedIOBase.readinto`, `io.BufferedIOBase.read`, or `io.TextIOBase.read`

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

sendfile(out_fd, in_fd, offset, count) sendfile(out_fd, in_fd, offset, count, headers=(), trailers=(), flags=0)

Copy *count* bytes from file descriptor *in_fd* to file descriptor *out_fd* starting at *offset*. Return the number of bytes sent. When EOF is reached return `0`.

The first function notation is supported by all platforms that define `sendfile`.

On Linux, if *offset* is given as `None`, the bytes are read from the current position of *in_fd* and the position of *in_fd* is updated.

The second case may be used on macOS and FreeBSD where *headers* and *trailers* are arbitrary sequences of buffers that are written before and after the data from *in_fd* is written. It returns the same as the first case.

On macOS and FreeBSD, a value of `0` for *count* specifies to send until the end of *in_fd* is reached.

All platforms support sockets as *out_fd* file descriptor, and some platforms allow other types (e.g. regular file, pipe) as well.

Cross-platform applications should not use *headers*, *trailers* and *flags* arguments.

<div class="availability">

Unix, not WASI.

</div>

> [!NOTE]
> For a higher-level wrapper of `sendfile`, see `socket.socket.sendfile`.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.9 Parameters *out* and *in* was renamed to *out_fd* and *in_fd*.

</div>

</div>

<div class="data">

SF_NODISKIO SF_MNOWAIT SF_SYNC

Parameters to the `sendfile` function, if the implementation supports them.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

SF_NOCACHE

Parameter to the `sendfile` function, if the implementation supports it. The data won't be cached in the virtual memory and will be freed afterwards.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

set_blocking(fd, blocking, /)

Set the blocking mode of the specified file descriptor. Set the `O_NONBLOCK` flag if blocking is `False`, clear the flag otherwise.

See also `get_blocking` and `socket.socket.setblocking`.

<div class="availability">

Unix, Windows.

The function is limited on WASI, see `wasm-availability` for more information.

On Windows, this function is limited to pipes.

</div>

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.12 Added support for pipes on Windows.

</div>

</div>

<div class="function">

splice(src, dst, count, offset_src=None, offset_dst=None, flags=0)

Transfer *count* bytes from file descriptor *src*, starting from offset *offset_src*, to file descriptor *dst*, starting from offset *offset_dst*.

The splicing behaviour can be modified by specifying a *flags* value. Any of the following variables may used, combined using bitwise OR (the `|` operator):

- If `SPLICE_F_MOVE` is specified, the kernel is asked to move pages instead of copying, but pages may still be copied if the kernel cannot move the pages from the pipe.
- If `SPLICE_F_NONBLOCK` is specified, the kernel is asked to not block on I/O. This makes the splice pipe operations nonblocking, but splice may nevertheless block because the spliced file descriptors may block.
- If `SPLICE_F_MORE` is specified, it hints to the kernel that more data will be coming in a subsequent splice.

At least one of the file descriptors must refer to a pipe. If *offset_src* is `None`, then *src* is read from the current position; respectively for *offset_dst*. The offset associated to the file descriptor that refers to a pipe must be `None`. The files pointed to by *src* and *dst* must reside in the same filesystem, otherwise an `OSError` is raised with `~OSError.errno` set to `errno.EXDEV`.

This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, some filesystems could implement extra optimizations. The copy is done as if both files are opened as binary.

Upon successful completion, returns the number of bytes spliced to or from the pipe. A return value of 0 means end of input. If *src* refers to a pipe, then this means that there was no data to transfer, and it would not make sense to block because there are no writers connected to the write end of the pipe.

<div class="seealso">

The `splice(2)` man page.

</div>

<div class="availability">

Linux \>= 2.6.17 with glibc \>= 2.5

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

SPLICE_F_MOVE SPLICE_F_NONBLOCK SPLICE_F_MORE

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

readv(fd, buffers, /)

Read from a file descriptor *fd* into a number of mutable `bytes-like
objects <bytes-like object>` *buffers*. Transfer data into each buffer until it is full and then move on to the next buffer in the sequence to hold the rest of the data.

Return the total number of bytes actually read which can be less than the total capacity of all the objects.

The operating system may set a limit (`sysconf` value `'SC_IOV_MAX'`) on the number of buffers that can be used.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

tcgetpgrp(fd, /)

Return the process group associated with the terminal given by *fd* (an open file descriptor as returned by `os.open`).

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

tcsetpgrp(fd, pg, /)

Set the process group associated with the terminal given by *fd* (an open file descriptor as returned by `os.open`) to *pg*.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

ttyname(fd, /)

Return a string which specifies the terminal device associated with file descriptor *fd*. If *fd* is not associated with a terminal device, an exception is raised.

<div class="availability">

Unix.

</div>

</div>

<div class="function">

unlockpt(fd, /)

Unlock the slave pseudo-terminal device associated with the master pseudo-terminal device to which the file descriptor *fd* refers. The file descriptor *fd* is not closed upon failure.

Calls the C standard library function `unlockpt`.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

write(fd, str, /)

Write the bytestring in *str* to file descriptor *fd*.

Return the number of bytes actually written.

> [!NOTE]
> This function is intended for low-level I/O and must be applied to a file descriptor as returned by `os.open` or `pipe`. To write a "file object" returned by the built-in function `open` or by `popen` or `fdopen`, or `sys.stdout` or `sys.stderr`, use its `~io.TextIOBase.write` method.

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the function now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="function">

writev(fd, buffers, /)

Write the contents of *buffers* to file descriptor *fd*. *buffers* must be a sequence of `bytes-like objects <bytes-like object>`. Buffers are processed in array order. Entire contents of the first buffer is written before proceeding to the second, and so on.

Returns the total number of bytes actually written.

The operating system may set a limit (`sysconf` value `'SC_IOV_MAX'`) on the number of buffers that can be used.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

### Querying the size of a terminal

<div class="versionadded">

3.3

</div>

<div class="function">

get_terminal_size(fd=STDOUT_FILENO, /)

Return the size of the terminal window as `(columns, lines)`, tuple of type `terminal_size`.

The optional argument `fd` (default `STDOUT_FILENO`, or standard output) specifies which file descriptor should be queried.

If the file descriptor is not connected to a terminal, an `OSError` is raised.

`shutil.get_terminal_size` is the high-level function which should normally be used, `os.get_terminal_size` is the low-level implementation.

<div class="availability">

Unix, Windows.

</div>

</div>

<div class="terminal_size">

A subclass of tuple, holding `(columns, lines)` of the terminal window size.

<div class="attribute">

columns

Width of the terminal window in characters.

</div>

<div class="attribute">

lines

Height of the terminal window in characters.

</div>

</div>

### Inheritance of File Descriptors

<div class="versionadded">

3.4

</div>

A file descriptor has an "inheritable" flag which indicates if the file descriptor can be inherited by child processes. Since Python 3.4, file descriptors created by Python are non-inheritable by default.

On UNIX, non-inheritable file descriptors are closed in child processes at the execution of a new program, other file descriptors are inherited. Note that non-inheritable file descriptors are still *inherited* by child processes on `os.fork`.

On Windows, non-inheritable handles and file descriptors are closed in child processes, except for standard streams (file descriptors 0, 1 and 2: stdin, stdout and stderr), which are always inherited. Using `spawn\* <spawnl>` functions, all inheritable handles and all inheritable file descriptors are inherited. Using the `subprocess` module, all file descriptors except standard streams are closed, and inheritable handles are only inherited if the *close_fds* parameter is `False`.

On WebAssembly platforms, the file descriptor cannot be modified.

<div class="function">

get_inheritable(fd, /)

Get the "inheritable" flag of the specified file descriptor (a boolean).

</div>

<div class="function">

set_inheritable(fd, inheritable, /)

Set the "inheritable" flag of the specified file descriptor.

</div>

<div class="function">

get_handle_inheritable(handle, /)

Get the "inheritable" flag of the specified handle (a boolean).

<div class="availability">

Windows.

</div>

</div>

<div class="function">

set_handle_inheritable(handle, inheritable, /)

Set the "inheritable" flag of the specified handle.

<div class="availability">

Windows.

</div>

</div>

## Files and Directories

On some Unix platforms, many of these functions support one or more of these features:

<div id="path_fd">

- **specifying a file descriptor:** Normally the *path* argument provided to functions in the `!os` module must be a string specifying a file path. However, some functions now alternatively accept an open file descriptor for their *path* argument. The function will then operate on the file referred to by the descriptor. For POSIX systems, Python will call the variant of the function prefixed with `f` (e.g. call `fchdir` instead of `chdir`).

  You can check whether or not *path* can be specified as a file descriptor for a particular function on your platform using `os.supports_fd`. If this functionality is unavailable, using it will raise a `NotImplementedError`.

  If the function also supports *dir_fd* or *follow_symlinks* arguments, it's an error to specify one of those when supplying *path* as a file descriptor.

</div>

<div id="dir_fd">

- **paths relative to directory descriptors:** If *dir_fd* is not `None`, it should be a file descriptor referring to a directory, and the path to operate on should be relative; path will then be relative to that directory. If the path is absolute, *dir_fd* is ignored. For POSIX systems, Python will call the variant of the function with an `at` suffix and possibly prefixed with `f` (e.g. call `faccessat` instead of `access`).

  You can check whether or not *dir_fd* is supported for a particular function on your platform using `os.supports_dir_fd`. If it's unavailable, using it will raise a `NotImplementedError`.

</div>

<div id="follow_symlinks">

- **not following symlinks:** If *follow_symlinks* is `False`, and the last element of the path to operate on is a symbolic link, the function will operate on the symbolic link itself rather than the file pointed to by the link. For POSIX systems, Python will call the `l...` variant of the function.

  You can check whether or not *follow_symlinks* is supported for a particular function on your platform using `os.supports_follow_symlinks`. If it's unavailable, using it will raise a `NotImplementedError`.

</div>

<div class="function">

access(path, mode, \*, dir_fd=None, effective_ids=False, follow_symlinks=True)

Use the real uid/gid to test for access to *path*. Note that most operations will use the effective uid/gid, therefore this routine can be used in a suid/sgid environment to test if the invoking user has the specified access to *path*. *mode* should be `F_OK` to test the existence of *path*, or it can be the inclusive OR of one or more of `R_OK`, `W_OK`, and `X_OK` to test permissions. Return `True` if access is allowed, `False` if not. See the Unix man page `access(2)` for more information.

This function can support specifying `paths relative to directory
descriptors <dir_fd>` and `not following symlinks <follow_symlinks>`.

If *effective_ids* is `True`, `access` will perform its access checks using the effective uid/gid instead of the real uid/gid. *effective_ids* may not be supported on your platform; you can check whether or not it is available using `os.supports_effective_ids`. If it is unavailable, using it will raise a `NotImplementedError`.

> [!NOTE]
> Using `access` to check if a user is authorized to e.g. open a file before actually doing so using `open` creates a security hole, because the user might exploit the short time interval between checking and opening the file to manipulate it. It's preferable to use `EAFP` techniques. For example:
>
>     if os.access("myfile", os.R_OK):
>         with open("myfile") as fp:
>             return fp.read()
>     return "some default data"
>
> is better written as:
>
>     try:
>         fp = open("myfile")
>     except PermissionError:
>         return "some default data"
>     else:
>         with fp:
>             return fp.read()

> [!NOTE]
> I/O operations may fail even when `access` indicates that they would succeed, particularly for operations on network filesystems which may have permissions semantics beyond the usual POSIX permission-bit model.

<div class="versionchanged">

3.3 Added the *dir_fd*, *effective_ids*, and *follow_symlinks* parameters.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="data">

F_OK R_OK W_OK X_OK

Values to pass as the *mode* parameter of `access` to test the existence, readability, writability and executability of *path*, respectively.

</div>

<div class="function">

chdir(path)

<div class="index">

single: directory; changing

</div>

Change the current working directory to *path*.

This function can support `specifying a file descriptor <path_fd>`. The descriptor must refer to an opened directory, not an open file.

This function can raise `OSError` and subclasses such as `FileNotFoundError`, `PermissionError`, and `NotADirectoryError`.

<div class="audit-event">

os.chdir path os.chdir

</div>

<div class="seealso">

The `contextlib.chdir` context manager, which changes the current working directory on entering and restores the previous one on exit.

</div>

<div class="versionchanged">

3.3 Added support for specifying *path* as a file descriptor on some platforms.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

chflags(path, flags, \*, follow_symlinks=True)

Set the flags of *path* to the numeric *flags*. *flags* may take a combination (bitwise OR) of the following values (as defined in the `stat` module):

- `stat.UF_NODUMP`
- `stat.UF_IMMUTABLE`
- `stat.UF_APPEND`
- `stat.UF_OPAQUE`
- `stat.UF_NOUNLINK`
- `stat.UF_COMPRESSED`
- `stat.UF_HIDDEN`
- `stat.SF_ARCHIVED`
- `stat.SF_IMMUTABLE`
- `stat.SF_APPEND`
- `stat.SF_NOUNLINK`
- `stat.SF_SNAPSHOT`

This function can support `not following symlinks <follow_symlinks>`.

<div class="audit-event">

os.chflags path,flags os.chflags

</div>

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.3 Added the *follow_symlinks* parameter.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

chmod(path, mode, \*, dir_fd=None, follow_symlinks=True)

Change the mode of *path* to the numeric *mode*. *mode* may take one of the following values (as defined in the `stat` module) or bitwise ORed combinations of them:

- `stat.S_ISUID`
- `stat.S_ISGID`
- `stat.S_ENFMT`
- `stat.S_ISVTX`
- `stat.S_IREAD`
- `stat.S_IWRITE`
- `stat.S_IEXEC`
- `stat.S_IRWXU`
- `stat.S_IRUSR`
- `stat.S_IWUSR`
- `stat.S_IXUSR`
- `stat.S_IRWXG`
- `stat.S_IRGRP`
- `stat.S_IWGRP`
- `stat.S_IXGRP`
- `stat.S_IRWXO`
- `stat.S_IROTH`
- `stat.S_IWOTH`
- `stat.S_IXOTH`

This function can support `specifying a file descriptor <path_fd>`, `paths relative to directory descriptors <dir_fd>` and `not
following symlinks <follow_symlinks>`.

> [!NOTE]
> Although Windows supports `chmod`, you can only set the file's read-only flag with it (via the `stat.S_IWRITE` and `stat.S_IREAD` constants or a corresponding integer value). All other bits are ignored. The default value of *follow_symlinks* is `False` on Windows.
>
> The function is limited on WASI, see `wasm-availability` for more information.

<div class="audit-event">

os.chmod path,mode,dir_fd os.chmod

</div>

<div class="versionchanged">

3.3 Added support for specifying *path* as an open file descriptor, and the *dir_fd* and *follow_symlinks* arguments.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.13 Added support for a file descriptor and the *follow_symlinks* argument on Windows.

</div>

</div>

<div class="function">

chown(path, uid, gid, \*, dir_fd=None, follow_symlinks=True)

Change the owner and group id of *path* to the numeric *uid* and *gid*. To leave one of the ids unchanged, set it to -1.

This function can support `specifying a file descriptor <path_fd>`, `paths relative to directory descriptors <dir_fd>` and `not
following symlinks <follow_symlinks>`.

See `shutil.chown` for a higher-level function that accepts names in addition to numeric ids.

<div class="audit-event">

os.chown path,uid,gid,dir_fd os.chown

</div>

<div class="availability">

Unix.

The function is limited on WASI, see `wasm-availability` for more information.

</div>

<div class="versionchanged">

3.3 Added support for specifying *path* as an open file descriptor, and the *dir_fd* and *follow_symlinks* arguments.

</div>

<div class="versionchanged">

3.6 Supports a `path-like object`.

</div>

</div>

<div class="function">

chroot(path)

Change the root directory of the current process to *path*.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

fchdir(fd)

Change the current working directory to the directory represented by the file descriptor *fd*. The descriptor must refer to an opened directory, not an open file. As of Python 3.3, this is equivalent to `os.chdir(fd)`.

<div class="audit-event">

os.chdir path os.fchdir

</div>

<div class="availability">

Unix.

</div>

</div>

<div class="function">

getcwd()

Return a string representing the current working directory.

</div>

<div class="function">

getcwdb()

Return a bytestring representing the current working directory.

<div class="versionchanged">

3.8 The function now uses the UTF-8 encoding on Windows, rather than the ANSI code page: see `529` for the rationale. The function is no longer deprecated on Windows.

</div>

</div>

<div class="function">

lchflags(path, flags)

Set the flags of *path* to the numeric *flags*, like `chflags`, but do not follow symbolic links. As of Python 3.3, this is equivalent to `os.chflags(path, flags, follow_symlinks=False)`.

<div class="audit-event">

os.chflags path,flags os.lchflags

</div>

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

lchmod(path, mode)

Change the mode of *path* to the numeric *mode*. If path is a symlink, this affects the symlink rather than the target. See the docs for `chmod` for possible values of *mode*. As of Python 3.3, this is equivalent to `os.chmod(path, mode, follow_symlinks=False)`.

`lchmod()` is not part of POSIX, but Unix implementations may have it if changing the mode of symbolic links is supported.

<div class="audit-event">

os.chmod path,mode,dir_fd os.lchmod

</div>

<div class="availability">

Unix, Windows, not Linux, FreeBSD \>= 1.3, NetBSD \>= 1.3, not OpenBSD

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.13 Added support on Windows.

</div>

</div>

<div class="function">

lchown(path, uid, gid)

Change the owner and group id of *path* to the numeric *uid* and *gid*. This function will not follow symbolic links. As of Python 3.3, this is equivalent to `os.chown(path, uid, gid, follow_symlinks=False)`.

<div class="audit-event">

os.chown path,uid,gid,dir_fd os.lchown

</div>

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

link(src, dst, \*, src_dir_fd=None, dst_dir_fd=None, follow_symlinks=True)

Create a hard link pointing to *src* named *dst*.

This function can support specifying *src_dir_fd* and/or *dst_dir_fd* to supply `paths relative to directory descriptors <dir_fd>`, and `not
following symlinks <follow_symlinks>`. The default value of *follow_symlinks* is `False` on Windows.

<div class="audit-event">

os.link src,dst,src_dir_fd,dst_dir_fd os.link

</div>

<div class="availability">

Unix, Windows.

</div>

<div class="versionchanged">

3.2 Added Windows support.

</div>

<div class="versionchanged">

3.3 Added the *src_dir_fd*, *dst_dir_fd*, and *follow_symlinks* parameters.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object` for *src* and *dst*.

</div>

</div>

<div class="function">

listdir(path='.')

Return a list containing the names of the entries in the directory given by *path*. The list is in arbitrary order, and does not include the special entries `'.'` and `'..'` even if they are present in the directory. If a file is removed from or added to the directory during the call of this function, whether a name for that file be included is unspecified.

*path* may be a `path-like object`. If *path* is of type `bytes` (directly or indirectly through the `PathLike` interface), the filenames returned will also be of type `bytes`; in all other circumstances, they will be of type `str`.

This function can also support `specifying a file descriptor
<path_fd>`; the file descriptor must refer to a directory.

<div class="audit-event">

os.listdir path os.listdir

</div>

> [!NOTE]
> To encode `str` filenames to `bytes`, use `~os.fsencode`.

<div class="seealso">

The `scandir` function returns directory entries along with file attribute information, giving better performance for many common use cases.

</div>

<div class="versionchanged">

3.2 The *path* parameter became optional.

</div>

<div class="versionchanged">

3.3 Added support for specifying *path* as an open file descriptor.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.15 `os.listdir(-1)` now fails with `OSError(errno.EBADF)` rather than listing the current directory.

</div>

</div>

<div class="function">

listdrives()

Return a list containing the names of drives on a Windows system.

A drive name typically looks like `'C:\\'`. Not every drive name will be associated with a volume, and some may be inaccessible for a variety of reasons, including permissions, network connectivity or missing media. This function does not test for access.

May raise `OSError` if an error occurs collecting the drive names.

<div class="audit-event">

os.listdrives "" os.listdrives

</div>

<div class="availability">

Windows

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

listmounts(volume)

Return a list containing the mount points for a volume on a Windows system.

*volume* must be represented as a GUID path, like those returned by `os.listvolumes`. Volumes may be mounted in multiple locations or not at all. In the latter case, the list will be empty. Mount points that are not associated with a volume will not be returned by this function.

The mount points return by this function will be absolute paths, and may be longer than the drive name.

Raises `OSError` if the volume is not recognized or if an error occurs collecting the paths.

<div class="audit-event">

os.listmounts volume os.listmounts

</div>

<div class="availability">

Windows

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

listvolumes()

Return a list containing the volumes in the system.

Volumes are typically represented as a GUID path that looks like `\\?\Volume{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}\`. Files can usually be accessed through a GUID path, permissions allowing. However, users are generally not familiar with them, and so the recommended use of this function is to retrieve mount points using `os.listmounts`.

May raise `OSError` if an error occurs collecting the volumes.

<div class="audit-event">

os.listvolumes "" os.listvolumes

</div>

<div class="availability">

Windows

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

lstat(path, \*, dir_fd=None)

Perform the equivalent of an `!lstat` system call on the given path. Similar to `~os.stat`, but does not follow symbolic links. Return a `stat_result` object.

On platforms that do not support symbolic links, this is an alias for `~os.stat`.

As of Python 3.3, this is equivalent to `os.stat(path, dir_fd=dir_fd, follow_symlinks=False)`.

This function can also support `paths relative to directory descriptors
<dir_fd>`.

<div class="seealso">

The `.stat` function.

</div>

<div class="versionchanged">

3.2 Added support for Windows 6.0 (Vista) symbolic links.

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.8 On Windows, now opens reparse points that represent another path (name surrogates), including symbolic links and directory junctions. Other kinds of reparse points are resolved by the operating system as for `~os.stat`.

</div>

</div>

<div class="function">

mkdir(path, mode=0o777, \*, dir_fd=None)

Create a directory named *path* with numeric mode *mode*.

If the directory already exists, `FileExistsError` is raised. If a parent directory in the path does not exist, `FileNotFoundError` is raised.

<div id="mkdir_modebits">

On some systems, *mode* is ignored. Where it is used, the current umask value is first masked out. If bits other than the last 9 (i.e. the last 3 digits of the octal representation of the *mode*) are set, their meaning is platform-dependent. On some platforms, they are ignored and you should call `chmod` explicitly to set them.

</div>

On Windows, a *mode* of `0o700` is specifically handled to apply access control to the new directory such that only the current user and administrators have access. Other values of *mode* are ignored.

This function can also support `paths relative to directory descriptors
<dir_fd>`.

It is also possible to create temporary directories; see the `tempfile` module's `tempfile.mkdtemp` function.

<div class="audit-event">

os.mkdir path,mode,dir_fd os.mkdir

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.13 Windows now handles a *mode* of `0o700`.

</div>

</div>

<div class="function">

makedirs(name, mode=0o777, exist_ok=False, \*, parent_mode=None)

<div class="index">

single: directory; creating single: UNC paths; and os.makedirs()

</div>

Recursive directory creation function. Like `mkdir`, but makes all intermediate-level directories needed to contain the leaf directory.

The *mode* parameter is passed to `mkdir` for creating the leaf directory; see `the mkdir() description <mkdir_modebits>` for how it is interpreted. To set the file permission bits of any newly created parent directories you can set the umask before invoking `makedirs`. The file permission bits of existing parent directories are not changed.

If *exist_ok* is `False` (the default), a `FileExistsError` is raised if the target directory already exists.

If *parent_mode* is not `None`, it is used as the mode for any newly-created, intermediate-level directories. Like *mode*, it is combined with the process's umask value; see `the mkdir()
description <mkdir_modebits>`. Otherwise, intermediate directories are created with the default mode, which is also subject to the umask.

> [!NOTE]
> `makedirs` will become confused if the path elements to create include `pardir` (eg. ".." on UNIX systems).

This function handles UNC paths correctly.

<div class="audit-event">

os.mkdir path,mode,dir_fd os.makedirs

</div>

<div class="versionchanged">

3.2 Added the *exist_ok* parameter.

</div>

<div class="versionchanged">

3.4.1

Before Python 3.4.1, if *exist_ok* was `True` and the directory existed, `makedirs` would still raise an error if *mode* did not match the mode of the existing directory. Since this behavior was impossible to implement safely, it was removed in Python 3.4.1. See `21082`.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.7 The *mode* argument no longer affects the file permission bits of newly created intermediate-level directories.

</div>

<div class="versionadded">

3.15 The *parent_mode* parameter. To match the behavior from Python 3.6 and earlier (where *mode* was applied to all created directories), pass `parent_mode=mode`.

</div>

</div>

<div class="function">

mkfifo(path, mode=0o666, \*, dir_fd=None)

Create a FIFO (a named pipe) named *path* with numeric mode *mode*. The current umask value is first masked out from the mode.

This function can also support `paths relative to directory descriptors
<dir_fd>`.

FIFOs are pipes that can be accessed like regular files. FIFOs exist until they are deleted (for example with `os.unlink`). Generally, FIFOs are used as rendezvous between "client" and "server" type processes: the server opens the FIFO for reading, and the client opens it for writing. Note that `mkfifo` doesn't open the FIFO --- it just creates the rendezvous point.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

mknod(path, mode=0o600, device=0, \*, dir_fd=None)

Create a filesystem node (file, device special file or named pipe) named *path*. *mode* specifies both the permissions to use and the type of node to be created, being combined (bitwise OR) with one of `stat.S_IFREG`, `stat.S_IFCHR`, `stat.S_IFBLK`, and `stat.S_IFIFO`. For `stat.S_IFCHR` and `stat.S_IFBLK`, *device* defines the newly created device special file (probably using `os.makedev`), otherwise it is ignored.

This function can also support `paths relative to directory descriptors
<dir_fd>`.

<div class="availability">

Unix, not WASI.

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

major(device, /)

Extract the device major number from a raw device number (usually the `~stat_result.st_dev` or `~stat_result.st_rdev` field from `stat`).

</div>

<div class="function">

minor(device, /)

Extract the device minor number from a raw device number (usually the `~stat_result.st_dev` or `~stat_result.st_rdev` field from `stat`).

</div>

<div class="function">

makedev(major, minor, /)

Compose a raw device number from the major and minor device numbers.

</div>

<div class="data">

NODEV

Non-existent device.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

pathconf(path, name)

Return system configuration information relevant to a named file. *name* specifies the configuration value to retrieve; it may be a string which is the name of a defined system value; these names are specified in a number of standards (POSIX.1, Unix 95, Unix 98, and others). Some platforms define additional names as well. The names known to the host operating system are given in the `pathconf_names` dictionary. For configuration variables not included in that mapping, passing an integer for *name* is also accepted.

If *name* is a string and is not known, `ValueError` is raised. If a specific value for *name* is not supported by the host system, even if it is included in `pathconf_names`, an `OSError` is raised with `errno.EINVAL` for the error number.

This function can support `specifying a file descriptor
<path_fd>`.

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="data">

pathconf_names

Dictionary mapping names accepted by `pathconf` and `fpathconf` to the integer values defined for those names by the host operating system. This can be used to determine the set of names known to the system.

<div class="availability">

Unix.

</div>

</div>

<div class="function">

readlink(path, \*, dir_fd=None)

Return a string representing the path to which the symbolic link points. The result may be either an absolute or relative pathname; if it is relative, it may be converted to an absolute pathname using `os.path.join(os.path.dirname(path), result)`.

This function can also support `paths relative to directory descriptors
<dir_fd>`.

On Linux, Android, macOS and iOS, *path* can also be a file descriptor referring to a symbolic link. In that case, *dir_fd* must be `None`. (On Linux and Android, such a file descriptor must be obtained through `os.open` with `os.O_RDONLY | os.O_PATH | os.O_NOFOLLOW`. On macOS and iOS, `os.open` must be called with `os.O_RDONLY | os.O_SYMLINK`.) On other operating systems, a `NotImplementedError` is raised if *path* is an integer.

If the *path* is a string object (directly or indirectly through a `PathLike` interface) or a file descriptor, the result will be a string object, and the call may raise a UnicodeDecodeError. If the *path* is a bytes object (direct or indirectly), the result will be a bytes object.

When trying to resolve a path that may contain links, use `~os.path.realpath` to properly handle recursion and platform differences.

<div class="availability">

Unix, Windows.

</div>

<div class="versionchanged">

3.2 Added support for Windows 6.0 (Vista) symbolic links.

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object` on Unix.

</div>

<div class="versionchanged">

3.8 Accepts a `path-like object` and a bytes object on Windows.

Added support for directory junctions, and changed to return the substitution path (which typically includes `\\?\` prefix) rather than the optional "print name" field that was previously returned.

</div>

<div class="versionchanged">

next Accepts file descriptors pointing to symbolic links as *path* on Linux, Android, macOS and iOS.

</div>

</div>

<div class="function">

remove(path, \*, dir_fd=None)

Remove (delete) the file *path*. If *path* is a directory, an `OSError` is raised. Use `rmdir` to remove directories. If the file does not exist, a `FileNotFoundError` is raised.

This function can support `paths relative to directory descriptors
<dir_fd>`.

On Windows, attempting to remove a file that is in use causes an exception to be raised; on Unix, the directory entry is removed but the storage allocated to the file is not made available until the original file is no longer in use.

This function is semantically identical to `unlink`.

<div class="audit-event">

os.remove path,dir_fd os.remove

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

removedirs(name)

<div class="index">

single: directory; deleting

</div>

Remove directories recursively. Works like `rmdir` except that, if the leaf directory is successfully removed, `removedirs` tries to successively remove every parent directory mentioned in *path* until an error is raised (which is ignored, because it generally means that a parent directory is not empty). For example, `os.removedirs('foo/bar/baz')` will first remove the directory `'foo/bar/baz'`, and then remove `'foo/bar'` and `'foo'` if they are empty. Raises `OSError` if the leaf directory could not be successfully removed.

<div class="audit-event">

os.remove path,dir_fd os.removedirs

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

rename(src, dst, \*, src_dir_fd=None, dst_dir_fd=None)

Rename the file or directory *src* to *dst*. If *dst* exists, the operation will fail with an `OSError` subclass in a number of cases:

On Windows, if *dst* exists a `FileExistsError` is always raised. The operation may fail if *src* and *dst* are on different filesystems. Use `shutil.move` to support moves to a different filesystem.

On Unix, if *src* is a file and *dst* is a directory or vice-versa, an `IsADirectoryError` or a `NotADirectoryError` will be raised respectively. If both are directories and *dst* is empty, *dst* will be silently replaced. If *dst* is a non-empty directory, an `OSError` is raised. If both are files, *dst* will be replaced silently if the user has permission. The operation may fail on some Unix flavors if *src* and *dst* are on different filesystems. If successful, the renaming will be an atomic operation (this is a POSIX requirement).

This function can support specifying *src_dir_fd* and/or *dst_dir_fd* to supply `paths relative to directory descriptors <dir_fd>`.

If you want cross-platform overwriting of the destination, use `replace`.

<div class="audit-event">

os.rename src,dst,src_dir_fd,dst_dir_fd os.rename

</div>

<div class="versionchanged">

3.3 Added the *src_dir_fd* and *dst_dir_fd* parameters.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object` for *src* and *dst*.

</div>

</div>

<div class="function">

renames(old, new)

Recursive directory or file renaming function. Works like `rename`, except creation of any intermediate directories needed to make the new pathname good is attempted first. After the rename, directories corresponding to rightmost path segments of the old name will be pruned away using `removedirs`.

> [!NOTE]
> This function can fail with the new directory structure made if you lack permissions needed to remove the leaf directory or file.

<div class="audit-event">

os.rename src,dst,src_dir_fd,dst_dir_fd os.renames

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object` for *old* and *new*.

</div>

</div>

<div class="function">

replace(src, dst, \*, src_dir_fd=None, dst_dir_fd=None)

Rename the file or directory *src* to *dst*. If *dst* is a non-empty directory, `OSError` will be raised. If *dst* exists and is a file, it will be replaced silently if the user has permission. The operation may fail if *src* and *dst* are on different filesystems. If successful, the renaming will be an atomic operation (this is a POSIX requirement).

This function can support specifying *src_dir_fd* and/or *dst_dir_fd* to supply `paths relative to directory descriptors <dir_fd>`.

<div class="audit-event">

os.rename src,dst,src_dir_fd,dst_dir_fd os.replace

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object` for *src* and *dst*.

</div>

</div>

<div class="function">

rmdir(path, \*, dir_fd=None)

Remove (delete) the directory *path*. If the directory does not exist or is not empty, a `FileNotFoundError` or an `OSError` is raised respectively. In order to remove whole directory trees, `shutil.rmtree` can be used.

This function can support `paths relative to directory descriptors
<dir_fd>`.

<div class="audit-event">

os.rmdir path,dir_fd os.rmdir

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

scandir(path='.')

Return an iterator of `os.DirEntry` objects corresponding to the entries in the directory given by *path*. The entries are yielded in arbitrary order, and the special entries `'.'` and `'..'` are not included. If a file is removed from or added to the directory after creating the iterator, whether an entry for that file be included is unspecified.

Using `scandir` instead of `listdir` can significantly increase the performance of code that also needs file type or file attribute information, because `os.DirEntry` objects expose this information if the operating system provides it when scanning a directory. All `os.DirEntry` methods may perform a system call, but `~os.DirEntry.is_dir` and `~os.DirEntry.is_file` usually only require a system call for symbolic links; `os.DirEntry.stat` always requires a system call on Unix but only requires one for symbolic links on Windows.

*path* may be a `path-like object`. If *path* is of type `bytes` (directly or indirectly through the `PathLike` interface), the type of the `~os.DirEntry.name` and `~os.DirEntry.path` attributes of each `os.DirEntry` will be `bytes`; in all other circumstances, they will be of type `str`.

This function can also support `specifying a file descriptor
<path_fd>`; the file descriptor must refer to a directory.

<div class="audit-event">

os.scandir path os.scandir

</div>

Sharing a `scandir` iterator between threads will not corrupt the iterator, but it is subject to `race conditions <race condition>`: which entries each thread receives is unspecified, and closing the iterator while another thread is iterating ends that iteration early.

The `scandir` iterator supports the `context manager` protocol and has the following method:

<div class="method">

scandir.close()

Close the iterator and free acquired resources.

This is called automatically when the iterator is `exhausted` or garbage collected, or when an error happens during iterating. However it is advisable to call it explicitly or use the `with` statement.

<div class="versionadded">

3.6

</div>

</div>

The following example shows a simple use of `scandir` to display all the files (excluding directories) in the given *path* that don't start with `'.'`. The `entry.is_file()` call will generally not make an additional system call:

    with os.scandir(path) as it:
        for entry in it:
            if not entry.name.startswith('.') and entry.is_file():
                print(entry.name)

> [!NOTE]
> On Unix-based systems, `scandir` uses the system's [opendir()](https://pubs.opengroup.org/onlinepubs/009695399/functions/opendir.html) and [readdir()](https://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html) functions. On Windows, it uses the Win32 [FindFirstFileW](https://msdn.microsoft.com/en-us/library/windows/desktop/aa364418(v=vs.85).aspx) and [FindNextFileW](https://msdn.microsoft.com/en-us/library/windows/desktop/aa364428(v=vs.85).aspx) functions.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.6 Added support for the `context manager` protocol and the `~scandir.close` method. If a `scandir` iterator is neither exhausted nor explicitly closed a `ResourceWarning` will be emitted in its destructor.

The function accepts a `path-like object`.

</div>

<div class="versionchanged">

3.7 Added support for `file descriptors <path_fd>` on Unix.

</div>

<div class="versionchanged">

3.15 `os.scandir(-1)` now fails with `OSError(errno.EBADF)` rather than listing the current directory.

</div>

</div>

<div class="DirEntry">

Object yielded by `scandir` to expose the file path and other file attributes of a directory entry.

`scandir` will provide as much of this information as possible without making additional system calls. When a `stat()` or `lstat()` system call is made, the `os.DirEntry` object will cache the result.

`os.DirEntry` instances are not intended to be stored in long-lived data structures; if you know the file metadata has changed or if a long time has elapsed since calling `scandir`, call `os.stat(entry.path)` to fetch up-to-date information.

Because the `os.DirEntry` methods can make operating system calls, they may also raise `OSError`. If you need very fine-grained control over errors, you can catch `OSError` when calling one of the `os.DirEntry` methods and handle as appropriate.

To be directly usable as a `path-like object`, `os.DirEntry` implements the `PathLike` interface.

`!DirEntry` objects are `generic <generics>` over the type of the path (`str` or `bytes`).

Attributes and methods on a `os.DirEntry` instance are as follows:

<div class="attribute">

name

The entry's base filename, relative to the `scandir` *path* argument.

The `name` attribute will be `bytes` if the `scandir` *path* argument is of type `bytes` and `str` otherwise. Use `~os.fsdecode` to decode byte filenames.

</div>

<div class="attribute">

path

The entry's path name: equivalent to `os.path.join(scandir_path, entry.name)` where *scandir_path* is the original `scandir` *path* argument. Apart from the filename, the path preserves the original `scandir` argument. If the `scandir` *path* argument was relative, the `path` attribute is also relative. Changing the current working directory after creating the `scandir` iterator may cause later uses of `path` to resolve differently. On some platforms, the constructed path may not be valid if the original `scandir` argument was usable for enumeration but not for joining with the entry name. If the `scandir` *path* argument was a `file descriptor <path_fd>`, the `path` attribute is the same as the `name` attribute.

The `path` attribute will be `bytes` if the `scandir` *path* argument is of type `bytes` and `str` otherwise. Use `~os.fsdecode` to decode byte filenames.

</div>

<div class="method">

inode()

Return the inode number of the entry.

The result is cached on the `os.DirEntry` object. Use `os.stat(entry.path, follow_symlinks=False).st_ino` to fetch up-to-date information.

On the first, uncached call, a system call is required on Windows but not on Unix.

</div>

<div class="method">

is_dir(\*, follow_symlinks=True)

Return `True` if this entry is a directory or a symbolic link pointing to a directory; return `False` if the entry is or points to any other kind of file, or if it doesn't exist anymore.

If *follow_symlinks* is `False`, return `True` only if this entry is a directory (without following symlinks); return `False` if the entry is any other kind of file or if it doesn't exist anymore.

The result is cached on the `os.DirEntry` object, with a separate cache for *follow_symlinks* `True` and `False`. Call `os.stat` along with `stat.S_ISDIR` to fetch up-to-date information.

On the first, uncached call, no system call is required in most cases. Specifically, for non-symlinks, neither Windows or Unix require a system call, except on certain Unix file systems, such as network file systems, that return `dirent.d_type == DT_UNKNOWN`. If the entry is a symlink, a system call will be required to follow the symlink unless *follow_symlinks* is `False`.

This method can raise `OSError`, such as `PermissionError`, but `FileNotFoundError` is caught and not raised.

</div>

<div class="method">

is_file(\*, follow_symlinks=True)

Return `True` if this entry is a file or a symbolic link pointing to a file; return `False` if the entry is or points to a directory or other non-file entry, or if it doesn't exist anymore.

If *follow_symlinks* is `False`, return `True` only if this entry is a file (without following symlinks); return `False` if the entry is a directory or other non-file entry, or if it doesn't exist anymore.

The result is cached on the `os.DirEntry` object. Caching, system calls made, and exceptions raised are as per `~os.DirEntry.is_dir`.

</div>

<div class="method">

is_symlink()

Return `True` if this entry is a symbolic link (even if broken); return `False` if the entry points to a directory or any kind of file, or if it doesn't exist anymore.

The result is cached on the `os.DirEntry` object. Call `os.path.islink` to fetch up-to-date information.

On the first, uncached call, no system call is required in most cases. Specifically, neither Windows or Unix require a system call, except on certain Unix file systems, such as network file systems, that return `dirent.d_type == DT_UNKNOWN`.

This method can raise `OSError`, such as `PermissionError`, but `FileNotFoundError` is caught and not raised.

</div>

<div class="method">

is_junction()

Return `True` if this entry is a junction (even if broken); return `False` if the entry points to a regular directory, any kind of file, a symlink, or if it doesn't exist anymore.

The result is cached on the `os.DirEntry` object. Call `os.path.isjunction` to fetch up-to-date information.

<div class="versionadded">

3.12

</div>

</div>

<div class="method">

stat(\*, follow_symlinks=True)

Return a `stat_result` object for this entry. This method follows symbolic links by default; to stat a symbolic link add the `follow_symlinks=False` argument.

On Unix, this method always requires a system call. On Windows, it only requires a system call if *follow_symlinks* is `True` and the entry is a reparse point (for example, a symbolic link or directory junction).

On Windows, the `st_ino`, `st_dev` and `st_nlink` attributes of the `stat_result` are always set to zero. Call `os.stat` to get these attributes.

The result is cached on the `os.DirEntry` object, with a separate cache for *follow_symlinks* `True` and `False`. Call `os.stat` to fetch up-to-date information.

</div>

Note that there is a nice correspondence between several attributes and methods of `os.DirEntry` and of `pathlib.Path`. In particular, the `name` attribute has the same meaning, as do the `is_dir()`, `is_file()`, `is_symlink()`, `is_junction()`, and `stat()` methods.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.6 Added support for the `~os.PathLike` interface. Added support for `bytes` paths on Windows.

</div>

<div class="versionchanged">

3.12 The `st_ctime` attribute of a stat result is deprecated on Windows. The file creation time is properly available as `st_birthtime`, and in the future `st_ctime` may be changed to return zero or the metadata change time, if available.

</div>

</div>

<div class="function">

stat(path, \*, dir_fd=None, follow_symlinks=True)

Get the status of a file or a file descriptor. Perform the equivalent of a `stat` system call on the given path. *path* may be specified as either a string or bytes -- directly or indirectly through the `PathLike` interface -- or as an open file descriptor. Return a `stat_result` object.

This function normally follows symlinks; to stat a symlink add the argument `follow_symlinks=False`, or use `lstat`.

This function can support `specifying a file descriptor <path_fd>` and `not following symlinks <follow_symlinks>`.

On Windows, passing `follow_symlinks=False` will disable following all name-surrogate reparse points, which includes symlinks and directory junctions. Other types of reparse points that do not resemble links or that the operating system is unable to follow will be opened directly. When following a chain of multiple links, this may result in the original link being returned instead of the non-link that prevented full traversal. To obtain stat results for the final path in this case, use the `os.path.realpath` function to resolve the path name as far as possible and call `lstat` on the result. This does not apply to dangling symlinks or junction points, which will raise the usual exceptions.

<div class="index">

pair: module; stat

</div>

Example:

    >>> import os
    >>> statinfo = os.stat('somefile.txt')
    >>> statinfo
    os.stat_result(st_mode=33188, st_ino=7876932, st_dev=234881026,
    st_nlink=1, st_uid=501, st_gid=501, st_size=264, st_atime=1297230295,
    st_mtime=1297230027, st_ctime=1297230027)
    >>> statinfo.st_size
    264

<div class="seealso">

`fstat` and `lstat` functions.

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* and *follow_symlinks* parameters, specifying a file descriptor instead of a path.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.8 On Windows, all reparse points that can be resolved by the operating system are now followed, and passing `follow_symlinks=False` disables following all name surrogate reparse points. If the operating system reaches a reparse point that it is not able to follow, *stat* now returns the information for the original path as if `follow_symlinks=False` had been specified instead of raising an error.

</div>

</div>

<div class="stat_result">

Object whose attributes correspond roughly to the members of the `stat` structure. It is used for the result of `os.stat`, `os.fstat` and `os.lstat`.

Attributes:

<div class="attribute">

st_mode

File mode: file type and file mode bits (permissions).

</div>

<div class="attribute">

st_ino

Platform dependent, but if non-zero, uniquely identifies the file for a given value of `st_dev`. Typically:

- the inode number on Unix,
- the [file index](https://msdn.microsoft.com/en-us/library/aa363788) on Windows

</div>

<div class="attribute">

st_dev

Identifier of the device on which this file resides.

</div>

<div class="attribute">

st_nlink

Number of hard links.

</div>

<div class="attribute">

st_uid

User identifier of the file owner.

</div>

<div class="attribute">

st_gid

Group identifier of the file owner.

</div>

<div class="attribute">

st_size

Size of the file in bytes, if it is a regular file or a symbolic link. The size of a symbolic link is the length of the pathname it contains, without a terminating null byte.

</div>

Timestamps:

<div class="attribute">

st_atime

Time of most recent access expressed in seconds.

</div>

<div class="attribute">

st_mtime

Time of most recent content modification expressed in seconds.

</div>

<div class="attribute">

st_ctime

Time of most recent metadata change expressed in seconds.

<div class="versionchanged">

3.12 `st_ctime` is deprecated on Windows. Use `st_birthtime` for the file creation time. In the future, `st_ctime` will contain the time of the most recent metadata change, as for other platforms.

</div>

</div>

<div class="attribute">

st_atime_ns

Time of most recent access expressed in nanoseconds as an integer.

<div class="versionadded">

3.3

</div>

</div>

<div class="attribute">

st_mtime_ns

Time of most recent content modification expressed in nanoseconds as an integer.

<div class="versionadded">

3.3

</div>

</div>

<div class="attribute">

st_ctime_ns

Time of most recent metadata change expressed in nanoseconds as an integer.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.12 `st_ctime_ns` is deprecated on Windows. Use `st_birthtime_ns` for the file creation time. In the future, `st_ctime` will contain the time of the most recent metadata change, as for other platforms.

</div>

</div>

<div class="attribute">

st_birthtime

Time of file creation expressed in seconds. This attribute is not always available, and may raise `AttributeError`.

<div class="versionchanged">

3.12 `st_birthtime` is now available on Windows.

</div>

</div>

<div class="attribute">

st_birthtime_ns

Time of file creation expressed in nanoseconds as an integer. This attribute is not always available, and may raise `AttributeError`.

<div class="versionadded">

3.12

</div>

</div>

> [!NOTE]
> The exact meaning and resolution of the `st_atime`, `st_mtime`, `st_ctime` and `st_birthtime` attributes depend on the operating system and the file system. For example, on Windows systems using the FAT32 file systems, `st_mtime` has 2-second resolution, and `st_atime` has only 1-day resolution. See your operating system documentation for details.
>
> Similarly, although `st_atime_ns`, `st_mtime_ns`, `st_ctime_ns` and `st_birthtime_ns` are always expressed in nanoseconds, many systems do not provide nanosecond precision. On systems that do provide nanosecond precision, the floating-point object used to store `st_atime`, `st_mtime`, `st_ctime` and `st_birthtime` cannot preserve all of it, and as such will be slightly inexact. If you need the exact timestamps you should always use `st_atime_ns`, `st_mtime_ns`, `st_ctime_ns` and `st_birthtime_ns`.

On some Unix systems (such as Linux), the following attributes may also be available:

<div class="attribute">

st_blocks

Number of 512-byte blocks allocated for file. This may be smaller than `st_size`/512 when the file has holes.

</div>

<div class="attribute">

st_blksize

"Preferred" blocksize for efficient file system I/O. Writing to a file in smaller chunks may cause an inefficient read-modify-rewrite.

</div>

<div class="attribute">

st_rdev

Type of device if an inode device.

</div>

<div class="attribute">

st_flags

User defined flags for file.

</div>

On other Unix systems (such as FreeBSD), the following attributes may be available (but may be only filled out if root tries to use them):

<div class="attribute">

st_gen

File generation number.

</div>

On Solaris and derivatives, the following attributes may also be available:

<div class="attribute">

st_fstype

String that uniquely identifies the type of the filesystem that contains the file.

</div>

On macOS systems, the following attributes may also be available:

<div class="attribute">

st_rsize

Real size of the file.

</div>

<div class="attribute">

st_creator

Creator of the file.

</div>

<div class="attribute">

st_type

File type.

</div>

On Windows systems, the following attributes are also available:

<div class="attribute">

st_file_attributes

Windows file attributes: `dwFileAttributes` member of the `BY_HANDLE_FILE_INFORMATION` structure returned by `!GetFileInformationByHandle`. See the `!FILE_ATTRIBUTE_* <stat.FILE_ATTRIBUTE_ARCHIVE>` constants in the `stat` module.

<div class="versionadded">

3.5

</div>

</div>

<div class="attribute">

st_reparse_tag

When `st_file_attributes` has the `~stat.FILE_ATTRIBUTE_REPARSE_POINT` set, this field contains the tag identifying the type of reparse point. See the `IO_REPARSE_TAG_* <stat.IO_REPARSE_TAG_SYMLINK>` constants in the `stat` module.

</div>

The standard module `stat` defines functions and constants that are useful for extracting information from a `stat` structure. (On Windows, some items are filled with dummy values.)

For backward compatibility, a `stat_result` instance is also accessible as a tuple of at least 10 integers giving the most important (and portable) members of the `stat` structure, in the order `st_mode`, `st_ino`, `st_dev`, `st_nlink`, `st_uid`, `st_gid`, `st_size`, `st_atime`, `st_mtime`, `st_ctime`. More items may be added at the end by some implementations. For compatibility with older Python versions, accessing `stat_result` as a tuple always returns integers.

<div class="versionchanged">

3.5 Windows now returns the file index as `st_ino` when available.

</div>

<div class="versionchanged">

3.7 Added the `st_fstype` member to Solaris/derivatives.

</div>

<div class="versionchanged">

3.8 Added the `st_reparse_tag` member on Windows.

</div>

<div class="versionchanged">

3.8 On Windows, the `st_mode` member now identifies special files as `~stat.S_IFCHR`, `~stat.S_IFIFO` or `~stat.S_IFBLK` as appropriate.

</div>

<div class="versionchanged">

3.12 On Windows, `st_ctime` is deprecated. Eventually, it will contain the last metadata change time, for consistency with other platforms, but for now still contains creation time. Use `st_birthtime` for the creation time.

On Windows, `st_ino` may now be up to 128 bits, depending on the file system. Previously it would not be above 64 bits, and larger file identifiers would be arbitrarily packed.

On Windows, `st_rdev` no longer returns a value. Previously it would contain the same as `st_dev`, which was incorrect.

Added the `st_birthtime` member on Windows.

</div>

</div>

<div class="function">

statx(path, mask, \*, flags=0, dir_fd=None, follow_symlinks=True)

Get the status of a file or file descriptor by performing a `!statx` system call on the given path.

*path* is a `path-like object` or an open file descriptor. *mask* is a combination of the module-level `STATX_* <STATX_TYPE>` constants specifying the information to retrieve. *flags* is a combination of the module-level `AT_STATX_* <AT_STATX_FORCE_SYNC>` constants and/or `AT_NO_AUTOMOUNT`. Returns a `statx_result` object whose `~os.statx_result.stx_mask` attribute specifies the information actually retrieved (which may differ from *mask*).

This function supports `specifying a file descriptor <path_fd>`, `paths relative to directory descriptors <dir_fd>`, and `not following symlinks <follow_symlinks>`.

<div class="seealso">

The `statx(2)` man page.

</div>

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="statx_result">

Information about a file returned by `os.statx`.

`!statx_result` has the following attributes:

<div class="attribute">

stx_atime

Time of most recent access expressed in seconds.

Equal to `None` if `STATX_ATIME` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_atime_ns

Time of most recent access expressed in nanoseconds as an integer.

Equal to `None` if `STATX_ATIME` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_atomic_write_segments_max

Maximum iovecs for direct I/O with torn-write protection.

Equal to `None` if `STATX_WRITE_ATOMIC` is missing from `~statx_result.stx_mask`.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28 and build-time kernel userspace API headers \>= 6.11.

</div>

</div>

<div class="attribute">

stx_atomic_write_unit_max

Maximum size for direct I/O with torn-write protection.

Equal to `None` if `STATX_WRITE_ATOMIC` is missing from `~statx_result.stx_mask`.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28 and build-time kernel userspace API headers \>= 6.11.

</div>

</div>

<div class="attribute">

stx_atomic_write_unit_max_opt

Maximum optimized size for direct I/O with torn-write protection.

Equal to `None` if `STATX_WRITE_ATOMIC` is missing from `~statx_result.stx_mask`.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28 and build-time kernel userspace API headers \>= 6.16.

</div>

</div>

<div class="attribute">

stx_atomic_write_unit_min

Minimum size for direct I/O with torn-write protection.

Equal to `None` if `STATX_WRITE_ATOMIC` is missing from `~statx_result.stx_mask`.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28 and build-time kernel userspace API headers \>= 6.11.

</div>

</div>

<div class="attribute">

stx_attributes

Bitmask of `STATX_ATTR_* <stat.STATX_ATTR_COMPRESSED>` constants specifying the attributes of this file.

</div>

<div class="attribute">

stx_attributes_mask

A mask indicating which bits in `stx_attributes` are supported by the VFS and the filesystem.

</div>

<div class="attribute">

stx_blksize

"Preferred" blocksize for efficient file system I/O. Writing to a file in smaller chunks may cause an inefficient read-modify-rewrite.

</div>

<div class="attribute">

stx_blocks

Number of 512-byte blocks allocated for file. This may be smaller than `stx_size`/512 when the file has holes.

Equal to `None` if `STATX_BLOCKS` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_btime

Time of file creation expressed in seconds.

Equal to `None` if `STATX_BTIME` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_btime_ns

Time of file creation expressed in nanoseconds as an integer.

Equal to `None` if `STATX_BTIME` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_ctime

Time of most recent metadata change expressed in seconds.

Equal to `None` if `STATX_CTIME` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_ctime_ns

Time of most recent metadata change expressed in nanoseconds as an integer.

Equal to `None` if `STATX_CTIME` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_dev

Identifier of the device on which this file resides.

</div>

<div class="attribute">

stx_dev_major

Major number of the device on which this file resides.

</div>

<div class="attribute">

stx_dev_minor

Minor number of the device on which this file resides.

</div>

<div class="attribute">

stx_dio_mem_align

Direct I/O memory buffer alignment requirement.

Equal to `None` if `STATX_DIOALIGN` is missing from `~statx_result.stx_mask`.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28 and build-time kernel userspace API headers \>= 6.1.

</div>

</div>

<div class="attribute">

stx_dio_offset_align

Direct I/O file offset alignment requirement.

Equal to `None` if `STATX_DIOALIGN` is missing from `~statx_result.stx_mask`.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28 and build-time kernel userspace API headers \>= 6.1.

</div>

</div>

<div class="attribute">

stx_dio_read_offset_align

Direct I/O file offset alignment requirement for reads.

Equal to `None` if `STATX_DIO_READ_ALIGN` is missing from `~statx_result.stx_mask`.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28 and build-time kernel userspace API headers \>= 6.14.

</div>

</div>

<div class="attribute">

stx_gid

Group identifier of the file owner.

Equal to `None` if `STATX_GID` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_ino

Inode number.

Equal to `None` if `STATX_INO` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_mask

Bitmask of `STATX_* <STATX_TYPE>` constants specifying the information retrieved, which may differ from what was requested.

</div>

<div class="attribute">

stx_mnt_id

Mount identifier.

Equal to `None` if `STATX_MNT_ID` is missing from `~statx_result.stx_mask`.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28 and build-time kernel userspace API headers \>= 5.8.

</div>

</div>

<div class="attribute">

stx_mode

File mode: file type and file mode bits (permissions).

Equal to `None` if `STATX_TYPE | STATX_MODE <STATX_TYPE>` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_mtime

Time of most recent content modification expressed in seconds.

Equal to `None` if `STATX_MTIME` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_mtime_ns

Time of most recent content modification expressed in nanoseconds as an integer.

Equal to `None` if `STATX_MTIME` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_nlink

Number of hard links.

Equal to `None` if `STATX_NLINK` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_rdev

Type of device if an inode device.

</div>

<div class="attribute">

stx_rdev_major

Major number of the device this file represents.

</div>

<div class="attribute">

stx_rdev_minor

Minor number of the device this file represents.

</div>

<div class="attribute">

stx_size

Size of the file in bytes, if it is a regular file or a symbolic link. The size of a symbolic link is the length of the pathname it contains, without a terminating null byte.

Equal to `None` if `STATX_SIZE` is missing from `~statx_result.stx_mask`.

</div>

<div class="attribute">

stx_subvol

Subvolume identifier.

Equal to `None` if `STATX_SUBVOL` is missing from `~statx_result.stx_mask`.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28 and build-time kernel userspace API headers \>= 6.10.

</div>

</div>

<div class="attribute">

stx_uid

User identifier of the file owner.

Equal to `None` if `STATX_UID` is missing from `~statx_result.stx_mask`.

</div>

<div class="seealso">

The `statx(2)` man page.

</div>

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

STATX_TYPE STATX_MODE STATX_NLINK STATX_UID STATX_GID STATX_ATIME STATX_MTIME STATX_CTIME STATX_INO STATX_SIZE STATX_BLOCKS STATX_BASIC_STATS STATX_BTIME STATX_MNT_ID STATX_DIOALIGN STATX_MNT_ID_UNIQUE STATX_SUBVOL STATX_WRITE_ATOMIC STATX_DIO_READ_ALIGN

Bitflags for use in the *mask* parameter to `os.statx`. Some of these flags may be available even when their corresponding members in `statx_result` are not available.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

AT_STATX_FORCE_SYNC

A flag for the `os.statx` function. Requests that the kernel return up-to-date information even when doing so is expensive (for example, requiring a round trip to the server for a file on a network filesystem).

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

AT_STATX_DONT_SYNC

A flag for the `os.statx` function. Requests that the kernel return cached information if possible.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

AT_STATX_SYNC_AS_STAT

A flag for the `os.statx` function. This flag is defined as `0`, so it has no effect, but it can be used to explicitly indicate neither `AT_STATX_FORCE_SYNC` nor `AT_STATX_DONT_SYNC` is being passed. In the absence of the other two flags, the kernel will generally return information as fresh as `os.stat` would return.

<div class="availability">

Linux \>= 4.11 with glibc \>= 2.28.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

AT_NO_AUTOMOUNT

If the final component of a path is an automount point, operate on the automount point instead of performing the automount. On Linux, `os.stat`, `os.fstat` and `os.lstat` always behave this way.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

statvfs(path)

Perform a `statvfs(3)` system call on the given path. The return value is a `statvfs_result` whose attributes describe the filesystem on the given path and correspond to the members of the `statvfs` structure.

This function can support `specifying a file descriptor <path_fd>`.

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.3 Added support for specifying *path* as an open file descriptor.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="statvfs_result">

Filesystem statistics returned by `os.statvfs` and `os.fstatvfs`. See `statvfs(3)` for more details.

<div class="attribute">

f_bsize

Block size.

</div>

<div class="attribute">

f_frsize

Fragment size.

</div>

<div class="attribute">

f_blocks

Number of `~statvfs_result.f_frsize` sized blocks the filesystem can contain.

</div>

<div class="attribute">

f_bfree

Number of free blocks.

</div>

<div class="attribute">

f_bavail

Number of free blocks for unprivileged users.

</div>

<div class="attribute">

f_files

Number of file entries, inodes, the filesystem can contain.

</div>

<div class="attribute">

f_ffree

Number of free files entries.

</div>

<div class="attribute">

f_favail

Number of free file entries for unprivileged users.

</div>

<div class="attribute">

f_flag

Bit-mask of mount flags. The following flags are defined: `ST_RDONLY`, `ST_NOSUID`, `ST_NODEV`, `ST_NOEXEC`, `ST_SYNCHRONOUS`, `ST_MANDLOCK`, `ST_WRITE`, `ST_APPEND`, `ST_IMMUTABLE`, `ST_NOATIME`, `ST_NODIRATIME`, and `ST_RELATIME`.

</div>

<div class="attribute">

f_namemax

Filesystem max filename length. OS specific limitations such as `Windows MAX_PATH <max-path>` and those described in Linux `pathname(7)` may exist.

</div>

<div class="attribute">

f_fsid

Filesystem ID.

<div class="versionadded">

3.7

</div>

</div>

</div>

The following flags are used in `statvfs_result.f_flag`.

<div class="data">

ST_RDONLY

Read-only filesystem.

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ST_NOSUID

Setuid/setgid bits are disabled or not supported.

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ST_NODEV

Disallow access to device special files.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

ST_NOEXEC

Disallow program execution.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

ST_SYNCHRONOUS

Writes are synced at once.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

ST_MANDLOCK

Allow mandatory locks on an FS.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

ST_WRITE

Write on file/directory/symlink.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

ST_APPEND

Append-only file.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

ST_IMMUTABLE

Immutable file.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

ST_NOATIME

Do not update access times.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

ST_NODIRATIME

Do not update directory access times.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

ST_RELATIME

Update atime relative to mtime/ctime.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

supports_dir_fd

A `set` object indicating which functions in the `!os` module accept an open file descriptor for their *dir_fd* parameter. Different platforms provide different features, and the underlying functionality Python uses to implement the *dir_fd* parameter is not available on all platforms Python supports. For consistency's sake, functions that may support *dir_fd* always allow specifying the parameter, but will throw an exception if the functionality is used when it's not locally available. (Specifying `None` for *dir_fd* is always supported on all platforms.)

To check whether a particular function accepts an open file descriptor for its *dir_fd* parameter, use the `in` operator on `supports_dir_fd`. As an example, this expression evaluates to `True` if `os.stat` accepts open file descriptors for *dir_fd* on the local platform:

    os.stat in os.supports_dir_fd

Currently *dir_fd* parameters only work on Unix platforms; none of them work on Windows.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

supports_effective_ids

A `set` object indicating whether `os.access` permits specifying `True` for its *effective_ids* parameter on the local platform. (Specifying `False` for *effective_ids* is always supported on all platforms.) If the local platform supports it, the collection will contain `os.access`; otherwise it will be empty.

This expression evaluates to `True` if `os.access` supports `effective_ids=True` on the local platform:

    os.access in os.supports_effective_ids

Currently *effective_ids* is only supported on Unix platforms; it does not work on Windows.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

supports_fd

A `set` object indicating which functions in the `!os` module permit specifying their *path* parameter as an open file descriptor on the local platform. Different platforms provide different features, and the underlying functionality Python uses to accept open file descriptors as *path* arguments is not available on all platforms Python supports.

To determine whether a particular function permits specifying an open file descriptor for its *path* parameter, use the `in` operator on `supports_fd`. As an example, this expression evaluates to `True` if `os.chdir` accepts open file descriptors for *path* on your local platform:

    os.chdir in os.supports_fd

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

supports_follow_symlinks

A `set` object indicating which functions in the `!os` module accept `False` for their *follow_symlinks* parameter on the local platform. Different platforms provide different features, and the underlying functionality Python uses to implement *follow_symlinks* is not available on all platforms Python supports. For consistency's sake, functions that may support *follow_symlinks* always allow specifying the parameter, but will throw an exception if the functionality is used when it's not locally available. (Specifying `True` for *follow_symlinks* is always supported on all platforms.)

To check whether a particular function accepts `False` for its *follow_symlinks* parameter, use the `in` operator on `supports_follow_symlinks`. As an example, this expression evaluates to `True` if you may specify `follow_symlinks=False` when calling `os.stat` on the local platform:

    os.stat in os.supports_follow_symlinks

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

symlink(src, dst, target_is_directory=False, \*, dir_fd=None)

Create a symbolic link pointing to *src* named *dst*.

The *src* parameter refers to the target of the link (the file or directory being linked to), and *dst* is the name of the link being created.

On Windows, a symlink represents either a file or a directory, and does not morph to the target dynamically. If the target is present, the type of the symlink will be created to match. Otherwise, the symlink will be created as a directory if *target_is_directory* is `True` or a file symlink (the default) otherwise. On non-Windows platforms, *target_is_directory* is ignored.

This function can support `paths relative to directory descriptors
<dir_fd>`.

> [!NOTE]
> On newer versions of Windows 10, unprivileged accounts can create symlinks if Developer Mode is enabled. When Developer Mode is not available/enabled, the *SeCreateSymbolicLinkPrivilege* privilege is required, or the process must be run as an administrator.
>
> `OSError` is raised when the function is called by an unprivileged user.

<div class="audit-event">

os.symlink src,dst,dir_fd os.symlink

</div>

<div class="availability">

Unix, Windows.

The function is limited on WASI, see `wasm-availability` for more information.

</div>

<div class="versionchanged">

3.2 Added support for Windows 6.0 (Vista) symbolic links.

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* parameter, and now allow *target_is_directory* on non-Windows platforms.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object` for *src* and *dst*.

</div>

<div class="versionchanged">

3.8 Added support for unelevated symlinks on Windows with Developer Mode.

</div>

</div>

<div class="function">

sync()

Force write of everything to disk.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

truncate(path, length)

Truncate the file corresponding to *path*, so that it is at most *length* bytes in size.

This function can support `specifying a file descriptor <path_fd>`.

<div class="audit-event">

os.truncate path,length os.truncate

</div>

<div class="availability">

Unix, Windows.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.5 Added support for Windows

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

unlink(path, \*, dir_fd=None)

Remove (delete) the file *path*. This function is semantically identical to `remove`; the `unlink` name is its traditional Unix name. Please see the documentation for `remove` for further information.

<div class="audit-event">

os.remove path,dir_fd os.unlink

</div>

<div class="versionchanged">

3.3 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

utime(path, times=None, \*\[, ns\], dir_fd=None, follow_symlinks=True)

Set the access and modified times of the file specified by *path*.

`utime` takes two optional parameters, *times* and *ns*. These specify the times set on *path* and are used as follows:

- If *ns* is specified, it must be a 2-tuple of the form `(atime_ns, mtime_ns)` where each member is an int expressing nanoseconds.
- If *times* is not `None`, it must be a 2-tuple of the form `(atime, mtime)` where each member is a real number expressing seconds, rounded down to nanoseconds.
- If *times* is `None` and *ns* is unspecified, this is equivalent to specifying `ns=(atime_ns, mtime_ns)` where both times are the current time.

It is an error to specify tuples for both *times* and *ns*.

Note that the exact times you set here may not be returned by a subsequent `~os.stat` call, depending on the resolution with which your operating system records access and modification times; see `~os.stat`. The best way to preserve exact times is to use the *st_atime_ns* and *st_mtime_ns* fields from the `os.stat` result object with the *ns* parameter to `utime`.

This function can support `specifying a file descriptor <path_fd>`, `paths relative to directory descriptors <dir_fd>` and `not
following symlinks <follow_symlinks>`.

<div class="audit-event">

os.utime path,times,ns,dir_fd os.utime

</div>

<div class="versionchanged">

3.3 Added support for specifying *path* as an open file descriptor, and the *dir_fd*, *follow_symlinks*, and *ns* parameters.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.15 Accepts any real numbers as *times*, not only integers or floats.

</div>

</div>

<div class="function">

walk(top, topdown=True, onerror=None, followlinks=False)

<div class="index">

single: directory; walking single: directory; traversal

</div>

Generate the file names in a directory tree by walking the tree either top-down or bottom-up. For each directory in the tree rooted at directory *top* (including *top* itself), it yields a 3-tuple `(dirpath, dirnames, filenames)`.

*dirpath* is a string, the path to the directory. *dirnames* is a list of the names of the subdirectories in *dirpath* (including symlinks to directories, and excluding `'.'` and `'..'`). *filenames* is a list of the names of the non-directory files in *dirpath*. Note that the names in the lists contain no path components. To get a full path (which begins with *top*) to a file or directory in *dirpath*, do `os.path.join(dirpath, name)`. Whether or not the lists are sorted depends on the file system. If a file is removed from or added to the *dirpath* directory during generating the lists, whether a name for that file be included is unspecified.

If optional argument *topdown* is `True` or not specified, the triple for a directory is generated before the triples for any of its subdirectories (directories are generated top-down). If *topdown* is `False`, the triple for a directory is generated after the triples for all of its subdirectories (directories are generated bottom-up). No matter the value of *topdown*, the list of subdirectories is retrieved before the tuples for the directory and its subdirectories are generated.

When *topdown* is `True`, the caller can modify the *dirnames* list in-place (perhaps using `del` or slice assignment), and `walk` will only recurse into the subdirectories whose names remain in *dirnames*; this can be used to prune the search, impose a specific order of visiting, or even to inform `walk` about directories the caller creates or renames before it resumes `walk` again. Modifying *dirnames* when *topdown* is `False` has no effect on the behavior of the walk, because in bottom-up mode the directories in *dirnames* are generated before *dirpath* itself is generated.

By default, errors from the `scandir` call are ignored. If optional argument *onerror* is specified, it should be a function; it will be called with one argument, an `OSError` instance. It can report the error to continue with the walk, or raise the exception to abort the walk. Note that the filename is available as the `filename` attribute of the exception object.

By default, `walk` will not walk down into symbolic links that resolve to directories. Set *followlinks* to `True` to visit directories pointed to by symlinks, on systems that support them.

> [!NOTE]
> Be aware that setting *followlinks* to `True` can lead to infinite recursion if a link points to a parent directory of itself. `walk` does not keep track of the directories it visited already.

> [!NOTE]
> If you pass a relative pathname, don't change the current working directory between resumptions of `walk`. `walk` never changes the current directory, and assumes that its caller doesn't either.

This example displays the number of bytes taken by non-directory files in each directory under the starting directory, except that it doesn't look under any `__pycache__` subdirectory:

    import os
    from os.path import join, getsize
    for root, dirs, files in os.walk('python/Lib/xml'):
        print(root, "consumes", end=" ")
        print(sum(getsize(join(root, name)) for name in files), end=" ")
        print("bytes in", len(files), "non-directory files")
        if '__pycache__' in dirs:
            dirs.remove('__pycache__')  # don't visit __pycache__ directories

In the next example (simple implementation of `shutil.rmtree`), walking the tree bottom-up is essential, `rmdir` doesn't allow deleting a directory before the directory is empty:

    # Delete everything reachable from the directory named in "top",
    # assuming there are no symbolic links.
    # CAUTION:  This is dangerous!  For example, if top == '/', it
    # could delete all your disk files.
    import os
    for root, dirs, files in os.walk(top, topdown=False):
        for name in files:
            os.remove(os.path.join(root, name))
        for name in dirs:
            os.rmdir(os.path.join(root, name))
    os.rmdir(top)

<div class="audit-event">

os.walk top,topdown,onerror,followlinks os.walk

</div>

<div class="versionchanged">

3.5 This function now calls `os.scandir` instead of `os.listdir`, making it faster by reducing the number of calls to `os.stat`.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

fwalk(top='.', topdown=True, onerror=None, \*, follow_symlinks=False, dir_fd=None)

<div class="index">

single: directory; walking single: directory; traversal

</div>

This behaves exactly like `walk`, except that it yields a 4-tuple `(dirpath, dirnames, filenames, dirfd)`, and it supports `dir_fd`.

*dirpath*, *dirnames* and *filenames* are identical to `walk` output, and *dirfd* is a file descriptor referring to the directory *dirpath*.

This function always supports `paths relative to directory descriptors
<dir_fd>` and `not following symlinks <follow_symlinks>`. Note however that, unlike other functions, the `fwalk` default value for *follow_symlinks* is `False`.

> [!NOTE]
> Since `fwalk` yields file descriptors, those are only valid until the next iteration step, so you should duplicate them (e.g. with `dup`) if you want to keep them longer.

This example displays the number of bytes taken by non-directory files in each directory under the starting directory, except that it doesn't look under any `__pycache__` subdirectory:

    import os
    for root, dirs, files, rootfd in os.fwalk('python/Lib/xml'):
        print(root, "consumes", end=" ")
        print(sum([os.stat(name, dir_fd=rootfd).st_size for name in files]),
              end=" ")
        print("bytes in", len(files), "non-directory files")
        if '__pycache__' in dirs:
            dirs.remove('__pycache__')  # don't visit __pycache__ directories

In the next example, walking the tree bottom-up is essential: `rmdir` doesn't allow deleting a directory before the directory is empty:

    # Delete everything reachable from the directory named in "top",
    # assuming there are no symbolic links.
    # CAUTION:  This is dangerous!  For example, if top == '/', it
    # could delete all your disk files.
    import os
    for root, dirs, files, rootfd in os.fwalk(top, topdown=False):
        for name in files:
            os.unlink(name, dir_fd=rootfd)
        for name in dirs:
            os.rmdir(name, dir_fd=rootfd)

<div class="audit-event">

os.fwalk top,topdown,onerror,follow_symlinks,dir_fd os.fwalk

</div>

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.7 Added support for `bytes` paths.

</div>

</div>

<div class="function">

memfd_create(name\[, flags=os.MFD_CLOEXEC\])

Create an anonymous file and return a file descriptor that refers to it. *flags* must be one of the `os.MFD_*` constants available on the system (or a bitwise ORed combination of them). By default, the new file descriptor is `non-inheritable <fd_inheritance>`.

The name supplied in *name* is used as a filename and will be displayed as the target of the corresponding symbolic link in the directory `/proc/self/fd/`. The displayed name is always prefixed with `memfd:` and serves only for debugging purposes. Names do not affect the behavior of the file descriptor, and as such multiple files can have the same name without any side effects.

<div class="availability">

Linux \>= 3.17.

</div>

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.16 The function is now also available when Python is built against a libc that lacks `memfd_create()`, such as glibc older than 2.27.

</div>

</div>

<div class="data">

MFD_CLOEXEC MFD_ALLOW_SEALING MFD_HUGETLB MFD_HUGE_SHIFT MFD_HUGE_MASK MFD_HUGE_64KB MFD_HUGE_512KB MFD_HUGE_1MB MFD_HUGE_2MB MFD_HUGE_8MB MFD_HUGE_16MB MFD_HUGE_32MB MFD_HUGE_256MB MFD_HUGE_512MB MFD_HUGE_1GB MFD_HUGE_2GB MFD_HUGE_16GB

These flags can be passed to `memfd_create`.

<div class="availability">

Linux \>= 3.17 with glibc \>= 2.27

The `MFD_HUGE*` flags are only available since Linux 4.14.

</div>

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

eventfd(initval\[, flags=os.EFD_CLOEXEC\])

Create and return an event file descriptor. The file descriptors supports raw `read` and `write` with a buffer size of 8, `~select.select`, `~select.poll` and similar. See man page `eventfd(2)` for more information. By default, the new file descriptor is `non-inheritable <fd_inheritance>`.

*initval* is the initial value of the event counter. The initial value must be a 32 bit unsigned integer. Please note that the initial value is limited to a 32 bit unsigned int although the event counter is an unsigned 64 bit integer with a maximum value of 2<sup>64</sup>-2.

*flags* can be constructed from `EFD_CLOEXEC`, `EFD_NONBLOCK`, and `EFD_SEMAPHORE`.

If `EFD_SEMAPHORE` is specified and the event counter is non-zero, `eventfd_read` returns 1 and decrements the counter by one.

If `EFD_SEMAPHORE` is not specified and the event counter is non-zero, `eventfd_read` returns the current event counter value and resets the counter to zero.

If the event counter is zero and `EFD_NONBLOCK` is not specified, `eventfd_read` blocks.

`eventfd_write` increments the event counter. Write blocks if the write operation would increment the counter to a value larger than 2<sup>64</sup>-2.

Example:

    import os

    # semaphore with start value '1'
    fd = os.eventfd(1, os.EFD_SEMAPHORE | os.EFD_CLOEXEC)
    try:
        # acquire semaphore
        v = os.eventfd_read(fd)
        try:
            do_work()
        finally:
            # release semaphore
            os.eventfd_write(fd, v)
    finally:
        os.close(fd)

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

eventfd_read(fd)

Read value from an `eventfd` file descriptor and return a 64 bit unsigned int. The function does not verify that *fd* is an `eventfd`.

<div class="availability">

Linux \>= 2.6.27

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

eventfd_write(fd, value)

Add value to an `eventfd` file descriptor. *value* must be a 64 bit unsigned int. The function does not verify that *fd* is an `eventfd`.

<div class="availability">

Linux \>= 2.6.27

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

EFD_CLOEXEC

Set close-on-exec flag for new `eventfd` file descriptor.

<div class="availability">

Linux \>= 2.6.27

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

EFD_NONBLOCK

Set `O_NONBLOCK` status flag for new `eventfd` file descriptor.

<div class="availability">

Linux \>= 2.6.27

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

EFD_SEMAPHORE

Provide semaphore-like semantics for reads from an `eventfd` file descriptor. On read the internal counter is decremented by one.

<div class="availability">

Linux \>= 2.6.30

</div>

<div class="versionadded">

3.10

</div>

</div>

### Timer File Descriptors

<div class="versionadded">

3.13

</div>

These functions provide support for Linux's *timer file descriptor* API. Naturally, they are all only available on Linux.

<div class="function">

timerfd_create(clockid, /, \*, flags=0)

Create and return a timer file descriptor (*timerfd*).

The file descriptor returned by `timerfd_create` supports:

- `read`
- `~select.select`
- `~select.poll`

The file descriptor's `read` method can be called with a buffer size of 8. If the timer has already expired one or more times, `read` returns the number of expirations with the host's endianness, which may be converted to an `int` by `int.from_bytes(x, byteorder=sys.byteorder)`.

`~select.select` and `~select.poll` can be used to wait until timer expires and the file descriptor is readable.

*clockid* must be a valid `clock ID <time-clock-id-constants>`, as defined in the `time` module:

- `time.CLOCK_REALTIME`
- `time.CLOCK_MONOTONIC`
- `time.CLOCK_BOOTTIME` (Since Linux 3.15 for timerfd_create)

If *clockid* is `time.CLOCK_REALTIME`, a settable system-wide real-time clock is used. If the system clock is changed, the timer setting needs to be updated. To cancel the timer when the system clock is changed, see `TFD_TIMER_CANCEL_ON_SET`.

If *clockid* is `time.CLOCK_MONOTONIC`, a non-settable monotonically increasing clock is used. Even if the system clock is changed, the timer setting will not be affected.

If *clockid* is `time.CLOCK_BOOTTIME`, it is the same as `time.CLOCK_MONOTONIC` except it includes any time that the system is suspended.

The file descriptor's behaviour can be modified by specifying a *flags* value. Any of the following variables may be used, combined using bitwise OR (the `|` operator):

- `TFD_NONBLOCK`
- `TFD_CLOEXEC`

If `TFD_NONBLOCK` is not set as a flag, `read` blocks until the timer expires. If it is set as a flag, `read` doesn't block, but if there hasn't been an expiration since the last call to read, `read` raises `OSError` with `errno` set to `errno.EAGAIN`.

`TFD_CLOEXEC` is always set by Python automatically.

The file descriptor must be closed with `os.close` when it is no longer needed, or else the file descriptor will be leaked.

<div class="seealso">

The `timerfd_create(2)` man page.

</div>

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

timerfd_settime(fd, /, \*, flags=0, initial=0.0, interval=0.0)

Alter a timer file descriptor's internal timer. This function operates the same interval timer as `timerfd_settime_ns`.

*fd* must be a valid timer file descriptor.

The timer's behaviour can be modified by specifying a *flags* value. Any of the following variables may be used, combined using bitwise OR (the `|` operator):

- `TFD_TIMER_ABSTIME`
- `TFD_TIMER_CANCEL_ON_SET`

The timer is disabled by setting *initial* to zero (`0`). If *initial* is greater than zero, the timer is enabled. If *initial* is less than zero, it raises an `OSError` exception with `errno` set to `errno.EINVAL`.

By default the timer will fire when *initial* seconds have elapsed.

However, if the `TFD_TIMER_ABSTIME` flag is set, the timer will fire when the timer's clock (set by *clockid* in `timerfd_create`) reaches *initial* seconds.

The timer's interval is set by the *interval* real number. If *interval* is zero, the timer only fires once, on the initial expiration. If *interval* is greater than zero, the timer fires every time *interval* seconds have elapsed since the previous expiration. If *interval* is less than zero, it raises `OSError` with `errno` set to `errno.EINVAL`.

If the `TFD_TIMER_CANCEL_ON_SET` flag is set along with `TFD_TIMER_ABSTIME` and the clock for this timer is `time.CLOCK_REALTIME`, the timer is marked as cancelable if the real-time clock is changed discontinuously. Reading the descriptor is aborted with the error `errno.ECANCELED`.

Linux manages system clock as UTC. A daylight-savings time transition is done by changing time offset only and doesn't cause discontinuous system clock change.

Discontinuous system clock change will be caused by the following events:

- `settimeofday`
- `clock_settime`
- set the system date and time by `date` command

Return a two-item tuple of (`next_expiration`, `interval`) from the previous timer state, before this function executed.

<div class="seealso">

`timerfd_create(2)`, `timerfd_settime(2)`, `settimeofday(2)`, `clock_settime(2)`, and `date(1)`.

</div>

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

timerfd_settime_ns(fd, /, \*, flags=0, initial=0, interval=0)

Similar to `timerfd_settime`, but use time as nanoseconds. This function operates the same interval timer as `timerfd_settime`.

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

timerfd_gettime(fd, /)

Return a two-item tuple of floats (`next_expiration`, `interval`).

`next_expiration` denotes the relative time until the timer next fires, regardless of if the `TFD_TIMER_ABSTIME` flag is set.

`interval` denotes the timer's interval. If zero, the timer will only fire once, after `next_expiration` seconds have elapsed.

<div class="seealso">

`timerfd_gettime(2)`

</div>

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

timerfd_gettime_ns(fd, /)

Similar to `timerfd_gettime`, but return time as nanoseconds.

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

TFD_NONBLOCK

A flag for the `timerfd_create` function, which sets the `O_NONBLOCK` status flag for the new timer file descriptor. If `TFD_NONBLOCK` is not set as a flag, `read` blocks.

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

TFD_CLOEXEC

A flag for the `timerfd_create` function, If `TFD_CLOEXEC` is set as a flag, set close-on-exec flag for new file descriptor.

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

TFD_TIMER_ABSTIME

A flag for the `timerfd_settime` and `timerfd_settime_ns` functions. If this flag is set, *initial* is interpreted as an absolute value on the timer's clock (in UTC seconds or nanoseconds since the Unix Epoch).

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

TFD_TIMER_CANCEL_ON_SET

A flag for the `timerfd_settime` and `timerfd_settime_ns` functions along with `TFD_TIMER_ABSTIME`. The timer is cancelled when the time of the underlying clock changes discontinuously.

<div class="availability">

Linux \>= 2.6.27 with glibc \>= 2.8

</div>

<div class="versionadded">

3.13

</div>

</div>

### Linux extended attributes

<div class="versionadded">

3.3

</div>

These functions are all available on Linux only.

<div class="function">

getxattr(path, attribute, \*, follow_symlinks=True)

Return the value of the extended filesystem attribute *attribute* for *path*. *attribute* can be bytes or str (directly or indirectly through the `PathLike` interface). If it is str, it is encoded with the filesystem encoding.

This function can support `specifying a file descriptor <path_fd>` and `not following symlinks <follow_symlinks>`.

<div class="audit-event">

os.getxattr path,attribute os.getxattr

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object` for *path* and *attribute*.

</div>

</div>

<div class="function">

listxattr(path=None, \*, follow_symlinks=True)

Return a list of the extended filesystem attributes on *path*. The attributes in the list are represented as strings decoded with the filesystem encoding. If *path* is `None`, `listxattr` will examine the current directory.

This function can support `specifying a file descriptor <path_fd>` and `not following symlinks <follow_symlinks>`.

<div class="audit-event">

os.listxattr path os.listxattr

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.15 `os.listxattr(-1)` now fails with `OSError(errno.EBADF)` rather than listing extended attributes of the current directory.

</div>

</div>

<div class="function">

removexattr(path, attribute, \*, follow_symlinks=True)

Removes the extended filesystem attribute *attribute* from *path*. *attribute* should be bytes or str (directly or indirectly through the `PathLike` interface). If it is a string, it is encoded with the `filesystem encoding and error handler`.

This function can support `specifying a file descriptor <path_fd>` and `not following symlinks <follow_symlinks>`.

<div class="audit-event">

os.removexattr path,attribute os.removexattr

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object` for *path* and *attribute*.

</div>

</div>

<div class="function">

setxattr(path, attribute, value, flags=0, \*, follow_symlinks=True)

Set the extended filesystem attribute *attribute* on *path* to *value*. *attribute* must be a bytes or str with no embedded NULs (directly or indirectly through the `PathLike` interface). If it is a str, it is encoded with the `filesystem encoding and error handler`. *flags* may be `XATTR_REPLACE` or `XATTR_CREATE`. If `XATTR_REPLACE` is given and the attribute does not exist, `ENODATA` will be raised. If `XATTR_CREATE` is given and the attribute already exists, the attribute will not be created and `EEXISTS` will be raised.

This function can support `specifying a file descriptor <path_fd>` and `not following symlinks <follow_symlinks>`.

> [!NOTE]
> A bug in Linux kernel versions less than 2.6.39 caused the flags argument to be ignored on some filesystems.

<div class="audit-event">

os.setxattr path,attribute,value,flags os.setxattr

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object` for *path* and *attribute*.

</div>

</div>

<div class="data">

XATTR_SIZE_MAX

The maximum size the value of an extended attribute can be. Currently, this is 64 KiB on Linux.

</div>

<div class="data">

XATTR_CREATE

This is a possible value for the flags argument in `setxattr`. It indicates the operation must create an attribute.

</div>

<div class="data">

XATTR_REPLACE

This is a possible value for the flags argument in `setxattr`. It indicates the operation must replace an existing attribute.

</div>

## Process Management

These functions may be used to create and manage processes.

The various `exec\* <execl>` functions take a list of arguments for the new program loaded into the process. In each case, the first of these arguments is passed to the new program as its own name rather than as an argument a user may have typed on a command line. For the C programmer, this is the `argv[0]` passed to a program's `main`. For example, `os.execv('/bin/echo', ['foo', 'bar'])` will only print `bar` on standard output; `foo` will seem to be ignored.

<div class="function">

abort()

Generate a `~signal.SIGABRT` signal to the current process. On Unix, the default behavior is to produce a core dump; on Windows, the process immediately returns an exit code of `3`. Be aware that calling this function will not call the Python signal handler registered for `~signal.SIGABRT` with `signal.signal`.

</div>

<div class="function">

add_dll_directory(path)

Add a path to the DLL search path.

This search path is used when resolving dependencies for imported extension modules (the module itself is resolved through `sys.path`), and also by `ctypes`.

Remove the directory by calling **close()** on the returned object or using it in a `with` statement.

See the [Microsoft documentation](https://msdn.microsoft.com/44228cf2-6306-466c-8f16-f513cd3ba8b5) for more information about how DLLs are loaded.

<div class="audit-event">

os.add_dll_directory path os.add_dll_directory

</div>

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.8 Previous versions of CPython would resolve DLLs using the default behavior for the current process. This led to inconsistencies, such as only sometimes searching `PATH` or the current working directory, and OS functions such as `AddDllDirectory` having no effect.

In 3.8, the two primary ways DLLs are loaded now explicitly override the process-wide behavior to ensure consistency. See the `porting notes <bpo-36085-whatsnew>` for information on updating libraries.

</div>

</div>

<div class="function">

execl(path, arg0, arg1, ...) execle(path, arg0, arg1, ..., env) execlp(file, arg0, arg1, ...) execlpe(file, arg0, arg1, ..., env) execv(path, args) execve(path, args, env) execvp(file, args) execvpe(file, args, env)

These functions all execute a new program, replacing the current process; they do not return. On Unix, the new executable is loaded into the current process, and will have the same process id as the caller. Errors will be reported as `OSError` exceptions.

The current process is replaced immediately. Open file objects and descriptors are not flushed, so if there may be data buffered on these open files, you should flush them using `~io.IOBase.flush` or `os.fsync` before calling an `exec\* <execl>` function.

The "l" and "v" variants of the `exec\* <execl>` functions differ in how command-line arguments are passed. The "l" variants are perhaps the easiest to work with if the number of parameters is fixed when the code is written; the individual parameters simply become additional parameters to the `!execl\*` functions. The "v" variants are good when the number of parameters is variable, with the arguments being passed in a list or tuple as the *args* parameter. In either case, the arguments to the child process should start with the name of the command being run, but this is not enforced.

The variants which include a "p" near the end (`execlp`, `execlpe`, `execvp`, and `execvpe`) will use the `PATH` environment variable to locate the program *file*. When the environment is being replaced (using one of the `exec\*e <execl>` variants, discussed in the next paragraph), the new environment is used as the source of the `PATH` variable. The other variants, `execl`, `execle`, `execv`, and `execve`, will not use the `PATH` variable to locate the executable; *path* must contain an appropriate absolute or relative path. Relative paths must include at least one slash, even on Windows, as plain names will not be resolved.

For `execle`, `execlpe`, `execve`, and `execvpe` (note that these all end in "e"), the *env* parameter must be a mapping which is used to define the environment variables for the new process (these are used instead of the current process' environment); the functions `execl`, `execlp`, `execv`, and `execvp` all cause the new process to inherit the environment of the current process.

For `execve` on some platforms, *path* may also be specified as an open file descriptor. This functionality may not be supported on your platform; you can check whether or not it is available using `os.supports_fd`. If it is unavailable, using it will raise a `NotImplementedError`.

<div class="audit-event">

os.exec path,args,env os.execl

</div>

<div class="availability">

Unix, Windows, not WASI, not Android, not iOS.

</div>

<div class="versionchanged">

3.3 Added support for specifying *path* as an open file descriptor for `execve`.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

<span id="exit">exit</span>(n)

Exit the process with status *n*, without calling cleanup handlers, flushing stdio buffers, etc.

> [!NOTE]
> The standard way to exit is `sys.exit(n) <sys.exit>`. `!_exit` should normally only be used in the child process after a `fork`.

</div>

The following exit codes are defined and can be used with `_exit`, although they are not required. These are typically used for system programs written in Python, such as a mail server's external command delivery program.

> [!NOTE]
> Some of these may not be available on all Unix platforms, since there is some variation. These constants are defined where they are defined by the underlying platform.

<div class="data">

EX_OK

Exit code that means no error occurred. May be taken from the defined value of `EXIT_SUCCESS` on some platforms. Generally has a value of zero.

<div class="availability">

Unix, Windows.

</div>

</div>

<div class="data">

EX_USAGE

Exit code that means the command was used incorrectly, such as when the wrong number of arguments are given.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_DATAERR

Exit code that means the input data was incorrect.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_NOINPUT

Exit code that means an input file did not exist or was not readable.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_NOUSER

Exit code that means a specified user did not exist.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_NOHOST

Exit code that means a specified host did not exist.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_UNAVAILABLE

Exit code that means that a required service is unavailable.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_SOFTWARE

Exit code that means an internal software error was detected.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_OSERR

Exit code that means an operating system error was detected, such as the inability to fork or create a pipe.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_OSFILE

Exit code that means some system file did not exist, could not be opened, or had some other kind of error.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_CANTCREAT

Exit code that means a user specified output file could not be created.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_IOERR

Exit code that means that an error occurred while doing I/O on some file.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_TEMPFAIL

Exit code that means a temporary failure occurred. This indicates something that may not really be an error, such as a network connection that couldn't be made during a retryable operation.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_PROTOCOL

Exit code that means that a protocol exchange was illegal, invalid, or not understood.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_NOPERM

Exit code that means that there were insufficient permissions to perform the operation (but not intended for file system problems).

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_CONFIG

Exit code that means that some kind of configuration error occurred.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="data">

EX_NOTFOUND

Exit code that means something like "an entry was not found".

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

fork()

Fork a child process. Return `0` in the child and the child's process id in the parent. If an error occurs `OSError` is raised.

Note that some platforms including FreeBSD \<= 6.3 and Cygwin have known issues when using `fork()` from a thread.

<div class="audit-event">

os.fork "" os.fork

</div>

> [!WARNING]
> If you use TLS sockets in an application calling `fork()`, see the warning in the `ssl` documentation.

> [!WARNING]
> On macOS the use of this function is unsafe when mixed with using higher-level system APIs, and that includes using `urllib.request`.

<div class="versionchanged">

3.8 Calling `fork()` in a subinterpreter is no longer supported (`RuntimeError` is raised).

</div>

<div class="versionchanged">

3.12 If Python is able to detect that your process has multiple threads, `os.fork` now raises a `DeprecationWarning`.

We chose to surface this as a warning, when detectable, to better inform developers of a design problem that the POSIX platform specifically notes as not supported. Even in code that *appears* to work, it has never been safe to mix threading with `os.fork` on POSIX platforms. The CPython runtime itself has always made API calls that are not safe for use in the child process when threads existed in the parent (such as `malloc` and `free`).

Users of macOS or users of libc or malloc implementations other than those typically found in glibc to date are among those already more likely to experience deadlocks running such code.

See [this discussion on fork being incompatible with threads](https://discuss.python.org/t/33555) for technical details of why we're surfacing this longstanding platform compatibility problem to developers.

</div>

<div class="availability">

POSIX, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

forkpty()

Fork a child process, using a new pseudo-terminal as the child's controlling terminal. Return a pair of `(pid, fd)`, where *pid* is `0` in the child, the new child's process id in the parent, and *fd* is the file descriptor of the master end of the pseudo-terminal. For a more portable approach, use the `pty` module. If an error occurs `OSError` is raised.

The returned file descriptor *fd* is `non-inheritable <fd_inheritance>`.

<div class="audit-event">

os.forkpty "" os.forkpty

</div>

> [!WARNING]
> On macOS the use of this function is unsafe when mixed with using higher-level system APIs, and that includes using `urllib.request`.

<div class="versionchanged">

3.8 Calling `forkpty()` in a subinterpreter is no longer supported (`RuntimeError` is raised).

</div>

<div class="versionchanged">

3.12 If Python is able to detect that your process has multiple threads, this now raises a `DeprecationWarning`. See the longer explanation on `os.fork`.

</div>

<div class="versionchanged">

3.15 The returned file descriptor is now made non-inheritable.

</div>

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

kill(pid, sig, /)

<div class="index">

single: process; killing single: process; signalling

</div>

Send signal *sig* to the process *pid*. Constants for the specific signals available on the host platform are defined in the `signal` module.

Windows: The `signal.CTRL_C_EVENT` and `signal.CTRL_BREAK_EVENT` signals are special signals which can only be sent to console processes which share a common console window, e.g., some subprocesses. Any other value for *sig* will cause the process to be unconditionally killed by the TerminateProcess API, and the exit code will be set to *sig*.

See also `signal.pthread_kill`.

<div class="audit-event">

os.kill pid,sig os.kill

</div>

<div class="availability">

Unix, Windows, not WASI, not iOS.

</div>

<div class="versionchanged">

3.2 Added Windows support.

</div>

</div>

<div class="function">

killpg(pgid, sig, /)

<div class="index">

single: process; killing single: process; signalling

</div>

Send the signal *sig* to the process group *pgid*.

<div class="audit-event">

os.killpg pgid,sig os.killpg

</div>

<div class="availability">

Unix, not WASI, not iOS.

</div>

</div>

<div class="function">

nice(increment, /)

Add *increment* to the process's "niceness". Return the new niceness.

<div class="availability">

Unix, not WASI.

</div>

</div>

<div class="function">

pidfd_open(pid, flags=0)

Return a file descriptor referring to the process *pid* with *flags* set. This descriptor can be used to perform process management without races and signals.

See the `pidfd_open(2)` man page for more details.

<div class="availability">

Linux \>= 5.3, Android \>= `build-time <sys.getandroidapilevel>` API level 31

</div>

<div class="versionadded">

3.9

</div>

<div class="data">

PIDFD_NONBLOCK

This flag indicates that the file descriptor will be non-blocking. If the process referred to by the file descriptor has not yet terminated, then an attempt to wait on the file descriptor using `waitid(2)` will immediately return the error `~errno.EAGAIN` rather than blocking.

</div>

<div class="availability">

Linux \>= 5.10

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

pidfd_getfd(pidfd, targetfd, \*, flags=0)

Duplicate *targetfd* from the process referred to by the process file descriptor *pidfd*, into the calling process. The returned file descriptor is `non-inheritable <fd_inheritance>`.

*flags* is reserved, and currently must be `0`.

See the `pidfd_getfd(2)` man page for more details.

<div class="availability">

Linux \>= 5.6, Android \>= `build-time <sys.getandroidapilevel>` API level 31

</div>

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

plock(op, /)

Lock program segments into memory. The value of *op* (defined in `<sys/lock.h>`) determines which segments are locked.

<div class="availability">

Unix, not WASI, not macOS, not iOS.

</div>

</div>

<div class="function">

popen(cmd, mode='r', buffering=-1)

Open a pipe to or from command *cmd*. The return value is an open file object connected to the pipe, which can be read or written depending on whether *mode* is `'r'` (default) or `'w'`. The *buffering* argument have the same meaning as the corresponding argument to the built-in `open` function. The returned file object reads or writes text strings rather than bytes.

The `close` method returns `None` if the subprocess exited successfully, or the subprocess's return code if there was an error. On POSIX systems, if the return code is positive it represents the return value of the process left-shifted by one byte. If the return code is negative, the process was terminated by the signal given by the negated value of the return code. (For example, the return value might be `- signal.SIGKILL` if the subprocess was killed.) On Windows systems, the return value contains the signed integer return code from the child process.

On Unix, `waitstatus_to_exitcode` can be used to convert the `close` method result (exit status) into an exit code if it is not `None`. On Windows, the `close` method result is directly the exit code (or `None`).

This is implemented using `subprocess.Popen`; see that class's documentation for more powerful ways to manage and communicate with subprocesses.

<div class="availability">

not WASI, not Android, not iOS.

</div>

> [!NOTE]
> The `Python UTF-8 Mode <utf8-mode>` affects encodings used for *cmd* and pipe contents.
>
> `popen` is a simple wrapper around `subprocess.Popen`. Use `subprocess.Popen` or `subprocess.run` to control options like encodings.

<div class="soft-deprecated">

3.14 The `subprocess` module is recommended instead.

</div>

</div>

<div class="function">

posix_spawn(path, argv, env, \*, file_actions=None, setpgroup=None, resetids=False, setsid=False, setsigmask=(), setsigdef=(), scheduler=None)

Wraps the `!posix_spawn` C library API for use from Python.

Most users should use `subprocess.run` instead of `posix_spawn`.

The positional-only arguments *path*, *args*, and *env* are similar to `execve`. *env* is allowed to be `None`, in which case current process' environment is used.

The *path* parameter is the path to the executable file. The *path* should contain a directory. Use `posix_spawnp` to pass an executable file without directory.

The *file_actions* argument may be a sequence of tuples describing actions to take on specific file descriptors in the child process between the C library implementation's `fork` and `exec` steps. The first item in each tuple must be one of the three type indicator listed below describing the remaining tuple elements:

<div class="data">

POSIX_SPAWN_OPEN

(`os.POSIX_SPAWN_OPEN`, *fd*, *path*, *flags*, *mode*)

Performs `os.dup2(os.open(path, flags, mode), fd)`.

</div>

<div class="data">

POSIX_SPAWN_CLOSE

(`os.POSIX_SPAWN_CLOSE`, *fd*)

Performs `os.close(fd)`.

</div>

<div class="data">

POSIX_SPAWN_DUP2

(`os.POSIX_SPAWN_DUP2`, *fd*, *new_fd*)

Performs `os.dup2(fd, new_fd)`.

</div>

<div class="data">

POSIX_SPAWN_CLOSEFROM

(`os.POSIX_SPAWN_CLOSEFROM`, *fd*)

Performs `os.closerange(fd, INF)`.

</div>

These tuples correspond to the C library `!posix_spawn_file_actions_addopen`, `!posix_spawn_file_actions_addclose`, `!posix_spawn_file_actions_adddup2`, and `!posix_spawn_file_actions_addclosefrom_np` API calls used to prepare for the `!posix_spawn` call itself.

The *setpgroup* argument will set the process group of the child to the value specified. If the value specified is 0, the child's process group ID will be made the same as its process ID. If the value of *setpgroup* is not set, the child will inherit the parent's process group ID. This argument corresponds to the C library `!POSIX_SPAWN_SETPGROUP` flag.

If the *resetids* argument is `True` it will reset the effective UID and GID of the child to the real UID and GID of the parent process. If the argument is `False`, then the child retains the effective UID and GID of the parent. In either case, if the set-user-ID and set-group-ID permission bits are enabled on the executable file, their effect will override the setting of the effective UID and GID. This argument corresponds to the C library `!POSIX_SPAWN_RESETIDS` flag.

If the *setsid* argument is `True`, it will create a new session ID for `posix_spawn`. *setsid* requires `!POSIX_SPAWN_SETSID` or `!POSIX_SPAWN_SETSID_NP` flag. Otherwise, `NotImplementedError` is raised.

The *setsigmask* argument will set the signal mask to the signal set specified. If the parameter is not used, then the child inherits the parent's signal mask. This argument corresponds to the C library `!POSIX_SPAWN_SETSIGMASK` flag.

The *sigdef* argument will reset the disposition of all signals in the set specified. This argument corresponds to the C library `!POSIX_SPAWN_SETSIGDEF` flag.

The *scheduler* argument must be a tuple containing the (optional) scheduler policy and an instance of `sched_param` with the scheduler parameters. A value of `None` in the place of the scheduler policy indicates that is not being provided. This argument is a combination of the C library `!POSIX_SPAWN_SETSCHEDPARAM` and `!POSIX_SPAWN_SETSCHEDULER` flags.

<div class="audit-event">

os.posix_spawn path,argv,env os.posix_spawn

</div>

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.13 *env* parameter accepts `None`. `os.POSIX_SPAWN_CLOSEFROM` is available on platforms where `!posix_spawn_file_actions_addclosefrom_np` exists.

</div>

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

posix_spawnp(path, argv, env, \*, file_actions=None, setpgroup=None, resetids=False, setsid=False, setsigmask=(), setsigdef=(), scheduler=None)

Wraps the `!posix_spawnp` C library API for use from Python.

Similar to `posix_spawn` except that the system searches for the *executable* file in the list of directories specified by the `PATH` environment variable (in the same way as for `execvp(3)`).

<div class="audit-event">

os.posix_spawn path,argv,env os.posix_spawnp

</div>

<div class="versionadded">

3.8

</div>

<div class="availability">

POSIX, not WASI, not Android, not iOS.

See `posix_spawn` documentation.

</div>

</div>

<div class="function">

register_at_fork(\*, before=None, after_in_parent=None, after_in_child=None)

Register callables to be executed when a new child process is forked using `os.fork` or similar process cloning APIs. The parameters are optional and keyword-only. Each specifies a different call point.

- *before* is a function called before forking a child process.
- *after_in_parent* is a function called from the parent process after forking a child process.
- *after_in_child* is a function called from the child process.

These calls are only made if control is expected to return to the Python interpreter. A typical `subprocess` launch will not trigger them as the child is not going to re-enter the interpreter.

Functions registered for execution before forking are called in reverse registration order. Functions registered for execution after forking (either in the parent or in the child) are called in registration order.

Note that `fork` calls made by third-party C code may not call those functions, unless it explicitly calls `PyOS_BeforeFork`, `PyOS_AfterFork_Parent` and `PyOS_AfterFork_Child`.

There is no way to unregister a function.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

spawnl(mode, path, ...) spawnle(mode, path, ..., env) spawnlp(mode, file, ...) spawnlpe(mode, file, ..., env) spawnv(mode, path, args) spawnve(mode, path, args, env) spawnvp(mode, file, args) spawnvpe(mode, file, args, env)

Execute the program *path* in a new process.

(Note that the `subprocess` module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using these functions. Check especially the `subprocess-replacements` section.)

If *mode* is `P_NOWAIT`, this function returns the process id of the new process; if *mode* is `P_WAIT`, returns the process's exit code if it exits normally, or `-signal`, where *signal* is the signal that killed the process. On Windows, the process id will actually be the process handle, so can be used with the `waitpid` function.

Note on VxWorks, this function doesn't return `-signal` when the new process is killed. Instead it raises OSError exception.

The "l" and "v" variants of the `spawn\* <spawnl>` functions differ in how command-line arguments are passed. The "l" variants are perhaps the easiest to work with if the number of parameters is fixed when the code is written; the individual parameters simply become additional parameters to the `!spawnl\*` functions. The "v" variants are good when the number of parameters is variable, with the arguments being passed in a list or tuple as the *args* parameter. In either case, the arguments to the child process must start with the name of the command being run.

The variants which include a second "p" near the end (`spawnlp`, `spawnlpe`, `spawnvp`, and `spawnvpe`) will use the `PATH` environment variable to locate the program *file*. When the environment is being replaced (using one of the `spawn\*e <spawnl>` variants, discussed in the next paragraph), the new environment is used as the source of the `PATH` variable. The other variants, `spawnl`, `spawnle`, `spawnv`, and `spawnve`, will not use the `PATH` variable to locate the executable; *path* must contain an appropriate absolute or relative path.

For `spawnle`, `spawnlpe`, `spawnve`, and `spawnvpe` (note that these all end in "e"), the *env* parameter must be a mapping which is used to define the environment variables for the new process (they are used instead of the current process' environment); the functions `spawnl`, `spawnlp`, `spawnv`, and `spawnvp` all cause the new process to inherit the environment of the current process. Note that keys and values in the *env* dictionary must be strings; invalid keys or values will cause the function to fail, with a return value of `127`.

As an example, the following calls to `spawnlp` and `spawnvpe` are equivalent:

    import os
    os.spawnlp(os.P_WAIT, 'cp', 'cp', 'index.html', '/dev/null')

    L = ['cp', 'index.html', '/dev/null']
    os.spawnvpe(os.P_WAIT, 'cp', L, os.environ)

<div class="audit-event">

os.spawn mode,path,args,env os.spawnl

</div>

<div class="availability">

Unix, Windows, not WASI, not Android, not iOS.

`spawnlp`, `spawnlpe`, `spawnvp` and `spawnvpe` are not available on Windows. `spawnle` and `spawnve` are not thread-safe on Windows; we advise you to use the `subprocess` module instead.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="soft-deprecated">

3.14 The `subprocess` module is recommended instead.

</div>

</div>

<div class="data">

P_NOWAIT P_NOWAITO

Possible values for the *mode* parameter to the `spawn\* <spawnl>` family of functions. If either of these values is given, the `spawn\* <spawnl>` functions will return as soon as the new process has been created, with the process id as the return value.

<div class="availability">

Unix, Windows.

</div>

</div>

<div class="data">

P_WAIT

Possible value for the *mode* parameter to the `spawn\* <spawnl>` family of functions. If this is given as *mode*, the `spawn\* <spawnl>` functions will not return until the new process has run to completion and will return the exit code of the process the run is successful, or `-signal` if a signal kills the process.

<div class="availability">

Unix, Windows.

</div>

</div>

<div class="data">

P_DETACH P_OVERLAY

Possible values for the *mode* parameter to the `spawn\* <spawnl>` family of functions. These are less portable than those listed above. `P_DETACH` is similar to `P_NOWAIT`, but the new process is detached from the console of the calling process. If `P_OVERLAY` is used, the current process will be replaced; the `spawn\* <spawnl>` function will not return.

<div class="availability">

Windows.

</div>

</div>

<div class="function">

startfile(path, \[operation\], \[arguments\], \[cwd\], \[show_cmd\])

Start a file with its associated application.

When *operation* is not specified, this acts like double-clicking the file in Windows Explorer, or giving the file name as an argument to the `start` command from the interactive command shell: the file is opened with whatever application (if any) its extension is associated.

When another *operation* is given, it must be a "command verb" that specifies what should be done with the file. Common verbs documented by Microsoft are `'open'`, `'print'` and `'edit'` (to be used on files) as well as `'explore'` and `'find'` (to be used on directories).

When launching an application, specify *arguments* to be passed as a single string. This argument may have no effect when using this function to launch a document.

The default working directory is inherited, but may be overridden by the *cwd* argument. This should be an absolute path. A relative *path* will be resolved against this argument.

Use *show_cmd* to override the default window style. Whether this has any effect will depend on the application being launched. Values are integers as supported by the Win32 `!ShellExecute` function.

`startfile` returns as soon as the associated application is launched. There is no option to wait for the application to close, and no way to retrieve the application's exit status. The *path* parameter is relative to the current directory or *cwd*. If you want to use an absolute path, make sure the first character is not a slash (`'/'`) Use `pathlib` or the `os.path.normpath` function to ensure that paths are properly encoded for Win32.

To reduce interpreter startup overhead, the Win32 `!ShellExecute` function is not resolved until this function is first called. If the function cannot be resolved, `NotImplementedError` will be raised.

<div class="audit-event">

os.startfile path,operation os.startfile

</div>

<div class="audit-event">

os.startfile/2 path,operation,arguments,cwd,show_cmd os.startfile

</div>

<div class="availability">

Windows.

</div>

<div class="versionchanged">

3.10 Added the *arguments*, *cwd* and *show_cmd* arguments, and the `os.startfile/2` audit event.

</div>

</div>

<div class="function">

system(command)

Execute the command (a string) in a subshell. This is implemented by calling the Standard C function `system`, and has the same limitations. Changes to `sys.stdin`, etc. are not reflected in the environment of the executed command. If *command* generates any output, it will be sent to the interpreter standard output stream. The C standard does not specify the meaning of the return value of the C function, so the return value of the Python function is system-dependent.

On Unix, the return value is the exit status of the process encoded in the format specified for `wait`.

On Windows, the return value is that returned by the system shell after running *command*. The shell is given by the Windows environment variable `COMSPEC`: it is usually `cmd.exe`, which returns the exit status of the command run; on systems using a non-native shell, consult your shell documentation.

The `subprocess` module provides more powerful facilities for spawning new processes and retrieving their results; using that module is recommended to using this function. See the `subprocess-replacements` section in the `subprocess` documentation for some helpful recipes.

On Unix, `waitstatus_to_exitcode` can be used to convert the result (exit status) into an exit code. On Windows, the result is directly the exit code.

<div class="audit-event">

os.system command os.system

</div>

<div class="availability">

Unix, Windows, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

times()

Returns the current global process times. The return value is an object with five attributes:

- `!user` - user time
- `!system` - system time
- `!children_user` - user time of all child processes
- `!children_system` - system time of all child processes
- `!elapsed` - elapsed real time since a fixed point in the past

For backwards compatibility, this object also behaves like a five-tuple containing `!user`, `!system`, `!children_user`, `!children_system`, and `!elapsed` in that order.

See the Unix manual page `times(2)` and [times(3)](https://man.freebsd.org/cgi/man.cgi?time(3)) manual page on Unix or [the GetProcessTimes MSDN](https://docs.microsoft.com/windows/win32/api/processthreadsapi/nf-processthreadsapi-getprocesstimes) on Windows. On Windows, only `!user` and `!system` are known; the other attributes are zero.

<div class="availability">

Unix, Windows.

</div>

<div class="versionchanged">

3.3 Return type changed from a tuple to a tuple-like object with named attributes.

</div>

</div>

<div class="function">

wait()

Wait for completion of a child process, and return a tuple containing its pid and exit status indication: a 16-bit number, whose low byte is the signal number that killed the process, and whose high byte is the exit status (if the signal number is zero); the high bit of the low byte is set if a core file was produced.

If there are no children that could be waited for, `ChildProcessError` is raised.

`waitstatus_to_exitcode` can be used to convert the exit status into an exit code.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

<div class="seealso">

The other `!wait*` functions documented below can be used to wait for the completion of a specific child process and have more options. `waitpid` is the only one also available on Windows.

</div>

</div>

<div class="function">

waitid(idtype, id, options, /)

Wait for the completion of a child process.

*idtype* can be `P_PID`, `P_PGID`, `P_ALL`, or (on Linux) `P_PIDFD`. The interpretation of *id* depends on it; see their individual descriptions.

*options* is an OR combination of flags. At least one of `WEXITED`, `WSTOPPED` or `WCONTINUED` is required; `WNOHANG` and `WNOWAIT` are additional optional flags.

The return value is an object representing the data contained in the `siginfo_t` structure with the following attributes:

- `!si_pid` (process ID)
- `!si_uid` (real user ID of the child)
- `!si_signo` (always `~signal.SIGCHLD`)
- `!si_status` (the exit status or signal number, depending on `!si_code`)
- `!si_code` (see `CLD_EXITED` for possible values)

If `WNOHANG` is specified and there are no matching children in the requested state, `None` is returned. Otherwise, if there are no matching children that could be waited for, `ChildProcessError` is raised.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.13 This function is now available on macOS as well.

</div>

</div>

<div class="function">

waitpid(pid, options, /)

The details of this function differ on Unix and Windows.

On Unix: Wait for completion of a child process given by process id *pid*, and return a tuple containing its process id and exit status indication (encoded as for `wait`). The semantics of the call are affected by the value of the integer *options*, which should be `0` for normal operation.

If *pid* is greater than `0`, `waitpid` requests status information for that specific process. If *pid* is `0`, the request is for the status of any child in the process group of the current process. If *pid* is `-1`, the request pertains to any child of the current process. If *pid* is less than `-1`, status is requested for any process in the process group `-pid` (the absolute value of *pid*).

*options* is an OR combination of flags. If it contains `WNOHANG` and there are no matching children in the requested state, `(0, 0)` is returned. Otherwise, if there are no matching children that could be waited for, `ChildProcessError` is raised. Other options that can be used are `WUNTRACED` and `WCONTINUED`.

On Windows: Wait for completion of a process given by process handle *pid*, and return a tuple containing *pid*, and its exit status shifted left by 8 bits (shifting makes cross-platform use of the function easier). A *pid* less than or equal to `0` has no special meaning on Windows, and raises an exception. The value of integer *options* has no effect. *pid* can refer to any process whose id is known, not necessarily a child process. The `spawn\* <spawnl>` functions called with `P_NOWAIT` return suitable process handles.

`waitstatus_to_exitcode` can be used to convert the exit status into an exit code.

<div class="availability">

Unix, Windows, not WASI, not Android, not iOS.

</div>

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the function now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="function">

wait3(options)

Similar to `waitpid`, except no process id argument is given and a 3-element tuple containing the child's process id, exit status indication, and resource usage information is returned. Refer to `resource.getrusage` for details on resource usage information. The *options* argument is the same as that provided to `waitpid` and `wait4`.

`waitstatus_to_exitcode` can be used to convert the exit status into an exitcode.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

wait4(pid, options)

Similar to `waitpid`, except a 3-element tuple, containing the child's process id, exit status indication, and resource usage information is returned. Refer to `resource.getrusage` for details on resource usage information. The arguments to `wait4` are the same as those provided to `waitpid`.

`waitstatus_to_exitcode` can be used to convert the exit status into an exitcode.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="data">

P_PID P_PGID P_ALL P_PIDFD

These are the possible values for *idtype* in `waitid`. They affect how *id* is interpreted:

- `!P_PID` - wait for the child whose PID is *id*.
- `!P_PGID` - wait for any child whose progress group ID is *id*.
- `!P_ALL` - wait for any child; *id* is ignored.
- `!P_PIDFD` - wait for the child identified by the file descriptor *id* (a process file descriptor created with `pidfd_open`).

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

> [!NOTE]
> `!P_PIDFD` is only available on Linux \>= 5.4.

<div class="versionadded">

3.3

</div>

<div class="versionadded">

3.9 The `!P_PIDFD` constant.

</div>

</div>

<div class="data">

WCONTINUED

This *options* flag for `waitpid`, `wait3`, `wait4`, and `waitid` causes child processes to be reported if they have been continued from a job control stop since they were last reported.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="data">

WEXITED

This *options* flag for `waitid` causes child processes that have terminated to be reported.

The other `wait*` functions always report children that have terminated, so this option is not available for them.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

WSTOPPED

This *options* flag for `waitid` causes child processes that have been stopped by the delivery of a signal to be reported.

This option is not available for the other `wait*` functions.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

WUNTRACED

This *options* flag for `waitpid`, `wait3`, and `wait4` causes child processes to also be reported if they have been stopped but their current state has not been reported since they were stopped.

This option is not available for `waitid`.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="data">

WNOHANG

This *options* flag causes `waitpid`, `wait3`, `wait4`, and `waitid` to return right away if no child process status is available immediately.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="data">

WNOWAIT

This *options* flag causes `waitid` to leave the child in a waitable state, so that a later `!wait*` call can be used to retrieve the child status information again.

This option is not available for the other `wait*` functions.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="data">

CLD_EXITED CLD_KILLED CLD_DUMPED CLD_TRAPPED CLD_STOPPED CLD_CONTINUED

These are the possible values for `!si_code` in the result returned by `waitid`.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.9 Added `CLD_KILLED` and `CLD_STOPPED` values.

</div>

</div>

<div class="function">

waitstatus_to_exitcode(status)

Convert a wait status to an exit code.

On Unix:

- If the process exited normally (if `WIFEXITED(status)` is true), return the process exit status (return `WEXITSTATUS(status)`): result greater than or equal to 0.
- If the process was terminated by a signal (if `WIFSIGNALED(status)` is true), return `-signum` where *signum* is the number of the signal that caused the process to terminate (return `-WTERMSIG(status)`): result less than 0.
- Otherwise, raise a `ValueError`.

On Windows, return *status* shifted right by 8 bits.

On Unix, if the process is being traced or if `waitpid` was called with `WUNTRACED` option, the caller must first check if `WIFSTOPPED(status)` is true. This function must not be called if `WIFSTOPPED(status)` is true.

<div class="seealso">

`WIFEXITED`, `WEXITSTATUS`, `WIFSIGNALED`, `WTERMSIG`, `WIFSTOPPED`, `WSTOPSIG` functions.

</div>

<div class="availability">

Unix, Windows, not WASI, not Android, not iOS.

</div>

<div class="versionadded">

3.9

</div>

</div>

The following functions take a process status code as returned by `system`, `wait`, or `waitpid` as a parameter. They may be used to determine the disposition of a process.

<div class="function">

WCOREDUMP(status, /)

Return `True` if a core dump was generated for the process, otherwise return `False`.

This function should be employed only if `WIFSIGNALED` is true.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

WIFCONTINUED(status)

Return `True` if a stopped child has been resumed by delivery of `~signal.SIGCONT` (if the process has been continued from a job control stop), otherwise return `False`.

See `WCONTINUED` option.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

WIFSTOPPED(status)

Return `True` if the process was stopped by delivery of a signal, otherwise return `False`.

`WIFSTOPPED` only returns `True` if the `waitpid` call was done using `WUNTRACED` option or when the process is being traced (see `ptrace(2)`).

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

WIFSIGNALED(status)

Return `True` if the process was terminated by a signal, otherwise return `False`.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

WIFEXITED(status)

Return `True` if the process exited terminated normally, that is, by calling `exit()` or `_exit()`, or by returning from `main()`; otherwise return `False`.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

WEXITSTATUS(status)

Return the process exit status.

This function should be employed only if `WIFEXITED` is true.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

WSTOPSIG(status)

Return the signal which caused the process to stop.

This function should be employed only if `WIFSTOPPED` is true.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

<div class="function">

WTERMSIG(status)

Return the number of the signal that caused the process to terminate.

This function should be employed only if `WIFSIGNALED` is true.

<div class="availability">

Unix, not WASI, not Android, not iOS.

</div>

</div>

## Interface to the scheduler

These functions control how a process is allocated CPU time by the operating system. They are only available on some Unix platforms. For more detailed information, consult your Unix manpages.

<div class="versionadded">

3.3

</div>

The following scheduling policies are exposed if they are supported by the operating system.

<div id="os-scheduling-policy">

<div class="data">

SCHED_OTHER

The default scheduling policy.

</div>

</div>

<div class="data">

SCHED_BATCH

Scheduling policy for CPU-intensive processes that tries to preserve interactivity on the rest of the computer.

</div>

<div class="data">

SCHED_DEADLINE

Scheduling policy for tasks with deadline constraints.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

SCHED_IDLE

Scheduling policy for extremely low priority background tasks.

</div>

<div class="data">

SCHED_NORMAL

Alias for `SCHED_OTHER`.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

SCHED_SPORADIC

Scheduling policy for sporadic server programs.

</div>

<div class="data">

SCHED_FIFO

A First In First Out scheduling policy.

</div>

<div class="data">

SCHED_RR

A round-robin scheduling policy.

</div>

<div class="data">

SCHED_RESET_ON_FORK

This flag can be OR'ed with any other scheduling policy. When a process with this flag set forks, its child's scheduling policy and priority are reset to the default.

</div>

<div class="sched_param(sched_priority)">

This class represents tunable scheduling parameters used in `sched_setparam`, `sched_setscheduler`, and `sched_getparam`. It is immutable.

At the moment, there is only one possible parameter:

<div class="attribute">

sched_priority

The scheduling priority for a scheduling policy.

</div>

</div>

<div class="function">

sched_get_priority_min(policy)

Get the minimum priority value for *policy*. *policy* is one of the scheduling policy constants above.

</div>

<div class="function">

sched_get_priority_max(policy)

Get the maximum priority value for *policy*. *policy* is one of the scheduling policy constants above.

</div>

<div class="function">

sched_setscheduler(pid, policy, param, /)

Set the scheduling policy for the process with PID *pid*. A *pid* of 0 means the calling process. *policy* is one of the scheduling policy constants above. *param* is a `sched_param` instance.

</div>

<div class="function">

sched_getscheduler(pid, /)

Return the scheduling policy for the process with PID *pid*. A *pid* of 0 means the calling process. The result is one of the scheduling policy constants above.

</div>

<div class="function">

sched_setparam(pid, param, /)

Set the scheduling parameters for the process with PID *pid*. A *pid* of 0 means the calling process. *param* is a `sched_param` instance.

</div>

<div class="function">

sched_getparam(pid, /)

Return the scheduling parameters as a `sched_param` instance for the process with PID *pid*. A *pid* of 0 means the calling process.

</div>

<div class="function">

sched_rr_get_interval(pid, /)

Return the round-robin quantum in seconds for the process with PID *pid*. A *pid* of 0 means the calling process.

</div>

<div class="function">

sched_yield()

Voluntarily relinquish the CPU. See `sched_yield(2)` for details.

</div>

<div class="function">

sched_setaffinity(pid, mask, /)

Restrict the process with PID *pid* (or the current process if zero) to a set of CPUs. *mask* is an iterable of integers representing the set of CPUs to which the process should be restricted.

</div>

<div class="function">

sched_getaffinity(pid, /)

Return the set of CPUs the process with PID *pid* is restricted to.

If *pid* is zero, return the set of CPUs the calling thread of the current process is restricted to.

See also the `process_cpu_count` function.

</div>

## Miscellaneous System Information

<div class="function">

confstr(name, /)

Return string-valued system configuration values. *name* specifies the configuration value to retrieve; it may be a string which is the name of a defined system value; these names are specified in a number of standards (POSIX, Unix 95, Unix 98, and others). Some platforms define additional names as well. The names known to the host operating system are given as the keys of the `confstr_names` dictionary. For configuration variables not included in that mapping, passing an integer for *name* is also accepted.

If the configuration value specified by *name* isn't defined, `None` is returned.

If *name* is a string and is not known, `ValueError` is raised. If a specific value for *name* is not supported by the host system, even if it is included in `confstr_names`, an `OSError` is raised with `errno.EINVAL` for the error number.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

confstr_names

Dictionary mapping names accepted by `confstr` to the integer values defined for those names by the host operating system. This can be used to determine the set of names known to the system.

<div class="availability">

Unix.

</div>

</div>

<div class="function">

cpu_count()

Return the number of logical CPUs in the **system**. Returns `None` if undetermined.

The `process_cpu_count` function can be used to get the number of logical CPUs usable by the calling thread of the **current process**.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.13 If `-X cpu_count <-X>` is given or `PYTHON_CPU_COUNT` is set, `cpu_count` returns the override value *n*.

</div>

</div>

<div class="function">

getloadavg()

Return the number of processes in the system run queue averaged over the last 1, 5, and 15 minutes or raises `OSError` if the load average was unobtainable.

<div class="availability">

Unix.

</div>

</div>

<div class="function">

process_cpu_count()

Get the number of logical CPUs usable by the calling thread of the **current process**. Returns `None` if undetermined. It can be less than `cpu_count` depending on the CPU affinity.

The `cpu_count` function can be used to get the number of logical CPUs in the **system**.

If `-X cpu_count <-X>` is given or `PYTHON_CPU_COUNT` is set, `process_cpu_count` returns the override value *n*.

See also the `sched_getaffinity` function.

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

sysconf(name, /)

Return integer-valued system configuration values. If the configuration value specified by *name* isn't defined, `-1` is returned. The comments regarding the *name* parameter for `confstr` apply here as well; the dictionary that provides information on the known names is given by `sysconf_names`.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

sysconf_names

Dictionary mapping names accepted by `sysconf` to the integer values defined for those names by the host operating system. This can be used to determine the set of names known to the system.

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.11 Add `'SC_MINSIGSTKSZ'` name.

</div>

</div>

The following data values are used to support path manipulation operations. These are defined for all platforms.

Higher-level operations on pathnames are defined in the `os.path` module.

<div class="index">

single: . (dot); in pathnames

</div>

<div class="data">

curdir

The constant string used by the operating system to refer to the current directory. This is `'.'` for Windows and POSIX. Also available via `os.path`.

</div>

<div class="index">

single: ..; in pathnames

</div>

<div class="data">

pardir

The constant string used by the operating system to refer to the parent directory. This is `'..'` for Windows and POSIX. Also available via `os.path`.

</div>

<div class="index">

single: / (slash); in pathnames

</div>

<div class="index">

single: (backslash); in pathnames (Windows)

</div>

<div class="data">

sep

The character used by the operating system to separate pathname components. This is `'/'` for POSIX and `'\\'` for Windows. Note that knowing this is not sufficient to be able to parse or concatenate pathnames --- use `os.path.split` and `os.path.join` --- but it is occasionally useful. Also available via `os.path`.

</div>

<div class="index">

single: / (slash); in pathnames

</div>

<div class="data">

altsep

An alternative character used by the operating system to separate pathname components, or `None` if only one separator character exists. This is set to `'/'` on Windows systems where `sep` is a backslash. Also available via `os.path`.

</div>

<div class="index">

single: . (dot); in pathnames

</div>

<div class="data">

extsep

The character which separates the base filename from the extension; for example, the `'.'` in `os.py`. Also available via `os.path`.

</div>

<div class="index">

single: : (colon); path separator (POSIX) single: ; (semicolon)

</div>

<div class="data">

pathsep

The character conventionally used by the operating system to separate search path components (as in `PATH`), such as `':'` for POSIX or `';'` for Windows. Also available via `os.path`.

</div>

<div class="data">

defpath

The default search path used by `exec\*p\* <execl>` and `spawn\*p\* <spawnl>` if the environment doesn't have a `'PATH'` key. Also available via `os.path`.

</div>

<div class="data">

linesep

The string used to separate (or, rather, terminate) lines on the current platform. This may be a single character, such as `'\n'` for POSIX, or multiple characters, for example, `'\r\n'` for Windows. Do not use *os.linesep* as a line terminator when writing files opened in text mode (the default); use a single `'\n'` instead, on all platforms.

</div>

<div class="data">

devnull

The file path of the null device. For example: `'/dev/null'` for POSIX, `'nul'` for Windows. Also available via `os.path`.

</div>

<div class="data">

RTLD_LAZY RTLD_NOW RTLD_GLOBAL RTLD_LOCAL RTLD_NODELETE RTLD_NOLOAD RTLD_DEEPBIND

Flags for use with the `~sys.setdlopenflags` and `~sys.getdlopenflags` functions. See the Unix manual page `dlopen(3)` for what the different flags mean.

<div class="versionadded">

3.3

</div>

</div>

## Random numbers

<div class="function">

getrandom(size, flags=0)

Get up to *size* random bytes. The function can return less bytes than requested.

These bytes can be used to seed user-space random number generators or for cryptographic purposes.

`getrandom()` relies on entropy gathered from device drivers and other sources of environmental noise. Unnecessarily reading large quantities of data will have a negative impact on other users of the `/dev/random` and `/dev/urandom` devices.

The flags argument is a bit mask that can contain zero or more of the following values ORed together: `os.GRND_RANDOM` and `GRND_NONBLOCK`.

See also the [Linux getrandom() manual page](https://man7.org/linux/man-pages/man2/getrandom.2.html).

<div class="availability">

Linux \>= 3.17.

</div>

<div class="versionadded">

3.6

</div>

</div>

<div class="function">

urandom(size, /)

Return a bytestring of *size* random bytes suitable for cryptographic use.

This function returns random bytes from an OS-specific randomness source. The returned data should be unpredictable enough for cryptographic applications, though its exact quality depends on the OS implementation.

On Linux, if the `getrandom()` syscall is available, it is used in blocking mode: block until the system urandom entropy pool is initialized (128 bits of entropy are collected by the kernel). See the `524` for the rationale. On Linux, the `getrandom` function can be used to get random bytes in non-blocking mode (using the `GRND_NONBLOCK` flag) or to poll until the system urandom entropy pool is initialized.

On a Unix-like system, random bytes are read from the `/dev/urandom` device. If the `/dev/urandom` device is not available or not readable, the `NotImplementedError` exception is raised.

On Windows, it will use `BCryptGenRandom()`.

<div class="seealso">

The `secrets` module provides higher level functions. For an easy-to-use interface to the random number generator provided by your platform, please see `random.SystemRandom`.

</div>

<div class="versionchanged">

3.5 On Linux 3.17 and newer, the `getrandom()` syscall is now used when available. On OpenBSD 5.6 and newer, the C `getentropy()` function is now used. These functions avoid the usage of an internal file descriptor.

</div>

<div class="versionchanged">

3.5.2 On Linux, if the `getrandom()` syscall blocks (the urandom entropy pool is not initialized yet), fall back on reading `/dev/urandom`.

</div>

<div class="versionchanged">

3.6 On Linux, `getrandom()` is now used in blocking mode to increase the security.

</div>

<div class="versionchanged">

3.11 On Windows, `BCryptGenRandom()` is used instead of `CryptGenRandom()` which is deprecated.

</div>

</div>

<div class="data">

GRND_NONBLOCK

By default, when reading from `/dev/random`, `getrandom` blocks if no random bytes are available, and when reading from `/dev/urandom`, it blocks if the entropy pool has not yet been initialized.

If the `GRND_NONBLOCK` flag is set, then `getrandom` does not block in these cases, but instead immediately raises `BlockingIOError`.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

GRND_RANDOM

If this bit is set, then random bytes are drawn from the `/dev/random` pool instead of the `/dev/urandom` pool.

<div class="versionadded">

3.6

</div>

</div>

# `!os.path` --- Common pathname manipulations

<div class="module" data-synopsis="Operations on pathnames.">

os.path

</div>

**Source code:** `Lib/genericpath.py`, `Lib/posixpath.py` (for POSIX) and `Lib/ntpath.py` (for Windows).

<div class="index">

single: path; operations

</div>

------------------------------------------------------------------------

This module implements some useful functions on pathnames. To read or write files see `open`, and for accessing the filesystem see the `os` module. The path parameters can be passed as strings, or bytes, or any object implementing the `os.PathLike` protocol.

Unlike a Unix shell, Python does not do any *automatic* path expansions. Functions such as `expanduser` and `expandvars` can be invoked explicitly when an application desires shell-like path expansion. (See also the `glob` module.)

<div class="seealso">

The `pathlib` module offers high-level path objects.

</div>

> [!NOTE]
> All of these functions accept either only bytes or only string objects as their parameters. The result is an object of the same type, if a path or file name is returned.

> [!NOTE]
> Since different operating systems have different path name conventions, there are several versions of this module in the standard library. The `!os.path` module is always the path module suitable for the operating system Python is running on, and therefore usable for local paths. However, you can also import and use the individual modules if you want to manipulate a path that is *always* in one of the different formats. They all have the same interface:
>
> - `!posixpath` for UNIX-style paths
> - `!ntpath` for Windows paths

<div class="versionchanged">

3.8

`exists`, `lexists`, `isdir`, `isfile`, `islink`, and `ismount` now return `False` instead of raising an exception for paths that contain characters or bytes unrepresentable at the OS level.

</div>

<div class="function">

abspath(path)

Return a normalized absolutized version of the pathname *path*. On most platforms, this is equivalent to calling `normpath(join(os.getcwd(), path))`.

On Windows the path is normalized by the operating system, therefore the result can differ from `normpath(join(os.getcwd(), path))`. A drive-relative path is resolved against the current directory of the specified drive, and the drive letter is capitalized. Trailing dots and spaces are stripped. For example:

    >>> os.path.abspath('c:spam')
    'C:\\Temp\\spam'
    >>> os.path.abspath('c:/temp/spam. . .')
    'c:\\temp\\spam'

<div class="seealso">

`os.path.join` and `os.path.normpath`.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

basename(path, /)

Return the base name of pathname *path*. This is the second element of the pair returned by passing *path* to the function `split`. Note that the result of this function is different from the Unix `basename` program; where `basename` for `'/foo/bar/'` returns `'bar'`, the `basename` function returns an empty string (`''`).

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

commonpath(paths)

Return the longest common sub-path of each pathname in the iterable *paths*. Raise `ValueError` if *paths* contain both absolute and relative pathnames, if *paths* are on different drives, or if *paths* is empty. Unlike `commonprefix`, this returns a valid path.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.6 Accepts a sequence of `path-like objects <path-like object>`.

</div>

<div class="versionchanged">

3.13 Any iterable can now be passed, rather than just sequences.

</div>

</div>

<div class="function">

commonprefix(list, /)

Return the longest string prefix (taken character-by-character) that is a prefix of all strings in *list*. If *list* is empty, return the empty string (`''`).

> [!WARNING]
> This function may return invalid paths because it works a character at a time. If you need a **common path prefix**, then the algorithm implemented in this function is not secure. Use `commonpath` for finding a common path prefix.
>
>     >>> os.path.commonprefix(['/usr/lib', '/usr/local/lib'])
>     '/usr/l'
>
>     >>> os.path.commonpath(['/usr/lib', '/usr/local/lib'])
>     '/usr'

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="deprecated">

3.15 Deprecated in favor of `os.path.commonpath` for path prefixes. The `os.path.commonprefix` function is being deprecated due to having a misleading name and module. The function is not safe to use for path prefixes despite being included in a module about path manipulation, meaning it is easy to accidentally introduce path traversal vulnerabilities into Python programs by using this function.

</div>

</div>

<div class="function">

dirname(path, /)

Return the directory name of pathname *path*. This is the first element of the pair returned by passing *path* to the function `split`.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

exists(path)

Return `True` if *path* refers to an existing path or an open file descriptor. Returns `False` for broken symbolic links. On some platforms, this function may return `False` if permission is not granted to execute `os.stat` on the requested file, even if the *path* physically exists.

<div class="versionchanged">

3.3 *path* can now be an integer: `True` is returned if it is an open file descriptor, `False` otherwise.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

lexists(path)

Return `True` if *path* refers to an existing path, including broken symbolic links. Equivalent to `exists` on platforms lacking `os.lstat`.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="index">

single: ~ (tilde); home directory expansion

</div>

<div class="function">

expanduser(path)

On Unix and Windows, return the argument with an initial component of `~` or `~user` replaced by that *user*'s home directory.

<div class="index">

pair: module; pwd

</div>

On Unix, an initial `~` is replaced by the environment variable `HOME` if it is set; otherwise the current user's home directory is looked up in the password directory through the built-in module `pwd`. An initial `~user` is looked up directly in the password directory.

On Windows, `USERPROFILE` will be used if set, otherwise a combination of `HOMEPATH` and `HOMEDRIVE` will be used. An initial `~user` is handled by checking that the last directory component of the current user's home directory matches `USERNAME`, and replacing it if so.

If the expansion fails or if the path does not begin with a tilde, the path is returned unchanged.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.8 No longer uses `HOME` on Windows.

</div>

</div>

<div class="index">

single: \$ (dollar); environment variables expansion single: % (percent); environment variables expansion (Windows)

</div>

<div class="function">

expandvars(path)

Return the argument with environment variables expanded. Substrings of the form `$name` or `${name}` are replaced by the value of environment variable *name*. Malformed variable names and references to non-existing variables are left unchanged.

On Windows, `%name%` expansions are supported in addition to `$name` and `${name}`.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

getatime(path, /)

Return the time of last access of *path*. The return value is a floating-point number giving the number of seconds since the epoch (see the `time` module). Raise `OSError` if the file does not exist or is inaccessible.

</div>

<div class="function">

getmtime(path, /)

Return the time of last modification of *path*. The return value is a floating-point number giving the number of seconds since the epoch (see the `time` module). Raise `OSError` if the file does not exist or is inaccessible.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

getctime(path, /)

Return the system's ctime which, on some systems (like Unix) is the time of the last metadata change, and, on others (like Windows), is the creation time for *path*. The return value is a number giving the number of seconds since the epoch (see the `time` module). Raise `OSError` if the file does not exist or is inaccessible.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

getsize(path, /)

Return the size, in bytes, of *path*. Raise `OSError` if the file does not exist or is inaccessible.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

isabs(path, /)

Return `True` if *path* is an absolute pathname. On Unix, that means it begins with a slash, on Windows that it begins with two (back)slashes, or a drive letter, colon, and (back)slash together.

<div class="seealso">

`abspath`

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.13 On Windows, returns `False` if the given path starts with exactly one (back)slash.

</div>

</div>

<div class="function">

isfile(path)

Return `True` if *path* is an `existing <exists>` regular file. This follows symbolic links, so both `islink` and `isfile` can be true for the same path.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

isdir(path, /)

Return `True` if *path* is an `existing <exists>` directory. This follows symbolic links, so both `islink` and `isdir` can be true for the same path.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

isjunction(path)

Return `True` if *path* refers to an `existing <lexists>` directory entry that is a junction. Always return `False` if junctions are not supported on the current platform.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

islink(path)

Return `True` if *path* refers to an `existing <exists>` directory entry that is a symbolic link. Always `False` if symbolic links are not supported by the Python runtime.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

ismount(path)

Return `True` if pathname *path* is a `mount point`: a point in a file system where a different file system has been mounted. On POSIX, the function checks whether *path*'s parent, `{path}/..`, is on a different device than *path*, or whether `{path}/..` and *path* point to the same i-node on the same device --- this should detect mount points for all Unix and POSIX variants. It is not able to reliably detect bind mounts on the same filesystem. On Linux systems, it will always return `True` for btrfs subvolumes, even if they aren't mount points. On Windows, a drive letter root and a share UNC are always mount points, and for any other path `GetVolumePathName` is called to see if it is different from the input path.

<div class="versionchanged">

3.4 Added support for detecting non-root mount points on Windows.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

isdevdrive(path)

Return `True` if pathname *path* is located on a Windows Dev Drive. A Dev Drive is optimized for developer scenarios, and offers faster performance for reading and writing files. It is recommended for use for source code, temporary build directories, package caches, and other IO-intensive operations.

May raise an error for an invalid path, for example, one without a recognizable drive, but returns `False` on platforms that do not support Dev Drives. See [the Windows documentation](https://learn.microsoft.com/windows/dev-drive/) for information on enabling and creating Dev Drives.

<div class="versionadded">

3.12

</div>

<div class="versionchanged">

3.13 The function is now available on all platforms, and will always return `False` on those that have no support for Dev Drives

</div>

</div>

<div class="function">

isreserved(path)

Return `True` if *path* is a reserved pathname on the current system.

On Windows, reserved filenames include those that end with a space or dot; those that contain colons (i.e. file streams such as "name:stream"), wildcard characters (i.e. `'*?"<>'`), pipe, or ASCII control characters; as well as DOS device names such as "NUL", "CON", "CONIN\$", "CONOUT\$", "AUX", "PRN", "COM1", and "LPT1".

> [!NOTE]
> This function approximates rules for reserved paths on most Windows systems. These rules change over time in various Windows releases. This function may be updated in future Python releases as changes to the rules become broadly available.

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

join(path, /, \*paths)

Join one or more path segments intelligently. The return value is the concatenation of *path* and all members of *\*paths*, with exactly one directory separator following each non-empty part, except the last. That is, the result will only end in a separator if the last part is either empty or ends in a separator.

If a segment is an absolute path (which on Windows requires both a drive and a root), then all previous segments are ignored and joining continues from the absolute path segment. On Linux, for example:

    >>> os.path.join('/home/foo', 'bar')
    '/home/foo/bar'
    >>> os.path.join('/home/foo', '/home/bar')
    '/home/bar'

On Windows, the drive is not reset when a rooted path segment (e.g., `r'\foo'`) is encountered. If a segment is on a different drive or is an absolute path, all previous segments are ignored and the drive is reset. For example:

    >>> os.path.join('c:\\', 'foo')
    'c:\\foo'
    >>> os.path.join('c:\\foo', 'd:\\bar')
    'd:\\bar'

Note that since there is a current directory for each drive, `os.path.join("c:", "foo")` represents a path relative to the current directory on drive `C:` (`c:foo`), not `c:\\foo`.

<div class="versionchanged">

3.6 Accepts a `path-like object` for *path* and *paths*.

</div>

</div>

<div class="function">

normcase(path, /)

Normalize the case of a pathname. On Windows, convert all characters in the pathname to lowercase, and also convert forward slashes to backward slashes. On other operating systems, return the path unchanged.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

normpath(path)

Normalize a pathname by collapsing redundant separators and up-level references so that `A//B`, `A/B/`, `A/./B` and `A/foo/../B` all become `A/B`. This string manipulation may change the meaning of a path that contains symbolic links. On Windows, it converts forward slashes to backward slashes. To normalize case, use `normcase`.

> [!NOTE]
> On POSIX systems, in accordance with [IEEE Std 1003.1 2013 Edition; 4.13 Pathname Resolution](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_13), if a pathname begins with exactly two slashes, the first component following the leading characters may be interpreted in an implementation-defined manner, although more than two leading characters shall be treated as a single character.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

realpath(path, /, \*, strict=False)

Return the canonical path of the specified filename, eliminating any symbolic links encountered in the path (if they are supported by the operating system). On Windows, this function will also resolve MS-DOS (also called 8.3) style names such as `C:\\PROGRA~1` to `C:\\Program Files`. The returned path uses the case reported by the operating system, which can differ from the case of *path*, in particular the drive letter is capitalized.

By default, the path is evaluated up to the first component that does not exist, is a symlink loop, or whose evaluation raises `OSError`. All such components are appended unchanged to the existing part of the path.

Some errors that are handled this way include "access denied", "not a directory", or "bad argument to internal function". Thus, the resulting path may be missing or inaccessible, may still contain links or loops, and may traverse non-directories.

This behavior can be modified by keyword arguments:

If *strict* is `True`, the first error encountered when evaluating the path is re-raised. In particular, `FileNotFoundError` is raised if *path* does not exist, or another `OSError` if it is otherwise inaccessible. If *strict* is `ALL_BUT_LAST`, the last component of the path is allowed to be missing, but all other errors are raised.

If *strict* is `os.path.ALLOW_MISSING`, errors other than `FileNotFoundError` are re-raised (as with `strict=True`). Thus, the returned path will not contain any symbolic links, but the named file and some of its parent directories may be missing.

> [!NOTE]
> This function emulates the operating system's procedure for making a path canonical, which differs slightly between Windows and UNIX with respect to how links and subsequent path components interact.
>
> Operating system APIs make paths canonical as needed, so it's not normally necessary to call this function.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

<div class="versionchanged">

3.8 Symbolic links and junctions are now resolved on Windows.

</div>

<div class="versionchanged">

3.10 The *strict* parameter was added.

</div>

<div class="versionchanged">

3.15 The `ALL_BUT_LAST` and `ALLOW_MISSING` values for the *strict* parameter was added.

</div>

</div>

<div class="data">

ALL_BUT_LAST

Special value used for the *strict* argument in `realpath`.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

ALLOW_MISSING

Special value used for the *strict* argument in `realpath`.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

relpath(path, start=os.curdir)

Return a relative filepath to *path* either from the current directory or from an optional *start* directory. This is a path computation: the filesystem is not accessed to confirm the existence or nature of *path* or *start*. On Windows, `ValueError` is raised when *path* and *start* are on different drives.

*start* defaults to `os.curdir`.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

samefile(path1, path2, /)

Return `True` if both pathname arguments refer to the same file or directory. This is determined by the device number and i-node number and raises an exception if an `os.stat` call on either pathname fails.

<div class="versionchanged">

3.2 Added Windows support.

</div>

<div class="versionchanged">

3.4 Windows now uses the same implementation as all other platforms.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

sameopenfile(fp1, fp2)

Return `True` if the file descriptors *fp1* and *fp2* refer to the same file.

<div class="versionchanged">

3.2 Added Windows support.

</div>

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

samestat(stat1, stat2, /)

Return `True` if the stat tuples *stat1* and *stat2* refer to the same file. These structures may have been returned by `os.fstat`, `os.lstat`, or `os.stat`. This function implements the underlying comparison used by `samefile` and `sameopenfile`.

<div class="versionchanged">

3.4 Added Windows support.

</div>

</div>

<div class="function">

split(path, /)

Split the pathname *path* into a pair, `(head, tail)` where *tail* is the last pathname component and *head* is everything leading up to that. The *tail* part will never contain a slash; if *path* ends in a slash, *tail* will be empty. If there is no slash in *path*, *head* will be empty. If *path* is empty, both *head* and *tail* are empty. Trailing slashes are stripped from *head* unless it is the root (one or more slashes only). In all cases, `join(head, tail)` returns a path to the same location as *path* (but the strings may differ). Also see the functions `join`, `dirname` and `basename`.

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

splitdrive(path, /)

Split the pathname *path* into a pair `(drive, tail)` where *drive* is either a mount point or the empty string. On systems which do not use drive specifications, *drive* will always be the empty string. In all cases, `drive + tail` will be the same as *path*.

On Windows, splits a pathname into drive/UNC sharepoint and relative path.

If the path contains a drive letter, drive will contain everything up to and including the colon:

    >>> splitdrive("c:/dir")
    ("c:", "/dir")

If the path contains a UNC path, drive will contain the host name and share:

    >>> splitdrive("//host/computer/dir")
    ("//host/computer", "/dir")

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="function">

splitroot(path, /)

Split the pathname *path* into a 3-item tuple `(drive, root, tail)` where *drive* is a device name or mount point, *root* is a string of separators after the drive, and *tail* is everything after the root. Any of these items may be the empty string. In all cases, `drive + root + tail` will be the same as *path*.

On POSIX systems, *drive* is always empty. The *root* may be empty (if *path* is relative), a single forward slash (if *path* is absolute), or two forward slashes (implementation-defined per [IEEE Std 1003.1-2017; 4.13 Pathname Resolution](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_13).) For example:

    >>> splitroot('/home/sam')
    ('', '/', 'home/sam')
    >>> splitroot('//home/sam')
    ('', '//', 'home/sam')
    >>> splitroot('///home/sam')
    ('', '/', '//home/sam')

On Windows, *drive* may be empty, a drive-letter name, a UNC share, or a device name. The *root* may be empty, a forward slash, or a backward slash. For example:

    >>> splitroot('C:/Users/Sam')
    ('C:', '/', 'Users/Sam')
    >>> splitroot('//Server/Share/Users/Sam')
    ('//Server/Share', '/', 'Users/Sam')

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

splitext(path, /)

Split the pathname *path* into a pair `(root, ext)` such that `root + ext == path`, and the extension, *ext*, is empty or begins with a period and contains at most one period.

If the path contains no extension, *ext* will be `''`:

    >>> splitext('bar')
    ('bar', '')

If the path contains an extension, then *ext* will be set to this extension, including the leading period. Note that previous periods will be ignored:

    >>> splitext('foo.bar.exe')
    ('foo.bar', '.exe')
    >>> splitext('/foo/bar.exe')
    ('/foo/bar', '.exe')

Leading periods of the last component of the path are considered to be part of the root:

    >>> splitext('.cshrc')
    ('.cshrc', '')
    >>> splitext('/foo/....jpg')
    ('/foo/....jpg', '')

<div class="versionchanged">

3.6 Accepts a `path-like object`.

</div>

</div>

<div class="data">

supports_unicode_filenames

`True` if arbitrary Unicode strings can be used as file names (within limitations imposed by the file system).

</div>

# `!ossaudiodev` --- Access to OSS-compatible audio devices

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

ossaudiodev

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!ossaudiodev` module was [Python 3.12](https://docs.python.org/3.12/library/ossaudiodev.html).

# `!pathlib` --- Object-oriented filesystem paths

<div class="module" data-synopsis="Object-oriented filesystem paths">

pathlib

</div>

<div class="versionadded">

3.4

</div>

**Source code:** `Lib/pathlib/`

<div class="index">

single: path; operations

</div>

------------------------------------------------------------------------

This module offers classes representing filesystem paths with semantics appropriate for different operating systems. Path classes are divided between `pure paths <pure-paths>`, which provide purely computational operations without I/O, and `concrete paths <concrete-paths>`, which inherit from pure paths but also provide I/O operations.

<img src="pathlib-inheritance.png" class="invert-in-dark-mode invert-in-dark-modealign-center" alt="Inheritance diagram showing the classes available in pathlib. The
most basic class is PurePath, which has three direct subclasses:
PurePosixPath, PureWindowsPath, and Path. Further to these four
classes, there are two classes that use multiple inheritance:
PosixPath subclasses PurePosixPath and Path, and WindowsPath
subclasses PureWindowsPath and Path." />

If you've never used this module before or just aren't sure which class is right for your task, `Path` is most likely what you need. It instantiates a `concrete path <concrete-paths>` for the platform the code is running on.

Pure paths are useful in some special cases; for example:

1.  If you want to manipulate Windows paths on a Unix machine (or vice versa). You cannot instantiate a `WindowsPath` when running on Unix, but you can instantiate `PureWindowsPath`.
2.  You want to make sure that your code only manipulates paths without actually accessing the OS. In this case, instantiating one of the pure classes may be useful since those simply don't have any OS-accessing operations.

<div class="seealso">

`428`: The pathlib module -- object-oriented filesystem paths.

</div>

<div class="seealso">

For low-level path manipulation on strings, you can also use the `os.path` module.

</div>

## Basic use

Importing the main class:

    >>> from pathlib import Path

Listing subdirectories:

    >>> p = Path('.')
    >>> [x for x in p.iterdir() if x.is_dir()]
    [PosixPath('.hg'), PosixPath('docs'), PosixPath('dist'),
     PosixPath('__pycache__'), PosixPath('build')]

Listing Python source files in this directory tree:

    >>> list(p.glob('**/*.py'))
    [PosixPath('test_pathlib.py'), PosixPath('setup.py'),
     PosixPath('pathlib.py'), PosixPath('docs/conf.py'),
     PosixPath('build/lib/pathlib.py')]

Navigating inside a directory tree:

    >>> p = Path('/etc')
    >>> q = p / 'init.d' / 'reboot'
    >>> q
    PosixPath('/etc/init.d/reboot')
    >>> q.resolve()
    PosixPath('/etc/rc.d/init.d/halt')

Querying path properties:

    >>> q.exists()
    True
    >>> q.is_dir()
    False

Opening a file:

    >>> with q.open() as f: f.readline()
    ...
    '#!/bin/bash\n'

## Exceptions

<div class="exception">

UnsupportedOperation

An exception inheriting `NotImplementedError` that is raised when an unsupported operation is called on a path object.

<div class="versionadded">

3.13

</div>

</div>

## Pure paths

Pure path objects provide path-handling operations which don't actually access a filesystem. There are three ways to access these classes, which we also call *flavours*:

<div class="PurePath(*pathsegments)">

A generic class that represents the system's path flavour (instantiating it creates either a `PurePosixPath` or a `PureWindowsPath`):

    >>> PurePath('setup.py')      # Running on a Unix machine
    PurePosixPath('setup.py')

Each element of *pathsegments* can be either a string representing a path segment, or an object implementing the `os.PathLike` interface where the `~os.PathLike.__fspath__` method returns a string, such as another path object:

    >>> PurePath('foo', 'some/path', 'bar')
    PurePosixPath('foo/some/path/bar')
    >>> PurePath(Path('foo'), Path('bar'))
    PurePosixPath('foo/bar')

When *pathsegments* is empty or consists only of empty strings, the current directory is assumed:

    >>> PurePath(), PurePath('')
    (PurePosixPath('.'), PurePosixPath('.'))

If a segment is an absolute path, all previous segments are ignored (like `os.path.join`):

    >>> PurePath('/etc', '/usr', 'lib64')
    PurePosixPath('/usr/lib64')
    >>> PureWindowsPath('c:/Windows', 'd:bar')
    PureWindowsPath('d:bar')

On Windows, the drive is not reset when a rooted relative path segment (e.g., `r'\foo'`) is encountered:

    >>> PureWindowsPath('c:/Windows', '/Program Files')
    PureWindowsPath('c:/Program Files')

Spurious slashes and single dots are collapsed, but double dots (`'..'`) and leading double slashes (`'//'`) are not, since this would change the meaning of a path for various reasons (e.g. symbolic links, UNC paths):

    >>> PurePath('foo//bar')
    PurePosixPath('foo/bar')
    >>> PurePath('//foo/bar')
    PurePosixPath('//foo/bar')
    >>> PurePath('foo/./bar')
    PurePosixPath('foo/bar')
    >>> PurePath('foo/../bar')
    PurePosixPath('foo/../bar')

(a naïve approach would make `PurePosixPath('foo/../bar')` equivalent to `PurePosixPath('bar')`, which is wrong if `foo` is a symbolic link to another directory)

Pure path objects implement the `os.PathLike` interface, allowing them to be used anywhere the interface is accepted.

<div class="versionchanged">

3.6 Added support for the `os.PathLike` interface.

</div>

</div>

<div class="PurePosixPath(*pathsegments)">

A subclass of `PurePath`, this path flavour represents non-Windows filesystem paths:

    >>> PurePosixPath('/etc/hosts')
    PurePosixPath('/etc/hosts')

*pathsegments* is specified similarly to `PurePath`.

</div>

<div class="PureWindowsPath(*pathsegments)">

A subclass of `PurePath`, this path flavour represents Windows filesystem paths, including [UNC paths](https://en.wikipedia.org/wiki/Path_(computing)#UNC):

    >>> PureWindowsPath('c:/', 'Users', 'Ximénez')
    PureWindowsPath('c:/Users/Ximénez')
    >>> PureWindowsPath('//server/share/file')
    PureWindowsPath('//server/share/file')

*pathsegments* is specified similarly to `PurePath`.

</div>

Regardless of the system you're running on, you can instantiate all of these classes, since they don't provide any operation that does system calls.

### General properties

Paths are immutable and `hashable`. Paths of a same flavour are comparable and orderable. These properties respect the flavour's case-folding semantics:

    >>> PurePosixPath('foo') == PurePosixPath('FOO')
    False
    >>> PureWindowsPath('foo') == PureWindowsPath('FOO')
    True
    >>> PureWindowsPath('FOO') in { PureWindowsPath('foo') }
    True
    >>> PureWindowsPath('C:') < PureWindowsPath('d:')
    True

Paths of a different flavour compare unequal and cannot be ordered:

    >>> PureWindowsPath('foo') == PurePosixPath('foo')
    False
    >>> PureWindowsPath('foo') < PurePosixPath('foo')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: '<' not supported between instances of 'PureWindowsPath' and 'PurePosixPath'

### Operators

The slash operator helps create child paths, like `os.path.join`. If the argument is an absolute path, the previous path is ignored. On Windows, the drive is not reset when the argument is a rooted relative path (e.g., `r'\foo'`):

    >>> p = PurePath('/etc')
    >>> p
    PurePosixPath('/etc')
    >>> p / 'init.d' / 'apache2'
    PurePosixPath('/etc/init.d/apache2')
    >>> q = PurePath('bin')
    >>> '/usr' / q
    PurePosixPath('/usr/bin')
    >>> p / '/an_absolute_path'
    PurePosixPath('/an_absolute_path')
    >>> PureWindowsPath('c:/Windows', '/Program Files')
    PureWindowsPath('c:/Program Files')

A path object can be used anywhere an object implementing `os.PathLike` is accepted:

    >>> import os
    >>> p = PurePath('/etc')
    >>> os.fspath(p)
    '/etc'

The string representation of a path is the raw filesystem path itself (in native form, e.g. with backslashes under Windows), which you can pass to any function taking a file path as a string:

    >>> p = PurePath('/etc')
    >>> str(p)
    '/etc'
    >>> p = PureWindowsPath('c:/Program Files')
    >>> str(p)
    'c:\\Program Files'

Similarly, calling `bytes` on a path gives the raw filesystem path as a bytes object, as encoded by `os.fsencode`:

    >>> bytes(p)
    b'/etc'

> [!NOTE]
> Calling `bytes` is only recommended under Unix. Under Windows, the unicode form is the canonical representation of filesystem paths.

### Accessing individual parts

To access the individual "parts" (components) of a path, use the following property:

<div class="attribute">

PurePath.parts

A tuple giving access to the path's various components:

    >>> p = PurePath('/usr/bin/python3')
    >>> p.parts
    ('/', 'usr', 'bin', 'python3')

    >>> p = PureWindowsPath('c:/Program Files/PSF')
    >>> p.parts
    ('c:\\', 'Program Files', 'PSF')

(note how the drive and local root are regrouped in a single part)

</div>

### Methods and properties

<div class="testsetup">

from pathlib import PurePath, PurePosixPath, PureWindowsPath

</div>

The `~PurePath.parent`, `~PurePath.name`, `~PurePath.stem`, and `~PurePath.suffix` properties describe different parts of a path. For example, given:

    >>> p = PurePosixPath('/srv/holy/grail.tar.gz')

<img src="pathlib-parts.svg" class="invert-in-dark-mode invert-in-dark-modealign-center" alt="For p = PurePosixPath(&#39;/srv/holy/grail.tar.gz&#39;), p.parent is
PurePosixPath(&#39;/srv/holy&#39;), p.name is &#39;grail.tar.gz&#39;,
p.stem is &#39;grail.tar&#39;, and p.suffix is &#39;.gz&#39;." />

The stem excludes only the final suffix: in this example, it includes `.tar`.

Pure paths provide the following methods and properties:

<div class="attribute">

PurePath.parser

The implementation of the `os.path` module used for low-level path parsing and joining: either `!posixpath` or `!ntpath`.

<div class="versionadded">

3.13

</div>

</div>

<div class="attribute">

PurePath.drive

A string representing the drive letter or name, if any:

    >>> PureWindowsPath('c:/Program Files/').drive
    'c:'
    >>> PureWindowsPath('/Program Files/').drive
    ''
    >>> PurePosixPath('/etc').drive
    ''

UNC shares are also considered drives:

    >>> PureWindowsPath('//host/share/foo.txt').drive
    '\\\\host\\share'

</div>

<div class="attribute">

PurePath.root

A string representing the (local or global) root, if any:

    >>> PureWindowsPath('c:/Program Files/').root
    '\\'
    >>> PureWindowsPath('c:Program Files/').root
    ''
    >>> PurePosixPath('/etc').root
    '/'

UNC shares always have a root:

    >>> PureWindowsPath('//host/share').root
    '\\'

If the path starts with more than two successive slashes, `~pathlib.PurePosixPath` collapses them:

    >>> PurePosixPath('//etc').root
    '//'
    >>> PurePosixPath('///etc').root
    '/'
    >>> PurePosixPath('////etc').root
    '/'

> [!NOTE]
> This behavior conforms to *The Open Group Base Specifications Issue 6*, paragraph [4.11 Pathname Resolution](https://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_11):
>
> *"A pathname that begins with two successive slashes may be interpreted in an implementation-defined manner, although more than two leading slashes shall be treated as a single slash."*

</div>

<div class="attribute">

PurePath.anchor

The concatenation of the drive and root:

    >>> PureWindowsPath('c:/Program Files/').anchor
    'c:\\'
    >>> PureWindowsPath('c:Program Files/').anchor
    'c:'
    >>> PurePosixPath('/etc').anchor
    '/'
    >>> PureWindowsPath('//host/share').anchor
    '\\\\host\\share\\'

</div>

<div class="attribute">

PurePath.parents

An immutable sequence providing access to the logical ancestors of the path:

    >>> p = PureWindowsPath('c:/foo/bar/setup.py')
    >>> p.parents[0]
    PureWindowsPath('c:/foo/bar')
    >>> p.parents[1]
    PureWindowsPath('c:/foo')
    >>> p.parents[2]
    PureWindowsPath('c:/')

<div class="versionchanged">

3.10 The parents sequence now supports `slices <slice>` and negative index values.

</div>

</div>

<div class="attribute">

PurePath.parent

The logical parent of the path:

    >>> p = PurePosixPath('/a/b/c/d')
    >>> p.parent
    PurePosixPath('/a/b/c')

You cannot go past an anchor, or empty path:

    >>> p = PurePosixPath('/')
    >>> p.parent
    PurePosixPath('/')
    >>> p = PurePosixPath('.')
    >>> p.parent
    PurePosixPath('.')

> [!NOTE]
> This is a purely lexical operation, hence the following behaviour:
>
>     >>> p = PurePosixPath('foo/..')
>     >>> p.parent
>
> PurePosixPath('foo')
>
> If you want to walk an arbitrary filesystem path upwards, it is recommended to first call `Path.resolve` so as to resolve symlinks and eliminate `".."` components.

</div>

<div class="attribute">

PurePath.name

A string representing the final path component, excluding the drive and root, if any:

    >>> PurePosixPath('my/library/setup.py').name
    'setup.py'

UNC drive names are not considered:

    >>> PureWindowsPath('//some/share/setup.py').name
    'setup.py'
    >>> PureWindowsPath('//some/share').name
    ''

</div>

<div class="attribute">

PurePath.suffix

The last dot-separated portion of the final component, if any:

    >>> PurePosixPath('my/library/setup.py').suffix
    '.py'
    >>> PurePosixPath('my/library.tar.gz').suffix
    '.gz'
    >>> PurePosixPath('my/library').suffix
    ''

This is commonly called the file extension.

<div class="versionchanged">

3.14

A single dot ("`.`") is considered a valid suffix.

</div>

</div>

<div class="attribute">

PurePath.suffixes

A list of the path's suffixes, often called file extensions:

    >>> PurePosixPath('my/library.tar.gar').suffixes
    ['.tar', '.gar']
    >>> PurePosixPath('my/library.tar.gz').suffixes
    ['.tar', '.gz']
    >>> PurePosixPath('my/library').suffixes
    []

<div class="versionchanged">

3.14

A single dot ("`.`") is considered a valid suffix.

</div>

</div>

<div class="attribute">

PurePath.stem

The final path component, without its suffix:

    >>> PurePosixPath('my/library.tar.gz').stem
    'library.tar'
    >>> PurePosixPath('my/library.tar').stem
    'library'
    >>> PurePosixPath('my/library').stem
    'library'

<div class="versionchanged">

3.14

A single dot ("`.`") is considered a valid suffix.

</div>

</div>

<div class="method">

PurePath.as_posix()

Return a string representation of the path with forward slashes (`/`):

    >>> p = PureWindowsPath('c:\\windows')
    >>> str(p)
    'c:\\windows'
    >>> p.as_posix()
    'c:/windows'

</div>

<div class="method">

PurePath.is_absolute()

Return whether the path is absolute or not. A path is considered absolute if it has both a root and (if the flavour allows) a drive:

    >>> PurePosixPath('/a/b').is_absolute()
    True
    >>> PurePosixPath('a/b').is_absolute()
    False

    >>> PureWindowsPath('c:/a/b').is_absolute()
    True
    >>> PureWindowsPath('/a/b').is_absolute()
    False
    >>> PureWindowsPath('c:').is_absolute()
    False
    >>> PureWindowsPath('//some/share').is_absolute()
    True

</div>

<div class="method">

PurePath.is_relative_to(other)

Return whether or not this path is relative to the *other* path.

> \>\>\> p = PurePath('/etc/passwd') \>\>\> p.is_relative_to('/etc') True \>\>\> p.is_relative_to('/usr') False

This method is string-based; it neither accesses the filesystem nor treats "`..`" segments specially. The following code is equivalent:

> \>\>\> u = PurePath('/usr') \>\>\> u == p or u in p.parents False

<div class="versionadded">

3.9

</div>

<div class="deprecated-removed">

3.12 3.14

Passing additional arguments is deprecated; if supplied, they are joined with *other*.

</div>

</div>

<div class="method">

PurePath.joinpath(\*pathsegments)

Calling this method is equivalent to combining the path with each of the given *pathsegments* in turn:

    >>> PurePosixPath('/etc').joinpath('passwd')
    PurePosixPath('/etc/passwd')
    >>> PurePosixPath('/etc').joinpath(PurePosixPath('passwd'))
    PurePosixPath('/etc/passwd')
    >>> PurePosixPath('/etc').joinpath('init.d', 'apache2')
    PurePosixPath('/etc/init.d/apache2')
    >>> PureWindowsPath('c:').joinpath('/Program Files')
    PureWindowsPath('c:/Program Files')

</div>

<div class="method">

PurePath.full_match(pattern, \*, case_sensitive=None)

Match this path against the provided glob-style pattern. Return `True` if matching is successful, `False` otherwise. For example:

    >>> PurePath('a/b.py').full_match('a/*.py')
    True
    >>> PurePath('a/b.py').full_match('*.py')
    False
    >>> PurePath('/a/b/c.py').full_match('**/*.py')
    True
    >>> PurePath('/a/b/c.py').full_match('/a/**')
    True

Patterns ending in "`/**`", as in the example above, may surprise when passed to `full_match`. Even though "`**`" can match 0 segments, "`.../**`" requires a separator to match. Hence "`/a/**`" does not match path "`/a`". Since `PurePath` drops trailing slashes, this pattern also does not match "`/a/`".

    >>> PurePath('/a').full_match('/a/**')
    False
    >>> PurePath('/a/').full_match('/a/**')
    False

<div class="seealso">

`pathlib-pattern-language` documentation.

</div>

As with other methods, case-sensitivity follows platform defaults:

    >>> PurePosixPath('b.py').full_match('*.PY')
    False
    >>> PureWindowsPath('b.py').full_match('*.PY')
    True

Set *case_sensitive* to `True` or `False` to override this behaviour.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

PurePath.match(pattern, \*, case_sensitive=None)

Match this path against the provided non-recursive glob-style pattern. Return `True` if matching is successful, `False` otherwise.

This method is similar to `~PurePath.full_match`, but empty patterns aren't allowed (`ValueError` is raised), the recursive wildcard "`**`" isn't supported (it acts like non-recursive "`*`"), and if a relative pattern is provided, then matching is done from the right:

    >>> PurePath('a/b.py').match('*.py')
    True
    >>> PurePath('/a/b/c.py').match('b/*.py')
    True
    >>> PurePath('/a/b/c.py').match('a/*.py')
    False

<div class="versionchanged">

3.12 The *pattern* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.12 The *case_sensitive* parameter was added.

</div>

</div>

<div class="method">

PurePath.relative_to(other, walk_up=False)

Compute a version of this path relative to the path represented by *other*. If it's impossible, `ValueError` is raised:

    >>> p = PurePosixPath('/etc/passwd')
    >>> p.relative_to('/')
    PurePosixPath('etc/passwd')
    >>> p.relative_to('/etc')
    PurePosixPath('passwd')
    >>> p.relative_to('/usr')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "pathlib.py", line 941, in relative_to
        raise ValueError(error_message.format(str(self), str(formatted)))
    ValueError: '/etc/passwd' is not in the subpath of '/usr' OR one path is relative and the other is absolute.

When *walk_up* is false (the default), the path must start with *other*. When the argument is true, `..` entries may be added to form the relative path. In all other cases, such as the paths referencing different drives, `ValueError` is raised.:

    >>> p.relative_to('/usr', walk_up=True)
    PurePosixPath('../etc/passwd')
    >>> p.relative_to('foo', walk_up=True)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "pathlib.py", line 941, in relative_to
        raise ValueError(error_message.format(str(self), str(formatted)))
    ValueError: '/etc/passwd' is not on the same drive as 'foo' OR one path is relative and the other is absolute.

> [!WARNING]
> This function is part of `PurePath` and works with strings. It does not check or access the underlying file structure. This can impact the *walk_up* option as it assumes that no symlinks are present in the path; call `~Path.resolve` first if necessary to resolve symlinks.

<div class="versionchanged">

3.12 The *walk_up* parameter was added (old behavior is the same as `walk_up=False`).

</div>

<div class="deprecated-removed">

3.12 3.14

Passing additional positional arguments is deprecated; if supplied, they are joined with *other*.

</div>

</div>

<div class="method">

PurePath.with_name(name)

Return a new path with the `name` changed. If the original path doesn't have a name, ValueError is raised:

    >>> p = PureWindowsPath('c:/Downloads/pathlib.tar.gz')
    >>> p.with_name('setup.py')
    PureWindowsPath('c:/Downloads/setup.py')
    >>> p = PureWindowsPath('c:/')
    >>> p.with_name('setup.py')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/antoine/cpython/default/Lib/pathlib.py", line 751, in with_name
        raise ValueError("%r has an empty name" % (self,))
    ValueError: PureWindowsPath('c:/') has an empty name

</div>

<div class="method">

PurePath.with_stem(stem)

Return a new path with the `stem` changed. If the original path doesn't have a name, ValueError is raised:

    >>> p = PureWindowsPath('c:/Downloads/draft.txt')
    >>> p.with_stem('final')
    PureWindowsPath('c:/Downloads/final.txt')
    >>> p = PureWindowsPath('c:/Downloads/pathlib.tar.gz')
    >>> p.with_stem('lib')
    PureWindowsPath('c:/Downloads/lib.gz')
    >>> p = PureWindowsPath('c:/')
    >>> p.with_stem('')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/antoine/cpython/default/Lib/pathlib.py", line 861, in with_stem
        return self.with_name(stem + self.suffix)
      File "/home/antoine/cpython/default/Lib/pathlib.py", line 851, in with_name
        raise ValueError("%r has an empty name" % (self,))
    ValueError: PureWindowsPath('c:/') has an empty name

<div class="versionadded">

3.9

</div>

</div>

<div class="method">

PurePath.with_suffix(suffix)

Return a new path with the `suffix` changed. If the original path doesn't have a suffix, the new *suffix* is appended instead. If the *suffix* is an empty string, the original suffix is removed:

    >>> p = PureWindowsPath('c:/Downloads/pathlib.tar.gz')
    >>> p.with_suffix('.bz2')
    PureWindowsPath('c:/Downloads/pathlib.tar.bz2')
    >>> p = PureWindowsPath('README')
    >>> p.with_suffix('.txt')
    PureWindowsPath('README.txt')
    >>> p = PureWindowsPath('README.txt')
    >>> p.with_suffix('')
    PureWindowsPath('README')

<div class="versionchanged">

3.14

A single dot ("`.`") is considered a valid suffix. In previous versions, `ValueError` is raised if a single dot is supplied.

</div>

</div>

<div class="method">

PurePath.with_segments(\*pathsegments)

Create a new path object of the same type by combining the given *pathsegments*. This method is called whenever a derivative path is created, such as from `parent` and `relative_to`. Subclasses may override this method to pass information to derivative paths, for example:

    from pathlib import PurePosixPath

    class MyPath(PurePosixPath):
        def __init__(self, *pathsegments, session_id):
            super().__init__(*pathsegments)
            self.session_id = session_id

        def with_segments(self, *pathsegments):
            return type(self)(*pathsegments, session_id=self.session_id)

    etc = MyPath('/etc', session_id=42)
    hosts = etc / 'hosts'
    print(hosts.session_id)  # 42

<div class="versionadded">

3.12

</div>

</div>

## Concrete paths

Concrete paths are subclasses of the pure path classes. In addition to operations provided by the latter, they also provide methods to do system calls on path objects. There are three ways to instantiate concrete paths:

<div class="Path(*pathsegments)">

A subclass of `PurePath`, this class represents concrete paths of the system's path flavour (instantiating it creates either a `PosixPath` or a `WindowsPath`):

    >>> Path('setup.py')
    PosixPath('setup.py')

*pathsegments* is specified similarly to `PurePath`.

</div>

<div class="PosixPath(*pathsegments)">

A subclass of `Path` and `PurePosixPath`, this class represents concrete non-Windows filesystem paths:

    >>> PosixPath('/etc/hosts')
    PosixPath('/etc/hosts')

*pathsegments* is specified similarly to `PurePath`.

<div class="versionchanged">

3.13 Raises `UnsupportedOperation` on Windows. In previous versions, `NotImplementedError` was raised instead.

</div>

</div>

<div class="WindowsPath(*pathsegments)">

A subclass of `Path` and `PureWindowsPath`, this class represents concrete Windows filesystem paths:

    >>> WindowsPath('c:/', 'Users', 'Ximénez')
    WindowsPath('c:/Users/Ximénez')

*pathsegments* is specified similarly to `PurePath`.

<div class="versionchanged">

3.13 Raises `UnsupportedOperation` on non-Windows platforms. In previous versions, `NotImplementedError` was raised instead.

</div>

</div>

You can only instantiate the class flavour that corresponds to your system (allowing system calls on non-compatible path flavours could lead to bugs or failures in your application):

    >>> import os
    >>> os.name
    'posix'
    >>> Path('setup.py')
    PosixPath('setup.py')
    >>> PosixPath('setup.py')
    PosixPath('setup.py')
    >>> WindowsPath('setup.py')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "pathlib.py", line 798, in __new__
        % (cls.__name__,))
    UnsupportedOperation: cannot instantiate 'WindowsPath' on your system

Some concrete path methods can raise an `OSError` if a system call fails (for example because the path doesn't exist).

### Parsing and generating URIs

Concrete path objects can be created from, and represented as, 'file' URIs conforming to `8089`.

> [!NOTE]
> File URIs are not portable across machines with different `filesystem encodings <filesystem-encoding>`.

<div class="classmethod">

Path.from_uri(uri)

Return a new path object from parsing a 'file' URI. For example:

    >>> p = Path.from_uri('file:///etc/hosts')
    PosixPath('/etc/hosts')

On Windows, DOS device and UNC paths may be parsed from URIs:

    >>> p = Path.from_uri('file:///c:/windows')
    WindowsPath('c:/windows')
    >>> p = Path.from_uri('file://server/share')
    WindowsPath('//server/share')

Several variant forms are supported:

    >>> p = Path.from_uri('file:////server/share')
    WindowsPath('//server/share')
    >>> p = Path.from_uri('file://///server/share')
    WindowsPath('//server/share')
    >>> p = Path.from_uri('file:c:/windows')
    WindowsPath('c:/windows')
    >>> p = Path.from_uri('file:/c|/windows')
    WindowsPath('c:/windows')

`ValueError` is raised if the URI does not start with `file:`, or the parsed path isn't absolute.

<div class="versionadded">

3.13

</div>

<div class="versionchanged">

3.14 The URL authority is discarded if it matches the local hostname. Otherwise, if the authority isn't empty or `localhost`, then on Windows a UNC path is returned (as before), and on other platforms a `ValueError` is raised.

</div>

</div>

<div class="method">

Path.as_uri()

Represent the path as a 'file' URI. `ValueError` is raised if the path isn't absolute.

```
>>> p = PosixPath('/etc/passwd')
>>> p.as_uri()
'file:///etc/passwd'
>>> p = WindowsPath('c:/Windows')
>>> p.as_uri()
'file:///c:/Windows'
```

<div class="deprecated-removed">

3.14 3.19

Calling this method from `PurePath` rather than `Path` is possible but deprecated. The method's use of `os.fsencode` makes it strictly impure.

</div>

</div>

### Expanding and resolving paths

<div class="classmethod">

Path.home()

Return a new path object representing the user's home directory (as returned by `os.path.expanduser` with `~` construct). If the home directory can't be resolved, `RuntimeError` is raised.

    >>> Path.home()
    PosixPath('/home/antoine')

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

Path.expanduser()

Return a new path with expanded `~` and `~user` constructs, as returned by `os.path.expanduser`. If a home directory can't be resolved, `RuntimeError` is raised.

    >>> p = PosixPath('~/films/Monty Python')
    >>> p.expanduser()
    PosixPath('/home/eric/films/Monty Python')

<div class="versionadded">

3.5

</div>

</div>

<div class="classmethod">

Path.cwd()

Return a new path object representing the current directory (as returned by `os.getcwd`):

    >>> Path.cwd()
    PosixPath('/home/antoine/pathlib')

</div>

<div class="method">

Path.absolute()

Make the path absolute, without normalization or resolving symlinks. Returns a new path object:

    >>> p = Path('tests')
    >>> p
    PosixPath('tests')
    >>> p.absolute()
    PosixPath('/home/antoine/pathlib/tests')

</div>

<div class="method">

Path.resolve(strict=False)

Make the path absolute, resolving any symlinks. A new path object is returned:

    >>> p = Path()
    >>> p
    PosixPath('.')
    >>> p.resolve()
    PosixPath('/home/antoine/pathlib')

"`..`" components are also eliminated (this is the only method to do so):

    >>> p = Path('docs/../setup.py')
    >>> p.resolve()
    PosixPath('/home/antoine/pathlib/setup.py')

If a path doesn't exist or a symlink loop is encountered, and *strict* is `True`, `OSError` is raised. If *strict* is `False`, the path is resolved as far as possible and any remainder is appended without checking whether it exists.

<div class="versionchanged">

3.6 The *strict* parameter was added (pre-3.6 behavior is strict).

</div>

<div class="versionchanged">

3.13 Symlink loops are treated like other errors: `OSError` is raised in strict mode, and no exception is raised in non-strict mode. In previous versions, `RuntimeError` is raised no matter the value of *strict*.

</div>

</div>

<div class="method">

Path.readlink()

Return the path to which the symbolic link points (as returned by `os.readlink`):

    >>> p = Path('mylink')
    >>> p.symlink_to('setup.py')
    >>> p.readlink()
    PosixPath('setup.py')

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.13 Raises `UnsupportedOperation` if `os.readlink` is not available. In previous versions, `NotImplementedError` was raised.

</div>

</div>

### Querying file type and status

<div class="versionchanged">

3.8

`~Path.exists`, `~Path.is_dir`, `~Path.is_file`, `~Path.is_mount`, `~Path.is_symlink`, `~Path.is_block_device`, `~Path.is_char_device`, `~Path.is_fifo`, `~Path.is_socket` now return `False` instead of raising an exception for paths that contain characters unrepresentable at the OS level.

</div>

<div class="versionchanged">

3.14

The methods given above now return `False` instead of raising any `OSError` exception from the operating system. In previous versions, some kinds of `OSError` exception are raised, and others suppressed. The new behaviour is consistent with `os.path.exists`, `os.path.isdir`, etc. Use `~Path.stat` to retrieve the file status without suppressing exceptions.

</div>

<div class="method">

Path.stat(\*, follow_symlinks=True)

Return an `os.stat_result` object containing information about this path, like `os.stat`. The result is looked up at each call to this method.

This method normally follows symlinks; to stat a symlink add the argument `follow_symlinks=False`, or use `~Path.lstat`.

    >>> p = Path('setup.py')
    >>> p.stat().st_size
    956
    >>> p.stat().st_mtime
    1327883547.852554

<div class="versionchanged">

3.10 The *follow_symlinks* parameter was added.

</div>

</div>

<div class="method">

Path.lstat()

Like `Path.stat` but, if the path points to a symbolic link, return the symbolic link's information rather than its target's.

</div>

<div class="method">

Path.exists(\*, follow_symlinks=True)

Return `True` if the path points to an existing file or directory and `False` if the path is invalid, inaccessible or missing. Use `Path.stat` to distinguish between these cases.

This method normally follows symlinks; to check if a symlink exists, add the argument `follow_symlinks=False`.

    >>> Path('').exists()  # The current directory.
    True
    >>> Path('.').exists()
    True
    >>> Path('setup.py').exists()
    True
    >>> Path('/etc').exists()
    True
    >>> Path('nonexistentfile').exists()
    False

<div class="versionchanged">

3.12 The *follow_symlinks* parameter was added.

</div>

</div>

<div class="method">

Path.is_file(\*, follow_symlinks=True)

Return `True` if the path points to a regular file. `False` will be returned if the path is invalid, inaccessible or missing, or if it points to something other than a regular file. Use `Path.stat` to distinguish between these cases.

This method normally follows symlinks; to exclude symlinks, add the argument `follow_symlinks=False`.

<div class="versionchanged">

3.13 The *follow_symlinks* parameter was added.

</div>

</div>

<div class="method">

Path.is_dir(\*, follow_symlinks=True)

Return `True` if the path points to a directory. `False` will be returned if the path is invalid, inaccessible or missing, or if it points to something other than a directory. Use `Path.stat` to distinguish between these cases.

This method normally follows symlinks; to exclude symlinks to directories, add the argument `follow_symlinks=False`.

<div class="versionchanged">

3.13 The *follow_symlinks* parameter was added.

</div>

</div>

<div class="method">

Path.is_symlink()

Return `True` if the path points to a symbolic link, even if that symlink is broken. `False` will be returned if the path is invalid, inaccessible or missing, or if it points to something other than a symbolic link. Use `Path.stat` to distinguish between these cases.

</div>

<div class="method">

Path.is_junction()

Return `True` if the path points to a junction, and `False` for any other type of file. Currently only Windows supports junctions.

<div class="versionadded">

3.12

</div>

</div>

<div class="method">

Path.is_mount()

Return `True` if the path is a `mount point`: a point in a file system where a different file system has been mounted. On POSIX, the function checks whether *path*'s parent, `path/..`, is on a different device than *path*, or whether `path/..` and *path* point to the same i-node on the same device --- this should detect mount points for all Unix and POSIX variants. On Windows, a mount point is considered to be a drive letter root (e.g. `c:\`), a UNC share (e.g. `\\server\share`), or a mounted filesystem directory.

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.12 Windows support was added.

</div>

</div>

<div class="method">

Path.is_socket()

Return `True` if the path points to a Unix socket. `False` will be returned if the path is invalid, inaccessible or missing, or if it points to something other than a Unix socket. Use `Path.stat` to distinguish between these cases.

</div>

<div class="method">

Path.is_fifo()

Return `True` if the path points to a FIFO. `False` will be returned if the path is invalid, inaccessible or missing, or if it points to something other than a FIFO. Use `Path.stat` to distinguish between these cases.

</div>

<div class="method">

Path.is_block_device()

Return `True` if the path points to a block device. `False` will be returned if the path is invalid, inaccessible or missing, or if it points to something other than a block device. Use `Path.stat` to distinguish between these cases.

</div>

<div class="method">

Path.is_char_device()

Return `True` if the path points to a character device. `False` will be returned if the path is invalid, inaccessible or missing, or if it points to something other than a character device. Use `Path.stat` to distinguish between these cases.

</div>

<div class="method">

Path.samefile(other_path)

Return whether this path points to the same file as *other_path*, which can be either a Path object, or a string. The semantics are similar to `os.path.samefile` and `os.path.samestat`.

An `OSError` can be raised if either file cannot be accessed for some reason.

    >>> p = Path('spam')
    >>> q = Path('eggs')
    >>> p.samefile(q)
    False
    >>> p.samefile('spam')
    True

<div class="versionadded">

3.5

</div>

</div>

<div class="attribute">

Path.info

A `~pathlib.types.PathInfo` object that supports querying file type information. The object exposes methods that cache their results, which can help reduce the number of system calls needed when switching on file type. For example:

    >>> p = Path('src')
    >>> if p.info.is_symlink():
    ...     print('symlink')
    ... elif p.info.is_dir():
    ...     print('directory')
    ... elif p.info.exists():
    ...     print('something else')
    ... else:
    ...     print('not found')
    ...
    directory

If the path was generated from `Path.iterdir` then this attribute is initialized with some information about the file type gleaned from scanning the parent directory. Merely accessing `Path.info` does not perform any filesystem queries.

To fetch up-to-date information, it's best to call `Path.is_dir`, `~Path.is_file` and `~Path.is_symlink` rather than methods of this attribute. There is no way to reset the cache; instead you can create a new path object with an empty info cache via `p = Path(p)`.

<div class="versionadded">

3.14

</div>

</div>

### Reading and writing files

<div class="method">

Path.open(mode='r', buffering=-1, encoding=None, errors=None, newline=None)

Open the file pointed to by the path, like the built-in `open` function does:

    >>> p = Path('setup.py')
    >>> with p.open() as f:
    ...     f.readline()
    ...
    '#!/usr/bin/env python3\n'

</div>

<div class="method">

Path.read_text(encoding=None, errors=None, newline=None)

Return the decoded contents of the pointed-to file as a string:

    >>> p = Path('my_text_file')
    >>> p.write_text('Text file contents')
    18
    >>> p.read_text()
    'Text file contents'

The file is opened and then closed. The optional parameters have the same meaning as in `open`.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.13 The *newline* parameter was added.

</div>

</div>

<div class="method">

Path.read_bytes()

Return the binary contents of the pointed-to file as a bytes object:

    >>> p = Path('my_binary_file')
    >>> p.write_bytes(b'Binary file contents')
    20
    >>> p.read_bytes()
    b'Binary file contents'

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

Path.write_text(data, encoding=None, errors=None, newline=None)

Open the file pointed to in text mode, write *data* to it, and close the file:

    >>> p = Path('my_text_file')
    >>> p.write_text('Text file contents')
    18
    >>> p.read_text()
    'Text file contents'

Return the number of characters written.

An existing file of the same name is overwritten. The optional parameters have the same meaning as in `open`.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.10 The *newline* parameter was added.

</div>

</div>

<div class="method">

Path.write_bytes(data)

Open the file pointed to in bytes mode, write *data* to it, and close the file:

    >>> p = Path('my_binary_file')
    >>> p.write_bytes(b'Binary file contents')
    20
    >>> p.read_bytes()
    b'Binary file contents'

Return the number of bytes written.

An existing file of the same name is overwritten.

<div class="versionadded">

3.5

</div>

</div>

### Reading directories

<div class="method">

Path.iterdir()

When the path points to a directory, yield path objects of the directory contents:

    >>> p = Path('docs')
    >>> for child in p.iterdir(): child
    ...
    PosixPath('docs/conf.py')
    PosixPath('docs/_templates')
    PosixPath('docs/make.bat')
    PosixPath('docs/index.rst')
    PosixPath('docs/_build')
    PosixPath('docs/_static')
    PosixPath('docs/Makefile')

The children are yielded in arbitrary order, and the special entries `'.'` and `'..'` are not included. If a file is removed from or added to the directory after creating the iterator, it is unspecified whether a path object for that file is included.

If the path is not a directory or otherwise inaccessible, `OSError` is raised.

</div>

<div class="method">

Path.glob(pattern, \*, case_sensitive=None, recurse_symlinks=False)

Glob the given relative *pattern* in the directory represented by this path, yielding all matching files (of any kind):

    >>> sorted(Path('.').glob('*.py'))
    [PosixPath('pathlib.py'), PosixPath('setup.py'), PosixPath('test_pathlib.py')]
    >>> sorted(Path('.').glob('*/*.py'))
    [PosixPath('docs/conf.py')]
    >>> sorted(Path('.').glob('**/*.py'))
    [PosixPath('build/lib/pathlib.py'),
     PosixPath('docs/conf.py'),
     PosixPath('pathlib.py'),
     PosixPath('setup.py'),
     PosixPath('test_pathlib.py')]

> [!NOTE]
> The paths are returned in no particular order. If you need a specific order, sort the results.

<div class="seealso">

`pathlib-pattern-language` documentation.

</div>

By default, or when the *case_sensitive* keyword-only argument is set to `None`, this method matches paths using platform-specific casing rules: typically, case-sensitive on POSIX, and case-insensitive on Windows. Set *case_sensitive* to `True` or `False` to override this behaviour.

By default, or when the *recurse_symlinks* keyword-only argument is set to `False`, this method follows symlinks except when expanding "`**`" wildcards. Set *recurse_symlinks* to `True` to always follow symlinks.

> [!NOTE]
> Any `OSError` exceptions raised from scanning the filesystem are suppressed. This includes `PermissionError` when accessing directories without read permission.

<div class="audit-event">

pathlib.Path.glob self,pattern pathlib.Path.glob

</div>

<div class="versionchanged">

3.12 The *case_sensitive* parameter was added.

</div>

<div class="versionchanged">

3.13 The *recurse_symlinks* parameter was added.

</div>

<div class="versionchanged">

3.13 The *pattern* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.13 Any `OSError` exceptions raised from scanning the filesystem are suppressed. In previous versions, such exceptions are suppressed in many cases, but not all.

</div>

</div>

<div class="method">

Path.rglob(pattern, \*, case_sensitive=None, recurse_symlinks=False)

Glob the given relative *pattern* recursively. This is like calling `Path.glob` with "`**/`" added in front of the *pattern*.

> [!NOTE]
> The paths are returned in no particular order. If you need a specific order, sort the results.

> [!NOTE]
> Any `OSError` exceptions raised from scanning the filesystem are suppressed. This includes `PermissionError` when accessing directories without read permission.

<div class="seealso">

`pathlib-pattern-language` and `Path.glob` documentation.

</div>

<div class="audit-event">

pathlib.Path.rglob self,pattern pathlib.Path.rglob

</div>

<div class="versionchanged">

3.12 The *case_sensitive* parameter was added.

</div>

<div class="versionchanged">

3.13 The *recurse_symlinks* parameter was added.

</div>

<div class="versionchanged">

3.13 The *pattern* parameter accepts a `path-like object`.

</div>

</div>

<div class="method">

Path.walk(top_down=True, on_error=None, follow_symlinks=False)

Generate the file names in a directory tree by walking the tree either top-down or bottom-up.

For each directory in the directory tree rooted at *self* (including *self* but excluding '.' and '..'), the method yields a 3-tuple of `(dirpath, dirnames, filenames)`.

*dirpath* is a `Path` to the directory currently being walked, *dirnames* is a list of strings for the names of subdirectories in *dirpath* (excluding `'.'` and `'..'`), and *filenames* is a list of strings for the names of the non-directory files in *dirpath*. To get a full path (which begins with *self*) to a file or directory in *dirpath*, do `dirpath / name`. Whether or not the lists are sorted is file system-dependent.

If the optional argument *top_down* is true (which is the default), the triple for a directory is generated before the triples for any of its subdirectories (directories are walked top-down). If *top_down* is false, the triple for a directory is generated after the triples for all of its subdirectories (directories are walked bottom-up). No matter the value of *top_down*, the list of subdirectories is retrieved before the triples for the directory and its subdirectories are walked.

When *top_down* is true, the caller can modify the *dirnames* list in-place (for example, using `del` or slice assignment), and `Path.walk` will only recurse into the subdirectories whose names remain in *dirnames*. This can be used to prune the search, or to impose a specific order of visiting, or even to inform `Path.walk` about directories the caller creates or renames before it resumes `Path.walk` again. Modifying *dirnames* when *top_down* is false has no effect on the behavior of `Path.walk` since the directories in *dirnames* have already been generated by the time *dirnames* is yielded to the caller.

By default, errors from `os.scandir` are ignored. If the optional argument *on_error* is specified, it should be a callable; it will be called with one argument, an `OSError` instance. The callable can handle the error to continue the walk or re-raise it to stop the walk. Note that the filename is available as the `filename` attribute of the exception object.

By default, `Path.walk` does not follow symbolic links, and instead adds them to the *filenames* list. Set *follow_symlinks* to true to resolve symlinks and place them in *dirnames* and *filenames* as appropriate for their targets, and consequently visit directories pointed to by symlinks (where supported).

> [!NOTE]
> Be aware that setting *follow_symlinks* to true can lead to infinite recursion if a link points to a parent directory of itself. `Path.walk` does not keep track of the directories it has already visited.

> [!NOTE]
> `Path.walk` assumes the directories it walks are not modified during execution. For example, if a directory from *dirnames* has been replaced with a symlink and *follow_symlinks* is false, `Path.walk` will still try to descend into it. To prevent such behavior, remove directories from *dirnames* as appropriate.

> [!NOTE]
> Unlike `os.walk`, `Path.walk` lists symlinks to directories in *filenames* if *follow_symlinks* is false.

This example displays the number of bytes used by all files in each directory, while ignoring `__pycache__` directories:

    from pathlib import Path
    for root, dirs, files in Path("cpython/Lib/concurrent").walk(on_error=print):
      print(
          root,
          "consumes",
          sum((root / file).stat().st_size for file in files),
          "bytes in",
          len(files),
          "non-directory files"
      )
      if '__pycache__' in dirs:
            dirs.remove('__pycache__')

This next example is a simple implementation of `shutil.rmtree`. Walking the tree bottom-up is essential as `rmdir` doesn't allow deleting a directory before it is empty:

    # Delete everything reachable from the directory "top".
    # CAUTION:  This is dangerous! For example, if top == Path('/'),
    # it could delete all of your files.
    for root, dirs, files in top.walk(top_down=False):
        for name in files:
            (root / name).unlink()
        for name in dirs:
            (root / name).rmdir()

<div class="versionadded">

3.12

</div>

</div>

### Creating files and directories

<div class="method">

Path.touch(mode=0o666, exist_ok=True)

Create a file at this given path. If *mode* is given, it is combined with the process's `umask` value to determine the file mode and access flags. If the file already exists, the function succeeds when *exist_ok* is true (and its modification time is updated to the current time), otherwise `FileExistsError` is raised.

<div class="seealso">

The `~Path.open`, `~Path.write_text` and `~Path.write_bytes` methods are often used to create files.

</div>

</div>

<div class="method">

Path.mkdir(mode=0o777, parents=False, exist_ok=False, \*, parent_mode=None)

Create a new directory at this given path. If *mode* is given, it is combined with the process's `umask` value to determine the file mode and access flags. If the path already exists, `FileExistsError` is raised.

If *parents* is true, any missing parents of this path are created as needed; they are created with the default permissions without taking *mode* into account (mimicking the POSIX `mkdir -p` command).

If *parent_mode* is not `None`, it is used as the mode for any newly-created, intermediate-level directories when *parents* is true. Like *mode*, it is combined with the process's `umask` value. Otherwise, intermediate directories are created with the default permissions (also subject to the umask).

If *parents* is false (the default), a missing parent raises `FileNotFoundError`.

If *exist_ok* is false (the default), `FileExistsError` is raised if the target directory already exists.

If *exist_ok* is true, `FileExistsError` will not be raised unless the given path already exists in the file system and is not a directory (same behavior as the POSIX `mkdir -p` command).

<div class="versionchanged">

3.5 The *exist_ok* parameter was added.

</div>

<div class="versionadded">

3.15 The *parent_mode* parameter.

</div>

</div>

<div class="method">

Path.symlink_to(target, target_is_directory=False)

Make this path a symbolic link pointing to *target*.

On Windows, a symlink represents either a file or a directory, and does not morph to the target dynamically. If the target is present, the type of the symlink will be created to match. Otherwise, the symlink will be created as a directory if *target_is_directory* is true or a file symlink (the default) otherwise. On non-Windows platforms, *target_is_directory* is ignored.

    >>> p = Path('mylink')
    >>> p.symlink_to('setup.py')
    >>> p.resolve()
    PosixPath('/home/antoine/pathlib/setup.py')
    >>> p.stat().st_size
    956
    >>> p.lstat().st_size
    8

> [!NOTE]
> The order of arguments (link, target) is the reverse of `os.symlink`'s.

<div class="versionchanged">

3.13 Raises `UnsupportedOperation` if `os.symlink` is not available. In previous versions, `NotImplementedError` was raised.

</div>

</div>

<div class="method">

Path.hardlink_to(target)

Make this path a hard link to the same file as *target*.

> [!NOTE]
> The order of arguments (link, target) is the reverse of `os.link`'s.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.13 Raises `UnsupportedOperation` if `os.link` is not available. In previous versions, `NotImplementedError` was raised.

</div>

</div>

### Copying, moving and deleting

<div class="method">

Path.copy(target, \*, follow_symlinks=True, preserve_metadata=False)

Copy this file or directory tree to the given *target*, and return a new `!Path` instance pointing to *target*.

If the source is a file, the target will be replaced if it is an existing file. If the source is a symlink and *follow_symlinks* is true (the default), the symlink's target is copied. Otherwise, the symlink is recreated at the destination.

If *preserve_metadata* is false (the default), only directory structures and file data are guaranteed to be copied. Set *preserve_metadata* to true to ensure that file and directory permissions, flags, last access and modification times, and extended attributes are copied where supported. This argument has no effect when copying files on Windows (where metadata is always preserved).

> [!NOTE]
> Where supported by the operating system and file system, this method performs a lightweight copy, where data blocks are only copied when modified. This is known as copy-on-write.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

Path.copy_into(target_dir, \*, follow_symlinks=True, preserve_metadata=False)

Copy this file or directory tree into the given *target_dir*, which should be an existing directory. Other arguments are handled identically to `Path.copy`. Returns a new `!Path` instance pointing to the copy.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

Path.rename(target)

Rename this file or directory to the given *target*, and return a new `!Path` instance pointing to *target*. On Unix, if *target* exists and is a file, it will be replaced silently if the user has permission. On Windows, if *target* exists, `FileExistsError` will be raised. *target* can be either a string or another path object:

    >>> p = Path('foo')
    >>> p.open('w').write('some text')
    9
    >>> target = Path('bar')
    >>> p.rename(target)
    PosixPath('bar')
    >>> target.open().read()
    'some text'

The target path may be absolute or relative. Relative paths are interpreted relative to the current working directory, *not* the directory of the `!Path` object.

It is implemented in terms of `os.rename` and gives the same guarantees.

<div class="versionchanged">

3.8 Added return value, return the new `!Path` instance.

</div>

</div>

<div class="method">

Path.replace(target)

Rename this file or directory to the given *target*, and return a new `!Path` instance pointing to *target*. If *target* points to an existing file or empty directory, it will be unconditionally replaced.

The target path may be absolute or relative. Relative paths are interpreted relative to the current working directory, *not* the directory of the `!Path` object.

<div class="versionchanged">

3.8 Added return value, return the new `!Path` instance.

</div>

</div>

<div class="method">

Path.move(target)

Move this file or directory tree to the given *target*, and return a new `!Path` instance pointing to *target*.

If the *target* doesn't exist it will be created. If both this path and the *target* are existing files, then the target is overwritten. If both paths point to the same file or directory, or the *target* is a non-empty directory, then `OSError` is raised.

If both paths are on the same filesystem, the move is performed with `os.replace`. Otherwise, this path is copied (preserving metadata and symlinks) and then deleted.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

Path.move_into(target_dir)

Move this file or directory tree into the given *target_dir*, which should be an existing directory. Returns a new `!Path` instance pointing to the moved path.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

Path.unlink(missing_ok=False)

Remove this file or symbolic link. If the path points to a directory, use `Path.rmdir` instead.

If *missing_ok* is false (the default), `FileNotFoundError` is raised if the path does not exist.

If *missing_ok* is true, `FileNotFoundError` exceptions will be ignored (same behavior as the POSIX `rm -f` command).

<div class="versionchanged">

3.8 The *missing_ok* parameter was added.

</div>

</div>

<div class="method">

Path.rmdir()

Remove this directory. The directory must be empty.

</div>

### Permissions and ownership

<div class="method">

Path.owner(\*, follow_symlinks=True)

Return the name of the user owning the file. `KeyError` is raised if the file's user identifier (UID) isn't found in the system database.

This method normally follows symlinks; to get the owner of the symlink, add the argument `follow_symlinks=False`.

<div class="versionchanged">

3.13 Raises `UnsupportedOperation` if the `pwd` module is not available. In earlier versions, `NotImplementedError` was raised.

</div>

<div class="versionchanged">

3.13 The *follow_symlinks* parameter was added.

</div>

</div>

<div class="method">

Path.group(\*, follow_symlinks=True)

Return the name of the group owning the file. `KeyError` is raised if the file's group identifier (GID) isn't found in the system database.

This method normally follows symlinks; to get the group of the symlink, add the argument `follow_symlinks=False`.

<div class="versionchanged">

3.13 Raises `UnsupportedOperation` if the `grp` module is not available. In earlier versions, `NotImplementedError` was raised.

</div>

<div class="versionchanged">

3.13 The *follow_symlinks* parameter was added.

</div>

</div>

<div class="method">

Path.chmod(mode, \*, follow_symlinks=True)

Change the file mode and permissions, like `os.chmod`.

This method normally follows symlinks. Some Unix flavours support changing permissions on the symlink itself; on these platforms you may add the argument `follow_symlinks=False`, or use `~Path.lchmod`.

    >>> p = Path('setup.py')
    >>> p.stat().st_mode
    33277
    >>> p.chmod(0o444)
    >>> p.stat().st_mode
    33060

<div class="versionchanged">

3.10 The *follow_symlinks* parameter was added.

</div>

</div>

<div class="method">

Path.lchmod(mode)

Like `Path.chmod` but, if the path points to a symbolic link, the symbolic link's mode is changed rather than its target's.

</div>

## Pattern language

The following wildcards are supported in patterns for `~PurePath.full_match`, `~Path.glob` and `~Path.rglob`:

`**` (entire segment)  
Matches any number of file or directory segments, including zero. Note that in `~PurePath.full_match`, "`a/**`" does not match "`a`", whereas `~Path.glob` yields "`a`" for that pattern.

`*` (entire segment)  
Matches one file or directory segment.

`*` (part of a segment)  
Matches any number of non-separator characters, including zero.

`?`  
Matches one non-separator character.

`[seq]`  
Matches one character in *seq*, where *seq* is a sequence of characters. Range expressions are supported; for example, `[a-z]` matches any lowercase ASCII letter. Multiple ranges can be combined: `[a-zA-Z0-9_]` matches any ASCII letter, digit, or underscore.

`[!seq]`  
Matches one character not in *seq*, where *seq* follows the same rules as above.

For a literal match, wrap the meta-characters in brackets. For example, `"[?]"` matches the character `"?"`.

The "`**`" wildcard enables recursive globbing. A few examples:

| Pattern | Meaning |
|----|----|
| "`**/*`" | Any path with at least one segment. |
| "`**/*.py`" | Any path with a final segment ending "`.py`". |
| "`assets/**`" | Any path starting with "`assets/`". |
| "`assets/**/*`" | Any path starting with "`assets/`", excluding "`assets/`" itself. |

> [!NOTE]
> Globbing with the "`**`" wildcard visits every directory in the tree. Large directory trees may take a long time to search.

<div class="versionchanged">

3.13 Globbing with a pattern that ends with "`**`" returns both files and directories. In previous versions, only directories were returned.

</div>

In `Path.glob` and `~Path.rglob`, a trailing slash may be added to the pattern to match only directories.

<div class="versionchanged">

3.11 Globbing with a pattern that ends with a pathname components separator (`~os.sep` or `~os.altsep`) returns only directories.

</div>

## Comparison to the `glob` module

The patterns accepted and results generated by `Path.glob` and `Path.rglob` differ slightly from those by the `glob` module:

1.  Files beginning with a dot are not special in pathlib. This is like passing `include_hidden=True` to `glob.glob`.
2.  "`**`" pattern components are always recursive in pathlib. This is like passing `recursive=True` to `glob.glob`.
3.  "`**`" pattern components do not follow symlinks by default in pathlib. This behaviour has no equivalent in `glob.glob`, but you can pass `recurse_symlinks=True` to `Path.glob` for compatible behaviour.
4.  Like all `PurePath` and `Path` objects, the values returned from `Path.glob` and `Path.rglob` don't include trailing slashes.
5.  The values returned from pathlib's `path.glob()` and `path.rglob()` include the *path* as a prefix, unlike the results of `glob.glob(root_dir=path)`.
6.  The values returned from pathlib's `path.glob()` and `path.rglob()` may include *path* itself, for example when globbing "`**`", whereas the results of `glob.glob(root_dir=path)` never include an empty string that would correspond to *path*.

## Comparison to the `os` and `os.path` modules

pathlib implements path operations using `PurePath` and `Path` objects, and so it's said to be *object-oriented*. On the other hand, the `os` and `os.path` modules supply functions that work with low-level `str` and `bytes` objects, which is a more *procedural* approach. Some users consider the object-oriented style to be more readable.

Many functions in `os` and `os.path` support `bytes` paths and `paths relative to directory descriptors <dir_fd>`. These features aren't available in pathlib.

Python's `str` and `bytes` types, and portions of the `os` and `os.path` modules, are written in C and are very speedy. pathlib is written in pure Python and is often slower, but rarely slow enough to matter.

pathlib's path normalization is slightly more opinionated and consistent than `os.path`. For example, whereas `os.path.abspath` eliminates "`..`" segments from a path, which may change its meaning if symlinks are involved, `Path.absolute` preserves these segments for greater safety.

pathlib's path normalization may render it unsuitable for some applications:

1.  pathlib normalizes `Path("my_folder/")` to `Path("my_folder")`, which changes a path's meaning when supplied to various operating system APIs and command-line utilities. Specifically, the absence of a trailing separator may allow the path to be resolved as either a file or directory, rather than a directory only.
2.  pathlib normalizes `Path("./my_program")` to `Path("my_program")`, which changes a path's meaning when used as an executable search path, such as in a shell or when spawning a child process. Specifically, the absence of a separator in the path may force it to be looked up in `PATH` rather than the current directory.

As a consequence of these differences, pathlib is not a drop-in replacement for `os.path`.

### Corresponding tools

Below is a table mapping various `os` functions to their corresponding `PurePath`/`Path` equivalent.

| `os` and `os.path`        | `!pathlib`                         |
|---------------------------|------------------------------------|
| `os.path.dirname`         | `PurePath.parent`                  |
| `os.path.basename`        | `PurePath.name`                    |
| `os.path.splitext`        | `PurePath.stem`, `PurePath.suffix` |
| `os.path.join`            | `PurePath.joinpath`                |
| `os.path.isabs`           | `PurePath.is_absolute`             |
| `os.path.relpath`         | `PurePath.relative_to`[^1]         |
| `os.path.expanduser`      | `Path.expanduser`[^2]              |
| `os.path.realpath`        | `Path.resolve`                     |
| `os.path.abspath`         | `Path.absolute`[^3]                |
| `os.path.exists`          | `Path.exists`                      |
| `os.path.isfile`          | `Path.is_file`                     |
| `os.path.isdir`           | `Path.is_dir`                      |
| `os.path.islink`          | `Path.is_symlink`                  |
| `os.path.isjunction`      | `Path.is_junction`                 |
| `os.path.ismount`         | `Path.is_mount`                    |
| `os.path.samefile`        | `Path.samefile`                    |
| `os.getcwd`               | `Path.cwd`                         |
| `os.stat`                 | `Path.stat`                        |
| `os.lstat`                | `Path.lstat`                       |
| `os.listdir`              | `Path.iterdir`                     |
| `os.walk`                 | `Path.walk`[^4]                    |
| `os.mkdir`, `os.makedirs` | `Path.mkdir`                       |
| `os.link`                 | `Path.hardlink_to`                 |
| `os.symlink`              | `Path.symlink_to`                  |
| `os.readlink`             | `Path.readlink`                    |
| `os.rename`               | `Path.rename`                      |
| `os.replace`              | `Path.replace`                     |
| `os.remove`, `os.unlink`  | `Path.unlink`                      |
| `os.rmdir`                | `Path.rmdir`                       |
| `os.chmod`                | `Path.chmod`                       |
| `os.lchmod`               | `Path.lchmod`                      |

**Footnotes**

## Protocols

<div class="module" data-synopsis="pathlib types for static type checking">

pathlib.types

</div>

The `!pathlib.types` module provides types for static type checking.

<div class="versionadded">

3.14

</div>

<div class="PathInfo()">

A `typing.Protocol` describing the `Path.info <pathlib.Path.info>` attribute. Implementations may return cached results from their methods.

<div class="method">

exists(\*, follow_symlinks=True)

Return `True` if the path is an existing file or directory, or any other kind of file; return `False` if the path doesn't exist.

If *follow_symlinks* is `False`, return `True` for symlinks without checking if their targets exist.

</div>

<div class="method">

is_dir(\*, follow_symlinks=True)

Return `True` if the path is a directory, or a symbolic link pointing to a directory; return `False` if the path is (or points to) any other kind of file, or if it doesn't exist.

If *follow_symlinks* is `False`, return `True` only if the path is a directory (without following symlinks); return `False` if the path is any other kind of file, or if it doesn't exist.

</div>

<div class="method">

is_file(\*, follow_symlinks=True)

Return `True` if the path is a file, or a symbolic link pointing to a file; return `False` if the path is (or points to) a directory or other non-file, or if it doesn't exist.

If *follow_symlinks* is `False`, return `True` only if the path is a file (without following symlinks); return `False` if the path is a directory or other non-file, or if it doesn't exist.

</div>

<div class="method">

is_symlink()

Return `True` if the path is a symbolic link (even if broken); return `False` if the path is a directory or any kind of file, or if it doesn't exist.

</div>

</div>

[^1]: `os.path.relpath` calls `~os.path.abspath` to make paths absolute and remove "`..`" parts, whereas `PurePath.relative_to` is a lexical operation that raises `ValueError` when its inputs' anchors differ (e.g. if one path is absolute and the other relative.)

[^2]: `os.path.expanduser` returns the path unchanged if the home directory can't be resolved, whereas `Path.expanduser` raises `RuntimeError`.

[^3]: `os.path.abspath` removes "`..`" components without resolving symlinks, which may change the meaning of the path, whereas `Path.absolute` leaves any "`..`" components in the path.

[^4]: `os.walk` always follows symlinks when categorizing paths into *dirnames* and *filenames*, whereas `Path.walk` categorizes all symlinks into *filenames* when *follow_symlinks* is false (the default.)

# `!pdb` --- The Python Debugger

<div class="module" data-synopsis="The Python debugger for interactive interpreters.">

pdb

</div>

**Source code:** `Lib/pdb.py`

<div class="index">

single: debugging

</div>

------------------------------------------------------------------------

The module `!pdb` defines an interactive source code debugger for Python programs. It supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame. It also supports post-mortem debugging and can be called under program control.

<div class="index">

single: Pdb (class in pdb) pair: module; bdb pair: module; cmd

</div>

The debugger is extensible -- it is actually defined as the class `Pdb`. This is currently undocumented but easily understood by reading the source. The extension interface uses the modules `bdb` and `cmd`.

<div class="seealso">

Module `faulthandler`  
Used to dump Python tracebacks explicitly, on a fault, after a timeout, or on a user signal.

Module `traceback`  
Standard interface to extract, format and print stack traces of Python programs.

</div>

The typical usage to break into the debugger is to insert:

    import pdb; pdb.set_trace()

Or:

    breakpoint()

at the location you want to break into the debugger, and then run the program. You can then step through the code following this statement, and continue running without the debugger using the `continue` command.

<div class="versionchanged">

3.7 The built-in `breakpoint`, when called with defaults, can be used instead of `import pdb; pdb.set_trace()`.

</div>

    def double(x):
       breakpoint()
       return x * 2
    val = 3
    print(f"{val} * 2 is {double(val)}")

The debugger's prompt is `(Pdb)`, which is the indicator that you are in debug mode:

    > ...(2)double()
    -> breakpoint()
    (Pdb) p x
    3
    (Pdb) continue
    3 * 2 is 6

<div class="versionchanged">

3.3 Tab-completion via the `readline` module is available for commands and command arguments, e.g. the current global and local names are offered as arguments of the `p` command.

</div>

## Command-line interface

<div class="program">

pdb

</div>

You can also invoke `!pdb` from the command line to debug other scripts. For example:

    python -m pdb [-c command] (-m module | -p pid | pyfile) [args ...]

When invoked as a module, pdb will automatically enter post-mortem debugging if the program being debugged exits abnormally. After post-mortem debugging (or after normal exit of the program), pdb will restart the program. Automatic restarting preserves pdb's state (such as breakpoints) and in most cases is more useful than quitting the debugger upon program's exit.

<div class="option">

`-c, --command <command>`  

To execute commands as if given in a `.pdbrc` file; see `debugger-commands`.

<div class="versionchanged">

3.2 Added the `-c` option.

</div>

</div>

<div class="option">

`-m <module>`  

To execute modules similar to the way `python -m` does. As with a script, the debugger will pause execution just before the first line of the module.

<div class="versionchanged">

3.7 Added the `-m` option.

</div>

</div>

<div class="option">

`-p, --pid <pid>`  

Attach to the process with the specified PID.

<div class="versionadded">

3.14

</div>

</div>

To attach to a running Python process for remote debugging, use the `-p` or `--pid` option with the target process's PID:

    python -m pdb -p 1234

> [!NOTE]
> Attaching to a process that is blocked in a system call or waiting for I/O will only work once the next bytecode instruction is executed or when the process receives a signal.

Typical usage to execute a statement under control of the debugger is:

    >>> import pdb
    >>> def f(x):
    ...     print(1 / x)
    >>> pdb.run("f(2)")
    > <string>(1)<module>()
    (Pdb) continue
    0.5
    >>>

The typical usage to inspect a crashed program is:

    >>> import pdb
    >>> def f(x):
    ...     print(1 / x)
    ...
    >>> f(0)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 2, in f
    ZeroDivisionError: division by zero
    >>> pdb.pm()
    > <stdin>(2)f()
    (Pdb) p x
    0
    (Pdb)

<div class="versionchanged">

3.13 The implementation of `667` means that name assignments made via `pdb` will immediately affect the active scope, even when running inside an `optimized scope`.

</div>

The module defines the following functions; each enters the debugger in a slightly different way:

<div class="function">

run(statement, globals=None, locals=None)

Execute the *statement* (given as a string or a code object) under debugger control. The debugger prompt appears before any code is executed; you can set breakpoints and type `continue`, or you can step through the statement using `step` or `next` (all these commands are explained below). The optional *globals* and *locals* arguments specify the environment in which the code is executed; by default the dictionary of the module `__main__` is used. (See the explanation of the built-in `exec` or `eval` functions.)

</div>

<div class="function">

runeval(expression, globals=None, locals=None)

Evaluate the *expression* (given as a string or a code object) under debugger control. When `runeval` returns, it returns the value of the *expression*. Otherwise this function is similar to `run`.

</div>

<div class="function">

runcall(function, *args,*\*kwds)

Call the *function* (a function or method object, not a string) with the given arguments. When `runcall` returns, it returns whatever the function call returned. The debugger prompt appears as soon as the function is entered.

</div>

<div class="function">

set_trace(\*, header=None, commands=None)

Enter the debugger at the calling stack frame. This is useful to hard-code a breakpoint at a given point in a program, even if the code is not otherwise being debugged (e.g. when an assertion fails). If given, *header* is printed to the console just before debugging begins. The *commands* argument, if given, is a list of commands to execute when the debugger starts.

<div class="versionchanged">

3.7 The keyword-only argument *header*.

</div>

<div class="versionchanged">

3.13 `set_trace` will enter the debugger immediately, rather than on the next line of code to be executed.

</div>

<div class="versionadded">

3.14 The *commands* argument.

</div>

</div>

<div class="awaitablefunction">

set_trace_async(\*, header=None, commands=None)

async version of `set_trace`. This function should be used inside an async function with `await`.

``` python
async def f():
    await pdb.set_trace_async()
```

`await` statements are supported if the debugger is invoked by this function.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

post_mortem(t=None)

Enter post-mortem debugging of the given exception or `traceback object <traceback-objects>`. If no value is given, it uses the exception that is currently being handled, or raises `ValueError` if there isn’t one.

<div class="versionchanged">

3.13 Support for exception objects was added.

</div>

</div>

<div class="function">

pm()

Enter post-mortem debugging of the exception found in `sys.last_exc`.

</div>

<div class="function">

set_default_backend(backend)

There are two supported backends for pdb: `'settrace'` and `'monitoring'`. See `bdb.Bdb` for details. The user can set the default backend to use if none is specified when instantiating `Pdb`. If no backend is specified, the default is `'settrace'`.

> [!NOTE]
> `breakpoint` and `set_trace` will not be affected by this function. They always use `'monitoring'` backend.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

get_default_backend()

Returns the default backend for pdb.

<div class="versionadded">

3.14

</div>

</div>

The `run*` functions and `set_trace` are aliases for instantiating the `Pdb` class and calling the method of the same name. If you want to access further features, you have to do this yourself:

<div class="Pdb(completekey='tab', stdin=None, stdout=None, skip=None, \ nosigint=False, readrc=True, mode=None, backend=None, colorize=False)">

`Pdb` is the debugger class.

The *completekey*, *stdin* and *stdout* arguments are passed to the underlying `cmd.Cmd` class; see the description there.

The *skip* argument, if given, must be an iterable of glob-style module name patterns. The debugger will not step into frames that originate in a module that matches one of these patterns.[^1]

By default, Pdb sets a handler for the SIGINT signal (which is sent when the user presses `Ctrl-C` on the console) when you give a `continue` command. This allows you to break into the debugger again by pressing `Ctrl-C`. If you want Pdb not to touch the SIGINT handler, set *nosigint* to true.

The *readrc* argument defaults to true and controls whether Pdb will load .pdbrc files from the filesystem.

The *mode* argument specifies how the debugger was invoked. It impacts the workings of some debugger commands. Valid values are `'inline'` (used by the breakpoint() builtin), `'cli'` (used by the command line invocation) or `None` (for backwards compatible behaviour, as before the *mode* argument was added).

The *backend* argument specifies the backend to use for the debugger. If `None` is passed, the default backend will be used. See `set_default_backend`. Otherwise the supported backends are `'settrace'` and `'monitoring'`.

The *colorize* argument, if set to `True`, will enable colorized output in the debugger, if color is supported. This will highlight source code displayed in pdb.

Example call to enable tracing with *skip*:

    import pdb; pdb.Pdb(skip=['django.*']).set_trace()

<div class="audit-event">

pdb.Pdb "" pdb.Pdb

</div>

<div class="versionchanged">

3.1 Added the *skip* parameter.

</div>

<div class="versionchanged">

3.2 Added the *nosigint* parameter. Previously, a SIGINT handler was never set by Pdb.

</div>

<div class="versionchanged">

3.6 The *readrc* argument.

</div>

<div class="versionadded">

3.14 Added the *mode* argument.

</div>

<div class="versionadded">

3.14 Added the *backend* argument.

</div>

<div class="versionadded">

3.14 Added the *colorize* argument.

</div>

<div class="versionchanged">

3.14 Inline breakpoints like `breakpoint` or `pdb.set_trace` will always stop the program at calling frame, ignoring the *skip* pattern (if any).

</div>

<div class="method">

run(statement, globals=None, locals=None) runeval(expression, globals=None, locals=None) runcall(function, *args,*\*kwds) set_trace()

See the documentation for the functions explained above.

</div>

</div>

## Debugger commands

The commands recognized by the debugger are listed below. Most commands can be abbreviated to one or two letters as indicated; e.g. `h(elp)` means that either `h` or `help` can be used to enter the help command (but not `he` or `hel`, nor `H` or `Help` or `HELP`). Arguments to commands must be separated by whitespace (spaces or tabs). Optional arguments are enclosed in square brackets (`[]`) in the command syntax; the square brackets must not be typed. Alternatives in the command syntax are separated by a vertical bar (`|`).

Entering a blank line repeats the last command entered. Exception: if the last command was a `list` command, the next 11 lines are listed.

Commands that the debugger doesn't recognize are assumed to be Python statements and are executed in the context of the program being debugged. Python statements can also be prefixed with an exclamation point (`!`). This is a powerful way to inspect the program being debugged; it is even possible to change a variable or call a function. When an exception occurs in such a statement, the exception name is printed but the debugger's state is not changed.

<div class="versionchanged">

3.13 Expressions/Statements whose prefix is a pdb command are now correctly identified and executed.

</div>

The debugger supports `aliases <debugger-aliases>`. Aliases can have parameters which allows one a certain level of adaptability to the context under examination.

Multiple commands may be entered on a single line, separated by `;;`. (A single `;` is not used as it is the separator for multiple commands in a line that is passed to the Python parser.) No intelligence is applied to separating the commands; the input is split at the first `;;` pair, even if it is in the middle of a quoted string. A workaround for strings with double semicolons is to use implicit string concatenation `';'';'` or `";"";"`.

To set a temporary global variable, use a *convenience variable*. A *convenience variable* is a variable whose name starts with `$`. For example, `$foo = 1` sets a global variable `$foo` which you can use in the debugger session. The *convenience variables* are cleared when the program resumes execution so it's less likely to interfere with your program compared to using normal variables like `foo = 1`.

There are four preset *convenience variables*:

- `$_frame`: the current frame you are debugging
- `$_retval`: the return value if the frame is returning
- `$_exception`: the exception if the frame is raising an exception
- `$_asynctask`: the asyncio task if pdb stops in an async function

<div class="versionadded">

3.12

Added the *convenience variable* feature.

</div>

<div class="versionadded">

3.14 Added the `$_asynctask` convenience variable.

</div>

<div class="index">

pair: .pdbrc; file triple: debugger; configuration; file

</div>

If a file `.pdbrc` exists in the user's home directory or in the current directory, it is read with `'utf-8'` encoding and executed as if it had been typed at the debugger prompt, with the exception that empty lines and lines starting with `#` are ignored. This is particularly useful for aliases. If both files exist, the one in the home directory is read first and aliases defined there can be overridden by the local file.

<div class="versionchanged">

3.2 `.pdbrc` can now contain commands that continue debugging, such as `continue` or `next`. Previously, these commands had no effect.

</div>

<div class="versionchanged">

3.11 `.pdbrc` is now read with `'utf-8'` encoding. Previously, it was read with the system locale encoding.

</div>

<div class="pdbcommand">

h(elp) \[command\]

Without argument, print the list of available commands. With a *command* as argument, print help about that command. `help pdb` displays the full documentation (the docstring of the `pdb` module). Since the *command* argument must be an identifier, `help exec` must be entered to get help on the `!` command.

</div>

<div class="pdbcommand">

w(here) \[count\]

Print a stack trace, with the most recent frame at the bottom. if *count* is 0, print the current frame entry. If *count* is negative, print the least recent - *count* frames. If *count* is positive, print the most recent *count* frames. An arrow (`>`) indicates the current frame, which determines the context of most commands.

<div class="versionchanged">

3.14 *count* argument is added.

</div>

</div>

<div class="pdbcommand">

d(own) \[count\]

Move the current frame *count* (default one) levels down in the stack trace (to a newer frame).

</div>

<div class="pdbcommand">

u(p) \[count\]

Move the current frame *count* (default one) levels up in the stack trace (to an older frame).

</div>

<div class="pdbcommand">

b(reak) \[(\[filename:\]lineno \| function) \[, condition\]\]

With a *lineno* argument, set a break at line *lineno* in the current file. The line number may be prefixed with a *filename* and a colon, to specify a breakpoint in another file (possibly one that hasn't been loaded yet). The file is searched on `sys.path`. Acceptable forms of *filename* are `/abspath/to/file.py`, `relpath/file.py`, `module` and `package.module`.

With a *function* argument, set a break at the first executable statement within that function. *function* can be any expression that evaluates to a function in the current namespace, or in the form `package.module.function` if `package.module` is importable.

If a second argument is present, it is an expression which must evaluate to true before the breakpoint is honored.

Without argument, list all breaks, including for each breakpoint, the number of times that breakpoint has been hit, the current ignore count, and the associated condition if any.

Each breakpoint is assigned a number to which all the other breakpoint commands refer.

</div>

<div class="pdbcommand">

tbreak \[(\[filename:\]lineno \| function) \[, condition\]\]

Temporary breakpoint, which is removed automatically when it is first hit. The arguments are the same as for `break`.

</div>

<div class="pdbcommand">

cl(ear) \[filename:lineno \| bpnumber ...\]

With a *filename:lineno* argument, clear all the breakpoints at this line. With a space separated list of breakpoint numbers, clear those breakpoints. Without argument, clear all breaks (but first ask confirmation).

</div>

<div class="pdbcommand">

disable bpnumber \[bpnumber ...\]

Disable the breakpoints given as a space separated list of breakpoint numbers. Disabling a breakpoint means it cannot cause the program to stop execution, but unlike clearing a breakpoint, it remains in the list of breakpoints and can be (re-)enabled.

</div>

<div class="pdbcommand">

enable bpnumber \[bpnumber ...\]

Enable the breakpoints specified.

</div>

<div class="pdbcommand">

ignore bpnumber \[count\]

Set the ignore count for the given breakpoint number. If *count* is omitted, the ignore count is set to 0. A breakpoint becomes active when the ignore count is zero. When non-zero, the *count* is decremented each time the breakpoint is reached and the breakpoint is not disabled and any associated condition evaluates to true.

</div>

<div class="pdbcommand">

condition bpnumber \[condition\]

Set a new *condition* for the breakpoint, an expression which must evaluate to true before the breakpoint is honored. If *condition* is absent, any existing condition is removed; i.e., the breakpoint is made unconditional.

</div>

<div class="pdbcommand">

commands \[bpnumber\]

Specify a list of commands for breakpoint number *bpnumber*. The commands themselves appear on the following lines. Type a line containing just `end` to terminate the commands. An example:

    (Pdb) commands 1
    (com) p some_variable
    (com) end
    (Pdb)

To remove all commands from a breakpoint, type `commands` and follow it immediately with `end`; that is, give no commands.

With no *bpnumber* argument, `commands` refers to the most recently set breakpoint that still exists.

You can use breakpoint commands to start your program up again. Simply use the `continue` command, or `step`, or any other command that resumes execution.

Specifying any command resuming execution (currently `continue`, `step`, `next`, `return`, `until`, `jump`, `quit` and their abbreviations) terminates the command list (as if that command was immediately followed by end). This is because any time you resume execution (even with a simple next or step), you may encounter another breakpoint—which could have its own command list, leading to ambiguities about which list to execute.

If the list of commands contains the `silent` command, or a command that resumes execution, then the breakpoint message containing information about the frame is not displayed.

<div class="versionchanged">

3.14 Frame information will not be displayed if a command that resumes execution is present in the command list.

</div>

</div>

<div class="pdbcommand">

s(tep)

Execute the current line, stop at the first possible occasion (either in a function that is called or on the next line in the current function).

</div>

<div class="pdbcommand">

n(ext)

Continue execution until the next line in the current function is reached or it returns. (The difference between `next` and `step` is that `step` stops inside a called function, while `next` executes called functions at (nearly) full speed, only stopping at the next line in the current function.)

</div>

<div class="pdbcommand">

unt(il) \[lineno\]

Without argument, continue execution until the line with a number greater than the current one is reached.

With *lineno*, continue execution until a line with a number greater or equal to *lineno* is reached. In both cases, also stop when the current frame returns.

<div class="versionchanged">

3.2 Allow giving an explicit line number.

</div>

</div>

<div class="pdbcommand">

r(eturn)

Continue execution until the current function returns.

</div>

<div class="pdbcommand">

c(ont(inue))

Continue execution, only stop when a breakpoint is encountered.

</div>

<div class="pdbcommand">

j(ump) lineno

Set the next line that will be executed. Only available in the bottom-most frame. This lets you jump back and execute code again, or jump forward to skip code that you don't want to run.

It should be noted that not all jumps are allowed -- for instance it is not possible to jump into the middle of a `for` loop or out of a `finally` clause.

</div>

<div class="pdbcommand">

l(ist) \[first\[, last\]\]

List source code for the current file. Without arguments, list 11 lines around the current line or continue the previous listing. With `.` as argument, list 11 lines around the current line. With one argument, list 11 lines around at that line. With two arguments, list the given range; if the second argument is less than the first, it is interpreted as a count.

The current line in the current frame is indicated by `->`. If an exception is being debugged, the line where the exception was originally raised or propagated is indicated by `>>`, if it differs from the current line.

<div class="versionchanged">

3.2 Added the `>>` marker.

</div>

</div>

<div class="pdbcommand">

ll \| longlist

List all source code for the current function or frame. Interesting lines are marked as for `list`.

<div class="versionadded">

3.2

</div>

</div>

<div class="pdbcommand">

a(rgs)

Print the arguments of the current function and their current values.

</div>

<div class="pdbcommand">

p expression

Evaluate *expression* in the current context and print its value.

> [!NOTE]
> `print()` can also be used, but is not a debugger command --- this executes the Python `print` function.

</div>

<div class="pdbcommand">

pp expression

Like the `p` command, except the value of *expression* is pretty-printed using the `pprint` module.

</div>

<div class="pdbcommand">

whatis expression

Print the type of *expression*.

</div>

<div class="pdbcommand">

source expression

Try to get source code of *expression* and display it.

<div class="versionadded">

3.2

</div>

</div>

<div class="pdbcommand">

display \[expression\]

Display the value of *expression* if it changed, each time execution stops in the current frame.

Without *expression*, list all display expressions for the current frame.

> [!NOTE]
> Display evaluates *expression* and compares to the result of the previous evaluation of *expression*, so when the result is mutable, display may not be able to pick up the changes.

Example:

    lst = []
    breakpoint()
    pass
    lst.append(1)
    print(lst)

Display won't realize `lst` has been changed because the result of evaluation is modified in place by `lst.append(1)` before being compared:

    > example.py(3)<module>()
    -> pass
    (Pdb) display lst
    display lst: []
    (Pdb) n
    > example.py(4)<module>()
    -> lst.append(1)
    (Pdb) n
    > example.py(5)<module>()
    -> print(lst)
    (Pdb)

You can do some tricks with copy mechanism to make it work:

    > example.py(3)<module>()
    -> pass
    (Pdb) display lst[:]
    display lst[:]: []
    (Pdb) n
    > example.py(4)<module>()
    -> lst.append(1)
    (Pdb) n
    > example.py(5)<module>()
    -> print(lst)
    display lst[:]: [1]  [old: []]
    (Pdb)

<div class="versionadded">

3.2

</div>

</div>

<div class="pdbcommand">

undisplay \[expression\]

Do not display *expression* anymore in the current frame. Without *expression*, clear all display expressions for the current frame.

<div class="versionadded">

3.2

</div>

</div>

<div class="pdbcommand">

interact

Start an interactive interpreter (using the `code` module) in a new global namespace initialised from the local and global namespaces for the current scope. Use `exit()` or `quit()` to exit the interpreter and return to the debugger.

> [!NOTE]
> As `interact` creates a new dedicated namespace for code execution, assignments to variables will not affect the original namespaces. However, modifications to any referenced mutable objects will be reflected in the original namespaces as usual.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.13 `exit()` and `quit()` can be used to exit the `interact` command.

</div>

<div class="versionchanged">

3.13 `interact` directs its output to the debugger's output channel rather than `sys.stderr`.

</div>

</div>

<div id="debugger-aliases">

<div class="pdbcommand">

alias \[name \[command\]\]

Create an alias called *name* that executes *command*. The *command* must *not* be enclosed in quotes. Replaceable parameters can be indicated by `%1`, `%2`, ... and `%9`, while `%*` is replaced by all the parameters. If *command* is omitted, the current alias for *name* is shown. If no arguments are given, all aliases are listed.

Aliases may be nested and can contain anything that can be legally typed at the pdb prompt. Note that internal pdb commands *can* be overridden by aliases. Such a command is then hidden until the alias is removed. Aliasing is recursively applied to the first word of the command line; all other words in the line are left alone.

As an example, here are two useful aliases (especially when placed in the `.pdbrc` file):

    # Print instance variables (usage "pi classInst")
    alias pi for k in %1.__dict__.keys(): print(f"%1.{k} = {%1.__dict__[k]}")
    # Print instance variables in self
    alias ps pi self

</div>

</div>

<div class="pdbcommand">

unalias name

Delete the specified alias *name*.

</div>

<div class="pdbcommand">

! statement

Execute the (one-line) *statement* in the context of the current stack frame. The exclamation point can be omitted unless the first word of the statement resembles a debugger command, e.g.:

```
(Pdb) ! n=42
(Pdb)
```

To set a global variable, you can prefix the assignment command with a `global` statement on the same line, e.g.:

```
(Pdb) global list_options; list_options = ['-l']
(Pdb)
```

</div>

<div class="pdbcommand">

run \[args ...\] restart \[args ...\]

Restart the debugged Python program. If *args* is supplied, it is split with `shlex` and the result is used as the new `sys.argv`. History, breakpoints, actions and debugger options are preserved. `restart` is an alias for `run`.

<div class="versionchanged">

3.14 `run` and `restart` commands are disabled when the debugger is invoked in `'inline'` mode.

</div>

</div>

<div class="pdbcommand">

q(uit)

Quit from the debugger. The program being executed is aborted. An end-of-file input is equivalent to `quit`.

A confirmation prompt will be shown if the debugger is invoked in `'inline'` mode. Either `y`, `Y`, `<Enter>` or `EOF` will confirm the quit.

<div class="versionchanged">

3.14 A confirmation prompt will be shown if the debugger is invoked in `'inline'` mode. After the confirmation, the debugger will call `sys.exit` immediately, instead of raising `bdb.BdbQuit` in the next trace event.

</div>

</div>

<div class="pdbcommand">

debug code

Enter a recursive debugger that steps through *code* (which is an arbitrary expression or statement to be executed in the current environment).

</div>

<div class="pdbcommand">

retval

Print the return value for the last return of the current function.

</div>

<div class="pdbcommand">

exceptions \[excnumber\]

List or jump between chained exceptions.

When using `pdb.pm()` or `Pdb.post_mortem(...)` with a chained exception instead of a traceback, it allows the user to move between the chained exceptions using `exceptions` command to list exceptions, and `exceptions <number>` to switch to that exception.

Example:

    def out():
        try:
            middle()
        except Exception as e:
            raise ValueError("reraise middle() error") from e

    def middle():
        try:
            return inner(0)
        except Exception as e:
            raise ValueError("Middle fail")

    def inner(x):
        1 / x

    out()

calling `pdb.pm()` will allow to move between exceptions:

    > example.py(5)out()
    -> raise ValueError("reraise middle() error") from e

    (Pdb) exceptions
      0 ZeroDivisionError('division by zero')
      1 ValueError('Middle fail')
    > 2 ValueError('reraise middle() error')

    (Pdb) exceptions 0
    > example.py(16)inner()
    -> 1 / x

    (Pdb) up
    > example.py(10)middle()
    -> return inner(0)

<div class="versionadded">

3.13

</div>

</div>

**Footnotes**

[^1]: Whether a frame is considered to originate in a certain module is determined by the `__name__` in the frame globals.

# Data Persistence

The modules described in this chapter support storing Python data in a persistent form on disk. The `pickle` and `marshal` modules can turn many Python data types into a stream of bytes and then recreate the objects from the bytes. The various DBM-related modules support a family of hash-based file formats that store a mapping of strings to other strings.

The list of modules described in this chapter is:

<div class="toctree">

pickle.rst copyreg.rst shelve.rst marshal.rst dbm.rst sqlite3.rst

</div>

# `!pickle` --- Python object serialization

<div class="module" data-synopsis="Convert Python objects to streams of bytes and back.">

pickle

</div>

**Source code:** `Lib/pickle.py`

<div class="index">

single: persistence pair: persistent; objects pair: serializing; objects pair: marshalling; objects pair: flattening; objects pair: pickling; objects

</div>

------------------------------------------------------------------------

The `!pickle` module implements binary protocols for serializing and de-serializing a Python object structure. *"Pickling"* is the process whereby a Python object hierarchy is converted into a byte stream, and *"unpickling"* is the inverse operation, whereby a byte stream (from a `binary file` or `bytes-like object`) is converted back into an object hierarchy. Pickling (and unpickling) is alternatively known as "serialization", "marshalling,"[^1] or "flattening"; however, to avoid confusion, the terms used here are "pickling" and "unpickling".

> [!WARNING]
> The `pickle` module **is not secure**. Only unpickle data you trust.
>
> It is possible to construct malicious pickle data which will **execute arbitrary code during unpickling**. Never unpickle data that could have come from an untrusted source, or that could have been tampered with.
>
> Consider signing data with `hmac` if you need to ensure that it has not been tampered with.
>
> Safer serialization formats such as `json` may be more appropriate if you are processing untrusted data. See `comparison-with-json`.

## Relationship to other Python modules

### Comparison with `marshal`

Python has a more primitive serialization module called `marshal`, but in general `!pickle` should always be the preferred way to serialize Python objects. `marshal` exists primarily to support Python's `.pyc` files.

The `!pickle` module differs from `marshal` in several significant ways:

- `marshal` cannot be used to serialize user-defined classes and their instances. `!pickle` can save and restore class instances transparently, however the class definition must be importable and live in the same module as when the object was pickled.
- The `marshal` serialization format is not guaranteed to be portable across Python versions. Because its primary job in life is to support `.pyc` files, the Python implementers reserve the right to change the serialization format in non-backwards compatible ways should the need arise. The `!pickle` serialization format is guaranteed to be backwards compatible across Python releases provided a compatible pickle protocol is chosen and pickling and unpickling code deals with Python 2 to Python 3 type differences if your data is crossing that unique breaking change language boundary.

### Comparison with `json`

There are fundamental differences between the pickle protocols and [JSON (JavaScript Object Notation)](https://json.org):

- JSON is a text serialization format (it outputs unicode text, although most of the time it is then encoded to `utf-8`), while pickle is a binary serialization format;
- JSON is human-readable, while pickle is not;
- JSON is interoperable and widely used outside of the Python ecosystem, while pickle is Python-specific;
- JSON, by default, can only represent a subset of the Python built-in types, and no custom classes; pickle can represent an extremely large number of Python types (many of them automatically, by clever usage of Python's introspection facilities; complex cases can be tackled by implementing `specific object APIs <pickle-inst>`);
- Unlike pickle, deserializing untrusted JSON does not in itself create an arbitrary code execution vulnerability.

<div class="seealso">

The `json` module: a standard library module allowing JSON serialization and deserialization.

</div>

## Data stream format

<div class="index">

single: External Data Representation

</div>

The data format used by `!pickle` is Python-specific. This has the advantage that there are no restrictions imposed by external standards such as JSON (which can't represent pointer sharing); however it means that non-Python programs may not be able to reconstruct pickled Python objects.

By default, the `!pickle` data format uses a relatively compact binary representation. If you need optimal size characteristics, you can efficiently `compress <archiving>` pickled data.

The module `pickletools` contains tools for analyzing data streams generated by `!pickle`. `pickletools` source code has extensive comments about opcodes used by pickle protocols.

There are currently 6 different protocols which can be used for pickling. The higher the protocol used, the more recent the version of Python needed to read the pickle produced.

- Protocol version 0 is the original "human-readable" protocol and is backwards compatible with earlier versions of Python.
- Protocol version 1 is an old binary format which is also compatible with earlier versions of Python.
- Protocol version 2 was introduced in Python 2.3. It provides much more efficient pickling of `new-style classes <new-style class>`. Refer to `307` for information about improvements brought by protocol 2.
- Protocol version 3 was added in Python 3.0. It has explicit support for `bytes` objects and cannot be unpickled by Python 2.x. This was the default protocol in Python 3.0--3.7.
- Protocol version 4 was added in Python 3.4. It adds support for very large objects, pickling more kinds of objects, and some data format optimizations. This was the default protocol in Python 3.8--3.13. Refer to `3154` for information about improvements brought by protocol 4.
- Protocol version 5 was added in Python 3.8. It adds support for out-of-band data and speedup for in-band data. It is the default protocol starting with Python 3.14. Refer to `574` for information about improvements brought by protocol 5.

> [!NOTE]
> Serialization is a more primitive notion than persistence; although `!pickle` reads and writes file objects, it does not handle the issue of naming persistent objects, nor the (even more complicated) issue of concurrent access to persistent objects. The `!pickle` module can transform a complex object into a byte stream and it can transform the byte stream into an object with the same internal structure. Perhaps the most obvious thing to do with these byte streams is to write them onto a file, but it is also conceivable to send them across a network or store them in a database. The `shelve` module provides a simple interface to pickle and unpickle objects on DBM-style database files.

## Module Interface

To serialize an object hierarchy, you simply call the `dumps` function. Similarly, to de-serialize a data stream, you call the `loads` function. However, if you want more control over serialization and de-serialization, you can create a `Pickler` or an `Unpickler` object, respectively.

The `!pickle` module provides the following constants:

<div class="data">

HIGHEST_PROTOCOL

An integer, the highest `protocol version <pickle-protocols>` available. This value can be passed as a *protocol* value to functions `dump` and `dumps` as well as the `Pickler` constructor.

</div>

<div class="data">

DEFAULT_PROTOCOL

An integer, the default `protocol version <pickle-protocols>` used for pickling. May be less than `HIGHEST_PROTOCOL`. Currently the default protocol is 5, introduced in Python 3.8 and incompatible with previous versions. This version introduces support for out-of-band buffers, where `3118`-compatible data can be transmitted separately from the main pickle stream.

<div class="versionchanged">

3.0

The default protocol is 3.

</div>

<div class="versionchanged">

3.8

The default protocol is 4.

</div>

<div class="versionchanged">

3.14

The default protocol is 5.

</div>

</div>

The `!pickle` module provides the following functions to make the pickling process more convenient:

<div class="function">

dump(obj, file, protocol=None, \*, fix_imports=True, buffer_callback=None)

Write the pickled representation of the object *obj* to the open `file object` *file*. This is equivalent to `Pickler(file, protocol).dump(obj)`.

Arguments *file*, *protocol*, *fix_imports* and *buffer_callback* have the same meaning as in the `Pickler` constructor.

<div class="versionchanged">

3.8 The *buffer_callback* argument was added.

</div>

</div>

<div class="function">

dumps(obj, protocol=None, \*, fix_imports=True, buffer_callback=None)

Return the pickled representation of the object *obj* as a `bytes` object, instead of writing it to a file.

Arguments *protocol*, *fix_imports* and *buffer_callback* have the same meaning as in the `Pickler` constructor.

<div class="versionchanged">

3.8 The *buffer_callback* argument was added.

</div>

</div>

<div class="function">

load(file, \*, fix_imports=True, encoding="ASCII", errors="strict", buffers=None)

Read the pickled representation of an object from the open `file object` *file* and return the reconstituted object hierarchy specified therein. This is equivalent to `Unpickler(file).load()`.

The protocol version of the pickle is detected automatically, so no protocol argument is needed. Bytes past the pickled representation of the object are ignored.

Arguments *file*, *fix_imports*, *encoding*, *errors*, *strict* and *buffers* have the same meaning as in the `Unpickler` constructor.

<div class="versionchanged">

3.8 The *buffers* argument was added.

</div>

</div>

<div class="function">

loads(data, /, \*, fix_imports=True, encoding="ASCII", errors="strict", buffers=None)

Return the reconstituted object hierarchy of the pickled representation *data* of an object. *data* must be a `bytes-like object`.

The protocol version of the pickle is detected automatically, so no protocol argument is needed. Bytes past the pickled representation of the object are ignored.

Arguments *fix_imports*, *encoding*, *errors*, *strict* and *buffers* have the same meaning as in the `Unpickler` constructor.

<div class="versionchanged">

3.8 The *buffers* argument was added.

</div>

</div>

The `!pickle` module defines three exceptions:

<div class="exception">

PickleError

Common base class for the other pickling exceptions. It inherits from `Exception`.

</div>

<div class="exception">

PicklingError

Error raised when an unpicklable object is encountered by `Pickler`. It inherits from `PickleError`.

Refer to `pickle-picklable` to learn what kinds of objects can be pickled.

</div>

<div class="exception">

UnpicklingError

Error raised when there is a problem unpickling an object, such as a data corruption or a security violation. It inherits from `PickleError`.

Note that other exceptions may also be raised during unpickling, including (but not necessarily limited to) AttributeError, EOFError, ImportError, and IndexError.

</div>

The `!pickle` module exports three classes, `Pickler`, `Unpickler` and `PickleBuffer`:

<div class="Pickler(file, protocol=None, *, fix_imports=True, buffer_callback=None)">

This takes a binary file for writing a pickle data stream.

The optional *protocol* argument, an integer, tells the pickler to use the given protocol; supported protocols are 0 to `HIGHEST_PROTOCOL`. If not specified, the default is `DEFAULT_PROTOCOL`. If a negative number is specified, `HIGHEST_PROTOCOL` is selected.

The *file* argument must have a write() method that accepts a single bytes argument. It can thus be an on-disk file opened for binary writing, an `io.BytesIO` instance, or any other custom object that meets this interface.

If *fix_imports* is true and *protocol* is less than 3, pickle will try to map the new Python 3 names to the old module names used in Python 2, so that the pickle data stream is readable with Python 2.

If *buffer_callback* is `None` (the default), buffer views are serialized into *file* as part of the pickle stream.

If *buffer_callback* is not `None`, then it can be called any number of times with a buffer view. If the callback returns a false value (such as `None`), the given buffer is `out-of-band <pickle-oob>`; otherwise the buffer is serialized in-band, i.e. inside the pickle stream.

It is an error if *buffer_callback* is not `None` and *protocol* is `None` or smaller than 5.

<div class="versionchanged">

3.8 The *buffer_callback* argument was added.

</div>

<div class="method">

dump(obj)

Write the pickled representation of *obj* to the open file object given in the constructor.

</div>

<div class="method">

persistent_id(obj)

Do nothing by default. This exists so a subclass can override it.

If `persistent_id` returns `None`, *obj* is pickled as usual. Any other value causes `Pickler` to emit the returned value as a persistent ID for *obj*. The meaning of this persistent ID should be defined by `Unpickler.persistent_load`. Note that the value returned by `persistent_id` cannot itself have a persistent ID.

See `pickle-persistent` for details and examples of uses.

<div class="versionchanged">

3.13 Add the default implementation of this method in the C implementation of `!Pickler`.

</div>

</div>

<div class="attribute">

dispatch_table

A pickler object's dispatch table is a registry of *reduction functions* of the kind which can be declared using `copyreg.pickle`. It is a mapping whose keys are classes and whose values are reduction functions. A reduction function takes a single argument of the associated class and should conform to the same interface as a `~object.__reduce__` method.

By default, a pickler object will not have a `dispatch_table` attribute, and it will instead use the global dispatch table managed by the `copyreg` module. However, to customize the pickling for a specific pickler object one can set the `dispatch_table` attribute to a dict-like object. Alternatively, if a subclass of `Pickler` has a `dispatch_table` attribute then this will be used as the default dispatch table for instances of that class.

See `pickle-dispatch` for usage examples.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

reducer_override(obj)

Special reducer that can be defined in `Pickler` subclasses. This method has priority over any reducer in the `dispatch_table`. It should conform to the same interface as a `~object.__reduce__` method, and can optionally return `NotImplemented` to fallback on `dispatch_table`-registered reducers to pickle `obj`.

For a detailed example, see `reducer_override`.

<div class="versionadded">

3.8

</div>

</div>

<div class="attribute">

fast

Deprecated. Enable fast mode if set to a true value. The fast mode disables the usage of memo, therefore speeding the pickling process by not generating superfluous PUT opcodes. It should not be used with self-referential objects, doing otherwise will cause `Pickler` to recurse infinitely.

Use `pickletools.optimize` if you need more compact pickles.

</div>

<div class="method">

clear_memo()

Clears the pickler's "memo".

The memo is the data structure that remembers which objects the pickler has already seen, so that shared or recursive objects are pickled by reference and not by value. This method is useful when re-using picklers.

</div>

</div>

<div class="Unpickler(file, *, fix_imports=True, encoding=&quot;ASCII&quot;, errors=&quot;strict&quot;, buffers=None)">

This takes a binary file for reading a pickle data stream.

The protocol version of the pickle is detected automatically, so no protocol argument is needed.

The argument *file* must have three methods, a read() method that takes an integer argument, a readinto() method that takes a buffer argument and a readline() method that requires no arguments, as in the `io.BufferedIOBase` interface. Thus *file* can be an on-disk file opened for binary reading, an `io.BytesIO` object, or any other custom object that meets this interface.

The optional arguments *fix_imports*, *encoding* and *errors* are used to control compatibility support for pickle stream generated by Python 2. If *fix_imports* is true, pickle will try to map the old Python 2 names to the new names used in Python 3. The *encoding* and *errors* tell pickle how to decode 8-bit string instances pickled by Python 2; these default to 'ASCII' and 'strict', respectively. The *encoding* can be 'bytes' to read these 8-bit string instances as bytes objects. Using `encoding='latin1'` is required for unpickling NumPy arrays and instances of `~datetime.datetime`, `~datetime.date` and `~datetime.time` pickled by Python 2.

If *buffers* is `None` (the default), then all data necessary for deserialization must be contained in the pickle stream. This means that the *buffer_callback* argument was `None` when a `Pickler` was instantiated (or when `dump` or `dumps` was called).

If *buffers* is not `None`, it should be an iterable of buffer-enabled objects that is consumed each time the pickle stream references an `out-of-band <pickle-oob>` buffer view. Such buffers have been given in order to the *buffer_callback* of a Pickler object.

<div class="versionchanged">

3.8 The *buffers* argument was added.

</div>

<div class="method">

load()

Read the pickled representation of an object from the open file object given in the constructor, and return the reconstituted object hierarchy specified therein. Bytes past the pickled representation of the object are ignored.

</div>

<div class="method">

persistent_load(pid)

Raise an `UnpicklingError` by default.

If defined, `persistent_load` should return the object specified by the persistent ID *pid*. If an invalid persistent ID is encountered, an `UnpicklingError` should be raised.

See `pickle-persistent` for details and examples of uses.

<div class="versionchanged">

3.13 Add the default implementation of this method in the C implementation of `!Unpickler`.

</div>

</div>

<div class="method">

find_class(module, name)

Import *module* if necessary and return the object called *name* from it, where the *module* and *name* arguments are `str` objects. Note, unlike its name suggests, `find_class` is also used for finding functions.

Subclasses may override this to gain control over what type of objects and how they can be loaded, potentially reducing security risks. Refer to `pickle-restrict` for details.

<div class="audit-event">

pickle.find_class module,name pickle.Unpickler.find_class

</div>

</div>

</div>

<div class="PickleBuffer(buffer)">

A wrapper for a buffer representing picklable data. *buffer* must be a `buffer-providing <bufferobjects>` object, such as a `bytes-like object` or a N-dimensional array.

`PickleBuffer` is itself a buffer provider, therefore it is possible to pass it to other APIs expecting a buffer-providing object, such as `memoryview`.

`PickleBuffer` objects can only be serialized using pickle protocol 5 or higher. They are eligible for `out-of-band serialization <pickle-oob>`.

<div class="versionadded">

3.8

</div>

<div class="method">

raw()

Return a `memoryview` of the memory area underlying this buffer. The returned object is a one-dimensional, C-contiguous memoryview with format `B` (unsigned bytes). `BufferError` is raised if the buffer is neither C- nor Fortran-contiguous.

</div>

<div class="method">

release()

Release the underlying buffer exposed by the PickleBuffer object.

</div>

</div>

## What can be pickled and unpickled?

The following types can be pickled:

- built-in constants (`None`, `True`, `False`, `Ellipsis`, and `NotImplemented`);
- integers, floating-point numbers, complex numbers;
- strings, bytes, bytearrays;
- tuples, lists, sets, and dictionaries containing only picklable objects;
- functions (built-in and user-defined) accessible from the top level of a module (using `def`, not `lambda`);
- classes accessible from the top level of a module;
- instances of such classes for which the result of calling `~object.__getstate__` is picklable (see section `pickle-inst` for details);
- weak references and the weak containers from the `weakref` module, provided that the weakly referenced objects are strongly referenced elsewhere in the same pickle (see `weakref.ref`).

Attempts to pickle unpicklable objects will raise the `PicklingError` exception; when this happens, an unspecified number of bytes may have already been written to the underlying file. Trying to pickle a highly recursive data structure may exceed the maximum recursion depth, a `RecursionError` will be raised in this case. You can carefully raise this limit with `sys.setrecursionlimit`.

Note that functions (built-in and user-defined) are pickled by fully `qualified name`, not by value.[^2] This means that only the function name is pickled, along with the name of the containing module and classes. Neither the function's code, nor any of its function attributes are pickled. Thus the defining module must be importable in the unpickling environment, and the module must contain the named object, otherwise an exception will be raised.[^3]

Similarly, classes are pickled by fully qualified name, so the same restrictions in the unpickling environment apply. Note that none of the class's code or data is pickled, so in the following example the class attribute `attr` is not restored in the unpickling environment:

    class Foo:
        attr = 'A class attribute'

    picklestring = pickle.dumps(Foo)

These restrictions are why picklable functions and classes must be defined at the top level of a module.

Similarly, when class instances are pickled, their class's code and data are not pickled along with them. Only the instance data are pickled. This is done on purpose, so you can fix bugs in a class or add methods to the class and still load objects that were created with an earlier version of the class. If you plan to have long-lived objects that will see many versions of a class, it may be worthwhile to put a version number in the objects so that suitable conversions can be made by the class's `~object.__setstate__` method.

## Pickling Class Instances

<div class="currentmodule">

None

</div>

In this section, we describe the general mechanisms available to you to define, customize, and control how class instances are pickled and unpickled.

In most cases, no additional code is needed to make instances picklable. By default, pickle will retrieve the class and the attributes of an instance via introspection. When a class instance is unpickled, its `~object.__init__` method is usually *not* invoked. The default behaviour first creates an uninitialized instance and then restores the saved attributes. The following code shows an implementation of this behaviour:

    def save(obj):
        return (obj.__class__, obj.__dict__)

    def restore(cls, attributes):
        obj = cls.__new__(cls)
        obj.__dict__.update(attributes)
        return obj

Classes can alter the default behaviour by providing one or several special methods:

<div class="method">

object.\_\_getnewargs_ex\_\_()

In protocols 2 and newer, classes that implement the `__getnewargs_ex__` method can dictate the values passed to the `__new__` method upon unpickling. The method must return a pair `(args, kwargs)` where *args* is a tuple of positional arguments and *kwargs* a dictionary of named arguments for constructing the object. Those will be passed to the `__new__` method upon unpickling.

You should implement this method if the `__new__` method of your class requires keyword-only arguments. Otherwise, it is recommended for compatibility to implement `__getnewargs__`.

<div class="versionchanged">

3.6 `__getnewargs_ex__` is now used in protocols 2 and 3.

</div>

</div>

<div class="method">

object.\_\_getnewargs\_\_()

This method serves a similar purpose as `__getnewargs_ex__`, but supports only positional arguments. It must return a tuple of arguments `args` which will be passed to the `__new__` method upon unpickling.

`__getnewargs__` will not be called if `__getnewargs_ex__` is defined.

<div class="versionchanged">

3.6 Before Python 3.6, `__getnewargs__` was called instead of `__getnewargs_ex__` in protocols 2 and 3.

</div>

</div>

<div class="method">

object.\_\_getstate\_\_()

Classes can further influence how their instances are pickled by overriding the method `__getstate__`. It is called and the returned object is pickled as the contents for the instance, instead of a default state. There are several cases:

- For a class that has no instance `~object.__dict__` and no `~object.__slots__`, the default state is `None`.
- For a class that has an instance `~object.__dict__` and no `~object.__slots__`, the default state is `self.__dict__`.
- For a class that has an instance `~object.__dict__` and `~object.__slots__`, the default state is a tuple consisting of two dictionaries: `self.__dict__`, and a dictionary mapping slot names to slot values. Only slots that have a value are included in the latter.
- For a class that has `~object.__slots__` and no instance `~object.__dict__`, the default state is a tuple whose first item is `None` and whose second item is a dictionary mapping slot names to slot values described in the previous bullet.

<div class="versionchanged">

3.11 Added the default implementation of the `__getstate__()` method in the `object` class.

</div>

</div>

<div class="method">

object.\_\_setstate\_\_(state)

Upon unpickling, if the class defines `__setstate__`, it is called with the unpickled state. In that case, there is no requirement for the state object to be a dictionary. Otherwise, the pickled state must be a dictionary and its items are assigned to the new instance's dictionary.

> [!NOTE]
> If `__reduce__` returns a state with value `None` at pickling, the `__setstate__` method will not be called upon unpickling.

</div>

Refer to the section `pickle-state` for more information about how to use the methods `~object.__getstate__` and `~object.__setstate__`.

> [!NOTE]
> At unpickling time, some methods like `~object.__getattr__`, `~object.__getattribute__`, or `~object.__setattr__` may be called upon the instance. In case those methods rely on some internal invariant being true, the type should implement `~object.__new__` to establish such an invariant, as `~object.__init__` is not called when unpickling an instance.

<div class="index">

pair: copy; protocol

</div>

As we shall see, pickle does not use directly the methods described above. In fact, these methods are part of the copy protocol which implements the `~object.__reduce__` special method. The copy protocol provides a unified interface for retrieving the data necessary for pickling and copying objects.[^4]

Although powerful, implementing `~object.__reduce__` directly in your classes is error prone. For this reason, class designers should use the high-level interface (i.e., `~object.__getnewargs_ex__`, `~object.__getstate__` and `~object.__setstate__`) whenever possible. We will show, however, cases where using `!__reduce__` is the only option or leads to more efficient pickling or both.

<div class="method">

object.\_\_reduce\_\_()

The interface is currently defined as follows. The `__reduce__` method takes no argument and shall return either a string or preferably a tuple (the returned object is often referred to as the "reduce value").

If a string is returned, the string should be interpreted as the name of a global variable. It should be the object's local name relative to its module; the pickle module searches the module namespace to determine the object's module: for a given `obj` to be pickled, the `__module__` attribute is looked up on `obj` directly, which falls back to a lookup on the type of `obj` if no `__module__` instance attribute is set. This behaviour is typically useful for singletons.

When a tuple is returned, it must be between two and six items long. Optional items can either be omitted, or `None` can be provided as their value. The semantics of each item are in order:

- A callable object that will be called to create the initial version of the object.

- A tuple of arguments for the callable object. An empty tuple must be given if the callable does not accept any argument.

- Optionally, the object's state, which will be passed to the object's `__setstate__` method as previously described. If the object has no such method then, the value must be a dictionary and it will be added to the object's `~object.__dict__` attribute.

- Optionally, an iterator (and not a sequence) yielding successive items. These items will be appended to the object either using `obj.append(item)` or, in batch, using `obj.extend(list_of_items)`. This is primarily used for list subclasses, but may be used by other classes as long as they have `~sequence.append` and `~sequence.extend` methods with the appropriate signature. (Whether `!append` or `!extend` is used depends on which pickle protocol version is used as well as the number of items to append, so both must be supported.)

- Optionally, an iterator (not a sequence) yielding successive key-value pairs. These items will be stored to the object using `obj[key] = value`. This is primarily used for dictionary subclasses, but may be used by other classes as long as they implement `__setitem__`.

- Optionally, a callable with a `(obj, state)` signature. This callable allows the user to programmatically control the state-updating behavior of a specific object, instead of using `obj`'s static `__setstate__` method. If not `None`, this callable will have priority over `obj`'s `__setstate__`.

  <div class="versionadded">

  3.8 The optional sixth tuple item, `(obj, state)`, was added.

  </div>

</div>

<div class="method">

object.\_\_reduce_ex\_\_(protocol)

Alternatively, a `__reduce_ex__` method may be defined. The only difference is this method should take a single integer argument, the protocol version. When defined, pickle will prefer it over the `__reduce__` method. In addition, `__reduce__` automatically becomes a synonym for the extended version. The main use for this method is to provide backwards-compatible reduce values for older Python releases.

</div>

<div class="currentmodule">

pickle

</div>

### Persistence of External Objects

<div class="index">

single: persistent_id (pickle protocol) single: persistent_load (pickle protocol)

</div>

For the benefit of object persistence, the `!pickle` module supports the notion of a reference to an object outside the pickled data stream. Such objects are referenced by a persistent ID, which should be either a string of alphanumeric characters (for protocol 0)[^5] or just an arbitrary object (for any newer protocol).

The resolution of such persistent IDs is not defined by the `!pickle` module; it will delegate this resolution to the user-defined methods on the pickler and unpickler, `~Pickler.persistent_id` and `~Unpickler.persistent_load` respectively.

To pickle objects that have an external persistent ID, the pickler must have a custom `~Pickler.persistent_id` method that takes an object as an argument and returns either `None` or the persistent ID for that object. When `None` is returned, the pickler simply pickles the object as normal. When a persistent ID string is returned, the pickler will pickle that object, along with a marker so that the unpickler will recognize it as a persistent ID.

To unpickle external objects, the unpickler must have a custom `~Unpickler.persistent_load` method that takes a persistent ID object and returns the referenced object.

Here is a comprehensive example presenting how persistent ID can be used to pickle external objects by reference.

<div class="literalinclude">

../includes/dbpickle.py

</div>

### Dispatch Tables

If one wants to customize pickling of some classes without disturbing any other code which depends on pickling, then one can create a pickler with a private dispatch table.

The global dispatch table managed by the `copyreg` module is available as `!copyreg.dispatch_table`. Therefore, one may choose to use a modified copy of `!copyreg.dispatch_table` as a private dispatch table.

For example :

    f = io.BytesIO()
    p = pickle.Pickler(f)
    p.dispatch_table = copyreg.dispatch_table.copy()
    p.dispatch_table[SomeClass] = reduce_SomeClass

creates an instance of `pickle.Pickler` with a private dispatch table which handles the `SomeClass` class specially. Alternatively, the code :

    class MyPickler(pickle.Pickler):
        dispatch_table = copyreg.dispatch_table.copy()
        dispatch_table[SomeClass] = reduce_SomeClass
    f = io.BytesIO()
    p = MyPickler(f)

does the same but all instances of `MyPickler` will by default share the private dispatch table. On the other hand, the code :

    copyreg.pickle(SomeClass, reduce_SomeClass)
    f = io.BytesIO()
    p = pickle.Pickler(f)

modifies the global dispatch table shared by all users of the `copyreg` module.

### Handling Stateful Objects

<div class="index">

single: \_\_getstate\_\_() (copy protocol) single: \_\_setstate\_\_() (copy protocol)

</div>

Here's an example that shows how to modify pickling behavior for a class. The `!TextReader` class below opens a text file, and returns the line number and line contents each time its `!readline` method is called. If a `!TextReader` instance is pickled, all attributes *except* the file object member are saved. When the instance is unpickled, the file is reopened, and reading resumes from the last location. The `!__setstate__` and `!__getstate__` methods are used to implement this behavior. :

    class TextReader:
        """Print and number lines in a text file."""

        def __init__(self, filename):
            self.filename = filename
            self.file = open(filename)
            self.lineno = 0

        def readline(self):
            self.lineno += 1
            line = self.file.readline()
            if not line:
                return None
            if line.endswith('\n'):
                line = line[:-1]
            return "%i: %s" % (self.lineno, line)

        def __getstate__(self):
            # Copy the object's state from self.__dict__ which contains
            # all our instance attributes. Always use the dict.copy()
            # method to avoid modifying the original state.
            state = self.__dict__.copy()
            # Remove the unpicklable entries.
            del state['file']
            return state

        def __setstate__(self, state):
            # Restore instance attributes (i.e., filename and lineno).
            self.__dict__.update(state)
            # Restore the previously opened file's state. To do so, we need to
            # reopen it and read from it until the line count is restored.
            file = open(self.filename)
            for _ in range(self.lineno):
                file.readline()
            # Finally, save the file.
            self.file = file

A sample usage might be something like this:

    >>> reader = TextReader("hello.txt")
    >>> reader.readline()
    '1: Hello world!'
    >>> reader.readline()
    '2: I am line number two.'
    >>> new_reader = pickle.loads(pickle.dumps(reader))
    >>> new_reader.readline()
    '3: Goodbye!'

## Custom Reduction for Types, Functions, and Other Objects

<div class="versionadded">

3.8

</div>

Sometimes, `~Pickler.dispatch_table` may not be flexible enough. In particular we may want to customize pickling based on another criterion than the object's type, or we may want to customize the pickling of functions and classes.

For those cases, it is possible to subclass from the `Pickler` class and implement a `~Pickler.reducer_override` method. This method can return an arbitrary reduction tuple (see `~object.__reduce__`). It can alternatively return `NotImplemented` to fallback to the traditional behavior.

If both the `~Pickler.dispatch_table` and `~Pickler.reducer_override` are defined, then `~Pickler.reducer_override` method takes priority.

> [!NOTE]
> For performance reasons, `~Pickler.reducer_override` may not be called for the following objects: `None`, `True`, `False`, and exact instances of `int`, `float`, `bytes`, `str`, `dict`, `set`, `frozenset`, `list` and `tuple`.

Here is a simple example where we allow pickling and reconstructing a given class:

    import io
    import pickle

    class MyClass:
        my_attribute = 1

    class MyPickler(pickle.Pickler):
        def reducer_override(self, obj):
            """Custom reducer for MyClass."""
            if getattr(obj, "__name__", None) == "MyClass":
                return type, (obj.__name__, obj.__bases__,
                              {'my_attribute': obj.my_attribute})
            else:
                # For any other object, fallback to usual reduction
                return NotImplemented

    f = io.BytesIO()
    p = MyPickler(f)
    p.dump(MyClass)

    del MyClass

    unpickled_class = pickle.loads(f.getvalue())

    assert isinstance(unpickled_class, type)
    assert unpickled_class.__name__ == "MyClass"
    assert unpickled_class.my_attribute == 1

## Out-of-band Buffers

<div class="versionadded">

3.8

</div>

In some contexts, the `!pickle` module is used to transfer massive amounts of data. Therefore, it can be important to minimize the number of memory copies, to preserve performance and resource consumption. However, normal operation of the `!pickle` module, as it transforms a graph-like structure of objects into a sequential stream of bytes, intrinsically involves copying data to and from the pickle stream.

This constraint can be eschewed if both the *provider* (the implementation of the object types to be transferred) and the *consumer* (the implementation of the communications system) support the out-of-band transfer facilities provided by pickle protocol 5 and higher.

### Provider API

The large data objects to be pickled must implement a `~object.__reduce_ex__` method specialized for protocol 5 and higher, which returns a `PickleBuffer` instance (instead of e.g. a `bytes` object) for any large data.

A `PickleBuffer` object *signals* that the underlying buffer is eligible for out-of-band data transfer. Those objects remain compatible with normal usage of the `!pickle` module. However, consumers can also opt-in to tell `!pickle` that they will handle those buffers by themselves.

### Consumer API

A communications system can enable custom handling of the `PickleBuffer` objects generated when serializing an object graph.

On the sending side, it needs to pass a *buffer_callback* argument to `Pickler` (or to the `dump` or `dumps` function), which will be called with each `PickleBuffer` generated while pickling the object graph. Buffers accumulated by the *buffer_callback* will not see their data copied into the pickle stream, only a cheap marker will be inserted.

On the receiving side, it needs to pass a *buffers* argument to `Unpickler` (or to the `load` or `loads` function), which is an iterable of the buffers which were passed to *buffer_callback*. That iterable should produce buffers in the same order as they were passed to *buffer_callback*. Those buffers will provide the data expected by the reconstructors of the objects whose pickling produced the original `PickleBuffer` objects.

Between the sending side and the receiving side, the communications system is free to implement its own transfer mechanism for out-of-band buffers. Potential optimizations include the use of shared memory or datatype-dependent compression.

### Example

Here is a trivial example where we implement a `bytearray` subclass able to participate in out-of-band buffer pickling:

    class ZeroCopyByteArray(bytearray):

        def __reduce_ex__(self, protocol):
            if protocol >= 5:
                return type(self)._reconstruct, (PickleBuffer(self),), None
            else:
                # PickleBuffer is forbidden with pickle protocols <= 4.
                return type(self)._reconstruct, (bytearray(self),)

        @classmethod
        def _reconstruct(cls, obj):
            with memoryview(obj) as m:
                # Get a handle over the original buffer object
                obj = m.obj
                if type(obj) is cls:
                    # Original buffer object is a ZeroCopyByteArray, return it
                    # as-is.
                    return obj
                else:
                    return cls(obj)

The reconstructor (the `_reconstruct` class method) returns the buffer's providing object if it has the right type. This is an easy way to simulate zero-copy behaviour on this toy example.

On the consumer side, we can pickle those objects the usual way, which when unserialized will give us a copy of the original object:

    b = ZeroCopyByteArray(b"abc")
    data = pickle.dumps(b, protocol=5)
    new_b = pickle.loads(data)
    print(b == new_b)  # True
    print(b is new_b)  # False: a copy was made

But if we pass a *buffer_callback* and then give back the accumulated buffers when unserializing, we are able to get back the original object:

    b = ZeroCopyByteArray(b"abc")
    buffers = []
    data = pickle.dumps(b, protocol=5, buffer_callback=buffers.append)
    new_b = pickle.loads(data, buffers=buffers)
    print(b == new_b)  # True
    print(b is new_b)  # True: no copy was made

This example is limited by the fact that `bytearray` allocates its own memory: you cannot create a `bytearray` instance that is backed by another object's memory. However, third-party datatypes such as NumPy arrays do not have this limitation, and allow use of zero-copy pickling (or making as few copies as possible) when transferring between distinct processes or systems.

<div class="seealso">

`574` -- Pickle protocol 5 with out-of-band data

</div>

## Restricting Globals

<div class="index">

single: find_class() (pickle protocol)

</div>

By default, unpickling will import any class or function that it finds in the pickle data. For many applications, this behaviour is unacceptable as it permits the unpickler to import and invoke arbitrary code. Just consider what this hand-crafted pickle data stream does when loaded:

    >>> import pickle
    >>> pickle.loads(b"cos\nsystem\n(S'echo hello world'\ntR.")
    hello world
    0

In this example, the unpickler imports the `os.system` function and then apply the string argument "echo hello world". Although this example is inoffensive, it is not difficult to imagine one that could damage your system.

For this reason, you may want to control what gets unpickled by customizing `Unpickler.find_class`. Unlike its name suggests, `Unpickler.find_class` is called whenever a global (i.e., a class or a function) is requested. Thus it is possible to either completely forbid globals or restrict them to a safe subset.

Here is an example of an unpickler allowing only few safe classes from the `builtins` module to be loaded:

    import builtins
    import io
    import pickle

    safe_builtins = {
        'range',
        'complex',
        'set',
        'frozenset',
        'slice',
    }

    class RestrictedUnpickler(pickle.Unpickler):

        def find_class(self, module, name):
            # Only allow safe classes from builtins.
            if module == "builtins" and name in safe_builtins:
                return getattr(builtins, name)
            # Forbid everything else.
            raise pickle.UnpicklingError("global '%s.%s' is forbidden" %
                                         (module, name))

    def restricted_loads(s):
        """Helper function analogous to pickle.loads()."""
        return RestrictedUnpickler(io.BytesIO(s)).load()

A sample usage of our unpickler working as intended:

    >>> restricted_loads(pickle.dumps([1, 2, range(15)]))
    [1, 2, range(0, 15)]
    >>> restricted_loads(b"cos\nsystem\n(S'echo hello world'\ntR.")
    Traceback (most recent call last):
      ...
    pickle.UnpicklingError: global 'os.system' is forbidden
    >>> restricted_loads(b'cbuiltins\neval\n'
    ...                  b'(S\'getattr(__import__("os"), "system")'
    ...                  b'("echo hello world")\'\ntR.')
    Traceback (most recent call last):
      ...
    pickle.UnpicklingError: global 'builtins.eval' is forbidden

As our examples shows, you have to be careful with what you allow to be unpickled. Therefore if security is a concern, you may want to consider alternatives such as the marshalling API in `xmlrpc.client` or third-party solutions.

## Performance

Recent versions of the pickle protocol (from protocol 2 and upwards) feature efficient binary encodings for several common features and built-in types. Also, the `!pickle` module has a transparent optimizer written in C.

## Examples

For the simplest code, use the `dump` and `load` functions. :

    import pickle

    # An arbitrary collection of objects supported by pickle.
    data = {
        'a': [1, 2.0, 3+4j],
        'b': ("character string", b"byte string"),
        'c': {None, True, False}
    }

    with open('data.pickle', 'wb') as f:
        # Pickle the 'data' dictionary using the highest protocol available.
        pickle.dump(data, f, pickle.HIGHEST_PROTOCOL)

The following example reads the resulting pickled data. :

    import pickle

    with open('data.pickle', 'rb') as f:
        # The protocol version used is detected automatically, so we do not
        # have to specify it.
        data = pickle.load(f)

## Command-line interface

The `!pickle` module can be invoked as a script from the command line, it will display contents of the pickle files. However, when the pickle file that you want to examine comes from an untrusted source, `-m pickletools` is a safer option because it does not execute pickle bytecode, see `pickletools CLI usage <pickletools-cli>`.

``` bash
python -m pickle pickle_file [pickle_file ...]
```

The following option is accepted:

<div class="program">

pickle

</div>

<div class="option">

pickle_file

A pickle file to read, or `-` to indicate reading from standard input.

</div>

<div class="seealso">

Module `copyreg`  
Pickle interface constructor registration for extension types.

Module `pickletools`  
Tools for working with and analyzing pickled data.

Module `shelve`  
Indexed databases of objects; uses `!pickle`.

Module `copy`  
Shallow and deep object copying.

Module `marshal`  
High-performance serialization of built-in types.

</div>

**Footnotes**

[^1]: Don't confuse this with the `marshal` module

[^2]: This is why `lambda` functions cannot be pickled: all `!lambda` functions share the same name: `<lambda>`.

[^3]: The exception raised will likely be an `ImportError` or an `AttributeError` but it could be something else.

[^4]: The `copy` module uses this protocol for shallow and deep copying operations.

[^5]: The limitation on alphanumeric characters is due to the fact that persistent IDs in protocol 0 are delimited by the newline character. Therefore if any kind of newline characters occurs in persistent IDs, the resulting pickled data will become unreadable.

# `!pickletools` --- Tools for pickle developers

<div class="module" data-synopsis="Contains extensive comments about the pickle protocols and
pickle-machine opcodes, as well as some useful functions.">

pickletools

</div>

**Source code:** `Lib/pickletools.py`

------------------------------------------------------------------------

This module contains various constants relating to the intimate details of the `pickle` module, some lengthy comments about the implementation, and a few useful functions for analyzing pickled data. The contents of this module are useful for Python core developers who are working on the `pickle`; ordinary users of the `pickle` module probably won't find the `!pickletools` module relevant.

## Command-line usage

<div class="versionadded">

3.2

</div>

When invoked from the command line, `python -m pickletools` will disassemble the contents of one or more pickle files. Note that if you want to see the Python object stored in the pickle rather than the details of pickle format, you may want to use `-m pickle` instead. However, when the pickle file that you want to examine comes from an untrusted source, `-m pickletools` is a safer option because it does not execute pickle bytecode.

For example, with a tuple `(1, 2)` pickled in file `x.pickle`:

```
$ python -m pickle x.pickle
(1, 2)

$ python -m pickletools x.pickle
    0: \x80 PROTO      3
    2: K    BININT1    1
    4: K    BININT1    2
    6: \x86 TUPLE2
    7: q    BINPUT     0
    9: .    STOP
highest protocol among opcodes = 2
```

### Command-line options

<div class="program">

pickletools

</div>

<div class="option">

`-a, --annotate`  

Annotate each line with a short opcode description.

</div>

<div class="option">

`-o, --output=<file>`  

Name of a file where the output should be written.

</div>

<div class="option">

`-l, --indentlevel=<num>`  

The number of blanks by which to indent a new MARK level.

</div>

<div class="option">

`-m, --memo`  

When multiple objects are disassembled, preserve memo between disassemblies.

</div>

<div class="option">

`-p, --preamble=<preamble>`  

When more than one pickle file is specified, print given preamble before each disassembly.

</div>

<div class="option">

pickle_file

A pickle file to read, or `-` to indicate reading from standard input.

</div>

<div class="versionadded">

3.15 Output is in color by default and can be `controlled using environment variables <using-on-controlling-color>`.

</div>

## Programmatic interface

<div class="function">

dis(pickle, out=None, memo=None, indentlevel=4, annotate=0)

Outputs a symbolic disassembly of the pickle to the file-like object *out*, defaulting to `sys.stdout`. *pickle* can be a string or a file-like object. *memo* can be a Python dictionary that will be used as the pickle's memo; it can be used to perform disassemblies across multiple pickles created by the same pickler. Successive levels, indicated by `MARK` opcodes in the stream, are indented by *indentlevel* spaces. If a nonzero value is given to *annotate*, each opcode in the output is annotated with a short description. The value of *annotate* is used as a hint for the column where annotation should start.

<div class="versionchanged">

3.2 Added the *annotate* parameter.

</div>

</div>

<div class="function">

genops(pickle)

Provides an `iterator` over all of the opcodes in a pickle, returning a sequence of `(opcode, arg, pos)` triples. *opcode* is an instance of an `OpcodeInfo` class; *arg* is the decoded value, as a Python object, of the opcode's argument; *pos* is the position at which this opcode is located. *pickle* can be a string or a file-like object.

</div>

<div class="function">

optimize(picklestring)

Returns a new equivalent pickle string after eliminating unused `PUT` opcodes. The optimized pickle is shorter, takes less transmission time, requires less storage space, and unpickles more efficiently.

</div>

# `!pipes` --- Interface to shell pipelines

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

pipes

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

Applications should use the `subprocess` module instead.

The last version of Python that provided the `!pipes` module was [Python 3.12](https://docs.python.org/3.12/library/pipes.html).

# `!pkgutil` --- Package extension utility

<div class="module" data-synopsis="Utilities for the import system.">

pkgutil

</div>

**Source code:** `Lib/pkgutil.py`

------------------------------------------------------------------------

This module provides utilities for the import system, in particular package support.

<div class="ModuleInfo(module_finder, name, ispkg)">

A namedtuple that holds a brief summary of a module's info.

<div class="versionadded">

3.6

</div>

</div>

<div class="function">

extend_path(path, name)

Extend the search path for the modules which comprise a package. Intended use is to place the following code in a package's `__init__.py`:

    from pkgutil import extend_path
    __path__ = extend_path(__path__, __name__)

For each directory on `sys.path` that has a subdirectory that matches the package name, add the subdirectory to the package's `~module.__path__`. This is useful if one wants to distribute different parts of a single logical package as multiple directories.

It also looks for `\*.pkg` files beginning where `*` matches the *name* argument. This feature is similar to `\*.pth` files (see the `site` module for more information), except that it doesn't special-case lines starting with `import`. A `\*.pkg` file is trusted at face value: apart from skipping blank lines and ignoring comments, all entries found in a `\*.pkg` file are added to the path, regardless of whether they exist on the filesystem (this is a feature).

If the input path is not a list (as is the case for frozen packages) it is returned unchanged. The input path is not modified; an extended copy is returned. Items are only appended to the copy at the end.

It is assumed that `sys.path` is a sequence. Items of `sys.path` that are not strings referring to existing directories are ignored. Unicode items on `sys.path` that cause errors when used as filenames may cause this function to raise an exception (in line with `os.path.isdir` behavior).

</div>

<div class="function">

get_importer(path_item)

Retrieve a `finder` for the given *path_item*.

The returned finder is cached in `sys.path_importer_cache` if it was newly created by a path hook.

The cache (or part of it) can be cleared manually if a rescan of `sys.path_hooks` is necessary.

<div class="versionchanged">

3.3 Updated to be based directly on `importlib` rather than relying on the package internal `302` import emulation.

</div>

</div>

<div class="function">

iter_importers(fullname='')

Yield `finder` objects for the given module name.

If *fullname* contains a `'.'`, the finders will be for the package containing *fullname*, otherwise they will be all registered top level finders (i.e. those on both `sys.meta_path` and `sys.path_hooks`).

If the named module is in a package, that package is imported as a side effect of invoking this function.

If no module name is specified, all top level finders are produced.

<div class="versionchanged">

3.3 Updated to be based directly on `importlib` rather than relying on the package internal `302` import emulation.

</div>

</div>

<div class="function">

iter_modules(path=None, prefix='')

Yields `ModuleInfo` for all submodules on *path*, or, if *path* is `None`, all top-level modules on `sys.path`.

*path* should be either `None` or a list of paths to look for modules in.

*prefix* is a string to output on the front of every module name on output.

> [!NOTE]
> Only works for a `finder` which defines an `iter_modules()` method. This interface is non-standard, so the module also provides implementations for `importlib.machinery.FileFinder` and `zipimport.zipimporter`.

<div class="versionchanged">

3.3 Updated to be based directly on `importlib` rather than relying on the package internal `302` import emulation.

</div>

</div>

<div class="function">

walk_packages(path=None, prefix='', onerror=None)

Yields `ModuleInfo` for all modules recursively on *path*, or, if *path* is `None`, all accessible modules.

*path* should be either `None` or a list of paths to look for modules in.

*prefix* is a string to output on the front of every module name on output.

Note that this function must import all *packages* (*not* all modules!) on the given *path*, in order to access the `__path__` attribute to find submodules.

*onerror* is a function which gets called with one argument (the name of the package which was being imported) if any exception occurs while trying to import a package. If no *onerror* function is supplied, `ImportError`s are caught and ignored, while all other exceptions are propagated, terminating the search.

Examples:

    # list all modules python can access
    walk_packages()

    # list all submodules of ctypes
    walk_packages(ctypes.__path__, ctypes.__name__ + '.')

> [!NOTE]
> Only works for a `finder` which defines an `iter_modules()` method. This interface is non-standard, so the module also provides implementations for `importlib.machinery.FileFinder` and `zipimport.zipimporter`.

<div class="versionchanged">

3.3 Updated to be based directly on `importlib` rather than relying on the package internal `302` import emulation.

</div>

</div>

<div class="function">

get_data(package, resource)

Get a resource from a package.

This is a wrapper for the `loader` `get_data <importlib.abc.ResourceLoader.get_data>` API. The *package* argument should be the name of a package, in standard module format (`foo.bar`). The *resource* argument should be in the form of a relative filename, using `/` as the path separator.

The function returns a binary string that is the contents of the specified resource.

This function uses the `loader` method `~importlib.abc.FileLoader.get_data` to support modules installed in the filesystem, but also in zip files, databases, or elsewhere.

For packages located in the filesystem, which have already been imported, this is the rough equivalent of:

    d = os.path.dirname(sys.modules[package].__file__)
    data = open(os.path.join(d, resource), 'rb').read()

Like the `open` function, `!get_data` can follow parent directories (`../`) and absolute paths (starting with `/` or `C:/`, for example). It can open compilation/installation artifacts like `.py` and `.pyc` files or files with `reserved filenames <os.path.isreserved>`. To be compatible with non-filesystem loaders, avoid using these features.

> [!WARNING]
> This function is intended for trusted input. It does not verify that *resource* "belongs" to *package*.

If you use a user-provided *resource* path, consider verifying it. For example, require an alphanumeric filename with a known extension, or install and check a list of known resources.

If the package cannot be located or loaded, or it uses a `loader` which does not support `get_data <importlib.abc.ResourceLoader.get_data>`, then `None` is returned. In particular, the `loader` for `namespace packages <namespace package>` does not support `get_data <importlib.abc.ResourceLoader.get_data>`.

<div class="seealso">

The `importlib.resources` module provides structured access to module resources.

</div>

</div>

<div class="function">

resolve_name(name, \*, strict=False)

Resolve a name to an object.

This functionality is used in numerous places in the standard library (see `12915`) - and equivalent functionality is also in widely used third-party packages such as setuptools, Django and Pyramid.

It is expected that *name* will be a string in one of the following formats, where W is shorthand for a valid Python identifier and dot stands for a literal period in these pseudo-regexes:

- `W(.W)*`
- `W(.W)*:(W(.W)*)?`
- `W(.W)*:(W(.W)*)`

The first form is intended for backward compatibility only. It assumes that some part of the dotted name is a package, and the rest is an object somewhere within that package, possibly nested inside other objects. Because the place where the package stops and the object hierarchy starts can't be inferred by inspection, repeated attempts to import must be done with this form.

In the second form, the caller makes the division point clear through the provision of a single colon: the dotted name to the left of the colon is a package to be imported, and the dotted name to the right is the object hierarchy within that package. Only one import is needed in this form. If it ends with the colon, then a module object is returned.

The first two forms are accepted when `strict=False` (the default).

The third form requires both the module name and callable, separated by a colon. Only this form is accepted when `strict=True`.

The function will return an object (which might be a module), or raise one of the following exceptions:

`ValueError` -- if *name* isn't in a recognised format.

`ImportError` -- if an import failed when it shouldn't have.

`AttributeError` -- If a failure occurred when traversing the object hierarchy within the imported package to get to the desired object.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.15

The optional keyword-only `strict` flag was added.

</div>

</div>

# `!platform` --- Access to underlying platform's identifying data

<div class="module" data-synopsis="Retrieves as much platform identifying data as possible.">

platform

</div>

**Source code:** `Lib/platform.py`

------------------------------------------------------------------------

> [!NOTE]
> Specific platforms listed alphabetically, with Linux included in the Unix section.

## Cross platform

<div class="function">

architecture(executable=sys.executable, bits='', linkage='')

Queries the given executable (defaults to the Python interpreter binary) for various architecture information.

Returns a tuple `(bits, linkage)` which contain information about the bit architecture and the linkage format used for the executable. Both values are returned as strings.

Values that cannot be determined are returned as given by the parameter presets. If bits is given as `''`, the `sizeof(pointer)` (or `sizeof(long)` on Python version \< 1.5.2) is used as indicator for the supported pointer size.

The function relies on the system's `file` command to do the actual work. This is available on most if not all Unix platforms and some non-Unix platforms and then only if the executable points to the Python interpreter. Reasonable defaults are used when the above needs are not met.

> [!NOTE]
> On macOS (and perhaps other platforms), executable files may be universal files containing multiple architectures.
>
> To get at the "64-bitness" of the current interpreter, it is more reliable to query the `sys.maxsize` attribute:
>
>     is_64bits = sys.maxsize > 2**32

</div>

<div class="function">

machine()

Returns the machine type, e.g. `'AMD64'`. An empty string is returned if the value cannot be determined.

The output is platform-dependent and may differ in casing and naming conventions.

</div>

<div class="function">

node()

Returns the computer's network name (may not be fully qualified!). An empty string is returned if the value cannot be determined.

</div>

<div class="function">

platform(aliased=False, terse=False)

Returns a single string identifying the underlying platform with as much useful information as possible.

The output is intended to be *human readable* rather than machine parseable. It may look different on different platforms and this is intended.

If *aliased* is true, the function will use aliases for various platforms that report system names which differ from their common names, for example SunOS will be reported as Solaris. The `system_alias` function is used to implement this.

Setting *terse* to true causes the function to return only the absolute minimum information needed to identify the platform.

<div class="versionchanged">

3.8 On macOS, the function now uses `mac_ver`, if it returns a non-empty release string, to get the macOS version rather than the darwin version.

</div>

</div>

<div class="function">

processor()

Returns the (real) processor name, e.g. `'amdk6'`.

An empty string is returned if the value cannot be determined. Note that many platforms do not provide this information or simply return the same value as for `machine`. NetBSD does this.

</div>

<div class="function">

python_build()

Returns a tuple `(buildno, builddate)` stating the Python build number and date as strings.

</div>

<div class="function">

python_compiler()

Returns a string identifying the compiler used for compiling Python.

</div>

<div class="function">

python_branch()

Returns a string identifying the Python implementation SCM branch.

</div>

<div class="function">

python_implementation()

Returns a string identifying the Python implementation. Possible return values are: 'CPython', 'IronPython', 'Jython', 'PyPy'.

</div>

<div class="function">

python_revision()

Returns a string identifying the Python implementation SCM revision.

</div>

<div class="function">

python_version()

Returns the Python version as string `'major.minor.patchlevel'`.

Note that unlike the Python `sys.version`, the returned value will always include the patchlevel (it defaults to 0).

</div>

<div class="function">

python_version_tuple()

Returns the Python version as tuple `(major, minor, patchlevel)` of strings.

Note that unlike the Python `sys.version`, the returned value will always include the patchlevel (it defaults to `'0'`).

</div>

<div class="function">

release()

Returns the system's release, e.g. `'2.2.0'` or `'NT'`. An empty string is returned if the value cannot be determined.

On iOS and Android, this is the user-facing OS release. To obtain the Darwin or Linux kernel release, use `os.uname`.

</div>

<div class="function">

system()

Returns the system/OS name, such as `'Linux'`, `'Darwin'`, `'Java'`, `'Windows'`. An empty string is returned if the value cannot be determined.

On iOS and Android, this returns the user-facing OS name (i.e, `'iOS`, `'iPadOS'` or `'Android'`). To obtain the kernel name (`'Darwin'` or `'Linux'`), use `os.uname`.

</div>

<div class="function">

system_alias(system, release, version)

Returns `(system, release, version)` aliased to common marketing names used for some systems. It also does some reordering of the information in some cases where it would otherwise cause confusion.

</div>

<div class="function">

version()

Returns the system's release version, e.g. `'#3 on degas'`. An empty string is returned if the value cannot be determined.

</div>

<div class="function">

uname()

Fairly portable uname interface. Returns a `~collections.namedtuple` containing six attributes: `system`, `node`, `release`, `version`, `machine`, and `processor`.

`processor` is resolved late, on demand.

Note: the first two attribute names differ from the names presented by `os.uname`, where they are named `!sysname` and `!nodename`.

Entries which cannot be determined are set to `''`.

<div class="versionchanged">

3.3 Result changed from a tuple to a `~collections.namedtuple`.

</div>

<div class="versionchanged">

3.9 `processor` is resolved late instead of immediately.

</div>

</div>

<div class="function">

invalidate_caches()

Clear out the internal cache of information, such as the `uname`. This is typically useful when the platform's `node` is changed by an external process and one needs to retrieve the updated value.

<div class="versionadded">

3.14

</div>

</div>

## Windows platform

<div class="function">

win32_ver(release='', version='', csd='', ptype='')

Get additional version information from the Windows Registry and return a tuple `(release, version, csd, ptype)` referring to OS release, version number, CSD level (service pack) and OS type (multi/single processor). Values which cannot be determined are set to the defaults given as parameters (which all default to an empty string).

As a hint: *ptype* is `'Uniprocessor Free'` on single processor NT machines and `'Multiprocessor Free'` on multi processor machines. The `'Free'` refers to the OS version being free of debugging code. It could also state `'Checked'` which means the OS version uses debugging code, i.e. code that checks arguments, ranges, etc.

</div>

<div class="function">

win32_edition()

Returns a string representing the current Windows edition, or `None` if the value cannot be determined. Possible values include but are not limited to `'Enterprise'`, `'IoTUAP'`, `'ServerStandard'`, and `'nanoserver'`.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

win32_is_iot()

Return `True` if the Windows edition returned by `win32_edition` is recognized as an IoT edition.

<div class="versionadded">

3.8

</div>

</div>

## macOS platform

<div class="function">

mac_ver(release='', versioninfo=('','',''), machine='')

Get macOS version information and return it as tuple `(release, versioninfo, machine)` with *versioninfo* being a tuple `(version, dev_stage, non_release_version)`.

Entries which cannot be determined are set to `''`. All tuple entries are strings.

</div>

## iOS platform

<div class="function">

ios_ver(system='', release='', model='', is_simulator=False)

Get iOS version information and return it as a `~collections.namedtuple` with the following attributes:

- `system` is the OS name; either `'iOS'` or `'iPadOS'`.
- `release` is the iOS version number as a string (e.g., `'17.2'`).
- `model` is the device model identifier; this will be a string like `'iPhone13,2'` for a physical device, or `'iPhone'` on a simulator.
- `is_simulator` is a boolean describing if the app is running on a simulator or a physical device.

Entries which cannot be determined are set to the defaults given as parameters.

</div>

## Unix platforms

<div class="function">

libc_ver(executable=sys.executable, lib='', version='', chunksize=16384)

Tries to determine the libc version against which the file executable (defaults to the Python interpreter) is linked. Returns a tuple of strings `(lib, version)` which default to the given parameters in case the lookup fails.

Note that this function has intimate knowledge of how different libc versions add symbols to the executable is probably only usable for executables compiled using `gcc`.

The file is read and scanned in chunks of *chunksize* bytes.

</div>

## Linux platforms

<div class="function">

freedesktop_os_release()

Get operating system identification from `os-release` file and return it as a dict. The `os-release` file is a [freedesktop.org standard](https://www.freedesktop.org/software/systemd/man/os-release.html) and is available in most Linux distributions. A noticeable exception is Android and Android-based distributions.

Raises `OSError` or subclass when neither `/etc/os-release` nor `/usr/lib/os-release` can be read.

On success, the function returns a dictionary where keys and values are strings. Values have their special characters like `"` and `$` unquoted. The fields `NAME`, `ID`, and `PRETTY_NAME` are always defined according to the standard. All other fields are optional. Vendors may include additional fields.

Note that fields like `NAME`, `VERSION`, and `VARIANT` are strings suitable for presentation to users. Programs should use fields like `ID`, `ID_LIKE`, `VERSION_ID`, or `VARIANT_ID` to identify Linux distributions.

Example:

    def get_like_distro():
        info = platform.freedesktop_os_release()
        ids = [info["ID"]]
        if "ID_LIKE" in info:
            # ids are space separated and ordered by precedence
            ids.extend(info["ID_LIKE"].split())
        return ids

<div class="versionadded">

3.10

</div>

</div>

## Android platform

<div class="function">

android_ver(release="", api_level=0, manufacturer="", model="", device="", is_emulator=False)

Get Android device information. Returns a `~collections.namedtuple` with the following attributes. Values which cannot be determined are set to the defaults given as parameters.

- `release` - Android version, as a string (e.g. `"14"`).
- `api_level` - API level of the running device, as an integer (e.g. `34` for Android 14). To get the API level which Python was built against, see `sys.getandroidapilevel`.
- `manufacturer` - [Manufacturer name](https://developer.android.com/reference/android/os/Build#MANUFACTURER).
- `model` - [Model name](https://developer.android.com/reference/android/os/Build#MODEL) – typically the marketing name or model number.
- `device` - [Device name](https://developer.android.com/reference/android/os/Build#DEVICE) – typically the model number or a codename.
- `is_emulator` - `True` if the device is an emulator; `False` if it's a physical device.

Google maintains a [list of known model and device names](https://storage.googleapis.com/play_public/supported_devices.html).

<div class="versionadded">

3.13

</div>

</div>

## Command-line usage

`!platform` can also be invoked directly using the `-m` switch of the interpreter:

    python -m platform [--terse] [--nonaliased] [{nonaliased,terse} ...]

The following options are accepted:

<div class="program">

platform

</div>

<div class="option">

`--terse`  

Print terse information about the platform. This is equivalent to calling `platform.platform` with the *terse* argument set to `True`.

</div>

<div class="option">

`--nonaliased`  

Print platform information without system/OS name aliasing. This is equivalent to calling `platform.platform` with the *aliased* argument set to `True`.

</div>

You can also pass one or more positional arguments (`terse`, `nonaliased`) to explicitly control the output format. These behave similarly to their corresponding options.

# `!plistlib` --- Generate and parse Apple `.plist` files

<div class="module" data-synopsis="Generate and parse Apple plist files.">

plistlib

</div>

**Source code:** `Lib/plistlib.py`

<div class="index">

pair: plist; file single: property list

</div>

------------------------------------------------------------------------

This module provides an interface for reading and writing the "property list" files used by Apple, primarily on macOS and iOS. This module supports both binary and XML plist files.

The property list (`.plist`) file format is a simple serialization supporting basic object types, like dictionaries, lists, numbers and strings. Usually the top level object is a dictionary or a frozen dictionary.

To write out and to parse a plist file, use the `dump` and `load` functions.

To work with plist data in bytes or string objects, use `dumps` and `loads`.

Values can be strings, integers, floats, booleans, tuples, lists, dictionaries (but only with string keys), `bytes`, `bytearray` or `datetime.datetime` objects.

<div class="versionchanged">

3.4 New API, old API deprecated. Support for binary format plists added.

</div>

<div class="versionchanged">

3.8 Support added for reading and writing `UID` tokens in binary plists as used by NSKeyedArchiver and NSKeyedUnarchiver.

</div>

<div class="versionchanged">

3.9 Old API removed.

</div>

<div class="seealso">

[PList manual page](https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/PropertyLists/)  
Apple's documentation of the file format.

</div>

This module defines the following functions:

<div class="function">

load(fp, \*, fmt=None, dict_type=dict, aware_datetime=False)

Read a plist file. *fp* should be a readable and binary file object. Return the unpacked root object (which usually is a dictionary).

The *fmt* is the format of the file and the following values are valid:

- `None`: Autodetect the file format
- `FMT_XML`: XML file format
- `FMT_BINARY`: Binary plist format

The *dict_type* is the type used for dictionaries that are read from the plist file.

When *aware_datetime* is true, fields with type `datetime.datetime` will be created as `aware object <datetime-naive-aware>`, with `!tzinfo` as `datetime.UTC`.

XML data for the `FMT_XML` format is parsed using the Expat parser from `xml.parsers.expat` -- see its documentation for possible exceptions on ill-formed XML. Unknown elements will simply be ignored by the plist parser.

The parser raises `InvalidFileException` when the file cannot be parsed.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.13 The keyword-only parameter *aware_datetime* has been added.

</div>

</div>

<div class="function">

loads(data, \*, fmt=None, dict_type=dict, aware_datetime=False)

Load a plist from a bytes or string object. See `load` for an explanation of the keyword arguments.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.13 *data* can be a string when *fmt* equals `FMT_XML`.

</div>

</div>

<div class="function">

dump(value, fp, \*, fmt=FMT_XML, sort_keys=True, skipkeys=False, aware_datetime=False)

Write *value* to a plist file. *fp* should be a writable, binary file object.

The *fmt* argument specifies the format of the plist file and can be one of the following values:

- `FMT_XML`: XML formatted plist file
- `FMT_BINARY`: Binary formatted plist file

When *sort_keys* is true (the default) the keys for dictionaries will be written to the plist in sorted order, otherwise they will be written in the iteration order of the dictionary.

When *skipkeys* is false (the default) the function raises `TypeError` when a key of a dictionary is not a string, otherwise such keys are skipped.

When *aware_datetime* is true and any field with type `datetime.datetime` is set as an `aware object <datetime-naive-aware>`, it will convert to UTC timezone before writing it.

A `TypeError` will be raised if the object is of an unsupported type or a container that contains objects of unsupported types.

An `OverflowError` will be raised for integer values that cannot be represented in (binary) plist files.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.13 The keyword-only parameter *aware_datetime* has been added.

</div>

</div>

<div class="function">

dumps(value, \*, fmt=FMT_XML, sort_keys=True, skipkeys=False, aware_datetime=False)

Return *value* as a plist-formatted bytes object. See the documentation for `dump` for an explanation of the keyword arguments of this function.

<div class="versionadded">

3.4

</div>

</div>

The following classes are available:

<div class="UID(data)">

Wraps an `int`. This is used when reading or writing NSKeyedArchiver encoded data, which contains UID (see PList manual).

<div class="attribute">

data

Int value of the UID. It must be in the range `0 <= data < 2**64`.

</div>

<div class="versionadded">

3.8

</div>

</div>

The following constants are available:

<div class="data">

FMT_XML

The XML format for plist files.

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

FMT_BINARY

The binary format for plist files

<div class="versionadded">

3.4

</div>

</div>

The module defines the following exceptions:

<div class="exception">

InvalidFileException

Raised when a file cannot be parsed.

<div class="versionadded">

3.4

</div>

</div>

## Examples

Generating a plist:

    import datetime as dt
    import plistlib

    pl = dict(
        aString = "Doodah",
        aList = ["A", "B", 12, 32.1, [1, 2, 3]],
        aFloat = 0.1,
        anInt = 728,
        aDict = dict(
            anotherString = "<hello & hi there!>",
            aThirdString = "M\xe4ssig, Ma\xdf",
            aTrueValue = True,
            aFalseValue = False,
        ),
        someData = b"<binary gunk>",
        someMoreData = b"<lots of binary gunk>" * 10,
        aDate = dt.datetime.now()
    )
    print(plistlib.dumps(pl).decode())

Parsing a plist:

    import plistlib

    plist = b"""<plist version="1.0">
    <dict>
        <key>foo</key>
        <string>bar</string>
    </dict>
    </plist>"""
    pl = plistlib.loads(plist)
    print(pl["foo"])

# `!poplib` --- POP3 protocol client

<div class="module" data-synopsis="POP3 protocol client (requires sockets).">

poplib

</div>

**Source code:** `Lib/poplib.py`

<div class="index">

pair: POP3; protocol

</div>

------------------------------------------------------------------------

This module defines a class, `POP3`, which encapsulates a connection to a POP3 server and implements the protocol as defined in `1939`. The `POP3` class supports both the minimal and optional command sets from `1939`. The `POP3` class also supports the `STLS` command introduced in `2595` to enable encrypted communication on an already established connection.

Additionally, this module provides a class `POP3_SSL`, which provides support for connecting to POP3 servers that use SSL as an underlying protocol layer.

Note that POP3, though widely supported, is obsolescent. The implementation quality of POP3 servers varies widely, and too many are quite poor. If your mailserver supports IMAP, you would be better off using the `imaplib.IMAP4` class, as IMAP servers tend to be better implemented.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

The `!poplib` module provides two classes:

<div class="POP3(host, port=POP3_PORT[, timeout])">

This class implements the actual POP3 protocol. The connection is created when the instance is initialized. If *port* is omitted, the standard POP3 port (110) is used. The optional *timeout* parameter specifies a timeout in seconds for the connection attempt (if not specified, the global default timeout setting will be used).

<div class="audit-event">

poplib.connect self,host,port poplib.POP3

</div>

<div class="audit-event">

poplib.putline self,line poplib.POP3

All commands will raise an `auditing event <auditing>` `poplib.putline` with arguments `self` and `line`, where `line` is the bytes about to be sent to the remote host.

</div>

<div class="versionchanged">

3.9 If the *timeout* parameter is set to be zero, it will raise a `ValueError` to prevent the creation of a non-blocking socket.

</div>

</div>

<div class="POP3_SSL(host, port=POP3_SSL_PORT, *, timeout=None, context=None)">

This is a subclass of `POP3` that connects to the server over an SSL encrypted socket. If *port* is not specified, 995, the standard POP3-over-SSL port is used. *timeout* works as in the `POP3` constructor. *context* is an optional `ssl.SSLContext` object which allows bundling SSL configuration options, certificates and private keys into a single (potentially long-lived) structure. Please read `ssl-security` for best practices.

<div class="audit-event">

poplib.connect self,host,port poplib.POP3_SSL

</div>

<div class="audit-event">

poplib.putline self,line poplib.POP3_SSL

All commands will raise an `auditing event <auditing>` `poplib.putline` with arguments `self` and `line`, where `line` is the bytes about to be sent to the remote host.

</div>

<div class="versionchanged">

3.2 *context* parameter added.

</div>

<div class="versionchanged">

3.4 The class now supports hostname check with `ssl.SSLContext.check_hostname` and *Server Name Indication* (see `ssl.HAS_SNI`).

</div>

<div class="versionchanged">

3.9 If the *timeout* parameter is set to be zero, it will raise a `ValueError` to prevent the creation of a non-blocking socket.

</div>

<div class="versionchanged">

3.12 The deprecated *keyfile* and *certfile* parameters have been removed.

</div>

</div>

One exception is defined as an attribute of the `!poplib` module:

<div class="exception">

error_proto

Exception raised on any errors from this module (errors from `socket` module are not caught). The reason for the exception is passed to the constructor as a string.

</div>

<div class="seealso">

Module `imaplib`  
The standard Python IMAP module.

[Frequently Asked Questions About Fetchmail](http://www.catb.org/~esr/fetchmail/fetchmail-FAQ.html)  
The FAQ for the `fetchmail` POP/IMAP client collects information on POP3 server variations and RFC noncompliance that may be useful if you need to write an application based on the POP protocol.

</div>

## POP3 Objects

All POP3 commands are represented by methods of the same name, in lowercase; most return the response text sent by the server.

A `POP3` instance has the following methods:

<div class="method">

POP3.set_debuglevel(level)

Set the instance's debugging level. This controls the amount of debugging output printed. The default, `0`, produces no debugging output. A value of `1` produces a moderate amount of debugging output, generally a single line per request. A value of `2` or higher produces the maximum amount of debugging output, logging each line sent and received on the control connection.

</div>

<div class="method">

POP3.getwelcome()

Returns the greeting string sent by the POP3 server.

</div>

<div class="method">

POP3.capa()

Query the server's capabilities as specified in `2449`. Returns a dictionary in the form `{'name': ['param'...]}`.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

POP3.user(username)

Send user command, response should indicate that a password is required.

</div>

<div class="method">

[POP3.pass]()(password)

Send password, response includes message count and mailbox size. Note: the mailbox on the server is locked until `~POP3.quit` is called.

</div>

<div class="method">

POP3.apop(user, secret)

Use the more secure APOP authentication to log into the POP3 server.

</div>

<div class="method">

POP3.rpop(user)

Use RPOP authentication (similar to UNIX r-commands) to log into POP3 server.

</div>

<div class="method">

POP3.stat()

Get mailbox status. The result is a tuple of 2 integers: `(message count, mailbox size)`.

</div>

<div class="method">

POP3.list(\[which\])

Request message list, result is in the form `(response, ['mesg_num octets', ...], octets)`. If *which* is set, it is the message to list.

</div>

<div class="method">

POP3.retr(which)

Retrieve whole message number *which*, and set its seen flag. Result is in form `(response, ['line', ...], octets)`.

</div>

<div class="method">

POP3.dele(which)

Flag message number *which* for deletion. On most servers deletions are not actually performed until QUIT (the major exception is Eudora QPOP, which deliberately violates the RFCs by doing pending deletes on any disconnect).

</div>

<div class="method">

POP3.rset()

Remove any deletion marks for the mailbox.

</div>

<div class="method">

POP3.noop()

Do nothing. Might be used as a keep-alive.

</div>

<div class="method">

POP3.quit()

Signoff: commit changes, unlock mailbox, drop connection.

</div>

<div class="method">

POP3.top(which, howmuch)

Retrieves the message header plus *howmuch* lines of the message after the header of message number *which*. Result is in form `(response, ['line', ...], octets)`.

The POP3 TOP command this method uses, unlike the RETR command, doesn't set the message's seen flag; unfortunately, TOP is poorly specified in the RFCs and is frequently broken in off-brand servers. Test this method by hand against the POP3 servers you will use before trusting it.

</div>

<div class="method">

POP3.uidl(which=None)

Return message digest (unique id) list. If *which* is specified, result contains the unique id for that message in the form `'response mesgnum uid`, otherwise result is list `(response, ['mesgnum uid', ...], octets)`.

</div>

<div class="method">

POP3.utf8()

Try to switch to UTF-8 mode. Returns the server response if successful, raises `error_proto` if not. Specified in [RFC 6856](http://www.faqs.org/rfcs/rfc6856.html "RFC 6856").

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

POP3.stls(context=None)

Start a TLS session on the active connection as specified in `2595`. This is only allowed before user authentication

*context* parameter is a `ssl.SSLContext` object which allows bundling SSL configuration options, certificates and private keys into a single (potentially long-lived) structure. Please read `ssl-security` for best practices.

This method supports hostname checking via `ssl.SSLContext.check_hostname` and *Server Name Indication* (see `ssl.HAS_SNI`).

<div class="versionadded">

3.4

</div>

</div>

Instances of `POP3_SSL` have no additional methods. The interface of this subclass is identical to its parent.

## POP3 Example

Here is a minimal example (without error checking) that opens a mailbox and retrieves and prints all messages:

    import getpass, poplib

    M = poplib.POP3('localhost')
    M.user(getpass.getuser())
    M.pass_(getpass.getpass())
    numMessages = len(M.list()[1])
    for i in range(numMessages):
        for j in M.retr(i+1)[1]:
            print(j)

At the end of the module, there is a test section that contains a more extensive example of usage.

# `!posix` --- The most common POSIX system calls

<div class="module" data-synopsis="The most common POSIX system calls (normally used via module os).">

posix

</div>

------------------------------------------------------------------------

This module provides access to operating system functionality that is standardized by the C Standard and the POSIX standard (a thinly disguised Unix interface).

<div class="availability">

Unix.

</div>

<div class="index">

pair: module; os

</div>

**Do not import this module directly.** Instead, import the module `os`, which provides a *portable* version of this interface. On Unix, the `os` module provides a superset of the `!posix` interface. On non-Unix operating systems the `!posix` module is not available, but a subset is always available through the `os` interface. Once `os` is imported, there is *no* performance penalty in using it instead of `!posix`. In addition, `os` provides some additional functionality, such as automatically calling `~os.putenv` when an entry in `os.environ` is changed.

Errors are reported as exceptions; the usual exceptions are given for type errors, while errors reported by the system calls raise `OSError`.

## Large File Support

<div class="index">

single: large files single: file; large files

</div>

Several operating systems (including AIX and Solaris) provide support for files that are larger than 2 GiB from a C programming model where `int` and `long` are 32-bit values. This is typically accomplished by defining the relevant size and offset types as 64-bit values. Such files are sometimes referred to as `large files`.

Large file support is enabled in Python when the size of an `off_t` is larger than a `long` and the `long long` is at least as large as an `off_t`. It may be necessary to configure and compile Python with certain compiler flags to enable this mode. For example, with Solaris 2.6 and 2.7 you need to do something like:

    CFLAGS="`getconf LFS_CFLAGS`" OPT="-g -O2 $CFLAGS" \
            ./configure

On large-file-capable Linux systems, this might work:

    CFLAGS='-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64' OPT="-g -O2 $CFLAGS" \
            ./configure

## Notable Module Contents

In addition to many functions described in the `os` module documentation, `!posix` defines the following data item:

<div class="data">

environ

A dictionary representing the string environment at the time the interpreter was started. Keys and values are bytes on Unix and str on Windows. For example, `environ[b'HOME']` (`environ['HOME']` on Windows) is the pathname of your home directory, equivalent to `getenv("HOME")` in C.

Modifying this dictionary does not affect the string environment passed on by `~os.execv`, `~os.popen` or `~os.system`; if you need to change the environment, pass `environ` to `~os.execve` or add variable assignments and export statements to the command string for `~os.system` or `~os.popen`.

<div class="versionchanged">

3.2 On Unix, keys and values are bytes.

</div>

> [!NOTE]
> The `os` module provides an alternate implementation of `environ` which updates the environment on modification. Note also that updating `os.environ` will render this dictionary obsolete. Use of the `os` module version of this is recommended over direct access to the `!posix` module.

</div>

# `!pprint` --- Data pretty printer

<div class="module" data-synopsis="Data pretty printer.">

pprint

</div>

**Source code:** `Lib/pprint.py`

------------------------------------------------------------------------

The `!pprint` module provides a capability to "pretty-print" arbitrary Python data structures in a form which can be used as input to the interpreter. If the formatted structures include objects which are not fundamental Python types, the representation may not be loadable. This may be the case if objects such as files, sockets or classes are included, as well as many other objects which are not representable as Python literals.

The formatted representation keeps objects on a single line if it can, and breaks them onto multiple lines if they don't fit within the allowed width, adjustable by the *width* parameter defaulting to 80 characters.

<div class="versionchanged">

3.9 Added support for pretty-printing `types.SimpleNamespace`.

</div>

<div class="versionchanged">

3.10 Added support for pretty-printing `dataclasses.dataclass`.

</div>

## Functions

<div class="function">

pp(object, stream=None, indent=1, width=80, depth=None, \*, compact=False, expand=False, sort_dicts=False, underscore_numbers=False)

Prints the formatted representation of *object*, followed by a newline. This function may be used in the interactive interpreter instead of the `print` function for inspecting values. Tip: you can reassign `print = pprint.pp` for use within a scope.

param object  
The object to be printed.

param stream  
A file-like object to which the output will be written by calling its `!write` method. If `None` (the default), `sys.stdout` is used.

type stream  
`file-like object` \| None

param int indent  
The amount of indentation added for each nesting level.

param int width  
The desired maximum number of characters per line in the output. If a structure cannot be formatted within the width constraint, a best effort will be made.

param depth  
The number of nesting levels which may be printed. If the data structure being printed is too deep, the next contained level is replaced by `...`. If `None` (the default), there is no constraint on the depth of the objects being formatted.

type depth  
int \| None

param bool compact  
Control the way long `sequences <sequence>` are formatted. If `False` (the default), each item of a sequence will be formatted on a separate line, otherwise as many items as will fit within the *width* will be formatted on each output line. Incompatible with *expand*.

param bool expand  
If `True`, opening parentheses and brackets will be followed by a newline and the following content will be indented by one level, similar to pretty-printed JSON. Incompatible with *compact*.

param bool sort_dicts  
If `True`, dictionaries will be formatted with their keys sorted, otherwise they will be displayed in insertion order (the default).

param bool underscore_numbers  
If `True`, integers will be formatted with the `_` character for a thousands separator, otherwise underscores are not displayed (the default).

\>\>\> import pprint \>\>\> stuff = \['spam', 'eggs', 'lumberjack', 'knights', 'ni'\] \>\>\> stuff.insert(0, stuff) \>\>\> pprint.pp(stuff) \[\<Recursion on list with id=...\>, 'spam', 'eggs', 'lumberjack', 'knights', 'ni'\]

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.10 Added the *underscore_numbers* parameter.

</div>

<div class="versionchanged">

3.11 No longer attempts to write to `!sys.stdout` if it is `None`.

</div>

<div class="versionchanged">

3.15 Added the *expand* parameter.

</div>

</div>

<div class="function">

pprint(object, stream=None, indent=1, width=80, depth=None, \*, compact=False, expand=False, sort_dicts=True, underscore_numbers=False)

Alias for `~pprint.pp` with *sort_dicts* set to `True` by default, which would automatically sort the dictionaries' keys, you might want to use `~pprint.pp` instead where it is `False` by default.

</div>

<div class="function">

pformat(object, indent=1, width=80, depth=None, \*, compact=False, expand=False, sort_dicts=True, underscore_numbers=False)

Return the formatted representation of *object* as a string. *indent*, *width*, *depth*, *compact*, *expand*, *sort_dicts* and *underscore_numbers* are passed to the `PrettyPrinter` constructor as formatting parameters and their meanings are as described in the documentation above.

</div>

<div class="function">

isreadable(object)

<div class="index">

pair: built-in function; eval

</div>

Determine if the formatted representation of *object* is "readable", or can be used to reconstruct the value using `eval`. This always returns `False` for recursive objects.

> \>\>\> pprint.isreadable(stuff) False

</div>

<div class="function">

isrecursive(object)

Determine if *object* requires a recursive representation. This function is subject to the same limitations as noted in `saferepr` below and may raise an `RecursionError` if it fails to detect a recursive object.

</div>

<div class="function">

saferepr(object)

Return a string representation of *object*, protected against recursion in some common data structures, namely instances of `dict`, `list` and `tuple` or subclasses whose `__repr__` has not been overridden. If the representation of object exposes a recursive entry, the recursive reference will be represented as `<Recursion on typename with id=number>`. The representation is not otherwise formatted.

\>\>\> pprint.saferepr(stuff) "\[\<Recursion on list with id=...\>, 'spam', 'eggs', 'lumberjack', 'knights', 'ni'\]"

</div>

## PrettyPrinter Objects

<div class="index">

single: ...; placeholder

</div>

<div class="PrettyPrinter(indent=1, width=80, depth=None, stream=None, *, \ compact=False, expand=False, sort_dicts=True, \ underscore_numbers=False)">

Construct a `PrettyPrinter` instance.

Arguments have the same meaning as for `~pprint.pp`. Note that they are in a different order, and that *sort_dicts* defaults to `True`.

\>\>\> import pprint \>\>\> stuff = \['spam', 'eggs', 'lumberjack', 'knights', 'ni'\] \>\>\> stuff.insert(0, stuff\[:\]) \>\>\> pp = pprint.PrettyPrinter(indent=4) \>\>\> pp.pprint(stuff) \[ \['spam', 'eggs', 'lumberjack', 'knights', 'ni'\], 'spam', 'eggs', 'lumberjack', 'knights', 'ni'\] \>\>\> pp = pprint.PrettyPrinter(width=41, compact=True) \>\>\> pp.pprint(stuff) \[\['spam', 'eggs', 'lumberjack', 'knights', 'ni'\], 'spam', 'eggs', 'lumberjack', 'knights', 'ni'\] \>\>\> pp = pprint.PrettyPrinter(width=41, expand=True, indent=3) \>\>\> pp.pprint(stuff) \[ \[ 'spam', 'eggs', 'lumberjack', 'knights', 'ni', \], 'spam', 'eggs', 'lumberjack', 'knights', 'ni', \] \>\>\> tup = ('spam', ('eggs', ('lumberjack', ('knights', ('ni', ('dead', ... ('parrot', ('fresh fruit',)))))))) \>\>\> pp = pprint.PrettyPrinter(depth=6) \>\>\> pp.pprint(tup) ('spam', ('eggs', ('lumberjack', ('knights', ('ni', ('dead', (...)))))))

<div class="versionchanged">

3.4 Added the *compact* parameter.

</div>

<div class="versionchanged">

3.8 Added the *sort_dicts* parameter.

</div>

<div class="versionchanged">

3.10 Added the *underscore_numbers* parameter.

</div>

<div class="versionchanged">

3.11 No longer attempts to write to `!sys.stdout` if it is `None`.

</div>

<div class="versionchanged">

3.15 Added the *expand* parameter.

</div>

</div>

`PrettyPrinter` instances have the following methods:

<div class="method">

PrettyPrinter.pformat(object)

Return the formatted representation of *object*. This takes into account the options passed to the `PrettyPrinter` constructor.

</div>

<div class="method">

PrettyPrinter.pprint(object)

Print the formatted representation of *object* on the configured stream, followed by a newline.

</div>

The following methods provide the implementations for the corresponding functions of the same names. Using these methods on an instance is slightly more efficient since new `PrettyPrinter` objects don't need to be created.

<div class="method">

PrettyPrinter.isreadable(object)

<div class="index">

pair: built-in function; eval

</div>

Determine if the formatted representation of the object is "readable," or can be used to reconstruct the value using `eval`. Note that this returns `False` for recursive objects. If the *depth* parameter of the `PrettyPrinter` is set and the object is deeper than allowed, this returns `False`.

</div>

<div class="method">

PrettyPrinter.isrecursive(object)

Determine if the object requires a recursive representation.

</div>

This method is provided as a hook to allow subclasses to modify the way objects are converted to strings. The default implementation uses the internals of the `saferepr` implementation.

<div class="method">

PrettyPrinter.format(object, context, maxlevels, level)

Returns three values: the formatted version of *object* as a string, a flag indicating whether the result is readable, and a flag indicating whether recursion was detected. The first argument is the object to be presented. The second is a dictionary which contains the `id` of objects that are part of the current presentation context (direct and indirect containers for *object* that are affecting the presentation) as the keys; if an object needs to be presented which is already represented in *context*, the third return value should be `True`. Recursive calls to the `.format` method should add additional entries for containers to this dictionary. The third argument, *maxlevels*, gives the requested limit to recursion; this will be `0` if there is no requested limit. This argument should be passed unmodified to recursive calls. The fourth argument, *level*, gives the current level; recursive calls should be passed a value less than that of the current call.

</div>

## Example

To demonstrate several uses of the `~pprint.pp` function and its parameters, let's fetch information about a project from [PyPI](https://pypi.org):

    >>> import json
    >>> import pprint
    >>> from urllib.request import urlopen
    >>> with urlopen('https://pypi.org/pypi/sampleproject/1.2.0/json') as resp:
    ...     project_info = json.load(resp)['info']

In its basic form, `~pprint.pp` shows the whole object:

    >>> pprint.pp(project_info)
    {'author': 'The Python Packaging Authority',
     'author_email': 'pypa-dev@googlegroups.com',
     'bugtrack_url': None,
     'classifiers': ['Development Status :: 3 - Alpha',
                     'Intended Audience :: Developers',
                     'License :: OSI Approved :: MIT License',
                     'Programming Language :: Python :: 2',
                     'Programming Language :: Python :: 2.6',
                     'Programming Language :: Python :: 2.7',
                     'Programming Language :: Python :: 3',
                     'Programming Language :: Python :: 3.2',
                     'Programming Language :: Python :: 3.3',
                     'Programming Language :: Python :: 3.4',
                     'Topic :: Software Development :: Build Tools'],
     'description': 'A sample Python project\n'
                    '=======================\n'
                    '\n'
                    'This is the description file for the project.\n'
                    '\n'
                    'The file should use UTF-8 encoding and be written using '
                    'ReStructured Text. It\n'
                    'will be used to generate the project webpage on PyPI, and '
                    'should be written for\n'
                    'that purpose.\n'
                    '\n'
                    'Typical contents for this file would include an overview of '
                    'the project, basic\n'
                    'usage examples, etc. Generally, including the project '
                    'changelog in here is not\n'
                    'a good idea, although a simple "What\'s New" section for the '
                    'most recent version\n'
                    'may be appropriate.',
     'description_content_type': None,
     'docs_url': None,
     'download_url': 'UNKNOWN',
     'downloads': {'last_day': -1, 'last_month': -1, 'last_week': -1},
     'home_page': 'https://github.com/pypa/sampleproject',
     'keywords': 'sample setuptools development',
     'license': 'MIT',
     'maintainer': None,
     'maintainer_email': None,
     'name': 'sampleproject',
     'package_url': 'https://pypi.org/project/sampleproject/',
     'platform': 'UNKNOWN',
     'project_url': 'https://pypi.org/project/sampleproject/',
     'project_urls': {'Download': 'UNKNOWN',
                      'Homepage': 'https://github.com/pypa/sampleproject'},
     'release_url': 'https://pypi.org/project/sampleproject/1.2.0/',
     'requires_dist': None,
     'requires_python': None,
     'summary': 'A sample Python project',
     'version': '1.2.0'}

The result can be limited to a certain *depth* (ellipsis is used for deeper contents):

    >>> pprint.pp(project_info, depth=1)
    {'author': 'The Python Packaging Authority',
     'author_email': 'pypa-dev@googlegroups.com',
     'bugtrack_url': None,
     'classifiers': [...],
     'description': 'A sample Python project\n'
                    '=======================\n'
                    '\n'
                    'This is the description file for the project.\n'
                    '\n'
                    'The file should use UTF-8 encoding and be written using '
                    'ReStructured Text. It\n'
                    'will be used to generate the project webpage on PyPI, and '
                    'should be written for\n'
                    'that purpose.\n'
                    '\n'
                    'Typical contents for this file would include an overview of '
                    'the project, basic\n'
                    'usage examples, etc. Generally, including the project '
                    'changelog in here is not\n'
                    'a good idea, although a simple "What\'s New" section for the '
                    'most recent version\n'
                    'may be appropriate.',
     'description_content_type': None,
     'docs_url': None,
     'download_url': 'UNKNOWN',
     'downloads': {...},
     'home_page': 'https://github.com/pypa/sampleproject',
     'keywords': 'sample setuptools development',
     'license': 'MIT',
     'maintainer': None,
     'maintainer_email': None,
     'name': 'sampleproject',
     'package_url': 'https://pypi.org/project/sampleproject/',
     'platform': 'UNKNOWN',
     'project_url': 'https://pypi.org/project/sampleproject/',
     'project_urls': {...},
     'release_url': 'https://pypi.org/project/sampleproject/1.2.0/',
     'requires_dist': None,
     'requires_python': None,
     'summary': 'A sample Python project',
     'version': '1.2.0'}

Additionally, maximum character *width* can be suggested. If a long object cannot be split, the specified width will be exceeded:

    >>> pprint.pp(project_info, depth=1, width=60)
    {'author': 'The Python Packaging Authority',
     'author_email': 'pypa-dev@googlegroups.com',
     'bugtrack_url': None,
     'classifiers': [...],
     'description': 'A sample Python project\n'
                    '=======================\n'
                    '\n'
                    'This is the description file for the '
                    'project.\n'
                    '\n'
                    'The file should use UTF-8 encoding and be '
                    'written using ReStructured Text. It\n'
                    'will be used to generate the project '
                    'webpage on PyPI, and should be written '
                    'for\n'
                    'that purpose.\n'
                    '\n'
                    'Typical contents for this file would '
                    'include an overview of the project, '
                    'basic\n'
                    'usage examples, etc. Generally, including '
                    'the project changelog in here is not\n'
                    'a good idea, although a simple "What\'s '
                    'New" section for the most recent version\n'
                    'may be appropriate.',
     'description_content_type': None,
     'docs_url': None,
     'download_url': 'UNKNOWN',
     'downloads': {...},
     'home_page': 'https://github.com/pypa/sampleproject',
     'keywords': 'sample setuptools development',
     'license': 'MIT',
     'maintainer': None,
     'maintainer_email': None,
     'name': 'sampleproject',
     'package_url': 'https://pypi.org/project/sampleproject/',
     'platform': 'UNKNOWN',
     'project_url': 'https://pypi.org/project/sampleproject/',
     'project_urls': {...},
     'release_url': 'https://pypi.org/project/sampleproject/1.2.0/',
     'requires_dist': None,
     'requires_python': None,
     'summary': 'A sample Python project',
     'version': '1.2.0'}

Lastly, we can format like pretty-printed JSON with the *expand* parameter. Best results are achieved with a higher *indent* value:

    >>> pprint.pp(project_info, indent=4, expand=True)
    {
       'author': 'The Python Packaging Authority',
       'author_email': 'pypa-dev@googlegroups.com',
       'bugtrack_url': None,
       'classifiers': [
          'Development Status :: 3 - Alpha',
          'Intended Audience :: Developers',
          'License :: OSI Approved :: MIT License',
          'Programming Language :: Python :: 2',
          'Programming Language :: Python :: 2.6',
          'Programming Language :: Python :: 2.7',
          'Programming Language :: Python :: 3',
          'Programming Language :: Python :: 3.2',
          'Programming Language :: Python :: 3.3',
          'Programming Language :: Python :: 3.4',
          'Topic :: Software Development :: Build Tools',
       ],
       'description': 'A sample Python project\n'
       '=======================\n'
       '\n'
       'This is the description file for the project.\n'
       '\n'
       'The file should use UTF-8 encoding and be written using ReStructured '
       'Text. It\n'
       'will be used to generate the project webpage on PyPI, and should be '
       'written for\n'
       'that purpose.\n'
       '\n'
       'Typical contents for this file would include an overview of the project, '
       'basic\n'
       'usage examples, etc. Generally, including the project changelog in here '
       'is not\n'
       'a good idea, although a simple "What\'s New" section for the most recent '
       'version\n'
       'may be appropriate.',
       'description_content_type': None,
       'docs_url': None,
       'download_url': 'UNKNOWN',
       'downloads': {'last_day': -1, 'last_month': -1, 'last_week': -1},
       'dynamic': None,
       'home_page': 'https://github.com/pypa/sampleproject',
       'keywords': 'sample setuptools development',
       'license': 'MIT',
       'license_expression': None,
       'license_files': None,
       'maintainer': None,
       'maintainer_email': None,
       'name': 'sampleproject',
       'package_url': 'https://pypi.org/project/sampleproject/',
       'platform': 'UNKNOWN',
       'project_url': 'https://pypi.org/project/sampleproject/',
       'project_urls': {
          'Download': 'UNKNOWN',
          'Homepage': 'https://github.com/pypa/sampleproject',
       },
       'provides_extra': None,
       'release_url': 'https://pypi.org/project/sampleproject/1.2.0/',
       'requires_dist': None,
       'requires_python': None,
       'summary': 'A sample Python project',
       'version': '1.2.0',
       'yanked': False,
       'yanked_reason': None,
    }

# `!profile` --- Pure Python profiler

<div class="module" data-synopsis="Pure Python profiler (deprecated)." data-deprecated="">

profile

</div>

**Source code:** `Lib/profile.py`

------------------------------------------------------------------------

<div class="deprecated-removed">

3.15 3.17

</div>

The `!profile` module is deprecated and will be removed in Python 3.17. Use `profiling.tracing` instead.

The `!profile` module provides a pure Python implementation of a deterministic profiler. While useful for understanding profiler internals or extending profiler behavior through subclassing, its pure Python implementation introduces significant overhead compared to the C-based `profiling.tracing` module.

For most profiling tasks, use:

- `profiling.sampling` for production debugging with zero overhead
- `profiling.tracing` for development and testing

## Migration

Migrating from `!profile` to `profiling.tracing` is straightforward. The APIs are compatible:

    # Old (deprecated)
    import profile
    profile.run('my_function()')

    # New (recommended)
    import profiling.tracing
    profiling.tracing.run('my_function()')

For most code, replacing `import profile` with `import profiling.tracing` (and using `profiling.tracing` instead of `profile` throughout) provides a straightforward migration path.

> [!NOTE]
> The `cProfile` module remains available as a backward-compatible alias to `profiling.tracing`. Existing code using `import cProfile` will continue to work without modification.

## `!profile` and `!profiling.tracing` module reference

Both the `!profile` and `profiling.tracing` modules provide the following functions:

<div class="function">

run(command, filename=None, sort=-1)

This function takes a single argument that can be passed to the `exec` function, and an optional file name. In all cases this routine executes:

    exec(command, __main__.__dict__, __main__.__dict__)

and gathers profiling statistics from the execution. If no file name is present, then this function automatically creates a `~pstats.Stats` instance and prints a simple profiling report. If the sort value is specified, it is passed to this `~pstats.Stats` instance to control how the results are sorted.

</div>

<div class="function">

runctx(command, globals, locals, filename=None, sort=-1)

This function is similar to `run`, with added arguments to supply the globals and locals mappings for the *command* string. This routine executes:

    exec(command, globals, locals)

and gathers profiling statistics as in the `run` function above.

</div>

<div class="Profile(timer=None, timeunit=0.0, subcalls=True, builtins=True)">

This class is normally only used if more precise control over profiling is needed than what the `profiling.tracing.run` function provides.

A custom timer can be supplied for measuring how long code takes to run via the *timer* argument. This must be a function that returns a single number representing the current time. If the number is an integer, the *timeunit* specifies a multiplier that specifies the duration of each unit of time. For example, if the timer returns times measured in thousands of seconds, the time unit would be `.001`.

Directly using the `Profile` class allows formatting profile results without writing the profile data to a file:

    import profiling.tracing
    import pstats
    import io
    from pstats import SortKey

    pr = profiling.tracing.Profile()
    pr.enable()
    # ... do something ...
    pr.disable()
    s = io.StringIO()
    sortby = SortKey.CUMULATIVE
    ps = pstats.Stats(pr, stream=s).sort_stats(sortby)
    ps.print_stats()
    print(s.getvalue())

The `Profile` class can also be used as a context manager (supported only in `profiling.tracing`, not in the deprecated `!profile` module; see `typecontextmanager`):

    import profiling.tracing

    with profiling.tracing.Profile() as pr:
        # ... do something ...

        pr.print_stats()

<div class="versionchanged">

3.8 Added context manager support.

</div>

<div class="method">

enable()

Start collecting profiling data. Only in `profiling.tracing`.

</div>

<div class="method">

disable()

Stop collecting profiling data. Only in `profiling.tracing`.

</div>

<div class="method">

create_stats()

Stop collecting profiling data and record the results internally as the current profile.

</div>

<div class="method">

print_stats(sort=-1)

Create a `~pstats.Stats` object based on the current profile and print the results to stdout.

The *sort* parameter specifies the sorting order of the displayed statistics. It accepts a single key or a tuple of keys to enable multi-level sorting, as in `pstats.Stats.sort_stats`.

<div class="versionadded">

3.13 `~Profile.print_stats` now accepts a tuple of keys.

</div>

</div>

<div class="method">

dump_stats(filename)

Write the results of the current profile to *filename*.

</div>

<div class="method">

run(cmd)

Profile the cmd via `exec`.

</div>

<div class="method">

runctx(cmd, globals, locals)

Profile the cmd via `exec` with the specified global and local environment.

</div>

<div class="method">

runcall(func, /, *args,*\*kwargs)

Profile `func(*args, **kwargs)`

</div>

</div>

Note that profiling will only work if the called command/function actually returns. If the interpreter is terminated (e.g. via a `sys.exit` call during the called command/function execution) no profiling results will be printed.

## Differences from `!profiling.tracing`

The `!profile` module differs from `profiling.tracing` in several ways:

**Higher overhead.** The pure Python implementation is significantly slower than the C implementation, making it unsuitable for profiling long-running programs or performance-sensitive code.

**Calibration support.** The `!profile` module supports calibration to compensate for profiling overhead. This is not needed in `profiling.tracing` because the C implementation has negligible overhead.

**Custom timers.** Both modules support custom timers, but `!profile` accepts timer functions that return tuples (like `os.times`), while `profiling.tracing` requires a function returning a single number.

**Subclassing.** The pure Python implementation is easier to subclass and extend for custom profiling behavior.

## What is deterministic profiling?

`Deterministic profiling` is meant to reflect the fact that all *function call*, *function return*, and *exception* events are monitored, and precise timings are made for the intervals between these events (during which time the user's code is executing). In contrast, `statistical profiling` (which is provided by the `profiling.sampling` module) periodically samples the effective instruction pointer, and deduces where time is being spent. The latter technique traditionally involves less overhead (as the code does not need to be instrumented), but provides only relative indications of where time is being spent.

In Python, since there is an interpreter active during execution, the presence of instrumented code is not required in order to do deterministic profiling. Python automatically provides a `hook` (optional callback) for each event. In addition, the interpreted nature of Python tends to add so much overhead to execution, that deterministic profiling tends to only add small processing overhead in typical applications. The result is that deterministic profiling is not that expensive, yet provides extensive run time statistics about the execution of a Python program.

Call count statistics can be used to identify bugs in code (surprising counts), and to identify possible inline-expansion points (high call counts). Internal time statistics can be used to identify "hot loops" that should be carefully optimized. Cumulative time statistics should be used to identify high level errors in the selection of algorithms. Note that the unusual handling of cumulative times in this profiler allows statistics for recursive implementations of algorithms to be directly compared to iterative implementations.

## Limitations

One limitation has to do with accuracy of timing information. There is a fundamental problem with deterministic profilers involving accuracy. The most obvious restriction is that the underlying "clock" is only ticking at a rate (typically) of about .001 seconds. Hence no measurements will be more accurate than the underlying clock. If enough measurements are taken, then the "error" will tend to average out. Unfortunately, removing this first error induces a second source of error.

The second problem is that it "takes a while" from when an event is dispatched until the profiler's call to get the time actually *gets* the state of the clock. Similarly, there is a certain lag when exiting the profiler event handler from the time that the clock's value was obtained (and then squirreled away), until the user's code is once again executing. As a result, functions that are called many times, or call many functions, will typically accumulate this error. The error that accumulates in this fashion is typically less than the accuracy of the clock (less than one clock tick), but it *can* accumulate and become very significant.

The problem is more important with the deprecated `!profile` module than with the lower-overhead `profiling.tracing`. For this reason, `!profile` provides a means of calibrating itself for a given platform so that this error can be probabilistically (on the average) removed. After the profiler is calibrated, it will be more accurate (in a least square sense), but it will sometimes produce negative numbers (when call counts are exceptionally low, and the gods of probability work against you :-). ) Do *not* be alarmed by negative numbers in the profile. They should *only* appear if you have calibrated your profiler, and the results are actually better than without calibration.

## Calibration

The profiler of the `!profile` module subtracts a constant from each event handling time to compensate for the overhead of calling the time function, and socking away the results. By default, the constant is 0. The following procedure can be used to obtain a better constant for a given platform (see `profile-limitations`). :

    import profile
    pr = profile.Profile()
    for i in range(5):
        print(pr.calibrate(10000))

The method executes the number of Python calls given by the argument, directly and again under the profiler, measuring the time for both. It then computes the hidden overhead per profiler event, and returns that as a float. For example, on a 1.8Ghz Intel Core i5 running macOS, and using Python's time.process_time() as the timer, the magical number is about 4.04e-6.

The object of this exercise is to get a fairly consistent result. If your computer is *very* fast, or your timer function has poor resolution, you might have to pass 100000, or even 1000000, to get consistent results.

When you have a consistent answer, there are three ways you can use it:

    import profile

    # 1. Apply computed bias to all Profile instances created hereafter.
    profile.Profile.bias = your_computed_bias

    # 2. Apply computed bias to a specific Profile instance.
    pr = profile.Profile()
    pr.bias = your_computed_bias

    # 3. Specify computed bias in instance constructor.
    pr = profile.Profile(bias=your_computed_bias)

If you have a choice, you are better off choosing a smaller constant, and then your results will "less often" show up as negative in profile statistics.

## Using a custom timer

If you want to change how current time is determined (for example, to force use of wall-clock time or elapsed process time), pass the timing function you want to the `Profile` class constructor:

    pr = profile.Profile(your_time_func)

The resulting profiler will then call `your_time_func`. Depending on whether you are using `profile.Profile` or `profiling.tracing.Profile`, `your_time_func`'s return value will be interpreted differently:

`profile.Profile`  
`your_time_func` should return a single number, or a list of numbers whose sum is the current time (like what `os.times` returns). If the function returns a single time number, or the list of returned numbers has length 2, then you will get an especially fast version of the dispatch routine.

Be warned that you should calibrate the profiler class for the timer function that you choose (see `profile-calibration`). For most machines, a timer that returns a lone integer value will provide the best results in terms of low overhead during profiling. (`os.times` is *pretty* bad, as it returns a tuple of floating-point values). If you want to substitute a better timer in the cleanest fashion, derive a class and hardwire a replacement dispatch method that best handles your timer call, along with the appropriate calibration constant.

`profiling.tracing.Profile`  
`your_time_func` should return a single number. If it returns integers, you can also invoke the class constructor with a second argument specifying the real duration of one unit of time. For example, if `your_integer_time_func` returns times measured in thousands of seconds, you would construct the `Profile` instance as follows:

    pr = profiling.tracing.Profile(your_integer_time_func, 0.001)

As the `profiling.tracing.Profile` class cannot be calibrated, custom timer functions should be used with care and should be as fast as possible. For the best results with a custom timer, it might be necessary to hard-code it in the C source of the internal `!_lsprof` module.

Python 3.3 adds several new functions in `time` that can be used to make precise measurements of process or wall-clock time. For example, see `time.perf_counter`.

<div class="seealso">

`profiling`  
Overview of Python profiling tools.

`profiling.tracing`  
Recommended replacement for this module.

`pstats`  
Statistical analysis and formatting for profile data.

</div>

# `!profiling` --- Python profilers

<div class="module" data-synopsis="Python profiling tools for performance analysis.">

profiling

</div>

<div class="versionadded">

3.15

</div>

**Source code:** `Lib/profiling/`

------------------------------------------------------------------------

<div class="index">

single: statistical profiling single: profiling, statistical single: deterministic profiling single: profiling, deterministic

</div>

## Introduction to profiling

A `profile` is a set of statistics that describes how often and for how long various parts of a program execute. These statistics help identify performance bottlenecks and guide optimization efforts. Python provides two fundamentally different approaches to collecting this information: statistical sampling and deterministic tracing.

The `!profiling` package organizes Python's built-in profiling tools under a single namespace. It contains two submodules, each implementing a different profiling methodology:

`profiling.sampling`  
A statistical profiler that periodically samples the call stack. Run scripts directly or attach to running processes by PID. Provides multiple output formats (flame graphs, heatmaps, Firefox Profiler), GIL analysis, GC tracking, and multiple profiling modes (wall-clock, CPU, GIL) with virtually no overhead.

`profiling.tracing`  
A deterministic profiler that traces every function call, return, and exception event. Provides exact call counts and precise timing information, capturing every invocation including very fast functions.

> [!NOTE]
> The profiler modules are designed to provide an execution profile for a given program, not for benchmarking purposes. For benchmarking, use the `timeit` module, which provides reasonably accurate timing measurements. This distinction is particularly important when comparing Python code against C code: deterministic profilers introduce overhead for Python code but not for C-level functions, which can skew comparisons.

## Choosing a profiler

For most performance analysis, use the statistical profiler (`profiling.sampling`). It has minimal overhead, works for both development and production, and provides rich visualization options including flame graphs, heatmaps, GIL analysis, and more.

Use the deterministic profiler (`profiling.tracing`) when you need **exact call counts** and cannot afford to miss any function calls. Since it instruments every function call and return, it will capture even very fast functions that complete between sampling intervals. The tradeoff is higher overhead.

The following table summarizes the key differences:

| Feature | Statistical sampling (`profiling.sampling`) | Deterministic (`profiling.tracing`) |
|----|----|----|
| **Overhead** | Virtually none | Moderate |
| **Accuracy** | Statistical estimate | Exact call counts |
| **Output formats** | pstats, flame graph, heatmap, gecko, collapsed | pstats |
| **Profiling modes** | Wall-clock, CPU, GIL | Wall-clock |
| **Special frames** | GC, native (C extensions) | N/A |
| **Attach to PID** | Yes | No |

### When to use statistical sampling

The statistical profiler (`profiling.sampling`) is recommended for most performance analysis tasks. Use it the same way you would use `profiling.tracing`:

``` sh
python -m profiling.sampling run script.py
```

One of the main strengths of the sampling profiler is its variety of output formats. Beyond traditional pstats tables, it can generate interactive flame graphs that visualize call hierarchies, line-level source heatmaps that show exactly where time is spent in your code, and Firefox Profiler output for timeline-based analysis.

The profiler also provides insight into Python interpreter behavior that deterministic profiling cannot capture. Use `--mode gil` to identify GIL contention in multi-threaded code, `--mode cpu` to measure actual CPU time excluding I/O waits, or inspect `<GC>` frames to understand garbage collection overhead. The `--native` option reveals time spent in C extensions, helping distinguish Python overhead from library performance.

For multi-threaded applications, the `-a` option samples all threads simultaneously, showing how work is distributed. And for production debugging, the `attach` command connects to any running Python process by PID without requiring a restart or code changes.

### When to use deterministic tracing

The deterministic profiler (`profiling.tracing`) instruments every function call and return. This approach has higher overhead than sampling, but guarantees complete coverage of program execution.

The primary reason to choose deterministic tracing is when you need exact call counts. Statistical profiling estimates frequency based on sampling, which may undercount short-lived functions that complete between samples. If you need to verify that an optimization actually reduced the number of function calls, or if you want to trace the complete call graph to understand caller-callee relationships, deterministic tracing is the right choice.

Deterministic tracing also excels at capturing functions that execute in microseconds. Such functions may not appear frequently enough in statistical samples, but deterministic tracing records every invocation regardless of duration.

## Quick start

This section provides the minimal steps needed to start profiling. For complete documentation, see the dedicated pages for each profiler.

### Statistical profiling

To profile a script, use the `profiling.sampling` module with the `run` command:

``` sh
python -m profiling.sampling run script.py
python -m profiling.sampling run -m mypackage.module
```

This runs the script under the profiler and prints a summary of where time was spent. For an interactive flame graph:

``` sh
python -m profiling.sampling run --flamegraph script.py
```

To profile an already-running process, use the `attach` command with the process ID:

``` sh
python -m profiling.sampling attach 1234
```

For custom settings, specify the sampling interval (in microseconds) and duration (in seconds):

``` sh
python -m profiling.sampling run -i 50 -d 30 script.py
```

### Deterministic profiling

To profile a script from the command line:

``` sh
python -m profiling.tracing script.py
```

To profile a piece of code programmatically:

``` python
import profiling.tracing
profiling.tracing.run('my_function()')
```

This executes the given code under the profiler and prints a summary showing exact function call counts and timing.

## Understanding profile output

Both profilers collect function-level statistics, though they present them in different formats. The sampling profiler offers multiple visualizations (flame graphs, heatmaps, Firefox Profiler, pstats tables), while the deterministic profiler produces pstats-compatible output. Regardless of format, the underlying concepts are the same.

Key profiling concepts:

**Direct time** (also called *self time* or *tottime*)  
Time spent executing code in the function itself, excluding time spent in functions it called. High direct time indicates the function contains expensive operations.

**Cumulative time** (also called *total time* or *cumtime*)  
Time spent in the function and all functions it called. This measures the total cost of calling a function, including its entire call subtree.

**Call count** (also called *ncalls* or *samples*)  
How many times the function was called (deterministic) or sampled (statistical). In deterministic profiling, this is exact. In statistical profiling, it represents the number of times the function appeared in a stack sample.

**Primitive calls**  
Calls that are not induced by recursion. When a function recurses, the total call count includes recursive invocations, but primitive calls counts only the initial entry. Displayed as `total/primitive` (for example, `3/1` means three total calls, one primitive).

**Caller/Callee relationships**  
Which functions called a given function (callers) and which functions it called (callees). Flame graphs visualize this as nested rectangles; pstats can display it via the `~pstats.Stats.print_callers` and `~pstats.Stats.print_callees` methods.

## Legacy compatibility

For backward compatibility, the `cProfile` module remains available as an alias to `profiling.tracing`. Existing code using `import cProfile` will continue to work without modification in all future Python versions.

<div class="deprecated">

3.15

The pure Python `profile` module is deprecated and will be removed in Python 3.17. Use `profiling.tracing` (or its alias `cProfile`) instead. See `profile` for migration guidance.

</div>

<div class="seealso">

`profiling.sampling`  
Statistical sampling profiler with flame graphs, heatmaps, and GIL analysis. Recommended for most users.

`profiling.tracing`  
Deterministic tracing profiler for exact call counts.

`pstats`  
Statistics analysis and formatting for profile data.

`timeit`  
Module for measuring execution time of small code snippets.

</div>

**Submodules**

<div class="toctree" data-maxdepth="1">

profiling.tracing.rst profiling.sampling.rst

</div>

# `!profiling.sampling` --- Statistical profiler

<div class="module" data-synopsis="Statistical sampling profiler for Python processes.">

profiling.sampling

</div>

<div class="versionadded">

3.15

</div>

**Source code:** `Lib/profiling/sampling/`

<div class="program">

profiling.sampling

</div>

------------------------------------------------------------------------

<img src="../../Lib/profiling/sampling/_assets/tachyon-logo.png" class="align-center" width="300" alt="Tachyon logo" />

The `!profiling.sampling` module, named **Tachyon**, provides statistical profiling of Python programs through periodic stack sampling. Tachyon can run scripts directly or attach to any running Python process without requiring code changes or restarts. Because sampling occurs externally to the target process, overhead is virtually zero, making Tachyon suitable for both development and production environments.

## What is statistical profiling?

Statistical profiling builds a picture of program behavior by periodically capturing snapshots of the call stack. Rather than instrumenting every function call and return as deterministic profilers do, Tachyon reads the call stack at regular intervals to record what code is currently running.

This approach rests on a simple principle: functions that consume significant CPU time will appear frequently in the collected samples. By gathering thousands of samples over a profiling session, Tachyon constructs an accurate statistical estimate of where time is spent. The more samples collected, the more precise this estimate becomes.

<div class="only">

html

The following interactive visualization demonstrates how sampling profiling works. Press **Play** to watch a Python program execute, and observe how the profiler periodically captures snapshots of the call stack. Adjust the **sample interval** to see how sampling frequency affects the results.

<div id="sampling-profiler-viz" class="sampling-profiler-viz"></div>

</div>

<div class="only">

not html

> [!NOTE]
> An interactive visualization of sampling profiling is available in the HTML version of this documentation.

</div>

### How time is estimated

The time values shown in Tachyon's output are **estimates derived from sample counts**, not direct measurements. Tachyon counts how many times each function appears in the collected samples, then multiplies by the sampling interval to estimate time.

For example, with a 10 kHz sampling rate over a 10-second profile, Tachyon collects approximately 100,000 samples. If a function appears in 5,000 samples (5% of total), Tachyon estimates it consumed 5% of the 10-second duration, or about 500 milliseconds. This is a statistical estimate, not a precise measurement.

The accuracy of these estimates depends on sample count. With 100,000 samples, a function showing 5% has a margin of error of roughly ±0.5%. With only 1,000 samples, the same 5% measurement could actually represent anywhere from 3% to 7% of real time.

This is why longer profiling durations and shorter sampling intervals produce more reliable results---they collect more samples. For most performance analysis, the default settings provide sufficient accuracy to identify bottlenecks and guide optimization efforts.

Because sampling is statistical, results will vary slightly between runs. A function showing 12% in one run might show 11% or 13% in the next. This is normal and expected. Focus on the overall pattern rather than exact percentages, and don't worry about small variations between runs.

### When to use a different approach

Statistical sampling is not ideal for every situation.

For very short scripts that complete in under one second, the profiler may not collect enough samples for reliable results. Use `profiling.tracing` instead, or run the script in a loop to extend profiling time.

When you need exact call counts, sampling cannot provide them. Sampling estimates frequency from snapshots, so if you need to know precisely how many times a function was called, use `profiling.tracing`.

When comparing two implementations where the difference might be only 1-2%, sampling noise can obscure real differences. Use `timeit` for micro-benchmarks or `profiling.tracing` for precise measurements.

The key difference from `profiling.tracing` is how measurement happens. A tracing profiler instruments your code, recording every function call and return. This provides exact call counts and precise timing but adds overhead to every function call. A sampling profiler, by contrast, observes the program from outside at fixed intervals without modifying its execution. Think of the difference like this: tracing is like having someone follow you and write down every step you take, while sampling is like taking photographs every second and inferring your path from those snapshots.

This external observation model is what makes sampling profiling practical for production use. The profiled program runs at full speed because there is no instrumentation code running inside it, and the target process is never stopped or paused during sampling---Tachyon reads the call stack directly from the process's memory while it continues to run. You can attach to a live server, collect data, and detach without the application ever knowing it was observed. The trade-off is that very short-lived functions may be missed if they happen to complete between samples.

Statistical profiling excels at answering the question, "Where is my program spending time?" It reveals hotspots and bottlenecks in production code where deterministic profiling overhead would be unacceptable. For exact call counts and complete call graphs, use `profiling.tracing` instead.

## Quick examples

Profile a script and see the results immediately:

``` sh
python -m profiling.sampling run script.py
```

Profile a module with arguments:

``` sh
python -m profiling.sampling run -m mypackage.module arg1 arg2
```

Generate an interactive flame graph:

``` sh
python -m profiling.sampling run --flamegraph -o profile.html script.py
```

Attach to a running process by PID:

``` sh
python -m profiling.sampling attach 12345
```

Print a single snapshot of a running process's stack:

``` sh
python -m profiling.sampling dump 12345
```

Use live mode for real-time monitoring (press `q` to quit):

``` sh
python -m profiling.sampling run --live script.py
```

Profile for 60 seconds with a faster sampling rate:

``` sh
python -m profiling.sampling run -d 60 -r 20khz script.py
```

Generate a line-by-line heatmap:

``` sh
python -m profiling.sampling run --heatmap script.py
```

Enable opcode-level profiling to see which bytecode instructions are executing:

``` sh
python -m profiling.sampling run --opcodes --flamegraph script.py
```

## Commands

Tachyon operates through several subcommands. `run` and `attach` collect samples over time; `dump` captures a single snapshot; `replay` converts binary profiles to other formats.

### The `run` command

The `run` command launches a Python script or module and profiles it from startup:

``` sh
python -m profiling.sampling run script.py
python -m profiling.sampling run -m mypackage.module
```

When profiling a script, the profiler starts the target in a subprocess, waits for it to initialize, then begins collecting samples. The `-m` flag indicates that the target should be run as a module (equivalent to `python -m`). Arguments after the target are passed through to the profiled program:

``` sh
python -m profiling.sampling run script.py --config settings.yaml
```

### The `attach` command

The `attach` command connects to an already-running Python process by its process ID:

``` sh
python -m profiling.sampling attach 12345
```

This command is particularly valuable for investigating performance issues in production systems. The target process requires no modification and need not be restarted. The profiler attaches, collects samples for the specified duration, then detaches and produces output.

``` sh
python -m profiling.sampling attach --live 12345
python -m profiling.sampling attach --flamegraph -d 30 -o profile.html 12345
```

On most systems, attaching to another process requires appropriate permissions. See `profiling-permissions` for platform-specific requirements.

### The `dump` command

The `dump` command prints a single snapshot of a running process's Python stack and exits, similar to a traceback:

``` sh
python -m profiling.sampling dump 12345
```

Unlike `attach`, `dump` does not run a sampling loop: it reads the stack once. This is useful for investigating hung or unresponsive processes, or for answering "what is this process doing right now?".

The output mirrors a traceback (most recent call last) and annotates each thread with its current state (main thread, has GIL, on CPU, waiting for GIL, has exception, or idle):

```
Stack dump for PID 12345, thread 140735 (main thread, has GIL, on CPU; most recent call last):
  File "server.py", line 28, in serve
    await handle_request(req)
  File "handler.py", line 91, in handle_request
    result = expensive_call(req)
```

When the target's source files are readable, `dump` prints the source line for each frame and highlights the executing expression.

Like `attach`, `dump` requires permission to read the target process's memory. See `profiling-permissions`.

The `dump` command supports the following options:

`-a`, `--all-threads`  
Dump every thread in the target process. Without this flag only the main thread is shown.

`--native`  
Include synthetic `<native>` frames marking transitions into C extensions or other non-Python code.

`--no-gc`  
Hide the synthetic `<GC>` frames that mark active garbage collection.

`--opcodes`  
Annotate each frame with the bytecode opcode the thread is currently executing (for example, `opcode=CALL_KW`). Useful for instruction-level investigation, including identifying specializations chosen by the adaptive interpreter.

`--async-aware`  
Reconstruct stacks across `await` boundaries. `dump` walks the task graph and emits one section per task, with `<task>` markers separating coroutines awaiting each other.

`--async-mode {running,all}`  
Controls which tasks are included when `--async-aware` is enabled. `running` shows only the task currently executing on each thread; `all` (the default for `dump`) also includes tasks suspended on a wait. `attach`'s default for this flag is `running`; `dump` defaults to `all` because a single snapshot is most useful when it shows the full task graph.

`--blocking`  
Pause every thread in the target while reading its stack and resume them after. Guarantees a fully consistent snapshot at the cost of briefly stopping the target. Without it, `dump` reads memory while the target keeps running, which is faster but can occasionally produce a torn stack.

### The `replay` command

The `replay` command converts binary profile files to other output formats:

``` sh
python -m profiling.sampling replay profile.bin
python -m profiling.sampling replay --flamegraph -o profile.html profile.bin
```

This command is useful when you have captured profiling data in binary format and want to analyze it later or convert it to a visualization format. Binary profiles can be replayed multiple times to different formats without re-profiling.

``` sh
# Convert binary to pstats (default, prints to stdout)
python -m profiling.sampling replay profile.bin

# Convert binary to flame graph
python -m profiling.sampling replay --flamegraph -o output.html profile.bin

# Convert binary to gecko format for Firefox Profiler
python -m profiling.sampling replay --gecko -o profile.json profile.bin

# Convert binary to heatmap
python -m profiling.sampling replay --heatmap -o my_heatmap profile.bin
```

### Profiling in production

The sampling profiler is designed for production use. It imposes no measurable overhead on the target process because it reads memory externally rather than instrumenting code. The target application continues running at full speed and is unaware it is being profiled.

When profiling production systems, keep these guidelines in mind:

Start with shorter durations (10-30 seconds) to get quick results, then extend if you need more statistical accuracy. By default, profiling runs until the target process completes, which is usually sufficient to identify major hotspots.

If possible, profile during representative load rather than peak traffic. Profiles collected during normal operation are easier to interpret than those collected during unusual spikes.

The profiler itself consumes some CPU on the machine where it runs (not on the target process). On the same machine, this is typically negligible. When profiling remote processes, network latency does not affect the target.

Results from production may differ from development due to different data sizes, concurrent load, or caching effects. This is expected and is often exactly what you want to capture.

### Platform requirements

The profiler reads the target process's memory to capture stack traces. This requires elevated permissions on most operating systems.

**Linux**

On Linux, the profiler uses `ptrace` or `process_vm_readv` to read the target process's memory. This typically requires one of:

- Running as root
- Having the `CAP_SYS_PTRACE` capability
- Adjusting the Yama ptrace scope: `/proc/sys/kernel/yama/ptrace_scope`

The default ptrace_scope of 1 restricts ptrace to parent processes only. To allow attaching to any process owned by the same user, set it to 0:

``` sh
echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
```

**macOS**

On macOS, the profiler uses `task_for_pid()` to access the target process. This requires one of:

- Running as root
- The profiler binary having the `com.apple.security.cs.debugger` entitlement
- System Integrity Protection (SIP) being disabled (not recommended)

**Windows**

On Windows, the profiler requires administrative privileges or the `SeDebugPrivilege` privilege to read another process's memory.

### Version compatibility

The profiler and target process must run the same Python minor version (for example, both Python 3.15). Attaching from Python 3.14 to a Python 3.15 process is not supported.

Additional restrictions apply to pre-release Python versions: if either the profiler or target is running a pre-release (alpha, beta, or release candidate), both must run the exact same version.

On free-threaded Python builds, the profiler cannot attach from a free-threaded build to a standard build, or vice versa.

## Sampling configuration

Before exploring the various output formats and visualization options, it is important to understand how to configure the sampling process itself. The profiler offers several options that control how frequently samples are collected, how long profiling runs, which threads are observed, and what additional context is captured in each sample.

The default configuration works well for most use cases:

| Option | Default |
|----|----|
| Default for `--sampling-rate` / `-r` | 1 kHz |
| Default for `--duration` / `-d` | Run to completion |
| Default for `--all-threads` / `-a` | Main thread only |
| Default for `--native` | No `<native>` frames (C code time attributed to caller) |
| Default for `--no-gc` | `<GC>` frames included when garbage collection is active |
| Default for `--mode` | Wall-clock mode (all samples recorded) |
| Default for `--realtime-stats` | Disabled |
| Default for `--subprocesses` | Disabled |
| Default for `--blocking` | Disabled (non-blocking sampling) |

### Sampling rate and duration

The two most fundamental parameters are the sampling rate and duration. Together, these determine how many samples will be collected during a profiling session.

The `--sampling-rate` option (`-r`) sets how frequently samples are collected. The default is 1 kHz (1,000 samples per second):

``` sh
python -m profiling.sampling run -r 20khz script.py
```

Higher rates capture more samples and provide finer-grained data at the cost of slightly higher profiler CPU usage. Lower rates reduce profiler overhead but may miss short-lived functions. For most applications, the default rate provides a good balance between accuracy and overhead.

The `--duration` option (`-d`) sets how long to profile in seconds. By default, profiling continues until the target process exits or is interrupted:

``` sh
python -m profiling.sampling run -d 60 script.py
```

Specifying a duration is useful when attaching to long-running processes or when you want to limit profiling to a specific time window. When profiling a script, the default behavior of running to completion is usually what you want.

### Thread selection

Python programs often use multiple threads, whether explicitly through the `threading` module or implicitly through libraries that manage thread pools.

By default, the profiler samples only the main thread. The `--all-threads` option (`-a`) enables sampling of all threads in the process:

``` sh
python -m profiling.sampling run -a script.py
```

Multi-thread profiling reveals how work is distributed across threads and can identify threads that are blocked or starved. Each thread's samples are combined in the output, with the ability to filter by thread in some formats. This option is particularly useful when investigating concurrency issues or when work is distributed across a thread pool.

### Blocking mode

By default, Tachyon reads the target process's memory without stopping it. This non-blocking approach is ideal for most profiling scenarios because it imposes virtually zero overhead on the target application: the profiled program runs at full speed and is unaware it is being observed.

However, non-blocking sampling can occasionally produce incomplete or inconsistent stack traces in applications with many generators or coroutines that rapidly switch between yield points, or in programs with very fast-changing call stacks where functions enter and exit between the start and end of a single stack read, resulting in reconstructed stacks that mix frames from different execution states or that never actually existed.

For these cases, the `--blocking` option stops the target process during each sample:

``` sh
python -m profiling.sampling run --blocking script.py
python -m profiling.sampling attach --blocking 12345
```

When blocking mode is enabled, the profiler suspends the target process, reads its stack, then resumes it. This guarantees that each captured stack represents a real, consistent snapshot of what the process was doing at that instant. The trade-off is that the target process runs slower because it is repeatedly paused.

> [!WARNING]
> Do not use very high sample rates (low `--interval` values) with blocking mode. Suspending and resuming a process takes time, and if the sampling interval is too short, the target will spend more time stopped than running. For blocking mode, intervals of 1000 microseconds (1 millisecond) or higher are recommended. The default 100 microsecond interval may cause noticeable slowdown in the target application.

Use blocking mode only when you observe inconsistent stacks in your profiles, particularly with generator-heavy or coroutine-heavy code. For most applications, the default non-blocking mode provides accurate results with zero impact on the target process.

### Special frames

The profiler can inject artificial frames into the captured stacks to provide additional context about what the interpreter is doing at the moment each sample is taken. These synthetic frames help distinguish different types of execution that would otherwise be invisible.

The `--native` option adds `<native>` frames to indicate when Python has called into C code (extension modules, built-in functions, or the interpreter itself):

``` sh
python -m profiling.sampling run --native script.py
```

These frames help distinguish time spent in Python code versus time spent in native libraries. Without this option, native code execution appears as time in the Python function that made the call. This is useful when optimizing code that makes heavy use of C extensions like NumPy or database drivers.

By default, the profiler includes `<GC>` frames when garbage collection is active. The `--no-gc` option suppresses these frames:

``` sh
python -m profiling.sampling run --no-gc script.py
```

GC frames help identify programs where garbage collection consumes significant time, which may indicate memory allocation patterns worth optimizing. If you see substantial time in `<GC>` frames, consider investigating object allocation rates or using object pooling.

### Opcode-aware profiling

The `--opcodes` option enables instruction-level profiling that captures which Python bytecode instructions are executing at each sample:

``` sh
python -m profiling.sampling run --opcodes --flamegraph script.py
```

This feature provides visibility into Python's bytecode execution, including adaptive specialization optimizations. When a generic instruction like `LOAD_ATTR` is specialized at runtime into a more efficient variant like `LOAD_ATTR_INSTANCE_VALUE`, the profiler shows both the specialized name and the base instruction.

Opcode information appears in several output formats:

- **Flame graphs**: Hovering over a frame displays a tooltip with a bytecode instruction breakdown, showing which opcodes consumed time in that function
- **Heatmap**: Expandable bytecode panels per source line show instruction breakdown with specialization percentages
- **Live mode**: An opcode panel shows instruction-level statistics for the selected function, accessible via keyboard navigation
- **Gecko format**: Opcode transitions are emitted as interval markers in the Firefox Profiler timeline

This level of detail is particularly useful for:

- Understanding the performance impact of Python's adaptive specialization
- Identifying hot bytecode instructions that might benefit from optimization
- Analyzing the effectiveness of different code patterns at the instruction level
- Debugging performance issues that occur at the bytecode level

The `--opcodes` option is compatible with `--live`, `--flamegraph`, `--heatmap`, and `--gecko` formats. It requires additional memory to store opcode information and may slightly reduce sampling performance, but provides unprecedented visibility into Python's execution model.

### Real-time statistics

The `--realtime-stats` option displays sampling rate statistics during profiling:

``` sh
python -m profiling.sampling run --realtime-stats script.py
```

This shows the actual achieved sampling rate, which may be lower than requested if the profiler cannot keep up. The statistics help verify that profiling is working correctly and that sufficient samples are being collected. See `sampling-efficiency` for details on interpreting these metrics.

### Subprocess profiling

The `--subprocesses` option enables automatic profiling of subprocesses spawned by the target:

``` sh
python -m profiling.sampling run --subprocesses script.py
python -m profiling.sampling attach --subprocesses 12345
```

When enabled, the profiler monitors the target process for child process creation. When a new Python child process is detected, a separate profiler instance is automatically spawned to profile it. This is useful for applications that use `multiprocessing`, `subprocess`, `concurrent.futures` with `~concurrent.futures.ProcessPoolExecutor`, or other process spawning mechanisms.

``` python
from concurrent.futures import ProcessPoolExecutor
import math

def compute_factorial(n):
    total = 0
    for i in range(50):
        total += math.factorial(n)
    return total

if __name__ == "__main__":
    numbers = [5000 + i * 100 for i in range(50)]
    with ProcessPoolExecutor(max_workers=4) as executor:
        results = list(executor.map(compute_factorial, numbers))
    print(f"Computed {len(results)} factorials")
```

``` sh
python -m profiling.sampling run --subprocesses --flamegraph worker_pool.py
```

This produces separate flame graphs for the main process and each worker process: `flamegraph_<main_pid>.html`, `flamegraph_<worker1_pid>.html`, and so on.

Each subprocess receives its own output file. The filename is derived from the specified output path (or the default) with the subprocess's process ID appended:

- If you specify `-o profile.html`, subprocesses produce `profile_12345.html`, `profile_12346.html`, and so on
- With default output, subprocesses produce files like `flamegraph_12345.html` or directories like `heatmap_12345`
- For pstats format (which defaults to stdout), subprocesses produce files like `profile_12345.pstats`

The subprocess profilers inherit most sampling options from the parent (sampling rate, duration, thread selection, native frames, GC frames, async-aware mode, and output format). All Python descendant processes are profiled recursively, including grandchildren and further descendants.

Subprocess detection works by periodically scanning for new descendants of the target process and checking whether each new process is a Python process by probing the process memory for Python runtime structures. Non-Python subprocesses (such as shell commands or external tools) are ignored.

There is a limit of 100 concurrent subprocess profilers to prevent resource exhaustion in programs that spawn many processes. If this limit is reached, additional subprocesses are not profiled and a warning is printed.

The `--subprocesses` option is incompatible with `--live` mode because live mode uses an interactive terminal interface that cannot accommodate multiple concurrent profiler displays.

### Sampling efficiency

Sampling efficiency metrics help assess the quality of the collected data. These metrics appear in the profiler's terminal output and in the flame graph sidebar.

**Sampling efficiency** is the percentage of sample attempts that succeeded. Each sample attempt reads the target process's call stack from memory. An attempt can fail if the process is in an inconsistent state at the moment of reading, such as during a context switch or while the interpreter is updating its internal structures. A low efficiency may indicate that the profiler could not keep up with the requested sampling rate, often due to system load or an overly aggressive interval setting.

**Missed samples** is the percentage of expected samples that were not collected. Based on the configured interval and duration, the profiler expects to collect a certain number of samples. Some samples may be missed if the profiler falls behind schedule, for example when the system is under heavy load. A small percentage of missed samples is normal and does not significantly affect the statistical accuracy of the profile.

Both metrics are informational. Even with some failed attempts or missed samples, the profile remains statistically valid as long as enough samples were collected. The profiler reports the actual number of samples captured, which you can use to judge whether the data is sufficient for your analysis.

## Profiling modes

The sampling profiler supports four modes that control which samples are recorded. The mode determines what the profile measures: total elapsed time, CPU execution time, time spent holding the global interpreter lock, or exception handling.

### Wall-clock mode

Wall-clock mode (`--mode``=wall`) captures all samples regardless of what the thread is doing. This is the default mode and provides a complete picture of where time passes during program execution:

``` sh
python -m profiling.sampling run --mode=wall script.py
```

In wall-clock mode, samples are recorded whether the thread is actively executing Python code, waiting for I/O, blocked on a lock, or sleeping. This makes wall-clock profiling ideal for understanding the overall time distribution in your program, including time spent waiting.

If your program spends significant time in I/O operations, network calls, or sleep, wall-clock mode will show these waits as time attributed to the calling function. This is often exactly what you want when optimizing end-to-end latency.

### CPU mode

CPU mode (`--mode``=cpu`) records samples only when the thread is actually executing on a CPU core:

``` sh
python -m profiling.sampling run --mode=cpu script.py
```

Samples taken while the thread is sleeping, blocked on I/O, or waiting for a lock are discarded. The resulting profile shows where CPU cycles are consumed, filtering out idle time.

CPU mode is useful when you want to focus on computational hotspots without being distracted by I/O waits. If your program alternates between computation and network calls, CPU mode reveals which computational sections are most expensive.

### Comparing wall-clock and CPU profiles

Running both wall-clock and CPU mode profiles can reveal whether a function's time is spent computing or waiting.

If a function appears prominently in both profiles, it is a true computational hotspot---actively using the CPU. Optimization should focus on algorithmic improvements or more efficient code.

If a function is high in wall-clock mode but low or absent in CPU mode, it is I/O-bound or waiting. The function spends most of its time waiting for network, disk, locks, or sleep. CPU optimization won't help here; consider async I/O, connection pooling, or reducing wait time instead.

``` python
import time

def do_sleep():
    time.sleep(2)

def do_compute():
    sum(i**2 for i in range(1000000))

if __name__ == "__main__":
    do_sleep()
    do_compute()
```

``` sh
python -m profiling.sampling run --mode=wall script.py  # do_sleep ~98%, do_compute ~1%
python -m profiling.sampling run --mode=cpu script.py   # do_sleep absent, do_compute dominates
```

### GIL mode

GIL mode (`--mode``=gil`) records samples only when the thread holds Python's global interpreter lock:

``` sh
python -m profiling.sampling run --mode=gil script.py
```

The GIL is held only while executing Python bytecode. When Python calls into C extensions, performs I/O operations, or executes native code, the GIL is typically released. This means GIL mode effectively measures time spent running Python code specifically, filtering out time in native libraries.

In multi-threaded programs, GIL mode reveals which code is preventing other threads from running Python bytecode. Since only one thread can hold the GIL at a time, functions that appear frequently in GIL mode profiles are monopolizing the interpreter.

GIL mode helps answer questions like "which functions are monopolizing the GIL?" and "why are my other threads starving?" It can also be useful in single-threaded programs to distinguish Python execution time from time spent in C extensions or I/O.

``` python
import hashlib

def hash_work():
    # C extension - releases GIL during computation
    for _ in range(200):
        hashlib.sha256(b"data" * 250000).hexdigest()

def python_work():
    # Pure Python - holds GIL during computation
    for _ in range(3):
        sum(i**2 for i in range(1000000))

if __name__ == "__main__":
    hash_work()
    python_work()
```

``` sh
python -m profiling.sampling run --mode=cpu script.py  # hash_work ~42%, python_work ~38%
python -m profiling.sampling run --mode=gil script.py  # hash_work ~5%, python_work ~60%
```

### Exception mode

Exception mode (`--mode=exception`) records samples only when a thread has an active exception:

``` sh
python -m profiling.sampling run --mode=exception script.py
```

Samples are recorded in two situations: when an exception is being propagated up the call stack (after `raise` but before being caught), or when code is executing inside an `except` block where exception information is still present in the thread state.

The following example illustrates which code regions are captured:

``` python
def example():
    try:
        raise ValueError("error")    # Captured: exception being raised
    except ValueError:
        process_error()              # Captured: inside except block
    finally:
        cleanup()                    # NOT captured: exception already handled

def example_propagating():
    try:
        try:
            raise ValueError("error")
        finally:
            cleanup()                # Captured: exception propagating through
    except ValueError:
        pass

def example_no_exception():
    try:
        do_work()
    finally:
        cleanup()                    # NOT captured: no exception involved
```

Note that `finally` blocks are only captured when an exception is actively propagating through them. Once an `except` block finishes executing, Python clears the exception information before running any subsequent `finally` block. Similarly, `finally` blocks that run during normal execution (when no exception was raised) are not captured because no exception state is present.

This mode is useful for understanding where your program spends time handling errors. Exception handling can be a significant source of overhead in code that uses exceptions for flow control (such as `StopIteration` in iterators) or in applications that process many error conditions (such as network servers handling connection failures).

Exception mode helps answer questions like "how much time is spent handling exceptions?" and "which exception handlers are the most expensive?" It can reveal hidden performance costs in code that catches and processes many exceptions, even when those exceptions are handled gracefully. For example, if a parsing library uses exceptions internally to signal format errors, this mode will capture time spent in those handlers even if the calling code never sees the exceptions.

## Output formats

The profiler produces output in several formats, each suited to different analysis workflows. The format is selected with a command-line flag, and output goes to stdout, a file, or a directory depending on the format.

### pstats format

The pstats format (`--pstats`) produces a text table similar to what deterministic profilers generate. This is the default output format:

``` sh
python -m profiling.sampling run script.py
python -m profiling.sampling run --pstats script.py
```

<figure class="align-center">
<img src="tachyon-pstats.png" style="width:100.0%" alt="Tachyon pstats terminal output" />
<figcaption>The pstats format displays profiling results in a color-coded table showing function hotspots, sample counts, and timing estimates.</figcaption>
</figure>

Output appears on stdout by default:

``` sh
Profile Stats (Mode: wall):
     nsamples  sample%    tottime (ms)  cumul%   cumtime (ms)  filename:lineno(function)
       234/892    11.7%       234.00     44.6%       892.00    server.py:145(handle_request)
       156/156     7.8%       156.00      7.8%       156.00    <built-in>:0(socket.recv)
        98/421     4.9%        98.00     21.1%       421.00    parser.py:67(parse_message)
```

The columns show sampling counts and estimated times:

- **nsamples**: Displayed as `direct/cumulative` (for example, `10/50`). Direct samples are when the function was at the top of the stack, actively executing. Cumulative samples are when the function appeared anywhere on the stack, including when it was waiting for functions it called. If a function shows `10/50`, it was directly executing in 10 samples and was on the call stack in 50 samples total.
- **sample%** and **cumul%**: Percentages of total samples for direct and cumulative counts respectively.
- **tottime** and **cumtime**: Estimated wall-clock time based on sample counts and the profiling duration. Time units are selected automatically based on the magnitude: seconds for large values, milliseconds for moderate values, or microseconds for small values.

The output includes a legend explaining each column and a summary of interesting functions that highlights:

- **Hot spots**: Functions with high direct/cumulative sample ratio (ratio close to 1.0). These functions spend most of their time executing their own code rather than waiting for callees. High ratios indicate where CPU time is actually consumed.
- **Indirect calls**: Functions with large differences between cumulative and direct samples. These are orchestration functions that delegate work to other functions. They appear frequently on the stack but rarely at the top.
- **Call magnification**: Functions where cumulative samples far exceed direct samples (high cumulative/direct multiplier). These are frequently-nested functions that appear deep in many call chains.

Use `--no-summary` to suppress both the legend and summary sections.

To save pstats output to a binary file instead of stdout:

``` sh
python -m profiling.sampling run -o profile.pstats script.py
```

The pstats format supports several options for controlling the display. The `--sort` option determines the column used for ordering results:

``` sh
python -m profiling.sampling run --sort=tottime script.py
python -m profiling.sampling run --sort=cumtime script.py
python -m profiling.sampling run --sort=nsamples script.py
```

The `--limit` option restricts output to the top N entries:

``` sh
python -m profiling.sampling run --limit=30 script.py
```

The `--no-summary` option suppresses the header summary that precedes the statistics table.

### Collapsed stacks format

Collapsed stacks format (`--collapsed`) produces one line per unique call stack, with a count of how many times that stack was sampled:

``` sh
python -m profiling.sampling run --collapsed script.py
```

The output looks like:

```
main;process_data;parse_json;decode_utf8 42
main;process_data;parse_json 156
main;handle_request;send_response 89
```

Each line contains semicolon-separated function names representing the call stack from bottom to top, followed by a space and the sample count. This format is designed for compatibility with external flame graph tools, particularly Brendan Gregg's `flamegraph.pl` script.

To generate a flame graph from collapsed stacks:

``` sh
python -m profiling.sampling run --collapsed script.py > stacks.txt
flamegraph.pl stacks.txt > profile.svg
```

The resulting SVG can be viewed in any web browser and provides an interactive visualization where you can click to zoom into specific call paths.

### Flame graph format

Flame graph format (`--flamegraph`) produces a self-contained HTML file with an interactive flame graph visualization:

``` sh
python -m profiling.sampling run --flamegraph script.py
python -m profiling.sampling run --flamegraph -o profile.html script.py
```

<figure class="align-center">
<img src="tachyon-flamegraph.png" style="width:100.0%" alt="Tachyon interactive flame graph" />
<figcaption>The flame graph visualization shows call stacks as nested rectangles, with width proportional to time spent. The sidebar displays runtime statistics, GIL metrics, and hotspot functions.</figcaption>
</figure>

<div class="only">

html

[Try the interactive example](../_static/tachyon-example-flamegraph.html)!

</div>

If no output file is specified, the profiler generates a filename based on the process ID (for example, `flamegraph.12345.html`).

The generated HTML file requires no external dependencies and can be opened directly in a web browser. The visualization displays call stacks as nested rectangles, with width proportional to time spent. Hovering over a rectangle shows details about that function including source code context, and clicking zooms into that portion of the call tree.

The flame graph interface includes:

- A sidebar showing profile summary, thread statistics, sampling efficiency metrics (see `sampling-efficiency`), and top hotspot functions
- Search functionality supporting both function name matching and `file.py:42` line patterns
- Per-thread filtering via dropdown
- Dark/light theme toggle (preference saved across sessions)
- SVG export for saving the current view

The thread statistics section shows runtime behavior metrics:

- **GIL Held**: percentage of samples where a thread held the global interpreter lock (actively running Python code)
- **GIL Released**: percentage of samples where no thread held the GIL
- **Waiting GIL**: percentage of samples where a thread was waiting to acquire the GIL
- **GC**: percentage of samples during garbage collection

These statistics help identify GIL contention and understand how time is distributed between Python execution, native code, and waiting.

Flame graphs are particularly effective for identifying deep call stacks and understanding the hierarchical structure of time consumption. Wide rectangles at the top indicate functions that consume significant time either directly or through their callees.

#### Differential flame graphs

Differential flame graphs compare two profiling runs to highlight where performance changed. This helps identify regressions introduced by code changes and validate that optimizations achieved their intended effect:

``` sh
# Capture baseline profile
python -m profiling.sampling run --binary -o baseline.bin script.py

# After modifying code, generate differential flamegraph
python -m profiling.sampling run --diff-flamegraph baseline.bin -o diff.html script.py
```

The visualization draws the current profile with frame widths showing current time consumption, then applies color to indicate how each function changed relative to the baseline.

**Color coding**:

- **Red**: Functions consuming more time (regressions). Lighter shades indicate modest increases, while darker shades show severe regressions.
- **Blue**: Functions consuming less time (improvements). Lighter shades for modest reductions, darker shades for significant speedups.
- **Gray**: Minimal or no change.
- **Purple**: New functions not present in the baseline.

Frame colors indicate changes in **direct time** (time when the function was at the top of the stack, actively executing), not cumulative time including callees. Hovering over a frame shows comparison details including baseline time, current time, and the percentage change.

Some call paths may disappear entirely between profiles. These are called **elided stacks** and occur when optimizations eliminate code paths or certain branches stop executing. If elided stacks are present, an elided toggle appears allowing you to switch between the main differential view and an elided-only view that shows just the removed paths (colored purple).

### Gecko format

Gecko format (`--gecko`) produces JSON output compatible with the Firefox Profiler:

``` sh
python -m profiling.sampling run --gecko script.py
python -m profiling.sampling run --gecko -o profile.json script.py
```

The [Firefox Profiler](https://profiler.firefox.com) is a sophisticated web-based tool originally built for profiling Firefox itself. It provides features beyond basic flame graphs, including a timeline view, call tree exploration, and marker visualization. See the [Firefox Profiler documentation](https://profiler.firefox.com/docs/#/) for detailed usage instructions.

To use the output, open the Firefox Profiler in your browser and load the JSON file. The profiler runs entirely client-side, so your profiling data never leaves your machine.

Gecko format automatically collects additional metadata about GIL state and CPU activity, enabling analysis features specific to Python's threading model. The profiler emits interval markers that appear as colored bands in the Firefox Profiler timeline:

- **GIL markers**: show when threads hold or release the global interpreter lock
- **CPU markers**: show when threads are executing on CPU versus idle
- **Code type markers**: distinguish Python code from native (C extension) code
- **GC markers**: indicate garbage collection activity

For this reason, the `--mode` option is not available with Gecko format; all relevant data is captured automatically.

<figure class="align-center">
<img src="tachyon-gecko-calltree.png" style="width:100.0%" alt="Firefox Profiler Call Tree view" />
<figcaption>The Call Tree view shows the complete call hierarchy with sample counts and percentages. The sidebar displays detailed statistics for the selected function including running time and sample distribution.</figcaption>
</figure>

<figure class="align-center">
<img src="tachyon-gecko-flamegraph.png" style="width:100.0%" alt="Firefox Profiler Flame Graph view" />
<figcaption>The Flame Graph visualization shows call stacks as nested rectangles. Functions names are visible in the call hierarchy.</figcaption>
</figure>

<figure class="align-center">
<img src="tachyon-gecko-opcodes.png" style="width:100.0%" alt="Firefox Profiler Marker Chart with opcodes" />
<figcaption>The Marker Chart displays interval markers including CPU state, GIL status, and opcodes. With <code>--opcodes</code> enabled, bytecode instructions like <code>BINARY_OP_ADD_FLOAT</code>, <code>CALL_PY_EXACT_ARGS</code>, and <code>CALL_LIST_APPEND</code> appear as markers showing execution over time.</figcaption>
</figure>

### Heatmap format

Heatmap format (`--heatmap`) generates an interactive HTML visualization showing sample counts at the source line level:

``` sh
python -m profiling.sampling run --heatmap script.py
python -m profiling.sampling run --heatmap -o my_heatmap script.py
```

<figure class="align-center">
<img src="tachyon-heatmap.png" style="width:100.0%" alt="Tachyon heatmap visualization" />
<figcaption>The heatmap overlays sample counts directly on your source code. Lines are color-coded from cool (few samples) to hot (many samples). Navigation buttons (▲▼) let you jump between callers and callees.</figcaption>
</figure>

Unlike other formats that produce a single file, heatmap output creates a directory containing HTML files for each profiled source file. If no output path is specified, the directory is named `heatmap_PID`.

The heatmap visualization displays your source code with a color gradient indicating how many samples were collected at each line. Hot lines (many samples) appear in warm colors, while cold lines (few or no samples) appear in cool colors. This view helps pinpoint exactly which lines of code are responsible for time consumption.

The heatmap interface provides several interactive features:

- **Coloring modes**: toggle between "Self Time" (direct execution) and "Total Time" (cumulative, including time in called functions)
- **Cold code filtering**: show all lines or only lines with samples
- **Call graph navigation**: each line shows navigation buttons (▲ for callers, ▼ for callees) that let you trace execution paths through your code. When multiple functions called or were called from a line, a menu appears showing all options with their sample counts.
- **Scroll minimap**: a vertical overview showing the heat distribution across the entire file
- **Hierarchical index**: files organized by type (stdlib, site-packages, project) with aggregate sample counts per folder
- **Dark/light theme**: toggle with preference saved across sessions
- **Line linking**: click line numbers to create shareable URLs

When opcode-level profiling is enabled with `--opcodes`, each hot line can be expanded to show which bytecode instructions consumed time:

<figure class="align-center">
<img src="tachyon-heatmap-with-opcodes.png" style="width:100.0%" alt="Heatmap with expanded bytecode panel" />
<figcaption>Expanding a hot line reveals the bytecode instructions executed, including specialized variants. The panel shows sample counts per instruction and the overall specialization percentage for the line.</figcaption>
</figure>

<div class="only">

html

[Try the interactive example](../_static/tachyon-example-heatmap.html)!

</div>

Heatmaps are especially useful when you know which file contains a performance issue but need to identify the specific lines. Many developers prefer this format because it maps directly to their source code, making it easy to read and navigate. For smaller scripts and focused analysis, heatmaps provide an intuitive view that shows exactly where time is spent without requiring interpretation of hierarchical visualizations.

### Binary format

Binary format (`--binary`) produces a compact binary file for efficient storage of profiling data:

``` sh
python -m profiling.sampling run --binary -o profile.bin script.py
python -m profiling.sampling attach --binary -o profile.bin 12345
```

The `--compression` option controls data compression:

- `auto` (default): Use zstd compression if available, otherwise no compression
- `zstd`: Force zstd compression (requires `compression.zstd` support)
- `none`: Disable compression

``` sh
python -m profiling.sampling run --binary --compression=zstd -o profile.bin script.py
```

To analyze binary profiles, use the `replay command <replay-command>` to convert them to other formats like flame graphs or pstats output.

## Record and replay workflow

The binary format combined with the replay command enables a record-and-replay workflow that separates data capture from analysis. Rather than generating visualizations during profiling, you capture raw data to a compact binary file and convert it to different formats later.

This approach has three main benefits:

- Sampling runs faster because the work of building data structures for visualization is deferred until replay.
- A single binary capture can be converted to multiple output formats without re-profiling: pstats for a quick overview, flame graph for visual exploration, heatmap for line-level detail.
- Binary files are compact and easy to share with colleagues who can convert them to their preferred format.

A typical workflow:

``` sh
# Capture profile in production or during tests
python -m profiling.sampling attach --binary -o profile.bin 12345

# Later, analyze with different formats
python -m profiling.sampling replay profile.bin
python -m profiling.sampling replay --flamegraph -o profile.html profile.bin
python -m profiling.sampling replay --heatmap -o heatmap profile.bin
```

## Live mode

Live mode (`--live`) provides a terminal-based real-time view of profiling data, similar to the `top` command for system processes:

``` sh
python -m profiling.sampling run --live script.py
python -m profiling.sampling attach --live 12345
```

<div class="only">

not latex

<figure class="align-center">
<img src="tachyon-live-mode-2.gif" style="width:100.0%" alt="Tachyon live mode showing all threads" />
</figure>

Live mode displays real-time profiling statistics, showing combined data from multiple threads in a multi-threaded application.

</div>

The display updates continuously as new samples arrive, showing the current hottest functions. This mode requires the `curses` module, which is available on Unix-like systems but not on Windows. The terminal must be at least 60 columns wide and 12 lines tall; larger terminals display more columns.

The header displays the top 3 hottest functions, sampling efficiency metrics, and thread status statistics (GIL held percentage, CPU usage, GC time). The main table shows function statistics with the currently sorted column indicated by an arrow (▼).

When `--opcodes` is enabled, an additional opcode panel appears below the main table, showing instruction-level statistics for the currently selected function. This panel displays which bytecode instructions are executing most frequently, including specialized variants and their base opcodes.

<div class="only">

not latex

<figure class="align-center">
<img src="tachyon-live-mode-1.gif" style="width:100.0%" alt="Tachyon live mode with opcode panel" />
</figure>

Live mode with `--opcodes` enabled shows an opcode panel with a bytecode instruction breakdown for the selected function.

</div>

### Keyboard commands

Within live mode, keyboard commands control the display:

`q`  
Quit the profiler and return to the shell.

`s` / `S`  
Cycle through sort orders forward/backward (sample count, percentage, total time, cumulative percentage, cumulative time).

`p`  
Pause or resume display updates. Sampling continues in the background while the display is paused, so you can freeze the view to examine results without stopping data collection.

`r`  
Reset all statistics and start fresh. This is disabled after profiling finishes to prevent accidental data loss.

`/`  
Enter filter mode to search for functions by name. The filter uses case-insensitive substring matching against the filename and function name. Type a pattern and press Enter to apply, or Escape to cancel. Glob patterns and regular expressions are not supported.

`c`  
Clear the current filter and show all functions again.

`t`  
Toggle between viewing all threads combined or per-thread statistics. In per-thread mode, a thread counter (for example, `1/4`) appears showing your position among the available threads.

`←` `→` or `↑` `↓`  
In per-thread view, navigate between threads. Navigation wraps around from the last thread to the first and vice versa.

`+` / `-`  
Increase or decrease the display refresh rate. The range is 0.05 seconds (20 Hz, very responsive) to 1.0 second (1 Hz, lower overhead). Faster refresh rates use more CPU. The default is 0.1 seconds (10 Hz).

`x`  
Toggle trend indicators that show whether functions are becoming hotter or cooler over time. When enabled, increasing metrics appear in green and decreasing metrics appear in red, comparing each update to the previous one.

`h` or `?`  
Show the help screen with all available commands.

`j` / `k` (or `Up` / `Down`)  
Navigate through opcode entries in the opcode panel (when `--opcodes` is enabled). These keys scroll through the instruction-level statistics for the currently selected function.

When profiling finishes (duration expires or target process exits), the display shows a "PROFILING COMPLETE" banner and freezes the final results. You can still navigate, sort, and filter the results before pressing `q` to exit.

Live mode is incompatible with output format options (`--collapsed`, `--flamegraph`, and so on) because it uses an interactive terminal interface rather than producing file output.

## Async-aware profiling

For programs using `asyncio`, the profiler offers async-aware mode (`--async-aware`) that reconstructs call stacks based on the task structure rather than the raw Python frames:

``` sh
python -m profiling.sampling run --async-aware async_script.py
```

Standard profiling of async code can be confusing because the physical call stack often shows event loop internals rather than the logical flow of your coroutines. Async-aware mode addresses this by tracking which task is running and presenting stacks that reflect the `await` chain.

``` python
import asyncio

async def fetch(url):
    await asyncio.sleep(0.1)
    return url

async def main():
    for _ in range(50):
        await asyncio.gather(fetch("a"), fetch("b"), fetch("c"))

if __name__ == "__main__":
    asyncio.run(main())
```

``` sh
python -m profiling.sampling run --async-aware --flamegraph -o out.html script.py
```

> [!NOTE]
> Async-aware profiling requires the target process to have the `asyncio` module loaded. If you profile a script before it imports asyncio, async-aware mode will not be able to capture task information.

### Async modes

The `--async-mode` option controls which tasks appear in the profile:

``` sh
python -m profiling.sampling run --async-aware --async-mode=running async_script.py
python -m profiling.sampling run --async-aware --async-mode=all async_script.py
```

With `--async-mode``=running` (the default), only the task currently executing on the CPU is profiled. This shows where your program is actively spending time and is the typical choice for performance analysis.

With `--async-mode``=all`, tasks that are suspended (awaiting I/O, locks, or other tasks) are also included. This mode is useful for understanding what your program is waiting on, but produces larger profiles since every suspended task appears in each sample.

### Task markers and stack reconstruction

In async-aware profiles, you will see `<task>` frames that mark boundaries between asyncio tasks. These are synthetic frames inserted by the profiler to show the task structure. The task name appears as the function name in these frames.

When a task awaits another task, the profiler reconstructs the logical call chain by following the `await` relationships. Only "leaf" tasks (tasks that no other task is currently awaiting) generate their own stack entries. Tasks being awaited by other tasks appear as part of their awaiter's stack instead.

If a task has multiple awaiters (a diamond pattern in the task graph), the profiler deterministically selects one parent and annotates the task marker with the number of parents, for example `MyTask (2 parents)`. This indicates that alternate execution paths exist but are not shown in this particular stack.

### Option restrictions

Async-aware mode uses a different stack reconstruction mechanism and is incompatible with: `--native`, `--no-gc`, `--all-threads`, and `--mode``=cpu` or `--mode``=gil`.

## Command-line interface

<div class="program">

profiling.sampling

</div>

The complete command-line interface for reference.

### Global options

<div class="option">

run

Run and profile a Python script or module.

</div>

<div class="option">

attach

Attach to and profile a running process by PID.

</div>

<div class="option">

dump

Print a single one-shot snapshot of a running process's Python stack.

</div>

<div class="option">

replay

Convert a binary profile file to another output format.

</div>

### Dump options

The following options apply to the `dump` subcommand:

<div class="option">

`-a, --all-threads`  

Dump all threads in the target process instead of just the main thread.

</div>

<div class="option">

`--native`  

Include `<native>` frames for non-Python code.

</div>

<div class="option">

`--no-gc`  

Exclude `<GC>` frames for active garbage collection.

</div>

<div class="option">

`--opcodes`  

Show bytecode opcode names when available.

</div>

<div class="option">

`--async-aware`  

Reconstruct the stack across `await` boundaries for asyncio applications.

</div>

<div class="option">

`--async-mode <mode>`  

Async stack mode: `running` (only the running task) or `all` (all tasks including waiting). Defaults to `all` for `dump`. Requires `--async-aware`.

</div>

<div class="option">

`--blocking`  

Pause all threads in the target process while reading the stack.

</div>

### Sampling options

<div class="option">

`-r <rate>, --sampling-rate <rate>`  

Sampling rate (for example, `10000`, `10khz`, `10k`). Default: `1khz`.

</div>

<div class="option">

`-d <seconds>, --duration <seconds>`  

Profiling duration in seconds. Default: run to completion.

</div>

<div class="option">

`-a, --all-threads`  

Sample all threads, not just the main thread.

</div>

<div class="option">

`--realtime-stats`  

Display sampling statistics during profiling.

</div>

<div class="option">

`--native`  

Include `<native>` frames for non-Python code.

</div>

<div class="option">

`--no-gc`  

Exclude `<GC>` frames for garbage collection.

</div>

<div class="option">

`--async-aware`  

Enable async-aware profiling for asyncio programs.

</div>

<div class="option">

`--opcodes`  

Gather bytecode opcode information for instruction-level profiling. Shows which bytecode instructions are executing, including specializations. Compatible with `--live`, `--flamegraph`, `--heatmap`, and `--gecko` formats only.

</div>

<div class="option">

`--subprocesses`  

Also profile subprocesses. Each subprocess gets its own profiler instance and output file. Incompatible with `--live`.

</div>

<div class="option">

`--blocking`  

Pause the target process during each sample. This ensures consistent stack traces at the cost of slowing down the target. Use with longer intervals (1000 µs or higher) to minimize impact. See `blocking-mode` for details.

</div>

### Mode options

<div class="option">

`--mode <mode>`  

Sampling mode: `wall` (default), `cpu`, `gil`, or `exception`. The `cpu`, `gil`, and `exception` modes are incompatible with `--async-aware`.

</div>

<div class="option">

`--async-mode <mode>`  

Async profiling mode: `running` (default) or `all`. Requires `--async-aware`.

</div>

### Output options

<div class="option">

`--pstats`  

Generate pstats statistics. This is the default. When written to stdout, the output is a text table; with `-o`, it is a binary pstats file.

</div>

<div class="option">

`--collapsed`  

Generate collapsed stack format for external flame graph tools.

</div>

<div class="option">

`--flamegraph`  

Generate self-contained HTML flame graph.

</div>

<div class="option">

`--diff-flamegraph <baseline.bin>`  

Generate differential flamegraph comparing to a baseline binary profile.

</div>

<div class="option">

`--gecko`  

Generate Gecko JSON format for Firefox Profiler.

</div>

<div class="option">

`--heatmap`  

Generate HTML heatmap with line-level sample counts.

</div>

<div class="option">

`--binary`  

Generate high-performance binary format for later conversion with the `replay` command.

</div>

<div class="option">

`--compression <type>`  

Compression for binary format: `auto` (use zstd if available, default), `zstd`, or `none`.

</div>

<div class="option">

`-o <path>, --output <path>`  

Output file or directory path. Default behavior varies by format: `--pstats` prints a text table to stdout, while `-o` writes a binary pstats file. Other formats generate a file named `<format>_<PID>.<ext>` (for example, `flamegraph_12345.html`). `--heatmap` creates a directory named `heatmap_<PID>`.

</div>

<div class="option">

`--browser`  

Automatically open HTML output (`--flamegraph` and `--heatmap`) in your default web browser after generation. When profiling with `--subprocesses`, only the main process opens the browser; subprocess outputs are never auto-opened.

</div>

### pstats display options

These options apply only to pstats format output.

<div class="option">

`--sort <key>`  

Sort order: `nsamples`, `tottime`, `cumtime`, `sample-pct`, `cumul-pct`, `nsamples-cumul`, or `name`. Default: `nsamples`.

</div>

<div class="option">

`-l <count>, --limit <count>`  

Maximum number of entries to display. Default: 15.

</div>

<div class="option">

`--no-summary`  

Omit the Legend and Summary of Interesting Functions sections from output.

</div>

### Run command options

<div class="option">

`-m, --module`  

Treat the target as a module name rather than a script path.

</div>

<div class="option">

`--live`  

Start interactive terminal interface instead of batch profiling.

</div>

<div class="seealso">

`profiling`  
Overview of Python profiling tools and guidance on choosing a profiler.

`profiling.tracing`  
Deterministic tracing profiler for exact call counts and timing.

`pstats`  
Statistics analysis for profile data.

[Firefox Profiler](https://profiler.firefox.com)  
Web-based profiler that accepts Gecko format output. See the [documentation](https://profiler.firefox.com/docs/#/) for usage details.

[FlameGraph](https://github.com/brendangregg/FlameGraph)  
Tools for generating flame graphs from collapsed stack format.

</div>

# `!profiling.tracing` --- Deterministic profiler

<div class="module" data-synopsis="Deterministic tracing profiler for Python programs.">

profiling.tracing

</div>

<div class="module" data-synopsis="Alias for profiling.tracing (backward compatibility)." data-noindex="">

cProfile

</div>

<div class="versionadded">

3.15

</div>

**Source code:** `Lib/profiling/tracing/`

------------------------------------------------------------------------

The `!profiling.tracing` module provides deterministic profiling of Python programs. It monitors every function call, function return, and exception event, recording precise timing for each. This approach provides exact call counts and complete visibility into program execution, making it ideal for development and testing scenarios.

> [!NOTE]
> This module is also available as `cProfile` for backward compatibility. The `cProfile` name will continue to work in all future Python versions. Use whichever import style suits your codebase:
>
>     # Preferred (new style)
>     import profiling.tracing
>     profiling.tracing.run('my_function()')
>
>     # Also works (backward compatible)
>     import cProfile
>     cProfile.run('my_function()')

## What is deterministic profiling?

`Deterministic profiling` captures every function call, function return, and exception event during program execution. The profiler measures the precise time intervals between these events, providing exact statistics about how the program behaves.

In contrast to `statistical profiling <profiling-sampling>`, which samples the call stack periodically to estimate where time is spent, deterministic profiling records every event. This means you get exact call counts rather than statistical approximations. The trade-off is that instrumenting every event introduces overhead that can slow down program execution.

Python's interpreted nature makes deterministic profiling practical. The interpreter already dispatches events for function calls and returns, so the profiler can hook into this mechanism without requiring code modification. The overhead tends to be moderate relative to the inherent cost of interpretation, making deterministic profiling suitable for most development workflows.

Deterministic profiling helps answer questions like:

- How many times was this function called?
- What is the complete call graph of my program?
- Which functions are called by a particular function?
- Are there unexpected function calls happening?

Call count statistics can identify bugs (surprising counts) and inline expansion opportunities (high call counts). Internal time statistics reveal "hot loops" that warrant optimization. Cumulative time statistics help identify algorithmic inefficiencies. The handling of cumulative times in this profiler allows direct comparison of recursive and iterative implementations.

## Command-line interface

<div class="program">

profiling.tracing

</div>

The `!profiling.tracing` module can be invoked as a script to profile another script or module:

```
python -m profiling.tracing [-o output_file] [-s sort_order] (-m module | script.py)
```

This runs the specified script or module under the profiler and prints the results to standard output (or saves them to a file).

<div class="option">

`-o <output_file>`  

Write the profile results to a file instead of standard output. The output file can be read by the `pstats` module for later analysis.

</div>

<div class="option">

`-s <sort_order>`  

Sort the output by the specified key. This accepts any of the sort keys recognized by `pstats.Stats.sort_stats`, such as `cumulative`, `time`, `calls`, or `name`. This option only applies when `-o <profiling.tracing -o>` is not specified.

</div>

<div class="option">

`-m <module>`  

Profile a module instead of a script. The module is located using the standard import mechanism.

<div class="versionadded">

3.7 The `-m` option for `cProfile`.

</div>

<div class="versionadded">

3.8 The `-m` option for `profile`.

</div>

</div>

## Programmatic usage examples

For more control over profiling, use the module's functions and classes directly.

### Basic profiling

The simplest approach uses the `!run` function:

    import profiling.tracing
    profiling.tracing.run('my_function()')

This profiles the given code string and prints a summary to standard output. To save results for later analysis:

    profiling.tracing.run('my_function()', 'output.prof')

### Using the `!Profile` class

The `!Profile` class provides fine-grained control:

    import profiling.tracing
    import pstats
    from io import StringIO

    pr = profiling.tracing.Profile()
    pr.enable()
    # ... code to profile ...
    pr.disable()

    # Print results
    s = StringIO()
    ps = pstats.Stats(pr, stream=s).sort_stats(pstats.SortKey.CUMULATIVE)
    ps.print_stats()
    print(s.getvalue())

The `!Profile` class also works as a context manager:

    import profiling.tracing

    with profiling.tracing.Profile() as pr:
        # ... code to profile ...

    pr.print_stats()

## Module reference

<div class="currentmodule">

profiling.tracing

</div>

<div class="function">

run(command, filename=None, sort=-1)

Profile execution of a command and print or save the results.

This function executes the *command* string using `exec` in the `__main__` module's namespace:

    exec(command, __main__.__dict__, __main__.__dict__)

If *filename* is not provided, the function creates a `pstats.Stats` instance and prints a summary to standard output. If *filename* is provided, the raw profile data is saved to that file for later analysis with `pstats`.

The *sort* argument specifies the sort order for printed output, accepting any value recognized by `pstats.Stats.sort_stats`.

</div>

<div class="function">

runctx(command, globals, locals, filename=None, sort=-1)

Profile execution of a command with explicit namespaces.

Like `run`, but executes the command with the specified *globals* and *locals* mappings instead of using the `__main__` module's namespace:

    exec(command, globals, locals)

</div>

<div class="Profile(timer=None, timeunit=0.0, subcalls=True, builtins=True)">

A profiler object that collects execution statistics.

The optional *timer* argument specifies a custom timing function. If not provided, the profiler uses a platform-appropriate default timer. When supplying a custom timer, it must return a single number representing the current time. If the timer returns integers, use *timeunit* to specify the duration of one time unit (for example, `0.001` for milliseconds).

The *subcalls* argument controls whether the profiler tracks call relationships between functions. The *builtins* argument controls whether built-in functions are profiled.

<div class="versionchanged">

3.8 Added context manager support.

</div>

<div class="method">

enable()

Start collecting profiling data.

</div>

<div class="method">

disable()

Stop collecting profiling data.

</div>

<div class="method">

create_stats()

Stop collecting data and record the results internally as the current profile.

</div>

<div class="method">

print_stats(sort=-1)

Create a `pstats.Stats` object from the current profile and print the results to standard output.

The *sort* argument specifies the sorting order. It accepts a single key or a tuple of keys for multi-level sorting, using the same values as `pstats.Stats.sort_stats`.

<div class="versionadded">

3.13 Support for a tuple of sort keys.

</div>

</div>

<div class="method">

dump_stats(filename)

Write the current profile data to *filename*. The file can be read by `pstats.Stats` for later analysis.

</div>

<div class="method">

run(cmd)

Profile the command string via `exec`.

</div>

<div class="method">

runctx(cmd, globals, locals)

Profile the command string via `exec` with the specified namespaces.

</div>

<div class="method">

runcall(func, /, *args,*\*kwargs)

Profile a function call. Returns whatever *func* returns:

    result = pr.runcall(my_function, arg1, arg2, keyword=value)

</div>

</div>

> [!NOTE]
> Profiling requires that the profiled code returns normally. If the interpreter terminates (for example, via `sys.exit`) during profiling, no results will be available.

## Using a custom timer

The `Profile` class accepts a custom timing function, allowing you to measure different aspects of execution such as wall-clock time or CPU time. Pass the timing function to the constructor:

    pr = profiling.tracing.Profile(my_timer_function)

The timer function must return a single number representing the current time. If it returns integers, also specify *timeunit* to indicate the duration of one unit:

    # Timer returns time in milliseconds
    pr = profiling.tracing.Profile(my_ms_timer, 0.001)

For best performance, the timer function should be as fast as possible. The profiler calls it frequently, so timer overhead directly affects profiling overhead.

The `time` module provides several functions suitable for use as custom timers:

- `time.perf_counter` for high-resolution wall-clock time
- `time.process_time` for CPU time (excluding sleep)
- `time.monotonic` for monotonic clock time

## Limitations

Deterministic profiling has inherent limitations related to timing accuracy.

The underlying timer typically has a resolution of about one millisecond. Measurements cannot be more accurate than this resolution. With enough measurements, timing errors tend to average out, but individual measurements may be imprecise.

There is also latency between when an event occurs and when the profiler captures the timestamp. Similarly, there is latency after reading the timestamp before user code resumes. Functions called frequently accumulate this latency, which can make them appear slower than they actually are. This error is typically less than one clock tick per call but can become significant for functions called many times.

The `!profiling.tracing` module (and its `cProfile` alias) is implemented as a C extension with low overhead, so these timing issues are less pronounced than with the deprecated pure Python `profile` module.

<div class="seealso">

`profiling`  
Overview of Python profiling tools and guidance on choosing a profiler.

`profiling.sampling`  
Statistical sampling profiler for production use.

`pstats`  
Statistics analysis and formatting for profile data.

`profile`  
Deprecated pure Python profiler (includes calibration documentation).

</div>

# `!pstats` --- Statistics for profilers

<div class="module" data-synopsis="Statistics object for analyzing profiler output.">

pstats

</div>

**Source code:** `Lib/pstats.py`

------------------------------------------------------------------------

The `!pstats` module provides tools for reading, manipulating, and displaying profiling statistics generated by Python's profilers. It reads output from both `profiling.tracing` (deterministic profiler) and `profiling.sampling` (statistical profiler).

## Reading and displaying profile data

The `Stats` class is the primary interface for working with profile data. It can read statistics from files or directly from a `~profiling.tracing.Profile` object.

Load statistics from a file and print a basic report:

    import pstats

    p = pstats.Stats('profile_output.prof')
    p.print_stats()

The `Stats` object provides methods for sorting and filtering the data before printing. For example, to see the ten functions with the highest cumulative time:

    from pstats import SortKey

    p = pstats.Stats('profile_output.prof')
    p.sort_stats(SortKey.CUMULATIVE).print_stats(10)

### Working with statistics

The `Stats` class supports method chaining, making it convenient to perform multiple operations:

    p = pstats.Stats('restats')
    p.strip_dirs().sort_stats(-1).print_stats()

The `~Stats.strip_dirs` method removes directory paths from filenames, making the output more compact. The `~Stats.sort_stats` method accepts various keys to control the sort order.

Different sort keys highlight different aspects of performance:

    from pstats import SortKey

    # Functions that consume the most cumulative time
    p.sort_stats(SortKey.CUMULATIVE).print_stats(10)

    # Functions that consume the most time in their own code
    p.sort_stats(SortKey.TIME).print_stats(10)

    # Functions sorted by name
    p.sort_stats(SortKey.NAME).print_stats()

### Filtering output

The `~Stats.print_stats` method accepts restrictions that filter which functions are displayed. Restrictions can be integers (limiting the count), floats between 0 and 1 (selecting a percentage), or strings (matching function names via regular expression).

Print only the top 10%:

    p.print_stats(.1)

Print only functions whose names contain "init":

    p.print_stats('init')

Combine restrictions (they apply sequentially):

    # Top 10%, then only those containing "init"
    p.print_stats(.1, 'init')

    # Functions in files matching "foo:", limited to top 50%
    p.sort_stats(SortKey.FILENAME).print_stats('foo:', .5)

### Analyzing call relationships

The `~Stats.print_callers` method shows which functions called each displayed function:

    p.print_callers()

The `~Stats.print_callees` method shows the opposite relationship, listing which functions each displayed function called:

    p.print_callees()

Both methods accept the same restriction arguments as `~Stats.print_stats`.

### Combining multiple profiles

Statistics from multiple profiling runs can be combined into a single `Stats` object:

    # Load multiple files at once
    p = pstats.Stats('run1.prof', 'run2.prof', 'run3.prof')

    # Or add files incrementally
    p = pstats.Stats('run1.prof')
    p.add('run2.prof')
    p.add('run3.prof')

When files are combined, statistics for identical functions (same file, line, and name) are accumulated, giving an aggregate view across all profiling runs.

## The `!Stats` class

<div class="Stats(*filenames_or_profile, stream=sys.stdout)">

Create a statistics object from profile data.

The arguments can be filenames (strings or path-like objects) or `~profiling.tracing.Profile` objects. If multiple sources are provided, their statistics are combined.

The *stream* argument specifies where output from `print_stats` and related methods is written. It defaults to `sys.stdout`.

The profile data format is specific to the Python version that created it. There is no compatibility guarantee between Python versions or between different profilers.

<div class="method">

strip_dirs()

Remove leading path information from all filenames.

This method modifies the object in place and returns it for method chaining. After stripping, the statistics are considered to be in random order.

If stripping causes two functions to become indistinguishable (same filename, line number, and function name), their statistics are combined into a single entry.

</div>

<div class="method">

add(\*filenames)

Add profiling data from additional files.

The files must have been created by the same profiler type. Statistics for identical functions are accumulated.

</div>

<div class="method">

dump_stats(filename)

Save the current statistics to a file.

The file is created if it does not exist and overwritten if it does. The saved data can be loaded by creating a new `Stats` object.

</div>

<div class="method">

sort_stats(\*keys)

Sort the statistics according to the specified criteria.

Each key can be a string or a `SortKey` enum member. When multiple keys are provided, later keys break ties in earlier keys.

Using `SortKey` enum members is preferred over strings as it provides better error checking:

    from pstats import SortKey
    p.sort_stats(SortKey.CUMULATIVE)

Valid sort keys:

| String         | Enum                 | Meaning              |
|----------------|----------------------|----------------------|
| `'calls'`      | `SortKey.CALLS`      | call count           |
| `'cumulative'` | `SortKey.CUMULATIVE` | cumulative time      |
| `'cumtime'`    | N/A                  | cumulative time      |
| `'file'`       | N/A                  | file name            |
| `'filename'`   | `SortKey.FILENAME`   | file name            |
| `'module'`     | N/A                  | file name            |
| `'ncalls'`     | N/A                  | call count           |
| `'pcalls'`     | `SortKey.PCALLS`     | primitive call count |
| `'line'`       | `SortKey.LINE`       | line number          |
| `'name'`       | `SortKey.NAME`       | function name        |
| `'nfl'`        | `SortKey.NFL`        | name/file/line       |
| `'stdname'`    | `SortKey.STDNAME`    | standard name        |
| `'time'`       | `SortKey.TIME`       | internal time        |
| `'tottime'`    | N/A                  | internal time        |

All sorts on statistics are in descending order (most time consuming first), while name, file, and line number sorts are ascending (alphabetical).

The difference between `SortKey.NFL` and `SortKey.STDNAME` is that NFL sorts line numbers numerically while STDNAME sorts them as strings. `sort_stats(SortKey.NFL)` is equivalent to `sort_stats(SortKey.NAME, SortKey.FILENAME, SortKey.LINE)`.

For backward compatibility, the numeric arguments `-1`, `0`, `1`, and `2` are also accepted, meaning `'stdname'`, `'calls'`, `'time'`, and `'cumulative'` respectively.

<div class="versionadded">

3.7 The `SortKey` enum.

</div>

</div>

<div class="method">

reverse_order()

Reverse the current sort order.

By default, the sort direction is chosen appropriately for the sort key (descending for time-based keys, ascending for name-based keys). This method inverts that choice.

</div>

<div class="method">

print_stats(\*restrictions)

Print a report of the profiling statistics.

The output includes a header line summarizing the data, followed by a table of function statistics sorted according to the last `sort_stats` call.

Restrictions filter the output. Each restriction is either:

- An integer: limits output to that many entries
- A float between 0.0 and 1.0: selects that fraction of entries
- A string: matches function names via regular expression

Restrictions are applied sequentially. For example:

    print_stats(.1, 'foo:')

First limits to the top 10%, then filters to functions matching 'foo:'.

</div>

<div class="method">

print_callers(\*restrictions)

Print the callers of each function in the statistics.

For each function in the filtered results, shows which functions called it and how often.

With `profiling.tracing` (or `cProfile`), each caller line shows three numbers: the number of calls from that caller, and the total and cumulative times for those specific calls.

Accepts the same restriction arguments as `print_stats`.

</div>

<div class="method">

print_callees(\*restrictions)

Print the functions called by each function in the statistics.

This is the inverse of `print_callers`, showing which functions each listed function called.

Accepts the same restriction arguments as `print_stats`.

</div>

<div class="method">

get_stats_profile()

Return a `StatsProfile` object containing the statistics.

The returned object provides programmatic access to the profile data, with function names mapped to `FunctionProfile` objects containing timing and call count information.

<div class="versionadded">

3.9

</div>

</div>

</div>

<div class="SortKey">

An enumeration of valid sort keys for `Stats.sort_stats`.

<div class="attribute">

CALLS

Sort by call count.

</div>

<div class="attribute">

CUMULATIVE

Sort by cumulative time.

</div>

<div class="attribute">

FILENAME

Sort by file name.

</div>

<div class="attribute">

LINE

Sort by line number.

</div>

<div class="attribute">

NAME

Sort by function name.

</div>

<div class="attribute">

NFL

Sort by name, then file, then line number (numeric line sort).

</div>

<div class="attribute">

PCALLS

Sort by primitive (non-recursive) call count.

</div>

<div class="attribute">

STDNAME

Sort by standard name (string-based line sort).

</div>

<div class="attribute">

TIME

Sort by internal time (time in function excluding subcalls).

</div>

</div>

## Command-line interface

The `!pstats` module can be invoked as a script to interactively browse profile data:

    python -m pstats profile_output.prof

This opens a line-oriented interface (built on `cmd`) for examining the statistics. Type `help` at the prompt for available commands.

<div class="seealso">

`profiling`  
Overview of Python profiling tools.

`profiling.tracing`  
Deterministic tracing profiler.

`profiling.sampling`  
Statistical sampling profiler.

</div>

# `!pty` --- Pseudo-terminal utilities

<div class="module" data-synopsis="Pseudo-Terminal Handling for Unix.">

pty

</div>

**Source code:** `Lib/pty.py`

------------------------------------------------------------------------

The `!pty` module defines operations for handling the pseudo-terminal concept: starting another process and being able to write to and read from its controlling terminal programmatically.

<div class="availability">

Unix.

</div>

Pseudo-terminal handling is highly platform dependent. This code is mainly tested on Linux, FreeBSD, and macOS (it is supposed to work on other POSIX platforms but it's not been thoroughly tested).

The `!pty` module defines the following functions:

<div class="function">

fork()

Fork. Connect the child's controlling terminal to a pseudo-terminal. Return value is `(pid, fd)`. Note that the child gets *pid* 0, and the *fd* is *invalid*. The parent's return value is the *pid* of the child, and *fd* is a file descriptor connected to the child's controlling terminal (and also to the child's standard input and output).

The returned file descriptor *fd* is `non-inheritable <fd_inheritance>`.

> [!WARNING]
> On macOS the use of this function is unsafe when mixed with using higher-level system APIs, and that includes using `urllib.request`.

<div class="versionchanged">

3.15 The returned file descriptor is now made non-inheritable.

</div>

</div>

<div class="function">

openpty()

Open a new pseudo-terminal pair, using `os.openpty` if possible, or emulation code for generic Unix systems. Return a pair of file descriptors `(master, slave)`, for the master and the slave end, respectively.

</div>

<div class="function">

spawn(argv\[, master_read\[, stdin_read\]\])

Spawn a process, and connect its controlling terminal with the current process's standard io. This is often used to baffle programs which insist on reading from the controlling terminal. It is expected that the process spawned behind the pty will eventually terminate, and when it does *spawn* will return.

A loop copies STDIN of the current process to the child and data received from the child to STDOUT of the current process. It is not signaled to the child if STDIN of the current process closes down.

The functions *master_read* and *stdin_read* are passed a file descriptor which they should read from, and they should always return a byte string. In order to force spawn to return before the child process exits an empty byte array should be returned to signal end of file.

The default implementation for both functions will read and return up to 1024 bytes each time the function is called. The *master_read* callback is passed the pseudoterminal’s master file descriptor to read output from the child process, and *stdin_read* is passed file descriptor 0, to read from the parent process's standard input.

Returning an empty byte string from either callback is interpreted as an end-of-file (EOF) condition, and that callback will not be called after that. If *stdin_read* signals EOF the controlling terminal can no longer communicate with the parent process OR the child process. Unless the child process will quit without any input, *spawn* will then loop forever. If *master_read* signals EOF the same behavior results (on linux at least).

Return the exit status value from `os.waitpid` on the child process.

`os.waitstatus_to_exitcode` can be used to convert the exit status into an exit code.

<div class="audit-event">

pty.spawn argv pty.spawn

</div>

<div class="versionchanged">

3.4 `spawn` now returns the status value from `os.waitpid` on the child process.

</div>

</div>

## Example

The following program acts like the Unix command `script(1)`, using a pseudo-terminal to record all input and output of a terminal session in a "typescript". :

    import argparse
    import os
    import pty
    import sys
    import time

    parser = argparse.ArgumentParser()
    parser.add_argument('-a', dest='append', action='store_true')
    parser.add_argument('-p', dest='use_python', action='store_true')
    parser.add_argument('filename', nargs='?', default='typescript')
    options = parser.parse_args()

    shell = sys.executable if options.use_python else os.environ.get('SHELL', 'sh')
    filename = options.filename
    mode = 'ab' if options.append else 'wb'

    with open(filename, mode) as script:
        def read(fd):
            data = os.read(fd, 1024)
            script.write(data)
            return data

        print('Script started, file is', filename)
        script.write(('Script started on %s\n' % time.asctime()).encode())

        pty.spawn(shell, read)

        script.write(('Script done on %s\n' % time.asctime()).encode())
        print('Script done, file is', filename)

# `!pwd` --- The password database

<div class="module" data-synopsis="The password database (getpwnam() and friends).">

pwd

</div>

------------------------------------------------------------------------

This module provides access to the Unix user account and password database. It is available on all Unix versions.

<div class="availability">

Unix, not WASI, not iOS.

</div>

Password database entries are reported as a tuple-like object, whose attributes correspond to the members of the `passwd` structure (Attribute field below, see `<pwd.h>`):

| Index | Attribute   | Meaning                     |
|-------|-------------|-----------------------------|
| 0     | `pw_name`   | Login name                  |
| 1     | `pw_passwd` | Optional encrypted password |
| 2     | `pw_uid`    | Numerical user ID           |
| 3     | `pw_gid`    | Numerical group ID          |
| 4     | `pw_gecos`  | User name or comment field  |
| 5     | `pw_dir`    | User home directory         |
| 6     | `pw_shell`  | User command interpreter    |

The uid and gid items are integers, all others are strings. `KeyError` is raised if the entry asked for cannot be found.

> [!NOTE]
> In traditional Unix the field `pw_passwd` usually contains a password encrypted with a DES derived algorithm. However most modern unices use a so-called *shadow password* system. On those unices the *pw_passwd* field only contains an asterisk (`'*'`) or the letter `'x'` where the encrypted password is stored in a file `/etc/shadow` which is not world readable. Whether the *pw_passwd* field contains anything useful is system-dependent.

It defines the following items:

<div class="function">

getpwuid(uid)

Return the password database entry for the given numeric user ID.

</div>

<div class="function">

getpwnam(name)

Return the password database entry for the given user name.

</div>

<div class="function">

getpwall()

Return a list of all available password database entries, in arbitrary order.

</div>

<div class="seealso">

Module `grp`  
An interface to the group database, similar to this.

</div>

# `!py_compile` --- Compile Python source files

<div class="module" data-synopsis="Generate byte-code files from Python source files.">

py_compile

</div>

**Source code:** `Lib/py_compile.py`

<div class="index">

pair: file; byte-code

</div>

------------------------------------------------------------------------

The `!py_compile` module provides a function to generate a byte-code file from a source file, and another function used when the module source file is invoked as a script.

Though not often needed, this function can be useful when installing modules for shared use, especially if some of the users may not have permission to write the byte-code cache files in the directory containing the source code.

<div class="exception">

PyCompileError

Exception raised when an error occurs while attempting to compile the file.

</div>

<div class="function">

compile(file, cfile=None, dfile=None, doraise=False, optimize=-1, invalidation_mode=PycInvalidationMode.TIMESTAMP, quiet=0)

Compile a source file to byte-code and write out the byte-code cache file. The source code is loaded from the file named *file*. The byte-code is written to *cfile*, which defaults to the `3147`/`488` path, ending in `.pyc`. For example, if *file* is `/foo/bar/baz.py` *cfile* will default to `/foo/bar/__pycache__/baz.cpython-32.pyc` for Python 3.2. If *dfile* is specified, it is used instead of *file* as the name of the source file from which source lines are obtained for display in exception tracebacks. If *doraise* is true, a `PyCompileError` is raised when an error is encountered while compiling *file*. If *doraise* is false (the default), an error string is written to `sys.stderr`, but no exception is raised. This function returns the path to byte-compiled file, i.e. whatever *cfile* value was used.

The *doraise* and *quiet* arguments determine how errors are handled while compiling file. If *quiet* is 0 or 1, and *doraise* is false, the default behaviour is enabled: an error string is written to `sys.stderr`, and the function returns `None` instead of a path. If *doraise* is true, a `PyCompileError` is raised instead. However if *quiet* is 2, no message is written, and *doraise* has no effect.

If the path that *cfile* becomes (either explicitly specified or computed) is a symlink or non-regular file, `FileExistsError` will be raised. This is to act as a warning that import will turn those paths into regular files if it is allowed to write byte-compiled files to those paths. This is a side-effect of import using file renaming to place the final byte-compiled file into place to prevent concurrent file writing issues.

*optimize* controls the optimization level and is passed to the built-in `compile` function. The default of `-1` selects the optimization level of the current interpreter.

*invalidation_mode* should be a member of the `PycInvalidationMode` enum and controls how the generated bytecode cache is invalidated at runtime. The default is `PycInvalidationMode.CHECKED_HASH` if the `SOURCE_DATE_EPOCH` environment variable is set, otherwise the default is `PycInvalidationMode.TIMESTAMP`.

<div class="versionchanged">

3.2 Changed default value of *cfile* to be [PEP 3147](http://www.python.org/dev/peps/pep-3147/ "PEP 3147")-compliant. Previous default was *file* + `'c'` (`'o'` if optimization was enabled). Also added the *optimize* parameter.

</div>

<div class="versionchanged">

3.4 Changed code to use `importlib` for the byte-code cache file writing. This means file creation/writing semantics now match what `importlib` does, e.g. permissions, write-and-move semantics, etc. Also added the caveat that `FileExistsError` is raised if *cfile* is a symlink or non-regular file.

</div>

<div class="versionchanged">

3.7 The *invalidation_mode* parameter was added as specified in `552`. If the `SOURCE_DATE_EPOCH` environment variable is set, *invalidation_mode* will be forced to `PycInvalidationMode.CHECKED_HASH`.

</div>

<div class="versionchanged">

3.7.2 The `SOURCE_DATE_EPOCH` environment variable no longer overrides the value of the *invalidation_mode* argument, and determines its default value instead.

</div>

<div class="versionchanged">

3.8 The *quiet* parameter was added.

</div>

</div>

<div class="PycInvalidationMode">

An enumeration of possible methods the interpreter can use to determine whether a bytecode file is up to date with a source file. The `.pyc` file indicates the desired invalidation mode in its header. See `pyc-invalidation` for more information on how Python invalidates `.pyc` files at runtime.

<div class="versionadded">

3.7

</div>

<div class="attribute">

TIMESTAMP

The `.pyc` file includes the timestamp and size of the source file, which Python will compare against the metadata of the source file at runtime to determine if the `.pyc` file needs to be regenerated.

</div>

<div class="attribute">

CHECKED_HASH

The `.pyc` file includes a hash of the source file content, which Python will compare against the source at runtime to determine if the `.pyc` file needs to be regenerated.

</div>

<div class="attribute">

UNCHECKED_HASH

Like `CHECKED_HASH`, the `.pyc` file includes a hash of the source file content. However, Python will at runtime assume the `.pyc` file is up to date and not validate the `.pyc` against the source file at all.

This option is useful when the `.pycs` are kept up to date by some system external to Python like a build system.

</div>

</div>

## Command-Line Interface

This module can be invoked as a script to compile several source files. The files named in *filenames* are compiled and the resulting bytecode is cached in the normal manner. This program does not search a directory structure to locate source files; it only compiles files named explicitly. The exit status is nonzero if one of the files could not be compiled.

<div class="program">

python -m py_compile

</div>

<div class="option">

\<file\> ... \<fileN\> -

Positional arguments are files to compile. If `-` is the only parameter, the list of files is taken from standard input.

</div>

<div class="option">

`-q, --quiet`  

Suppress errors output.

</div>

<div class="versionchanged">

3.2 Added support for `-`.

</div>

<div class="versionchanged">

3.10 Added support for `-q`.

</div>

<div class="seealso">

Module `compileall`  
Utilities to compile all Python source files in a directory tree.

</div>

# `!pyclbr` --- Python module browser support

<div class="module" data-synopsis="Supports information extraction for a Python module browser.">

pyclbr

</div>

**Source code:** `Lib/pyclbr.py`

------------------------------------------------------------------------

The `!pyclbr` module provides limited information about the functions, classes, and methods defined in a Python-coded module. The information is sufficient to implement a module browser. The information is extracted from the Python source code rather than by importing the module, so this module is safe to use with untrusted code. This restriction makes it impossible to use this module with modules not implemented in Python, including all standard and optional extension modules.

<div class="function">

readmodule(module, path=None)

Return a dictionary mapping module-level class names to class descriptors. If possible, descriptors for imported base classes are included. Parameter *module* is a string with the name of the module to read; it may be the name of a module within a package. If given, *path* is a sequence of directory paths prepended to `sys.path`, which is used to locate the module source code.

This function is the original interface and is only kept for back compatibility. It returns a filtered version of the following.

</div>

<div class="function">

readmodule_ex(module, path=None)

Return a dictionary-based tree containing a function or class descriptors for each function and class defined in the module with a `def` or `class` statement. The returned dictionary maps module-level function and class names to their descriptors. Nested objects are entered into the children dictionary of their parent. As with readmodule, *module* names the module to be read and *path* is prepended to sys.path. If the module being read is a package, the returned dictionary has a key `'__path__'` whose value is a list containing the package search path.

</div>

<div class="versionadded">

3.7 Descriptors for nested definitions. They are accessed through the new children attribute. Each has a new parent attribute.

</div>

The descriptors returned by these functions are instances of Function and Class classes. Users are not expected to create instances of these classes.

## Function Objects

<div class="Function">

Class `!Function` instances describe functions defined by def statements. They have the following attributes:

<div class="attribute">

file

Name of the file in which the function is defined.

</div>

<div class="attribute">

module

The name of the module defining the function described.

</div>

<div class="attribute">

name

The name of the function.

</div>

<div class="attribute">

lineno

The line number in the file where the definition starts.

</div>

<div class="attribute">

parent

For top-level functions, `None`. For nested functions, the parent.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

children

A `dictionary <dict>` mapping names to descriptors for nested functions and classes.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

is_async

`True` for functions that are defined with the `async <async def>` prefix, `False` otherwise.

<div class="versionadded">

3.10

</div>

</div>

</div>

## Class Objects

<div class="Class">

Class `!Class` instances describe classes defined by class statements. They have the same attributes as `Functions <Function>` and two more.

<div class="attribute">

file

Name of the file in which the class is defined.

</div>

<div class="attribute">

module

The name of the module defining the class described.

</div>

<div class="attribute">

name

The name of the class.

</div>

<div class="attribute">

lineno

The line number in the file where the definition starts.

</div>

<div class="attribute">

parent

For top-level classes, `None`. For nested classes, the parent.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

children

A dictionary mapping names to descriptors for nested functions and classes.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

super

A list of `!Class` objects which describe the immediate base classes of the class being described. Classes which are named as superclasses but which are not discoverable by `readmodule_ex` are listed as a string with the class name instead of as `!Class` objects.

</div>

<div class="attribute">

methods

A `dictionary <dict>` mapping method names to line numbers. This can be derived from the newer `children` dictionary, but remains for back-compatibility.

</div>

</div>

# `!pydoc` --- Documentation generator and online help system

<div class="module" data-synopsis="Documentation generator and online help system.">

pydoc

</div>

**Source code:** `Lib/pydoc.py`

<div class="index">

single: documentation; generation single: documentation; online single: help; online

</div>

------------------------------------------------------------------------

The `!pydoc` module automatically generates documentation from Python modules. The documentation can be presented as pages of text on the console, served to a web browser, or saved to HTML files.

For modules, classes, functions and methods, the displayed documentation is derived from the docstring (i.e. the `~definition.__doc__` attribute) of the object, and recursively of its documentable members. If there is no docstring, `!pydoc` tries to obtain a description from the block of comment lines just above the definition of the class, function or method in the source file, or at the top of the module (see `inspect.getcomments`).

The built-in function `help` invokes the online help system in the interactive interpreter, which uses `!pydoc` to generate its documentation as text on the console. The same text documentation can also be viewed from outside the Python interpreter by running `pydoc` as a script at the operating system's command prompt. For example, running :

    python -m pydoc sys

at a shell prompt will display documentation on the `sys` module, in a style similar to the manual pages shown by the Unix `man` command. The argument to `pydoc` can be the name of a function, module, or package, or a dotted reference to a class, method, or function within a module or module in a package. If the argument to `pydoc` looks like a path (that is, it contains the path separator for your operating system, such as a slash in Unix), and refers to an existing Python source file, then documentation is produced for that file.

> [!NOTE]
> In order to find objects and their documentation, `!pydoc` imports the module(s) to be documented. Therefore, any code on module level will be executed on that occasion. Use an `if __name__ == '__main__':` guard to only execute code when a file is invoked as a script and not just imported.

When printing output to the console, `pydoc` attempts to paginate the output for easier reading. If either the `MANPAGER` or the `PAGER` environment variable is set, `pydoc` will use its value as a pagination program. When both are set, `MANPAGER` is used.

Specifying a `-w` flag before the argument will cause HTML documentation to be written out to a file in the current directory, instead of displaying text on the console.

Specifying a `-k` flag before the argument will search the synopsis lines of all available modules for the keyword given as the argument, again in a manner similar to the Unix `man` command. The synopsis line of a module is the first line of its documentation string.

You can also use `pydoc` to start an HTTP server on the local machine that will serve documentation to visiting web browsers. `python -m pydoc -p 1234` will start a HTTP server on port 1234, allowing you to browse the documentation at `http://localhost:1234/` in your preferred web browser. Specifying `0` as the port number will select an arbitrary unused port.

> [!WARNING]
> The `!pydoc` HTTP server is intended for local use during development and is not suitable for production use.

`python -m pydoc -n <hostname>` will start the server listening at the given hostname. By default the hostname is 'localhost' but if you want the server to be reached from other machines, you may want to change the host name that the server responds to. During development this is especially useful if you want to run pydoc from within a container.

`python -m pydoc -b` will start the server and additionally open a web browser to a module index page. Each served page has a navigation bar at the top where you can *Get* help on an individual item, *Search* all modules with a keyword in their synopsis line, and go to the *Module index*, *Topics* and *Keywords* pages.

When `pydoc` generates documentation, it uses the current environment and path to locate modules. Thus, invoking `pydoc spam` documents precisely the version of the module you would get if you started the Python interpreter and typed `import spam`.

Module docs for core modules are assumed to reside in `https://docs.python.org/X.Y/library/` where `X` and `Y` are the major and minor version numbers of the Python interpreter. This can be overridden by setting the `!PYTHONDOCS` environment variable to a different URL or to a local directory containing the Library Reference Manual pages.

<div class="versionchanged">

3.2 Added the `-b` option.

</div>

<div class="versionchanged">

3.3 The `-g` command line option was removed.

</div>

<div class="versionchanged">

3.4 `!pydoc` now uses `inspect.signature` rather than `inspect.getfullargspec` to extract signature information from callables.

</div>

<div class="versionchanged">

3.7 Added the `-n` option.

</div>

# `!xml.parsers.expat` --- Fast XML parsing using Expat

<div class="module" data-synopsis="An interface to the Expat non-validating XML parser.">

xml.parsers.expat

</div>

------------------------------------------------------------------------

> Many of the attributes of the XMLParser objects are callbacks. Since signature information must be presented, these are described using the method directive. Since they are attributes which are set by client code, in-text references to these attributes should be marked using the :member: role.

> [!NOTE]
> If you need to parse untrusted or unauthenticated data, see `xml-security`.

<div class="index">

single: Expat

</div>

The `!xml.parsers.expat` module is a Python interface to the Expat non-validating XML parser. The module provides a single extension type, `xmlparser`, that represents the current state of an XML parser. After an `xmlparser` object has been created, various attributes of the object can be set to handler functions. When an XML document is then fed to the parser, the handler functions are called for the character data and markup in the XML document.

<div class="index">

pair: module; pyexpat

</div>

This module uses the `pyexpat` module to provide access to the Expat parser. Direct use of the `pyexpat` module is deprecated.

This module provides the following exception, type object and data items:

<div class="exception">

ExpatError

The exception raised when Expat reports an error. See section `expaterror-objects` for more information on interpreting Expat errors.

</div>

<div class="exception">

error

Alias for `ExpatError`.

</div>

<div class="data">

XMLParserType

The type of the return values from the `ParserCreate` function.

</div>

<div class="data">

EXPAT_VERSION

The version string of the Expat library loaded by the interpreter, like `'expat_2.8.4'`.

</div>

<div class="data">

VERSION_INFO

A named tuple containing the three components of the Expat library version that was used for building the module: *major*, *minor*, and *micro*. All values are integers. The components can also be accessed by name, so `xml.parsers.expat.VERSION_INFO[0]` is equivalent to `xml.parsers.expat.VERSION_INFO.major` and so on. This may be different from the Expat library actually used at runtime, which is available as `version_info`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

version_info

A named tuple containing the version of the Expat library loaded by the interpreter, with the same fields as `VERSION_INFO`.

<div class="versionchanged">

3.16 It is now a named tuple.

</div>

</div>

<div class="data">

features

The list of the features with which the loaded Expat library was compiled, as `(name, value)` pairs. The value is only meaningful for features which have one, like `'XML_CONTEXT_BYTES'` or the default protection limits `'XML_BLAP_ACT_THRES'` and `'XML_AT_MAX_AMP'`; for other features, like `'XML_DTD'` and `'XML_NS'`, the value is `0` and only the presence of the name is significant.

</div>

The `!xml.parsers.expat` module contains two functions:

<div class="function">

ErrorString(errno)

Returns an explanatory string for a given error number *errno*.

</div>

<div class="function">

ParserCreate(encoding=None, namespace_separator=None, intern=None)

Creates and returns a new `xmlparser` object. *encoding*[^1], if specified, must be a string naming the encoding used by the XML data. If it is given it will override the implicit or explicit encoding of the document.

<div class="impl-detail">

Expat natively understands and processes UTF-8, UTF-16, UTF-16BE, UTF-16LE, ISO-8859-1, and US-ASCII. For other encodings (including aliases like Latin1 and ASCII) it falls back to Python. It supports most of 8-bit encodings and many multi-byte encodings like Shift_JIS, although only BMP characters (`U+0000-U+FFFF`) are supported with non-native encodings (this restriction is also applied to aliases like UTF8). These restrictions only apply if *encoding* is not given.

<div class="versionchanged">

3.16 Added support for multi-byte encodings.

</div>

</div>

<div id="xmlparser-non-root">

Parsers created through `!ParserCreate` are called "root" parsers, in the sense that they do not have any parent parser attached. Non-root parsers are created by `parser.ExternalEntityParserCreate
<xmlparser.ExternalEntityParserCreate>`.

</div>

Expat can optionally do XML namespace processing for you, enabled by providing a value for *namespace_separator*. The value must be a one-character string; a `ValueError` will be raised if the string has an illegal length (`None` is considered the same as omission). When namespace processing is enabled, element type names and attribute names that belong to a namespace will be expanded. The element name passed to the element handlers `StartElementHandler` and `EndElementHandler` will be the concatenation of the namespace URI, the namespace separator character, and the local part of the name. If the namespace separator is a zero byte (`chr(0)`) then the namespace URI and the local part will be concatenated without any separator.

For example, if *namespace_separator* is set to a space character (`' '`) and the following document is parsed:

``` xml
<?xml version="1.0"?>
<root xmlns    = "http://default-namespace.org/"
      xmlns:py = "http://www.python.org/ns/">
  <py:elem1 />
  <elem2 xmlns="" />
</root>
```

`StartElementHandler` will receive the following strings for each element:

    http://default-namespace.org/ root
    http://www.python.org/ns/ elem1
    elem2

*intern*, if given, must be a dictionary. It is used to intern the names of elements and attributes, and is available as the `~xmlparser.intern` attribute. By default a new empty dictionary is created for every parser.

Due to limitations in the `Expat` library used by `pyexpat`, the `xmlparser` instance returned can only be used to parse a single XML document. Call `ParserCreate` for each document to provide unique parser instances.

</div>

<div class="seealso">

[The Expat XML Parser](http://www.libexpat.org/)  
Home page of the Expat project.

</div>

## XMLParser Objects

`xmlparser` objects have the following methods:

<div class="method">

xmlparser.Parse(data\[, isfinal\])

Parses the contents of *data*, calling the appropriate handler functions to process the parsed data. *data* can be a `bytes-like object` or a string. If it is a string, the encoding declaration in the XML data is ignored, and the data is parsed as already decoded text. *isfinal* must be true on the final call to this method; it allows the parsing of a single file in fragments, not the submission of multiple files. *data* can be empty at any time.

</div>

<div class="method">

xmlparser.ParseFile(file)

Parse XML data reading from the object *file*. *file* only needs to provide the `read(nbytes)` method, which returns bytes, and an empty bytes object when there's no more data. Text files are not supported; use `Parse` for data which is already decoded.

</div>

<div class="method">

xmlparser.SetBase(base)

Sets the base to be used for resolving relative URIs in system identifiers in declarations. Resolving relative identifiers is left to the application: this value will be passed through as the *base* argument to the `ExternalEntityRefHandler`, `NotationDeclHandler`, and `UnparsedEntityDeclHandler` functions.

</div>

<div class="method">

xmlparser.GetBase()

Returns a string containing the base set by a previous call to `SetBase`, or `None` if `SetBase` hasn't been called.

</div>

<div class="method">

xmlparser.GetSpecifiedAttributeCount()

Return the index just past the attributes given in the start tag. Attributes defaulted from the DTD follow the specified ones, so attributes at lower indices in the list passed to `StartElementHandler` were given in the start tag. Each attribute takes two items in that list, its name and its value. Only meaningful inside a `StartElementHandler` call, and only if `ordered_attributes` is true.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

xmlparser.GetInputContext()

Returns the input data which generated the current event as a `bytes` object. The data is in the encoding of the entity which contains the text. It extends to the end of the currently buffered input, therefore it can contain also the data of the following events, and if the event was generated by a large amount of text, not all of it may be available. When called while an event handler is not active, the return value is `None`.

</div>

<div class="method">

xmlparser.ExternalEntityParserCreate(context\[, encoding\])

Create a "child" parser which can be used to parse an external parsed entity referred to by content parsed by the parent parser. The *context* parameter should be the string passed to the `ExternalEntityRefHandler` handler function, described below. The child parser is created with the `ordered_attributes` and `specified_attributes` set to the values of this parser.

</div>

<div class="method">

xmlparser.SetParamEntityParsing(flag)

Control parsing of parameter entities (including the external DTD subset). Possible *flag* values are `XML_PARAM_ENTITY_PARSING_NEVER`, `XML_PARAM_ENTITY_PARSING_UNLESS_STANDALONE` and `XML_PARAM_ENTITY_PARSING_ALWAYS`. Return true if setting the flag was successful.

</div>

<div class="method">

xmlparser.UseForeignDTD(\[flag\])

Calling this with a true value for *flag* (the default) will cause Expat to call the `ExternalEntityRefHandler` with `None` for all arguments to allow an alternate DTD to be loaded. If the document does not contain a document type declaration, the `ExternalEntityRefHandler` will still be called, but the `StartDoctypeDeclHandler` and `EndDoctypeDeclHandler` will not be called.

Passing a false value for *flag* will cancel a previous call that passed a true value, but otherwise has no effect.

This method can only be called before the `Parse` or `ParseFile` methods are called; calling it after either of those have been called causes `ExpatError` to be raised with the `code` attribute set to `errors.codes[errors.XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING]`.

</div>

<div class="method">

xmlparser.SetReparseDeferralEnabled(enabled)

> [!WARNING]
> Calling `SetReparseDeferralEnabled(False)` has security implications, as detailed below; please make sure to understand these consequences prior to using the `SetReparseDeferralEnabled` method.

Expat 2.6.0 introduced a security mechanism called "reparse deferral" where instead of causing denial of service through quadratic runtime from reparsing large tokens, reparsing of unfinished tokens is now delayed by default until a sufficient amount of input is reached. Due to this delay, registered handlers may — depending of the sizing of input chunks pushed to Expat — no longer be called right after pushing new input to the parser. Where immediate feedback and taking over responsibility of protecting against denial of service from large tokens are both wanted, calling `SetReparseDeferralEnabled(False)` disables reparse deferral for the current Expat parser instance, temporarily or altogether. Calling `SetReparseDeferralEnabled(True)` allows re-enabling reparse deferral.

`!SetReparseDeferralEnabled` has been backported to some prior releases of CPython as a security fix. Check for availability using `hasattr` if used in code running across a variety of Python versions.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

xmlparser.GetReparseDeferralEnabled()

Returns whether reparse deferral is currently enabled for the given Expat parser instance.

<div class="versionadded">

3.13

</div>

</div>

`!xmlparser` objects have the following methods to tune protections against some common XML vulnerabilities.

<div class="method">

xmlparser.SetBillionLaughsAttackProtectionActivationThreshold(threshold, /)

Sets the number of output bytes needed to activate protection against [billion laughs](https://en.wikipedia.org/wiki/Billion_laughs_attack) attacks.

The number of output bytes includes amplification from entity expansion and reading DTD files.

Parser objects usually have a protection activation threshold of 8 MiB, but the actual default value depends on the underlying Expat library.

An `ExpatError` is raised if this method is called on a `non-root <xmlparser-non-root>` parser. The corresponding `~ExpatError.lineno` and `~ExpatError.offset` should not be used as they may have no special meaning.

`!SetBillionLaughsAttackProtectionActivationThreshold` has been backported to some prior releases of CPython as a security fix. Check for availability using `hasattr` if used in code running across a variety of Python versions.

> [!NOTE]
> Activation thresholds below 4 MiB are known to break support for DITA 1.3 payload and are hence not recommended.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

xmlparser.SetBillionLaughsAttackProtectionMaximumAmplification(max_factor, /)

Sets the maximum tolerated amplification factor for protection against [billion laughs](https://en.wikipedia.org/wiki/Billion_laughs_attack) attacks.

The amplification factor is calculated as `(direct + indirect) / direct` while parsing, where `direct` is the number of bytes read from the primary document in parsing and `indirect` is the number of bytes added by expanding entities and reading of external DTD files.

The *max_factor* value must be a non-NaN `float` value greater than or equal to 1.0. Peak amplifications of factor 15,000 for the entire payload and of factor 30,000 in the middle of parsing have been observed with small benign files in practice. In particular, the activation threshold should be carefully chosen to avoid false positives.

Parser objects usually have a maximum amplification factor of 100, but the actual default value depends on the underlying Expat library.

An `ExpatError` is raised if this method is called on a `non-root <xmlparser-non-root>` parser or if *max_factor* is outside the valid range. The corresponding `~ExpatError.lineno` and `~ExpatError.offset` should not be used as they may have no special meaning.

`!SetBillionLaughsAttackProtectionMaximumAmplification` has been backported to some prior releases of CPython as a security fix. Check for availability using `hasattr` if used in code running across a variety of Python versions.

> [!NOTE]
> The maximum amplification factor is only considered if the threshold that can be adjusted by `.SetBillionLaughsAttackProtectionActivationThreshold` is exceeded.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

xmlparser.SetAllocTrackerActivationThreshold(threshold, /)

Sets the number of allocated bytes of dynamic memory needed to activate protection against disproportionate use of RAM.

Parser objects usually have an allocation activation threshold of 64 MiB, but the actual default value depends on the underlying Expat library.

An `ExpatError` is raised if this method is called on a `non-root <xmlparser-non-root>` parser. The corresponding `~ExpatError.lineno` and `~ExpatError.offset` should not be used as they may have no special meaning.

`!SetAllocTrackerActivationThreshold` has been backported to some prior releases of CPython as a security fix. Check for availability using `hasattr` if used in code running across a variety of Python versions.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

xmlparser.SetAllocTrackerMaximumAmplification(max_factor, /)

Sets the maximum amplification factor between direct input and bytes of dynamic memory allocated.

The amplification factor is calculated as `allocated / direct` while parsing, where `direct` is the number of bytes read from the primary document in parsing and `allocated` is the number of bytes of dynamic memory allocated in the parser hierarchy.

The *max_factor* value must be a non-NaN `float` value greater than or equal to 1.0. Amplification factors greater than 100.0 can be observed near the start of parsing even with benign files in practice. In particular, the activation threshold should be carefully chosen to avoid false positives.

Parser objects usually have a maximum amplification factor of 100, but the actual default value depends on the underlying Expat library.

An `ExpatError` is raised if this method is called on a `non-root <xmlparser-non-root>` parser or if *max_factor* is outside the valid range. The corresponding `~ExpatError.lineno` and `~ExpatError.offset` should not be used as they may have no special meaning.

`!SetAllocTrackerMaximumAmplification` has been backported to some prior releases of CPython as a security fix. Check for availability using `hasattr` if used in code running across a variety of Python versions.

> [!NOTE]
> The maximum amplification factor is only considered if the threshold that can be adjusted by `.SetAllocTrackerActivationThreshold` is exceeded.

<div class="versionadded">

3.15

</div>

</div>

`xmlparser` objects have the following attributes:

<div class="attribute">

xmlparser.buffer_size

The size of the buffer used when `buffer_text` is true. A new buffer size can be set by assigning a new integer value to this attribute. When the size is changed, the buffer will be flushed.

</div>

<div class="attribute">

xmlparser.buffer_text

Setting this to true causes the `xmlparser` object to buffer textual content returned by Expat to avoid multiple calls to the `CharacterDataHandler` callback whenever possible. This can improve performance substantially since Expat normally breaks character data into chunks at every line ending. This attribute is false by default, and may be changed at any time. Note that when it is false, data that does not contain newlines may be chunked too.

</div>

<div class="attribute">

xmlparser.buffer_used

If `buffer_text` is enabled, the number of bytes stored in the buffer. These bytes represent UTF-8 encoded text. This attribute has no meaningful interpretation when `buffer_text` is false.

</div>

<div class="attribute">

xmlparser.ordered_attributes

Setting this attribute to a non-zero integer causes the attributes to be reported as a list rather than a dictionary. The attributes are presented in the order found in the document text. For each attribute, two list entries are presented: the attribute name and the attribute value. (Older versions of this module also used this format.) By default, this attribute is false; it may be changed at any time.

</div>

<div class="attribute">

xmlparser.specified_attributes

If set to a non-zero integer, the parser will report only those attributes which were specified in the document instance and not those which were derived from attribute declarations. Applications which set this need to be especially careful to use what additional information is available from the declarations as needed to comply with the standards for the behavior of XML processors. By default, this attribute is false; it may be changed at any time.

</div>

<div class="attribute">

xmlparser.intern

The dictionary used to intern the names of elements and attributes. It is either the dictionary passed as the *intern* argument of `ParserCreate`, or a new dictionary created for this parser.

</div>

<div class="attribute">

xmlparser.namespace_prefixes

If set to a true value, and namespace processing is enabled, the namespace prefix is reported as the third part of the expanded name, separated by the namespace separator. Names which have no prefix are not affected. By default, this attribute is false; it may be changed at any time.

</div>

The following attributes contain values relating to the most recent error encountered by an `xmlparser` object, and will only have correct values once a call to `Parse` or `ParseFile` has raised an `xml.parsers.expat.ExpatError` exception.

<div class="attribute">

xmlparser.ErrorByteIndex

Byte index at which an error occurred.

</div>

<div class="attribute">

xmlparser.ErrorCode

Numeric code specifying the problem. This value can be passed to the `ErrorString` function, or compared to one of the constants defined in the `errors` object.

</div>

<div class="attribute">

xmlparser.ErrorColumnNumber

Column number at which an error occurred.

</div>

<div class="attribute">

xmlparser.ErrorLineNumber

Line number at which an error occurred.

</div>

The following attributes contain values relating to the current parse location in an `xmlparser` object. During a callback reporting a parse event they indicate the location of the first of the sequence of characters that generated the event. When called outside of a callback, the position indicated will be just past the last parse event (regardless of whether there was an associated callback).

<div class="attribute">

xmlparser.CurrentByteIndex

Current byte index in the parser input.

</div>

<div class="attribute">

xmlparser.CurrentColumnNumber

Current column number in the parser input.

</div>

<div class="attribute">

xmlparser.CurrentLineNumber

Current line number in the parser input.

</div>

Here is the list of handlers that can be set. To set a handler on an `xmlparser` object *o*, use `o.handlername = func`. *handlername* must be taken from the following list, and *func* must be a callable object accepting the correct number of arguments. The arguments are all strings, unless otherwise stated.

<div class="method">

xmlparser.XmlDeclHandler(version, encoding, standalone)

Called when the XML declaration is parsed. The XML declaration is the (optional) declaration of the applicable version of the XML recommendation, the encoding of the document text, and an optional "standalone" declaration. *version* and *encoding* will be strings, and *standalone* will be `1` if the document is declared standalone, `0` if it is declared not to be standalone, or `-1` if the standalone clause was omitted.

</div>

<div class="method">

xmlparser.StartDoctypeDeclHandler(doctypeName, systemId, publicId, has_internal_subset)

Called when Expat begins parsing the document type declaration (`<!DOCTYPE ...`). The *doctypeName* is provided exactly as presented. The *systemId* and *publicId* parameters give the system and public identifiers if specified, or `None` if omitted. *has_internal_subset* will be true if the document contains an internal document declaration subset.

</div>

<div class="method">

xmlparser.EndDoctypeDeclHandler()

Called when Expat is done parsing the document type declaration.

</div>

<div class="method">

xmlparser.ElementDeclHandler(name, model)

Called once for each element type declaration. *name* is the name of the element type, and *model* is a representation of the content model.

</div>

<div class="method">

xmlparser.AttlistDeclHandler(elname, attname, type, default, required)

Called for each declared attribute for an element type. If an attribute list declaration declares three attributes, this handler is called three times, once for each attribute. *elname* is the name of the element to which the declaration applies and *attname* is the name of the attribute declared. The The attribute type is a string passed as *type*: `'CDATA'`, `'ID'`, `'IDREF'`, `'IDREFS'`, `'ENTITY'`, `'ENTITIES'`, `'NMTOKEN'` or `'NMTOKENS'`, an enumeration like `'(x|y)'`, or a notation list like `'NOTATION(n1|n2)'`. *default* gives the default value for the attribute used when the attribute is not specified by the document instance, or `None` if there is no default value (`#IMPLIED` values). If the attribute is required to be given in the document instance, *required* will be true.

</div>

<div class="method">

xmlparser.StartElementHandler(name, attributes)

Called for the start of every element. *name* is a string containing the element name, and *attributes* is the element attributes. If `ordered_attributes` is true, this is a list (see `ordered_attributes` for a full description). Otherwise it's a dictionary mapping names to values.

</div>

<div class="method">

xmlparser.EndElementHandler(name)

Called for the end of every element.

</div>

<div class="method">

xmlparser.ProcessingInstructionHandler(target, data)

Called for every processing instruction.

</div>

<div class="method">

xmlparser.CharacterDataHandler(data)

Called for character data. This will be called for normal character data, CDATA marked content, and ignorable whitespace. Applications which must distinguish these cases can use the `StartCdataSectionHandler`, `EndCdataSectionHandler`, and `ElementDeclHandler` callbacks to collect the required information. Note that the character data may be chunked even if it is short and so you may receive more than one call to `CharacterDataHandler`. Set the `buffer_text` instance attribute to `True` to avoid that.

</div>

<div class="method">

xmlparser.UnparsedEntityDeclHandler(entityName, base, systemId, publicId, notationName)

Called for unparsed (NDATA) entity declarations. If this handler is not set, such declarations are reported by `EntityDeclHandler`, which is preferred for new code. (The underlying function in the Expat library has been declared obsolete.)

</div>

<div class="method">

xmlparser.EntityDeclHandler(entityName, is_parameter_entity, value, base, systemId, publicId, notationName)

Called for all entity declarations. For parameter and internal entities, *value* will be a string giving the declared contents of the entity; this will be `None` for external entities. The *notationName* parameter will be `None` for parsed entities, and the name of the notation for unparsed entities. *is_parameter_entity* will be true if the entity is a parameter entity or false for general entities (most applications only need to be concerned with general entities).

</div>

<div class="method">

xmlparser.NotationDeclHandler(notationName, base, systemId, publicId)

Called for notation declarations. *notationName*, *base*, and *systemId*, and *publicId* are strings if given. If the public identifier is omitted, *publicId* will be `None`.

</div>

<div class="method">

xmlparser.StartNamespaceDeclHandler(prefix, uri)

Called when an element contains a namespace declaration. Namespace declarations are processed before the `StartElementHandler` is called for the element on which declarations are placed.

</div>

<div class="method">

xmlparser.EndNamespaceDeclHandler(prefix)

Called when the closing tag is reached for an element that contained a namespace declaration. This is called once for each namespace declaration on the element in the reverse of the order for which the `StartNamespaceDeclHandler` was called to indicate the start of each namespace declaration's scope. Calls to this handler are made after the corresponding `EndElementHandler` for the end of the element.

</div>

<div class="method">

xmlparser.CommentHandler(data)

Called for comments. *data* is the text of the comment, excluding the leading `'<!-``-'` and trailing `'-``->'`.

</div>

<div class="method">

xmlparser.StartCdataSectionHandler()

Called at the start of a CDATA section. This and `EndCdataSectionHandler` are needed to be able to identify the syntactical start and end for CDATA sections.

</div>

<div class="method">

xmlparser.EndCdataSectionHandler()

Called at the end of a CDATA section.

</div>

<div class="method">

xmlparser.DefaultHandler(data)

Called for any characters in the XML document for which no applicable handler has been specified. This means characters that are part of a construct which could be reported, but for which no handler has been supplied.

</div>

<div class="method">

xmlparser.DefaultHandlerExpand(data)

This is the same as the `DefaultHandler`, but doesn't inhibit expansion of internal entities. The entity reference will not be passed to the default handler.

</div>

<div class="method">

xmlparser.NotStandaloneHandler()

Called if the XML document hasn't been declared as being a standalone document. This happens when there is an external subset or a reference to a parameter entity, but the XML declaration does not set standalone to `yes` in an XML declaration. If this handler returns `0`, then the parser will raise an `XML_ERROR_NOT_STANDALONE` error. If this handler is not set, no exception is raised by the parser for this condition.

</div>

<div class="method">

xmlparser.ExternalEntityRefHandler(context, base, systemId, publicId)

> [!WARNING]
> Implementing a handler that accesses local files and/or the network may create a vulnerability to [external entity attacks](https://en.wikipedia.org/wiki/XML_external_entity_attack) if `xmlparser` is used with user-provided XML content. Please reflect on your [threat model](https://en.wikipedia.org/wiki/Threat_model) before implementing this handler.

Called for references to external entities. *base* is the current base, as set by a previous call to `SetBase`. The public and system identifiers, *systemId* and *publicId*, are strings if given; if the public identifier is not given, *publicId* will be `None`. The *context* value is opaque and should only be used as described below.

For external entities to be parsed, this handler must be implemented. It is responsible for creating the sub-parser using `ExternalEntityParserCreate(context)`, initializing it with the appropriate callbacks, and parsing the entity. This handler should return an integer; if it returns `0`, the parser will raise an `XML_ERROR_EXTERNAL_ENTITY_HANDLING` error, otherwise parsing will continue.

If this handler is not provided, external entities are reported by the `DefaultHandler` callback, if provided.

</div>

<div class="method">

xmlparser.SkippedEntityHandler(entityName, is_parameter_entity)

Called for entity references which are not expanded, because the parser did not read the declaration of the entity. This happens when the external DTD subset or an external parameter entity is not parsed. *is_parameter_entity* is true for a parameter entity and false for a general entity.

</div>

## ExpatError Exceptions

`ExpatError` exceptions have a number of interesting attributes:

<div class="attribute">

ExpatError.code

Expat's internal error number for the specific error. The `errors.messages <xml.parsers.expat.errors.messages>` dictionary maps these error numbers to Expat's error messages. For example:

    from xml.parsers.expat import ParserCreate, ExpatError, errors

    p = ParserCreate()
    try:
        p.Parse(some_xml_document)
    except ExpatError as err:
        print("Error:", errors.messages[err.code])

The `~xml.parsers.expat.errors` module also provides error message constants and a dictionary `~xml.parsers.expat.errors.codes` mapping these messages back to the error codes, see below.

</div>

<div class="attribute">

ExpatError.lineno

Line number on which the error was detected. The first line is numbered `1`.

</div>

<div class="attribute">

ExpatError.offset

Character offset into the line where the error occurred. The first column is numbered `0`.

</div>

## Example

The following program defines three handlers that just print out their arguments. :

    import xml.parsers.expat

    # 3 handler functions
    def start_element(name, attrs):
        print('Start element:', name, attrs)
    def end_element(name):
        print('End element:', name)
    def char_data(data):
        print('Character data:', repr(data))

    p = xml.parsers.expat.ParserCreate()

    p.StartElementHandler = start_element
    p.EndElementHandler = end_element
    p.CharacterDataHandler = char_data

    p.Parse("""<?xml version="1.0"?>
    <parent id="top"><child1 name="paul">Text goes here</child1>
    <child2 name="fred">More text</child2>
    </parent>""", 1)

The output from this program is:

    Start element: parent {'id': 'top'}
    Start element: child1 {'name': 'paul'}
    Character data: 'Text goes here'
    End element: child1
    Character data: '\n'
    Start element: child2 {'name': 'fred'}
    Character data: 'More text'
    End element: child2
    Character data: '\n'
    End element: parent

## Content Model Descriptions

<div class="module">

xml.parsers.expat.model

</div>

Content models are described using nested tuples. Each tuple contains four values: the type, the quantifier, the name, and a tuple of children. Children are simply additional content model descriptions.

The values of the first two fields are constants defined in the `!xml.parsers.expat.model` module. These constants can be collected in two groups: the model type group and the quantifier group.

The constants in the model type group are:

<div class="data">

XML_CTYPE_ANY

The element named by the model name was declared to have a content model of `ANY`.

</div>

<div class="data">

XML_CTYPE_CHOICE

The named element allows a choice from a number of options; this is used for content models such as `(A | B | C)`.

</div>

<div class="data">

XML_CTYPE_EMPTY

Elements which are declared to be `EMPTY` have this model type.

</div>

<div class="data">

XML_CTYPE_MIXED

The named element allows character data, optionally interspersed with the named children; this is used for content models such as `(#PCDATA)` and `(#PCDATA | A | B)*`.

</div>

<div class="data">

XML_CTYPE_NAME

The model names a single element, as for `A`.

</div>

<div class="data">

XML_CTYPE_SEQ

Models which represent a series of models which follow one after the other are indicated with this model type. This is used for models such as `(A, B, C)`.

</div>

The constants in the quantifier group are:

<div class="data">

XML_CQUANT_NONE

No modifier is given, so it can appear exactly once, as for `A`.

</div>

<div class="data">

XML_CQUANT_OPT

The model is optional: it can appear once or not at all, as for `A?`.

</div>

<div class="data">

XML_CQUANT_PLUS

The model must occur one or more times (like `A+`).

</div>

<div class="data">

XML_CQUANT_REP

The model must occur zero or more times, as for `A*`.

</div>

## Expat error constants

<div class="module">

xml.parsers.expat.errors

</div>

The following constants are provided in the `!xml.parsers.expat.errors` module. These constants are useful in interpreting some of the attributes of the `ExpatError` exception objects raised when an error has occurred. Since for backwards compatibility reasons, the constants' value is the error *message* and not the numeric error *code*, you do this by comparing its `code` attribute with `errors.codes[errors.XML_ERROR_{CONSTANT_NAME}]`.

The `errors` module has the following attributes:

<div class="data">

codes

A dictionary mapping string descriptions to their error codes.

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

messages

A dictionary mapping numeric error codes to their string descriptions.

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

XML_ERROR_ASYNC_ENTITY

</div>

<div class="data">

XML_ERROR_ATTRIBUTE_EXTERNAL_ENTITY_REF

An entity reference in an attribute value referred to an external entity instead of an internal entity.

</div>

<div class="data">

XML_ERROR_BAD_CHAR_REF

A character reference referred to a character which is illegal in XML (for example, character `0`, or '`&#0;`').

</div>

<div class="data">

XML_ERROR_BINARY_ENTITY_REF

An entity reference referred to an entity which was declared with a notation, so cannot be parsed.

</div>

<div class="data">

XML_ERROR_DUPLICATE_ATTRIBUTE

An attribute was used more than once in a start tag.

</div>

<div class="data">

XML_ERROR_INCORRECT_ENCODING

</div>

<div class="data">

XML_ERROR_INVALID_TOKEN

Raised when an input byte could not properly be assigned to a character; for example, a NUL byte (value `0`) in a UTF-8 input stream.

</div>

<div class="data">

XML_ERROR_JUNK_AFTER_DOC_ELEMENT

Something other than whitespace occurred after the document element.

</div>

<div class="data">

XML_ERROR_MISPLACED_XML_PI

An XML declaration was found somewhere other than the start of the input data.

</div>

<div class="data">

XML_ERROR_NO_ELEMENTS

The document contains no elements (XML requires all documents to contain exactly one top-level element).

</div>

<div class="data">

XML_ERROR_NO_MEMORY

Expat was not able to allocate memory internally.

</div>

<div class="data">

XML_ERROR_PARAM_ENTITY_REF

A parameter entity reference was found where it was not allowed.

</div>

<div class="data">

XML_ERROR_PARTIAL_CHAR

An incomplete character was found in the input.

</div>

<div class="data">

XML_ERROR_RECURSIVE_ENTITY_REF

An entity reference contained another reference to the same entity; possibly via a different name, and possibly indirectly.

</div>

<div class="data">

XML_ERROR_SYNTAX

Some unspecified syntax error was encountered.

</div>

<div class="data">

XML_ERROR_TAG_MISMATCH

An end tag did not match the innermost open start tag.

</div>

<div class="data">

XML_ERROR_UNCLOSED_TOKEN

Some token (such as a start tag) was not closed before the end of the stream or the next token was encountered.

</div>

<div class="data">

XML_ERROR_UNDEFINED_ENTITY

A reference was made to an entity which was not defined.

</div>

<div class="data">

XML_ERROR_UNKNOWN_ENCODING

The document encoding is not supported by Expat.

</div>

<div class="data">

XML_ERROR_UNCLOSED_CDATA_SECTION

A CDATA marked section was not closed.

</div>

<div class="data">

XML_ERROR_EXTERNAL_ENTITY_HANDLING

</div>

<div class="data">

XML_ERROR_NOT_STANDALONE

The parser determined that the document was not "standalone" though it declared itself to be in the XML declaration, and the `NotStandaloneHandler` was set and returned `0`.

</div>

<div class="data">

XML_ERROR_UNEXPECTED_STATE

</div>

<div class="data">

XML_ERROR_ENTITY_DECLARED_IN_PE

</div>

<div class="data">

XML_ERROR_FEATURE_REQUIRES_XML_DTD

An operation was requested that requires DTD support to be compiled in, but Expat was configured without DTD support. This should never be reported by a standard build of the `!xml.parsers.expat` module.

</div>

<div class="data">

XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING

A behavioral change was requested after parsing started that can only be changed before parsing has started. This is (currently) only raised by `UseForeignDTD`.

</div>

<div class="data">

XML_ERROR_UNBOUND_PREFIX

An undeclared prefix was found when namespace processing was enabled.

</div>

<div class="data">

XML_ERROR_UNDECLARING_PREFIX

The document attempted to remove the namespace declaration associated with a prefix.

</div>

<div class="data">

XML_ERROR_INCOMPLETE_PE

A parameter entity contained incomplete markup.

</div>

<div class="data">

XML_ERROR_XML_DECL

There was an error parsing the XML declaration.

</div>

<div class="data">

XML_ERROR_TEXT_DECL

There was an error parsing a text declaration in an external entity.

</div>

<div class="data">

XML_ERROR_PUBLICID

Characters were found in the public id that are not allowed.

</div>

<div class="data">

XML_ERROR_SUSPENDED

The requested operation was made on a suspended parser, but isn't allowed. This includes attempts to provide additional input or to stop the parser.

</div>

<div class="data">

XML_ERROR_NOT_SUSPENDED

An attempt to resume the parser was made when the parser had not been suspended.

</div>

<div class="data">

XML_ERROR_ABORTED

This should not be reported to Python applications.

</div>

<div class="data">

XML_ERROR_FINISHED

The requested operation was made on a parser which was finished parsing input, but isn't allowed. This includes attempts to provide additional input or to stop the parser.

</div>

<div class="data">

XML_ERROR_SUSPEND_PE

</div>

<div class="data">

XML_ERROR_RESERVED_PREFIX_XML

An attempt was made to undeclare reserved namespace prefix `xml` or to bind it to another namespace URI.

</div>

<div class="data">

XML_ERROR_RESERVED_PREFIX_XMLNS

An attempt was made to declare or undeclare reserved namespace prefix `xmlns`.

</div>

<div class="data">

XML_ERROR_RESERVED_NAMESPACE_URI

An attempt was made to bind the URI of one the reserved namespace prefixes `xml` and `xmlns` to another namespace prefix.

</div>

<div class="data">

XML_ERROR_INVALID_ARGUMENT

This should not be reported to Python applications.

</div>

<div class="data">

XML_ERROR_NO_BUFFER

This should not be reported to Python applications.

</div>

<div class="data">

XML_ERROR_AMPLIFICATION_LIMIT_BREACH

The limit on input amplification factor (from DTD and entities) has been breached.

</div>

<div class="data">

XML_ERROR_NOT_STARTED

The parser was tried to be stopped or suspended before it started.

<div class="versionadded">

3.14

</div>

</div>

**Footnotes**

[^1]: The encoding string included in XML output should conform to the appropriate standards. For example, "UTF-8" is valid, but "UTF8" is not valid in an XML document's declaration, even though Python accepts it as an encoding name. See <https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl> and <https://www.iana.org/assignments/character-sets/character-sets.xhtml>.

# Python Runtime Services

The modules described in this chapter provide a wide range of services related to the Python interpreter and its interaction with its environment. Here's an overview:

<div class="toctree">

sys.rst sys.monitoring.rst sysconfig.rst builtins.rst \_\_main\_\_.rst warnings.rst dataclasses.rst contextlib.rst abc.rst atexit.rst traceback.rst \_\_future\_\_.rst gc.rst inspect.rst annotationlib.rst site.rst

</div>

<div class="seealso">

- See the `concurrent.interpreters` module, which similarly exposes core runtime functionality.

</div>

# `!queue` --- A synchronized queue class

<div class="module" data-synopsis="A synchronized queue class.">

queue

</div>

**Source code:** `Lib/queue.py`

------------------------------------------------------------------------

The `!queue` module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads. The `Queue` class in this module implements all the required locking semantics.

The module implements three types of queue, which differ only in the order in which the entries are retrieved. In a `FIFO (first-in, first-out)` queue, the first tasks added are the first retrieved. In a `LIFO (last-in, first-out)` queue, the most recently added entry is the first retrieved (operating like a stack). With a priority queue, the entries are kept sorted (using the `heapq` module) and the lowest valued entry is retrieved first.

Internally, those three types of queues use locks to temporarily block competing threads; however, they are not designed to handle reentrancy within a thread.

In addition, the module implements a "simple" `FIFO (first-in, first-out)` queue type, `SimpleQueue`, whose specific implementation provides additional guarantees in exchange for the smaller functionality.

The `!queue` module defines the following classes and exceptions:

<div class="Queue(maxsize=0)">

Constructor for a `FIFO (first-in, first-out)` queue. *maxsize* is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If *maxsize* is less than or equal to zero, the queue size is infinite.

</div>

<div class="LifoQueue(maxsize=0)">

Constructor for a `LIFO (last-in, first-out)` queue. *maxsize* is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If *maxsize* is less than or equal to zero, the queue size is infinite.

</div>

<div class="PriorityQueue(maxsize=0)">

Constructor for a priority queue. *maxsize* is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If *maxsize* is less than or equal to zero, the queue size is infinite.

The lowest valued entries are retrieved first (the lowest valued entry is the one that would be returned by `min(entries)`). A typical pattern for entries is a tuple in the form: `(priority_number, data)`.

If the *data* elements are not comparable, the data can be wrapped in a class that ignores the data item and only compares the priority number:

    from dataclasses import dataclass, field
    from typing import Any

    @dataclass(order=True)
    class PrioritizedItem:
        priority: int
        item: Any=field(compare=False)

</div>

<div class="SimpleQueue()">

Constructor for an unbounded `FIFO (first-in, first-out)` queue. Simple queues lack advanced functionality such as task tracking.

Simple queues are `generic <generics>` over the type of their items.

<div class="versionadded">

3.7

</div>

</div>

<div class="exception">

Empty

Exception raised when non-blocking `~Queue.get` (or `~Queue.get_nowait`) is called on a `Queue` object which is empty.

</div>

<div class="exception">

Full

Exception raised when non-blocking `~Queue.put` (or `~Queue.put_nowait`) is called on a `Queue` object which is full.

</div>

<div class="exception">

ShutDown

Exception raised when `~Queue.put` or `~Queue.get` is called on a `Queue` object which has been shut down.

<div class="versionadded">

3.13

</div>

</div>

## Queue Objects

Queue objects (`Queue`, `LifoQueue`, or `PriorityQueue`) provide the public methods described below.

<div class="method">

Queue.qsize()

Return the approximate size of the queue. Note, qsize() \> 0 doesn't guarantee that a subsequent get() will not block, nor will qsize() \< maxsize guarantee that put() will not block.

</div>

<div class="method">

Queue.empty()

Return `True` if the queue is empty, `False` otherwise. If empty() returns `True` it doesn't guarantee that a subsequent call to put() will not block. Similarly, if empty() returns `False` it doesn't guarantee that a subsequent call to get() will not block.

</div>

<div class="method">

Queue.full()

Return `True` if the queue is full, `False` otherwise. If full() returns `True` it doesn't guarantee that a subsequent call to get() will not block. Similarly, if full() returns `False` it doesn't guarantee that a subsequent call to put() will not block.

</div>

<div class="method">

Queue.put(item, block=True, timeout=None)

Put *item* into the queue. If optional args *block* is true and *timeout* is `None` (the default), block if necessary until a free slot is available. If *timeout* is a positive number, it blocks at most *timeout* seconds and raises the `Full` exception if no free slot was available within that time. Otherwise (*block* is false), put an item on the queue if a free slot is immediately available, else raise the `Full` exception (*timeout* is ignored in that case).

Raises `ShutDown` if the queue has been shut down.

</div>

<div class="method">

Queue.put_nowait(item)

Equivalent to `put(item, block=False)`.

</div>

<div class="method">

Queue.get(block=True, timeout=None)

Remove and return an item from the queue. If optional args *block* is true and *timeout* is `None` (the default), block if necessary until an item is available. If *timeout* is a positive number, it blocks at most *timeout* seconds and raises the `Empty` exception if no item was available within that time. Otherwise (*block* is false), return an item if one is immediately available, else raise the `Empty` exception (*timeout* is ignored in that case).

Prior to 3.0 on POSIX systems, and for all versions on Windows, if *block* is true and *timeout* is `None`, this operation goes into an uninterruptible wait on an underlying lock. This means that no exceptions can occur, and in particular a SIGINT will not trigger a `KeyboardInterrupt`.

Raises `ShutDown` if the queue has been shut down and is empty, or if the queue has been shut down immediately.

</div>

<div class="method">

Queue.get_nowait()

Equivalent to `get(block=False)`.

</div>

Two methods are offered to support tracking whether enqueued tasks have been fully processed by daemon consumer threads.

<div class="method">

Queue.task_done()

Indicate that a formerly enqueued task is complete. Used by queue consumer threads. For each `get` used to fetch a task, a subsequent call to `task_done` tells the queue that the processing on the task is complete.

If a `join` is currently blocking, it will resume when all items have been processed (meaning that a `task_done` call was received for every item that had been `put` into the queue).

Raises a `ValueError` if called more times than there were items placed in the queue.

</div>

<div class="method">

Queue.join()

Blocks until all items in the queue have been gotten and processed.

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls `task_done` to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, `join` unblocks.

</div>

### Waiting for task completion

Example of how to wait for enqueued tasks to be completed:

    import threading
    import queue

    q = queue.Queue()

    def worker():
        while True:
            item = q.get()
            print(f'Working on {item}')
            print(f'Finished {item}')
            q.task_done()

    # Turn-on the worker thread.
    threading.Thread(target=worker, daemon=True).start()

    # Send thirty task requests to the worker.
    for item in range(30):
        q.put(item)

    # Block until all tasks are done.
    q.join()
    print('All work completed')

### Terminating queues

When no longer needed, `Queue` objects can be wound down until empty or terminated immediately with a hard shutdown.

<div class="method">

Queue.shutdown(immediate=False)

Put a `Queue` instance into a shutdown mode.

The queue can no longer grow. Future calls to `~Queue.put` raise `ShutDown`. Currently blocked callers of `~Queue.put` will be unblocked and will raise `ShutDown` in the formerly blocked thread.

If *immediate* is false (the default), the queue can be wound down normally with `~Queue.get` calls to extract tasks that have already been loaded.

And if `~Queue.task_done` is called for each remaining task, a pending `~Queue.join` will be unblocked normally.

Once the queue is empty, future calls to `~Queue.get` will raise `ShutDown`.

If *immediate* is true, the queue is terminated immediately. The queue is drained to be completely empty and the count of unfinished tasks is reduced by the number of tasks drained. If unfinished tasks is zero, callers of `~Queue.join` are unblocked. Also, blocked callers of `~Queue.get` are unblocked and will raise `ShutDown` because the queue is empty.

Use caution when using `~Queue.join` with *immediate* set to true. This unblocks the join even when no work has been done on the tasks, violating the usual invariant for joining a queue.

<div class="versionadded">

3.13

</div>

</div>

## SimpleQueue Objects

`SimpleQueue` objects provide the public methods described below.

<div class="method">

SimpleQueue.qsize()

Return the approximate size of the queue. Note, qsize() \> 0 doesn't guarantee that a subsequent get() will not block.

</div>

<div class="method">

SimpleQueue.empty()

Return `True` if the queue is empty, `False` otherwise. If empty() returns `False` it doesn't guarantee that a subsequent call to get() will not block.

</div>

<div class="method">

SimpleQueue.put(item, block=True, timeout=None)

Put *item* into the queue. The method never blocks and always succeeds (except for potential low-level errors such as failure to allocate memory). The optional args *block* and *timeout* are ignored and only provided for compatibility with `Queue.put`.

<div class="impl-detail">

This method has a C implementation which is reentrant. That is, a `put()` or `get()` call can be interrupted by another `put()` call in the same thread without deadlocking or corrupting internal state inside the queue. This makes it appropriate for use in destructors such as `__del__` methods or `weakref` callbacks.

</div>

</div>

<div class="method">

SimpleQueue.put_nowait(item)

Equivalent to `put(item, block=False)`, provided for compatibility with `Queue.put_nowait`.

</div>

<div class="method">

SimpleQueue.get(block=True, timeout=None)

Remove and return an item from the queue. If optional args *block* is true and *timeout* is `None` (the default), block if necessary until an item is available. If *timeout* is a positive number, it blocks at most *timeout* seconds and raises the `Empty` exception if no item was available within that time. Otherwise (*block* is false), return an item if one is immediately available, else raise the `Empty` exception (*timeout* is ignored in that case).

</div>

<div class="method">

SimpleQueue.get_nowait()

Equivalent to `get(False)`.

</div>

<div class="seealso">

Class `multiprocessing.Queue`  
A queue class for use in a multi-processing (rather than multi-threading) context.

`collections.deque` is an alternative implementation of unbounded queues with fast atomic `~collections.deque.append` and `~collections.deque.popleft` operations that do not require locking and also support indexing.

</div>

# `!quopri` --- Encode and decode MIME quoted-printable data

<div class="module" data-synopsis="Encode and decode files using the MIME quoted-printable encoding.">

quopri

</div>

**Source code:** `Lib/quopri.py`

<div class="index">

pair: quoted-printable; encoding single: MIME; quoted-printable encoding

</div>

------------------------------------------------------------------------

This module performs quoted-printable transport encoding and decoding, as defined in `1521`: "MIME (Multipurpose Internet Mail Extensions) Part One: Mechanisms for Specifying and Describing the Format of Internet Message Bodies". The quoted-printable encoding is designed for data where there are relatively few nonprintable characters; the base64 encoding scheme available via the `base64` module is more compact if there are many such characters, as when sending a graphics file.

<div class="function">

decode(input, output, header=False, \*, strip_ws=False)

Decode the contents of the *input* file and write the resulting decoded binary data to the *output* file. *input* and *output* must be `binary file objects
<file object>`. If the optional argument *header* is present and true, underscore will be decoded as space. This is used to decode "Q"-encoded headers as described in `1522`: "MIME (Multipurpose Internet Mail Extensions) Part Two: Message Header Extensions for Non-ASCII Text". If the optional argument *strip_ws* is true, trailing whitespace is stripped from each line, as required by `2045`.

<div class="versionchanged">

next Added the *strip_ws* parameter.

</div>

</div>

<div class="function">

encode(input, output, quotetabs, header=False)

Encode the contents of the *input* file and write the resulting quoted-printable data to the *output* file. *input* and *output* must be `binary file objects <file object>`. *quotetabs*, a non-optional flag which controls whether to encode embedded spaces and tabs; when true it encodes such embedded whitespace, and when false it leaves them unencoded. Note that spaces and tabs appearing at the end of lines are always encoded, as per `1521`. *header* is a flag which controls if spaces are encoded as underscores as per `1522`.

</div>

<div class="function">

decodestring(s, header=False, \*, strip_ws=False)

Like `decode`, except that it accepts a source `bytes` and returns the corresponding decoded `bytes`.

<div class="versionchanged">

next Added the *strip_ws* parameter.

</div>

</div>

<div class="function">

encodestring(s, quotetabs=False, header=False)

Like `encode`, except that it accepts a source `bytes` and returns the corresponding encoded `bytes`. By default, it sends a `False` value to *quotetabs* parameter of the `encode` function.

</div>

<div class="seealso">

Module `base64`  
Encode and decode MIME base64 data

</div>

# `!random` --- Generate pseudo-random numbers

<div class="module" data-synopsis="Generate pseudo-random numbers with various common distributions.">

random

</div>

**Source code:** `Lib/random.py`

------------------------------------------------------------------------

This module implements pseudo-random number generators for various distributions.

For integers, there is uniform selection from a range. For sequences, there is uniform selection of a random element, a function to generate a random permutation of a list in-place, and a function for random sampling without replacement.

On the real line, there are functions to compute uniform, normal (Gaussian), lognormal, negative exponential, gamma, and beta distributions. For generating distributions of angles, the von Mises distribution is available.

Almost all module functions depend on the basic function `.random`, which generates a random float uniformly in the half-open range `0.0 <= X < 1.0`. Python uses the Mersenne Twister as the core generator. It produces 53-bit precision floats and has a period of 2\*\*19937-1. The underlying implementation in C is both fast and threadsafe. The Mersenne Twister is one of the most extensively tested random number generators in existence. However, being completely deterministic, it is not suitable for all purposes, and is completely unsuitable for cryptographic purposes.

The functions supplied by this module are actually bound methods of a hidden instance of the `random.Random` class. You can instantiate your own instances of `Random` to get generators that don't share state.

Class `Random` can also be subclassed if you want to use a different basic generator of your own devising: see the documentation on that class for more details.

The `!random` module also provides the `SystemRandom` class which uses the system function `os.urandom` to generate random numbers from sources provided by the operating system.

> [!WARNING]
> The pseudo-random generators of this module should not be used for security purposes. For security or cryptographic uses, see the `secrets` module.

<div class="seealso">

M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally equidistributed uniform pseudorandom number generator", ACM Transactions on Modeling and Computer Simulation Vol. 8, No. 1, January pp.3--30 1998.

[Complementary-Multiply-with-Carry recipe](https://code.activestate.com/recipes/576707-long-period-random-number-generator/) for a compatible alternative random number generator with a long period and comparatively simple update operations.

</div>

> [!NOTE]
> The global random number generator and instances of `Random` are thread-safe. However, in the free-threaded build, concurrent calls to the global generator or to the same instance of `Random` may encounter contention and poor performance. Consider using separate instances of `Random` per thread instead.

## Bookkeeping functions

<div class="function">

seed(a=None, version=2)

Initialize the random number generator.

If *a* is omitted or `None`, the current system time is used. If randomness sources are provided by the operating system, they are used instead of the system time (see the `os.urandom` function for details on availability).

If *a* is an int, its absolute value is used directly.

With version 2 (the default), a `str`, `bytes`, or `bytearray` object gets converted to an `int` and all of its bits are used.

With version 1 (provided for reproducing random sequences from older versions of Python), the algorithm for `str` and `bytes` generates a narrower range of seeds.

<div class="versionchanged">

3.2 Moved to the version 2 scheme which uses all of the bits in a string seed.

</div>

<div class="versionchanged">

3.11 The *seed* must be one of the following types: `None`, `int`, `float`, `str`, `bytes`, or `bytearray`.

</div>

</div>

<div class="function">

getstate()

Return an object capturing the current internal state of the generator. This object can be passed to `setstate` to restore the state.

</div>

<div class="function">

setstate(state)

*state* should have been obtained from a previous call to `getstate`, and `setstate` restores the internal state of the generator to what it was at the time `getstate` was called.

</div>

## Functions for bytes

<div class="function">

randbytes(n)

Generate *n* random bytes.

This method should not be used for generating security tokens. Use `secrets.token_bytes` instead.

<div class="versionadded">

3.9

</div>

</div>

## Functions for integers

<div class="function">

randrange(stop) randrange(start, stop\[, step\])

Return a randomly selected element from `range(start, stop, step)`.

This is roughly equivalent to `choice(range(start, stop, step))` but supports arbitrarily large ranges and is optimized for common cases.

The positional argument pattern matches the `range` function.

Keyword arguments should not be used because they can be interpreted in unexpected ways. For example `randrange(start=100)` is interpreted as `randrange(0, 100, 1)`.

<div class="versionchanged">

3.2 `randrange` is more sophisticated about producing equally distributed values. Formerly it used a style like `int(random()*n)` which could produce slightly uneven distributions.

</div>

<div class="versionchanged">

3.12 Automatic conversion of non-integer types is no longer supported. Calls such as `randrange(10.0)` and `randrange(Fraction(10, 1))` now raise a `TypeError`.

</div>

</div>

<div class="function">

randint(a, b)

Return a random integer *N* such that `a <= N <= b`. Alias for `randrange(a, b+1)`.

</div>

<div class="function">

getrandbits(k)

Returns a non-negative Python integer with *k* random bits. This method is supplied with the Mersenne Twister generator and some other generators may also provide it as an optional part of the API. When available, `getrandbits` enables `randrange` to handle arbitrarily large ranges.

<div class="versionchanged">

3.9 This method now accepts zero for *k*.

</div>

</div>

## Functions for sequences

<div class="function">

choice(seq)

Return a random element from the non-empty sequence *seq*. If *seq* is empty, raises `IndexError`.

</div>

<div class="function">

choices(population, weights=None, \*, cum_weights=None, k=1)

Return a *k* sized list of elements chosen from the *population* with replacement. If the *population* is empty, raises `IndexError`.

If a *weights* sequence is specified, selections are made according to the relative weights. Alternatively, if a *cum_weights* sequence is given, the selections are made according to the cumulative weights (perhaps computed using `itertools.accumulate`). For example, the relative weights `[10, 5, 30, 5]` are equivalent to the cumulative weights `[10, 15, 45, 50]`. Internally, the relative weights are converted to cumulative weights before making selections, so supplying the cumulative weights saves work.

If neither *weights* nor *cum_weights* are specified, selections are made with equal probability. If a weights sequence is supplied, it must be the same length as the *population* sequence. It is a `TypeError` to specify both *weights* and *cum_weights*.

The *weights* or *cum_weights* can use any numeric type that interoperates with the `float` values returned by `~random.random` (that includes integers, floats, and fractions but excludes decimals). Weights are assumed to be non-negative and finite. A `ValueError` is raised if all weights are zero.

For a given seed, the `choices` function with equal weighting typically produces a different sequence than repeated calls to `choice`. The algorithm used by `choices` uses floating-point arithmetic for internal consistency and speed. The algorithm used by `choice` defaults to integer arithmetic with repeated selections to avoid small biases from round-off error.

<div class="versionadded">

3.6

</div>

<div class="versionchanged">

3.9 Raises a `ValueError` if all weights are zero.

</div>

</div>

<div class="function">

shuffle(x)

Shuffle the sequence *x* in place.

To shuffle an immutable sequence and return a new shuffled list, use `sample(x, k=len(x))` instead.

Note that even for small `len(x)`, the total number of permutations of *x* can quickly grow larger than the period of most random number generators. This implies that most permutations of a long sequence can never be generated. For example, a sequence of length 2080 is the largest that can fit within the period of the Mersenne Twister random number generator.

<div class="versionchanged">

3.11 Removed the optional parameter *random*.

</div>

</div>

<div class="function">

sample(population, k, \*, counts=None)

Return a *k* length list of unique elements chosen from the population sequence. Used for random sampling without replacement.

Returns a new list containing elements from the population while leaving the original population unchanged. The resulting list is in selection order so that all sub-slices will also be valid random samples. This allows raffle winners (the sample) to be partitioned into grand prize and second place winners (the subslices).

Members of the population need not be `hashable` or unique. If the population contains repeats, then each occurrence is a possible selection in the sample.

Repeated elements can be specified one at a time or with the optional keyword-only *counts* parameter. For example, `sample(['red', 'blue'], counts=[4, 2], k=5)` is equivalent to `sample(['red', 'red', 'red', 'red', 'blue', 'blue'], k=5)`.

To choose a sample from a range of integers, use a `range` object as an argument. This is especially fast and space efficient for sampling from a large population: `sample(range(10000000), k=60)`.

If the sample size is larger than the population size, a `ValueError` is raised.

<div class="versionchanged">

3.9 Added the *counts* parameter.

</div>

<div class="versionchanged">

3.11

The *population* must be a sequence. Automatic conversion of sets to lists is no longer supported.

</div>

</div>

## Discrete distributions

The following function generates a discrete distribution.

<div class="function">

binomialvariate(n=1, p=0.5)

[Binomial distribution](https://mathworld.wolfram.com/BinomialDistribution.html). Return the number of successes for *n* independent trials with the probability of success in each trial being *p*:

Mathematically equivalent to:

    sum(random() < p for i in range(n))

The number of trials *n* should be a non-negative integer. The probability of success *p* should be between `0.0 <= p <= 1.0`. The result is an integer in the range `0 <= X <= n`.

<div class="versionadded">

3.12

</div>

</div>

## Real-valued distributions

The following functions generate specific real-valued distributions. Function parameters are named after the corresponding variables in the distribution's equation, as used in common mathematical practice; most of these equations can be found in any statistics text.

<div class="function">

random()

Return the next random floating-point number in the range `0.0 <= X < 1.0`

</div>

<div class="function">

uniform(a, b)

Return a random floating-point number *N* such that `a <= N <= b` for `a <= b` and `b <= N <= a` for `b < a`.

The end-point value `b` may or may not be included in the range depending on floating-point rounding in the expression `a + (b-a) * random()`.

</div>

<div class="function">

triangular(low, high, mode)

Return a random floating-point number *N* such that `low <= N <= high` and with the specified *mode* between those bounds. The *low* and *high* bounds default to zero and one. The *mode* argument defaults to the midpoint between the bounds, giving a symmetric distribution.

</div>

<div class="function">

betavariate(alpha, beta)

Beta distribution. Conditions on the parameters are `alpha > 0` and `beta > 0`. Returned values range between 0 and 1.

</div>

<div class="function">

expovariate(lambd = 1.0)

Exponential distribution. *lambd* is 1.0 divided by the desired mean. It should be nonzero. (The parameter would be called "lambda", but that is a reserved word in Python.) Returned values range from 0 to positive infinity if *lambd* is positive, and from negative infinity to 0 if *lambd* is negative.

<div class="versionchanged">

3.12 Added the default value for `lambd`.

</div>

</div>

<div class="function">

gammavariate(alpha, beta)

Gamma distribution. (*Not* the gamma function!) The shape and scale parameters, *alpha* and *beta*, must have positive values. (Calling conventions vary and some sources define 'beta' as the inverse of the scale).

The probability distribution function is:

    x ** (alpha - 1) * math.exp(-x / beta)
    pdf(x) =  --------------------------------------
      math.gamma(alpha) * beta ** alpha

</div>

<div class="function">

gauss(mu=0.0, sigma=1.0)

Normal distribution, also called the Gaussian distribution. *mu* is the mean, and *sigma* is the standard deviation. This is slightly faster than the `normalvariate` function defined below.

Multithreading note: When two threads call this function simultaneously, it is possible that they will receive the same return value. This can be avoided in three ways. 1) Have each thread use a different instance of the random number generator. 2) Put locks around all calls. 3) Use the slower, but thread-safe `normalvariate` function instead.

<div class="versionchanged">

3.11 *mu* and *sigma* now have default arguments.

</div>

</div>

<div class="function">

lognormvariate(mu, sigma)

Log normal distribution. If you take the natural logarithm of this distribution, you'll get a normal distribution with mean *mu* and standard deviation *sigma*. *mu* can have any value, and *sigma* must be greater than zero.

</div>

<div class="function">

normalvariate(mu=0.0, sigma=1.0)

Normal distribution. *mu* is the mean, and *sigma* is the standard deviation.

<div class="versionchanged">

3.11 *mu* and *sigma* now have default arguments.

</div>

</div>

<div class="function">

vonmisesvariate(mu, kappa)

*mu* is the mean angle, expressed in radians between 0 and 2\**pi*, and *kappa* is the concentration parameter, which must be greater than or equal to zero. If *kappa* is equal to zero, this distribution reduces to a uniform random angle over the range 0 to 2\**pi*.

</div>

<div class="function">

paretovariate(alpha)

Pareto distribution. *alpha* is the shape parameter.

</div>

<div class="function">

weibullvariate(alpha, beta)

Weibull distribution. *alpha* is the scale parameter and *beta* is the shape parameter.

</div>

## Alternative Generator

<div class="Random([seed])">

Class that implements the default pseudo-random number generator used by the `!random` module.

<div class="versionchanged">

3.11 Formerly the *seed* could be any hashable object. Now it is limited to: `None`, `int`, `float`, `str`, `bytes`, or `bytearray`.

</div>

Subclasses of `!Random` should override the following methods if they wish to make use of a different basic generator:

<div class="method">

Random.seed(a=None, version=2)

Override this method in subclasses to customise the `~random.seed` behaviour of `!Random` instances.

</div>

<div class="method">

Random.getstate()

Override this method in subclasses to customise the `~random.getstate` behaviour of `!Random` instances.

</div>

<div class="method">

Random.setstate(state)

Override this method in subclasses to customise the `~random.setstate` behaviour of `!Random` instances.

</div>

<div class="method">

Random.random()

Override this method in subclasses to customise the `~random.random` behaviour of `!Random` instances.

</div>

Optionally, a custom generator subclass can also supply the following method:

<div class="method">

Random.getrandbits(k)

Override this method in subclasses to customise the `~random.getrandbits` behaviour of `!Random` instances.

</div>

<div class="method">

Random.randbytes(n)

Override this method in subclasses to customise the `~random.randbytes` behaviour of `!Random` instances.

</div>

</div>

<div class="SystemRandom([seed])">

Class that uses the `os.urandom` function for generating random numbers from sources provided by the operating system. Not available on all systems. Does not rely on software state, and sequences are not reproducible. Accordingly, the `seed` method has no effect and is ignored. The `getstate` and `setstate` methods raise `NotImplementedError` if called.

</div>

## Notes on Reproducibility

Sometimes it is useful to be able to reproduce the sequences given by a pseudo-random number generator. By reusing a seed value, the same sequence should be reproducible from run to run as long as multiple threads are not running.

Most of the random module's algorithms and seeding functions are subject to change across Python versions, but two aspects are guaranteed not to change:

- If a new seeding method is added, then a backward compatible seeder will be offered.
- The generator's `~Random.random` method will continue to produce the same sequence when the compatible seeder is given the same seed.

## Examples

Basic examples:

    >>> random()                          # Random float:  0.0 <= x < 1.0
    0.37444887175646646

    >>> uniform(2.5, 10.0)                # Random float:  2.5 <= x <= 10.0
    3.1800146073117523

    >>> expovariate(1 / 5)                # Interval between arrivals averaging 5 seconds
    5.148957571865031

    >>> randrange(10)                     # Integer from 0 to 9 inclusive
    7

    >>> randrange(0, 101, 2)              # Even integer from 0 to 100 inclusive
    26

    >>> choice(['win', 'lose', 'draw'])   # Single random element from a sequence
    'draw'

    >>> deck = 'ace two three four'.split()
    >>> shuffle(deck)                     # Shuffle a list
    >>> deck
    ['four', 'two', 'ace', 'three']

    >>> sample([10, 20, 30, 40, 50], k=4) # Four samples without replacement
    [40, 10, 50, 30]

Simulations:

    >>> # Six roulette wheel spins (weighted sampling with replacement)
    >>> choices(['red', 'black', 'green'], [18, 18, 2], k=6)
    ['red', 'green', 'black', 'black', 'red', 'black']

    >>> # Deal 20 cards without replacement from a deck
    >>> # of 52 playing cards, and determine the proportion of cards
    >>> # with a ten-value:  ten, jack, queen, or king.
    >>> deal = sample(['tens', 'low cards'], counts=[16, 36], k=20)
    >>> deal.count('tens') / 20
    0.15

    >>> # Estimate the probability of getting 5 or more heads from 7 spins
    >>> # of a biased coin that settles on heads 60% of the time.
    >>> sum(binomialvariate(n=7, p=0.6) >= 5 for i in range(10_000)) / 10_000
    0.4169

    >>> # Probability of the median of 5 samples being in middle two quartiles
    >>> def trial():
    ...     return 2_500 <= sorted(choices(range(10_000), k=5))[2] < 7_500
    ...
    >>> sum(trial() for i in range(10_000)) / 10_000
    0.7958

Example of [statistical bootstrapping](https://en.wikipedia.org/wiki/Bootstrapping_(statistics)) using resampling with replacement to estimate a confidence interval for the mean of a sample:

    # https://www.thoughtco.com/example-of-bootstrapping-3126155
    from statistics import fmean as mean
    from random import choices

    data = [41, 50, 29, 37, 81, 30, 73, 63, 20, 35, 68, 22, 60, 31, 95]
    means = sorted(mean(choices(data, k=len(data))) for i in range(100))
    print(f'The sample mean of {mean(data):.1f} has a 90% confidence '
          f'interval from {means[5]:.1f} to {means[94]:.1f}')

Example of a [resampling permutation test](https://en.wikipedia.org/wiki/Resampling_(statistics)#Permutation_tests) to determine the statistical significance or [p-value](https://en.wikipedia.org/wiki/P-value) of an observed difference between the effects of a drug versus a placebo:

    # Example from "Statistics is Easy" by Dennis Shasha and Manda Wilson
    from statistics import fmean as mean
    from random import shuffle

    drug = [54, 73, 53, 70, 73, 68, 52, 65, 65]
    placebo = [54, 51, 58, 44, 55, 52, 42, 47, 58, 46]
    observed_diff = mean(drug) - mean(placebo)

    n = 10_000
    count = 0
    combined = drug + placebo
    for i in range(n):
        shuffle(combined)
        new_diff = mean(combined[:len(drug)]) - mean(combined[len(drug):])
        count += (new_diff >= observed_diff)

    print(f'{n} label reshufflings produced only {count} instances with a difference')
    print(f'at least as extreme as the observed difference of {observed_diff:.1f}.')
    print(f'The one-sided p-value of {count / n:.4f} leads us to reject the null')
    print(f'hypothesis that there is no difference between the drug and the placebo.')

Simulation of arrival times and service deliveries for a multiserver queue:

    from heapq import heapify, heapreplace
    from random import expovariate, gauss
    from statistics import mean, quantiles

    average_arrival_interval = 5.6
    average_service_time = 15.0
    stdev_service_time = 3.5
    num_servers = 3

    waits = []
    arrival_time = 0.0
    servers = [0.0] * num_servers  # time when each server becomes available
    heapify(servers)
    for i in range(1_000_000):
        arrival_time += expovariate(1.0 / average_arrival_interval)
        next_server_available = servers[0]
        wait = max(0.0, next_server_available - arrival_time)
        waits.append(wait)
        service_duration = max(0.0, gauss(average_service_time, stdev_service_time))
        service_completed = arrival_time + wait + service_duration
        heapreplace(servers, service_completed)

    print(f'Mean wait: {mean(waits):.1f}   Max wait: {max(waits):.1f}')
    print('Quartiles:', [round(q, 1) for q in quantiles(waits)])

<div class="seealso">

[Statistics for Hackers](https://www.youtube.com/watch?v=Iq9DzN6mvYA) a video tutorial by [Jake Vanderplas](https://us.pycon.org/2016/speaker/profile/295/) on statistical analysis using just a few fundamental concepts including simulation, sampling, shuffling, and cross-validation.

[Economics Simulation](https://nbviewer.org/url/norvig.com/ipython/Economics.ipynb) a simulation of a marketplace by [Peter Norvig](https://norvig.com/bio.html) that shows effective use of many of the tools and distributions provided by this module (gauss, uniform, sample, betavariate, choice, triangular, and randrange).

[A Concrete Introduction to Probability (using Python)](https://nbviewer.org/url/norvig.com/ipython/Probability.ipynb) a tutorial by [Peter Norvig](https://norvig.com/bio.html) covering the basics of probability theory, how to write simulations, and how to perform data analysis using Python.

</div>

## Recipes

These recipes show how to efficiently make random selections from the combinatoric iterators in the `itertools` module or the `more-itertools` project:

<div class="testcode">

import random

def random_product(*iterables, repeat=1): "Random selection from itertools.product(*iterables, repeat=repeat)"  
pools = tuple(map(tuple, iterables)) \* repeat return tuple(map(random.choice, pools))

def random_permutation(iterable, r=None):  
"Random selection from itertools.permutations(iterable, r)" pool = tuple(iterable) r = len(pool) if r is None else r return tuple(random.sample(pool, r))

def random_combination(iterable, r):  
"Random selection from itertools.combinations(iterable, r)" pool = tuple(iterable) n = len(pool) indices = sorted(random.sample(range(n), r)) return tuple(pool\[i\] for i in indices)

def random_combination_with_replacement(iterable, r):  
"Choose r elements with replacement. Order the result to match the iterable." \# Result will be in set(itertools.combinations_with_replacement(iterable, r)). pool = tuple(iterable) n = len(pool) indices = sorted(random.choices(range(n), k=r)) return tuple(pool\[i\] for i in indices)

def random_derangement(iterable):  
"Choose a permutation where no element stays in its original position." seq = tuple(iterable) if len(seq) \< 2: if not seq: return () raise IndexError('No derangments to choose from') perm = list(range(len(seq))) start = tuple(perm) while True: random.shuffle(perm) if all(p != q for p, q in zip(start, perm)): return tuple(\[seq\[i\] for i in perm\])

</div>

<div class="doctest" data-hide="">

\>\>\> import random

\>\>\> random.seed(8675309) \>\>\> random_product('ABCDEFG', repeat=5) ('D', 'B', 'E', 'F', 'E')

\>\>\> random.seed(8675309) \>\>\> random_permutation('ABCDEFG') ('D', 'B', 'E', 'C', 'G', 'A', 'F') \>\>\> random_permutation('ABCDEFG', 5) ('A', 'G', 'D', 'C', 'B')

\>\>\> random.seed(8675309) \>\>\> random_combination('ABCDEFG', 7) ('A', 'B', 'C', 'D', 'E', 'F', 'G') \>\>\> random_combination('ABCDEFG', 6) ('A', 'B', 'C', 'D', 'F', 'G') \>\>\> random_combination('ABCDEFG', 5) ('A', 'B', 'C', 'E', 'F') \>\>\> random_combination('ABCDEFG', 4) ('B', 'C', 'D', 'G') \>\>\> random_combination('ABCDEFG', 3) ('B', 'E', 'G') \>\>\> random_combination('ABCDEFG', 2) ('E', 'G') \>\>\> random_combination('ABCDEFG', 1) ('C',) \>\>\> random_combination('ABCDEFG', 0) ()

\>\>\> random.seed(8675309) \>\>\> random_combination_with_replacement('ABCDEFG', 7) ('B', 'C', 'D', 'E', 'E', 'E', 'G') \>\>\> random_combination_with_replacement('ABCDEFG', 3) ('A', 'B', 'E') \>\>\> random_combination_with_replacement('ABCDEFG', 2) ('A', 'G') \>\>\> random_combination_with_replacement('ABCDEFG', 1) ('E',) \>\>\> random_combination_with_replacement('ABCDEFG', 0) ()

\>\>\> random.seed(8675309) \>\>\> random_derangement('') () \>\>\> random_derangement('A') Traceback (most recent call last): ... IndexError: No derangments to choose from \>\>\> random_derangement('AB') ('B', 'A') \>\>\> random_derangement('ABC') ('C', 'A', 'B') \>\>\> random_derangement('ABCD') ('B', 'A', 'D', 'C') \>\>\> random_derangement('ABCDE') ('B', 'C', 'A', 'E', 'D') \>\>\> \# Identical inputs treated as distinct \>\>\> identical = 20 \>\>\> random_derangement((10, identical, 30, identical)) (20, 30, 10, 20)

</div>

The default `.random` returns multiples of 2⁻⁵³ in the range *0.0 ≤ x \< 1.0*. All such numbers are evenly spaced and are exactly representable as Python floats. However, many other representable floats in that interval are not possible selections. For example, `0.05954861408025609` isn't an integer multiple of 2⁻⁵³.

The following recipe takes a different approach. All floats in the interval are possible selections. The mantissa comes from a uniform distribution of integers in the range *2⁵² ≤ mantissa \< 2⁵³*. The exponent comes from a geometric distribution where exponents smaller than *-53* occur half as often as the next larger exponent.

    from random import Random
    from math import ldexp

    class FullRandom(Random):

        def random(self):
            mantissa = 0x10_0000_0000_0000 | self.getrandbits(52)
            exponent = -53
            x = 0
            while not x:
                x = self.getrandbits(32)
                exponent += x.bit_length() - 32
            return ldexp(mantissa, exponent)

All `real valued distributions <real-valued-distributions>` in the class will use the new method:

    >>> fr = FullRandom()
    >>> fr.random()
    0.05954861408025609
    >>> fr.expovariate(0.25)
    8.87925541791544

The recipe is conceptually equivalent to an algorithm that chooses from all the multiples of 2⁻¹⁰⁷⁴ in the range *0.0 ≤ x \< 1.0*. All such numbers are evenly spaced, but most have to be rounded down to the nearest representable Python float. (The value 2⁻¹⁰⁷⁴ is the smallest positive unnormalized float and is equal to `math.ulp(0.0)`.)

<div class="seealso">

[Generating Pseudo-random Floating-Point Values](https://allendowney.com/research/rand/downey07randfloat.pdf) a paper by Allen B. Downey describing ways to generate more fine-grained floats than normally generated by `.random`.

</div>

## Command-line usage

<div class="versionadded">

3.13

</div>

The `!random` module can be executed from the command line.

``` sh
python -m random [-h] [-c CHOICE [CHOICE ...] | -i N | -f N] [input ...]
```

The following options are accepted:

<div class="program">

random

</div>

<div class="option">

`-h, --help`  

Show the help message and exit.

</div>

<div class="option">

-c CHOICE \[CHOICE ...\] --choice CHOICE \[CHOICE ...\]

Print a random choice, using `choice`.

</div>

<div class="option">

`-i <N>`  

`--integer <N>`  

Print a random integer between 1 and N inclusive, using `randint`.

</div>

<div class="option">

`-f <N>`  

`--float <N>`  

Print a random floating-point number between 0 and N inclusive, using `uniform`.

</div>

If no options are given, the output depends on the input:

- String or multiple: same as `--choice`.
- Integer: same as `--integer`.
- Float: same as `--float`.

## Command-line example

Here are some examples of the `!random` command-line interface:

```
$ # Choose one at random
$ python -m random egg bacon sausage spam "Lobster Thermidor aux crevettes with a Mornay sauce"
Lobster Thermidor aux crevettes with a Mornay sauce

$ # Random integer
$ python -m random 6
6

$ # Random floating-point number
$ python -m random 1.8
1.7080016272295635

$ # With explicit arguments
$ python  -m random --choice egg bacon sausage spam "Lobster Thermidor aux crevettes with a Mornay sauce"
egg

$ python -m random --integer 6
3

$ python -m random --float 1.8
1.5666339105010318

$ python -m random --integer 6
5

$ python -m random --float 6
3.1942323316565915
```

# re

tocdepth  
3

# `!re` --- Regular expression operations

<div class="module" data-synopsis="Regular expression operations.">

re

</div>

**Source code:** `Lib/re/`

------------------------------------------------------------------------

This module provides regular expression matching operations similar to those found in Perl.

Both patterns and strings to be searched can be Unicode strings (`str`) as well as 8-bit strings (`bytes`). However, Unicode strings and 8-bit strings cannot be mixed: that is, you cannot match a Unicode string with a bytes pattern or vice-versa; similarly, when asking for a substitution, the replacement string must be of the same type as both the pattern and the search string.

Regular expressions use the backslash character (`'\'`) to indicate special forms or to allow special characters to be used without invoking their special meaning. This collides with Python's usage of the same character for the same purpose in string literals; for example, to match a literal backslash, one might have to write `'\\\\'` as the pattern string, because the regular expression must be `\\`, and each backslash must be expressed as `\\` inside a regular Python string literal. Also, please note that any invalid escape sequences in Python's usage of the backslash in string literals now generate a `SyntaxWarning` and in the future this will become a `SyntaxError`. This behaviour will happen even if it is a valid escape sequence for a regular expression.

The solution is to use Python's raw string notation for regular expression patterns; backslashes are not handled in any special way in a string literal prefixed with `'r'`. So `r"\n"` is a two-character string containing `'\'` and `'n'`, while `"\n"` is a one-character string containing a newline. Usually patterns will be expressed in Python code using this raw string notation.

It is important to note that most regular expression operations are available as module-level functions and methods on `compiled regular expressions <re-objects>`. The functions are shortcuts that don't require you to compile a regex object first, but miss some fine-tuning parameters.

<div class="seealso">

The third-party `regex` module, which has an API compatible with the standard library `!re` module, but offers additional functionality and a more thorough Unicode support.

</div>

## Regular expression syntax

A regular expression (or RE) specifies a set of strings that matches it; the functions in this module let you check if a particular string matches a given regular expression (or if a given regular expression matches a particular string, which comes down to the same thing).

Regular expressions can be concatenated to form new regular expressions; if *A* and *B* are both regular expressions, then *AB* is also a regular expression. In general, if a string *p* matches *A* and another string *q* matches *B*, the string *pq* will match AB. This holds unless *A* or *B* contain low precedence operations; boundary conditions between *A* and *B*; or have numbered group references. Thus, complex expressions can easily be constructed from simpler primitive expressions like the ones described here. For details of the theory and implementation of regular expressions, consult the Friedl book <a href="#Frie09" class="citation">[Frie09]</a>, or almost any textbook about compiler construction.

A brief explanation of the format of regular expressions follows. For further information and a gentler presentation, consult the `regex-howto`.

Regular expressions can contain both special and ordinary characters. Most ordinary characters, like `'A'`, `'a'`, or `'0'`, are the simplest regular expressions; they simply match themselves. You can concatenate ordinary characters, so `last` matches the string `'last'`. (In the rest of this section, we'll write RE's in `this special style`, usually without quotes, and strings to be matched `'in single quotes'`.)

Some characters, like `'|'` or `'('`, are special. Special characters either stand for classes of ordinary characters, or affect how the regular expressions around them are interpreted.

Repetition operators or quantifiers (`*`, `+`, `?`, `{m,n}`, etc) cannot be directly nested. This avoids ambiguity with the non-greedy modifier suffix `?`, and with other modifiers in other implementations. To apply a second repetition to an inner repetition, parentheses may be used. For example, the expression `(?:a{6})*` matches any multiple of six `'a'` characters.

### Special characters

The special characters are:

<div id="re-syntax-dot" class="index">

single: . (dot); in regular expressions

</div>

#### `.`

(Dot.) In the default mode, this matches any character except a newline. If the `DOTALL` flag has been specified, this matches any character including a newline. `(?s:.)` matches any character regardless of flags.

<div id="re-syntax-caret" class="index">

single: ^ (caret); in regular expressions

</div>

#### `^`

(Caret.) Matches the start of the string, and in `MULTILINE` mode also matches immediately after each newline.

<div id="re-syntax-dollar" class="index">

single: \$ (dollar); in regular expressions

</div>

#### `$`

Matches the end of the string or just before the newline at the end of the string, and in `MULTILINE` mode also matches before a newline. `foo` matches both 'foo' and 'foobar', while the regular expression `foo$` matches only 'foo'. More interestingly, searching for `foo.$` in `'foo1\nfoo2\n'` matches 'foo2' normally, but 'foo1' in `MULTILINE` mode; searching for a single `$` in `'foo\n'` will find two (empty) matches: one just before the newline, and one at the end of the string.

<div id="re-syntax-asterisk" class="index">

single: \* (asterisk); in regular expressions

</div>

#### `*`

Causes the resulting RE to match 0 or more repetitions of the preceding RE, as many repetitions as are possible. `ab*` will match 'a', 'ab', or 'a' followed by any number of 'b's.

<div id="re-syntax-plus" class="index">

single: + (plus); in regular expressions

</div>

#### `+`

Causes the resulting RE to match 1 or more repetitions of the preceding RE. `ab+` will match 'a' followed by any non-zero number of 'b's; it will not match just 'a'.

<div id="re-syntax-question-mark" class="index">

single: ? (question mark); in regular expressions

</div>

#### `?`

Causes the resulting RE to match 0 or 1 repetitions of the preceding RE. `ab?` will match either 'a' or 'ab'.

<div id="re-syntax-non-greedy-quantifiers" class="index">

single: \*?; in regular expressions single: +?; in regular expressions single: ??; in regular expressions

</div>

#### `*?`, `+?`, `??`

The `'*'`, `'+'`, and `'?'` quantifiers are all `greedy`; they match as much text as possible. Sometimes this behaviour isn't desired; if the RE `<.*>` is matched against `'<a> b <c>'`, it will match the entire string, and not just `'<a>'`. Adding `?` after the quantifier makes it perform the match in `non-greedy` or `minimal` fashion; as *few* characters as possible will be matched. Using the RE `<.*?>` will match only `'<a>'`.

<div id="re-syntax-possessive-quantifiers" class="index">

single: \*+; in regular expressions single: ++; in regular expressions single: ?+; in regular expressions

</div>

#### `*+`, `++`, `?+`

Like the `'*'`, `'+'`, and `'?'` quantifiers, those where `'+'` is appended also match as many times as possible. However, unlike the true greedy quantifiers, these do not allow back-tracking when the expression following it fails to match. These are known as `possessive` quantifiers. For example, `a*a` will match `'aaaa'` because the `a*` will match all 4 `'a'`s, but, when the final `'a'` is encountered, the expression is backtracked so that in the end the `a*` ends up matching 3 `'a'`s total, and the fourth `'a'` is matched by the final `'a'`. However, when `a*+a` is used to match `'aaaa'`, the `a*+` will match all 4 `'a'`, but when the final `'a'` fails to find any more characters to match, the expression cannot be backtracked and will thus fail to match. `x*+`, `x++` and `x?+` are equivalent to `(?>x*)`, `(?>x+)` and `(?>x?)` correspondingly.

<div class="versionadded">

3.11

</div>

<div id="re-syntax-curly-brackets" class="index">

single: {} (curly brackets); in regular expressions

</div>

#### `{m}`

Specifies that exactly *m* copies of the previous RE should be matched; fewer matches cause the entire RE not to match. For example, `a{6}` will match exactly six `'a'` characters, but not five.

#### `{m,n}`

Causes the resulting RE to match from *m* to *n* repetitions of the preceding RE, attempting to match as many repetitions as possible. For example, `a{3,5}` will match from 3 to 5 `'a'` characters. Omitting *m* specifies a lower bound of zero, and omitting *n* specifies an infinite upper bound. As an example, `a{4,}b` will match `'aaaab'` or a thousand `'a'` characters followed by a `'b'`, but not `'aaab'`. The comma may not be omitted or the modifier would be confused with the previously described form.

#### `{m,n}?`

Causes the resulting RE to match from *m* to *n* repetitions of the preceding RE, attempting to match as *few* repetitions as possible. This is the non-greedy version of the previous quantifier. For example, on the 6-character string `'aaaaaa'`, `a{3,5}` will match 5 `'a'` characters, while `a{3,5}?` will only match 3 characters.

#### `{m,n}+`

Causes the resulting RE to match from *m* to *n* repetitions of the preceding RE, attempting to match as many repetitions as possible *without* establishing any backtracking points. This is the possessive version of the quantifier above. For example, on the 6-character string `'aaaaaa'`, `a{3,5}+aa` attempts to match 5 `'a'` characters, then, requiring 2 more `'a'`s, will need more characters than available and thus fail, while `a{3,5}aa` will match with `a{3,5}` capturing 5, then 4 `'a'`s by backtracking and then the final 2 `'a'`s are matched by the final `aa` in the pattern. `x{m,n}+` is equivalent to `(?>x{m,n})`.

<div class="versionadded">

3.11

</div>

<div id="re-syntax-backslash" class="index">

single: (backslash); in regular expressions

</div>

#### `\`

Either escapes special characters (permitting you to match characters like `'*'`, `'?'`, and so forth), or signals a special sequence; special sequences are discussed below.

If you're not using a raw string to express the pattern, remember that Python also uses the backslash as an escape sequence in string literals; if the escape sequence isn't recognized by Python's parser, the backslash and subsequent character are included in the resulting string. However, if Python would recognize the resulting sequence, the backslash should be repeated twice. This is complicated and hard to understand, so it's highly recommended that you use raw strings for all but the simplest expressions.

<div id="re-syntax-square-brackets" class="index">

single: \[\] (square brackets); in regular expressions

</div>

#### `[]`

Used to indicate a set of characters. In a set:

- Characters can be listed individually, e.g. `[amk]` will match `'a'`, `'m'`, or `'k'`.

<div id="re-syntax-square-brackets-minus" class="index">

single: - (minus); in regular expressions

</div>

- Ranges of characters can be indicated by giving two characters and separating them by a `'-'`, for example `[a-z]` will match any lowercase ASCII letter, `[0-5][0-9]` will match all the two-digits numbers from `00` to `59`, and `[0-9A-Fa-f]` will match any hexadecimal digit. If `-` is escaped (e.g. `[a\-z]`) or if it's placed as the first or last character (e.g. `[-a]` or `[a-]`), it will match a literal `'-'`.
- Special characters except backslash lose their special meaning inside sets. For example, `[(+*)]` will match any of the literal characters `'('`, `'+'`, `'*'`, or `')'`.

<div id="re-syntax-square-brackets-backslash" class="index">

single: (backslash); in regular expressions

</div>

- Backslash either escapes characters which have special meaning in a set such as `'-'`, `']'`, `'^'` and `'\\'` itself or signals a special sequence which represents a single character such as `\xa0` or `\n` or a character class such as `\w` or `\S` (defined below). Note that `\b` represents a single "backspace" character, not a word boundary as outside a set, and numeric escapes such as `\1` are always octal escapes, not group references. Special sequences which do not match a single character such as `\A` and `\z` are not allowed.

<div id="re-syntax-square-brackets-caret" class="index">

single: ^ (caret); in regular expressions

</div>

- Characters that are not within a range can be matched by `complementing` the set. If the first character of the set is `'^'`, all the characters that are *not* in the set will be matched. For example, `[^5]` will match any character except `'5'`, and `[^^]` will match any character except `'^'`. `^` has no special meaning if it's not the first character in the set.
- To match a literal `']'` inside a set, precede it with a backslash, or place it at the beginning of the set. For example, both `[()[\]{}]` and `[]()[{}]` will match a right bracket, as well as left bracket, braces, and parentheses.

<div id="re-syntax-square-brackets-nested-sets" class="index">

single: --; in regular expressions single: &&; in regular expressions single: \|\|; in regular expressions

</div>

- A character set may contain a nested set written in square brackets, and two sets may be combined with a set operator, as in [Unicode Technical Standard \#18](https://unicode.org/reports/tr18/):

  - `[A--B]` (*difference*) matches a character that is in *A* but not in *B*; for example `[a-z--[aeiou]]` matches an ASCII lowercase consonant.
  - `[A&&B]` (*intersection*) matches a character that is in both *A* and *B*; for example `[\w&&[a-z]]` matches an ASCII lowercase letter.
  - `[A||B]` (*union*) matches a character that is in *A* or in *B*; this is the same as listing the members of both sets in a single set, but allows combining nested sets.

  Operators have no precedence and are applied from left to right. To group, write a nested set as the operand after an operator, as in `[a-z--[aeiou]]`. A leading `'^'` complements the whole result. A `'['` begins a nested set only immediately after a set operator; anywhere else -- including at the start of a character set -- it is an ordinary character, so existing patterns keep their meaning. Escape it as `'\['` to include a literal `'['` right after an operator.

> [!NOTE]
> Symmetric difference (`A~~B`) is not yet supported; a literal `'~~'` in a character set still raises a `FutureWarning`.

<div class="versionchanged">

3.7 `FutureWarning` is raised if a character set contains constructs that will change semantically in the future.

</div>

<div class="versionchanged">

3.16 Added support for nested sets and the set operators `--`, `&&` and `||`.

</div>

<div id="re-syntax-vertical-bar" class="index">

single: \| (vertical bar); in regular expressions

</div>

#### `|`

`A|B`, where *A* and *B* can be arbitrary REs, creates a regular expression that will match either *A* or *B*. An arbitrary number of REs can be separated by the `'|'` in this way. This can be used inside groups (see below) as well. As the target string is scanned, REs separated by `'|'` are tried from left to right. When one pattern completely matches, that branch is accepted. This means that once *A* matches, *B* will not be tested further, even if it would produce a longer overall match. In other words, the `'|'` operator is never greedy. To match a literal `'|'`, use `\|`, or enclose it inside a character class, as in `[|]`.

<div id="re-syntax-parentheses" class="index">

single: () (parentheses); in regular expressions

</div>

#### `(...)`

Matches whatever regular expression is inside the parentheses, and indicates the start and end of a group; the contents of a group can be retrieved after a match has been performed, and can be matched later in the string with the `\number` special sequence, described below. To match the literals `'('` or `')'`, use `\(` or `\)`, or enclose them inside a character class: `[(]`, `[)]`.

<div id="re-syntax-extension" class="index">

single: (?; in regular expressions

</div>

#### `(?...)`

This is an extension notation (a `'?'` following a `'('` is not meaningful otherwise). The first character after the `'?'` determines what the meaning and further syntax of the construct is. Extensions usually do not create a new group; `(?P<name>...)` is the only exception to this rule. Following are the currently supported extensions.

#### `(?aiLmsux)`

(One or more letters from the set `'a'`, `'i'`, `'L'`, `'m'`, `'s'`, `'u'`, `'x'`.) The group matches the empty string; the letters set the corresponding flags for the entire regular expression:

- `re.A` (ASCII-only matching)
- `re.I` (ignore case)
- `re.L` (locale dependent)
- `re.M` (multi-line)
- `re.S` (dot matches all)
- `re.U` (Unicode matching)
- `re.X` (verbose)

(The flags are described in `contents-of-module-re`.) This is useful if you wish to include the flags as part of the regular expression, instead of passing a *flag* argument to the `re.compile` function. Flags should be used first in the expression string.

<div class="versionchanged">

3.11 This construction can only be used at the start of the expression.

</div>

<div id="re-syntax-non-capturing" class="index">

single: (?:; in regular expressions

</div>

#### `(?:...)`

A non-capturing version of regular parentheses. Matches whatever regular expression is inside the parentheses, but the substring matched by the group *cannot* be retrieved after performing a match or referenced later in the pattern.

#### `(?aiLmsux-imsx:...)`

(Zero or more letters from the set `'a'`, `'i'`, `'L'`, `'m'`, `'s'`, `'u'`, `'x'`, optionally followed by `'-'` followed by one or more letters from the `'i'`, `'m'`, `'s'`, `'x'`.) The letters set or remove the corresponding flags for the part of the expression:

- `re.A` (ASCII-only matching)
- `re.I` (ignore case)
- `re.L` (locale dependent)
- `re.M` (multi-line)
- `re.S` (dot matches all)
- `re.U` (Unicode matching)
- `re.X` (verbose)

(The flags are described in `contents-of-module-re`.)

The letters `'a'`, `'L'` and `'u'` are mutually exclusive when used as inline flags, so they can't be combined or follow `'-'`. Instead, when one of them appears in an inline group, it overrides the matching mode in the enclosing group. In Unicode patterns `(?a:...)` switches to ASCII-only matching, and `(?u:...)` switches to Unicode matching (default). In bytes patterns `(?L:...)` switches to locale dependent matching, and `(?a:...)` switches to ASCII-only matching (default). This override is only in effect for the narrow inline group, and the original matching mode is restored outside of the group.

<div class="versionadded">

3.6

</div>

<div class="versionchanged">

3.7 The letters `'a'`, `'L'` and `'u'` also can be used in a group.

</div>

<div id="re-syntax-atomic-group" class="index">

single: (?\>; in regular expressions

</div>

#### `(?>...)`

Attempts to match `...` as if it was a separate regular expression, and if successful, continues to match the rest of the pattern following it. If the subsequent pattern fails to match, the stack can only be unwound to a point *before* the `(?>...)` because once exited, the expression, known as an `atomic group`, has thrown away all stack points within itself. Thus, `(?>.*).` would never match anything because first the `.*` would match all characters possible, then, having nothing left to match, the final `.` would fail to match. Since there are no stack points saved in the Atomic Group, and there is no stack point before it, the entire expression would thus fail to match.

<div class="versionadded">

3.11

</div>

<div id="re-syntax-named-capture" class="index">

single: (?P\<; in regular expressions

</div>

#### `(?P<name>...)`

Similar to regular parentheses, but the substring matched by the group is accessible via the symbolic group name *name*. Group names must be valid Python identifiers, and in `bytes` patterns they can only contain bytes in the ASCII range. Each group name must be defined only once within a regular expression. A symbolic group is also a numbered group, just as if the group were not named.

Named groups can be referenced in three contexts. If the pattern is `(?P<quote>['"]).*?(?P=quote)` (i.e. matching a string quoted with either single or double quotes):

<table>
<colgroup>
<col style="width: 53%" />
<col style="width: 46%" />
</colgroup>
<thead>
<tr>
<th><p>Context of reference to group "quote"</p></th>
<th><p>Ways to reference it</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p>in the same pattern itself</p></td>
<td><ul>
<li><code>(?P=quote)</code> (as shown)</li>
<li><code>\1</code></li>
</ul></td>
</tr>
<tr>
<td><p>when processing match object <em>m</em></p></td>
<td><ul>
<li><code>m.group('quote')</code></li>
<li><code>m.end('quote')</code> (etc.)</li>
</ul></td>
</tr>
<tr>
<td><p>in a string passed to the <em>repl</em> argument of <code>re.sub()</code></p></td>
<td><ul>
<li><code>\g&lt;quote&gt;</code></li>
<li><code>\g&lt;1&gt;</code></li>
<li><code>\1</code></li>
</ul></td>
</tr>
</tbody>
</table>

<div class="versionchanged">

3.12 In `bytes` patterns, group *name* can only contain bytes in the ASCII range (`b'\x00'`-`b'\x7f'`).

</div>

<div id="re-syntax-named-backreference" class="index">

single: (?P=; in regular expressions

</div>

#### `(?P=name)`

A backreference to a named group; it matches whatever text was matched by the earlier group named *name*.

<div id="re-syntax-comment" class="index">

single: (?#; in regular expressions

</div>

#### `(?#...)`

A comment; the contents of the parentheses are simply ignored.

<div id="re-syntax-lookahead" class="index">

single: (?=; in regular expressions

</div>

#### `(?=...)`

Matches if `...` matches next, but doesn't consume any of the string. This is called a `lookahead assertion`. For example, `Isaac (?=Asimov)` will match `'Isaac '` only if it's followed by `'Asimov'`.

<div id="re-syntax-negative-lookahead" class="index">

single: (?!; in regular expressions

</div>

#### `(?!...)`

Matches if `...` doesn't match next. This is a `negative lookahead assertion`. For example, `Isaac (?!Asimov)` will match `'Isaac '` only if it's *not* followed by `'Asimov'`.

<div id="re-syntax-lookbehind" class="index">

single: (?\<=; in regular expressions

</div>

#### `(?<=...)`

Matches if the current position in the string is preceded by a match for `...` that ends at the current position. This is called a `positive lookbehind
assertion`. `(?<=abc)def` will find a match in `'abcdef'`, since the lookbehind will back up 3 characters and check if the contained pattern matches. The contained pattern must only match strings of some fixed length, meaning that `abc` or `a|b` are allowed, but `a*` and `a{3,4}` are not. Note that patterns which start with positive lookbehind assertions will not match at the beginning of the string being searched; you will most likely want to use the `search` function rather than the `match` function:

> \>\>\> import re \>\>\> m = re.search('(?\<=abc)def', 'abcdef') \>\>\> m.group(0) 'def'

This example looks for a word following a hyphen:

> \>\>\> m = re.search(r'(?\<=-)w+', 'spam-egg') \>\>\> m.group(0) 'egg'

<div class="versionchanged">

3.5 Added support for group references of fixed length.

</div>

<div id="re-syntax-negative-lookbehind" class="index">

single: (?\<!; in regular expressions

</div>

#### `(?<!...)`

Matches if the current position in the string is not preceded by a match for `...`. This is called a `negative lookbehind assertion`. Similar to positive lookbehind assertions, the contained pattern must only match strings of some fixed length. Patterns which start with negative lookbehind assertions may match at the beginning of the string being searched.

#### `(?(id/name)yes-pattern|no-pattern)`

Will try to match with `yes-pattern` if the group with given *id* or *name* exists, and with `no-pattern` if it doesn't. `no-pattern` is optional and can be omitted. For example, `(<)?(\w+@\w+(?:\.\w+)+)(?(1)>|$)` is a poor email matching pattern, which matches `'<user@host.com>'` as well as `'user@host.com'`, but does not match `'<user@host.com'` nor `'user@host.com>'` in their entirety (`re.search` finds only `'user@host.com'` in the former).

<div class="versionchanged">

3.12 Group *id* can only contain ASCII digits. In `bytes` patterns, group *name* can only contain bytes in the ASCII range (`b'\x00'`-`b'\x7f'`).

</div>

### Special sequences

The special sequences consist of `'\'` and a character from the list below. If the ordinary character is not an ASCII digit or an ASCII letter, then the resulting RE will match the second character. For example, `\$` matches the character `'$'`.

<div id="re-syntax-special-group-reference" class="index">

single: (backslash); in regular expressions

</div>

#### `\number`

Matches the contents of the group of the same number. Groups are numbered starting from 1. For example, `(.+) \1` matches `'the the'` or `'55 55'`, but not `'thethe'` (note the space after the group). This special sequence can only be used to match one of the first 99 groups. If the first digit of *number* is 0, or *number* is 3 octal digits long, it will not be interpreted as a group match, but as the character with octal value *number*. Inside the `'['` and `']'` of a character class, all numeric escapes are treated as characters.

<div id="re-syntax-special-start" class="index">

single: A; in regular expressions

</div>

#### `\A`

Matches only at the start of the string.

<div id="re-syntax-special-word-boundary" class="index">

single: b; in regular expressions

</div>

#### `\b`

Matches the empty string, but only at the beginning or end of a word. A word is defined as a sequence of word characters. Note that formally, `\b` is defined as the boundary between a `\w` and a `\W` character (or vice versa), or between `\w` and the beginning or end of the string. This means that `r'\bat\b'` matches `'at'`, `'at.'`, `'(at)'`, and `'as at ay'` but not `'attempt'` or `'atlas'`.

The default word characters in Unicode (str) patterns are Unicode alphanumerics and the underscore, but this can be changed by using the `~re.ASCII` flag. Word boundaries are determined by the current locale if the `~re.LOCALE` flag is used.

> [!NOTE]
> Inside a character range, `\b` represents the backspace character, for compatibility with Python's string literals.

<div id="re-syntax-special-inside-word" class="index">

single: B; in regular expressions

</div>

#### `\B`

Matches the empty string, but only when it is *not* at the beginning or end of a word. This means that `r'at\B'` matches `'athens'`, `'atom'`, `'attorney'`, but not `'at'`, `'at.'`, or `'at!'`. `\B` is the opposite of `\b`, so word characters in Unicode (str) patterns are Unicode alphanumerics or the underscore, although this can be changed by using the `~re.ASCII` flag. Word boundaries are determined by the current locale if the `~re.LOCALE` flag is used.

<div class="versionchanged">

3.14 `\B` now matches empty input string.

</div>

<div id="re-syntax-special-digits" class="index">

single: d; in regular expressions

</div>

#### `\d`

For Unicode (str) patterns:  
Matches any Unicode decimal digit (that is, any character in Unicode character category [\[Nd\]](https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-4/#G124142)). This includes `[0-9]`, and also many other digit characters.

Matches `[0-9]` if the `~re.ASCII` flag is used.

For 8-bit (bytes) patterns:  
Matches any decimal digit in the ASCII character set; this is equivalent to `[0-9]`.

<div id="re-syntax-special-non-digits" class="index">

single: D; in regular expressions

</div>

#### `\D`

Matches any character which is not a decimal digit. This is the opposite of `\d`.

Matches `[^0-9]` if the `~re.ASCII` flag is used.

<div id="re-syntax-special-whitespace" class="index">

single: s; in regular expressions

</div>

#### `\s`

For Unicode (str) patterns:  
Matches Unicode whitespace characters (as defined by `str.isspace`). This includes `[ \t\n\r\f\v]`, and also many other characters, for example the non-breaking spaces mandated by typography rules in many languages.

Matches `[ \t\n\r\f\v]` if the `~re.ASCII` flag is used.

For 8-bit (bytes) patterns:  
Matches characters considered whitespace in the ASCII character set; this is equivalent to `[ \t\n\r\f\v]`.

<div id="re-syntax-special-non-whitespace" class="index">

single: S; in regular expressions

</div>

#### `\S`

Matches any character which is not a whitespace character. This is the opposite of `\s`.

Matches `[^ \t\n\r\f\v]` if the `~re.ASCII` flag is used.

<div id="re-syntax-special-word" class="index">

single: w; in regular expressions

</div>

#### `\w`

For Unicode (str) patterns:  
Matches Unicode word characters; this includes all Unicode alphanumeric characters (as defined by `str.isalnum`), as well as the underscore (`_`).

Matches `[a-zA-Z0-9_]` if the `~re.ASCII` flag is used.

For 8-bit (bytes) patterns:  
Matches characters considered alphanumeric in the ASCII character set; this is equivalent to `[a-zA-Z0-9_]`. If the `~re.LOCALE` flag is used, matches characters considered alphanumeric in the current locale and the underscore.

<div id="re-syntax-special-non-word" class="index">

single: W; in regular expressions

</div>

#### `\W`

Matches any character which is not a word character. This is the opposite of `\w`. By default, matches non-underscore (`_`) characters for which `str.isalnum` returns `False`.

Matches `[^a-zA-Z0-9_]` if the `~re.ASCII` flag is used.

If the `~re.LOCALE` flag is used, matches characters which are neither alphanumeric in the current locale nor the underscore.

<div id="re-syntax-special-unicode-property" class="index">

single: p; in regular expressions single: P; in regular expressions

</div>

#### `\p{property=value}`, `\p{value}`

Matches any character with the given Unicode property (see [Unicode Technical Standard \#18](https://unicode.org/reports/tr18/), requirement RL1.2 "Properties"). Property and value names are matched loosely: case, whitespace, `'-'` and `'_'` are ignored. The following properties are supported:

- The `General_Category` property (short name `gc`), spelled `\p{Lu}`, `\p{gc=Lu}` or, for a one-letter group, `\p{L}`. The supported values are the groups `L`, `N`, `Z` and `C` and the values `Lu`, `Lt`, `Lm`, `Nd`, `Nl`, `No`, `Zs`, `Zl`, `Zp`, `Cc`, `Cf`, `Cs`, `Co` and `Cn`.
- The binary properties `XID_Start`, `XID_Continue`, `Alphabetic`, `Lowercase`, `Uppercase`, `Numeric`, `Printable`, `Cased` and `Case_Ignorable`. A binary property may also be spelled `\p{name=yes}` or `\p{name=no}`.
- The POSIX compatibility classes `alpha`, `alnum`, `blank`, `cntrl`, `digit`, `graph`, `lower`, `print`, `space`, `upper`, `word` and `xdigit`.
- The properties `ASCII`, `Any`, `Assigned`, `Noncharacter_Code_Point`, `Join_Control`, `Regional_Indicator`, `ASCII_Hex_Digit`, `Hex_Digit`, `Pattern_Syntax` and `Pattern_White_Space`.

Where a supported property corresponds to a `unicodedata` accessor or `str` method, the set of characters it matches is exactly the one they report. For consistency with these, `space` follows `str.isspace` (like `\s`) and `xdigit` matches only the ASCII hexadecimal digits.

This is only recognized in Unicode (str) patterns. In bytes patterns it is an error.

<div class="versionadded">

3.16

</div>

#### `\P{...}`

Matches any character which does *not* have the given Unicode property. This is the opposite of `\p`.

<div class="versionadded">

3.16

</div>

<div id="re-syntax-special-end" class="index">

single: z; in regular expressions single: Z; in regular expressions

</div>

#### `\z`

Matches only at the end of the string.

<div class="versionadded">

3.14

</div>

#### `\Z`

The same as `\z`. For compatibility with old Python versions.

<div id="re-syntax-standard-escapes" class="index">

single: a; in regular expressions single: b; in regular expressions single: f; in regular expressions single: n; in regular expressions single: N; in regular expressions single: r; in regular expressions single: t; in regular expressions single: u; in regular expressions single: U; in regular expressions single: v; in regular expressions single: x; in regular expressions single: \\; in regular expressions

</div>

### Standard escapes

Most of the `escape sequences <escape-sequences>` supported by Python string literals are also accepted by the regular expression parser:

    \a      \b      \f      \n
    \N      \r      \t      \u
    \U      \v      \x      \\

(Note that `\b` is used to represent word boundaries, and means "backspace" only inside character classes.)

`'\u'`, `'\U'`, and `'\N'` escape sequences are only recognized in Unicode (str) patterns. In bytes patterns they are errors. Unknown escapes of ASCII letters are reserved for future use and treated as errors.

Octal escapes are included in a limited form. If the first digit is a 0, or if there are three octal digits, it is considered an octal escape. Otherwise, it is a group reference. As for string literals, octal escapes are always at most three digits in length.

<div class="versionchanged">

3.3 The `'\u'` and `'\U'` escape sequences have been added.

</div>

<div class="versionchanged">

3.6 Unknown escapes consisting of `'\'` and an ASCII letter now are errors.

</div>

<div class="versionchanged">

3.8 The `'\\N\\{{name}\\}'` escape sequence has been added. As in string literals, it expands to the named Unicode character (e.g. `'\N{EM DASH}'`).

</div>

## Module contents

The module defines several functions, constants, and an exception. Some of the functions are simplified versions of the full featured methods for compiled regular expressions. Most non-trivial applications always use the compiled form.

### Flags

<div class="versionchanged">

3.6 Flag constants are now instances of `RegexFlag`, which is a subclass of `enum.IntFlag`.

</div>

<div class="RegexFlag">

An `enum.IntFlag` class containing the regex options listed below.

<div class="versionadded">

3.11 - added to `__all__`

</div>

</div>

<div class="data">

A ASCII

Make `\w`, `\W`, `\b`, `\B`, `\d`, `\D`, `\s` and `\S` perform ASCII-only matching instead of full Unicode matching. This is only meaningful for Unicode (str) patterns, and is ignored for bytes patterns.

Corresponds to the inline flag `(?a)`.

> [!NOTE]
> The `~re.U` flag still exists for backward compatibility, but is redundant in Python 3 since matches are Unicode by default for `str` patterns, and Unicode matching isn't allowed for bytes patterns. `~re.UNICODE` and the inline flag `(?u)` are similarly redundant.

</div>

<div class="data">

DEBUG

Display debug information about compiled expression.

No corresponding inline flag.

</div>

<div class="data">

I IGNORECASE

Perform case-insensitive matching; expressions like `[A-Z]` will also match lowercase letters. Full Unicode matching (such as `Ü` matching `ü`) also works unless the `~re.ASCII` flag is used to disable non-ASCII matches. The current locale does not change the effect of this flag unless the `~re.LOCALE` flag is also used.

Corresponds to the inline flag `(?i)`.

Note that when the Unicode patterns `[a-z]` or `[A-Z]` are used in combination with the `IGNORECASE` flag, they will match the 52 ASCII letters and 4 additional non-ASCII letters: 'İ' (U+0130, Latin capital letter I with dot above), 'ı' (U+0131, Latin small letter dotless i), 'ſ' (U+017F, Latin small letter long s) and 'K' (U+212A, Kelvin sign). If the `~re.ASCII` flag is used, only letters 'a' to 'z' and 'A' to 'Z' are matched.

</div>

<div class="data">

L LOCALE

Make `\w`, `\W`, `\b`, `\B` and case-insensitive matching dependent on the current locale. This flag can be used only with bytes patterns.

Corresponds to the inline flag `(?L)`.

> [!WARNING]
> This flag is discouraged; consider Unicode matching instead. The locale mechanism is very unreliable as it only handles one "culture" at a time and only works with 8-bit locales. Unicode matching is enabled by default for Unicode (str) patterns and it is able to handle different locales and languages.

<div class="versionchanged">

3.6 `~re.LOCALE` can be used only with bytes patterns and is not compatible with `~re.ASCII`.

</div>

<div class="versionchanged">

3.7 Compiled regular expression objects with the `~re.LOCALE` flag no longer depend on the locale at compile time. Only the locale at matching time affects the result of matching.

</div>

</div>

<div class="data">

M MULTILINE

When specified, the pattern character `'^'` matches at the beginning of the string and at the beginning of each line (immediately following each newline); and the pattern character `'$'` matches at the end of the string and at the end of each line (immediately preceding each newline). By default, `'^'` matches only at the beginning of the string, and `'$'` only at the end of the string and immediately before the newline (if any) at the end of the string.

Corresponds to the inline flag `(?m)`.

</div>

<div class="data">

NOFLAG

Indicates no flag being applied, the value is `0`. This flag may be used as a default value for a function keyword argument or as a base value that will be conditionally ORed with other flags. Example of use as a default value:

    def myfunc(pattern, text, flag=re.NOFLAG):
        return re.search(pattern, text, flag)

<div class="versionadded">

3.11

</div>

</div>

<div class="data">

S DOTALL

Make the `'.'` special character match any character at all, including a newline; without this flag, `'.'` will match anything *except* a newline.

Corresponds to the inline flag `(?s)`.

</div>

<div class="data">

U UNICODE

In Python 3, Unicode characters are matched by default for `str` patterns. This flag is therefore redundant with **no effect** and is only kept for backward compatibility.

See `~re.ASCII` to restrict matching to ASCII characters instead.

</div>

<div class="data">

X VERBOSE

<div class="index">

single: \# (hash); in regular expressions

</div>

This flag allows you to write regular expressions that look nicer and are more readable by allowing you to visually separate logical sections of the pattern and add comments. Whitespace within the pattern is ignored, except when in a character class, or when preceded by an unescaped backslash, or within tokens like `*?`, `(?:` or `(?P<...>`. For example, `(? :` and `* ?` are not allowed. When a line contains a `#` that is not in a character class and is not preceded by an unescaped backslash, all characters from the leftmost such `#` through the end of the line are ignored.

This means that the two following regular expression objects that match a decimal number are functionally equal:

    a = re.compile(r"""\d +  # the integral part
                       \.    # the decimal point
                       \d *  # some fractional digits""", re.X)
    b = re.compile(r"\d+\.\d*")

Corresponds to the inline flag `(?x)`.

</div>

### Functions

<div class="function">

compile(pattern, flags=0)

Compile a regular expression pattern into a `regular expression object
<re-objects>`, which can be used for matching using its `~Pattern.prefixmatch`, `~Pattern.search`, and other methods, described below.

The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the [flags](#flags) variables, combined using bitwise OR (the `|` operator).

The sequence :

    prog = re.compile(pattern)
    result = prog.search(string)

is equivalent to :

    result = re.search(pattern, string)

but using `re.compile` and saving the resulting regular expression object for reuse is more efficient when the expression will be used several times in a single program.

> [!NOTE]
> The compiled versions of the most recent patterns passed to `re.compile` and the module-level matching functions are cached, so programs that use only a few regular expressions at a time needn't worry about compiling regular expressions.

</div>

<div class="function">

search(pattern, string, flags=0)

Scan through *string* looking for the first location where the regular expression *pattern* produces a match, and return a corresponding `~re.Match`. Return `None` if no position in the string matches the pattern; note that this is different from finding a zero-length match at some point in the string.

The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the [flags](#flags) variables, combined using bitwise OR (the `|` operator).

</div>

<div class="function">

prefixmatch(pattern, string, flags=0)

If zero or more characters at the beginning of *string* match the regular expression *pattern*, return a corresponding `~re.Match`. Return `None` if the string does not match the pattern; note that this is different from a zero-length match.

> [!NOTE]
> Even in `MULTILINE` mode, this will only match at the beginning of the string and not at the beginning of each line.

If you want to locate a match anywhere in *string*, use `search` instead (see also `search-vs-match`).

The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the [flags](#flags) variables, combined using bitwise OR (the `|` operator).

This function now has two names and has long been known as `~re.match`. Use that name when you need to retain compatibility with older Python versions.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

match(pattern, string, flags=0)

<div class="soft-deprecated">

3.15 `~re.match` has been `soft deprecated` in favor of the alternate `~re.prefixmatch` name of this API which is more explicitly descriptive. Use it to better express intent. The norm in other languages and regular expression implementations is to use the term *match* to refer to the behavior of what Python has always called `~re.search`. See `prefixmatch-vs-match`.

</div>

</div>

<div class="function">

fullmatch(pattern, string, flags=0)

If the whole *string* matches the regular expression *pattern*, return a corresponding `~re.Match`. Return `None` if the string does not match the pattern; note that this is different from a zero-length match.

The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the [flags](#flags) variables, combined using bitwise OR (the `|` operator).

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

split(pattern, string, maxsplit=0, flags=0)

Split *string* by the occurrences of *pattern*. If capturing parentheses are used in *pattern*, then the text of all groups in the pattern are also returned as part of the resulting list. If *maxsplit* is nonzero, at most *maxsplit* splits occur, and the remainder of the string is returned as the final element of the list. :

    >>> re.split(r'\W+', 'Words, words, words.')
    ['Words', 'words', 'words', '']
    >>> re.split(r'(\W+)', 'Words, words, words.')
    ['Words', ', ', 'words', ', ', 'words', '.', '']
    >>> re.split(r'\W+', 'Words, words, words.', maxsplit=1)
    ['Words', 'words, words.']
    >>> re.split('[a-f]+', '0a3B9', flags=re.IGNORECASE)
    ['0', '3', '9']

If there are capturing groups in the separator and it matches at the start of the string, the result will start with an empty string. The same holds for the end of the string:

    >>> re.split(r'(\W+)', '...words, words...')
    ['', '...', 'words', ', ', 'words', '...', '']

That way, separator components are always found at the same relative indices within the result list.

Adjacent empty matches are not possible, but an empty match can occur immediately after a non-empty match.

```
>>> re.split(r'\b', 'Words, words, words.')
['', 'Words', ', ', 'words', ', ', 'words', '.']
>>> re.split(r'\W*', '...words...')
['', '', 'w', 'o', 'r', 'd', 's', '', '']
>>> re.split(r'(\W*)', '...words...')
['', '...', '', '', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '', '', '']
```

The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the [flags](#flags) variables, combined using bitwise OR (the `|` operator).

<div class="versionchanged">

3.1 Added the optional flags argument.

</div>

<div class="versionchanged">

3.7 Added support of splitting on a pattern that could match an empty string.

</div>

<div class="deprecated">

3.13 Passing *maxsplit* and *flags* as positional arguments is deprecated. In future Python versions they will be `keyword-only parameters <keyword-only_parameter>`.

</div>

</div>

<div class="function">

findall(pattern, string, flags=0)

Return all non-overlapping matches of *pattern* in *string*, as a list of strings or tuples. The *string* is scanned left-to-right, and matches are returned in the order found. Empty matches are included in the result.

The result depends on the number of capturing groups in the pattern. If there are no groups, return a list of strings matching the whole pattern. If there is exactly one group, return a list of strings matching that group. If multiple groups are present, return a list of tuples of strings matching the groups. Non-capturing groups do not affect the form of the result.

> \>\>\> re.findall(r'bf\[a-z\]\*', 'which foot or hand fell fastest') \['foot', 'fell', 'fastest'\] \>\>\> re.findall(r'(w+)=(d+)', 'set width=20 and height=10') \[('width', '20'), ('height', '10')\]

The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the [flags](#flags) variables, combined using bitwise OR (the `|` operator).

<div class="versionchanged">

3.7 Non-empty matches can now start just after a previous empty match.

</div>

</div>

<div class="function">

finditer(pattern, string, flags=0)

Return an `iterator` yielding `~re.Match` objects over all non-overlapping matches for the RE *pattern* in *string*. The *string* is scanned left-to-right, and matches are returned in the order found. Empty matches are included in the result.

The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the [flags](#flags) variables, combined using bitwise OR (the `|` operator).

<div class="versionchanged">

3.7 Non-empty matches can now start just after a previous empty match.

</div>

</div>

<div class="function">

sub(pattern, repl, string, count=0, flags=0)

Return the string obtained by replacing the leftmost non-overlapping occurrences of *pattern* in *string* by the replacement *repl*. If the pattern isn't found, *string* is returned unchanged. *repl* can be a string or a function; if it is a string, any backslash escapes in it are processed. That is, `\n` is converted to a single newline character, `\r` is converted to a carriage return, and so forth. Unknown escapes of ASCII letters are reserved for future use and treated as errors. Other unknown escapes such as `\&` are left alone. Backreferences, such as `\6`, are replaced with the substring matched by group 6 in the pattern. For example:

    >>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
    ...        r'static PyObject*\npy_\1(void)\n{',
    ...        'def myfunc():')
    'static PyObject*\npy_myfunc(void)\n{'

If *repl* is a function, it is called for every non-overlapping occurrence of *pattern*. The function takes a single `~re.Match` argument, and returns the replacement string. For example:

    >>> def dashrepl(matchobj):
    ...     if matchobj.group(0) == '-': return ' '
    ...     else: return '-'
    ...
    >>> re.sub('-{1,2}', dashrepl, 'pro----gram-files')
    'pro--gram files'
    >>> re.sub(r'\sAND\s', ' & ', 'Baked Beans And Spam', flags=re.IGNORECASE)
    'Baked Beans & Spam'

The pattern may be a string or a `~re.Pattern`.

The optional argument *count* is the maximum number of pattern occurrences to be replaced; *count* must be a non-negative integer. If omitted or zero, all occurrences will be replaced.

Adjacent empty matches are not possible, but an empty match can occur immediately after a non-empty match. As a result, `sub('x*', '-', 'abxd')` returns `'-a-b--d-'` instead of `'-a-b-d-'`.

<div class="index">

single: g; in regular expressions

</div>

In string-type *repl* arguments, in addition to the character escapes and backreferences described above, `\g<name>` will use the substring matched by the group named `name`, as defined by the `(?P<name>...)` syntax. `\g<number>` uses the corresponding group number; `\g<2>` is therefore equivalent to `\2`, but isn't ambiguous in a replacement such as `\g<2>0`. `\20` would be interpreted as a reference to group 20, not a reference to group 2 followed by the literal character `'0'`. The backreference `\g<0>` substitutes in the entire substring matched by the RE.

The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the [flags](#flags) variables, combined using bitwise OR (the `|` operator).

<div class="versionchanged">

3.1 Added the optional flags argument.

</div>

<div class="versionchanged">

3.5 Unmatched groups are replaced with an empty string.

</div>

<div class="versionchanged">

3.6 Unknown escapes in *pattern* consisting of `'\'` and an ASCII letter now are errors.

</div>

<div class="versionchanged">

3.7 Unknown escapes in *repl* consisting of `'\'` and an ASCII letter now are errors. An empty match can occur immediately after a non-empty match.

</div>

<div class="versionchanged">

3.12 Group *id* can only contain ASCII digits. In `bytes` replacement strings, group *name* can only contain bytes in the ASCII range (`b'\x00'`-`b'\x7f'`).

</div>

<div class="deprecated">

3.13 Passing *count* and *flags* as positional arguments is deprecated. In future Python versions they will be `keyword-only parameters <keyword-only_parameter>`.

</div>

</div>

<div class="function">

subn(pattern, repl, string, count=0, flags=0)

Perform the same operation as `sub`, but return a tuple `(new_string, number_of_subs_made)`.

The expression's behaviour can be modified by specifying a *flags* value. Values can be any of the [flags](#flags) variables, combined using bitwise OR (the `|` operator).

</div>

<div class="function">

escape(pattern)

Escape special characters in *pattern*. This is useful if you want to match an arbitrary literal string that may have regular expression metacharacters in it. For example:

    >>> print(re.escape('https://www.python.org'))
    https://www\.python\.org

    >>> legal_chars = string.ascii_lowercase + string.digits + "!#$%&'*+-.^_`|~:"
    >>> print('[%s]+' % re.escape(legal_chars))
    [abcdefghijklmnopqrstuvwxyz0123456789!\#\$%\&'\*\+\-\.\^_`\|\~:]+

    >>> operators = ['+', '-', '*', '/', '**']
    >>> print('|'.join(map(re.escape, sorted(operators, reverse=True))))
    /|\-|\+|\*\*|\*

This function must not be used for the replacement string in `sub` and `subn`, only backslashes should be escaped. For example:

    >>> digits_re = r'\d+'
    >>> sample = '/usr/sbin/sendmail - 0 errors, 12 warnings'
    >>> print(re.sub(digits_re, digits_re.replace('\\', r'\\'), sample))
    /usr/sbin/sendmail - \d+ errors, \d+ warnings

<div class="versionchanged">

3.3 The `'_'` character is no longer escaped.

</div>

<div class="versionchanged">

3.7 Only characters that can have special meaning in a regular expression are escaped. As a result, `'!'`, `'"'`, `'%'`, `"'"`, `','`, `'/'`, `':'`, `';'`, `'<'`, `'='`, `'>'`, `'@'`, and `` "`" `` are no longer escaped.

</div>

</div>

<div class="function">

purge()

Clear the regular expression cache.

</div>

### Exceptions

<div class="exception">

PatternError(msg, pattern=None, pos=None)

Exception raised when a string passed to one of the functions here is not a valid regular expression (for example, it might contain unmatched parentheses) or when some other error occurs during compilation or matching. It is never an error if a string contains no match for a pattern. The `PatternError` instance has the following additional attributes:

<div class="attribute">

msg

The unformatted error message.

</div>

<div class="attribute">

pattern

The regular expression pattern.

</div>

<div class="attribute">

pos

The index in *pattern* where compilation failed (may be `None`).

</div>

<div class="attribute">

lineno

The line corresponding to *pos* (may be `None`).

</div>

<div class="attribute">

colno

The column corresponding to *pos* (may be `None`).

</div>

<div class="versionchanged">

3.5 Added additional attributes.

</div>

<div class="versionchanged">

3.13 `PatternError` was originally named `error`; the latter is kept as an alias for backward compatibility.

</div>

</div>

## Regular expression objects

<div class="Pattern">

Compiled regular expression object returned by `re.compile`.

Patterns are `generic <generics>` over the type of string they handle (`str` or `bytes`).

<div class="versionchanged">

3.9 `re.Pattern` supports `[]` to indicate a Unicode (str) or bytes pattern. See `types-genericalias`.

</div>

</div>

<div class="method">

Pattern.search(string\[, pos\[, endpos\]\])

Scan through *string* looking for the first location where this regular expression produces a match, and return a corresponding `~re.Match`. Return `None` if no position in the string matches the pattern; note that this is different from finding a zero-length match at some point in the string.

The optional second parameter *pos* gives an index in the string where the search is to start; it defaults to `0`. This is not completely equivalent to slicing the string; the `'^'` pattern character matches at the real beginning of the string and at positions just after a newline, but not necessarily at the index where the search is to start.

The optional parameter *endpos* limits how far the string will be searched; it will be as if the string is *endpos* characters long, so only the characters from *pos* to `endpos - 1` will be searched for a match. If *endpos* is less than *pos*, no match will be found; otherwise, if *rx* is a compiled regular expression object, `rx.search(string, 0, 50)` is equivalent to `rx.search(string[:50], 0)`. :

    >>> pattern = re.compile("d")
    >>> pattern.search("dog")     # Match at index 0
    <re.Match object; span=(0, 1), match='d'>
    >>> pattern.search("dog", 1)  # No match; search doesn't include the "d"

</div>

<div class="method">

Pattern.prefixmatch(string\[, pos\[, endpos\]\])

If zero or more characters at the *beginning* of *string* match this regular expression, return a corresponding `~re.Match`. Return `None` if the string does not match the pattern; note that this is different from a zero-length match.

Note that even in `MULTILINE` mode, this will only match at the beginning of the string and not at the beginning of each line.

The optional *pos* and *endpos* parameters have the same meaning as for the `~Pattern.search` method. :

    >>> pattern = re.compile("o")
    >>> pattern.prefixmatch("dog")     # No match as "o" is not at the start of "dog".
    >>> pattern.prefixmatch("dog", 1)  # Match as "o" is the 2nd character of "dog".
    <re.Match object; span=(1, 2), match='o'>

If you want to locate a match anywhere in *string*, use `~Pattern.search` instead (see also `search-vs-match`).

This method now has two names and has long been known as `~Pattern.match`. Use that name when you need to retain compatibility with older Python versions.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

Pattern.match(string\[, pos\[, endpos\]\])

<div class="soft-deprecated">

3.15 `~Pattern.match` has been `soft deprecated` in favor of the alternate `~Pattern.prefixmatch` name of this API which is more explicitly descriptive. Use it to better express intent. The norm in other languages and regular expression implementations is to use the term *match* to refer to the behavior of what Python has always called `~Pattern.search`. See `prefixmatch-vs-match`.

</div>

</div>

<div class="method">

Pattern.fullmatch(string\[, pos\[, endpos\]\])

If the whole *string* matches this regular expression, return a corresponding `~re.Match`. Return `None` if the string does not match the pattern; note that this is different from a zero-length match.

The optional *pos* and *endpos* parameters have the same meaning as for the `~Pattern.search` method. :

    >>> pattern = re.compile("o[gh]")
    >>> pattern.fullmatch("dog")      # No match as "o" is not at the start of "dog".
    >>> pattern.fullmatch("ogre")     # No match as not the full string matches.
    >>> pattern.fullmatch("doggie", 1, 3)   # Matches within given limits.
    <re.Match object; span=(1, 3), match='og'>

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

Pattern.split(string, maxsplit=0)

Identical to the `split` function, using the compiled pattern.

</div>

<div class="method">

Pattern.findall(string\[, pos\[, endpos\]\])

Similar to the `findall` function, using the compiled pattern, but also accepts optional *pos* and *endpos* parameters that limit the search region like for `search`.

</div>

<div class="method">

Pattern.finditer(string\[, pos\[, endpos\]\])

Similar to the `finditer` function, using the compiled pattern, but also accepts optional *pos* and *endpos* parameters that limit the search region like for `search`.

</div>

<div class="method">

Pattern.sub(repl, string, count=0)

Identical to the `sub` function, using the compiled pattern.

</div>

<div class="method">

Pattern.subn(repl, string, count=0)

Identical to the `subn` function, using the compiled pattern.

</div>

<div class="attribute">

Pattern.flags

The regex matching flags. This is a combination of the flags given to `.compile`, any `(?...)` inline flags in the pattern, and implicit flags such as `~re.UNICODE` if the pattern is a Unicode string.

</div>

<div class="attribute">

Pattern.groups

The number of capturing groups in the pattern.

</div>

<div class="attribute">

Pattern.groupindex

A dictionary mapping any symbolic group names defined by `(?P<id>)` to group numbers. The dictionary is empty if no symbolic groups were used in the pattern.

</div>

<div class="attribute">

Pattern.pattern

The pattern string from which the pattern object was compiled.

</div>

<div class="versionchanged">

3.7 Added support of `copy.copy` and `copy.deepcopy`. Compiled regular expression objects are considered atomic.

</div>

## Match objects

Match objects always have a boolean value of `True`. Since `~Pattern.match` and `~Pattern.search` return `None` when there is no match, you can test whether there was a match with a simple `if` statement:

    if match := re.search(pattern, string):
        process(match)

<div class="Match">

Match object returned by successful `match`es and `search`es.

Matches are `generic <generics>` over the type of string which was matched (`str` or `bytes`).

<div class="versionchanged">

3.9 `re.Match` supports `[]` to indicate a Unicode (str) or bytes match. See `types-genericalias`.

</div>

</div>

<div class="method">

Match.expand(template)

Return the string obtained by doing backslash substitution on the template string *template*, as done by the `~Pattern.sub` method. Escapes such as `\n` are converted to the appropriate characters, and numeric backreferences (`\1`, `\2`) and named backreferences (`\g<1>`, `\g<name>`) are replaced by the contents of the corresponding group. The backreference `\g<0>` will be replaced by the entire match.

<div class="versionchanged">

3.5 Unmatched groups are replaced with an empty string.

</div>

</div>

<div class="method">

Match.group(\[group1, ...\])

Returns one or more subgroups of the match. If there is a single argument, the result is a single string; if there are multiple arguments, the result is a tuple with one item per argument. Without arguments, *group1* defaults to zero (the whole match is returned). If a *groupN* argument is zero, the corresponding return value is the entire matching string; if it is a positive integer, it is the string matching the corresponding parenthesized group. If a group number is negative or larger than the number of groups defined in the pattern, an `IndexError` exception is raised. If a group is contained in a part of the pattern that did not match, the corresponding result is `None`. If a group is contained in a part of the pattern that matched multiple times, the last match is returned. :

    >>> m = re.search(r"\A(\w+) (\w+)", "Norwegian Blue, pining for the fjords")
    >>> m.group(0)       # The entire match
    'Norwegian Blue'
    >>> m.group(1)       # The first parenthesized subgroup.
    'Norwegian'
    >>> m.group(2)       # The second parenthesized subgroup.
    'Blue'
    >>> m.group(1, 2)    # Multiple arguments give us a tuple.
    ('Norwegian', 'Blue')

If the regular expression uses the `(?P<name>...)` syntax, the *groupN* arguments may also be strings identifying groups by their group name. If a string argument is not used as a group name in the pattern, an `IndexError` exception is raised.

A moderately complicated example:

    >>> m = re.search(r"(?P<adjective>\w+) (?P<animal>\w+)", "killer rabbit")
    >>> m.group('adjective')
    'killer'
    >>> m.group('animal')
    'rabbit'

Named groups can also be referred to by their index:

    >>> m.group(1)
    'killer'
    >>> m.group(2)
    'rabbit'

If a group matches multiple times, only the last match is accessible:

    >>> m = re.search(r"(..)+", "a1b2c3")  # Matches 3 times.
    >>> m.group(1)                         # Returns only the last match.
    'c3'

</div>

<div class="method">

Match.\_\_getitem\_\_(g)

This is identical to `m.group(g)`. This allows easier access to an individual group from a match:

    >>> m = re.search(r"(\w+) (\w+)", "Norwegian Blue, pining for the fjords")
    >>> m[0]       # The entire match
    'Norwegian Blue'
    >>> m[1]       # The first parenthesized subgroup.
    'Norwegian'
    >>> m[2]       # The second parenthesized subgroup.
    'Blue'

Named groups are supported as well:

    >>> m = re.search(r"(?P<adjective>\w+) (?P<animal>\w+)", "killer rabbit")
    >>> m['adjective']
    'killer'
    >>> m['animal']
    'rabbit'

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

Match.groups(default=None)

Return a tuple containing all the subgroups of the match, from 1 up to however many groups are in the pattern. The *default* argument is used for groups that did not participate in the match; it defaults to `None`.

For example:

    >>> m = re.search(r"(\d+)\.(\d+)", "24.1632")
    >>> m.groups()
    ('24', '1632')

If we make the decimal place and everything after it optional, not all groups might participate in the match. These groups will default to `None` unless the *default* argument is given:

    >>> m = re.search(r"(\d+)\.?(\d+)?", "24")
    >>> m.groups()      # Second group defaults to None.
    ('24', None)
    >>> m.groups('0')   # Now, the second group defaults to '0'.
    ('24', '0')

</div>

<div class="method">

Match.groupdict(default=None)

Return a dictionary containing all the *named* subgroups of the match, keyed by the subgroup name. The *default* argument is used for groups that did not participate in the match; it defaults to `None`. For example:

    >>> m = re.search(r"(?P<adjective>\w+) (?P<animal>\w+)", "killer rabbit")
    >>> m.groupdict()
    {'adjective': 'killer', 'animal': 'rabbit'}

</div>

<div class="method">

Match.start(\[group\]) Match.end(\[group\])

Return the indices of the start and end of the substring matched by *group*; *group* defaults to zero (meaning the whole matched substring). Return `-1` if *group* exists but did not contribute to the match. For a match object *m*, and a group *g* that did contribute to the match, the substring matched by group *g* (equivalent to `m.group(g)`) is :

    m.string[m.start(g):m.end(g)]

Note that `m.start(group)` will equal `m.end(group)` if *group* matched a null string. For example, after `m = re.search('b(c?)', 'cba')`, `m.start(0)` is 1, `m.end(0)` is 2, `m.start(1)` and `m.end(1)` are both 2, and `m.start(2)` raises an `IndexError` exception.

An example that will remove *remove_this* from email addresses:

    >>> email = "tony@tiremove_thisger.net"
    >>> m = re.search("remove_this", email)
    >>> email[:m.start()] + email[m.end():]
    'tony@tiger.net'

</div>

<div class="method">

Match.span(\[group\])

For a match *m*, return the 2-tuple `(m.start(group), m.end(group))`. Note that if *group* did not contribute to the match, this is `(-1, -1)`. *group* defaults to zero, the entire match.

</div>

<div class="attribute">

Match.pos

The value of *pos* which was passed to the `~Pattern.search` or `~Pattern.match` method of a `regex object <re-objects>`. This is the index into the string at which the RE engine started looking for a match.

</div>

<div class="attribute">

Match.endpos

The value of *endpos* which was passed to the `~Pattern.search` or `~Pattern.match` method of a `regex object <re-objects>`. This is the index into the string beyond which the RE engine will not go.

</div>

<div class="attribute">

Match.lastindex

The integer index of the last matched capturing group, or `None` if no group was matched at all. For example, the expressions `(a)b`, `((a)(b))`, and `((ab))` will have `lastindex == 1` if applied to the string `'ab'`, while the expression `(a)(b)` will have `lastindex == 2`, if applied to the same string.

</div>

<div class="attribute">

Match.lastgroup

The name of the last matched capturing group, or `None` if the group didn't have a name, or if no group was matched at all.

</div>

<div class="attribute">

Match.re

The `regular expression object <re-objects>` whose `~Pattern.match` or `~Pattern.search` method produced this match instance.

</div>

<div class="attribute">

Match.string

The string passed to `~Pattern.match` or `~Pattern.search`.

</div>

<div class="versionchanged">

3.7 Added support of `copy.copy` and `copy.deepcopy`. Match objects are considered atomic.

</div>

## Regular expression examples

### Checking for a pair

In this example, we'll use the following helper function to display match objects a little more gracefully:

    def displaymatch(match):
        if match is None:
            return None
        return '<Match: %r, groups=%r>' % (match.group(), match.groups())

Suppose you are writing a poker program where a player's hand is represented as a 5-character string with each character representing a card, "a" for ace, "k" for king, "q" for queen, "j" for jack, "t" for 10, and "2" through "9" representing the card with that value.

To see if a given string is a valid hand, one could do the following:

    >>> valid_hand_re = re.compile(r"^[a2-9tjqk]{5}$")
    >>> displaymatch(valid_hand_re.search("akt5q"))  # Valid.
    "<Match: 'akt5q', groups=()>"
    >>> displaymatch(valid_hand_re.search("akt5e"))  # Invalid.
    >>> displaymatch(valid_hand_re.search("akt"))    # Invalid.
    >>> displaymatch(valid_hand_re.search("727ak"))  # Valid.
    "<Match: '727ak', groups=()>"

That last hand, `"727ak"`, contained a pair, or two of the same valued cards. To match this with a regular expression, one could use backreferences as such:

    >>> pair_re = re.compile(r".*(.).*\1")
    >>> displaymatch(pair_re.prefixmatch("717ak"))     # Pair of 7s.
    "<Match: '717', groups=('7',)>"
    >>> displaymatch(pair_re.prefixmatch("718ak"))     # No pairs.
    >>> displaymatch(pair_re.prefixmatch("354aa"))     # Pair of aces.
    "<Match: '354aa', groups=('a',)>"

To find out what card the pair consists of, one could use the `~Match.group` method of the match object in the following manner:

    >>> pair_re = re.compile(r".*(.).*\1")
    >>> pair_re.prefixmatch("717ak").group(1)
    '7'

    # Error because prefixmatch() returns None, which doesn't have a group() method:
    >>> pair_re.prefixmatch("718ak").group(1)
    Traceback (most recent call last):
      File "<pyshell#23>", line 1, in <module>
        pair_re.prefixmatch("718ak").group(1)
    AttributeError: 'NoneType' object has no attribute 'group'

    >>> pair_re.prefixmatch("354aa").group(1)
    'a'

### Simulating scanf()

<div class="index">

single: scanf (C function)

</div>

Python does not currently have an equivalent to `!scanf`. Regular expressions are generally more powerful, though also more verbose, than `!scanf` format strings. The table below offers some more-or-less equivalent mappings between `!scanf` format tokens and regular expressions.

| `!scanf` Token         | Regular Expression                        |
|------------------------|-------------------------------------------|
| `%c`                   | `.`                                       |
| `%5c`                  | `.{5}`                                    |
| `%d`                   | `[-+]?\d+`                                |
| `%e`, `%E`, `%f`, `%g` | `[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?` |
| `%i`                   | `[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`     |
| `%o`                   | `[-+]?[0-7]+`                             |
| `%s`                   | `\S+`                                     |
| `%u`                   | `\d+`                                     |
| `%x`, `%X`             | `[-+]?(0[xX])?[\dA-Fa-f]+`                |

To extract the filename and numbers from a string like:

```
/usr/sbin/sendmail - 0 errors, 4 warnings
```

you would use a `!scanf` format like:

```
%s - %d errors, %d warnings
```

The equivalent regular expression would be:

```
(\S+) - (\d+) errors, (\d+) warnings
```

### search() vs. prefixmatch()

Python offers different primitive operations based on regular expressions:

- `re.prefixmatch` checks for a match only at the beginning of the string
- `re.search` checks for a match anywhere in the string (this is what Perl does by default)
- `re.fullmatch` checks for entire string to be a match

For example:

    >>> re.prefixmatch("c", "abcdef")  # No match
    >>> re.search("c", "abcdef")       # Match
    <re.Match object; span=(2, 3), match='c'>
    >>> re.fullmatch("p.*n", "python") # Match
    <re.Match object; span=(0, 6), match='python'>
    >>> re.fullmatch("r.*n", "python") # No match

Regular expressions beginning with `'^'` can be used with `search` to restrict the match at the beginning of the string:

    >>> re.prefixmatch("c", "abcdef")  # No match
    >>> re.search("^c", "abcdef")      # No match
    >>> re.search("^a", "abcdef")      # Match
    <re.Match object; span=(0, 1), match='a'>

Note however that in `MULTILINE` mode `prefixmatch` only matches at the beginning of the string, whereas using `search` with a regular expression beginning with `'^'` will match at the beginning of each line. :

    >>> re.prefixmatch("X", "A\nB\nX", re.MULTILINE)  # No match
    >>> re.search("^X", "A\nB\nX", re.MULTILINE)  # Match
    <re.Match object; span=(4, 5), match='X'>

### prefixmatch() vs. match()

Why is the `~re.match` function and method discouraged in favor of the longer `~re.prefixmatch` spelling?

Many other languages have gained regex support libraries since regular expressions were added to Python. However in the most popular of those, they use the term *match* in their APIs to mean the unanchored behavior provided in Python by `~re.search`. Thus use of the plain term *match* can be unclear to those used to other languages when reading or writing code and not familiar with the Python API's divergence from what otherwise become the industry norm.

Quoting from the Zen Of Python (`python3 -m this`): *"Explicit is better than implicit"*. Anyone reading the name `!prefixmatch` is likely to understand the intended semantics. When reading `!match` there remains a seed of doubt about the intended behavior to anyone not already familiar with this old Python gotcha.

We **do not** plan to remove the older `!match` name, as it has been used in code for over 30 years. It has been `soft deprecated`: code supporting older versions of Python should continue to use `!match`, while new code should prefer `!prefixmatch`.

<div class="versionadded">

3.15 `!prefixmatch`

</div>

<div class="soft-deprecated">

3.15 `!match`

</div>

### Making a phonebook

`split` splits a string into a list delimited by the passed pattern. The method is invaluable for converting textual data into data structures that can be easily read and modified by Python as demonstrated in the following example that creates a phonebook.

First, here is the input. Normally it may come from a file, here we are using triple-quoted string syntax

<div class="doctest">

\>\>\> text = """Ross McFluff: 834.345.1254 155 Elm Street ... ... Ronald Heathmore: 892.345.3428 436 Finley Avenue ... Frank Burger: 925.541.7625 662 South Dogwood Way ... ... ... Heather Albrecht: 548.326.4584 919 Park Place"""

</div>

The entries are separated by one or more newlines. Now we convert the string into a list with each nonempty line having its own entry:

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> entries = re.split("n+", text) \>\>\> entries \['Ross McFluff: 834.345.1254 155 Elm Street', 'Ronald Heathmore: 892.345.3428 436 Finley Avenue', 'Frank Burger: 925.541.7625 662 South Dogwood Way', 'Heather Albrecht: 548.326.4584 919 Park Place'\]

</div>

Finally, split each entry into a list with first name, last name, telephone number, and address. We use the `maxsplit` parameter of `split` because the address has spaces, our splitting pattern, in it:

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> \[re.split(":? ", entry, maxsplit=3) for entry in entries\] \[\['Ross', 'McFluff', '834.345.1254', '155 Elm Street'\], \['Ronald', 'Heathmore', '892.345.3428', '436 Finley Avenue'\], \['Frank', 'Burger', '925.541.7625', '662 South Dogwood Way'\], \['Heather', 'Albrecht', '548.326.4584', '919 Park Place'\]\]

</div>

The `:?` pattern matches the colon after the last name, so that it does not occur in the result list. With a `maxsplit` of `4`, we could separate the house number from the street name:

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> \[re.split(":? ", entry, maxsplit=4) for entry in entries\] \[\['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'\], \['Ronald', 'Heathmore', '892.345.3428', '436', 'Finley Avenue'\], \['Frank', 'Burger', '925.541.7625', '662', 'South Dogwood Way'\], \['Heather', 'Albrecht', '548.326.4584', '919', 'Park Place'\]\]

</div>

### Text munging

`sub` replaces every occurrence of a pattern with a string or the result of a function. This example demonstrates using `sub` with a function to "munge" text, or randomize the order of all the characters in each word of a sentence except for the first and last characters:

    >>> def repl(m):
    ...     inner_word = list(m.group(2))
    ...     random.shuffle(inner_word)
    ...     return m.group(1) + "".join(inner_word) + m.group(3)
    ...
    >>> text = "Professor Abdolmalek, please report your absences promptly."
    >>> re.sub(r"(\w)(\w+)(\w)", repl, text)
    'Poefsrosr Aealmlobdk, pslaee reorpt your abnseces plmrptoy.'
    >>> re.sub(r"(\w)(\w+)(\w)", repl, text)
    'Pofsroser Aodlambelk, plasee reoprt yuor asnebces potlmrpy.'

### Finding all adverbs

`findall` matches *all* occurrences of a pattern, not just the first one as `search` does. For example, if a writer wanted to find all of the adverbs in some text, they might use `findall` in the following manner:

    >>> text = "He was carefully disguised but captured quickly by police."
    >>> re.findall(r"\w+ly\b", text)
    ['carefully', 'quickly']

### Finding all adverbs and their positions

If one wants more information about all matches of a pattern than the matched text, `finditer` is useful as it provides `~re.Match` objects instead of strings. Continuing with the previous example, if a writer wanted to find all of the adverbs *and their positions* in some text, they would use `finditer` in the following manner:

    >>> text = "He was carefully disguised but captured quickly by police."
    >>> for m in re.finditer(r"\w+ly\b", text):
    ...     print('%02d-%02d: %s' % (m.start(), m.end(), m.group(0)))
    07-16: carefully
    40-47: quickly

### Raw string notation

Raw string notation (`r"text"`) keeps regular expressions sane. Without it, every backslash (`'\'`) in a regular expression would have to be prefixed with another one to escape it. For example, the two following lines of code are functionally identical:

    >>> re.search(r"\W(.)\1\W", " ff ")
    <re.Match object; span=(0, 4), match=' ff '>
    >>> re.search("\\W(.)\\1\\W", " ff ")
    <re.Match object; span=(0, 4), match=' ff '>

When one wants to match a literal backslash, it must be escaped in the regular expression. With raw string notation, this means `r"\\"`. Without raw string notation, one must use `"\\\\"`, making the following lines of code functionally identical:

    >>> re.search(r"\\", r"\\")
    <re.Match object; span=(0, 1), match='\\'>
    >>> re.search("\\\\", r"\\")
    <re.Match object; span=(0, 1), match='\\'>

### Writing a tokenizer

A [tokenizer or scanner](https://en.wikipedia.org/wiki/Lexical_analysis) analyzes a string to categorize groups of characters. This is a useful first step in writing a compiler or interpreter.

The text categories are specified with regular expressions. The technique is to combine those into a single master regular expression and to loop over successive matches:

    from typing import NamedTuple
    import re

    class Token(NamedTuple):
        type: str
        value: int | float | str
        line: int
        column: int

    def tokenize(code):
        keywords = {'IF', 'THEN', 'ENDIF', 'FOR', 'NEXT', 'GOSUB', 'RETURN'}
        token_specification = [
            ('NUMBER',   r'\d+(\.\d*)?'),  # Integer or decimal number
            ('ASSIGN',   r':='),           # Assignment operator
            ('END',      r';'),            # Statement terminator
            ('ID',       r'[A-Za-z]+'),    # Identifiers
            ('OP',       r'[+\-*/]'),      # Arithmetic operators
            ('NEWLINE',  r'\n'),           # Line endings
            ('SKIP',     r'[ \t]+'),       # Skip over spaces and tabs
            ('MISMATCH', r'.'),            # Any other character
        ]
        tok_regex = '|'.join('(?P<%s>%s)' % pair for pair in token_specification)
        line_num = 1
        line_start = 0
        for mo in re.finditer(tok_regex, code):
            kind = mo.lastgroup
            value = mo.group()
            column = mo.start() - line_start
            if kind == 'NUMBER':
                value = float(value) if '.' in value else int(value)
            elif kind == 'ID' and value in keywords:
                kind = value
            elif kind == 'NEWLINE':
                line_start = mo.end()
                line_num += 1
                continue
            elif kind == 'SKIP':
                continue
            elif kind == 'MISMATCH':
                raise RuntimeError(f'{value!r} unexpected on line {line_num}')
            yield Token(kind, value, line_num, column)

    statements = '''
        IF quantity THEN
            total := total + price * quantity;
            tax := price * 0.05;
        ENDIF;
    '''

    for token in tokenize(statements):
        print(token)

The tokenizer produces the following output:

    Token(type='IF', value='IF', line=2, column=4)
    Token(type='ID', value='quantity', line=2, column=7)
    Token(type='THEN', value='THEN', line=2, column=16)
    Token(type='ID', value='total', line=3, column=8)
    Token(type='ASSIGN', value=':=', line=3, column=14)
    Token(type='ID', value='total', line=3, column=17)
    Token(type='OP', value='+', line=3, column=23)
    Token(type='ID', value='price', line=3, column=25)
    Token(type='OP', value='*', line=3, column=31)
    Token(type='ID', value='quantity', line=3, column=33)
    Token(type='END', value=';', line=3, column=41)
    Token(type='ID', value='tax', line=4, column=8)
    Token(type='ASSIGN', value=':=', line=4, column=12)
    Token(type='ID', value='price', line=4, column=15)
    Token(type='OP', value='*', line=4, column=21)
    Token(type='NUMBER', value=0.05, line=4, column=23)
    Token(type='END', value=';', line=4, column=27)
    Token(type='ENDIF', value='ENDIF', line=5, column=4)
    Token(type='END', value=';', line=5, column=9)

<div id="citations">

<span id="Frie09" class="citation-label">Frie09</span>  
Friedl, Jeffrey. Mastering Regular Expressions. 3rd ed., O'Reilly Media, 2009. The third edition of the book no longer covers Python at all, but the first edition covered writing good regular expression patterns in great detail.

</div>

# `!readline` --- GNU readline interface

<div class="module" data-synopsis="GNU readline support for Python.">

readline

</div>

------------------------------------------------------------------------

The `!readline` module defines a number of functions to facilitate completion and reading/writing of history files from the Python interpreter. This module can be used directly, or via the `rlcompleter` module, which supports completion of Python identifiers at the interactive prompt. Settings made using this module affect the behaviour of both the interpreter's interactive prompt and the prompts offered by the built-in `input` function.

Readline keybindings may be configured via an initialization file, typically `.inputrc` in your home directory. See [Readline Init File](https://tiswww.cwru.edu/php/chet/readline/rluserman.html#Readline-Init-File) in the GNU Readline manual for information about the format and allowable constructs of that file, and the capabilities of the Readline library in general.

<div class="availability">

not Android, not iOS, not WASI.

This module is not supported on `mobile platforms <mobile-availability>` or `WebAssembly platforms <wasm-availability>`.

</div>

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

<div class="availability">

Unix.

</div>

> [!NOTE]
> The underlying Readline library API may be implemented by the `editline` (`libedit`) library instead of GNU readline. On macOS the `!readline` module detects which library is being used at run time.
>
> The configuration file for `editline` is different from that of GNU readline. If you programmatically load configuration strings you can use `backend` to determine which library is being used.
>
> If you use `editline`/`libedit` readline emulation on macOS, the initialization file located in your home directory is named `.editrc`. For example, the following content in `~/.editrc` will turn ON *vi* keybindings and TAB completion:
>
>     python:bind -v
>     python:bind ^I rl_complete
>
> Also note that different libraries may use different history file formats. When switching the underlying library, existing history files may become unusable.

<div class="data">

backend

The name of the underlying Readline library being used, either `"readline"` or `"editline"`.

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

READLINE_VERSION_INFO

A named tuple containing the two components of the Readline library version that was used for building the module: *major* and *minor*. Both values are integers. The components can also be accessed by name, so `readline.READLINE_VERSION_INFO[0]` is equivalent to `readline.READLINE_VERSION_INFO.major` and so on. This may be different from the Readline library actually used at runtime, which is available as `readline_version_info`.

With the `editline` backend, this is the version of the Readline interface emulated by libedit, not the version of libedit.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

readline_version_info

A named tuple containing the version of the Readline library actually loaded by the interpreter, with the same fields as `READLINE_VERSION_INFO`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

readline_version

The version string of the Readline library actually loaded by the interpreter, like `'8.3'`. With the `editline` backend, this is `'EditLine wrapper'`.

<div class="versionadded">

3.16

</div>

</div>

## Init file

The following functions relate to the init file and user configuration:

<div class="function">

parse_and_bind(string)

Execute the init line provided in the *string* argument. This calls `!rl_parse_and_bind` in the underlying library.

</div>

<div class="function">

read_init_file(\[filename\])

Execute a readline initialization file. The default filename is the last filename used. This calls `!rl_read_init_file` in the underlying library. It raises an `auditing event <auditing>` `open` with the file name if given, and `"<readline_init_file>"` otherwise, regardless of which file the library resolves.

<div class="versionchanged">

3.14 The auditing event was added.

</div>

</div>

## Line buffer

The following functions operate on the line buffer:

<div class="function">

get_line_buffer()

Return the current contents of the line buffer (`!rl_line_buffer` in the underlying library).

</div>

<div class="function">

insert_text(string)

Insert text into the line buffer at the cursor position. This calls `!rl_insert_text` in the underlying library, but ignores the return value.

</div>

<div class="function">

redisplay()

Change what's displayed on the screen to reflect the current contents of the line buffer. This calls `!rl_redisplay` in the underlying library.

</div>

## History file

The following functions operate on a history file:

<div class="function">

read_history_file(\[filename\])

Load a readline history file, and append it to the history list. The default filename is `~/.history`. This calls `!read_history` in the underlying library and raises an `auditing event <auditing>` `open` with the file name if given and `"~/.history"` otherwise.

<div class="versionchanged">

3.14 The auditing event was added.

</div>

</div>

<div class="function">

write_history_file(\[filename\])

Save the history list to a readline history file, overwriting any existing file. The default filename is `~/.history`. This calls `!write_history` in the underlying library and raises an `auditing event <auditing>` `open` with the file name if given and `"~/.history"` otherwise.

<div class="versionchanged">

3.14 The auditing event was added.

</div>

</div>

<div class="function">

append_history_file(nelements\[, filename\])

Append the last *nelements* items of history to a file. The default filename is `~/.history`. The file must already exist. This calls `!append_history` in the underlying library. This function only exists if Python was compiled for a version of the library that supports it. It raises an `auditing event <auditing>` `open` with the file name if given and `"~/.history"` otherwise.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.14 The auditing event was added.

</div>

</div>

<div class="function">

get_history_length() set_history_length(length)

Set or return the desired number of lines to save in the history file. The `write_history_file` function uses this value to truncate the history file, by calling `!history_truncate_file` in the underlying library. Negative values imply unlimited history file size.

</div>

## History list

The following functions operate on a global history list:

<div class="function">

clear_history()

Clear the current history. This calls `!clear_history` in the underlying library. The Python function only exists if Python was compiled for a version of the library that supports it.

</div>

<div class="function">

get_current_history_length()

Return the number of items currently in the history. (This is different from `get_history_length`, which returns the maximum number of lines that will be written to a history file.)

</div>

<div class="function">

get_history_item(index)

Return the current contents of history item at *index*. The item index is one-based. This calls `!history_get` in the underlying library.

</div>

<div class="function">

remove_history_item(pos)

Remove history item specified by its position from the history. The position is zero-based. This calls `!remove_history` in the underlying library.

</div>

<div class="function">

replace_history_item(pos, line)

Replace history item specified by its position with *line*. The position is zero-based. This calls `!replace_history_entry` in the underlying library.

</div>

<div class="function">

add_history(line)

Append *line* to the history buffer, as if it was the last line typed. This calls `!add_history` in the underlying library.

</div>

<div class="function">

set_auto_history(enabled)

Enable or disable automatic calls to `!add_history` when reading input via readline. The *enabled* argument should be a Boolean value that when true, enables auto history, and that when false, disables auto history.

<div class="versionadded">

3.6

</div>

<div class="impl-detail">

Auto history is enabled by default, and changes to this do not persist across multiple sessions.

</div>

</div>

## Startup hooks

<div class="function">

set_startup_hook(\[function\])

Set or remove the function invoked by the `!rl_startup_hook` callback of the underlying library. If *function* is specified, it will be used as the new hook function; if omitted or `None`, any function already installed is removed. The hook is called with no arguments just before readline prints the first prompt.

</div>

<div class="function">

set_pre_input_hook(\[function\])

Set or remove the function invoked by the `!rl_pre_input_hook` callback of the underlying library. If *function* is specified, it will be used as the new hook function; if omitted or `None`, any function already installed is removed. The hook is called with no arguments after the first prompt has been printed and just before readline starts reading input characters. This function only exists if Python was compiled for a version of the library that supports it.

</div>

<div class="function">

get_pre_input_hook()

Get the current pre-input hook function, or `None` if no pre-input hook function has been set. This function only exists if Python was compiled for a version of the library that supports it.

<div class="versionadded">

3.15

</div>

</div>

## Completion

The following functions relate to implementing a custom word completion function. This is typically operated by the Tab key, and can suggest and automatically complete a word being typed. By default, Readline is set up to be used by `rlcompleter` to complete Python identifiers for the interactive interpreter. If the `!readline` module is to be used with a custom completer, a different set of word delimiters should be set.

<div class="function">

set_completer(\[function\])

Set or remove the completer function. If *function* is specified, it will be used as the new completer function; if omitted or `None`, any completer function already installed is removed. The completer function is called as `function(text, state)`, for *state* in `0`, `1`, `2`, ..., until it returns a non-string value. It should return the next possible completion starting with *text*.

The installed completer function is invoked by the *entry_func* callback passed to `!rl_completion_matches` in the underlying library. The *text* string comes from the first parameter to the `!rl_attempted_completion_function` callback of the underlying library.

</div>

<div class="function">

get_completer()

Get the completer function, or `None` if no completer function has been set.

</div>

<div class="function">

get_completion_type()

Get the type of completion being attempted. This returns the `!rl_completion_type` variable in the underlying library as an integer.

</div>

<div class="function">

get_begidx() get_endidx()

Get the beginning or ending index of the completion scope. These indexes are the *start* and *end* arguments passed to the `!rl_attempted_completion_function` callback of the underlying library. The values may be different in the same input editing scenario based on the underlying C readline implementation. Ex: libedit is known to behave differently than libreadline.

</div>

<div class="function">

set_completer_delims(string) get_completer_delims()

Set or get the word delimiters for completion. These determine the start of the word to be considered for completion (the completion scope). These functions access the `!rl_completer_word_break_characters` variable in the underlying library.

</div>

<div class="function">

set_completion_display_matches_hook(\[function\])

Set or remove the completion display function. If *function* is specified, it will be used as the new completion display function; if omitted or `None`, any completion display function already installed is removed. This sets or clears the `!rl_completion_display_matches_hook` callback in the underlying library. The completion display function is called as `function(substitution, [matches], longest_match_length)` once each time matches need to be displayed.

</div>

## Example

The following example demonstrates how to use the `!readline` module's history reading and writing functions to automatically load and save a history file named `.python_history` from the user's home directory. The code below would normally be executed automatically during interactive sessions from the user's `PYTHONSTARTUP` file. :

    import atexit
    import os
    import readline

    histfile = os.path.join(os.path.expanduser("~"), ".python_history")
    try:
        readline.read_history_file(histfile)
        # default history len is -1 (infinite), which may grow unruly
        readline.set_history_length(1000)
    except FileNotFoundError:
        pass

    atexit.register(readline.write_history_file, histfile)

This code is actually automatically run when Python is run in `interactive mode <tut-interactive>` (see `rlcompleter-config`).

The following example achieves the same goal but supports concurrent interactive sessions, by only appending the new history. :

    import atexit
    import os
    import readline
    histfile = os.path.join(os.path.expanduser("~"), ".python_history")

    try:
        readline.read_history_file(histfile)
        h_len = readline.get_current_history_length()
    except FileNotFoundError:
        open(histfile, 'wb').close()
        h_len = 0

    def save(prev_h_len, histfile):
        new_h_len = readline.get_current_history_length()
        readline.set_history_length(1000)
        readline.append_history_file(new_h_len - prev_h_len, histfile)
    atexit.register(save, h_len, histfile)

The following example extends the `code.InteractiveConsole` class to support history save/restore. :

    import atexit
    import code
    import os
    import readline

    class HistoryConsole(code.InteractiveConsole):
        def __init__(self, locals=None, filename="<console>",
                     histfile=os.path.expanduser("~/.console-history")):
            code.InteractiveConsole.__init__(self, locals, filename)
            self.init_history(histfile)

        def init_history(self, histfile):
            readline.parse_and_bind("tab: complete")
            if hasattr(readline, "read_history_file"):
                try:
                    readline.read_history_file(histfile)
                except FileNotFoundError:
                    pass
                atexit.register(self.save_history, histfile)

        def save_history(self, histfile):
            readline.set_history_length(1000)
            readline.write_history_file(histfile)

> [!NOTE]
> The new `REPL` introduced in version 3.13 doesn't support readline. However, readline can still be used by setting the `PYTHON_BASIC_REPL` environment variable.

# removed

tocdepth  
1

# Removed Modules

The modules described in this chapter have been removed from the Python standard library. They are documented here to help people find replacements.

<div class="toctree" data-maxdepth="1">

aifc.rst asynchat.rst asyncore.rst audioop.rst cgi.rst cgitb.rst chunk.rst crypt.rst distutils.rst imghdr.rst imp.rst mailcap.rst msilib.rst nis.rst nntplib.rst ossaudiodev.rst pipes.rst smtpd.rst sndhdr.rst spwd.rst sunau.rst telnetlib.rst uu.rst xdrlib.rst

</div>

# `!reprlib` --- Alternate `repr` implementation

<div class="module" data-synopsis="Alternate repr() implementation with size limits.">

reprlib

</div>

**Source code:** `Lib/reprlib.py`

------------------------------------------------------------------------

The `!reprlib` module provides a means for producing object representations with limits on the size of the resulting strings. This is used in the Python debugger and may be useful in other contexts as well.

This module provides a class, an instance, and a function:

<div class="Repr(*, maxlevel=6, maxtuple=6, maxlist=6, maxarray=5, maxdict=4, \ maxset=6, maxfrozenset=6, maxdeque=6, maxstring=30, maxlong=40, \ maxother=30, fillvalue=&quot;...&quot;, indent=None)">

Class which provides formatting services useful in implementing functions similar to the built-in `repr`; size limits for different object types are added to avoid the generation of representations which are excessively long.

The keyword arguments of the constructor can be used as a shortcut to set the attributes of the `Repr` instance. Which means that the following initialization:

    aRepr = reprlib.Repr(maxlevel=3)

Is equivalent to:

    aRepr = reprlib.Repr()
    aRepr.maxlevel = 3

See section [Repr Objects](#repr-objects) for more information about `Repr` attributes.

<div class="versionchanged">

3.12 Allow attributes to be set via keyword arguments.

</div>

</div>

<div class="data">

aRepr

This is an instance of `Repr` which is used to provide the `.repr` function described below. Changing the attributes of this object will affect the size limits used by `.repr` and the Python debugger.

</div>

<div class="function">

repr(obj)

This is the `~Repr.repr` method of `aRepr`. It returns a string similar to that returned by the built-in function of the same name, but with limits on most sizes.

</div>

In addition to size-limiting tools, the module also provides a decorator for detecting recursive calls to `~object.__repr__` and substituting a placeholder string instead.

<div class="index">

single: ...; placeholder

</div>

<div class="decorator">

recursive_repr(fillvalue="...")

Decorator for `~object.__repr__` methods to detect recursive calls within the same thread. If a recursive call is made, the *fillvalue* is returned, otherwise, the usual `!__repr__` call is made. For example:

<div class="doctest">

\>\>\> from reprlib import recursive_repr \>\>\> class MyList(list): ... @recursive_repr() ... def \_\_repr\_\_(self): ... return '\<' + '\|'.join(map(repr, self)) + '\>' ... \>\>\> m = MyList('abc') \>\>\> m.append(m) \>\>\> m.append('x') \>\>\> print(m) \<'a'\|'b'\|'c'\|...\|'x'\>

</div>

<div class="versionadded">

3.2

</div>

</div>

## Repr Objects

`Repr` instances provide several attributes which can be used to provide size limits for the representations of different object types, and methods which format specific object types.

<div class="attribute">

Repr.fillvalue

This string is displayed for recursive references. It defaults to `...`.

<div class="versionadded">

3.11

</div>

</div>

<div class="attribute">

Repr.maxlevel

Depth limit on the creation of recursive representations. The default is `6`.

</div>

<div class="attribute">

Repr.maxdict Repr.maxlist Repr.maxtuple Repr.maxset Repr.maxfrozenset Repr.maxdeque Repr.maxarray

Limits on the number of entries represented for the named object type. The default is `4` for `maxdict`, `5` for `maxarray`, and `6` for the others.

</div>

<div class="attribute">

Repr.maxlong

Maximum number of characters in the representation for an integer. Digits are dropped from the middle. The default is `40`.

</div>

<div class="attribute">

Repr.maxstring

Limit on the number of characters in the representation of the string. Note that the "normal" representation of the string is used as the character source: if escape sequences are needed in the representation, these may be mangled when the representation is shortened. The default is `30`.

</div>

<div class="attribute">

Repr.maxother

This limit is used to control the size of object types for which no specific formatting method is available on the `Repr` object. It is applied in a similar manner as `maxstring`. The default is `20`.

</div>

<div class="attribute">

Repr.indent

If this attribute is set to `None` (the default), the output is formatted with no line breaks or indentation, like the standard `repr`. For example:

<div class="doctest">

indent

\>\>\> example = \[ ... 1, 'spam', {'a': 2, 'b': 'spam eggs', 'c': {3: 4.5, 6: \[\]}}, 'ham'\] \>\>\> import reprlib \>\>\> aRepr = reprlib.Repr() \>\>\> print(aRepr.repr(example)) \[1, 'spam', {'a': 2, 'b': 'spam eggs', 'c': {3: 4.5, 6: \[\]}}, 'ham'\]

</div>

If `~Repr.indent` is set to a string, each recursion level is placed on its own line, indented by that string:

<div class="doctest">

indent

\>\>\> aRepr.indent = '--\>' \>\>\> print(aRepr.repr(example)) \[ --\>1, --\>'spam', --\>{ --\>--\>'a': 2, --\>--\>'b': 'spam eggs', --\>--\>'c': { --\>--\>--\>3: 4.5, --\>--\>--\>6: \[\], --\>--\>}, --\>}, --\>'ham', \]

</div>

Setting `~Repr.indent` to a positive integer value behaves as if it was set to a string with that number of spaces:

<div class="doctest">

indent

\>\>\> aRepr.indent = 4 \>\>\> print(aRepr.repr(example)) \[ 1, 'spam', { 'a': 2, 'b': 'spam eggs', 'c': { 3: 4.5, 6: \[\], }, }, 'ham', \]

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="method">

Repr.repr(obj)

The equivalent to the built-in `repr` that uses the formatting imposed by the instance.

</div>

<div class="method">

Repr.repr1(obj, level)

Recursive implementation used by `.repr`. This uses the type of *obj* to determine which formatting method to call, passing it *obj* and *level*. The type-specific methods should call `repr1` to perform recursive formatting, with `level - 1` for the value of *level* in the recursive call.

</div>

<div class="method" data-noindex="">

Repr.repr_TYPE(obj, level)

Formatting methods for specific types are implemented as methods with a name based on the type name. In the method name, **TYPE** is replaced by `'_'.join(type(obj).__name__.split())`. Dispatch to these methods is handled by `repr1`. Type-specific methods which need to recursively format a value should call `self.repr1(subobj, level - 1)`.

</div>

## Subclassing Repr Objects

The use of dynamic dispatching by `Repr.repr1` allows subclasses of `Repr` to add support for additional built-in object types or to modify the handling of types already supported. This example shows how special support for file objects could be added:

<div class="testcode">

import reprlib import sys

class MyRepr(reprlib.Repr):

> def repr_TextIOWrapper(self, obj, level):  
> if obj.name in {'\<stdin\>', '\<stdout\>', '\<stderr\>'}:  
> return obj.name
>
> return repr(obj)

aRepr = MyRepr() print(aRepr.repr(sys.stdin)) \# prints '\<stdin\>'

</div>

<div class="testoutput">

\<stdin\>

</div>

# `!resource` --- Resource usage information

<div class="module" data-synopsis="An interface to provide resource usage information on the current process.">

resource

</div>

------------------------------------------------------------------------

This module provides basic mechanisms for measuring and controlling system resources utilized by a program.

<div class="availability">

Unix, not WASI.

</div>

Symbolic constants are used to specify particular system resources and to request usage information about either the current process or its children.

An `OSError` is raised on syscall failure.

<div class="exception">

error

A deprecated alias of `OSError`.

<div class="versionchanged">

3.3 Following `3151`, this class was made an alias of `OSError`.

</div>

</div>

## Resource Limits

Resources usage can be limited using the `setrlimit` function described below. Each resource is controlled by a pair of limits: a soft limit and a hard limit. The soft limit is the current limit, and may be lowered or raised by a process over time. The soft limit can never exceed the hard limit. The hard limit can be lowered to any value greater than the soft limit, but not raised. (Only processes with the effective UID of the super-user can raise a hard limit.)

The specific resources that can be limited are system dependent. They are described in the `getrlimit(2)` man page. The resources listed below are supported when the underlying operating system supports them; resources which cannot be checked or controlled by the operating system are not defined in this module for those platforms.

<div class="data">

RLIM_INFINITY

Constant used to represent the limit for an unlimited resource.

<div class="versionchanged">

3.15 It is now always positive. Previously, it could be negative, such as -1 or -3.

</div>

</div>

<div class="data">

RLIM_SAVED_CUR

</div>

<div class="data">

RLIM_SAVED_MAX

Constants used to represent the soft and hard limit values if they cannot be represented in the `rlim_t` value in C. Can be equal to `RLIM_INFINITY`.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

getrlimit(resource)

Returns a tuple `(soft, hard)` with the current soft and hard limits of *resource*. Raises `ValueError` if an invalid resource is specified, or `error` if the underlying system call fails unexpectedly.

</div>

<div class="function">

setrlimit(resource, limits)

Sets new limits of consumption of *resource*. The *limits* argument must be a tuple `(soft, hard)` of two integers describing the new limits. A value of `~resource.RLIM_INFINITY` can be used to request a limit that is unlimited.

Raises `ValueError` if an invalid resource is specified, if the new soft limit exceeds the hard limit, or if a process tries to raise its hard limit. Specifying a limit of `~resource.RLIM_INFINITY` when the hard or system limit for that resource is not unlimited will result in a `ValueError`. A process with the effective UID of super-user can request any valid limit value, including unlimited, but `ValueError` will still be raised if the requested limit exceeds the system imposed limit.

`setrlimit` may also raise `error` if the underlying system call fails.

VxWorks only supports setting `RLIMIT_NOFILE`.

<div class="audit-event">

resource.setrlimit resource,limits resource.setrlimit

</div>

</div>

<div class="function">

prlimit(pid, resource\[, limits\])

Combines `setrlimit` and `getrlimit` in one function and supports to get and set the resources limits of an arbitrary process. If *pid* is 0, then the call applies to the current process. *resource* and *limits* have the same meaning as in `setrlimit`, except that *limits* is optional.

When *limits* is not given the function returns the *resource* limit of the process *pid*. When *limits* is given the *resource* limit of the process is set and the former resource limit is returned.

Raises `ProcessLookupError` when *pid* can't be found and `PermissionError` when the user doesn't have `CAP_SYS_RESOURCE` for the process.

<div class="audit-event">

resource.prlimit pid,resource,limits resource.prlimit

</div>

<div class="availability">

Linux \>= 2.6.36 with glibc \>= 2.13.

</div>

<div class="versionadded">

3.4

</div>

</div>

These symbols define resources whose consumption can be controlled using the `setrlimit` and `getrlimit` functions described below. The values of these symbols are exactly the constants used by C programs.

The Unix man page for `getrlimit(2)` lists the available resources. Note that not all systems use the same symbol or same value to denote the same resource. This module does not attempt to mask platform differences --- symbols not defined for a platform will not be available from this module on that platform.

<div class="data">

RLIMIT_CORE

The maximum size (in bytes) of a core file that the current process can create. This may result in the creation of a partial core file if a larger core would be required to contain the entire process image.

</div>

<div class="data">

RLIMIT_CPU

The maximum amount of processor time (in seconds) that a process can use. If this limit is exceeded, a `~signal.SIGXCPU` signal is sent to the process. (See the `signal` module documentation for information about how to catch this signal and do something useful, e.g. flush open files to disk.)

</div>

<div class="data">

RLIMIT_FSIZE

The maximum size of a file which the process may create.

</div>

<div class="data">

RLIMIT_DATA

The maximum size (in bytes) of the process's heap.

</div>

<div class="data">

RLIMIT_STACK

The maximum size (in bytes) of the call stack for the current process. This only affects the stack of the main thread in a multi-threaded process.

</div>

<div class="data">

RLIMIT_RSS

The maximum resident set size that should be made available to the process.

</div>

<div class="data">

RLIMIT_NPROC

The maximum number of processes the current process may create.

</div>

<div class="data">

RLIMIT_NOFILE

The maximum number of open file descriptors for the current process.

</div>

<div class="data">

RLIMIT_OFILE

The BSD name for `RLIMIT_NOFILE`.

</div>

<div class="data">

RLIMIT_MEMLOCK

The maximum address space which may be locked in memory.

</div>

<div class="data">

RLIMIT_VMEM

The largest area of mapped memory which the process may occupy. Usually an alias of `RLIMIT_AS`.

<div class="availability">

Solaris, FreeBSD, NetBSD.

</div>

</div>

<div class="data">

RLIMIT_AS

The maximum area (in bytes) of address space which may be taken by the process.

</div>

<div class="data">

RLIMIT_MSGQUEUE

The number of bytes that can be allocated for POSIX message queues.

<div class="availability">

Linux \>= 2.6.8.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

RLIMIT_NICE

The ceiling for the process's nice level (calculated as 20 - rlim_cur).

<div class="availability">

Linux \>= 2.6.12.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

RLIMIT_RTPRIO

The ceiling of the real-time priority.

<div class="availability">

Linux \>= 2.6.12.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

RLIMIT_RTTIME

The time limit (in microseconds) on CPU time that a process can spend under real-time scheduling without making a blocking syscall.

<div class="availability">

Linux \>= 2.6.25.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

RLIMIT_SIGPENDING

The number of signals which the process may queue.

<div class="availability">

Linux \>= 2.6.8.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

RLIMIT_SBSIZE

The maximum size (in bytes) of socket buffer usage for this user. This limits the amount of network memory, and hence the amount of mbufs, that this user may hold at any time.

<div class="availability">

FreeBSD, NetBSD.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

RLIMIT_SWAP

The maximum size (in bytes) of the swap space that may be reserved or used by all of this user id's processes. This limit is enforced only if bit 1 of the vm.overcommit sysctl is set. Please see [tuning(7)](https://man.freebsd.org/cgi/man.cgi?query=tuning&sektion=7) for a complete description of this sysctl.

<div class="availability">

FreeBSD \>= 8.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

RLIMIT_NPTS

The maximum number of pseudo-terminals created by this user id.

<div class="availability">

FreeBSD \>= 8.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

RLIMIT_KQUEUES

The maximum number of kqueues this user id is allowed to create.

<div class="availability">

FreeBSD \>= 11.

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

RLIMIT_NTHR

The maximum number of threads for this user id, not counting the main and kernel threads.

<div class="availability">

NetBSD \>= 7.0.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

RLIMIT_PIPEBUF

The maximum total size of in-kernel buffers for bi-directional pipes/fifos that this user id is allowed to consume.

<div class="availability">

FreeBSD \>= 14.2.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

RLIMIT_THREADS

The maximum number of threads each process can create.

<div class="availability">

AIX.

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

RLIMIT_UMTXP

The limit of the number of process-shared Posix thread library objects allocated by user id.

<div class="availability">

FreeBSD \>= 11.

</div>

<div class="versionadded">

3.15

</div>

</div>

## Resource Usage

These functions are used to retrieve resource usage information:

<div class="function">

getrusage(who)

This function returns an object that describes the resources consumed by either the current process or its children, as specified by the *who* parameter. The *who* parameter should be specified using one of the `!RUSAGE_\*` constants described below.

A simple example:

    from resource import *
    import time

    # a non CPU-bound task
    time.sleep(3)
    print(getrusage(RUSAGE_SELF))

    # a CPU-bound task
    for i in range(10 ** 8):
       _ = 1 + 1
    print(getrusage(RUSAGE_SELF))

The fields of the return value each describe how a particular system resource has been used, e.g. amount of time spent running in user mode or number of times the process was swapped out of main memory. Some values are dependent on the clock tick interval, e.g. the amount of memory the process is using.

For backward compatibility, the return value is also accessible as a tuple of 16 elements.

The fields `!ru_utime` and `!ru_stime` of the return value are floating-point values representing the amount of time spent executing in user mode and the amount of time spent executing in system mode, respectively. The remaining values are integers. Consult the `getrusage(2)` man page for detailed information about these values. A brief summary is presented here:

| Index | Field          | Resource                            |
|-------|----------------|-------------------------------------|
| `0`   | `!ru_utime`    | time in user mode (float seconds)   |
| `1`   | `!ru_stime`    | time in system mode (float seconds) |
| `2`   | `!ru_maxrss`   | maximum resident set size           |
| `3`   | `!ru_ixrss`    | shared memory size                  |
| `4`   | `!ru_idrss`    | unshared memory size                |
| `5`   | `!ru_isrss`    | unshared stack size                 |
| `6`   | `!ru_minflt`   | page faults not requiring I/O       |
| `7`   | `!ru_majflt`   | page faults requiring I/O           |
| `8`   | `!ru_nswap`    | number of swap outs                 |
| `9`   | `!ru_inblock`  | block input operations              |
| `10`  | `!ru_oublock`  | block output operations             |
| `11`  | `!ru_msgsnd`   | messages sent                       |
| `12`  | `!ru_msgrcv`   | messages received                   |
| `13`  | `!ru_nsignals` | signals received                    |
| `14`  | `!ru_nvcsw`    | voluntary context switches          |
| `15`  | `!ru_nivcsw`   | involuntary context switches        |

This function will raise a `ValueError` if an invalid *who* parameter is specified. It may also raise `error` exception in unusual circumstances.

</div>

<div class="function">

getpagesize()

Returns the number of bytes in a system page. (This need not be the same as the hardware page size.)

</div>

The following `!RUSAGE_\*` symbols are passed to the `getrusage` function to specify which processes information should be provided for.

<div class="data">

RUSAGE_SELF

Pass to `getrusage` to request resources consumed by the calling process, which is the sum of resources used by all threads in the process.

</div>

<div class="data">

RUSAGE_CHILDREN

Pass to `getrusage` to request resources consumed by child processes of the calling process which have been terminated and waited for.

</div>

<div class="data">

RUSAGE_BOTH

Pass to `getrusage` to request resources consumed by both the current process and child processes. May not be available on all systems.

</div>

<div class="data">

RUSAGE_THREAD

Pass to `getrusage` to request resources consumed by the current thread. May not be available on all systems.

<div class="versionadded">

3.2

</div>

</div>

# `!rlcompleter` --- Completion function for GNU readline

<div class="module" data-synopsis="Python identifier completion, suitable for the GNU readline library.">

rlcompleter

</div>

**Source code:** `Lib/rlcompleter.py`

------------------------------------------------------------------------

The `!rlcompleter` module defines a completion function suitable to be passed to `~readline.set_completer` in the `readline` module.

When this module is imported on a Unix platform with the `readline` module available, an instance of the `Completer` class is automatically created and its `~Completer.complete` method is set as the `readline completer <readline-completion>`. The method provides completion of valid Python `identifiers and keywords <identifiers>`.

Example:

    >>> import rlcompleter
    >>> import readline
    >>> readline.parse_and_bind("tab: complete")
    >>> readline. <TAB PRESSED>
    readline.__doc__          readline.get_line_buffer(  readline.read_init_file(
    readline.__file__         readline.insert_text(      readline.set_completer(
    readline.__name__         readline.parse_and_bind(
    >>> readline.

The `!rlcompleter` module is designed for use with Python's `interactive mode <tut-interactive>`. Unless Python is run with the `-S` option, the module is automatically imported and configured (see `rlcompleter-config`).

On platforms without `readline`, the `Completer` class defined by this module can still be used for custom purposes.

<div id="completer-objects">

<div class="Completer">

Completer objects have the following method:

<div class="method">

Completer.complete(text, state)

Return the next possible completion for *text*.

When called by the `readline` module, this method is called successively with `state == 0, 1, 2, ...` until the method returns `None`.

If called for *text* that doesn't include a period character (`'.'`), it will complete from names currently defined in `__main__`, `builtins` and keywords (as defined by the `keyword` module).

If called for a dotted name, it will try to evaluate anything without obvious side-effects (functions will not be evaluated, but it can generate calls to `~object.__getattr__`) up to the last part, and find matches for the rest via the `dir` function. Any exception raised during the evaluation of the expression is caught, silenced and `None` is returned.

</div>

</div>

</div>

# `!runpy` --- Locating and executing Python modules

<div class="module" data-synopsis="Locate and run Python modules without importing them first.">

runpy

</div>

**Source code:** `Lib/runpy.py`

------------------------------------------------------------------------

The `!runpy` module is used to locate and run Python modules without importing them first. Its main use is to implement the `-m` command line switch that allows scripts to be located using the Python module namespace rather than the filesystem.

Note that this is *not* a sandbox module - all code is executed in the current process, and any side effects (such as cached imports of other modules) will remain in place after the functions have returned.

Furthermore, any functions and classes defined by the executed code are not guaranteed to work correctly after a `!runpy` function has returned. If that limitation is not acceptable for a given use case, `importlib` is likely to be a more suitable choice than this module.

The `!runpy` module provides two functions:

<div class="function">

run_module(mod_name, init_globals=None, run_name=None, alter_sys=False)

<div class="index">

pair: module; \_\_main\_\_

</div>

Execute the code of the specified module and return the resulting module's globals dictionary. The module's code is first located using the standard import mechanism (refer to `302` for details) and then executed in a fresh module namespace.

The *mod_name* argument should be an absolute module name. If the module name refers to a package rather than a normal module, then that package is imported and the `__main__` submodule within that package is then executed and the resulting module globals dictionary returned.

The optional dictionary argument *init_globals* may be used to pre-populate the module's globals dictionary before the code is executed. *init_globals* will not be modified. If any of the special global variables below are defined in *init_globals*, those definitions are overridden by `run_module`.

The special global variables `__name__`, `__spec__`, `__file__`, `__loader__` and `__package__` are set in the globals dictionary before the module code is executed. (Note that this is a minimal set of variables - other variables may be set implicitly as an interpreter implementation detail.)

`__name__` is set to *run_name* if this optional argument is not `None`, to `mod_name + '.__main__'` if the named module is a package and to the *mod_name* argument otherwise.

`__spec__` will be set appropriately for the *actually* imported module (that is, `__spec__.name` will always be *mod_name* or `mod_name + '.__main__'`, never *run_name*).

`__file__`, `__loader__` and `__package__` are `set as normal <import-mod-attrs>` based on the module spec.

If the argument *alter_sys* is supplied and evaluates to `True`, then `sys.argv[0]` is updated with the value of `__file__` and `sys.modules[__name__]` is updated with a temporary module object for the module being executed. Both `sys.argv[0]` and `sys.modules[__name__]` are restored to their original values before the function returns.

Note that this manipulation of `sys` is not thread-safe. Other threads may see the partially initialised module, as well as the altered list of arguments. It is recommended that the `sys` module be left alone when invoking this function from threaded code.

<div class="seealso">

The `-m` option offering equivalent functionality from the command line.

</div>

<div class="versionchanged">

3.1 Added ability to execute packages by looking for a `__main__` submodule.

</div>

<div class="versionchanged">

3.2 Added `__cached__` global variable (see `3147`).

</div>

<div class="versionchanged">

3.4 Updated to take advantage of the module spec feature added by `451`. This allows `__cached__` to be set correctly for modules run this way, as well as ensuring the real module name is always accessible as `__spec__.name`.

</div>

<div class="versionchanged">

3.12 The setting of `__cached__`, `__loader__`, and `__package__` are deprecated. See `~importlib.machinery.ModuleSpec` for alternatives.

</div>

<div class="versionchanged">

3.15 The global variable `__cached__` is no longer set.

</div>

</div>

<div class="function">

run_path(path_name, init_globals=None, run_name=None)

<div class="index">

pair: module; \_\_main\_\_

</div>

Execute the code at the named filesystem location and return the resulting module's globals dictionary. As with a script name supplied to the CPython command line, *path_name* may refer to a Python source file, a compiled bytecode file or a valid `sys.path` entry containing a `__main__` module (e.g. a zipfile containing a top-level `__main__.py` file).

For a simple script, the specified code is simply executed in a fresh module namespace. For a valid `sys.path` entry (typically a zipfile or directory), the entry is first added to the beginning of `sys.path`. The function then looks for and executes a `__main__` module using the updated path. Note that there is no special protection against invoking an existing `__main__` entry located elsewhere on `sys.path` if there is no such module at the specified location.

The optional dictionary argument *init_globals* may be used to pre-populate the module's globals dictionary before the code is executed. *init_globals* will not be modified. If any of the special global variables below are defined in *init_globals*, those definitions are overridden by `run_path`.

The special global variables `__name__`, `__spec__`, `__file__`, `__loader__` and `__package__` are set in the globals dictionary before the module code is executed. (Note that this is a minimal set of variables - other variables may be set implicitly as an interpreter implementation detail.)

`__name__` is set to *run_name* if this optional argument is not `None` and to `'<run_path>'` otherwise.

If *path_name* directly references a script file (whether as source or as precompiled byte code), then `__file__` will be set to *path_name*, and `__spec__`, `__loader__` and `__package__` will all be set to `None`.

If *path_name* is a reference to a valid `sys.path` entry, then `__spec__` will be set appropriately for the imported `__main__` module (that is, `__spec__.name` will always be `__main__`). `__file__`, `__loader__` and `__package__` will be `set as normal <import-mod-attrs>` based on the module spec.

A number of alterations are also made to the `sys` module. Firstly, `sys.path` may be altered as described above. `sys.argv[0]` is updated with the value of *path_name* and `sys.modules[__name__]` is updated with a temporary module object for the module being executed. All modifications to items in `sys` are reverted before the function returns.

Note that, unlike `run_module`, the alterations made to `sys` are not optional in this function as these adjustments are essential to allowing the execution of `sys.path` entries. As the thread-safety limitations still apply, use of this function in threaded code should be either serialised with the import lock or delegated to a separate process.

<div class="seealso">

`using-on-interface-options` for equivalent functionality on the command line (`python path/to/script`).

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.4 Updated to take advantage of the module spec feature added by `451`. This allows `__cached__` to be set correctly in the case where `__main__` is imported from a valid `sys.path` entry rather than being executed directly.

</div>

<div class="versionchanged">

3.12 The setting of `__cached__`, `__loader__`, and `__package__` are deprecated.

</div>

<div class="versionchanged">

3.15 The global variable `__cached__` is no longer set.

</div>

</div>

<div class="seealso">

`338` -- Executing modules as scripts  
PEP written and implemented by Nick Coghlan.

`366` -- Main module explicit relative imports  
PEP written and implemented by Nick Coghlan.

`451` -- A ModuleSpec Type for the Import System  
PEP written and implemented by Eric Snow

`using-on-general` - CPython command line details

The `importlib.import_module` function

</div>

# `!sched` --- Event scheduler

<div class="module" data-synopsis="General purpose event scheduler.">

sched

</div>

**Source code:** `Lib/sched.py`

<div class="index">

single: event scheduling

</div>

------------------------------------------------------------------------

The `!sched` module defines a class which implements a general purpose event scheduler:

<div class="scheduler(timefunc=time.monotonic, delayfunc=time.sleep)">

The `scheduler` class defines a generic interface to scheduling events. It needs two functions to actually deal with the "outside world" --- *timefunc* should be callable without arguments, and return a number (the "time", in any units whatsoever). The *delayfunc* function should be callable with one argument, compatible with the output of *timefunc*, and should delay that many time units. *delayfunc* will also be called with the argument `0` after each event is run to allow other threads an opportunity to run in multi-threaded applications.

<div class="versionchanged">

3.3 *timefunc* and *delayfunc* parameters are optional.

</div>

<div class="versionchanged">

3.3 `scheduler` class can be safely used in multi-threaded environments.

</div>

</div>

Example:

    >>> import sched, time
    >>> s = sched.scheduler(time.time, time.sleep)
    >>> def print_time(a='default'):
    ...     print("From print_time", time.time(), a)
    ...
    >>> def print_some_times():
    ...     print(time.time())
    ...     s.enter(10, 1, print_time)
    ...     s.enter(5, 2, print_time, argument=('positional',))
    ...     # despite having higher priority, 'keyword' runs after 'positional' as enter() is relative
    ...     s.enter(5, 1, print_time, kwargs={'a': 'keyword'})
    ...     s.enterabs(1_650_000_000, 10, print_time, argument=("first enterabs",))
    ...     s.enterabs(1_650_000_000, 5, print_time, argument=("second enterabs",))
    ...     s.run()
    ...     print(time.time())
    ...
    >>> print_some_times()
    1652342830.3640375
    From print_time 1652342830.3642538 second enterabs
    From print_time 1652342830.3643398 first enterabs
    From print_time 1652342835.3694863 positional
    From print_time 1652342835.3696074 keyword
    From print_time 1652342840.369612 default
    1652342840.3697174

## Scheduler Objects

`scheduler` instances have the following methods and attributes:

<div class="method">

scheduler.enterabs(time, priority, action, argument=(), kwargs={})

Schedule a new event. The *time* argument should be a numeric type compatible with the return value of the *timefunc* function passed to the constructor. Events scheduled for the same *time* will be executed in the order of their *priority*. A lower number represents a higher priority.

Executing the event means executing `action(*argument, **kwargs)`. *argument* is a sequence holding the positional arguments for *action*. *kwargs* is a dictionary holding the keyword arguments for *action*.

Return value is an event which may be used for later cancellation of the event (see `cancel`).

<div class="versionchanged">

3.3 *argument* parameter is optional.

</div>

<div class="versionchanged">

3.3 *kwargs* parameter was added.

</div>

</div>

<div class="method">

scheduler.enter(delay, priority, action, argument=(), kwargs={})

Schedule an event for *delay* more time units. Other than the relative time, the other arguments, the effect and the return value are the same as those for `enterabs`.

<div class="versionchanged">

3.3 *argument* parameter is optional.

</div>

<div class="versionchanged">

3.3 *kwargs* parameter was added.

</div>

</div>

<div class="method">

scheduler.cancel(event)

Remove the event from the queue. If *event* is not an event currently in the queue, this method will raise a `ValueError`.

</div>

<div class="method">

scheduler.empty()

Return `True` if the event queue is empty.

</div>

<div class="method">

scheduler.run(blocking=True)

Run all scheduled events. This method will wait (using the *delayfunc* function passed to the constructor) for the next event, then execute it and so on until there are no more scheduled events.

If *blocking* is false, immediately executes all events in the queue which have a time value less than or equal to the current *timefunc* value (if any) and returns the difference between the current *timefunc* value and the time value of the next scheduled event in the scheduler's event queue. If the queue is empty, returns `None`.

Either *action* or *delayfunc* can raise an exception. In either case, the scheduler will maintain a consistent state and propagate the exception. If an exception is raised by *action*, the event will not be attempted in future calls to `run`.

If a sequence of events takes longer to run than the time available before the next event, the scheduler will simply fall behind. No events will be dropped; the calling code is responsible for canceling events which are no longer pertinent.

<div class="versionchanged">

3.3 *blocking* parameter was added.

</div>

</div>

<div class="attribute">

scheduler.queue

Read-only attribute returning a list of upcoming events in the order they will be run. Each event is shown as a `named tuple` with the following fields: time, priority, action, argument, kwargs.

</div>

# `!secrets` --- Generate secure random numbers for managing secrets

<div class="module" data-synopsis="Generate secure random numbers for managing secrets.">

secrets

</div>

<div class="versionadded">

3.6

</div>

<div class="testsetup">

from secrets import \* \_\_name\_\_ = '\<doctest\>'

</div>

**Source code:** `Lib/secrets.py`

------------------------------------------------------------------------

The `!secrets` module is used for generating cryptographically strong random numbers suitable for managing data such as passwords, account authentication, security tokens, and related secrets.

In particular, `!secrets` should be used in preference to the default pseudo-random number generator in the `random` module, which is designed for modelling and simulation, not security or cryptography.

<div class="seealso">

`506`

</div>

## Random numbers

The `!secrets` module provides access to the most secure source of randomness that your operating system provides.

<div class="SystemRandom">

A class for generating random numbers using the highest-quality sources provided by the operating system. See `random.SystemRandom` for additional details.

</div>

<div class="function">

choice(seq)

Return a randomly chosen element from a non-empty sequence.

</div>

<div class="function">

randbelow(exclusive_upper_bound)

Return a random int in the range \[0, *exclusive_upper_bound*).

</div>

<div class="function">

randbits(k)

Return a non-negative int with *k* random bits.

</div>

## Generating tokens

The `!secrets` module provides functions for generating secure tokens, suitable for applications such as password resets, hard-to-guess URLs, and similar.

<div class="function">

token_bytes(nbytes=None)

Return a random byte string containing *nbytes* number of bytes.

If *nbytes* is not specified or `None`, `DEFAULT_ENTROPY` is used instead.

<div class="doctest">

\>\>\> token_bytes(16) \# doctest: +SKIP b'xebrx17D\*txaexd4xe3Sxb6xe2xebP1x8b'

</div>

</div>

<div class="function">

token_hex(nbytes=None)

Return a random text string, in hexadecimal. The string has *nbytes* random bytes, each byte converted to two hex digits.

If *nbytes* is not specified or `None`, `DEFAULT_ENTROPY` is used instead.

<div class="doctest">

\>\>\> token_hex(16) \# doctest: +SKIP 'f9bf78b9a18ce6d46a0cd2b0b86df9da'

</div>

</div>

<div class="function">

token_urlsafe(nbytes=None)

Return a random URL-safe text string, containing *nbytes* random bytes. The text is Base64 encoded, so on average each byte results in approximately 1.3 characters.

If *nbytes* is not specified or `None`, `DEFAULT_ENTROPY` is used instead.

<div class="doctest">

\>\>\> token_urlsafe(16) \# doctest: +SKIP 'Drmhze6EPcv0fN_81Bj-nA'

</div>

</div>

### How many bytes should tokens use?

To be secure against [brute-force attacks](https://en.wikipedia.org/wiki/Brute-force_attack), tokens need to have sufficient randomness. Unfortunately, what is considered sufficient will necessarily increase as computers get more powerful and able to make more guesses in a shorter period. As of 2015, it is believed that 32 bytes (256 bits) of randomness is sufficient for the typical use-case expected for the `!secrets` module.

For those who want to manage their own token length, you can explicitly specify how much randomness is used for tokens by giving an `int` argument to the various `token_*` functions. That argument is taken as the number of bytes of randomness to use.

Otherwise, if no argument is provided, or if the argument is `None`, the `token_*` functions use `DEFAULT_ENTROPY` instead.

<div class="data">

DEFAULT_ENTROPY

Default number of bytes of randomness used by the `token_*` functions.

The exact value is subject to change at any time, including during maintenance releases.

</div>

## Other functions

<div class="function">

compare_digest(a, b)

Return `True` if strings or `bytes-like objects <bytes-like object>` *a* and *b* are equal, otherwise `False`, using a "constant-time compare" to reduce the risk of [timing attacks](https://web.archive.org/web/20250815071532/https://codahale.com/a-lesson-in-timing-attacks/). See `hmac.compare_digest` for additional details.

</div>

## Recipes and best practices

This section shows recipes and best practices for using `!secrets` to manage a basic level of security.

Generate an eight-character alphanumeric password:

<div class="testcode">

import string import secrets alphabet = string.ascii_letters + string.digits password = ''.join(secrets.choice(alphabet) for i in range(8))

</div>

> [!NOTE]
> Applications should not `store passwords in a recoverable format <257>`, whether plain text or encrypted. They should be salted and hashed using a cryptographically strong one-way (irreversible) hash function.

Generate a ten-character alphanumeric password with at least one lowercase character, at least one uppercase character, and at least three digits:

<div class="testcode">

import string import secrets alphabet = string.ascii_letters + string.digits while True: password = ''.join(secrets.choice(alphabet) for i in range(10)) if (any(c.islower() for c in password) and any(c.isupper() for c in password) and sum(c.isdigit() for c in password) \>= 3): break

</div>

Generate an [XKCD-style passphrase](https://xkcd.com/936/):

<div class="testcode">

import secrets \# On standard Linux systems, use a convenient dictionary file. \# Other platforms may need to provide their own word-list. with open('/usr/share/dict/words') as f: words = \[word.strip() for word in f\] password = ' '.join(secrets.choice(words) for i in range(4))

</div>

Generate a hard-to-guess temporary URL containing a security token suitable for password recovery applications:

<div class="testcode">

import secrets url = 'https://example.com/reset=' + secrets.token_urlsafe()

</div>

# security_warnings

<div id="security-warnings">

<div class="index">

single: security considerations

</div>

</div>

# Security Considerations

The following modules have specific security considerations:

- `base64`: `base64 security considerations <base64-security>` in `4648`
- `hashlib`: `all constructors take a "usedforsecurity" keyword-only
  argument disabling known insecure and blocked algorithms
  <hashlib-usedforsecurity>`
- `http.server` is not suitable for production use, only implementing basic security checks. See the `security considerations <http.server-security>`.
- `logging`: `Logging configuration uses eval()
  <logging-eval-security>`
- `multiprocessing`: `Connection.recv() uses pickle
  <multiprocessing-recv-pickle-security>`
- `pickle`: `Restricting globals in pickle <pickle-restrict>`
- `random` shouldn't be used for security purposes, use `secrets` instead
- `shelve`: `shelve is based on pickle and thus unsuitable for
  dealing with untrusted sources <shelve-security>`
- `ssl`: `SSL/TLS security considerations <ssl-security>`
- `subprocess`: `Subprocess security considerations
  <subprocess-security>`
- `tempfile`: `mktemp is deprecated due to vulnerability to race
  conditions <tempfile-mktemp-deprecated>`
- `xml`: `XML security <xml-security>`
- `zipfile`: `maliciously prepared .zip files can cause disk volume
  exhaustion <zipfile-resources-limitations>`

The `-I` command line option can be used to run Python in isolated mode. When it cannot be used, the `-P` option or the `PYTHONSAFEPATH` environment variable can be used to not prepend a potentially unsafe path to `sys.path` such as the current directory, the script's directory or an empty string.

# `!select` --- Waiting for I/O completion

<div class="module" data-synopsis="Wait for I/O completion on multiple streams.">

select

</div>

------------------------------------------------------------------------

This module provides access to the `!select` and `!poll` functions available in most operating systems, `!devpoll` available on Solaris and derivatives, `!epoll` available on Linux 2.5+ and `!kqueue` available on most BSD. Note that on Windows, it only works for sockets; on other operating systems, it also works for other file types (in particular, on Unix, it works on pipes). It cannot be used on regular files to determine whether a file has grown since it was last read.

> [!NOTE]
> The `selectors` module allows high-level and efficient I/O multiplexing, built upon the `!select` module primitives. Users are encouraged to use the `selectors` module instead, unless they want precise control over the OS-level primitives used.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

The module defines the following:

<div class="exception">

error

A deprecated alias of `OSError`.

<div class="versionchanged">

3.3 Following `3151`, this class was made an alias of `OSError`.

</div>

</div>

<div class="function">

devpoll()

Returns a `/dev/poll` polling object; see section `devpoll-objects` below for the methods supported by devpoll objects.

`!devpoll` objects are linked to the number of file descriptors allowed at the time of instantiation. If your program reduces this value, `!devpoll` will fail. If your program increases this value, `!devpoll` may return an incomplete list of active file descriptors.

The new file descriptor is `non-inheritable <fd_inheritance>`.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.4 The new file descriptor is now non-inheritable.

</div>

<div class="availability">

Solaris.

</div>

</div>

<div class="function">

epoll(sizehint=-1, flags=0)

Return an edge polling object, which can be used as Edge or Level Triggered interface for I/O events.

*sizehint* informs epoll about the expected number of events to be registered. It must be positive, or `-1` to use the default. It is only used on older systems where `epoll_create1(2)` is not available; otherwise it has no effect (though its value is still checked).

*flags* is deprecated and completely ignored. However, when supplied, its value must be `0` or `select.EPOLL_CLOEXEC`, otherwise `OSError` is raised.

See the `epoll-objects` section below for the methods supported by epolling objects.

`epoll` objects support the context management protocol: when used in a `with` statement, the new file descriptor is automatically closed at the end of the block.

The new file descriptor is `non-inheritable <fd_inheritance>`.

<div class="versionchanged">

3.3 Added the *flags* parameter.

</div>

<div class="versionchanged">

3.4 Support for the `with` statement was added. The new file descriptor is now non-inheritable.

</div>

<div class="deprecated">

3.4 The *flags* parameter. `select.EPOLL_CLOEXEC` is used by default now. Use `os.set_inheritable` to make the file descriptor inheritable.

</div>

<div class="versionchanged">

3.15

When CPython is built, this function may be disabled using `--disable-epoll`.

</div>

<div class="availability">

Linux \>= 2.5.44.

</div>

</div>

<div class="function">

poll()

Returns a polling object, which supports registering and unregistering file descriptors, and then polling them for I/O events; see section `poll-objects` below for the methods supported by polling objects.

<div class="availability">

Unix.

</div>

</div>

<div class="function">

kqueue()

Returns a kernel queue object; see section `kqueue-objects` below for the methods supported by kqueue objects.

The new file descriptor is `non-inheritable <fd_inheritance>`.

<div class="versionchanged">

3.4 The new file descriptor is now non-inheritable.

</div>

<div class="availability">

BSD, macOS.

</div>

</div>

<div class="function">

kevent(ident, filter=KQ_FILTER_READ, flags=KQ_EV_ADD, fflags=0, data=0, udata=0)

Returns a kernel event object; see section `kevent-objects` below for the methods supported by kevent objects.

<div class="availability">

BSD, macOS.

</div>

</div>

<div class="function">

select(rlist, wlist, xlist, timeout=None)

This is a straightforward interface to the Unix `!select` system call. The first three arguments are iterables of 'waitable objects': either integers representing file descriptors or objects with a parameterless method named `~io.IOBase.fileno` returning such an integer:

- *rlist*: wait until ready for reading
- *wlist*: wait until ready for writing
- *xlist*: wait for an "exceptional condition" (see the manual page for what your system considers such a condition)

Empty iterables are allowed, but acceptance of three empty iterables is platform-dependent. (It is known to work on Unix but not on Windows.) The optional *timeout* argument specifies a time-out in seconds; it may be a non-integer to specify fractions of seconds. When the *timeout* argument is omitted or `None`, the function blocks until at least one file descriptor is ready. A time-out value of zero specifies a poll and never blocks.

The return value is a triple of lists of objects that are ready: subsets of the first three arguments. When the time-out is reached without a file descriptor becoming ready, three empty lists are returned.

<div class="index">

single: socket() (in module socket) single: popen() (in module os)

</div>

Among the acceptable object types in the iterables are Python `file
objects <file object>` (e.g. `sys.stdin`, or objects returned by `open` or `os.popen`), socket objects returned by `socket.socket`. You may also define a `wrapper` class yourself, as long as it has an appropriate `~io.IOBase.fileno` method (that really returns a file descriptor, not just a random integer).

> [!NOTE]
> <div class="index">
>
> single: WinSock
>
> </div>
>
> File objects on Windows are not acceptable, but sockets are. On Windows, the underlying `!select` function is provided by the WinSock library, and does not handle file descriptors that don't originate from WinSock.

<div class="versionchanged">

3.5 The function is now retried with a recomputed timeout when interrupted by a signal, except if the signal handler raises an exception (see `475` for the rationale), instead of raising `InterruptedError`.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float.

</div>

</div>

<div class="data">

PIPE_BUF

The minimum number of bytes which can be written without blocking to a pipe when the pipe has been reported as ready for writing by `~select.select`, `!poll` or another interface in this module. This doesn't apply to other kinds of file-like objects such as sockets.

This value is guaranteed by POSIX to be at least 512.

<div class="availability">

Unix

</div>

<div class="versionadded">

3.2

</div>

</div>

## `/dev/poll` polling objects

Solaris and derivatives have `/dev/poll`. While `!select` is *O*(*highest file descriptor*) and `!poll` is *O*(*number of file descriptors*), `/dev/poll` is *O*(*active file descriptors*).

`/dev/poll` behaviour is very close to the standard `!poll` object.

<div class="method">

devpoll.close()

Close the file descriptor of the polling object.

<div class="versionadded">

3.4

</div>

</div>

<div class="attribute">

devpoll.closed

`True` if the polling object is closed.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

devpoll.fileno()

Return the file descriptor number of the polling object.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

devpoll.register(fd\[, eventmask\])

Register a file descriptor with the polling object. Future calls to the `poll` method will then check whether the file descriptor has any pending I/O events. *fd* can be either an integer, or an object with a `~io.IOBase.fileno` method that returns an integer. File objects implement `!fileno`, so they can also be used as the argument.

*eventmask* is an optional bitmask describing the type of events you want to check for. The constants are the same as with `!poll` object. The default value is a combination of the constants `POLLIN`, `POLLPRI`, and `POLLOUT`.

> [!WARNING]
> Registering a file descriptor that's already registered is not an error, but the result is undefined. The appropriate action is to unregister or modify it first. This is an important difference compared with `!poll`.

</div>

<div class="method">

devpoll.modify(fd\[, eventmask\])

This method does an `unregister` followed by a `register`. It is (a bit) more efficient than doing the same explicitly.

</div>

<div class="method">

devpoll.unregister(fd)

Remove a file descriptor being tracked by a polling object. Just like the `register` method, *fd* can be an integer or an object with a `~io.IOBase.fileno` method that returns an integer.

Attempting to remove a file descriptor that was never registered is safely ignored.

</div>

<div class="method">

devpoll.poll(\[timeout\])

Polls the set of registered file descriptors, and returns a possibly empty list containing `(fd, event)` 2-tuples for the descriptors that have events or errors to report. *fd* is the file descriptor, and *event* is a bitmask with bits set for the reported events for that descriptor --- `POLLIN` for waiting input, `POLLOUT` to indicate that the descriptor can be written to, and so forth. An empty list indicates that the call timed out and no file descriptors had any events to report. If *timeout* is given, it specifies the length of time in milliseconds which the system will wait for events before returning. If *timeout* is omitted, -1, or `None`, the call will block until there is an event for this poll object.

<div class="versionchanged">

3.5 The function is now retried with a recomputed timeout when interrupted by a signal, except if the signal handler raises an exception (see `475` for the rationale), instead of raising `InterruptedError`.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float.

</div>

</div>

## Edge and level trigger polling (epoll) objects

> <https://linux.die.net/man/4/epoll>
>
> The *eventmask* is a bit mask using the following constants:
>
> | Constant | Meaning |
> |----|----|
> | `EPOLLIN` | Available for read. |
> | `EPOLLOUT` | Available for write. |
> | `EPOLLPRI` | Urgent data for read. |
> | `EPOLLERR` | Error condition happened on the associated fd. |
> | `EPOLLHUP` | Hang up happened on the associated fd. |
> | `EPOLLET` | Set Edge Trigger behavior, the default is Level Trigger behavior. |
> | `EPOLLONESHOT` | Set one-shot behavior. After one event is pulled out, the fd is internally disabled. |
> | `EPOLLEXCLUSIVE` | Wake only one epoll object when the associated fd has an event. The default (if this flag is not set) is to wake all epoll objects polling on an fd. |
> | `EPOLLRDHUP` | Stream socket peer closed connection or shut down writing half of connection. |
> | `EPOLLRDNORM` | Equivalent to `EPOLLIN` |
> | `EPOLLRDBAND` | Priority data band can be read. |
> | `EPOLLWRNORM` | Equivalent to `EPOLLOUT`. |
> | `EPOLLWRBAND` | Priority data may be written. |
> | `EPOLLMSG` | Ignored. |
> | `EPOLLWAKEUP` | Prevents sleep during event waiting. |
>
> <div class="versionadded">
>
> 3.6 `EPOLLEXCLUSIVE` was added. It's only supported by Linux Kernel 4.5 or later.
>
> </div>
>
> <div class="versionadded">
>
> 3.14 `EPOLLWAKEUP` was added. It's only supported by Linux Kernel 3.5 or later.
>
> </div>

<div class="method">

epoll.close()

Close the control file descriptor of the epoll object.

</div>

<div class="attribute">

epoll.closed

`True` if the epoll object is closed.

</div>

<div class="method">

epoll.fileno()

Return the file descriptor number of the control fd.

</div>

<div class="method">

epoll.fromfd(fd)

Create an epoll object from a given file descriptor.

</div>

<div class="method">

epoll.register(fd\[, eventmask\])

Register a file descriptor *fd* with the epoll object.

</div>

<div class="method">

epoll.modify(fd, eventmask)

Modify a registered file descriptor *fd*.

</div>

<div class="method">

epoll.unregister(fd)

Remove a registered file descriptor from the epoll object.

<div class="versionchanged">

3.9 The method no longer ignores the `~errno.EBADF` error.

</div>

</div>

<div class="method">

epoll.poll(timeout=None, maxevents=-1)

Wait for events. If *timeout* is given, it specifies the length of time in seconds (may be non-integer) which the system will wait for events before returning.

<div class="versionchanged">

3.5 The function is now retried with a recomputed timeout when interrupted by a signal, except if the signal handler raises an exception (see `475` for the rationale), instead of raising `InterruptedError`.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float.

</div>

</div>

## Polling objects

The `!poll` system call, supported on most Unix systems, provides better scalability for network servers that service many, many clients at the same time. `!poll` scales better because the system call only requires listing the file descriptors of interest, while `!select` builds a bitmap, turns on bits for the fds of interest, and then afterward the whole bitmap has to be linearly scanned again. `!select` is *O*(*highest file descriptor*), while `!poll` is *O*(*number of file descriptors*).

<div class="method">

poll.register(fd\[, eventmask\])

Register a file descriptor with the polling object. Future calls to the `poll` method will then check whether the file descriptor has any pending I/O events. *fd* can be either an integer, or an object with a `~io.IOBase.fileno` method that returns an integer. File objects implement `!fileno`, so they can also be used as the argument.

*eventmask* is an optional bitmask describing the type of events you want to check for, and can be a combination of the constants `POLLIN`, `POLLPRI`, and `POLLOUT`, described in the table below. If not specified, the default value used will check for all 3 types of events.

| Constant | Meaning |
|----|----|
| `POLLIN` | There is data to read. |
| `POLLPRI` | There is urgent data to read. |
| `POLLOUT` | Ready for output: writing will not block. |
| `POLLERR` | Error condition of some sort. |
| `POLLHUP` | Hung up. |
| `POLLRDHUP` | Stream socket peer closed connection, or shut down writing half of connection. |
| `POLLNVAL` | Invalid request: descriptor not open. |

Registering a file descriptor that's already registered is not an error, and has the same effect as registering the descriptor exactly once.

</div>

<div class="method">

poll.modify(fd, eventmask)

Modifies an already registered fd. This has the same effect as `register(fd, eventmask)`. Attempting to modify a file descriptor that was never registered causes an `OSError` exception with errno `ENOENT` to be raised.

</div>

<div class="method">

poll.unregister(fd)

Remove a file descriptor being tracked by a polling object. Just like the `register` method, *fd* can be an integer or an object with a `~io.IOBase.fileno` method that returns an integer.

Attempting to remove a file descriptor that was never registered causes a `KeyError` exception to be raised.

</div>

<div class="method">

poll.poll(\[timeout\])

Polls the set of registered file descriptors, and returns a possibly empty list containing `(fd, event)` 2-tuples for the descriptors that have events or errors to report. *fd* is the file descriptor, and *event* is a bitmask with bits set for the reported events for that descriptor --- `POLLIN` for waiting input, `POLLOUT` to indicate that the descriptor can be written to, and so forth. An empty list indicates that the call timed out and no file descriptors had any events to report. If *timeout* is given, it specifies the length of time in milliseconds which the system will wait for events before returning. If *timeout* is omitted, negative, or `None`, the call will block until there is an event for this poll object.

<div class="versionchanged">

3.5 The function is now retried with a recomputed timeout when interrupted by a signal, except if the signal handler raises an exception (see `475` for the rationale), instead of raising `InterruptedError`.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float. If `ppoll()` function is available, *timeout* has a resolution of `1` ns (`1e-6` ms) instead of `1` ms.

</div>

</div>

## Kqueue objects

<div class="method">

kqueue.close()

Close the control file descriptor of the kqueue object.

</div>

<div class="attribute">

kqueue.closed

`True` if the kqueue object is closed.

</div>

<div class="method">

kqueue.fileno()

Return the file descriptor number of the control fd.

</div>

<div class="method">

kqueue.fromfd(fd)

Create a kqueue object from a given file descriptor.

</div>

<div class="method">

kqueue.control(changelist, max_events\[, timeout\]) -\> eventlist

Low level interface to kevent

- changelist must be an iterable of kevent objects or `None`
- max_events must be 0 or a positive integer
- timeout in seconds (non-integers are possible); the default is `None`, to wait forever

<div class="versionchanged">

3.5 The function is now retried with a recomputed timeout when interrupted by a signal, except if the signal handler raises an exception (see `475` for the rationale), instead of raising `InterruptedError`.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float.

</div>

</div>

## Kevent objects

<https://man.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2>

<div class="attribute">

kevent.ident

Value used to identify the event. The interpretation depends on the filter but it's usually the file descriptor. In the constructor ident can either be an int or an object with a `~io.IOBase.fileno` method. kevent stores the integer internally.

</div>

<div class="attribute">

kevent.filter

Name of the kernel filter.

| Constant | Meaning |
|----|----|
| `KQ_FILTER_READ` | Takes a descriptor and returns whenever there is data available to read. |
| `KQ_FILTER_WRITE` | Takes a descriptor and returns whenever there is data available to write. |
| `KQ_FILTER_AIO` | AIO requests. |
| `KQ_FILTER_VNODE` | Returns when one or more of the requested events watched in *fflags* occurs. |
| `KQ_FILTER_PROC` | Watch for events on a process ID. |
| `KQ_FILTER_NETDEV` | Watch for events on a network device (not available on macOS). |
| `KQ_FILTER_SIGNAL` | Returns whenever the watched signal is delivered to the process. |
| `KQ_FILTER_TIMER` | Establishes an arbitrary timer. |

</div>

<div class="attribute">

kevent.flags

Filter action.

| Constant         | Meaning                                      |
|------------------|----------------------------------------------|
| `KQ_EV_ADD`      | Adds or modifies an event.                   |
| `KQ_EV_DELETE`   | Removes an event from the queue.             |
| `KQ_EV_ENABLE`   | Permits control() to return the event.       |
| `KQ_EV_DISABLE`  | Disables event.                              |
| `KQ_EV_ONESHOT`  | Removes event after first occurrence.        |
| `KQ_EV_CLEAR`    | Reset the state after an event is retrieved. |
| `KQ_EV_SYSFLAGS` | Internal event.                              |
| `KQ_EV_FLAG1`    | Internal event.                              |
| `KQ_EV_EOF`      | Filter-specific EOF condition.               |
| `KQ_EV_ERROR`    | See return values.                           |

</div>

<div class="attribute">

kevent.fflags

Filter-specific flags.

`KQ_FILTER_READ` and `KQ_FILTER_WRITE` filter flags:

| Constant        | Meaning                            |
|-----------------|------------------------------------|
| `KQ_NOTE_LOWAT` | Low water mark of a socket buffer. |

`KQ_FILTER_VNODE` filter flags:

| Constant         | Meaning                         |
|------------------|---------------------------------|
| `KQ_NOTE_DELETE` | *unlink()* was called.          |
| `KQ_NOTE_WRITE`  | A write occurred.               |
| `KQ_NOTE_EXTEND` | The file was extended.          |
| `KQ_NOTE_ATTRIB` | An attribute was changed.       |
| `KQ_NOTE_LINK`   | The link count has changed.     |
| `KQ_NOTE_RENAME` | The file was renamed.           |
| `KQ_NOTE_REVOKE` | Access to the file was revoked. |

`KQ_FILTER_PROC` filter flags:

| Constant            | Meaning                                         |
|---------------------|-------------------------------------------------|
| `KQ_NOTE_EXIT`      | The process has exited.                         |
| `KQ_NOTE_FORK`      | The process has called *fork()*.                |
| `KQ_NOTE_EXEC`      | The process has executed a new process.         |
| `KQ_NOTE_PCTRLMASK` | Internal filter flag.                           |
| `KQ_NOTE_PDATAMASK` | Internal filter flag.                           |
| `KQ_NOTE_TRACK`     | Follow a process across *fork()*.               |
| `KQ_NOTE_CHILD`     | Returned on the child process for *NOTE_TRACK*. |
| `KQ_NOTE_TRACKERR`  | Unable to attach to a child.                    |

`KQ_FILTER_NETDEV` filter flags (not available on macOS):

| Constant           | Meaning                |
|--------------------|------------------------|
| `KQ_NOTE_LINKUP`   | Link is up.            |
| `KQ_NOTE_LINKDOWN` | Link is down.          |
| `KQ_NOTE_LINKINV`  | Link state is invalid. |

</div>

<div class="attribute">

kevent.data

Filter-specific data.

</div>

<div class="attribute">

kevent.udata

User-defined value.

</div>

# `!selectors` --- High-level I/O multiplexing

<div class="module" data-synopsis="High-level I/O multiplexing.">

selectors

</div>

<div class="versionadded">

3.4

</div>

**Source code:** `Lib/selectors.py`

------------------------------------------------------------------------

## Introduction

This module allows high-level and efficient I/O multiplexing, built upon the `select` module primitives. Users are encouraged to use this module instead, unless they want precise control over the OS-level primitives used.

It defines a `BaseSelector` abstract base class, along with several concrete implementations (`KqueueSelector`, `EpollSelector`...), that can be used to wait for I/O readiness notification on multiple file objects. In the following, "file object" refers to any object with a `~io.IOBase.fileno` method, or a raw file descriptor. See `file object`.

`DefaultSelector` is an alias to the most efficient implementation available on the current platform: this should be the default choice for most users.

> [!NOTE]
> The type of file objects supported depends on the platform: on Windows, sockets are supported, but not pipes, whereas on Unix, both are supported (some other types may be supported as well, such as fifos or special file devices).

<div class="seealso">

`select`  
Low-level I/O multiplexing module.

</div>

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

## Classes

Classes hierarchy:

    BaseSelector
    +-- SelectSelector
    +-- PollSelector
    +-- EpollSelector
    +-- DevpollSelector
    +-- KqueueSelector

In the following, *events* is a bitwise mask indicating which I/O events should be waited for on a given file object. It can be a combination of the module's constants below:

> <table>
> <colgroup>
> <col style="width: 33%" />
> <col style="width: 66%" />
> </colgroup>
> <thead>
> <tr>
> <th><p>Constant</p></th>
> <th><p>Meaning</p></th>
> </tr>
> </thead>
> <tbody>
> <tr>
> <td><div class="data">
> <p>EVENT_READ</p>
> </div></td>
> <td><p>Available for read</p></td>
> </tr>
> <tr>
> <td><div class="data">
> <p>EVENT_WRITE</p>
> </div></td>
> <td><p>Available for write</p></td>
> </tr>
> </tbody>
> </table>

<div class="SelectorKey">

A `SelectorKey` is a `~collections.namedtuple` used to associate a file object to its underlying file descriptor, selected event mask and attached data. It is returned by several `BaseSelector` methods.

<div class="attribute">

fileobj

File object registered.

</div>

<div class="attribute">

fd

Underlying file descriptor.

</div>

<div class="attribute">

events

Events that must be waited for on this file object.

</div>

<div class="attribute">

data

Optional opaque data associated to this file object: for example, this could be used to store a per-client session ID.

</div>

</div>

<div class="BaseSelector">

A `BaseSelector` is used to wait for I/O event readiness on multiple file objects. It supports file stream registration, unregistration, and a method to wait for I/O events on those streams, with an optional timeout. It's an abstract base class, so cannot be instantiated. Use `DefaultSelector` instead, or one of `SelectSelector`, `KqueueSelector` etc. if you want to specifically use an implementation, and your platform supports it. `BaseSelector` and its concrete implementations support the `context manager` protocol.

<div class="method" data-abstractmethod="">

register(fileobj, events, data=None)

Register a file object for selection, monitoring it for I/O events.

*fileobj* is the file object to monitor. It may either be an integer file descriptor or an object with a `fileno()` method. *events* is a bitwise mask of events to monitor. *data* is an opaque object.

This returns a new `SelectorKey` instance, or raises a `ValueError` in case of invalid event mask or file descriptor, or `KeyError` if the file object is already registered.

</div>

<div class="method" data-abstractmethod="">

unregister(fileobj)

Unregister a file object from selection, removing it from monitoring. A file object shall be unregistered prior to being closed.

*fileobj* must be a file object previously registered.

This returns the associated `SelectorKey` instance, or raises a `KeyError` if *fileobj* is not registered. It will raise `ValueError` if *fileobj* is invalid (e.g. it has no `fileno()` method or its `fileno()` method has an invalid return value).

</div>

<div class="method">

modify(fileobj, events, data=None)

Change a registered file object's monitored events or attached data.

This is equivalent to `BaseSelector.unregister(fileobj)` followed by `BaseSelector.register(fileobj, events, data)`, except that it can be implemented more efficiently.

This returns a new `SelectorKey` instance, or raises a `ValueError` in case of invalid event mask or file descriptor, or `KeyError` if the file object is not registered.

</div>

<div class="method" data-abstractmethod="">

select(timeout=None)

Wait until some registered file objects become ready, or the timeout expires.

If `timeout > 0`, this specifies the maximum wait time, in seconds. If `timeout <= 0`, the call won't block, and will report the currently ready file objects. If *timeout* is `None`, the call will block until a monitored file object becomes ready.

This returns a list of `(key, events)` tuples, one for each ready file object.

*key* is the `SelectorKey` instance corresponding to a ready file object. *events* is a bitmask of events ready on this file object.

> [!NOTE]
> This method can return before any file object becomes ready or the timeout has elapsed if the current process receives a signal: in this case, an empty list will be returned.

<div class="versionchanged">

3.5 The selector is now retried with a recomputed timeout when interrupted by a signal if the signal handler did not raise an exception (see `475` for the rationale), instead of returning an empty list of events before the timeout.

</div>

</div>

<div class="method">

close()

Close the selector.

This must be called to make sure that any underlying resource is freed. The selector shall not be used once it has been closed.

</div>

<div class="method">

get_key(fileobj)

Return the key associated with a registered file object.

This returns the `SelectorKey` instance associated to this file object, or raises `KeyError` if the file object is not registered.

</div>

<div class="method" data-abstractmethod="">

get_map()

Return a mapping of file objects to selector keys.

This returns a `~collections.abc.Mapping` instance mapping registered file objects to their associated `SelectorKey` instance.

</div>

</div>

<div class="DefaultSelector()">

The default selector class, using the most efficient implementation available on the current platform. This should be the default choice for most users.

</div>

<div class="SelectSelector()">

`select.select`-based selector.

</div>

<div class="PollSelector()">

`select.poll`-based selector.

</div>

<div class="EpollSelector()">

`select.epoll`-based selector.

<div class="method">

fileno()

This returns the file descriptor used by the underlying `select.epoll` object.

</div>

</div>

<div class="DevpollSelector()">

`select.devpoll`-based selector.

<div class="method">

fileno()

This returns the file descriptor used by the underlying `select.devpoll` object.

</div>

<div class="versionadded">

3.5

</div>

</div>

<div class="KqueueSelector()">

`select.kqueue`-based selector.

<div class="method">

fileno()

This returns the file descriptor used by the underlying `select.kqueue` object.

</div>

</div>

## Examples

Here is a simple echo server implementation:

    import selectors
    import socket

    sel = selectors.DefaultSelector()

    def accept(sock, mask):
        conn, addr = sock.accept()  # Should be ready
        print('accepted', conn, 'from', addr)
        conn.setblocking(False)
        sel.register(conn, selectors.EVENT_READ, read)

    def read(conn, mask):
        data = conn.recv(1000)  # Should be ready
        if data:
            print('echoing', repr(data), 'to', conn)
            conn.send(data)  # Hope it won't block
        else:
            print('closing', conn)
            sel.unregister(conn)
            conn.close()

    sock = socket.socket()
    sock.bind(('localhost', 1234))
    sock.listen(100)
    sock.setblocking(False)
    sel.register(sock, selectors.EVENT_READ, accept)

    while True:
        events = sel.select()
        for key, mask in events:
            callback = key.data
            callback(key.fileobj, mask)

# `!shelve` --- Python object persistence

<div class="module" data-synopsis="Python object persistence.">

shelve

</div>

**Source code:** `Lib/shelve.py`

<div class="index">

pair: module; pickle

</div>

------------------------------------------------------------------------

A "shelf" is a persistent, dictionary-like object. The difference with "dbm" databases is that the values (not the keys!) in a shelf can be essentially arbitrary Python objects --- anything that the `pickle` module can handle. This includes most class instances, recursive data types, and objects containing lots of shared sub-objects. The keys are ordinary strings.

<div class="function">

open(filename, flag='c', protocol=None, writeback=False, \*, serializer=None, deserializer=None)

Open a persistent dictionary. The filename specified is the base filename for the underlying database. As a side-effect, an extension may be added to the filename and more than one file may be created. By default, the underlying database file is opened for reading and writing. The optional *flag* parameter has the same interpretation as the *flag* parameter of `dbm.open`.

By default, pickles created with `pickle.DEFAULT_PROTOCOL` are used to serialize values. The version of the pickle protocol can be specified with the *protocol* parameter.

Because of Python semantics, a shelf cannot know when a mutable persistent-dictionary entry is modified. By default modified objects are written *only* when assigned to the shelf (see `shelve-example`). If the optional *writeback* parameter is set to `True`, all entries accessed are also cached in memory, and written back on `~Shelf.sync` and `~Shelf.close`; this can make it handier to mutate mutable entries in the persistent dictionary, but, if many entries are accessed, it can consume vast amounts of memory for the cache, and it can make the close operation very slow since all accessed entries are written back (there is no way to determine which accessed entries are mutable, nor which ones were actually mutated).

By default, `!shelve` uses `pickle.dumps` and `pickle.loads` for serializing and deserializing. This can be changed by supplying *serializer* and *deserializer*, respectively.

The *serializer* argument must be a callable which takes an object `obj` and the *protocol* as inputs and returns the representation `obj` as a `bytes-like object`; the *protocol* value may be ignored by the serializer.

The *deserializer* argument must be a callable which takes a serialized object given as a `bytes` object and returns the corresponding object.

A `ShelveError` is raised if *serializer* is given but *deserializer* is not, or vice-versa.

<div class="versionchanged">

3.10 `pickle.DEFAULT_PROTOCOL` is now used as the default pickle protocol.

</div>

<div class="versionchanged">

3.11 Accepts `path-like object` for filename.

</div>

<div class="versionchanged">

3.15 Accepts custom *serializer* and *deserializer* functions in place of `pickle.dumps` and `pickle.loads`.

</div>

> [!NOTE]
> Do not rely on the shelf being closed automatically; always call `~Shelf.close` explicitly when you don't need it any more, or use `shelve.open` as a context manager:
>
>     with shelve.open('spam') as db:
>         db['eggs'] = 'eggs'

</div>

<div id="shelve-security">

> [!WARNING]
> Because the `!shelve` module is backed by `pickle`, it is insecure to load a shelf from an untrusted source. Like with pickle, loading a shelf can execute arbitrary code.

</div>

Shelf objects support most of the methods and operations supported by dictionaries (except copying, constructors and operators `|` and `|=`). This eases the transition from dictionary based scripts to those requiring persistent storage.

Two additional methods are supported:

<div class="method">

Shelf.sync()

Write back all entries in the cache if the shelf was opened with *writeback* set to `True`. Also empty the cache and synchronize the persistent dictionary on disk, if feasible. This is called automatically when `reorganize` is called or the shelf is closed with `close`.

</div>

<div class="method">

Shelf.reorganize()

Calls `sync` and attempts to shrink space used on disk by removing empty space resulting from deletions.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

Shelf.close()

Synchronize and close the persistent *dict* object. Operations on a closed shelf will fail with a `ValueError`.

</div>

<div class="seealso">

[Persistent dictionary recipe](https://code.activestate.com/recipes/576642-persistent-dict-with-multiple-standard-file-format/) with widely supported storage formats and having the speed of native dictionaries.

</div>

## Restrictions

<div class="index">

pair: module; dbm.ndbm pair: module; dbm.gnu

</div>

- The choice of which database package will be used (such as `dbm.ndbm` or `dbm.gnu`) depends on which interface is available. Therefore it is not safe to open the database directly using `dbm`. The database is also (unfortunately) subject to the limitations of `dbm`, if it is used --- this means that (the pickled representation of) the objects stored in the database should be fairly small, and in rare cases key collisions may cause the database to refuse updates.
- The `!shelve` module does not support *concurrent* read/write access to shelved objects. (Multiple simultaneous read accesses are safe.) When a program has a shelf open for writing, no other program should have it open for reading or writing. Unix file locking can be used to solve this, but this differs across Unix versions and requires knowledge about the database implementation used.
- On macOS `dbm.ndbm` can silently corrupt the database file on updates, which can cause hard crashes when trying to read from the database.
- `Shelf.reorganize` may not be available for all database packages and may temporarily increase resource usage (especially disk space) when called. Additionally, it will never run automatically and instead needs to be called explicitly.

<div class="Shelf(dict, protocol=None, writeback=False, \ keyencoding='utf-8', *, serializer=None, deserializer=None)">

A subclass of `collections.abc.MutableMapping` which stores pickled values in the *dict* object.

By default, pickles created with `pickle.DEFAULT_PROTOCOL` are used to serialize values. The version of the pickle protocol can be specified with the *protocol* parameter. See the `pickle` documentation for a discussion of the pickle protocols.

If the *writeback* parameter is `True`, the object will hold a cache of all entries accessed and write them back to the *dict* at sync and close times. This allows natural operations on mutable entries, but can consume much more memory and make sync and close take a long time.

The *keyencoding* parameter is the encoding used to encode keys before they are used with the underlying dict.

The *serializer* and *deserializer* parameters have the same interpretation as in `~shelve.open`.

A `Shelf` object can also be used as a context manager, in which case it will be automatically closed when the `with` block ends.

<div class="versionchanged">

3.2 Added the *keyencoding* parameter; previously, keys were always encoded in UTF-8.

</div>

<div class="versionchanged">

3.4 Added context manager support.

</div>

<div class="versionchanged">

3.10 `pickle.DEFAULT_PROTOCOL` is now used as the default pickle protocol.

</div>

<div class="versionchanged">

3.15 Added the *serializer* and *deserializer* parameters.

</div>

</div>

<div class="BsdDbShelf(dict, protocol=None, writeback=False, \ keyencoding='utf-8', *, \ serializer=None, deserializer=None)">

A subclass of `Shelf` which exposes `!first`, `!next`, `!previous`, `!last` and `!set_location` methods. These are available in the third-party `!bsddb` module from [pybsddb](https://www.jcea.es/programacion/pybsddb.htm) but not in other database modules. The *dict* object passed to the constructor must support those methods. This is generally accomplished by calling one of `!bsddb.hashopen`, `!bsddb.btopen` or `!bsddb.rnopen`. The optional *protocol*, *writeback*, *keyencoding*, *serializer* and *deserializer* parameters have the same interpretation as in `~shelve.open`.

<div class="versionchanged">

3.15 Added the *serializer* and *deserializer* parameters.

</div>

</div>

<div class="DbfilenameShelf(filename, flag='c', protocol=None, \ writeback=False, *, serializer=None, \ deserializer=None)">

A subclass of `Shelf` which accepts a *filename* instead of a dict-like object. The underlying file will be opened using `dbm.open`. By default, the file will be created and opened for both read and write. The optional *flag* parameter has the same interpretation as for the `.open` function. The optional *protocol*, *writeback*, *serializer* and *deserializer* parameters have the same interpretation as in `~shelve.open`.

<div class="versionchanged">

3.15 Added the *serializer* and *deserializer* parameters.

</div>

</div>

## Example

To summarize the interface (`key` is a string, `data` is an arbitrary object):

    import shelve

    d = shelve.open(filename)  # open -- file may get suffix added by low-level
                               # library

    d[key] = data              # store data at key (overwrites old data if
                               # using an existing key)
    data = d[key]              # retrieve a COPY of data at key (raise KeyError
                               # if no such key)
    del d[key]                 # delete data stored at key (raises KeyError
                               # if no such key)

    flag = key in d            # true if the key exists
    klist = list(d.keys())     # a list of all existing keys (slow!)

    # as d was opened WITHOUT writeback=True, beware:
    d['xx'] = [0, 1, 2]        # this works as expected, but...
    d['xx'].append(3)          # *this doesn't!* -- d['xx'] is STILL [0, 1, 2]!

    # having opened d without writeback=True, you need to code carefully:
    temp = d['xx']             # extracts the copy
    temp.append(5)             # mutates the copy
    d['xx'] = temp             # stores the copy right back, to persist it

    # or, d=shelve.open(filename,writeback=True) would let you just code
    # d['xx'].append(5) and have it work as expected, BUT it would also
    # consume more memory and make the d.close() operation slower.

    d.close()                  # close it

## Exceptions

<div class="exception">

ShelveError

Exception raised when one of the arguments *deserializer* and *serializer* is missing in the `~shelve.open`, `Shelf`, `BsdDbShelf` and `DbfilenameShelf`.

The *deserializer* and *serializer* arguments must be given together.

<div class="versionadded">

3.15

</div>

</div>

<div class="seealso">

Module `dbm`  
Generic interface to `dbm`-style databases.

Module `pickle`  
Object serialization used by `!shelve`.

</div>

# `!shlex` --- Simple lexical analysis

<div class="module" data-synopsis="Simple lexical analysis for Unix shell-like languages.">

shlex

</div>

**Source code:** `Lib/shlex.py`

------------------------------------------------------------------------

The `~shlex.shlex` class makes it easy to write lexical analyzers for simple syntaxes resembling that of the Unix shell. This will often be useful for writing minilanguages, (for example, in run control files for Python applications) or for parsing quoted strings.

The `!shlex` module defines the following functions:

<div class="function">

split(s, comments=False, posix=True)

Split the string *s* using shell-like syntax. If *comments* is `False` (the default), the parsing of comments in the given string will be disabled (setting the `~shlex.commenters` attribute of the `~shlex.shlex` instance to the empty string). This function operates in POSIX mode by default, but uses non-POSIX mode if the *posix* argument is false.

<div class="versionchanged">

3.12 Passing `None` for *s* argument now raises an exception, rather than reading `sys.stdin`.

</div>

</div>

<div class="function">

join(split_command)

Concatenate the tokens of the list *split_command* and return a string. This function is the inverse of `split`.

> \>\>\> from shlex import join \>\>\> print(join(\['echo', '-n', 'Multiple words'\])) echo -n 'Multiple words'

The returned value is shell-escaped to protect against injection vulnerabilities (see `quote`).

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

quote(s, \*, force=False)

Return a shell-escaped version of the string *s*. The returned value is a string that can safely be used as one token in a shell command line, for cases where you cannot use a list.

If *force* is `True`, then *s* is unconditionally quoted, even if it is already safe for a shell without being quoted.

<div id="shlex-quote-warning">

> [!WARNING]
> The `shlex` module is **only designed for Unix shells**.
>
> The `quote` function is not guaranteed to be correct on non-POSIX compliant shells or shells from other operating systems such as Windows. Executing commands quoted by this module on such shells can open up the possibility of a command injection vulnerability.
>
> Consider using functions that pass command arguments with lists such as `subprocess.run` with `shell=False`.

</div>

This idiom would be unsafe:

> \>\>\> filename = 'somefile; rm -rf ~' \>\>\> command = 'ls -l {}'.format(filename) \>\>\> print(command) \# executed by a shell: boom! ls -l somefile; rm -rf ~

`quote` lets you plug the security hole:

> \>\>\> from shlex import quote \>\>\> command = 'ls -l {}'.format(quote(filename)) \>\>\> print(command) ls -l 'somefile; rm -rf ~' \>\>\> remote_command = 'ssh home {}'.format(quote(command)) \>\>\> print(remote_command) ssh home 'ls -l '"'"'somefile; rm -rf ~'"'"''

The quoting is compatible with UNIX shells and with `split`:

> \>\>\> from shlex import split \>\>\> remote_command = split(remote_command) \>\>\> remote_command \['ssh', 'home', "ls -l 'somefile; rm -rf ~'"\] \>\>\> command = split(remote_command\[-1\]) \>\>\> command \['ls', '-l', 'somefile; rm -rf ~'\]

The *force* keyword can be used to produce consistent behavior when escaping multiple strings:

> \>\>\> from shlex import quote \>\>\> filenames = \['my first file', 'file2', 'file 3'\] \>\>\> filenames_some_escaped = \[quote(f) for f in filenames\] \>\>\> filenames_some_escaped \["'my first file'", 'file2', "'file 3'"\] \>\>\> filenames_all_escaped = \[quote(f, force=True) for f in filenames\] \>\>\> filenames_all_escaped \["'my first file'", "'file2'", "'file 3'"\]

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.16 The *force* keyword was added.

</div>

</div>

The `!shlex` module defines the following class:

<div class="shlex(instream=None, infile=None, posix=False, punctuation_chars=False)">

A `~shlex.shlex` instance or subclass instance is a lexical analyzer object. The initialization argument, if present, specifies where to read characters from. It must be a file-/stream-like object with `~io.TextIOBase.read` and `~io.TextIOBase.readline` methods, or a string. If no argument is given, input will be taken from `sys.stdin`. The second optional argument is a filename string, which sets the initial value of the `~shlex.infile` attribute. If the *instream* argument is omitted or equal to `sys.stdin`, this second argument defaults to "stdin". The *posix* argument defines the operational mode: when *posix* is not true (default), the `~shlex.shlex` instance will operate in compatibility mode. When operating in POSIX mode, `~shlex.shlex` will try to be as close as possible to the POSIX shell parsing rules. The *punctuation_chars* argument provides a way to make the behaviour even closer to how real shells parse. This can take a number of values: the default value, `False`, preserves the behaviour seen under Python 3.5 and earlier. If set to `True`, then parsing of the characters `();<>|&` is changed: any run of these characters (considered punctuation characters) is returned as a single token. If set to a non-empty string of characters, those characters will be used as the punctuation characters. Any characters in the `wordchars` attribute that appear in *punctuation_chars* will be removed from `wordchars`. See `improved-shell-compatibility` for more information. *punctuation_chars* can be set only upon `~shlex.shlex` instance creation and can't be modified later.

<div class="versionchanged">

3.6 The *punctuation_chars* parameter was added.

</div>

</div>

<div class="seealso">

Module `configparser`  
Parser for configuration files similar to the Windows `.ini` files.

</div>

## shlex Objects

A `~shlex.shlex` instance has the following methods:

<div class="method">

shlex.get_token()

Return a token. If tokens have been stacked using `push_token`, pop a token off the stack. Otherwise, read one from the input stream. If reading encounters an immediate end-of-file, `eof` is returned (the empty string (`''`) in non-POSIX mode, and `None` in POSIX mode).

</div>

<div class="method">

shlex.push_token(str)

Push the argument onto the token stack.

</div>

<div class="method">

shlex.read_token()

Read a raw token. Ignore the pushback stack, and do not interpret source requests. (This is not ordinarily a useful entry point, and is documented here only for the sake of completeness.)

</div>

<div class="method">

shlex.sourcehook(filename)

When `~shlex.shlex` detects a source request (see `source` below) this method is given the following token as argument, and expected to return a tuple consisting of a filename and an open file-like object.

Normally, this method first strips any quotes off the argument. If the result is an absolute pathname, or there was no previous source request in effect, or the previous source was a stream (such as `sys.stdin`), the result is left alone. Otherwise, if the result is a relative pathname, the directory part of the name of the file immediately before it on the source inclusion stack is prepended (this behavior is like the way the C preprocessor handles `#include "file.h"`).

The result of the manipulations is treated as a filename, and returned as the first component of the tuple, with `open` called on it to yield the second component. (Note: this is the reverse of the order of arguments in instance initialization!)

This hook is exposed so that you can use it to implement directory search paths, addition of file extensions, and other namespace hacks. There is no corresponding 'close' hook, but a shlex instance will call the `~io.IOBase.close` method of the sourced input stream when it returns EOF.

For more explicit control of source stacking, use the `push_source` and `pop_source` methods.

</div>

<div class="method">

shlex.push_source(newstream, newfile=None)

Push an input source stream onto the input stack. If the filename argument is specified it will later be available for use in error messages. This is the same method used internally by the `sourcehook` method.

</div>

<div class="method">

shlex.pop_source()

Pop the last-pushed input source from the input stack. This is the same method used internally when the lexer reaches EOF on a stacked input stream.

</div>

<div class="method">

shlex.error_leader(infile=None, lineno=None)

This method generates an error message leader in the format of a Unix C compiler error label; the format is `'"%s", line %d: '`, where the `%s` is replaced with the name of the current source file and the `%d` with the current input line number (the optional arguments can be used to override these).

This convenience is provided to encourage `!shlex` users to generate error messages in the standard, parseable format understood by Emacs and other Unix tools.

</div>

Instances of `~shlex.shlex` subclasses have some public instance variables which either control lexical analysis or can be used for debugging:

<div class="attribute">

shlex.commenters

The string of characters that are recognized as comment beginners. All characters from the comment beginner to end of line are ignored. Includes just `'#'` by default.

</div>

<div class="attribute">

shlex.wordchars

The string of characters that will accumulate into multi-character tokens. By default, includes all ASCII alphanumerics and underscore. In POSIX mode, the accented characters in the Latin-1 set are also included. If `punctuation_chars` is not empty, the characters `~-./*?=`, which can appear in filename specifications and command line parameters, will also be included in this attribute, and any characters which appear in `punctuation_chars` will be removed from `wordchars` if they are present there. If `whitespace_split` is set to `True`, this will have no effect.

</div>

<div class="attribute">

shlex.whitespace

Characters that will be considered whitespace and skipped. Whitespace bounds tokens. By default, includes space, tab, linefeed and carriage-return.

</div>

<div class="attribute">

shlex.escape

Characters that will be considered as escape. This will be only used in POSIX mode, and includes just `'\'` by default.

</div>

<div class="attribute">

shlex.quotes

Characters that will be considered string quotes. The token accumulates until the same quote is encountered again (thus, different quote types protect each other as in the shell.) By default, includes ASCII single and double quotes.

</div>

<div class="attribute">

shlex.escapedquotes

Characters in `quotes` that will interpret escape characters defined in `escape`. This is only used in POSIX mode, and includes just `'"'` by default.

</div>

<div class="attribute">

shlex.whitespace_split

If `True`, tokens will only be split in whitespaces. This is useful, for example, for parsing command lines with `~shlex.shlex`, getting tokens in a similar way to shell arguments. When used in combination with `punctuation_chars`, tokens will be split on whitespace in addition to those characters.

<div class="versionchanged">

3.8 The `punctuation_chars` attribute was made compatible with the `whitespace_split` attribute.

</div>

</div>

<div class="attribute">

shlex.infile

The name of the current input file, as initially set at class instantiation time or stacked by later source requests. It may be useful to examine this when constructing error messages.

</div>

<div class="attribute">

shlex.instream

The input stream from which this `~shlex.shlex` instance is reading characters.

</div>

<div class="attribute">

shlex.source

This attribute is `None` by default. If you assign a string to it, that string will be recognized as a lexical-level inclusion request similar to the `source` keyword in various shells. That is, the immediately following token will be opened as a filename and input will be taken from that stream until EOF, at which point the `~io.IOBase.close` method of that stream will be called and the input source will again become the original input stream. Source requests may be stacked any number of levels deep.

</div>

<div class="attribute">

shlex.debug

If this attribute is numeric and `1` or more, a `~shlex.shlex` instance will print verbose progress output on its behavior. If you need to use this, you can read the module source code to learn the details.

</div>

<div class="attribute">

shlex.lineno

Source line number (count of newlines seen so far plus one).

</div>

<div class="attribute">

shlex.token

The token buffer. It may be useful to examine this when catching exceptions.

</div>

<div class="attribute">

shlex.eof

Token used to determine end of file. This will be set to the empty string (`''`), in non-POSIX mode, and to `None` in POSIX mode.

</div>

<div class="attribute">

shlex.punctuation_chars

A read-only property. Characters that will be considered punctuation. Runs of punctuation characters will be returned as a single token. However, note that no semantic validity checking will be performed: for example, '\>\>\>' could be returned as a token, even though it may not be recognised as such by shells.

<div class="versionadded">

3.6

</div>

</div>

## Parsing Rules

When operating in non-POSIX mode, `~shlex.shlex` will try to obey the following rules.

- Quote characters are not recognized within words (`Do"Not"Separate` is parsed as the single word `Do"Not"Separate`);
- Escape characters are not recognized;
- Enclosing characters in quotes preserve the literal value of all characters within the quotes;
- Closing quotes separate words (`"Do"Separate` is parsed as `"Do"` and `Separate`);
- If `~shlex.whitespace_split` is `False`, any character not declared to be a word character, whitespace, or a quote will be returned as a single-character token. If it is `True`, `~shlex.shlex` will only split words in whitespaces;
- EOF is signaled with an empty string (`''`);
- It's not possible to parse empty strings, even if quoted.

When operating in POSIX mode, `~shlex.shlex` will try to obey the following parsing rules.

- Quotes are stripped out, and do not separate words (`"Do"Not"Separate"` is parsed as the single word `DoNotSeparate`);
- Non-quoted escape characters (e.g. `'\'`) preserve the literal value of the next character that follows;
- Enclosing characters in quotes which are not part of `~shlex.escapedquotes` (e.g. `"'"`) preserve the literal value of all characters within the quotes;
- Enclosing characters in quotes which are part of `~shlex.escapedquotes` (e.g. `'"'`) preserves the literal value of all characters within the quotes, with the exception of the characters mentioned in `~shlex.escape`. The escape characters retain their special meaning only when followed by the quote in use, or the escape character itself. Otherwise the escape character will be considered a normal character.
- EOF is signaled with a `None` value;
- Quoted empty strings (`''`) are allowed.

## Improved Compatibility with Shells

<div class="versionadded">

3.6

</div>

The `~shlex.shlex` class provides compatibility with the parsing performed by common Unix shells like `bash`, `dash`, and `sh`. To take advantage of this compatibility, specify the `punctuation_chars` argument in the constructor. This defaults to `False`, which preserves pre-3.6 behaviour. However, if it is set to `True`, then parsing of the characters `();<>|&` is changed: any run of these characters is returned as a single token. While this is short of a full parser for shells (which would be out of scope for the standard library, given the multiplicity of shells out there), it does allow you to perform processing of command lines more easily than you could otherwise. To illustrate, you can see the difference in the following snippet:

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> import shlex \>\>\> text = "a && b; c && d \|\| e; f \>'abc'; (def "ghi")" \>\>\> s = shlex.shlex(text, posix=True) \>\>\> s.whitespace_split = True \>\>\> list(s) \['a', '&&', 'b;', 'c', '&&', 'd', '\|\|', 'e;', 'f', '\>abc;', '(def', 'ghi)'\] \>\>\> s = shlex.shlex(text, posix=True, punctuation_chars=True) \>\>\> s.whitespace_split = True \>\>\> list(s) \['a', '&&', 'b', ';', 'c', '&&', 'd', '\|\|', 'e', ';', 'f', '\>', 'abc', ';', '(', 'def', 'ghi', ')'\]

</div>

Of course, tokens will be returned which are not valid for shells, and you'll need to implement your own error checks on the returned tokens.

Instead of passing `True` as the value for the punctuation_chars parameter, you can pass a string with specific characters, which will be used to determine which characters constitute punctuation. For example:

    >>> import shlex
    >>> s = shlex.shlex("a && b || c", punctuation_chars="|")
    >>> list(s)
    ['a', '&', '&', 'b', '||', 'c']

> [!NOTE]
> When `punctuation_chars` is specified, the `~shlex.wordchars` attribute is augmented with the characters `~-./*?=`. That is because these characters can appear in file names (including wildcards) and command-line arguments (e.g. `--color=auto`). Hence:
>
>     >>> import shlex
>     >>> s = shlex.shlex('~/a && b-c --color=auto || d *.py?',
>
> ... punctuation_chars=True) \>\>\> list(s) \['~/a', '&&', 'b-c', '--color=auto', '\|\|', 'd', '\*.py?'\]
>
> However, to match the shell as closely as possible, it is recommended to always use `posix` and `~shlex.whitespace_split` when using `~shlex.punctuation_chars`, which will negate `~shlex.wordchars` entirely.

For best effect, `punctuation_chars` should be set in conjunction with `posix=True`. (Note that `posix=False` is the default for `~shlex.shlex`.)

# `!shutil` --- High-level file operations

<div class="module" data-synopsis="High-level file operations, including copying.">

shutil

</div>

**Source code:** `Lib/shutil.py`

<div class="index">

single: file; copying single: copying files

</div>

------------------------------------------------------------------------

The `!shutil` module offers a number of high-level operations on files and collections of files. In particular, functions are provided which support file copying and removal. For operations on individual files, see also the `os` module.

> [!WARNING]
> Even the higher-level file copying functions (`shutil.copy`, `shutil.copy2`) cannot copy all file metadata.
>
> On POSIX platforms, this means that file owner and group are lost as well as ACLs. On Mac OS, the resource fork and other metadata are not used. This means that resources will be lost and file type and creator codes will not be correct. On Windows, file owners, ACLs and alternate data streams are not copied.

## Directory and files operations

<div class="function">

copyfileobj(fsrc, fdst\[, length\])

Copy the contents of the `file-like object <file object>` *fsrc* to the file-like object *fdst*. The integer *length*, if given, is the buffer size. In particular, a negative *length* value means to copy the data without looping over the source data in chunks; by default the data is read in chunks to avoid uncontrolled memory consumption. Note that if the current file position of the *fsrc* object is not 0, only the contents from the current file position to the end of the file will be copied.

`copyfileobj` will *not* guarantee that the destination stream has been flushed on completion of the copy. If you want to read from the destination at the completion of the copy operation (for example, reading the contents of a temporary file that has been copied from a HTTP stream), you must ensure that you have called `~io.IOBase.flush` or `~io.IOBase.close` on the file-like object before attempting to read the destination file.

</div>

<div class="function">

copyfile(src, dst, \*, follow_symlinks=True)

Copy the contents (no metadata) of the file named *src* to a file named *dst* and return *dst* in the most efficient way possible. *src* and *dst* are `path-like objects <path-like object>` or path names given as strings.

*dst* must be the complete target file name; look at `~shutil.copy` for a copy that accepts a target directory path. If *src* and *dst* specify the same file, `SameFileError` is raised.

The destination location must be writable; otherwise, an `OSError` exception will be raised. If *dst* already exists, it will be replaced. Special files such as character or block devices, pipes, and sockets cannot be copied with this function.

If *follow_symlinks* is false and *src* is a symbolic link, a new symbolic link will be created instead of copying the file *src* points to.

<div class="audit-event">

shutil.copyfile src,dst shutil.copyfile

</div>

<div class="versionchanged">

3.3 `IOError` used to be raised instead of `OSError`. Added *follow_symlinks* argument. Now returns *dst*.

</div>

<div class="versionchanged">

3.4 Raise `SameFileError` instead of `Error`. Since the former is a subclass of the latter, this change is backward compatible.

</div>

<div class="versionchanged">

3.8 Platform-specific fast-copy syscalls may be used internally in order to copy the file more efficiently. See `shutil-platform-dependent-efficient-copy-operations` section.

</div>

<div class="versionchanged">

3.15 `SpecialFileError` is now also raised for sockets and device files.

</div>

</div>

<div class="exception">

SpecialFileError

This exception is raised when `copyfile` or `copytree` attempt to copy a named pipe, socket, or device file.

<div class="versionadded">

2.7

</div>

</div>

<div class="exception">

SameFileError

This exception is raised if source and destination in `copyfile` are the same file.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

copymode(src, dst, \*, follow_symlinks=True)

Copy the permission bits from *src* to *dst*. The file contents, owner, and group are unaffected. *src* and *dst* are `path-like objects <path-like object>` or path names given as strings. If *follow_symlinks* is false, and both *src* and *dst* are symbolic links, `copymode` will attempt to modify the mode of *dst* itself (rather than the file it points to). This functionality is not available on every platform; please see `copystat` for more information. If `copymode` cannot modify symbolic links on the local platform, and it is asked to do so, it will do nothing and return.

<div class="audit-event">

shutil.copymode src,dst shutil.copymode

</div>

<div class="versionchanged">

3.3 Added *follow_symlinks* argument.

</div>

</div>

<div class="function">

copystat(src, dst, \*, follow_symlinks=True)

Copy the permission bits, last access time, last modification time, and flags from *src* to *dst*. On Linux, `copystat` also copies the "extended attributes" where possible. The file contents, owner, and group are unaffected. *src* and *dst* are `path-like objects <path-like object>` or path names given as strings.

If *follow_symlinks* is false, and *src* and *dst* both refer to symbolic links, `copystat` will operate on the symbolic links themselves rather than the files the symbolic links refer to—reading the information from the *src* symbolic link, and writing the information to the *dst* symbolic link.

> [!NOTE]
> Not all platforms provide the ability to examine and modify symbolic links. Python itself can tell you what functionality is locally available.
>
> - If `os.chmod in os.supports_follow_symlinks` is `True`, `copystat` can modify the permission bits of a symbolic link.
> - If `os.utime in os.supports_follow_symlinks` is `True`, `copystat` can modify the last access and modification times of a symbolic link.
> - If `os.chflags in os.supports_follow_symlinks` is `True`, `copystat` can modify the flags of a symbolic link. (`os.chflags` is not available on all platforms.)
>
> On platforms where some or all of this functionality is unavailable, when asked to modify a symbolic link, `copystat` will copy everything it can. `copystat` never returns failure.
>
> Please see `os.supports_follow_symlinks` for more information.

<div class="audit-event">

shutil.copystat src,dst shutil.copystat

</div>

<div class="versionchanged">

3.3 Added *follow_symlinks* argument and support for Linux extended attributes.

</div>

</div>

<div class="function">

copy(src, dst, \*, follow_symlinks=True)

Copies the file *src* to the file or directory *dst*. *src* and *dst* should be `path-like objects <path-like object>` or strings. If *dst* specifies a directory, the file will be copied into *dst* using the base filename from *src*. If *dst* specifies a file that already exists, it will be replaced. Returns the path to the newly created file.

If *follow_symlinks* is false, and *src* is a symbolic link, *dst* will be created as a symbolic link. If *follow_symlinks* is true and *src* is a symbolic link, *dst* will be a copy of the file *src* refers to.

`~shutil.copy` copies the file data and the file's permission mode (see `os.chmod`). Other metadata, like the file's creation and modification times, is not preserved. To preserve all file metadata from the original, use `~shutil.copy2` instead.

<div class="audit-event">

shutil.copyfile src,dst shutil.copy

</div>

<div class="audit-event">

shutil.copymode src,dst shutil.copy

</div>

<div class="versionchanged">

3.3 Added *follow_symlinks* argument. Now returns path to the newly created file.

</div>

<div class="versionchanged">

3.8 Platform-specific fast-copy syscalls may be used internally in order to copy the file more efficiently. See `shutil-platform-dependent-efficient-copy-operations` section.

</div>

</div>

<div class="function">

copy2(src, dst, \*, follow_symlinks=True)

Identical to `~shutil.copy` except that `copy2` also attempts to preserve file metadata.

When *follow_symlinks* is false, and *src* is a symbolic link, `copy2` attempts to copy all metadata from the *src* symbolic link to the newly created *dst* symbolic link. However, this functionality is not available on all platforms. On platforms where some or all of this functionality is unavailable, `copy2` will preserve all the metadata it can; `copy2` never raises an exception because it cannot preserve file metadata.

`copy2` uses `copystat` to copy the file metadata. Please see `copystat` for more information about platform support for modifying symbolic link metadata.

<div class="audit-event">

shutil.copyfile src,dst shutil.copy2

</div>

<div class="audit-event">

shutil.copystat src,dst shutil.copy2

</div>

<div class="versionchanged">

3.3 Added *follow_symlinks* argument, try to copy extended file system attributes too (currently Linux only). Now returns path to the newly created file.

</div>

<div class="versionchanged">

3.8 Platform-specific fast-copy syscalls may be used internally in order to copy the file more efficiently. See `shutil-platform-dependent-efficient-copy-operations` section.

</div>

</div>

<div class="function">

ignore_patterns(\*patterns)

This factory function creates a function that can be used as a callable for `copytree`'s *ignore* argument, ignoring files and directories that match one of the glob-style *patterns* provided. See the example below.

</div>

<div class="function">

copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2, ignore_dangling_symlinks=False, dirs_exist_ok=False, \*, copy_dir_stat=True)

Recursively copy an entire directory tree rooted at *src* to a directory named *dst* and return the destination directory. All intermediate directories needed to contain *dst* will also be created by default.

Permissions and times of directories are copied with `copystat`, individual files are copied using `~shutil.copy2`.

If *copy_dir_stat* is false, directory metadata is not copied with `copystat`, e.g. to copy a read-only tree into a writable one. It defaults to true. Note that file metadata is always handled by *copy_function*: use e.g. `copy_function=shutil.copyfile` to skip that as well.

If *symlinks* is true, symbolic links in the source tree are represented as symbolic links in the new tree and the metadata of the original links will be copied as far as the platform allows; if false or omitted, the contents and metadata of the linked files are copied to the new tree.

When *symlinks* is false, if the file pointed to by the symlink doesn't exist, an exception will be added in the list of errors raised in an `Error` exception at the end of the copy process. You can set the optional *ignore_dangling_symlinks* flag to true if you want to silence this exception. Notice that this option has no effect on platforms that don't support `os.symlink`.

If *ignore* is given, it must be a callable that will receive as its arguments the directory being visited by `copytree`, and a list of its contents, as returned by `os.listdir`. Since `copytree` is called recursively, the *ignore* callable will be called once for each directory that is copied. The callable must return a sequence of directory and file names relative to the current directory (i.e. a subset of the items in its second argument); these names will then be ignored in the copy process. `ignore_patterns` can be used to create such a callable that ignores names based on glob-style patterns.

If exception(s) occur, an `Error` is raised with a list of reasons.

If *copy_function* is given, it must be a callable that will be used to copy each file. It will be called with the source path and the destination path as arguments. By default, `~shutil.copy2` is used, but any function that supports the same signature (like `~shutil.copy`) can be used.

If *dirs_exist_ok* is false (the default) and *dst* already exists, a `FileExistsError` is raised. If *dirs_exist_ok* is true, the copying operation will continue if it encounters existing directories, and files within the *dst* tree will be overwritten by corresponding files from the *src* tree.

<div class="audit-event">

shutil.copytree src,dst shutil.copytree

</div>

<div class="versionchanged">

3.2 Added the *copy_function* argument to be able to provide a custom copy function. Added the *ignore_dangling_symlinks* argument to silence dangling symlinks errors when *symlinks* is false.

</div>

<div class="versionchanged">

3.3 Copy metadata when *symlinks* is false. Now returns *dst*.

</div>

<div class="versionchanged">

3.8 Platform-specific fast-copy syscalls may be used internally in order to copy the file more efficiently. See `shutil-platform-dependent-efficient-copy-operations` section.

</div>

<div class="versionchanged">

3.8 Added the *dirs_exist_ok* parameter.

</div>

<div class="versionchanged">

3.16 Added the *copy_dir_stat* parameter.

</div>

</div>

<div class="function">

rmtree(path, ignore_errors=False, onerror=None, \*, onexc=None, dir_fd=None)

<div class="index">

single: directory; deleting

</div>

Delete an entire directory tree; *path* must point to a directory (but not a symbolic link to a directory). If *ignore_errors* is true, errors resulting from failed removals will be ignored; if false or omitted, such errors are handled by calling a handler specified by *onexc* or *onerror* or, if both are omitted, exceptions are propagated to the caller.

If *path* is a symbolic link to a directory followed by a trailing path separator, such as `'link/'`, the operating system resolves the link, so `!rmtree` deletes the contents of the directory it points to and then fails to remove *path* itself. `pathlib.Path` strips trailing separators, so `rmtree(Path('link/'))` refuses the link as usual.

This function can support `paths relative to directory descriptors
<dir_fd>`.

> [!NOTE]
> On platforms that support the necessary fd-based functions a symlink attack resistant version of `rmtree` is used by default. On other platforms, the `rmtree` implementation is susceptible to a symlink attack: given proper timing and circumstances, attackers can manipulate symlinks on the filesystem to delete files they wouldn't be able to access otherwise. Applications can use the `rmtree.avoids_symlink_attacks` function attribute to determine which case applies.

If *onexc* is provided, it must be a callable that accepts three parameters: *function*, *path*, and *excinfo*.

The first parameter, *function*, is the function which raised the exception; it depends on the platform and implementation. The second parameter, *path*, will be the path name passed to *function*. The third parameter, *excinfo*, is the exception that was raised. Exceptions raised by *onexc* will not be caught.

The deprecated *onerror* is similar to *onexc*, except that the third parameter it receives is the tuple returned from `sys.exc_info`.

<div class="seealso">

`shutil-rmtree-example` for an example of handling the removal of a directory tree that contains read-only files.

</div>

<div class="audit-event">

shutil.rmtree path,dir_fd shutil.rmtree

</div>

<div class="versionchanged">

3.3 Added a symlink attack resistant version that is used automatically if platform supports fd-based functions.

</div>

<div class="versionchanged">

3.8 On Windows, will no longer delete the contents of a directory junction before removing the junction.

</div>

<div class="versionchanged">

3.11 Added the *dir_fd* parameter.

</div>

<div class="versionchanged">

3.12 Added the *onexc* parameter, deprecated *onerror*.

</div>

<div class="versionchanged">

3.13 `!rmtree` now ignores `FileNotFoundError` exceptions for all but the top-level path. Exceptions other than `OSError` and subclasses of `!OSError` are now always propagated to the caller.

</div>

<div class="attribute">

rmtree.avoids_symlink_attacks

Indicates whether the current platform and implementation provides a symlink attack resistant version of `rmtree`. Currently this is only true for platforms supporting fd-based directory access functions.

<div class="versionadded">

3.3

</div>

</div>

</div>

<div class="function">

move(src, dst, copy_function=copy2)

Recursively move a file or directory (*src*) to another location and return the destination.

If *dst* is an existing directory or a symlink to a directory, then *src* is moved inside that directory. The destination path in that directory must not already exist.

If *dst* already exists but is not a directory, it may be overwritten depending on `os.rename` semantics.

`os.rename` is preferably used internally when *src* and the destination are on the same filesystem. In case `os.rename` fails due to `OSError` (e.g. the user has write permission to the destination file but not to its parent directory), this method falls back to using *copy_function*, in which case *src* is copied to the destination using *copy_function* and then removed.

In case of symlinks, a new symlink pointing to the target of *src* will be created in or as the destination, and *src* will be removed.

If *copy_function* is given, it must be a callable that takes two arguments, *src* and the destination, and will be used to copy *src* to the destination if `os.rename` cannot be used. If the source is a directory, `copytree` is called, passing it the *copy_function*. The default *copy_function* is `copy2`. Using `~shutil.copy` as the *copy_function* allows the move to succeed when it is not possible to also copy the metadata, at the expense of not copying any of the metadata.

<div class="audit-event">

shutil.move src,dst shutil.move

</div>

<div class="versionchanged">

3.3 Added explicit symlink handling for foreign filesystems, thus adapting it to the behavior of GNU's `mv`. Now returns *dst*.

</div>

<div class="versionchanged">

3.5 Added the *copy_function* keyword argument.

</div>

<div class="versionchanged">

3.8 Platform-specific fast-copy syscalls may be used internally in order to copy the file more efficiently. See `shutil-platform-dependent-efficient-copy-operations` section.

</div>

<div class="versionchanged">

3.9 Accepts a `path-like object` for both *src* and *dst*.

</div>

</div>

<div class="function">

disk_usage(path)

Return disk usage statistics about the given path as a `named tuple` with the attributes *total*, *used* and *free*, which are the amount of total, used and free space, in bytes. *path* may be a file or a directory.

> [!NOTE]
> On Unix filesystems, *path* must point to a path within a **mounted** filesystem partition. On those platforms, CPython doesn't attempt to retrieve disk usage information from non-mounted filesystems.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.8 On Windows, *path* can now be a file or directory.

</div>

<div class="availability">

Unix, Windows.

</div>

</div>

<div class="function">

chown(path, user=None, group=None, \*, dir_fd=None, follow_symlinks=True)

Change owner *user* and/or *group* of the given *path*.

*user* can be a system user name or a uid; the same applies to *group*. At least one argument is required.

See also `os.chown`, the underlying function.

<div class="audit-event">

shutil.chown path,user,group shutil.chown

</div>

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.13 Added *dir_fd* and *follow_symlinks* parameters.

</div>

</div>

<div class="function">

which(cmd, mode=os.F_OK \| os.X_OK, path=None)

Return the path to an executable which would be run if the given *cmd* was called. If no *cmd* would be called, return `None`.

*mode* is a permission mask passed to `os.access`, by default determining if the file exists and is executable.

*path* is a "`PATH` string" specifying the directories to look in, delimited by `os.pathsep`. When no *path* is specified, the `PATH` environment variable is read from `os.environ`, falling back to `os.defpath` if it is not set.

If *cmd* contains a directory component, `!which` only checks the specified path directly and does not search the directories listed in *path* or in the system's `PATH` environment variable.

On Windows, the current directory is prepended to the *path* if *mode* does not include `os.X_OK`. When the *mode* does include `os.X_OK`, the Windows API `NeedCurrentDirectoryForExePathW` will be consulted to determine if the current directory should be prepended to *path*. To avoid consulting the current working directory for executables: set the environment variable `NoDefaultCurrentDirectoryInExePath`.

Also on Windows, the `PATHEXT` environment variable is used to resolve commands that may not already include an extension. For example, if you call `shutil.which("python")`, `which` will search `PATHEXT` to know that it should look for `python.exe` within the *path* directories. For example, on Windows:

    >>> shutil.which("python")
    'C:\\Python33\\python.EXE'

This is also applied when *cmd* is a path that contains a directory component:

    >>> shutil.which("C:\\Python33\\python")
    'C:\\Python33\\python.EXE'

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.8 The `bytes` type is now accepted. If *cmd* type is `bytes`, the result type is also `bytes`.

</div>

<div class="versionchanged">

3.12 On Windows, the current directory is no longer prepended to the search path if *mode* includes `os.X_OK` and WinAPI `NeedCurrentDirectoryForExePathW(cmd)` is false, else the current directory is prepended even if it is already in the search path; `PATHEXT` is used now even when *cmd* includes a directory component or ends with an extension that is in `PATHEXT`; and filenames that have no extension can now be found.

</div>

</div>

<div class="exception">

Error

Subclass of `OSError` collecting exceptions raised during a multi-file operation. For `copytree`, the exception argument is a list of 3-tuples (*srcname*, *dstname*, *exception*).

</div>

### Platform-dependent efficient copy operations

Starting from Python 3.8, all functions involving a file copy (`copyfile`, `~shutil.copy`, `copy2`, `copytree`, and `move`) may use platform-specific "fast-copy" syscalls in order to copy the file more efficiently (see `33671`). "fast-copy" means that the copying operation occurs within the kernel, avoiding the use of userspace buffers in Python as in "`outfd.write(infd.read())`".

On macOS [fcopyfile](http://www.manpagez.com/man/3/copyfile/) is used to copy the file content (not metadata).

On Linux `os.copy_file_range` or `os.sendfile` is used.

On Solaris `os.sendfile` is used.

On Windows `shutil.copyfile` uses a bigger default buffer size (1 MiB instead of 64 KiB) and a `memoryview`-based variant of `shutil.copyfileobj` is used, which still reads and writes in a loop. `shutil.copy2` uses the native `CopyFile2` call on Windows, which is the most efficient method, supports copy-on-write, and preserves metadata.

If the fast-copy operation fails and no data was written in the destination file then shutil will silently fall back to less efficient `copyfileobj` function internally.

<div class="versionchanged">

3.8

</div>

<div class="versionchanged">

3.14 Solaris now uses `os.sendfile`.

</div>

<div class="versionchanged">

3.14 Copy-on-write or server-side copy may be used internally via `os.copy_file_range` on supported Linux filesystems.

</div>

### copytree example

An example that uses the `ignore_patterns` helper:

    from shutil import copytree, ignore_patterns

    copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*'))

This will copy everything except `.pyc` files and files or directories whose name starts with `tmp`.

Another example that uses the *ignore* argument to add a logging call:

    from shutil import copytree
    import logging

    def _logpath(path, names):
        logging.info('Working in %s', path)
        return []   # nothing will be ignored

    copytree(source, destination, ignore=_logpath)

### rmtree example

This example shows how to remove a directory tree on Windows where some of the files have their read-only bit set. It uses the onexc callback to clear the readonly bit and reattempt the remove. Any subsequent failure will propagate. :

    import os, stat
    import shutil

    def remove_readonly(func, path, _):
        "Clear the readonly bit and reattempt the removal"
        os.chmod(path, stat.S_IWRITE)
        func(path)

    shutil.rmtree(directory, onexc=remove_readonly)

## Archiving operations

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.5 Added support for the *xztar* format.

</div>

High-level utilities to create and read compressed and archived files are also provided. They rely on the `zipfile` and `tarfile` modules.

<div class="function">

make_archive(base_name, format, \[root_dir, \[base_dir, \[verbose, \[dry_run, \[owner, \[group, \[logger\]\]\]\]\]\]\])

Create an archive file (such as zip or tar) and return its name.

*base_name* is a string or `path-like object` specifying the name of the file to create, including the path, minus any format-specific extension.

*format* is the archive format: one of "zip" (if the `zlib` module is available), "tar", "gztar" (if the `zlib` module is available), "bztar" (if the `bz2` module is available), "xztar" (if the `lzma` module is available), or "zstdtar" (if the `compression.zstd` module is available).

*root_dir* is a string or `path-like object` specifying a directory that will be the root directory of the archive, all paths in the archive will be relative to it; for example, we typically chdir into *root_dir* before creating the archive.

*base_dir* is a string or `path-like object` specifying a directory where we start archiving from; i.e. *base_dir* will be the common prefix of all files and directories in the archive. *base_dir* must be given relative to *root_dir*. See `shutil-archiving-example-with-basedir` for how to use *base_dir* and *root_dir* together.

*root_dir* and *base_dir* both default to the current directory.

If *dry_run* is true, no archive is created, but the operations that would be executed are logged to *logger*.

*owner* and *group* are used when creating a tar archive. By default, uses the current owner and group.

*logger* must be an object compatible with `282`, usually an instance of `logging.Logger`.

The *verbose* argument is unused and deprecated.

<div class="audit-event">

shutil.make_archive base_name,format,root_dir,base_dir shutil.make_archive

</div>

> [!NOTE]
> This function is not thread-safe when custom archivers registered with `register_archive_format` do not support the *root_dir* argument. In this case it temporarily changes the current working directory of the process to *root_dir* to perform archiving.

<div class="versionchanged">

3.8 The modern pax (POSIX.1-2001) format is now used instead of the legacy GNU format for archives created with `format="tar"`.

</div>

<div class="versionchanged">

3.10.6 This function is now made thread-safe during creation of standard `.zip` and tar archives.

</div>

<div class="versionchanged">

3.15 Accepts a `path-like object` for *base_name*, *root_dir* and *base_dir*.

</div>

</div>

<div class="function">

get_archive_formats()

Return a list of supported formats for archiving. Each element of the returned sequence is a tuple `(name, description)`.

By default `!shutil` provides these formats:

- *zip*: ZIP file (if the `zlib` module is available).
- *tar*: Uncompressed tar file. Uses POSIX.1-2001 pax format for new archives.
- *gztar*: gzip'ed tar-file (if the `zlib` module is available).
- *bztar*: bzip2'ed tar-file (if the `bz2` module is available).
- *xztar*: xz'ed tar-file (if the `lzma` module is available).
- *zstdtar*: Zstandard compressed tar-file (if the `compression.zstd` module is available).

You can register new formats or provide your own archiver for any existing formats, by using `register_archive_format`.

</div>

<div class="function">

register_archive_format(name, function, \[extra_args, \[description\]\])

Register an archiver for the format *name*.

*function* is the callable that will be used to create archives. The callable will receive the *base_name* of the file to create, followed by the *base_dir* (which defaults to `os.curdir`) to start archiving from. Further arguments are passed as keyword arguments: *owner*, *group*, *dry_run* and *logger* (as passed in `make_archive`).

If *function* has the custom attribute `function.supports_root_dir` set to `True`, the *root_dir* argument is passed as a keyword argument. Otherwise the current working directory of the process is temporarily changed to *root_dir* before calling *function*. In this case `make_archive` is not thread-safe.

If given, *extra_args* is a sequence of `(name, value)` pairs that will be used as extra keywords arguments when the archiver callable is used.

*description* is used by `get_archive_formats` which returns the list of archivers. Defaults to an empty string.

<div class="versionchanged">

3.12 Added support for functions supporting the *root_dir* argument.

</div>

</div>

<div class="function">

unregister_archive_format(name)

Remove the archive format *name* from the list of supported formats.

</div>

<div class="function">

unpack_archive(filename\[, extract_dir\[, format\[, filter\]\]\])

Unpack an archive. *filename* is the full path of the archive.

*extract_dir* is the name of the target directory where the archive is unpacked. If not provided, the current working directory is used.

*format* is the archive format: one of "zip", "tar", "gztar", "bztar", "xztar", "zstdtar", or "whl". Or any other format registered with `register_unpack_format`. If not provided, `unpack_archive` will use the archive file name extension and see if an unpacker was registered for that extension. In case none is found, a `ValueError` is raised.

The keyword-only *filter* argument is passed to the underlying unpacking function. For zip files, *filter* is not accepted. For tar files, it is recommended to use `'data'` (default since Python 3.14), unless using features specific to tar and UNIX-like filesystems. (See `tarfile-extraction-filter` for details.)

<div class="audit-event">

shutil.unpack_archive filename,extract_dir,format shutil.unpack_archive

</div>

> [!WARNING]
> Never extract archives from untrusted sources without prior inspection. It is possible that files are created outside of the path specified in the *extract_dir* argument, for example, members that have absolute filenames or filenames with ".." components.
>
> Since Python 3.14, the defaults for both built-in formats (zip and tar files) will prevent the most dangerous of such security issues, but will not prevent *all* unintended behavior. Read the `tarfile-further-verification` section for tar-specific details.

<div class="versionchanged">

3.7 Accepts a `path-like object` for *filename* and *extract_dir*.

</div>

<div class="versionchanged">

3.12 Added the *filter* argument.

</div>

<div class="versionchanged">

next Added the "whl" format.

</div>

</div>

<div class="function">

register_unpack_format(name, extensions, function\[, extra_args\[, description\]\])

Registers an unpack format. *name* is the name of the format and *extensions* is a list of extensions corresponding to the format, like `.zip` for Zip files.

*function* is the callable that will be used to unpack archives. The callable will receive:

- the path of the archive, as a positional argument;
- the directory the archive must be extracted to, as a positional argument;
- possibly a *filter* keyword argument, if it was given to `unpack_archive`;
- additional keyword arguments, specified by *extra_args* as a sequence of `(name, value)` tuples.

*description* can be provided to describe the format, and will be returned by the `get_unpack_formats` function.

</div>

<div class="function">

unregister_unpack_format(name)

Unregister an unpack format. *name* is the name of the format.

</div>

<div class="function">

get_unpack_formats()

Return a list of all registered formats for unpacking. Each element of the returned sequence is a tuple `(name, extensions, description)`.

By default `!shutil` provides these formats:

- *zip*: ZIP file (unpacking compressed files works only if the corresponding module is available).
- *tar*: uncompressed tar file.
- *gztar*: gzip'ed tar-file (if the `zlib` module is available).
- *bztar*: bzip2'ed tar-file (if the `bz2` module is available).
- *xztar*: xz'ed tar-file (if the `lzma` module is available).
- *zstdtar*: Zstandard compressed tar-file (if the `compression.zstd` module is available).
- *whl*: [wheel file](https://packaging.python.org/en/latest/discussions/package-formats/#what-is-a-wheel) (unpacking compressed files works only if the corresponding module is available).

You can register new formats or provide your own unpacker for any existing formats, by using `register_unpack_format`.

<div class="versionchanged">

next Added the *whl* format.

</div>

</div>

### Archiving example

In this example, we create a gzip'ed tar-file archive containing all files found in the `.ssh` directory of the user:

    >>> from shutil import make_archive
    >>> import os
    >>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive'))
    >>> root_dir = os.path.expanduser(os.path.join('~', '.ssh'))
    >>> make_archive(archive_name, 'gztar', root_dir)
    '/Users/tarek/myarchive.tar.gz'

The resulting archive contains:

```
$ tar -tzvf /Users/tarek/myarchive.tar.gz
drwx------ tarek/staff       0 2010-02-01 16:23:40 ./
-rw-r--r-- tarek/staff     609 2008-06-09 13:26:54 ./authorized_keys
-rwxr-xr-x tarek/staff      65 2008-06-09 13:26:54 ./config
-rwx------ tarek/staff     668 2008-06-09 13:26:54 ./id_dsa
-rwxr-xr-x tarek/staff     609 2008-06-09 13:26:54 ./id_dsa.pub
-rw------- tarek/staff    1675 2008-06-09 13:26:54 ./id_rsa
-rw-r--r-- tarek/staff     397 2008-06-09 13:26:54 ./id_rsa.pub
-rw-r--r-- tarek/staff   37192 2010-02-06 18:23:10 ./known_hosts
```

### Archiving example with *base_dir*

In this example, similar to the [one above](#shutil-archiving-example), we show how to use `make_archive`, but this time with the usage of *base_dir*. We now have the following directory structure:

```
$ tree tmp
tmp
└── root
    └── structure
        ├── content
            └── please_add.txt
        └── do_not_add.txt
```

In the final archive, `please_add.txt` should be included, but `do_not_add.txt` should not. Therefore we use the following:

    >>> from shutil import make_archive
    >>> import os
    >>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive'))
    >>> make_archive(
    ...     archive_name,
    ...     'tar',
    ...     root_dir='tmp/root',
    ...     base_dir='structure/content',
    ... )
    '/Users/tarek/myarchive.tar'

Listing the files in the resulting archive gives us:

```
$ python -m tarfile -l /Users/tarek/myarchive.tar
structure/content/
structure/content/please_add.txt
```

## Querying the size of the output terminal

<div class="function">

get_terminal_size(fallback=(columns, lines))

Get the size of the terminal window.

For each of the two dimensions, the environment variable, `COLUMNS` and `LINES` respectively, is checked. If the variable is defined and the value is a positive integer, it is used.

When `COLUMNS` or `LINES` is not defined, which is the common case, the terminal connected to `sys.__stdout__` is queried by invoking `os.get_terminal_size`.

If the terminal size cannot be successfully queried, either because the system doesn't support querying, or because we are not connected to a terminal, the value given in `fallback` parameter is used. `fallback` defaults to `(80, 24)` which is the default size used by many terminal emulators.

The value returned is a named tuple of type `os.terminal_size`.

See also: The Single UNIX Specification, Version 2, [Other Environment Variables](https://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html#tag_002_003).

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.11 The `fallback` values are also used if `os.get_terminal_size` returns zeroes.

</div>

</div>

# `!signal` --- Set handlers for asynchronous events

<div class="module" data-synopsis="Set handlers for asynchronous events.">

signal

</div>

**Source code:** `Lib/signal.py`

------------------------------------------------------------------------

This module provides mechanisms to use signal handlers in Python.

## General rules

The `signal.signal` function allows defining custom handlers to be executed when a signal is received. A small number of default handlers are installed: `SIGPIPE` is ignored (so write errors on pipes and sockets can be reported as ordinary Python exceptions) and `SIGINT` is translated into a `KeyboardInterrupt` exception if the parent process has not changed it.

A handler for a particular signal, once set, remains installed until it is explicitly reset (Python emulates the BSD style interface regardless of the underlying implementation), with the exception of the handler for `SIGCHLD`, which follows the underlying implementation.

On WebAssembly platforms, signals are emulated and therefore behave differently. Several functions and signals are not available on these platforms.

### Execution of Python signal handlers

A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the `virtual machine` to execute the corresponding Python signal handler at a later point (for example, at the next `bytecode` instruction). This has consequences:

- It makes little sense to catch synchronous errors like `SIGFPE` or `SIGSEGV` that are caused by an invalid operation in C code. Python will return from the signal handler to the C code, which is likely to raise the same signal again, causing Python to apparently hang. From Python 3.3 onwards, you can use the `faulthandler` module to report on synchronous errors.
- A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.
- If the handler raises an exception, it will be raised "out of thin air" in the main thread. See the `note below <handlers-and-exceptions>` for a discussion.

### Signals and threads

Python signal handlers are always executed in the main Python thread of the main interpreter, even if the signal was received in another thread. This means that signals can't be used as a means of inter-thread communication. You can use the synchronization primitives from the `threading` module instead.

Besides, only the main thread of the main interpreter is allowed to set a new signal handler.

> [!WARNING]
> Synchronization primitives such as `threading.Lock` should not be used within signal handlers. Doing so can lead to unexpected deadlocks.

## Module contents

<div class="versionchanged">

3.5 signal (SIG\*), handler (`SIG_DFL`, `SIG_IGN`) and sigmask (`SIG_BLOCK`, `SIG_UNBLOCK`, `SIG_SETMASK`) related constants listed below were turned into `enums <enum.IntEnum>` (`Signals`, `Handlers` and `Sigmasks` respectively). `getsignal`, `pthread_sigmask`, `sigpending` and `sigwait` functions return human-readable `enums <enum.IntEnum>` as `Signals` objects.

</div>

The signal module defines three enums:

<div class="Signals">

`enum.IntEnum` collection of SIG\* constants and the [CTRL]()\* constants.

<div class="versionadded">

3.5

</div>

</div>

<div class="Handlers">

`enum.IntEnum` collection of the constants `SIG_DFL` and `SIG_IGN`.

<div class="versionadded">

3.5

</div>

</div>

<div class="Sigmasks">

`enum.IntEnum` collection of the constants `SIG_BLOCK`, `SIG_UNBLOCK` and `SIG_SETMASK`.

<div class="availability">

Unix.

See the man page `sigprocmask(2)` and `pthread_sigmask(3)` for further information.

</div>

<div class="versionadded">

3.5

</div>

</div>

The variables defined in the `!signal` module are:

<div class="data">

SIG_DFL

This is one of two standard signal handling options; it will simply perform the default function for the signal. For example, on most systems the default action for `SIGQUIT` is to dump core and exit, while the default action for `SIGCHLD` is to simply ignore it.

</div>

<div class="data">

SIG_IGN

This is another standard signal handler, which will simply ignore the given signal.

</div>

<div class="data">

SIGABRT

Abort signal from `abort(3)`.

</div>

<div class="data">

SIGALRM

Timer signal from `alarm(2)`.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGBREAK

Interrupt from keyboard (CTRL + BREAK).

<div class="availability">

Windows.

</div>

</div>

<div class="data">

SIGBUS

Bus error (bad memory access).

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGCHLD

Child process stopped or terminated.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGCLD

Alias to `SIGCHLD`.

<div class="availability">

not macOS.

</div>

</div>

<div class="data">

SIGCONT

Continue the process if it is currently stopped

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGFPE

Floating-point exception. For example, division by zero.

<div class="seealso">

`ZeroDivisionError` is raised when the second argument of a division or modulo operation is zero.

</div>

</div>

<div class="data">

SIGHUP

Hangup detected on controlling terminal or death of controlling process.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGILL

Illegal instruction.

</div>

<div class="data">

SIGINT

Interrupt from keyboard (CTRL + C).

Default action is to raise `KeyboardInterrupt`.

</div>

<div class="data">

SIGKILL

Kill signal.

It cannot be caught, blocked, or ignored.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGPIPE

Broken pipe: write to pipe with no readers.

Default action is to ignore the signal.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGPROF

Profiling timer expired.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGQUIT

Terminal quit signal.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGSEGV

Segmentation fault: invalid memory reference.

</div>

<div class="data">

SIGSTOP

Stop executing (cannot be caught or ignored).

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGSTKFLT

Stack fault on coprocessor. The Linux kernel does not raise this signal: it can only be raised in user space.

<div class="availability">

Linux.

On architectures where the signal is available. See the man page `signal(7)` for further information.

</div>

<div class="versionadded">

3.11

</div>

</div>

<div class="data">

SIGTERM

Termination signal.

</div>

<div class="data">

SIGUSR1

User-defined signal 1.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGUSR2

User-defined signal 2.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGVTALRM

Virtual timer expired.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGWINCH

Window resize signal.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIGXCPU

CPU time limit exceeded.

<div class="availability">

Unix.

</div>

</div>

<div class="data">

SIG\*

All the signal numbers are defined symbolically. For example, the hangup signal is defined as `signal.SIGHUP`; the variable names are identical to the names used in C programs, as found in `<signal.h>`. The Unix man page for '`signal`' lists the existing signals (on some systems this is `signal(2)`, on others the list is in `signal(7)`). Note that not all systems define the same set of signal names; only those names defined by the system are defined by this module.

</div>

<div class="data">

CTRL_C_EVENT

The signal corresponding to the `Ctrl+C` keystroke event. This signal can only be used with `os.kill`.

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

CTRL_BREAK_EVENT

The signal corresponding to the `Ctrl+Break` keystroke event. This signal can only be used with `os.kill`.

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

NSIG

One more than the number of the highest signal number. Use `valid_signals` to get valid signal numbers.

</div>

<div class="data">

ITIMER_REAL

Decrements interval timer in real time, and delivers `SIGALRM` upon expiration.

</div>

<div class="data">

ITIMER_VIRTUAL

Decrements interval timer only when the process is executing, and delivers SIGVTALRM upon expiration.

</div>

<div class="data">

ITIMER_PROF

Decrements interval timer both when the process executes and when the system is executing on behalf of the process. Coupled with ITIMER_VIRTUAL, this timer is usually used to profile the time spent by the application in user and kernel space. SIGPROF is delivered upon expiration.

</div>

<div class="data">

SIG_BLOCK

A possible value for the *how* parameter to `pthread_sigmask` indicating that signals are to be blocked.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

SIG_UNBLOCK

A possible value for the *how* parameter to `pthread_sigmask` indicating that signals are to be unblocked.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

SIG_SETMASK

A possible value for the *how* parameter to `pthread_sigmask` indicating that the signal mask is to be replaced.

<div class="versionadded">

3.3

</div>

</div>

The `!signal` module defines one exception:

<div class="exception">

ItimerError

Raised to signal an error from the underlying `setitimer` or `getitimer` implementation. Expect this error if an invalid interval timer or a negative time is passed to `setitimer`. This error is a subtype of `OSError`.

<div class="versionadded">

3.3 This error used to be a subtype of `IOError`, which is now an alias of `OSError`.

</div>

</div>

The `!signal` module defines the following functions:

<div class="function">

alarm(time)

If *time* is non-zero, this function requests that a `SIGALRM` signal be sent to the process in *time* seconds. Any previously scheduled alarm is canceled (only one alarm can be scheduled at any time). The returned value is then the number of seconds before any previously set alarm was to have been delivered. If *time* is zero, no alarm is scheduled, and any scheduled alarm is canceled. If the return value is zero, no alarm is currently scheduled.

<div class="availability">

Unix.

See the man page `alarm(2)` for further information.

</div>

</div>

<div class="function">

getsignal(signalnum)

Return the current signal handler for the signal *signalnum*. The returned value may be a callable Python object, or one of the special values `signal.SIG_IGN`, `signal.SIG_DFL` or `None`. Here, `signal.SIG_IGN` means that the signal was previously ignored, `signal.SIG_DFL` means that the default way of handling the signal was previously in use, and `None` means that the previous signal handler was not installed from Python.

</div>

<div class="function">

strsignal(signalnum)

Returns the description of signal *signalnum*, such as "Interrupt" for `SIGINT`. Returns `None` if *signalnum* has no description. Raises `ValueError` if *signalnum* is invalid.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

valid_signals()

Return the set of valid signal numbers on this platform. This can be less than `range(1, NSIG)` if some signals are reserved by the system for internal use.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

pause()

Cause the process to sleep until a signal is received; the appropriate handler will then be called. Returns nothing.

<div class="availability">

Unix.

See the man page `signal(2)` for further information.

</div>

See also `sigwait`, `sigwaitinfo`, `sigtimedwait` and `sigpending`.

</div>

<div class="function">

raise_signal(signum)

Sends a signal to the calling process. Returns nothing.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

pidfd_send_signal(pidfd, sig, siginfo=None, flags=0)

Send signal *sig* to the process referred to by file descriptor *pidfd*. Python does not currently support the *siginfo* parameter; it must be `None`. The *flags* argument is provided for future extensions; no flag values are currently defined.

See the `pidfd_send_signal(2)` man page for more information.

<div class="availability">

Linux \>= 5.1, Android \>= `build-time <sys.getandroidapilevel>` API level 31

</div>

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

pthread_kill(thread_id, signalnum)

Send the signal *signalnum* to the thread *thread_id*, another thread in the same process as the caller. The target thread can be executing any code (Python or not). However, if the target thread is executing the Python interpreter, the Python signal handlers will be `executed by the main
thread of the main interpreter <signals-and-threads>`. Therefore, the only point of sending a signal to a particular Python thread would be to force a running system call to fail with `InterruptedError`.

Use `threading.get_ident` or the `~threading.Thread.ident` attribute of `threading.Thread` objects to get a suitable value for *thread_id*.

If *signalnum* is 0, then no signal is sent, but error checking is still performed; this can be used to check if the target thread is still running.

<div class="audit-event">

signal.pthread_kill thread_id,signalnum signal.pthread_kill

</div>

<div class="availability">

Unix.

See the man page `pthread_kill(3)` for further information.

</div>

See also `os.kill`.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

pthread_sigmask(how, mask)

Fetch and/or change the signal mask of the calling thread. The signal mask is the set of signals whose delivery is currently blocked for the caller. Return the old signal mask as a set of signals.

The behavior of the call is dependent on the value of *how*, as follows.

- `SIG_BLOCK`: The set of blocked signals is the union of the current set and the *mask* argument.
- `SIG_UNBLOCK`: The signals in *mask* are removed from the current set of blocked signals. It is permissible to attempt to unblock a signal which is not blocked.
- `SIG_SETMASK`: The set of blocked signals is set to the *mask* argument.

*mask* is a set of signal numbers (e.g. {`signal.SIGINT`, `signal.SIGTERM`}). Use `~signal.valid_signals` for a full mask including all signals.

For example, `signal.pthread_sigmask(signal.SIG_BLOCK, [])` reads the signal mask of the calling thread.

`SIGKILL` and `SIGSTOP` cannot be blocked.

<div class="availability">

Unix.

See the man page `sigprocmask(2)` and `pthread_sigmask(3)` for further information.

</div>

See also `pause`, `sigpending` and `sigwait`.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

setitimer(which, seconds, interval=0)

Sets given interval timer (one of `signal.ITIMER_REAL`, `signal.ITIMER_VIRTUAL` or `signal.ITIMER_PROF`) specified by *which* to fire after *seconds* (rounded up to microseconds, different from `alarm`) and after that every *interval* seconds (if *interval* is non-zero). The interval timer specified by *which* can be cleared by setting *seconds* to zero.

When an interval timer fires, a signal is sent to the process. The signal sent is dependent on the timer being used; `signal.ITIMER_REAL` will deliver `SIGALRM`, `signal.ITIMER_VIRTUAL` sends `SIGVTALRM`, and `signal.ITIMER_PROF` will deliver `SIGPROF`.

The old values are returned as a two-tuple of floats: (`delay`, `interval`).

Attempting to pass an invalid interval timer will cause an `ItimerError`.

<div class="availability">

Unix.

</div>

<div class="versionchanged">

3.15 Accepts any real numbers as *seconds* and *interval*, not only integers or floats.

</div>

</div>

<div class="function">

getitimer(which)

Returns current value of a given interval timer specified by *which*.

<div class="availability">

Unix.

</div>

</div>

<div class="function">

set_wakeup_fd(fd, \*, warn_on_full_buffer=True)

Set the wakeup file descriptor to *fd*. When a signal your program has registered a signal handler for is received, the signal number is written as a single byte into the fd. If you haven't registered a signal handler for the signals you care about, then nothing will be written to the wakeup fd. This can be used by a library to wakeup a poll or select call, allowing the signal to be fully processed.

The old wakeup fd is returned (or -1 if file descriptor wakeup was not enabled). If *fd* is -1, file descriptor wakeup is disabled. If not -1, *fd* must be non-blocking. It is up to the library to remove any bytes from *fd* before calling poll or select again.

When threads are enabled, this function can only be called from `the main thread of the main interpreter <signals-and-threads>`; attempting to call it from other threads will cause a `ValueError` exception to be raised.

There are two common ways to use this function. In both approaches, you use the fd to wake up when a signal arrives, but then they differ in how they determine *which* signal or signals have arrived.

In the first approach, we read the data out of the fd's buffer, and the byte values give you the signal numbers. This is simple, but in rare cases it can run into a problem: generally the fd will have a limited amount of buffer space, and if too many signals arrive too quickly, then the buffer may become full, and some signals may be lost. If you use this approach, then you should set `warn_on_full_buffer=True`, which will at least cause a warning to be printed to stderr when signals are lost.

In the second approach, we use the wakeup fd *only* for wakeups, and ignore the actual byte values. In this case, all we care about is whether the fd's buffer is empty or non-empty; a full buffer doesn't indicate a problem at all. If you use this approach, then you should set `warn_on_full_buffer=False`, so that your users are not confused by spurious warning messages.

<div class="versionchanged">

3.5 On Windows, the function now also supports socket handles.

</div>

<div class="versionchanged">

3.7 Added `warn_on_full_buffer` parameter.

</div>

</div>

<div class="function">

siginterrupt(signalnum, flag)

Change system call restart behaviour: if *flag* is `False`, system calls will be restarted when interrupted by signal *signalnum*, otherwise system calls will be interrupted. Returns nothing.

<div class="availability">

Unix.

See the man page `siginterrupt(3)` for further information.

</div>

Note that installing a signal handler with `~signal.signal` will reset the restart behaviour to interruptible by implicitly calling `!siginterrupt` with a true *flag* value for the given signal.

</div>

<div class="function">

signal(signalnum, handler)

Set the handler for signal *signalnum* to the function *handler*. *handler* can be a callable Python object taking two arguments (see below), or one of the special values `signal.SIG_IGN` or `signal.SIG_DFL`. The previous signal handler will be returned (see the description of `getsignal` above). (See the Unix man page `signal(2)` for further information.)

When threads are enabled, this function can only be called from `the main thread of the main interpreter <signals-and-threads>`; attempting to call it from other threads will cause a `ValueError` exception to be raised.

The *handler* is called with two arguments: the signal number and the current stack frame (`None` or a frame object; for a description of frame objects, see the `description in the type hierarchy <frame-objects>` or see the attribute descriptions in the `inspect` module).

On Windows, `~signal.signal` can only be called with `SIGABRT`, `SIGFPE`, `SIGILL`, `SIGINT`, `SIGSEGV`, `SIGTERM`, or `SIGBREAK`. A `ValueError` will be raised in any other case. Note that not all systems define the same set of signal names; an `AttributeError` will be raised if a signal name is not defined as `SIG*` module level constant.

</div>

<div class="function">

sigpending()

Examine the set of signals that are pending for delivery to the calling thread (i.e., the signals which have been raised while blocked). Return the set of the pending signals.

<div class="availability">

Unix.

See the man page `sigpending(2)` for further information.

</div>

See also `pause`, `pthread_sigmask` and `sigwait`.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

sigwait(sigset)

Suspend execution of the calling thread until the delivery of one of the signals specified in the signal set *sigset*. The function accepts the signal (removes it from the pending list of signals), and returns the signal number.

<div class="availability">

Unix.

See the man page `sigwait(3)` for further information.

</div>

See also `pause`, `pthread_sigmask`, `sigpending`, `sigwaitinfo` and `sigtimedwait`.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

sigwaitinfo(sigset)

Suspend execution of the calling thread until the delivery of one of the signals specified in the signal set *sigset*. The function accepts the signal and removes it from the pending list of signals. If one of the signals in *sigset* is already pending for the calling thread, the function will return immediately with information about that signal. The signal handler is not called for the delivered signal. The function raises an `InterruptedError` if it is interrupted by a signal that is not in *sigset*.

The return value is an object representing the data contained in the `siginfo_t` structure, namely: `si_signo`, `si_code`, `si_errno`, `si_pid`, `si_uid`, `si_status`, `si_band`.

<div class="availability">

Unix.

See the man page `sigwaitinfo(2)` for further information.

</div>

See also `pause`, `sigwait` and `sigtimedwait`.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.5 The function is now retried if interrupted by a signal not in *sigset* and the signal handler does not raise an exception (see `475` for the rationale).

</div>

</div>

<div class="function">

sigtimedwait(sigset, timeout)

Like `sigwaitinfo`, but takes an additional *timeout* argument specifying a timeout. If *timeout* is specified as `0`, a poll is performed. Returns `None` if a timeout occurs.

<div class="availability">

Unix.

See the man page `sigtimedwait(2)` for further information.

</div>

See also `pause`, `sigwait` and `sigwaitinfo`.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.5 The function is now retried with the recomputed *timeout* if interrupted by a signal not in *sigset* and the signal handler does not raise an exception (see `475` for the rationale).

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float.

</div>

</div>

## Examples

Here is a minimal example program. It uses the `alarm` function to limit the time spent waiting to open a file; this is useful if the file is for a serial device that may not be turned on, which would normally cause the `os.open` to hang indefinitely. The solution is to set a 5-second alarm before opening the file; if the operation takes too long, the alarm signal will be sent, and the handler raises an exception. :

    import signal, os

    def handler(signum, frame):
        signame = signal.Signals(signum).name
        print(f'Signal handler called with signal {signame} ({signum})')
        raise OSError("Couldn't open device!")

    # Set the signal handler and a 5-second alarm
    signal.signal(signal.SIGALRM, handler)
    signal.alarm(5)

    # This open() may hang indefinitely
    fd = os.open('/dev/ttyS0', os.O_RDWR)

    signal.alarm(0)          # Disable the alarm

## Note on SIGPIPE

Piping output of your program to tools like `head(1)` will cause a `SIGPIPE` signal to be sent to your process when the receiver of its standard output closes early. This results in an exception like `BrokenPipeError: [Errno 32] Broken pipe`. To handle this case, wrap your entry point to catch this exception as follows:

    import os
    import sys

    def main():
        try:
            # simulate large output (your code replaces this loop)
            for x in range(10000):
                print("y")
            # flush output here to force SIGPIPE to be triggered
            # while inside this try block.
            sys.stdout.flush()
        except BrokenPipeError:
            # Python flushes standard streams on exit; redirect remaining output
            # to devnull to avoid another BrokenPipeError at shutdown
            devnull = os.open(os.devnull, os.O_WRONLY)
            os.dup2(devnull, sys.stdout.fileno())
            sys.exit(1)  # Python exits with error code 1 on EPIPE

    if __name__ == '__main__':
        main()

Do not set `SIGPIPE`'s disposition to `SIG_DFL` in order to avoid `BrokenPipeError`. Doing that would cause your program to exit unexpectedly whenever any socket connection is interrupted while your program is still writing to it.

## Note on Signal Handlers and Exceptions

If a signal handler raises an exception, the exception will be propagated to the main thread and may be raised after any `bytecode` instruction. Most notably, a `KeyboardInterrupt` may appear at any point during execution. Most Python code, including the standard library, cannot be made robust against this, and so a `KeyboardInterrupt` (or any other exception resulting from a signal handler) may on rare occasions put the program in an unexpected state.

To illustrate this issue, consider the following code:

    class SpamContext:
        def __init__(self):
            self.lock = threading.Lock()

        def __enter__(self):
            # If KeyboardInterrupt occurs here, everything is fine
            self.lock.acquire()
            # If KeyboardInterrupt occurs here, __exit__ will not be called
            ...
            # KeyboardInterrupt could occur just before the function returns

        def __exit__(self, exc_type, exc_val, exc_tb):
            ...
            self.lock.release()

For many programs, especially those that merely want to exit on `KeyboardInterrupt`, this is not a problem, but applications that are complex or require high reliability should avoid raising exceptions from signal handlers. They should also avoid catching `KeyboardInterrupt` as a means of gracefully shutting down. Instead, they should install their own `SIGINT` handler. Below is an example of an HTTP server that avoids `KeyboardInterrupt`:

    import signal
    import socket
    from selectors import DefaultSelector, EVENT_READ
    from http.server import HTTPServer, SimpleHTTPRequestHandler

    interrupt_read, interrupt_write = socket.socketpair()

    def handler(signum, frame):
        print('Signal handler called with signal', signum)
        interrupt_write.send(b'\0')
    signal.signal(signal.SIGINT, handler)

    def serve_forever(httpd):
        sel = DefaultSelector()
        sel.register(interrupt_read, EVENT_READ)
        sel.register(httpd, EVENT_READ)

        while True:
            for key, _ in sel.select():
                if key.fileobj == interrupt_read:
                    interrupt_read.recv(1)
                    return
                if key.fileobj == httpd:
                    httpd.handle_request()

    print("Serving on port 8000")
    httpd = HTTPServer(('', 8000), SimpleHTTPRequestHandler)
    serve_forever(httpd)
    print("Shutdown...")

# `!site` --- Site-specific configuration hook

<div class="module" data-synopsis="Module responsible for site-specific configuration.">

site

</div>

**Source code:** `Lib/site.py`

------------------------------------------------------------------------

**This module is automatically imported during initialization.** The automatic import can be suppressed using the interpreter's `-S` option.

<div class="index">

triple: module; search; path

</div>

Importing this module normally appends site-specific paths to the module search path and adds `callables <site-consts>`, including `help` to the built-in namespace. However, Python startup option `-S` blocks this, and this module can be safely imported with no automatic modifications to the module search path or additions to the builtins. To explicitly trigger the usual site-specific additions, call the `main` function.

<div class="versionchanged">

3.3 Importing the module used to trigger paths manipulation even when using `-S`.

</div>

<div class="index">

pair: site-packages; directory

</div>

It starts by constructing up to four directories from a head and a tail part. For the head part, it uses `sys.prefix` and `sys.exec_prefix`; empty heads are skipped. For the tail part, it uses the empty string and then `lib/site-packages` (on Windows) or `lib/python{X.Y[t]}/site-packages` (on Unix and macOS). (The optional suffix "t" indicates the `free-threaded build`, and is appended if `"t"` is present in the `sys.abiflags` constant.) For each of the distinct head-tail combinations, it sees if it refers to an existing directory, and if so, adds it to `sys.path` and also inspects the newly added path for configuration files.

<div class="versionchanged">

3.5 Support for the "site-python" directory has been removed.

</div>

<div class="versionchanged">

3.13 On Unix, `Free threading <free threading>` Python installations are identified by the "t" suffix in the version-specific directory name, such as `lib/python3.13t/`.

</div>

<div class="versionchanged">

3.14

`!site` is no longer responsible for updating `sys.prefix` and `sys.exec_prefix` on `sys-path-init-virtual-environments`. This is now done during the `path initialization <sys-path-init>`. As a result, under `sys-path-init-virtual-environments`, `sys.prefix` and `sys.exec_prefix` no longer depend on the `!site` initialization, and are therefore unaffected by `-S`.

</div>

<div id="site-virtual-environments-configuration">

When running under a `virtual environment <sys-path-init-virtual-environments>`, the `pyvenv.cfg` file in `sys.prefix` is checked for site-specific configurations. If the `include-system-site-packages` key exists and is set to `true` (case-insensitive), the system-level prefixes will be searched for site-packages, otherwise they won't. If the system-level prefixes are not searched then the user site prefixes are also implicitly not searched for site-packages.

</div>

<div class="index">

single: \# (hash); comment pair: statement; import

</div>

The `!site` module recognizes two startup configuration files of the form `{name}.pth` for path configurations, and `{name}.start` for pre-first-line code execution. Both files can exist in one of the four directories mentioned above. Within each directory, these files are sorted alphabetically by filename, then parsed in sorted order.

## Path extensions (`.pth` files)

`{name}.pth` contains additional items (one per line) to be appended to `sys.path`. Items that name non-existing directories are never added to `sys.path`, and no check is made that the item refers to a directory rather than a file. No item is added to `sys.path` more than once. Blank lines and lines beginning with `#` are skipped.

For backward compatibility, lines starting with `import` (followed by space or tab) are executed with `exec`.

<div class="versionchanged">

3.13

The `.pth` files are now decoded by UTF-8 at first and then by the `locale encoding` if it fails.

</div>

<div class="versionchanged">

3.15

`.pth` file lines starting with `import` are deprecated. During the deprecation period, such lines are still executed (except in the case below), but a diagnostic message is emitted only when the `-v` flag is given.

`import` lines in `{name}.pth` are silently ignored when a `matching <site-start-files>` `{name}.start` file exists.

Errors on individual lines no longer abort processing of the rest of the file. Each error is reported and the remaining lines continue to be processed.

</div>

<div class="deprecated-removed">

3.15 3.20

Decoding `{name}.pth` files in any encoding other than `utf-8-sig` is deprecated in Python 3.15, and support for decoding from the locale encoding will be removed in Python 3.20.

`import` lines in `{name}.pth` files are deprecated and will be silently ignored in Python 3.18 and 3.19. In Python 3.20 a warning will be produced for `import` lines in `{name}.pth` files.

</div>

## Startup entry points (`.start` files)

<div class="versionadded">

3.15

</div>

A startup entry point file is a file whose name has the form `{name}.start` and exists in one of the site-packages directories described above. Each file specifies entry points to be called during interpreter startup, using the `pkg.mod:callable` syntax understood by `pkgutil.resolve_name`.

Each non-blank line that does not begin with `#` must contain an entry point reference in the form `pkg.mod:callable`. The colon and callable portion are mandatory. Each callable is invoked with no arguments, and any return value is discarded.

`.start` files are processed after all `.pth` path extensions have been applied to `sys.path`, ensuring that paths are available before any startup code runs.

Unlike `sys.path` extensions from `.pth` files, duplicate entry points are **not** de-duplicated --- if an entry point appears more than once, it will be called more than once.

If an exception occurs during resolution or invocation of an entry point, a traceback is printed to `sys.stderr` and processing continues with the remaining entry points.

`.start` files must be encoded in UTF-8.

`829` defined the original specification for these features.

> [!NOTE]
> If a `{name}.start` file exists alongside a `{name}.pth` file with the same base name, any `import` lines in the `.pth` file are ignored in favor of the entry points in the `.start` file.

> [!NOTE]
> Executable lines (`import` lines in `{name}.pth` files and `{name}.start` file entry points) are always run at Python startup (unless `-S` is given to disable the `site.py` module entirely), regardless of whether a particular module is actually going to be used.

> [!NOTE]
> `{name}.start` files invoke `pkgutil.resolve_name` with `strict=True`, which requires the full `pkg.mod:callable` form.

<div class="index">

single: package triple: path; configuration; file

</div>

## Startup file examples

For example, suppose `sys.prefix` and `sys.exec_prefix` are set to `/usr/local`. The Python X.Y library is then installed in `/usr/local/lib/python{X.Y}`. Suppose this has a subdirectory `/usr/local/lib/python{X.Y}/site-packages` with three sub-subdirectories, `foo`, `bar` and `spam`, and two path configuration files, `foo.pth` and `bar.pth`. Assume `foo.pth` contains the following:

```
# foo package configuration

foo
bar
bletch
```

and `bar.pth` contains:

```
# bar package configuration

bar
```

Then the following version-specific directories are added to `sys.path`, in this order:

```
/usr/local/lib/pythonX.Y/site-packages/bar
/usr/local/lib/pythonX.Y/site-packages/foo
```

Note that `bletch` is omitted because it doesn't exist; the `bar` directory precedes the `foo` directory because `bar.pth` comes alphabetically before `foo.pth`; and `spam` is omitted because it is not mentioned in either path configuration file.

Let's say that there is also a `foo.start` file containing the following:

```
# foo package startup code

foo.submod:initialize
```

Now, after `sys.path` has been extended as above, and before Python turns control over to user code, the `foo.submod` module is imported and the `initialize()` function from that module is called.

## Migrating from `import` lines in `.pth` files to `.start` files

If your package currently ships a `{name}.pth` file, you can keep all `sys.path` extension lines unchanged. Only `import` lines need to be migrated.

To migrate, create a callable (taking zero arguments) within an importable module in your package. Reference it as a `pkg.mod:callable` entry point in a matching `{name}.start` file. Move everything on your `import` line after the first semi-colon into the `callable()` function.

If your package must straddle older Pythons that do not support `829` and newer Pythons that do, change the `import` lines in your `{name}.pth` to use the following form:

``` python
import pkg.mod; pkg.mod.callable()
```

Older Pythons will execute these `import` lines, while newer Pythons will ignore them in favor of the `{name}.start` file. After the straddling period, remove all `import` lines from your `.pth` files.

## `!sitecustomize`

<div class="module">

sitecustomize

</div>

After these path manipulations, an attempt is made to import a module named `!sitecustomize`, which can perform arbitrary site-specific customizations. It is typically created by a system administrator in the site-packages directory. If this import fails with an `ImportError` or its subclass exception, and the exception's `~ImportError.name` attribute equals `'sitecustomize'`, it is silently ignored. If Python is started without output streams available, as with `pythonw.exe` on Windows (which is used by default to start IDLE), attempted output from `!sitecustomize` is ignored. Any other exception causes a silent and perhaps mysterious failure of the process.

## `!usercustomize`

<div class="module">

usercustomize

</div>

After this, an attempt is made to import a module named `!usercustomize`, which can perform arbitrary user-specific customizations, if `~site.ENABLE_USER_SITE` is true. This file is intended to be created in the user site-packages directory (see below), which is part of `sys.path` unless disabled by `-s`. If this import fails with an `ImportError` or its subclass exception, and the exception's `~ImportError.name` attribute equals `'usercustomize'`, it is silently ignored.

Note that for some non-Unix systems, `sys.prefix` and `sys.exec_prefix` are empty, and the path manipulations are skipped; however the import of `sitecustomize` and `!usercustomize` is still attempted.

<div class="currentmodule">

site

</div>

## Readline configuration

On systems that support `readline`, this module will also import and configure the `rlcompleter` module, if Python is started in `interactive mode <tut-interactive>` and without the `-S` option. The default behavior is to enable tab completion and to use `~/.python_history` as the history save file. To disable it, delete (or override) the `sys.__interactivehook__` attribute in your `sitecustomize` or `usercustomize` module or your `PYTHONSTARTUP` file.

<div class="versionchanged">

3.4 Activation of rlcompleter and history was made automatic.

</div>

## Module contents

<div class="data">

PREFIXES

A list of prefixes for site-packages directories.

</div>

<div class="data">

ENABLE_USER_SITE

Flag showing the status of the user site-packages directory. `True` means that it is enabled and was added to `sys.path`. `False` means that it was disabled by user request (with `-s` or `PYTHONNOUSERSITE`). `None` means it was disabled for security reasons (mismatch between user or group id and effective id) or by an administrator.

</div>

<div class="data">

USER_SITE

Path to the user site-packages for the running Python. Can be `None` if `getusersitepackages` hasn't been called yet. Default value is `~/.local/lib/python{X.Y}[t]/site-packages` for UNIX and non-framework macOS builds, `~/Library/Python/{X.Y}/lib/python/site-packages` for macOS framework builds, and `{%APPDATA%}\\Python\\Python{XY}\\site-packages` on Windows. The optional "t" indicates the free-threaded build. This directory is a site directory, which means that `.pth` files in it will be processed.

</div>

<div class="data">

USER_BASE

Path to the base directory for the user site-packages. Can be `None` if `getuserbase` hasn't been called yet. Default value is `~/.local` for UNIX and macOS non-framework builds, `~/Library/Python/{X.Y}` for macOS framework builds, and `{%APPDATA%}\\Python` for Windows. This value is used to compute the installation directories for scripts, data files, Python modules, etc. for the `user installation scheme <sysconfig-user-scheme>`. See also `PYTHONUSERBASE`.

</div>

<div class="function">

main()

Adds all the standard site-specific directories to the module search path. This function is called automatically when this module is imported, unless the Python interpreter was started with the `-S` flag.

<div class="versionchanged">

3.3 This function used to be called unconditionally.

</div>

</div>

<div class="function">

makepath(\*paths)

Join *paths* with `os.path.join`, attempt to make the result absolute with `os.path.abspath`, and return a 2-tuple containing the absolute path and its case-normalized form as produced by `os.path.normcase`. If `os.path.abspath` raises `OSError`, the joined path is used unchanged for the case-normalization step.

The second element of the returned tuple is the form used throughout the `!site` module to compare paths on case-insensitive file systems, and is what populates the `known_paths` sets that prevent duplicate `sys.path` entries in various APIs within this module.

</div>

<div class="StartupState(known_paths=None)">

Instances of this class accumulate interpreter startup configuration data from one or more site directories. They are the preferred interface for batching the processing of `.pth` and `.start` files across multiple site directories, so that every `sys.path` extension is visible before any startup code runs.

The optional *known_paths* argument is a set of case-normalized paths (which can be produced by `makepath`) used to prevent duplicate `sys.path` entries. When `None` (the default), the set is built from the current `sys.path`. `main` implicitly uses an instance of this class.

Typical use:

``` python
state = site.StartupState()
for sitedir in site_dirs:
    state.addsitedir(sitedir)
state.process()
```

<div class="versionadded">

3.15

</div>

<div class="method">

addsitedir(sitedir)

Read the `.pth` and `.start` files in *sitedir* and record their `sys.path` extensions, deprecated `.pth` `import` lines, and `.start` entry points on this state. The recorded data is not applied until `process` is called.

</div>

<div class="method">

addusersitepackages()

Add the per-user site-packages directory, if enabled and if it exists. The directory's startup data is accumulated for later processing by `process`.

</div>

<div class="method">

addsitepackages(prefixes=None)

Add global site-packages directories, computed from *prefixes* or from the global `PREFIXES` when *prefixes* is `None`. Each directory's startup data is accumulated for later processing by `process`.

</div>

<div class="method">

process()

Apply the accumulated state by first adding the path extensions to `sys.path`, then executing the `.start` file entry points and `.pth` file `import` lines (`deprecated
<site-pth-files>`).

This method is not idempotent and must not be called more than once on the same instance. Doing so will apply the accumulated state more than once, re-running entry points and `import` lines.

</div>

</div>

<div class="function">

addsitedir(sitedir, known_paths=None)

Add a directory to sys.path and parse the `.pth` and `.start` files found in that directory. Typically used in `sitecustomize` or `usercustomize` (see above).

The *known_paths* argument is an optional set of case-normalized paths used to prevent duplicate `sys.path` entries. When `None` (the default), the set is built from the current `sys.path`.

For batched processing across multiple site directories, build a `StartupState` explicitly and call `StartupState.addsitedir` on it; that defers `.pth` and `.start` processing until a single `StartupState.process` call, ensuring every `sys.path` extension is visible before any startup code runs.

<div class="versionchanged">

3.15

Also processes `.start` files. See `site-start-files`.

</div>

</div>

<div class="function">

getsitepackages(prefixes=None)

Return a list containing all global site-packages directories.

For each directory given in *prefixes* (or `PREFIXES` if *prefixes* is `None`), this function will compute its site-packages subdirectory depending on the system environment, and will return a list of full paths, which are not checked for existence.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Added the optional *prefixes* parameter.

</div>

</div>

<div class="function">

getuserbase()

Return the path of the user base directory, `USER_BASE`. If it is not initialized yet, this function will also set it, respecting `PYTHONUSERBASE`.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

getusersitepackages()

Return the path of the user-specific site-packages directory, `USER_SITE`. If it is not initialized yet, this function will also set it, respecting `USER_BASE`. To determine if the user-specific site-packages was added to `sys.path` `ENABLE_USER_SITE` should be used.

<div class="versionadded">

3.2

</div>

</div>

## Command-line interface

<div class="program">

site

</div>

The `!site` module also provides a way to get the user directories from the command line:

```
$ python -m site --user-site
/home/user/.local/lib/python3.11/site-packages
```

If it is called without arguments, it will print the contents of `sys.path` on the standard output, followed by the value of `USER_BASE` and whether the directory exists, then the same thing for `USER_SITE`, and finally the value of `ENABLE_USER_SITE`.

<div class="option">

`--user-base`  

Print the path to the user base directory.

</div>

<div class="option">

`--user-site`  

Print the path to the user site-packages directory.

</div>

If both options are given, user base and user site will be printed (always in this order), separated by `os.pathsep`.

If any option is given, the script will exit with one of these values: `0` if the user site-packages directory is enabled, `1` if it was disabled by the user, `2` if it is disabled for security reasons or by an administrator, and a value greater than 2 if there is an error.

<div class="seealso">

- `370` -- Per user site-packages directory
- `829` -- Startup entry points and the deprecation of import lines in `.pth` files
- `sys-path-init` -- The initialization of `sys.path`.

</div>

# `!smtpd` --- SMTP Server

<div class="module" data-synopsis="Removed in 3.12." data-deprecated="">

smtpd

</div>

<div class="deprecated-removed">

3.6 3.12

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.12 <whatsnew312-removed>` after being deprecated in Python 3.6. The removal was decided in `594`.

A possible replacement is the third-party `aiosmtpd` library. This library is not maintained or supported by the Python core team.

The last version of Python that provided the `!smtpd` module was [Python 3.11](https://docs.python.org/3.11/library/smtpd.html).

# `!smtplib` --- SMTP protocol client

<div class="module" data-synopsis="SMTP protocol client (requires sockets).">

smtplib

</div>

**Source code:** `Lib/smtplib.py`

<div class="index">

pair: SMTP; protocol single: Simple Mail Transfer Protocol

</div>

------------------------------------------------------------------------

The `!smtplib` module defines an SMTP client session object that can be used to send mail to any internet machine with an SMTP or ESMTP listener daemon. For details of SMTP and ESMTP operation, consult `821` (Simple Mail Transfer Protocol) and `1869` (SMTP Service Extensions).

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

<div class="SMTP(host='', port=0, local_hostname=None[, timeout], source_address=None)">

An `SMTP` instance encapsulates an SMTP connection. It has methods that support a full repertoire of SMTP and ESMTP operations.

If the host parameter is set to a truthy value, `SMTP.connect` is called with host and port automatically when the object is created; otherwise, `!connect` must be called manually.

If specified, *local_hostname* is used as the FQDN of the local host in the HELO/EHLO command. Otherwise, the local hostname is found using `socket.getfqdn`. If the `connect` call returns anything other than a success code, an `SMTPConnectError` is raised. The optional *timeout* parameter specifies a timeout in seconds for blocking operations like the connection attempt (if not specified, the global default timeout setting will be used). If the timeout expires, `TimeoutError` is raised. The optional *source_address* parameter allows binding to some specific source address in a machine with multiple network interfaces, and/or to some specific source TCP port. It takes a 2-tuple `(host, port)`, for the socket to bind to as its source address before connecting. If omitted (or if *host* or *port* are `''` and/or `0` respectively) the OS default behavior will be used.

For normal use, you should only require the initialization/connect, `sendmail`, and `SMTP.quit` methods. An example is included below.

The `SMTP` class supports the `with` statement. When used like this, the SMTP `QUIT` command is issued automatically when the `!with` statement exits. E.g.:

    >>> from smtplib import SMTP
    >>> with SMTP("domain.org") as smtp:
    ...     smtp.noop()
    ...
    (250, b'Ok')
    >>>

<div class="audit-event">

smtplib.send self,data smtplib.SMTP

All commands will raise an `auditing event <auditing>` `smtplib.SMTP.send` with arguments `self` and `data`, where `data` is the bytes about to be sent to the remote host.

</div>

<div class="attribute">

SMTP.default_port

The default port used for SMTP connections (25).

</div>

<div class="versionchanged">

3.3 Support for the `with` statement was added.

</div>

<div class="versionchanged">

3.3 *source_address* argument was added.

</div>

<div class="versionadded">

3.5 The SMTPUTF8 extension (`6531`) is now supported.

</div>

<div class="versionchanged">

3.9 If the *timeout* parameter is set to be zero, it will raise a `ValueError` to prevent the creation of a non-blocking socket.

</div>

</div>

<div class="SMTP_SSL(host='', port=0, local_hostname=None, * [, timeout], \ context=None, source_address=None)">

An `SMTP_SSL` instance behaves exactly the same as instances of `SMTP`. `SMTP_SSL` should be used for situations where SSL is required from the beginning of the connection and using `SMTP.starttls` is not appropriate.

If the host parameter is set to a truthy value, `SMTP.connect` is called with host and port automatically when the object is created; otherwise, `!SMTP.connect` must be called manually.

The optional arguments *local_hostname*, *timeout* and *source_address* have the same meaning as they do in the `SMTP` class. *context*, also optional, can contain a `~ssl.SSLContext` and allows configuring various aspects of the secure connection. Please read `ssl-security` for best practices.

<div class="attribute">

SMTP_SSL.default_port

The default port used for SMTP-over-SSL connections (465).

</div>

<div class="versionchanged">

3.3 *context* was added.

</div>

<div class="versionchanged">

3.3 The *source_address* argument was added.

</div>

<div class="versionchanged">

3.4 The class now supports hostname check with `ssl.SSLContext.check_hostname` and *Server Name Indication* (see `ssl.HAS_SNI`).

</div>

<div class="versionchanged">

3.9 If the *timeout* parameter is set to be zero, it will raise a `ValueError` to prevent the creation of a non-blocking socket

</div>

<div class="versionchanged">

3.12 The deprecated *keyfile* and *certfile* parameters have been removed.

</div>

</div>

<div class="LMTP(host='', port=LMTP_PORT, local_hostname=None, \ source_address=None[, timeout])">

The LMTP protocol, which is very similar to ESMTP, is heavily based on the standard SMTP client. It's common to use Unix sockets for LMTP, so our `~SMTP.connect` method must support that as well as a regular host:port server. The optional arguments *local_hostname* and *source_address* have the same meaning as they do in the `SMTP` class. To specify a Unix socket, you must use an absolute path for *host*, starting with a '/'.

Authentication is supported, using the regular SMTP mechanism. When using a Unix socket, LMTP generally don't support or require any authentication, but your mileage might vary.

<div class="versionchanged">

3.9 The optional *timeout* parameter was added.

</div>

</div>

A nice selection of exceptions is defined as well:

<div class="exception">

SMTPException

Subclass of `OSError` that is the base exception class for all the other exceptions provided by this module.

<div class="versionchanged">

3.4 SMTPException became subclass of `OSError`

</div>

</div>

<div class="exception">

SMTPServerDisconnected

This exception is raised when the server unexpectedly disconnects, or when an attempt is made to use the `SMTP` instance before connecting it to a server.

</div>

<div class="exception">

SMTPResponseException

Base class for all exceptions that include an SMTP error code. These exceptions are generated in some instances when the SMTP server returns an error code.

<div class="attribute">

smtp_code

The error code.

</div>

<div class="attribute">

smtp_error

The error message.

</div>

</div>

<div class="exception">

SMTPSenderRefused

Sender address refused. In addition to the attributes set by on all `SMTPResponseException` exceptions, this sets 'sender' to the string that the SMTP server refused.

</div>

<div class="exception">

SMTPRecipientsRefused

All recipient addresses refused.

<div class="attribute">

recipients

A dictionary of exactly the same sort as returned by `SMTP.sendmail` containing the errors for each recipient.

</div>

</div>

<div class="exception">

SMTPDataError

The SMTP server refused to accept the message data.

</div>

<div class="exception">

SMTPConnectError

Error occurred during establishment of a connection with the server.

</div>

<div class="exception">

SMTPHeloError

The server refused our `HELO` message.

</div>

<div class="exception">

SMTPNotSupportedError

The command or option attempted is not supported by the server.

<div class="versionadded">

3.5

</div>

</div>

<div class="exception">

SMTPAuthenticationError

SMTP authentication went wrong. Most probably the server didn't accept the username/password combination provided.

</div>

<div class="seealso">

`821` - Simple Mail Transfer Protocol  
Protocol definition for SMTP. This document covers the model, operating procedure, and protocol details for SMTP.

`1869` - SMTP Service Extensions  
Definition of the ESMTP extensions for SMTP. This describes a framework for extending SMTP with new commands, supporting dynamic discovery of the commands provided by the server, and defines a few additional commands.

</div>

## SMTP Objects

An `SMTP` instance has the following methods:

<div class="method">

SMTP.set_debuglevel(level)

Set the debug output level. A value of 1 or `True` for *level* results in debug messages for connection and for all messages sent to and received from the server. A value of 2 for *level* results in these messages being timestamped.

<div class="versionchanged">

3.5 Added debuglevel 2.

</div>

</div>

<div class="method">

SMTP.docmd(cmd, args='')

Send a command *cmd* to the server. The optional argument *args* is simply concatenated to the command, separated by a space.

This returns a 2-tuple composed of a numeric response code and the actual response line (multiline responses are joined into one long line.)

In normal operation it should not be necessary to call this method explicitly. It is used to implement other methods and may be useful for testing private extensions.

If the connection to the server is lost while waiting for the reply, `SMTPServerDisconnected` will be raised.

</div>

<div class="method">

SMTP.connect(host='localhost', port=0)

Connect to a host on a given port. The defaults are to connect to the local host at the standard SMTP port (25). If the hostname ends with a colon (`':'`) followed by a number, that suffix will be stripped off and the number interpreted as the port number to use. This method is automatically invoked by the constructor if a host is specified during instantiation. Returns a 2-tuple of the response code and message sent by the server in its connection response.

If port is not changed from its default value of 0, the value of the `default_port` attribute is used.

<div class="audit-event">

smtplib.connect self,host,port smtplib.SMTP.connect

</div>

</div>

<div class="method">

SMTP.helo(name='')

Identify yourself to the SMTP server using `HELO`. The hostname argument defaults to the fully qualified domain name of the local host. The message returned by the server is stored as the `helo_resp` attribute of the object.

In normal operation it should not be necessary to call this method explicitly. It will be implicitly called by the `sendmail` when necessary.

</div>

<div class="method">

SMTP.ehlo(name='')

Identify yourself to an ESMTP server using `EHLO`. The hostname argument defaults to the fully qualified domain name of the local host. Examine the response for ESMTP option and store them for use by `has_extn`. Also sets several informational attributes: the message returned by the server is stored as the `ehlo_resp` attribute, `does_esmtp` is set to `True` or `False` depending on whether the server supports ESMTP, and `esmtp_features` will be a dictionary containing the names of the SMTP service extensions this server supports, and their parameters (if any).

Unless you wish to use `has_extn` before sending mail, it should not be necessary to call this method explicitly. It will be implicitly called by `sendmail` when necessary.

</div>

<div class="method">

SMTP.ehlo_or_helo_if_needed()

This method calls `ehlo` and/or `helo` if there has been no previous `EHLO` or `HELO` command this session. It tries ESMTP `EHLO` first.

`SMTPHeloError`  
The server didn't reply properly to the `HELO` greeting.

</div>

<div class="method">

SMTP.has_extn(name)

Return `True` if *name* is in the set of SMTP service extensions returned by the server, `False` otherwise. Case is ignored.

</div>

<div class="method">

SMTP.verify(address)

Check the validity of an address on this server using SMTP `VRFY`. Returns a tuple consisting of code 250 and a full `822` address (including human name) if the user address is valid. Otherwise returns an SMTP error code of 400 or greater and an error string.

> [!NOTE]
> Many sites disable SMTP `VRFY` in order to foil spammers.

</div>

<div class="method">

SMTP.login(user, password, \*, initial_response_ok=True)

Log in on an SMTP server that requires authentication. The arguments are the username and the password to authenticate with. If there has been no previous `EHLO` or `HELO` command this session, this method tries ESMTP `EHLO` first. This method will return normally if the authentication was successful, or may raise the following exceptions:

`SMTPHeloError`  
The server didn't reply properly to the `HELO` greeting.

`SMTPAuthenticationError`  
The server didn't accept the username/password combination.

`SMTPNotSupportedError`  
The `AUTH` command is not supported by the server.

`SMTPException`  
No suitable authentication method was found.

Each of the authentication methods supported by `!smtplib` are tried in turn if they are advertised as supported by the server. See `auth` for a list of supported authentication methods. *initial_response_ok* is passed through to `auth`.

Optional keyword argument *initial_response_ok* specifies whether, for authentication methods that support it, an "initial response" as specified in `4954` can be sent along with the `AUTH` command, rather than requiring a challenge/response.

<div class="versionchanged">

3.5 `SMTPNotSupportedError` may be raised, and the *initial_response_ok* parameter was added.

</div>

</div>

<div class="method">

SMTP.auth(mechanism, authobject, \*, initial_response_ok=True)

Issue an `SMTP` `AUTH` command for the specified authentication *mechanism*, and handle the challenge response via *authobject*.

*mechanism* specifies which authentication mechanism is to be used as argument to the `AUTH` command; the valid values are those listed in the `auth` element of `esmtp_features`.

*authobject* must be a callable object taking an optional single argument:

    data = authobject(challenge=None)

If optional keyword argument *initial_response_ok* is true, `authobject()` will be called first with no argument. It can return the `4954` "initial response" ASCII `str` which will be encoded and sent with the `AUTH` command as below. If the `authobject()` does not support an initial response (e.g. because it requires a challenge), it should return `None` when called with `challenge=None`. If *initial_response_ok* is false, then `authobject()` will not be called first with `None`.

If the initial response check returns `None`, or if *initial_response_ok* is false, `authobject()` will be called to process the server's challenge response; the *challenge* argument it is passed will be a `bytes`. It should return ASCII `str` *data* that will be base64 encoded and sent to the server.

The `SMTP` class provides `authobjects` for the `CRAM-MD5`, `PLAIN`, and `LOGIN` mechanisms; they are named `SMTP.auth_cram_md5`, `SMTP.auth_plain`, and `SMTP.auth_login` respectively. They all require that the `user` and `password` properties of the `SMTP` instance are set to appropriate values.

User code does not normally need to call `auth` directly, but can instead call the `login` method, which will try each of the above mechanisms in turn, in the order listed. `auth` is exposed to facilitate the implementation of authentication methods not (or not yet) supported directly by `!smtplib`.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

SMTP.starttls(\*, context=None)

Put the SMTP connection in TLS (Transport Layer Security) mode. All SMTP commands that follow will be encrypted. You should then call `ehlo` again.

If *keyfile* and *certfile* are provided, they are used to create an `ssl.SSLContext`.

Optional *context* parameter is an `ssl.SSLContext` object; This is an alternative to using a keyfile and a certfile and if specified both *keyfile* and *certfile* should be `None`.

If there has been no previous `EHLO` or `HELO` command this session, this method tries ESMTP `EHLO` first.

<div class="versionchanged">

3.12 The deprecated *keyfile* and *certfile* parameters have been removed.

</div>

`SMTPHeloError`  
The server didn't reply properly to the `HELO` greeting.

`SMTPNotSupportedError`  
The server does not support the STARTTLS extension.

`RuntimeError`  
SSL/TLS support is not available to your Python interpreter.

<div class="versionchanged">

3.3 *context* was added.

</div>

<div class="versionchanged">

3.4 The method now supports hostname check with `ssl.SSLContext.check_hostname` and *Server Name Indicator* (see `~ssl.HAS_SNI`).

</div>

<div class="versionchanged">

3.5 The error raised for lack of STARTTLS support is now the `SMTPNotSupportedError` subclass instead of the base `SMTPException`.

</div>

</div>

<div class="method">

SMTP.sendmail(from_addr, to_addrs, msg, mail_options=(), rcpt_options=())

Send mail. The required arguments are an `822` from-address string, a list of `822` to-address strings (a bare string will be treated as a list with 1 address), and a message string. The caller may pass a list of ESMTP options (such as `"8bitmime"`) to be used in `MAIL FROM` commands as *mail_options*. ESMTP options (such as `DSN` commands) that should be used with all `RCPT` commands can be passed as *rcpt_options*. Each option should be passed as a string containing the full text of the option, including any potential key (for instance, `"NOTIFY=SUCCESS,FAILURE"`). (If you need to use different ESMTP options to different recipients you have to use the low-level methods such as `!mail`, `!rcpt` and `!data` to send the message.)

> [!NOTE]
> The *from_addr* and *to_addrs* parameters are used to construct the message envelope used by the transport agents. `sendmail` does not modify the message headers in any way.

*msg* may be a string containing characters in the ASCII range, or a byte string. A string is encoded to bytes using the ascii codec, and lone `\r` and `\n` characters are converted to `\r\n` characters. A byte string is not modified.

If there has been no previous `EHLO` or `HELO` command this session, this method tries ESMTP `EHLO` first. If the server does ESMTP, message size and each of the specified options will be passed to it (if the option is in the feature set the server advertises). If `EHLO` fails, `HELO` will be tried and ESMTP options suppressed.

This method will return normally if the mail is accepted for at least one recipient. Otherwise it will raise an exception. That is, if this method does not raise an exception, then someone should get your mail. If this method does not raise an exception, it returns a dictionary, with one entry for each recipient that was refused. Each entry contains a tuple of the SMTP error code and the accompanying error message sent by the server.

If `SMTPUTF8` is included in *mail_options*, and the server supports it, *from_addr* and *to_addrs* may contain non-ASCII characters.

This method may raise the following exceptions:

`SMTPRecipientsRefused`  
All recipients were refused. Nobody got the mail.

`SMTPHeloError`  
The server didn't reply properly to the `HELO` greeting.

`SMTPSenderRefused`  
The server didn't accept the *from_addr*.

`SMTPDataError`  
The server replied with an unexpected error code (other than a refusal of a recipient).

`SMTPNotSupportedError`  
`SMTPUTF8` was given in the *mail_options* but is not supported by the server.

Unless otherwise noted, the connection will be open even after an exception is raised.

<div class="versionchanged">

3.2 *msg* may be a byte string.

</div>

<div class="versionchanged">

3.5 `SMTPUTF8` support added, and `SMTPNotSupportedError` may be raised if `SMTPUTF8` is specified but the server does not support it.

</div>

</div>

<div class="method">

SMTP.send_message(msg, from_addr=None, to_addrs=None, mail_options=(), rcpt_options=())

This is a convenience method for calling `sendmail` with the message represented by an `email.message.Message` object. The arguments have the same meaning as for `sendmail`, except that *msg* is a `Message` object.

If *from_addr* is `None` or *to_addrs* is `None`, `send_message` fills those arguments with addresses extracted from the headers of *msg* as specified in `5322`: *from_addr* is set to the `Sender` field if it is present, and otherwise to the `From` field. *to_addrs* combines the values (if any) of the `To`, `Cc`, and `Bcc` fields from *msg*. If exactly one set of `Resent-*` headers appear in the message, the regular headers are ignored and the `Resent-*` headers are used instead. If the message contains more than one set of `Resent-*` headers, a `ValueError` is raised, since there is no way to unambiguously detect the most recent set of `Resent-` headers.

`send_message` serializes *msg* using `~email.generator.BytesGenerator` with `\r\n` as the *linesep*, and calls `sendmail` to transmit the resulting message. Regardless of the values of *from_addr* and *to_addrs*, `send_message` does not transmit any `Bcc` or `Resent-Bcc` headers that may appear in *msg*. If any of the addresses in *from_addr* and *to_addrs* contain non-ASCII characters and the server does not advertise `SMTPUTF8` support, an `SMTPNotSupportedError` is raised. Otherwise the `Message` is serialized with a clone of its `~email.policy` with the `~email.policy.EmailPolicy.utf8` attribute set to `True`, and `SMTPUTF8` and `BODY=8BITMIME` are added to *mail_options*.

<div class="versionadded">

3.2

</div>

<div class="versionadded">

3.5 Support for internationalized addresses (`SMTPUTF8`).

</div>

</div>

<div class="method">

SMTP.quit()

Terminate the SMTP session and close the connection. Return the result of the SMTP `QUIT` command.

</div>

Low-level methods corresponding to the standard SMTP/ESMTP commands `HELP`, `RSET`, `NOOP`, `MAIL`, `RCPT`, and `DATA` are also supported. Normally these do not need to be called directly, so they are not documented here. For details, consult the module code.

Additionally, an SMTP instance has the following attributes:

<div class="attribute">

SMTP.helo_resp

The response to the `HELO` command, see `helo`.

</div>

<div class="attribute">

SMTP.ehlo_resp

The response to the `EHLO` command, see `ehlo`.

</div>

<div class="attribute">

SMTP.does_esmtp

A boolean value indicating whether the server supports ESMTP, see `ehlo`.

</div>

<div class="attribute">

SMTP.esmtp_features

A dictionary of the names of SMTP service extensions supported by the server, see `ehlo`.

</div>

## SMTP Example

This example prompts the user for addresses needed in the message envelope ('To' and 'From' addresses), and the message to be delivered. Note that the headers to be included with the message must be included in the message as entered; this example doesn't do any processing of the `822` headers. In particular, the 'To' and 'From' addresses must be included in the message headers explicitly:

    import smtplib

    def prompt(title):
        return input(title).strip()

    from_addr = prompt("From: ")
    to_addrs  = prompt("To: ").split()
    print("Enter message, end with ^D (Unix) or ^Z (Windows):")

    # Add the From: and To: headers at the start!
    lines = [f"From: {from_addr}", f"To: {', '.join(to_addrs)}", ""]
    while True:
        try:
            line = input()
        except EOFError:
            break
        else:
            lines.append(line)

    msg = "\r\n".join(lines)
    print("Message length is", len(msg))

    server = smtplib.SMTP("localhost")
    server.set_debuglevel(1)
    server.sendmail(from_addr, to_addrs, msg)
    server.quit()

> [!NOTE]
> In general, you will want to use the `email` package's features to construct an email message, which you can then send via `~smtplib.SMTP.send_message`; see `email-examples`.

# `!sndhdr` --- Determine type of sound file

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

sndhdr

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

Possible replacements are third-party modules from PyPI: `filetype`, `puremagic`, or `python-magic`. These are not supported or maintained by the Python core team.

The last version of Python that provided the `!sndhdr` module was [Python 3.12](https://docs.python.org/3.12/library/sndhdr.html).

# `!socket` --- Low-level networking interface

<div class="module" data-synopsis="Low-level networking interface.">

socket

</div>

**Source code:** `Lib/socket.py`

------------------------------------------------------------------------

This module provides access to the BSD *socket* interface. It is available on all modern Unix systems, Windows, MacOS, and probably additional platforms.

> [!NOTE]
> Some behavior may be platform dependent, since calls are made to the operating system socket APIs.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

<div class="index">

pair: object; socket

</div>

The Python interface is a straightforward transliteration of the Unix system call and library interface for sockets to Python's object-oriented style: the `~socket.socket` function returns a `socket object` whose methods implement the various socket system calls. Parameter types are somewhat higher-level than in the C interface: as with `read` and `write` operations on Python files, buffer allocation on receive operations is automatic, and buffer length is implicit on send operations.

<div class="seealso">

Module `socketserver`  
Classes that simplify writing network servers.

Module `ssl`  
A TLS/SSL wrapper for socket objects.

</div>

## Socket families

Depending on the system and the build options, various socket families are supported by this module.

The address format required by a particular socket object is automatically selected based on the address family specified when the socket object was created. Socket addresses are represented as follows:

- The address of an `AF_UNIX` socket bound to a file system node is represented as a string, using the file system encoding and the `'surrogateescape'` error handler (see `383`). An address in Linux's abstract namespace is returned as a `bytes-like object` with an initial null byte; note that sockets in this namespace can communicate with normal file system sockets, so programs intended to run on Linux may need to deal with both types of address. A string or bytes-like object can be used for either type of address when passing it as an argument.

  <div class="versionchanged">

  3.3 Previously, `AF_UNIX` socket paths were assumed to use UTF-8 encoding.

  </div>

  <div class="versionchanged">

  3.5 Writable `bytes-like object` is now accepted.

  </div>

<div id="host_port">

- A pair `(host, port)` is used for the `AF_INET` address family, where *host* is a string representing either a hostname in internet domain notation like `'daring.cwi.nl'` or an IPv4 address like `'100.50.200.5'`, and *port* is an integer.

  - For IPv4 addresses, two special forms are accepted instead of a host address: `''` represents `INADDR_ANY`, which is used to bind to all interfaces, and the string `'<broadcast>'` represents `INADDR_BROADCAST`. This behavior is not compatible with IPv6, therefore, you may want to avoid these if you intend to support IPv6 with your Python programs.

- For `AF_INET6` address family, a four-tuple `(host, port, flowinfo, scope_id)` is used, where *flowinfo* and *scope_id* represent the `sin6_flowinfo` and `sin6_scope_id` members in `struct sockaddr_in6` in C. For `!socket` module methods, *flowinfo* and *scope_id* can be omitted just for backward compatibility. Note, however, omission of *scope_id* can cause problems in manipulating scoped IPv6 addresses.

  <div class="versionchanged">

  3.7 For multicast addresses (with *scope_id* meaningful) *address* may not contain `%scope_id` (or `zone id`) part. This information is superfluous and may be safely omitted (recommended).

  </div>

- `AF_NETLINK` sockets are represented as pairs `(pid, groups)`.

- Linux-only support for TIPC is available using the `AF_TIPC` address family. TIPC is an open, non-IP based networked protocol designed for use in clustered computer environments. Addresses are represented by a tuple, and the fields depend on the address type. The general tuple form is `(addr_type, v1, v2, v3 [, scope])`, where:

  - *addr_type* is one of `TIPC_ADDR_NAMESEQ`, `TIPC_ADDR_NAME`, or `TIPC_ADDR_ID`.

  - *scope* is one of `TIPC_ZONE_SCOPE`, `TIPC_CLUSTER_SCOPE`, and `TIPC_NODE_SCOPE`.

  - If *addr_type* is `TIPC_ADDR_NAME`, then *v1* is the server type, *v2* is the port identifier, and *v3* should be 0.

    If *addr_type* is `TIPC_ADDR_NAMESEQ`, then *v1* is the server type, *v2* is the lower port number, and *v3* is the upper port number.

    If *addr_type* is `TIPC_ADDR_ID`, then *v1* is the node, *v2* is the reference, and *v3* should be set to 0.

- A tuple `(interface, )` is used for the `AF_CAN` address family, where *interface* is a string representing a network interface name like `'can0'`. The network interface name `''` can be used to receive packets from all network interfaces of this family.

  - `CAN_ISOTP` protocol requires a tuple `(interface, rx_addr, tx_addr)` where both additional parameters are unsigned long integer that represent a CAN identifier (standard or extended).
  - `CAN_J1939` protocol requires a tuple `(interface, name, pgn, addr)` where additional parameters are 64-bit unsigned integer representing the ECU name, a 32-bit unsigned integer representing the Parameter Group Number (PGN), and an 8-bit integer representing the address.

- A string or a tuple `(id, unit)` is used for the `SYSPROTO_CONTROL` protocol of the `PF_SYSTEM` family. The string is the name of a kernel control using a dynamically assigned ID. The tuple can be used if ID and unit number of the kernel control are known or if a registered ID is used.

  <div class="versionadded">

  3.3

  </div>

- `AF_BLUETOOTH` supports the following protocols and address formats:

  - `BTPROTO_L2CAP` accepts a tuple `(bdaddr, psm[, cid[, bdaddr_type]])` where:

    - `bdaddr` is a string specifying the Bluetooth address.
    - `psm` is an integer specifying the Protocol/Service Multiplexer.
    - `cid` is an optional integer specifying the Channel Identifier. If not given, defaults to zero.
    - `bdaddr_type` is an optional integer specifying the address type; one of `BDADDR_BREDR` (default), `BDADDR_LE_PUBLIC`, `BDADDR_LE_RANDOM`.

    <div class="versionchanged">

    3.14 Added `cid` and `bdaddr_type` fields.

    </div>

  - `BTPROTO_RFCOMM` accepts `(bdaddr, channel)` where `bdaddr` is the Bluetooth address as a string and `channel` is an integer.

  - `BTPROTO_HCI` accepts a format that depends on your OS.

    - On Linux it accepts an integer `device_id` or a tuple `(device_id, [channel])` where `device_id` specifies the number of the Bluetooth device, and `channel` is an optional integer specifying the HCI channel (`HCI_CHANNEL_RAW` by default).
    - On FreeBSD, NetBSD and DragonFly BSD it accepts `bdaddr` where `bdaddr` is the Bluetooth address as a string.

    <div class="versionchanged">

    3.2 NetBSD and DragonFlyBSD support added.

    </div>

    <div class="versionchanged">

    3.13.3 FreeBSD support added.

    </div>

    <div class="versionchanged">

    3.14 Added `channel` field. `device_id` not packed in a tuple is now accepted.

    </div>

  - `BTPROTO_SCO` accepts `bdaddr` where `bdaddr` is the Bluetooth address as a string or a `bytes` object. (ex. `'12:23:34:45:56:67'` or `b'12:23:34:45:56:67'`)

    <div class="versionchanged">

    3.14 FreeBSD support added.

    </div>

- `AF_ALG` is a Linux-only socket based interface to Kernel cryptography. An algorithm socket is configured with a tuple of two to four elements `(type, name [, feat [, mask]])`, where:

  - *type* is the algorithm type as string, e.g. `aead`, `hash`, `skcipher` or `rng`.
  - *name* is the algorithm name and operation mode as string, e.g. `sha256`, `hmac(sha256)`, `cbc(aes)` or `drbg_nopr_ctr_aes256`.
  - *feat* and *mask* are unsigned 32bit integers.

  <div class="availability">

  Linux \>= 2.6.38.

  Some algorithm types require more recent Kernels.

  </div>

  <div class="versionadded">

  3.6

  </div>

- `AF_VSOCK` allows communication between virtual machines and their hosts. The sockets are represented as a `(CID, port)` tuple where the context ID or CID and port are integers.

  <div class="availability">

  Linux \>= 3.9

  See `vsock(7)`

  </div>

  <div class="versionadded">

  3.7

  </div>

- `AF_PACKET` is a low-level interface directly to network devices. The addresses are represented by the tuple `(ifname, proto[, pkttype[, hatype[, addr]]])` where:

  - *ifname* - String specifying the device name.
  - *proto* - The Ethernet protocol number. May be `ETH_P_ALL` to capture all protocols, one of the `ETHERTYPE_* constants <socket-ethernet-types>` or any other Ethernet protocol number.
  - *pkttype* - Optional integer specifying the packet type:
    - `PACKET_HOST` (the default) - Packet addressed to the local host.
    - `PACKET_BROADCAST` - Physical-layer broadcast packet.
    - `PACKET_MULTICAST` - Packet sent to a physical-layer multicast address.
    - `PACKET_OTHERHOST` - Packet to some other host that has been caught by a device driver in promiscuous mode.
    - `PACKET_OUTGOING` - Packet originating from the local host that is looped back to a packet socket.
  - *hatype* - Optional integer specifying the ARP hardware address type.
  - *addr* - Optional bytes-like object specifying the hardware physical address, whose interpretation depends on the device.

  <div class="availability">

  Linux \>= 2.2.

  </div>

- `AF_QIPCRTR` is a Linux-only socket based interface for communicating with services running on co-processors in Qualcomm platforms. The address family is represented as a `(node, port)` tuple where the *node* and *port* are non-negative integers.

  <div class="availability">

  Linux \>= 4.7.

  </div>

  <div class="versionadded">

  3.8

  </div>

- `IPPROTO_UDPLITE` is a variant of UDP which allows you to specify what portion of a packet is covered with the checksum. It adds two socket options that you can change. `self.setsockopt(IPPROTO_UDPLITE, UDPLITE_SEND_CSCOV, length)` will change what portion of outgoing packets are covered by the checksum and `self.setsockopt(IPPROTO_UDPLITE, UDPLITE_RECV_CSCOV, length)` will filter out packets which cover too little of their data. In both cases `length` should be in `range(8, 2**16, 8)`.

  Such a socket should be constructed with `socket(AF_INET, SOCK_DGRAM, IPPROTO_UDPLITE)` for IPv4 or `socket(AF_INET6, SOCK_DGRAM, IPPROTO_UDPLITE)` for IPv6.

  <div class="availability">

  Linux \>= 2.6.20, FreeBSD \>= 10.1

  </div>

  <div class="versionadded">

  3.9

  </div>

- `AF_HYPERV` is a Windows-only socket based interface for communicating with Hyper-V hosts and guests. The address family is represented as a `(vm_id, service_id)` tuple where the `vm_id` and `service_id` are UUID strings.

  The `vm_id` is the virtual machine identifier or a set of known VMID values if the target is not a specific virtual machine. Known VMID constants defined on `socket` are:

  - `HV_GUID_ZERO`
  - `HV_GUID_BROADCAST`
  - `HV_GUID_WILDCARD` - Used to bind on itself and accept connections from all partitions.
  - `HV_GUID_CHILDREN` - Used to bind on itself and accept connection from child partitions.
  - `HV_GUID_LOOPBACK` - Used as a target to itself.
  - `HV_GUID_PARENT` - When used as a bind accepts connection from the parent partition. When used as an address target it will connect to the parent partition.

  The `service_id` is the service identifier of the registered service.

  <div class="versionadded">

  3.12

  </div>

</div>

If you use a hostname in the *host* portion of IPv4/v6 socket address, the program may show a nondeterministic behavior, as Python uses the first address returned from the DNS resolution. The socket address will be resolved differently into an actual IPv4/v6 address, depending on the results from DNS resolution and/or the host configuration. For deterministic behavior use a numeric address in *host* portion.

All errors raise exceptions. The normal exceptions for invalid argument types and out-of-memory conditions can be raised. Errors related to socket or address semantics raise `OSError` or one of its subclasses.

Non-blocking mode is supported through `~socket.setblocking`. A generalization of this based on timeouts is supported through `~socket.settimeout`.

## Module contents

The module `!socket` exports the following elements.

### Exceptions

<div class="exception">

error

A deprecated alias of `OSError`.

<div class="versionchanged">

3.3 Following `3151`, this class was made an alias of `OSError`.

</div>

</div>

<div class="exception">

herror

A subclass of `OSError`, this exception is raised for address-related errors, i.e. for functions that use *h_errno* in the POSIX C API, including `gethostbyname_ex` and `gethostbyaddr`. The accompanying value is a pair `(h_errno, string)` representing an error returned by a library call. *h_errno* is a numeric value, while *string* represents the description of *h_errno*, as returned by the `hstrerror` C function.

<div class="versionchanged">

3.3 This class was made a subclass of `OSError`.

</div>

</div>

<div class="exception">

gaierror

A subclass of `OSError`, this exception is raised for address-related errors by `getaddrinfo` and `getnameinfo`. The accompanying value is a pair `(error, string)` representing an error returned by a library call. *string* represents the description of *error*, as returned by the `gai_strerror` C function. The numeric *error* value will match one of the `!EAI_\*` constants defined in this module.

<div class="versionchanged">

3.3 This class was made a subclass of `OSError`.

</div>

</div>

<div class="exception">

timeout

A deprecated alias of `TimeoutError`.

A subclass of `OSError`, this exception is raised when a timeout occurs on a socket which has had timeouts enabled via a prior call to `~socket.settimeout` (or implicitly through `~socket.setdefaulttimeout`). The accompanying value is a string whose value is currently always "timed out".

<div class="versionchanged">

3.3 This class was made a subclass of `OSError`.

</div>

<div class="versionchanged">

3.10 This class was made an alias of `TimeoutError`.

</div>

</div>

### Constants

The [AF]()\* and [SOCK]()\* constants are now `AddressFamily` and `SocketKind` `.IntEnum` collections.

<div class="versionadded">

3.4

</div>

<div class="data">

AF_UNIX AF_INET AF_INET6

These constants represent the address (and protocol) families, used for the first argument to `~socket.socket`. If the `AF_UNIX` constant is not defined then this protocol is unsupported. More constants may be available depending on the system.

</div>

<div class="data">

AF_UNSPEC

`AF_UNSPEC` means that `getaddrinfo` should return socket addresses for any address family (either IPv4, IPv6, or any other) that can be used.

</div>

<div class="data">

SOCK_STREAM SOCK_DGRAM SOCK_RAW SOCK_RDM SOCK_SEQPACKET

These constants represent the socket types, used for the second argument to `~socket.socket`. More constants may be available depending on the system. (Only `SOCK_STREAM` and `SOCK_DGRAM` appear to be generally useful.)

</div>

<div class="data">

SOCK_CLOEXEC SOCK_NONBLOCK

These two constants, if defined, can be combined with the socket types and allow you to set some flags atomically (thus avoiding possible race conditions and the need for separate calls).

<div class="seealso">

[Secure File Descriptor Handling](https://udrepper.livejournal.com/20407.html) for a more thorough explanation.

</div>

<div class="availability">

Linux \>= 2.6.27.

</div>

<div class="versionadded">

3.2

</div>

</div>

<div id="socket-unix-constants">

<div class="data">

[SO]()\* SOMAXCONN [MSG]()\* [SOL]()\* [SCM]()\* [IPPROTO]()\* [IPPORT]()\* [INADDR]()\* [IP]()\* [IPV6]()\* [EAI]()\* [AI]()\* [NI]()\* [TCP]()\*

Many constants of these forms, documented in the Unix documentation on sockets and/or the IP protocol, are also defined in the socket module. They are generally used in arguments to the `~socket.setsockopt` and `~socket.getsockopt` methods of socket objects. In most cases, only those symbols that are defined in the Unix header files are defined; for a few symbols, default values are provided.

<div class="versionchanged">

3.6 `SO_DOMAIN`, `SO_PROTOCOL`, `SO_PEERSEC`, `SO_PASSSEC`, `TCP_USER_TIMEOUT`, `TCP_CONGESTION` were added.

</div>

<div class="versionchanged">

3.6.5 Added support for `TCP_FASTOPEN`, `TCP_KEEPCNT` on Windows platforms when available.

</div>

<div class="versionchanged">

3.7 `TCP_NOTSENT_LOWAT` was added.

Added support for `TCP_KEEPIDLE`, `TCP_KEEPINTVL` on Windows platforms when available.

</div>

<div class="versionchanged">

3.10 `IP_RECVTOS` was added. Added `TCP_KEEPALIVE`. On MacOS this constant can be used in the same way that `TCP_KEEPIDLE` is used on Linux.

</div>

<div class="versionchanged">

3.11 Added `TCP_CONNECTION_INFO`. On MacOS this constant can be used in the same way that `TCP_INFO` is used on Linux and BSD.

</div>

<div class="versionchanged">

3.12 Added `SO_RTABLE` and `SO_USER_COOKIE`. On OpenBSD and FreeBSD respectively those constants can be used in the same way that `SO_MARK` is used on Linux. Also added missing TCP socket options from Linux: `TCP_MD5SIG`, `TCP_THIN_LINEAR_TIMEOUTS`, `TCP_THIN_DUPACK`, `TCP_REPAIR`, `TCP_REPAIR_QUEUE`, `TCP_QUEUE_SEQ`, `TCP_REPAIR_OPTIONS`, `TCP_TIMESTAMP`, `TCP_CC_INFO`, `TCP_SAVE_SYN`, `TCP_SAVED_SYN`, `TCP_REPAIR_WINDOW`, `TCP_FASTOPEN_CONNECT`, `TCP_ULP`, `TCP_MD5SIG_EXT`, `TCP_FASTOPEN_KEY`, `TCP_FASTOPEN_NO_COOKIE`, `TCP_ZEROCOPY_RECEIVE`, `TCP_INQ`, `TCP_TX_DELAY`. Added `IP_PKTINFO`, `IP_UNBLOCK_SOURCE`, `IP_BLOCK_SOURCE`, `IP_ADD_SOURCE_MEMBERSHIP`, `IP_DROP_SOURCE_MEMBERSHIP`.

</div>

<div class="versionchanged">

3.13 Added `SO_BINDTOIFINDEX`. On Linux this constant can be used in the same way that `SO_BINDTODEVICE` is used, but with the index of a network interface instead of its name.

</div>

<div class="versionchanged">

3.14 Added missing `IP_FREEBIND`, `IP_RECVERR`, `IPV6_RECVERR`, `IP_RECVTTL`, and `IP_RECVORIGDSTADDR` on Linux.

</div>

<div class="versionchanged">

3.14 Added support for `TCP_QUICKACK` on Windows platforms when available.

</div>

<div class="versionchanged">

3.15 `IPV6_HDRINCL` was added. Added support for `SO_PASSRIGHTS` on Linux platforms when available.

</div>

</div>

</div>

<div class="data">

AF_CAN PF_CAN [SOL_CAN]()\* [CAN]()\*

Many constants of these forms, documented in the Linux documentation, are also defined in the socket module.

<div class="availability">

Linux \>= 2.6.25, NetBSD \>= 8.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.11 NetBSD support was added.

</div>

<div class="versionchanged">

3.14 Restored missing `CAN_RAW_ERR_FILTER` on Linux.

</div>

</div>

<div class="data">

CAN_BCM [CAN_BCM]()\*

CAN_BCM, in the CAN protocol family, is the broadcast manager (BCM) protocol. Broadcast manager constants, documented in the Linux documentation, are also defined in the socket module.

<div class="availability">

Linux \>= 2.6.25.

</div>

> [!NOTE]
> The `CAN_BCM_CAN_FD_FRAME` flag is only available on Linux \>= 4.8.

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

CAN_RAW_FD_FRAMES

Enables CAN FD support in a CAN_RAW socket. This is disabled by default. This allows your application to send both CAN and CAN FD frames; however, you must accept both CAN and CAN FD frames when reading from the socket.

This constant is documented in the Linux documentation.

<div class="availability">

Linux \>= 3.6.

</div>

<div class="versionadded">

3.5

</div>

</div>

<div class="data">

CAN_RAW_JOIN_FILTERS

Joins the applied CAN filters such that only CAN frames that match all given CAN filters are passed to user space.

This constant is documented in the Linux documentation.

<div class="availability">

Linux \>= 4.1.

</div>

<div class="versionadded">

3.9

</div>

</div>

<div class="data">

CAN_ISOTP

CAN_ISOTP, in the CAN protocol family, is the ISO-TP (ISO 15765-2) protocol. ISO-TP constants, documented in the Linux documentation.

<div class="availability">

Linux \>= 2.6.25.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

CAN_J1939

CAN_J1939, in the CAN protocol family, is the SAE J1939 protocol. J1939 constants, documented in the Linux documentation.

<div class="availability">

Linux \>= 5.4.

</div>

<div class="versionadded">

3.9

</div>

</div>

<div class="data">

AF_DIVERT PF_DIVERT

These two constants, documented in the FreeBSD divert(4) manual page, are also defined in the socket module.

<div class="availability">

FreeBSD \>= 14.0.

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="data">

AF_PACKET PF_PACKET [PACKET]()\*

Many constants of these forms, documented in the Linux documentation, are also defined in the socket module.

<div class="availability">

Linux \>= 2.2.

</div>

</div>

<div class="data">

ETH_P_ALL

`!ETH_P_ALL` can be used in the `~socket.socket` constructor as *proto* for the `AF_PACKET` family in order to capture every packet, regardless of protocol.

For more information, see the `packet(7)` manpage.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="data">

AF_RDS PF_RDS SOL_RDS [RDS]()\*

Many constants of these forms, documented in the Linux documentation, are also defined in the socket module.

<div class="availability">

Linux \>= 2.6.30.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

SIO_RCVALL SIO_KEEPALIVE_VALS SIO_LOOPBACK_FAST_PATH [RCVALL]()\*

Constants for Windows' WSAIoctl(). The constants are used as arguments to the `~socket.socket.ioctl` method of socket objects.

<div class="versionchanged">

3.6 `SIO_LOOPBACK_FAST_PATH` was added.

</div>

</div>

<div class="data">

[TIPC]()\*

TIPC related constants, matching the ones exported by the C socket API. See the TIPC documentation for more information.

</div>

<div class="data">

AF_ALG SOL_ALG [ALG]()\*

Constants for Linux Kernel cryptography.

<div class="availability">

Linux \>= 2.6.38.

</div>

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

AF_VSOCK IOCTL_VM_SOCKETS_GET_LOCAL_CID VMADDR\* SO_VM\*

Constants for Linux host/guest communication.

<div class="availability">

Linux \>= 4.8.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

AF_LINK

<div class="availability">

BSD, macOS.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

has_ipv6

This constant contains a boolean value which indicates if IPv6 is supported on this platform.

</div>

<div class="data">

AF_BLUETOOTH BTPROTO_L2CAP BTPROTO_RFCOMM BTPROTO_HCI BTPROTO_SCO

Integer constants for use with Bluetooth addresses.

</div>

<div class="data">

BDADDR_ANY BDADDR_LOCAL

These are string constants containing Bluetooth addresses with special meanings. For example, `BDADDR_ANY` can be used to indicate any address when specifying the binding socket with `BTPROTO_RFCOMM`.

</div>

<div class="data">

BDADDR_BREDR BDADDR_LE_PUBLIC BDADDR_LE_RANDOM

These constants describe the Bluetooth address type when binding or connecting a `BTPROTO_L2CAP` socket.

<div class="availability">

Linux, FreeBSD

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

SOL_RFCOMM SOL_L2CAP SOL_HCI SOL_SCO SOL_BLUETOOTH

Used in the level argument to the `~socket.setsockopt` and `~socket.getsockopt` methods of Bluetooth socket objects.

`SOL_BLUETOOTH` is only available on Linux. Other constants are available if the corresponding protocol is supported.

</div>

<div class="data">

[SO_L2CAP]()\* L2CAP_LM [L2CAP_LM]()\* [SO_RFCOMM]()\* [RFCOMM_LM]()\* [SO_SCO]()\* [SO_BTH]()\* [BT]()\*

Used in the option name and value argument to the `~socket.setsockopt` and `~socket.getsockopt` methods of Bluetooth socket objects.

`!BT_*` and `L2CAP_LM` are only available on Linux. `!SO_BTH_*` are only available on Windows. Other constants may be available on Linux and various BSD platforms.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

HCI_FILTER HCI_TIME_STAMP HCI_DATA_DIR SO_HCI_EVT_FILTER SO_HCI_PKT_FILTER

Option names for use with `BTPROTO_HCI`. Availability and format of the option values depend on platform.

<div class="versionchanged">

3.14 Added `!SO_HCI_EVT_FILTER` and `!SO_HCI_PKT_FILTER` on NetBSD and DragonFly BSD. Added `!HCI_DATA_DIR` on FreeBSD, NetBSD and DragonFly BSD.

</div>

</div>

<div class="data">

HCI_DEV_NONE

The `device_id` value used to create an HCI socket that isn't specific to a single Bluetooth adapter.

<div class="availability">

Linux

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

HCI_CHANNEL_RAW HCI_CHANNEL_USER HCI_CHANNEL_MONITOR HCI_CHANNEL_CONTROL HCI_CHANNEL_LOGGING

Possible values for `channel` field in the `BTPROTO_HCI` address.

<div class="availability">

Linux

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

AF_QIPCRTR

Constant for Qualcomm's IPC router protocol, used to communicate with service providing remote processors.

<div class="availability">

Linux \>= 4.7.

</div>

</div>

<div class="data">

SCM_CREDS2 LOCAL_CREDS LOCAL_CREDS_PERSISTENT

LOCAL_CREDS and LOCAL_CREDS_PERSISTENT can be used with SOCK_DGRAM, SOCK_STREAM sockets, equivalent to Linux/DragonFlyBSD SO_PASSCRED, while LOCAL_CREDS sends the credentials at first read, LOCAL_CREDS_PERSISTENT sends for each read, SCM_CREDS2 must be then used for the latter for the message type.

<div class="versionadded">

3.11

</div>

<div class="availability">

FreeBSD.

</div>

</div>

<div class="data">

SO_INCOMING_CPU

Constant to optimize CPU locality, to be used in conjunction with `SO_REUSEPORT`.

<div class="versionadded">

3.11

</div>

<div class="availability">

Linux \>= 3.9

</div>

</div>

<div class="data">

SO_REUSEPORT_LB

Constant to enable duplicate address and port bindings with load balancing.

<div class="versionadded">

3.14

</div>

<div class="availability">

FreeBSD \>= 12.0

</div>

</div>

<div class="data">

AF_HYPERV HV_PROTOCOL_RAW HVSOCKET_CONNECT_TIMEOUT HVSOCKET_CONNECT_TIMEOUT_MAX HVSOCKET_CONNECTED_SUSPEND HVSOCKET_ADDRESS_FLAG_PASSTHRU HV_GUID_ZERO HV_GUID_WILDCARD HV_GUID_BROADCAST HV_GUID_CHILDREN HV_GUID_LOOPBACK HV_GUID_PARENT

Constants for Windows Hyper-V sockets for host/guest communications.

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.12

</div>

</div>

<div id="socket-ethernet-types">

<div class="data">

ETHERTYPE_ARP ETHERTYPE_IP ETHERTYPE_IPV6 ETHERTYPE_VLAN

[IEEE 802.3 protocol number](https://www.iana.org/assignments/ieee-802-numbers/ieee-802-numbers.txt). constants.

<div class="availability">

Linux, FreeBSD, macOS.

</div>

<div class="versionadded">

3.12

</div>

</div>

</div>

<div class="data">

SHUT_RD SHUT_WR SHUT_RDWR

These constants are used by the `~socket.socket.shutdown` method of socket objects.

<div class="availability">

not WASI.

</div>

</div>

### Functions

#### Creating sockets

The following functions all create `socket objects <socket-objects>`.

The `socket <socket.socket>` class constructor creates a new socket directly; see `socket-objects` for its parameters and full description.

<div class="function">

socketpair(\[family\[, type\[, proto\]\]\])

Build a pair of connected socket objects using the given address family, socket type, and protocol number. Address family, socket type, and protocol number are as for the `~socket.socket` function. The default family is `AF_UNIX` if defined on the platform; otherwise, the default is `AF_INET`.

The newly created sockets are `non-inheritable <fd_inheritance>`.

<div class="versionchanged">

3.2 The returned socket objects now support the whole socket API, rather than a subset.

</div>

<div class="versionchanged">

3.4 The returned sockets are now non-inheritable.

</div>

<div class="versionchanged">

3.5 Windows support added.

</div>

</div>

<div class="function">

create_connection(address, timeout=GLOBAL_DEFAULT, source_address=None, \*, all_errors=False)

Connect to a TCP service listening on the internet *address* (a 2-tuple `(host, port)`), and return the socket object. This is a higher-level function than `socket.connect`: if *host* is a non-numeric hostname, it will try to resolve it for both `AF_INET` and `AF_INET6`, and then try to connect to all possible addresses in turn until a connection succeeds. This makes it easy to write clients that are compatible to both IPv4 and IPv6.

Passing the optional *timeout* parameter will set the timeout on the socket instance before attempting to connect. If no *timeout* is supplied, the global default timeout setting returned by `getdefaulttimeout` is used.

If supplied, *source_address* must be a 2-tuple `(host, port)` for the socket to bind to as its source address before connecting. If host or port are '' or 0 respectively the OS default behavior will be used.

When a connection cannot be created, an exception is raised. By default, it is the exception from the last address in the list. If *all_errors* is `True`, it is an `ExceptionGroup` containing the errors of all attempts.

<div class="versionchanged">

3.2 *source_address* was added.

</div>

<div class="versionchanged">

3.11 *all_errors* was added.

</div>

</div>

<div class="function">

create_server(address, \*, family=AF_INET, backlog=None, reuse_port=False, dualstack_ipv6=False)

Convenience function which creates a TCP socket bound to *address* (a 2-tuple `(host, port)`) and returns the socket object.

*family* should be either `AF_INET` or `AF_INET6`. *backlog* is the queue size passed to `socket.listen`; if not specified , a default reasonable value is chosen. *reuse_port* dictates whether to set the `SO_REUSEPORT` socket option.

If *dualstack_ipv6* is true, *family* is `AF_INET6` and the platform supports it the socket will be able to accept both IPv4 and IPv6 connections, else it will raise `ValueError`. Most POSIX platforms and Windows are supposed to support this functionality. When this functionality is enabled the address returned by `socket.getpeername` when an IPv4 connection occurs will be an IPv6 address represented as an IPv4-mapped IPv6 address. If *dualstack_ipv6* is false it will explicitly disable this functionality on platforms that enable it by default (e.g. Linux). This parameter can be used in conjunction with `has_dualstack_ipv6`:

    import socket

    addr = ("", 8080)  # all interfaces, port 8080
    if socket.has_dualstack_ipv6():
        s = socket.create_server(addr, family=socket.AF_INET6, dualstack_ipv6=True)
    else:
        s = socket.create_server(addr)

> [!NOTE]
> On POSIX platforms the `SO_REUSEADDR` socket option is set in order to immediately reuse previous sockets which were bound on the same *address* and remained in TIME_WAIT state.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

has_dualstack_ipv6()

Return `True` if the platform supports creating a TCP socket which can handle both IPv4 and IPv6 connections.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

fromfd(fd, family, type, proto=0)

Duplicate the file descriptor *fd* (an integer as returned by a file object's `~io.IOBase.fileno` method) and build a socket object from the result. Address family, socket type and protocol number are as for the `~socket.socket` function. The file descriptor should refer to a socket, but this is not checked --- subsequent operations on the object may fail if the file descriptor is invalid. This function is rarely needed, but can be used to get or set socket options on a socket passed to a program as standard input or output (such as a server started by the Unix inet daemon). The socket is assumed to be in blocking mode.

The newly created socket is `non-inheritable <fd_inheritance>`.

<div class="versionchanged">

3.4 The returned socket is now non-inheritable.

</div>

</div>

<div class="function">

fromshare(data)

Instantiate a socket from data obtained from the `socket.share` method. The socket is assumed to be in blocking mode.

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.3

</div>

</div>

#### Other functions

The `!socket` module also offers various network-related services:

<div class="function">

close(fd)

Close a socket file descriptor. This is like `os.close`, but for sockets. On some platforms (most notably Windows) `os.close` does not work for socket file descriptors.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

getaddrinfo(host, port, family=AF_UNSPEC, type=0, proto=0, flags=0)

This function wraps the C function `getaddrinfo` of the underlying system.

Translate the *host*/\*port\* argument into a sequence of 5-tuples that contain all the necessary arguments for creating a socket connected to that service. *host* is a domain name, a string representation of an IPv4/v6 address or `None`. *port* is a string service name such as `'http'`, a numeric port number or `None`. By passing `None` as the value of *host* and *port*, you can pass `NULL` to the underlying C API.

The *family*, *type* and *proto* arguments can be optionally specified in order to provide options and limit the list of addresses returned. Pass their default values (`AF_UNSPEC`, 0, and 0, respectively) to not limit the results. See the note below for details.

The *flags* argument can be one or several of the `AI_*` constants, and will influence how results are computed and returned. For example, `AI_NUMERICHOST` will disable domain name resolution and will raise an error if *host* is a domain name.

The function returns a list of 5-tuples with the following structure:

`(family, type, proto, canonname, sockaddr)`

In these tuples, *family*, *type*, *proto* are all integers and are meant to be passed to the `~socket.socket` function. *canonname* will be a string representing the canonical name of the *host* if `AI_CANONNAME` is part of the *flags* argument; else *canonname* will be empty. *sockaddr* is a tuple describing a socket address, whose format depends on the returned *family* and flags Python was compiled with, and is meant to be passed to the `socket.connect` method.

*sockaddr* can be one of the following:

- a `(address, port)` 2-tuple for `AF_INET`
- a `(address, port, flowinfo, scope_id)` 4-tuple for `AF_INET6` if Python was compiled with `--enable-ipv6` (the default)
- a 2-tuple containing raw data for `AF_INET6` if Python was compiled with `--disable-ipv6`

> [!NOTE]
> If you intend to use results from `!getaddrinfo` to create a socket (rather than, for example, retrieve *canonname*), consider limiting the results by *type* (e.g. `SOCK_STREAM` or `SOCK_DGRAM`) and/or *proto* (e.g. `IPPROTO_TCP` or `IPPROTO_UDP`) that your application can handle.
>
> The behavior with default values of *family*, *type*, *proto* and *flags* is system-specific.
>
> Many systems (for example, most Linux configurations) will return a sorted list of all matching addresses. These addresses should generally be tried in order until a connection succeeds (possibly tried in parallel, for example, using a [Happy Eyeballs](https://en.wikipedia.org/wiki/Happy_Eyeballs) algorithm). In these cases, limiting the *type* and/or *proto* can help eliminate unsuccessful or unusable connection attempts.
>
> Some systems will, however, only return a single address. (For example, this was reported on Solaris and AIX configurations.) On these systems, limiting the *type* and/or *proto* helps ensure that this address is usable.

<div class="audit-event">

socket.getaddrinfo host,port,family,type,protocol socket.getaddrinfo

</div>

The following example fetches address information for a hypothetical TCP connection to `example.org` on port 80 (results may differ on your system if IPv6 isn't enabled):

    >>> socket.getaddrinfo("example.org", 80, proto=socket.IPPROTO_TCP)
    [(socket.AF_INET6, socket.SOCK_STREAM,
     6, '', ('2606:2800:220:1:248:1893:25c8:1946', 80, 0, 0)),
     (socket.AF_INET, socket.SOCK_STREAM,
     6, '', ('93.184.216.34', 80))]

<div class="versionchanged">

3.2 parameters can now be passed using keyword arguments.

</div>

<div class="versionchanged">

3.7 for IPv6 multicast addresses, string representing an address will not contain `%scope_id` part.

</div>

<div class="versionchanged">

next Null characters are now rejected in the *host* and *port* parameters.

</div>

</div>

<div class="function">

getfqdn(\[name\])

Return a fully qualified domain name for *name*. If *name* is omitted or empty, it is interpreted as the local host. To find the fully qualified name, the hostname returned by `gethostbyaddr` is checked, followed by aliases for the host, if available. The first name which includes a period is selected. In case no fully qualified domain name is available and *name* was provided, it is returned unchanged. If *name* was empty or equal to `'0.0.0.0'`, the hostname from `gethostname` is returned.

</div>

<div class="function">

gethostbyname(hostname)

Translate a host name to IPv4 address format. The IPv4 address is returned as a string, such as `'100.50.200.5'`. If the host name is an IPv4 address itself it is returned unchanged. See `gethostbyname_ex` for a more complete interface. `gethostbyname` does not support IPv6 name resolution, and `getaddrinfo` should be used instead for IPv4/v6 dual stack support.

<div class="audit-event">

socket.gethostbyname hostname socket.gethostbyname

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="function">

gethostbyname_ex(hostname)

Translate a host name to IPv4 address format, extended interface. Return a 3-tuple `(hostname, aliaslist, ipaddrlist)` where *hostname* is the host's primary host name, *aliaslist* is a (possibly empty) list of alternative host names for the same address, and *ipaddrlist* is a list of IPv4 addresses for the same interface on the same host (often but not always a single address). `gethostbyname_ex` does not support IPv6 name resolution, and `getaddrinfo` should be used instead for IPv4/v6 dual stack support.

<div class="audit-event">

socket.gethostbyname hostname socket.gethostbyname_ex

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="function">

gethostname()

Return a string containing the hostname of the machine where the Python interpreter is currently executing.

<div class="audit-event">

socket.gethostname "" socket.gethostname

</div>

Note: `gethostname` doesn't always return the fully qualified domain name; use `getfqdn` for that.

<div class="availability">

not WASI.

</div>

</div>

<div class="function">

gethostbyaddr(ip_address)

Return a 3-tuple `(hostname, aliaslist, ipaddrlist)` where *hostname* is the primary host name responding to the given *ip_address*, *aliaslist* is a (possibly empty) list of alternative host names for the same address, and *ipaddrlist* is a list of IPv4/v6 addresses for the same interface on the same host (most likely containing only a single address). To find the fully qualified domain name, use the function `getfqdn`. `gethostbyaddr` supports both IPv4 and IPv6.

<div class="audit-event">

socket.gethostbyaddr ip_address socket.gethostbyaddr

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="function">

getnameinfo(sockaddr, flags)

Translate a socket address *sockaddr* into a 2-tuple `(host, port)`. Depending on the settings of *flags*, the result can contain a fully qualified domain name or numeric address representation in *host*. Similarly, *port* can contain a string port name or a numeric port number.

For IPv6 addresses, `%scope_id` is appended to the host part if *sockaddr* contains meaningful *scope_id*. Usually this happens for multicast addresses.

For more information about *flags* you can consult `getnameinfo(3)`.

<div class="audit-event">

socket.getnameinfo sockaddr socket.getnameinfo

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="function">

getprotobyname(protocolname)

Translate an internet protocol name (for example, `'icmp'`) to a constant suitable for passing as the (optional) third argument to the `~socket.socket` function. This is usually only needed for sockets opened in "raw" mode (`SOCK_RAW`); for the normal socket modes, the correct protocol is chosen automatically if the protocol is omitted or zero.

<div class="availability">

not WASI.

</div>

</div>

<div class="function">

getservbyname(servicename\[, protocolname\])

Translate an internet service name and protocol name to a port number for that service. The optional protocol name, if given, should be `'tcp'` or `'udp'`, otherwise any protocol will match.

<div class="audit-event">

socket.getservbyname servicename,protocolname socket.getservbyname

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="function">

getservbyport(port\[, protocolname\])

Translate an internet port number and protocol name to a service name for that service. The optional protocol name, if given, should be `'tcp'` or `'udp'`, otherwise any protocol will match.

<div class="audit-event">

socket.getservbyport port,protocolname socket.getservbyport

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="function">

ntohl(x)

Convert 32-bit positive integers from network to host byte order. On machines where the host byte order is the same as network byte order, this is a no-op; otherwise, it performs a 4-byte swap operation.

</div>

<div class="function">

ntohs(x)

Convert 16-bit positive integers from network to host byte order. On machines where the host byte order is the same as network byte order, this is a no-op; otherwise, it performs a 2-byte swap operation.

<div class="versionchanged">

3.10 Raises `OverflowError` if *x* does not fit in a 16-bit unsigned integer.

</div>

</div>

<div class="function">

htonl(x)

Convert 32-bit positive integers from host to network byte order. On machines where the host byte order is the same as network byte order, this is a no-op; otherwise, it performs a 4-byte swap operation.

</div>

<div class="function">

htons(x)

Convert 16-bit positive integers from host to network byte order. On machines where the host byte order is the same as network byte order, this is a no-op; otherwise, it performs a 2-byte swap operation.

<div class="versionchanged">

3.10 Raises `OverflowError` if *x* does not fit in a 16-bit unsigned integer.

</div>

</div>

<div class="function">

inet_aton(ip_string)

Convert an IPv4 address from dotted-quad string format (for example, '123.45.67.89') to 32-bit packed binary format, as a bytes object four characters in length. This is useful when conversing with a program that uses the standard C library and needs objects of type `in_addr`, which is the C type for the 32-bit packed binary this function returns.

`inet_aton` also accepts strings with less than three dots; see the Unix manual page `inet(3)` for details.

If the IPv4 address string passed to this function is invalid, `OSError` will be raised. Note that exactly what is valid depends on the underlying C implementation of `inet_aton`.

`inet_aton` does not support IPv6, and `inet_pton` should be used instead for IPv4/v6 dual stack support.

</div>

<div class="function">

inet_ntoa(packed_ip)

Convert a 32-bit packed IPv4 address (a `bytes-like object` four bytes in length) to its standard dotted-quad string representation (for example, '123.45.67.89'). This is useful when conversing with a program that uses the standard C library and needs objects of type `in_addr`, which is the C type for the 32-bit packed binary data this function takes as an argument.

If the byte sequence passed to this function is not exactly 4 bytes in length, `OSError` will be raised. `inet_ntoa` does not support IPv6, and `inet_ntop` should be used instead for IPv4/v6 dual stack support.

<div class="versionchanged">

3.5 Writable `bytes-like object` is now accepted.

</div>

</div>

<div class="function">

inet_pton(address_family, ip_string)

Convert an IP address from its family-specific string format to a packed, binary format. `inet_pton` is useful when a library or network protocol calls for an object of type `in_addr` (similar to `inet_aton`) or `in6_addr`.

Supported values for *address_family* are currently `AF_INET` and `AF_INET6`. If the IP address string *ip_string* is invalid, `OSError` will be raised. Note that exactly what is valid depends on both the value of *address_family* and the underlying implementation of `inet_pton`.

<div class="availability">

Unix, Windows.

</div>

<div class="versionchanged">

3.4 Windows support added

</div>

</div>

<div class="function">

inet_ntop(address_family, packed_ip)

Convert a packed IP address (a `bytes-like object` of some number of bytes) to its standard, family-specific string representation (for example, `'7.10.0.5'` or `'5aef:2b::8'`). `inet_ntop` is useful when a library or network protocol returns an object of type `in_addr` (similar to `inet_ntoa`) or `in6_addr`.

Supported values for *address_family* are currently `AF_INET` and `AF_INET6`. If the bytes object *packed_ip* is not the correct length for the specified address family, `ValueError` will be raised. `OSError` is raised for errors from the call to `inet_ntop`.

<div class="availability">

Unix, Windows.

</div>

<div class="versionchanged">

3.4 Windows support added

</div>

<div class="versionchanged">

3.5 Writable `bytes-like object` is now accepted.

</div>

</div>

<div class="function">

CMSG_LEN(length)

Return the total length, without trailing padding, of an ancillary data item with associated data of the given *length*. This value can often be used as the buffer size for `~socket.recvmsg` to receive a single item of ancillary data, but `3542` requires portable applications to use `CMSG_SPACE` and thus include space for padding, even when the item will be the last in the buffer. Raises `OverflowError` if *length* is outside the permissible range of values.

<div class="availability">

Unix, not WASI.

Most Unix platforms.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

CMSG_SPACE(length)

Return the buffer size needed for `~socket.recvmsg` to receive an ancillary data item with associated data of the given *length*, along with any trailing padding. The buffer space needed to receive multiple items is the sum of the `CMSG_SPACE` values for their associated data lengths. Raises `OverflowError` if *length* is outside the permissible range of values.

Note that some systems might support ancillary data without providing this function. Also note that setting the buffer size using the results of this function may not precisely limit the amount of ancillary data that can be received, since additional data may be able to fit into the padding area.

<div class="availability">

Unix, not WASI.

most Unix platforms.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

getdefaulttimeout()

Return the default timeout in seconds (float) for new socket objects. A value of `None` indicates that new socket objects have no timeout. When the socket module is first imported, the default is `None`.

</div>

<div class="function">

setdefaulttimeout(timeout)

Set the default timeout in seconds (real number) for new socket objects. When the socket module is first imported, the default is `None`. See `~socket.settimeout` for possible values and their respective meanings.

<div class="versionchanged">

3.15 Accepts any real number, not only integer or float.

</div>

</div>

<div class="function">

sethostname(name)

Set the machine's hostname to *name*. This will raise an `OSError` if you don't have enough rights.

<div class="audit-event">

socket.sethostname name socket.sethostname

</div>

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.16 Support for Android now exists.

</div>

</div>

<div class="function">

if_nameindex()

Return a list of network interface information (index int, name string) tuples. `OSError` if the system call fails.

<div class="availability">

Unix, Windows, not WASI.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.8 Windows support was added.

</div>

> [!NOTE]
> On Windows network interfaces have different names in different contexts (all names are examples):
>
> - UUID: `{FB605B73-AAC2-49A6-9A2F-25416AEA0573}`
> - name: `ethernet_32770`
> - friendly name: `vEthernet (nat)`
> - description: `Hyper-V Virtual Ethernet Adapter`
>
> This function returns names of the second form from the list, `ethernet_32770` in this example case.

</div>

<div class="function">

if_nametoindex(if_name)

Return a network interface index number corresponding to an interface name. `OSError` if no interface with the given name exists.

<div class="availability">

Unix, Windows, not WASI.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.8 Windows support was added.

</div>

<div class="seealso">

"Interface name" is a name as documented in `if_nameindex`.

</div>

</div>

<div class="function">

if_indextoname(if_index)

Return a network interface name corresponding to an interface index number. `OSError` if no interface with the given index exists.

<div class="availability">

Unix, Windows, not WASI.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.8 Windows support was added.

</div>

<div class="seealso">

"Interface name" is a name as documented in `if_nameindex`.

</div>

</div>

<div class="function">

send_fds(sock, buffers, fds\[, flags\[, address\]\])

Send the list of file descriptors *fds* over an `AF_UNIX` socket *sock*. The *fds* parameter is a sequence of file descriptors. Consult `~socket.sendmsg` for the documentation of these parameters.

<div class="availability">

Unix, not WASI.

Unix platforms supporting `~socket.sendmsg` and `SCM_RIGHTS` mechanism.

</div>

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

recv_fds(sock, bufsize, maxfds\[, flags\])

Receive up to *maxfds* file descriptors from an `AF_UNIX` socket *sock*. Return `(msg, list(fds), flags, addr)`. Consult `~socket.recvmsg` for the documentation of these parameters.

<div class="availability">

Unix, not WASI.

Unix platforms supporting `~socket.recvmsg` and `SCM_RIGHTS` mechanism.

</div>

<div class="versionadded">

3.9

</div>

> [!NOTE]
> Any truncated integers at the end of the list of file descriptors.

</div>

## Socket Objects

<div class="socket(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None)">

Create a new socket using the given address family, socket type and protocol number. The address family should be `AF_INET` (the default), `AF_INET6`, `AF_UNIX`, `AF_CAN`, `AF_PACKET`, or `AF_RDS`. The socket type should be `SOCK_STREAM` (the default), `SOCK_DGRAM`, `SOCK_RAW` or perhaps one of the other `SOCK_` constants. The protocol number is usually zero and may be omitted or in the case where the address family is `AF_CAN` the protocol should be one of `CAN_RAW`, `CAN_BCM`, `CAN_ISOTP` or `CAN_J1939`.

If *fileno* is specified, the values for *family*, *type*, and *proto* are auto-detected from the specified file descriptor. Auto-detection can be overruled by calling the function with explicit *family*, *type*, or *proto* arguments. This only affects how Python represents e.g. the return value of `socket.getpeername` but not the actual OS resource. Unlike `socket.fromfd`, *fileno* will return the same socket and not a duplicate. This may help close a detached socket using `~socket.socket.close`.

The newly created socket is `non-inheritable <fd_inheritance>`.

<div class="audit-event">

socket.\_\_new\_\_ self,family,type,protocol socket.socket

</div>

<div class="versionchanged">

3.3 The AF_CAN family was added. The AF_RDS family was added.

</div>

<div class="versionchanged">

3.4 The CAN_BCM protocol was added.

</div>

<div class="versionchanged">

3.4 The returned socket is now non-inheritable.

</div>

<div class="versionchanged">

3.7 The CAN_ISOTP protocol was added.

</div>

<div class="versionchanged">

3.7 When `SOCK_NONBLOCK` or `SOCK_CLOEXEC` bit flags are applied to *type* they are cleared, and `socket.type` will not reflect them. They are still passed to the underlying system `socket()` call. Therefore,

    sock = socket.socket(
        socket.AF_INET,
        socket.SOCK_STREAM | socket.SOCK_NONBLOCK)

will still create a non-blocking socket on OSes that support `SOCK_NONBLOCK`, but `sock.type` will be set to `socket.SOCK_STREAM`.

</div>

<div class="versionchanged">

3.9 The CAN_J1939 protocol was added.

</div>

<div class="versionchanged">

3.10 The IPPROTO_MPTCP protocol was added.

</div>

Socket objects have the following methods. Except for `~socket.makefile`, these correspond to Unix system calls applicable to sockets.

<div class="versionchanged">

3.2 Support for the `context manager` protocol was added. Exiting the context manager is equivalent to calling `~socket.socket.close`.

</div>

<div class="method">

accept()

Accept a connection. The socket must be bound to an address and listening for connections. The return value is a pair `(conn, address)` where *conn* is a *new* socket object usable to send and receive data on the connection, and *address* is the address bound to the socket on the other end of the connection.

The newly created socket is `non-inheritable <fd_inheritance>`.

<div class="versionchanged">

3.4 The socket is now non-inheritable.

</div>

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="method">

bind(address)

Bind the socket to *address*. The socket must not already be bound. The format of *address* depends on the address family --- see `socket-addresses`.

<div class="audit-event">

socket.bind self,address socket.socket.bind

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="method">

close()

Mark the socket closed. The underlying system resource (e.g. a file descriptor) is also closed when all file objects from `makefile` are closed. Once that happens, all future operations on the socket object will fail. The remote end will receive no more data (after queued data is flushed).

Sockets are automatically closed when they are garbage-collected, but it is recommended to `close` them explicitly, or to use a `with` statement around them.

<div class="versionchanged">

3.6 `OSError` is now raised if an error occurs when the underlying `!close` call is made.

</div>

> [!NOTE]
> `close` releases the resource associated with a connection but does not necessarily close the connection immediately. If you want to close the connection in a timely fashion, call `shutdown` before `close`.

</div>

<div class="method">

connect(address)

Connect to a remote socket at *address*. The format of *address* depends on the address family --- see `socket-addresses`.

If the connection is interrupted by a signal, the method waits until the connection completes, or raises a `TimeoutError` on timeout, if the signal handler doesn't raise an exception and the socket is blocking or has a timeout. For non-blocking sockets, the method raises an `InterruptedError` exception if the connection is interrupted by a signal (or the exception raised by the signal handler).

<div class="audit-event">

socket.connect self,address socket.socket.connect

</div>

<div class="versionchanged">

3.5 The method now waits until the connection completes instead of raising an `InterruptedError` exception if the connection is interrupted by a signal, the signal handler doesn't raise an exception and the socket is blocking or has a timeout (see the `475` for the rationale).

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="method">

connect_ex(address)

Like `connect(address)`, but return an error indicator instead of raising an exception for errors returned by the C-level `!connect` call (other problems, such as "host not found," can still raise exceptions). The error indicator is `0` if the operation succeeded, otherwise the value of the `errno` variable. This is useful to support, for example, asynchronous connects.

<div class="audit-event">

socket.connect self,address socket.socket.connect_ex

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="method">

detach()

Put the socket object into closed state without actually closing the underlying file descriptor. The file descriptor is returned, and can be reused for other purposes.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

dup()

Duplicate the socket.

The newly created socket is `non-inheritable <fd_inheritance>`.

<div class="versionchanged">

3.4 The socket is now non-inheritable.

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="method">

fileno()

Return the socket's file descriptor (a small integer), or -1 on failure. This is useful with `select.select`.

Under Windows the small integer returned by this method cannot be used where a file descriptor can be used (such as `os.fdopen`). Unix does not have this limitation.

</div>

<div class="method">

get_inheritable()

Get the `inheritable flag <fd_inheritance>` of the socket's file descriptor or socket's handle: `True` if the socket can be inherited in child processes, `False` if it cannot.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

getpeername()

Return the remote address to which the socket is connected. This is useful to find out the port number of a remote IPv4/v6 socket, for instance. The format of the address returned depends on the address family --- see `socket-addresses`. On some systems this function is not supported.

</div>

<div class="method">

getsockname()

Return the socket's own address. This is useful to find out the port number of an IPv4/v6 socket, for instance. The format of the address returned depends on the address family --- see `socket-addresses`.

</div>

<div class="method">

getsockopt(level, optname\[, buflen\])

Return the value of the given socket option (see the Unix man page `getsockopt(2)`). The needed symbolic constants (`SO_\* etc. <socket-unix-constants>`) are defined in this module. If *buflen* is absent, an integer option is assumed and its integer value is returned by the function. If *buflen* is present, it specifies the maximum length of the buffer used to receive the option in, and this buffer is returned as a bytes object. It is up to the caller to decode the contents of the buffer (see the optional built-in module `struct` for a way to decode C structures encoded as byte strings).

<div class="availability">

not WASI.

</div>

</div>

<div class="method">

getblocking()

Return `True` if socket is in blocking mode, `False` if in non-blocking.

This is equivalent to checking `socket.gettimeout() != 0`.

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

gettimeout()

Return the timeout in seconds (float) associated with socket operations, or `None` if no timeout is set. This reflects the last call to `setblocking` or `settimeout`.

</div>

<div class="method">

ioctl(control, option)

The `ioctl` method is a limited interface to the WSAIoctl system interface. Please refer to the [Win32 documentation](https://msdn.microsoft.com/en-us/library/ms741621%28VS.85%29.aspx) for more information.

On other platforms, the generic `fcntl.fcntl` and `fcntl.ioctl` functions may be used; they accept a socket object as their first argument.

Currently only the following control codes are supported: `SIO_RCVALL`, `SIO_KEEPALIVE_VALS`, and `SIO_LOOPBACK_FAST_PATH`.

<div class="availability">

Windows

</div>

<div class="versionchanged">

3.6 `SIO_LOOPBACK_FAST_PATH` was added.

</div>

</div>

<div class="method">

listen(\[backlog\])

Enable a server to accept connections. If *backlog* is specified, it must be at least 0 (if it is lower, it is set to 0); it specifies the number of unaccepted connections that the system will allow before refusing new connections. If not specified, a default reasonable value is chosen.

<div class="availability">

not WASI.

</div>

<div class="versionchanged">

3.5 The *backlog* parameter is now optional.

</div>

</div>

<div class="method">

makefile(mode='r', buffering=None, \*, encoding=None, errors=None, newline=None)

<div class="index">

single: I/O control; buffering

</div>

Return a `file object` associated with the socket. The exact returned type depends on the arguments given to `makefile`. These arguments are interpreted the same way as by the built-in `open` function, except the only supported *mode* values are `'r'` (default), `'w'`, `'b'`, or a combination of those.

The socket must be in blocking mode; it can have a timeout, but the file object's internal buffer may end up in an inconsistent state if a timeout occurs.

Closing the file object returned by `makefile` won't close the original socket unless all other file objects have been closed and `~socket.socket.close` has been called on the socket object.

> [!NOTE]
> On Windows, the file-like object created by `makefile` cannot be used where a file object with a file descriptor is expected, such as the stream arguments of `subprocess.Popen`.

</div>

<div class="method">

recv(bufsize\[, flags\])

Receive data from the socket. The return value is a bytes object representing the data received. The maximum amount of data to be received at once is specified by *bufsize*. A returned empty bytes object indicates that the client has disconnected. See the Unix manual page `recv(2)` for the meaning of the optional argument *flags*; it defaults to zero.

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="method">

recvfrom(bufsize\[, flags\])

Receive data from the socket. The return value is a pair `(bytes, address)` where *bytes* is a bytes object representing the data received and *address* is the address of the socket sending the data. See the Unix manual page `recv(2)` for the meaning of the optional argument *flags*; it defaults to zero. The format of *address* depends on the address family --- see `socket-addresses`.

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

<div class="versionchanged">

3.7 For multicast IPv6 address, first item of *address* does not contain `%scope_id` part anymore. In order to get full IPv6 address use `getnameinfo`.

</div>

</div>

<div class="method">

recvmsg(bufsize\[, ancbufsize\[, flags\]\])

Receive normal data (up to *bufsize* bytes) and ancillary data from the socket. The *ancbufsize* argument sets the size in bytes of the internal buffer used to receive the ancillary data; it defaults to 0, meaning that no ancillary data will be received. Appropriate buffer sizes for ancillary data can be calculated using `CMSG_SPACE` or `CMSG_LEN`, and items which do not fit into the buffer might be truncated or discarded. The *flags* argument defaults to 0 and has the same meaning as for `recv`.

The return value is a 4-tuple: `(data, ancdata, msg_flags, address)`. The *data* item is a `bytes` object holding the non-ancillary data received. The *ancdata* item is a list of zero or more tuples `(cmsg_level, cmsg_type, cmsg_data)` representing the ancillary data (control messages) received: *cmsg_level* and *cmsg_type* are integers specifying the protocol level and protocol-specific type respectively, and *cmsg_data* is a `bytes` object holding the associated data. The *msg_flags* item is the bitwise OR of various flags indicating conditions on the received message; see your system documentation for details. If the receiving socket is unconnected, *address* is the address of the sending socket, if available; otherwise, its value is unspecified.

On some systems, `sendmsg` and `recvmsg` can be used to pass file descriptors between processes over an `AF_UNIX` socket. When this facility is used (it is often restricted to `SOCK_STREAM` sockets), `recvmsg` will return, in its ancillary data, items of the form `(socket.SOL_SOCKET, socket.SCM_RIGHTS, fds)`, where *fds* is a `bytes` object representing the new file descriptors as a binary array of the native C `int` type. If `recvmsg` raises an exception after the system call returns, it will first attempt to close any file descriptors received via this mechanism.

Some systems do not indicate the truncated length of ancillary data items which have been only partially received. If an item appears to extend beyond the end of the buffer, `recvmsg` will issue a `RuntimeWarning`, and will return the part of it which is inside the buffer provided it has not been truncated before the start of its associated data.

On systems which support the `!SCM_RIGHTS` mechanism, the following function will receive up to *maxfds* file descriptors, returning the message data and a list containing the descriptors (while ignoring unexpected conditions such as unrelated control messages being received). See also `sendmsg`. :

    import socket, array

    def recv_fds(sock, msglen, maxfds):
        fds = array.array("i")   # Array of ints
        msg, ancdata, flags, addr = sock.recvmsg(msglen, socket.CMSG_LEN(maxfds * fds.itemsize))
        for cmsg_level, cmsg_type, cmsg_data in ancdata:
            if cmsg_level == socket.SOL_SOCKET and cmsg_type == socket.SCM_RIGHTS:
                # Append data, ignoring any truncated integers at the end.
                fds.frombytes(cmsg_data[:len(cmsg_data) - (len(cmsg_data) % fds.itemsize)])
        return msg, list(fds)

<div class="availability">

Unix.

Most Unix platforms.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="method">

recvmsg_into(buffers\[, ancbufsize\[, flags\]\])

Receive normal data and ancillary data from the socket, behaving as `recvmsg` would, but scatter the non-ancillary data into a series of buffers instead of returning a new bytes object. The *buffers* argument must be an iterable of objects that export writable buffers (e.g. `bytearray` objects); these will be filled with successive chunks of the non-ancillary data until it has all been written or there are no more buffers. The operating system may set a limit (`~os.sysconf` value `SC_IOV_MAX`) on the number of buffers that can be used. The *ancbufsize* and *flags* arguments have the same meaning as for `recvmsg`.

The return value is a 4-tuple: `(nbytes, ancdata, msg_flags, address)`, where *nbytes* is the total number of bytes of non-ancillary data written into the buffers, and *ancdata*, *msg_flags* and *address* are the same as for `recvmsg`.

Example:

    >>> import socket
    >>> s1, s2 = socket.socketpair()
    >>> b1 = bytearray(b'----')
    >>> b2 = bytearray(b'0123456789')
    >>> b3 = bytearray(b'--------------')
    >>> s1.send(b'Mary had a little lamb')
    22
    >>> s2.recvmsg_into([b1, memoryview(b2)[2:9], b3])
    (22, [], 0, None)
    >>> [b1, b2, b3]
    [bytearray(b'Mary'), bytearray(b'01 had a 9'), bytearray(b'little lamb---')]

<div class="availability">

Unix.

Most Unix platforms.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

recvfrom_into(buffer\[, nbytes\[, flags\]\])

Receive data from the socket, writing it into *buffer* instead of creating a new bytestring. The return value is a pair `(nbytes, address)` where *nbytes* is the number of bytes received and *address* is the address of the socket sending the data. See the Unix manual page `recv(2)` for the meaning of the optional argument *flags*; it defaults to zero. The format of *address* depends on the address family --- see `socket-addresses`.

</div>

<div class="method">

recv_into(buffer\[, nbytes\[, flags\]\])

Receive up to *nbytes* bytes from the socket, storing the data into a buffer rather than creating a new bytestring. If *nbytes* is not specified (or 0), receive up to the size available in the given buffer. Returns the number of bytes received. See the Unix manual page `recv(2)` for the meaning of the optional argument *flags*; it defaults to zero.

</div>

<div class="method">

send(bytes\[, flags\])

Send data to the socket. The socket must be connected to a remote socket. The optional *flags* argument has the same meaning as for `recv`. Returns the number of bytes sent. Applications are responsible for checking that all data has been sent; if only some of the data was transmitted, the application needs to attempt delivery of the remaining data. For further information on this topic, consult the `socket-howto`.

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="method">

sendall(bytes\[, flags\])

Send data to the socket. The socket must be connected to a remote socket. The optional *flags* argument has the same meaning as for `recv`. Unlike `send`, this method continues to send data from *bytes* until either all data has been sent or an error occurs. `None` is returned on success. On error, an exception is raised, and there is no way to determine how much data, if any, was successfully sent.

<div class="versionchanged">

3.5 The socket timeout is no longer reset each time data is sent successfully. The socket timeout is now the maximum total duration to send all data.

</div>

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="method">

sendto(bytes, address) sendto(bytes, flags, address)

Send data to the socket. The socket should not be connected to a remote socket, since the destination socket is specified by *address*. The optional *flags* argument has the same meaning as for `recv`. Return the number of bytes sent. The format of *address* depends on the address family --- see `socket-addresses`.

<div class="audit-event">

socket.sendto self,address socket.socket.sendto

</div>

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="method">

sendmsg(buffers\[, ancdata\[, flags\[, address\]\]\])

Send normal and ancillary data to the socket, gathering the non-ancillary data from a series of buffers and concatenating it into a single message. The *buffers* argument specifies the non-ancillary data as an iterable of `bytes-like objects <bytes-like object>` (e.g. `bytes` objects); the operating system may set a limit (`~os.sysconf` value `SC_IOV_MAX`) on the number of buffers that can be used. The *ancdata* argument specifies the ancillary data (control messages) as an iterable of zero or more tuples `(cmsg_level, cmsg_type, cmsg_data)`, where *cmsg_level* and *cmsg_type* are integers specifying the protocol level and protocol-specific type respectively, and *cmsg_data* is a bytes-like object holding the associated data. Note that some systems (in particular, systems without `CMSG_SPACE`) might support sending only one control message per call. The *flags* argument defaults to 0 and has the same meaning as for `send`. If *address* is supplied and not `None`, it sets a destination address for the message. The return value is the number of bytes of non-ancillary data sent.

The following function sends the list of file descriptors *fds* over an `AF_UNIX` socket, on systems which support the `!SCM_RIGHTS` mechanism. See also `recvmsg`. :

    import socket, array

    def send_fds(sock, msg, fds):
        return sock.sendmsg([msg], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, array.array("i", fds))])

<div class="availability">

Unix, not WASI.

Most Unix platforms.

</div>

<div class="audit-event">

socket.sendmsg self,address socket.socket.sendmsg

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.5 If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an `InterruptedError` exception (see `475` for the rationale).

</div>

</div>

<div class="method">

sendmsg_afalg(\[msg\], \*, op\[, iv\[, assoclen\[, flags\]\]\])

Specialized version of `~socket.sendmsg` for `AF_ALG` socket. Set mode, IV, AEAD associated data length and flags for `AF_ALG` socket.

<div class="availability">

Linux \>= 2.6.38.

</div>

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

sendfile(file, offset=0, count=None)

Send a file until EOF is reached by using high-performance `os.sendfile` and return the total number of bytes which were sent. *file* must be a regular file object opened in binary mode. If `os.sendfile` is not available (e.g. Windows) or *file* is not a regular file `send` will be used instead. *offset* tells from where to start reading the file. If specified, *count* is the total number of bytes to transmit as opposed to sending the file until EOF is reached. File position is updated on return or also in case of error in which case `file.tell() <io.IOBase.tell>` can be used to figure out the number of bytes which were sent. The socket must be of `SOCK_STREAM` type. Non-blocking sockets are not supported.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

set_inheritable(inheritable)

Set the `inheritable flag <fd_inheritance>` of the socket's file descriptor or socket's handle.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

setblocking(flag)

Set blocking or non-blocking mode of the socket: if *flag* is false, the socket is set to non-blocking, else to blocking mode.

This method is a shorthand for certain `~socket.settimeout` calls:

- `sock.setblocking(True)` is equivalent to `sock.settimeout(None)`
- `sock.setblocking(False)` is equivalent to `sock.settimeout(0.0)`

<div class="versionchanged">

3.7 The method no longer applies `SOCK_NONBLOCK` flag on `socket.type`.

</div>

</div>

<div class="method">

settimeout(value)

Set a timeout on blocking socket operations. The *value* argument can be a nonnegative real number expressing seconds, or `None`. If a non-zero value is given, subsequent socket operations will raise a `timeout` exception if the timeout period *value* has elapsed before the operation has completed. If zero is given, the socket is put in non-blocking mode. If `None` is given, the socket is put in blocking mode.

For further information, please consult the `notes on socket timeouts <socket-timeouts>`.

<div class="versionchanged">

3.7 The method no longer toggles `SOCK_NONBLOCK` flag on `socket.type`.

</div>

<div class="versionchanged">

3.15 Accepts any real number, not only integer or float.

</div>

</div>

<div class="method">

setsockopt(level, optname, value: int \| Buffer) setsockopt(level, optname, None, optlen: int)

<div class="index">

pair: module; struct

</div>

Set the value of the given socket option (see the Unix manual page `setsockopt(2)`). The needed symbolic constants are defined in this module (`!SO_\* etc. <socket-unix-constants>`). The value can be an integer, `None` or a `bytes-like object` representing a buffer. In the latter case it is up to the caller to ensure that the bytestring contains the proper bits (see the optional built-in module `struct` for a way to encode C structures as bytestrings). When *value* is set to `None`, *optlen* argument is required. It's equivalent to calling `!setsockopt` C function with `optval=NULL` and `optlen=optlen`.

<div class="versionchanged">

3.5 Writable `bytes-like object` is now accepted.

</div>

<div class="versionchanged">

3.6 setsockopt(level, optname, None, optlen: int) form added.

</div>

<div class="availability">

not WASI.

</div>

</div>

<div class="method">

shutdown(how)

Shut down one or both halves of the connection. If *how* is `SHUT_RD`, further receives are disallowed. If *how* is `SHUT_WR`, further sends are disallowed. If *how* is `SHUT_RDWR`, further sends and receives are disallowed.

<div class="availability">

not WASI.

</div>

</div>

<div class="method">

share(process_id)

Duplicate a socket and prepare it for sharing with a target process. The target process must be provided with *process_id*. The resulting bytes object can then be passed to the target process using some form of interprocess communication and the socket can be recreated there using `fromshare`. Once this method has been called, it is safe to close the socket since the operating system has already duplicated it for the target process.

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.3

</div>

</div>

Note that there are no methods `!read` or `!write`; use `~socket.recv` and `~socket.send` without *flags* argument instead.

Socket objects also have these (read-only) attributes that correspond to the values given to the `~socket.socket` constructor.

<div class="attribute">

family

The socket family.

</div>

<div class="attribute">

type

The socket type.

</div>

<div class="attribute">

proto

The socket protocol.

</div>

</div>

<div class="SocketType">

The base class of the `~socket.socket` type, re-exported from `!_socket`. An instance check such as `isinstance(socket(...), SocketType)` is true, but `SocketType` is not the same as `type(socket(...))`, which is `~socket.socket` itself.

</div>

## Notes on socket timeouts

A socket object can be in one of three modes: blocking, non-blocking, or timeout. Sockets are by default always created in blocking mode, but this can be changed by calling `setdefaulttimeout`.

- In *blocking mode*, operations block until complete or the system returns an error (such as connection timed out).
- In *non-blocking mode*, operations fail (with an error that is unfortunately system-dependent) if they cannot be completed immediately: functions from the `select` module can be used to know when and whether a socket is available for reading or writing.
- In *timeout mode*, operations fail if they cannot be completed within the timeout specified for the socket (they raise a `timeout` exception) or if the system returns an error.

> [!NOTE]
> At the operating system level, sockets in *timeout mode* are internally set in non-blocking mode. Also, the blocking and timeout modes are shared between file descriptors and socket objects that refer to the same network endpoint. This implementation detail can have visible consequences if e.g. you decide to use the `~socket.fileno` of a socket.

### Timeouts and the `connect` method

The `~socket.connect` operation is also subject to the timeout setting, and in general it is recommended to call `~socket.settimeout` before calling `~socket.connect` or pass a timeout parameter to `create_connection`. However, the system network stack may also return a connection timeout error of its own regardless of any Python socket timeout setting.

### Timeouts and the `accept` method

If `getdefaulttimeout` is not `None`, sockets returned by the `~socket.accept` method inherit that timeout. Otherwise, the behaviour depends on settings of the listening socket:

- if the listening socket is in *blocking mode* or in *timeout mode*, the socket returned by `~socket.accept` is in *blocking mode*;
- if the listening socket is in *non-blocking mode*, whether the socket returned by `~socket.accept` is in blocking or non-blocking mode is operating system-dependent. If you want to ensure cross-platform behaviour, it is recommended you manually override this setting.

## Example

Here are four minimal example programs using the TCP/IP protocol: a server that echoes all data that it receives back (servicing only one client), and a client using it. Note that a server must perform the sequence `~socket.socket`, `~socket.bind`, `~socket.listen`, `~socket.accept` (possibly repeating the `~socket.accept` to service more than one client), while a client only needs the sequence `~socket.socket`, `~socket.connect`. Also note that the server does not `~socket.sendall`/`~socket.recv` on the socket it is listening on but on the new socket returned by `~socket.accept`.

The first two examples support IPv4 only. :

    # Echo server program
    import socket

    HOST = ''                 # Symbolic name meaning all available interfaces
    PORT = 50007              # Arbitrary non-privileged port
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
        s.bind((HOST, PORT))
        s.listen(1)
        conn, addr = s.accept()
        with conn:
            print('Connected by', addr)
            while True:
                data = conn.recv(1024)
                if not data: break
                conn.sendall(data)

    # Echo client program
    import socket

    HOST = 'daring.cwi.nl'    # The remote host
    PORT = 50007              # The same port as used by the server
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
        s.connect((HOST, PORT))
        s.sendall(b'Hello, world')
        data = s.recv(1024)
    print('Received', repr(data))

The next two examples are identical to the above two, but support both IPv4 and IPv6. The server side will listen to the first address family available (it should listen to both instead). On most of IPv6-ready systems, IPv6 will take precedence and the server may not accept IPv4 traffic. The client side will try to connect to all the addresses returned as a result of the name resolution, and sends traffic to the first one connected successfully. :

    # Echo server program
    import socket
    import sys

    HOST = None               # Symbolic name meaning all available interfaces
    PORT = 50007              # Arbitrary non-privileged port
    s = None
    for res in socket.getaddrinfo(HOST, PORT, socket.AF_UNSPEC,
                                  socket.SOCK_STREAM, 0, socket.AI_PASSIVE):
        af, socktype, proto, canonname, sa = res
        try:
            s = socket.socket(af, socktype, proto)
        except OSError as msg:
            s = None
            continue
        try:
            s.bind(sa)
            s.listen(1)
        except OSError as msg:
            s.close()
            s = None
            continue
        break
    if s is None:
        print('could not open socket')
        sys.exit(1)
    conn, addr = s.accept()
    with conn:
        print('Connected by', addr)
        while True:
            data = conn.recv(1024)
            if not data: break
            conn.send(data)

    # Echo client program
    import socket
    import sys

    HOST = 'daring.cwi.nl'    # The remote host
    PORT = 50007              # The same port as used by the server
    s = None
    for res in socket.getaddrinfo(HOST, PORT, socket.AF_UNSPEC, socket.SOCK_STREAM):
        af, socktype, proto, canonname, sa = res
        try:
            s = socket.socket(af, socktype, proto)
        except OSError as msg:
            s = None
            continue
        try:
            s.connect(sa)
        except OSError as msg:
            s.close()
            s = None
            continue
        break
    if s is None:
        print('could not open socket')
        sys.exit(1)
    with s:
        s.sendall(b'Hello, world')
        data = s.recv(1024)
    print('Received', repr(data))

The next example shows how to write a very simple network sniffer with raw sockets on Windows. The example requires administrator privileges to modify the interface:

    import socket

    # the public network interface
    HOST = socket.gethostbyname(socket.gethostname())

    # create a raw socket and bind it to the public interface
    s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
    s.bind((HOST, 0))

    # Include IP headers
    s.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)

    # receive all packets
    s.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)

    # receive a packet
    print(s.recvfrom(65565))

    # disabled promiscuous mode
    s.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)

The next example shows how to use the socket interface to communicate to a CAN network using the raw socket protocol. To use CAN with the broadcast manager protocol instead, open a socket with:

    socket.socket(socket.AF_CAN, socket.SOCK_DGRAM, socket.CAN_BCM)

After binding (`CAN_RAW`) or connecting (`CAN_BCM`) the socket, you can use the `socket.send` and `socket.recv` operations (and their counterparts) on the socket object as usual.

This last example might require special privileges:

    import socket
    import struct


    # CAN frame packing/unpacking (see 'struct can_frame' in <linux/can.h>)

    can_frame_fmt = "=IB3x8s"
    can_frame_size = struct.calcsize(can_frame_fmt)

    def build_can_frame(can_id, data):
        can_dlc = len(data)
        data = data.ljust(8, b'\x00')
        return struct.pack(can_frame_fmt, can_id, can_dlc, data)

    def dissect_can_frame(frame):
        can_id, can_dlc, data = struct.unpack(can_frame_fmt, frame)
        return (can_id, can_dlc, data[:can_dlc])


    # create a raw socket and bind it to the 'vcan0' interface
    s = socket.socket(socket.AF_CAN, socket.SOCK_RAW, socket.CAN_RAW)
    s.bind(('vcan0',))

    while True:
        cf, addr = s.recvfrom(can_frame_size)

        print('Received: can_id=%x, can_dlc=%x, data=%s' % dissect_can_frame(cf))

        try:
            s.send(cf)
        except OSError:
            print('Error sending CAN frame')

        try:
            s.send(build_can_frame(0x01, b'\x01\x02\x03'))
        except OSError:
            print('Error sending CAN frame')

Running an example several times with too small delay between executions, could lead to this error:

    OSError: [Errno 98] Address already in use

This is because the previous execution has left the socket in a `TIME_WAIT` state, and can't be immediately reused.

There is a `!socket` flag to set, in order to prevent this, `socket.SO_REUSEADDR`:

    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind((HOST, PORT))

the `SO_REUSEADDR` flag tells the kernel to reuse a local socket in `TIME_WAIT` state, without waiting for its natural timeout to expire.

<div class="seealso">

For an introduction to socket programming (in C), see the following papers:

- *An Introductory 4.3BSD Interprocess Communication Tutorial*, by Stuart Sechrest
- *An Advanced 4.3BSD Interprocess Communication Tutorial*, by Samuel J. Leffler et al,

both in the UNIX Programmer's Manual, Supplementary Documents 1 (sections PS1:7 and PS1:8). The platform-specific reference material for the various socket-related system calls are also a valuable source of information on the details of socket semantics. For Unix, refer to the manual pages; for Windows, see the WinSock (or Winsock 2) specification. For IPv6-ready APIs, readers may want to refer to `3493` titled Basic Socket Interface Extensions for IPv6.

</div>

# `!socketserver` --- A framework for network servers

<div class="module" data-synopsis="A framework for network servers.">

socketserver

</div>

**Source code:** `Lib/socketserver.py`

------------------------------------------------------------------------

The `!socketserver` module simplifies the task of writing network servers.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

There are four basic concrete server classes:

<div class="TCPServer(server_address, RequestHandlerClass, bind_and_activate=True)">

This uses the internet TCP protocol, which provides for continuous streams of data between the client and server. If *bind_and_activate* is true, the constructor automatically attempts to invoke `~BaseServer.server_bind` and `~BaseServer.server_activate`. The other parameters are passed to the `BaseServer` base class.

<div class="versionchanged">

3.15 The default queue size is now `socket.SOMAXCONN` for `socketserver.TCPServer`.

</div>

</div>

<div class="UDPServer(server_address, RequestHandlerClass, bind_and_activate=True)">

This uses datagrams, which are discrete packets of information that may arrive out of order or be lost while in transit. The parameters are the same as for `TCPServer`.

</div>

<div class="UnixStreamServer(server_address, RequestHandlerClass, bind_and_activate=True) UnixDatagramServer(server_address, RequestHandlerClass, bind_and_activate=True)">

These more infrequently used classes are similar to the TCP and UDP classes, but use Unix domain sockets; they're not available on non-Unix platforms. The parameters are the same as for `TCPServer`.

</div>

These four classes process requests `synchronously`; each request must be completed before the next request can be started. This isn't suitable if each request takes a long time to complete, because it requires a lot of computation, or because it returns a lot of data which the client is slow to process. The solution is to create a separate process or thread to handle each request; the `ForkingMixIn` and `ThreadingMixIn` mix-in classes can be used to support asynchronous behaviour.

Creating a server requires several steps. First, you must create a request handler class by subclassing the `BaseRequestHandler` class and overriding its `~BaseRequestHandler.handle` method; this method will process incoming requests. Second, you must instantiate one of the server classes, passing it the server's address and the request handler class. It is recommended to use the server in a `with` statement. Then call the `~BaseServer.handle_request` or `~BaseServer.serve_forever` method of the server object to process one or many requests. Finally, call `~BaseServer.server_close` to close the socket (unless you used a `!with` statement).

When inheriting from `ThreadingMixIn` for threaded connection behavior, you should explicitly declare how you want your threads to behave on an abrupt shutdown. The `ThreadingMixIn` class defines an attribute *daemon_threads*, which indicates whether or not the server should wait for thread termination. You should set the flag explicitly if you would like threads to behave autonomously; the default is `False`, meaning that Python will not exit until all threads created by `ThreadingMixIn` have exited.

Server classes have the same external methods and attributes, no matter what network protocol they use.

## Server Creation Notes

There are five classes in an inheritance diagram, four of which represent synchronous servers of four types:

    +------------+
    | BaseServer |
    +------------+
          |
          v
    +-----------+        +------------------+
    | TCPServer |------->| UnixStreamServer |
    +-----------+        +------------------+
          |
          v
    +-----------+        +--------------------+
    | UDPServer |------->| UnixDatagramServer |
    +-----------+        +--------------------+

Note that `UnixDatagramServer` derives from `UDPServer`, not from `UnixStreamServer` --- the only difference between an IP and a Unix server is the address family.

<div class="ForkingMixIn ThreadingMixIn">

Forking and threading versions of each type of server can be created using these mix-in classes. For instance, `ThreadingUDPServer` is created as follows:

    class ThreadingUDPServer(ThreadingMixIn, UDPServer):
        pass

The mix-in class comes first, since it overrides a method defined in `UDPServer`. Setting the various attributes also changes the behavior of the underlying server mechanism.

`ForkingMixIn` and the Forking classes mentioned below are only available on POSIX platforms that support `~os.fork`.

<div class="attribute">

block_on_close

`ForkingMixIn.server_close <BaseServer.server_close>` waits until all child processes complete, except if `block_on_close` attribute is `False`.

`ThreadingMixIn.server_close <BaseServer.server_close>` waits until all non-daemon threads complete, except if `block_on_close` attribute is `False`.

</div>

<div class="attribute">

max_children

Specify how many child processes will exist to handle requests at a time for `ForkingMixIn`. If the limit is reached, new requests will wait until one child process has finished.

</div>

<div class="attribute">

daemon_threads

For `ThreadingMixIn` use daemonic threads by setting `ThreadingMixIn.daemon_threads <daemon_threads>` to `True` to not wait until threads complete.

</div>

<div class="versionchanged">

3.7

`ForkingMixIn.server_close <BaseServer.server_close>` and `ThreadingMixIn.server_close <BaseServer.server_close>` now waits until all child processes and non-daemonic threads complete. Add a new `ForkingMixIn.block_on_close <block_on_close>` class attribute to opt-in for the pre-3.7 behaviour.

</div>

</div>

<div class="ForkingTCPServer ForkingUDPServer ThreadingTCPServer ThreadingUDPServer ForkingUnixStreamServer ForkingUnixDatagramServer ThreadingUnixStreamServer ThreadingUnixDatagramServer">

These classes are pre-defined using the mix-in classes.

</div>

<div class="versionadded">

3.12 The `ForkingUnixStreamServer` and `ForkingUnixDatagramServer` classes were added.

</div>

To implement a service, you must derive a class from `BaseRequestHandler` and redefine its `~BaseRequestHandler.handle` method. You can then run various versions of the service by combining one of the server classes with your request handler class. The request handler class must be different for datagram or stream services. This can be hidden by using the handler subclasses `StreamRequestHandler` or `DatagramRequestHandler`.

Of course, you still have to use your head! For instance, it makes no sense to use a forking server if the service contains state in memory that can be modified by different requests, since the modifications in the child process would never reach the initial state kept in the parent process and passed to each child. In this case, you can use a threading server, but you will probably have to use locks to protect the integrity of the shared data.

On the other hand, if you are building an HTTP server where all data is stored externally (for instance, in the file system), a synchronous class will essentially render the service "deaf" while one request is being handled -- which may be for a very long time if a client is slow to receive all the data it has requested. Here a threading or forking server is appropriate.

In some cases, it may be appropriate to process part of a request synchronously, but to finish processing in a forked child depending on the request data. This can be implemented by using a synchronous server and doing an explicit fork in the request handler class `~BaseRequestHandler.handle` method.

Another approach to handling multiple simultaneous requests in an environment that supports neither threads nor `~os.fork` (or where these are too expensive or inappropriate for the service) is to maintain an explicit table of partially finished requests and to use `selectors` to decide which request to work on next (or whether to handle a new incoming request). This is particularly important for stream services where each client can potentially be connected for a long time (if threads or subprocesses cannot be used).

## Server Objects

<div class="BaseServer(server_address, RequestHandlerClass)">

This is the superclass of all Server objects in the module. It defines the interface, given below, but does not implement most of the methods, which is done in subclasses. The two parameters are stored in the respective `server_address` and `RequestHandlerClass` attributes.

<div class="method">

fileno()

Return an integer file descriptor for the socket on which the server is listening. This function is most commonly passed to `selectors`, to allow monitoring multiple servers in the same process.

</div>

<div class="method">

handle_request()

Process a single request. This function calls the following methods in order: `get_request`, `verify_request`, and `process_request`. If the user-provided `~BaseRequestHandler.handle` method of the handler class raises an exception, the server's `handle_error` method will be called. If no request is received within `timeout` seconds, `handle_timeout` will be called and `handle_request` will return.

</div>

<div class="method">

serve_forever(poll_interval=0.5)

Handle requests until an explicit `shutdown` request. Poll for shutdown every *poll_interval* seconds. Ignores the `timeout` attribute. It also calls `service_actions`, which may be used by a subclass or mixin to provide actions specific to a given service. For example, the `ForkingMixIn` class uses `service_actions` to clean up zombie child processes.

<div class="versionchanged">

3.3 Added `service_actions` call to the `serve_forever` method.

</div>

</div>

<div class="method">

service_actions()

This is called in the `serve_forever` loop. This method can be overridden by subclasses or mixin classes to perform actions specific to a given service, such as cleanup actions.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

shutdown()

Tell the `serve_forever` loop to stop and wait until it does. `shutdown` must be called while `serve_forever` is running in a different thread otherwise it will deadlock.

</div>

<div class="method">

server_close()

Clean up the server. May be overridden.

</div>

<div class="attribute">

address_family

The family of protocols to which the server's socket belongs. Common examples are `socket.AF_INET`, `socket.AF_INET6`, and `socket.AF_UNIX`. Subclass the TCP or UDP server classes in this module with class attribute `address_family = AF_INET6` set if you want IPv6 server classes.

</div>

<div class="attribute">

RequestHandlerClass

The user-provided request handler class; an instance of this class is created for each request.

</div>

<div class="attribute">

server_address

The address on which the server is listening. The format of addresses varies depending on the protocol family; see the documentation for the `socket` module for details. For internet protocols, this is a tuple containing a string giving the address, and an integer port number: `('127.0.0.1', 80)`, for example.

</div>

<div class="attribute">

socket

The socket object on which the server will listen for incoming requests.

</div>

The server classes support the following class variables:

<div class="attribute">

allow_reuse_address

Whether the server will allow the reuse of an address. This defaults to `False`, and can be set in subclasses to change the policy.

</div>

<div class="attribute">

request_queue_size

The size of the request queue. If it takes a long time to process a single request, any requests that arrive while the server is busy are placed into a queue, up to `request_queue_size` requests. Once the queue is full, further requests from clients will get a "Connection denied" error. The default value is usually 5, but this can be overridden by subclasses.

</div>

<div class="attribute">

socket_type

The type of socket used by the server; `socket.SOCK_STREAM` and `socket.SOCK_DGRAM` are two common values.

</div>

<div class="attribute">

timeout

Timeout duration, measured in seconds, or `None` if no timeout is desired. If `handle_request` receives no incoming requests within the timeout period, the `handle_timeout` method is called.

</div>

There are various server methods that can be overridden by subclasses of base server classes like `TCPServer`; these methods aren't useful to external users of the server object.

<div class="method">

finish_request(request, client_address)

Actually processes the request by instantiating `RequestHandlerClass` and calling its `~BaseRequestHandler.handle` method.

</div>

<div class="method">

get_request()

Must accept a request from the socket, and return a 2-tuple containing the *new* socket object to be used to communicate with the client, and the client's address.

</div>

<div class="method">

handle_error(request, client_address)

This function is called if the `~BaseRequestHandler.handle` method of a `RequestHandlerClass` instance raises an exception. The default action is to print the traceback to standard error and continue handling further requests.

<div class="versionchanged">

3.6 Now only called for exceptions derived from the `Exception` class.

</div>

</div>

<div class="method">

handle_timeout()

This function is called when the `timeout` attribute has been set to a value other than `None` and the timeout period has passed with no requests being received. The default action for forking servers is to collect the status of any child processes that have exited, while in threading servers this method does nothing.

</div>

<div class="method">

process_request(request, client_address)

Calls `finish_request` to create an instance of the `RequestHandlerClass`. If desired, this function can create a new process or thread to handle the request; the `ForkingMixIn` and `ThreadingMixIn` classes do this.

</div>

<div class="method">

server_activate()

Called by the server's constructor to activate the server. The default behavior for a TCP server just invokes `~socket.socket.listen` on the server's socket. May be overridden.

</div>

<div class="method">

server_bind()

Called by the server's constructor to bind the socket to the desired address. May be overridden.

</div>

<div class="method">

verify_request(request, client_address)

Must return a Boolean value; if the value is `True`, the request will be processed, and if it's `False`, the request will be denied. This function can be overridden to implement access controls for a server. The default implementation always returns `True`.

</div>

<div class="versionchanged">

3.6 Support for the `context manager` protocol was added. Exiting the context manager is equivalent to calling `server_close`.

</div>

</div>

## Request Handler Objects

<div class="BaseRequestHandler">

This is the superclass of all request handler objects. It defines the interface, given below. A concrete request handler subclass must define a new `handle` method, and can override any of the other methods. A new instance of the subclass is created for each request.

<div class="method">

setup()

Called before the `handle` method to perform any initialization actions required. The default implementation does nothing.

</div>

<div class="method">

handle()

This function must do all the work required to service a request. The default implementation does nothing. Several instance attributes are available to it; the request is available as `request`; the client address as `client_address`; and the server instance as `server`, in case it needs access to per-server information.

The type of `request` is different for datagram or stream services. For stream services, `request` is a socket object; for datagram services, `request` is a pair of string and socket.

</div>

<div class="method">

finish()

Called after the `handle` method to perform any clean-up actions required. The default implementation does nothing. If `setup` raises an exception, this function will not be called.

</div>

<div class="attribute">

request

The *new* `socket.socket` object to be used to communicate with the client.

</div>

<div class="attribute">

client_address

Client address returned by `BaseServer.get_request`.

</div>

<div class="attribute">

server

`BaseServer` object used for handling the request.

</div>

</div>

<div class="StreamRequestHandler DatagramRequestHandler">

These `BaseRequestHandler` subclasses override the `~BaseRequestHandler.setup` and `~BaseRequestHandler.finish` methods, and provide `rfile` and `wfile` attributes.

<div class="attribute">

rfile

A file object from which receives the request is read. Support the `io.BufferedIOBase` readable interface.

</div>

<div class="attribute">

wfile

A file object to which the reply is written. Support the `io.BufferedIOBase` writable interface

</div>

<div class="versionchanged">

3.6 `wfile` also supports the `io.BufferedIOBase` writable interface.

</div>

</div>

## Examples

### `socketserver.TCPServer` Example

This is the server side:

    import socketserver

    class MyTCPHandler(socketserver.BaseRequestHandler):
        """
        The request handler class for our server.

        It is instantiated once per connection to the server, and must
        override the handle() method to implement communication to the
        client.
        """

        def handle(self):
            # self.request is the TCP socket connected to the client
            pieces = [b'']
            total = 0
            while b'\n' not in pieces[-1] and total < 10_000:
                pieces.append(self.request.recv(2000))
                total += len(pieces[-1])
            self.data = b''.join(pieces)
            print(f"Received from {self.client_address[0]}:")
            print(self.data.decode("utf-8"))
            # just send back the same data, but upper-cased
            self.request.sendall(self.data.upper())
            # after we return, the socket will be closed.

    if __name__ == "__main__":
        HOST, PORT = "localhost", 9999

        # Create the server, binding to localhost on port 9999
        with socketserver.TCPServer((HOST, PORT), MyTCPHandler) as server:
            # Activate the server; this will keep running until you
            # interrupt the program with Ctrl-C
            server.serve_forever()

An alternative request handler class that makes use of streams (file-like objects that simplify communication by providing the standard file interface):

    class MyTCPHandler(socketserver.StreamRequestHandler):

        def handle(self):
            # self.rfile is a file-like object created by the handler.
            # We can now use e.g. readline() instead of raw recv() calls.
            # We limit ourselves to 10000 bytes to avoid abuse by the sender.
            self.data = self.rfile.readline(10000).rstrip()
            print(f"{self.client_address[0]} wrote:")
            print(self.data.decode("utf-8"))
            # Likewise, self.wfile is a file-like object used to write back
            # to the client
            self.wfile.write(self.data.upper())

The difference is that the `readline()` call in the second handler will call `recv()` multiple times until it encounters a newline character, while the first handler had to use a `recv()` loop to accumulate data until a newline itself. If it had just used a single `recv()` without the loop it would just have returned what has been received so far from the client. TCP is stream based: data arrives in the order it was sent, but there is no correlation between client `send()` or `sendall()` calls and the number of `recv()` calls on the server required to receive it.

This is the client side:

    import socket
    import sys

    HOST, PORT = "localhost", 9999
    data = " ".join(sys.argv[1:])

    # Create a socket (SOCK_STREAM means a TCP socket)
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        # Connect to server and send data
        sock.connect((HOST, PORT))
        sock.sendall(bytes(data, "utf-8"))
        sock.sendall(b"\n")

        # Receive data from the server and shut down
        received = str(sock.recv(1024), "utf-8")

    print("Sent:    ", data)
    print("Received:", received)

The output of the example should look something like this:

Server:

```
$ python TCPServer.py
127.0.0.1 wrote:
b'hello world with TCP'
127.0.0.1 wrote:
b'python is nice'
```

Client:

```
$ python TCPClient.py hello world with TCP
Sent:     hello world with TCP
Received: HELLO WORLD WITH TCP
$ python TCPClient.py python is nice
Sent:     python is nice
Received: PYTHON IS NICE
```

### `socketserver.UDPServer` Example

This is the server side:

    import socketserver

    class MyUDPHandler(socketserver.BaseRequestHandler):
        """
        This class works similar to the TCP handler class, except that
        self.request consists of a pair of data and client socket, and since
        there is no connection the client address must be given explicitly
        when sending data back via sendto().
        """

        def handle(self):
            data = self.request[0].strip()
            socket = self.request[1]
            print(f"{self.client_address[0]} wrote:")
            print(data)
            socket.sendto(data.upper(), self.client_address)

    if __name__ == "__main__":
        HOST, PORT = "localhost", 9999
        with socketserver.UDPServer((HOST, PORT), MyUDPHandler) as server:
            server.serve_forever()

This is the client side:

    import socket
    import sys

    HOST, PORT = "localhost", 9999
    data = " ".join(sys.argv[1:])

    # SOCK_DGRAM is the socket type to use for UDP sockets
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

    # As you can see, there is no connect() call; UDP has no connections.
    # Instead, data is directly sent to the recipient via sendto().
    sock.sendto(bytes(data + "\n", "utf-8"), (HOST, PORT))
    received = str(sock.recv(1024), "utf-8")

    print("Sent:    ", data)
    print("Received:", received)

The output of the example should look exactly like for the TCP server example.

### Asynchronous Mixins

To build asynchronous handlers, use the `ThreadingMixIn` and `ForkingMixIn` classes.

An example for the `ThreadingMixIn` class:

    import socket
    import threading
    import socketserver

    class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):

        def handle(self):
            data = str(self.request.recv(1024), 'ascii')
            cur_thread = threading.current_thread()
            response = bytes("{}: {}".format(cur_thread.name, data), 'ascii')
            self.request.sendall(response)

    class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
        pass

    def client(ip, port, message):
        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
            sock.connect((ip, port))
            sock.sendall(bytes(message, 'ascii'))
            response = str(sock.recv(1024), 'ascii')
            print("Received: {}".format(response))

    if __name__ == "__main__":
        # Port 0 means to select an arbitrary unused port
        HOST, PORT = "localhost", 0

        server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
        with server:
            ip, port = server.server_address

            # Start a thread with the server -- that thread will then start one
            # more thread for each request
            server_thread = threading.Thread(target=server.serve_forever)
            # Exit the server thread when the main thread terminates
            server_thread.daemon = True
            server_thread.start()
            print("Server loop running in thread:", server_thread.name)

            client(ip, port, "Hello World 1")
            client(ip, port, "Hello World 2")
            client(ip, port, "Hello World 3")

            server.shutdown()

The output of the example should look something like this:

```
$ python ThreadedTCPServer.py
Server loop running in thread: Thread-1
Received: Thread-2: Hello World 1
Received: Thread-3: Hello World 2
Received: Thread-4: Hello World 3
```

The `ForkingMixIn` class is used in the same way, except that the server will spawn a new process for each request. Available only on POSIX platforms that support `~os.fork`.

# `!spwd` --- The shadow password database

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

spwd

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

A possible replacement is the third-party library `python-pam`. This library is not supported or maintained by the Python core team.

The last version of Python that provided the `!spwd` module was [Python 3.12](https://docs.python.org/3.12/library/spwd.html).

# `!sqlite3` --- DB-API 2.0 interface for SQLite databases

<div class="module" data-synopsis="A DB-API 2.0 implementation using SQLite 3.x.">

sqlite3

</div>

**Source code:** `Lib/sqlite3/`

<div class="testsetup">

import sqlite3 src = sqlite3.connect(":memory:", isolation_level=None) dst = sqlite3.connect("tutorial.db", isolation_level=None) src.backup(dst) src.close() dst.close() del src, dst

</div>

<div id="sqlite3-intro">

SQLite is a C library that provides a lightweight disk-based database that doesn't require a separate server process and allows accessing the database using a nonstandard variant of the SQL query language. Some applications can use SQLite for internal data storage. It's also possible to prototype an application using SQLite and then port the code to a larger database such as PostgreSQL or Oracle.

</div>

The `!sqlite3` module was written by Gerhard Häring. It provides an SQL interface compliant with the DB-API 2.0 specification described by `249`, and requires the third-party [SQLite](https://sqlite.org/) library.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

This document includes four main sections:

- `sqlite3-tutorial` teaches how to use the `!sqlite3` module.
- `sqlite3-reference` describes the classes and functions this module defines.
- `sqlite3-howtos` details how to handle specific tasks.
- `sqlite3-explanation` provides in-depth background on transaction control.

<div class="seealso">

<https://www.sqlite.org>  
The SQLite web page; the documentation describes the syntax and the available data types for the supported SQL dialect.

<https://www.w3schools.com/sql/>  
Tutorial, reference and examples for learning SQL syntax.

`249` - Database API Specification 2.0  
PEP written by Marc-André Lemburg.

</div>

## Tutorial

In this tutorial, you will create a database of Monty Python movies using basic `!sqlite3` functionality. It assumes a fundamental understanding of database concepts, including [cursors](https://en.wikipedia.org/wiki/Cursor_(databases)) and [transactions](https://en.wikipedia.org/wiki/Database_transaction).

First, we need to create a new database and open a database connection to allow `!sqlite3` to work with it. Call `sqlite3.connect` to create a connection to the database `tutorial.db` in the current working directory, implicitly creating it if it does not exist:

<div class="testcode">

import sqlite3 con = sqlite3.connect("tutorial.db")

</div>

The returned `Connection` object `con` represents the connection to the on-disk database.

In order to execute SQL statements and fetch results from SQL queries, we will need to use a database cursor. Call `con.cursor() <Connection.cursor>` to create the `Cursor`:

<div class="testcode">

cur = con.cursor()

</div>

Now that we've got a database connection and a cursor, we can create a database table `movie` with columns for title, release year, and review score. For simplicity, we can just use column names in the table declaration -- thanks to the [flexible typing](https://www.sqlite.org/flextypegood.html) feature of SQLite, specifying the data types is optional. Execute the `CREATE TABLE` statement by calling `cur.execute(...) <Cursor.execute>`:

<div class="testcode">

cur.execute("CREATE TABLE movie(title, year, score)")

</div>

We can verify that the new table has been created by querying the `sqlite_master` table built-in to SQLite, which should now contain an entry for the `movie` table definition (see [The Schema Table](https://www.sqlite.org/schematab.html) for details). Execute that query by calling `cur.execute(...) <Cursor.execute>`, assign the result to `res`, and call `res.fetchone() <Cursor.fetchone>` to fetch the resulting row:

<div class="doctest">

\>\>\> res = cur.execute("SELECT name FROM sqlite_master") \>\>\> res.fetchone() ('movie',)

</div>

We can see that the table has been created, as the query returns a `tuple` containing the table's name. If we query `sqlite_master` for a non-existent table `spam`, `!res.fetchone` will return `None`:

<div class="doctest">

\>\>\> res = cur.execute("SELECT name FROM sqlite_master WHERE name='spam'") \>\>\> res.fetchone() is None True

</div>

Now, add two rows of data supplied as SQL literals by executing an `INSERT` statement, once again by calling `cur.execute(...) <Cursor.execute>`:

<div class="testcode">

cur.execute("""  
INSERT INTO movie VALUES  
('Monty Python and the Holy Grail', 1975, 8.2), ('And Now for Something Completely Different', 1971, 7.5)

""")

</div>

The `INSERT` statement implicitly opens a transaction, which needs to be committed before changes are saved in the database (see `sqlite3-controlling-transactions` for details). Call `con.commit() <Connection.commit>` on the connection object to commit the transaction:

<div class="testcode">

con.commit()

</div>

We can verify that the data was inserted correctly by executing a `SELECT` query. Use the now-familiar `cur.execute(...) <Cursor.execute>` to assign the result to `res`, and call `res.fetchall() <Cursor.fetchall>` to return all resulting rows:

<div class="doctest">

\>\>\> res = cur.execute("SELECT score FROM movie") \>\>\> res.fetchall() \[(8.2,), (7.5,)\]

</div>

The result is a `list` of two `!tuple`s, one per row, each containing that row's `score` value.

Now, insert three more rows by calling `cur.executemany(...) <Cursor.executemany>`:

<div class="testcode">

data = \[  
("Monty Python Live at the Hollywood Bowl", 1982, 7.9), ("Monty Python's The Meaning of Life", 1983, 7.5), ("Monty Python's Life of Brian", 1979, 8.0),

\] cur.executemany("INSERT INTO movie VALUES(?, ?, ?)", data) con.commit() \# Remember to commit the transaction after executing INSERT.

</div>

Notice that `?` placeholders are used to bind `data` to the query. Always use placeholders instead of `string formatting <tut-formatting>` to bind Python values to SQL statements, to avoid [SQL injection attacks](https://en.wikipedia.org/wiki/SQL_injection) (see `sqlite3-placeholders` for more details).

We can verify that the new rows were inserted by executing a `SELECT` query, this time iterating over the results of the query:

<div class="doctest">

\>\>\> for row in cur.execute("SELECT year, title FROM movie ORDER BY year"): ... print(row) (1971, 'And Now for Something Completely Different') (1975, 'Monty Python and the Holy Grail') (1979, "Monty Python's Life of Brian") (1982, 'Monty Python Live at the Hollywood Bowl') (1983, "Monty Python's The Meaning of Life")

</div>

Each row is a two-item `tuple` of `(year, title)`, matching the columns selected in the query.

Finally, verify that the database has been written to disk by calling `con.close() <Connection.close>` to close the existing connection, opening a new one, creating a new cursor, then querying the database:

<div class="doctest">

\>\>\> con.close() \>\>\> new_con = sqlite3.connect("tutorial.db") \>\>\> new_cur = new_con.cursor() \>\>\> res = new_cur.execute("SELECT title, year FROM movie ORDER BY score DESC") \>\>\> title, year = res.fetchone() \>\>\> print(f'The highest scoring Monty Python movie is {title!r}, released in {year}') The highest scoring Monty Python movie is 'Monty Python and the Holy Grail', released in 1975 \>\>\> new_con.close()

</div>

You've now created an SQLite database using the `!sqlite3` module, inserted data and retrieved values from it in multiple ways.

<div class="seealso">

- `sqlite3-howtos` for further reading:
  - `sqlite3-placeholders`
  - `sqlite3-adapters`
  - `sqlite3-converters`
  - `sqlite3-connection-context-manager`
  - `sqlite3-howto-row-factory`
- `sqlite3-explanation` for in-depth background on transaction control.

</div>

## Reference

### Module functions<span id="sqlite3-module-contents"></span>

<div class="function">

connect(database, \*, timeout=5.0, detect_types=0, isolation_level="DEFERRED", check_same_thread=True, factory=sqlite3.Connection, cached_statements=128, uri=False, autocommit=sqlite3.LEGACY_TRANSACTION_CONTROL)

Open a connection to an SQLite database.

param database  
The path to the database file to be opened. You can pass `":memory:"` to create an [SQLite database existing only in memory](https://sqlite.org/inmemorydb.html), and open a connection to it.

type database  
`path-like object`

param float timeout  
How many seconds the connection should wait before raising an `OperationalError` when a table is locked. If another connection opens a transaction to modify a table, that table will be locked until the transaction is committed. Default five seconds.

param int detect_types  
Control whether and how data types not `natively supported by SQLite <sqlite3-types>` are looked up to be converted to Python types, using the converters registered with `register_converter`. Set it to any combination (using `|`, bitwise or) of `PARSE_DECLTYPES` and `PARSE_COLNAMES` to enable this. Column names take precedence over declared types if both flags are set. By default (`0`), type detection is disabled.

param isolation_level  
Control legacy transaction handling behaviour. See `Connection.isolation_level` and `sqlite3-transaction-control-isolation-level` for more information. Can be `"DEFERRED"` (default), `"EXCLUSIVE"` or `"IMMEDIATE"`; or `None` to disable opening transactions implicitly. Has no effect unless `Connection.autocommit` is set to `~sqlite3.LEGACY_TRANSACTION_CONTROL` (the default).

type isolation_level  
str \| None

param bool check_same_thread  
If `True` (default), `ProgrammingError` will be raised if the database connection is used by a thread other than the one that created it. If `False`, the connection may be accessed in multiple threads; write operations may need to be serialized by the user to avoid data corruption. See `threadsafety` for more information.

param ~sqlite3.Connection factory  
A custom subclass of `Connection` to create the connection with, if not the default `Connection` class.

param int cached_statements  
The number of statements that `!sqlite3` should internally cache for this connection, to avoid parsing overhead. By default, 128 statements.

param bool uri  
If set to `True`, *database* is interpreted as a `URI (Uniform Resource Identifier)` with a file path and an optional query string. The scheme part *must* be `"file:"`, and the path can be relative or absolute. The query string allows passing parameters to SQLite, enabling various `sqlite3-uri-tricks`.

param autocommit  
Control `249` transaction handling behaviour. See `Connection.autocommit` and `sqlite3-transaction-control-autocommit` for more information. *autocommit* currently defaults to `~sqlite3.LEGACY_TRANSACTION_CONTROL`. The default will change to `False` in a future Python release.

type autocommit  
bool

rtype  
~sqlite3.Connection

<div class="audit-event">

sqlite3.connect database sqlite3.connect

</div>

<div class="audit-event">

sqlite3.connect/handle connection_handle sqlite3.connect

</div>

<div class="versionchanged">

3.4 Added the *uri* parameter.

</div>

<div class="versionchanged">

3.7 *database* can now also be a `path-like object`, not only a string.

</div>

<div class="versionchanged">

3.10 Added the `sqlite3.connect/handle` auditing event.

</div>

<div class="versionchanged">

3.12 Added the *autocommit* parameter.

</div>

<div class="versionchanged">

3.15 All parameters except *database* are now keyword-only.

</div>

</div>

<div class="function">

complete_statement(statement)

Return `True` if the string *statement* appears to contain one or more complete SQL statements. No syntactic verification or parsing of any kind is performed, other than checking that there are no unclosed string literals and the statement is terminated by a semicolon.

For example:

<div class="doctest">

\>\>\> sqlite3.complete_statement("SELECT foo FROM bar;") True \>\>\> sqlite3.complete_statement("SELECT foo") False

</div>

This function may be useful during command-line input to determine if the entered text seems to form a complete SQL statement, or if additional input is needed before calling `~Cursor.execute`.

See `!runsource` in `Lib/sqlite3/__main__.py` for real-world use.

</div>

<div class="function">

enable_callback_tracebacks(flag, /)

Enable or disable callback tracebacks. By default you will not get any tracebacks in user-defined functions, aggregates, converters, authorizer callbacks etc. If you want to debug them, you can call this function with *flag* set to `True`. Afterwards, you will get tracebacks from callbacks on `sys.stderr`. Use `False` to disable the feature again.

> [!NOTE]
> Errors in user-defined function callbacks are logged as unraisable exceptions. Use an `unraisable hook handler <sys.unraisablehook>` for introspection of the failed callback.

</div>

<div class="function">

register_adapter(type, adapter, /)

Register an *adapter* `callable` to adapt the Python type *type* into an SQLite type. The adapter is called with a Python object of type *type* as its sole argument, and must return a value of a `type that SQLite natively understands <sqlite3-types>`.

</div>

<div class="function">

register_converter(typename, converter, /)

Register the *converter* `callable` to convert SQLite objects of type *typename* into a Python object of a specific type. The converter is invoked for all SQLite values of type *typename*; it is passed a `bytes` object and should return an object of the desired Python type. Consult the parameter *detect_types* of `connect` for information regarding how type detection works.

Note: *typename* and the name of the type in your query are matched case-insensitively.

</div>

### Module constants

<div class="data">

LEGACY_TRANSACTION_CONTROL

Set `~Connection.autocommit` to this constant to select old style (pre-Python 3.12) transaction control behaviour. See `sqlite3-transaction-control-isolation-level` for more information.

</div>

<div class="data">

PARSE_DECLTYPES

Pass this flag value to the *detect_types* parameter of `connect` to look up a converter function using the declared types for each column. The types are declared when the database table is created. `!sqlite3` will look up a converter function using the first word of the declared type as the converter dictionary key. For example:

``` sql
CREATE TABLE test(
   i integer primary key,  ! will look up a converter named "integer"
   p point,                ! will look up a converter named "point"
   n number(10)            ! will look up a converter named "number"
 )
```

This flag may be combined with `PARSE_COLNAMES` using the `|` (bitwise or) operator.

> [!NOTE]
> Generated fields (for example `MAX(p)`) are returned as `str`. Use `!PARSE_COLNAMES` to enforce types for such queries.

</div>

<div class="data">

PARSE_COLNAMES

Pass this flag value to the *detect_types* parameter of `connect` to look up a converter function by using the type name, parsed from the query column name, as the converter dictionary key. The query column name must be wrapped in double quotes (`"`) and the type name must be wrapped in square brackets (`[]`).

``` sql
SELECT MAX(p) as "p [point]" FROM test;  ! will look up converter "point"
```

This flag may be combined with `PARSE_DECLTYPES` using the `|` (bitwise or) operator.

</div>

<div class="data">

SQLITE_OK SQLITE_DENY SQLITE_IGNORE

Flags that should be returned by the *authorizer_callback* `callable` passed to `Connection.set_authorizer`, to indicate whether:

- Access is allowed (`!SQLITE_OK`),
- The SQL statement should be aborted with an error (`!SQLITE_DENY`)
- The column should be treated as a `NULL` value (`!SQLITE_IGNORE`)

</div>

<div class="data">

apilevel

String constant stating the supported DB-API level. Required by the DB-API. Hard-coded to `"2.0"`.

</div>

<div class="data">

paramstyle

String constant stating the type of parameter marker formatting expected by the `!sqlite3` module. Required by the DB-API. Hard-coded to `"qmark"`.

> [!NOTE]
> The `named` DB-API parameter style is also supported.

</div>

<div class="data">

SQLITE_VERSION

The version string of the SQLite library that was used for building the module. This may be different from the SQLite library actually used at runtime, which is available as `sqlite_version`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

sqlite_version

Version number of the runtime SQLite library as a `string <str>`.

</div>

<div class="data">

SQLITE_VERSION_INFO

A named tuple containing the three components of the SQLite library version that was used for building the module: *major*, *minor*, and *patch*. All values are integers. The components can also be accessed by name, so `sqlite3.SQLITE_VERSION_INFO[0]` is equivalent to `sqlite3.SQLITE_VERSION_INFO.major` and so on. This may be different from the SQLite library actually used at runtime, which is available as `sqlite_version_info`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

sqlite_version_info

A named tuple containing the version of the runtime SQLite library, with the same fields as `SQLITE_VERSION_INFO`.

<div class="versionchanged">

3.16 It is now a named tuple.

</div>

</div>

<div class="data">

SQLITE_KEYWORDS

A `tuple` containing all SQLite keywords.

This constant is only available if Python was compiled with SQLite 3.24.0 or greater.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

threadsafety

Integer constant required by the DB-API 2.0, stating the level of thread safety the `!sqlite3` module supports. This attribute is set based on the default [threading mode](https://sqlite.org/threadsafe.html) the underlying SQLite library is compiled with. The SQLite threading modes are:

1.  **Single-thread**: In this mode, all mutexes are disabled and SQLite is unsafe to use in more than a single thread at once.
2.  **Multi-thread**: In this mode, SQLite can be safely used by multiple threads provided that no single database connection is used simultaneously in two or more threads.
3.  **Serialized**: In serialized mode, SQLite can be safely used by multiple threads with no restriction.

The mappings from SQLite threading modes to DB-API 2.0 threadsafety levels are as follows:

| SQLite threading mode | `threadsafety
<0249#threadsafety>` | [SQLITE_THREADSAFE](https://sqlite.org/compile.html#threadsafe) | DB-API 2.0 meaning |
|----|----|----|----|
| single-thread | 0 | 0 | Threads may not share the module |
| multi-thread | 1 | 2 | Threads may share the module, but not connections |
| serialized | 3 | 1 | Threads may share the module, connections and cursors |

<div class="versionchanged">

3.11 Set *threadsafety* dynamically instead of hard-coding it to `1`.

</div>

</div>

<div id="sqlite3-dbconfig-constants">

<div class="data">

SQLITE_DBCONFIG_DEFENSIVE SQLITE_DBCONFIG_DQS_DDL SQLITE_DBCONFIG_DQS_DML SQLITE_DBCONFIG_ENABLE_FKEY SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION SQLITE_DBCONFIG_ENABLE_QPSG SQLITE_DBCONFIG_ENABLE_TRIGGER SQLITE_DBCONFIG_ENABLE_VIEW SQLITE_DBCONFIG_LEGACY_ALTER_TABLE SQLITE_DBCONFIG_LEGACY_FILE_FORMAT SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE SQLITE_DBCONFIG_RESET_DATABASE SQLITE_DBCONFIG_TRIGGER_EQP SQLITE_DBCONFIG_TRUSTED_SCHEMA SQLITE_DBCONFIG_WRITABLE_SCHEMA

These constants are used for the `Connection.setconfig` and `~Connection.getconfig` methods.

The availability of these constants varies depending on the version of SQLite Python was compiled with.

<div class="versionadded">

3.12

</div>

<div class="seealso">

<https://www.sqlite.org/c3ref/c_dbconfig_defensive.html>  
SQLite docs: Database Connection Configuration Options

</div>

</div>

</div>

<div class="deprecated-removed">

3.12 3.14 The `!version` and `!version_info` constants.

</div>

### Connection objects

<div class="Connection">

Each open SQLite database is represented by a `Connection` object, which is created using `sqlite3.connect`. Their main purpose is creating `Cursor` objects, and `sqlite3-controlling-transactions`.

<div class="seealso">

- `sqlite3-connection-shortcuts`
- `sqlite3-connection-context-manager`

</div>

<div class="versionchanged">

3.13

A `ResourceWarning` is emitted if `close` is not called before a `!Connection` object is deleted.

</div>

An SQLite database connection has the following attributes and methods:

<div class="method">

cursor(factory=Cursor)

Create and return a `Cursor` object. The cursor method accepts a single optional parameter *factory*. If supplied, this must be a `callable` returning an instance of `Cursor` or its subclasses.

</div>

<div class="method">

blobopen(table, column, rowid, /, \*, readonly=False, name="main")

Open a `Blob` handle to an existing `BLOB (Binary Large OBject)`.

param str table  
The name of the table where the blob is located.

param str column  
The name of the column where the blob is located.

param int rowid  
The row id where the blob is located.

param bool readonly  
Set to `True` if the blob should be opened without write permissions. Defaults to `False`.

param str name  
The name of the database where the blob is located. Defaults to `"main"`.

raises OperationalError  
When trying to open a blob in a `WITHOUT ROWID` table.

rtype  
Blob

> [!NOTE]
> The blob size cannot be changed using the `Blob` class. Use the SQL function `zeroblob` to create a blob with a fixed size.

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

commit()

Commit any pending transaction to the database. If `autocommit` is `True`, or there is no open transaction, this method does nothing. If `!autocommit` is `False`, a new transaction is implicitly opened if a pending transaction was committed by this method.

</div>

<div class="method">

rollback()

Roll back to the start of any pending transaction. If `autocommit` is `True`, or there is no open transaction, this method does nothing. If `!autocommit` is `False`, a new transaction is implicitly opened if a pending transaction was rolled back by this method.

</div>

<div class="method">

close()

Close the database connection. If `autocommit` is `False`, any pending transaction is implicitly rolled back. If `!autocommit` is `True` or `LEGACY_TRANSACTION_CONTROL`, no implicit transaction control is executed. Make sure to `commit` before closing to avoid losing pending changes.

</div>

<div class="method">

execute(sql, parameters=(), /)

Create a new `Cursor` object and call `~Cursor.execute` on it with the given *sql* and *parameters*. Return the new cursor object.

</div>

<div class="method">

executemany(sql, parameters, /)

Create a new `Cursor` object and call `~Cursor.executemany` on it with the given *sql* and *parameters*. Return the new cursor object.

</div>

<div class="method">

executescript(sql_script, /)

Create a new `Cursor` object and call `~Cursor.executescript` on it with the given *sql_script*. Return the new cursor object.

</div>

<div class="method">

create_function(name, narg, func, /, \*, deterministic=False)

Create or remove a user-defined SQL function.

param str name  
The name of the SQL function.

param int narg  
The number of arguments the SQL function can accept. If `-1`, it may take any number of arguments.

param func  
A `callable` that is called when the SQL function is invoked. The callable must return `a type natively supported by SQLite
<sqlite3-types>`. Set to `None` to remove an existing SQL function.

type func  
`callback` \| None

param bool deterministic  
If `True`, the created SQL function is marked as [deterministic](https://sqlite.org/deterministic.html), which allows SQLite to perform additional optimizations.

<div class="versionchanged">

3.8 Added the *deterministic* parameter.

</div>

<div class="versionchanged">

3.15 The first three parameters are now positional-only.

</div>

Example:

<div class="doctest">

\>\>\> import hashlib \>\>\> def md5sum(t): ... return hashlib.md5(t).hexdigest() \>\>\> con = sqlite3.connect(":memory:") \>\>\> con.create_function("md5", 1, md5sum) \>\>\> for row in con.execute("SELECT md5(?)", (b"foo",)): ... print(row) ('acbd18db4cc2f85cedef654fccc4a4d8',) \>\>\> con.close()

</div>

</div>

<div class="method">

create_aggregate(name, n_arg, aggregate_class, /)

Create or remove a user-defined SQL aggregate function.

param str name  
The name of the SQL aggregate function.

param int n_arg  
The number of arguments the SQL aggregate function can accept. If `-1`, it may take any number of arguments.

param aggregate_class  
A class must implement the following methods:

- `step()`: Add a row to the aggregate.
- `finalize()`: Return the final result of the aggregate as `a type natively supported by SQLite <sqlite3-types>`.

The number of arguments that the `step()` method must accept is controlled by *n_arg*.

Set to `None` to remove an existing SQL aggregate function.

type aggregate_class  
`class` \| None

<div class="versionchanged">

3.15 All three parameters are now positional-only.

</div>

Example:

<div class="testcode">

class MySum:  
def \_\_init\_\_(self):  
self.count = 0

def step(self, value):  
self.count += value

def finalize(self):  
return self.count

con = sqlite3.connect(":memory:") con.create_aggregate("mysum", 1, MySum) cur = con.execute("CREATE TABLE test(i)") cur.execute("INSERT INTO test(i) VALUES(1)") cur.execute("INSERT INTO test(i) VALUES(2)") cur.execute("SELECT mysum(i) FROM test") print(cur.fetchone()\[0\])

con.close()

</div>

<div class="testoutput" data-hide="">

3

</div>

</div>

<div class="method">

create_window_function(name, num_params, aggregate_class, /)

Create or remove a user-defined aggregate window function.

param str name  
The name of the SQL aggregate window function to create or remove.

param int num_params  
The number of arguments the SQL aggregate window function can accept. If `-1`, it may take any number of arguments.

param aggregate_class  
A class that must implement the following methods:

- `step()`: Add a row to the current window.
- `value()`: Return the current value of the aggregate.
- `inverse()`: Remove a row from the current window.
- `finalize()`: Return the final result of the aggregate as `a type natively supported by SQLite <sqlite3-types>`.

The number of arguments that the `step()` and `value()` methods must accept is controlled by *num_params*.

Set to `None` to remove an existing SQL aggregate window function.

raises NotSupportedError  
If used with a version of SQLite older than 3.25.0, which does not support aggregate window functions.

type aggregate_class  
`class` \| None

<div class="versionadded">

3.11

</div>

Example:

<div class="testcode">

\# Example taken from <https://www.sqlite.org/windowfunctions.html#udfwinfunc> class WindowSumInt: def \_\_init\_\_(self): self.count = 0

> def step(self, value):  
> """Add a row to the current window.""" self.count += value
>
> def value(self):  
> """Return the current value of the aggregate.""" return self.count
>
> def inverse(self, value):  
> """Remove a row from the current window.""" self.count -= value
>
> def finalize(self):  
> """Return the final value of the aggregate.
>
> Any clean-up actions should be placed here. """ return self.count

con = sqlite3.connect(":memory:") cur = con.execute("CREATE TABLE test(x, y)") values = \[ ("a", 4), ("b", 5), ("c", 3), ("d", 8), ("e", 1), \] cur.executemany("INSERT INTO test VALUES(?, ?)", values) con.create_window_function("sumint", 1, WindowSumInt) cur.execute(""" SELECT x, sumint(y) OVER ( ORDER BY x ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING ) AS sum_y FROM test ORDER BY x """) print(cur.fetchall()) con.close()

</div>

<div class="testoutput" data-hide="">

\[('a', 9), ('b', 12), ('c', 16), ('d', 12), ('e', 9)\]

</div>

</div>

<div class="method">

create_collation(name, callable, /)

Create a collation named *name* using the collating function *callable*. *callable* is passed two `string <str>` arguments, and it should return an `integer <int>`:

- `1` if the first is ordered higher than the second
- `-1` if the first is ordered lower than the second
- `0` if they are ordered equal

The following example shows a reverse sorting collation:

<div class="testcode">

def collate_reverse(string1, string2):  
if string1 == string2:  
return 0

elif string1 \< string2:  
return 1

else:  
return -1

con = sqlite3.connect(":memory:") con.create_collation("reverse", collate_reverse)

cur = con.execute("CREATE TABLE test(x)") cur.executemany("INSERT INTO test(x) VALUES(?)", \[("a",), ("b",)\]) cur.execute("SELECT x FROM test ORDER BY x COLLATE reverse") for row in cur: print(row) con.close()

</div>

<div class="testoutput" data-hide="">

('b',) ('a',)

</div>

Remove a collation function by setting *callable* to `None`.

<div class="versionchanged">

3.11 The collation name can contain any Unicode character. Earlier, only ASCII characters were allowed.

</div>

</div>

<div class="method">

interrupt()

Call this method from a different thread to abort any queries that might be executing on the connection. Aborted queries will raise an `OperationalError`.

</div>

<div class="method">

set_authorizer(authorizer_callback, /)

Register `callable` *authorizer_callback* to be invoked for each attempt to access a column of a table in the database. The callback should return one of `SQLITE_OK`, `SQLITE_DENY`, or `SQLITE_IGNORE` to signal how access to the column should be handled by the underlying SQLite library.

The first argument to the callback signifies what kind of operation is to be authorized. The second and third argument will be arguments or `None` depending on the first argument. The 4th argument is the name of the database ("main", "temp", etc.) if applicable. The 5th argument is the name of the inner-most trigger or view that is responsible for the access attempt or `None` if this access attempt is directly from input SQL code.

Please consult the SQLite documentation about the possible values for the first argument and the meaning of the second and third argument depending on the first one. All necessary constants are available in the `!sqlite3` module.

Passing `None` as *authorizer_callback* will disable the authorizer.

<div class="versionchanged">

3.11 Added support for disabling the authorizer using `None`.

</div>

<div class="versionchanged">

3.15 The only parameter is now positional-only.

</div>

</div>

<div class="method">

set_progress_handler(progress_handler, /, n)

Register `callable` *progress_handler* to be invoked for every *n* instructions of the SQLite virtual machine. This is useful if you want to get called from SQLite during long-running operations, for example to update a GUI.

If you want to clear any previously installed progress handler, call the method with `None` for *progress_handler*.

Returning a non-zero value from the handler function will terminate the currently executing query and cause it to raise a `DatabaseError` exception.

<div class="versionchanged">

3.15 The first parameter is now positional-only.

</div>

</div>

<div class="method">

set_trace_callback(trace_callback, /)

Register `callable` *trace_callback* to be invoked for each SQL statement that is actually executed by the SQLite backend.

The only argument passed to the callback is the statement (as `str`) that is being executed. The return value of the callback is ignored. Note that the backend does not only run statements passed to the `Cursor.execute` methods. Other sources include the `transaction management <sqlite3-controlling-transactions>` of the `!sqlite3` module and the execution of triggers defined in the current database.

Passing `None` as *trace_callback* will disable the trace callback.

> [!NOTE]
> Exceptions raised in the trace callback are not propagated. As a development and debugging aid, use `~sqlite3.enable_callback_tracebacks` to enable printing tracebacks from exceptions raised in the trace callback.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.15 The first parameter is now positional-only.

</div>

</div>

<div class="method">

enable_load_extension(enabled, /)

Enable the SQLite engine to load SQLite extensions from shared libraries if *enabled* is `True`; else, disallow loading SQLite extensions. SQLite extensions can define new functions, aggregates or whole new virtual table implementations. One well-known extension is the fulltext-search extension distributed with SQLite.

> [!NOTE]
> The `!sqlite3` module is not built with loadable extension support by default, because some platforms (notably macOS) have SQLite libraries which are compiled without this feature. To get loadable extension support, you must pass the `--enable-loadable-sqlite-extensions` option to `configure`.

<div class="audit-event">

sqlite3.enable_load_extension connection,enabled sqlite3.Connection.enable_load_extension

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.10 Added the `sqlite3.enable_load_extension` auditing event.

</div>

    con.enable_load_extension(True)

    # Load the fulltext search extension
    con.execute("select load_extension('./fts3.so')")

    # alternatively you can load the extension using an API call:
    # con.load_extension("./fts3.so")

    # disable extension loading again
    con.enable_load_extension(False)

    # example from SQLite wiki
    con.execute("CREATE VIRTUAL TABLE recipe USING fts3(name, ingredients)")
    con.executescript("""
        INSERT INTO recipe (name, ingredients) VALUES('broccoli stew', 'broccoli peppers cheese tomatoes');
        INSERT INTO recipe (name, ingredients) VALUES('pumpkin stew', 'pumpkin onions garlic celery');
        INSERT INTO recipe (name, ingredients) VALUES('broccoli pie', 'broccoli cheese onions flour');
        INSERT INTO recipe (name, ingredients) VALUES('pumpkin pie', 'pumpkin sugar flour butter');
        """)
    for row in con.execute("SELECT rowid, name, ingredients FROM recipe WHERE name MATCH 'pie'"):
        print(row)

</div>

<div class="method">

load_extension(path, /, \*, entrypoint=None)

Load an SQLite extension from a shared library. Enable extension loading with `enable_load_extension` before calling this method.

param str path  

> The path to the SQLite extension.

param entrypoint  

> Entry point name. If `None` (the default), SQLite will come up with an entry point name of its own; see the SQLite docs [Loading an Extension](https://www.sqlite.org/loadext.html#loading_an_extension) for details.

type entrypoint  
str \| None

<div class="audit-event">

sqlite3.load_extension connection,path sqlite3.Connection.load_extension

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.10 Added the `sqlite3.load_extension` auditing event.

</div>

<div class="versionchanged">

3.12 Added the *entrypoint* parameter.

</div>

</div>

<div class="method">

iterdump(\*, filter=None)

Return an `iterator` to dump the database as SQL source code. Useful when saving an in-memory database for later restoration. Similar to the `.dump` command in the `sqlite3` shell.

param filter  

> An optional `LIKE` pattern for database objects to dump, e.g. `prefix_%`. If `None` (the default), all database objects will be included.

type filter  
str \| None

Example:

<div class="testcode">

\# Convert file example.db to SQL dump file dump.sql con = sqlite3.connect('example.db') with open('dump.sql', 'w') as f: for line in con.iterdump(): f.write('%sn' % line) con.close()

</div>

<div class="seealso">

`sqlite3-howto-encoding`

</div>

<div class="versionchanged">

3.13 Added the *filter* parameter.

</div>

</div>

<div class="method">

backup(target, \*, pages=-1, progress=None, name="main", sleep=0.250)

Create a backup of an SQLite database.

Works even if the database is being accessed by other clients or concurrently by the same connection.

param ~sqlite3.Connection target  
The database connection to save the backup to.

param int pages  
The number of pages to copy at a time. If equal to or less than `0`, the entire database is copied in a single step. Defaults to `-1`.

param progress  
If set to a `callable`, it is invoked with three integer arguments for every backup iteration: the *status* of the last iteration, the *remaining* number of pages still to be copied, and the *total* number of pages. Defaults to `None`.

type progress  
`callback` \| None

param str name  
The name of the database to back up. Either `"main"` (the default) for the main database, `"temp"` for the temporary database, or the name of a custom database as attached using the `ATTACH DATABASE` SQL statement.

param float sleep  
The number of seconds to sleep between successive attempts to back up remaining pages.

Example 1, copy an existing database into another:

<div class="testcode">

def progress(status, remaining, total):  
print(f'Copied {total-remaining} of {total} pages...')

src = sqlite3.connect('example.db') dst = sqlite3.connect('backup.db') with dst: src.backup(dst, pages=1, progress=progress) dst.close() src.close()

</div>

<div class="testoutput" data-hide="">

Copied 0 of 0 pages...

</div>

Example 2, copy an existing database into a transient copy:

<div class="testcode">

src = sqlite3.connect('example.db') dst = sqlite3.connect(':memory:') src.backup(dst) dst.close() src.close()

</div>

<div class="versionadded">

3.7

</div>

<div class="seealso">

`sqlite3-howto-encoding`

</div>

</div>

<div class="method">

getlimit(category, /)

Get a connection runtime limit.

param int category  
The [SQLite limit category](https://www.sqlite.org/c3ref/c_limit_attached.html) to be queried.

rtype  
int

raises ProgrammingError  
If *category* is not recognised by the underlying SQLite library.

Example, query the maximum length of an SQL statement for `Connection` `con` (the default is 1000000000):

<div class="testsetup">

sqlite3.limits

import sqlite3 con = sqlite3.connect(":memory:") con.setlimit(sqlite3.SQLITE_LIMIT_SQL_LENGTH, 1_000_000_000) con.setlimit(sqlite3.SQLITE_LIMIT_ATTACHED, 10)

</div>

<div class="doctest">

sqlite3.limits

\>\>\> con.getlimit(sqlite3.SQLITE_LIMIT_SQL_LENGTH) 1000000000

</div>

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

setlimit(category, limit, /)

Set a connection runtime limit. Attempts to increase a limit above its hard upper bound are silently truncated to the hard upper bound. Regardless of whether or not the limit was changed, the prior value of the limit is returned.

param int category  
The [SQLite limit category](https://www.sqlite.org/c3ref/c_limit_attached.html) to be set.

param int limit  
The value of the new limit. If negative, the current limit is unchanged.

rtype  
int

raises ProgrammingError  
If *category* is not recognised by the underlying SQLite library.

Example, limit the number of attached databases to 1 for `Connection` `con` (the default limit is 10):

<div class="doctest">

sqlite3.limits

\>\>\> con.setlimit(sqlite3.SQLITE_LIMIT_ATTACHED, 1) 10 \>\>\> con.getlimit(sqlite3.SQLITE_LIMIT_ATTACHED) 1

</div>

<div class="testcleanup">

sqlite3.limits

con.close()

</div>

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

getconfig(op, /)

Query a boolean connection configuration option.

param int op  
A `SQLITE_DBCONFIG code <sqlite3-dbconfig-constants>`.

rtype  
bool

<div class="versionadded">

3.12

</div>

</div>

<div class="method">

setconfig(op, enable=True, /)

Set a boolean connection configuration option.

param int op  
A `SQLITE_DBCONFIG code <sqlite3-dbconfig-constants>`.

param bool enable  
`True` if the configuration option should be enabled (default); `False` if it should be disabled.

<div class="versionadded">

3.12

</div>

</div>

<div class="method">

serialize(\*, name="main")

Serialize a database into a `bytes` object. For an ordinary on-disk database file, the serialization is just a copy of the disk file. For an in-memory database or a "temp" database, the serialization is the same sequence of bytes which would be written to disk if that database were backed up to disk.

param str name  
The database name to be serialized. Defaults to `"main"`.

rtype  
bytes

> [!NOTE]
> This method is only available if the underlying SQLite library has the serialize API.

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

deserialize(data, /, \*, name="main")

Deserialize a `serialized <serialize>` database into a `Connection`. This method causes the database connection to disconnect from database *name*, and reopen *name* as an in-memory database based on the serialization contained in *data*.

param bytes data  
A serialized database.

param str name  
The database name to deserialize into. Defaults to `"main"`.

raises OperationalError  
If the database connection is currently involved in a read transaction or a backup operation.

raises DatabaseError  
If *data* does not contain a valid SQLite database.

raises OverflowError  
If `len(data) <len>` is larger than `2**63 - 1`.

> [!NOTE]
> This method is only available if the underlying SQLite library has the deserialize API.

<div class="versionadded">

3.11

</div>

</div>

<div class="attribute">

autocommit

This attribute controls `249`-compliant transaction behaviour. `!autocommit` has three allowed values:

- `False`: Select `249`-compliant transaction behaviour, implying that `!sqlite3` ensures a transaction is always open. Use `commit` and `rollback` to close transactions.

  This is the recommended value of `!autocommit`.

- `True`: Use SQLite's [autocommit mode](https://www.sqlite.org/lang_transaction.html#implicit_versus_explicit_transactions). `commit` and `rollback` have no effect in this mode.

- `LEGACY_TRANSACTION_CONTROL`: Pre-Python 3.12 (non-`249`-compliant) transaction control. See `isolation_level` for more details.

  This is currently the default value of `!autocommit`.

Changing `!autocommit` to `False` will open a new transaction, and changing it to `True` will commit any pending transaction.

See `sqlite3-transaction-control-autocommit` for more details.

> [!NOTE]
> The `isolation_level` attribute has no effect unless `autocommit` is `LEGACY_TRANSACTION_CONTROL`.

<div class="versionadded">

3.12

</div>

</div>

<div class="attribute">

in_transaction

This read-only attribute corresponds to the low-level SQLite [autocommit mode](https://www.sqlite.org/lang_transaction.html#implicit_versus_explicit_transactions).

`True` if a transaction is active (there are uncommitted changes), `False` otherwise.

<div class="versionadded">

3.2

</div>

</div>

<div class="attribute">

isolation_level

Controls the `legacy transaction handling mode
<sqlite3-transaction-control-isolation-level>` of `!sqlite3`. If set to `None`, transactions are never implicitly opened. If set to one of `"DEFERRED"`, `"IMMEDIATE"`, or `"EXCLUSIVE"`, corresponding to the underlying [SQLite transaction behaviour](https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions), `implicit transaction management
<sqlite3-transaction-control-isolation-level>` is performed.

If not overridden by the *isolation_level* parameter of `connect`, the default is `""`, which is an alias for `"DEFERRED"`.

> [!NOTE]
> Using `autocommit` to control transaction handling is recommended over using `!isolation_level`. `!isolation_level` has no effect unless `autocommit` is set to `LEGACY_TRANSACTION_CONTROL` (the default).

</div>

<div class="attribute">

row_factory

The initial `~Cursor.row_factory` for `Cursor` objects created from this connection. Assigning to this attribute does not affect the `!row_factory` of existing cursors belonging to this connection, only new ones. Is `None` by default, meaning each row is returned as a `tuple`.

See `sqlite3-howto-row-factory` for more details.

<div class="versionchanged">

3.16 Deleting the `row_factory` attribute is no longer allowed.

</div>

</div>

<div class="attribute">

text_factory

A `callable` that accepts a `bytes` parameter and returns a text representation of it. The callable is invoked for SQLite values with the `TEXT` data type. By default, this attribute is set to `str`.

See `sqlite3-howto-encoding` for more details.

<div class="versionchanged">

3.16 Deleting the `text_factory` attribute is no longer allowed.

</div>

</div>

<div class="attribute">

total_changes

Return the total number of database rows that have been modified, inserted, or deleted since the database connection was opened.

</div>

</div>

### Cursor objects

> A `Cursor` object represents a [database cursor](https://en.wikipedia.org/wiki/Cursor_(databases)) which is used to execute SQL statements, and manage the context of a fetch operation. Cursors are created using `Connection.cursor`, or by using any of the `connection shortcut methods
> <sqlite3-connection-shortcuts>`.
>
> Cursor objects are `iterators <iterator>`, meaning that if you `~Cursor.execute` a `SELECT` query, you can simply iterate over the cursor to fetch the resulting rows:
>
> <div class="testsetup">
>
> sqlite3.cursor
>
> import sqlite3 con = sqlite3.connect(":memory:", isolation_level=None) cur = con.execute("CREATE TABLE data(t)") cur.execute("INSERT INTO data VALUES(1)")
>
> </div>
>
> <div class="testcode">
>
> sqlite3.cursor
>
> for row in cur.execute("SELECT t FROM data"):  
> print(row)
>
> </div>
>
> <div class="testoutput" data-hide="">
>
> sqlite3.cursor
>
> (1,)
>
> </div>

<div class="Cursor">

A `Cursor` instance has the following attributes and methods.

<div class="index">

single: ? (question mark); in SQL statements

</div>

<div class="index">

single: : (colon); in SQL statements

</div>

<div class="method">

execute(sql, parameters=(), /)

Execute a single SQL statement, optionally binding Python values using `placeholders <sqlite3-placeholders>`.

param str sql  
A single SQL statement.

param parameters  
Python values to bind to placeholders in *sql*. A `!dict` if named placeholders are used. A `!sequence` if unnamed placeholders are used. See `sqlite3-placeholders`.

type parameters  
`dict` \| `sequence`

raises ProgrammingError  
When *sql* contains more than one SQL statement. When `named placeholders <sqlite3-placeholders>` are used and *parameters* is a sequence instead of a `dict`.

If `~Connection.autocommit` is `LEGACY_TRANSACTION_CONTROL`, `~Connection.isolation_level` is not `None`, *sql* is an `INSERT`, `UPDATE`, `DELETE`, or `REPLACE` statement, and there is no open transaction, a transaction is implicitly opened before executing *sql*.

<div class="versionchanged">

3.14

`ProgrammingError` is emitted if `named placeholders <sqlite3-placeholders>` are used and *parameters* is a sequence instead of a `dict`.

</div>

Use `executescript` to execute multiple SQL statements.

</div>

<div class="method">

executemany(sql, parameters, /)

For every item in *parameters*, repeatedly execute the `parameterized <sqlite3-placeholders>` `DML (Data Manipulation Language)` SQL statement *sql*.

Uses the same implicit transaction handling as `~Cursor.execute`.

param str sql  
A single SQL DML statement.

param parameters  
An `!iterable` of parameters to bind with the placeholders in *sql*. See `sqlite3-placeholders`.

type parameters  
`iterable`

raises ProgrammingError  
When *sql* contains more than one SQL statement or is not a DML statement, When `named placeholders <sqlite3-placeholders>` are used and the items in *parameters* are sequences instead of `dict`s.

Example:

<div class="testcode">

sqlite3.cursor

rows = \[  
("row1",), ("row2",),

\] \# cur is an sqlite3.Cursor object cur.executemany("INSERT INTO data VALUES(?)", rows)

</div>

<div class="testcleanup">

sqlite3.cursor

con.close()

</div>

> [!NOTE]
> Any resulting rows are discarded, including DML statements with [RETURNING clauses](https://www.sqlite.org/lang_returning.html).

<div class="versionchanged">

3.14

`ProgrammingError` is emitted if `named placeholders <sqlite3-placeholders>` are used and the items in *parameters* are sequences instead of `dict`s.

</div>

</div>

<div class="method">

executescript(sql_script, /)

Execute the SQL statements in *sql_script*. If the `~Connection.autocommit` is `LEGACY_TRANSACTION_CONTROL` and there is a pending transaction, an implicit `COMMIT` statement is executed first. No other implicit transaction control is performed; any transaction control must be added to *sql_script*.

*sql_script* must be a `string <str>`.

Example:

<div class="testcode">

sqlite3.cursor

\# cur is an sqlite3.Cursor object cur.executescript(""" BEGIN; CREATE TABLE person(firstname, lastname, age); CREATE TABLE book(title, author, published); CREATE TABLE publisher(name, address); COMMIT; """)

</div>

</div>

<div class="method">

fetchone()

If `~Cursor.row_factory` is `None`, return the next row query result set as a `tuple`. Else, pass it to the row factory and return its result. Return `None` if no more data is available.

</div>

<div class="method">

fetchmany(size=cursor.arraysize)

Return the next set of rows of a query result as a `list`. Return an empty list if no more rows are available.

The number of rows to fetch per call is specified by the *size* parameter. If *size* is not given, `arraysize` determines the number of rows to be fetched. If fewer than *size* rows are available, as many rows as are available are returned.

Note there are performance considerations involved with the *size* parameter. For optimal performance, it is usually best to use the arraysize attribute. If the *size* parameter is used, then it is best for it to retain the same value from one `fetchmany` call to the next.

<div class="versionchanged">

3.15 Negative *size* values are rejected by raising `ValueError`.

</div>

</div>

<div class="method">

fetchall()

Return all (remaining) rows of a query result as a `list`. Return an empty list if no rows are available. Note that the `arraysize` attribute can affect the performance of this operation.

</div>

<div class="method">

close()

Close the cursor now (rather than whenever `__del__` is called).

The cursor will be unusable from this point forward; a `ProgrammingError` exception will be raised if any operation is attempted with the cursor.

</div>

<div class="method">

setinputsizes(sizes, /)

Required by the DB-API. Does nothing in `!sqlite3`.

</div>

<div class="method">

setoutputsize(size, column=None, /)

Required by the DB-API. Does nothing in `!sqlite3`.

</div>

<div class="attribute">

arraysize

Read/write attribute that controls the number of rows returned by `fetchmany`. The default value is 1 which means a single row would be fetched per call.

<div class="versionchanged">

3.15 Negative values are rejected by raising `ValueError`.

</div>

</div>

<div class="attribute">

connection

Read-only attribute that provides the SQLite database `Connection` belonging to the cursor. A `Cursor` object created by calling `con.cursor() <Connection.cursor>` will have a `connection` attribute that refers to *con*:

<div class="doctest">

\>\>\> con = sqlite3.connect(":memory:") \>\>\> cur = con.cursor() \>\>\> cur.connection == con True \>\>\> con.close()

</div>

</div>

<div class="attribute">

description

Read-only attribute that provides the column names of the last query. To remain compatible with the Python DB API, it returns a 7-tuple for each column where the last six items of each tuple are `None`.

It is set for `SELECT` statements without any matching rows as well.

</div>

<div class="attribute">

lastrowid

Read-only attribute that provides the row id of the last inserted row. It is only updated after successful `INSERT` or `REPLACE` statements using the `execute` method. For other statements, after `executemany` or `executescript`, or if the insertion failed, the value of `lastrowid` is left unchanged. The initial value of `lastrowid` is `None`.

> [!NOTE]
> Inserts into `WITHOUT ROWID` tables are not recorded.

<div class="versionchanged">

3.6 Added support for the `REPLACE` statement.

</div>

</div>

<div class="attribute">

rowcount

Read-only attribute that provides the number of modified rows for `INSERT`, `UPDATE`, `DELETE`, and `REPLACE` statements; is `-1` for other statements, including `CTE (Common Table Expression)` queries. It is only updated by the `execute` and `executemany` methods, after the statement has run to completion. This means that any resulting rows must be fetched in order for `!rowcount` to be updated.

</div>

<div class="attribute">

row_factory

Control how a row fetched from this `!Cursor` is represented. If `None`, a row is represented as a `tuple`. Can be set to the included `sqlite3.Row`; or a `callable` that accepts two arguments, a `Cursor` object and the `!tuple` of row values, and returns a custom object representing an SQLite row.

Defaults to what `Connection.row_factory` was set to when the `!Cursor` was created. Assigning to this attribute does not affect `Connection.row_factory` of the parent connection.

See `sqlite3-howto-row-factory` for more details.

<div class="versionchanged">

3.16 Deleting the `row_factory` attribute is no longer allowed.

</div>

</div>

</div>

### Row objects<span id="sqlite3-columns-by-name"></span>

<div class="Row">

A `!Row` instance serves as a highly optimized `~Connection.row_factory` for `Connection` objects. It supports iteration, equality testing, `len`, and `mapping` access by column name and index.

Two `!Row` objects compare equal if they have identical column names and values.

See `sqlite3-howto-row-factory` for more details.

<div class="method">

keys

Return a `list` of column names as `strings <str>`. Immediately after a query, it is the first member of each tuple in `Cursor.description`.

</div>

<div class="versionchanged">

3.5 Added support of slicing.

</div>

</div>

### Blob objects

<div class="Blob">

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.16 `Blob` now supports negative-step slices (e.g. `blob[9:0:-2]`) for both reading and writing.

</div>

A `Blob` instance is a `file-like object` that can read and write data in an SQLite `BLOB (Binary Large OBject)`. Call `len(blob) <len>` to get the size (number of bytes) of the blob. Use indices and `slices <slice>` for direct access to the blob data.

Use the `Blob` as a `context manager` to ensure that the blob handle is closed after use.

<div class="testcode">

con = sqlite3.connect(":memory:") con.execute("CREATE TABLE test(blob_col blob)") con.execute("INSERT INTO test(blob_col) VALUES(zeroblob(13))")

\# Write to our blob, using two write operations: with con.blobopen("test", "blob_col", 1) as blob: blob.write(b"hello, ") blob.write(b"world.") \# Modify the first and last bytes of our blob blob\[0\] = ord("H") blob\[-1\] = ord("!")

\# Read the contents of our blob with con.blobopen("test", "blob_col", 1) as blob: greeting = blob.read()

print(greeting) \# outputs "b'Hello, world!'" con.close()

</div>

<div class="testoutput" data-hide="">

b'Hello, world!'

</div>

<div class="method">

close()

Close the blob.

The blob will be unusable from this point onward. An `~sqlite3.Error` (or subclass) exception will be raised if any further operation is attempted with the blob.

</div>

<div class="method">

read(length=-1, /)

Read *length* bytes of data from the blob at the current offset position. If the end of the blob is reached, the data up to `EOF (End of File)` will be returned. When *length* is not specified, or is negative, `~Blob.read` will read until the end of the blob.

</div>

<div class="method">

write(data, /)

Write *data* to the blob at the current offset. This function cannot change the blob length. Writing beyond the end of the blob will raise `ValueError`.

</div>

<div class="method">

tell()

Return the current access position of the blob.

</div>

<div class="method">

seek(offset, origin=os.SEEK_SET, /)

Set the current access position of the blob to *offset*. The *origin* argument defaults to `os.SEEK_SET` (absolute blob positioning). Other values for *origin* are `os.SEEK_CUR` (seek relative to the current position) and `os.SEEK_END` (seek relative to the blob’s end).

</div>

</div>

### PrepareProtocol objects

<div class="PrepareProtocol">

The PrepareProtocol type's single purpose is to act as a `246` style adaption protocol for objects that can `adapt themselves
<sqlite3-conform>` to `native SQLite types <sqlite3-types>`.

</div>

### Exceptions

The exception hierarchy is defined by the DB-API 2.0 (`249`).

<div class="exception">

Warning

This exception is not currently raised by the `!sqlite3` module, but may be raised by applications using `!sqlite3`, for example if a user-defined function truncates data while inserting. `Warning` is a subclass of `Exception`.

</div>

<div class="exception">

Error

The base class of the other exceptions in this module. Use this to catch all errors with one single `except` statement. `Error` is a subclass of `Exception`.

If the exception originated from within the SQLite library, the following two attributes are added to the exception:

<div class="attribute">

sqlite_errorcode

The numeric error code from the [SQLite API](https://sqlite.org/rescode.html)

<div class="versionadded">

3.11

</div>

</div>

<div class="attribute">

sqlite_errorname

The symbolic name of the numeric error code from the [SQLite API](https://sqlite.org/rescode.html)

<div class="versionadded">

3.11

</div>

</div>

</div>

<div class="exception">

InterfaceError

Exception raised for misuse of the low-level SQLite C API. In other words, if this exception is raised, it probably indicates a bug in the `!sqlite3` module. `InterfaceError` is a subclass of `Error`.

</div>

<div class="exception">

DatabaseError

Exception raised for errors that are related to the database. This serves as the base exception for several types of database errors. It is only raised implicitly through the specialised subclasses. `DatabaseError` is a subclass of `Error`.

</div>

<div class="exception">

DataError

Exception raised for errors caused by problems with the processed data, like numeric values out of range, and strings which are too long. `DataError` is a subclass of `DatabaseError`.

</div>

<div class="exception">

OperationalError

Exception raised for errors that are related to the database's operation, and not necessarily under the control of the programmer. For example, the database path is not found, or a transaction could not be processed. `OperationalError` is a subclass of `DatabaseError`.

</div>

<div class="exception">

IntegrityError

Exception raised when the relational integrity of the database is affected, e.g. a foreign key check fails. It is a subclass of `DatabaseError`.

</div>

<div class="exception">

InternalError

Exception raised when SQLite encounters an internal error. If this is raised, it may indicate that there is a problem with the runtime SQLite library. `InternalError` is a subclass of `DatabaseError`.

</div>

<div class="exception">

ProgrammingError

Exception raised for `!sqlite3` API programming errors, for example supplying the wrong number of bindings to a query, or trying to operate on a closed `Connection`. `ProgrammingError` is a subclass of `DatabaseError`.

</div>

<div class="exception">

NotSupportedError

Exception raised in case a method or database API is not supported by the underlying SQLite library. For example, setting *deterministic* to `True` in `~Connection.create_function`, if the underlying SQLite library does not support deterministic functions. `NotSupportedError` is a subclass of `DatabaseError`.

</div>

### SQLite and Python types

SQLite natively supports the following types: `NULL`, `INTEGER`, `REAL`, `TEXT`, `BLOB`.

The following Python types can thus be sent to SQLite without any problem:

| Python type | SQLite type |
|-------------|-------------|
| `None`      | `NULL`      |
| `int`       | `INTEGER`   |
| `float`     | `REAL`      |
| `str`       | `TEXT`      |
| `bytes`     | `BLOB`      |

This is how SQLite types are converted to Python types by default:

| SQLite type | Python type                                             |
|-------------|---------------------------------------------------------|
| `NULL`      | `None`                                                  |
| `INTEGER`   | `int`                                                   |
| `REAL`      | `float`                                                 |
| `TEXT`      | depends on `~Connection.text_factory`, `str` by default |
| `BLOB`      | `bytes`                                                 |

The type system of the `!sqlite3` module is extensible in two ways: you can store additional Python types in an SQLite database via `object adapters <sqlite3-adapters>`, and you can let the `!sqlite3` module convert SQLite types to Python types via `converters <sqlite3-converters>`.

### Default adapters and converters (deprecated)

> [!NOTE]
> The default adapters and converters are deprecated as of Python 3.12. Instead, use the `sqlite3-adapter-converter-recipes` and tailor them to your needs.

The deprecated default adapters and converters consist of:

- An adapter for `datetime.date` objects to `strings <str>` in [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) format.
- An adapter for `datetime.datetime` objects to strings in ISO 8601 format.
- A converter for `declared <sqlite3-converters>` "date" types to `datetime.date` objects.
- A converter for declared "timestamp" types to `datetime.datetime` objects. Fractional parts will be truncated to 6 digits (microsecond precision).

> [!NOTE]
> The default "timestamp" converter ignores UTC offsets in the database and always returns a naive `datetime.datetime` object. To preserve UTC offsets in timestamps, either leave converters disabled, or register an offset-aware converter with `register_converter`.

<div class="deprecated">

3.12

</div>

### Command-line interface

The `!sqlite3` module can be invoked as a script, using the interpreter's `-m` switch, in order to provide a simple SQLite shell. The argument signature is as follows:

    python -m sqlite3 [-h] [-v] [filename] [sql]

Type `.quit` or CTRL-D to exit the shell.

<div class="program">

python -m sqlite3 \[-h\] \[-v\] \[filename\] \[sql\]

</div>

<div class="option">

`-h, --help`  

Print CLI help.

</div>

<div class="option">

`-v, --version`  

Print underlying SQLite library version.

</div>

<div class="versionadded">

3.12

</div>

## How-to guides

### How to use placeholders to bind values in SQL queries

SQL operations usually need to use values from Python variables. However, beware of using Python's string operations to assemble queries, as they are vulnerable to [SQL injection attacks](https://en.wikipedia.org/wiki/SQL_injection). For example, an attacker can simply close the single quote and inject `OR TRUE` to select all rows:

    >>> # Never do this -- insecure!
    >>> symbol = input()
    ' OR TRUE; --
    >>> sql = "SELECT * FROM stocks WHERE symbol = '%s'" % symbol
    >>> print(sql)
    SELECT * FROM stocks WHERE symbol = '' OR TRUE; --'
    >>> cur.execute(sql)

Instead, use the DB-API's parameter substitution. To insert a variable into a query string, use a placeholder in the string, and substitute the actual values into the query by providing them as a `tuple` of values to the second argument of the cursor's `~Cursor.execute` method.

An SQL statement may use one of two kinds of placeholders: question marks (qmark style) or named placeholders (named style). For the qmark style, *parameters* must be a `sequence` whose length must match the number of placeholders, or a `ProgrammingError` is raised. For the named style, *parameters* must be an instance of a `dict` (or a subclass), which must contain keys for all named parameters; any extra items are ignored. Here's an example of both styles:

<div class="testcode">

con = sqlite3.connect(":memory:") cur = con.execute("CREATE TABLE lang(name, first_appeared)")

\# This is the named style used with executemany(): data = ( {"name": "C", "year": 1972}, {"name": "Fortran", "year": 1957}, {"name": "Python", "year": 1991}, {"name": "Go", "year": 2009}, ) cur.executemany("INSERT INTO lang VALUES(:name, :year)", data)

\# This is the qmark style used in a SELECT query: params = (1972,) cur.execute("SELECT \* FROM lang WHERE first_appeared = ?", params) print(cur.fetchall()) con.close()

</div>

<div class="testoutput" data-hide="">

\[('C', 1972)\]

</div>

> [!NOTE]
> `249` numeric placeholders are *not* supported. If used, they will be interpreted as named placeholders.

### How to adapt custom Python types to SQLite values

SQLite supports only a limited set of data types natively. To store custom Python types in SQLite databases, *adapt* them to one of the `Python types SQLite natively understands <sqlite3-types>`.

There are two ways to adapt Python objects to SQLite types: letting your object adapt itself, or using an *adapter callable*. The latter will take precedence above the former. For a library that exports a custom type, it may make sense to enable that type to adapt itself. As an application developer, it may make more sense to take direct control by registering custom adapter functions.

#### How to write adaptable objects

Suppose we have a `!Point` class that represents a pair of coordinates, `x` and `y`, in a Cartesian coordinate system. The coordinate pair will be stored as a text string in the database, using a semicolon to separate the coordinates. This can be implemented by adding a `__conform__(self, protocol)` method which returns the adapted value. The object passed to *protocol* will be of type `PrepareProtocol`.

<div class="testcode">

class Point:  
def \_\_init\_\_(self, x, y):  
self.x, self.y = x, y

def \_\_conform\_\_(self, protocol):  
if protocol is sqlite3.PrepareProtocol:  
return f"{self.x};{self.y}"

con = sqlite3.connect(":memory:") cur = con.cursor()

cur.execute("SELECT ?", (Point(4.0, -3.2),)) print(cur.fetchone()\[0\]) con.close()

</div>

<div class="testoutput" data-hide="">

4.0;-3.2

</div>

#### How to register adapter callables

The other possibility is to create a function that converts the Python object to an SQLite-compatible type. This function can then be registered using `register_adapter`.

<div class="testcode">

class Point:  
def \_\_init\_\_(self, x, y):  
self.x, self.y = x, y

def adapt_point(point):  
return f"{point.x};{point.y}"

sqlite3.register_adapter(Point, adapt_point)

con = sqlite3.connect(":memory:") cur = con.cursor()

cur.execute("SELECT ?", (Point(1.0, 2.5),)) print(cur.fetchone()\[0\]) con.close()

</div>

<div class="testoutput" data-hide="">

1.0;2.5

</div>

### How to convert SQLite values to custom Python types

Writing an adapter lets you convert *from* custom Python types *to* SQLite values. To be able to convert *from* SQLite values *to* custom Python types, we use *converters*.

Let's go back to the `!Point` class. We stored the x and y coordinates separated via semicolons as strings in SQLite.

First, we'll define a converter function that accepts the string as a parameter and constructs a `!Point` object from it.

> [!NOTE]
> Converter functions are **always** passed a `bytes` object, no matter the underlying SQLite data type.

<div class="testcode">

def convert_point(s):  
x, y = map(float, s.split(b";")) return Point(x, y)

</div>

We now need to tell `!sqlite3` when it should convert a given SQLite value. This is done when connecting to a database, using the *detect_types* parameter of `connect`. There are three options:

- Implicit: set *detect_types* to `PARSE_DECLTYPES`
- Explicit: set *detect_types* to `PARSE_COLNAMES`
- Both: set *detect_types* to `sqlite3.PARSE_DECLTYPES | sqlite3.PARSE_COLNAMES`. Column names take precedence over declared types.

The following example illustrates the implicit and explicit approaches:

<div class="testcode">

class Point:  
def \_\_init\_\_(self, x, y):  
self.x, self.y = x, y

def \_\_repr\_\_(self):  
return f"Point({self.x}, {self.y})"

def adapt_point(point):  
return f"{point.x};{point.y}"

def convert_point(s):  
x, y = list(map(float, s.split(b";"))) return Point(x, y)

\# Register the adapter and converter sqlite3.register_adapter(Point, adapt_point) sqlite3.register_converter("point", convert_point)

\# 1) Parse using declared types p = Point(4.0, -3.2) con = sqlite3.connect(":memory:", detect_types=sqlite3.PARSE_DECLTYPES) cur = con.execute("CREATE TABLE test(p point)")

cur.execute("INSERT INTO test(p) VALUES(?)", (p,)) cur.execute("SELECT p FROM test") print("with declared types:", cur.fetchone()\[0\]) cur.close() con.close()

\# 2) Parse using column names con = sqlite3.connect(":memory:", detect_types=sqlite3.PARSE_COLNAMES) cur = con.execute("CREATE TABLE test(p)")

cur.execute("INSERT INTO test(p) VALUES(?)", (p,)) cur.execute('SELECT p AS "p \[point\]" FROM test') print("with column names:", cur.fetchone()\[0\]) cur.close() con.close()

</div>

<div class="testoutput" data-hide="">

with declared types: Point(4.0, -3.2) with column names: Point(4.0, -3.2)

</div>

### Adapter and converter recipes

This section shows recipes for common adapters and converters.

<div class="testcode">

import datetime as dt import sqlite3

def adapt_date_iso(val):  
"""Adapt datetime.date to ISO 8601 date.""" return val.isoformat()

def adapt_datetime_iso(val):  
"""Adapt datetime.datetime to timezone-naive ISO 8601 date.""" return val.replace(tzinfo=None).isoformat()

def adapt_datetime_epoch(val):  
"""Adapt datetime.datetime to Unix timestamp.""" return int(val.timestamp())

sqlite3.register_adapter(dt.date, adapt_date_iso) sqlite3.register_adapter(dt.datetime, adapt_datetime_iso) sqlite3.register_adapter(dt.datetime, adapt_datetime_epoch)

def convert_date(val):  
"""Convert ISO 8601 date to datetime.date object.""" return dt.date.fromisoformat(val.decode())

def convert_datetime(val):  
"""Convert ISO 8601 datetime to datetime.datetime object.""" return dt.datetime.fromisoformat(val.decode())

def convert_timestamp(val):  
"""Convert Unix epoch timestamp to datetime.datetime object.""" return dt.datetime.fromtimestamp(int(val))

sqlite3.register_converter("date", convert_date) sqlite3.register_converter("datetime", convert_datetime) sqlite3.register_converter("timestamp", convert_timestamp)

</div>

<div class="testcode" data-hide="">

when = dt.datetime(2019, 5, 18, 15, 17, 8, 123456)

assert adapt_date_iso(when.date()) == "2019-05-18" assert convert_date(b"2019-05-18") == when.date()

assert adapt_datetime_iso(when) == "2019-05-18T15:17:08.123456" assert convert_datetime(b"2019-05-18T15:17:08.123456") == when

\# Using current time as fromtimestamp() returns local date/time. \# Dropping microseconds as adapt_datetime_epoch truncates fractional second part. now = dt.datetime.now().replace(microsecond=0) current_timestamp = int(now.timestamp())

assert adapt_datetime_epoch(now) == current_timestamp assert convert_timestamp(str(current_timestamp).encode()) == now

</div>

### How to use connection shortcut methods

Using the `~Connection.execute`, `~Connection.executemany`, and `~Connection.executescript` methods of the `Connection` class, your code can be written more concisely because you don't have to create the (often superfluous) `Cursor` objects explicitly. Instead, the `Cursor` objects are created implicitly and these shortcut methods return the cursor objects. This way, you can execute a `SELECT` statement and iterate over it directly using only a single call on the `Connection` object.

<div class="testcode">

\# Create and fill the table. con = sqlite3.connect(":memory:") con.execute("CREATE TABLE lang(name, first_appeared)") data = \[ ("C++", 1985), ("Objective-C", 1984), \] con.executemany("INSERT INTO lang(name, first_appeared) VALUES(?, ?)", data)

\# Print the table contents for row in con.execute("SELECT name, first_appeared FROM lang"): print(row)

print("I just deleted", con.execute("DELETE FROM lang").rowcount, "rows")

\# close() is not a shortcut method and it's not called automatically; \# the connection object should be closed manually con.close()

</div>

<div class="testoutput" data-hide="">

('C++', 1985) ('Objective-C', 1984) I just deleted 2 rows

</div>

### How to use the connection context manager

A `Connection` object can be used as a context manager that automatically commits or rolls back open transactions when leaving the body of the context manager. If the body of the `with` statement finishes without exceptions, the transaction is committed. If this commit fails, or if the body of the `with` statement raises an uncaught exception, the transaction is rolled back. If `~Connection.autocommit` is `False`, a new transaction is implicitly opened after committing or rolling back.

If there is no open transaction upon leaving the body of the `with` statement, or if `~Connection.autocommit` is `True`, the context manager does nothing.

> [!NOTE]
> The context manager neither implicitly opens a new transaction nor closes the connection. If you need a closing context manager, consider using `contextlib.closing`.

<div class="testcode">

con = sqlite3.connect(":memory:") con.execute("CREATE TABLE lang(id INTEGER PRIMARY KEY, name VARCHAR UNIQUE)")

\# Successful, con.commit() is called automatically afterwards with con: con.execute("INSERT INTO lang(name) VALUES(?)", ("Python",))

\# con.rollback() is called after the with block finishes with an exception, \# the exception is still raised and must be caught try: with con: con.execute("INSERT INTO lang(name) VALUES(?)", ("Python",)) except sqlite3.IntegrityError: print("couldn't add Python twice")

\# Connection object used as context manager only commits or rollbacks transactions, \# so the connection object should be closed manually con.close()

</div>

<div class="testoutput" data-hide="">

couldn't add Python twice

</div>

### How to work with SQLite URIs

Some useful URI tricks include:

- Open a database in read-only mode:

<div class="doctest">

\>\>\> con = sqlite3.connect("file:tutorial.db?mode=ro", uri=True) \>\>\> con.execute("CREATE TABLE readonly(data)") Traceback (most recent call last): OperationalError: attempt to write a readonly database \>\>\> con.close()

</div>

- Do not implicitly create a new database file if it does not already exist; will raise `~sqlite3.OperationalError` if unable to create a new file:

<div class="doctest">

\>\>\> con = sqlite3.connect("file:nosuchdb.db?mode=rw", uri=True) Traceback (most recent call last): OperationalError: unable to open database file

</div>

- Create a shared named in-memory database:

<div class="testcode">

db = "file:mem1?mode=memory&cache=shared" con1 = sqlite3.connect(db, uri=True) con2 = sqlite3.connect(db, uri=True) with con1: con1.execute("CREATE TABLE shared(data)") con1.execute("INSERT INTO shared VALUES(28)") res = con2.execute("SELECT data FROM shared") assert res.fetchone() == (28,)

con1.close() con2.close()

</div>

More information about this feature, including a list of parameters, can be found in the [SQLite URI documentation](https://www.sqlite.org/uri.html).

### How to create and use row factories

By default, `!sqlite3` represents each row as a `tuple`. If a `!tuple` does not suit your needs, you can use the `sqlite3.Row` class or a custom `~Cursor.row_factory`.

While `!row_factory` exists as an attribute both on the `Cursor` and the `Connection`, it is recommended to set `Connection.row_factory`, so all cursors created from the connection will use the same row factory.

`!Row` provides indexed and case-insensitive named access to columns, with minimal memory overhead and performance impact over a `!tuple`. To use `!Row` as a row factory, assign it to the `!row_factory` attribute:

<div class="doctest">

\>\>\> con = sqlite3.connect(":memory:") \>\>\> con.row_factory = sqlite3.Row

</div>

Queries now return `!Row` objects:

<div class="doctest">

\>\>\> res = con.execute("SELECT 'Earth' AS name, 6378 AS radius") \>\>\> row = res.fetchone() \>\>\> row.keys() \['name', 'radius'\] \>\>\> row\[0\] \# Access by index. 'Earth' \>\>\> row\["name"\] \# Access by name. 'Earth' \>\>\> row\["RADIUS"\] \# Column names are case-insensitive. 6378 \>\>\> con.close()

</div>

> [!NOTE]
> The `FROM` clause can be omitted in the `SELECT` statement, as in the above example. In such cases, SQLite returns a single row with columns defined by expressions, e.g. literals, with the given aliases `expr AS alias`.

You can create a custom `~Cursor.row_factory` that returns each row as a `dict`, with column names mapped to values:

<div class="testcode">

def dict_factory(cursor, row):  
fields = \[column\[0\] for column in cursor.description\] return {key: value for key, value in zip(fields, row)}

</div>

Using it, queries now return a `!dict` instead of a `!tuple`:

<div class="doctest">

\>\>\> con = sqlite3.connect(":memory:") \>\>\> con.row_factory = dict_factory \>\>\> for row in con.execute("SELECT 1 AS a, 2 AS b"): ... print(row) {'a': 1, 'b': 2} \>\>\> con.close()

</div>

The following row factory returns a `named tuple`:

<div class="testcode">

from collections import namedtuple

def namedtuple_factory(cursor, row):  
fields = \[column\[0\] for column in cursor.description\] cls = namedtuple("Row", fields) return cls.\_make(row)

</div>

`!namedtuple_factory` can be used as follows:

<div class="doctest">

\>\>\> con = sqlite3.connect(":memory:") \>\>\> con.row_factory = namedtuple_factory \>\>\> cur = con.execute("SELECT 1 AS a, 2 AS b") \>\>\> row = cur.fetchone() \>\>\> row Row(a=1, b=2) \>\>\> row\[0\] \# Indexed access. 1 \>\>\> row.b \# Attribute access. 2 \>\>\> con.close()

</div>

With some adjustments, the above recipe can be adapted to use a `~dataclasses.dataclass`, or any other custom class, instead of a `~collections.namedtuple`.

### How to handle non-UTF-8 text encodings

By default, `!sqlite3` uses `str` to adapt SQLite values with the `TEXT` data type. This works well for UTF-8 encoded text, but it might fail for other encodings and invalid UTF-8. You can use a custom `~Connection.text_factory` to handle such cases.

Because of SQLite's [flexible typing](https://www.sqlite.org/flextypegood.html), it is not uncommon to encounter table columns with the `TEXT` data type containing non-UTF-8 encodings, or even arbitrary data. To demonstrate, let's assume we have a database with ISO-8859-2 (Latin-2) encoded text, for example a table of Czech-English dictionary entries. Assuming we now have a `Connection` instance `!con` connected to this database, we can decode the Latin-2 encoded text using this `~Connection.text_factory`:

<div class="testcode">

con.text_factory = lambda data: str(data, encoding="latin2")

</div>

For invalid UTF-8 or arbitrary data in stored in `TEXT` table columns, you can use the following technique, borrowed from the `unicode-howto`:

<div class="testcode">

con.text_factory = lambda data: str(data, errors="surrogateescape")

</div>

> [!NOTE]
> The `!sqlite3` module API does not support strings containing surrogates.

<div class="seealso">

`unicode-howto`

</div>

## Explanation

### Transaction control<span id="sqlite3-transaction-control"></span>

`!sqlite3` offers multiple methods of controlling whether, when and how database transactions are opened and closed. `sqlite3-transaction-control-autocommit` is recommended, while `sqlite3-transaction-control-isolation-level` retains the pre-Python 3.12 behaviour.

#### Transaction control via the `autocommit` attribute

The recommended way of controlling transaction behaviour is through the `Connection.autocommit` attribute, which should preferably be set using the *autocommit* parameter of `connect`.

It is suggested to set *autocommit* to `False`, which implies `249`-compliant transaction control. This means:

- `!sqlite3` ensures that a transaction is always open, so `connect`, `Connection.commit`, and `Connection.rollback` will implicitly open a new transaction (immediately after closing the pending one, for the latter two). `!sqlite3` uses `BEGIN DEFERRED` statements when opening transactions.
- Transactions should be committed explicitly using `!commit`.
- Transactions should be rolled back explicitly using `!rollback`.
- An implicit rollback is performed if the database is `~Connection.close`-ed with pending changes.

Set *autocommit* to `True` to enable SQLite's [autocommit mode](https://www.sqlite.org/lang_transaction.html#implicit_versus_explicit_transactions). In this mode, `Connection.commit` and `Connection.rollback` have no effect. Note that SQLite's autocommit mode is distinct from the `249`-compliant `Connection.autocommit` attribute; use `Connection.in_transaction` to query the low-level SQLite autocommit mode.

Set *autocommit* to `LEGACY_TRANSACTION_CONTROL` to leave transaction control behaviour to the `Connection.isolation_level` attribute. See `sqlite3-transaction-control-isolation-level` for more information.

#### Transaction control via the `isolation_level` attribute

> [!NOTE]
> The recommended way of controlling transactions is via the `~Connection.autocommit` attribute. See `sqlite3-transaction-control-autocommit`.

If `Connection.autocommit` is set to `LEGACY_TRANSACTION_CONTROL` (the default), transaction behaviour is controlled using the `Connection.isolation_level` attribute. Otherwise, `!isolation_level` has no effect.

If the connection attribute `~Connection.isolation_level` is not `None`, new transactions are implicitly opened before `~Cursor.execute` and `~Cursor.executemany` executes `INSERT`, `UPDATE`, `DELETE`, or `REPLACE` statements; for other statements, no implicit transaction handling is performed. Use the `~Connection.commit` and `~Connection.rollback` methods to respectively commit and roll back pending transactions. You can choose the underlying [SQLite transaction behaviour](https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions) — that is, whether and what type of `BEGIN` statements `!sqlite3` implicitly executes – via the `~Connection.isolation_level` attribute.

If `~Connection.isolation_level` is set to `None`, no transactions are implicitly opened at all. This leaves the underlying SQLite library in [autocommit mode](https://www.sqlite.org/lang_transaction.html#implicit_versus_explicit_transactions), but also allows the user to perform their own transaction handling using explicit SQL statements. The underlying SQLite library autocommit mode can be queried using the `~Connection.in_transaction` attribute.

The `~Cursor.executescript` method implicitly commits any pending transaction before execution of the given SQL script, regardless of the value of `~Connection.isolation_level`.

<div class="versionchanged">

3.6 `!sqlite3` used to implicitly commit an open transaction before DDL statements. This is no longer the case.

</div>

<div class="versionchanged">

3.12 The recommended way of controlling transactions is now via the `~Connection.autocommit` attribute.

</div>

<div class="testcleanup">

import os os.remove("backup.db") os.remove("dump.sql") os.remove("example.db") os.remove("tutorial.db")

</div>

# `!ssl` --- TLS/SSL wrapper for socket objects

<div class="module" data-synopsis="TLS/SSL wrapper for socket objects">

ssl

</div>

**Source code:** `Lib/ssl.py`

<div class="index">

single: OpenSSL; (use in module ssl)

</div>

<div class="index">

TLS, SSL, Transport Layer Security, Secure Sockets Layer

</div>

------------------------------------------------------------------------

This module provides access to Transport Layer Security (often known as "Secure Sockets Layer") encryption and peer authentication facilities for network sockets, both client-side and server-side. This module uses the OpenSSL library.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

> [!NOTE]
> Some behavior may be platform dependent, since calls are made to the operating system socket APIs. The installed version of OpenSSL may also cause variations in behavior. For example, TLSv1.3 comes with OpenSSL version 1.1.1.

> [!WARNING]
> Don't use this module without reading the `ssl-security`. Doing so may lead to a false sense of security, as the default settings of the ssl module are not necessarily appropriate for your application.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

This section documents the objects and functions in the `ssl` module; for more general information about TLS, SSL, and certificates, the reader is referred to the documents in the "See Also" section at the bottom.

This module provides a class, `ssl.SSLSocket`, which is derived from the `socket.socket` type, and provides a socket-like wrapper that also encrypts and decrypts the data going over the socket with SSL. It supports additional methods such as `getpeercert`, which retrieves the certificate of the other side of the connection, `cipher`, which retrieves the cipher being used for the secure connection or `get_verified_chain`, `get_unverified_chain` which retrieves certificate chain.

For more sophisticated applications, the `ssl.SSLContext` class helps manage settings and certificates, which can then be inherited by SSL sockets created through the `SSLContext.wrap_socket` method.

<div class="versionchanged">

3.5.3 Updated to support linking with OpenSSL 1.1.0

</div>

<div class="versionchanged">

3.6

OpenSSL 0.9.8, 1.0.0 and 1.0.1 are deprecated and no longer supported. In the future the ssl module will require at least OpenSSL 1.0.2 or 1.1.0.

</div>

<div class="versionchanged">

3.10

`644` has been implemented. The ssl module requires OpenSSL 1.1.1 or newer.

Use of deprecated constants and functions result in deprecation warnings.

</div>

## Functions, constants, and exceptions

### Socket creation

Instances of `SSLSocket` must be created using the `SSLContext.wrap_socket` method. The helper function `create_default_context` returns a new context with secure default settings.

Client socket example with default context and IPv4/IPv6 dual stack:

    import socket
    import ssl

    hostname = 'www.python.org'
    context = ssl.create_default_context()

    with socket.create_connection((hostname, 443)) as sock:
        with context.wrap_socket(sock, server_hostname=hostname) as ssock:
            print(ssock.version())

Client socket example with custom context and IPv4:

    hostname = 'www.python.org'
    # PROTOCOL_TLS_CLIENT requires valid cert chain and hostname
    context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    context.load_verify_locations('path/to/cabundle.pem')

    with socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) as sock:
        with context.wrap_socket(sock, server_hostname=hostname) as ssock:
            print(ssock.version())

Server socket example listening on localhost IPv4:

    context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    context.load_cert_chain('/path/to/certchain.pem', '/path/to/private.key')

    with socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) as sock:
        sock.bind(('127.0.0.1', 8443))
        sock.listen(5)
        with context.wrap_socket(sock, server_side=True) as ssock:
            conn, addr = ssock.accept()
            ...

### Context creation

A convenience function helps create `SSLContext` objects for common purposes.

<div class="function">

create_default_context(purpose=Purpose.SERVER_AUTH, \*,cafile=None, capath=None, cadata=None)

Return a new `SSLContext` object with default settings for the given *purpose*. The settings are chosen by the `!ssl` module, and usually represent a higher security level than when calling the `SSLContext` constructor directly.

*cafile*, *capath*, *cadata* represent optional CA certificates to trust for certificate verification, as in `SSLContext.load_verify_locations`. If all three are `None`, this function can choose to trust the system's default CA certificates instead.

The settings are: `PROTOCOL_TLS_CLIENT` or `PROTOCOL_TLS_SERVER`, `OP_NO_SSLv2`, and `OP_NO_SSLv3` with high encryption cipher suites without RC4 and without unauthenticated cipher suites. Passing `~Purpose.SERVER_AUTH` as *purpose* sets `~SSLContext.verify_mode` to `CERT_REQUIRED` and either loads CA certificates (when at least one of *cafile*, *capath* or *cadata* is given) or uses `SSLContext.load_default_certs` to load default CA certificates.

When the environment variable `!SSLKEYLOGFILE` is set, `create_default_context` enables key logging by setting `~SSLContext.keylog_filename` to the variable's value.

The default settings for this context include `VERIFY_X509_PARTIAL_CHAIN` and `VERIFY_X509_STRICT`. These make the underlying OpenSSL implementation behave more like a conforming implementation of `5280`, in exchange for a small amount of incompatibility with older X.509 certificates.

> [!NOTE]
> The protocol, options, cipher and other settings may change to more restrictive values anytime without prior deprecation. The values represent a fair balance between compatibility and security.
>
> If your application needs specific settings, you should create a `SSLContext` and apply the settings yourself.

> [!NOTE]
> If you find that when certain older clients or servers attempt to connect with a `SSLContext` created by this function that they get an error stating "Protocol or cipher suite mismatch", it may be that they only support SSL3.0 which this function excludes using the `OP_NO_SSLv3`. SSL3.0 is widely considered to be [completely broken](https://en.wikipedia.org/wiki/POODLE). If you still wish to continue to use this function but still allow SSL 3.0 connections you can re-enable them using:
>
> ctx = ssl.create_default_context(Purpose.CLIENT_AUTH) ctx.options &= ~ssl.OP_NO_SSLv3

> [!NOTE]
> This context enables `VERIFY_X509_STRICT` by default, which may reject pre-`5280` or malformed certificates that the underlying OpenSSL implementation otherwise would accept. While disabling this is not recommended, you can do so using:
>
> ctx = ssl.create_default_context() ctx.verify_flags &= ~ssl.VERIFY_X509_STRICT

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.4.4

RC4 was dropped from the default cipher string.

</div>

<div class="versionchanged">

3.6

ChaCha20/Poly1305 was added to the default cipher string.

3DES was dropped from the default cipher string.

</div>

<div class="versionchanged">

3.8

Support for key logging to `SSLKEYLOGFILE` was added.

</div>

<div class="versionchanged">

3.10

The context now uses `PROTOCOL_TLS_CLIENT` or `PROTOCOL_TLS_SERVER` protocol instead of generic `PROTOCOL_TLS`.

</div>

<div class="versionchanged">

3.13

The context now uses `VERIFY_X509_PARTIAL_CHAIN` and `VERIFY_X509_STRICT` in its default verify flags.

</div>

</div>

### Signature algorithms

<div class="function">

get_sigalgs()

Return a list of available TLS signature algorithm names used by servers to complete the TLS handshake or clients requesting certificate-based authentication. For example:

    >>> ssl.get_sigalgs()  # doctest: +SKIP
    ['ecdsa_secp256r1_sha256', 'ecdsa_secp384r1_sha384', ...]

These names can be used when building string values to pass to the `SSLContext.set_client_sigalgs` and `SSLContext.set_server_sigalgs` methods.

<div class="versionadded">

3.15

</div>

</div>

### Exceptions

<div class="exception">

SSLError

Raised to signal an error from the underlying SSL implementation (currently provided by the OpenSSL library). This signifies some problem in the higher-level encryption and authentication layer that's superimposed on the underlying network connection. This error is a subtype of `OSError`. The error code and message of `SSLError` instances are provided by the OpenSSL library.

<div class="versionchanged">

3.3 `SSLError` used to be a subtype of `socket.error`.

</div>

<div class="attribute">

library

A string mnemonic designating the OpenSSL submodule in which the error occurred, such as `SSL`, `PEM` or `X509`. The range of possible values depends on the OpenSSL version.

<div class="versionadded">

3.3

</div>

</div>

<div class="attribute">

reason

A string mnemonic designating the reason this error occurred, for example `CERTIFICATE_VERIFY_FAILED`. The range of possible values depends on the OpenSSL version.

<div class="versionadded">

3.3

</div>

</div>

</div>

<div class="exception">

SSLZeroReturnError

A subclass of `SSLError` raised when trying to read or write and the SSL connection has been closed cleanly. Note that this doesn't mean that the underlying transport (read TCP) has been closed.

<div class="versionadded">

3.3

</div>

</div>

<div class="exception">

SSLWantReadError

A subclass of `SSLError` raised by a `non-blocking SSL socket
<ssl-nonblocking>` when trying to read or write data, but more data needs to be received on the underlying TCP transport before the request can be fulfilled.

<div class="versionadded">

3.3

</div>

</div>

<div class="exception">

SSLWantWriteError

A subclass of `SSLError` raised by a `non-blocking SSL socket
<ssl-nonblocking>` when trying to read or write data, but more data needs to be sent on the underlying TCP transport before the request can be fulfilled.

<div class="versionadded">

3.3

</div>

</div>

<div class="exception">

SSLSyscallError

A subclass of `SSLError` raised when a system error was encountered while trying to fulfill an operation on a SSL socket. Unfortunately, there is no easy way to inspect the original errno number.

<div class="versionadded">

3.3

</div>

</div>

<div class="exception">

SSLEOFError

A subclass of `SSLError` raised when the SSL connection has been terminated abruptly. Generally, you shouldn't try to reuse the underlying transport when this error is encountered.

<div class="versionadded">

3.3

</div>

</div>

<div class="exception">

SSLCertVerificationError

A subclass of `SSLError` raised when certificate validation has failed.

<div class="versionadded">

3.7

</div>

<div class="attribute">

verify_code

A numeric error number that denotes the verification error.

</div>

<div class="attribute">

verify_message

A human readable string of the verification error.

</div>

</div>

<div class="exception">

SSLECHRejectedError

A subclass of `SSLError` raised when the handshake fails because the server rejected `Encrypted Client Hello <ssl-encrypted-client-hello>`. The connection cannot be used. If `retry_config` is not empty, the client should retry once with a new connection using that configuration. If it is empty, the authenticated server has securely disabled ECH, and the client should retry with a new connection without ECH (see `9849`, section 6.1.6).

If certificate verification for the outer server name fails, the handshake raises `SSLCertVerificationError` instead.

This exception is only raised when `HAS_ECH` is `True`.

<div class="attribute">

retry_config

The server's ECH retry configuration as an ECHConfigList, or `b""` if the server did not provide one.

</div>

<div class="versionadded">

next

</div>

</div>

<div class="exception">

CertificateError

An alias for `SSLCertVerificationError`.

<div class="versionchanged">

3.7 The exception is now an alias for `SSLCertVerificationError`.

</div>

</div>

### Random generation

<div class="function">

RAND_bytes(num, /)

Return *num* cryptographically strong pseudo-random bytes. Raises an `SSLError` if the PRNG has not been seeded with enough data or if the operation is not supported by the current RAND method. `RAND_status` can be used to check the status of the PRNG and `RAND_add` can be used to seed the PRNG.

For almost all applications `os.urandom` is preferable.

Read the Wikipedia article, [Cryptographically secure pseudorandom number generator (CSPRNG)](https://en.wikipedia.org/wiki/Cryptographically_secure_pseudorandom_number_generator), to get the requirements of a cryptographically strong generator.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

RAND_status()

Return `True` if the SSL pseudo-random number generator has been seeded with 'enough' randomness, and `False` otherwise. Use `ssl.RAND_add` to increase the randomness of the pseudo-random number generator.

</div>

<div class="function">

RAND_add(bytes, entropy, /)

Mix the given *bytes* into the SSL pseudo-random number generator. The parameter *entropy* (a float) is a lower bound on the entropy contained in string (so you can always use `0.0`). See `1750` for more information on sources of entropy.

<div class="versionchanged">

3.5 Writable `bytes-like object` is now accepted.

</div>

</div>

### Certificate handling

<div class="testsetup">

import ssl

</div>

<div class="function">

cert_time_to_seconds(cert_time)

Return the time in seconds since the epoch, given the `cert_time` string representing the "notBefore" or "notAfter" date from a certificate in `"%b %d %H:%M:%S %Y %Z"` strptime format (C locale).

Here's an example:

<div class="doctest">

newcontext

\>\>\> import ssl \>\>\> import datetime as dt \>\>\> timestamp = ssl.cert_time_to_seconds("Jan 5 09:34:43 2018 GMT") \>\>\> timestamp \# doctest: +SKIP 1515144883 \>\>\> print(dt.datetime.fromtimestamp(timestamp, dt.UTC)) \# doctest: +SKIP 2018-01-05 09:34:43+00:00

</div>

"notBefore" or "notAfter" dates must use GMT (`5280`).

<div class="versionchanged">

3.5 Interpret the input time as a time in UTC as specified by 'GMT' timezone in the input string. Local timezone was used previously. Return an integer (no fractions of a second in the input format)

</div>

</div>

<div class="function">

get_server_certificate(addr, ssl_version=PROTOCOL_TLS_CLIENT, ca_certs=None\[, timeout\])

Given the address `addr` of an SSL-protected server, as a (*hostname*, *port-number*) pair, fetches the server's certificate, and returns it as a PEM-encoded string. If `ssl_version` is specified, uses that version of the SSL protocol to attempt to connect to the server. If *ca_certs* is specified, it should be a file containing a list of root certificates, the same format as used for the *cafile* parameter in `SSLContext.load_verify_locations`. The call will attempt to validate the server certificate against that set of root certificates, and will fail if the validation attempt fails. A timeout can be specified with the `timeout` parameter.

<div class="versionchanged">

3.3 This function is now IPv6-compatible.

</div>

<div class="versionchanged">

3.5 The default *ssl_version* is changed from `PROTOCOL_SSLv3` to `PROTOCOL_TLS` for maximum compatibility with modern servers.

</div>

<div class="versionchanged">

3.10 The *timeout* parameter was added.

</div>

</div>

<div class="function">

DER_cert_to_PEM_cert(der_cert_bytes)

Given a certificate as a DER-encoded blob of bytes, returns a PEM-encoded string version of the same certificate.

</div>

<div class="function">

PEM_cert_to_DER_cert(pem_cert_string)

Given a certificate as an ASCII PEM string, returns a DER-encoded sequence of bytes for that same certificate.

</div>

<div class="function">

get_default_verify_paths()

Returns a named tuple with paths to OpenSSL's default cafile and capath. The paths are the same as used by `SSLContext.set_default_verify_paths`. The return value is a `named tuple` `DefaultVerifyPaths`:

- `cafile` - resolved path to cafile or `None` if the file doesn't exist,
- `capath` - resolved path to capath or `None` if the directory doesn't exist,
- `openssl_cafile_env` - OpenSSL's environment key that points to a cafile,
- `openssl_cafile` - hard coded path to a cafile,
- `openssl_capath_env` - OpenSSL's environment key that points to a capath,
- `openssl_capath` - hard coded path to a capath directory

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

enum_certificates(store_name)

Retrieve certificates from Windows' system cert store. *store_name* may be one of `CA`, `ROOT` or `MY`. Windows may provide additional cert stores, too.

The function returns a list of (cert_bytes, encoding_type, trust) tuples. The encoding_type specifies the encoding of cert_bytes. It is either `'x509_asn'` for X.509 ASN.1 data or `'pkcs_7_asn'` for PKCS#7 ASN.1 data. Trust specifies the purpose of the certificate as a set of OIDS or exactly `True` if the certificate is trustworthy for all purposes.

Example:

    >>> ssl.enum_certificates("CA")
    [(b'data...', 'x509_asn', {'1.3.6.1.5.5.7.3.1', '1.3.6.1.5.5.7.3.2'}),
     (b'data...', 'x509_asn', True)]

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

enum_crls(store_name)

Retrieve CRLs from Windows' system cert store. *store_name* may be one of `CA`, `ROOT` or `MY`. Windows may provide additional cert stores, too.

The function returns a list of (cert_bytes, encoding_type) tuples. The encoding_type specifies the encoding of cert_bytes. It is either `'x509_asn'` for X.509 ASN.1 data or `'pkcs_7_asn'` for PKCS#7 ASN.1 data.

<div class="availability">

Windows.

</div>

<div class="versionadded">

3.4

</div>

</div>

### Constants

> All constants are now `enum.IntEnum` or `enum.IntFlag` collections.
>
> <div class="versionadded">
>
> 3.6
>
> </div>

<div class="data">

CERT_NONE

Possible value for `SSLContext.verify_mode`. Except for `PROTOCOL_TLS_CLIENT`, it is the default mode. With client-side sockets, just about any cert is accepted. Validation errors, such as untrusted or expired cert, are ignored and do not abort the TLS/SSL handshake.

In server mode, no certificate is requested from the client, so the client does not send any for client cert authentication.

See the discussion of `ssl-security` below.

</div>

<div class="data">

CERT_OPTIONAL

Possible value for `SSLContext.verify_mode`. In client mode, `CERT_OPTIONAL` has the same meaning as `CERT_REQUIRED`. It is recommended to use `CERT_REQUIRED` for client-side sockets instead.

In server mode, a client certificate request is sent to the client. The client may either ignore the request or send a certificate in order perform TLS client cert authentication. If the client chooses to send a certificate, it is verified. Any verification error immediately aborts the TLS handshake.

Use of this setting requires a valid set of CA certificates to be passed to `SSLContext.load_verify_locations`.

</div>

<div class="data">

CERT_REQUIRED

Possible value for `SSLContext.verify_mode`. In this mode, certificates are required from the other side of the socket connection; an `SSLError` will be raised if no certificate is provided, or if its validation fails. This mode is **not** sufficient to verify a certificate in client mode as it does not match hostnames. `~SSLContext.check_hostname` must be enabled as well to verify the authenticity of a cert. `PROTOCOL_TLS_CLIENT` uses `CERT_REQUIRED` and enables `~SSLContext.check_hostname` by default.

With server socket, this mode provides mandatory TLS client cert authentication. A client certificate request is sent to the client and the client must provide a valid and trusted certificate.

Use of this setting requires a valid set of CA certificates to be passed to `SSLContext.load_verify_locations`.

</div>

<div class="VerifyMode">

`enum.IntEnum` collection of [CERT]()\* constants.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

VERIFY_DEFAULT

Possible value for `SSLContext.verify_flags`. In this mode, certificate revocation lists (CRLs) are not checked. By default OpenSSL does neither require nor verify CRLs.

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

VERIFY_CRL_CHECK_LEAF

Possible value for `SSLContext.verify_flags`. In this mode, only the peer cert is checked but none of the intermediate CA certificates. The mode requires a valid CRL that is signed by the peer cert's issuer (its direct ancestor CA). If no proper CRL has been loaded with `SSLContext.load_verify_locations`, validation will fail.

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

VERIFY_CRL_CHECK_CHAIN

Possible value for `SSLContext.verify_flags`. In this mode, CRLs of all certificates in the peer cert chain are checked.

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

VERIFY_X509_STRICT

Possible value for `SSLContext.verify_flags` to disable workarounds for broken X.509 certificates.

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

VERIFY_ALLOW_PROXY_CERTS

Possible value for `SSLContext.verify_flags` to enables proxy certificate verification.

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

VERIFY_X509_TRUSTED_FIRST

Possible value for `SSLContext.verify_flags`. It instructs OpenSSL to prefer trusted certificates when building the trust chain to validate a certificate. This flag is enabled by default.

<div class="versionadded">

3.4.4

</div>

</div>

<div class="data">

VERIFY_X509_PARTIAL_CHAIN

Possible value for `SSLContext.verify_flags`. It instructs OpenSSL to accept intermediate CAs in the trust store to be treated as trust-anchors, in the same way as the self-signed root CA certificates. This makes it possible to trust certificates issued by an intermediate CA without having to trust its ancestor root CA.

<div class="versionadded">

3.10

</div>

</div>

<div class="VerifyFlags">

`enum.IntFlag` collection of [VERIFY]()\* constants.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

PROTOCOL_TLS

Selects the highest protocol version that both the client and server support. Despite the name, this option can select both "SSL" and "TLS" protocols.

<div class="versionadded">

3.6

</div>

<div class="deprecated">

3.10

TLS clients and servers require different default settings for secure communication. The generic TLS protocol constant is deprecated in favor of `PROTOCOL_TLS_CLIENT` and `PROTOCOL_TLS_SERVER`.

</div>

</div>

<div class="data">

PROTOCOL_TLS_CLIENT

Auto-negotiate the highest protocol version that both the client and server support, and configure the context client-side connections. The protocol enables `CERT_REQUIRED` and `~SSLContext.check_hostname` by default.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

PROTOCOL_TLS_SERVER

Auto-negotiate the highest protocol version that both the client and server support, and configure the context server-side connections.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

PROTOCOL_SSLv23

Alias for `PROTOCOL_TLS`.

<div class="deprecated">

3.6

Use `PROTOCOL_TLS` instead.

</div>

</div>

<div class="data">

PROTOCOL_SSLv3

Selects SSL version 3 as the channel encryption protocol.

This protocol is not available if OpenSSL is compiled with the `no-ssl3` option.

> [!WARNING]
> SSL version 3 is insecure. Its use is highly discouraged.

<div class="deprecated">

3.6

OpenSSL has deprecated all version specific protocols. Use the default protocol `PROTOCOL_TLS_SERVER` or `PROTOCOL_TLS_CLIENT` with `SSLContext.minimum_version` and `SSLContext.maximum_version` instead.

</div>

</div>

<div class="data">

PROTOCOL_TLSv1

Selects TLS version 1.0 as the channel encryption protocol.

<div class="deprecated">

3.6

OpenSSL has deprecated all version specific protocols.

</div>

</div>

<div class="data">

PROTOCOL_TLSv1_1

Selects TLS version 1.1 as the channel encryption protocol. Available only with openssl version 1.0.1+.

<div class="versionadded">

3.4

</div>

<div class="deprecated">

3.6

OpenSSL has deprecated all version specific protocols.

</div>

</div>

<div class="data">

PROTOCOL_TLSv1_2

Selects TLS version 1.2 as the channel encryption protocol. Available only with openssl version 1.0.1+.

<div class="versionadded">

3.4

</div>

<div class="deprecated">

3.6

OpenSSL has deprecated all version specific protocols.

</div>

</div>

<div class="data">

OP_ALL

Enables workarounds for various bugs present in other SSL implementations. This option is set by default. It does not necessarily set the same flags as OpenSSL's `SSL_OP_ALL` constant.

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

OP_NO_SSLv2

Prevents an SSLv2 connection. This option is only applicable in conjunction with `PROTOCOL_TLS`. It prevents the peers from choosing SSLv2 as the protocol version.

<div class="versionadded">

3.2

</div>

<div class="deprecated">

3.6

SSLv2 is deprecated

</div>

</div>

<div class="data">

OP_NO_SSLv3

Prevents an SSLv3 connection. This option is only applicable in conjunction with `PROTOCOL_TLS`. It prevents the peers from choosing SSLv3 as the protocol version.

<div class="versionadded">

3.2

</div>

<div class="deprecated">

3.6

SSLv3 is deprecated

</div>

</div>

<div class="data">

OP_NO_TLSv1

Prevents a TLSv1 connection. This option is only applicable in conjunction with `PROTOCOL_TLS`. It prevents the peers from choosing TLSv1 as the protocol version.

<div class="versionadded">

3.2

</div>

<div class="deprecated">

3.7 The option is deprecated since OpenSSL 1.1.0, use the new `SSLContext.minimum_version` and `SSLContext.maximum_version` instead.

</div>

</div>

<div class="data">

OP_NO_TLSv1_1

Prevents a TLSv1.1 connection. This option is only applicable in conjunction with `PROTOCOL_TLS`. It prevents the peers from choosing TLSv1.1 as the protocol version. Available only with openssl version 1.0.1+.

<div class="versionadded">

3.4

</div>

<div class="deprecated">

3.7 The option is deprecated since OpenSSL 1.1.0.

</div>

</div>

<div class="data">

OP_NO_TLSv1_2

Prevents a TLSv1.2 connection. This option is only applicable in conjunction with `PROTOCOL_TLS`. It prevents the peers from choosing TLSv1.2 as the protocol version. Available only with openssl version 1.0.1+.

<div class="versionadded">

3.4

</div>

<div class="deprecated">

3.7 The option is deprecated since OpenSSL 1.1.0.

</div>

</div>

<div class="data">

OP_NO_TLSv1_3

Prevents a TLSv1.3 connection. This option is only applicable in conjunction with `PROTOCOL_TLS`. It prevents the peers from choosing TLSv1.3 as the protocol version. TLS 1.3 is available with OpenSSL 1.1.1 or later. When Python has been compiled against an older version of OpenSSL, the flag defaults to *0*.

<div class="versionadded">

3.6.3

</div>

<div class="deprecated">

3.7 The option is deprecated since OpenSSL 1.1.0. It was added to 2.7.15 and 3.6.3 for backwards compatibility with OpenSSL 1.0.2.

</div>

</div>

<div class="data">

OP_NO_RENEGOTIATION

Disable all renegotiation in TLSv1.2 and earlier. Do not send HelloRequest messages, and ignore renegotiation requests via ClientHello.

This option is only available with OpenSSL 1.1.0h and later.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

OP_CIPHER_SERVER_PREFERENCE

Use the server's cipher ordering preference, rather than the client's. This option has no effect on client sockets and SSLv2 server sockets.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

OP_SINGLE_DH_USE

Prevents reuse of the same DH key for distinct SSL sessions. This improves forward secrecy but requires more computational resources. This option only applies to server sockets.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

OP_SINGLE_ECDH_USE

Prevents reuse of the same ECDH key for distinct SSL sessions. This improves forward secrecy but requires more computational resources. This option only applies to server sockets.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

OP_ENABLE_MIDDLEBOX_COMPAT

Send dummy Change Cipher Spec (CCS) messages in TLS 1.3 handshake to make a TLS 1.3 connection look more like a TLS 1.2 connection.

This option is only available with OpenSSL 1.1.1 and later.

<div class="versionadded">

3.8

</div>

</div>

<div class="data">

OP_NO_COMPRESSION

Disable compression on the SSL channel. This is useful if the application protocol supports its own compression scheme.

<div class="versionadded">

3.3

</div>

</div>

<div class="Options">

`enum.IntFlag` collection of [OP]()\* constants.

</div>

<div class="data">

OP_NO_TICKET

Prevent client side from requesting a session ticket.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

OP_IGNORE_UNEXPECTED_EOF

Ignore unexpected shutdown of TLS connections.

This option is only available with OpenSSL 3.0.0 and later.

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

OP_ENABLE_KTLS

Enable the use of the kernel TLS. To benefit from the feature, OpenSSL must have been compiled with support for it, and the negotiated cipher suites and extensions must be supported by it (a list of supported ones may vary by platform and kernel version).

Note that with enabled kernel TLS some cryptographic operations are performed by the kernel directly and not via any available OpenSSL Providers. This might be undesirable if, for example, the application requires all cryptographic operations to be performed by the FIPS provider.

This option is only available with OpenSSL 3.0.0 and later.

<div class="versionadded">

3.12

</div>

</div>

<div class="data">

OP_LEGACY_SERVER_CONNECT

Allow legacy insecure renegotiation between OpenSSL and unpatched servers only.

<div class="versionadded">

3.12

</div>

</div>

<div class="data">

OP_ECH_GREASE

Include a GREASE ECH extension in ClientHello messages when ECH is not being attempted, so these connections resemble connections that use ECH. Clients must ignore any ECH retry configurations received in response to GREASE (see `9849`, section 6.2.1).

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="data">

OP_ECH_TRIALDECRYPT

Attempt server-side ECH decryption with all available ECH keys, rather than only those with a matching configuration identifier.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="data">

OP_ECH_IGNORE_CID

Use a random ECH configuration identifier in client connections.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="data">

OP_ECH_GREASE_RETRY_CONFIG

Include GREASE entries in the ECH retry configurations sent by a server.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="data">

HAS_ALPN

Whether the OpenSSL library has built-in support for the *Application-Layer Protocol Negotiation* TLS extension as described in `7301`.

<div class="versionadded">

3.5

</div>

</div>

<div class="data">

HAS_NEVER_CHECK_COMMON_NAME

Whether the OpenSSL library has built-in support not checking subject common name and `SSLContext.hostname_checks_common_name` is writeable.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

HAS_ECDH

Whether the OpenSSL library has built-in support for the Elliptic Curve-based Diffie-Hellman key exchange. This should be true unless the feature was explicitly disabled by the distributor.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

HAS_SNI

Whether the OpenSSL library has built-in support for the *Server Name Indication* extension (as defined in `6066`).

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

HAS_NPN

Whether the OpenSSL library has built-in support for the *Next Protocol Negotiation* as described in the [Application Layer Protocol Negotiation](https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation). When true, you can use the `SSLContext.set_npn_protocols` method to advertise which protocols you want to support.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

HAS_SSLv2

Whether the OpenSSL library has built-in support for the SSL 2.0 protocol.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

HAS_SSLv3

Whether the OpenSSL library has built-in support for the SSL 3.0 protocol.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

HAS_TLSv1

Whether the OpenSSL library has built-in support for the TLS 1.0 protocol.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

HAS_TLSv1_1

Whether the OpenSSL library has built-in support for the TLS 1.1 protocol.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

HAS_TLSv1_2

Whether the OpenSSL library has built-in support for the TLS 1.2 protocol.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

HAS_TLSv1_3

Whether the OpenSSL library has built-in support for the TLS 1.3 protocol.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

HAS_PSK

Whether the OpenSSL library has built-in support for TLS-PSK.

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

HAS_PSK_TLS13

Whether the OpenSSL library has built-in support for External PSKs in TLS 1.3 as described in `9258`.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

HAS_PHA

Whether the OpenSSL library has built-in support for TLS-PHA.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

HAS_ECH

Whether the OpenSSL library has built-in support for Encrypted Client Hello (ECH). This requires OpenSSL 4.0 or later with ECH enabled.

<div class="versionadded">

next

</div>

</div>

<div class="data">

CHANNEL_BINDING_TYPES

List of supported TLS channel binding types. Strings in this list can be used as arguments to `SSLSocket.get_channel_binding`.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

OPENSSL_VERSION

The version string of the OpenSSL library loaded by the interpreter:

    >>> ssl.OPENSSL_VERSION
    'OpenSSL 1.0.2k  26 Jan 2017'

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

OPENSSL_VERSION_INFO

A named tuple of five integers representing version information about the OpenSSL library loaded by the interpreter: *major*, *minor*, *fix*, *patch* and *status*:

    >>> ssl.OPENSSL_VERSION_INFO
    ssl.OPENSSL_VERSION_INFO(major=3, minor=0, fix=0, patch=13, status=0)

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.16 It is now a named tuple.

</div>

</div>

<div class="data">

OPENSSL_API_VERSION_INFO

A named tuple containing the version of the OpenSSL library that was used for building the module, with the same fields as `OPENSSL_VERSION_INFO`. This may be different from the OpenSSL library actually used at runtime.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

OPENSSL_VERSION_NUMBER

The raw version number of the OpenSSL library, as a single integer:

    >>> ssl.OPENSSL_VERSION_NUMBER
    268443839
    >>> hex(ssl.OPENSSL_VERSION_NUMBER)
    '0x100020bf'

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

ALERT_DESCRIPTION_HANDSHAKE_FAILURE ALERT_DESCRIPTION_INTERNAL_ERROR [ALERT_DESCRIPTION]()\*

Alert Descriptions from `5246` and others. The [IANA TLS Alert Registry](https://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-6) contains this list and references to the RFCs where their meaning is defined.

Used as the return value of the callback function in `SSLContext.set_servername_callback`.

<div class="versionadded">

3.4

</div>

</div>

<div class="AlertDescription">

`enum.IntEnum` collection of [ALERT_DESCRIPTION]()\* constants.

<div class="versionadded">

3.6

</div>

</div>

<div class="ECHStatus">

`enum.IntEnum` collection of Encrypted Client Hello (ECH) statuses returned by `SSLSocket.get_ech_status`.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_SUCCESS

ECH was accepted without a certificate verification error.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_FAILED

An internal or protocol error occurred, or ECH was rejected without a retry configuration.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_FAILED_ECH

ECH was rejected and the server supplied an authenticated retry configuration.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_BAD_NAME

ECH was accepted, but verification of the peer's certificate failed.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_FAILED_ECH_BAD_NAME

ECH was rejected and a retry configuration was received, but certificate verification for the outer server name failed. The retry configuration must not be used.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_NOT_TRIED

ECH is configured but has not been attempted (such as before the handshake starts).

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_NOT_CONFIGURED

No ECH configuration is set for this connection.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_GREASE

ECH GREASE is enabled or was used, but no retry configuration was received.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_GREASE_ECH

The client sent a GREASE ECH extension and received a retry configuration.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_BACKEND

The server received a ClientHello marked as an inner ClientHello, as used by a backend server in an ECH split deployment.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

ECHStatus.ECH_STATUS_BAD_CALL

Invalid arguments were supplied to OpenSSL's ECH status API.

<div class="versionadded">

next

</div>

</div>

<div class="data">

Purpose.SERVER_AUTH

Option for `create_default_context` and `SSLContext.load_default_certs`. This value indicates that the context may be used to authenticate web servers (therefore, it will be used to create client-side sockets).

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

Purpose.CLIENT_AUTH

Option for `create_default_context` and `SSLContext.load_default_certs`. This value indicates that the context may be used to authenticate web clients (therefore, it will be used to create server-side sockets).

<div class="versionadded">

3.4

</div>

</div>

<div class="SSLErrorNumber">

`enum.IntEnum` collection of [SSL_ERROR]()\* constants.

<div class="versionadded">

3.6

</div>

</div>

<div class="TLSVersion">

`enum.IntEnum` collection of SSL and TLS versions for `SSLContext.maximum_version` and `SSLContext.minimum_version`.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

TLSVersion.MINIMUM_SUPPORTED

</div>

<div class="attribute">

TLSVersion.MAXIMUM_SUPPORTED

The minimum or maximum supported SSL or TLS version. These are magic constants. Their values don't reflect the lowest and highest available TLS/SSL versions.

</div>

<div class="attribute">

TLSVersion.SSLv3

</div>

<div class="attribute">

TLSVersion.TLSv1

</div>

<div class="attribute">

TLSVersion.TLSv1_1

</div>

<div class="attribute">

TLSVersion.TLSv1_2

</div>

<div class="attribute">

TLSVersion.TLSv1_3

SSL 3.0 to TLS 1.3.

<div class="deprecated">

3.10

All `TLSVersion` members except `TLSVersion.TLSv1_2` and `TLSVersion.TLSv1_3` are deprecated.

</div>

</div>

## SSL sockets

<div class="SSLSocket(socket.socket)">

SSL sockets provide the following methods of `socket-objects`:

- `~socket.socket.accept`
- `~socket.socket.bind`
- `~socket.socket.close`
- `~socket.socket.connect`
- `~socket.socket.detach`
- `~socket.socket.fileno`
- `~socket.socket.getpeername`, `~socket.socket.getsockname`
- `~socket.socket.getsockopt`, `~socket.socket.setsockopt`
- `~socket.socket.gettimeout`, `~socket.socket.settimeout`, `~socket.socket.setblocking`
- `~socket.socket.listen`
- `~socket.socket.makefile`
- `~socket.socket.recv`, `~socket.socket.recv_into` (but passing a non-zero `flags` argument is not allowed)
- `~socket.socket.send`, `~socket.socket.sendall` (with the same limitation)
- `~socket.socket.sendfile` (it may be high-performant only when the kernel TLS is enabled by setting `~ssl.OP_ENABLE_KTLS` or when a socket is plain-text, else `~socket.socket.send` will be used)
- `~socket.socket.shutdown`

However, since the SSL (and TLS) protocol has its own framing atop of TCP, the SSL sockets abstraction can, in certain respects, diverge from the specification of normal, OS-level sockets. See especially the `notes on non-blocking sockets <ssl-nonblocking>`.

Instances of `SSLSocket` must be created using the `SSLContext.wrap_socket` method.

<div class="versionchanged">

3.5 The `sendfile` method was added.

</div>

<div class="versionchanged">

3.5 The `shutdown` does not reset the socket timeout each time bytes are received or sent. The socket timeout is now the maximum total duration of the shutdown.

</div>

<div class="deprecated">

3.6 It is deprecated to create a `SSLSocket` instance directly, use `SSLContext.wrap_socket` to wrap a socket.

</div>

<div class="versionchanged">

3.7 `SSLSocket` instances must be created with `~SSLContext.wrap_socket`. In earlier versions, it was possible to create instances directly. This was never documented or officially supported.

</div>

<div class="versionchanged">

3.10 Python now uses `SSL_read_ex` and `SSL_write_ex` internally. The functions support reading and writing of data larger than 2 GB. Writing zero-length data no longer fails with a protocol violation error.

</div>

<div class="versionchanged">

3.15 Python now uses `SSL_sendfile` internally when possible. The function sends a file more efficiently because it performs TLS encryption in the kernel to avoid additional context switches.

</div>

</div>

SSL sockets also have the following additional methods and attributes:

<div class="method">

SSLSocket.read(len=1024, buffer=None)

Read up to *len* bytes of data from the SSL socket and return the result as a `bytes` instance. If *buffer* is specified, then read into the buffer instead, and return the number of bytes read.

Raise `SSLWantReadError` or `SSLWantWriteError` if the socket is `non-blocking <ssl-nonblocking>` and the read would block.

As at any time a re-negotiation is possible, a call to `read` can also cause write operations.

<div class="versionchanged">

3.5 The socket timeout is no longer reset each time bytes are received or sent. The socket timeout is now the maximum total duration to read up to *len* bytes.

</div>

<div class="deprecated">

3.6 Use `~SSLSocket.recv` instead of `~SSLSocket.read`.

</div>

</div>

<div class="method">

SSLSocket.write(data)

Write *data* to the SSL socket and return the number of bytes written. The *data* argument must be an object supporting the buffer interface.

Raise `SSLWantReadError` or `SSLWantWriteError` if the socket is `non-blocking <ssl-nonblocking>` and the write would block.

As at any time a re-negotiation is possible, a call to `write` can also cause read operations.

<div class="versionchanged">

3.5 The socket timeout is no longer reset each time bytes are received or sent. The socket timeout is now the maximum total duration to write *data*.

</div>

<div class="deprecated">

3.6 Use `~SSLSocket.send` instead of `~SSLSocket.write`.

</div>

</div>

> [!NOTE]
> The `~SSLSocket.read` and `~SSLSocket.write` methods are the low-level methods that read and write unencrypted, application-level data and decrypt/encrypt it to encrypted, wire-level data. These methods require an active SSL connection, i.e. the handshake was completed and `SSLSocket.unwrap` was not called.
>
> Normally you should use the socket API methods like `~socket.socket.recv` and `~socket.socket.send` instead of these methods.

<div class="method">

SSLSocket.do_handshake(block=False)

Perform the SSL setup handshake.

If *block* is true and the timeout obtained by `~socket.socket.gettimeout` is zero, the socket is set in blocking mode until the handshake is performed.

<div class="versionchanged">

3.4 The handshake method also performs `!match_hostname` when the `~SSLContext.check_hostname` attribute of the socket's `~SSLSocket.context` is true.

</div>

<div class="versionchanged">

3.5 The socket timeout is no longer reset each time bytes are received or sent. The socket timeout is now the maximum total duration of the handshake.

</div>

<div class="versionchanged">

3.7 Hostname or IP address is matched by OpenSSL during handshake. The function `!match_hostname` is no longer used. In case OpenSSL refuses a hostname or IP address, the handshake is aborted early and a TLS alert message is sent to the peer.

</div>

</div>

<div class="method">

SSLSocket.getpeercert(binary_form=False)

If there is no certificate for the peer on the other end of the connection, return `None`. If the SSL handshake hasn't been done yet, raise `ValueError`.

If the `binary_form` parameter is `False`, and a certificate was received from the peer, this method returns a `dict` instance. If the certificate was not validated, the dict is empty. If the certificate was validated, it returns a dict with several keys, amongst them `subject` (the principal for which the certificate was issued) and `issuer` (the principal issuing the certificate). If a certificate contains an instance of the *Subject Alternative Name* extension (see `3280`), there will also be a `subjectAltName` key in the dictionary.

The `subject` and `issuer` fields are tuples containing the sequence of relative distinguished names (RDNs) given in the certificate's data structure for the respective fields, and each RDN is a sequence of name-value pairs. Here is a real-world example:

    {'issuer': ((('countryName', 'IL'),),
                (('organizationName', 'StartCom Ltd.'),),
                (('organizationalUnitName',
                  'Secure Digital Certificate Signing'),),
                (('commonName',
                  'StartCom Class 2 Primary Intermediate Server CA'),)),
     'notAfter': 'Nov 22 08:15:19 2013 GMT',
     'notBefore': 'Nov 21 03:09:52 2011 GMT',
     'serialNumber': '95F0',
     'subject': ((('description', '571208-SLe257oHY9fVQ07Z'),),
                 (('countryName', 'US'),),
                 (('stateOrProvinceName', 'California'),),
                 (('localityName', 'San Francisco'),),
                 (('organizationName', 'Electronic Frontier Foundation, Inc.'),),
                 (('commonName', '*.eff.org'),),
                 (('emailAddress', 'hostmaster@eff.org'),)),
     'subjectAltName': (('DNS', '*.eff.org'), ('DNS', 'eff.org')),
     'version': 3}

If the `binary_form` parameter is `True`, and a certificate was provided, this method returns the DER-encoded form of the entire certificate as a sequence of bytes, or `None` if the peer did not provide a certificate. Whether the peer provides a certificate depends on the SSL socket's role:

- for a client SSL socket, the server will always provide a certificate, regardless of whether validation was required;
- for a server SSL socket, the client will only provide a certificate when requested by the server; therefore `getpeercert` will return `None` if you used `CERT_NONE` (rather than `CERT_OPTIONAL` or `CERT_REQUIRED`).

See also `SSLContext.check_hostname`.

<div class="versionchanged">

3.2 The returned dictionary includes additional items such as `issuer` and `notBefore`.

</div>

<div class="versionchanged">

3.4 `ValueError` is raised when the handshake isn't done. The returned dictionary includes additional X509v3 extension items such as `crlDistributionPoints`, `caIssuers` and `OCSP` URIs.

</div>

<div class="versionchanged">

3.9 IPv6 address strings no longer have a trailing new line.

</div>

</div>

<div class="method">

SSLSocket.get_verified_chain()

Returns verified certificate chain provided by the other end of the SSL channel as a list of DER-encoded bytes. If certificate verification was disabled method acts the same as `~SSLSocket.get_unverified_chain`.

The verified chain is not stored with a session, so the list is empty when the session is resumed (see `~SSLSocket.session_reused`).

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

SSLSocket.get_unverified_chain()

Returns raw certificate chain provided by the other end of the SSL channel as a list of DER-encoded bytes.

No certificates are exchanged when a session is resumed (see `~SSLSocket.session_reused`); the list then holds what was stored with the session, which for a session resumed from a ticket on the server side is the peer's own certificate only.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

SSLSocket.cipher()

Returns a three-value tuple containing the name of the cipher being used, the version of the SSL protocol that defines its use, and the number of secret bits being used. If no connection has been established, returns `None`.

</div>

<div class="method">

SSLSocket.shared_ciphers()

Return the list of ciphers available in both the client and server. Each entry of the returned list is a three-value tuple containing the name of the cipher, the version of the SSL protocol that defines its use, and the number of secret bits the cipher uses. `~SSLSocket.shared_ciphers` returns `None` if no connection has been established or the socket is a client socket.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

SSLSocket.group()

Return the group used for doing key agreement on this connection. If no connection has been established, returns `None`.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

SSLSocket.client_sigalg()

Return the signature algorithm used for performing certificate-based client authentication on this connection, or `None` if no connection has been established or client authentication didn't occur.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

SSLSocket.server_sigalg()

Return the signature algorithm used by the server to complete the TLS handshake on this connection, or `None` if no connection has been established or the cipher suite has no signature.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

SSLSocket.compression()

Return the compression algorithm being used as a string, or `None` if the connection isn't compressed.

If the higher-level protocol supports its own compression mechanism, you can use `OP_NO_COMPRESSION` to disable SSL-level compression.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

SSLSocket.get_channel_binding(cb_type="tls-unique")

Get channel binding data for current connection, as a bytes object. Returns `None` if not connected or the handshake has not been completed.

The *cb_type* parameter allow selection of the desired channel binding type. Valid channel binding types are listed in the `CHANNEL_BINDING_TYPES` list. Currently only the 'tls-unique' channel binding, defined by `5929`, is supported. `ValueError` will be raised if an unsupported channel binding type is requested.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

SSLSocket.get_ech_retry_config()

Return the server's ECH retry configuration as `bytes`, or `b""` if no usable retry configuration is available.

Return `None` if the socket has no underlying SSL object, such as before it connects or after it is closed.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="method">

SSLSocket.get_ech_status()

Return the status of Encrypted Client Hello (ECH) processing as an `ECHStatus` instance.

Return `None` if the socket has no underlying SSL object, such as before it connects or after it is closed.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="method">

SSLSocket.set_ech_config(ech_config)

Set the Encrypted Client Hello (ECH) configuration for this connection only, replacing any configuration set on the context with `SSLContext.set_ech_config`. The *ech_config* parameter accepts the same values.

The configuration must be set before the handshake starts, so this is mostly useful with `do_handshake_on_connect=False` or on a socket that is not connected yet.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="method">

SSLSocket.selected_alpn_protocol()

Return the protocol that was selected during the TLS handshake. If `SSLContext.set_alpn_protocols` was not called, if the other party does not support ALPN, if this socket does not support any of the client's proposed protocols, or if the handshake has not happened yet, `None` is returned.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

SSLSocket.selected_npn_protocol()

Return the higher-level protocol that was selected during the TLS/SSL handshake. If `SSLContext.set_npn_protocols` was not called, or if the other party does not support NPN, or if the handshake has not yet happened, this will return `None`.

<div class="versionadded">

3.3

</div>

<div class="deprecated">

3.10

NPN has been superseded by ALPN

</div>

</div>

<div class="method">

SSLSocket.unwrap()

Performs the SSL shutdown handshake, which removes the TLS layer from the underlying socket, and returns the underlying socket object. This can be used to go from encrypted operation over a connection to unencrypted. The returned socket should always be used for further communication with the other side of the connection, rather than the original socket.

</div>

<div class="method">

SSLSocket.verify_client_post_handshake()

Requests post-handshake authentication (PHA) from a TLS 1.3 client. PHA can only be initiated for a TLS 1.3 connection from a server-side socket, after the initial TLS handshake and with PHA enabled on both sides, see `SSLContext.post_handshake_auth`.

The method does not perform a cert exchange immediately. The server-side sends a CertificateRequest during the next write event and expects the client to respond with a certificate on the next read event.

If any precondition isn't met (e.g. not TLS 1.3, PHA not enabled), an `SSLError` is raised.

> [!NOTE]
> Only available with OpenSSL 1.1.1 and TLS 1.3 enabled. Without TLS 1.3 support, the method raises `NotImplementedError`.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

SSLSocket.version()

Return the actual SSL protocol version negotiated by the connection as a string, or `None` if no secure connection is established. As of this writing, possible return values include `"SSLv2"`, `"SSLv3"`, `"TLSv1"`, `"TLSv1.1"` and `"TLSv1.2"`. Recent OpenSSL versions may define more return values.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

SSLSocket.pending()

Returns the number of already decrypted bytes available for read, pending on the connection.

</div>

<div class="attribute">

SSLSocket.context

The `SSLContext` object this SSL socket is tied to.

<div class="versionadded">

3.2

</div>

</div>

<div class="attribute">

SSLSocket.outer_server_hostname

The server name used in the outer ClientHello when Encrypted Client Hello (ECH) is used, as a `str`. Available on both client-side and server-side sockets. Returns `None` if no outer server name is available.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

SSLSocket.server_side

A boolean which is `True` for server-side sockets and `False` for client-side sockets.

<div class="versionadded">

3.2

</div>

</div>

<div class="attribute">

SSLSocket.server_hostname

Hostname of the server: `str` type, or `None` for server-side socket or if the hostname was not specified in the constructor.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.7 The attribute is now always ASCII text. When `server_hostname` is an internationalized domain name (IDN), this attribute now stores the A-label form (`"xn--pythn-mua.org"`), rather than the U-label form (`"pythön.org"`).

</div>

</div>

<div class="attribute">

SSLSocket.session

The `SSLSession` for this SSL connection. The session is available for client and server side sockets after the TLS handshake has been performed. For client sockets the session can be set before `~SSLSocket.do_handshake` has been called to reuse a session.

<div class="versionadded">

3.6

</div>

</div>

<div class="attribute">

SSLSocket.session_reused

<div class="versionadded">

3.6

</div>

</div>

## SSL contexts

<div class="versionadded">

3.2

</div>

An SSL context holds various data longer-lived than single SSL connections, such as SSL configuration options, certificate(s) and private key(s). It also manages a cache of SSL sessions for server-side sockets, in order to speed up repeated connections from the same clients.

<div class="SSLContext(protocol=None)">

Create a new SSL context. You may pass *protocol* which must be one of the `PROTOCOL_*` constants defined in this module. The parameter specifies which version of the SSL protocol to use. Typically, the server chooses a particular protocol version, and the client must adapt to the server's choice. Most of the versions are not interoperable with the other versions. If not specified, the default is `PROTOCOL_TLS`; it provides the most compatibility with other versions.

Here's a table showing which versions in a client (down the side) can connect to which versions in a server (along the top):

|  |  |  |  |  |  |  |
|----|----|----|----|----|----|----|
| *client* / **server** | **SSLv2** | **SSLv3** | **TLS**[^1] | **TLSv1** | **TLSv1.1** | **TLSv1.2** |
| *SSLv2* | yes | no | no[^2] | no | no | no |
| *SSLv3* | no | yes | no[^3] | no | no | no |
| *TLS* (*SSLv23*)[^4] | no[^5] | no[^6] | yes | yes | yes | yes |
| *TLSv1* | no | no | yes | yes | no | no |
| *TLSv1.1* | no | no | yes | no | yes | no |
| *TLSv1.2* | no | no | yes | no | no | yes |

**Footnotes**

<div class="seealso">

`create_default_context` lets the `!ssl` module choose security settings for a given purpose.

</div>

<div class="versionchanged">

3.6

The context is created with secure default values. The options `OP_NO_COMPRESSION`, `OP_CIPHER_SERVER_PREFERENCE`, `OP_SINGLE_DH_USE`, `OP_SINGLE_ECDH_USE`, `OP_NO_SSLv2`, and `OP_NO_SSLv3` (except for `PROTOCOL_SSLv3`) are set by default. The initial cipher suite list contains only `HIGH` ciphers, no `NULL` ciphers and no `MD5` ciphers.

</div>

<div class="deprecated">

3.10

`SSLContext` without protocol argument is deprecated. The context class will either require `PROTOCOL_TLS_CLIENT` or `PROTOCOL_TLS_SERVER` protocol in the future.

</div>

<div class="versionchanged">

3.10

The default cipher suites now include only secure AES and ChaCha20 ciphers with forward secrecy and security level 2. RSA and DH keys with less than 2048 bits and ECC keys with less than 224 bits are prohibited. `PROTOCOL_TLS`, `PROTOCOL_TLS_CLIENT`, and `PROTOCOL_TLS_SERVER` use TLS 1.2 as minimum TLS version.

</div>

> [!NOTE]
> `SSLContext` only supports limited mutation once it has been used by a connection. Adding new certificates to the internal trust store is allowed, but changing ciphers, verification settings, or mTLS certificates may result in surprising behavior.

> [!NOTE]
> `SSLContext` is designed to be shared and used by multiple connections. Thus, it is thread-safe as long as it is not reconfigured after being used by a connection.

</div>

`SSLContext` objects have the following methods and attributes:

<div class="method">

SSLContext.cert_store_stats()

Get statistics about quantities of loaded X.509 certificates, count of X.509 certificates flagged as CA certificates and certificate revocation lists as dictionary.

Example for a context with one CA cert and one other cert:

    >>> context.cert_store_stats()
    {'crl': 0, 'x509_ca': 1, 'x509': 2}

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

SSLContext.load_cert_chain(certfile, keyfile=None, password=None)

Load a private key and the corresponding certificate. The *certfile* string must be the path to a single file in PEM format containing the certificate as well as any number of CA certificates needed to establish the certificate's authenticity. The *keyfile* string, if present, must point to a file containing the private key. Otherwise the private key will be taken from *certfile* as well. See the discussion of `ssl-certificates` for more information on how the certificate is stored in the *certfile*.

The *password* argument may be a function to call to get the password for decrypting the private key. It will only be called if the private key is encrypted and a password is necessary. It will be called with no arguments, and it should return a string, bytes, or bytearray. If the return value is a string it will be encoded as UTF-8 before using it to decrypt the key. Alternatively a string, bytes, or bytearray value may be supplied directly as the *password* argument. It will be ignored if the private key is not encrypted and no password is needed.

If the *password* argument is not specified and a password is required, OpenSSL's built-in password prompting mechanism will be used to interactively prompt the user for a password.

An `SSLError` is raised if the private key doesn't match with the certificate.

<div class="versionchanged">

3.3 New optional argument *password*.

</div>

</div>

<div class="method">

SSLContext.load_default_certs(purpose=Purpose.SERVER_AUTH)

Load a set of default "certification authority" (CA) certificates from default locations. On Windows it loads CA certs from the `CA` and `ROOT` system stores. On all systems it calls `SSLContext.set_default_verify_paths`. In the future the method may load CA certificates from other locations, too.

The *purpose* flag specifies what kind of CA certificates are loaded. The default settings `Purpose.SERVER_AUTH` loads certificates, that are flagged and trusted for TLS web server authentication (client side sockets). `Purpose.CLIENT_AUTH` loads CA certificates for client certificate verification on the server side.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

SSLContext.load_ech_keys(keyfile, password=None, \*, for_retry=True)

Load a server's Encrypted Client Hello (ECH) private key and configuration. The *keyfile* must be the path to a PEM file containing a private key followed by its matching ECHConfigList.

The *password* argument is used to decrypt encrypted private keys. It accepts a string, bytes, bytearray, or a callable returning one of those types, as in `load_cert_chain`.

Each call adds keys and configurations to those already loaded into the context. If *for_retry* is true, the new configurations are included in ECH retry configurations sent to clients.

Raises `SSLError` if the context uses `PROTOCOL_TLS_CLIENT`.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="method">

SSLContext.load_verify_locations(cafile=None, capath=None, cadata=None)

Load a set of "certification authority" (CA) certificates used to validate other peers' certificates when `verify_mode` is other than `CERT_NONE`. At least one of *cafile* or *capath* must be specified.

This method can also load certification revocation lists (CRLs) in PEM or DER format. In order to make use of CRLs, `SSLContext.verify_flags` must be configured properly.

The *cafile* string, if present, is the path to a file of concatenated CA certificates in PEM format. See the discussion of `ssl-certificates` for more information about how to arrange the certificates in this file.

The *capath* string, if present, is the path to a directory containing several CA certificates in PEM format, following an [OpenSSL specific layout](https://docs.openssl.org/master/man3/SSL_CTX_load_verify_locations/).

The *cadata* object, if present, is either an ASCII string of one or more PEM-encoded certificates or a `bytes-like object` of DER-encoded certificates. Like with *capath* extra lines around PEM-encoded certificates are ignored but at least one certificate must be present.

<div class="versionchanged">

3.4 New optional argument *cadata*

</div>

</div>

<div class="method">

SSLContext.get_ca_certs(binary_form=False)

Get a list of loaded "certification authority" (CA) certificates. If the `binary_form` parameter is `False` each list entry is a dict like the output of `SSLSocket.getpeercert`. Otherwise the method returns a list of DER-encoded certificates. The returned list does not contain certificates from *capath* unless a certificate was requested and loaded by a SSL connection.

> [!NOTE]
> Certificates in a capath directory aren't loaded unless they have been used at least once.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

SSLContext.get_ciphers()

Get a list of enabled ciphers. The list is in order of cipher priority. See `SSLContext.set_ciphers`.

Example:

    >>> ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
    >>> ctx.set_ciphers('ECDHE+AESGCM:!ECDSA')
    >>> ctx.get_ciphers()
    [{'aead': True,
      'alg_bits': 256,
      'auth': 'auth-rsa',
      'description': 'ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  '
                     'Enc=AESGCM(256) Mac=AEAD',
      'digest': None,
      'id': 50380848,
      'kea': 'kx-ecdhe',
      'name': 'ECDHE-RSA-AES256-GCM-SHA384',
      'protocol': 'TLSv1.2',
      'strength_bits': 256,
      'symmetric': 'aes-256-gcm'},
     {'aead': True,
      'alg_bits': 128,
      'auth': 'auth-rsa',
      'description': 'ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  '
                     'Enc=AESGCM(128) Mac=AEAD',
      'digest': None,
      'id': 50380847,
      'kea': 'kx-ecdhe',
      'name': 'ECDHE-RSA-AES128-GCM-SHA256',
      'protocol': 'TLSv1.2',
      'strength_bits': 128,
      'symmetric': 'aes-128-gcm'}]

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

SSLContext.get_groups(\*, include_aliases=False)

Get a list of groups implemented for key agreement, taking into account the current TLS `~SSLContext.minimum_version` and `~SSLContext.maximum_version` values. For example:

    >>> ctx = ssl.create_default_context()
    >>> ctx.minimum_version = ssl.TLSVersion.TLSv1_3
    >>> ctx.maximum_version = ssl.TLSVersion.TLSv1_3
    >>> ctx.get_groups()  # doctest: +SKIP
    ['secp256r1', 'secp384r1', 'secp521r1', 'x25519', 'x448', ...]

By default, this method returns only the preferred IANA names for the available groups. However, if the `include_aliases` parameter is set to `True` this method will also return any associated aliases such as the ECDH curve names supported in older versions of OpenSSL.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

SSLContext.set_default_verify_paths()

Load a set of default "certification authority" (CA) certificates from a filesystem path defined when building the OpenSSL library. Unfortunately, there's no easy way to know whether this method succeeds: no error is returned if no certificates are to be found. When the OpenSSL library is provided as part of the operating system, though, it is likely to be configured properly.

</div>

<div class="method">

SSLContext.set_ciphers(ciphers, /)

Set the allowed ciphers for sockets created with this context when connecting using TLS 1.2 and earlier. The *ciphers* argument should be a string in the [OpenSSL cipher list format](https://docs.openssl.org/master/man1/ciphers/). To set allowed TLS 1.3 ciphers, use `SSLContext.set_ciphersuites`.

If no cipher can be selected (because compile-time options or other configuration forbids use of all the specified ciphers), an `SSLError` will be raised.

> [!NOTE]
> When connected, the `SSLSocket.cipher` method of SSL sockets will return details about the negotiated cipher.

</div>

<div class="method">

SSLContext.set_ciphersuites(ciphersuites, /)

Set the allowed ciphers for sockets created with this context when connecting using TLS 1.3. The *ciphersuites* argument should be a colon-separate string of TLS 1.3 cipher names. If no cipher can be selected (because compile-time options or other configuration forbids use of all the specified ciphers), an `SSLError` will be raised.

> [!NOTE]
> When connected, the `SSLSocket.cipher` method of SSL sockets will return details about the negotiated cipher.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

SSLContext.set_groups(groups, /)

Set the groups allowed for key agreement for sockets created with this context. It should be a string in the [OpenSSL group list format](https://docs.openssl.org/master/man3/SSL_CTX_set1_groups_list/).

> [!NOTE]
> When connected, the `SSLSocket.group` method of SSL sockets will return the group used for key agreement on that connection.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

SSLContext.set_client_sigalgs(sigalgs, /)

Set the signature algorithms allowed for certificate-based client authentication. It should be a string in the [OpenSSL client sigalgs list format](https://docs.openssl.org/master/man3/SSL_CTX_set1_client_sigalgs_list/).

> [!NOTE]
> When connected, the `SSLSocket.client_sigalg` method of SSL sockets will return the signature algorithm used for performing certificate-based client authentication on that connection.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

SSLContext.set_server_sigalgs(sigalgs, /)

Set the signature algorithms allowed for the server to complete the TLS handshake. It should be a string in the [OpenSSL sigalgs list format](https://docs.openssl.org/master/man3/SSL_CTX_set1_sigalgs_list/).

> [!NOTE]
> When connected, the `SSLSocket.server_sigalg` method of SSL sockets will return the signature algorithm used by the server to complete the TLS handshake on that connection.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

SSLContext.set_alpn_protocols(alpn_protocols)

Specify which protocols the socket should advertise during the SSL/TLS handshake. It should be a list of ASCII strings, like `['http/1.1', 'spdy/2']`, ordered by preference. The selection of a protocol will happen during the handshake, and will play out according to `7301`. After a successful handshake, the `SSLSocket.selected_alpn_protocol` method will return the agreed-upon protocol.

This method will raise `NotImplementedError` if `HAS_ALPN` is `False`.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

SSLContext.set_ech_config(ech_config)

Sets an Encrypted Client Hello (ECH) configuration, which may be discovered from an HTTPS resource record in DNS or from `SSLECHRejectedError.retry_config` after ECH rejection.

The configuration applies to every connection made from the context, and a server that cannot decrypt it rejects the connection with `SSLECHRejectedError`. Because ECH configurations are published per host, a context with a configuration should only be used for connections to the hosts that publish it. To use a configuration for a single connection, pass it as the *ech_config* argument of `wrap_socket` or `wrap_bio` instead.

Raises `SSLError` if the input is invalid or contains no usable configuration.

The *ech_config* parameter should be a `bytes-like object` containing an ECHConfigList in binary or Base64 encoding.

> [!NOTE]
> OpenSSL 4.0 can reject the entire list with `SSLError` if one entry is unusable. It also rejects binary lists whose first entry uses an unsupported ECH version, although unknown versions after a supported entry are skipped. These limitations prevent falling back to usable configurations as described in `9849`.

Raises `SSLError` if the context uses `PROTOCOL_TLS_SERVER`.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="method">

SSLContext.set_npn_protocols(npn_protocols)

Specify which protocols the socket should advertise during the SSL/TLS handshake. It should be a list of strings, like `['http/1.1', 'spdy/2']`, ordered by preference. The selection of a protocol will happen during the handshake, and will play out according to the [Application Layer Protocol Negotiation](https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation). After a successful handshake, the `SSLSocket.selected_npn_protocol` method will return the agreed-upon protocol.

This method will raise `NotImplementedError` if `HAS_NPN` is `False`.

<div class="versionadded">

3.3

</div>

<div class="deprecated">

3.10

NPN has been superseded by ALPN

</div>

</div>

<div class="method">

SSLContext.set_outer_alpn_protocols(protocols)

Specify which protocols the socket should advertise during the TLS handshake in the outer ClientHello when ECH is used. The *protocols* argument accepts the same values as for `~SSLContext.set_alpn_protocols`.

> [!NOTE]
> OpenSSL 4.0 validates the server's ALPN selection against the inner protocol list even when ECH is rejected. If the server selects an outer protocol absent from the inner list, the handshake fails with a `SSLError`. To support ECH rejection and retry on affected versions, choose outer protocols that also appear in the inner list.

Only available when `HAS_ECH` is `True`.

<div class="versionadded">

next

</div>

</div>

<div class="attribute">

SSLContext.sni_callback

Register a callback function that will be called after the TLS Client Hello handshake message has been received by the SSL/TLS server when the TLS client specifies a server name indication. The server name indication mechanism is specified in `6066` section 3 - Server Name Indication.

Only one callback can be set per `SSLContext`. If *sni_callback* is set to `None` then the callback is disabled. Calling this function a subsequent time will disable the previously registered callback.

The callback function will be called with three arguments; the first being the `ssl.SSLSocket`, the second is a string that represents the server name that the client is intending to communicate (or `None` if the TLS Client Hello does not contain a server name) and the third argument is the original `SSLContext`. The server name argument is text. For internationalized domain name, the server name is an IDN A-label (`"xn--pythn-mua.org"`).

A typical use of this callback is to change the `ssl.SSLSocket`'s `SSLSocket.context` attribute to a new object of type `SSLContext` representing a certificate chain that matches the server name.

If the callback assigns a new context to `SSLSocket.context`, any further ClientHello message on the same connection (for example after a TLS 1.3 HelloRetryRequest) is dispatched to the new context's *sni_callback*, if it has one; the original callback is not called again for that connection.

Due to the early negotiation phase of the TLS connection, only limited methods and attributes are usable like `SSLSocket.selected_alpn_protocol` and `SSLSocket.context`. The `SSLSocket.getpeercert`, `SSLSocket.get_verified_chain`, `SSLSocket.get_unverified_chain` `SSLSocket.cipher` and `SSLSocket.compression` methods require that the TLS connection has progressed beyond the TLS Client Hello and therefore will not return meaningful values nor can they be called safely.

The *sni_callback* function must return `None` to allow the TLS negotiation to continue. If a TLS failure is required, a constant `ALERT_DESCRIPTION_* <ALERT_DESCRIPTION_INTERNAL_ERROR>` can be returned. Other return values will result in a TLS fatal error with `ALERT_DESCRIPTION_INTERNAL_ERROR`.

If an exception is raised from the *sni_callback* function the TLS connection will terminate with a fatal TLS alert message `ALERT_DESCRIPTION_HANDSHAKE_FAILURE`.

This method will raise `NotImplementedError` if the OpenSSL library had OPENSSL_NO_TLSEXT defined when it was built.

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.16 After the callback assigns a new `SSLSocket.context`, later ClientHello messages on the connection are dispatched to the new context's *sni_callback*.

</div>

</div>

<div class="method">

SSLContext.set_servername_callback(server_name_callback)

This is a legacy API retained for backwards compatibility. When possible, you should use `sni_callback` instead. The given *server_name_callback* is similar to *sni_callback*, except that when the server hostname is an IDN-encoded internationalized domain name, the *server_name_callback* receives a decoded U-label (`"pythön.org"`).

If there is a decoding error on the server name, the TLS connection will terminate with an `ALERT_DESCRIPTION_INTERNAL_ERROR` fatal TLS alert message to the client.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

SSLContext.load_dh_params(dhfile, /)

Load the key generation parameters for Diffie-Hellman (DH) key exchange. Using DH key exchange improves forward secrecy at the expense of computational resources (both on the server and on the client). The *dhfile* parameter should be the path to a file containing DH parameters in PEM format.

This setting doesn't apply to client sockets. You can also use the `OP_SINGLE_DH_USE` option to further improve security.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

SSLContext.set_ecdh_curve(curve_name, /)

Set the curve name for Elliptic Curve-based Diffie-Hellman (ECDH) key exchange. ECDH is significantly faster than regular DH while arguably as secure. The *curve_name* parameter should be a string describing a well-known elliptic curve, for example `prime256v1` for a widely supported curve.

This setting doesn't apply to client sockets. You can also use the `OP_SINGLE_ECDH_USE` option to further improve security.

This method is not available if `HAS_ECDH` is `False`.

<div class="versionadded">

3.3

</div>

<div class="seealso">

[SSL/TLS & Perfect Forward Secrecy](https://vincent.bernat.ch/en/blog/2011-ssl-perfect-forward-secrecy) Vincent Bernat.

</div>

</div>

<div class="method">

SSLContext.wrap_socket(sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, session=None, ech_config=None)

Wrap an existing Python socket *sock* and return an instance of `SSLContext.sslsocket_class` (default `SSLSocket`). The returned SSL socket is tied to the context, its settings and certificates. *sock* must be a `~socket.SOCK_STREAM` socket; other socket types are unsupported.

The parameter `server_side` is a boolean which identifies whether server-side or client-side behavior is desired from this socket.

For client-side sockets, the context construction is lazy; if the underlying socket isn't connected yet, the context construction will be performed after `connect` is called on the socket. For server-side sockets, if the socket has no remote peer, it is assumed to be a listening socket, and the server-side SSL wrapping is automatically performed on client connections accepted via the `accept` method. The method may raise `SSLError`.

On client connections, the optional parameter *server_hostname* specifies the hostname of the service which we are connecting to. This allows a single server to host multiple SSL-based services with distinct certificates, quite similarly to HTTP virtual hosts. Specifying *server_hostname* will raise a `ValueError` if *server_side* is true.

The parameter `do_handshake_on_connect` specifies whether to do the SSL handshake automatically after doing a `socket.connect`, or whether the application program will call it explicitly, by invoking the `SSLSocket.do_handshake` method. Calling `SSLSocket.do_handshake` explicitly gives the program control over the blocking behavior of the socket I/O involved in the handshake.

The parameter `suppress_ragged_eofs` specifies how the `SSLSocket.recv` method should signal unexpected EOF from the other end of the connection. If specified as `True` (the default), it returns a normal EOF (an empty bytes object) in response to unexpected EOF errors raised from the underlying socket; if `False`, it will raise the exceptions back to the caller.

*session*, see `~SSLSocket.session`.

*ech_config*, see `SSLSocket.set_ech_config`. Specifying *ech_config* will raise a `ValueError` if *server_side* is true, and a `NotImplementedError` if `HAS_ECH` is `False`.

To wrap an `SSLSocket` in another `SSLSocket`, use `SSLContext.wrap_bio`.

<div class="versionchanged">

3.5 Always allow a server_hostname to be passed, even if OpenSSL does not have SNI.

</div>

<div class="versionchanged">

3.6 *session* argument was added.

</div>

<div class="versionchanged">

3.7 The method returns an instance of `SSLContext.sslsocket_class` instead of hard-coded `SSLSocket`.

</div>

<div class="versionchanged">

next *ech_config* argument was added.

</div>

</div>

<div class="attribute">

SSLContext.sslsocket_class

The return type of `SSLContext.wrap_socket`, defaults to `SSLSocket`. The attribute can be assigned to on instances of `SSLContext` in order to return a custom subclass of `SSLSocket`.

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

SSLContext.wrap_bio(incoming, outgoing, server_side=False, server_hostname=None, session=None, ech_config=None)

Wrap the BIO objects *incoming* and *outgoing* and return an instance of `SSLContext.sslobject_class` (default `SSLObject`). The SSL routines will read input data from the incoming BIO and write data to the outgoing BIO.

The *server_side*, *server_hostname*, *session* and *ech_config* parameters have the same meaning as in `SSLContext.wrap_socket`, and are validated in the same way: in particular a `ValueError` is raised when `~SSLContext.check_hostname` is enabled but no *server_hostname* is given, since there would be no name to match the peer's certificate against.

<div class="versionchanged">

3.6 *session* argument was added.

</div>

<div class="versionchanged">

3.7 The method returns an instance of `SSLContext.sslobject_class` instead of hard-coded `SSLObject`.

</div>

<div class="versionchanged">

3.16 The *server_side*, *server_hostname* and *session* parameters are now validated as `SSLContext.wrap_socket` validates them. Previously a context with `~SSLContext.check_hostname` enabled and no *server_hostname* was accepted, and verified the certificate chain but never the peer's identity.

</div>

<div class="versionchanged">

next *ech_config* argument was added.

</div>

</div>

<div class="attribute">

SSLContext.sslobject_class

The return type of `SSLContext.wrap_bio`, defaults to `SSLObject`. The attribute can be overridden on instance of class in order to return a custom subclass of `SSLObject`.

<div class="versionadded">

3.7

</div>

</div>

<div class="method">

SSLContext.session_stats()

Get statistics about the SSL sessions created or managed by this context. A dictionary is returned which maps the names of each [piece of information](https://docs.openssl.org/1.1.1/man3/SSL_CTX_sess_number/) to their numeric values. For example, here is the total number of hits and misses in the session cache since the context was created:

    >>> stats = context.session_stats()
    >>> stats['hits'], stats['misses']
    (0, 0)

</div>

<div class="attribute">

SSLContext.check_hostname

Whether to match the peer cert's hostname in `SSLSocket.do_handshake`. The context's `~SSLContext.verify_mode` must be set to `CERT_OPTIONAL` or `CERT_REQUIRED`, and you must pass *server_hostname* to `~SSLContext.wrap_socket` in order to match the hostname. Enabling hostname checking automatically sets `~SSLContext.verify_mode` from `CERT_NONE` to `CERT_REQUIRED`. It cannot be set back to `CERT_NONE` as long as hostname checking is enabled. The `PROTOCOL_TLS_CLIENT` protocol enables hostname checking by default. With other protocols, hostname checking must be enabled explicitly.

Example:

    import socket, ssl

    context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
    context.verify_mode = ssl.CERT_REQUIRED
    context.check_hostname = True
    context.load_default_certs()

    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    ssl_sock = context.wrap_socket(s, server_hostname='www.verisign.com')
    ssl_sock.connect(('www.verisign.com', 443))

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.7

`~SSLContext.verify_mode` is now automatically changed to `CERT_REQUIRED` when hostname checking is enabled and `~SSLContext.verify_mode` is `CERT_NONE`. Previously the same operation would have failed with a `ValueError`.

</div>

</div>

<div class="attribute">

SSLContext.keylog_filename

Write TLS keys to a keylog file, whenever key material is generated or received. The keylog file is designed for debugging purposes only. The file format is specified by NSS and used by many traffic analyzers such as Wireshark. The log file is opened in append-only mode. Writes are synchronized between threads, but not between processes.

<div class="versionadded">

3.8

</div>

</div>

<div class="attribute">

SSLContext.maximum_version

A `TLSVersion` enum member representing the highest supported TLS version. The value defaults to `TLSVersion.MAXIMUM_SUPPORTED`. The attribute is read-only for protocols other than `PROTOCOL_TLS`, `PROTOCOL_TLS_CLIENT`, and `PROTOCOL_TLS_SERVER`.

The attributes `~SSLContext.maximum_version`, `~SSLContext.minimum_version` and `SSLContext.options` all affect the supported SSL and TLS versions of the context. The implementation does not prevent invalid combinations. For example a context with `OP_NO_TLSv1_2` in `~SSLContext.options` and `~SSLContext.maximum_version` set to `TLSVersion.TLSv1_2` will not be able to establish a TLS 1.2 connection.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

SSLContext.minimum_version

Like `SSLContext.maximum_version` except it is the lowest supported version or `TLSVersion.MINIMUM_SUPPORTED`.

<div class="versionadded">

3.7

</div>

</div>

<div class="attribute">

SSLContext.num_tickets

Control the number of TLS 1.3 session tickets of a `PROTOCOL_TLS_SERVER` context. The setting has no impact on TLS 1.0 to 1.2 connections.

<div class="versionadded">

3.8

</div>

</div>

<div class="attribute">

SSLContext.options

An integer representing the set of SSL options enabled on this context. The default value is `OP_ALL`, but you can specify other options such as `OP_NO_SSLv2` by ORing them together.

<div class="versionchanged">

3.6 `SSLContext.options` returns `Options` flags:

\>\>\> ssl.create_default_context().options \# doctest: +SKIP \<Options.OP_ALL\|OP_NO_SSLv3\|OP_NO_SSLv2\|OP_NO_COMPRESSION: 2197947391\>

</div>

<div class="deprecated">

3.7

All `OP_NO_SSL*` and `OP_NO_TLS*` options have been deprecated since Python 3.7. Use `SSLContext.minimum_version` and `SSLContext.maximum_version` instead.

</div>

</div>

<div class="attribute">

SSLContext.post_handshake_auth

Enable TLS 1.3 post-handshake client authentication. Post-handshake auth is disabled by default and a server can only request a TLS client certificate during the initial handshake. When enabled, a server may request a TLS client certificate at any time after the handshake.

When enabled on client-side sockets, the client signals the server that it supports post-handshake authentication.

When enabled on server-side sockets, `SSLContext.verify_mode` must be set to `CERT_OPTIONAL` or `CERT_REQUIRED`, too. The actual client cert exchange is delayed until `SSLSocket.verify_client_post_handshake` is called and some I/O is performed.

<div class="versionadded">

3.8

</div>

</div>

<div class="attribute">

SSLContext.protocol

The protocol version chosen when constructing the context. This attribute is read-only.

</div>

<div class="attribute">

SSLContext.hostname_checks_common_name

Whether `~SSLContext.check_hostname` falls back to verify the cert's subject common name in the absence of a subject alternative name extension (default: true).

<div class="versionadded">

3.7

</div>

<div class="versionchanged">

3.10

The flag had no effect with OpenSSL before version 1.1.1l. Python 3.8.9, 3.9.3, and 3.10 include workarounds for previous versions.

</div>

</div>

<div class="attribute">

SSLContext.security_level

An integer representing the [security level](https://docs.openssl.org/master/man3/SSL_CTX_get_security_level/) for the context. This attribute is read-only.

<div class="versionadded">

3.10

</div>

</div>

<div class="attribute">

SSLContext.verify_flags

The flags for certificate verification operations. You can set flags like `VERIFY_CRL_CHECK_LEAF` by ORing them together. By default OpenSSL does neither require nor verify certificate revocation lists (CRLs).

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.6 `SSLContext.verify_flags` returns `VerifyFlags` flags:

\>\>\> ssl.create_default_context().verify_flags \# doctest: +SKIP \<VerifyFlags.VERIFY_X509_TRUSTED_FIRST: 32768\>

</div>

</div>

<div class="attribute">

SSLContext.verify_mode

Whether to try to verify other peers' certificates and how to behave if verification fails. This attribute must be one of `CERT_NONE`, `CERT_OPTIONAL` or `CERT_REQUIRED`.

<div class="versionchanged">

3.6 `SSLContext.verify_mode` returns `VerifyMode` enum:

\>\>\> ssl.create_default_context().verify_mode \# doctest: +SKIP \<VerifyMode.CERT_REQUIRED: 2\>

</div>

</div>

<div class="method">

SSLContext.set_psk_client_callback(callback)

Enables TLS-PSK (pre-shared key) authentication on a client-side connection.

In general, certificate based authentication should be preferred over this method.

The parameter `callback` is a callable object with the signature: `def callback(hint: str | None) -> tuple[str | None, bytes]`. The `hint` parameter is an optional identity hint sent by the server. The return value is a tuple in the form (client-identity, psk). Client-identity is an optional string which may be used by the server to select a corresponding PSK for the client. The string must be less than or equal to `256` octets when UTF-8 encoded. PSK is a `bytes-like object` representing the pre-shared key. With TLS 1.2 and earlier, return a zero length PSK to reject the connection.

Setting `callback` to `None` removes any existing callback.

> [!NOTE]
> When using TLS 1.3:
>
> - the `hint` parameter is always `None`.
> - client-identity must be a non-empty string.
> - the PSK is only offered to the server, which may decline it and authenticate with a certificate instead. That certificate is checked according to `~SSLContext.verify_mode`, so a client relying on the PSK to authenticate the server should not set it to `CERT_NONE`, or should check `SSLSocket.session_reused`, which is true if the PSK was used. A PSK handshake does not need any CA certificates to be loaded when using `CERT_REQUIRED`.
> - a zero length PSK does not reject the connection, it only means that no PSK is offered. An exception raised by the callback has the same effect.
> - the PSK is used with the SHA-256 handshake digest, so a server can only accept it with a cipher suite that uses SHA-256.

Example usage:

    context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    context.check_hostname = False
    context.maximum_version = ssl.TLSVersion.TLSv1_2
    context.set_ciphers('PSK')

    # A simple lambda:
    psk = bytes.fromhex('c0ffee')
    context.set_psk_client_callback(lambda hint: (None, psk))

    # A table using the hint from the server:
    psk_table = { 'ServerId_1': bytes.fromhex('c0ffee'),
                  'ServerId_2': bytes.fromhex('facade')
    }
    def callback(hint):
        return 'ClientId_1', psk_table.get(hint, b'')
    context.set_psk_client_callback(callback)

This method will raise `NotImplementedError` if `HAS_PSK` is `False`.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

SSLContext.set_psk_server_callback(callback, identity_hint=None)

Enables TLS-PSK (pre-shared key) authentication on a server-side connection.

In general, certificate based authentication should be preferred over this method.

The parameter `callback` is a callable object with the signature: `def callback(identity: str | None) -> bytes`. The `identity` parameter is an optional identity sent by the client which can be used to select a corresponding PSK. The return value is a `bytes-like object` representing the pre-shared key. With TLS 1.2 and earlier, return a zero length PSK to reject the connection.

Setting `callback` to `None` removes any existing callback.

The parameter `identity_hint` is an optional identity hint string sent to the client. The string must be less than or equal to `256` octets when UTF-8 encoded.

> [!NOTE]
> When using TLS 1.3:
>
> - the `identity_hint` parameter is not sent to the client.
> - a zero length PSK, or an exception raised by the callback, only declines the client's PSK. If a certificate was loaded with `~SSLContext.load_cert_chain`, the handshake continues with certificate authentication instead of failing, so a server relying on the PSK to authenticate clients should not load one, or should check `SSLSocket.session_reused`, which is true if the PSK was used.
> - the PSK can only be accepted with a cipher suite that uses SHA-256. When a certificate is loaded, OpenSSL's default TLS 1.3 cipher suite order selects a SHA-384 suite instead, so the PSK is declined even if it matches.

Example usage:

    context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    context.maximum_version = ssl.TLSVersion.TLSv1_2
    context.set_ciphers('PSK')

    # A simple lambda:
    psk = bytes.fromhex('c0ffee')
    context.set_psk_server_callback(lambda identity: psk)

    # A table using the identity of the client:
    psk_table = { 'ClientId_1': bytes.fromhex('c0ffee'),
                  'ClientId_2': bytes.fromhex('facade')
    }
    def callback(identity):
        return psk_table.get(identity, b'')
    context.set_psk_server_callback(callback, 'ServerId_1')

This method will raise `NotImplementedError` if `HAS_PSK` is `False`.

<div class="versionadded">

3.13

</div>

</div>

<div class="index">

single: certificates

</div>

<div class="index">

single: X509 certificate

</div>

## Certificates

Certificates in general are part of a public-key / private-key system. In this system, each *principal*, (which may be a machine, or a person, or an organization) is assigned a unique two-part encryption key. One part of the key is public, and is called the *public key*; the other part is kept secret, and is called the *private key*. The two parts are related, in that if you encrypt a message with one of the parts, you can decrypt it with the other part, and **only** with the other part.

A certificate contains information about two principals. It contains the name of a *subject*, and the subject's public key. It also contains a statement by a second principal, the *issuer*, that the subject is who they claim to be, and that this is indeed the subject's public key. The issuer's statement is signed with the issuer's private key, which only the issuer knows. However, anyone can verify the issuer's statement by finding the issuer's public key, decrypting the statement with it, and comparing it to the other information in the certificate. The certificate also contains information about the time period over which it is valid. This is expressed as two fields, called "notBefore" and "notAfter".

In the Python use of certificates, a client or server can use a certificate to prove who they are. The other side of a network connection can also be required to produce a certificate, and that certificate can be validated to the satisfaction of the client or server that requires such validation. The connection attempt can be set to raise an exception if the validation fails. Validation is done automatically, by the underlying OpenSSL framework; the application need not concern itself with its mechanics. But the application does usually need to provide sets of certificates to allow this process to take place.

Python uses files to contain certificates. They should be formatted as "PEM" (see `1422`), which is a base-64 encoded form wrapped with a header line and a footer line:

    -----BEGIN CERTIFICATE-----
    ... (certificate in base64 PEM encoding) ...
    -----END CERTIFICATE-----

### Certificate chains

The Python files which contain certificates can contain a sequence of certificates, sometimes called a *certificate chain*. This chain should start with the specific certificate for the principal who "is" the client or server, and then the certificate for the issuer of that certificate, and then the certificate for the issuer of *that* certificate, and so on up the chain till you get to a certificate which is *self-signed*, that is, a certificate which has the same subject and issuer, sometimes called a *root certificate*. The certificates should just be concatenated together in the certificate file. For example, suppose we had a three certificate chain, from our server certificate to the certificate of the certification authority that signed our server certificate, to the root certificate of the agency which issued the certification authority's certificate:

    -----BEGIN CERTIFICATE-----
    ... (certificate for your server)...
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    ... (the certificate for the CA)...
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    ... (the root certificate for the CA's issuer)...
    -----END CERTIFICATE-----

### CA certificates

If you are going to require validation of the other side of the connection's certificate, you need to provide a "CA certs" file, filled with the certificate chains for each issuer you are willing to trust. Again, this file just contains these chains concatenated together. For validation, Python will use the first chain it finds in the file which matches. The platform's certificates file can be used by calling `SSLContext.load_default_certs`, this is done automatically with `.create_default_context`.

### Combined key and certificate

Often the private key is stored in the same file as the certificate; in this case, only the `certfile` parameter to `SSLContext.load_cert_chain` needs to be passed. If the private key is stored with the certificate, it should come before the first certificate in the certificate chain:

    -----BEGIN RSA PRIVATE KEY-----
    ... (private key in base64 encoding) ...
    -----END RSA PRIVATE KEY-----
    -----BEGIN CERTIFICATE-----
    ... (certificate in base64 PEM encoding) ...
    -----END CERTIFICATE-----

### Self-signed certificates

If you are going to create a server that provides SSL-encrypted connection services, you will need to acquire a certificate for that service. There are many ways of acquiring appropriate certificates, such as buying one from a certification authority. Another common practice is to generate a self-signed certificate. The simplest way to do this is with the OpenSSL package, using something like the following:

    % openssl req -new -x509 -days 365 -nodes -out cert.pem -keyout cert.pem
    Generating a 1024 bit RSA private key
    .......++++++
    .............................++++++
    writing new private key to 'cert.pem'
    -----
    You are about to be asked to enter information that will be incorporated
    into your certificate request.
    What you are about to enter is what is called a Distinguished Name or a DN.
    There are quite a few fields but you can leave some blank
    For some fields there will be a default value,
    If you enter '.', the field will be left blank.
    -----
    Country Name (2 letter code) [AU]:US
    State or Province Name (full name) [Some-State]:MyState
    Locality Name (eg, city) []:Some City
    Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Organization, Inc.
    Organizational Unit Name (eg, section) []:My Group
    Common Name (eg, YOUR name) []:myserver.mygroup.myorganization.com
    Email Address []:ops@myserver.mygroup.myorganization.com
    %

The disadvantage of a self-signed certificate is that it is its own root certificate, and no one else will have it in their cache of known (and trusted) root certificates.

## Examples

### Testing for SSL support

To test for the presence of SSL support in a Python installation, user code should use the following idiom:

    try:
        import ssl
    except ImportError:
        pass
    else:
        ...  # do something that requires SSL support

### Client-side operation

This example creates a SSL context with the recommended security settings for client sockets, including automatic certificate verification:

    >>> context = ssl.create_default_context()

If you prefer to tune security settings yourself, you might create a context from scratch (but beware that you might not get the settings right):

    >>> context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    >>> context.load_verify_locations("/etc/ssl/certs/ca-bundle.crt")

(this snippet assumes your operating system places a bundle of all CA certificates in `/etc/ssl/certs/ca-bundle.crt`; if not, you'll get an error and have to adjust the location)

The `PROTOCOL_TLS_CLIENT` protocol configures the context for cert validation and hostname verification. `~SSLContext.verify_mode` is set to `CERT_REQUIRED` and `~SSLContext.check_hostname` is set to `True`. All other protocols create SSL contexts with insecure defaults.

When you use the context to connect to a server, `CERT_REQUIRED` and `~SSLContext.check_hostname` validate the server certificate: it ensures that the server certificate was signed with one of the CA certificates, checks the signature for correctness, and verifies other properties like validity and identity of the hostname:

    >>> conn = context.wrap_socket(socket.socket(socket.AF_INET),
    ...                            server_hostname="www.python.org")
    >>> conn.connect(("www.python.org", 443))

You may then fetch the certificate:

    >>> cert = conn.getpeercert()

Visual inspection shows that the certificate does identify the desired service (that is, the HTTPS host `www.python.org`):

    >>> pprint.pprint(cert)
    {'OCSP': ('http://ocsp.digicert.com',),
     'caIssuers': ('http://cacerts.digicert.com/DigiCertSHA2ExtendedValidationServerCA.crt',),
     'crlDistributionPoints': ('http://crl3.digicert.com/sha2-ev-server-g1.crl',
                               'http://crl4.digicert.com/sha2-ev-server-g1.crl'),
     'issuer': ((('countryName', 'US'),),
                (('organizationName', 'DigiCert Inc'),),
                (('organizationalUnitName', 'www.digicert.com'),),
                (('commonName', 'DigiCert SHA2 Extended Validation Server CA'),)),
     'notAfter': 'Sep  9 12:00:00 2016 GMT',
     'notBefore': 'Sep  5 00:00:00 2014 GMT',
     'serialNumber': '01BB6F00122B177F36CAB49CEA8B6B26',
     'subject': ((('businessCategory', 'Private Organization'),),
                 (('1.3.6.1.4.1.311.60.2.1.3', 'US'),),
                 (('1.3.6.1.4.1.311.60.2.1.2', 'Delaware'),),
                 (('serialNumber', '3359300'),),
                 (('streetAddress', '16 Allen Rd'),),
                 (('postalCode', '03894-4801'),),
                 (('countryName', 'US'),),
                 (('stateOrProvinceName', 'NH'),),
                 (('localityName', 'Wolfeboro'),),
                 (('organizationName', 'Python Software Foundation'),),
                 (('commonName', 'www.python.org'),)),
     'subjectAltName': (('DNS', 'www.python.org'),
                        ('DNS', 'python.org'),
                        ('DNS', 'pypi.org'),
                        ('DNS', 'docs.python.org'),
                        ('DNS', 'testpypi.org'),
                        ('DNS', 'bugs.python.org'),
                        ('DNS', 'wiki.python.org'),
                        ('DNS', 'hg.python.org'),
                        ('DNS', 'mail.python.org'),
                        ('DNS', 'packaging.python.org'),
                        ('DNS', 'pythonhosted.org'),
                        ('DNS', 'www.pythonhosted.org'),
                        ('DNS', 'test.pythonhosted.org'),
                        ('DNS', 'us.pycon.org'),
                        ('DNS', 'id.python.org')),
     'version': 3}

Now the SSL channel is established and the certificate verified, you can proceed to talk with the server:

    >>> conn.sendall(b"HEAD / HTTP/1.0\r\nHost: linuxfr.org\r\n\r\n")
    >>> pprint.pprint(conn.recv(1024).split(b"\r\n"))
    [b'HTTP/1.1 200 OK',
     b'Date: Sat, 18 Oct 2014 18:27:20 GMT',
     b'Server: nginx',
     b'Content-Type: text/html; charset=utf-8',
     b'X-Frame-Options: SAMEORIGIN',
     b'Content-Length: 45679',
     b'Accept-Ranges: bytes',
     b'Via: 1.1 varnish',
     b'Age: 2188',
     b'X-Served-By: cache-lcy1134-LCY',
     b'X-Cache: HIT',
     b'X-Cache-Hits: 11',
     b'Vary: Cookie',
     b'Strict-Transport-Security: max-age=63072000; includeSubDomains',
     b'Connection: close',
     b'',
     b'']

See the discussion of `ssl-security` below.

### Server-side operation

For server operation, typically you'll need to have a server certificate, and private key, each in a file. You'll first create a context holding the key and the certificate, so that clients can check your authenticity. Then you'll open a socket, bind it to a port, call `listen` on it, and start waiting for clients to connect:

    import socket, ssl

    context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    context.load_cert_chain(certfile="mycertfile", keyfile="mykeyfile")

    bindsocket = socket.socket()
    bindsocket.bind(('myaddr.example.com', 10023))
    bindsocket.listen(5)

When a client connects, you'll call `accept` on the socket to get the new socket from the other end, and use the context's `SSLContext.wrap_socket` method to create a server-side SSL socket for the connection:

    while True:
        newsocket, fromaddr = bindsocket.accept()
        connstream = context.wrap_socket(newsocket, server_side=True)
        try:
            deal_with_client(connstream)
        finally:
            connstream.shutdown(socket.SHUT_RDWR)
            connstream.close()

Then you'll read data from the `connstream` and do something with it till you are finished with the client (or the client is finished with you):

    def deal_with_client(connstream):
        data = connstream.recv(1024)
        # empty data means the client is finished with us
        while data:
            if not do_something(connstream, data):
                # we'll assume do_something returns False
                # when we're finished with client
                break
            data = connstream.recv(1024)
        # finished with client

And go back to listening for new client connections (of course, a real server would probably handle each client connection in a separate thread, or put the sockets in `non-blocking mode <ssl-nonblocking>` and use an event loop).

## Notes on non-blocking sockets

SSL sockets behave slightly different than regular sockets in non-blocking mode. When working with non-blocking sockets, there are thus several things you need to be aware of:

- Most `SSLSocket` methods will raise either `SSLWantWriteError` or `SSLWantReadError` instead of `BlockingIOError` if an I/O operation would block. `SSLWantReadError` will be raised if a read operation on the underlying socket is necessary, and `SSLWantWriteError` for a write operation on the underlying socket. Note that attempts to *write* to an SSL socket may require *reading* from the underlying socket first, and attempts to *read* from the SSL socket may require a prior *write* to the underlying socket.

  <div class="versionchanged">

  3.5

  In earlier Python versions, the `!SSLSocket.send` method returned zero instead of raising `SSLWantWriteError` or `SSLWantReadError`.

  </div>

- Calling `~select.select` tells you that the OS-level socket can be read from (or written to), but it does not imply that there is sufficient data at the upper SSL layer. For example, only part of an SSL frame might have arrived. Therefore, you must be ready to handle `SSLSocket.recv` and `SSLSocket.send` failures, and retry after another call to `~select.select`.

- Conversely, since the SSL layer has its own framing, a SSL socket may still have data available for reading without `~select.select` being aware of it. Therefore, you should first call `SSLSocket.recv` to drain any potentially available data, and then only block on a `~select.select` call if still necessary.

  (of course, similar provisions apply when using other primitives such as `~select.poll`, or those in the `selectors` module)

- The SSL handshake itself will be non-blocking: the `SSLSocket.do_handshake` method has to be retried until it returns successfully. Here is a synopsis using `~select.select` to wait for the socket's readiness:

      while True:
          try:
              sock.do_handshake()
              break
          except ssl.SSLWantReadError:
              select.select([sock], [], [])
          except ssl.SSLWantWriteError:
              select.select([], [sock], [])

<div class="seealso">

The `asyncio` module supports `non-blocking SSL sockets
<ssl-nonblocking>` and provides a higher level `Streams API <asyncio-streams>`. It polls for events using the `selectors` module and handles `SSLWantWriteError`, `SSLWantReadError` and `BlockingIOError` exceptions. It runs the SSL handshake asynchronously as well.

</div>

## Memory BIO support

<div class="versionadded">

3.5

</div>

Ever since the SSL module was introduced in Python 2.6, the `SSLSocket` class has provided two related but distinct areas of functionality:

- SSL protocol handling
- Network IO

The network IO API is identical to that provided by `socket.socket`, from which `SSLSocket` also inherits. This allows an SSL socket to be used as a drop-in replacement for a regular socket, making it very easy to add SSL support to an existing application.

Combining SSL protocol handling and network IO usually works well, but there are some cases where it doesn't. An example is async IO frameworks that want to use a different IO multiplexing model than the "select/poll on a file descriptor" (readiness based) model that is assumed by `socket.socket` and by the internal OpenSSL socket IO routines. This is mostly relevant for platforms like Windows where this model is not efficient. For this purpose, a reduced scope variant of `SSLSocket` called `SSLObject` is provided.

<div class="SSLObject">

A reduced-scope variant of `SSLSocket` representing an SSL protocol instance that does not contain any network IO methods. This class is typically used by framework authors that want to implement asynchronous IO for SSL through memory buffers.

This class implements an interface on top of a low-level SSL object as implemented by OpenSSL. This object captures the state of an SSL connection but does not provide any network IO itself. IO needs to be performed through separate "BIO" objects which are OpenSSL's IO abstraction layer.

This class has no public constructor. An `SSLObject` instance must be created using the `~SSLContext.wrap_bio` method. This method will create the `SSLObject` instance and bind it to a pair of BIOs. The *incoming* BIO is used to pass data from Python to the SSL protocol instance, while the *outgoing* BIO is used to pass data the other way around.

The following methods are available:

- `~SSLSocket.context`
- `~SSLSocket.server_side`
- `~SSLSocket.server_hostname`
- `~SSLSocket.session`
- `~SSLSocket.session_reused`
- `~SSLSocket.read`
- `~SSLSocket.write`
- `~SSLSocket.getpeercert`
- `~SSLSocket.get_verified_chain`
- `~SSLSocket.get_unverified_chain`
- `~SSLSocket.selected_alpn_protocol`
- `~SSLSocket.selected_npn_protocol`
- `~SSLSocket.cipher`
- `~SSLSocket.shared_ciphers`
- `~SSLSocket.compression`
- `~SSLSocket.pending`
- `~SSLSocket.do_handshake`
- `~SSLSocket.verify_client_post_handshake`
- `~SSLSocket.unwrap`
- `~SSLSocket.get_channel_binding`
- `~SSLSocket.set_ech_config`
- `~SSLSocket.get_ech_retry_config`
- `~SSLSocket.get_ech_status`
- `~SSLSocket.version`

When compared to `SSLSocket`, this object lacks the following features:

- Any form of network IO; `recv()` and `send()` read and write only to the underlying `MemoryBIO` buffers.
- There is no *do_handshake_on_connect* machinery. You must always manually call `~SSLSocket.do_handshake` to start the handshake.
- There is no handling of *suppress_ragged_eofs*. All end-of-file conditions that are in violation of the protocol are reported via the `SSLEOFError` exception.
- The method `~SSLSocket.unwrap` call does not return anything, unlike for an SSL socket where it returns the underlying socket.
- The *server_name_callback* callback passed to `SSLContext.set_servername_callback` will get an `SSLObject` instance instead of a `SSLSocket` instance as its first parameter.

Some notes related to the use of `SSLObject`:

- All IO on an `SSLObject` is `non-blocking <ssl-nonblocking>`. This means that for example `~SSLSocket.read` will raise an `SSLWantReadError` if it needs more data than the incoming BIO has available.

<div class="versionchanged">

3.7 `SSLObject` instances must be created with `~SSLContext.wrap_bio`. In earlier versions, it was possible to create instances directly. This was never documented or officially supported.

</div>

</div>

An SSLObject communicates with the outside world using memory buffers. The class `MemoryBIO` provides a memory buffer that can be used for this purpose. It wraps an OpenSSL memory BIO (Basic IO) object:

<div class="MemoryBIO">

A memory buffer that can be used to pass data between Python and an SSL protocol instance.

<div class="attribute">

MemoryBIO.pending

Return the number of bytes currently in the memory buffer.

</div>

<div class="attribute">

MemoryBIO.eof

A boolean indicating whether the memory BIO is current at the end-of-file position.

</div>

<div class="method">

MemoryBIO.read(n=-1, /)

Read up to *n* bytes from the memory buffer. If *n* is not specified or negative, all bytes are returned.

</div>

<div class="method">

MemoryBIO.write(buf, /)

Write the bytes from *buf* to the memory BIO. The *buf* argument must be an object supporting the buffer protocol.

The return value is the number of bytes written, which is always equal to the length of *buf*.

</div>

<div class="method">

MemoryBIO.write_eof()

Write an EOF marker to the memory BIO. After this method has been called, it is illegal to call `~MemoryBIO.write`. The attribute `eof` will become true after all data currently in the buffer has been read.

</div>

</div>

## SSL session

<div class="versionadded">

3.6

</div>

<div class="SSLSession">

Session object used by `~SSLSocket.session`.

<div class="attribute">

id

</div>

<div class="attribute">

time

</div>

<div class="attribute">

timeout

</div>

<div class="attribute">

ticket_lifetime_hint

</div>

<div class="attribute">

has_ticket

</div>

</div>

## Security considerations

### Best defaults

For **client use**, if you don't have any special requirements for your security policy, it is highly recommended that you use the `create_default_context` function to create your SSL context. It will load the system's trusted CA certificates, enable certificate validation and hostname checking, and try to choose reasonably secure protocol and cipher settings.

For example, here is how you would use the `smtplib.SMTP` class to create a trusted, secure connection to a SMTP server:

    >>> import ssl, smtplib
    >>> smtp = smtplib.SMTP("mail.python.org", port=587)
    >>> context = ssl.create_default_context()
    >>> smtp.starttls(context=context)
    (220, b'2.0.0 Ready to start TLS')

If a client certificate is needed for the connection, it can be added with `SSLContext.load_cert_chain`.

By contrast, if you create the SSL context by calling the `SSLContext` constructor yourself, it will not have certificate validation nor hostname checking enabled by default. If you do so, please read the paragraphs below to achieve a good security level.

### Manual settings

#### Verifying certificates

When calling the `SSLContext` constructor directly, `CERT_NONE` is the default. Since it does not authenticate the other peer, it can be insecure, especially in client mode where most of the time you would like to ensure the authenticity of the server you're talking to. Therefore, when in client mode, it is highly recommended to use `CERT_REQUIRED`. However, it is in itself not sufficient; you also have to check that the server certificate, which can be obtained by calling `SSLSocket.getpeercert`, matches the desired service. For many protocols and applications, the service can be identified by the hostname. This common check is automatically performed when `SSLContext.check_hostname` is enabled.

<div class="versionchanged">

3.7 Hostname matchings is now performed by OpenSSL. Python no longer uses `!match_hostname`.

</div>

In server mode, if you want to authenticate your clients using the SSL layer (rather than using a higher-level authentication mechanism), you'll also have to specify `CERT_REQUIRED` and similarly check the client certificate.

#### Protocol versions

SSL versions 2 and 3 are considered insecure and are therefore dangerous to use. If you want maximum compatibility between clients and servers, it is recommended to use `PROTOCOL_TLS_CLIENT` or `PROTOCOL_TLS_SERVER` as the protocol version. SSLv2 and SSLv3 are disabled by default.

    >>> client_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    >>> client_context.minimum_version = ssl.TLSVersion.TLSv1_2
    >>> client_context.maximum_version = ssl.TLSVersion.TLSv1_3

The SSL client context created above will only allow TLSv1.2 and TLSv1.3 (if supported by your system) connections to a server. `PROTOCOL_TLS_CLIENT` implies certificate validation and hostname checks by default. You have to load certificates into the context.

#### Cipher selection

If you have advanced security requirements, fine-tuning of the ciphers enabled when negotiating a SSL session is possible through the `SSLContext.set_ciphers` method. Starting from Python 3.2.3, the ssl module disables certain weak ciphers by default, but you may want to further restrict the cipher choice. Be sure to read OpenSSL's documentation about the [cipher list format](https://docs.openssl.org/1.1.1/man1/ciphers/#cipher-list-format). If you want to check which ciphers are enabled by a given cipher list, use `SSLContext.get_ciphers` or the `openssl ciphers` command on your system.

### Multi-processing

If using this module as part of a multi-processed application (using, for example the `multiprocessing` or `concurrent.futures` modules), be aware that OpenSSL's internal random number generator does not properly handle forked processes. Applications must change the PRNG state of the parent process if they use any SSL feature with `os.fork`. Any successful call of `~ssl.RAND_add` or `~ssl.RAND_bytes` is sufficient.

## TLS 1.3

<div class="versionadded">

3.7

</div>

The TLS 1.3 protocol behaves slightly differently than previous version of TLS/SSL. Some new TLS 1.3 features are not yet available.

- TLS 1.3 uses a disjunct set of cipher suites. All AES-GCM and ChaCha20 cipher suites are enabled by default. To restrict which TLS 1.3 ciphers are allowed, the `SSLContext.set_ciphersuites` method should be called instead of `SSLContext.set_ciphers`, which only affects ciphers in older TLS versions. The `SSLContext.get_ciphers` method returns information about ciphers for both TLS 1.3 and earlier versions and the method `SSLSocket.cipher` returns information about the negotiated cipher for both TLS 1.3 and earlier versions once a connection is established.
- Session tickets are no longer sent as part of the initial handshake and are handled differently. `SSLSocket.session` and `SSLSession` are not compatible with TLS 1.3.
- Client-side certificates are also no longer verified during the initial handshake. A server can request a certificate at any time. Clients process certificate requests while they send or receive application data from the server.
- TLS 1.3 features like early data, deferred TLS client cert request, and rekeying are not supported yet.

### Encrypted Client Hello

<div class="versionadded">

next

</div>

Encrypted Client Hello (ECH) encrypts values in the ClientHello record sent when establishing a TLS connection. It requires an ECHConfigList containing algorithm parameters, a public key, and a public name used in the unencrypted outer ClientHello. ECH requires OpenSSL 4.0 or later with ECH enabled; check `HAS_ECH` before using it.

Servers can load a PEM file containing their ECH private key and configuration with `SSLContext.load_ech_keys`. Clients use the corresponding public ECHConfigList, published in DNS for each host, as the *ech_config* argument of `SSLContext.wrap_socket` or `SSLContext.wrap_bio`, or for every connection from a context with `SSLContext.set_ech_config`.

A server that has rotated its ECH keys rejects the connection and provides a retry configuration, which the handshake raises as `SSLECHRejectedError`. The following example uses a binary-encoded ECHConfigList discovered through DNS and retries once with the server's `SSLECHRejectedError.retry_config` if it is not empty, or without ECH if the authenticated server has securely disabled it.

The following example shows how to connect to a website with HTTP with an ECH configuration:

    import http.client
    import socket
    import ssl


    def connect_with_tls_ech(hostname, ech_config):
        context = ssl.create_default_context()
        context.set_ech_config(ech_config)
        sock = socket.create_connection((hostname, 443), timeout=10)
        family = sock.family
        address = sock.getpeername()
        for attempt in range(2):
            try:
                if attempt:
                    sock = socket.socket(family, socket.SOCK_STREAM)
                    sock.settimeout(10)
                    sock.connect(address)
                return context.wrap_socket(sock, server_hostname=hostname)
            except ssl.SSLECHRejectedError as exc:
                sock.close()
                if attempt:
                    raise
                if exc.retry_config:
                    context.set_ech_config(exc.retry_config)
                else:
                    context = ssl.create_default_context()
            except BaseException:
                sock.close()
                raise

    hostname = "python.org"
    ech_config = ...  # Replace with an ECHConfigList discovered through DNS.

    with connect_with_tls_ech(hostname, ech_config) as ssock:
        print("ECH status:", ssock.get_ech_status().name)
        request = (
            "GET / HTTP/1.1\r\n"
            f"Host: {hostname}\r\n"
            "Connection: close\r\n\r\n"
        )
        ssock.sendall(request.encode("ascii"))
        with http.client.HTTPResponse(ssock) as response:
            response.begin()
            print(f"HTTP {response.status} {response.reason}")
            for name, value in response.getheaders():
                print(f"{name}: {value}")
            print()
            print(response.read().decode("utf-8", errors="replace"))

The following classes, methods, and attributes will be useful for using ECH:

> - `ECHStatus`
> - `SSLECHRejectedError`
> - `SSLContext.load_ech_keys`
> - `SSLContext.set_ech_config`
> - `SSLContext.set_outer_alpn_protocols`
> - `SSLSocket.set_ech_config`
> - `SSLSocket.get_ech_status`
> - `SSLSocket.get_ech_retry_config`

<div class="seealso">

Class `socket.socket`  
Documentation of underlying `socket` class

[SSL/TLS Strong Encryption: An Introduction](https://httpd.apache.org/docs/trunk/en/ssl/ssl_intro.html)  
Intro from the Apache HTTP Server documentation

`RFC 1422: Privacy Enhancement for Internet Electronic Mail: Part II: Certificate-Based Key Management <1422>`  
Steve Kent

`RFC 4086: Randomness Requirements for Security <4086>`  
Donald E. Eastlake, Jeffrey I. Schiller, Steve Crocker

`RFC 5280: Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile <5280>`  
David Cooper et al.

`RFC 5246: The Transport Layer Security (TLS) Protocol Version 1.2 <5246>`  
Tim Dierks and Eric Rescorla.

`RFC 6066: Transport Layer Security (TLS) Extensions <6066>`  
Donald E. Eastlake

[IANA TLS: Transport Layer Security (TLS) Parameters](https://www.iana.org/assignments/tls-parameters/tls-parameters.xml)  
IANA

`RFC 7525: Recommendations for Secure Use of Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS) <7525>`  
IETF

[Mozilla's Server Side TLS recommendations](https://wiki.mozilla.org/Security/Server_Side_TLS)  
Mozilla

</div>

[^1]: TLS 1.3 protocol will be available with `PROTOCOL_TLS` in OpenSSL \>= 1.1.1. There is no dedicated PROTOCOL constant for just TLS 1.3.

[^2]: `SSLContext` disables SSLv2 with `OP_NO_SSLv2` by default.

[^3]: `SSLContext` disables SSLv3 with `OP_NO_SSLv3` by default.

[^4]: TLS 1.3 protocol will be available with `PROTOCOL_TLS` in OpenSSL \>= 1.1.1. There is no dedicated PROTOCOL constant for just TLS 1.3.

[^5]: `SSLContext` disables SSLv2 with `OP_NO_SSLv2` by default.

[^6]: `SSLContext` disables SSLv3 with `OP_NO_SSLv3` by default.

# `!stat` --- Interpreting `~os.stat` results

<div class="module" data-synopsis="Utilities for interpreting the results of os.stat(),
os.lstat() and os.fstat().">

stat

</div>

**Source code:** `Lib/stat.py`

------------------------------------------------------------------------

The `!stat` module defines constants and functions for interpreting the results of `os.stat`, `os.fstat` and `os.lstat` (if they exist). For complete details about the `stat`, `!fstat` and `!lstat` calls, consult the documentation for your system.

<div class="versionchanged">

3.4 The stat module is backed by a C implementation.

</div>

The `!stat` module defines the following functions to test for specific file types:

<div class="function">

S_ISDIR(mode)

Return non-zero if the mode is from a directory.

</div>

<div class="function">

S_ISCHR(mode)

Return non-zero if the mode is from a character special device file.

</div>

<div class="function">

S_ISBLK(mode)

Return non-zero if the mode is from a block special device file.

</div>

<div class="function">

S_ISREG(mode)

Return non-zero if the mode is from a regular file.

</div>

<div class="function">

S_ISFIFO(mode)

Return non-zero if the mode is from a FIFO (named pipe).

</div>

<div class="function">

S_ISLNK(mode)

Return non-zero if the mode is from a symbolic link.

</div>

<div class="function">

S_ISSOCK(mode)

Return non-zero if the mode is from a socket.

</div>

<div class="function">

S_ISDOOR(mode)

Return non-zero if the mode is from a door.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

S_ISPORT(mode)

Return non-zero if the mode is from an event port.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

S_ISWHT(mode)

Return non-zero if the mode is from a whiteout.

<div class="versionadded">

3.4

</div>

</div>

Two additional functions are defined for more general manipulation of the file's mode:

<div class="function">

S_IMODE(mode)

Return the portion of the file's mode that can be set by `os.chmod`---that is, the file's permission bits, plus the sticky bit, set-group-id, and set-user-id bits (on systems that support them).

</div>

<div class="function">

S_IFMT(mode)

Return the portion of the file's mode that describes the file type (used by the `!S_IS\*` functions above).

</div>

Normally, you would use the `!os.path.is\*` functions for testing the type of a file; the functions here are useful when you are doing multiple tests of the same file and wish to avoid the overhead of the `stat` system call for each test. These are also useful when checking for information about a file that isn't handled by `os.path`, like the tests for block and character devices.

Example:

    import os, sys
    from stat import *

    def walktree(top, callback):
        '''recursively descend the directory tree rooted at top,
           calling the callback function for each regular file'''

        for f in os.listdir(top):
            pathname = os.path.join(top, f)
            mode = os.lstat(pathname).st_mode
            if S_ISDIR(mode):
                # It's a directory, recurse into it
                walktree(pathname, callback)
            elif S_ISREG(mode):
                # It's a file, call the callback function
                callback(pathname)
            else:
                # Unknown file type, print a message
                print('Skipping %s' % pathname)

    def visitfile(file):
        print('visiting', file)

    if __name__ == '__main__':
        walktree(sys.argv[1], visitfile)

An additional utility function is provided to convert a file's mode in a human readable string:

<div class="function">

filemode(mode)

Convert a file's mode to a string of the form '-rwxrwxrwx'.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.4 The function supports `S_IFDOOR`, `S_IFPORT` and `S_IFWHT`.

</div>

</div>

All the variables below are simply symbolic indexes into the 10-tuple returned by `os.stat`, `os.fstat` or `os.lstat`.

<div class="data">

ST_MODE

Inode protection mode.

</div>

<div class="data">

ST_INO

Inode number.

</div>

<div class="data">

ST_DEV

Device inode resides on.

</div>

<div class="data">

ST_NLINK

Number of links to the inode.

</div>

<div class="data">

ST_UID

User id of the owner.

</div>

<div class="data">

ST_GID

Group id of the owner.

</div>

<div class="data">

ST_SIZE

Size in bytes of a plain file; amount of data waiting on some special files.

</div>

<div class="data">

ST_ATIME

Time of last access.

</div>

<div class="data">

ST_MTIME

Time of last modification.

</div>

<div class="data">

ST_CTIME

The "ctime" as reported by the operating system. On some systems (like Unix) is the time of the last metadata change, and, on others (like Windows), is the creation time (see platform documentation for details).

</div>

The interpretation of "file size" changes according to the file type. For plain files this is the size of the file in bytes. For FIFOs and sockets under most flavors of Unix (including Linux in particular), the "size" is the number of bytes waiting to be read at the time of the call to `os.stat`, `os.fstat`, or `os.lstat`; this can sometimes be useful, especially for polling one of these special files after a non-blocking open. The meaning of the size field for other character and block devices varies more, depending on the implementation of the underlying system call.

The variables below define the flags used in the `ST_MODE` field.

Use of the functions above is more portable than use of the first set of flags:

<div class="data">

S_IFSOCK

Socket.

</div>

<div class="data">

S_IFLNK

Symbolic link.

</div>

<div class="data">

S_IFREG

Regular file.

</div>

<div class="data">

S_IFBLK

Block device.

</div>

<div class="data">

S_IFDIR

Directory.

</div>

<div class="data">

S_IFCHR

Character device.

</div>

<div class="data">

S_IFIFO

FIFO.

</div>

<div class="data">

S_IFDOOR

Door.

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

S_IFPORT

Event port.

<div class="versionadded">

3.4

</div>

</div>

<div class="data">

S_IFWHT

Whiteout.

<div class="versionadded">

3.4

</div>

</div>

> [!NOTE]
> `S_IFDOOR`, `S_IFPORT` or `S_IFWHT` are defined as 0 when the platform does not have support for the file types.

The following flags can also be used in the *mode* argument of `os.chmod`:

<div class="data">

S_ISUID

Set UID bit.

</div>

<div class="data">

S_ISGID

Set-group-ID bit. This bit has several special uses. For a directory it indicates that BSD semantics is to be used for that directory: files created there inherit their group ID from the directory, not from the effective group ID of the creating process, and directories created there will also get the `S_ISGID` bit set. For a file that does not have the group execution bit (`S_IXGRP`) set, the set-group-ID bit indicates mandatory file/record locking (see also `S_ENFMT`).

</div>

<div class="data">

S_ISVTX

Sticky bit. When this bit is set on a directory it means that a file in that directory can be renamed or deleted only by the owner of the file, by the owner of the directory, or by a privileged process.

</div>

<div class="data">

S_IRWXU

Mask for file owner permissions.

</div>

<div class="data">

S_IRUSR

Owner has read permission.

</div>

<div class="data">

S_IWUSR

Owner has write permission.

</div>

<div class="data">

S_IXUSR

Owner has execute permission.

</div>

<div class="data">

S_IRWXG

Mask for group permissions.

</div>

<div class="data">

S_IRGRP

Group has read permission.

</div>

<div class="data">

S_IWGRP

Group has write permission.

</div>

<div class="data">

S_IXGRP

Group has execute permission.

</div>

<div class="data">

S_IRWXO

Mask for permissions for others (not in group).

</div>

<div class="data">

S_IROTH

Others have read permission.

</div>

<div class="data">

S_IWOTH

Others have write permission.

</div>

<div class="data">

S_IXOTH

Others have execute permission.

</div>

<div class="data">

S_ENFMT

System V file locking enforcement. This flag is shared with `S_ISGID`: file/record locking is enforced on files that do not have the group execution bit (`S_IXGRP`) set.

</div>

<div class="data">

S_IREAD

Unix V7 synonym for `S_IRUSR`.

</div>

<div class="data">

S_IWRITE

Unix V7 synonym for `S_IWUSR`.

</div>

<div class="data">

S_IEXEC

Unix V7 synonym for `S_IXUSR`.

</div>

The following flags can be used in the *flags* argument of `os.chflags`:

<div class="data">

UF_SETTABLE

All user settable flags.

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

UF_NODUMP

Do not dump the file.

</div>

<div class="data">

UF_IMMUTABLE

The file may not be changed.

</div>

<div class="data">

UF_APPEND

The file may only be appended to.

</div>

<div class="data">

UF_OPAQUE

The directory is opaque when viewed through a union stack.

</div>

<div class="data">

UF_NOUNLINK

The file may not be renamed or deleted.

</div>

<div class="data">

UF_COMPRESSED

The file is stored compressed (macOS 10.6+).

</div>

<div class="data">

UF_TRACKED

Used for handling document IDs (macOS)

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

UF_DATAVAULT

The file needs an entitlement for reading or writing (macOS 10.13+)

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

UF_HIDDEN

The file should not be displayed in a GUI (macOS 10.5+).

</div>

<div class="data">

SF_SETTABLE

All super-user changeable flags

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

SF_SUPPORTED

All super-user supported flags

<div class="availability">

macOS

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

SF_SYNTHETIC

All super-user read-only synthetic flags

<div class="availability">

macOS

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

SF_ARCHIVED

The file may be archived.

</div>

<div class="data">

SF_IMMUTABLE

The file may not be changed.

</div>

<div class="data">

SF_APPEND

The file may only be appended to.

</div>

<div class="data">

SF_RESTRICTED

The file needs an entitlement to write to (macOS 10.13+)

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

SF_NOUNLINK

The file may not be renamed or deleted.

</div>

<div class="data">

SF_SNAPSHOT

The file is a snapshot file.

</div>

<div class="data">

SF_FIRMLINK

The file is a firmlink (macOS 10.15+)

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

SF_DATALESS

The file is a dataless object (macOS 10.15+)

<div class="versionadded">

3.13

</div>

</div>

See the \*BSD or macOS systems man page `chflags(2)` for more information.

On Windows, the following file attribute constants are available for use when testing bits in the `st_file_attributes` member returned by `os.stat`. See the [Windows API documentation](https://msdn.microsoft.com/en-us/library/windows/desktop/gg258117.aspx) for more detail on the meaning of these constants.

<div class="data">

FILE_ATTRIBUTE_ARCHIVE FILE_ATTRIBUTE_COMPRESSED FILE_ATTRIBUTE_DEVICE FILE_ATTRIBUTE_DIRECTORY FILE_ATTRIBUTE_ENCRYPTED FILE_ATTRIBUTE_HIDDEN FILE_ATTRIBUTE_INTEGRITY_STREAM FILE_ATTRIBUTE_NORMAL FILE_ATTRIBUTE_NOT_CONTENT_INDEXED FILE_ATTRIBUTE_NO_SCRUB_DATA FILE_ATTRIBUTE_OFFLINE FILE_ATTRIBUTE_READONLY FILE_ATTRIBUTE_REPARSE_POINT FILE_ATTRIBUTE_SPARSE_FILE FILE_ATTRIBUTE_SYSTEM FILE_ATTRIBUTE_TEMPORARY FILE_ATTRIBUTE_VIRTUAL

<div class="versionadded">

3.5

</div>

</div>

On Windows, the following constants are available for comparing against the `st_reparse_tag` member returned by `os.lstat`. These are well-known constants, but are not an exhaustive list.

<div class="data">

IO_REPARSE_TAG_SYMLINK IO_REPARSE_TAG_MOUNT_POINT IO_REPARSE_TAG_APPEXECLINK

<div class="versionadded">

3.8

</div>

</div>

On Linux, the following file attribute constants are available for use when testing bits in the `~os.statx_result.stx_attributes` and `~os.statx_result.stx_attributes_mask` members returned by `os.statx`. See the `statx(2)` man page for more detail on the meaning of these constants.

<div class="data">

STATX_ATTR_COMPRESSED STATX_ATTR_IMMUTABLE STATX_ATTR_APPEND STATX_ATTR_NODUMP STATX_ATTR_ENCRYPTED STATX_ATTR_AUTOMOUNT STATX_ATTR_MOUNT_ROOT STATX_ATTR_VERITY STATX_ATTR_DAX STATX_ATTR_WRITE_ATOMIC

<div class="versionadded">

3.15

</div>

</div>

# `!statistics` --- Mathematical statistics functions

<div class="module" data-synopsis="Mathematical statistics functions">

statistics

</div>

<div class="versionadded">

3.4

</div>

**Source code:** `Lib/statistics.py`

<div class="testsetup">

- 

from statistics import \* import math \_\_name\_\_ = '\<doctest\>'

</div>

------------------------------------------------------------------------

This module provides functions for calculating mathematical statistics of numeric (`~numbers.Real`-valued) data.

The module is not intended to be a competitor to third-party libraries such as [NumPy](https://numpy.org), [SciPy](https://scipy.org/), or proprietary full-featured statistics packages aimed at professional statisticians such as Minitab, SAS and Matlab. It is aimed at the level of graphing and scientific calculators.

Unless explicitly noted, these functions support `int`, `float`, `~decimal.Decimal` and `~fractions.Fraction`. Behaviour with other types (whether in the numeric tower or not) is currently unsupported. Collections with a mix of types are also undefined and implementation-dependent. If your input data consists of mixed types, you may be able to use `map` to ensure a consistent result, for example: `map(float, input_data)`.

Some datasets use `NaN` (not a number) values to represent missing data. Since NaNs have unusual comparison semantics, they cause surprising or undefined behaviors in the statistics functions that sort data or that count occurrences. The functions affected are `median()`, `median_low()`, `median_high()`, `median_grouped()`, `mode()`, `multimode()`, and `quantiles()`. The `NaN` values should be stripped before calling these functions:

    >>> from statistics import median
    >>> from math import isnan
    >>> from itertools import filterfalse

    >>> data = [20.7, float('NaN'),19.2, 18.3, float('NaN'), 14.4]
    >>> sorted(data)  # This has surprising behavior
    [20.7, nan, 14.4, 18.3, 19.2, nan]
    >>> median(data)  # This result is unexpected
    16.35

    >>> sum(map(isnan, data))    # Number of missing values
    2
    >>> clean = list(filterfalse(isnan, data))  # Strip NaN values
    >>> clean
    [20.7, 19.2, 18.3, 14.4]
    >>> sorted(clean)  # Sorting now works as expected
    [14.4, 18.3, 19.2, 20.7]
    >>> median(clean)       # This result is now well defined
    18.75

## Averages and measures of central location

These functions calculate an average or typical value from a population or sample.

|  |  |
|----|----|
| `mean` | Arithmetic mean ("average") of data. |
| `fmean` | Fast, floating-point arithmetic mean, with optional weighting. |
| `geometric_mean` | Geometric mean of data. |
| `harmonic_mean` | Harmonic mean of data. |
| `kde` | Estimate the probability density distribution of the data. |
| `kde_random` | Random sampling from the PDF generated by kde(). |
| `median` | Median (middle value) of data. |
| `median_low` | Low median of data. |
| `median_high` | High median of data. |
| `median_grouped` | Median (50th percentile) of grouped data. |
| `mode` | Single mode (most common value) of discrete or nominal data. |
| `multimode` | List of modes (most common values) of discrete or nominal data. |
| `quantiles` | Divide data into intervals with equal probability. |

## Measures of spread

These functions calculate a measure of how much the population or sample tends to deviate from the typical or average values.

|             |                                        |
|-------------|----------------------------------------|
| `pstdev`    | Population standard deviation of data. |
| `pvariance` | Population variance of data.           |
| `stdev`     | Sample standard deviation of data.     |
| `variance`  | Sample variance of data.               |

## Statistics for relations between two inputs

These functions calculate statistics regarding relations between two inputs.

|                     |                                                   |
|---------------------|---------------------------------------------------|
| `covariance`        | Sample covariance for two variables.              |
| `correlation`       | Pearson and Spearman's correlation coefficients.  |
| `linear_regression` | Slope and intercept for simple linear regression. |

## Function details

Note: The functions do not require the data given to them to be sorted. However, for reading convenience, most of the examples show sorted sequences.

<div class="function">

mean(data)

Return the sample arithmetic mean of *data* which can be a sequence or iterable.

The arithmetic mean is the sum of the data divided by the number of data points. It is commonly called "the average", although it is only one of many different mathematical averages. It is a measure of the central location of the data.

If *data* is empty, `StatisticsError` will be raised.

Some examples of use:

<div class="doctest">

\>\>\> mean(\[1, 2, 3, 4, 4\]) 2.8 \>\>\> mean(\[-1.0, 2.5, 3.25, 5.75\]) 2.625

\>\>\> from fractions import Fraction as F \>\>\> mean(\[F(3, 7), F(1, 21), F(5, 3), F(1, 3)\]) Fraction(13, 21)

\>\>\> from decimal import Decimal as D \>\>\> mean(\[D("0.5"), D("0.75"), D("0.625"), D("0.375")\]) Decimal('0.5625')

</div>

> [!NOTE]
> The mean is strongly affected by [outliers](https://en.wikipedia.org/wiki/Outlier) and is not necessarily a typical example of the data points. For a more robust, although less efficient, measure of [central tendency](https://en.wikipedia.org/wiki/Central_tendency), see `median`.
>
> The sample mean gives an unbiased estimate of the true population mean, so that when taken on average over all the possible samples, `mean(sample)` converges on the true mean of the entire population. If *data* represents the entire population rather than a sample, then `mean(data)` is equivalent to calculating the true population mean μ.

</div>

<div class="function">

fmean(data, weights=None)

Convert *data* to floats and compute the arithmetic mean.

This runs faster than the `mean` function and it always returns a `float`. The *data* may be a sequence or iterable. If the input dataset is empty, raises a `StatisticsError`.

<div class="doctest">

\>\>\> fmean(\[3.5, 4.0, 5.25\]) 4.25

</div>

Optional weighting is supported. For example, a professor assigns a grade for a course by weighting quizzes at 20%, homework at 20%, a midterm exam at 30%, and a final exam at 30%:

<div class="doctest">

\>\>\> grades = \[85, 92, 83, 91\] \>\>\> weights = \[0.20, 0.20, 0.30, 0.30\] \>\>\> fmean(grades, weights) 87.6

</div>

If *weights* is supplied, it must be the same length as the *data* or a `ValueError` will be raised.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.11 Added support for *weights*.

</div>

</div>

<div class="function">

geometric_mean(data)

Convert *data* to floats and compute the geometric mean.

The geometric mean indicates the central tendency or typical value of the *data* using the product of the values (as opposed to the arithmetic mean which uses their sum).

Raises a `StatisticsError` if the input dataset is empty, if it contains a zero, or if it contains a negative value. The *data* may be a sequence or iterable.

No special efforts are made to achieve exact results. (However, this may change in the future.)

<div class="doctest">

\>\>\> round(geometric_mean(\[54, 24, 36\]), 1) 36.0

</div>

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

harmonic_mean(data, weights=None)

Return the harmonic mean of *data*, a sequence or iterable of real-valued numbers. If *weights* is omitted or `None`, then equal weighting is assumed.

The harmonic mean is the reciprocal of the arithmetic `mean` of the reciprocals of the data. For example, the harmonic mean of three values *a*, *b* and *c* will be equivalent to `3/(1/a + 1/b + 1/c)`. If one of the values is zero, the result will be zero.

The harmonic mean is a type of average, a measure of the central location of the data. It is often appropriate when averaging ratios or rates, for example speeds.

Suppose a car travels 10 km at 40 km/hr, then another 10 km at 60 km/hr. What is the average speed?

<div class="doctest">

\>\>\> harmonic_mean(\[40, 60\]) 48.0

</div>

Suppose a car travels 40 km/hr for 5 km, and when traffic clears, speeds-up to 60 km/hr for the remaining 30 km of the journey. What is the average speed?

<div class="doctest">

\>\>\> harmonic_mean(\[40, 60\], weights=\[5, 30\]) 56.0

</div>

`StatisticsError` is raised if *data* is empty, any element is less than zero, or if the weighted sum isn't positive.

The current algorithm has an early-out when it encounters a zero in the input. This means that the subsequent inputs are not tested for validity. (This behavior may change in the future.)

<div class="versionadded">

3.6

</div>

<div class="versionchanged">

3.10 Added support for *weights*.

</div>

</div>

<div class="function">

kde(data, h, kernel='normal', \*, cumulative=False)

[Kernel Density Estimation (KDE)](https://www.itm-conferences.org/articles/itmconf/pdf/2018/08/itmconf_sam2018_00037.pdf): Create a continuous probability density function or cumulative distribution function from discrete samples.

The basic idea is to smooth the data using [a kernel function](https://en.wikipedia.org/wiki/Kernel_(statistics)). to help draw inferences about a population from a sample.

The degree of smoothing is controlled by the scaling parameter *h* which is called the bandwidth. Smaller values emphasize local features while larger values give smoother results.

The *kernel* determines the relative weights of the sample data points. Generally, the choice of kernel shape does not matter as much as the more influential bandwidth smoothing parameter.

Kernels that give some weight to every sample point include *normal* (*gauss*), *logistic*, and *sigmoid*.

Kernels that only give weight to sample points within the bandwidth include *rectangular* (*uniform*), *triangular*, *parabolic* (*epanechnikov*), *quartic* (*biweight*), *triweight*, and *cosine*.

If *cumulative* is true, will return a cumulative distribution function.

A `StatisticsError` will be raised if the *data* sequence is empty.

[Wikipedia has an example](https://en.wikipedia.org/wiki/Kernel_density_estimation#Example) where we can use `kde` to generate and plot a probability density function estimated from a small sample:

<div class="doctest">

\>\>\> sample = \[-2.1, -1.3, -0.4, 1.9, 5.1, 6.2\] \>\>\> f_hat = kde(sample, h=1.5) \>\>\> xarr = \[i/100 for i in range(-750, 1100)\] \>\>\> yarr = \[f_hat(x) for x in xarr\]

</div>

The points in `xarr` and `yarr` can be used to make a PDF plot:

![Scatter plot of the estimated probability density function.](kde_example.png)

Because the returned `f_hat` function is typically called many times, it caches the *data* for performance. To support dynamic datasets, this cache automatically refreshes whenever the length of the *data* changes. This allows new samples to be added as they become available.

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

kde_random(data, h, kernel='normal', \*, seed=None)

Return a function that makes a random selection from the estimated probability density function produced by `kde(data, h, kernel)`.

Providing a *seed* allows reproducible selections. In the future, the values may change slightly as more accurate kernel inverse CDF estimates are implemented. The seed may be an integer, float, str, or bytes.

A `StatisticsError` will be raised if the *data* sequence is empty.

Continuing the example for `kde`, we can use `kde_random` to generate new random selections from an estimated probability density function:

> \>\>\> data = \[-2.1, -1.3, -0.4, 1.9, 5.1, 6.2\] \>\>\> rand = kde_random(data, h=1.5, seed=8675309) \>\>\> new_selections = \[rand() for i in range(10)\] \>\>\> \[round(x, 1) for x in new_selections\] \[0.7, 6.2, 1.2, 6.9, 7.0, 1.8, 2.5, -0.5, -1.8, 5.6\]

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

median(data)

Return the median (middle value) of numeric data, using the common "mean of middle two" method. If *data* is empty, `StatisticsError` is raised. *data* can be a sequence or iterable.

The median is a robust measure of central location and is less affected by the presence of outliers. When the number of data points is odd, the middle data point is returned:

<div class="doctest">

\>\>\> median(\[1, 3, 5\]) 3

</div>

When the number of data points is even, the median is interpolated by taking the average of the two middle values:

<div class="doctest">

\>\>\> median(\[1, 3, 5, 7\]) 4.0

</div>

This is suited for when your data is discrete, and you don't mind that the median may not be an actual data point.

If the data is ordinal (supports order operations) but not numeric (doesn't support addition), consider using `median_low` or `median_high` instead.

</div>

<div class="function">

median_low(data)

Return the low median of numeric data. If *data* is empty, `StatisticsError` is raised. *data* can be a sequence or iterable.

The low median is always a member of the data set. When the number of data points is odd, the middle value is returned. When it is even, the smaller of the two middle values is returned.

<div class="doctest">

\>\>\> median_low(\[1, 3, 5\]) 3 \>\>\> median_low(\[1, 3, 5, 7\]) 3

</div>

Use the low median when your data are discrete and you prefer the median to be an actual data point rather than interpolated.

</div>

<div class="function">

median_high(data)

Return the high median of data. If *data* is empty, `StatisticsError` is raised. *data* can be a sequence or iterable.

The high median is always a member of the data set. When the number of data points is odd, the middle value is returned. When it is even, the larger of the two middle values is returned.

<div class="doctest">

\>\>\> median_high(\[1, 3, 5\]) 3 \>\>\> median_high(\[1, 3, 5, 7\]) 5

</div>

Use the high median when your data are discrete and you prefer the median to be an actual data point rather than interpolated.

</div>

<div class="function">

median_grouped(data, interval=1.0)

Estimates the median for numeric data that has been [grouped or binned](https://en.wikipedia.org/wiki/Data_binning) around the midpoints of consecutive, fixed-width intervals.

The *data* can be any iterable of numeric data with each value being exactly the midpoint of a bin. At least one value must be present.

The *interval* is the width of each bin.

For example, demographic information may have been summarized into consecutive ten-year age groups with each group being represented by the 5-year midpoints of the intervals:

<div class="doctest">

\>\>\> from collections import Counter \>\>\> demographics = Counter({ ... 25: 172, \# 20 to 30 years old ... 35: 484, \# 30 to 40 years old ... 45: 387, \# 40 to 50 years old ... 55: 22, \# 50 to 60 years old ... 65: 6, \# 60 to 70 years old ... }) ...

</div>

The 50th percentile (median) is the 536th person out of the 1071 member cohort. That person is in the 30 to 40 year old age group.

The regular `median` function would assume that everyone in the tricenarian age group was exactly 35 years old. A more tenable assumption is that the 484 members of that age group are evenly distributed between 30 and 40. For that, we use `median_grouped`:

<div class="doctest">

\>\>\> data = list(demographics.elements()) \>\>\> median(data) 35 \>\>\> round(median_grouped(data, interval=10), 1) 37.5

</div>

The caller is responsible for making sure the data points are separated by exact multiples of *interval*. This is essential for getting a correct result. The function does not check this precondition.

Inputs may be any numeric type that can be coerced to a float during the interpolation step.

</div>

<div class="function">

mode(data)

Return the single most common data point from discrete or nominal *data*. The mode (when it exists) is the most typical value and serves as a measure of central location.

If there are multiple modes with the same frequency, returns the first one encountered in the *data*. If the smallest or largest of those is desired instead, use `min(multimode(data))` or `max(multimode(data))`. If the input *data* is empty, `StatisticsError` is raised.

`mode` assumes discrete data and returns a single value. This is the standard treatment of the mode as commonly taught in schools:

<div class="doctest">

\>\>\> mode(\[1, 1, 2, 3, 3, 3, 3, 4\]) 3

</div>

The mode is unique in that it is the only statistic in this package that also applies to nominal (non-numeric) data:

<div class="doctest">

\>\>\> mode(\["red", "blue", "blue", "red", "green", "red", "red"\]) 'red'

</div>

Only hashable inputs are supported. To handle type `set`, consider casting to `frozenset`. To handle type `list`, consider casting to `tuple`. For mixed or nested inputs, consider using this slower quadratic algorithm that only depends on equality tests: `max(data, key=data.count)`.

<div class="versionchanged">

3.8 Now handles multimodal datasets by returning the first mode encountered. Formerly, it raised `StatisticsError` when more than one mode was found.

</div>

</div>

<div class="function">

multimode(data)

Return a list of the most frequently occurring values in the order they were first encountered in the *data*. Will return more than one result if there are multiple modes or an empty list if the *data* is empty:

<div class="doctest">

\>\>\> multimode('aabbbbccddddeeffffgg') \['b', 'd', 'f'\] \>\>\> multimode('') \[\]

</div>

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

pstdev(data, mu=None)

Return the population standard deviation (the square root of the population variance). See `pvariance` for arguments and other details.

<div class="doctest">

\>\>\> pstdev(\[1.5, 2.5, 2.5, 2.75, 3.25, 4.75\]) 0.986893273527251

</div>

</div>

<div class="function">

pvariance(data, mu=None)

Return the population variance of *data*, a non-empty sequence or iterable of real-valued numbers. Variance, or second moment about the mean, is a measure of the variability (spread or dispersion) of data. A large variance indicates that the data is spread out; a small variance indicates it is clustered closely around the mean.

If the optional second argument *mu* is given, it should be the *population* mean of the *data*. It can also be used to compute the second moment around a point that is not the mean. If it is missing or `None` (the default), the arithmetic mean is automatically calculated.

Use this function to calculate the variance from the entire population. To estimate the variance from a sample, the `variance` function is usually a better choice.

Raises `StatisticsError` if *data* is empty.

Examples:

<div class="doctest">

\>\>\> data = \[0.0, 0.25, 0.25, 1.25, 1.5, 1.75, 2.75, 3.25\] \>\>\> pvariance(data) 1.25

</div>

If you have already calculated the mean of your data, you can pass it as the optional second argument *mu* to avoid recalculation:

<div class="doctest">

\>\>\> mu = mean(data) \>\>\> pvariance(data, mu) 1.25

</div>

Decimals and Fractions are supported:

<div class="doctest">

\>\>\> from decimal import Decimal as D \>\>\> pvariance(\[D("27.5"), D("30.25"), D("30.25"), D("34.5"), D("41.75")\]) Decimal('24.815')

\>\>\> from fractions import Fraction as F \>\>\> pvariance(\[F(1, 4), F(5, 4), F(1, 2)\]) Fraction(13, 72)

</div>

> [!NOTE]
> When called with the entire population, this gives the population variance σ². When called on a sample instead, this is the biased sample variance s², also known as variance with N degrees of freedom.
>
> If you somehow know the true population mean μ, you may use this function to calculate the variance of a sample, giving the known population mean as the second argument. Provided the data points are a random sample of the population, the result will be an unbiased estimate of the population variance.

</div>

<div class="function">

stdev(data, xbar=None)

Return the sample standard deviation (the square root of the sample variance). See `variance` for arguments and other details.

<div class="doctest">

\>\>\> stdev(\[1.5, 2.5, 2.5, 2.75, 3.25, 4.75\]) 1.0810874155219827

</div>

</div>

<div class="function">

variance(data, xbar=None)

Return the sample variance of *data*, an iterable of at least two real-valued numbers. Variance, or second moment about the mean, is a measure of the variability (spread or dispersion) of data. A large variance indicates that the data is spread out; a small variance indicates it is clustered closely around the mean.

If the optional second argument *xbar* is given, it should be the *sample* mean of *data*. If it is missing or `None` (the default), the mean is automatically calculated.

Use this function when your data is a sample from a population. To calculate the variance from the entire population, see `pvariance`.

Raises `StatisticsError` if *data* has fewer than two values.

Examples:

<div class="doctest">

\>\>\> data = \[2.75, 1.75, 1.25, 0.25, 0.5, 1.25, 3.5\] \>\>\> variance(data) 1.3720238095238095

</div>

If you have already calculated the sample mean of your data, you can pass it as the optional second argument *xbar* to avoid recalculation:

<div class="doctest">

\>\>\> m = mean(data) \>\>\> variance(data, m) 1.3720238095238095

</div>

This function does not attempt to verify that you have passed the actual mean as *xbar*. Using arbitrary values for *xbar* can lead to invalid or impossible results.

Decimal and Fraction values are supported:

<div class="doctest">

\>\>\> from decimal import Decimal as D \>\>\> variance(\[D("27.5"), D("30.25"), D("30.25"), D("34.5"), D("41.75")\]) Decimal('31.01875')

\>\>\> from fractions import Fraction as F \>\>\> variance(\[F(1, 6), F(1, 2), F(5, 3)\]) Fraction(67, 108)

</div>

> [!NOTE]
> This is the sample variance s² with Bessel's correction, also known as variance with N-1 degrees of freedom. Provided that the data points are representative (e.g. independent and identically distributed), the result should be an unbiased estimate of the true population variance.
>
> If you somehow know the actual population mean μ you should pass it to the `pvariance` function as the *mu* parameter to get the variance of a sample.

</div>

<div class="function">

quantiles(data, \*, n=4, method='exclusive')

Divide *data* into *n* continuous intervals with equal probability. Returns a list of `n - 1` cut points separating the intervals.

Set *n* to 4 for quartiles (the default). Set *n* to 10 for deciles. Set *n* to 100 for percentiles which gives the 99 cuts points that separate *data* into 100 equal sized groups. Raises `StatisticsError` if *n* is not least 1.

The *data* can be any iterable containing sample data. For meaningful results, the number of data points in *data* should be larger than *n*. Raises `StatisticsError` if there is not at least one data point.

The cut points are linearly interpolated from the two nearest data points. For example, if a cut point falls one-third of the distance between two sample values, `100` and `112`, the cut-point will evaluate to `104`.

The *method* for computing quantiles can be varied depending on whether the *data* includes or excludes the lowest and highest possible values from the population.

The default *method* is "exclusive" and is used for data sampled from a population that can have more extreme values than found in the samples. The portion of the population falling below the *i-th* of *m* sorted data points is computed as `i / (m + 1)`. Given nine sample values, the method sorts them and assigns the following percentiles: 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%.

Setting the *method* to "inclusive" is used for describing population data or for samples that are known to include the most extreme values from the population. The minimum value in *data* is treated as the 0th percentile and the maximum value is treated as the 100th percentile. The portion of the population falling below the *i-th* of *m* sorted data points is computed as `(i - 1) / (m - 1)`. Given 11 sample values, the method sorts them and assigns the following percentiles: 0%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100%.

<div class="doctest">

\# Decile cut points for empirically sampled data \>\>\> data = \[105, 129, 87, 86, 111, 111, 89, 81, 108, 92, 110, ... 100, 75, 105, 103, 109, 76, 119, 99, 91, 103, 129, ... 106, 101, 84, 111, 74, 87, 86, 103, 103, 106, 86, ... 111, 75, 87, 102, 121, 111, 88, 89, 101, 106, 95, ... 103, 107, 101, 81, 109, 104\] \>\>\> \[round(q, 1) for q in quantiles(data, n=10)\] \[81.0, 86.2, 89.0, 99.4, 102.5, 103.6, 106.0, 109.8, 111.0\]

</div>

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.13 No longer raises an exception for an input with only a single data point. This allows quantile estimates to be built up one sample point at a time becoming gradually more refined with each new data point.

</div>

</div>

<div class="function">

covariance(x, y, /)

Return the sample covariance of two sequence inputs *x* and *y*. Covariance is a measure of the joint variability of two inputs.

Both inputs must be of the same length (no less than two), otherwise `StatisticsError` is raised.

Examples:

<div class="doctest">

\>\>\> x = \[1, 2, 3, 4, 5, 6, 7, 8, 9\] \>\>\> y = \[1, 2, 3, 1, 2, 3, 1, 2, 3\] \>\>\> covariance(x, y) 0.75 \>\>\> z = \[9, 8, 7, 6, 5, 4, 3, 2, 1\] \>\>\> covariance(x, z) -7.5 \>\>\> covariance(z, x) -7.5

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

correlation(x, y, /, \*, method='linear')

Return the [Pearson's correlation coefficient](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient) for two sequence inputs. Pearson's correlation coefficient *r* takes values between -1 and +1. It measures the strength and direction of a linear relationship.

If *method* is "ranked", computes [Spearman's rank correlation coefficient](https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient) for two inputs. The data is replaced by ranks. Ties are averaged so that equal values receive the same rank. The resulting coefficient measures the strength of a monotonic relationship.

Spearman's correlation coefficient is appropriate for ordinal data or for continuous data that doesn't meet the linear proportion requirement for Pearson's correlation coefficient.

Both inputs must be of the same length (no less than two), and need not to be constant, otherwise `StatisticsError` is raised.

Example with [Kepler's laws of planetary motion](https://en.wikipedia.org/wiki/Kepler's_laws_of_planetary_motion):

<div class="doctest">

\>\>\> \# Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, and Neptune \>\>\> orbital_period = \[88, 225, 365, 687, 4331, 10_756, 30_687, 60_190\] \# days \>\>\> dist_from_sun = \[58, 108, 150, 228, 778, 1_400, 2_900, 4_500\] \# million km

\>\>\> \# Show that a perfect monotonic relationship exists \>\>\> correlation(orbital_period, dist_from_sun, method='ranked') 1.0

\>\>\> \# Observe that a linear relationship is imperfect \>\>\> round(correlation(orbital_period, dist_from_sun), 4) 0.9882

\>\>\> \# Demonstrate Kepler's third law: There is a linear correlation \>\>\> \# between the square of the orbital period and the cube of the \>\>\> \# distance from the sun. \>\>\> period_squared = \[p \* p for p in orbital_period\] \>\>\> dist_cubed = \[d \* d \* d for d in dist_from_sun\] \>\>\> round(correlation(period_squared, dist_cubed), 4) 1.0

</div>

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.12 Added support for Spearman's rank correlation coefficient.

</div>

</div>

<div class="function">

linear_regression(x, y, /, \*, proportional=False)

Return the slope and intercept of [simple linear regression](https://en.wikipedia.org/wiki/Simple_linear_regression) parameters estimated using ordinary least squares. Simple linear regression describes the relationship between an independent variable *x* and a dependent variable *y* in terms of this linear function:

> *y = slope \* x + intercept + noise*

where `slope` and `intercept` are the regression parameters that are estimated, and `noise` represents the variability of the data that was not explained by the linear regression (it is equal to the difference between predicted and actual values of the dependent variable).

Both inputs must be sequences of the same length (no less than two), and the independent variable *x* cannot be constant; otherwise a `StatisticsError` is raised.

For example, we can use the [release dates of the Monty Python films](https://en.wikipedia.org/wiki/Monty_Python#Films) to predict the cumulative number of Monty Python films that would have been produced by 2019 assuming that they had kept the pace.

<div class="doctest">

\>\>\> year = \[1971, 1975, 1979, 1982, 1983\] \>\>\> films_total = \[1, 2, 3, 4, 5\] \>\>\> slope, intercept = linear_regression(year, films_total) \>\>\> round(slope \* 2019 + intercept) 16

</div>

If *proportional* is true, the independent variable *x* and the dependent variable *y* are assumed to be directly proportional. The data is fit to a line passing through the origin. Since the *intercept* will always be 0.0, the underlying linear function simplifies to:

> *y = slope \* x + noise*

Continuing the example from `correlation`, we look to see how well a model based on major planets can predict the orbital distances for dwarf planets:

<div class="doctest">

\>\>\> model = linear_regression(period_squared, dist_cubed, proportional=True) \>\>\> slope = model.slope

\>\>\> \# Dwarf planets: Pluto, Eris, Makemake, Haumea, Ceres \>\>\> orbital_periods = \[90_560, 204_199, 111_845, 103_410, 1_680\] \# days \>\>\> predicted_dist = \[math.cbrt(slope \* (p \* p)) for p in orbital_periods\] \>\>\> list(map(round, predicted_dist)) \[5912, 10166, 6806, 6459, 414\]

\>\>\> \[5_906, 10_152, 6_796, 6_450, 414\] \# actual distance in million km \[5906, 10152, 6796, 6450, 414\]

</div>

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.11 Added support for *proportional*.

</div>

</div>

## Exceptions

A single exception is defined:

<div class="exception">

StatisticsError

Subclass of `ValueError` for statistics-related exceptions.

</div>

## `NormalDist` objects

`NormalDist` is a tool for creating and manipulating normal distributions of a [random variable](http://www.stat.yale.edu/Courses/1997-98/101/ranvar.htm). It is a class that treats the mean and standard deviation of data measurements as a single entity.

Normal distributions arise from the [Central Limit Theorem](https://en.wikipedia.org/wiki/Central_limit_theorem) and have a wide range of applications in statistics.

<div class="NormalDist(mu=0.0, sigma=1.0)">

Returns a new *NormalDist* object where *mu* represents the [arithmetic mean](https://en.wikipedia.org/wiki/Arithmetic_mean) and *sigma* represents the [standard deviation](https://en.wikipedia.org/wiki/Standard_deviation).

If *sigma* is negative, raises `StatisticsError`.

<div class="attribute">

mean

A read-only property for the [arithmetic mean](https://en.wikipedia.org/wiki/Arithmetic_mean) of a normal distribution.

</div>

<div class="attribute">

median

A read-only property for the [median](https://en.wikipedia.org/wiki/Median) of a normal distribution.

</div>

<div class="attribute">

mode

A read-only property for the [mode](https://en.wikipedia.org/wiki/Mode_(statistics)) of a normal distribution.

</div>

<div class="attribute">

stdev

A read-only property for the [standard deviation](https://en.wikipedia.org/wiki/Standard_deviation) of a normal distribution.

</div>

<div class="attribute">

variance

A read-only property for the [variance](https://en.wikipedia.org/wiki/Variance) of a normal distribution. Equal to the square of the standard deviation.

</div>

<div class="classmethod">

NormalDist.from_samples(data)

Makes a normal distribution instance with *mu* and *sigma* parameters estimated from the *data* using `fmean` and `stdev`.

The *data* can be any `iterable` and should consist of values that can be converted to type `float`. If *data* does not contain at least two elements, raises `StatisticsError` because it takes at least one point to estimate a central value and at least two points to estimate dispersion.

</div>

<div class="method">

NormalDist.samples(n, \*, seed=None)

Generates *n* random samples for a given mean and standard deviation. Returns a `list` of `float` values.

If *seed* is given, creates a new instance of the underlying random number generator. This is useful for creating reproducible results, even in a multi-threading context.

<div class="versionchanged">

3.13

</div>

Switched to a faster algorithm. To reproduce samples from previous versions, use `random.seed` and `random.gauss`.

</div>

<div class="method">

NormalDist.pdf(x)

Using a [probability density function (pdf)](https://en.wikipedia.org/wiki/Probability_density_function), compute the relative likelihood that a random variable *X* will be near the given value *x*. Mathematically, it is the limit of the ratio `P(x <= X < x+dx) / dx` as *dx* approaches zero.

The relative likelihood is computed as the probability of a sample occurring in a narrow range divided by the width of the range (hence the word "density"). Since the likelihood is relative to other points, its value can be greater than `1.0`.

</div>

<div class="method">

NormalDist.cdf(x)

Using a [cumulative distribution function (cdf)](https://en.wikipedia.org/wiki/Cumulative_distribution_function), compute the probability that a random variable *X* will be less than or equal to *x*. Mathematically, it is written `P(X <= x)`.

</div>

<div class="method">

NormalDist.inv_cdf(p)

Compute the inverse cumulative distribution function, also known as the [quantile function](https://en.wikipedia.org/wiki/Quantile_function) or the [percent-point](https://web.archive.org/web/20190203145224/https://www.statisticshowto.datasciencecentral.com/inverse-distribution-function/) function. Mathematically, it is written `x : P(X <= x) = p`.

Finds the value *x* of the random variable *X* such that the probability of the variable being less than or equal to that value equals the given probability *p*.

</div>

<div class="method">

NormalDist.overlap(other)

Measures the agreement between two normal probability distributions. Returns a value between 0.0 and 1.0 giving [the overlapping area for the two probability density functions](https://www.rasch.org/rmt/rmt101r.htm).

</div>

<div class="method">

NormalDist.quantiles(n=4)

Divide the normal distribution into *n* continuous intervals with equal probability. Returns a list of (n - 1) cut points separating the intervals.

Set *n* to 4 for quartiles (the default). Set *n* to 10 for deciles. Set *n* to 100 for percentiles which gives the 99 cuts points that separate the normal distribution into 100 equal sized groups.

</div>

<div class="method">

NormalDist.zscore(x)

Compute the [Standard Score](https://www.statisticshowto.com/probability-and-statistics/z-score/) describing *x* in terms of the number of standard deviations above or below the mean of the normal distribution: `(x - mean) / stdev`.

<div class="versionadded">

3.9

</div>

</div>

Instances of `NormalDist` support addition, subtraction, multiplication and division by a constant. These operations are used for translation and scaling. For example:

<div class="doctest">

\>\>\> temperature_february = NormalDist(5, 2.5) \# Celsius \>\>\> temperature_february \* (9/5) + 32 \# Fahrenheit NormalDist(mu=41.0, sigma=4.5)

</div>

Dividing a constant by an instance of `NormalDist` is not supported because the result wouldn't be normally distributed.

Since normal distributions arise from additive effects of independent variables, it is possible to [add and subtract two independent normally distributed random variables](https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables) represented as instances of `NormalDist`. For example:

<div class="doctest">

\>\>\> birth_weights = NormalDist.from_samples(\[2.5, 3.1, 2.1, 2.4, 2.7, 3.5\]) \>\>\> drug_effects = NormalDist(0.4, 0.15) \>\>\> combined = birth_weights + drug_effects \>\>\> round(combined.mean, 1) 3.1 \>\>\> round(combined.stdev, 1) 0.5

</div>

<div class="versionadded">

3.8

</div>

</div>

## Examples and Recipes

### Classic probability problems

`NormalDist` readily solves classic probability problems.

For example, given [historical data for SAT exams](https://nces.ed.gov/programs/digest/d17/tables/dt17_226.40.asp) showing that scores are normally distributed with a mean of 1060 and a standard deviation of 195, determine the percentage of students with test scores between 1100 and 1200, after rounding to the nearest whole number:

<div class="doctest">

\>\>\> sat = NormalDist(1060, 195) \>\>\> fraction = sat.cdf(1200 + 0.5) - sat.cdf(1100 - 0.5) \>\>\> round(fraction \* 100.0, 1) 18.4

</div>

Find the [quartiles](https://en.wikipedia.org/wiki/Quartile) and [deciles](https://en.wikipedia.org/wiki/Decile) for the SAT scores:

<div class="doctest">

\>\>\> list(map(round, sat.quantiles())) \[928, 1060, 1192\] \>\>\> list(map(round, sat.quantiles(n=10))) \[810, 896, 958, 1011, 1060, 1109, 1162, 1224, 1310\]

</div>

### Monte Carlo inputs for simulations

To estimate the distribution for a model that isn't easy to solve analytically, `NormalDist` can generate input samples for a [Monte Carlo simulation](https://en.wikipedia.org/wiki/Monte_Carlo_method):

<div class="doctest">

\>\>\> def model(x, y, z): ... return (3\*x + 7\*x\*y - 5\*y) / (11 \* z) ... \>\>\> n = 100_000 \>\>\> X = NormalDist(10, 2.5).samples(n, seed=3652260728) \>\>\> Y = NormalDist(15, 1.75).samples(n, seed=4582495471) \>\>\> Z = NormalDist(50, 1.25).samples(n, seed=6582483453) \>\>\> quantiles(map(model, X, Y, Z)) \# doctest: +SKIP \[1.4591308524824727, 1.8035946855390597, 2.175091447274739\]

</div>

### Approximating binomial distributions

Normal distributions can be used to approximate [Binomial distributions](https://mathworld.wolfram.com/BinomialDistribution.html) when the sample size is large and when the probability of a successful trial is near 50%.

For example, an open source conference has 750 attendees and two rooms with a 500 person capacity. There is a talk about Python and another about Ruby. In previous conferences, 65% of the attendees preferred to listen to Python talks. Assuming the population preferences haven't changed, what is the probability that the Python room will stay within its capacity limits?

<div class="doctest">

\>\>\> n = 750 \# Sample size \>\>\> p = 0.65 \# Preference for Python \>\>\> q = 1.0 - p \# Preference for Ruby \>\>\> k = 500 \# Room capacity

\>\>\> \# Approximation using the cumulative normal distribution \>\>\> from math import sqrt \>\>\> round(NormalDist(mu=n\*p, sigma=sqrt(n\*p\*q)).cdf(k + 0.5), 4) 0.8402

\>\>\> \# Exact solution using the cumulative binomial distribution \>\>\> from math import comb, fsum \>\>\> round(fsum(comb(n, r) \* p\*\*r \* q\*\*(n-r) for r in range(k+1)), 4) 0.8402

\>\>\> \# Approximation using a simulation \>\>\> from random import seed, binomialvariate \>\>\> seed(8675309) \>\>\> mean(binomialvariate(n, p) \<= k for i in range(10_000)) 0.8406

</div>

### Naive bayesian classifier

Normal distributions commonly arise in machine learning problems.

Wikipedia has a [nice example of a Naive Bayesian Classifier](https://en.wikipedia.org/wiki/Naive_Bayes_classifier#Person_classification). The challenge is to predict a person's gender from measurements of normally distributed features including height, weight, and foot size.

We're given a training dataset with measurements for eight people. The measurements are assumed to be normally distributed, so we summarize the data with `NormalDist`:

<div class="doctest">

\>\>\> height_male = NormalDist.from_samples(\[6, 5.92, 5.58, 5.92\]) \>\>\> height_female = NormalDist.from_samples(\[5, 5.5, 5.42, 5.75\]) \>\>\> weight_male = NormalDist.from_samples(\[180, 190, 170, 165\]) \>\>\> weight_female = NormalDist.from_samples(\[100, 150, 130, 150\]) \>\>\> foot_size_male = NormalDist.from_samples(\[12, 11, 12, 10\]) \>\>\> foot_size_female = NormalDist.from_samples(\[6, 8, 7, 9\])

</div>

Next, we encounter a new person whose feature measurements are known but whose gender is unknown:

<div class="doctest">

\>\>\> ht = 6.0 \# height \>\>\> wt = 130 \# weight \>\>\> fs = 8 \# foot size

</div>

Starting with a 50% [prior probability](https://en.wikipedia.org/wiki/Prior_probability) of being male or female, we compute the posterior as the prior times the product of likelihoods for the feature measurements given the gender:

<div class="doctest">

\>\>\> prior_male = 0.5 \>\>\> prior_female = 0.5 \>\>\> posterior_male = (prior_male \* height_male.pdf(ht) \* ... weight_male.pdf(wt) \* foot_size_male.pdf(fs))

\>\>\> posterior_female = (prior_female \* height_female.pdf(ht) \* ... weight_female.pdf(wt) \* foot_size_female.pdf(fs))

</div>

The final prediction goes to the largest posterior. This is known as the [maximum a posteriori](https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation) or MAP:

<div class="doctest">

\>\>\> 'male' if posterior_male \> posterior_female else 'female' 'female'

</div>

# `!string` --- Common string operations

<div class="module" data-synopsis="Common string operations.">

string

</div>

**Source code:** `Lib/string/__init__.py`

------------------------------------------------------------------------

<div class="seealso">

`textseq`

`string-methods`

</div>

## String constants

The constants defined in this module are:

<div class="data">

ascii_letters

The concatenation of the `ascii_lowercase` and `ascii_uppercase` constants described below. This value is not locale-dependent.

</div>

<div class="data">

ascii_lowercase

The lowercase letters `'abcdefghijklmnopqrstuvwxyz'`. This value is not locale-dependent and will not change.

</div>

<div class="data">

ascii_uppercase

The uppercase letters `'ABCDEFGHIJKLMNOPQRSTUVWXYZ'`. This value is not locale-dependent and will not change.

</div>

<div class="data">

digits

The string `'0123456789'`.

</div>

<div class="data">

hexdigits

The string `'0123456789abcdefABCDEF'`.

</div>

<div class="data">

octdigits

The string `'01234567'`.

</div>

<div class="data">

punctuation

String of ASCII characters which are considered punctuation characters in the `C` locale: `` !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~ ``.

</div>

<div class="data">

printable

String of ASCII characters which are considered printable by Python. This is a combination of `digits`, `ascii_letters`, `punctuation`, and `whitespace`.

> [!NOTE]
> By design, `string.printable.isprintable() <str.isprintable>` returns `False`. In particular, `string.printable` is not printable in the POSIX sense (see `LC_CTYPE <locale(5)>`).

</div>

<div class="data">

whitespace

A string containing all ASCII characters that are considered whitespace. This includes the characters space, tab, linefeed, return, formfeed, and vertical tab.

</div>

## Custom string formatting

The built-in string class provides the ability to do complex variable substitutions and value formatting via the `~str.format` method described in `3101`. The `Formatter` class in the `!string` module allows you to create and customize your own string formatting behaviors using the same implementation as the built-in `~str.format` method.

<div class="Formatter">

The `Formatter` class has the following public methods:

<div class="method">

format(format_string, /, *args,*\*kwargs)

The primary API method. It takes a format string and an arbitrary set of positional and keyword arguments. It is just a wrapper that calls `vformat`.

<div class="versionchanged">

3.7 A format string argument is now `positional-only
<positional-only_parameter>`.

</div>

</div>

<div class="method">

vformat(format_string, args, kwargs)

This function does the actual work of formatting. It is exposed as a separate function for cases where you want to pass in a predefined dictionary of arguments, rather than unpacking and repacking the dictionary as individual arguments using the `*args` and `**kwargs` syntax. `vformat` does the work of breaking up the format string into character data and replacement fields. It calls the various methods described below.

</div>

In addition, the `Formatter` defines a number of methods that are intended to be replaced by subclasses:

<div class="method">

parse(format_string)

Loop over the format_string and return an iterable of tuples (*literal_text*, *field_name*, *format_spec*, *conversion*). This is used by `vformat` to break the string into either literal text, or replacement fields.

The values in the tuple conceptually represent a span of literal text followed by a single replacement field. If there is no literal text (which can happen if two replacement fields occur consecutively), then *literal_text* will be a zero-length string. If there is no replacement field, then the values of *field_name*, *format_spec* and *conversion* will be `None`. The value of *field_name* is unmodified and auto-numbering of non-numbered positional fields is done by `vformat`.

</div>

<div class="method">

get_field(field_name, args, kwargs)

Given *field_name*, convert it to an object to be formatted. Auto-numbering of *field_name* returned from `parse` is done by `vformat` before calling this method. Returns a tuple (obj, used_key). The default version takes strings of the form defined in `3101`, such as "0\[name\]" or "label.title". *args* and *kwargs* are as passed in to `vformat`. The return value *used_key* has the same meaning as the *key* parameter to `get_value`.

</div>

<div class="method">

get_value(key, args, kwargs)

Retrieve a given field value. The *key* argument will be either an integer or a string. If it is an integer, it represents the index of the positional argument in *args*; if it is a string, then it represents a named argument in *kwargs*.

The *args* parameter is set to the list of positional arguments to `vformat`, and the *kwargs* parameter is set to the dictionary of keyword arguments.

For compound field names, these functions are only called for the first component of the field name; subsequent components are handled through normal attribute and indexing operations.

So for example, the field expression '0.name' would cause `get_value` to be called with a *key* argument of 0. The `name` attribute will be looked up after `get_value` returns by calling the built-in `getattr` function.

If the index or keyword refers to an item that does not exist, then an `IndexError` or `KeyError` should be raised.

</div>

<div class="method">

check_unused_args(used_args, args, kwargs)

Implement checking for unused arguments if desired. The arguments to this function is the set of all argument keys that were actually referred to in the format string (integers for positional arguments, and strings for named arguments), and a reference to the *args* and *kwargs* that was passed to vformat. The set of unused args can be calculated from these parameters. `check_unused_args` is assumed to raise an exception if the check fails.

</div>

<div class="method">

format_field(value, format_spec)

`format_field` simply calls the global `format` built-in. The method is provided so that subclasses can override it.

</div>

<div class="method">

convert_field(value, conversion)

Converts the value (returned by `get_field`) given a conversion type (as in the tuple returned by the `parse` method). The default version understands 's' (str), 'r' (repr) and 'a' (ascii) conversion types.

</div>

</div>

## Format string syntax

The `str.format` method and the `Formatter` class share the same syntax for format strings (although in the case of `Formatter`, subclasses can define their own format string syntax). The syntax is related to that of `formatted string literals <f-strings>` and `template string literals <t-strings>`, but it is less sophisticated and, in particular, does not support arbitrary expressions in interpolations.

<div class="index">

single: {} (curly brackets); in string formatting single: . (dot); in string formatting single: \[\] (square brackets); in string formatting single: ! (exclamation); in string formatting single: : (colon); in string formatting

</div>

Format strings contain "replacement fields" surrounded by curly braces `{}`. Anything that is not contained in braces is considered literal text, which is copied unchanged to the output. If you need to include a brace character in the literal text, it can be escaped by doubling: `{{` and `}}`.

The grammar for a replacement field is as follows:

<div class="productionlist">

format-string replacement_field: "{" \[<span class="title-ref">field_name</span>\] \["!" <span class="title-ref">conversion</span>\] \[":" <span class="title-ref">format_spec</span>\] "}" field_name: <span class="title-ref">arg_name</span> ("." <span class="title-ref">attribute_name</span> \| "\[" <span class="title-ref">element_index</span> "\]")\* arg_name: \[<span class="title-ref">~python-grammar:identifier</span> \| <span class="title-ref">~python-grammar:digit</span>+\] attribute_name: <span class="title-ref">~python-grammar:identifier</span> element_index: <span class="title-ref">~python-grammar:digit</span>+ \| <span class="title-ref">index_string</span> index_string: \<any source character except "\]"\> + conversion: "r" \| "s" \| "a" format_spec: <span class="title-ref">format-spec:format_spec</span>

</div>

In less formal terms, the replacement field can start with a *field_name* that specifies the object whose value is to be formatted and inserted into the output instead of the replacement field. The *field_name* is optionally followed by a *conversion* field, which is preceded by an exclamation point `'!'`, and a *format_spec*, which is preceded by a colon `':'`. These specify a non-default format for the replacement value.

See also the `formatspec` section.

The *field_name* itself begins with an *arg_name* that is either a number or a keyword. If it's a number, it refers to a positional argument, and if it's a keyword, it refers to a named keyword argument. An *arg_name* is treated as a number if a call to `str.isdecimal` on the string would return true. If the numerical arg_names in a format string are 0, 1, 2, ... in sequence, they can all be omitted (not just some) and the numbers 0, 1, 2, ... will be automatically inserted in that order. Because *arg_name* is not quote-delimited, it is not possible to specify arbitrary dictionary keys (e.g., the strings `'10'` or `':-]'`) within a format string. The *arg_name* can be followed by any number of index or attribute expressions. An expression of the form `'.name'` selects the named attribute using `getattr`, while an expression of the form `'[index]'` does an index lookup using `~object.__getitem__`.

<div class="versionchanged">

3.1 The positional argument specifiers can be omitted for `str.format`, so `'{} {}'.format(a, b)` is equivalent to `'{0} {1}'.format(a, b)`.

</div>

<div class="versionchanged">

3.4 The positional argument specifiers can be omitted for `Formatter`.

</div>

Some simple format string examples:

    "First, thou shalt count to {0}"  # References first positional argument
    "Bring me a {}"                   # Implicitly references the first positional argument
    "From {} to {}"                   # Same as "From {0} to {1}"
    "My quest is {name}"              # References keyword argument 'name'
    "Weight in tons {0.weight}"       # 'weight' attribute of first positional arg
    "Units destroyed: {players[0]}"   # First element of keyword argument 'players'.

<div id="formatstrings-conversion">

The *conversion* field causes a type coercion before formatting. Normally, the job of formatting a value is done by the `~object.__format__` method of the value itself. However, in some cases it is desirable to force a type to be formatted as a string, overriding its own definition of formatting. By converting the value to a string before calling `~object.__format__`, the normal formatting logic is bypassed.

</div>

Three conversion flags are currently supported: `'!s'` which calls `str` on the value, `'!r'` which calls `repr` and `'!a'` which calls `ascii`.

Some examples:

    "Harold's a clever {0!s}"        # Calls str() on the argument first
    "Bring out the holy {name!r}"    # Calls repr() on the argument first
    "More {!a}"                      # Calls ascii() on the argument first

The *format_spec* field contains a specification of how the value should be presented, including such details as field width, alignment, padding, decimal precision and so on. Each value type can define its own "formatting mini-language" or interpretation of the *format_spec*.

Most built-in types support a common formatting mini-language, which is described in the next section.

A *format_spec* field can also include nested replacement fields within it. These nested replacement fields may contain a field name, conversion flag and format specification, but deeper nesting is not allowed. The replacement fields within the format_spec are substituted before the *format_spec* string is interpreted. This allows the formatting of a value to be dynamically specified.

See the `formatexamples` section for some examples.

### Format specification mini-language

"Format specifications" are used within replacement fields contained within a format string to define how individual values are presented (see `formatstrings`, `f-strings`, and `t-strings`). They can also be passed directly to the built-in `format` function. Each formattable type may define how the format specification is to be interpreted.

Most built-in types implement the following options for format specifications, although some of the formatting options are only supported by the numeric types.

A general convention is that an empty format specification produces the same result as if you had called `str` on the value. A non-empty format specification typically modifies the result.

The general form of a *standard format specifier* is:

<div class="productionlist">

format-spec format_spec: \[<span class="title-ref">options</span>\]\[<span class="title-ref">width_and_precision</span>\]\[<span class="title-ref">type</span>\] options: \[\[<span class="title-ref">fill</span>\]\`align\`\]\[<span class="title-ref">sign</span>\]\["z"\]\["#"\]\["0"\] fill: \<any character\> align: "\<" \| "\>" \| "=" \| "^" sign: "+" \| "-" \| " " width_and_precision: \[<span class="title-ref">width_with_grouping</span>\]\[<span class="title-ref">precision_with_grouping</span>\] width_with_grouping: \[<span class="title-ref">width</span>\]\[<span class="title-ref">grouping</span>\] precision_with_grouping: "." \[<span class="title-ref">precision</span>\]\[<span class="title-ref">grouping</span>\] \| "." <span class="title-ref">grouping</span> width: <span class="title-ref">~python-grammar:digit</span>+ precision: <span class="title-ref">~python-grammar:digit</span>+ grouping: "," \| "\_" type: "b" \| "c" \| "d" \| "e" \| "E" \| "f" \| "F" \| "g" : \| "G" \| "n" \| "o" \| "s" \| "x" \| "X" \| "%"

</div>

If a valid *align* value is specified, it can be preceded by a *fill* character that can be any character and defaults to a space if omitted. It is not possible to use a literal curly brace ("`{`" or "`}`") as the *fill* character in a `formatted string literal
<f-strings>` or when using the `str.format` method. However, it is possible to insert a curly brace with a nested replacement field. This limitation doesn't affect the `format` function.

The meaning of the various alignment options is as follows:

<div class="index">

single: \< (less); in string formatting single: \> (greater); in string formatting single: = (equals); in string formatting single: ^ (caret); in string formatting

</div>

| Option | Meaning |
|----|----|
| `'<'` | Forces the field to be left-aligned within the available space (this is the default for most objects). |
| `'>'` | Forces the field to be right-aligned within the available space (this is the default for numbers). |
| `'='` | Forces the padding to be placed after the sign (if any) but before the digits. This is used for printing fields in the form '+000000120'. This alignment option is only valid for numeric types, excluding `complex`. It becomes the default for numbers when '0' immediately precedes the field width. |
| `'^'` | Forces the field to be centered within the available space. |

Note that unless a minimum field width is defined, the field width will always be the same size as the data to fill it, so that the alignment option has no meaning in this case.

The *sign* option is only valid for number types, and can be one of the following:

<div class="index">

single: + (plus); in string formatting single: - (minus); in string formatting single: space; in string formatting

</div>

| Option | Meaning |
|----|----|
| `'+'` | Indicates that a sign should be used for both positive as well as negative numbers. |
| `'-'` | Indicates that a sign should be used only for negative numbers (this is the default behavior). |
| space | Indicates that a leading space should be used on positive numbers, and a minus sign on negative numbers. |

<div class="index">

single: z; in string formatting

</div>

The `'z'` option coerces negative zero floating-point values to positive zero after rounding to the format precision. This option is only valid for floating-point presentation types.

<div class="versionchanged">

3.11 Added the `'z'` option (see also `682`).

</div>

<div class="index">

single: \# (hash); in string formatting

</div>

The `'#'` option causes the "alternate form" to be used for the conversion. The alternate form is defined differently for different types. This option is only valid for integer, float and complex types. For integers, when binary, octal, or hexadecimal output is used, this option adds the respective prefix `'0b'`, `'0o'`, `'0x'`, or `'0X'` to the output value. For float and complex the alternate form causes the result of the conversion to always contain a decimal-point character, even if no digits follow it. Normally, a decimal-point character appears in the result of these conversions only if a digit follows it. In addition, for `'g'` and `'G'` conversions, trailing zeros are not removed from the result.

The *width* is a decimal integer defining the minimum total field width, including any prefixes, separators, and other formatting characters. If not specified, then the field width will be determined by the content.

When no explicit alignment is given, preceding the *width* field by a zero (`'0'`) character enables sign-aware zero-padding for numeric types, excluding `complex`. This is equivalent to a *fill* character of `'0'` with an *alignment* type of `'='`.

<div class="versionchanged">

3.10 Preceding the *width* field by `'0'` no longer affects the default alignment for strings.

</div>

The *precision* is a decimal integer indicating how many digits should be displayed after the decimal point for presentation types `'f'` and `'F'`, or before and after the decimal point for presentation types `'g'` or `'G'`. For string presentation types the field indicates the maximum field size - in other words, how many characters will be used from the field content. The *precision* is not allowed for integer presentation types.

The *grouping* option after *width* and *precision* fields specifies a digit group separator for the integral and fractional parts of a number respectively. It can be one of the following:

<div class="index">

single: , (comma); in string formatting single: \_ (underscore); in string formatting

</div>

| Option | Meaning |
|----|----|
| `','` | Inserts a comma every 3 digits for integer presentation type `'d'` and floating-point presentation types, excluding `'n'`. For other presentation types, this option is not supported. |
| `'_'` | Inserts an underscore every 3 digits for integer presentation type `'d'` and floating-point presentation types, excluding `'n'`. For integer presentation types `'b'`, `'o'`, `'x'`, and `'X'`, underscores are inserted every 4 digits. For other presentation types, this option is not supported. |

For a locale-aware separator, use the `'n'` `float presentation type <n-format-float>` or `integer presentation type <n-format-integer>` instead.

<div class="versionchanged">

3.1 Added the `','` option (see also `378`).

</div>

<div class="versionchanged">

3.6 Added the `'_'` option (see also `515`).

</div>

<div class="versionchanged">

3.14 Support the *grouping* option for the fractional part.

</div>

Finally, the *type* determines how the data should be presented.

The available string presentation types are:

> | Type | Meaning |
> |----|----|
> | `'s'` | String format. This is the default type for strings and may be omitted. |
> | None | The same as `'s'`. |

The available integer presentation types are:

> <table style="width:95%;">
> <colgroup>
> <col style="width: 13%" />
> <col style="width: 81%" />
> </colgroup>
> <thead>
> <tr>
> <th><p>Type</p></th>
> <th><p>Meaning</p></th>
> </tr>
> </thead>
> <tbody>
> <tr>
> <td><p><code>'b'</code></p></td>
> <td><p>Binary format. Outputs the number in base 2.</p></td>
> </tr>
> <tr>
> <td><p><code>'c'</code></p></td>
> <td><p>Character. Converts the integer to the corresponding unicode character before printing.</p></td>
> </tr>
> <tr>
> <td><p><code>'d'</code></p></td>
> <td><p>Decimal Integer. Outputs the number in base 10.</p></td>
> </tr>
> <tr>
> <td><p><code>'o'</code></p></td>
> <td><p>Octal format. Outputs the number in base 8.</p></td>
> </tr>
> <tr>
> <td><p><code>'x'</code></p></td>
> <td><p>Hex format. Outputs the number in base 16, using lower-case letters for the digits above 9.</p></td>
> </tr>
> <tr>
> <td><p><code>'X'</code></p></td>
> <td><p>Hex format. Outputs the number in base 16, using upper-case letters for the digits above 9. In case <code>'#'</code> is specified, the prefix <code>'0x'</code> will be upper-cased to <code>'0X'</code> as well.</p></td>
> </tr>
> <tr>
> <td><p><code>'n'</code></p></td>
> <td><div id="n-format-integer">
> <p>Number. This is the same as <code>'d'</code>, except that it uses the current locale setting to insert the appropriate digit group separators. Note that the default locale is not the system locale. Depending on your use case, you may wish to set <code class="interpreted-text" role="const">~locale.LC_NUMERIC</code> with <code class="interpreted-text" role="func">locale.setlocale</code> before using <code>'n'</code>.</p>
> </div></td>
> </tr>
> <tr>
> <td><p>None</p></td>
> <td><p>The same as <code>'d'</code>.</p></td>
> </tr>
> </tbody>
> </table>

In addition to the above presentation types, integers can be formatted with the floating-point presentation types listed below (except `'n'` and `None`). When doing so, `float` is used to convert the integer to a floating-point number before formatting.

The available presentation types for `float` and `~decimal.Decimal` values are:

> <table style="width:95%;">
> <colgroup>
> <col style="width: 13%" />
> <col style="width: 81%" />
> </colgroup>
> <thead>
> <tr>
> <th><p>Type</p></th>
> <th><p>Meaning</p></th>
> </tr>
> </thead>
> <tbody>
> <tr>
> <td><p><code>'e'</code></p></td>
> <td><p>Scientific notation. For a given precision <code>p</code>, formats the number in scientific notation with the letter 'e' separating the coefficient from the exponent. The coefficient has one digit before and <code>p</code> digits after the decimal point, for a total of <code>p + 1</code> significant digits. With no precision given, uses a precision of <code>6</code> digits after the decimal point for <code class="interpreted-text" role="class">float</code>, and shows all coefficient digits for <code class="interpreted-text" role="class">~decimal.Decimal</code>. If <code>p=0</code>, the decimal point is omitted unless the <code>#</code> option is used.</p>
> <p>For <code class="interpreted-text" role="class">float</code>, the exponent always contains at least two digits, and is zero if the value is zero.</p></td>
> </tr>
> <tr>
> <td><p><code>'E'</code></p></td>
> <td><p>Scientific notation. Same as <code>'e'</code> except it uses an upper case 'E' as the separator character.</p></td>
> </tr>
> <tr>
> <td><p><code>'f'</code></p></td>
> <td><p>Fixed-point notation. For a given precision <code>p</code>, formats the number as a decimal number with exactly <code>p</code> digits following the decimal point. With no precision given, uses a precision of <code>6</code> digits after the decimal point for <code class="interpreted-text" role="class">float</code>, and uses a precision large enough to show all coefficient digits for <code class="interpreted-text" role="class">~decimal.Decimal</code>. If <code>p=0</code>, the decimal point is omitted unless the <code>#</code> option is used.</p></td>
> </tr>
> <tr>
> <td><p><code>'F'</code></p></td>
> <td><p>Fixed-point notation. Same as <code>'f'</code>, but converts <code>nan</code> to <code>NAN</code> and <code>inf</code> to <code>INF</code>.</p></td>
> </tr>
> <tr>
> <td><p><code>'g'</code></p></td>
> <td><p>General format. For a given precision <code>p &gt;= 1</code>, this rounds the number to <code>p</code> significant digits and then formats the result in either fixed-point format or in scientific notation, depending on its magnitude. A precision of <code>0</code> is treated as equivalent to a precision of <code>1</code>.</p>
> <p>The precise rules are as follows: suppose that the result formatted with presentation type <code>'e'</code> and precision <code>p-1</code> would have exponent <code>exp</code>. Then, if <code>m &lt;= exp &lt; p</code>, where <code>m</code> is -4 for floats and -6 for <code class="interpreted-text" role="class">Decimals &lt;decimal.Decimal&gt;</code>, the number is formatted with presentation type <code>'f'</code> and precision <code>p-1-exp</code>. Otherwise, the number is formatted with presentation type <code>'e'</code> and precision <code>p-1</code>. In both cases insignificant trailing zeros are removed from the significand, and the decimal point is also removed if there are no remaining digits following it, unless the <code>'#'</code> option is used.</p>
> <p>With no precision given, uses a precision of <code>6</code> significant digits for <code class="interpreted-text" role="class">float</code>. For <code class="interpreted-text" role="class">~decimal.Decimal</code>, the coefficient of the result is formed from the coefficient digits of the value; scientific notation is used for values smaller than <code>1e-6</code> in absolute value and values where the place value of the least significant digit is larger than 1, and fixed-point notation is used otherwise.</p>
> <p>Positive and negative infinity, positive and negative zero, and nans, are formatted as <code>inf</code>, <code>-inf</code>, <code>0</code>, <code>-0</code> and <code>nan</code> respectively, regardless of the precision.</p></td>
> </tr>
> <tr>
> <td><p><code>'G'</code></p></td>
> <td><p>General format. Same as <code>'g'</code> except switches to <code>'E'</code> if the number gets too large. The representations of infinity and NaN are uppercased, too.</p></td>
> </tr>
> <tr>
> <td><p><code>'n'</code></p></td>
> <td><div id="n-format-float">
> <p>Number. This is the same as <code>'g'</code>, except that it uses the current locale setting to insert the appropriate digit group separators for the integral part of a number. Note that the default locale is not the system locale. Depending on your use case, you may wish to set <code class="interpreted-text" role="const">~locale.LC_NUMERIC</code> with <code class="interpreted-text" role="func">locale.setlocale</code> before using <code>'n'</code>.</p>
> </div></td>
> </tr>
> <tr>
> <td><p><code>'%'</code></p></td>
> <td><p>Percentage. Multiplies the number by 100 and displays in fixed (<code>'f'</code>) format, followed by a percent sign.</p></td>
> </tr>
> <tr>
> <td><p>None</p></td>
> <td><p>For <code class="interpreted-text" role="class">float</code> this is like the <code>'g'</code> type, except that when fixed-point notation is used to format the result, it always includes at least one digit past the decimal point, and switches to the scientific notation when <code>exp &gt;= p - 1</code>. When the precision is not specified, the latter will be as large as needed to represent the given value faithfully.</p>
> <p>For <code class="interpreted-text" role="class">~decimal.Decimal</code>, this is the same as either <code>'g'</code> or <code>'G'</code> depending on the value of <code>context.capitals</code> for the current decimal context.</p>
> <p>The overall effect is to match the output of <code class="interpreted-text" role="func">str</code> as altered by the other format modifiers.</p></td>
> </tr>
> </tbody>
> </table>

The result should be correctly rounded to a given precision `p` of digits after the decimal point. The rounding mode for `float` matches that of the `round` builtin. For `~decimal.Decimal`, the rounding mode of the current `context <decimal-context>` will be used.

The available presentation types for `complex` are the same as those for `float` (`'%'` is not allowed). Both the real and imaginary components of a complex number are formatted as floating-point numbers, according to the specified presentation type. They are separated by the mandatory sign of the imaginary part, the latter being terminated by a `j` suffix. If the presentation type is missing, the result will match the output of `str` (complex numbers with a non-zero real part are also surrounded by parentheses), possibly altered by other format modifiers.

### Format examples

This section contains examples of the `str.format` syntax and comparison with the old `%`-formatting.

In most of the cases the syntax is similar to the old `%`-formatting, with the addition of the `{}` and with `:` used instead of `%`. For example, `'%03.2f'` can be translated to `'{:03.2f}'`.

The new format syntax also supports new and different options, shown in the following examples.

Accessing arguments by position:

    >>> '{0}, {1}, {2}'.format('a', 'b', 'c')
    'a, b, c'
    >>> '{}, {}, {}'.format('a', 'b', 'c')  # 3.1+ only
    'a, b, c'
    >>> '{2}, {1}, {0}'.format('a', 'b', 'c')
    'c, b, a'
    >>> '{2}, {1}, {0}'.format(*'abc')      # unpacking argument sequence
    'c, b, a'
    >>> '{0}{1}{0}'.format('abra', 'cad')   # arguments' indices can be repeated
    'abracadabra'

Accessing arguments by name:

    >>> 'Coordinates: {latitude}, {longitude}'.format(latitude='37.24N', longitude='-115.81W')
    'Coordinates: 37.24N, -115.81W'
    >>> coord = {'latitude': '37.24N', 'longitude': '-115.81W'}
    >>> 'Coordinates: {latitude}, {longitude}'.format(**coord)
    'Coordinates: 37.24N, -115.81W'

Accessing arguments' attributes:

    >>> c = 3-5j
    >>> ('The complex number {0} is formed from the real part {0.real} '
    ...  'and the imaginary part {0.imag}.').format(c)
    'The complex number (3-5j) is formed from the real part 3.0 and the imaginary part -5.0.'
    >>> class Point:
    ...     def __init__(self, x, y):
    ...         self.x, self.y = x, y
    ...     def __str__(self):
    ...         return 'Point({self.x}, {self.y})'.format(self=self)
    ...
    >>> str(Point(4, 2))
    'Point(4, 2)'

Accessing arguments' items:

    >>> coord = (3, 5)
    >>> 'X: {0[0]};  Y: {0[1]}'.format(coord)
    'X: 3;  Y: 5'

Replacing `%s` and `%r`:

    >>> "repr() shows quotes: {!r}; str() doesn't: {!s}".format('test1', 'test2')
    "repr() shows quotes: 'test1'; str() doesn't: test2"

Aligning the text and specifying a width:

    >>> '{:<30}'.format('left aligned')
    'left aligned                  '
    >>> '{:>30}'.format('right aligned')
    '                 right aligned'
    >>> '{:^30}'.format('centered')
    '           centered           '
    >>> '{:*^30}'.format('centered')  # use '*' as a fill char
    '***********centered***********'

Replacing `%+f`, `%-f`, and `% f` and specifying a sign:

    >>> '{:+f}; {:+f}'.format(3.14, -3.14)  # show it always
    '+3.140000; -3.140000'
    >>> '{: f}; {: f}'.format(3.14, -3.14)  # show a space for positive numbers
    ' 3.140000; -3.140000'
    >>> '{:-f}; {:-f}'.format(3.14, -3.14)  # show only the minus -- same as '{:f}; {:f}'
    '3.140000; -3.140000'

Replacing `%x` and `%o` and converting the value to different bases:

    >>> # format also supports binary numbers
    >>> "int: {0:d};  hex: {0:x};  oct: {0:o};  bin: {0:b}".format(42)
    'int: 42;  hex: 2a;  oct: 52;  bin: 101010'
    >>> # with 0x, 0o, or 0b as prefix:
    >>> "int: {0:d};  hex: {0:#x};  oct: {0:#o};  bin: {0:#b}".format(42)
    'int: 42;  hex: 0x2a;  oct: 0o52;  bin: 0b101010'

Using the comma or the underscore as a digit group separator:

    >>> '{:,}'.format(1234567890)
    '1,234,567,890'
    >>> '{:_}'.format(1234567890)
    '1_234_567_890'
    >>> '{:_b}'.format(1234567890)
    '100_1001_1001_0110_0000_0010_1101_0010'
    >>> '{:_x}'.format(1234567890)
    '4996_02d2'
    >>> '{:_}'.format(123456789.123456789)
    '123_456_789.12345679'
    >>> '{:.,}'.format(123456789.123456789)
    '123456789.123,456,79'
    >>> '{:,._}'.format(123456789.123456789)
    '123,456,789.123_456_79'

Expressing a percentage:

    >>> points = 19
    >>> total = 22
    >>> 'Correct answers: {:.2%}'.format(points/total)
    'Correct answers: 86.36%'

Using type-specific formatting:

    >>> import datetime as dt
    >>> d = dt.datetime(2010, 7, 4, 12, 15, 58)
    >>> '{:%Y-%m-%d %H:%M:%S}'.format(d)
    '2010-07-04 12:15:58'

Nesting arguments and more complex examples:

    >>> for align, text in zip('<^>', ['left', 'center', 'right']):
    ...     '{0:{fill}{align}16}'.format(text, fill=align, align=align)
    ...
    'left<<<<<<<<<<<<'
    '^^^^^center^^^^^'
    '>>>>>>>>>>>right'
    >>>
    >>> octets = [192, 168, 0, 1]
    >>> '{:02X}{:02X}{:02X}{:02X}'.format(*octets)
    'C0A80001'
    >>> int(_, 16)
    3232235521
    >>>
    >>> width = 5
    >>> for num in range(5,12): #doctest: +NORMALIZE_WHITESPACE
    ...     for base in 'dXob':
    ...         print('{0:{width}{base}}'.format(num, base=base, width=width), end=' ')
    ...     print()
    ...
        5     5     5   101
        6     6     6   110
        7     7     7   111
        8     8    10  1000
        9     9    11  1001
       10     A    12  1010
       11     B    13  1011

## Template strings (\$-strings)

> [!NOTE]
> The feature described here was introduced in Python 2.4; a simple templating method based upon regular expressions. It predates `str.format`, `formatted string literals <f-strings>`, and `template string literals <template-strings>`.
>
> It is unrelated to template string literals (t-strings), which were introduced in Python 3.14. These evaluate to `string.templatelib.Template` objects, found in the `string.templatelib` module.

Template strings provide simpler string substitutions as described in `292`. A primary use case for template strings is for internationalization (i18n) since in that context, the simpler syntax and functionality makes it easier to translate than other built-in string formatting facilities in Python. As an example of a library built on template strings for i18n, see the [flufl.i18n](https://flufli18n.readthedocs.io/en/latest/) package.

<div class="index">

single: \$ (dollar); in template strings

</div>

Template strings support `$`-based substitutions, using the following rules:

- `$$` is an escape; it is replaced with a single `$`.
- `$identifier` names a substitution placeholder matching a mapping key of `"identifier"`. By default, `"identifier"` is restricted to any case-insensitive ASCII alphanumeric string (including underscores) that starts with an underscore or ASCII letter. The first non-identifier character after the `$` character terminates this placeholder specification.
- `${identifier}` is equivalent to `$identifier`. It is required when valid identifier characters follow the placeholder but are not part of the placeholder, such as `"${noun}ification"`.

Any other appearance of `$` in the string will result in a `ValueError` being raised.

The `!string` module provides a `Template` class that implements these rules. The methods of `Template` are:

<div class="Template(template)">

The constructor takes a single argument which is the template string.

<div class="method">

substitute(mapping={}, /, \*\*kwds)

Performs the template substitution, returning a new string. *mapping* is any dictionary-like object with keys that match the placeholders in the template. Alternatively, you can provide keyword arguments, where the keywords are the placeholders. When both *mapping* and *kwds* are given and there are duplicates, the placeholders from *kwds* take precedence.

</div>

<div class="method">

safe_substitute(mapping={}, /, \*\*kwds)

Like `substitute`, except that if placeholders are missing from *mapping* and *kwds*, instead of raising a `KeyError` exception, the original placeholder will appear in the resulting string intact. Also, unlike with `substitute`, any other appearances of the `$` will simply return `$` instead of raising `ValueError`.

While other exceptions may still occur, this method is called "safe" because it always tries to return a usable string instead of raising an exception. In another sense, `safe_substitute` may be anything other than safe, since it will silently ignore malformed templates containing dangling delimiters, unmatched braces, or placeholders that are not valid Python identifiers.

</div>

<div class="method">

is_valid()

Returns `False` if the template has invalid placeholders that will cause `substitute` to raise `ValueError`.

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

get_identifiers()

Returns a list of the valid identifiers in the template, in the order they first appear, ignoring any invalid identifiers.

<div class="versionadded">

3.11

</div>

</div>

`Template` instances also provide one public data attribute:

<div class="attribute">

template

This is the object passed to the constructor's *template* argument. In general, you shouldn't change it, but read-only access is not enforced.

</div>

</div>

Here is an example of how to use a Template:

    >>> from string import Template
    >>> s = Template('$who likes $what')
    >>> s.substitute(who='tim', what='kung pao')
    'tim likes kung pao'
    >>> d = dict(who='tim')
    >>> Template('Give $who $100').substitute(d)
    Traceback (most recent call last):
    ...
    ValueError: Invalid placeholder in string: line 1, col 11
    >>> Template('$who likes $what').substitute(d)
    Traceback (most recent call last):
    ...
    KeyError: 'what'
    >>> Template('$who likes $what').safe_substitute(d)
    'tim likes $what'

Advanced usage: you can derive subclasses of `Template` to customize the placeholder syntax, delimiter character, or the entire regular expression used to parse template strings. To do this, you can override these class attributes:

- *delimiter* -- This is the literal string describing a placeholder introducing delimiter. The default value is `$`. Note that this should *not* be a regular expression, as the implementation will call `re.escape` on this string as needed. Note further that you cannot change the delimiter after class creation (i.e. a different delimiter must be set in the subclass's class namespace).

- *idpattern* -- This is the regular expression describing the pattern for non-braced placeholders. The default value is the regular expression `(?a:[_a-z][_a-z0-9]*)`. If this is given and *braceidpattern* is `None` this pattern will also apply to braced placeholders.

  > [!NOTE]
  > Since default *flags* is `re.IGNORECASE`, pattern `[a-z]` can match with some non-ASCII characters. That's why we use the local `a` flag here.

  <div class="versionchanged">

  3.7 *braceidpattern* can be used to define separate patterns used inside and outside the braces.

  </div>

- *braceidpattern* -- This is like *idpattern* but describes the pattern for braced placeholders. Defaults to `None` which means to fall back to *idpattern* (i.e. the same pattern is used both inside and outside braces). If given, this allows you to define different patterns for braced and unbraced placeholders.

  <div class="versionadded">

  3.7

  </div>

- *flags* -- The regular expression flags that will be applied when compiling the regular expression used for recognizing substitutions. The default value is `re.IGNORECASE`. Note that `re.VERBOSE` will always be added to the flags, so custom *idpattern*s must follow conventions for verbose regular expressions.

  <div class="versionadded">

  3.2

  </div>

Alternatively, you can provide the entire regular expression pattern by overriding the class attribute *pattern*. If you do this, the value must be a regular expression pattern string, or a compiled regular expression object, with four named capturing groups. The capturing groups correspond to the rules given above, along with the invalid placeholder rule:

- *escaped* -- This group matches the escape sequence, e.g. `$$`, in the default pattern.
- *named* -- This group matches the unbraced placeholder name; it should not include the delimiter in capturing group.
- *braced* -- This group matches the brace enclosed placeholder name; it should not include either the delimiter or braces in the capturing group.
- *invalid* -- This group matches any other delimiter pattern (usually a single delimiter), and it should appear last in the regular expression.

The methods on this class will raise `ValueError` if the pattern matches the template without one of these named groups matching.

## Helper functions

<div class="function">

capwords(s, sep=None)

Split the argument into words using `str.split`, capitalize each word using `str.capitalize`, and join the capitalized words using `str.join`. If the optional second argument *sep* is absent or `None`, runs of whitespace characters are replaced by a single space and leading and trailing whitespace are removed, otherwise *sep* is used to split and join the words.

</div>

# `!string.templatelib` --- Support for template string literals

<div class="module" data-synopsis="Support for template string literals.">

string.templatelib

</div>

**Source code:** `Lib/string/templatelib.py`

------------------------------------------------------------------------

<div class="seealso">

- `Format strings <f-strings>`
- `Template string literal (t-string) syntax <t-strings>`
- `750`

</div>

## Template strings

<div class="versionadded">

3.14

</div>

Template strings are a mechanism for custom string processing. They have the full flexibility of Python's `f-strings`, but return a `Template` instance that gives access to the static and interpolated (in curly brackets) parts of a string *before* they are combined.

To write a t-string, use a `'t'` prefix instead of an `'f'`, like so:

```
>>> pi = 3.14
>>> t't-strings are new in Python {pi!s}!'
Template(
   strings=('t-strings are new in Python ', '!'),
   interpolations=(Interpolation(3.14, 'pi', 's', ''),)
)
```

## Types

<div class="Template">

The `!Template` class describes the contents of a template string. It is immutable, meaning that attributes of a template cannot be reassigned.

The most common way to create a `!Template` instance is to use the `template string literal syntax <t-strings>`. This syntax is identical to that of `f-strings <f-strings>`, except that it uses a `t` prefix in place of an `f`:

\>\>\> cheese = 'Red Leicester' \>\>\> template = t"We're fresh out of {cheese}, sir." \>\>\> type(template) \<class 'string.templatelib.Template'\>

Templates are stored as sequences of literal `~Template.strings` and dynamic `~Template.interpolations`. A `~Template.values` attribute holds the values of the interpolations:

\>\>\> cheese = 'Camembert' \>\>\> template = t'Ah! We do have {cheese}.' \>\>\> template.strings ('Ah! We do have ', '.') \>\>\> template.interpolations (Interpolation('Camembert', ...),) \>\>\> template.values ('Camembert',)

The `!strings` tuple has one more element than `!interpolations` and `!values`; the interpolations “belong” between the strings. This may be easier to understand when tuples are aligned

``` python
template.strings:  ('Ah! We do have ',              '.')
template.values:   (                   'Camembert',    )
```

**Attributes**

<div class="attribute" type="tuple[str, ...]">

strings

A `tuple` of the static strings in the template.

\>\>\> cheese = 'Camembert' \>\>\> template = t'Ah! We do have {cheese}.' \>\>\> template.strings ('Ah! We do have ', '.')

Empty strings *are* included in the tuple:

\>\>\> response = 'We do have ' \>\>\> cheese = 'Camembert' \>\>\> template = t'Ah! {response}{cheese}.' \>\>\> template.strings ('Ah! ', '', '.')

The `strings` tuple is never empty, and always contains one more string than the `interpolations` and `values` tuples:

\>\>\> t''.strings ('',) \>\>\> t''.values () \>\>\> t'{'cheese'}'.strings ('', '') \>\>\> t'{'cheese'}'.values ('cheese',)

</div>

<div class="attribute" type="tuple[Interpolation, ...]">

interpolations

A `tuple` of the interpolations in the template.

\>\>\> cheese = 'Camembert' \>\>\> template = t'Ah! We do have {cheese}.' \>\>\> template.interpolations (Interpolation('Camembert', 'cheese', None, ''),)

The `interpolations` tuple may be empty and always contains one fewer values than the `strings` tuple:

\>\>\> t'Red Leicester'.interpolations ()

</div>

<div class="attribute" type="tuple[object, ...]">

values

A tuple of all interpolated values in the template.

\>\>\> cheese = 'Camembert' \>\>\> template = t'Ah! We do have {cheese}.' \>\>\> template.values ('Camembert',)

The `values` tuple always has the same length as the `interpolations` tuple. It is always equivalent to `tuple(i.value for i in template.interpolations)`.

</div>

**Methods**

<div class="method">

\_\_new\_\_(\*args: str \| Interpolation)

While literal syntax is the most common way to create a `!Template`, it is also possible to create them directly using the constructor:

\>\>\> from string.templatelib import Interpolation, Template \>\>\> cheese = 'Camembert' \>\>\> template = Template( ... 'Ah! We do have ', Interpolation(cheese, 'cheese'), '.' ... ) \>\>\> list(template) \['Ah! We do have ', Interpolation('Camembert', 'cheese', None, ''), '.'\]

If multiple strings are passed consecutively, they will be concatenated into a single value in the `~Template.strings` attribute. For example, the following code creates a `Template` with a single final string:

\>\>\> from string.templatelib import Template \>\>\> template = Template('Ah! We do have ', 'Camembert', '.') \>\>\> template.strings ('Ah! We do have Camembert.',)

If multiple interpolations are passed consecutively, they will be treated as separate interpolations and an empty string will be inserted between them. For example, the following code creates a template with empty placeholders in the `~Template.strings` attribute:

\>\>\> from string.templatelib import Interpolation, Template \>\>\> template = Template( ... Interpolation('Camembert', 'cheese'), ... Interpolation('.', 'punctuation'), ... ) \>\>\> template.strings ('', '', '')

</div>

<div class="describe">

iter(template)

Iterate over the template, yielding each non-empty string and `Interpolation` in the correct order:

\>\>\> cheese = 'Camembert' \>\>\> list(t'Ah! We do have {cheese}.') \['Ah! We do have ', Interpolation('Camembert', 'cheese', None, ''), '.'\]

> [!CAUTION]
> Empty strings are **not** included in the iteration:
>
> \>\>\> response = 'We do have ' \>\>\> cheese = 'Camembert' \>\>\> list(t'Ah! {response}{cheese}.') \# doctest: +NORMALIZE_WHITESPACE \['Ah! ', Interpolation('We do have ', 'response', None, ''), Interpolation('Camembert', 'cheese', None, ''), '.'\]

</div>

<div class="describe">

template + other template += other

Concatenate this template with another, returning a new `!Template` instance:

\>\>\> cheese = 'Camembert' \>\>\> list(t'Ah! ' + t'We do have {cheese}.') \['Ah! We do have ', Interpolation('Camembert', 'cheese', None, ''), '.'\]

Concatenating a `!Template` and a `str` is **not** supported. This is because it is unclear whether the string should be treated as a static string or an interpolation. If you want to concatenate a `!Template` with a string, you should either wrap the string directly in a `!Template` (to treat it as a static string) or use an `!Interpolation` (to treat it as dynamic):

\>\>\> from string.templatelib import Interpolation, Template \>\>\> template = t'Ah! ' \>\>\> \# Treat 'We do have ' as a static string \>\>\> template += Template('We do have ') \>\>\> \# Treat cheese as an interpolation \>\>\> cheese = 'Camembert' \>\>\> template += Template(Interpolation(cheese, 'cheese')) \>\>\> list(template) \['Ah! We do have ', Interpolation('Camembert', 'cheese', None, '')\]

</div>

</div>

<div class="Interpolation">

The `!Interpolation` type represents an expression inside a template string. It is immutable, meaning that attributes of an interpolation cannot be reassigned.

Interpolations support pattern matching, allowing you to match against their attributes with the `match statement <match>`:

\>\>\> from string.templatelib import Interpolation \>\>\> interpolation = t'{1. + 2.:.2f}'.interpolations\[0\] \>\>\> interpolation Interpolation(3.0, '1. + 2.', None, '.2f') \>\>\> match interpolation: ... case Interpolation(value, expression, conversion, format_spec): ... print(value, expression, conversion, format_spec, sep=' \| ') ... 3.0 \| 1. + 2. \| None \| .2f

Interpolations are `generic <generics>` over the types of their values.

**Attributes**

<div class="attribute" type="object">

value

The evaluated value of the interpolation.

\>\>\> t'{1 + 2}'.interpolations\[0\].value 3

</div>

<div class="attribute" type="str">

expression

For interpolations created by t-string literals, `!expression` is the expression text found inside the curly brackets (`{` & `}`), including any whitespace, excluding the curly brackets themselves, and ending before the first `!`, `:`, or `=` if any is present. For manually created interpolations, `!expression` is the arbitrary string provided when constructing the interpolation instance.

We recommend using valid Python expressions or the empty string for the `expression` field of manually created `!Interpolation` instances, although this is not enforced at runtime.

\>\>\> t'{1 + 2}'.interpolations\[0\].expression '1 + 2'

</div>

<div class="attribute" type="typing.Literal['a', 'r', 's'] | None">

conversion

The conversion to apply to the value, or `None`.

The `!conversion` is the optional conversion to apply to the value:

\>\>\> t'{1 + 2!a}'.interpolations\[0\].conversion 'a'

> [!NOTE]
> Unlike f-strings, where conversions are applied automatically, the expected behavior with t-strings is that code that *processes* the `!Template` will decide how to interpret and whether to apply the `!conversion`. For convenience, the `convert` function can be used to mimic f-string conversion semantics.

</div>

<div class="attribute" type="str">

format_spec

The format specification to apply to the value.

The `!format_spec` is an optional, arbitrary string used as the format specification to present the value:

\>\>\> t'{1 + 2:.2f}'.interpolations\[0\].format_spec '.2f'

> [!NOTE]
> Unlike f-strings, where format specifications are applied automatically via the `format` protocol, the expected behavior with t-strings is that code that *processes* the interpolation will decide how to interpret and whether to apply the format specification. As a result, `!format_spec` values in interpolations can be arbitrary strings, including those that do not conform to the `format` protocol.

</div>

**Methods**

<div class="method">

\_\_new\_\_(value: object, expression: str, conversion: typing.Literal\['a', 'r', 's'\] \| None = None, format_spec: str = '')

Create a new `!Interpolation` object from component parts.

param value  
The evaluated, in-scope result of the interpolation.

param expression  
The text of a valid Python expression, or an empty string.

param conversion  
The `conversion <formatstrings>` to be used, one of `None`, `'a'`, `'r'`, or `'s'`.

param format_spec  
An optional, arbitrary string used as the `format specification <formatspec>` to present the value.

</div>

</div>

## Helper functions

<div class="function">

convert(obj, /, conversion)

Applies formatted string literal `conversion <formatstrings-conversion>` semantics to the given object *obj*. This is frequently useful for custom template string processing logic.

Three conversion flags are currently supported:

- `'s'` which calls `str` on the value (like `!s`),
- `'r'` which calls `repr` (like `!r`), and
- `'a'` which calls `ascii` (like `!a`).

If the conversion flag is `None`, *obj* is returned unchanged.

</div>

# `!stringprep` --- Internet String Preparation

<div class="module" data-synopsis="String preparation, as per RFC 3453">

stringprep

</div>

**Source code:** `Lib/stringprep.py`

------------------------------------------------------------------------

When identifying things (such as host names) in the internet, it is often necessary to compare such identifications for "equality". Exactly how this comparison is executed may depend on the application domain, e.g. whether it should be case-insensitive or not. It may be also necessary to restrict the possible identifications, to allow only identifications consisting of "printable" characters.

`3454` defines a procedure for "preparing" Unicode strings in internet protocols. Before passing strings onto the wire, they are processed with the preparation procedure, after which they have a certain normalized form. The RFC defines a set of tables, which can be combined into profiles. Each profile must define which tables it uses, and what other optional parts of the `stringprep` procedure are part of the profile. One example of a `stringprep` profile is `nameprep`, which is used for internationalized domain names.

The module `!stringprep` only exposes the tables from `3454`. As these tables would be very large to represent as dictionaries or lists, the module uses the Unicode character database internally. The module source code itself was generated using the `mkstringprep.py` utility.

As a result, these tables are exposed as functions, not as data structures. There are two kinds of tables in the RFC: sets and mappings. For a set, `!stringprep` provides the "characteristic function", i.e. a function that returns `True` if the parameter is part of the set. For mappings, it provides the mapping function: given the key, it returns the associated value. Below is a list of all functions available in the module.

<div class="function">

in_table_a1(code)

Determine whether *code* is in tableA.1 (Unassigned code points in Unicode 3.2).

</div>

<div class="function">

in_table_b1(code)

Determine whether *code* is in tableB.1 (Commonly mapped to nothing).

</div>

<div class="function">

map_table_b2(code)

Return the mapped value for *code* according to tableB.2 (Mapping for case-folding used with NFKC).

</div>

<div class="function">

map_table_b3(code)

Return the mapped value for *code* according to tableB.3 (Mapping for case-folding used with no normalization).

</div>

<div class="function">

in_table_c11(code)

Determine whether *code* is in tableC.1.1 (ASCII space characters).

</div>

<div class="function">

in_table_c12(code)

Determine whether *code* is in tableC.1.2 (Non-ASCII space characters).

</div>

<div class="function">

in_table_c11_c12(code)

Determine whether *code* is in tableC.1 (Space characters, union of C.1.1 and C.1.2).

</div>

<div class="function">

in_table_c21(code)

Determine whether *code* is in tableC.2.1 (ASCII control characters).

</div>

<div class="function">

in_table_c22(code)

Determine whether *code* is in tableC.2.2 (Non-ASCII control characters).

</div>

<div class="function">

in_table_c21_c22(code)

Determine whether *code* is in tableC.2 (Control characters, union of C.2.1 and C.2.2).

</div>

<div class="function">

in_table_c3(code)

Determine whether *code* is in tableC.3 (Private use).

</div>

<div class="function">

in_table_c4(code)

Determine whether *code* is in tableC.4 (Non-character code points).

</div>

<div class="function">

in_table_c5(code)

Determine whether *code* is in tableC.5 (Surrogate codes).

</div>

<div class="function">

in_table_c6(code)

Determine whether *code* is in tableC.6 (Inappropriate for plain text).

</div>

<div class="function">

in_table_c7(code)

Determine whether *code* is in tableC.7 (Inappropriate for canonical representation).

</div>

<div class="function">

in_table_c8(code)

Determine whether *code* is in tableC.8 (Change display properties or are deprecated).

</div>

<div class="function">

in_table_c9(code)

Determine whether *code* is in tableC.9 (Tagging characters).

</div>

<div class="function">

in_table_d1(code)

Determine whether *code* is in tableD.1 (Characters with bidirectional property "R" or "AL").

</div>

<div class="function">

in_table_d2(code)

Determine whether *code* is in tableD.2 (Characters with bidirectional property "L").

</div>

# `!struct` --- Interpret bytes as packed binary data

<div class="testsetup">

- 

from struct import \*

</div>

<div class="module" data-synopsis="Interpret bytes as packed binary data.">

struct

</div>

**Source code:** `Lib/struct.py`

<div class="index">

pair: C; structures triple: packing; binary; data

</div>

------------------------------------------------------------------------

This module converts between Python values and C structs represented as Python `bytes` objects. Compact `format strings <struct-format-strings>` describe the intended conversions to/from Python values. The module's functions and objects can be used for two largely distinct applications, data exchange with external sources (files or network connections), or data transfer between the Python application and the C layer.

> [!NOTE]
> When no prefix character is given, native mode is the default. It packs or unpacks data based on the platform and compiler on which the Python interpreter was built. The result of packing a given C struct includes pad bytes which maintain proper alignment for the C types involved; similarly, alignment is taken into account when unpacking. In contrast, when communicating data between external sources, the programmer is responsible for defining byte ordering and padding between elements. See `struct-alignment` for details.

Several `!struct` functions (and methods of `Struct`) take a *buffer* argument. This refers to objects that implement the `bufferobjects` and provide either a readable or read-writable buffer. The most common types used for that purpose are `bytes` and `bytearray`, but many other types that can be viewed as an array of bytes implement the buffer protocol, so that they can be read/filled without additional copying from a `bytes` object.

## Functions and Exceptions

The module defines the following exception and functions:

<div class="exception">

error

Exception raised on various occasions; argument is a string describing what is wrong.

</div>

<div class="function">

pack(format, v1, v2, ...)

Return a bytes object containing the values *v1*, *v2*, ... packed according to the format string *format*. The arguments must match the values required by the format exactly.

</div>

<div class="function">

pack_into(format, buffer, offset, v1, v2, ...)

Pack the values *v1*, *v2*, ... according to the format string *format* and write the packed bytes into the writable buffer *buffer* starting at position *offset*. Note that *offset* is a required argument. A negative *offset* counts from the end of *buffer*.

</div>

<div class="function">

unpack(format, buffer)

Unpack from the buffer *buffer* (presumably packed by `pack(format, ...)`) according to the format string *format*. The result is a tuple even if it contains exactly one item. The buffer's size in bytes must match the size required by the format, as reflected by `calcsize`.

</div>

<div class="function">

unpack_from(format, /, buffer, offset=0)

Unpack from *buffer* starting at position *offset*, according to the format string *format*. The result is a tuple even if it contains exactly one item. The buffer's size in bytes, starting at position *offset*, must be at least the size required by the format, as reflected by `calcsize`. A negative *offset* counts from the end of *buffer*.

</div>

<div class="function">

iter_unpack(format, buffer)

Iteratively unpack from the buffer *buffer* according to the format string *format*. This function returns an iterator which will read equally sized chunks from the buffer until all its contents have been consumed. The buffer's size in bytes must be a multiple of the size required by the format, as reflected by `calcsize`.

Each iteration yields a tuple as specified by the format string.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

calcsize(format)

Return the size of the struct (and hence of the bytes object produced by `pack(format, ...)`) corresponding to the format string *format*.

</div>

## Format Strings

Format strings describe the data layout when packing and unpacking data. They are built up from `type codes <type-codes>`, which specify the type of data being packed/unpacked. In addition, special characters control the `byte order, size and alignment<struct-alignment>`. Each format string consists of an optional prefix character which describes the overall properties of the data and one or more format characters which describe the actual data values and padding.

### Byte Order, Size, and Alignment

By default, C types are represented in the machine's native format and byte order, and properly aligned by skipping pad bytes if necessary (according to the rules used by the C compiler). This behavior is chosen so that the bytes of a packed struct correspond exactly to the memory layout of the corresponding C struct. Whether to use native byte ordering and padding or standard formats depends on the application.

<div class="index">

single: @ (at); in struct format strings single: = (equals); in struct format strings single: \< (less); in struct format strings single: \> (greater); in struct format strings single: ! (exclamation); in struct format strings

</div>

Alternatively, the first character of the format string can be used to indicate the byte order, size and alignment of the packed data, according to the following table:

| Character | Byte order             | Size     | Alignment |
|-----------|------------------------|----------|-----------|
| `@`       | native                 | native   | native    |
| `=`       | native                 | standard | none      |
| `<`       | little-endian          | standard | none      |
| `>`       | big-endian             | standard | none      |
| `!`       | network (= big-endian) | standard | none      |

If the first character is not one of these, `'@'` is assumed.

> [!NOTE]
> The number 1023 (`0x3ff` in hexadecimal) has the following byte representations:
>
> - `03 ff` in big-endian (`>`)
> - `ff 03` in little-endian (`<`)
>
> Python example:
>
> > \>\>\> import struct \>\>\> struct.pack('\>h', 1023) b'x03xff' \>\>\> struct.pack('\<h', 1023) b'xffx03'

Native byte order is big-endian or little-endian, depending on the host system. For example, Intel x86, AMD64 (x86-64), and Apple M1 are little-endian; IBM z and many legacy architectures are big-endian. Use `sys.byteorder` to check the endianness of your system.

Native size and alignment are determined using the C compiler's `sizeof` expression. This is always combined with native byte order.

Standard size depends only on the type code; see the table in the `type-codes` section.

Note the difference between `'@'` and `'='`: both use native byte order, but the size and alignment of the latter is standardized.

The form `'!'` represents the network byte order which is always big-endian as defined in [IETF RFC 1700](https://datatracker.ietf.org/doc/html/rfc1700).

There is no way to indicate non-native byte order (force byte-swapping); use the appropriate choice of `'<'` or `'>'`.

Notes:

1)  Padding is only automatically added between successive structure members. No padding is added at the beginning or the end of the encoded struct.
2)  No padding is added when using non-native size and alignment, e.g. with '\<', '\>', '=', and '!'.
3)  To align the end of a structure to the alignment requirement of a particular type, end the format with the code for that type with a repeat count of zero. See `struct-examples`.

### Type Codes<span id="type-codes"></span>

Type codes (or format codes) have the following meaning; the conversion between C and Python values should be obvious given their types. The 'Standard size' column refers to the size of the packed value in bytes when using standard size; that is, when the format string starts with one of `'<'`, `'>'`, `'!'` or `'='`. When using native size, the size of the packed value is platform-dependent.

| Format | C Type           | Python type       | Standard size | Notes    |
|--------|------------------|-------------------|---------------|----------|
| `x`    | pad byte         | no value          |               | \(7\)    |
| `c`    | `char`           | bytes of length 1 | 1             |          |
| `b`    | `signed char`    | int               | 1             | \(2\)    |
| `B`    | `unsigned char`  | int               | 1             | \(2\)    |
| `?`    | `_Bool`          | bool              | 1             | \(1\)    |
| `h`    | `short`          | int               | 2             | \(2\)    |
| `H`    | `unsigned short` | int               | 2             | \(2\)    |
| `i`    | `int`            | int               | 4             | \(2\)    |
| `I`    | `unsigned int`   | int               | 4             | \(2\)    |
| `l`    | `long`           | int               | 4             | \(2\)    |
| `L`    | `unsigned long`  | int               | 4             | \(2\)    |
| `q`    | `long long`      | int               | 8             | \(2\)    |
| `Q`    | `unsigned long   
          long`             | int               | 8             | \(2\)    |
| `n`    | `ssize_t`        | int               |               | (2), (3) |
| `N`    | `size_t`         | int               |               | (2), (3) |
| `e`    | `_Float16`       | float             | 2             | (4), (6) |
| `f`    | `float`          | float             | 4             | \(4\)    |
| `d`    | `double`         | float             | 8             | \(4\)    |
| `Zf`   | `float complex`  | complex           | 8             | \(10\)   |
| `Zd`   | `double complex` | complex           | 16            | \(10\)   |
| `s`    | `char[]`         | bytes             |               | \(9\)    |
| `p`    | `char[]`         | bytes             |               | \(8\)    |
| `P`    | `void \*`        | int               |               | (2), (5) |

<div class="versionchanged">

3.3 Added support for the `'n'` and `'N'` formats.

</div>

<div class="versionchanged">

3.6 Added support for the `'e'` format.

</div>

<div class="versionchanged">

3.14 Added support for the `'F'` and `'D'` formats.

</div>

<div class="versionchanged">

3.15 Added support for the `'Zf'` and `'Zd'` formats.

</div>

<div class="versionchanged">

3.16 `'F'` and `'D'` formats are deprecated.

</div>

<div class="seealso">

The `array` and `ctypes <ctypes-fundamental-data-types>` modules, as well as third-party modules like [numpy](https://numpy.org/doc/stable/reference/arrays.interface.html#object.__array_interface__), use similar -- but slightly different -- type codes.

</div>

Notes:

\(1\)  
<div class="index">

single: ? (question mark); in struct format strings

</div>

The `'?'` conversion code corresponds to the `_Bool` type defined by C standards since C99. In standard mode, it is represented by one byte.

\(2\)  
When attempting to pack a non-integer using any of the integer conversion codes, if the non-integer has a `~object.__index__` method then that method is called to convert the argument to an integer before packing.

<div class="versionchanged">

3.2 Added use of the `~object.__index__` method for non-integers.

</div>

\(3\)  
The `'n'` and `'N'` conversion codes are only available for the native size (selected as the default or with the `'@'` byte order character). For the standard size, you can use whichever of the other integer formats fits your application.

\(4\)  
For the `'f'`, `'d'` and `'e'` conversion codes, the packed representation uses the IEEE 754 binary32, binary64 or binary16 format (for `'f'`, `'d'` or `'e'` respectively), regardless of the floating-point format used by the platform.

\(5\)  
The `'P'` type code is only available for the native byte ordering (selected as the default or with the `'@'` byte order character). The byte order character `'='` chooses to use little- or big-endian ordering based on the host system. The struct module does not interpret this as native ordering, so the `'P'` format is not available.

\(6\)  
The IEEE 754 binary16 "half precision" type was introduced in the 2008 revision of the [IEEE 754 standard](https://en.wikipedia.org/wiki/IEEE_754-2008_revision). It has a sign bit, a 5-bit exponent and 11-bit precision (with 10 bits explicitly stored), and can represent numbers between approximately `6.1e-05` and `6.5e+04` at full precision. This type is not widely supported by C compilers: it's available as `_Float16` type, if the compiler supports the Annex H of the C23 standard. On a typical machine, an unsigned short can be used for storage, but not for math operations. See the Wikipedia page on the [half-precision floating-point format](https://en.wikipedia.org/wiki/Half-precision_floating-point_format) for more information.

\(7\)  
When packing, `'x'` inserts one NUL byte.

\(8\)  
The `'p'` type code encodes a "Pascal string", meaning a short variable-length string stored in a *fixed number of bytes*, given by the count. The first byte stored is the length of the string, or 255, whichever is smaller. The bytes of the string follow. If the byte string passed in to `pack` is too long (longer than the count minus 1), only the leading `count-1` bytes of the string are stored. If the byte string is shorter than `count-1`, it is padded with null bytes so that exactly count bytes in all are used. Note that for `unpack`, the `'p'` type code consumes `count` bytes, but that the `!bytes` object returned can never contain more than 255 bytes. When packing, arguments of types `bytes` and `bytearray` are accepted.

\(9\)  
For the `'s'` type code, the count is interpreted as the length of the byte string, not a repeat count like for the other type codes; for example, `'10s'` means a single 10-byte string mapping to or from a single Python byte string, while `'10c'` means 10 separate one byte character elements (e.g., `cccccccccc`) mapping to or from ten different Python byte objects. (See `struct-examples` for a concrete demonstration of the difference.) If a count is not given, it defaults to 1. For packing, the byte string is truncated or padded with null bytes as appropriate to make it fit. For unpacking, the resulting `!bytes` object always has exactly the specified number of bytes. As a special case, `'0s'` means a single, empty byte string (while `'0c'` means 0 characters). When packing, arguments of types `bytes` and `bytearray` are accepted.

\(10\)  
For the `'Zf'` and `'Zd'` type codes, the packed representation uses the IEEE 754 binary32 and binary64 format for components of the complex number, regardless of the floating-point format used by the platform. Note that complex types are available unconditionally, despite complex types being an optional feature in C. As specified in the C11 standard, each complex type is represented by a two-element C array containing, respectively, the real and imaginary parts. The `'F'` and `'D'` (for `'Zf'` and `'Zd'`, respectively) format characters are supported for compatibility.

A type code may be preceded by an integral repeat count. For example, the format string `'4h'` means exactly the same as `'hhhh'`.

Whitespace characters between formats are ignored; a count and its format must not contain whitespace though.

When packing a value `x` using one of the integer formats (`'b'`, `'B'`, `'h'`, `'H'`, `'i'`, `'I'`, `'l'`, `'L'`, `'q'`, `'Q'`), if `x` is outside the valid range for that format then `struct.error` is raised.

<div class="versionchanged">

3.1 Previously, some of the integer formats wrapped out-of-range values and raised `DeprecationWarning` instead of `struct.error`.

</div>

<div class="index">

single: ? (question mark); in struct format strings

</div>

For the `'?'` type code, the return value is either `True` or `False`. When packing, the truth value of the argument object is used. Either 0 or 1 in the native or standard bool representation will be packed, and any non-zero value will be `True` when unpacking.

### Examples

> [!NOTE]
> Native byte order examples (designated by the `'@'` format prefix or lack of any prefix character) may not match what the reader's machine produces as that depends on the platform and compiler.

Pack and unpack integers of three different sizes, using big endian ordering:

    >>> from struct import *
    >>> pack(">bhl", 1, 2, 3)
    b'\x01\x00\x02\x00\x00\x00\x03'
    >>> unpack('>bhl', b'\x01\x00\x02\x00\x00\x00\x03')
    (1, 2, 3)
    >>> calcsize('>bhl')
    7

Attempt to pack an integer which is too large for the defined field:

    >>> pack(">h", 99999)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    struct.error: 'h' format requires -32768 <= number <= 32767

Demonstrate the difference between `'s'` and `'c'` format characters:

    >>> pack("@ccc", b'1', b'2', b'3')
    b'123'
    >>> pack("@3s", b'123')
    b'123'

Unpacked fields can be named by assigning them to variables or by wrapping the result in a named tuple:

    >>> record = b'raymond   \x32\x12\x08\x01\x08'
    >>> name, serialnum, school, gradelevel = unpack('<10sHHb', record)

    >>> from collections import namedtuple
    >>> Student = namedtuple('Student', 'name serialnum school gradelevel')
    >>> Student._make(unpack('<10sHHb', record))
    Student(name=b'raymond   ', serialnum=4658, school=264, gradelevel=8)

The ordering of type codes may have an impact on size in native mode since padding is implicit. In standard mode, the user is responsible for inserting any desired padding. Note in the first `pack` call below that three NUL bytes were added after the packed `'#'` to align the following integer on a four-byte boundary. In this example, the output was produced on a little endian machine:

    >>> pack('@ci', b'#', 0x12131415)
    b'#\x00\x00\x00\x15\x14\x13\x12'
    >>> pack('@ic', 0x12131415, b'#')
    b'\x15\x14\x13\x12#'
    >>> calcsize('@ci')
    8
    >>> calcsize('@ic')
    5

The following format `'llh0l'` results in two pad bytes being added at the end, assuming the platform's longs are aligned on 4-byte boundaries:

    >>> pack('@llh0l', 1, 2, 3)
    b'\x00\x00\x00\x01\x00\x00\x00\x02\x00\x03\x00\x00'

<div class="seealso">

Module `array`  
Packed binary storage of homogeneous data.

Module `json`  
JSON encoder and decoder.

Module `pickle`  
Python object serialization.

</div>

## Applications

Two main applications for the `!struct` module exist, data interchange between Python and C code within an application or another application compiled using the same compiler (`native formats<struct-native-formats>`), and data interchange between applications using agreed upon data layout (`standard formats<struct-standard-formats>`). Generally speaking, the format strings constructed for these two domains are distinct.

### Native Formats

When constructing format strings which mimic native layouts, the compiler and machine architecture determine byte ordering and padding. In such cases, the `@` format character should be used to specify native byte ordering and data sizes. Internal pad bytes are normally inserted automatically. It is possible that a zero-repeat type code will be needed at the end of a format string to round up to the correct byte boundary for proper alignment of consecutive chunks of data.

Consider these two simple examples (on a 64-bit, little-endian machine):

    >>> calcsize('@lhl')
    24
    >>> calcsize('@llh')
    18

Data is not padded to an 8-byte boundary at the end of the second format string without the use of extra padding. A zero-repeat format code solves that problem:

    >>> calcsize('@llh0l')
    24

The `'x'` type code can be used to specify the repeat, but for native formats it is better to use a zero-repeat format like `'0l'`.

By default, native byte ordering and alignment is used, but it is better to be explicit and use the `'@'` prefix character.

### Standard Formats

When exchanging data beyond your process such as networking or storage, be precise. Specify the exact byte order, size, and alignment. Do not assume they match the native order of a particular machine. For example, network byte order is big-endian, while many popular CPUs are little-endian. By defining this explicitly, the user need not care about the specifics of the platform their code is running on. The first character should typically be `<` or `>` (or `!`). Padding is the responsibility of the programmer. The zero-repeat format character won't work. Instead, the user must explicitly add `'x'` pad bytes where needed. Revisiting the examples from the previous section, we have:

    >>> calcsize('<qh6xq')
    24
    >>> pack('<qh6xq', 1, 2, 3) == pack('@lhl', 1, 2, 3)
    True
    >>> calcsize('@llh')
    18
    >>> pack('@llh', 1, 2, 3) == pack('<qqh', 1, 2, 3)
    True
    >>> calcsize('<qqh6x')
    24
    >>> calcsize('@llh0l')
    24
    >>> pack('@llh0l', 1, 2, 3) == pack('<qqh6x', 1, 2, 3)
    True

The above results (executed on a 64-bit machine) aren't guaranteed to match when executed on different machines. For example, the examples below were executed on a 32-bit machine:

    >>> calcsize('<qqh6x')
    24
    >>> calcsize('@llh0l')
    12
    >>> pack('@llh0l', 1, 2, 3) == pack('<qqh6x', 1, 2, 3)
    False

## Classes

The `!struct` module also defines the following type:

<div class="Struct(format)">

Return a new Struct object which writes and reads binary data according to the format string *format*. Creating a `Struct` object once and calling its methods is more efficient than calling module-level functions with the same format since the format string is only compiled once.

> [!NOTE]
> The compiled versions of the most recent format strings passed to the module-level functions are cached, so programs that use only a few format strings needn't worry about reusing a single `Struct` instance.

Compiled Struct objects support the following methods and attributes:

<div class="method">

pack(v1, v2, ...)

Identical to the `pack` function, using the compiled format. (`len(result)` will equal `size`.)

</div>

<div class="method">

pack_into(buffer, offset, v1, v2, ...)

Identical to the `pack_into` function, using the compiled format.

</div>

<div class="method">

unpack(buffer)

Identical to the `unpack` function, using the compiled format. The buffer's size in bytes must equal `size`.

</div>

<div class="method">

unpack_from(buffer, offset=0)

Identical to the `unpack_from` function, using the compiled format. The buffer's size in bytes, starting at position *offset*, must be at least `size`.

</div>

<div class="method">

iter_unpack(buffer)

Identical to the `iter_unpack` function, using the compiled format. The buffer's size in bytes must be a multiple of `size`.

<div class="versionadded">

3.4

</div>

</div>

<div class="attribute">

format

The format string used to construct this Struct object.

<div class="versionchanged">

3.7 The format string type is now `str` instead of `bytes`.

</div>

</div>

<div class="attribute">

size

The calculated size of the struct (and hence of the bytes object produced by the `pack` method) corresponding to `format`.

</div>

<div class="versionchanged">

3.13 The *repr()* of structs has changed. It is now:

\>\>\> Struct('i') Struct('i')

</div>

</div>

# `!subprocess` --- Subprocess management

<div class="module" data-synopsis="Subprocess management.">

subprocess

</div>

**Source code:** `Lib/subprocess.py`

------------------------------------------------------------------------

The `!subprocess` module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several older modules and functions:

    os.system
    os.spawn*

Information about how the `!subprocess` module can be used to replace these modules and functions can be found in the following sections.

<div class="seealso">

`324` -- PEP proposing the subprocess module

</div>

<div class="availability">

not Android, not iOS, not WASI.

This module is not supported on `mobile platforms <mobile-availability>` or `WebAssembly platforms <wasm-availability>`.

</div>

## Using the `!subprocess` Module

The recommended approach to invoking subprocesses is to use the `run` function for all use cases it can handle. For more advanced use cases, the underlying `Popen` interface can be used directly.

<div class="function">

run(args, *, stdin=None, input=None, stdout=None, stderr=None,capture_output=False, shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None, text=None, env=None, universal_newlines=None,*\*other_popen_kwargs)

Run the command described by *args*. Wait for command to complete, then return a `CompletedProcess` instance.

The arguments shown above are merely the most common ones, described below in `frequently-used-arguments` (hence the use of keyword-only notation in the abbreviated signature). The full function signature is largely the same as that of the `Popen` constructor - most of the arguments to this function are passed through to that interface. (*timeout*, *input*, *check*, and *capture_output* are not.)

If *capture_output* is true, stdout and stderr will be captured. When used, the internal `Popen` object is automatically created with *stdout* and *stderr* both set to `~subprocess.PIPE`. The *stdout* and *stderr* arguments may not be supplied at the same time as *capture_output*. If you wish to capture and combine both streams into one, set *stdout* to `~subprocess.PIPE` and *stderr* to `~subprocess.STDOUT`, instead of using *capture_output*.

A *timeout* may be specified in seconds, it is internally passed on to `Popen.communicate`. If the timeout expires, the child process will be killed and waited for. The `TimeoutExpired` exception will be re-raised after the child process has terminated. The initial process creation itself cannot be interrupted on many platform APIs so you are not guaranteed to see a timeout exception until at least after however long process creation takes.

The *input* argument is passed to `Popen.communicate` and thus to the subprocess's stdin. If used it must be a byte sequence, or a string if *encoding* or *errors* is specified or *text* is true. When used, the internal `Popen` object is automatically created with *stdin* set to `~subprocess.PIPE`, and the *stdin* argument may not be used as well.

If *check* is true, and the process exits with a non-zero exit code, a `CalledProcessError` exception will be raised. Attributes of that exception hold the arguments, the exit code, and stdout and stderr if they were captured.

If *encoding* or *errors* are specified, or *text* is true, file objects for stdin, stdout and stderr are opened in text mode using the specified *encoding* and *errors* or the `io.TextIOWrapper` default. The *universal_newlines* argument is equivalent to *text* and is provided for backwards compatibility. By default, file objects are opened in binary mode.

If *env* is not `None`, it must be a mapping that defines the environment variables for the new process; these are used instead of the default behavior of inheriting the current process' environment. It is passed directly to `Popen`. This mapping can be str to str on any platform or bytes to bytes on POSIX platforms much like `os.environ` or `os.environb`.

Examples:

    >>> subprocess.run(["ls", "-l"])  # doesn't capture output
    CompletedProcess(args=['ls', '-l'], returncode=0)

    >>> subprocess.run("exit 1", shell=True, check=True)
    Traceback (most recent call last):
      ...
    subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1

    >>> subprocess.run(["ls", "-l", "/dev/null"], capture_output=True)
    CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0,
    stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n', stderr=b'')

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.6

Added *encoding* and *errors* parameters

</div>

<div class="versionchanged">

3.7

Added the *text* parameter, as a more understandable alias of *universal_newlines*. Added the *capture_output* parameter.

</div>

<div class="versionchanged">

3.12

Changed Windows shell search order for `shell=True`. The current directory and `%PATH%` are replaced with `%COMSPEC%` and `%SystemRoot%\System32\cmd.exe`. As a result, dropping a malicious program named `cmd.exe` into a current directory no longer works.

</div>

</div>

<div class="CompletedProcess">

The return value from `run`, representing a process that has finished.

<div class="attribute">

args

The arguments used to launch the process. This may be a list or a string.

</div>

<div class="attribute">

returncode

Exit status of the child process. Typically, an exit status of 0 indicates that it ran successfully.

A negative value `-N` indicates that the child was terminated by signal `N` (POSIX only).

</div>

<div class="attribute">

stdout

Captured stdout from the child process. A bytes sequence, or a string if `run` was called with an encoding, errors, or text=True. `None` if stdout was not captured.

If you ran the process with `stderr=subprocess.STDOUT`, stdout and stderr will be combined in this attribute, and `stderr` will be `None`.

</div>

<div class="attribute">

stderr

Captured stderr from the child process. A bytes sequence, or a string if `run` was called with an encoding, errors, or text=True. `None` if stderr was not captured.

</div>

<div class="method">

check_returncode()

If `returncode` is non-zero, raise a `CalledProcessError`.

</div>

<div class="versionadded">

3.5

</div>

</div>

<div class="data">

DEVNULL

Special value that can be used as the *stdin*, *stdout* or *stderr* argument to `Popen` and indicates that the special file `os.devnull` will be used.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

PIPE

Special value that can be used as the *stdin*, *stdout* or *stderr* argument to `Popen` and indicates that a pipe to the standard stream should be opened. Most useful with `Popen.communicate`.

</div>

<div class="data">

STDOUT

Special value that can be used as the *stderr* argument to `Popen` and indicates that standard error should go into the same handle as standard output.

</div>

<div class="exception">

SubprocessError

Base class for all other exceptions from this module.

<div class="versionadded">

3.3

</div>

</div>

<div class="exception">

TimeoutExpired

Subclass of `SubprocessError`, raised when a timeout expires while waiting for a child process.

<div class="attribute">

cmd

Command that was used to spawn the child process.

</div>

<div class="attribute">

timeout

Timeout in seconds.

</div>

<div class="attribute">

output

Output of the child process if it was captured by `run` or `check_output`. Otherwise, `None`. This is always `bytes` when any output was captured regardless of the `text=True` setting. It may remain `None` instead of `b''` when no output was observed.

</div>

<div class="attribute">

stdout

Alias for output, for symmetry with `stderr`.

</div>

<div class="attribute">

stderr

Stderr output of the child process if it was captured by `run`. Otherwise, `None`. This is always `bytes` when stderr output was captured regardless of the `text=True` setting. It may remain `None` instead of `b''` when no stderr output was observed.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.5 *stdout* and *stderr* attributes added

</div>

</div>

<div class="exception">

CalledProcessError

Subclass of `SubprocessError`, raised when a process run by `check_call`, `check_output`, or `run` (with `check=True`) returns a non-zero exit status.

<div class="attribute">

returncode

Exit status of the child process, an integer. If the process exited due to a signal, this will be the negative signal number.

</div>

<div class="attribute">

cmd

Command that was used to spawn the child process.

</div>

<div class="attribute">

output

Output of the child process if it was captured by `run` or `check_output`. Otherwise, `None`.

</div>

<div class="attribute">

stdout

Alias for output, for symmetry with `stderr`.

</div>

<div class="attribute">

stderr

Stderr output of the child process if it was captured by `run`. Otherwise, `None`.

</div>

<div class="versionchanged">

3.5 *stdout* and *stderr* attributes added

</div>

</div>

### Frequently Used Arguments

To support a wide variety of use cases, the `Popen` constructor (and the convenience functions) accept a large number of optional arguments. For most typical use cases, many of these arguments can be safely left at their default values. The arguments that are most commonly needed are:

> *args* is required for all calls and should be a string, or a sequence of program arguments. Providing a sequence of arguments is generally preferred, as it allows the module to take care of any required escaping and quoting of arguments (e.g. to permit spaces in file names). If passing a single string, either *shell* must be `True` (see below) or else the string must simply name the program to be executed without specifying any arguments.
>
> *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values are `None`, `PIPE`, `DEVNULL`, an existing file descriptor (a positive integer), and an existing `file object` with a valid file descriptor. With the default settings of `None`, no redirection will occur. `PIPE` indicates that a new pipe to the child should be created. `DEVNULL` indicates that the special file `os.devnull` will be used. Additionally, *stderr* can be `STDOUT`, which indicates that the stderr data from the child process should be captured into the same file handle as for *stdout*.
>
> <div class="index">
>
> single: universal newlines; subprocess module
>
> </div>
>
> If *encoding* or *errors* are specified, or *text* (also known as *universal_newlines*) is true, the file objects *stdin*, *stdout* and *stderr* will be opened in text mode using the *encoding* and *errors* specified in the call or the defaults for `io.TextIOWrapper`.
>
> For *stdin*, line ending characters `'\n'` in the input will be converted to the default line separator `os.linesep`. For *stdout* and *stderr*, all line endings in the output will be converted to `'\n'`. For more information see the documentation of the `io.TextIOWrapper` class when the *newline* argument to its constructor is `None`.
>
> If text mode is not used, *stdin*, *stdout* and *stderr* will be opened as binary streams. No encoding or line ending conversion is performed.
>
> <div class="versionchanged">
>
> 3.6 Added the *encoding* and *errors* parameters.
>
> </div>
>
> <div class="versionchanged">
>
> 3.7 Added the *text* parameter as an alias for *universal_newlines*.
>
> </div>
>
> > [!NOTE]
> > The newlines attribute of the file objects `Popen.stdin`, `Popen.stdout` and `Popen.stderr` are not updated by the `Popen.communicate` method.
>
> If *shell* is `True`, the specified command will be executed through the shell. This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want convenient access to other shell features such as shell pipes, filename wildcards, environment variable expansion, and expansion of `~` to a user's home directory. However, note that Python itself offers implementations of many shell-like features (in particular, `glob`, `fnmatch`, `os.walk`, `os.path.expandvars`, `os.path.expanduser`, and `shutil`).
>
> <div class="versionchanged">
>
> 3.3 When *universal_newlines* is `True`, the class uses the encoding `locale.getpreferredencoding(False) <locale.getpreferredencoding>` instead of `locale.getpreferredencoding()`. See the `io.TextIOWrapper` class for more information on this change.
>
> </div>
>
> > [!NOTE]
> > Read the [Security Considerations](#security-considerations) section before using `shell=True`.

These options, along with all of the other options, are described in more detail in the `Popen` constructor documentation.

### Popen Constructor

The underlying process creation and management in this module is handled by the `Popen` class. It offers a lot of flexibility so that developers are able to handle the less common cases not covered by the convenience functions.

<div class="Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, \ stderr=None, preexec_fn=None, close_fds=True, shell=False, \ cwd=None, env=None, universal_newlines=None, \ startupinfo=None, creationflags=0, restore_signals=True, \ start_new_session=False, pass_fds=(), *, group=None, \ extra_groups=None, user=None, umask=-1, \ encoding=None, errors=None, text=None, pipesize=-1, \ process_group=None)">

Execute a child program in a new process. On POSIX, the class uses `os.execvpe`-like behavior to execute the child program. On Windows, the class uses the Windows `CreateProcess()` function. The arguments to `Popen` are as follows.

*args* should be a sequence of program arguments or else a single string or `path-like object`. By default, the program to execute is the first item in *args* if *args* is a sequence. If *args* is a string, the interpretation is platform-dependent and described below. See the *shell* and *executable* arguments for additional differences from the default behavior. Unless otherwise stated, it is recommended to pass *args* as a sequence.

> [!WARNING]
> For maximum reliability, use a fully qualified path for the executable. To search for an unqualified name on `PATH`, use `shutil.which`. On all platforms, passing `sys.executable` is the recommended way to launch the current Python interpreter again, and use the `-m` command-line format to launch an installed module.
>
> Resolving the path of *executable* (or the first item of *args*) is platform dependent. For POSIX, see `os.execvpe`, and note that when resolving or searching for the executable path, *cwd* overrides the current working directory and *env* can override the `PATH` environment variable. For Windows, see the documentation of the `lpApplicationName` and `lpCommandLine` parameters of WinAPI `CreateProcess`, and note that when resolving or searching for the executable path with `shell=False`, *cwd* does not override the current working directory and *env* cannot override the `PATH` environment variable. Using a full path avoids all of these variations.

An example of passing some arguments to an external program as a sequence is:

    Popen(["/usr/bin/git", "commit", "-m", "Fixes a bug."])

On POSIX, if *args* is a string, the string is interpreted as the name or path of the program to execute. However, this can only be done if not passing arguments to the program.

> [!NOTE]
> It may not be obvious how to break a shell command into a sequence of arguments, especially in complex cases. `shlex.split` can illustrate how to determine the correct tokenization for *args*:
>
>     >>> import shlex, subprocess
>     >>> command_line = input()
>     /bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'"
>     >>> args = shlex.split(command_line)
>     >>> print(args)
>     ['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]
>     >>> p = subprocess.Popen(args) # Success!
>
> Note in particular that options (such as *-input*) and arguments (such as *eggs.txt*) that are separated by whitespace in the shell go in separate list elements, while arguments that need quoting or backslash escaping when used in the shell (such as filenames containing spaces or the *echo* command shown above) are single list elements.

On Windows, if *args* is a sequence, it will be converted to a string in a manner described in `converting-argument-sequence`. This is because the underlying `CreateProcess()` operates on strings.

<div class="versionchanged">

3.6 *args* parameter accepts a `path-like object` if *shell* is `False` and a sequence containing path-like objects on POSIX.

</div>

<div class="versionchanged">

3.8 *args* parameter accepts a `path-like object` if *shell* is `False` and a sequence containing bytes and path-like objects on Windows.

</div>

The *shell* argument (which defaults to `False`) specifies whether to use the shell as the program to execute. If *shell* is `True`, it is recommended to pass *args* as a string rather than as a sequence.

On POSIX with `shell=True`, the shell defaults to `/bin/sh`. If *args* is a string, the string specifies the command to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If *args* is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself. That is to say, `Popen` does the equivalent of:

    Popen(['/bin/sh', '-c', args[0], args[1], ...])

On Windows with `shell=True`, the `COMSPEC` environment variable specifies the default shell. The only time you need to specify `shell=True` on Windows is when the command you wish to execute is built into the shell (e.g. `dir` or `copy`). You do not need `shell=True` to run a batch file or console-based executable.

> [!NOTE]
> Read the [Security Considerations](#security-considerations) section before using `shell=True`.

*bufsize* will be supplied as the corresponding argument to the `open` function when creating the stdin/stdout/stderr pipe file objects:

- `0` means unbuffered (read and write are one system call and can return short)
- `1` means line buffered (only usable if `text=True` or `universal_newlines=True`)
- any other positive value means use a buffer of approximately that size
- negative bufsize (the default) means the system default of io.DEFAULT_BUFFER_SIZE will be used.

<div class="versionchanged">

3.3.1 *bufsize* now defaults to -1 to enable buffering by default to match the behavior that most code expects. In versions prior to Python 3.2.4 and 3.3.1 it incorrectly defaulted to `0` which was unbuffered and allowed short reads. This was unintentional and did not match the behavior of Python 2 as most code expected.

</div>

The *executable* argument specifies a replacement program to execute. It is very seldom needed. When `shell=False`, *executable* replaces the program to execute specified by *args*. However, the original *args* is still passed to the program. Most programs treat the program specified by *args* as the command name, which can then be different from the program actually executed. On POSIX, the *args* name becomes the display name for the executable in utilities such as `ps`. If `shell=True`, on POSIX the *executable* argument specifies a replacement shell for the default `/bin/sh`.

<div class="versionchanged">

3.6 *executable* parameter accepts a `path-like object` on POSIX.

</div>

<div class="versionchanged">

3.8 *executable* parameter accepts a bytes and `path-like object` on Windows.

</div>

<div class="versionchanged">

3.12

Changed Windows shell search order for `shell=True`. The current directory and `%PATH%` are replaced with `%COMSPEC%` and `%SystemRoot%\System32\cmd.exe`. As a result, dropping a malicious program named `cmd.exe` into a current directory no longer works.

</div>

*stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values are `None`, `PIPE`, `DEVNULL`, an existing file descriptor (a positive integer), and an existing `file object` with a valid file descriptor. With the default settings of `None`, no redirection will occur. `PIPE` indicates that a new pipe to the child should be created. `DEVNULL` indicates that the special file `os.devnull` will be used. Additionally, *stderr* can be `STDOUT`, which indicates that the stderr data from the applications should be captured into the same file handle as for *stdout*.

If *preexec_fn* is set to a callable object, this object will be called in the child process just before the child is executed. (POSIX only)

> [!WARNING]
> The *preexec_fn* parameter is NOT SAFE to use in the presence of threads in your application. The child process could deadlock before exec is called.

> [!NOTE]
> If you need to modify the environment for the child use the *env* parameter rather than doing it in a *preexec_fn*. The *start_new_session* and *process_group* parameters should take the place of code using *preexec_fn* to call `os.setsid` or `os.setpgid` in the child.

<div class="versionchanged">

3.8

The *preexec_fn* parameter is no longer supported in subinterpreters. The use of the parameter in a subinterpreter raises `RuntimeError`. The new restriction may affect applications that are deployed in mod_wsgi, uWSGI, and other embedded environments.

</div>

If *close_fds* is true, all file descriptors except `0`, `1` and `2` will be closed before the child process is executed. Otherwise when *close_fds* is false, file descriptors obey their inheritable flag as described in `fd_inheritance`.

On Windows, if *close_fds* is true then no handles will be inherited by the child process unless explicitly passed in the `handle_list` element of `STARTUPINFO.lpAttributeList`, or by standard handle redirection.

<div class="versionchanged">

3.2 The default for *close_fds* was changed from `False` to what is described above.

</div>

<div class="versionchanged">

3.7 On Windows the default for *close_fds* was changed from `False` to `True` when redirecting the standard handles. It's now possible to set *close_fds* to `True` when redirecting the standard handles.

</div>

*pass_fds* is an optional sequence of file descriptors to keep open between the parent and child. Providing any *pass_fds* forces *close_fds* to be `True`. (POSIX only)

<div class="versionchanged">

3.2 The *pass_fds* parameter was added.

</div>

If *cwd* is not `None`, the function changes the working directory to *cwd* before executing the child. *cwd* can be a string, bytes or `path-like <path-like object>` object. On POSIX, the function looks for *executable* (or for the first item in *args*) relative to *cwd* if the executable path is a relative path.

<div class="versionchanged">

3.6 *cwd* parameter accepts a `path-like object` on POSIX.

</div>

<div class="versionchanged">

3.7 *cwd* parameter accepts a `path-like object` on Windows.

</div>

<div class="versionchanged">

3.8 *cwd* parameter accepts a bytes object on Windows.

</div>

If *restore_signals* is true (the default) all signals that Python has set to SIG_IGN are restored to SIG_DFL in the child process before the exec. Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. (POSIX only)

<div class="versionchanged">

3.2 *restore_signals* was added.

</div>

If *start_new_session* is true the `setsid()` system call will be made in the child process prior to the execution of the subprocess.

<div class="availability">

POSIX

</div>

<div class="versionchanged">

3.2 *start_new_session* was added.

</div>

If *process_group* is a non-negative integer, the `setpgid(0, value)` system call will be made in the child process prior to the execution of the subprocess.

<div class="availability">

POSIX

</div>

<div class="versionchanged">

3.11 *process_group* was added.

</div>

If *group* is not `None`, the setregid() system call will be made in the child process prior to the execution of the subprocess. If the provided value is a string, it will be looked up via `grp.getgrnam` and the value in `gr_gid` will be used. If the value is an integer, it will be passed verbatim. (POSIX only)

<div class="availability">

POSIX

</div>

<div class="versionadded">

3.9

</div>

If *extra_groups* is not `None`, the setgroups() system call will be made in the child process prior to the execution of the subprocess. Strings provided in *extra_groups* will be looked up via `grp.getgrnam` and the values in `gr_gid` will be used. Integer values will be passed verbatim. (POSIX only)

<div class="availability">

POSIX

</div>

<div class="versionadded">

3.9

</div>

If *user* is not `None`, the setreuid() system call will be made in the child process prior to the execution of the subprocess. If the provided value is a string, it will be looked up via `pwd.getpwnam` and the value in `pw_uid` will be used. If the value is an integer, it will be passed verbatim. (POSIX only)

> [!NOTE]
> Specifying *user* will not drop existing supplementary group memberships! The caller must also pass `extra_groups=()` to reduce the group membership of the child process for security purposes.

<div class="availability">

POSIX

</div>

<div class="versionadded">

3.9

</div>

If *umask* is not negative, the umask() system call will be made in the child process prior to the execution of the subprocess.

<div class="availability">

POSIX

</div>

<div class="versionadded">

3.9

</div>

If *env* is not `None`, it must be a mapping that defines the environment variables for the new process; these are used instead of the default behavior of inheriting the current process' environment. This mapping can be str to str on any platform or bytes to bytes on POSIX platforms much like `os.environ` or `os.environb`.

> [!NOTE]
> If specified, *env* must provide any variables required for the program to execute. On Windows, in order to run a [side-by-side assembly](https://en.wikipedia.org/wiki/Side-by-Side_Assembly) the specified *env* **must** include a valid `%SystemRoot%`.

If *encoding* or *errors* are specified, or *text* is true, the file objects *stdin*, *stdout* and *stderr* are opened in text mode with the specified *encoding* and *errors*, as described above in `frequently-used-arguments`. The *universal_newlines* argument is equivalent to *text* and is provided for backwards compatibility. By default, file objects are opened in binary mode.

<div class="versionadded">

3.6 *encoding* and *errors* were added.

</div>

<div class="versionadded">

3.7 *text* was added as a more readable alias for *universal_newlines*.

</div>

If given, *startupinfo* will be a `STARTUPINFO` object, which is passed to the underlying `CreateProcess` function.

If given, *creationflags*, can be one or more of the following flags:

- `CREATE_NEW_CONSOLE`
- `CREATE_NEW_PROCESS_GROUP`
- `ABOVE_NORMAL_PRIORITY_CLASS`
- `BELOW_NORMAL_PRIORITY_CLASS`
- `HIGH_PRIORITY_CLASS`
- `IDLE_PRIORITY_CLASS`
- `NORMAL_PRIORITY_CLASS`
- `REALTIME_PRIORITY_CLASS`
- `CREATE_NO_WINDOW`
- `DETACHED_PROCESS`
- `CREATE_DEFAULT_ERROR_MODE`
- `CREATE_BREAKAWAY_FROM_JOB`

*pipesize* can be used to change the size of the pipe when `PIPE` is used for *stdin*, *stdout* or *stderr*. The size of the pipe is only changed on platforms that support this (only Linux at this time of writing). Other platforms will ignore this parameter.

<div class="versionchanged">

3.10 Added the *pipesize* parameter.

</div>

Popen objects are supported as context managers via the `with` statement: on exit, standard file descriptors are closed, and the process is waited for. :

    with Popen(["ifconfig"], stdout=PIPE) as proc:
        log.write(proc.stdout.read())

<div class="audit-event">

subprocess.Popen executable,args,cwd,env subprocess.Popen

Popen and the other functions in this module that use it raise an `auditing event <auditing>` `subprocess.Popen` with arguments `executable`, `args`, `cwd`, and `env`. The value for `args` may be a single string or a list of strings, depending on platform.

</div>

<div class="versionchanged">

3.2 Added context manager support.

</div>

<div class="versionchanged">

3.6 Popen destructor now emits a `ResourceWarning` warning if the child process is still running.

</div>

<div class="versionchanged">

3.8 Popen can use `os.posix_spawn` in some cases for better performance. On Windows Subsystem for Linux and QEMU User Emulation, Popen constructor using `os.posix_spawn` no longer raise an exception on errors like missing program, but the child process fails with a non-zero `~Popen.returncode`.

</div>

</div>

### Exceptions

Exceptions raised in the child process, before the new program has started to execute, will be re-raised in the parent.

The most common exception raised is `OSError`. This occurs, for example, when trying to execute a non-existent file. Applications should prepare for `OSError` exceptions. Note that, when `shell=True`, `OSError` will be raised by the child only if the selected shell itself was not found. To determine if the shell failed to find the requested application, it is necessary to check the return code or output from the subprocess.

A `ValueError` will be raised if `Popen` is called with invalid arguments.

`check_call` and `check_output` will raise `CalledProcessError` if the called process returns a non-zero return code.

All of the functions and methods that accept a *timeout* parameter, such as `run` and `Popen.communicate` will raise `TimeoutExpired` if the timeout expires before the process exits.

Exceptions defined in this module all inherit from `SubprocessError`.

<div class="versionadded">

3.3 The `SubprocessError` base class was added.

</div>

## Security Considerations

Unlike some other popen functions, this library will not implicitly choose to call a system shell. This means that all characters, including shell metacharacters, can safely be passed to child processes. If the shell is invoked explicitly, via `shell=True`, it is the application's responsibility to ensure that all whitespace and metacharacters are quoted appropriately to avoid [shell injection](https://en.wikipedia.org/wiki/Shell_injection#Shell_injection) vulnerabilities. On `some platforms <shlex-quote-warning>`, it is possible to use `shlex.quote` for this escaping.

On Windows, batch files (`*.bat` or `*.cmd`) may be launched by the operating system in a system shell regardless of the arguments passed to this library. This could result in arguments being parsed according to shell rules, but without any escaping added by Python. If you are intentionally launching a batch file with arguments from untrusted sources, consider passing `shell=True` to allow Python to escape special characters. See `114539` for additional discussion.

## Popen Objects

Instances of the `Popen` class have the following methods:

<div class="method">

Popen.poll()

Check if child process has terminated. Set and return `~Popen.returncode` attribute. Otherwise, returns `None`.

</div>

<div class="method">

Popen.wait(timeout=None)

Wait for child process to terminate. Set and return `~Popen.returncode` attribute.

If the process does not terminate after *timeout* seconds, raise a `TimeoutExpired` exception. It is safe to catch this exception and retry the wait.

> [!NOTE]
> This will deadlock when using `stdout=PIPE` or `stderr=PIPE` and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use `Popen.communicate` when using pipes to avoid that.

> [!NOTE]
> When `timeout` is not `None` and the platform supports it, an efficient event-driven mechanism is used to wait for process termination:
>
> - Linux \>= 5.3 uses `os.pidfd_open` + `select.poll`
> - macOS and other BSD variants use `select.kqueue` + `KQ_FILTER_PROC` + `KQ_NOTE_EXIT`
> - Windows uses `WaitForSingleObject`
>
> If none of these mechanisms are available, the function falls back to a busy loop (non-blocking call and short sleeps).

> [!NOTE]
> Use the `asyncio` module for an asynchronous wait: see `asyncio.create_subprocess_exec`.

<div class="versionchanged">

3.3 *timeout* was added.

</div>

<div class="versionchanged">

3.15 if *timeout* is not `None`, use efficient event-driven implementation on Linux \>= 5.3 and macOS / BSD.

</div>

</div>

<div class="method">

Popen.communicate(input=None, timeout=None)

Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate and set the `~Popen.returncode` attribute. The optional *input* argument should be data to be sent to the child process, or `None`, if no data should be sent to the child. If streams were opened in text mode, *input* must be a string. Otherwise, it must be bytes.

`communicate` returns a tuple `(stdout_data, stderr_data)`. The data will be strings if streams were opened in text mode; otherwise, bytes.

Note that if you want to send data to the process's stdin, you need to create the Popen object with `stdin=PIPE`. Similarly, to get anything other than `None` in the result tuple, you need to give `stdout=PIPE` and/or `stderr=PIPE` too.

If the process does not terminate after *timeout* seconds, a `TimeoutExpired` exception will be raised. Catching this exception and retrying communication will not lose any output. Supplying *input* to a subsequent post-timeout `communicate` call is in undefined behavior and may become an error in the future.

The child process is not killed if the timeout expires, so in order to cleanup properly a well-behaved application should kill the child process and finish communication:

    proc = subprocess.Popen(...)
    try:
        outs, errs = proc.communicate(timeout=15)
    except TimeoutExpired:
        proc.kill()
        outs, errs = proc.communicate()

After a call to `~Popen.communicate` raises `TimeoutExpired`, do not call `~Popen.wait`. Use an additional `~Popen.communicate` call to finish handling pipes and populate the `~Popen.returncode` attribute.

> [!NOTE]
> The data read is buffered in memory, so do not use this method if the data size is large or unlimited.

<div class="versionchanged">

3.3 *timeout* was added.

</div>

</div>

<div class="method">

Popen.send_signal(signal)

Sends the signal *signal* to the child.

Do nothing if the process completed.

> [!NOTE]
> On Windows, SIGTERM is an alias for `terminate`. CTRL_C_EVENT and CTRL_BREAK_EVENT can be sent to processes started with a *creationflags* parameter which includes `CREATE_NEW_PROCESS_GROUP`.

</div>

<div class="method">

Popen.terminate()

Stop the child. On POSIX OSs the method sends `~signal.SIGTERM` to the child. On Windows the Win32 API function `!TerminateProcess` is called to stop the child.

</div>

<div class="method">

Popen.kill()

Kills the child. On POSIX OSs the function sends SIGKILL to the child. On Windows `kill` is an alias for `terminate`.

</div>

The following attributes are also set by the class for you to access. Reassigning them to new values is unsupported:

<div class="attribute">

Popen.args

The *args* argument as it was passed to `Popen` -- a sequence of program arguments or else a single string.

<div class="versionadded">

3.3

</div>

</div>

<div class="attribute">

Popen.stdin

If the *stdin* argument was `PIPE`, this attribute is a writeable stream object as returned by `open`. If the *encoding* or *errors* arguments were specified or the *text* or *universal_newlines* argument was `True`, the stream is a text stream, otherwise it is a byte stream. If the *stdin* argument was not `PIPE`, this attribute is `None`.

</div>

<div class="attribute">

Popen.stdout

If the *stdout* argument was `PIPE`, this attribute is a readable stream object as returned by `open`. Reading from the stream provides output from the child process. If the *encoding* or *errors* arguments were specified or the *text* or *universal_newlines* argument was `True`, the stream is a text stream, otherwise it is a byte stream. If the *stdout* argument was not `PIPE`, this attribute is `None`.

</div>

<div class="attribute">

Popen.stderr

If the *stderr* argument was `PIPE`, this attribute is a readable stream object as returned by `open`. Reading from the stream provides error output from the child process. If the *encoding* or *errors* arguments were specified or the *text* or *universal_newlines* argument was `True`, the stream is a text stream, otherwise it is a byte stream. If the *stderr* argument was not `PIPE`, this attribute is `None`.

</div>

> [!WARNING]
> Use `~Popen.communicate` rather than `.stdin.write <Popen.stdin>`, `.stdout.read <Popen.stdout>` or `.stderr.read <Popen.stderr>` to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

<div class="attribute">

Popen.pid

The process ID of the child process.

Note that if you set the *shell* argument to `True`, this is the process ID of the spawned shell.

</div>

<div class="attribute">

Popen.returncode

The child return code. Initially `None`, `returncode` is set by a call to the `poll`, `wait`, or `communicate` methods if they detect that the process has terminated.

A `None` value indicates that the process hadn't yet terminated at the time of the last method call.

A negative value `-N` indicates that the child was terminated by signal `N` (POSIX only).

When `shell=True`, the return code reflects the exit status of the shell itself (e.g. `/bin/sh`), which may map signals to codes such as `128+N`. See the documentation of the shell (for example, the Bash manual's Exit Status) for details.

</div>

## Windows Popen Helpers

The `STARTUPINFO` class and following constants are only available on Windows.

<div class="STARTUPINFO(*, dwFlags=0, hStdInput=None, hStdOutput=None, \ hStdError=None, wShowWindow=0, lpAttributeList=None)">

Partial support of the Windows [STARTUPINFO](https://msdn.microsoft.com/en-us/library/ms686331(v=vs.85).aspx) structure is used for `Popen` creation. The following attributes can be set by passing them as keyword-only arguments.

<div class="versionchanged">

3.7 Keyword-only argument support was added.

</div>

<div class="attribute">

dwFlags

A bit field that determines whether certain `STARTUPINFO` attributes are used when the process creates a window. :

    si = subprocess.STARTUPINFO()
    si.dwFlags = subprocess.STARTF_USESTDHANDLES | subprocess.STARTF_USESHOWWINDOW

</div>

<div class="attribute">

hStdInput

If `dwFlags` specifies `STARTF_USESTDHANDLES`, this attribute is the standard input handle for the process. If `STARTF_USESTDHANDLES` is not specified, the default for standard input is the keyboard buffer.

</div>

<div class="attribute">

hStdOutput

If `dwFlags` specifies `STARTF_USESTDHANDLES`, this attribute is the standard output handle for the process. Otherwise, this attribute is ignored and the default for standard output is the console window's buffer.

</div>

<div class="attribute">

hStdError

If `dwFlags` specifies `STARTF_USESTDHANDLES`, this attribute is the standard error handle for the process. Otherwise, this attribute is ignored and the default for standard error is the console window's buffer.

</div>

<div class="attribute">

wShowWindow

If `dwFlags` specifies `STARTF_USESHOWWINDOW`, this attribute can be any of the values that can be specified in the `nCmdShow` parameter for the [ShowWindow](https://msdn.microsoft.com/en-us/library/ms633548(v=vs.85).aspx) function, except for `SW_SHOWDEFAULT`. Otherwise, this attribute is ignored.

`SW_HIDE` is provided for this attribute. It is used when `Popen` is called with `shell=True`.

</div>

<div class="attribute">

lpAttributeList

A dictionary of additional attributes for process creation as given in `STARTUPINFOEX`, see [UpdateProcThreadAttribute](https://msdn.microsoft.com/en-us/library/windows/desktop/ms686880(v=vs.85).aspx).

Supported attributes:

**handle_list**  
Sequence of handles that will be inherited. *close_fds* must be true if non-empty.

The handles must be temporarily made inheritable by `os.set_handle_inheritable` when passed to the `Popen` constructor, else `OSError` will be raised with Windows error `ERROR_INVALID_PARAMETER` (87).

> [!WARNING]
> In a multithreaded process, use caution to avoid leaking handles that are marked inheritable when combining this feature with concurrent calls to other process creation functions that inherit all handles such as `os.system`. This also applies to standard handle redirection, which temporarily creates inheritable handles.

<div class="versionadded">

3.7

</div>

</div>

</div>

### Windows Constants

The `!subprocess` module exposes the following constants.

<div class="data">

STD_INPUT_HANDLE

The standard input device. Initially, this is the console input buffer, `CONIN$`.

</div>

<div class="data">

STD_OUTPUT_HANDLE

The standard output device. Initially, this is the active console screen buffer, `CONOUT$`.

</div>

<div class="data">

STD_ERROR_HANDLE

The standard error device. Initially, this is the active console screen buffer, `CONOUT$`.

</div>

<div class="data">

SW_HIDE

Hides the window. Another window will be activated.

</div>

<div class="data">

STARTF_USESTDHANDLES

Specifies that the `STARTUPINFO.hStdInput`, `STARTUPINFO.hStdOutput`, and `STARTUPINFO.hStdError` attributes contain additional information.

</div>

<div class="data">

STARTF_USESHOWWINDOW

Specifies that the `STARTUPINFO.wShowWindow` attribute contains additional information.

</div>

<div class="data">

STARTF_FORCEONFEEDBACK

A `STARTUPINFO.dwFlags` parameter to specify that the *Working in Background* mouse cursor will be displayed while a process is launching. This is the default behavior for GUI processes.

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

STARTF_FORCEOFFFEEDBACK

A `STARTUPINFO.dwFlags` parameter to specify that the mouse cursor will not be changed when launching a process.

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

CREATE_NEW_CONSOLE

The new process has a new console, instead of inheriting its parent's console (the default).

</div>

<div class="data">

CREATE_NEW_PROCESS_GROUP

A `Popen` `creationflags` parameter to specify that a new process group will be created. This flag is necessary for using `os.kill` on the subprocess.

This flag is ignored if `CREATE_NEW_CONSOLE` is specified.

</div>

<div class="data">

ABOVE_NORMAL_PRIORITY_CLASS

A `Popen` `creationflags` parameter to specify that a new process will have an above average priority.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

BELOW_NORMAL_PRIORITY_CLASS

A `Popen` `creationflags` parameter to specify that a new process will have a below average priority.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

HIGH_PRIORITY_CLASS

A `Popen` `creationflags` parameter to specify that a new process will have a high priority.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

IDLE_PRIORITY_CLASS

A `Popen` `creationflags` parameter to specify that a new process will have an idle (lowest) priority.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

NORMAL_PRIORITY_CLASS

A `Popen` `creationflags` parameter to specify that a new process will have a normal priority. (default)

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

REALTIME_PRIORITY_CLASS

A `Popen` `creationflags` parameter to specify that a new process will have realtime priority. You should almost never use REALTIME_PRIORITY_CLASS, because this interrupts system threads that manage mouse input, keyboard input, and background disk flushing. This class can be appropriate for applications that "talk" directly to hardware or that perform brief tasks that should have limited interruptions.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

CREATE_NO_WINDOW

A `Popen` `creationflags` parameter to specify that a new process will not create a window.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

DETACHED_PROCESS

A `Popen` `creationflags` parameter to specify that a new process will not inherit its parent's console. This value cannot be used with CREATE_NEW_CONSOLE.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

CREATE_DEFAULT_ERROR_MODE

A `Popen` `creationflags` parameter to specify that a new process does not inherit the error mode of the calling process. Instead, the new process gets the default error mode. This feature is particularly useful for multithreaded shell applications that run with hard errors disabled.

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

CREATE_BREAKAWAY_FROM_JOB

A `Popen` `creationflags` parameter to specify that a new process is not associated with the job.

<div class="versionadded">

3.7

</div>

</div>

## Older high-level API

Prior to Python 3.5, these three functions comprised the high level API to subprocess. You can now use `run` in many cases, but lots of existing code calls these functions.

<div class="function">

call(args, *, stdin=None, stdout=None, stderr=None, shell=False, cwd=None, timeout=None,*\*other_popen_kwargs)

Run the command described by *args*. Wait for command to complete, then return the `~Popen.returncode` attribute.

Code needing to capture stdout or stderr should use `run` instead:

    run(...).returncode

To suppress stdout or stderr, supply a value of `DEVNULL`.

The arguments shown above are merely some common ones. The full function signature is the same as that of the `Popen` constructor - this function passes all supplied arguments other than *timeout* directly through to that interface.

> [!NOTE]
> Do not use `stdout=PIPE` or `stderr=PIPE` with this function. The child process will block if it generates enough output to a pipe to fill up the OS pipe buffer as the pipes are not being read from.

<div class="versionchanged">

3.3 *timeout* was added.

</div>

<div class="versionchanged">

3.12

Changed Windows shell search order for `shell=True`. The current directory and `%PATH%` are replaced with `%COMSPEC%` and `%SystemRoot%\System32\cmd.exe`. As a result, dropping a malicious program named `cmd.exe` into a current directory no longer works.

</div>

</div>

<div class="function">

check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False, cwd=None, timeout=None,*\*other_popen_kwargs)

Run command with arguments. Wait for command to complete. If the return code was zero then return, otherwise raise `CalledProcessError`. The `CalledProcessError` object will have the return code in the `~CalledProcessError.returncode` attribute. If `check_call` was unable to start the process it will propagate the exception that was raised.

Code needing to capture stdout or stderr should use `run` instead:

    run(..., check=True)

To suppress stdout or stderr, supply a value of `DEVNULL`.

The arguments shown above are merely some common ones. The full function signature is the same as that of the `Popen` constructor - this function passes all supplied arguments other than *timeout* directly through to that interface.

> [!NOTE]
> Do not use `stdout=PIPE` or `stderr=PIPE` with this function. The child process will block if it generates enough output to a pipe to fill up the OS pipe buffer as the pipes are not being read from.

<div class="versionchanged">

3.3 *timeout* was added.

</div>

<div class="versionchanged">

3.12

Changed Windows shell search order for `shell=True`. The current directory and `%PATH%` are replaced with `%COMSPEC%` and `%SystemRoot%\System32\cmd.exe`. As a result, dropping a malicious program named `cmd.exe` into a current directory no longer works.

</div>

</div>

<div class="function">

check_output(args, *, stdin=None, stderr=None, shell=False, cwd=None, encoding=None, errors=None, universal_newlines=None, timeout=None, text=None,*\*other_popen_kwargs)

Run command with arguments and return its output.

If the return code was non-zero it raises a `CalledProcessError`. The `CalledProcessError` object will have the return code in the `~CalledProcessError.returncode` attribute and any output in the `~CalledProcessError.output` attribute.

This is equivalent to:

    run(..., check=True, stdout=PIPE).stdout

The arguments shown above are merely some common ones. The full function signature is largely the same as that of `run` - most arguments are passed directly through to that interface. One API deviation from `run` behavior exists: passing `input=None` will behave the same as `input=b''` (or `input=''`, depending on other arguments) rather than using the parent's standard input file handle.

By default, this function will return the data as encoded bytes. The actual encoding of the output data may depend on the command being invoked, so the decoding to text will often need to be handled at the application level.

This behaviour may be overridden by setting *text*, *encoding*, *errors*, or *universal_newlines* to `True` as described in `frequently-used-arguments` and `run`.

To also capture standard error in the result, use `stderr=subprocess.STDOUT`:

    >>> subprocess.check_output(
    ...     "ls non_existent_file; exit 0",
    ...     stderr=subprocess.STDOUT,
    ...     shell=True)
    'ls: non_existent_file: No such file or directory\n'

<div class="versionadded">

3.1

</div>

<div class="versionchanged">

3.3 *timeout* was added.

</div>

<div class="versionchanged">

3.4 Support for the *input* keyword argument was added.

</div>

<div class="versionchanged">

3.6 *encoding* and *errors* were added. See `run` for details.

</div>

<div class="versionadded">

3.7 *text* was added as a more readable alias for *universal_newlines*.

</div>

<div class="versionchanged">

3.12

Changed Windows shell search order for `shell=True`. The current directory and `%PATH%` are replaced with `%COMSPEC%` and `%SystemRoot%\System32\cmd.exe`. As a result, dropping a malicious program named `cmd.exe` into a current directory no longer works.

</div>

</div>

## Replacing Older Functions with the `!subprocess` Module

In this section, "a becomes b" means that b can be used as a replacement for a.

> [!NOTE]
> All "a" functions in this section fail (more or less) silently if the executed program cannot be found; the "b" replacements raise `OSError` instead.
>
> In addition, the replacements using `check_output` will fail with a `CalledProcessError` if the requested operation produces a non-zero return code. The output is still available as the `~CalledProcessError.output` attribute of the raised exception.

In the following examples, we assume that the relevant functions have already been imported from the `!subprocess` module.

### Replacing `/bin/sh` shell command substitution

``` bash
output=$(mycmd myarg)
```

becomes:

    output = check_output(["mycmd", "myarg"])

### Replacing shell pipeline

``` bash
output=$(dmesg | grep hda)
```

becomes:

    p1 = Popen(["dmesg"], stdout=PIPE)
    p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
    p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
    output = p2.communicate()[0]

The `p1.stdout.close()` call after starting the p2 is important in order for p1 to receive a SIGPIPE if p2 exits before p1.

Alternatively, for trusted input, the shell's own pipeline support may still be used directly:

``` bash
output=$(dmesg | grep hda)
```

becomes:

    output = check_output("dmesg | grep hda", shell=True)

### Replacing `os.system`

    sts = os.system("mycmd" + " myarg")
    # becomes
    retcode = call("mycmd" + " myarg", shell=True)

Notes:

- Calling the program through the shell is usually not required.
- The `call` return value is encoded differently to that of `os.system`.
- The `os.system` function ignores SIGINT and SIGQUIT signals while the command is running, but the caller must do this separately when using the `!subprocess` module.

A more realistic example would look like this:

    try:
        retcode = call("mycmd" + " myarg", shell=True)
        if retcode < 0:
            print("Child was terminated by signal", -retcode, file=sys.stderr)
        else:
            print("Child returned", retcode, file=sys.stderr)
    except OSError as e:
        print("Execution failed:", e, file=sys.stderr)

### Replacing the `os.spawn <os.spawnl>` family

P_NOWAIT example:

    pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
    ==>
    pid = Popen(["/bin/mycmd", "myarg"]).pid

P_WAIT example:

    retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
    ==>
    retcode = call(["/bin/mycmd", "myarg"])

Vector example:

    os.spawnvp(os.P_NOWAIT, path, args)
    ==>
    Popen([path] + args[1:])

Environment example:

    os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
    ==>
    Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})

### Replacing `os.popen`

Return code handling translates as follows:

    pipe = os.popen(cmd, 'w')
    ...
    rc = pipe.close()
    if rc is not None and rc >> 8:
        print("There were some errors")
    ==>
    process = Popen(cmd, stdin=PIPE)
    ...
    process.stdin.close()
    if process.wait() != 0:
        print("There were some errors")

## Legacy Shell Invocation Functions

This module also provides the following legacy functions from the 2.x `commands` module. These operations implicitly invoke the system shell and none of the guarantees described above regarding security and exception handling consistency are valid for these functions.

<div class="function">

getstatusoutput(cmd, \*, encoding=None, errors=None)

Return `(exitcode, output)` of executing *cmd* in a shell.

Execute the string *cmd* in a shell with `check_output` and return a 2-tuple `(exitcode, output)`. *encoding* and *errors* are used to decode output; see the notes on `frequently-used-arguments` for more details.

A trailing newline is stripped from the output. The exit code for the command can be interpreted as the return code of subprocess. Example:

    >>> subprocess.getstatusoutput('ls /bin/ls')
    (0, '/bin/ls')
    >>> subprocess.getstatusoutput('cat /bin/junk')
    (1, 'cat: /bin/junk: No such file or directory')
    >>> subprocess.getstatusoutput('/bin/junk')
    (127, 'sh: /bin/junk: not found')
    >>> subprocess.getstatusoutput('/bin/kill $$')
    (-15, '')

<div class="availability">

Unix, Windows.

</div>

<div class="versionchanged">

3.3.4 Windows support was added.

The function now returns (exitcode, output) instead of (status, output) as it did in Python 3.3.3 and earlier. exitcode has the same value as `~Popen.returncode`.

</div>

<div class="versionchanged">

3.11 Added the *encoding* and *errors* parameters.

</div>

</div>

<div class="function">

getoutput(cmd, \*, encoding=None, errors=None)

Return output (stdout and stderr) of executing *cmd* in a shell.

Like `getstatusoutput`, except the exit code is ignored and the return value is a string containing the command's output. Example:

    >>> subprocess.getoutput('ls /bin/ls')
    '/bin/ls'

<div class="availability">

Unix, Windows.

</div>

<div class="versionchanged">

3.3.4 Windows support added

</div>

<div class="versionchanged">

3.11 Added the *encoding* and *errors* parameters.

</div>

</div>

## Notes

### Timeout Behavior

When using the `timeout` parameter in functions like `run`, `Popen.wait`, or `Popen.communicate`, users should be aware of the following behaviors:

1.  **Process Creation Delay**: The initial process creation itself cannot be interrupted on many platform APIs. This means that even when specifying a timeout, you are not guaranteed to see a timeout exception until at least after however long process creation takes.
2.  **Extremely Small Timeout Values**: Setting very small timeout values (such as a few milliseconds) may result in almost immediate `TimeoutExpired` exceptions because process creation and system scheduling inherently require time.

### Converting an argument sequence to a string on Windows

On Windows, an *args* sequence is converted to a string that can be parsed using the following rules (which correspond to the rules used by the MS C runtime):

1.  Arguments are delimited by white space, which is either a space or a tab.
2.  A string surrounded by double quotation marks is interpreted as a single argument, regardless of white space contained within. A quoted string can be embedded in an argument.
3.  A double quotation mark preceded by a backslash is interpreted as a literal double quotation mark.
4.  Backslashes are interpreted literally, unless they immediately precede a double quotation mark.
5.  If backslashes immediately precede a double quotation mark, every pair of backslashes is interpreted as a literal backslash. If the number of backslashes is odd, the last backslash escapes the next double quotation mark as described in rule 3.

<div class="seealso">

`shlex`  
Module which provides function to parse and escape command lines.

</div>

### Disable use of `posix_spawn()`

On Linux, `!subprocess` defaults to using the `vfork()` system call internally when it is safe to do so rather than `fork()`. This greatly improves performance.

    subprocess._USE_POSIX_SPAWN = False  # See CPython issue gh-NNNNNN.

It is safe to set this to false on any Python version. It will have no effect on older or newer versions where unsupported. Do not assume the attribute is available to read. Despite the name, a true value does not indicate the corresponding function will be used, only that it may be.

Please file issues any time you have to use these private knobs with a way to reproduce the issue you were seeing. Link to that issue from a comment in your code.

<div class="versionadded">

3.8 `_USE_POSIX_SPAWN`

</div>

# `!sunau` --- Read and write Sun AU files

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

sunau

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!sunau` module was [Python 3.12](https://docs.python.org/3.12/library/sunau.html).

# Superseded modules

The modules described in this chapter have been superseded by other modules for most use cases, and are retained primarily to preserve backwards compatibility.

Modules may appear in this chapter because they only cover a limited subset of a problem space, and a more generally applicable solution is available elsewhere in the standard library (for example, `getopt` covers the very specific task of "mimic the C `!getopt` API in Python", rather than the broader command line option parsing and argument parsing capabilities offered by `optparse` and `argparse`).

Alternatively, modules may appear in this chapter because they are deprecated outright, and awaiting removal in a future release, or they are `soft deprecated` and their use is actively discouraged in new projects. With the removal of various obsolete modules through `594`, there are currently no modules in this latter category.

<div class="toctree" data-maxdepth="1">

getopt.rst profile.rst

</div>

# `!symtable` --- Access to the compiler's symbol tables

<div class="module" data-synopsis="Interface to the compiler's internal symbol tables.">

symtable

</div>

**Source code:** `Lib/symtable.py`

------------------------------------------------------------------------

Symbol tables are generated by the compiler from AST just before bytecode is generated. The symbol table is responsible for calculating the scope of every identifier in the code. `!symtable` provides an interface to examine these tables.

## Generating Symbol Tables

<div class="function">

symtable(code, filename, compile_type, \*, module=None)

Return the toplevel `SymbolTable` for the Python source *code*. *code* can be a string, a bytes object, or an AST object, as for the builtin `compile`. *filename* is the name of the file containing the code. *compile_type* is like the *mode* argument to `compile`. The optional argument *module* specifies the module name. It is needed to unambiguous `filter <warning-filter>` syntax warnings by module name.

<div class="versionadded">

3.15 Added the *module* parameter.

</div>

<div class="versionchanged">

3.16 *code* can now be an AST object.

</div>

</div>

## Examining Symbol Tables

<div class="SymbolTableType">

An enumeration indicating the type of a `SymbolTable` object.

<div class="attribute" value="&quot;module&quot;">

MODULE

Used for the symbol table of a module.

</div>

<div class="attribute" value="&quot;function&quot;">

FUNCTION

Used for the symbol table of a function.

</div>

<div class="attribute" value="&quot;class&quot;">

CLASS

Used for the symbol table of a class.

</div>

<div class="attribute" value="&quot;inlined comprehension&quot;">

INLINED_COMPREHENSION

Used for the symbol table of a list, set or dict comprehension that is inlined into the enclosing code unit (see `709`). A symbol table of this type represents a sub-scope of the enclosing code unit's scope, and it does not correspond to a separate compilation unit.

<div class="versionadded">

3.16

</div>

</div>

The following members refer to different flavors of `annotation scopes <annotation-scopes>`.

<div class="attribute" value="&quot;annotation&quot;">

ANNOTATION

Used for annotations if `from __future__ import annotations` is active.

</div>

<div class="attribute" value="&quot;type alias&quot;">

TYPE_ALIAS

Used for the symbol table of `type` constructions.

</div>

<div class="attribute" value="&quot;type parameters&quot;">

TYPE_PARAMETERS

Used for the symbol table of `generic functions <generic-functions>` or `generic classes <generic-classes>`.

</div>

<div class="attribute" value="&quot;type variable&quot;">

TYPE_VARIABLE

Used for the symbol table of the bound, the constraint tuple or the default value of a single type variable in the formal sense, i.e., a TypeVar, a TypeVarTuple or a ParamSpec object (the latter two do not support a bound or a constraint tuple).

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="SymbolTable">

A namespace table for a block. The constructor is not public.

<div class="method">

get_type()

Return the type of the symbol table. Possible values are members of the `SymbolTableType` enumeration.

<div class="versionchanged">

3.12 Added `'annotation'`, `'TypeVar bound'`, `'type alias'`, and `'type parameter'` as possible return values.

</div>

<div class="versionchanged">

3.13 Return values are members of the `SymbolTableType` enumeration.

The exact values of the returned string may change in the future, and thus, it is recommended to use `SymbolTableType` members instead of hard-coded strings.

</div>

</div>

<div class="method">

get_id()

Return the table's identifier.

</div>

<div class="method">

get_name()

Return the table's name. This is the name of the class if the table is for a class, the name of the function if the table is for a function, or `'top'` if the table is global (`get_type` returns `'module'`). For type parameter scopes (which are used for generic classes, functions, and type aliases), it is the name of the underlying class, function, or type alias. For type alias scopes, it is the name of the type alias. For `~typing.TypeVar` bound scopes, it is the name of the `TypeVar`.

</div>

<div class="method">

get_lineno()

Return the number of the first line in the block this table represents.

</div>

<div class="method">

is_optimized()

Return `True` if the locals in this table can be optimized.

</div>

<div class="method">

is_nested()

Return `True` if the block is a nested class or function.

</div>

<div class="method">

has_children()

Return `True` if the block has nested namespaces within it. These can be obtained with `get_children`.

</div>

<div class="method">

get_identifiers()

Return a view object containing the names of symbols in the table. See the `documentation of view objects <dict-views>`.

</div>

<div class="method">

lookup(name)

Lookup *name* in the table and return a `Symbol` instance.

</div>

<div class="method">

get_symbols()

Return a list of `Symbol` instances for names in the table.

</div>

<div class="method">

get_children()

Return a list of the nested symbol tables.

</div>

</div>

<div class="Function">

A namespace for a function or method. This class inherits from `SymbolTable`.

<div class="method">

get_parameters()

Return a tuple containing names of parameters to this function.

</div>

<div class="method">

get_locals()

Return a tuple containing names of locals in this function.

</div>

<div class="method">

get_globals()

Return a tuple containing names of globals in this function.

</div>

<div class="method">

get_nonlocals()

Return a tuple containing names of explicitly declared nonlocals in this function.

</div>

<div class="method">

get_frees()

Return a tuple containing names of `free (closure) variables <closure variable>` in this function.

</div>

<div class="method">

get_cells()

Return a tuple containing names of `cell variables <closure variable>` in this table.

<div class="versionadded">

3.15

</div>

</div>

</div>

<div class="Class">

A namespace of a class. This class inherits from `SymbolTable`.

</div>

<div class="Symbol">

An entry in a `SymbolTable` corresponding to an identifier in the source. The constructor is not public.

<div class="method">

get_name()

Return the symbol's name.

</div>

<div class="method">

is_referenced()

Return `True` if the symbol is used in its block.

</div>

<div class="method">

is_imported()

Return `True` if the symbol is created from an import statement.

</div>

<div class="method">

is_parameter()

Return `True` if the symbol is a parameter.

</div>

<div class="method">

is_type_parameter()

Return `True` if the symbol is a type parameter.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

is_global()

Return `True` if the symbol is global.

</div>

<div class="method">

is_nonlocal()

Return `True` if the symbol is nonlocal.

</div>

<div class="method">

is_declared_global()

Return `True` if the symbol is declared global with a global statement.

</div>

<div class="method">

is_local()

Return `True` if the symbol is local to its block.

</div>

<div class="method">

is_annotated()

Return `True` if the symbol is annotated.

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

is_free()

Return `True` if the symbol is referenced in its block, but not assigned to.

</div>

<div class="method">

is_cell()

Return `True` if the symbol is referenced but not assigned in a nested block.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

is_free_class()

Return *True* if a class-scoped symbol is free from the perspective of a method.

Consider the following example:

    def f():
        x = 1  # function-scoped
        class C:
            x = 2  # class-scoped
            def method(self):
                return x

In this example, the class-scoped symbol `x` is considered to be free from the perspective of `C.method`, thereby allowing the latter to return *1* at runtime and not *2*.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

is_assigned()

Return `True` if the symbol is assigned to in its block.

</div>

<div class="method">

is_comp_iter()

Return `True` if the symbol is a comprehension iteration variable.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

is_comp_cell()

Return `True` if the symbol is a cell in an inlined comprehension.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

is_namespace()

Return `True` if name binding introduces new namespace.

If the name is used as the target of a function or class statement, this will be true.

For example:

    >>> table = symtable.symtable("def some_func(): pass", "string", "exec")
    >>> table.lookup("some_func").is_namespace()
    True

Note that a single name can be bound to multiple objects. If the result is `True`, the name may also be bound to other objects, like an int or list, that does not introduce a new namespace.

</div>

<div class="method">

get_namespaces()

Return a list of namespaces bound to this name.

</div>

<div class="method">

get_namespace()

Return the namespace bound to this name. If more than one or no namespace is bound to this name, a `ValueError` is raised.

</div>

</div>

## Command-Line Usage

<div class="versionadded">

3.13

</div>

The `!symtable` module can be executed as a script from the command line.

``` sh
python -m symtable [infile...]
```

Symbol tables are generated for the specified Python source files and dumped to stdout. If no input file is specified, the content is read from stdin.

# `!sys` --- System-specific parameters and functions

<div class="module" data-synopsis="Access system-specific parameters and functions.">

sys

</div>

------------------------------------------------------------------------

This module provides access to some variables used or maintained by the interpreter and to functions that interact strongly with the interpreter. It is always available. Unless explicitly noted otherwise, all variables are read-only.

<div class="data">

abi_info

<div class="versionadded">

3.15

</div>

An object containing information about the ABI of the currently running Python interpreter. It should include information that affect the CPython ABI in ways that require a specific build of the interpreter chosen from variants that can co-exist on a single machine. For example, it does not encode the base OS (Linux or Windows), but does include pointer size since some systems support both 32- and 64-bit builds. The available entries are the same on all platforms; e.g. *pointer_size* is available even on 64-bit-only architectures.

The following attributes are available:

<div class="attribute">

abi_info.pointer_bits

The width of pointers in bits, as an integer, equivalent to `8 * sizeof(void *)`. Usually, this is `32` or `64`.

</div>

<div class="attribute">

abi_info.free_threaded

A Boolean indicating whether the interpreter was built with `free threading` support. This reflects either the presence of the `--disable-gil` `configure` option (on Unix) or setting the `DisableGil` property (on Windows).

</div>

<div class="attribute">

abi_info.debug

A Boolean indicating whether the interpreter was built in `debug mode <debug-build>`. This reflects either the presence of the `--with-pydebug` `configure` option (on Unix) or the `Debug` configuration (on Windows).

</div>

<div class="attribute">

abi_info.byteorder

A string indicating the native byte order, either `'big'` or `'little'`. This is the same as the `byteorder` attribute.

</div>

</div>

<div class="data">

abiflags

On POSIX systems where Python was built with the standard `configure` script, this contains the ABI flags as specified by `3149`.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.8 Default flags became an empty string (`m` flag for pymalloc has been removed).

</div>

<div class="availability">

Unix.

</div>

</div>

<div class="function">

addaudithook(hook)

Append the callable *hook* to the list of active auditing hooks for the current (sub)interpreter.

When an auditing event is raised through the `sys.audit` function, each hook will be called in the order it was added with the event name and the tuple of arguments. Native hooks added by `PySys_AddAuditHook` are called first, followed by hooks added in the current (sub)interpreter. Hooks can then log the event, raise an exception to abort the operation, or terminate the process entirely.

Note that audit hooks are primarily for collecting information about internal or otherwise unobservable actions, whether by Python or libraries written in Python. They are not suitable for implementing a "sandbox". In particular, malicious code can trivially disable or bypass hooks added using this function. At a minimum, any security-sensitive hooks must be added using the C API `PySys_AddAuditHook` before initialising the runtime, and any modules allowing arbitrary memory modification (such as `ctypes`) should be completely removed or closely monitored.

<div class="audit-event">

sys.addaudithook "" sys.addaudithook

Calling `sys.addaudithook` will itself raise an auditing event named `sys.addaudithook` with no arguments. If any existing hooks raise an exception derived from `RuntimeError`, the new hook will not be added and the exception suppressed. As a result, callers cannot assume that their hook has been added unless they control all existing hooks.

</div>

See the `audit events table <audit-events>` for all events raised by CPython, and `578` for the original design discussion.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.8.1

Exceptions derived from `Exception` but not `RuntimeError` are no longer suppressed.

</div>

<div class="impl-detail">

When tracing is enabled (see `settrace`), Python hooks are only traced if the callable has a `__cantrace__` member that is set to a true value. Otherwise, trace functions will skip the hook.

</div>

</div>

<div class="data">

argv

The list of command line arguments passed to a Python script. `argv[0]` is the script name (it is operating system dependent whether this is a full pathname or not). If the command was executed using the `-c` command line option to the interpreter, `argv[0]` is set to the string `'-c'`. If no script name was passed to the Python interpreter, `argv[0]` is the empty string.

To loop over the standard input, or the list of files given on the command line, see the `fileinput` module.

See also `sys.orig_argv`.

> [!NOTE]
> On Unix, command line arguments are passed by bytes from OS. Python decodes them with filesystem encoding and "surrogateescape" error handler. When you need original bytes, you can get it by `[os.fsencode(arg) for arg in sys.argv]`.

</div>

<div id="auditing">

<div class="function">

audit(event, \*args)

<div class="index">

single: auditing

</div>

Raise an auditing event and trigger any active auditing hooks. *event* is a string identifying the event, and *args* may contain optional arguments with more information about the event. The number and types of arguments for a given event are considered a public and stable API and should not be modified between releases.

For example, one auditing event is named `os.chdir`. This event has one argument called *path* that will contain the requested new working directory.

`sys.audit` will call the existing auditing hooks, passing the event name and arguments, and will re-raise the first exception from any hook. In general, if an exception is raised, it should not be handled and the process should be terminated as quickly as possible. This allows hook implementations to decide how to respond to particular events: they can merely log the event or abort the operation by raising an exception.

Hooks are added using the `sys.addaudithook` or `PySys_AddAuditHook` functions.

The native equivalent of this function is `PySys_Audit`. Using the native function is preferred when possible.

See the `audit events table <audit-events>` for all events raised by CPython.

<div class="versionadded">

3.8

</div>

</div>

</div>

<div class="data">

base_exec_prefix

Equivalent to `exec_prefix`, but referring to the base Python installation.

When running under `sys-path-init-virtual-environments`, `exec_prefix` gets overwritten to the virtual environment prefix. `base_exec_prefix`, conversely, does not change, and always points to the base Python installation. Refer to `sys-path-init-virtual-environments` for more information.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

base_prefix

Equivalent to `prefix`, but referring to the base Python installation.

When running under `virtual environment <venv-def>`, `prefix` gets overwritten to the virtual environment prefix. `base_prefix`, conversely, does not change, and always points to the base Python installation. Refer to `sys-path-init-virtual-environments` for more information.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

byteorder

An indicator of the native byte order. This will have the value `'big'` on big-endian (most-significant byte first) platforms, and `'little'` on little-endian (least-significant byte first) platforms.

</div>

<div class="data">

builtin_module_names

A tuple of strings containing the names of all modules that are compiled into this Python interpreter. (This information is not available in any other way --- `modules.keys()` only lists the imported modules.)

See also the `sys.stdlib_module_names` list.

</div>

<div class="function">

call_tracing(func, args)

Call `func(*args)`, while tracing is enabled. The tracing state is saved, and restored afterwards. This is intended to be called from a debugger from a checkpoint, to recursively debug or profile some other code.

Tracing is suspended while calling a tracing function set by `settrace` or `setprofile` to avoid infinite recursion. `!call_tracing` enables explicit recursion of the tracing function.

</div>

<div class="data">

copyright

A string containing the copyright pertaining to the Python interpreter.

</div>

<div class="function">

<span id="clear_type_cache">clear_type_cache</span>()

This function is a no-op. It used to clear the internal type cache, which is now implemented per-type.

<div class="deprecated">

3.13 Use the more general `_clear_internal_caches` function instead.

</div>

<div class="versionchanged">

3.16 This function is now a no-op.

</div>

</div>

<div class="function">

<span id="clear_internal_caches">clear_internal_caches</span>()

Clear all internal performance-related caches. Use this function *only* to release unnecessary references and memory blocks when hunting for leaks.

<div class="versionadded">

3.13

</div>

<div class="versionchanged">

3.16 The type cache is no longer cleared, as it is now implemented per-type.

</div>

</div>

<div class="function">

<span id="current_frames">current_frames</span>()

Return a dictionary mapping each thread's identifier to the topmost stack frame currently active in that thread at the time the function is called. Note that functions in the `traceback` module can build the call stack given such a frame.

This is most useful for debugging deadlock: this function does not require the deadlocked threads' cooperation, and such threads' call stacks are frozen for as long as they remain deadlocked. The frame returned for a non-deadlocked thread may bear no relationship to that thread's current activity by the time calling code examines the frame.

This function should be used for internal and specialized purposes only.

<div class="audit-event">

sys.\_current_frames "" sys.\_current_frames

</div>

</div>

<div class="function">

<span id="current_exceptions">current_exceptions</span>()

Return a dictionary mapping each thread's identifier to the topmost exception currently active in that thread at the time the function is called. If a thread is not currently handling an exception, it is not included in the result dictionary.

This is most useful for statistical profiling.

This function should be used for internal and specialized purposes only.

<div class="audit-event">

sys.\_current_exceptions "" sys.\_current_exceptions

</div>

<div class="versionchanged">

3.12 Each value in the dictionary is now a single exception instance, rather than a 3-tuple as returned from `sys.exc_info()`.

</div>

</div>

<div class="function">

breakpointhook()

This hook function is called by built-in `breakpoint`. By default, it drops you into the `pdb` debugger, but it can be set to any other function so that you can choose which debugger gets used.

The signature of this function is dependent on what it calls. For example, the default binding (e.g. `pdb.set_trace()`) expects no arguments, but you might bind it to a function that expects additional arguments (positional and/or keyword). The built-in `breakpoint()` function passes its `*args` and `**kws` straight through. Whatever `breakpointhooks()` returns is returned from `breakpoint()`.

The default implementation first consults the environment variable `PYTHONBREAKPOINT`. If that is set to `"0"` then this function returns immediately; i.e. it is a no-op. If the environment variable is not set, or is set to the empty string, `pdb.set_trace()` is called. Otherwise this variable should name a function to run, using Python's dotted-import nomenclature, e.g. `package.subpackage.module.function`. In this case, `package.subpackage.module` would be imported and the resulting module must have a callable named `function()`. This is run, passing in `*args` and `**kws`, and whatever `function()` returns, `sys.breakpointhook()` returns to the built-in `breakpoint` function.

Note that if anything goes wrong while importing the callable named by `PYTHONBREAKPOINT`, a `RuntimeWarning` is reported and the breakpoint is ignored.

Also note that if `sys.breakpointhook()` is overridden programmatically, `PYTHONBREAKPOINT` is *not* consulted.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

<span id="debugmallocstats">debugmallocstats</span>()

Print low-level information to stderr about the state of CPython's memory allocator.

If Python is `built in debug mode <debug-build>` (`configure
--with-pydebug option <--with-pydebug>`), it also performs some expensive internal consistency checks.

<div class="versionadded">

3.3

</div>

<div class="impl-detail">

This function is specific to CPython. The exact output format is not defined here, and may change.

</div>

</div>

<div class="data">

dllhandle

Integer specifying the handle of the Python DLL.

<div class="availability">

Windows.

</div>

</div>

<div class="function">

displayhook(value)

If *value* is not `None`, this function prints `repr(value)` to `sys.stdout`, and saves *value* in `builtins._`. If `repr(value)` is not encodable to `sys.stdout.encoding` with `sys.stdout.errors` error handler (which is probably `'strict'`), encode it to `sys.stdout.encoding` with `'backslashreplace'` error handler.

`sys.displayhook` is called on the result of evaluating an `expression` entered in an interactive Python session. The display of these values can be customized by assigning another one-argument function to `sys.displayhook`.

Pseudo-code:

    def displayhook(value):
        if value is None:
            return
        # Set '_' to None to avoid recursion
        builtins._ = None
        text = repr(value)
        try:
            sys.stdout.write(text)
        except UnicodeEncodeError:
            bytes = text.encode(sys.stdout.encoding, 'backslashreplace')
            if hasattr(sys.stdout, 'buffer'):
                sys.stdout.buffer.write(bytes)
            else:
                text = bytes.decode(sys.stdout.encoding, 'strict')
                sys.stdout.write(text)
        sys.stdout.write("\n")
        builtins._ = value

<div class="versionchanged">

3.2 Use `'backslashreplace'` error handler on `UnicodeEncodeError`.

</div>

</div>

<div class="data">

dont_write_bytecode

If this is true, Python won't try to write `.pyc` files on the import of source modules. This value is initially set to `True` or `False` depending on the `-B` command line option and the `PYTHONDONTWRITEBYTECODE` environment variable, but you can set it yourself to control bytecode file generation.

</div>

<div class="data">

<span id="emscripten_info">emscripten_info</span>

A `named tuple` holding information about the environment on the *wasm32-emscripten* platform. The named tuple is provisional and may change in the future.

<div class="attribute">

<span id="emscripten_info.emscripten_version">emscripten_info.emscripten_version</span>

Emscripten version as tuple of ints (major, minor, micro), e.g. `(3, 1, 8)`.

</div>

<div class="attribute">

<span id="emscripten_info.runtime">emscripten_info.runtime</span>

Runtime string, e.g. browser user agent, `'Node.js v14.18.2'`, or `'UNKNOWN'`.

</div>

<div class="attribute">

<span id="emscripten_info.pthreads">emscripten_info.pthreads</span>

`True` if Python is compiled with Emscripten pthreads support.

</div>

<div class="attribute">

<span id="emscripten_info.shared_memory">emscripten_info.shared_memory</span>

`True` if Python is compiled with shared memory support.

</div>

<div class="availability">

Emscripten.

</div>

<div class="versionadded">

3.11

</div>

</div>

<div class="data">

<span id="wasi_info">wasi_info</span>

An object containing information about the build on the WebAssembly WASI platform. The object is provisional and may change in the future.

<div class="attribute">

<span id="wasi_info.wasi_version">wasi_info.wasi_version</span>

The WASI target version as a string: `'p1'`, `'p2'`, or `'p3'`.

</div>

<div class="attribute">

<span id="wasi_info.wasi_sdk_version">wasi_info.wasi_sdk_version</span>

The full WASI SDK version as a string, such as `'34.0'`.

</div>

<div class="attribute">

<span id="wasi_info.cooperative_threads">wasi_info.cooperative_threads</span>

`True` if wasi-libc was compiled with cooperative multithreading support, `False` otherwise.

</div>

<div class="availability">

WASI.

</div>

<div class="versionadded">

next

</div>

</div>

<div class="data">

pycache_prefix

If this is set (not `None`), Python will write bytecode-cache `.pyc` files to (and read them from) a parallel directory tree rooted at this directory, rather than from `__pycache__` directories in the source code tree. Any `__pycache__` directories in the source code tree will be ignored and new `.pyc` files written within the pycache prefix. Thus if you use `compileall` as a pre-build step, you must ensure you run it with the same pycache prefix (if any) that you will use at runtime.

A relative path is interpreted relative to the current working directory.

This value is initially set based on the value of the `-X` `pycache_prefix=PATH` command-line option or the `PYTHONPYCACHEPREFIX` environment variable (command-line takes precedence). If neither are set, it is `None`.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

excepthook(type, value, traceback)

This function prints out a given traceback and exception to `sys.stderr`.

When an exception other than `SystemExit` is raised and uncaught, the interpreter calls `sys.excepthook` with three arguments, the exception class, exception instance, and a traceback object. In an interactive session this happens just before control is returned to the prompt; in a Python program this happens just before the program exits. The handling of such top-level exceptions can be customized by assigning another three-argument function to `sys.excepthook`.

<div class="audit-event">

sys.excepthook hook,type,value,traceback sys.excepthook

Raise an auditing event `sys.excepthook` with arguments `hook`, `type`, `value`, `traceback` when an uncaught exception occurs. If no hook has been set, `hook` may be `None`. If any hook raises an exception derived from `RuntimeError` the call to the hook will be suppressed. Otherwise, the audit hook exception will be reported as unraisable and `sys.excepthook` will be called.

</div>

<div class="seealso">

The `sys.unraisablehook` function handles unraisable exceptions and the `threading.excepthook` function handles exception raised by `threading.Thread.run`.

</div>

</div>

<div class="data">

\_\_breakpointhook\_\_ \_\_displayhook\_\_ \_\_excepthook\_\_ \_\_unraisablehook\_\_

These objects contain the original values of `breakpointhook`, `displayhook`, `excepthook`, and `unraisablehook` at the start of the program. They are saved so that `breakpointhook`, `displayhook` and `excepthook`, `unraisablehook` can be restored in case they happen to get replaced with broken or alternative objects.

<div class="versionadded">

3.7 \_\_breakpointhook\_\_

</div>

<div class="versionadded">

3.8 \_\_unraisablehook\_\_

</div>

</div>

<div class="function">

exception()

This function, when called while an exception handler is executing (such as an `except` or `except*` clause), returns the exception instance that was caught by this handler. When exception handlers are nested within one another, only the exception handled by the innermost handler is accessible.

If no exception handler is executing, this function returns `None`.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

exc_info()

This function returns the old-style representation of the handled exception. If an exception `e` is currently handled (so `exception` would return `e`), `exc_info` returns the tuple `(type(e), e, e.__traceback__)`. That is, a tuple containing the type of the exception (a subclass of `BaseException`), the exception itself, and a `traceback
object <traceback-objects>` which typically encapsulates the call stack at the point where the exception last occurred.

<div class="index">

pair: object; traceback

</div>

If no exception is being handled anywhere on the stack, this function return a tuple containing three `None` values.

<div class="versionchanged">

3.11 The `type` and `traceback` fields are now derived from the `value` (the exception instance), so when an exception is modified while it is being handled, the changes are reflected in the results of subsequent calls to `exc_info`.

</div>

</div>

<div class="data">

exec_prefix

A string giving the site-specific directory prefix where the platform-dependent Python files are installed; by default, this is also `'/usr/local'`. This can be set at build time with the `--exec-prefix` argument to the `configure` script. Specifically, all configuration files (e.g. the `pyconfig.h` header file) are installed in the directory `{exec_prefix}/lib/python{X.Y}/config`, and shared library modules are installed in `{exec_prefix}/lib/python{X.Y}/lib-dynload`, where *X.Y* is the version number of Python, for example `3.2`.

> [!NOTE]
> If a `virtual environment <venv-def>` is in effect, this `exec_prefix` will point to the virtual environment. The value for the Python installation will still be available, via `base_exec_prefix`. Refer to `sys-path-init-virtual-environments` for more information.

<div class="versionchanged">

3.14

When running under a `virtual environment <venv-def>`, `prefix` and `exec_prefix` are now set to the virtual environment prefix by the `path initialization <sys-path-init>`, instead of `site`. This means that `prefix` and `exec_prefix` always point to the virtual environment, even when `site` is disabled (`-S`).

</div>

</div>

<div class="data">

executable

A string giving the absolute path of the executable binary for the Python interpreter, on systems where this makes sense. If Python is unable to retrieve the real path to its executable, `sys.executable` will be an empty string or `None`.

</div>

<div class="function">

exit(\[arg\])

Raise a `SystemExit` exception, signaling an intention to exit the interpreter.

The optional argument *arg* can be an integer giving the exit status (defaulting to zero), or another type of object. If it is an integer, zero is considered "successful termination" and any nonzero value is considered "abnormal termination" by shells and the like. Most systems require it to be in the range 0--127, and produce undefined results otherwise. Some systems have a convention for assigning specific meanings to specific exit codes, but these are generally underdeveloped; Unix programs generally use 2 for command line syntax errors and 1 for all other kinds of errors. If another type of object is passed, `None` is equivalent to passing zero, and any other object is printed to `stderr` and results in an exit code of 1. In particular, `sys.exit("some error message")` is a quick way to exit a program when an error occurs.

Since `~sys.exit` ultimately "only" raises an exception, it will only exit the process when called from the main thread, and the exception is not intercepted. Cleanup actions specified by `finally` clauses of `try` statements are honored, and it is possible to intercept the exit attempt at an outer level.

<div class="versionchanged">

3.6 If an error occurs in the cleanup after the Python interpreter has caught `SystemExit` (such as an error flushing buffered data in the standard streams), the exit status is changed to 120.

</div>

</div>

<div class="data">

flags

The `named tuple` *flags* exposes the status of command line flags. Flags should only be accessed only by name and not by index. The attributes are read only.

<table>
<tbody>
<tr>
<td><div class="attribute">
<p>flags.debug</p>
</div></td>
<td><code class="interpreted-text" role="option">-d</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.inspect</p>
</div></td>
<td><code class="interpreted-text" role="option">-i</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.interactive</p>
</div></td>
<td><code class="interpreted-text" role="option">-i</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.isolated</p>
</div></td>
<td><code class="interpreted-text" role="option">-I</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.optimize</p>
</div></td>
<td><code class="interpreted-text" role="option">-O</code> or <code class="interpreted-text" role="option">-OO</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.dont_write_bytecode</p>
</div></td>
<td><code class="interpreted-text" role="option">-B</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.no_user_site</p>
</div></td>
<td><code class="interpreted-text" role="option">-s</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.no_site</p>
</div></td>
<td><code class="interpreted-text" role="option">-S</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.ignore_environment</p>
</div></td>
<td><code class="interpreted-text" role="option">-E</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.verbose</p>
</div></td>
<td><code class="interpreted-text" role="option">-v</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.bytes_warning</p>
</div></td>
<td><code class="interpreted-text" role="option">-b</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.quiet</p>
</div></td>
<td><code class="interpreted-text" role="option">-q</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.hash_randomization</p>
</div></td>
<td><code class="interpreted-text" role="option">-R</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.dev_mode</p>
</div></td>
<td><code class="interpreted-text" role="option">-X dev &lt;-X&gt;</code> (<code class="interpreted-text" role="ref">Python Development Mode &lt;devmode&gt;</code>)</td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.utf8_mode</p>
</div></td>
<td><code class="interpreted-text" role="option">-X utf8 &lt;-X&gt;</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.safe_path</p>
</div></td>
<td><code class="interpreted-text" role="option">-P</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.int_max_str_digits</p>
</div></td>
<td><code class="interpreted-text" role="option">-X int_max_str_digits &lt;-X&gt;</code> (<code class="interpreted-text" role="ref">integer string conversion length limitation &lt;int_max_str_digits&gt;</code>)</td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.warn_default_encoding</p>
</div></td>
<td><code class="interpreted-text" role="option">-X warn_default_encoding &lt;-X&gt;</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.gil</p>
</div></td>
<td><code class="interpreted-text" role="option">-X gil &lt;-X&gt;</code> and <code class="interpreted-text" role="envvar">PYTHON_GIL</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.thread_inherit_context</p>
</div></td>
<td><code class="interpreted-text" role="option">-X thread_inherit_context &lt;-X&gt;</code> and <code class="interpreted-text" role="envvar">PYTHON_THREAD_INHERIT_CONTEXT</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.context_aware_warnings</p>
</div></td>
<td><code class="interpreted-text" role="option">-X context_aware_warnings &lt;-X&gt;</code> and <code class="interpreted-text" role="envvar">PYTHON_CONTEXT_AWARE_WARNINGS</code></td>
</tr>
<tr>
<td><div class="attribute">
<p>flags.lazy_imports</p>
</div></td>
<td><code class="interpreted-text" role="option">-X lazy_imports &lt;-X&gt;</code> and <code class="interpreted-text" role="envvar">PYTHON_LAZY_IMPORTS</code> (<code>1</code> for <code>all</code>, <code>-1</code> otherwise; not changed by <code class="interpreted-text" role="func">set_lazy_imports</code>)</td>
</tr>
</tbody>
</table>

<div class="versionchanged">

3.2 Added `quiet` attribute for the new `-q` flag.

</div>

<div class="versionadded">

3.2.3 The `hash_randomization` attribute.

</div>

<div class="versionchanged">

3.3 Removed obsolete `division_warning` attribute.

</div>

<div class="versionchanged">

3.4 Added `isolated` attribute for `-I` `isolated` flag.

</div>

<div class="versionchanged">

3.7 Added the `dev_mode` attribute for the new `Python Development
Mode <devmode>` and the `utf8_mode` attribute for the new `-X` `utf8` flag.

</div>

<div class="versionchanged">

3.10 Added `warn_default_encoding` attribute for `-X` `warn_default_encoding` flag.

</div>

<div class="versionchanged">

3.11 Added the `safe_path` attribute for `-P` option.

</div>

<div class="versionchanged">

3.11 Added the `int_max_str_digits` attribute.

</div>

<div class="versionchanged">

3.13 Added the `gil` attribute.

</div>

<div class="versionchanged">

3.14 Added the `thread_inherit_context` attribute.

</div>

<div class="versionchanged">

3.14 Added the `context_aware_warnings` attribute.

</div>

<div class="versionchanged">

3.15 Added the `lazy_imports` attribute.

</div>

</div>

<div class="data">

float_info

A `named tuple` holding information about the float type. It contains low level information about the precision and internal representation. The values correspond to the various floating-point constants defined in the standard header file `float.h` for the 'C' programming language; see section 5.2.4.2.2 of the 1999 ISO/IEC C standard <a href="#C99" class="citation">[C99]</a>, 'Characteristics of floating types', for details.

<table>
<caption>Attributes of the <code class="interpreted-text" role="data">!float_info</code> <code class="interpreted-text" role="term">named tuple</code></caption>
<thead>
<tr>
<th>attribute</th>
<th>float.h macro</th>
<th>explanation</th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="attribute">
<p>float_info.epsilon</p>
</div></td>
<td><p><code class="interpreted-text" role="c:macro">!DBL_EPSILON</code></p></td>
<td><p>difference between 1.0 and the least value greater than 1.0 that is representable as a float.</p>
<p>See also <code class="interpreted-text" role="func">math.ulp</code>.</p></td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.dig</p>
</div></td>
<td><code class="interpreted-text" role="c:macro">!DBL_DIG</code></td>
<td>The maximum number of decimal digits that can be faithfully represented in a float; see below.</td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.mant_dig</p>
</div></td>
<td><code class="interpreted-text" role="c:macro">!DBL_MANT_DIG</code></td>
<td>Float precision: the number of base-<code>radix</code> digits in the significand of a float.</td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.max</p>
</div></td>
<td><code class="interpreted-text" role="c:macro">!DBL_MAX</code></td>
<td>The maximum representable positive finite float.</td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.max_exp</p>
</div></td>
<td><code class="interpreted-text" role="c:macro">!DBL_MAX_EXP</code></td>
<td>The maximum integer <em>e</em> such that <code>radix**(e-1)</code> is a representable finite float.</td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.max_10_exp</p>
</div></td>
<td><code class="interpreted-text" role="c:macro">!DBL_MAX_10_EXP</code></td>
<td>The maximum integer <em>e</em> such that <code>10**e</code> is in the range of representable finite floats.</td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.min</p>
</div></td>
<td><p><code class="interpreted-text" role="c:macro">!DBL_MIN</code></p></td>
<td><p>The minimum representable positive <em>normalized</em> float.</p>
<p>Use <code class="interpreted-text" role="func">math.ulp(0.0) &lt;math.ulp&gt;</code> to get the smallest positive <em>denormalized</em> representable float.</p></td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.min_exp</p>
</div></td>
<td><code class="interpreted-text" role="c:macro">!DBL_MIN_EXP</code></td>
<td>The minimum integer <em>e</em> such that <code>radix**(e-1)</code> is a normalized float.</td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.min_10_exp</p>
</div></td>
<td><code class="interpreted-text" role="c:macro">!DBL_MIN_10_EXP</code></td>
<td>The minimum integer <em>e</em> such that <code>10**e</code> is a normalized float.</td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.radix</p>
</div></td>
<td><code class="interpreted-text" role="c:macro">!FLT_RADIX</code></td>
<td>The radix of exponent representation.</td>
</tr>
<tr>
<td><div class="attribute">
<p>float_info.rounds</p>
</div></td>
<td><p><code class="interpreted-text" role="c:macro">!FLT_ROUNDS</code></p></td>
<td><p>An integer representing the rounding mode for floating-point arithmetic. This reflects the value of the system <code class="interpreted-text" role="c:macro">!FLT_ROUNDS</code> macro at interpreter startup time:</p>
<ul>
<li><code>-1</code>: indeterminable</li>
<li><code>0</code>: toward zero</li>
<li><code>1</code>: to nearest</li>
<li><code>2</code>: toward positive infinity</li>
<li><code>3</code>: toward negative infinity</li>
</ul>
<p>All other values for <code class="interpreted-text" role="c:macro">!FLT_ROUNDS</code> characterize implementation-defined rounding behavior.</p></td>
</tr>
</tbody>
</table>

The attribute `sys.float_info.dig` needs further explanation. If `s` is any string representing a decimal number with at most `!sys.float_info.dig` significant digits, then converting `s` to a float and back again will recover a string representing the same decimal value:

    >>> import sys
    >>> sys.float_info.dig
    15
    >>> s = '3.14159265358979'    # decimal string with 15 significant digits
    >>> format(float(s), '.15g')  # convert to float and back -> same value
    '3.14159265358979'

But for strings with more than `sys.float_info.dig` significant digits, this isn't always true:

    >>> s = '9876543211234567'    # 16 significant digits is too many!
    >>> format(float(s), '.16g')  # conversion changes value
    '9876543211234568'

</div>

<div class="data">

float_repr_style

A string indicating how the `repr` function behaves for floats. If the string has value `'short'` then for a finite float `x`, `repr(x)` aims to produce a short string with the property that `float(repr(x)) == x`. This is the usual behaviour in Python 3.1 and later. Otherwise, `float_repr_style` has value `'legacy'` and `repr(x)` behaves in the same way as it did in versions of Python prior to 3.1.

<div class="versionadded">

3.1

</div>

</div>

<div class="function">

getallocatedblocks()

Return the number of memory blocks currently allocated by the interpreter, regardless of their size. This function is mainly useful for tracking and debugging memory leaks. Because of the interpreter's internal caches, the result can vary from call to call; you may have to call `_clear_internal_caches` and `gc.collect` to get more predictable results.

If a Python build or implementation cannot reasonably compute this information, `getallocatedblocks` is allowed to return 0 instead.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

getunicodeinternedsize()

Return the number of unicode objects that have been interned.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

getandroidapilevel()

Return the build-time API level of Android as an integer. This represents the minimum version of Android this build of Python can run on. For runtime version information, see `platform.android_ver`.

<div class="availability">

Android.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

getdefaultencoding()

Return `'utf-8'`. This is the name of the default string encoding, used in methods like `str.encode`.

</div>

<div class="function">

getdlopenflags()

Return the current value of the flags that are used for `dlopen` calls. Symbolic names for the flag values can be found in the `os` module (`RTLD_{xxx}` constants, e.g. `os.RTLD_LAZY`).

<div class="availability">

Unix.

</div>

</div>

<div class="function">

getfilesystemencoding()

Get the `filesystem encoding <filesystem encoding and error handler>`: the encoding used with the `filesystem error handler <filesystem
encoding and error handler>` to convert between Unicode filenames and bytes filenames. The filesystem error handler is returned from `getfilesystemencodeerrors`.

For best compatibility, str should be used for filenames in all cases, although representing filenames as bytes is also supported. Functions accepting or returning filenames should support either str or bytes and internally convert to the system's preferred representation.

`os.fsencode` and `os.fsdecode` should be used to ensure that the correct encoding and errors mode are used.

The `filesystem encoding and error handler` are configured at Python startup by the `PyConfig_Read` function: see `~PyConfig.filesystem_encoding` and `~PyConfig.filesystem_errors` members of `PyConfig`.

<div class="versionchanged">

3.2 `getfilesystemencoding` result cannot be `None` anymore.

</div>

<div class="versionchanged">

3.6 Windows is no longer guaranteed to return `'mbcs'`. See `529` for more information.

</div>

<div class="versionchanged">

3.7 Return `'utf-8'` if the `Python UTF-8 Mode <utf8-mode>` is enabled.

</div>

</div>

<div class="function">

getfilesystemencodeerrors()

Get the `filesystem error handler <filesystem encoding and error
handler>`: the error handler used with the `filesystem encoding
<filesystem encoding and error handler>` to convert between Unicode filenames and bytes filenames. The filesystem encoding is returned from `getfilesystemencoding`.

`os.fsencode` and `os.fsdecode` should be used to ensure that the correct encoding and errors mode are used.

The `filesystem encoding and error handler` are configured at Python startup by the `PyConfig_Read` function: see `~PyConfig.filesystem_encoding` and `~PyConfig.filesystem_errors` members of `PyConfig`.

<div class="versionadded">

3.6

</div>

</div>

<div class="function">

get_int_max_str_digits()

Returns the current value for the `integer string conversion length
limitation <int_max_str_digits>`. See also `set_int_max_str_digits`.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

get_lazy_imports()

Returns the current lazy imports mode as a string.

- `"normal"`: Only imports explicitly marked with the `lazy` keyword are lazy
- `"all"`: All top-level imports are potentially lazy

See also `set_lazy_imports` and `810`.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

get_lazy_imports_filter()

Returns the current lazy imports filter function, or `None` if no filter is set.

The filter function is called for every potentially lazy import to determine whether it should actually be lazy. See `set_lazy_imports_filter` for details on the filter function signature.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

getrefcount(object)

Return the reference count of the *object*. The count returned is generally one higher than you might expect, because it includes the (temporary) reference as an argument to `getrefcount`.

Note that the returned value may not actually reflect how many references to the object are actually held. For example, some objects are `immortal` and have a very high refcount that does not reflect the actual number of references. Consequently, do not rely on the returned value to be accurate, other than a value of 0 or 1.

<div class="impl-detail">

`Immortal <immortal>` objects with a large reference count can be identified via `_is_immortal`.

</div>

<div class="versionchanged">

3.12 Immortal objects have very large refcounts that do not match the actual number of references to the object.

</div>

</div>

<div class="function">

getrecursionlimit()

Return the current value of the recursion limit, the maximum depth of the Python interpreter stack. This limit prevents infinite recursion from causing an overflow of the C stack and crashing Python. It can be set by `setrecursionlimit`.

</div>

<div class="function">

getsizeof(object\[, default\])

Return the size of an object in bytes. The object can be any type of object. All built-in objects will return correct results, but this does not have to hold true for third-party extensions as it is implementation specific.

Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to.

If given, *default* will be returned if the object does not provide means to retrieve the size. Otherwise a `TypeError` will be raised.

`getsizeof` calls the object's `__sizeof__` method and adds an additional garbage collector overhead if the object is managed by the garbage collector.

See [recursive sizeof recipe](https://code.activestate.com/recipes/577504-compute-memory-footprint-of-an-object-and-its-cont/) for an example of using `getsizeof` recursively to find the size of containers and all their contents.

</div>

<div class="function">

getswitchinterval()

Return the interpreter's "thread switch interval" in seconds; see `setswitchinterval`.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

<span id="getframe">getframe</span>(\[depth\])

Return a frame object from the call stack. If optional integer *depth* is given, return the frame object that many calls below the top of the stack. If that is deeper than the call stack, `ValueError` is raised. The default for *depth* is zero, returning the frame at the top of the call stack.

<div class="audit-event">

sys.\_getframe frame sys.\_getframe

</div>

<div class="impl-detail">

This function should be used for internal and specialized purposes only. It is not guaranteed to exist in all implementations of Python.

</div>

</div>

<div class="function">

<span id="getframemodulename">getframemodulename</span>(\[depth\])

Return the name of a module from the call stack. If optional integer *depth* is given, return the module that many calls below the top of the stack. If that is deeper than the call stack, or if the module is unidentifiable, `None` is returned. The default for *depth* is zero, returning the module at the top of the call stack.

<div class="audit-event">

sys.\_getframemodulename depth sys.\_getframemodulename

</div>

<div class="impl-detail">

This function should be used for internal and specialized purposes only. It is not guaranteed to exist in all implementations of Python.

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

getobjects(limit\[, type\])

This function only exists if CPython was built using the specialized configure option `--with-trace-refs`. It is intended only for debugging garbage-collection issues.

Return a list of up to *limit* dynamically allocated Python objects. If *type* is given, only objects of that exact type (not subtypes) are included.

Objects from the list are not safe to use. Specifically, the result will include objects from all interpreters that share their object allocator state (that is, ones created with `PyInterpreterConfig.use_main_obmalloc` set to 1 or using `Py_NewInterpreter`, and the `main interpreter <sub-interpreter-support>`). Mixing objects from different interpreters may lead to crashes or other unexpected behavior.

<div class="impl-detail">

This function should be used for specialized purposes only. It is not guaranteed to exist in all implementations of Python.

</div>

<div class="versionchanged">

3.14

The result may include objects from other interpreters.

</div>

</div>

<div class="function">

getprofile()

<div class="index">

single: profile function single: profiler

</div>

Get the profiler function as set by `setprofile`.

</div>

<div class="function">

gettrace()

<div class="index">

single: trace function single: debugger

</div>

Get the trace function as set by `settrace`.

<div class="impl-detail">

The `gettrace` function is intended only for implementing debuggers, profilers, coverage tools and the like. Its behavior is part of the implementation platform, rather than part of the language definition, and thus may not be available in all Python implementations.

</div>

</div>

<div class="function">

getwindowsversion()

Return a named tuple describing the Windows version currently running. The named elements are *major*, *minor*, *build*, *platform*, *service_pack*, *service_pack_minor*, *service_pack_major*, *suite_mask*, *product_type* and *platform_version*. *service_pack* contains a string, *platform_version* a 3-tuple and all other values are integers. The components can also be accessed by name, so `sys.getwindowsversion()[0]` is equivalent to `sys.getwindowsversion().major`. For compatibility with prior versions, only the first 5 elements are retrievable by indexing.

*platform* will be `2` (VER_PLATFORM_WIN32_NT).

*product_type* may be one of the following values:

| Constant | Meaning |
|----|----|
| `1` (VER_NT_WORKSTATION) | The system is a workstation. |
| `2` (VER_NT_DOMAIN_CONTROLLER) | The system is a domain controller. |
| `3` (VER_NT_SERVER) | The system is a server, but not a domain controller. |

This function wraps the Win32 `!GetVersionEx` function; see the Microsoft documentation on `!OSVERSIONINFOEX` for more information about these fields.

*platform_version* returns the major version, minor version and build number of the current operating system, rather than the version that is being emulated for the process. It is intended for use in logging rather than for feature detection.

> [!NOTE]
> *platform_version* derives the version from kernel32.dll which can be of a different version than the OS version. Please use `platform` module for achieving accurate OS version.

<div class="availability">

Windows.

</div>

<div class="versionchanged">

3.2 Changed to a named tuple and added *service_pack_minor*, *service_pack_major*, *suite_mask*, and *product_type*.

</div>

<div class="versionchanged">

3.6 Added *platform_version*

</div>

</div>

<div class="function">

get_asyncgen_hooks()

Returns an *asyncgen_hooks* object, which is similar to a `~collections.namedtuple` of the form `(firstiter, finalizer)`, where *firstiter* and *finalizer* are expected to be either `None` or functions which take an `asynchronous generator iterator` as an argument, and are used to schedule finalization of an asynchronous generator by an event loop.

<div class="versionadded">

3.6 See `525` for more details.

</div>

> [!NOTE]
> This function has been added on a provisional basis (see `411` for details.)

</div>

<div class="function">

get_coroutine_origin_tracking_depth()

Get the current coroutine origin tracking depth, as set by `set_coroutine_origin_tracking_depth`.

<div class="versionadded">

3.7

</div>

> [!NOTE]
> This function has been added on a provisional basis (see `411` for details.) Use it only for debugging purposes.

</div>

<div class="data">

hash_info

A `named tuple` giving parameters of the numeric hash implementation. For more details about hashing of numeric types, see `numeric-hash`.

<div class="attribute">

hash_info.width

The width in bits used for hash values

</div>

<div class="attribute">

hash_info.modulus

The prime modulus P used for numeric hash scheme

</div>

<div class="attribute">

hash_info.inf

The hash value returned for a positive infinity

</div>

<div class="attribute">

hash_info.nan

(This attribute is no longer used)

</div>

<div class="attribute">

hash_info.imag

The multiplier used for the imaginary part of a complex number

</div>

<div class="attribute">

hash_info.algorithm

The name of the algorithm for hashing of str, bytes, and memoryview

</div>

<div class="attribute">

hash_info.hash_bits

The internal output size of the hash algorithm

</div>

<div class="attribute">

hash_info.seed_bits

The size of the seed key of the hash algorithm

</div>

<div class="attribute">

hash_info.cutoff

Cutoff for small string DJBX33A optimization in range `[1, cutoff)`.

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.4 Added *algorithm*, *hash_bits*, *seed_bits*, and *cutoff*.

</div>

</div>

<div class="data">

hexversion

The version number encoded as a single integer. This is guaranteed to increase with each version, including proper support for non-production releases. For example, to test that the Python interpreter is at least version 1.5.2, use:

    if sys.hexversion >= 0x010502F0:
        # use some advanced feature
        ...
    else:
        # use an alternative implementation or warn the user
        ...

This is called `hexversion` since it only really looks meaningful when viewed as the result of passing it to the built-in `hex` function. The `named tuple` `sys.version_info` may be used for a more human-friendly encoding of the same information.

More details of `hexversion` can be found at `apiabiversion`.

</div>

<div class="data">

implementation

An object containing information about the implementation of the currently running Python interpreter. The following attributes are required to exist in all Python implementations.

*name* is the implementation's identifier, e.g. `'cpython'`. The actual string is defined by the Python implementation, but it is guaranteed to be lower case.

*version* is a named tuple, in the same format as `sys.version_info`. It represents the version of the Python *implementation*. This has a distinct meaning from the specific version of the Python *language* to which the currently running interpreter conforms, which `sys.version_info` represents. For example, for PyPy 1.8 `sys.implementation.version` might be `sys.version_info(1, 8, 0, 'final', 0)`, whereas `sys.version_info` would be `sys.version_info(2, 7, 2, 'final', 0)`. For CPython they are the same value, since it is the reference implementation.

*hexversion* is the implementation version in hexadecimal format, like `sys.hexversion`.

*cache_tag* is the tag used by the import machinery in the filenames of cached modules. By convention, it would be a composite of the implementation's name and version, like `'cpython-33'`. However, a Python implementation may use some other value if appropriate. If `cache_tag` is set to `None`, it indicates that module caching should be disabled.

*supports_isolated_interpreters* is a boolean value, whether this implementation supports multiple isolated interpreters. It is `True` for CPython on most platforms. Platforms with this support implement the low-level `!_interpreters` module.

<div class="seealso">

`684`, `734`, and `concurrent.interpreters`.

</div>

`sys.implementation` may contain additional attributes specific to the Python implementation. These non-standard attributes must start with an underscore, and are not described here. Regardless of its contents, `sys.implementation` will not change during a run of the interpreter, nor between implementation versions. (It may change between Python language versions, however.) See `421` for more information.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.14 Added `supports_isolated_interpreters` field.

</div>

> [!NOTE]
> The addition of new required attributes must go through the normal PEP process. See `421` for more information.

</div>

<div class="data">

int_info

A `named tuple` that holds information about Python's internal representation of integers. The attributes are read only.

<div class="attribute">

int_info.bits_per_digit

The number of bits held in each digit. Python integers are stored internally in base `2**int_info.bits_per_digit`.

</div>

<div class="attribute">

int_info.sizeof_digit

The size in bytes of the C type used to represent a digit.

</div>

<div class="attribute">

int_info.default_max_str_digits

The default value for `sys.get_int_max_str_digits` when it is not otherwise explicitly configured.

</div>

<div class="attribute">

int_info.str_digits_check_threshold

The minimum non-zero value for `sys.set_int_max_str_digits`, `PYTHONINTMAXSTRDIGITS`, or `-X int_max_str_digits <-X>`.

</div>

<div class="versionadded">

3.1

</div>

<div class="versionchanged">

3.11

Added `~int_info.default_max_str_digits` and `~int_info.str_digits_check_threshold`.

</div>

</div>

<div class="data">

\_\_interactivehook\_\_

When this attribute exists, its value is automatically called (with no arguments) when the interpreter is launched in `interactive mode
<tut-interactive>`. This is done after the `PYTHONSTARTUP` file is read, so that you can set this hook there. The `site` module `sets this <rlcompleter-config>`.

<div class="audit-event">

cpython.run_interactivehook hook sys.\_\_interactivehook\_\_

Raises an `auditing event <auditing>` `cpython.run_interactivehook` with the hook object as the argument when the hook is called on startup.

</div>

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

intern(string)

Enter *string* in the table of "interned" strings and return the interned string -- which is *string* itself or a copy. Interning strings is useful to gain a little performance on dictionary lookup -- if the keys in a dictionary are interned, and the lookup key is interned, the key comparisons (after hashing) can be done by a pointer compare instead of a string compare. Normally, the names used in Python programs are automatically interned, and the dictionaries used to hold module, class or instance attributes have interned keys.

Interned strings are not `immortal`; you must keep a reference to the return value of `intern` around to benefit from it.

</div>

<div class="function">

<span id="is_gil_enabled">is_gil_enabled</span>()

Return `True` if the `GIL` is enabled and `False` if it is disabled.

<div class="versionadded">

3.13

</div>

<div class="impl-detail">

It is not guaranteed to exist in all implementations of Python.

</div>

</div>

<div class="function">

is_finalizing()

Return `True` if the main Python interpreter is `shutting down <interpreter shutdown>`. Return `False` otherwise.

See also the `PythonFinalizationError` exception.

<div class="versionadded">

3.5

</div>

</div>

<div class="data">

<span id="jit">jit</span>

Utilities for observing just-in-time compilation.

<div class="impl-detail">

JIT compilation is an *experimental implementation detail* of CPython. `sys._jit` is not guaranteed to exist or behave the same way in all Python implementations, versions, or build configurations.

</div>

<div class="versionadded">

3.14

</div>

<div class="function">

<span id="jit.is_available">jit.is_available</span>()

Return `True` if the current Python executable supports JIT compilation, and `False` otherwise. This can be controlled by building CPython with the `--experimental-jit` option on Windows, and the `--enable-experimental-jit` option on all other platforms.

</div>

<div class="function">

<span id="jit.is_enabled">jit.is_enabled</span>()

Return `True` if JIT compilation is enabled for the current Python process (implies `sys._jit.is_available`), and `False` otherwise. If JIT compilation is available, this can be controlled by setting the `PYTHON_JIT` environment variable to `0` (disabled) or `1` (enabled) at interpreter startup.

</div>

<div class="function">

<span id="jit.is_active">jit.is_active</span>()

Return `True` if the topmost Python frame is currently executing JIT code (implies `sys._jit.is_enabled`), and `False` otherwise.

> [!NOTE]
> This function is intended for testing and debugging the JIT itself. It should be avoided for any other purpose.

> [!NOTE]
> Due to the nature of tracing JIT compilers, repeated calls to this function may give surprising results. For example, branching on its return value will likely lead to unexpected behavior (if doing so causes JIT code to be entered or exited):
>
> ```
> >>> for warmup in range(BIG_NUMBER):
> ...     # This line is "hot", and is eventually JIT-compiled:
> ...     if sys._jit.is_active():
> ...         # This line is "cold", and is run in the interpreter:
> ...         assert sys._jit.is_active()
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 5, in <module>
>     assert sys._jit.is_active()
>            ~~~~~~~~~~~~~~~~~~^^
> AssertionError
> ```

</div>

</div>

<div class="data">

last_exc

This variable is not always defined; it is set to the exception instance when an exception is not handled and the interpreter prints an error message and a stack traceback. Its intended use is to allow an interactive user to import a debugger module and engage in post-mortem debugging without having to re-execute the command that caused the error. (Typical use is `import pdb; pdb.pm()` to enter the post-mortem debugger; see `pdb` module for more information.)

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

<span id="is_immortal">is_immortal</span>(op)

Return `True` if the given object is `immortal`, `False` otherwise.

> [!NOTE]
> Objects that are immortal (and thus return `True` upon being passed to this function) are not guaranteed to be immortal in future versions, and vice versa for mortal objects.

<div class="versionadded">

3.14

</div>

<div class="impl-detail">

This function should be used for specialized purposes only. It is not guaranteed to exist in all implementations of Python.

</div>

</div>

<div class="function">

<span id="is_interned">is_interned</span>(string)

Return `True` if the given string is "interned", `False` otherwise.

<div class="versionadded">

3.13

</div>

<div class="impl-detail">

It is not guaranteed to exist in all implementations of Python.

</div>

</div>

<div class="data">

last_type last_value last_traceback

These three variables are deprecated; use `sys.last_exc` instead. They hold the legacy representation of `sys.last_exc`, as returned from `exc_info` above.

</div>

<div class="data">

lazy_modules

A `set` of fully qualified module name strings that have been lazily imported in the current interpreter but not yet loaded. When a lazily imported module is accessed for the first time, its name is typically removed from this set.

The set may contain some additional strings. It is intended for debugging and introspection, and consumers are expected to verify each entry's status.

<div class="impl-detail">

Currently, `!lazy_modules` may also contain:

- names of *attributes* (non-modules), such as `"pathlib.Path"` after running `lazy from pathlib import Path`, and
- names of items than have already been accessed.

In future versions of Python, these may be removed, and/or additional extras may be added.

</div>

See also `set_lazy_imports` and `810`.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

maxsize

An integer giving the maximum value a variable of type `Py_ssize_t` can take. It's usually `2**31 - 1` on a 32-bit platform and `2**63 - 1` on a 64-bit platform.

</div>

<div class="data">

maxunicode

An integer giving the value of the largest Unicode code point, i.e. `1114111` (`0x10FFFF` in hexadecimal).

<div class="versionchanged">

3.3 Before `393`, `sys.maxunicode` used to be either `0xFFFF` or `0x10FFFF`, depending on the configuration option that specified whether Unicode characters were stored as UCS-2 or UCS-4.

</div>

</div>

<div class="data">

meta_path

A list of `meta path finder` objects that have their `~importlib.abc.MetaPathFinder.find_spec` methods called to see if one of the objects can find the module to be imported. By default, it holds entries that implement Python's default import semantics. The `~importlib.abc.MetaPathFinder.find_spec` method is called with at least the absolute name of the module being imported. If the module to be imported is contained in a package, then the parent package's `~module.__path__` attribute is passed in as a second argument. The method returns a `module spec`, or `None` if the module cannot be found.

<div class="seealso">

`importlib.abc.MetaPathFinder`  
The abstract base class defining the interface of finder objects on `meta_path`.

`importlib.machinery.ModuleSpec`  
The concrete class which `~importlib.abc.MetaPathFinder.find_spec` should return instances of.

</div>

<div class="versionchanged">

3.4

`Module specs <module spec>` were introduced in Python 3.4, by `451`.

</div>

<div class="versionchanged">

3.12

Removed the fallback that looked for a `!find_module` method if a `meta_path` entry didn't have a `~importlib.abc.MetaPathFinder.find_spec` method.

</div>

</div>

<div class="data">

modules

This is a dictionary that maps module names to modules which have already been loaded. This can be manipulated to force reloading of modules and other tricks. However, replacing the dictionary will not necessarily work as expected and deleting essential items from the dictionary may cause Python to fail. If you want to iterate over this global dictionary always use `sys.modules.copy()` or `tuple(sys.modules)` to avoid exceptions as its size may change during iteration as a side effect of code or activity in other threads.

</div>

<div class="data">

orig_argv

The list of the original command line arguments passed to the Python executable.

The elements of `sys.orig_argv` are the arguments to the Python interpreter, while the elements of `sys.argv` are the arguments to the user's program. Arguments consumed by the interpreter itself will be present in `sys.orig_argv` and missing from `sys.argv`.

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

path

<div class="index">

triple: module; search; path

</div>

A list of strings that specifies the search path for modules. Initialized from the environment variable `PYTHONPATH`, plus an installation-dependent default.

By default, as initialized upon program startup, a potentially unsafe path is prepended to `sys.path` (*before* the entries inserted as a result of `PYTHONPATH`):

- `python -m module` command line: prepend the current working directory.
- `python script.py` command line: prepend the script's directory. If it's a symbolic link, resolve symbolic links.
- `python -c code` and `python` (REPL) command lines: prepend an empty string, which means the current working directory.

To not prepend this potentially unsafe path, use the `-P` command line option or the `PYTHONSAFEPATH` environment variable.

A program is free to modify this list for its own purposes. Only strings should be added to `sys.path`; all other data types are ignored during import.

<div class="seealso">

\* Module `site` This describes how to use .pth files to extend `sys.path`.

</div>

</div>

<div class="data">

path_hooks

A list of callables that take a path argument to try to create a `finder` for the path. If a finder can be created, it is to be returned by the callable, else raise `ImportError`.

Originally specified in `302`.

</div>

<div class="data">

path_importer_cache

A dictionary acting as a cache for `finder` objects. The keys are paths that have been passed to `sys.path_hooks` and the values are the finders that are found. If a path is a valid file system path but no finder is found on `sys.path_hooks` then `None` is stored.

Originally specified in `302`.

</div>

<div class="data">

platform

A string containing a platform identifier. Known values are:

| System         | `platform` value |
|----------------|------------------|
| AIX            | `'aix'`          |
| Android        | `'android'`      |
| Emscripten     | `'emscripten'`   |
| FreeBSD        | `'freebsd'`      |
| iOS            | `'ios'`          |
| Linux          | `'linux'`        |
| macOS          | `'darwin'`       |
| Windows        | `'win32'`        |
| Windows/Cygwin | `'cygwin'`       |
| WASI           | `'wasi'`         |

On Unix systems not listed in the table, the value is the lowercased OS name as returned by `uname -s`, with the first part of the version as returned by `uname -r` appended, e.g. `'sunos5'`, *at the time when Python was built*. Unless you want to test for a specific system version, it is therefore recommended to use the following idiom:

    if sys.platform.startswith('sunos'):
        # SunOS-specific code here...

<div class="versionchanged">

3.3 On Linux, `sys.platform` doesn't contain the major version anymore. It is always `'linux'`, instead of `'linux2'` or `'linux3'`.

</div>

<div class="versionchanged">

3.8 On AIX, `sys.platform` doesn't contain the major version anymore. It is always `'aix'`, instead of `'aix5'` or `'aix7'`.

</div>

<div class="versionchanged">

3.13 On Android, `sys.platform` now returns `'android'` rather than `'linux'`.

</div>

<div class="versionchanged">

3.14 On FreeBSD, `sys.platform` doesn't contain the major version anymore. It is always `'freebsd'`, instead of `'freebsd13'` or `'freebsd14'`.

</div>

<div class="seealso">

`os.name` has a coarser granularity. `os.uname` gives system-dependent version information.

The `platform` module provides detailed checks for the system's identity.

</div>

</div>

<div class="data">

platlibdir

Name of the platform-specific library directory. It is used to build the path of standard library and the paths of installed extension modules.

It is equal to `"lib"` on most platforms. On Fedora and SuSE, it is equal to `"lib64"` on 64-bit platforms which gives the following `sys.path` paths (where `X.Y` is the Python `major.minor` version):

- `/usr/lib64/pythonX.Y/`: Standard library (like `os.py` of the `os` module)
- `/usr/lib64/pythonX.Y/lib-dynload/`: C extension modules of the standard library (like the `errno` module, the exact filename is platform specific)
- `/usr/lib/pythonX.Y/site-packages/` (always use `lib`, not `sys.platlibdir`): Third-party modules
- `/usr/lib64/pythonX.Y/site-packages/`: C extension modules of third-party packages

<div class="versionadded">

3.9

</div>

</div>

<div class="data">

prefix

A string giving the site-specific directory prefix where the platform independent Python files are installed; on Unix, the default is `/usr/local`. This can be set at build time with the `--prefix` argument to the `configure` script. See `installation_paths` for derived paths.

> [!NOTE]
> If a `virtual environment <venv-def>` is in effect, this `prefix` will point to the virtual environment. The value for the Python installation will still be available, via `base_prefix`. Refer to `sys-path-init-virtual-environments` for more information.

<div class="versionchanged">

3.14

When running under a `virtual environment <venv-def>`, `prefix` and `exec_prefix` are now set to the virtual environment prefix by the `path initialization <sys-path-init>`, instead of `site`. This means that `prefix` and `exec_prefix` always point to the virtual environment, even when `site` is disabled (`-S`).

</div>

</div>

<div class="data">

ps1 ps2

<div class="index">

single: interpreter prompts single: prompts, interpreter single: \>\>\>; interpreter prompt single: ...; interpreter prompt

</div>

Strings specifying the primary and secondary prompt of the interpreter. These are only defined if the interpreter is in interactive mode. Their initial values in this case are `'>>> '` and `'... '`. If a non-string object is assigned to either variable, its `str` is re-evaluated each time the interpreter prepares to read a new interactive command; this can be used to implement a dynamic prompt.

</div>

<div class="function">

setdlopenflags(n)

Set the flags used by the interpreter for `dlopen` calls, such as when the interpreter loads extension modules. Among other things, this will enable a lazy resolving of symbols when importing a module, if called as `sys.setdlopenflags(0)`. To share symbols across extension modules, call as `sys.setdlopenflags(os.RTLD_GLOBAL)`. Symbolic names for the flag values can be found in the `os` module (`RTLD_{xxx}` constants, e.g. `os.RTLD_LAZY`).

<div class="availability">

Unix.

</div>

</div>

<div class="function">

set_int_max_str_digits(maxdigits)

Set the `integer string conversion length limitation
<int_max_str_digits>` used by this interpreter. See also `get_int_max_str_digits`.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

set_lazy_imports(mode)

Sets the global lazy imports mode. The *mode* parameter must be one of the following strings:

- `"normal"`: Only imports explicitly marked with the `lazy` keyword are lazy
- `"all"`: All top-level imports become potentially lazy

This function is intended for advanced users who need to control lazy imports across their entire application. Library developers should generally not use this function as it affects the runtime execution of applications.

In addition to the mode, lazy imports can be controlled via the filter provided by `set_lazy_imports_filter`.

See also `get_lazy_imports` and `810`.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

set_lazy_imports_filter(filter)

Sets the lazy imports filter callback. The *filter* parameter must be a callable or `None` to clear the filter.

The filter function is called for every potentially lazy import to determine whether it should actually be lazy. It should have the following signature:

    def filter(importing_module: str, imported_module: str,
               fromlist: tuple[str, ...] | None) -> bool

The function is called with three positional arguments:

- *importing_module* is the name of the module doing the import
- *imported_module* is the resolved name of the module being imported (for example, `lazy from .spam import eggs` passes `package.spam`)
- *fromlist* is the tuple of names being imported (for `from ... import` statements), or `None` for regular imports

The filter should return `True` to allow the import to be lazy, or `False` to force an eager import.

This is an advanced feature intended for specialized users who need fine-grained control over lazy import behavior.

See also `get_lazy_imports_filter` and `810`.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

setprofile(profilefunc)

<div class="index">

single: profile function single: profiler

</div>

Set the system's profile function, which allows you to implement a Python source code profiler in Python. See chapter `profile` for more information on the Python profiler. The system's profile function is called similarly to the system's trace function (see `settrace`), but it is called with different events, for example it isn't called for each executed line of code (only on call and return, but the return event is reported even when an exception has been set). The function is thread-specific, but there is no way for the profiler to know about context switches between threads, so it does not make sense to use this in the presence of multiple threads. Also, its return value is not used, so it can simply return `None`. Error in the profile function will cause itself unset.

> [!NOTE]
> The same tracing mechanism is used for `!setprofile` as `settrace`. To trace calls with `!setprofile` inside a tracing function (e.g. in a debugger breakpoint), see `call_tracing`.

Profile functions should have three arguments: *frame*, *event*, and *arg*. *frame* is the current stack frame. *event* is a string: `'call'`, `'return'`, `'c_call'`, `'c_return'`, or `'c_exception'`. *arg* depends on the event type.

The events have the following meaning:

`'call'`  
A function is called (or some other code block entered). The profile function is called; *arg* is `None`.

`'return'`  
A function (or other code block) is about to return. The profile function is called; *arg* is the value that will be returned, or `None` if the event is caused by an exception being raised.

`'c_call'`  
A C function is about to be called. This may be an extension function or a built-in. *arg* is the C function object.

`'c_return'`  
A C function has returned. *arg* is the C function object.

`'c_exception'`  
A C function has raised an exception. *arg* is the C function object.

<div class="audit-event">

sys.setprofile "" sys.setprofile

</div>

</div>

<div class="function">

setrecursionlimit(limit)

Set the maximum depth of the Python interpreter stack to *limit*. This limit prevents infinite recursion from causing an overflow of the C stack and crashing Python.

The highest possible limit is platform-dependent. A user may need to set the limit higher when they have a program that requires deep recursion and a platform that supports a higher limit. This should be done with care, because a too-high limit can lead to a crash.

If the new limit is too low at the current recursion depth, a `RecursionError` exception is raised.

<div class="versionchanged">

3.5.1 A `RecursionError` exception is now raised if the new limit is too low at the current recursion depth.

</div>

</div>

<div class="function">

setswitchinterval(interval)

Set the interpreter's thread switch interval (in seconds). This floating-point value determines the ideal duration of the "timeslices" allocated to concurrently running Python threads. Please note that the actual value can be higher, especially if long-running internal functions or methods are used. Also, which thread becomes scheduled at the end of the interval is the operating system's decision. The interpreter doesn't have its own scheduler.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

settrace(tracefunc)

<div class="index">

single: trace function single: debugger

</div>

Set the system's trace function, which allows you to implement a Python source code debugger in Python. The function is thread-specific; for a debugger to support multiple threads, it must register a trace function using `settrace` for each thread being debugged or use `threading.settrace`.

Trace functions should have three arguments: *frame*, *event*, and *arg*. *frame* is the `current stack frame <frame-objects>`. *event* is a string: `'call'`, `'line'`, `'return'`, `'exception'` or `'opcode'`. *arg* depends on the event type.

The trace function is invoked (with *event* set to `'call'`) whenever a new local scope is entered; it should return a reference to a local trace function to be used for the new scope, or `None` if the scope shouldn't be traced.

The local trace function should return a reference to itself, or to another function which would then be used as the local trace function for the scope.

If there is any error occurred in the trace function, it will be unset, just like `settrace(None)` is called.

> [!NOTE]
> Tracing is disabled while calling the trace function (e.g. a function set by `!settrace`). For recursive tracing see `call_tracing`.

The events have the following meaning:

`'call'`  
A function is called (or some other code block entered). The global trace function is called; *arg* is `None`; the return value specifies the local trace function.

`'line'`  
The interpreter is about to execute a new line of code or re-execute the condition of a loop. The local trace function is called; *arg* is `None`; the return value specifies the new local trace function. See `InternalDocs/code_objects.md` for a detailed explanation of how this works. Per-line events may be disabled for a frame by setting `~frame.f_trace_lines` to `False` on that `frame <frame-objects>`.

`'return'`  
A function (or other code block) is about to return. The local trace function is called; *arg* is the value that will be returned, or `None` if the event is caused by an exception being raised. The trace function's return value is ignored.

`'exception'`  
An exception has occurred. The local trace function is called; *arg* is a tuple `(exception, value, traceback)`; the return value specifies the new local trace function.

`'opcode'`  
The interpreter is about to execute a new opcode (see `dis` for opcode details). The local trace function is called; *arg* is `None`; the return value specifies the new local trace function. Per-opcode events are not emitted by default: they must be explicitly requested by setting `~frame.f_trace_opcodes` to `True` on the `frame <frame-objects>`.

Note that as an exception is propagated down the chain of callers, an `'exception'` event is generated at each level.

For more fine-grained usage, it's possible to set a trace function by assigning `frame.f_trace = tracefunc` explicitly, rather than relying on it being set indirectly via the return value from an already installed trace function. This is also required for activating the trace function on the current frame, which `settrace` doesn't do. Note that in order for this to work, a global tracing function must have been installed with `settrace` in order to enable the runtime tracing machinery, but it doesn't need to be the same tracing function (e.g. it could be a low overhead tracing function that simply returns `None` to disable itself immediately on each frame).

For more information on code and frame objects, refer to `types`.

<div class="audit-event">

sys.settrace "" sys.settrace

</div>

<div class="impl-detail">

The `settrace` function is intended only for implementing debuggers, profilers, coverage tools and the like. Its behavior is part of the implementation platform, rather than part of the language definition, and thus may not be available in all Python implementations.

</div>

<div class="versionchanged">

3.7

`'opcode'` event type added; `~frame.f_trace_lines` and `~frame.f_trace_opcodes` attributes added to frames

</div>

</div>

<div class="function">

set_asyncgen_hooks(\[firstiter\] \[, finalizer\])

Accepts two optional keyword arguments which are callables that accept an `asynchronous generator iterator` as an argument. The *firstiter* callable will be called when an asynchronous generator is iterated for the first time. The *finalizer* will be called when an asynchronous generator is about to be garbage collected.

<div class="audit-event">

sys.set_asyncgen_hooks_firstiter "" sys.set_asyncgen_hooks

</div>

<div class="audit-event">

sys.set_asyncgen_hooks_finalizer "" sys.set_asyncgen_hooks

</div>

Two auditing events are raised because the underlying API consists of two calls, each of which must raise its own event.

<div class="versionadded">

3.6 See `525` for more details, and for a reference example of a *finalizer* method see the implementation of `asyncio.Loop.shutdown_asyncgens` in `Lib/asyncio/base_events.py`

</div>

> [!NOTE]
> This function has been added on a provisional basis (see `411` for details.)

</div>

<div class="function">

set_coroutine_origin_tracking_depth(depth)

Allows enabling or disabling coroutine origin tracking. When enabled, the `cr_origin` attribute on coroutine objects will contain a tuple of (filename, line number, function name) tuples describing the traceback where the coroutine object was created, with the most recent call first. When disabled, `cr_origin` will be `None`.

To enable, pass a *depth* value greater than zero; this sets the number of frames whose information will be captured. To disable, set *depth* to zero.

This setting is thread-specific.

<div class="versionadded">

3.7

</div>

> [!NOTE]
> This function has been added on a provisional basis (see `411` for details.) Use it only for debugging purposes.

</div>

<div class="function">

activate_stack_trampoline(backend, /)

Activate the stack profiler trampoline *backend*. The only supported backend is `"perf"`.

Stack trampolines cannot be activated if the JIT is active.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.12

</div>

<div class="seealso">

- `perf_profiling`
- <https://perf.wiki.kernel.org>

</div>

</div>

<div class="function">

deactivate_stack_trampoline()

Deactivate the current stack profiler trampoline backend.

If no stack profiler is activated, this function has no effect.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

is_stack_trampoline_active()

Return `True` if a stack profiler trampoline is active.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

remote_exec(pid, script)

Executes *script*, a file containing Python code in the remote process with the given *pid*.

This function returns immediately, and the code will be executed by the target process's main thread at the next available opportunity, similarly to how signals are handled. There is no interface to determine when the code has been executed. The caller is responsible for making sure that the file still exists whenever the remote process tries to read it and that it hasn't been overwritten.

The remote process must be running a CPython interpreter of the same major and minor version as the local process. If either the local or remote interpreter is pre-release (alpha, beta, or release candidate) then the local and remote interpreters must be the same exact version.

See `remote-debugging` for more information about the remote debugging mechanism.

<div class="audit-event">

sys.remote_exec pid,script_path

When the code is executed in the remote process, an `auditing event <auditing>` `sys.remote_exec` is raised with the *pid* and the path to the script file. This event is raised in the process that called `sys.remote_exec`.

</div>

<div class="audit-event">

cpython.remote_debugger_script script_path

When the script is executed in the remote process, an `auditing event <auditing>` `cpython.remote_debugger_script` is raised with the path in the remote process. This event is raised in the remote process, not the one that called `sys.remote_exec`.

</div>

<div class="availability">

Unix, Windows.

</div>

<div class="versionadded">

3.14 See `768` for more details.

</div>

</div>

<div class="data">

stdin stdout stderr

`File objects <file object>` used by the interpreter for standard input, output and errors:

- `stdin` is used for all interactive input (including calls to `input`);
- `stdout` is used for the output of `print` and `expression` statements and for the prompts of `input`;
- The interpreter's own prompts and its error messages go to `stderr`.

These streams are regular `text files <text file>` like those returned by the `open` function. Their parameters are chosen as follows:

- The encoding and error handling are initialized from `PyConfig.stdio_encoding` and `PyConfig.stdio_errors`.

  On Windows, UTF-8 is used for the console device. Non-character devices such as disk files and pipes use the system locale encoding (i.e. the ANSI codepage). Non-console character devices such as NUL (i.e. where `isatty()` returns `True`) use the value of the console input and output codepages at startup, respectively for stdin and stdout/stderr. This defaults to the system `locale encoding` if the process is not initially attached to a console.

  The special behaviour of the console can be overridden by setting the environment variable PYTHONLEGACYWINDOWSSTDIO before starting Python. In that case, the console codepages are used as for any other character device.

  Under all platforms, you can override the character encoding by setting the `PYTHONIOENCODING` environment variable before starting Python or by using the new `-X` `utf8` command line option and `PYTHONUTF8` environment variable. However, for the Windows console, this only applies when `PYTHONLEGACYWINDOWSSTDIO` is also set.

- When interactive, the `stdout` stream is line-buffered. Otherwise, it is block-buffered like regular text files. The `stderr` stream is line-buffered in both cases. You can make both streams unbuffered by passing the `-u` command-line option or setting the `PYTHONUNBUFFERED` environment variable.

<div class="versionchanged">

3.9 Non-interactive `stderr` is now line-buffered instead of fully buffered.

</div>

> [!NOTE]
> To write or read binary data from/to the standard streams, use the underlying binary `~io.TextIOBase.buffer` object. For example, to write bytes to `stdout`, use `sys.stdout.buffer.write(b'abc')`.
>
> However, if you are writing a library (and do not control in which context its code will be executed), be aware that the standard streams may be replaced with file-like objects like `io.StringIO` which do not support the `!buffer` attribute.

</div>

<div class="data">

\_\_stdin\_\_ \_\_stdout\_\_ \_\_stderr\_\_

These objects contain the original values of `stdin`, `stderr` and `stdout` at the start of the program. They are used during finalization, and could be useful to print to the actual standard stream no matter if the `sys.std*` object has been redirected.

It can also be used to restore the actual files to known working file objects in case they have been overwritten with a broken object. However, the preferred way to do this is to explicitly save the previous stream before replacing it, and restore the saved object.

> [!NOTE]
> Under some conditions `stdin`, `stdout` and `stderr` as well as the original values `__stdin__`, `__stdout__` and `__stderr__` can be `None`. It is usually the case for Windows GUI apps that aren't connected to a console and Python apps started with `pythonw`.

</div>

<div class="data">

stdlib_module_names

A frozenset of strings containing the names of standard library modules.

It is the same on all platforms. Modules which are not available on some platforms and modules disabled at Python build are also listed. All module kinds are listed: pure Python, built-in, frozen and extension modules. Test modules are excluded.

For packages, only the main package is listed: sub-packages and sub-modules are not listed. For example, the `email` package is listed, but the `email.mime` sub-package and the `email.message` sub-module are not listed.

See also the `sys.builtin_module_names` list.

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

thread_info

A `named tuple` holding information about the thread implementation.

<div class="attribute">

thread_info.name

The name of the thread implementation:

- `"nt"`: Windows threads
- `"pthread"`: POSIX threads
- `"pthread-stubs"`: stub POSIX threads (on WebAssembly platforms without threading support)
- `"solaris"`: Solaris threads

</div>

<div class="attribute">

thread_info.lock

The name of the lock implementation:

- `"semaphore"`: a lock uses a semaphore (Python 3.14 and older)
- `"mutex+cond"`: a lock uses a mutex and a condition variable (Python 3.14 and older)
- `"pymutex"`: a lock uses the `PyMutex` implementation (Python 3.15 and newer)
- `None` if this information is unknown

</div>

<div class="attribute">

thread_info.version

The name and version of the thread library. It is a string, or `None` if this information is unknown.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

tracebacklimit

When this variable is set to an integer value, it determines the maximum number of levels of traceback information printed when an unhandled exception occurs. The default is `1000`. When set to `0` or less, all traceback information is suppressed and only the exception type and value are printed.

</div>

<div class="function">

unraisablehook(unraisable, /)

Handle an unraisable exception.

Called when an exception has occurred but there is no way for Python to handle it. For example, when a destructor raises an exception or during garbage collection (`gc.collect`).

The *unraisable* argument has the following attributes:

- `!exc_type`: Exception type.
- `!exc_value`: Exception value, can be `None`.
- `!exc_traceback`: Exception traceback, can be `None`.
- `!err_msg`: Error message, can be `None`.
- `!object`: Object causing the exception, can be `None`.

The default hook formats `!err_msg` and `!object` as: `f'{err_msg}: {object!r}'`; use "Exception ignored in" error message if `!err_msg` is `None`. Similar to the `traceback` module, this adds color to exceptions by default. This can be disabled using `environment variables <using-on-controlling-color>`.

`sys.unraisablehook` can be overridden to control how unraisable exceptions are handled.

<div class="versionchanged">

3.15 Exceptions are now printed with colorful text.

</div>

<div class="seealso">

`excepthook` which handles uncaught exceptions.

</div>

> [!WARNING]
> Storing `!exc_value` using a custom hook can create a reference cycle. It should be cleared explicitly to break the reference cycle when the exception is no longer needed.
>
> Storing `!object` using a custom hook can resurrect it if it is set to an object which is being finalized. Avoid storing `!object` after the custom hook completes to avoid resurrecting objects.

<div class="audit-event">

sys.unraisablehook hook,unraisable sys.unraisablehook

Raise an auditing event `sys.unraisablehook` with arguments *hook*, *unraisable* when an exception that cannot be handled occurs. The *unraisable* object is the same as what will be passed to the hook. If no hook has been set, *hook* may be `None`.

</div>

<div class="versionadded">

3.8

</div>

</div>

<div class="data">

version

A string containing the version number of the Python interpreter plus additional information on the build number and compiler used. This string is displayed when the interactive interpreter is started. Do not extract version information out of it, rather, use `version_info` and the functions provided by the `platform` module.

</div>

<div class="data">

api_version

The C API version, equivalent to the C macro `PYTHON_API_VERSION`. Defined for backwards compatibility.

Currently, this constant is not updated in new Python versions, and is not useful for versioning. This may change in the future.

</div>

<div class="data">

version_info

A tuple containing the five components of the version number: *major*, *minor*, *micro*, *releaselevel*, and *serial*. All values except *releaselevel* are integers; the release level is `'alpha'`, `'beta'`, `'candidate'`, or `'final'`. The `version_info` value corresponding to the Python version 2.0 is `(2, 0, 0, 'final', 0)`. The components can also be accessed by name, so `sys.version_info[0]` is equivalent to `sys.version_info.major` and so on.

<div class="versionchanged">

3.1 Added named component attributes.

</div>

</div>

<div class="data">

warnoptions

This is an implementation detail of the warnings framework; do not modify this value. Refer to the `warnings` module for more information on the warnings framework.

</div>

<div class="data">

winver

The version number used to form registry keys on Windows platforms. This is stored as string resource 1000 in the Python DLL. The value is normally the major and minor versions of the running Python interpreter. It is provided in the `!sys` module for informational purposes; modifying this value has no effect on the registry keys used by Python.

<div class="availability">

Windows.

</div>

</div>

<div class="data" data-noindex="">

monitoring

Namespace containing functions and constants for register callbacks and controlling monitoring events. See `sys.monitoring` for details.

</div>

<div class="data">

<span id="xoptions">xoptions</span>

A dictionary of the various implementation-specific flags passed through the `-X` command-line option. Option names are either mapped to their values, if given explicitly, or to `True`. Example:

```
$ ./python -Xa=b -Xc
Python 3.2a3+ (py3k, Oct 16 2010, 20:14:50)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys._xoptions
{'a': 'b', 'c': True}
```

<div class="impl-detail">

This is a CPython-specific way of accessing options passed through `-X`. Other implementations may export them through other means, or not at all.

</div>

<div class="versionadded">

3.2

</div>

</div>

<div class="data">

EXIT_SUCCESS

An int you can use to indicate your program ended gracefully.

``` python
sys.exit(sys.EXIT_SUCCESS)
```

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

EXIT_FAILURE

An int you can use to indicate your program encountered an error.

``` python
sys.exit(sys.EXIT_FAILURE)
```

<div class="versionadded">

3.16

</div>

</div>

**Citations**

<div id="citations">

<span id="C99" class="citation-label">C99</span>  
ISO/IEC 9899:1999. "Programming languages -- C." A public draft of this standard is available at [https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf\\](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf\) .

</div>

# `!sys.monitoring` --- Execution event monitoring

<div class="module" data-synopsis="Access and control event monitoring">

sys.monitoring

</div>

<div class="versionadded">

3.12

</div>

------------------------------------------------------------------------

> [!NOTE]
> `!sys.monitoring` is a namespace within the `sys` module, not an independent module, and `import sys.monitoring` would fail with a `ModuleNotFoundError`. Instead, simply `import sys` and then use `sys.monitoring`.

This namespace provides access to the functions and constants necessary to activate and control event monitoring.

As programs execute, events occur that might be of interest to tools that monitor execution. The `!sys.monitoring` namespace provides means to receive callbacks when events of interest occur.

The monitoring API consists of three components:

- [Tool identifiers](#tool-identifiers)
- [Events](#events)
- `Callbacks <callbacks>`

## Tool identifiers

A tool identifier is an integer and the associated name. Tool identifiers are used to discourage tools from interfering with each other and to allow multiple tools to operate at the same time. Currently tools are completely independent and cannot be used to monitor each other. This restriction may be lifted in the future.

Before registering or activating events, a tool should choose an identifier. Identifiers are integers in the range 0 to 5 inclusive.

### Registering and using tools

<div class="function">

use_tool_id(tool_id: int, name: str, /) -\> None

Must be called before *tool_id* can be used. *tool_id* must be in the range 0 to 5 inclusive. Raises a `ValueError` if *tool_id* is in use.

</div>

<div class="function">

clear_tool_id(tool_id: int, /) -\> None

Unregister all events and callback functions associated with *tool_id*.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

free_tool_id(tool_id: int, /) -\> None

Should be called once a tool no longer requires *tool_id*. Will call `clear_tool_id` before releasing *tool_id*.

<div class="versionchanged">

3.14 Now calls `clear_tool_id` before releasing *tool_id*. Previously, it would not disable global or local events associated with *tool_id*, nor unregister any callback functions.

</div>

</div>

<div class="function">

get_tool(tool_id: int, /) -\> str \| None

Returns the name of the tool if *tool_id* is in use, otherwise it returns `None`. *tool_id* must be in the range 0 to 5 inclusive.

</div>

All IDs are treated the same by the VM with regard to events, but the following IDs are pre-defined to make co-operation of tools easier:

    sys.monitoring.DEBUGGER_ID = 0
    sys.monitoring.COVERAGE_ID = 1
    sys.monitoring.PROFILER_ID = 2
    sys.monitoring.OPTIMIZER_ID = 5

## Events

The following events are supported:

<div class="monitoring-event">

BRANCH_LEFT

A conditional branch goes left.

It is up to the tool to determine how to present "left" and "right" branches. There is no guarantee which branch is "left" and which is "right", except that it will be consistent for the duration of the program.

</div>

<div class="monitoring-event">

BRANCH_RIGHT

A conditional branch goes right.

</div>

<div class="monitoring-event">

CALL

A call in Python code (event occurs before the call).

</div>

<div class="monitoring-event">

C_RAISE

An exception raised from any callable, except for Python functions (event occurs after the exit).

</div>

<div class="monitoring-event">

C_RETURN

Return from any callable, except for Python functions (event occurs after the return).

</div>

<div class="monitoring-event">

EXCEPTION_HANDLED

An exception is handled.

</div>

<div class="monitoring-event">

INSTRUCTION

A VM instruction is about to be executed.

</div>

<div class="monitoring-event">

JUMP

An unconditional jump in the control flow graph is made.

</div>

<div class="monitoring-event">

LINE

An instruction is about to be executed that has a different line number from the preceding instruction.

</div>

<div class="monitoring-event">

PY_RESUME

Resumption of a Python function (for generator and coroutine functions), except for `throw()` calls.

</div>

<div class="monitoring-event">

PY_RETURN

Return from a Python function (occurs immediately before the return, the callee's frame will be on the stack).

</div>

<div class="monitoring-event">

PY_START

Start of a Python function (occurs immediately after the call, the callee's frame will be on the stack)

</div>

<div class="monitoring-event">

PY_THROW

A Python function is resumed by a `throw()` call.

</div>

<div class="monitoring-event">

PY_UNWIND

Exit from a Python function during exception unwinding. This includes exceptions raised directly within the function and that are allowed to continue to propagate.

</div>

<div class="monitoring-event">

PY_YIELD

Yield from a Python function (occurs immediately before the yield, the callee's frame will be on the stack).

</div>

<div class="monitoring-event">

RAISE

An exception is raised, except those that cause a `STOP_ITERATION` event.

</div>

<div class="monitoring-event">

RERAISE

An exception is re-raised, for example at the end of a `finally` block.

</div>

<div class="monitoring-event">

STOP_ITERATION

An artificial `StopIteration` is raised; see [the STOP_ITERATION event](#the-stop_iteration-event).

</div>

More events may be added in the future.

These events are attributes of the `!sys.monitoring.events` namespace. Each event is represented as a power-of-2 integer constant. To define a set of events, simply bitwise OR the individual events together. For example, to specify both `PY_RETURN` and `PY_START` events, use the expression `PY_RETURN | PY_START`.

<div class="monitoring-event">

NO_EVENTS

An alias for `0` so users can do explicit comparisons like:

    if get_events(DEBUGGER_ID) == NO_EVENTS:
        ...

Setting this event deactivates all events.

</div>

### Local events

Local events are associated with normal execution of the program and happen at clearly defined locations. All local events can be disabled per location. The local events are:

- `PY_START`
- `PY_RESUME`
- `PY_RETURN`
- `PY_YIELD`
- `CALL`
- `LINE`
- `INSTRUCTION`
- `JUMP`
- `BRANCH_LEFT`
- `BRANCH_RIGHT`
- `STOP_ITERATION`

### Deprecated event

- `BRANCH`

The `BRANCH` event is deprecated in 3.14. Using `BRANCH_LEFT` and `BRANCH_RIGHT` events will give much better performance as they can be disabled independently.

### Ancillary events

Ancillary events can be monitored like other events, but are controlled by another event:

- `C_RAISE`
- `C_RETURN`

The `C_RETURN` and `C_RAISE` events are controlled by the `CALL` event. `C_RETURN` and `C_RAISE` events will only be seen if the corresponding `CALL` event is being monitored.

### Other events

Other events are not necessarily tied to a specific location in the program and cannot be individually disabled per location.

The other events that can be monitored are:

- `PY_THROW`
- `PY_UNWIND`
- `RAISE`
- `EXCEPTION_HANDLED`
- `RERAISE`

<div class="versionchanged">

3.15 Other events can now be turned on and disabled on a per code object basis. Returning `DISABLE` from a callback disables the event for the entire code object (for the current tool).

</div>

### The STOP_ITERATION event

`PEP 380 <380#use-of-stopiteration-to-return-values>` specifies that a `StopIteration` exception is raised when returning a value from a generator or coroutine. However, this is a very inefficient way to return a value, so some Python implementations, notably CPython 3.12+, do not raise an exception unless it would be visible to other code.

To allow tools to monitor for real exceptions without slowing down generators and coroutines, the `STOP_ITERATION` event is provided. `STOP_ITERATION` can be locally disabled.

Note that the `STOP_ITERATION` event and the `RAISE` event for a `StopIteration` exception are equivalent, and are treated as interchangeable when generating events. Implementations will favor `STOP_ITERATION` for performance reasons, but may generate a `RAISE` event with a `StopIteration`.

## Turning events on and off

In order to monitor an event, it must be turned on and a corresponding callback must be registered. Events can be turned on or off by setting the events either globally and/or for a particular code object. An event will trigger only once, even if it is turned on both globally and locally.

### Setting events globally

Events can be controlled globally by modifying the set of events being monitored.

<div class="function">

get_events(tool_id: int, /) -\> int

Returns the `int` representing all the active events.

</div>

<div class="function">

set_events(tool_id: int, event_set: int, /) -\> None

Activates all events which are set in *event_set*. Raises a `ValueError` if *tool_id* is not in use.

</div>

No events are active by default.

### Per code object events

Events can also be controlled on a per code object basis. The functions defined below which accept a `types.CodeType` should be prepared to accept a look-alike object from functions which are not defined in Python (see `c-api-monitoring`).

<div class="function">

get_local_events(tool_id: int, code: CodeType, /) -\> int

Returns all the `local events <monitoring-event-local>` for *code*

</div>

<div class="function">

set_local_events(tool_id: int, code: CodeType, event_set: int, /) -\> None

Activates all the `local events <monitoring-event-local>` for *code* which are set in *event_set*. Raises a `ValueError` if *tool_id* is not in use.

</div>

### Disabling events

<div class="data">

DISABLE

A special value that can be returned from a callback function to disable events for the current code location.

</div>

`Local events <monitoring-event-local>` can be disabled for a specific code location by returning `sys.monitoring.DISABLE` from a callback function. This does not change which events are set, or any other code locations for the same event.

`Other events <monitoring-event-global>` can be disabled on a per code object basis by returning `sys.monitoring.DISABLE` from a callback function. This disables the event for the entire code object (for the current tool).

Disabling events for specific locations is very important for high performance monitoring. For example, a program can be run under a debugger with no overhead if the debugger disables all monitoring except for a few breakpoints.

<div class="function">

restart_events() -\> None

Enable all the events that were disabled by `sys.monitoring.DISABLE` for all tools.

</div>

## Registering callback functions

<div class="function">

register_callback(tool_id: int, event: int, func: Callable \| None, /) -\> Callable \| None

Registers the callable *func* for the *event* with the given *tool_id*

If another callback was registered for the given *tool_id* and *event*, it is unregistered and returned. Otherwise `register_callback` returns `None`.

<div class="audit-event">

sys.monitoring.register_callback func sys.monitoring.register_callback

</div>

</div>

Functions can be unregistered by calling `sys.monitoring.register_callback(tool_id, event, None)`.

Callback functions can be registered and unregistered at any time.

Callbacks are called only once regardless if the event is turned on both globally and locally. As such, if an event could be turned on for both global and local events by your code then the callback needs to be written to handle either trigger.

### Callback function arguments

<div class="data">

MISSING

A special value that is passed to a callback function to indicate that there are no arguments to the call.

</div>

When an active event occurs, the registered callback function is called. Callback functions returning an object other than `DISABLE` will have no effect. Different events will provide the callback function with different arguments, as follows:

- `PY_START` and `PY_RESUME`:

      func(code: CodeType, instruction_offset: int) -> object

- `PY_RETURN` and `PY_YIELD`:

      func(code: CodeType, instruction_offset: int, retval: object) -> object

- `CALL`, `C_RAISE` and `C_RETURN` (*arg0* can be `MISSING` specifically):

      func(code: CodeType, instruction_offset: int, callable: object, arg0: object) -> object

  *code* represents the code object where the call is being made, while *callable* is the object that is about to be called (and thus triggered the event). If there are no arguments, *arg0* is set to `sys.monitoring.MISSING`.

  For instance methods, *callable* will be the function object as found on the class with *arg0* set to the instance (i.e. the `self` argument to the method).

- `RAISE`, `RERAISE`, `EXCEPTION_HANDLED`, `PY_UNWIND`, `PY_THROW` and `STOP_ITERATION`:

      func(code: CodeType, instruction_offset: int, exception: BaseException) -> object

- `LINE`:

      func(code: CodeType, line_number: int) -> object

- `BRANCH_LEFT`, `BRANCH_RIGHT` and `JUMP`:

      func(code: CodeType, instruction_offset: int, destination_offset: int) -> object

  Note that the *destination_offset* is where the code will next execute.

- `INSTRUCTION`:

      func(code: CodeType, instruction_offset: int) -> object

# The initialization of the `sys.path` module search path

A module search path is initialized when Python starts. This module search path may be accessed at `sys.path`.

The first entry in the module search path is the directory that contains the input script, if there is one. Otherwise, the first entry is the current directory, which is the case when executing the interactive shell, a `-c` command, or `-m` module.

The `PYTHONPATH` environment variable is often used to add directories to the search path. If this environment variable is found then the contents are added to the module search path.

> [!NOTE]
> `PYTHONPATH` will affect all installed Python versions/environments. Be wary of setting this in your shell profile or global environment variables. The `site` module offers more nuanced techniques as mentioned below.

The next items added are the directories containing standard Python modules as well as any `extension module`s that these modules depend on. Extension modules are `.pyd` files on Windows and `.so` files on other platforms. The directory with the platform-independent Python modules is called `prefix`. The directory with the extension modules is called `exec_prefix`.

The `PYTHONHOME` environment variable may be used to set the `prefix` and `exec_prefix` locations. Otherwise these directories are found by using the Python executable as a starting point and then looking for various 'landmark' files and directories. Note that any symbolic links are followed so the real Python executable location is used as the search starting point. The Python executable location is called `home`.

Once `home` is determined, the `prefix` directory is found by first looking for `python{majorversion}{minorversion}.zip` (`python311.zip`). On Windows the zip archive is searched for in `home` and on Unix the archive is expected to be in `lib`. Note that the expected zip archive location is added to the module search path even if the archive does not exist. If no archive was found, Python on Windows will continue the search for `prefix` by looking for `Lib\\os.py`. Python on Unix will look for `lib/python{majorversion}.{minorversion}/os.py` (`lib/python3.11/os.py`). On Windows `prefix` and `exec_prefix` are the same, however on other platforms `lib/python{majorversion}.{minorversion}/lib-dynload` (`lib/python3.11/lib-dynload`) is searched for and used as an anchor for `exec_prefix`. On some platforms `lib` may be `lib64` or another value, see `sys.platlibdir` and `PYTHONPLATLIBDIR`.

Once found, `prefix` and `exec_prefix` are available at `sys.base_prefix` and `sys.base_exec_prefix` respectively.

If `PYTHONHOME` is not set, and a `pyvenv.cfg` file is found alongside the main executable, or in its parent directory, `sys.prefix` and `sys.exec_prefix` get set to the directory containing `pyvenv.cfg`, otherwise they are set to the same value as `sys.base_prefix` and `sys.base_exec_prefix`, respectively. This is used by `sys-path-init-virtual-environments`.

Finally, the `site` module is processed and `site-packages` directories are added to the module search path. The `PYTHONUSERBASE` environment variable controls where is searched for user site-packages and the `PYTHONNOUSERSITE` environment variable prevents searching for user site-packages all together. A common way to customize the search path is to create `sitecustomize` or `usercustomize` modules as described in the `site` module documentation.

> [!NOTE]
> The command line options `-E`, `-P`, `-I`, `-S` and `-s` further affect path calculations, see their documentation for details.

<div class="versionchanged">

3.14

`sys.prefix` and `sys.exec_prefix` are now set to the `pyvenv.cfg` directory during the path initialization. This was previously done by `site`, therefore affected by `-S`.

</div>

## Virtual Environments

Virtual environments place a `pyvenv.cfg` file in their prefix, which causes `sys.prefix` and `sys.exec_prefix` to point to them, instead of the base installation.

The `prefix` and `exec_prefix` values of the base installation are available at `sys.base_prefix` and `sys.base_exec_prefix`.

As well as being used as a marker to identify virtual environments, `pyvenv.cfg` may also be used to configure the `site` initialization. Please refer to `site`'s `virtual environments documentation <site-virtual-environments-configuration>`.

> [!NOTE]
> `PYTHONHOME` overrides the `pyvenv.cfg` detection.

> [!NOTE]
> There are other ways "virtual environments" could be implemented. This documentation refers to implementations based on the `pyvenv.cfg` mechanism, such as `venv`, that many virtual environment implementations follow.

## <span id="pth">pth</span> files

To completely override `sys.path` create a `._pth` file with the same name as the shared library or executable (`python._pth` or `python311._pth`). The shared library path is always known on Windows, however it may not be available on other platforms. In the `._pth` file specify one line for each path to add to `sys.path`. The file based on the shared library name overrides the one based on the executable, which allows paths to be restricted for any program loading the runtime if desired.

When the file exists, all registry and environment variables are ignored, isolated mode is enabled, and `site` is not imported unless one line in the file specifies `import site`. Blank paths and lines starting with `#` are ignored. Each path may be absolute or relative to the location of the file. Import statements other than to `site` are not permitted, and arbitrary code cannot be specified.

Note that `.pth` files (without leading underscore) will be processed normally by the `site` module when `import site` has been specified.

## Embedded Python

If Python is embedded within another application `Py_InitializeFromConfig` and the `PyConfig` structure can be used to initialize Python. The path specific details are described at `init-path-config`.

<div class="seealso">

- `windows_finding_modules` for detailed Windows notes.
- `using-on-unix` for Unix details.

</div>

# `!sysconfig` --- Provide access to Python's configuration information

<div class="module" data-synopsis="Python's configuration information">

sysconfig

</div>

<div class="versionadded">

3.2

</div>

**Source code:** `Lib/sysconfig`

<div class="index">

single: configuration information

</div>

------------------------------------------------------------------------

The `!sysconfig` module provides access to Python's configuration information like the list of installation paths and the configuration variables relevant for the current platform.

## Configuration variables

A Python distribution contains a `Makefile` and a `pyconfig.h` header file that are necessary to build both the Python binary itself and third-party C extensions compiled using `setuptools`.

`!sysconfig` puts all variables found in these files in a dictionary that can be accessed using `get_config_vars` or `get_config_var`.

Notice that on Windows, it's a much smaller set.

<div class="function">

get_config_vars(\*args)

With no arguments, return a dictionary of all configuration variables relevant for the current platform.

With arguments, return a list of values that result from looking up each argument in the configuration variable dictionary.

For each argument, if the value is not found, return `None`.

</div>

<div class="function">

get_config_var(name)

Return the value of a single variable *name*. Equivalent to `get_config_vars().get(name)`.

If *name* is not found, return `None`.

</div>

Example of usage:

    >>> import sysconfig
    >>> sysconfig.get_config_var('Py_ENABLE_SHARED')
    0
    >>> sysconfig.get_config_var('LIBDIR')
    '/usr/local/lib'
    >>> sysconfig.get_config_vars('AR', 'CXX')
    ['ar', 'g++']

## Installation paths

Python uses an installation scheme that differs depending on the platform and on the installation options. These schemes are stored in `!sysconfig` under unique identifiers based on the value returned by `os.name`. The schemes are used by package installers to determine where to copy files to.

Python currently supports nine schemes:

- *posix_prefix*: scheme for POSIX platforms like Linux or macOS. This is the default scheme used when Python or a component is installed.
- *posix_home*: scheme for POSIX platforms, when the *home* option is used. This scheme defines paths located under a specific home prefix.
- *posix_user*: scheme for POSIX platforms, when the *user* option is used. This scheme defines paths located under the user's home directory (`site.USER_BASE`).
- *posix_venv*: scheme for `Python virtual environments <venv>` on POSIX platforms; by default it is the same as *posix_prefix*.
- *nt*: scheme for Windows. This is the default scheme used when Python or a component is installed.
- *nt_user*: scheme for Windows, when the *user* option is used.
- *nt_venv*: scheme for `Python virtual environments <venv>` on Windows; by default it is the same as *nt*.
- *venv*: a scheme with values from either *posix_venv* or *nt_venv* depending on the platform Python runs on.
- *osx_framework_user*: scheme for macOS, when the *user* option is used.

Each scheme is itself composed of a series of paths and each path has a unique identifier. Python currently uses eight paths:

- *stdlib*: directory containing the standard Python library files that are not platform-specific.
- *platstdlib*: directory containing the standard Python library files that are platform-specific.
- *platlib*: directory for site-specific, platform-specific files.
- *purelib*: directory for site-specific, non-platform-specific files ('pure' Python).
- *include*: directory for non-platform-specific header files for the Python C-API.
- *platinclude*: directory for platform-specific header files for the Python C-API.
- *scripts*: directory for script files.
- *data*: directory for data files.

## User scheme

This scheme is designed to be the most convenient solution for users that don't have write permission to the global site-packages directory or don't want to install into it.

Files will be installed into subdirectories of `site.USER_BASE` (written as `{userbase}` hereafter). This scheme installs pure Python modules and extension modules in the same location (also known as `site.USER_SITE`).

### `posix_user`

| Path         | Installation directory                     |
|--------------|--------------------------------------------|
| *stdlib*     | `{userbase}/lib/python{X.Y}`               |
| *platstdlib* | `{userbase}/lib/python{X.Y}`               |
| *platlib*    | `{userbase}/lib/python{X.Y}/site-packages` |
| *purelib*    | `{userbase}/lib/python{X.Y}/site-packages` |
| *include*    | `{userbase}/include/python{X.Y}`           |
| *scripts*    | `{userbase}/bin`                           |
| *data*       | `{userbase}`                               |

### `nt_user`

| Path         | Installation directory                  |
|--------------|-----------------------------------------|
| *stdlib*     | `{userbase}\\Python{XY}`                |
| *platstdlib* | `{userbase}\\Python{XY}`                |
| *platlib*    | `{userbase}\\Python{XY}\\site-packages` |
| *purelib*    | `{userbase}\\Python{XY}\\site-packages` |
| *include*    | `{userbase}\\Python{XY}\\Include`       |
| *scripts*    | `{userbase}\\Python{XY}\\Scripts`       |
| *data*       | `{userbase}`                            |

### `osx_framework_user`

| Path         | Installation directory                |
|--------------|---------------------------------------|
| *stdlib*     | `{userbase}/lib/python`               |
| *platstdlib* | `{userbase}/lib/python`               |
| *platlib*    | `{userbase}/lib/python/site-packages` |
| *purelib*    | `{userbase}/lib/python/site-packages` |
| *include*    | `{userbase}/include/python{X.Y}`      |
| *scripts*    | `{userbase}/bin`                      |
| *data*       | `{userbase}`                          |

## Home scheme

The idea behind the "home scheme" is that you build and maintain a personal stash of Python modules. This scheme's name is derived from the idea of a "home" directory on Unix, since it's not unusual for a Unix user to make their home directory have a layout similar to `/usr/` or `/usr/local/`. This scheme can be used by anyone, regardless of the operating system they are installing for.

### `posix_home`

| Path          | Installation directory  |
|---------------|-------------------------|
| *stdlib*      | `{home}/lib/python`     |
| *platstdlib*  | `{home}/lib/python`     |
| *platlib*     | `{home}/lib/python`     |
| *purelib*     | `{home}/lib/python`     |
| *include*     | `{home}/include/python` |
| *platinclude* | `{home}/include/python` |
| *scripts*     | `{home}/bin`            |
| *data*        | `{home}`                |

## Prefix scheme

The "prefix scheme" is useful when you wish to use one Python installation to perform the build/install (i.e., to run the setup script), but install modules into the third-party module directory of a different Python installation (or something that looks like a different Python installation). If this sounds a trifle unusual, it is---that's why the user and home schemes come before. However, there are at least two known cases where the prefix scheme will be useful.

First, consider that many Linux distributions put Python in `/usr`, rather than the more traditional `/usr/local`. This is entirely appropriate, since in those cases Python is part of "the system" rather than a local add-on. However, if you are installing Python modules from source, you probably want them to go in `/usr/local/lib/python2.{X}` rather than `/usr/lib/python2.{X}`.

Another possibility is a network filesystem where the name used to write to a remote directory is different from the name used to read it: for example, the Python interpreter accessed as `/usr/local/bin/python` might search for modules in `/usr/local/lib/python2.{X}`, but those modules would have to be installed to, say, `/mnt/{@server}/export/lib/python2.{X}`.

### `posix_prefix`

| Path          | Installation directory                   |
|---------------|------------------------------------------|
| *stdlib*      | `{prefix}/lib/python{X.Y}`               |
| *platstdlib*  | `{prefix}/lib/python{X.Y}`               |
| *platlib*     | `{prefix}/lib/python{X.Y}/site-packages` |
| *purelib*     | `{prefix}/lib/python{X.Y}/site-packages` |
| *include*     | `{prefix}/include/python{X.Y}`           |
| *platinclude* | `{prefix}/include/python{X.Y}`           |
| *scripts*     | `{prefix}/bin`                           |
| *data*        | `{prefix}`                               |

### `nt`

| Path          | Installation directory         |
|---------------|--------------------------------|
| *stdlib*      | `{prefix}\\Lib`                |
| *platstdlib*  | `{prefix}\\Lib`                |
| *platlib*     | `{prefix}\\Lib\\site-packages` |
| *purelib*     | `{prefix}\\Lib\\site-packages` |
| *include*     | `{prefix}\\Include`            |
| *platinclude* | `{prefix}\\Include`            |
| *scripts*     | `{prefix}\\Scripts`            |
| *data*        | `{prefix}`                     |

## Installation path functions

`!sysconfig` provides some functions to determine these installation paths.

<div class="function">

get_scheme_names()

Return a tuple containing all schemes currently supported in `!sysconfig`.

</div>

<div class="function">

get_default_scheme()

Return the default scheme name for the current platform.

<div class="versionadded">

3.10 This function was previously named `_get_default_scheme()` and considered an implementation detail.

</div>

<div class="versionchanged">

3.11 When Python runs from a virtual environment, the *venv* scheme is returned.

</div>

</div>

<div class="function">

get_preferred_scheme(key)

Return a preferred scheme name for an installation layout specified by *key*.

*key* must be either `"prefix"`, `"home"`, or `"user"`.

The return value is a scheme name listed in `get_scheme_names`. It can be passed to `!sysconfig` functions that take a *scheme* argument, such as `get_paths`.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.11 When Python runs from a virtual environment and `key="prefix"`, the *venv* scheme is returned.

</div>

</div>

<div class="function">

<span id="get_preferred_schemes">get_preferred_schemes</span>()

Return a dict containing preferred scheme names on the current platform. Python implementers and redistributors may add their preferred schemes to the `_INSTALL_SCHEMES` module-level global value, and modify this function to return those scheme names, to e.g. provide different schemes for system and language package managers to use, so packages installed by either do not mix with those by the other.

End users should not use this function, but `get_default_scheme` and `get_preferred_scheme` instead.

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

get_path_names()

Return a tuple containing all path names currently supported in `!sysconfig`.

</div>

<div class="function">

get_path(name, \[scheme, \[vars, \[expand\]\]\])

Return an installation path corresponding to the path *name*, from the install scheme named *scheme*.

*name* has to be a value from the list returned by `get_path_names`.

`!sysconfig` stores installation paths corresponding to each path name, for each platform, with variables to be expanded. For instance the *stdlib* path for the *nt* scheme is: `{base}/Lib`.

`get_path` will use the variables returned by `get_config_vars` to expand the path. All variables have default values for each platform so one may call this function and get the default value.

If *scheme* is provided, it must be a value from the list returned by `get_scheme_names`. Otherwise, the default scheme for the current platform is used.

If *vars* is provided, it must be a dictionary of variables that will update the dictionary returned by `get_config_vars`.

If *expand* is set to `False`, the path will not be expanded using the variables.

If *name* is not found, raise a `KeyError`.

</div>

<div class="function">

get_paths(\[scheme, \[vars, \[expand\]\]\])

Return a dictionary containing all installation paths corresponding to an installation scheme. See `get_path` for more information.

If *scheme* is not provided, will use the default scheme for the current platform.

If *vars* is provided, it must be a dictionary of variables that will update the dictionary used to expand the paths.

If *expand* is set to false, the paths will not be expanded.

If *scheme* is not an existing scheme, `get_paths` will raise a `KeyError`.

</div>

## Other functions

<div class="function">

get_python_version()

Return the `MAJOR.MINOR` Python version number as a string. Similar to `'%d.%d' % sys.version_info[:2]`.

</div>

<div class="function">

get_platform()

Return a string that identifies the current platform.

This is used mainly to distinguish platform-specific build directories and platform-specific built distributions. Typically includes the OS name and version and the architecture (as supplied by `os.uname`), although the exact information included depends on the OS; e.g., on Linux, the kernel version isn't particularly important.

Examples of returned values:

- linux-x86_64
- linux-aarch64
- solaris-2.6-sun4u

Windows:

- win-amd64 (64-bit Windows on AMD64, aka x86_64, Intel64, and EM64T)
- win-arm64 (64-bit Windows on ARM64, aka AArch64)
- win32 (all others - specifically, sys.platform is returned)

POSIX based OS:

- linux-x86_64
- macosx-15.5-arm64
- macosx-26.0-universal2 (macOS on Apple Silicon or Intel)
- android-24-arm64_v8a

For other non-POSIX platforms, currently just returns `sys.platform`.

</div>

<div class="function">

is_python_build()

Return `True` if the running Python interpreter was built from source and is being run from its built location, and not from a location resulting from e.g. running `make install` or installing via a binary installer.

</div>

<div class="function">

parse_config_h(fp\[, vars\])

Parse a `config.h`-style file.

*fp* is a file-like object pointing to the `config.h`-like file.

A dictionary containing name/value pairs is returned. If an optional dictionary is passed in as the second argument, it is used instead of a new dictionary, and updated with the values read in the file.

</div>

<div class="function">

get_config_h_filename()

Return the path of `pyconfig.h`.

</div>

<div class="function">

get_makefile_filename()

Return the path of `Makefile`.

</div>

## Command-line usage<span id="sysconfig-cli"></span>

You can use `!sysconfig` as a script with Python's *-m* option:

```
$ python -m sysconfig
Platform: "macosx-10.4-i386"
Python version: "3.2"
Current installation scheme: "posix_prefix"

Paths:
        data = "/usr/local"
        include = "/Users/tarek/Dev/svn.python.org/py3k/Include"
        platinclude = "."
        platlib = "/usr/local/lib/python3.2/site-packages"
        platstdlib = "/usr/local/lib/python3.2"
        purelib = "/usr/local/lib/python3.2/site-packages"
        scripts = "/usr/local/bin"
        stdlib = "/usr/local/lib/python3.2"

Variables:
        AC_APPLE_UNIVERSAL_BUILD = "0"
        AIX_GENUINE_CPLUSPLUS = "0"
        AR = "ar"
        ARFLAGS = "rc"
        ...
```

This call will print in the standard output the information returned by `get_platform`, `get_python_version`, `get_path` and `get_config_vars`.

# `!syslog` --- Unix syslog library routines

<div class="module" data-synopsis="An interface to the Unix syslog library routines.">

syslog

</div>

------------------------------------------------------------------------

This module provides an interface to the Unix `syslog` library routines. Refer to the Unix manual pages for a detailed description of the `syslog` facility.

<div class="availability">

Unix, not WASI, not iOS.

</div>

This module wraps the system `syslog` family of routines. A pure Python library that can speak to a syslog server is available in the `logging.handlers` module as `~logging.handlers.SysLogHandler`.

The module defines the following functions:

<div class="function">

syslog(message) syslog(priority, message)

Send the string *message* to the system logger. A trailing newline is added if necessary. Each message is tagged with a priority composed of a *facility* and a *level*. The optional *priority* argument, which defaults to `LOG_INFO`, determines the message priority. If the facility is not encoded in *priority* using logical-or (`LOG_INFO | LOG_USER`), the value given in the `openlog` call is used.

If `openlog` has not been called prior to the call to `~syslog.syslog`, `openlog` will be called with no arguments.

<div class="audit-event">

syslog.syslog priority,message syslog.syslog

</div>

<div class="versionchanged">

3.2 In previous versions, `openlog` would not be called automatically if it wasn't called prior to the call to `~syslog.syslog`, deferring to the syslog implementation to call `openlog()`.

</div>

<div class="versionchanged">

3.12 This function is restricted in subinterpreters. (Only code that runs in multiple interpreters is affected and the restriction is not relevant for most users.) `openlog` must be called in the main interpreter before `~syslog.syslog` may be used in a subinterpreter. Otherwise it will raise `RuntimeError`.

</div>

</div>

<div class="function">

openlog(\[ident\[, logoption\[, facility\]\]\])

Logging options of subsequent `~syslog.syslog` calls can be set by calling `openlog`. `~syslog.syslog` will call `openlog` with no arguments if the log is not currently open.

The optional *ident* keyword argument is a string which is prepended to every message, and defaults to `sys.argv[0]` with leading path components stripped. The optional *logoption* keyword argument (default is 0) is a bit field -- see below for possible values to combine. The optional *facility* keyword argument (default is `LOG_USER`) sets the default facility for messages which do not have a facility explicitly encoded.

<div class="audit-event">

syslog.openlog ident,logoption,facility syslog.openlog

</div>

<div class="versionchanged">

3.2 In previous versions, keyword arguments were not allowed, and *ident* was required.

</div>

<div class="versionchanged">

3.12 This function is restricted in subinterpreters. (Only code that runs in multiple interpreters is affected and the restriction is not relevant for most users.) This may only be called in the main interpreter. It will raise `RuntimeError` if called in a subinterpreter.

</div>

</div>

<div class="function">

closelog()

Reset the syslog module values and call the system library `closelog()`.

This causes the module to behave as it does when initially imported. For example, `openlog` will be called on the first `~syslog.syslog` call (if `openlog` hasn't already been called), and *ident* and other `openlog` parameters are reset to defaults.

<div class="audit-event">

syslog.closelog "" syslog.closelog

</div>

<div class="versionchanged">

3.12 This function is restricted in subinterpreters. (Only code that runs in multiple interpreters is affected and the restriction is not relevant for most users.) This may only be called in the main interpreter. It will raise `RuntimeError` if called in a subinterpreter.

</div>

</div>

<div class="function">

setlogmask(maskpri)

Set the priority mask to *maskpri* and return the previous mask value. Calls to `~syslog.syslog` with a priority level not set in *maskpri* are ignored. The default is to log all priorities. The function `LOG_MASK(pri)` calculates the mask for the individual priority *pri*. The function `LOG_UPTO(pri)` calculates the mask for all priorities up to and including *pri*.

<div class="audit-event">

syslog.setlogmask maskpri syslog.setlogmask

</div>

</div>

The module defines the following constants:

<div class="data">

LOG_EMERG LOG_ALERT LOG_CRIT LOG_ERR LOG_WARNING LOG_NOTICE LOG_INFO LOG_DEBUG

Priority levels (high to low).

</div>

<div class="data">

LOG_AUTH LOG_AUTHPRIV LOG_CRON LOG_DAEMON LOG_FTP LOG_INSTALL LOG_KERN LOG_LAUNCHD LOG_LPR LOG_MAIL LOG_NETINFO LOG_NEWS LOG_RAS LOG_REMOTEAUTH LOG_SYSLOG LOG_USER LOG_UUCP LOG_LOCAL0 LOG_LOCAL1 LOG_LOCAL2 LOG_LOCAL3 LOG_LOCAL4 LOG_LOCAL5 LOG_LOCAL6 LOG_LOCAL7

Facilities, depending on availability in `<syslog.h>` for `LOG_AUTHPRIV`, `LOG_FTP`, `LOG_NETINFO`, `LOG_REMOTEAUTH`, `LOG_INSTALL` and `LOG_RAS`.

<div class="versionchanged">

3.13 Added `LOG_FTP`, `LOG_NETINFO`, `LOG_REMOTEAUTH`, `LOG_INSTALL`, `LOG_RAS`, and `LOG_LAUNCHD`.

</div>

</div>

<div class="data">

LOG_PID LOG_CONS LOG_NDELAY LOG_ODELAY LOG_NOWAIT LOG_PERROR

Log options, depending on availability in `<syslog.h>` for `LOG_ODELAY`, `LOG_NOWAIT` and `LOG_PERROR`.

</div>

## Examples

### Simple example

A simple set of examples:

    import syslog

    syslog.syslog('Processing started')
    if error:
        syslog.syslog(syslog.LOG_ERR, 'Processing started')

An example of setting some log options, these would include the process ID in logged messages, and write the messages to the destination facility used for mail logging:

    syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_MAIL)
    syslog.syslog('E-mail processing initiated...')

# `!tabnanny` --- Detection of ambiguous indentation

<div class="module" data-synopsis="Tool for detecting white space related problems in Python
source files in a directory tree.">

tabnanny

</div>

**Source code:** `Lib/tabnanny.py`

------------------------------------------------------------------------

For the time being this module is intended to be called as a script. However it is possible to import it into an IDE and use the function `check` described below.

> [!NOTE]
> The API provided by this module is likely to change in future releases; such changes may not be backward compatible.

<div class="function">

check(file_or_dir)

If *file_or_dir* is a directory and not a symbolic link, then recursively descend the directory tree named by *file_or_dir*, checking all `.py` files along the way. If *file_or_dir* is an ordinary Python source file, it is checked for whitespace related problems. The diagnostic messages are written to standard output using the `print` function.

</div>

<div class="data">

verbose

Flag indicating whether to print verbose messages. This is incremented by the `-v` option if called as a script.

</div>

<div class="data">

filename_only

Flag indicating whether to print only the filenames of files containing whitespace related problems. This is set to true by the `-q` option if called as a script.

</div>

<div class="exception">

NannyNag

Raised by `process_tokens` if detecting an ambiguous indent. Captured and handled in `check`.

</div>

<div class="function">

process_tokens(tokens)

This function is used by `check` to process tokens generated by the `tokenize` module.

</div>

<div class="seealso">

Module `tokenize`  
Lexical scanner for Python source code.

</div>

# `!tarfile` --- Read and write tar archive files

<div class="module" data-synopsis="Read and write tar-format archive files.">

tarfile

</div>

**Source code:** `Lib/tarfile.py`

------------------------------------------------------------------------

The `!tarfile` module makes it possible to read and write tar archives, including those using gzip, bz2 and lzma compression. Use the `zipfile` module to read or write `.zip` files, or the higher-level functions in `shutil <archiving-operations>`.

Some facts and figures:

- reads and writes `gzip`, `bz2`, `compression.zstd`, and `lzma` compressed archives if the respective modules are available.

  If any of these `optional modules <optional module>` are missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

- read/write support for the POSIX.1-1988 (ustar) format.

- read/write support for the GNU tar format including *longname* and *longlink* extensions, read-only support for all variants of the *sparse* extension including restoration of sparse files.

- read/write support for the POSIX.1-2001 (pax) format.

- handles directories, regular files, hardlinks, symbolic links, fifos, character devices and block devices and is able to acquire and restore file information like timestamp, access permissions and owner.

<div class="versionchanged">

3.3 Added support for `lzma` compression.

</div>

<div class="versionchanged">

3.12 Archives are extracted using a `filter <tarfile-extraction-filter>`, which makes it possible to either limit surprising/dangerous features, or to acknowledge that they are expected and the archive is fully trusted.

</div>

<div class="versionchanged">

3.14 Set the default extraction filter to `data <data_filter>`, which disallows some dangerous features such as links to absolute paths or paths outside of the destination. Previously, the filter strategy was equivalent to `fully_trusted <fully_trusted_filter>`.

</div>

<div class="versionchanged">

3.14

Added support for Zstandard compression using `compression.zstd`.

</div>

<div class="function">

open(name=None, mode='r', fileobj=None, bufsize=10240, \*\*kwargs)

Return a `TarFile` object for the pathname *name*. For detailed information on `TarFile` objects and the keyword arguments that are allowed, see `tarfile-objects`.

*mode* has to be a string of the form `'filemode[:compression]'`, it defaults to `'r'`. Here is a full list of mode combinations:

| mode | action |
|----|----|
| `'r'` or `'r:*'` | Open for reading with transparent compression (recommended). |
| `'r:'` | Open for reading exclusively without compression. |
| `'r:gz'` | Open for reading with gzip compression. |
| `'r:bz2'` | Open for reading with bzip2 compression. |
| `'r:xz'` | Open for reading with lzma compression. |
| `'r:zst'` | Open for reading with Zstandard compression. |
| `'x'` or `'x:'` | Create a tarfile exclusively without compression. Raise a `FileExistsError` exception if it already exists. |
| `'x:gz'` | Create a tarfile with gzip compression. Raise a `FileExistsError` exception if it already exists. |
| `'x:bz2'` | Create a tarfile with bzip2 compression. Raise a `FileExistsError` exception if it already exists. |
| `'x:xz'` | Create a tarfile with lzma compression. Raise a `FileExistsError` exception if it already exists. |
| `'x:zst'` | Create a tarfile with Zstandard compression. Raise a `FileExistsError` exception if it already exists. |
| `'a'` or `'a:'` | Open for appending with no compression. The file is created if it does not exist. |
| `'w'` or `'w:'` | Open for uncompressed writing. |
| `'w:gz'` | Open for gzip compressed writing. |
| `'w:bz2'` | Open for bzip2 compressed writing. |
| `'w:xz'` | Open for lzma compressed writing. |
| `'w:zst'` | Open for Zstandard compressed writing. |

Note that `'a:gz'`, `'a:bz2'` or `'a:xz'` is not possible. If *mode* is not suitable to open a certain (compressed) file for reading, `ReadError` is raised. Use *mode* `'r'` to avoid this. If a compression method is not supported, `CompressionError` is raised.

> [!NOTE]
> Compressed archives opened with modes like `'r:gz'`, `'r:bz2'`, `'r:xz'`, or `'r:zst'` support random access, but seeking backwards in the underlying compressed stream requires re-decompressing from the beginning. Extracting members in a different order than they appear in the archive can therefore be significantly slower — proportional to the total compressed data read rather than just the target member's size. For best performance, extract members in archive order or use `TarFile.extractall`.

If *fileobj* is specified, it is used as an alternative to a `file object` opened in binary mode for *name*. It is supposed to be at position 0.

For modes `'w:gz'`, `'x:gz'`, `'w|gz'`, `'w:bz2'`, `'x:bz2'`, `'w|bz2'`, `tarfile.open` accepts the keyword argument *compresslevel* (default `6`) to specify the compression level of the file.

For modes `'w:xz'`, `'x:xz'` and `'w|xz'`, `tarfile.open` accepts the keyword argument *preset* to specify the compression level of the file.

For modes `'w:zst'`, `'x:zst'` and `'w|zst'`, `tarfile.open` accepts the keyword argument *level* to specify the compression level of the file. The keyword argument *options* may also be passed, providing advanced Zstandard compression parameters described by `~compression.zstd.CompressionParameter`. The keyword argument *zstd_dict* can be passed to provide a `~compression.zstd.ZstdDict`, a Zstandard dictionary used to improve compression of smaller amounts of data.

For modes `'w:gz'` and `'w|gz'`, `tarfile.open` accepts the keyword argument *mtime* to create a gzip archive header with that mtime. By default, the mtime is set to the time of creation of the archive. Use *mtime* `0` to generate a compressed stream that does not depend on creation time, for reproducible output.

For special purposes, there is a second format for *mode*: `'filemode|[compression]'`. `tarfile.open` will return a `TarFile` object that processes its data as a stream of blocks. No random seeking will be done on the file. If given, *fileobj* may be any object that has a `~io.RawIOBase.read` or `~io.RawIOBase.write` method (depending on the *mode*) that works with bytes. *bufsize* specifies the blocksize and defaults to `20 * 512` bytes. Use this variant in combination with e.g. `sys.stdin.buffer`, a socket `file object` or a tape device. However, such a `TarFile` object is limited in that it does not allow random access, see `tar-examples`. The currently possible modes:

| Mode | Action |
|----|----|
| `'r|*'` | Open a *stream* of tar blocks for reading with transparent compression. |
| `'r|'` | Open a *stream* of uncompressed tar blocks for reading. |
| `'r|gz'` | Open a gzip compressed *stream* for reading. |
| `'r|bz2'` | Open a bzip2 compressed *stream* for reading. |
| `'r|xz'` | Open an lzma compressed *stream* for reading. |
| `'r|zst'` | Open a Zstandard compressed *stream* for reading. |
| `'w|'` | Open an uncompressed *stream* for writing. |
| `'w|gz'` | Open a gzip compressed *stream* for writing. |
| `'w|bz2'` | Open a bzip2 compressed *stream* for writing. |
| `'w|xz'` | Open an lzma compressed *stream* for writing. |
| `'w|zst'` | Open a Zstandard compressed *stream* for writing. |

<div class="versionchanged">

3.5 The `'x'` (exclusive creation) mode was added.

</div>

<div class="versionchanged">

3.6 The *name* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.12 The *compresslevel* keyword argument also works for streams.

</div>

<div class="versionchanged">

3.14 The *preset* keyword argument also works for streams.

</div>

<div class="versionchanged">

3.15 The default compression level was reduced to 6 (down from 9). It is the default level used by most compression tools and a better tradeoff between speed and performance.

</div>

</div>

<div class="TarFile" data-noindex="">

Class for reading and writing tar archives. Do not use this class directly: use `tarfile.open` instead. See `tarfile-objects`.

</div>

<div class="function">

is_tarfile(name)

Return `True` if *name* is a tar archive file, that the `!tarfile` module can read. *name* may be a `str`, file, or file-like object.

<div class="versionchanged">

3.9 Support for file and file-like objects.

</div>

</div>

The `!tarfile` module defines the following exceptions:

<div class="exception">

TarError

Base class for all `!tarfile` exceptions.

</div>

<div class="exception">

ReadError

Is raised when a tar archive is opened, that either cannot be handled by the `!tarfile` module or is somehow invalid.

</div>

<div class="exception">

CompressionError

Is raised when a compression method is not supported or when the data cannot be decoded properly.

</div>

<div class="exception">

StreamError

Is raised for the limitations that are typical for stream-like `TarFile` objects.

</div>

<div class="exception">

ExtractError

Is raised for *non-fatal* errors when using `TarFile.extract`, but only if `TarFile.errorlevel``== 2`.

</div>

<div class="exception">

HeaderError

Is raised by `TarInfo.frombuf` if the buffer it gets is invalid.

</div>

<div class="exception">

FilterError

Base class for members `refused <tarfile-extraction-refuse>` by filters.

<div class="attribute">

tarinfo

Information about the member that the filter refused to extract, as `TarInfo <tarinfo-objects>`.

</div>

</div>

<div class="exception">

AbsolutePathError

Raised to refuse extracting a member with an absolute path.

</div>

<div class="exception">

OutsideDestinationError

Raised to refuse extracting a member outside the destination directory.

</div>

<div class="exception">

SpecialFileError

Raised to refuse extracting a special file (e.g. a device or pipe).

</div>

<div class="exception">

AbsoluteLinkError

Raised to refuse extracting a symbolic link with an absolute path.

</div>

<div class="exception">

LinkOutsideDestinationError

Raised to refuse extracting a symbolic link pointing outside the destination directory.

</div>

<div class="exception">

LinkFallbackError

Raised to refuse emulating a link (hard or symbolic) by extracting another archive member, when that member would be rejected by the filter location. The exception that was raised to reject the replacement member is available as `!BaseException.__context__`.

<div class="versionadded">

3.15

</div>

</div>

The following constants are available at the module level:

<div class="data">

ENCODING

The default character encoding: `'utf-8'` on Windows, the value returned by `sys.getfilesystemencoding` otherwise.

</div>

<div class="data">

REGTYPE AREGTYPE

A regular file `~TarInfo.type`.

</div>

<div class="data">

LNKTYPE

A link (inside tarfile) `~TarInfo.type`.

</div>

<div class="data">

SYMTYPE

A symbolic link `~TarInfo.type`.

</div>

<div class="data">

CHRTYPE

A character special device `~TarInfo.type`.

</div>

<div class="data">

BLKTYPE

A block special device `~TarInfo.type`.

</div>

<div class="data">

DIRTYPE

A directory `~TarInfo.type`.

</div>

<div class="data">

FIFOTYPE

A FIFO special device `~TarInfo.type`.

</div>

<div class="data">

CONTTYPE

A contiguous file `~TarInfo.type`.

</div>

<div class="data">

GNUTYPE_LONGNAME

A GNU tar longname `~TarInfo.type`.

</div>

<div class="data">

GNUTYPE_LONGLINK

A GNU tar longlink `~TarInfo.type`.

</div>

<div class="data">

GNUTYPE_SPARSE

A GNU tar sparse file `~TarInfo.type`.

</div>

Each of the following constants defines a tar archive format that the `!tarfile` module is able to create. See section `tar-formats` for details.

<div class="data">

USTAR_FORMAT

POSIX.1-1988 (ustar) format.

</div>

<div class="data">

GNU_FORMAT

GNU tar format.

</div>

<div class="data">

PAX_FORMAT

POSIX.1-2001 (pax) format.

</div>

<div class="data">

DEFAULT_FORMAT

The default format for creating archives. This is currently `PAX_FORMAT`.

<div class="versionchanged">

3.8 The default format for new archives was changed to `PAX_FORMAT` from `GNU_FORMAT`.

</div>

</div>

<div class="seealso">

Module `zipfile`  
Documentation of the `zipfile` standard module.

`archiving-operations`  
Documentation of the higher-level archiving facilities provided by the standard `shutil` module.

[GNU tar manual, Basic Tar Format](https://www.gnu.org/software/tar/manual/html_node/Standard.html)  
Documentation for tar archive files, including GNU tar extensions.

</div>

## TarFile Objects

The `TarFile` object provides an interface to a tar archive. A tar archive is a sequence of blocks. An archive member (a stored file) is made up of a header block followed by data blocks. It is possible to store a file in a tar archive several times. Each archive member is represented by a `TarInfo` object, see `tarinfo-objects` for details.

A `TarFile` object can be used as a context manager in a `with` statement. It will automatically be closed when the block is completed. Please note that in the event of an exception an archive opened for writing will not be finalized; only the internally used file object will be closed. See the `tar-examples` section for a use case.

<div class="versionadded">

3.2 Added support for the context management protocol.

</div>

<div class="TarFile(name=None, mode='r', fileobj=None, format=DEFAULT_FORMAT, tarinfo=TarInfo, dereference=False, ignore_zeros=False, encoding=ENCODING, errors='surrogateescape', pax_headers=None, debug=0, errorlevel=1, stream=False)">

All following arguments are optional and can be accessed as instance attributes as well.

*name* is the pathname of the archive. *name* may be a `path-like object`. It can be omitted if *fileobj* is given. In this case, the file object's `!name` attribute is used if it exists.

*mode* is either `'r'` to read from an existing archive, `'a'` to append data to an existing file, `'w'` to create a new file overwriting an existing one, or `'x'` to create a new file only if it does not already exist.

If *fileobj* is given, it is used for reading or writing data. If it can be determined, *mode* is overridden by *fileobj*'s mode. *fileobj* will be used from position 0.

> [!NOTE]
> *fileobj* is not closed, when `TarFile` is closed.

*format* controls the archive format for writing. It must be one of the constants `USTAR_FORMAT`, `GNU_FORMAT` or `PAX_FORMAT` that are defined at module level. When reading, format will be automatically detected, even if different formats are present in a single archive.

The *tarinfo* argument can be used to replace the default `TarInfo` class with a different one.

If *dereference* is `False`, add symbolic and hard links to the archive. If it is `True`, add the content of the target files to the archive. This has no effect on systems that do not support symbolic links.

If *ignore_zeros* is `False`, treat an empty block as the end of the archive. If it is `True`, skip empty (and invalid) blocks and try to get as many members as possible. This is only useful for reading concatenated or damaged archives.

*debug* can be set from `0` (no debug messages) up to `3` (all debug messages). The messages are written to `sys.stderr`.

*errorlevel* controls how extraction errors are handled, see `the corresponding attribute <TarFile.errorlevel>`.

The *encoding* and *errors* arguments define the character encoding to be used for reading or writing the archive and how conversion errors are going to be handled. The default settings will work for most users. See section `tar-unicode` for in-depth information.

The *pax_headers* argument is an optional dictionary of strings which will be added as a pax global header if *format* is `PAX_FORMAT`.

If *stream* is set to `True` then while reading the archive info about files in the archive are not cached, saving memory.

<div class="versionchanged">

3.2 Use `'surrogateescape'` as the default for the *errors* argument.

</div>

<div class="versionchanged">

3.5 The `'x'` (exclusive creation) mode was added.

</div>

<div class="versionchanged">

3.6 The *name* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.13 Add the *stream* parameter.

</div>

</div>

<div class="classmethod">

TarFile.open(...)

Alternative constructor. The `tarfile.open` function is actually a shortcut to this classmethod.

</div>

<div class="method">

TarFile.getmember(name)

Return a `TarInfo` object for member *name*. If *name* can not be found in the archive, `KeyError` is raised.

> [!NOTE]
> If a member occurs more than once in the archive, its last occurrence is assumed to be the most up-to-date version.

</div>

<div class="method">

TarFile.getmembers()

Return the members of the archive as a list of `TarInfo` objects. The list has the same order as the members in the archive.

</div>

<div class="method">

TarFile.getnames()

Return the members as a list of their names. It has the same order as the list returned by `getmembers`.

</div>

<div class="method">

TarFile.list(verbose=True, \*, members=None)

Print a table of contents to `sys.stdout`. If *verbose* is `False`, only the names of the members are printed. If it is `True`, output similar to that of `ls -l` is produced. If optional *members* is given, it must be a subset of the list returned by `getmembers`.

<div class="versionchanged">

3.5 Added the *members* parameter.

</div>

</div>

<div class="method">

TarFile.next()

Return the next member of the archive as a `TarInfo` object, when `TarFile` is opened for reading. Return `None` if there is no more available.

</div>

<div class="method">

TarFile.extractall(path=".", members=None, \*, numeric_owner=False, filter=None)

Extract all members from the archive to the current working directory or directory *path*. If optional *members* is given, it must be a subset of the list returned by `getmembers`. Directory information like owner, modification time and permissions are set after all members have been extracted. This is done to work around two problems: A directory's modification time is reset each time a file is created in it. And, if a directory's permissions do not allow writing, extracting files to it will fail.

If *numeric_owner* is `True`, the uid and gid numbers from the tarfile are used to set the owner/group for the extracted files. Otherwise, the named values from the tarfile are used.

The *filter* argument specifies how `members` are modified or rejected before extraction. See `tarfile-extraction-filter` for details. It is recommended to set this explicitly only if specific *tar* features are required, or as `filter='data'` to support Python versions with a less secure default (3.13 and lower).

> [!WARNING]
> Never extract archives from untrusted sources without prior inspection.
>
> Since Python 3.14, the default (`data <data_filter>`) will prevent the most dangerous security issues. However, it will not prevent *all* unintended or insecure behavior. Read the `tarfile-extraction-filter` section for details.

<div class="versionchanged">

3.5 Added the *numeric_owner* parameter.

</div>

<div class="versionchanged">

3.6 The *path* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.12 Added the *filter* parameter.

</div>

<div class="versionchanged">

3.14 The *filter* parameter now defaults to `'data'`.

</div>

</div>

<div class="method">

TarFile.extract(member, path="", set_attrs=True, \*, numeric_owner=False, filter=None)

Extract a member from the archive to the current working directory, using its full name. Its file information is extracted as accurately as possible. *member* may be a filename or a `TarInfo` object. You can specify a different directory using *path*. *path* may be a `path-like object`. File attributes (owner, mtime, mode) are set unless *set_attrs* is false.

The *numeric_owner* and *filter* arguments are the same as for `extractall`.

> [!NOTE]
> The `extract` method does not take care of several extraction issues. In most cases you should consider using the `extractall` method.

> [!WARNING]
> Never extract archives from untrusted sources without prior inspection. See the warning for `extractall` for details.

<div class="versionchanged">

3.2 Added the *set_attrs* parameter.

</div>

<div class="versionchanged">

3.5 Added the *numeric_owner* parameter.

</div>

<div class="versionchanged">

3.6 The *path* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.12 Added the *filter* parameter.

</div>

</div>

<div class="method">

TarFile.extractfile(member)

Extract a member from the archive as a file object. *member* may be a filename or a `TarInfo` object. If *member* is a regular file or a link, an `io.BufferedReader` object is returned. For all other existing members, `None` is returned. If *member* does not appear in the archive, `KeyError` is raised.

<div class="versionchanged">

3.3 Return an `io.BufferedReader` object.

</div>

<div class="versionchanged">

3.13 The returned `io.BufferedReader` object has the `!mode` attribute which is always equal to `'rb'`.

</div>

</div>

<div class="attribute" type="int">

TarFile.errorlevel

If *errorlevel* is `0`, errors are ignored when using `TarFile.extract` and `TarFile.extractall`. Nevertheless, they appear as error messages in the debug output when *debug* is greater than 0. If `1` (the default), all *fatal* errors are raised as `OSError` or `FilterError` exceptions. If `2`, all *non-fatal* errors are raised as `TarError` exceptions as well.

Some exceptions, e.g. ones caused by wrong argument types or data corruption, are always raised.

Custom `extraction filters <tarfile-extraction-filter>` should raise `FilterError` for *fatal* errors and `ExtractError` for *non-fatal* ones.

Note that when an exception is raised, the archive may be partially extracted. It is the user’s responsibility to clean up.

</div>

<div class="attribute">

TarFile.extraction_filter

<div class="versionadded">

3.12

</div>

The `extraction filter <tarfile-extraction-filter>` used as a default for the *filter* argument of `~TarFile.extract` and `~TarFile.extractall`.

The attribute may be `None` or a callable. String names are not allowed for this attribute, unlike the *filter* argument to `~TarFile.extract`.

If `extraction_filter` is `None` (the default), extraction methods will use the `data <data_filter>` filter by default.

The attribute may be set on instances or overridden in subclasses. It also is possible to set it on the `TarFile` class itself to set a global default, although, since it affects all uses of *tarfile*, it is best practice to only do so in top-level applications or `site configuration <site>`. To set a global default this way, a filter function needs to be wrapped in `staticmethod` to prevent injection of a `self` argument.

<div class="versionchanged">

3.14

The default filter is set to `data <data_filter>`, which disallows some dangerous features such as links to absolute paths or paths outside of the destination. Previously, the default was equivalent to `fully_trusted <fully_trusted_filter>`.

</div>

</div>

<div class="method">

TarFile.add(name, arcname=None, recursive=True, \*, filter=None)

Add the file *name* to the archive. *name* may be any type of file (directory, fifo, symbolic link, etc.). If given, *arcname* specifies an alternative name for the file in the archive. Directories are added recursively by default. This can be avoided by setting *recursive* to `False`. Recursion adds entries in sorted order. If *filter* is given, it should be a function that takes a `TarInfo` object argument and returns the changed `TarInfo` object. If it instead returns `None` the `TarInfo` object will be excluded from the archive. See `tar-examples` for an example.

<div class="versionchanged">

3.2 Added the *filter* parameter.

</div>

<div class="versionchanged">

3.7 Recursion adds entries in sorted order.

</div>

</div>

<div class="method">

TarFile.addfile(tarinfo, fileobj=None)

Add the `TarInfo` object *tarinfo* to the archive. If *tarinfo* represents a non zero-size regular file, the *fileobj* argument should be a `binary file`, and `tarinfo.size` bytes are read from it and added to the archive. You can create `TarInfo` objects directly, or by using `gettarinfo`.

<div class="versionchanged">

3.13

*fileobj* must be given for non-zero-sized regular files.

</div>

</div>

<div class="method">

TarFile.gettarinfo(name=None, arcname=None, fileobj=None)

Create a `TarInfo` object from the result of `os.stat` or equivalent on an existing file. The file is either named by *name*, or specified as a `file object` *fileobj* with a file descriptor. *name* may be a `path-like object`. If given, *arcname* specifies an alternative name for the file in the archive, otherwise, the name is taken from *fileobj*’s `~io.FileIO.name` attribute, or the *name* argument. The name should be a text string.

You can modify some of the `TarInfo`’s attributes before you add it using `addfile`. If the file object is not an ordinary file object positioned at the beginning of the file, attributes such as `~TarInfo.size` may need modifying. This is the case for objects such as `~gzip.GzipFile`. The `~TarInfo.name` may also be modified, in which case *arcname* could be a dummy string.

<div class="versionchanged">

3.6 The *name* parameter accepts a `path-like object`.

</div>

</div>

<div class="method">

TarFile.close()

Close the `TarFile`. In write mode, two finishing zero blocks are appended to the archive.

</div>

<div class="attribute" type="dict">

TarFile.pax_headers

A dictionary containing key-value pairs of pax global headers.

</div>

## TarInfo Objects

A `TarInfo` object represents one member in a `TarFile`. Aside from storing all required attributes of a file (like file type, size, time, permissions, owner etc.), it provides some useful methods to determine its type. It does *not* contain the file's data itself.

`TarInfo` objects are returned by `TarFile`'s methods `~TarFile.getmember`, `~TarFile.getmembers` and `~TarFile.gettarinfo`.

Modifying the objects returned by `~TarFile.getmember` or `~TarFile.getmembers` will affect all subsequent operations on the archive. For cases where this is unwanted, you can use `copy.copy() <copy>` or call the `~TarInfo.replace` method to create a modified copy in one step.

Several attributes can be set to `None` to indicate that a piece of metadata is unused or unknown. Different `TarInfo` methods handle `None` differently:

- The `~TarFile.extract` or `~TarFile.extractall` methods will ignore the corresponding metadata, leaving it set to a default.
- `~TarFile.addfile` will fail.
- `~TarFile.list` will print a placeholder string.

<div class="TarInfo(name=&quot;&quot;)">

Create a `TarInfo` object.

</div>

<div class="classmethod">

TarInfo.frombuf(buf, encoding, errors)

Create and return a `TarInfo` object from string buffer *buf*.

Raises `HeaderError` if the buffer is invalid.

</div>

<div class="classmethod">

TarInfo.fromtarfile(tarfile)

Read the next member from the `TarFile` object *tarfile* and return it as a `TarInfo` object.

</div>

<div class="method">

TarInfo.tobuf(format=DEFAULT_FORMAT, encoding=ENCODING, errors='surrogateescape')

Create a string buffer from a `TarInfo` object. For information on the arguments see the constructor of the `TarFile` class.

<div class="versionchanged">

3.2 Use `'surrogateescape'` as the default for the *errors* argument.

</div>

</div>

A `TarInfo` object has the following public data attributes:

<div class="attribute" type="str">

TarInfo.name

Name of the archive member.

</div>

<div class="attribute" type="int">

TarInfo.size

Size in bytes.

</div>

<div class="attribute" type="int | float">

TarInfo.mtime

Time of last modification in seconds since the `epoch <epoch>`, as in `os.stat_result.st_mtime`.

<div class="versionchanged">

3.12

Can be set to `None` for `~TarFile.extract` and `~TarFile.extractall`, causing extraction to skip applying this attribute.

</div>

</div>

<div class="attribute" type="int">

TarInfo.mode

Permission bits, as for `os.chmod`.

<div class="versionchanged">

3.12

Can be set to `None` for `~TarFile.extract` and `~TarFile.extractall`, causing extraction to skip applying this attribute.

</div>

</div>

<div class="attribute">

TarInfo.type

File type. *type* is usually one of these constants: `REGTYPE`, `AREGTYPE`, `LNKTYPE`, `SYMTYPE`, `DIRTYPE`, `FIFOTYPE`, `CONTTYPE`, `CHRTYPE`, `BLKTYPE`, `GNUTYPE_SPARSE`. To determine the type of a `TarInfo` object more conveniently, use the `is*()` methods below.

</div>

<div class="attribute" type="str">

TarInfo.linkname

Name of the target file name, which is only present in `TarInfo` objects of type `LNKTYPE` and `SYMTYPE`.

For symbolic links (`SYMTYPE`), the *linkname* is relative to the directory that contains the link. For hard links (`LNKTYPE`), the *linkname* is relative to the root of the archive.

</div>

<div class="attribute" type="int">

TarInfo.uid

User ID of the user who originally stored this member.

<div class="versionchanged">

3.12

Can be set to `None` for `~TarFile.extract` and `~TarFile.extractall`, causing extraction to skip applying this attribute.

</div>

</div>

<div class="attribute" type="int">

TarInfo.gid

Group ID of the user who originally stored this member.

<div class="versionchanged">

3.12

Can be set to `None` for `~TarFile.extract` and `~TarFile.extractall`, causing extraction to skip applying this attribute.

</div>

</div>

<div class="attribute" type="str">

TarInfo.uname

User name.

<div class="versionchanged">

3.12

Can be set to `None` for `~TarFile.extract` and `~TarFile.extractall`, causing extraction to skip applying this attribute.

</div>

</div>

<div class="attribute" type="str">

TarInfo.gname

Group name.

<div class="versionchanged">

3.12

Can be set to `None` for `~TarFile.extract` and `~TarFile.extractall`, causing extraction to skip applying this attribute.

</div>

</div>

<div class="attribute" type="int">

TarInfo.chksum

Header checksum.

</div>

<div class="attribute" type="int">

TarInfo.devmajor

Device major number.

</div>

<div class="attribute" type="int">

TarInfo.devminor

Device minor number.

</div>

<div class="attribute" type="int">

TarInfo.offset

The tar header starts here.

</div>

<div class="attribute" type="int">

TarInfo.offset_data

The file's data starts here.

</div>

<div class="attribute">

TarInfo.sparse

Sparse member information.

</div>

<div class="attribute" type="dict">

TarInfo.pax_headers

A dictionary containing key-value pairs of an associated pax extended header.

</div>

<div class="method">

TarInfo.replace(name=..., mtime=..., mode=..., linkname=..., uid=..., gid=..., uname=..., gname=..., deep=True)

<div class="versionadded">

3.12

</div>

Return a *new* copy of the `!TarInfo` object with the given attributes changed. For example, to return a `TarInfo` with the group name set to `'staff'`, use:

    new_tarinfo = old_tarinfo.replace(gname='staff')

By default, a deep copy is made. If *deep* is false, the copy is shallow, i.e. `pax_headers` and any custom attributes are shared with the original `TarInfo` object.

This method is also used by `copy.replace`.

<div class="versionchanged">

3.16 Added support for `copy.replace`.

</div>

</div>

A `TarInfo` object also provides some convenient query methods:

<div class="method">

TarInfo.isfile()

Return `True` if the `TarInfo` object is a regular file.

</div>

<div class="method">

TarInfo.isreg()

Same as `isfile`.

</div>

<div class="method">

TarInfo.isdir()

Return `True` if it is a directory.

</div>

<div class="method">

TarInfo.issym()

Return `True` if it is a symbolic link.

</div>

<div class="method">

TarInfo.islnk()

Return `True` if it is a hard link.

</div>

<div class="method">

TarInfo.ischr()

Return `True` if it is a character device.

</div>

<div class="method">

TarInfo.isblk()

Return `True` if it is a block device.

</div>

<div class="method">

TarInfo.isfifo()

Return `True` if it is a FIFO.

</div>

<div class="method">

TarInfo.isdev()

Return `True` if it is one of character device, block device or FIFO.

</div>

## Extraction filters

<div class="versionadded">

3.12

</div>

The *tar* format is designed to capture all details of a UNIX-like filesystem, which makes it very powerful. Unfortunately, the features make it easy to create tar files that have unintended -- and possibly malicious -- effects when extracted. For example, extracting a tar file can overwrite arbitrary files in various ways (e.g. by using absolute paths, `..` path components, or symlinks that affect later members).

In most cases, the full functionality is not needed. Therefore, *tarfile* supports extraction filters: a mechanism to limit functionality, and thus mitigate some of the security issues.

> [!WARNING]
> None of the available filters blocks *all* dangerous archive features. Never extract archives from untrusted sources without prior inspection. See also `tarfile-further-verification`.

<div class="seealso">

`706`  
Contains further motivation and rationale behind the design.

</div>

The *filter* argument to `TarFile.extract` or `~TarFile.extractall` can be:

- the string `'fully_trusted'`: Honor all metadata as specified in the archive. Should be used if the user trusts the archive completely, or implements their own complex verification.

- the string `'tar'`: Honor most *tar*-specific features (i.e. features of UNIX-like filesystems), but block features that are very likely to be surprising or malicious. Note that this filter does not limit the targets of symbolic links. See `tar_filter` for details.

- the string `'data'`: Ignore or block most features specific to UNIX-like filesystems. Intended for extracting cross-platform data archives. See `data_filter` for details.

- `None` (default): Use `TarFile.extraction_filter`.

  If that is also `None` (the default), the `'data'` filter will be used.

  > <div class="versionchanged">
  >
  > 3.14
  >
  > The default filter is set to `data <data_filter>`. Previously, the default was equivalent to `fully_trusted <fully_trusted_filter>`.
  >
  > </div>

- A callable which will be called for each extracted member with a `TarInfo <tarinfo-objects>` describing the member and the destination path to where the archive is extracted (i.e. the same path is used for all members):

      filter(member: TarInfo, path: str, /) -> TarInfo | None

  The callable is called just before each member is extracted, so it can take the current state of the disk into account. It can:

  - return a `TarInfo` object which will be used instead of the metadata in the archive, or
  - return `None`, in which case the member will be skipped, or
  - raise an exception to abort the operation or skip the member, depending on `~TarFile.errorlevel`. Note that when extraction is aborted, `~TarFile.extractall` may leave the archive partially extracted. It does not attempt to clean up.

### Default named filters

The pre-defined, named filters are available as functions, so they can be reused in custom filters:

<div class="function">

fully_trusted_filter(member, path)

Return *member* unchanged.

This implements the `'fully_trusted'` filter.

</div>

<div class="function">

tar_filter(member, path)

Implements the `'tar'` filter.

- Strip leading slashes (`/` and `os.sep`) from filenames.
- `Refuse <tarfile-extraction-refuse>` to extract files with absolute paths (in case the name is absolute even after stripping slashes, e.g. `C:/foo` on Windows). This raises `~tarfile.AbsolutePathError`.
- Normalize filenames (`TarInfo.name`) that contain `..` components using `os.path.normpath`. Note that this removes internal `..` components, which may change the meaning of the name if it traverses symbolic links.
- `Refuse <tarfile-extraction-refuse>` to extract files whose absolute path (after following symlinks) would end up outside the destination. This raises `~tarfile.OutsideDestinationError`.
- `Refuse <tarfile-extraction-refuse>` to extract links (hard or soft) whose name refers to the destination directory itself. This raises `~tarfile.OutsideDestinationError`. Note that such files are refused even on platforms that do not support symbolic links.
- Clear high mode bits (setuid, setgid, sticky) and group/other write bits (`~stat.S_IWGRP` \| `~stat.S_IWOTH`).

Return the modified `TarInfo` member.

> [!WARNING]
> This filter does not check link targets (`TarInfo.linkname`). It allows benign cases (like links to `/dev/null`), but it also allows links (symbolic and hard) to arbitrary files on the filesystem, potentially granting unintended access to these files.

<div class="versionchanged">

next

Filenames containing `..` components are now normalized.

</div>

</div>

<div class="function">

data_filter(member, path)

Implements the `'data'` filter. In addition to what `tar_filter` does:

- Normalize link targets (`TarInfo.linkname`) using `os.path.normpath`. Note that this removes internal `..` components, which may change the meaning of the link if the path in `!TarInfo.linkname` traverses symbolic links.

- `Refuse <tarfile-extraction-refuse>` to extract links (hard or soft) that link to absolute paths, or ones that link outside the destination.

  This raises `~tarfile.AbsoluteLinkError` or `~tarfile.LinkOutsideDestinationError`.

  Note that such files are refused even on platforms that do not support symbolic links.

- `Refuse <tarfile-extraction-refuse>` to extract device files (including pipes). This raises `~tarfile.SpecialFileError`.

- For regular files, including hard links:

  - Set the owner read and write permissions (`~stat.S_IRUSR` \| `~stat.S_IWUSR`).
  - Remove the group & other executable permission (`~stat.S_IXGRP` \| `~stat.S_IXOTH`) if the owner doesn’t have it (`~stat.S_IXUSR`).

- For other files (directories), set `mode` to `None`, so that extraction methods skip applying permission bits.

- Set user and group info (`uid`, `gid`, `uname`, `gname`) to `None`, so that extraction methods skip setting it.

Return the modified `TarInfo` member.

Note that this filter does not block *all* dangerous archive features. See `tarfile-further-verification` for details.

<div class="versionchanged">

3.15

Link targets are now normalized.

</div>

</div>

### Filter errors

When a filter refuses to extract a file, it will raise an appropriate exception, a subclass of `~tarfile.FilterError`. This will abort the extraction if `TarFile.errorlevel` is 1 or more. With `errorlevel=0` the error will be logged and the member will be skipped, but extraction will continue.

### Hints for further verification

Even with `filter='data'`, *tarfile* is not suited for extracting untrusted files without prior inspection. Among other issues, the pre-defined filters do not prevent denial-of-service attacks. Users should do additional checks.

Here is an incomplete list of things to consider:

- Extract to a `new temporary directory <tempfile.mkdtemp>` to prevent e.g. exploiting pre-existing links, and to make it easier to clean up after a failed extraction.
- Disallow symbolic links if you do not need the functionality.
- When working with untrusted data, use external (e.g. OS-level) limits on disk, memory and CPU usage.
- Check filenames against an allow-list of characters (to filter out control characters, confusables, foreign path separators, and so on).
- Check for platform-specific filename semantics. For example, on Windows some names can have reserved meanings.
- Check that filenames have expected extensions (discouraging files that execute when you “click on them”, or extension-less files like Windows special device names).
- Limit the number of extracted files, total size of extracted data, filename length (including symlink length), and size of individual files.
- Check for files that would be shadowed on case-insensitive filesystems.

Also note that:

- Tar files may contain multiple versions of the same file. Later ones are expected to overwrite any earlier ones. This feature is crucial to allow updating tape archives, but can be abused maliciously.
- *tarfile* does not protect against issues with “live” data, e.g. an attacker tinkering with the destination (or source) directory while extraction (or archiving) is in progress.

### Supporting older Python versions

Extraction filters were added to Python 3.12, but may be backported to older versions as security updates. To check whether the feature is available, use e.g. `hasattr(tarfile, 'data_filter')` rather than checking the Python version.

The following examples show how to support Python versions with and without the feature. Note that setting `extraction_filter` will affect any subsequent operations.

- Fully trusted archive:

      my_tarfile.extraction_filter = (lambda member, path: member)
      my_tarfile.extractall()

- Use the `'data'` filter if available, but revert to Python 3.11 behavior (`'fully_trusted'`) if this feature is not available:

      my_tarfile.extraction_filter = getattr(tarfile, 'data_filter',
                                             (lambda member, path: member))
      my_tarfile.extractall()

- Use the `'data'` filter; *fail* if it is not available:

      my_tarfile.extractall(filter=tarfile.data_filter)

  or:

      my_tarfile.extraction_filter = tarfile.data_filter
      my_tarfile.extractall()

- Use the `'data'` filter; *warn* if it is not available:

      if hasattr(tarfile, 'data_filter'):
          my_tarfile.extractall(filter='data')
      else:
          # remove this when no longer needed
          warn_the_user('Extracting may be unsafe; consider updating Python')
          my_tarfile.extractall()

### Stateful extraction filter example

While *tarfile*'s extraction methods take a simple *filter* callable, custom filters may be more complex objects with an internal state. It may be useful to write these as context managers, to be used like this:

    with StatefulFilter() as filter_func:
        tar.extractall(path, filter=filter_func)

Such a filter can be written as, for example:

    class StatefulFilter:
        def __init__(self):
            self.file_count = 0

        def __enter__(self):
            return self

        def __call__(self, member, path):
            self.file_count += 1
            return member

        def __exit__(self, *exc_info):
            print(f'{self.file_count} files extracted')

## Command-Line Interface

<div class="versionadded">

3.4

</div>

The `!tarfile` module provides a simple command-line interface to interact with tar archives.

If you want to create a new tar archive, specify its name after the `-c` option and then list the filename(s) that should be included:

```
$ python -m tarfile -c monty.tar  spam.txt eggs.txt
```

Passing a directory is also acceptable:

```
$ python -m tarfile -c monty.tar life-of-brian_1979/
```

If you want to extract a tar archive into the current directory, use the `-e` option:

```
$ python -m tarfile -e monty.tar
```

You can also extract a tar archive into a different directory by passing the directory's name:

```
$ python -m tarfile -e monty.tar  other-dir/
```

For a list of the files in a tar archive, use the `-l` option:

```
$ python -m tarfile -l monty.tar
```

### Command-line options

<div class="option">

`-l <tarfile>`  

`--list <tarfile>`  

List files in a tarfile.

</div>

<div class="option">

-c \<tarfile\> \<source1\> ... \<sourceN\> --create \<tarfile\> \<source1\> ... \<sourceN\>

Create tarfile from source files.

</div>

<div class="option">

-e \<tarfile\> \[\<output_dir\>\] --extract \<tarfile\> \[\<output_dir\>\]

Extract tarfile into the current directory if *output_dir* is not specified.

</div>

<div class="option">

`-t <tarfile>`  

`--test <tarfile>`  

Test whether the tarfile is valid or not.

</div>

<div class="option">

`-v, --verbose`  

Verbose output.

</div>

<div class="option">

`--filter <filtername>`  

Specifies the *filter* for `--extract`. See `tarfile-extraction-filter` for details. Only string names are accepted (that is, `fully_trusted`, `tar`, and `data`).

</div>

## Examples

### Reading examples

How to extract an entire tar archive to the current working directory:

    import tarfile
    tar = tarfile.open("sample.tar.gz")
    tar.extractall(filter='data')
    tar.close()

How to extract a subset of a tar archive with `TarFile.extractall` using a generator function instead of a list:

    import os
    import tarfile

    def py_files(members):
        for tarinfo in members:
            if os.path.splitext(tarinfo.name)[1] == ".py":
                yield tarinfo

    tar = tarfile.open("sample.tar.gz")
    tar.extractall(members=py_files(tar))
    tar.close()

How to read a gzip compressed tar archive and display some member information:

    import tarfile
    tar = tarfile.open("sample.tar.gz", "r:gz")
    for tarinfo in tar:
        print(tarinfo.name, "is", tarinfo.size, "bytes in size and is ", end="")
        if tarinfo.isreg():
            print("a regular file.")
        elif tarinfo.isdir():
            print("a directory.")
        else:
            print("something else.")
    tar.close()

### Writing examples

How to create an uncompressed tar archive from a list of filenames:

    import tarfile
    tar = tarfile.open("sample.tar", "w")
    for name in ["foo", "bar", "quux"]:
        tar.add(name)
    tar.close()

The same example using the `with` statement:

    import tarfile
    with tarfile.open("sample.tar", "w") as tar:
        for name in ["foo", "bar", "quux"]:
            tar.add(name)

How to create and write an archive to stdout using `sys.stdout.buffer <sys.stdout>` in the *fileobj* parameter in `TarFile.add`:

    import sys
    import tarfile
    with tarfile.open("sample.tar.gz", "w|gz", fileobj=sys.stdout.buffer) as tar:
        for name in ["foo", "bar", "quux"]:
            tar.add(name)

How to create an archive and reset the user information using the *filter* parameter in `TarFile.add`:

    import tarfile
    def reset(tarinfo):
        tarinfo.uid = tarinfo.gid = 0
        tarinfo.uname = tarinfo.gname = "root"
        return tarinfo
    tar = tarfile.open("sample.tar.gz", "w:gz")
    tar.add("foo", filter=reset)
    tar.close()

## Supported tar formats

There are three tar formats that can be created with the `!tarfile` module:

- The POSIX.1-1988 ustar format (`USTAR_FORMAT`). It supports filenames up to a length of at best 256 characters and linknames up to 100 characters. The maximum file size is 8 GiB. This is an old and limited but widely supported format.

- The GNU tar format (`GNU_FORMAT`). It supports long filenames and linknames, files bigger than 8 GiB and sparse files. It is the de facto standard on GNU/Linux systems. `!tarfile` fully supports the GNU tar extensions for long names, sparse file support is read-only.

- The POSIX.1-2001 pax format (`PAX_FORMAT`). It is the most flexible format with virtually no limits. It supports long filenames and linknames, large files and stores pathnames in a portable way. Modern tar implementations, including GNU tar, bsdtar/libarchive and star, fully support extended *pax* features; some old or unmaintained libraries may not, but should treat *pax* archives as if they were in the universally supported *ustar* format. It is the current default format for new archives.

  It extends the existing *ustar* format with extra headers for information that cannot be stored otherwise. There are two flavours of pax headers: Extended headers only affect the subsequent file header, global headers are valid for the complete archive and affect all following files. All the data in a pax header is encoded in *UTF-8* for portability reasons.

There are some more variants of the tar format which can be read, but not created:

- The ancient V7 format. This is the first tar format from Unix Seventh Edition, storing only regular files and directories. Names must not be longer than 100 characters, there is no user/group name information. Some archives have miscalculated header checksums in case of fields with non-ASCII characters.
- The SunOS tar extended format. This format is a variant of the POSIX.1-2001 pax format, but is not compatible.

## Unicode issues

The tar format was originally conceived to make backups on tape drives with the main focus on preserving file system information. Nowadays tar archives are commonly used for file distribution and exchanging archives over networks. One problem of the original format (which is the basis of all other formats) is that there is no concept of supporting different character encodings. For example, an ordinary tar archive created on a *UTF-8* system cannot be read correctly on a *Latin-1* system if it contains non-*ASCII* characters. Textual metadata (like filenames, linknames, user/group names) will appear damaged. Unfortunately, there is no way to autodetect the encoding of an archive. The pax format was designed to solve this problem. It stores non-ASCII metadata using the universal character encoding *UTF-8*.

The details of character conversion in `!tarfile` are controlled by the *encoding* and *errors* keyword arguments of the `TarFile` class.

*encoding* defines the character encoding to use for the metadata in the archive. The default value is `sys.getfilesystemencoding` or `'ascii'` as a fallback. Depending on whether the archive is read or written, the metadata must be either decoded or encoded. If *encoding* is not set appropriately, this conversion may fail.

The *errors* argument defines how characters are treated that cannot be converted. Possible values are listed in section `error-handlers`. The default scheme is `'surrogateescape'` which Python also uses for its file system calls, see `os-filenames`.

For `PAX_FORMAT` archives (the default), *encoding* is generally not needed because all the metadata is stored using *UTF-8*. *encoding* is only used in the rare cases when binary pax headers are decoded or when strings with surrogate characters are stored.

# `!telnetlib` --- Telnet client

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

telnetlib

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

Possible replacements are third-party libraries from PyPI: `telnetlib3` or `Exscript`. These are not supported or maintained by the Python core team.

The last version of Python that provided the `!telnetlib` module was [Python 3.12](https://docs.python.org/3.12/library/telnetlib.html).

# `!tempfile` --- Generate temporary files and directories

<div class="module" data-synopsis="Generate temporary files and directories.">

tempfile

</div>

**Source code:** `Lib/tempfile.py`

<div class="index">

pair: temporary; file name pair: temporary; file

</div>

------------------------------------------------------------------------

This module creates temporary files and directories. It works on all supported platforms. `TemporaryFile`, `NamedTemporaryFile`, `TemporaryFileWrapper`, `TemporaryDirectory`, and `SpooledTemporaryFile` are high-level interfaces which provide automatic cleanup and can be used as `context managers
<context manager>`. `mkstemp` and `mkdtemp` are lower-level functions which require manual cleanup.

All the user-callable functions and constructors take additional arguments which allow direct control over the location and name of temporary files and directories. Files names used by this module include a string of random characters which allows those files to be securely created in shared temporary directories. To maintain backward compatibility, the argument order is somewhat odd; it is recommended to use keyword arguments for clarity.

The module defines the following user-callable items:

<div class="function">

TemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, \*, errors=None)

Return a `file-like object` that can be used as a temporary storage area. The file is created securely, using the same rules as `mkstemp`. It will be destroyed as soon as it is closed (including an implicit close when the object is garbage collected). Under Unix, the directory entry for the file is either not created at all or is removed immediately after the file is created. Other platforms do not support this; your code should not rely on a temporary file created using this function having or not having a visible name in the file system.

The resulting object can be used as a `context manager` (see `tempfile-examples`). On completion of the context or destruction of the file object the temporary file will be removed from the filesystem.

The *mode* parameter defaults to `'w+b'` so that the file created can be read and written without being closed. Binary mode is used so that it behaves consistently on all platforms without regard for the data that is stored. *buffering*, *encoding*, *errors* and *newline* are interpreted as for `open`.

The *dir*, *prefix* and *suffix* parameters have the same meaning and defaults as with `mkstemp`.

The returned object is a true file object on POSIX platforms. On other platforms, it is a file-like object whose `!file` attribute is the underlying true file object.

The `os.O_TMPFILE` flag is used if it is available and works (Linux-specific, requires Linux kernel 3.11 or later).

On platforms that are neither Posix nor Cygwin, TemporaryFile is an alias for NamedTemporaryFile.

<div class="audit-event">

tempfile.mkstemp fullpath tempfile.TemporaryFile

</div>

<div class="versionchanged">

3.5

The `os.O_TMPFILE` flag is now used if available.

</div>

<div class="versionchanged">

3.8 Added *errors* parameter.

</div>

</div>

<div class="function">

NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, \*, errors=None, delete_on_close=True)

This function operates exactly as `TemporaryFile` does, except the following differences:

- This function returns a file that is guaranteed to have a visible name in the file system.
- To manage the named file, it extends the parameters of `TemporaryFile` with *delete* and *delete_on_close* parameters that determine whether and how the named file should be automatically deleted.

The returned object is always a `TemporaryFileWrapper` instance (a `file-like object`) whose `~TemporaryFileWrapper.file` attribute is the underlying true file object. This file-like object can be used in a `with` statement, just like a normal file. The name of the temporary file can be retrieved from the `!name` attribute of the returned file-like object. On Unix, unlike with the `TemporaryFile`, the directory entry does not get unlinked immediately after the file creation.

If *delete* is true (the default) and *delete_on_close* is true (the default), the file is deleted as soon as it is closed. If *delete* is true and *delete_on_close* is false, the file is deleted on context manager exit only, or else when the `file-like object` is finalized. Deletion is not always guaranteed in this case (see `object.__del__`). If *delete* is false, the value of *delete_on_close* is ignored.

Therefore to use the name of the temporary file to reopen the file after closing it, either make sure not to delete the file upon closure (set the *delete* parameter to be false) or, in case the temporary file is created in a `with` statement, set the *delete_on_close* parameter to be false. The latter approach is recommended as it provides assistance in automatic cleaning of the temporary file upon the context manager exit.

Opening the temporary file again by its name while it is still open works as follows:

- On POSIX the file can always be opened again.
- On Windows, make sure that at least one of the following conditions are fulfilled:
  - *delete* is false
  - additional open shares delete access (e.g. by calling `os.open` with the flag `O_TEMPORARY`)
  - *delete* is true but *delete_on_close* is false. Note, that in this case the additional opens that do not share delete access (e.g. created via builtin `open`) must be closed before exiting the context manager, else the `os.unlink` call on context manager exit will fail with a `PermissionError`.

On Windows, if *delete_on_close* is false, and the file is created in a directory for which the user lacks delete access, then the `os.unlink` call on exit of the context manager will fail with a `PermissionError`. This cannot happen when *delete_on_close* is true because delete access is requested by the open, which fails immediately if the requested access is not granted.

On POSIX (only), a process that is terminated abruptly with SIGKILL cannot automatically delete any NamedTemporaryFiles it created.

<div class="audit-event">

tempfile.mkstemp fullpath tempfile.NamedTemporaryFile

</div>

<div class="versionchanged">

3.8 Added *errors* parameter.

</div>

<div class="versionchanged">

3.12 Added *delete_on_close* parameter.

</div>

</div>

<div class="TemporaryFileWrapper(file, name, delete=True, delete_on_close=True)">

A mutable wrapper returned by `NamedTemporaryFile`. It wraps the underlying file object, delegating attribute access to it, and ensures the temporary file is deleted when appropriate.

<div class="attribute">

file

The underlying `file-like object`.

</div>

<div class="attribute">

name

The file name of the temporary file.

</div>

<div class="method">

close()

Close the temporary file, possibly deleting it depending on the *delete* and *delete_on_close* arguments passed to `NamedTemporaryFile`.

</div>

> [!NOTE]
> `tempfile._TemporaryFileWrapper` is kept as a backwards compatible deprecated alias for this class. It will be removed in Python 3.21

<div class="versionadded">

3.16

</div>

</div>

<div class="SpooledTemporaryFile(max_size=0, mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None)">

This class operates exactly as `TemporaryFile` does, except that data is spooled in memory until the file size exceeds *max_size*, or until the file's `~io.IOBase.fileno` method is called, at which point the contents are written to disk and operation proceeds as with `TemporaryFile`.

<div class="method">

SpooledTemporaryFile.rollover

The resulting file has one additional method, `!rollover`, which causes the file to roll over to an on-disk file regardless of its size.

</div>

The returned object is a file-like object whose `!_file` attribute is either an `io.BytesIO` or `io.TextIOWrapper` object (depending on whether binary or text *mode* was specified) or a true file object, depending on whether `rollover` has been called. This file-like object can be used in a `with` statement, just like a normal file.

<div class="versionchanged">

3.3 the truncate method now accepts a *size* argument.

</div>

<div class="versionchanged">

3.8 Added *errors* parameter.

</div>

<div class="versionchanged">

3.11 Fully implements the `io.BufferedIOBase` and `io.TextIOBase` abstract base classes (depending on whether binary or text *mode* was specified).

</div>

</div>

<div class="TemporaryDirectory(suffix=None, prefix=None, dir=None, ignore_cleanup_errors=False, *, delete=True)">

This class securely creates a temporary directory using the same rules as `mkdtemp`. The resulting object can be used as a `context manager` (see `tempfile-examples`). On completion of the context or destruction of the temporary directory object, the newly created temporary directory and all its contents are removed from the filesystem.

<div class="attribute">

TemporaryDirectory.name

The directory name can be retrieved from the `!name` attribute of the returned object. When the returned object is used as a `context manager`, the `!name` will be assigned to the target of the `!as` clause in the `with` statement, if there is one.

</div>

<div class="method">

TemporaryDirectory.cleanup

The directory can be explicitly cleaned up by calling the `!cleanup` method. If *ignore_cleanup_errors* is true, any unhandled exceptions during explicit or implicit cleanup (such as a `PermissionError` removing open files on Windows) will be ignored, and the remaining removable items deleted on a "best-effort" basis. Otherwise, errors will be raised in whatever context cleanup occurs (the `!cleanup` call, exiting the context manager, when the object is garbage-collected or during interpreter shutdown).

</div>

The *delete* parameter can be used to disable cleanup of the directory tree upon exiting the context. While it may seem unusual for a context manager to disable the action taken when exiting the context, it can be useful during debugging or when you need your cleanup behavior to be conditional based on other logic.

> [!WARNING]
> Cleanup is not robust against the tree being modified while it is removed. Files outside of the tree may have their permissions and file flags reset.
>
> On systems where `shutil.rmtree.avoids_symlink_attacks` is false, manipulating symbolic links during cleanup may cause files outside of the tree to be removed.

<div class="audit-event">

tempfile.mkdtemp fullpath tempfile.TemporaryDirectory

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.10 Added *ignore_cleanup_errors* parameter.

</div>

<div class="versionchanged">

3.12 Added the *delete* parameter.

</div>

</div>

<div class="function">

mkstemp(suffix=None, prefix=None, dir=None, text=False)

Creates a temporary file in the most secure manner possible. There are no race conditions in the file's creation, assuming that the platform properly implements the `os.O_EXCL` flag for `os.open`. The file is readable and writable only by the creating user ID. If the platform uses permission bits to indicate whether a file is executable, the file is executable by no one.

The file descriptor is `not inherited by child processes <fd_inheritance>`.

Unlike `TemporaryFile`, the user of `mkstemp` is responsible for closing the file descriptor (for example, using `os.close`) and deleting the temporary file (for example, using `os.remove`).

If *suffix* is not `None`, the file name will end with that suffix, otherwise there will be no suffix. `mkstemp` does not put a dot between the file name and the suffix; if you need one, put it at the beginning of *suffix*.

If *prefix* is not `None`, the file name will begin with that prefix; otherwise, a default prefix is used. The default is the return value of `gettempprefix` or `gettempprefixb`, as appropriate.

If *dir* is not `None`, the file will be created in that directory; otherwise, a default directory is used. The default directory is chosen from a platform-dependent list, but the user of the application can control the directory location by setting the *TMPDIR*, *TEMP* or *TMP* environment variables. There is thus no guarantee that the generated filename will have any nice properties, such as not requiring quoting when passed to external commands via `os.popen()`.

If any of *suffix*, *prefix*, and *dir* are not `None`, they must be the same type. If they are bytes, the returned name will be bytes instead of str. If you want to force a bytes return value with otherwise default behavior, pass `suffix=b''`.

If *text* is specified and true, the file is opened in text mode. Otherwise, (the default) the file is opened in binary mode.

`mkstemp` returns a tuple containing an OS-level handle to an open file (as would be returned by `os.open`) and the absolute pathname of that file, in that order.

<div class="audit-event">

tempfile.mkstemp fullpath tempfile.mkstemp

</div>

<div class="versionchanged">

3.5 *suffix*, *prefix*, and *dir* may now be supplied in bytes in order to obtain a bytes return value. Prior to this, only str was allowed. *suffix* and *prefix* now accept and default to `None` to cause an appropriate default value to be used.

</div>

<div class="versionchanged">

3.6 The *dir* parameter now accepts a `path-like object`.

</div>

</div>

<div class="function">

mkdtemp(suffix=None, prefix=None, dir=None)

Creates a temporary directory in the most secure manner possible. There are no race conditions in the directory's creation. The directory is readable, writable, and searchable only by the creating user ID.

The user of `mkdtemp` is responsible for deleting the temporary directory and its contents when done with it.

The *prefix*, *suffix*, and *dir* arguments are the same as for `mkstemp`.

`mkdtemp` returns the absolute pathname of the new directory.

<div class="audit-event">

tempfile.mkdtemp fullpath tempfile.mkdtemp

</div>

<div class="versionchanged">

3.5 *suffix*, *prefix*, and *dir* may now be supplied in bytes in order to obtain a bytes return value. Prior to this, only str was allowed. *suffix* and *prefix* now accept and default to `None` to cause an appropriate default value to be used.

</div>

<div class="versionchanged">

3.6 The *dir* parameter now accepts a `path-like object`.

</div>

<div class="versionchanged">

3.12 `mkdtemp` now always returns an absolute path, even if *dir* is relative.

</div>

</div>

<div class="function">

gettempdir()

Return the name of the directory used for temporary files. This defines the default value for the *dir* argument to all functions in this module.

Python searches a standard list of directories to find one which the calling user can create files in. The list is:

1.  The directory named by the `TMPDIR` environment variable.
2.  The directory named by the `TEMP` environment variable.
3.  The directory named by the `TMP` environment variable.
4.  A platform-specific location:
    - On Windows, the directories `%USERPROFILE%\\AppData\\Local\\Temp`, `%SYSTEMROOT%\\Temp`, `C:\\TEMP`, `C:\\TMP`, `\\TEMP`, and `\\TMP`, in that order.
    - On all other platforms, the directories `/tmp`, `/var/tmp`, and `/usr/tmp`, in that order.
5.  As a last resort, the current working directory.

The result of this search is cached, see the description of `tempdir` below.

<div class="versionchanged">

3.10

Always returns a str. Previously it would return any `tempdir` value regardless of type so long as it was not `None`.

</div>

</div>

<div class="function">

gettempdirb()

Same as `gettempdir` but the return value is in bytes.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

gettempprefix()

Return the filename prefix used to create temporary files. This does not contain the directory component.

</div>

<div class="function">

gettempprefixb()

Same as `gettempprefix` but the return value is in bytes.

<div class="versionadded">

3.5

</div>

</div>

The module uses a global variable to store the name of the directory used for temporary files returned by `gettempdir`. It can be set directly to override the selection process, but this is discouraged. All functions in this module take a *dir* argument which can be used to specify the directory. This is the recommended approach that does not surprise other unsuspecting code by changing global API behavior.

<div class="data">

tempdir

When set to a value other than `None`, this variable defines the default value for the *dir* argument to the functions defined in this module, including its type, bytes or str. It cannot be a `path-like object`.

If `tempdir` is `None` (the default) at any call to any of the above functions except `gettempprefix` it is initialized following the algorithm described in `gettempdir`.

> [!NOTE]
> Beware that if you set `tempdir` to a bytes value, there is a nasty side effect: The global default return type of `mkstemp` and `mkdtemp` changes to bytes when no explicit `prefix`, `suffix`, or `dir` arguments of type str are supplied. Please do not write code expecting or depending on this. This awkward behavior is maintained for compatibility with the historical implementation.

</div>

## Examples

Here are some examples of typical usage of the `!tempfile` module:

    >>> import tempfile

    # create a temporary file and write some data to it
    >>> fp = tempfile.TemporaryFile()
    >>> fp.write(b'Hello world!')
    # read data from file
    >>> fp.seek(0)
    >>> fp.read()
    b'Hello world!'
    # close the file, it will be removed
    >>> fp.close()

    # create a temporary file using a context manager
    >>> with tempfile.TemporaryFile() as fp:
    ...     fp.write(b'Hello world!')
    ...     fp.seek(0)
    ...     fp.read()
    b'Hello world!'
    >>>
    # file is now closed and removed

    # create a temporary file using a context manager
    # close the file, use the name to open the file again
    >>> with tempfile.NamedTemporaryFile(delete_on_close=False) as fp:
    ...     fp.write(b'Hello world!')
    ...     fp.close()
    ... # the file is closed, but not removed
    ... # open the file again by using its name
    ...     with open(fp.name, mode='rb') as f:
    ...         f.read()
    b'Hello world!'
    >>>
    # file is now removed

    # create a temporary directory using the context manager
    >>> with tempfile.TemporaryDirectory() as tmpdirname:
    ...     print('created temporary directory', tmpdirname)
    >>>
    # directory and contents have been removed

## Deprecated functions and variables

A historical way to create temporary files was to first generate a file name with the `mktemp` function and then create a file using this name. Unfortunately this is not secure, because a different process may create a file with this name in the time between the call to `mktemp` and the subsequent attempt to create the file by the first process. The solution is to combine the two steps and create the file immediately. This approach is used by `mkstemp` and the other functions described above.

<div class="function">

mktemp(suffix='', prefix='tmp', dir=None)

<div class="deprecated">

2.3 Use `mkstemp` instead.

</div>

Return an absolute pathname of a file that did not exist at the time the call is made. The *prefix*, *suffix*, and *dir* arguments are similar to those of `mkstemp`, except that bytes file names, `suffix=None` and `prefix=None` are not supported.

> [!WARNING]
> Use of this function may introduce a security hole in your program. By the time you get around to doing anything with the file name it returns, someone else may have beaten you to the punch. `mktemp` usage can be replaced easily with `NamedTemporaryFile`, passing it the `delete=False` parameter:
>
>     >>> f = NamedTemporaryFile(delete=False)
>     >>> f.name
>     '/tmp/tmptjujjt'
>     >>> f.write(b"Hello World!\n")
>     13
>     >>> f.close()
>     >>> os.unlink(f.name)
>     >>> os.path.exists(f.name)
>     False

</div>

# `!termios` --- POSIX style tty control

<div class="module" data-synopsis="POSIX style tty control.">

termios

</div>

<div class="index">

pair: POSIX; I/O control pair: tty; I/O control

</div>

------------------------------------------------------------------------

This module provides an interface to the POSIX calls for tty I/O control. For a complete description of these calls, see `termios(3)` Unix manual page. It is only available for those Unix versions that support POSIX *termios* style tty I/O control configured during installation.

<div class="availability">

Unix.

</div>

All functions in this module take a file descriptor *fd* as their first argument. This can be an integer file descriptor, such as returned by `sys.stdin.fileno()`, or a `file object`, such as `sys.stdin` itself.

This module also defines all the constants needed to work with the functions provided here; these have the same name as their counterparts in C. Please refer to your system documentation for more information on using these terminal control interfaces.

The module defines the following functions:

<div class="function">

tcgetattr(fd)

Return a list containing the tty attributes for file descriptor *fd*, as follows: `[iflag, oflag, cflag, lflag, ispeed, ospeed, cc]` where *cc* is a list of the tty special characters (each a string of length 1, except the items with indices `VMIN` and `VTIME`, which are integers when these fields are defined). The interpretation of the flags and the speeds as well as the indexing in the *cc* array must be done using the symbolic constants defined in the `!termios` module.

<div class="data">

VMIN VTIME

Indices into the *cc* array controlling a non-canonical read: the minimum number of characters to read and the timeout in tenths of a second, respectively.

</div>

</div>

<div class="function">

tcsetattr(fd, when, attributes)

Set the tty attributes for file descriptor *fd* from the *attributes*, which is a list like the one returned by `tcgetattr`. The *when* argument determines when the attributes are changed:

<div class="data">

TCSANOW

Change attributes immediately.

</div>

<div class="data">

TCSADRAIN

Change attributes after transmitting all queued output.

</div>

<div class="data">

TCSAFLUSH

Change attributes after transmitting all queued output and discarding all queued input.

</div>

</div>

<div class="function">

tcsendbreak(fd, duration)

Send a break on file descriptor *fd*. A zero *duration* sends a break for 0.25--0.5 seconds; a nonzero *duration* has a system dependent meaning.

</div>

<div class="function">

tcdrain(fd)

Wait until all output written to file descriptor *fd* has been transmitted.

</div>

<div class="function">

tcflush(fd, queue)

Discard queued data on file descriptor *fd*. The *queue* selector specifies which queue: `TCIFLUSH` for the input queue, `TCOFLUSH` for the output queue, or `TCIOFLUSH` for both queues.

<div class="data">

TCIFLUSH TCOFLUSH TCIOFLUSH

Values for the *queue* selector: the input queue, the output queue, or both queues, respectively.

</div>

</div>

<div class="function">

tcflow(fd, action)

Suspend or resume input or output on file descriptor *fd*. The *action* argument can be `TCOOFF` to suspend output, `TCOON` to restart output, `TCIOFF` to suspend input, or `TCION` to restart input.

<div class="data">

TCOOFF TCOON TCIOFF TCION

Values for the *action* argument: suspend output, restart output, suspend input, or restart input, respectively.

</div>

</div>

<div class="function">

tcgetwinsize(fd)

Return a tuple `(ws_row, ws_col)` containing the tty window size for file descriptor *fd*. Requires `termios.TIOCGWINSZ` or `termios.TIOCGSIZE`.

<div class="versionadded">

3.11

</div>

<div class="data">

TIOCGWINSZ TIOCGSIZE

`ioctl()` requests for querying the tty window size. Not defined on all platforms; `tcgetwinsize` requires at least one of them.

</div>

</div>

<div class="function">

tcsetwinsize(fd, winsize)

Set the tty window size for file descriptor *fd* from *winsize*, which is a two-item tuple `(ws_row, ws_col)` like the one returned by `tcgetwinsize`. Requires at least one of the pairs (`termios.TIOCGWINSZ`, `termios.TIOCSWINSZ`); (`termios.TIOCGSIZE`, `termios.TIOCSSIZE`) to be defined.

<div class="versionadded">

3.11

</div>

<div class="data">

TIOCSWINSZ TIOCSSIZE

`ioctl()` requests for setting the tty window size. Not defined on all platforms; see `tcsetwinsize` for the required pairs.

</div>

</div>

<div class="seealso">

Module `tty`  
Convenience functions for common terminal control operations.

</div>

## Example

Here's a function that prompts for a password with echoing turned off. Note the technique using a separate `tcgetattr` call and a `try` ... `finally` statement to ensure that the old tty attributes are restored exactly no matter what happens:

    def getpass(prompt="Password: "):
        import termios, sys
        fd = sys.stdin.fileno()
        old = termios.tcgetattr(fd)
        new = termios.tcgetattr(fd)
        new[3] = new[3] & ~termios.ECHO          # lflags
        try:
            termios.tcsetattr(fd, termios.TCSADRAIN, new)
            passwd = input(prompt)
        finally:
            termios.tcsetattr(fd, termios.TCSADRAIN, old)
        return passwd

# `!test` --- Regression tests package for Python

<div class="module" data-synopsis="Regression tests package containing the testing suite for Python.">

test

</div>

> [!NOTE]
> The `!test` package is meant for internal use by Python only. It is documented for the benefit of the core developers of Python. Any use of this package outside of Python's standard library is discouraged as code mentioned here can change or be removed without notice between releases of Python.

------------------------------------------------------------------------

The `!test` package contains all regression tests for Python as well as the modules `test.support` and `test.regrtest`. `test.support` is used to enhance your tests while `test.regrtest` drives the testing suite.

Each module in the `!test` package whose name starts with `test_` is a testing suite for a specific module or feature. All new tests should be written using the `unittest` or `doctest` module. Some older tests are written using a "traditional" testing style that compares output printed to `sys.stdout`; this style of test is considered deprecated.

<div class="seealso">

Module `unittest`  
Writing PyUnit regression tests.

Module `doctest`  
Tests embedded in documentation strings.

</div>

## Writing Unit Tests for the `!test` package

It is preferred that tests that use the `unittest` module follow a few guidelines. One is to name the test module by starting it with `test_` and end it with the name of the module being tested. The test methods in the test module should start with `test_` and end with a description of what the method is testing. This is needed so that the methods are recognized by the test driver as test methods. Also, no documentation string for the method should be included. A comment (such as `# Tests function returns only True or False`) should be used to provide documentation for test methods. This is done because documentation strings get printed out if they exist and thus what test is being run is not stated.

A basic boilerplate is often used:

    import unittest
    from test import support

    class MyTestCase1(unittest.TestCase):

        # Only use setUp() and tearDown() if necessary

        def setUp(self):
            ... code to execute in preparation for tests ...

        def tearDown(self):
            ... code to execute to clean up after tests ...

        def test_feature_one(self):
            # Test feature one.
            ... testing code ...

        def test_feature_two(self):
            # Test feature two.
            ... testing code ...

        ... more test methods ...

    class MyTestCase2(unittest.TestCase):
        ... same structure as MyTestCase1 ...

    ... more test classes ...

    if __name__ == '__main__':
        unittest.main()

This code pattern allows the testing suite to be run by `test.regrtest`, on its own as a script that supports the `unittest` CLI, or via the `python -m unittest` CLI.

The goal for regression testing is to try to break code. This leads to a few guidelines to be followed:

- The testing suite should exercise all classes, functions, and constants. This includes not just the external API that is to be presented to the outside world but also "private" code.

- Whitebox testing (examining the code being tested when the tests are being written) is preferred. Blackbox testing (testing only the published user interface) is not complete enough to make sure all boundary and edge cases are tested.

- Make sure all possible values are tested including invalid ones. This makes sure that not only all valid values are acceptable but also that improper values are handled correctly.

- Exhaust as many code paths as possible. Test where branching occurs and thus tailor input to make sure as many different paths through the code are taken.

- Add an explicit test for any bugs discovered for the tested code. This will make sure that the error does not crop up again if the code is changed in the future.

- Make sure to clean up after your tests (such as close and remove all temporary files).

- If a test is dependent on a specific condition of the operating system then verify the condition already exists before attempting the test.

- Import as few modules as possible and do it as soon as possible. This minimizes external dependencies of tests and also minimizes possible anomalous behavior from side-effects of importing a module.

- Try to maximize code reuse. On occasion, tests will vary by something as small as what type of input is used. Minimize code duplication by subclassing a basic test class with a class that specifies the input:

      class TestFuncAcceptsSequencesMixin:

          func = mySuperWhammyFunction

          def test_func(self):
              self.func(self.arg)

      class AcceptLists(TestFuncAcceptsSequencesMixin, unittest.TestCase):
          arg = [1, 2, 3]

      class AcceptStrings(TestFuncAcceptsSequencesMixin, unittest.TestCase):
          arg = 'abc'

      class AcceptTuples(TestFuncAcceptsSequencesMixin, unittest.TestCase):
          arg = (1, 2, 3)

  When using this pattern, remember that all classes that inherit from `unittest.TestCase` are run as tests. The `!TestFuncAcceptsSequencesMixin` class in the example above does not have any data and so can't be run by itself, thus it does not inherit from `unittest.TestCase`.

<div class="seealso">

Test Driven Development  
A book by Kent Beck on writing tests before code.

</div>

## Running tests using the command-line interface

<div class="module" data-synopsis="Drives the regression test suite.">

test.regrtest

</div>

The `!test` package can be run as a script to drive Python's regression test suite, thanks to the `-m` option: `python -m test`. Under the hood, it uses `!test.regrtest`; the call `python -m
test.regrtest` used in previous Python versions still works. Running the script by itself automatically starts running all regression tests in the `!test` package. It does this by finding modules and packages whose names start with `test_`, importing each one, and loading its tests with `unittest.TestLoader.loadTestsFromModule`. See the `load_tests protocol <load_tests-protocol>` for how a module or package can customize what is loaded. Defining a module-level `test_main()` function is not supported and causes the test to fail. The names of tests to execute may also be passed to the script. Specifying a single regression test (`python -m test test_spam`) will minimize output and only print whether the test passed or failed.

Running `!test` directly allows what resources are available for tests to use to be set. You do this by using the `-u` command-line option. Specifying `all` as the value for the `-u` option enables all possible resources: `python -m test -uall`. If all but one resource is desired (a more common case), a comma-separated list of resources that are not desired may be listed after `all`. The command `python -m test -uall,-audio,-largefile` will run `!test` with all resources except the `audio` and `largefile` resources. For a list of all resources and more command-line options, run `python -m test -h`.

Some other ways to execute the regression tests depend on what platform the tests are being executed on. On Unix, you can run `make test` at the top-level directory where Python was built. On Windows, executing `rt.bat` from your `PCbuild` directory will run all regression tests.

<div class="versionadded">

3.14 Output is colorized by default and can be `controlled using environment variables <using-on-controlling-color>`.

</div>

# `!test.support` --- Utilities for the Python test suite

<div class="module" data-synopsis="Support for Python's regression test suite.">

test.support

</div>

The `!test.support` module provides support for Python's regression test suite.

> [!NOTE]
> `!test.support` is not a public module. It is documented here to help Python developers write tests. The API of this module is subject to change without backwards compatibility concerns between releases.

This module defines the following exceptions:

<div class="exception">

TestFailed

Exception to be raised when a test fails. This is deprecated in favor of `unittest`-based tests and `unittest.TestCase`'s assertion methods.

</div>

<div class="exception">

ResourceDenied

Subclass of `unittest.SkipTest`. Raised when a resource (such as a network connection) is not available. Raised by the `requires` function.

</div>

The `!test.support` module defines the following constants:

<div class="data">

verbose

How verbose the output is: the number of `!-v` options which `test.regrtest` was run with, and therefore `0` when verbose output is not enabled. Should be checked when more detailed information is desired about a running test.

</div>

<div class="data">

is_jython

`True` if the running interpreter is Jython.

</div>

<div class="data">

is_android

`True` if `sys.platform` is `android`.

</div>

<div class="data">

is_emscripten

`True` if `sys.platform` is `emscripten`.

</div>

<div class="data">

is_wasi

`True` if `sys.platform` is `wasi`.

</div>

<div class="data">

is_apple_mobile

`True` if `sys.platform` is `ios`, `tvos`, or `watchos`.

</div>

<div class="data">

is_apple

`True` if `sys.platform` is `darwin` or `is_apple_mobile` is `True`.

</div>

<div class="data">

unix_shell

Path for shell if not on Windows; otherwise `None`.

</div>

<div class="data">

LOOPBACK_TIMEOUT

Timeout in seconds for tests using a network server listening on the network local loopback interface like `127.0.0.1`.

The timeout is long enough to prevent test failure: it takes into account that the client and the server can run in different threads or even different processes.

The timeout should be long enough for `~socket.socket.connect`, `~socket.socket.recv` and `~socket.socket.send` methods of `socket.socket`.

Its default value is 10 seconds.

See also `INTERNET_TIMEOUT`.

</div>

<div class="data">

INTERNET_TIMEOUT

Timeout in seconds for network requests going to the internet.

The timeout is short enough to prevent a test to wait for too long if the internet request is blocked for whatever reason.

Usually, a timeout using `INTERNET_TIMEOUT` should not mark a test as failed, but skip the test instead: see `~test.support.socket_helper.transient_internet`.

Its default value is 1 minute.

See also `LOOPBACK_TIMEOUT`.

</div>

<div class="data">

SHORT_TIMEOUT

Timeout in seconds to mark a test as failed if the test takes "too long".

The timeout value depends on the regrtest `--timeout` command line option.

If a test using `SHORT_TIMEOUT` starts to fail randomly on slow buildbots, use `LONG_TIMEOUT` instead.

Its default value is 30 seconds.

</div>

<div class="data">

LONG_TIMEOUT

Timeout in seconds to detect when a test hangs.

It is long enough to reduce the risk of test failure on the slowest Python buildbots. It should not be used to mark a test as failed if the test takes "too long". The timeout value depends on the regrtest `--timeout` command line option.

Its default value is 5 minutes.

See also `LOOPBACK_TIMEOUT`, `INTERNET_TIMEOUT` and `SHORT_TIMEOUT`.

</div>

<div class="data">

PGO

Set when tests can be skipped when they are not useful for PGO.

</div>

<div class="data">

PIPE_MAX_SIZE

A constant that is likely larger than the underlying OS pipe buffer size, to make writes blocking.

</div>

<div class="data">

Py_DEBUG

`True` if Python was built with the `Py_DEBUG` macro defined, that is, if Python was `built in debug mode <debug-build>`.

<div class="versionadded">

3.12

</div>

</div>

<div class="data">

SOCK_MAX_SIZE

A constant that is likely larger than the underlying OS socket buffer size, to make writes blocking.

</div>

<div class="data">

TEST_SUPPORT_DIR

Set to the top level directory that contains `!test.support`.

</div>

<div class="data">

TEST_HOME_DIR

Set to the top level directory for the test package.

</div>

<div class="data">

TEST_DATA_DIR

Set to the `data` directory within the test package.

</div>

<div class="data">

MAX_Py_ssize_t

Set to `sys.maxsize` for big memory tests.

</div>

<div class="data">

max_memuse

Set by `set_memlimit` as the memory limit for big memory tests. Limited by `MAX_Py_ssize_t`.

</div>

<div class="data">

real_max_memuse

Set by `set_memlimit` as the memory limit for big memory tests. Not limited by `MAX_Py_ssize_t`.

</div>

<div class="data">

MISSING_C_DOCSTRINGS

Set to `True` if Python is built without docstrings (the `WITH_DOC_STRINGS` macro is not defined). See the `configure --without-doc-strings <--without-doc-strings>` option.

See also the `HAVE_DOCSTRINGS` variable.

</div>

<div class="data">

HAVE_DOCSTRINGS

Set to `True` if function docstrings are available. See the `python -OO <-O>` option, which strips docstrings of functions implemented in Python.

See also the `MISSING_C_DOCSTRINGS` variable.

</div>

<div class="data">

TEST_HTTP_URL

Define the URL of a dedicated HTTP server for the network tests.

</div>

<div class="data">

ALWAYS_EQ

Object that is equal to anything. Used to test mixed type comparison.

</div>

<div class="data">

NEVER_EQ

Object that is not equal to anything (even to `ALWAYS_EQ`). Used to test mixed type comparison.

</div>

<div class="data">

LARGEST

Object that is greater than anything (except itself). Used to test mixed type comparison.

</div>

<div class="data">

SMALLEST

Object that is less than anything (except itself). Used to test mixed type comparison.

</div>

The `!test.support` module defines the following functions:

<div class="function">

busy_retry(timeout, err_msg=None, /, \*, error=True)

Run the loop body until `break` stops the loop.

After *timeout* seconds, raise an `AssertionError` if *error* is true, or just stop the loop if *error* is false.

Example:

    for _ in support.busy_retry(support.SHORT_TIMEOUT):
        if check():
            break

Example of error=False usage:

    for _ in support.busy_retry(support.SHORT_TIMEOUT, error=False):
        if check():
            break
    else:
        raise RuntimeError('my custom error')

</div>

<div class="function">

sleeping_retry(timeout, err_msg=None, /, \*, init_delay=0.010, max_delay=1.0, error=True)

Wait strategy that applies exponential backoff.

Run the loop body until `break` stops the loop. Sleep at each loop iteration, but not at the first iteration. The sleep delay is doubled at each iteration (up to *max_delay* seconds).

See `busy_retry` documentation for the parameters usage.

Example raising an exception after SHORT_TIMEOUT seconds:

    for _ in support.sleeping_retry(support.SHORT_TIMEOUT):
        if check():
            break

Example of error=False usage:

    for _ in support.sleeping_retry(support.SHORT_TIMEOUT, error=False):
        if check():
            break
    else:
        raise RuntimeError('my custom error')

</div>

<div class="function">

is_resource_enabled(resource)

Return `True` if *resource* is enabled and available. The list of available resources is only set when `test.regrtest` is executing the tests.

</div>

<div class="function">

get_resource_value(resource)

Return the value specified for *resource* (as `-u {resource}={value}`). Return `None` if *resource* is disabled or no value is specified.

</div>

<div class="function">

python_is_optimized()

Return `True` if Python was not built with `-O0` or `-Og`.

</div>

<div class="function">

with_pymalloc()

Return `True` if Python was built with `pymalloc <pymalloc>` enabled and is not a `free-threaded build`.

</div>

<div class="function">

requires(resource, msg=None)

Raise `ResourceDenied` if *resource* is not available. *msg* is the argument to `ResourceDenied` if it is raised. Always returns `True` if called by a function whose `__name__` is `'__main__'`. Used when tests are executed by `test.regrtest`.

</div>

<div class="function">

sortdict(dict)

Return a repr of *dict* with keys sorted.

</div>

<div class="function">

findfile(filename, subdir=None)

Return the path to the file named *filename*. If no match is found *filename* is returned. This does not equal a failure since it could be the path to the file.

Setting *subdir* indicates a relative path to use to find the file rather than looking directly in the path directories.

</div>

<div class="function">

get_pagesize()

Get size of a page in bytes.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

setswitchinterval(interval)

Set the `sys.setswitchinterval` to the given *interval*. Defines a minimum interval for Android systems to prevent the system from hanging.

</div>

<div class="function">

check_impl_detail(\*\*guards)

Use this check to guard CPython's implementation-specific tests or to run them only on the implementations guarded by the arguments. This function returns `True` or `False` depending on the host platform. Example usage:

    check_impl_detail()               # Only on CPython (default).
    check_impl_detail(jython=True)    # Only on Jython.
    check_impl_detail(cpython=False)  # Everywhere except CPython.

</div>

<div class="function">

set_memlimit(limit)

Set the values for `max_memuse` and `real_max_memuse` for big memory tests.

</div>

<div class="function">

record_original_stdout(stdout)

Store the value from *stdout*. It is meant to hold the stdout at the time the regrtest began.

</div>

<div class="function">

get_original_stdout()

Return the original stdout set by `record_original_stdout` or `sys.stdout` if it's not set.

</div>

<div class="function">

args_from_interpreter_flags()

Return a list of command line arguments reproducing the current settings in `sys.flags` and `sys.warnoptions`.

</div>

<div class="function">

optim_args_from_interpreter_flags()

Return a list of command line arguments reproducing the current optimization settings in `sys.flags`.

</div>

<div class="function">

captured_stdin() captured_stdout() captured_stderr()

A context managers that temporarily replaces the named stream with `io.StringIO` object.

Example use with output streams:

    with captured_stdout() as stdout, captured_stderr() as stderr:
        print("hello")
        print("error", file=sys.stderr)
    assert stdout.getvalue() == "hello\n"
    assert stderr.getvalue() == "error\n"

Example use with input stream:

    with captured_stdin() as stdin:
        stdin.write('hello\n')
        stdin.seek(0)
        # call test code that consumes from sys.stdin
        captured = input()
    self.assertEqual(captured, "hello")

</div>

<div class="function">

disable_faulthandler()

A context manager that temporary disables `faulthandler`.

</div>

<div class="function">

gc_collect()

Force as many objects as possible to be collected. This is needed because timely deallocation is not guaranteed by the garbage collector. This means that `__del__` methods may be called later than expected and weakrefs may remain alive for longer than expected.

</div>

<div class="function">

disable_gc()

A context manager that disables the garbage collector on entry. On exit, the garbage collector is restored to its prior state.

</div>

<div class="function">

swap_attr(obj, attr, new_val)

Context manager to swap out an attribute with a new object.

Usage:

    with swap_attr(obj, "attr", 5):
        ...

This will set `obj.attr` to 5 for the duration of the `with` block, restoring the old value at the end of the block. If `attr` doesn't exist on `obj`, it will be created and then deleted at the end of the block.

The old value (or `None` if it doesn't exist) will be assigned to the target of the "as" clause, if there is one.

</div>

<div class="function">

swap_item(obj, attr, new_val)

Context manager to swap out an item with a new object.

Usage:

    with swap_item(obj, "item", 5):
        ...

This will set `obj["item"]` to 5 for the duration of the `with` block, restoring the old value at the end of the block. If `item` doesn't exist on `obj`, it will be created and then deleted at the end of the block.

The old value (or `None` if it doesn't exist) will be assigned to the target of the "as" clause, if there is one.

</div>

<div class="function">

flush_std_streams()

Call the `flush()` method on `sys.stdout` and then on `sys.stderr`. It can be used to make sure that the logs order is consistent before writing into stderr.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

print_warning(msg)

Print a warning into `sys.__stderr__`. Format the message as: `f"Warning -- {msg}"`. If *msg* is made of multiple lines, add `"Warning -- "` prefix to each line.

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

wait_process(pid, \*, exitcode, timeout=None)

Wait until process *pid* completes and check that the process exit code is *exitcode*.

Raise an `AssertionError` if the process exit code is not equal to *exitcode*.

If the process runs longer than *timeout* seconds (`SHORT_TIMEOUT` by default), kill the process and raise an `AssertionError`. The timeout feature is not available on Windows.

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

calcobjsize(fmt)

Return the size of the `PyObject` whose structure members are defined by *fmt*. The returned value includes the size of the Python object header and alignment.

</div>

<div class="function">

calcvobjsize(fmt)

Return the size of the `PyVarObject` whose structure members are defined by *fmt*. The returned value includes the size of the Python object header and alignment.

</div>

<div class="function">

checksizeof(test, o, size)

For testcase *test*, assert that the `sys.getsizeof` for *o* plus the GC header size equals *size*.

</div>

<div class="decorator">

anticipate_failure(condition)

A decorator to conditionally mark tests with `unittest.expectedFailure`. Any use of this decorator should have an associated comment identifying the relevant tracker issue.

</div>

<div class="function">

system_must_validate_cert(f)

A decorator that skips the decorated test on TLS certification validation failures.

</div>

<div class="decorator">

run_with_locale(catstr, \*locales)

A decorator for running a function in a different locale, correctly resetting it after it has finished. *catstr* is the locale category as a string (for example `"LC_ALL"`). The *locales* passed will be tried sequentially, and the first valid locale will be used.

</div>

<div class="decorator">

run_with_tz(tz)

A decorator for running a function in a specific timezone, correctly resetting it after it has finished.

</div>

<div class="decorator">

requires_freebsd_version(\*min_version)

Decorator for the minimum version when running test on FreeBSD. If the FreeBSD version is less than the minimum, the test is skipped.

</div>

<div class="decorator">

requires_linux_version(\*min_version)

Decorator for the minimum version when running test on Linux. If the Linux version is less than the minimum, the test is skipped.

</div>

<div class="decorator">

requires_mac_version(\*min_version)

Decorator for the minimum version when running test on macOS. If the macOS version is less than the minimum, the test is skipped.

</div>

<div class="decorator">

requires_gil_enabled

Decorator for skipping tests on the free-threaded build. If the `GIL` is disabled, the test is skipped.

</div>

<div class="decorator">

requires_IEEE_754

Decorator for skipping tests on non-IEEE 754 platforms.

</div>

<div class="decorator">

requires_zlib

Decorator for skipping tests if `zlib` doesn't exist.

</div>

<div class="decorator">

requires_gzip

Decorator for skipping tests if `gzip` doesn't exist.

</div>

<div class="decorator">

requires_bz2

Decorator for skipping tests if `bz2` doesn't exist.

</div>

<div class="decorator">

requires_lzma

Decorator for skipping tests if `lzma` doesn't exist.

</div>

<div class="decorator">

requires_resource(resource)

Decorator for skipping tests if *resource* is not available.

</div>

<div class="decorator">

requires_docstrings

Decorator for only running the test if `HAVE_DOCSTRINGS`.

</div>

<div class="decorator">

requires_limited_api

Decorator for only running the test if `Limited C API <limited-c-api>` is available.

</div>

<div class="decorator">

cpython_only

Decorator for tests only applicable to CPython.

</div>

<div class="decorator">

impl_detail(msg=None, \*\*guards)

Decorator for invoking `check_impl_detail` on *guards*. If that returns `False`, then uses *msg* as the reason for skipping the test.

</div>

<div class="decorator">

thread_unsafe(reason=None)

Decorator for marking tests as thread-unsafe. This test always runs in one thread even when invoked with `--parallel-threads`.

</div>

<div class="decorator">

no_tracing

Decorator to temporarily turn off tracing for the duration of the test.

</div>

<div class="decorator">

refcount_test

Decorator for tests which involve reference counting. The decorator does not run the test if it is not run by CPython. Any trace function is unset for the duration of the test to prevent unexpected refcounts caused by the trace function.

</div>

<div class="decorator">

bigmemtest(size, memuse, dry_run=True)

Decorator for bigmem tests.

*size* is a requested size for the test (in arbitrary, test-interpreted units.) *memuse* is the number of bytes per unit for the test, or a good estimate of it. For example, a test that needs two byte buffers, of 4 GiB each, could be decorated with `@bigmemtest(size=_4G, memuse=2)`.

The *size* argument is normally passed to the decorated test method as an extra argument. If *dry_run* is `True`, the value passed to the test method may be less than the requested value. If *dry_run* is `False`, it means the test doesn't support dummy runs when `-M` is not specified.

</div>

<div class="decorator">

bigaddrspacetest

Decorator for tests that fill the address space.

</div>

<div class="function">

linked_to_musl()

Return `False` if there is no evidence the interpreter was compiled with `musl`, otherwise return a version triple, either `(0, 0, 0)` if the version is unknown, or the actual version if it is known. Intended for use in `skip` decorators. `emscripten` and `wasi` are assumed to be compiled with `musl`; otherwise `platform.libc_ver` is checked.

</div>

<div class="function">

check_syntax_error(testcase, statement, errtext='', \*, lineno=None, offset=None)

Test for syntax errors in *statement* by attempting to compile *statement*. *testcase* is the `unittest` instance for the test. *errtext* is the regular expression which should match the string representation of the raised `SyntaxError`. If *lineno* is not `None`, compares to the line of the exception. If *offset* is not `None`, compares to the offset of the exception.

</div>

<div class="function">

open_urlresource(url, *args,*\*kw)

Open *url*. If open fails, raises `TestFailed`.

</div>

<div class="function">

reap_children()

Call this from `~unittest.TestCase.setUp`, `~unittest.TestCase.tearDown`, or `tearDownModule()` in tests that start subprocesses. This will help ensure that no extra children (zombies) stick around to hog resources and create problems when looking for refleaks.

</div>

<div class="function">

get_attribute(obj, name)

Get an attribute, raising `unittest.SkipTest` if `AttributeError` is raised.

</div>

<div class="function">

catch_unraisable_exception()

Context manager catching unraisable exception using `sys.unraisablehook`.

Storing the exception value (`cm.unraisable.exc_value`) creates a reference cycle. The reference cycle is broken explicitly when the context manager exits.

Storing the object (`cm.unraisable.object`) can resurrect it if it is set to an object which is being finalized. Exiting the context manager clears the stored object.

Usage:

    with support.catch_unraisable_exception() as cm:
        # code creating an "unraisable exception"
        ...

        # check the unraisable exception: use cm.unraisable
        ...

    # cm.unraisable attribute no longer exists at this point
    # (to break a reference cycle)

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

load_package_tests(pkg_dir, loader, standard_tests, pattern)

Generic implementation of the `unittest` `load_tests` protocol for use in test packages. *pkg_dir* is the root directory of the package; *loader*, *standard_tests*, and *pattern* are the arguments expected by `load_tests`. In simple cases, the test package's `__init__.py` can be the following:

    import os
    from test.support import load_package_tests

    def load_tests(*args):
        return load_package_tests(os.path.dirname(__file__), *args)

</div>

<div class="function">

detect_api_mismatch(ref_api, other_api, \*, ignore=())

Returns the set of attributes, functions or methods of *ref_api* not found on *other_api*, except for a defined list of items to be ignored in this check specified in *ignore*.

By default this skips private attributes beginning with '\_' but includes all magic methods, i.e. those starting and ending in '\_\_'.

<div class="versionadded">

3.5

</div>

</div>

<div class="function">

patch(test_instance, object_to_patch, attr_name, new_value)

Override *object_to_patch.attr_name* with *new_value*. Also add cleanup procedure to *test_instance* to restore *object_to_patch* for *attr_name*. The *attr_name* should be a valid attribute for *object_to_patch*.

</div>

<div class="function">

run_in_subinterp(code)

Run *code* in subinterpreter. Raise `unittest.SkipTest` if `tracemalloc` is enabled.

</div>

<div class="currentmodule">

test.support.isolation

</div>

<div class="decorator">

runInSubprocess(\*, options=(), env=None, timeout=None)

Decorator that runs the decorated test in a fresh interpreter subprocess, in isolation, so that it does not share global or interpreter state with the rest of the test run. It can decorate a test method or a whole `~unittest.TestCase` subclass. Decorated methods must take no extra arguments. A failure, error or skip in the subprocess is reported for the corresponding test, and individual `subtests
<unittest.TestCase.subTest>` that fail or are skipped are reported individually. A reported failure or error shows the original subprocess traceback as the cause of the exception.

When a **method** is decorated, only that method runs in a subprocess; all fixtures (`~unittest.TestCase.setUp` / `~unittest.TestCase.tearDown`, `~unittest.TestCase.setUpClass` / `~unittest.TestCase.tearDownClass` and `setUpModule()` / `tearDownModule()`) run both in the parent process (as usual) and in the subprocess around the method.

When a **class** is decorated, the whole class runs in a single subprocess, and `~unittest.TestCase.setUpClass`, `~unittest.TestCase.tearDownClass`, `~unittest.TestCase.setUp` and `~unittest.TestCase.tearDown` run once each in the subprocess and are skipped in the parent process. A failure or skip of `~unittest.TestCase.setUpClass` in the subprocess is reported for the whole class. `setUpModule()` cannot be controlled by a class decorator, so it still runs in the parent process too; test it with `runningInSubprocess` if needed.

The subprocess inherits the enabled resources (`-u`), memory limit (`-M`) and verbosity (`-v`) of the parent test run, so that `~test.support.requires_resource`, `~test.support.requires`, `~test.support.bigmemtest` and the like behave consistently in both processes.

*options* is a sequence of interpreter command line options to run the subprocess with, and *env* is a mapping of environment variables to set in it, on top of the inherited environment. A value of `None` in *env* unsets the variable. Note that `-E` and `-I` make the subprocess ignore the `PYTHON*` environment variables, including `PYTHONPATH`.

*timeout* is the number of seconds to wait for the subprocess; the test is reported as an error if it does not complete in time. By default there is no timeout, and a hung test is left to the timeout of the test runner.

The test is skipped on platforms without subprocess support.

</div>

<div class="data">

runningInSubprocess

`True` while the code runs in the isolated subprocess spawned by `runInSubprocess`, and `False` otherwise (including in the parent process and in a normal, non-isolated test run). Fixtures such as `~unittest.TestCase.setUp`, `~unittest.TestCase.tearDown`, `~unittest.TestCase.setUpClass`, `~unittest.TestCase.tearDownClass`, `setUpModule()` and `tearDownModule()` can test it to choose which code to run in the subprocess.

</div>

<div class="currentmodule">

test.support

</div>

<div class="function">

check_free_after_iterating(test, iter, cls, args=())

Assert instances of *cls* are deallocated after iterating.

</div>

<div class="function">

missing_compiler_executable(cmd_names=\[\])

Check for the existence of the compiler executables whose names are listed in *cmd_names* or all the compiler executables when *cmd_names* is empty and return the first missing executable or `None` when none is found missing.

</div>

<div class="function">

check\_\_all\_\_(test_case, module, name_of_module=None, extra=(), not_exported=())

Assert that the `__all__` variable of *module* contains all public names.

The module's public names (its API) are detected automatically based on whether they match the public name convention and were defined in *module*.

The *name_of_module* argument can specify (as a string or tuple thereof) what module(s) an API could be defined in order to be detected as a public API. One case for this is when *module* imports part of its public API from other modules, possibly a C backend (like `csv` and its `_csv`).

The *extra* argument can be a set of names that wouldn't otherwise be automatically detected as "public", like objects without a proper `~definition.__module__` attribute. If provided, it will be added to the automatically detected ones.

The *not_exported* argument can be a set of names that must not be treated as part of the public API even though their names indicate otherwise.

Example use:

    import bar
    import foo
    import unittest
    from test import support

    class MiscTestCase(unittest.TestCase):
        def test__all__(self):
            support.check__all__(self, foo)

    class OtherTestCase(unittest.TestCase):
        def test__all__(self):
            extra = {'BAR_CONST', 'FOO_CONST'}
            not_exported = {'baz'}  # Undocumented name.
            # bar imports part of its API from _bar.
            support.check__all__(self, bar, ('bar', '_bar'),
                                 extra=extra, not_exported=not_exported)

<div class="versionadded">

3.6

</div>

</div>

<div class="function">

check_deprecated_module_attributes(test_case, module, \*names, remove)

Assert that accessing each attribute in *names* on *module* emits a `DeprecationWarning` naming the attribute and the Python version *remove*, given as a `(major, minor)` tuple, in which it will be removed.

</div>

<div class="function">

skip_if_broken_multiprocessing_synchronize()

Skip tests if the `!multiprocessing.synchronize` module is missing, if there is no available semaphore implementation, or if creating a lock raises an `OSError`.

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

check_disallow_instantiation(test_case, tp, *args,*\*kwds)

Assert that type *tp* cannot be instantiated using *args* and *kwds*.

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

adjust_int_max_str_digits(max_digits)

This function returns a context manager that will change the global `sys.set_int_max_str_digits` setting for the duration of the context to allow execution of test code that needs a different limit on the number of digits when converting between an integer and string.

<div class="versionadded">

3.11

</div>

</div>

The `!test.support` module defines the following classes:

<div class="SuppressCrashReport()">

A context manager used to try to prevent crash dialog popups on tests that are expected to crash a subprocess.

On Windows, it disables Windows Error Reporting dialogs using [SetErrorMode](https://msdn.microsoft.com/en-us/library/windows/desktop/ms680621.aspx).

On UNIX, `resource.setrlimit` is used to set `resource.RLIMIT_CORE`'s soft limit to 0 to prevent coredump file creation.

On both platforms, the old value is restored by `~object.__exit__`.

</div>

<div class="SaveSignals()">

Class to save and restore signal handlers registered by the Python signal handler.

<div class="method">

save(self)

Save the signal handlers to a dictionary mapping signal numbers to the current signal handler.

</div>

<div class="method">

restore(self)

Set the signal numbers from the `save` dictionary to the saved handler.

</div>

</div>

<div class="Matcher()">

<div class="method">

matches(self, d, \*\*kwargs)

Try to match a single dict with the supplied arguments.

</div>

<div class="method">

match_value(self, k, dv, v)

Try to match a single stored value (*dv*) with a supplied value (*v*).

</div>

</div>

# `!test.support.socket_helper` --- Utilities for socket tests

<div class="module" data-synopsis="Support for socket tests.">

test.support.socket_helper

</div>

The `!test.support.socket_helper` module provides support for socket tests.

<div class="versionadded">

3.9

</div>

<div class="data">

IPV6_ENABLED

Set to `True` if IPv6 is enabled on this host, `False` otherwise.

</div>

<div class="function">

find_unused_port(family=socket.AF_INET, socktype=socket.SOCK_STREAM)

Returns an unused port that should be suitable for binding. This is achieved by creating a temporary socket with the same family and type as the `sock` parameter (default is `~socket.AF_INET`, `~socket.SOCK_STREAM`), and binding it to the specified host address (defaults to `0.0.0.0`) with the port set to 0, eliciting an unused ephemeral port from the OS. The temporary socket is then closed and deleted, and the ephemeral port is returned.

Either this method or `bind_port` should be used for any tests where a server socket needs to be bound to a particular port for the duration of the test. Which one to use depends on whether the calling code is creating a Python socket, or if an unused port needs to be provided in a constructor or passed to an external program (i.e. the `-accept` argument to openssl's s_server mode). Always prefer `bind_port` over `find_unused_port` where possible. Using a hard coded port is discouraged since it can make multiple instances of the test impossible to run simultaneously, which is a problem for buildbots.

</div>

<div class="function">

bind_port(sock, host=HOST)

Bind the socket to a free port and return the port number. Relies on ephemeral ports in order to ensure we are using an unbound port. This is important as many tests may be running simultaneously, especially in a buildbot environment. This method raises an exception if the `sock.family` is `~socket.AF_INET` and `sock.type` is `~socket.SOCK_STREAM`, and the socket has `~socket.SO_REUSEADDR` or `~socket.SO_REUSEPORT` set on it. Tests should never set these socket options for TCP/IP sockets. The only case for setting these options is testing multicasting via multiple UDP sockets.

Additionally, if the `~socket.SO_EXCLUSIVEADDRUSE` socket option is available (i.e. on Windows), it will be set on the socket. This will prevent anyone else from binding to our host/port for the duration of the test.

</div>

<div class="function">

bind_unix_socket(sock, addr)

Bind a Unix socket, raising `unittest.SkipTest` if `PermissionError` is raised.

</div>

<div class="decorator">

skip_unless_bind_unix_socket

A decorator for running tests that require a functional `bind()` for Unix sockets.

</div>

<div class="function">

transient_internet(resource_name, \*, timeout=30.0, errnos=())

A context manager that raises `~test.support.ResourceDenied` when various issues with the internet connection manifest themselves as exceptions.

</div>

# `!test.support.script_helper` --- Utilities for the Python execution tests

<div class="module" data-synopsis="Support for Python's script execution tests.">

test.support.script_helper

</div>

The `!test.support.script_helper` module provides support for Python's script execution tests.

<div class="function">

interpreter_requires_environment()

Return `True` if `sys.executable interpreter` requires environment variables in order to be able to run at all.

This is designed to be used with `@unittest.skipIf()` to annotate tests that need to use an `assert_python*()` function to launch an isolated mode (`-I`) or no environment mode (`-E`) sub-interpreter process.

A normal build & test does not run into this situation but it can happen when trying to run the standard library test suite from an interpreter that doesn't have an obvious home with Python's current home finding logic.

Setting `PYTHONHOME` is one way to get most of the testsuite to run in that situation. `PYTHONPATH` or `PYTHONNOUSERSITE` are other common environment variables that might impact whether or not the interpreter can start.

</div>

<div class="function">

run_python_until_end(*args,*\*env_vars)

Set up the environment based on *env_vars* for running the interpreter in a subprocess. The values can include `__isolated`, `__cleanenv`, `__cwd`, and `TERM`.

<div class="versionchanged">

3.9 The function no longer strips whitespaces from *stderr*.

</div>

</div>

<div class="function">

assert_python_ok(*args,*\*env_vars)

Assert that running the interpreter with *args* and optional environment variables *env_vars* succeeds (`rc == 0`) and return a `(return code, stdout, stderr)` tuple.

If the *\_\_cleanenv* keyword-only parameter is set, *env_vars* is used as a fresh environment.

Python is started in isolated mode (command line option `-I`), except if the *\_\_isolated* keyword-only parameter is set to `False`.

<div class="versionchanged">

3.9 The function no longer strips whitespaces from *stderr*.

</div>

</div>

<div class="function">

assert_python_failure(*args,*\*env_vars)

Assert that running the interpreter with *args* and optional environment variables *env_vars* fails (`rc != 0`) and return a `(return code, stdout, stderr)` tuple.

See `assert_python_ok` for more options.

<div class="versionchanged">

3.9 The function no longer strips whitespaces from *stderr*.

</div>

</div>

<div class="function">

spawn_python(*args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,*\*kw)

Run a Python subprocess with the given arguments.

*kw* is extra keyword args to pass to `subprocess.Popen`. Returns a `subprocess.Popen` object.

</div>

<div class="function">

kill_python(p)

Run the given `subprocess.Popen` process until completion and return stdout.

</div>

<div class="function">

make_script(script_dir, script_basename, source, omit_suffix=False)

Create script containing *source* in path *script_dir* and *script_basename*. If *omit_suffix* is `False`, append `.py` to the name. Return the full script path.

</div>

<div class="function">

make_zip_script(zip_dir, zip_basename, script_name, name_in_zip=None)

Create zip file at *zip_dir* and *zip_basename* with extension `zip` which contains the files in *script_name*. *name_in_zip* is the archive name. Return a tuple containing `(full path, full path of archive name)`.

</div>

<div class="function">

make_pkg(pkg_dir, init_source='')

Create a directory named *pkg_dir* containing an `__init__` file with *init_source* as its contents.

</div>

<div class="function">

make_zip_pkg(zip_dir, zip_basename, pkg_name, script_basename, source, depth=1, compiled=False)

Create a zip package directory with a path of *zip_dir* and *zip_basename* containing an empty `__init__` file and a file *script_basename* containing the *source*. If *compiled* is `True`, both source files will be compiled and added to the zip package. Return a tuple of the full zip path and the archive name for the zip file.

</div>

# `!test.support.bytecode_helper` --- Support tools for testing correct bytecode generation

<div class="module" data-synopsis="Support tools for testing correct bytecode generation.">

test.support.bytecode_helper

</div>

The `!test.support.bytecode_helper` module provides support for testing and inspecting bytecode generation.

<div class="versionadded">

3.9

</div>

The module defines the following class:

<div class="BytecodeTestCase(unittest.TestCase)">

This class has custom assertion methods for inspecting bytecode.

</div>

<div class="method">

BytecodeTestCase.get_disassembly_as_string(co)

Return the disassembly of *co* as string.

</div>

<div class="method">

BytecodeTestCase.assertInBytecode(x, opname, argval=\_UNSPECIFIED)

Return instr if *opname* is found, otherwise throws `AssertionError`.

</div>

<div class="method">

BytecodeTestCase.assertNotInBytecode(x, opname, argval=\_UNSPECIFIED)

Throws `AssertionError` if *opname* is found.

</div>

# `!test.support.threading_helper` --- Utilities for threading tests

<div class="module" data-synopsis="Support for threading tests.">

test.support.threading_helper

</div>

The `!test.support.threading_helper` module provides support for threading tests.

<div class="versionadded">

3.10

</div>

<div class="function">

join_thread(thread, timeout=None)

Join a *thread* within *timeout*. Raise an `AssertionError` if thread is still alive after *timeout* seconds.

</div>

<div class="decorator">

reap_threads

Decorator to ensure the threads are cleaned up even if the test fails.

</div>

<div class="function">

start_threads(threads, unlock=None)

Context manager to start *threads*, which is a sequence of threads. *unlock* is a function called after the threads are started, even if an exception was raised; an example would be `threading.Event.set`. `start_threads` will attempt to join the started threads upon exit.

</div>

<div class="function">

threading_cleanup(\*original_values)

Cleanup up threads not specified in *original_values*. Designed to emit a warning if a test leaves running threads in the background.

</div>

<div class="function">

threading_setup()

Return current thread count and copy of dangling threads.

</div>

<div class="function">

wait_threads_exit(timeout=None)

Context manager to wait until all threads created in the `with` statement exit.

</div>

<div class="function">

catch_threading_exception()

Context manager catching `threading.Thread` exception using `threading.excepthook`.

Attributes set when an exception is caught:

- `exc_type`
- `exc_value`
- `exc_traceback`
- `thread`

See `threading.excepthook` documentation.

These attributes are deleted at the context manager exit.

Usage:

    with threading_helper.catch_threading_exception() as cm:
        # code spawning a thread which raises an exception
        ...

        # check the thread exception, use cm attributes:
        # exc_type, exc_value, exc_traceback, thread
        ...

    # exc_type, exc_value, exc_traceback, thread attributes of cm no longer
    # exists at this point
    # (to avoid reference cycles)

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

run_concurrently(worker_func, nthreads, args=(), kwargs={})

Run the worker function concurrently in multiple threads. Re-raises an exception if any thread raises one, after all threads have finished.

</div>

# `!test.support.os_helper` --- Utilities for os tests

<div class="module" data-synopsis="Support for os tests.">

test.support.os_helper

</div>

The `!test.support.os_helper` module provides support for os tests.

<div class="versionadded">

3.10

</div>

<div class="data">

FS_NONASCII

A non-ASCII character encodable by `os.fsencode`.

</div>

<div class="data">

SAVEDCWD

Set to `os.getcwd`.

</div>

<div class="data">

TESTFN

Set to a name that is safe to use as the name of a temporary file. Any temporary file that is created should be closed and unlinked (removed).

</div>

<div class="data">

TESTFN_NONASCII

Set to a filename containing the `FS_NONASCII` character, if it exists. This guarantees that if the filename exists, it can be encoded and decoded with the default filesystem encoding. This allows tests that require a non-ASCII filename to be easily skipped on platforms where they can't work.

</div>

<div class="data">

TESTFN_UNENCODABLE

Set to a filename (str type) that should not be able to be encoded by file system encoding in strict mode. It may be `None` if it's not possible to generate such a filename.

</div>

<div class="data">

TESTFN_UNDECODABLE

Set to a filename (bytes type) that should not be able to be decoded by file system encoding in strict mode. It may be `None` if it's not possible to generate such a filename.

</div>

<div class="data">

TESTFN_UNICODE

Set to a non-ASCII name for a temporary file.

</div>

<div class="EnvironmentVarGuard()">

Class used to temporarily set or unset environment variables. Instances can be used as a context manager and have a complete dictionary interface for querying/modifying the underlying `os.environ`. After exit from the context manager all changes to environment variables done through this instance will be rolled back.

<div class="versionchanged">

3.1 Added dictionary interface.

</div>

</div>

<div class="FakePath(path)">

Simple `path-like object`. It implements the `~os.PathLike.__fspath__` method which just returns the *path* argument. If *path* is an exception, it will be raised in `!__fspath__`.

</div>

<div class="method">

EnvironmentVarGuard.set(envvar, value)

Temporarily set the environment variable `envvar` to the value of `value`.

</div>

<div class="method">

EnvironmentVarGuard.unset(envvar, \*others)

Temporarily unset one or more environment variables.

<div class="versionchanged">

3.14 More than one environment variable can be unset.

</div>

</div>

<div class="function">

can_symlink()

Return `True` if the OS supports symbolic links, `False` otherwise.

</div>

<div class="function">

can_xattr()

Return `True` if the OS supports xattr, `False` otherwise.

</div>

<div class="function">

change_cwd(path, quiet=False)

A context manager that temporarily changes the current working directory to *path* and yields the directory.

If *quiet* is `False`, the context manager raises an exception on error. Otherwise, it issues only a warning and keeps the current working directory the same.

</div>

<div class="function">

create_empty_file(filename)

Create an empty file with *filename*. If it already exists, truncate it.

</div>

<div class="function">

fd_count()

Count the number of open file descriptors.

</div>

<div class="function">

fs_is_case_insensitive(directory)

Return `True` if the file system for *directory* is case-insensitive.

</div>

<div class="function">

make_bad_fd()

Create an invalid file descriptor by opening and closing a temporary file, and returning its descriptor.

</div>

<div class="function">

rmdir(filename)

Call `os.rmdir` on *filename*. On Windows platforms, this is wrapped with a wait loop that checks for the existence of the file, which is needed due to antivirus programs that can hold files open and prevent deletion.

</div>

<div class="function">

rmtree(path)

Call `shutil.rmtree` on *path* or call `os.lstat` and `os.rmdir` to remove a path and its contents. As with `rmdir`, on Windows platforms this is wrapped with a wait loop that checks for the existence of the files.

</div>

<div class="decorator">

skip_unless_symlink

A decorator for running tests that require support for symbolic links.

</div>

<div class="decorator">

skip_unless_xattr

A decorator for running tests that require support for xattr.

</div>

<div class="function">

temp_cwd(name='tempcwd', quiet=False)

A context manager that temporarily creates a new directory and changes the current working directory (CWD).

The context manager creates a temporary directory in the current directory with name *name* before temporarily changing the current working directory. If *name* is `None`, the temporary directory is created using `tempfile.mkdtemp`.

If *quiet* is `False` and it is not possible to create or change the CWD, an error is raised. Otherwise, only a warning is raised and the original CWD is used.

</div>

<div class="function">

temp_dir(path=None, quiet=False)

A context manager that creates a temporary directory at *path* and yields the directory.

If *path* is `None`, the temporary directory is created using `tempfile.mkdtemp`. If *quiet* is `False`, the context manager raises an exception on error. Otherwise, if *path* is specified and cannot be created, only a warning is issued.

</div>

<div class="function">

temp_umask(umask)

A context manager that temporarily sets the process umask.

</div>

<div class="function">

unlink(filename)

Call `os.unlink` on *filename*. As with `rmdir`, on Windows platforms, this is wrapped with a wait loop that checks for the existence of the file.

</div>

<div class="decorator">

with_source_date_epoch(\*, epoch=123456789)

A decorator for running tests with the `SOURCE_DATE_EPOCH` environment variable set to *epoch*.

</div>

<div class="decorator">

without_source_date_epoch

A decorator for running tests with the `SOURCE_DATE_EPOCH` environment variable unset.

</div>

<div class="SourceDateEpochTestMeta">

Metaclass wrapping all test methods of the class with `with_source_date_epoch` if the *source_date_epoch* keyword class argument is true, or with `without_source_date_epoch` otherwise. For example:

    class TestsWithSourceEpoch(Tests,
                               metaclass=SourceDateEpochTestMeta,
                               source_date_epoch=True):
        pass

</div>

# `!test.support.import_helper` --- Utilities for import tests

<div class="module" data-synopsis="Support for import tests.">

test.support.import_helper

</div>

The `!test.support.import_helper` module provides support for import tests.

<div class="versionadded">

3.10

</div>

<div class="function">

forget(module_name)

Remove the module named *module_name* from `sys.modules` and delete any byte-compiled files of the module.

</div>

<div class="function">

import_fresh_module(name, fresh=(), blocked=(), deprecated=False)

This function imports and returns a fresh copy of the named Python module by removing the named module from `sys.modules` before doing the import. Note that unlike `importlib.reload`, the original module is not affected by this operation.

*fresh* is an iterable of additional module names that are also removed from the `sys.modules` cache before doing the import.

*blocked* is an iterable of module names that are replaced with `None` in the module cache during the import to ensure that attempts to import them raise `ImportError`.

The named module and any modules named in the *fresh* and *blocked* parameters are saved before starting the import and then reinserted into `sys.modules` when the fresh import is complete.

Module and package deprecation messages are suppressed during this import if *deprecated* is `True`.

This function will raise `ImportError` if the named module cannot be imported.

Example use:

    # Get copies of the warnings module for testing without affecting the
    # version being used by the rest of the test suite. One copy uses the
    # C implementation, the other is forced to use the pure Python fallback
    # implementation
    py_warnings = import_fresh_module('warnings', blocked=['_warnings'])
    c_warnings = import_fresh_module('warnings', fresh=['_warnings'])

<div class="versionadded">

3.1

</div>

</div>

<div class="function">

import_module(name, deprecated=False, \*, required_on=())

This function imports and returns the named module. Unlike a normal import, this function raises `unittest.SkipTest` if the module cannot be imported.

Module and package deprecation messages are suppressed during this import if *deprecated* is `True`. If a module is required on a platform but optional for others, set *required_on* to an iterable of platform prefixes which will be compared against `sys.platform`.

<div class="versionadded">

3.1

</div>

</div>

<div class="function">

modules_setup()

Return a copy of `sys.modules`.

</div>

<div class="function">

modules_cleanup(oldmodules)

Remove modules except for *oldmodules* and `encodings` in order to preserve internal cache.

</div>

<div class="function">

unload(name)

Delete *name* from `sys.modules`.

</div>

<div class="function">

make_legacy_pyc(source)

Move a `3147`/`488` pyc file to its legacy pyc location and return the file system path to the legacy pyc file. The *source* value is the file system path to the source file. It does not need to exist, however the PEP 3147/488 pyc file must exist.

</div>

<div class="CleanImport(*module_names)">

A context manager to force import to return a new module reference. This is useful for testing module-level behaviors, such as the emission of a `DeprecationWarning` on import. Example usage:

    with CleanImport('foo'):
        importlib.import_module('foo')  # New reference.

</div>

<div class="DirsOnSysPath(*paths)">

A context manager to temporarily add directories to `sys.path`.

This makes a copy of `sys.path`, appends any directories given as positional arguments, then reverts `sys.path` to the copied settings when the context ends.

Note that *all* `sys.path` modifications in the body of the context manager, including replacement of the object, will be reverted at the end of the block.

</div>

# `!test.support.warnings_helper` --- Utilities for warnings tests

<div class="module" data-synopsis="Support for warnings tests.">

test.support.warnings_helper

</div>

The `!test.support.warnings_helper` module provides support for warnings tests.

<div class="versionadded">

3.10

</div>

<div class="function">

ignore_warnings(\*, category)

Suppress warnings that are instances of *category*, which must be `Warning` or a subclass. Roughly equivalent to `warnings.catch_warnings` with `warnings.simplefilter('ignore', category=category) <warnings.simplefilter>`. For example:

    @warning_helper.ignore_warnings(category=DeprecationWarning)
    def test_suppress_warning():
        # do something

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

check_no_resource_warning(testcase)

Context manager to check that no `ResourceWarning` was raised. You must remove the object which may emit `ResourceWarning` before the end of the context manager.

</div>

<div class="function">

check_syntax_warning(testcase, statement, errtext='', \*, lineno=1, offset=None)

Test for syntax warning in *statement* by attempting to compile *statement*. Test also that the `SyntaxWarning` is emitted only once, and that it will be converted to a `SyntaxError` when turned into error. *testcase* is the `unittest` instance for the test. *errtext* is the regular expression which should match the string representation of the emitted `SyntaxWarning` and raised `SyntaxError`. If *lineno* is not `None`, compares to the line of the warning and exception. If *offset* is not `None`, compares to the offset of the exception.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

check_warnings(\*filters, quiet=True)

A convenience wrapper for `warnings.catch_warnings` that makes it easier to test that a warning was correctly raised. It is approximately equivalent to calling `warnings.catch_warnings(record=True)` with `warnings.simplefilter` set to `always` and with the option to automatically validate the results that are recorded.

`check_warnings` accepts 2-tuples of the form `("message regexp", WarningCategory)` as positional arguments. If one or more *filters* are provided, or if the optional keyword argument *quiet* is `False`, it checks to make sure the warnings are as expected: each specified filter must match at least one of the warnings raised by the enclosed code or the test fails, and if any warnings are raised that do not match any of the specified filters the test fails. To disable the first of these checks, set *quiet* to `True`.

If no arguments are specified, it defaults to:

    check_warnings(("", Warning), quiet=True)

In this case all warnings are caught and no errors are raised.

On entry to the context manager, a `WarningsRecorder` instance is returned. The recorded warnings are available via its `~WarningsRecorder.warnings` attribute, and its `~WarningsRecorder.reset` method discards previously recorded warnings.

The context manager is designed to be used like this:

    with check_warnings(("assertion is always true", SyntaxWarning),
                        ("", UserWarning)):
        exec('assert(False, "Hey!")')
        warnings.warn(UserWarning("Hide me!"))

In this case if either warning was not raised, or some other warning was raised, `check_warnings` would raise an error.

When a test needs to look more deeply into the warnings, rather than just checking whether or not they occurred, code like this can be used:

    with check_warnings(quiet=True) as w:
        warnings.warn("foo")
        assert str(w.message) == "foo"
        warnings.warn("bar")
        assert str(w.message) == "bar"
        assert str(w.warnings[0].message) == "foo"
        assert str(w.warnings[1].message) == "bar"
        w.reset()
        assert len(w.warnings) == 0

Here all warnings will be caught, and the test code tests the captured warnings directly.

<div class="versionchanged">

3.2 New optional arguments *filters* and *quiet*.

</div>

</div>

<div class="WarningsRecorder(warnings_list)">

Class used to record warnings for unit tests. Wraps the warnings list returned by `warnings.catch_warnings` when recording is enabled. `check_warnings` returns an instance of this class.

As a convenience, the attributes of the object representing the most recent warning can also be accessed directly through the recorder object. If no warning has been raised since the recorder was created or last reset, these attributes return `None`.

<div class="attribute">

warnings

A list of warnings recorded since the recorder was created or last reset.

</div>

<div class="method">

reset()

Discard previously recorded warnings from the recorder's view. The underlying warnings list is not modified.

</div>

</div>

# Text Processing Services<span id="stringservices"></span>

The modules described in this chapter provide a wide range of string manipulation operations and other text processing services.

The `codecs` module described under `binaryservices` is also highly relevant to text processing. In addition, see the documentation for Python's built-in string type in `textseq`.

<div class="toctree">

string.rst string.templatelib.rst re.rst difflib.rst textwrap.rst unicodedata.rst stringprep.rst readline.rst rlcompleter.rst

</div>

# `!textwrap` --- Text wrapping and filling

<div class="module" data-synopsis="Text wrapping and filling">

textwrap

</div>

**Source code:** `Lib/textwrap.py`

------------------------------------------------------------------------

The `!textwrap` module provides some convenience functions, as well as `TextWrapper`, the class that does all the work. If you're just wrapping or filling one or two text strings, the convenience functions should be good enough; otherwise, you should use an instance of `TextWrapper` for efficiency.

<div class="function">

wrap(text, width=70, \*, initial_indent="", subsequent_indent="", expand_tabs=True, replace_whitespace=True, fix_sentence_endings=False, break_long_words=True, drop_whitespace=True, break_on_hyphens=True, tabsize=8, max_lines=None, placeholder=' \[...\]')

Wraps the single paragraph in *text* (a string) so every line is at most *width* characters long. Returns a list of output lines, without final newlines.

Optional keyword arguments correspond to the instance attributes of `TextWrapper`, documented below.

See the `TextWrapper.wrap` method for additional details on how `wrap` behaves.

</div>

<div class="function">

fill(text, width=70, \*, initial_indent="", subsequent_indent="", expand_tabs=True, replace_whitespace=True, fix_sentence_endings=False, break_long_words=True, drop_whitespace=True, break_on_hyphens=True, tabsize=8, max_lines=None, placeholder=' \[...\]')

Wraps the single paragraph in *text*, and returns a single string containing the wrapped paragraph. `fill` is shorthand for :

    "\n".join(wrap(text, ...))

In particular, `fill` accepts exactly the same keyword arguments as `wrap`.

</div>

<div class="function">

shorten(text, width, \*, fix_sentence_endings=False, break_long_words=True, break_on_hyphens=True, placeholder=' \[...\]')

Collapse and truncate the given *text* to fit in the given *width*.

First the whitespace in *text* is collapsed (all whitespace is replaced by single spaces). If the result fits in the *width*, it is returned. Otherwise, enough words are dropped from the end so that the remaining words plus the *placeholder* fit within *width*:

    >>> textwrap.shorten("Hello  world!", width=12)
    'Hello world!'
    >>> textwrap.shorten("Hello  world!", width=11)
    'Hello [...]'
    >>> textwrap.shorten("Hello world", width=10, placeholder="...")
    'Hello...'

Optional keyword arguments correspond to the instance attributes of `TextWrapper`, documented below. Note that the whitespace is collapsed before the text is passed to the `TextWrapper` `fill` function, so changing the value of `.tabsize`, `.expand_tabs`, `.drop_whitespace`, and `.replace_whitespace` will have no effect.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

dedent(text)

Remove any common leading whitespace from every line in *text*.

This can be used to make triple-quoted strings line up with the left edge of the display, while still presenting them in the source code in indented form.

Note that tabs and spaces are both treated as whitespace, but they are not equal: the lines `"  hello"` and `"\thello"` are considered to have no common leading whitespace.

Lines containing only whitespace are ignored in the input and normalized to a single newline character in the output.

For example:

    def test():
        # end first line with \ to avoid the empty line!
        s = '''\
        hello
          world
        '''
        print(repr(s))          # prints '    hello\n      world\n    '
        print(repr(dedent(s)))  # prints 'hello\n  world\n'

<div class="versionchanged">

3.14 The `!dedent` function now correctly normalizes blank lines containing only whitespace characters. Previously, the implementation only normalized blank lines containing tabs and spaces.

</div>

</div>

<div class="function">

indent(text, prefix, predicate=None)

Add *prefix* to the beginning of selected lines in *text*.

Lines are separated by calling `text.splitlines(True)`.

By default, *prefix* is added to all lines that do not consist solely of whitespace (including any line endings).

For example:

    >>> s = 'hello\n\n \nworld'
    >>> indent(s, '  ')
    '  hello\n\n \n  world'

The optional *predicate* argument can be used to control which lines are indented. For example, it is easy to add *prefix* to even empty and whitespace-only lines:

    >>> print(indent(s, '+ ', lambda line: True))
    + hello
    +
    +
    + world

<div class="versionadded">

3.3

</div>

</div>

`wrap`, `fill` and `shorten` work by creating a `TextWrapper` instance and calling a single method on it. That instance is not reused, so for applications that process many text strings using `wrap` and/or `fill`, it may be more efficient to create your own `TextWrapper` object.

Text is preferably wrapped on whitespaces and right after the hyphens in hyphenated words; only then will long words be broken if necessary, unless `TextWrapper.break_long_words` is set to false.

<div class="TextWrapper(**kwargs)">

The `TextWrapper` constructor accepts a number of optional keyword arguments. Each keyword argument corresponds to an instance attribute, so for example :

    wrapper = TextWrapper(initial_indent="* ")

is the same as :

    wrapper = TextWrapper()
    wrapper.initial_indent = "* "

You can reuse the same `TextWrapper` object many times, and you can change any of its options through direct assignment to instance attributes between uses.

The `TextWrapper` instance attributes (and keyword arguments to the constructor) are as follows:

<div class="attribute">

width

(default: `70`) The maximum length of wrapped lines. As long as there are no individual words in the input text longer than `width`, `TextWrapper` guarantees that no output line will be longer than `width` characters.

</div>

<div class="attribute">

expand_tabs

(default: `True`) If true, then all tab characters in *text* will be expanded to spaces using the `~str.expandtabs` method of *text*.

</div>

<div class="attribute">

tabsize

(default: `8`) If `expand_tabs` is true, then all tab characters in *text* will be expanded to zero or more spaces, depending on the current column and the given tab size.

<div class="versionadded">

3.3

</div>

</div>

<div class="attribute">

replace_whitespace

(default: `True`) If true, after tab expansion but before wrapping, the `wrap` method will replace each whitespace character with a single space. The whitespace characters replaced are as follows: tab, newline, vertical tab, formfeed, and carriage return (`'\t\n\v\f\r'`).

> [!NOTE]
> If `expand_tabs` is false and `replace_whitespace` is true, each tab character will be replaced by a single space, which is *not* the same as tab expansion.

> [!NOTE]
> If `replace_whitespace` is false, newlines may appear in the middle of a line and cause strange output. For this reason, text should be split into paragraphs (using `str.splitlines` or similar) which are wrapped separately.

</div>

<div class="attribute">

drop_whitespace

(default: `True`) If true, whitespace at the beginning and ending of every line (after wrapping but before indenting) is dropped. Whitespace at the beginning of the paragraph, however, is not dropped if non-whitespace follows it. If whitespace being dropped takes up an entire line, the whole line is dropped.

</div>

<div class="attribute">

initial_indent

(default: `''`) String that will be prepended to the first line of wrapped output. Counts towards the length of the first line. The empty string is not indented.

</div>

<div class="attribute">

subsequent_indent

(default: `''`) String that will be prepended to all lines of wrapped output except the first. Counts towards the length of each line except the first.

</div>

<div class="attribute">

fix_sentence_endings

(default: `False`) If true, `TextWrapper` attempts to detect sentence endings and ensure that sentences are always separated by exactly two spaces. This is generally desired for text in a monospaced font. However, the sentence detection algorithm is imperfect: it assumes that a sentence ending consists of a lowercase letter followed by one of `'.'`, `'!'`, or `'?'`, possibly followed by one of `'"'` or `"'"`, followed by a space. One problem with this algorithm is that it is unable to detect the difference between "Dr." in :

    [...] Dr. Frankenstein's monster [...]

and "Spot." in :

    [...] See Spot. See Spot run [...]

`fix_sentence_endings` is false by default.

Since the sentence detection algorithm relies on `string.lowercase` for the definition of "lowercase letter", and a convention of using two spaces after a period to separate sentences on the same line, it is specific to English-language texts.

</div>

<div class="attribute">

break_long_words

(default: `True`) If true, then words longer than `width` will be broken in order to ensure that no lines are longer than `width`. If it is false, long words will not be broken, and some lines may be longer than `width`. (Long words will be put on a line by themselves, in order to minimize the amount by which `width` is exceeded.)

</div>

<div class="attribute">

break_on_hyphens

(default: `True`) If true, wrapping will occur preferably on whitespaces and right after hyphens in compound words, as it is customary in English. If false, only whitespaces will be considered as potentially good places for line breaks, but you need to set `break_long_words` to false if you want truly insecable words. Default behaviour in previous versions was to always allow breaking hyphenated words.

</div>

<div class="attribute">

max_lines

(default: `None`) If not `None`, then the output will contain at most *max_lines* lines, with *placeholder* appearing at the end of the output.

<div class="versionadded">

3.4

</div>

</div>

<div class="index">

single: ...; placeholder

</div>

<div class="attribute">

placeholder

(default: `' [...]'`) String that will appear at the end of the output text if it has been truncated.

<div class="versionadded">

3.4

</div>

</div>

`TextWrapper` also provides some public methods, analogous to the module-level convenience functions:

<div class="method">

wrap(text)

Wraps the single paragraph in *text* (a string) so every line is at most `width` characters long. All wrapping options are taken from instance attributes of the `TextWrapper` instance. Returns a list of output lines, without final newlines. If the wrapped output has no content, the returned list is empty.

</div>

<div class="method">

fill(text)

Wraps the single paragraph in *text*, and returns a single string containing the wrapped paragraph.

</div>

</div>

# `!threading` --- Thread-based parallelism

<div class="module" data-synopsis="Thread-based parallelism.">

threading

</div>

**Source code:** `Lib/threading.py`

------------------------------------------------------------------------

This module constructs higher-level threading interfaces on top of the lower level `_thread` module.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

## Introduction

The `!threading` module provides a way to run multiple [threads](https://en.wikipedia.org/wiki/Thread_(computing)) (smaller units of a process) concurrently within a single process. It allows for the creation and management of threads, making it possible to execute tasks in parallel, sharing memory space. Threads are particularly useful when tasks are I/O bound, such as file operations or making network requests, where much of the time is spent waiting for external resources.

A typical use case for `!threading` includes managing a pool of worker threads that can process multiple tasks concurrently. Here's a basic example of creating and starting threads using `~threading.Thread`:

    import threading
    import time

    def crawl(link, delay=3):
        print(f"crawl started for {link}")
        time.sleep(delay)  # Blocking I/O (simulating a network request)
        print(f"crawl ended for {link}")

    links = [
        "https://python.org",
        "https://docs.python.org",
        "https://peps.python.org",
    ]

    # Start threads for each link
    threads = []
    for link in links:
        # Using `args` to pass positional arguments and `kwargs` for keyword arguments
        t = threading.Thread(target=crawl, args=(link,), kwargs={"delay": 2})
        threads.append(t)

    # Start each thread
    for t in threads:
        t.start()

    # Wait for all threads to finish
    for t in threads:
        t.join()

<div class="versionchanged">

3.7 This module used to be optional, it is now always available.

</div>

<div class="seealso">

`concurrent.futures.ThreadPoolExecutor` offers a higher level interface to push tasks to a background thread without blocking execution of the calling thread, while still being able to retrieve their results when needed.

`queue` provides a thread-safe interface for exchanging data between running threads.

`asyncio` offers an alternative approach to achieving task level concurrency without requiring the use of multiple operating system threads.

</div>

> [!NOTE]
> In the Python 2.x series, this module contained `camelCase` names for some methods and functions. These are deprecated as of Python 3.10, but they are still supported for compatibility with Python 2.5 and lower.

<div class="impl-detail">

In CPython, due to the `Global Interpreter Lock
<global interpreter lock>`, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation). If you want your application to make better use of the computational resources of multi-core machines, you are advised to use `multiprocessing` or `concurrent.futures.ProcessPoolExecutor`. However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously.

</div>

## GIL and performance considerations

Unlike the `multiprocessing` module, which uses separate processes to bypass the `global interpreter lock` (GIL), the threading module operates within a single process, meaning that all threads share the same memory space. However, the GIL limits the performance gains of threading when it comes to CPU-bound tasks, as only one thread can execute Python bytecode at a time. Despite this, threads remain a useful tool for achieving concurrency in many scenarios.

As of Python 3.13, `free-threaded <free threading>` builds can disable the GIL, enabling true parallel execution of threads, but this feature is not available by default (see `703`).

## Reference

This module defines the following functions:

<div class="function">

active_count()

Return the number of `Thread` objects currently alive. The returned count is equal to the length of the list returned by `.enumerate`.

The function `activeCount` is a deprecated alias for this function.

</div>

<div class="function">

current_thread()

Return the current `Thread` object, corresponding to the caller's thread of control. If the caller's thread of control was not created through the `!threading` module, a dummy thread object with limited functionality is returned.

The function `currentThread` is a deprecated alias for this function.

</div>

<div class="function">

excepthook(args, /)

Handle uncaught exception raised by `Thread.run`.

The *args* argument has the following attributes:

- *exc_type*: Exception type.
- *exc_value*: Exception value, can be `None`.
- *exc_traceback*: Exception traceback, can be `None`.
- *thread*: Thread which raised the exception, can be `None`.

If *exc_type* is `SystemExit`, the exception is silently ignored. Otherwise, the exception is printed out on `sys.stderr`.

If this function raises an exception, `sys.excepthook` is called to handle it.

`threading.excepthook` can be overridden to control how uncaught exceptions raised by `Thread.run` are handled.

Storing *exc_value* using a custom hook can create a reference cycle. It should be cleared explicitly to break the reference cycle when the exception is no longer needed.

Storing *thread* using a custom hook can resurrect it if it is set to an object which is being finalized. Avoid storing *thread* after the custom hook completes to avoid resurrecting objects.

<div class="seealso">

`sys.excepthook` handles uncaught exceptions.

</div>

<div class="versionadded">

3.8

</div>

</div>

<div class="data">

\_\_excepthook\_\_

Holds the original value of `threading.excepthook`. It is saved so that the original value can be restored in case they happen to get replaced with broken or alternative objects.

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

get_ident()

Return the 'thread identifier' of the current thread. This is a nonzero integer. Its value has no direct meaning; it is intended as a magic cookie to be used e.g. to index a dictionary of thread-specific data. Thread identifiers may be recycled when a thread exits and another thread is created.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

get_native_id()

Return the native integral Thread ID of the current thread assigned by the kernel. This is a non-negative integer. Its value may be used to uniquely identify this particular thread system-wide (until the thread terminates, after which the value may be recycled by the OS).

<div class="availability">

Windows, FreeBSD, Linux, macOS, OpenBSD, NetBSD, AIX, DragonFlyBSD, GNU/kFreeBSD, Solaris.

</div>

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.13 Added support for GNU/kFreeBSD.

</div>

<div class="versionchanged">

3.15 Added support for Solaris.

</div>

</div>

<div class="function">

enumerate()

Return a list of all `Thread` objects currently active. The list includes daemonic threads and dummy thread objects created by `current_thread`. It excludes terminated threads and threads that have not yet been started. However, the main thread is always part of the result, even when terminated.

</div>

<div class="function">

main_thread()

Return the main `Thread` object. In normal conditions, the main thread is the thread from which the Python interpreter was started.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

settrace(func)

<div class="index">

single: trace function

</div>

Set a trace function for all threads started from the `!threading` module. The *func* will be passed to `sys.settrace` for each thread, before its `~Thread.run` method is called.

</div>

<div class="function">

settrace_all_threads(func)

Set a trace function for all threads started from the `!threading` module and all Python threads that are currently executing.

The *func* will be passed to `sys.settrace` for each thread, before its `~Thread.run` method is called.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

gettrace()

<div class="index">

single: trace function single: debugger

</div>

Get the trace function as set by `settrace`.

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

setprofile(func)

<div class="index">

single: profile function

</div>

Set a profile function for all threads started from the `!threading` module. The *func* will be passed to `sys.setprofile` for each thread, before its `~Thread.run` method is called.

</div>

<div class="function">

setprofile_all_threads(func)

Set a profile function for all threads started from the `!threading` module and all Python threads that are currently executing.

The *func* will be passed to `sys.setprofile` for each thread, before its `~Thread.run` method is called.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

getprofile()

<div class="index">

single: profile function

</div>

Get the profiler function as set by `setprofile`.

<div class="versionadded">

3.10

</div>

</div>

<div class="function">

stack_size(\[size\])

Return the thread stack size used when creating new threads. The optional *size* argument specifies the stack size to be used for subsequently created threads, and must be 0 (use platform or configured default) or a positive integer value of at least 32,768 (32 KiB). If *size* is not specified, 0 is used. If changing the thread stack size is unsupported, a `RuntimeError` is raised. If the specified stack size is invalid, a `ValueError` is raised and the stack size is unmodified. 32 KiB is currently the minimum supported stack size value to guarantee sufficient stack space for the interpreter itself. Note that some platforms may have particular restrictions on values for the stack size, such as requiring a minimum stack size \> 32 KiB or requiring allocation in multiples of the system memory page size - platform documentation should be referred to for more information (4 KiB pages are common; using multiples of 4096 for the stack size is the suggested approach in the absence of more specific information).

<div class="availability">

Windows, pthreads.

Unix platforms with POSIX threads support.

</div>

</div>

This module also defines the following constant:

<div class="data">

TIMEOUT_MAX

The maximum value allowed for the *timeout* parameter of blocking functions (`Lock.acquire`, `RLock.acquire`, `Condition.wait`, etc.). Specifying a timeout greater than this value will raise an `OverflowError`.

<div class="versionadded">

3.2

</div>

</div>

This module defines a number of classes, which are detailed in the sections below.

The design of this module is loosely based on Java's threading model. However, where Java makes locks and condition variables basic behavior of every object, they are separate objects in Python. Python's `Thread` class supports a subset of the behavior of Java's Thread class; currently, there are no priorities, no thread groups, and threads cannot be destroyed, stopped, suspended, resumed, or interrupted. The static methods of Java's Thread class, when implemented, are mapped to module-level functions.

All of the methods described below are executed atomically.

### Thread-local data

Thread-local data is data whose values are thread specific. If you have data that you want to be local to a thread, create a `local` object and use its attributes:

    >>> mydata = local()
    >>> mydata.number = 42
    >>> mydata.number
    42

You can also access the `local`-object's dictionary:

    >>> mydata.__dict__
    {'number': 42}
    >>> mydata.__dict__.setdefault('widgets', [])
    []
    >>> mydata.widgets
    []

If we access the data in a different thread:

    >>> log = []
    >>> def f():
    ...     items = sorted(mydata.__dict__.items())
    ...     log.append(items)
    ...     mydata.number = 11
    ...     log.append(mydata.number)

    >>> import threading
    >>> thread = threading.Thread(target=f)
    >>> thread.start()
    >>> thread.join()
    >>> log
    [[], 11]

we get different data. Furthermore, changes made in the other thread don't affect data seen in this thread:

    >>> mydata.number
    42

Of course, values you get from a `local` object, including their `~object.__dict__` attribute, are for whatever thread was current at the time the attribute was read. For that reason, you generally don't want to save these values across threads, as they apply only to the thread they came from.

You can create custom `local` objects by subclassing the `local` class:

    >>> class MyLocal(local):
    ...     number = 2
    ...     def __init__(self, /, **kw):
    ...         self.__dict__.update(kw)
    ...     def squared(self):
    ...         return self.number ** 2

This can be useful to support default values, methods and initialization. Note that if you define an `~object.__init__` method, it will be called each time the `local` object is used in a separate thread. This is necessary to initialize each thread's dictionary.

Now if we create a `local` object:

    >>> mydata = MyLocal(color='red')

we have a default number:

    >>> mydata.number
    2

an initial color:

    >>> mydata.color
    'red'
    >>> del mydata.color

And a method that operates on the data:

    >>> mydata.squared()
    4

As before, we can access the data in a separate thread:

    >>> log = []
    >>> thread = threading.Thread(target=f)
    >>> thread.start()
    >>> thread.join()
    >>> log
    [[('color', 'red')], 11]

without affecting this thread's data:

    >>> mydata.number
    2
    >>> mydata.color
    Traceback (most recent call last):
    ...
    AttributeError: 'MyLocal' object has no attribute 'color'

Note that subclasses can define `__slots__`, but they are not thread local. They are shared across threads:

    >>> class MyLocal(local):
    ...     __slots__ = 'number'

    >>> mydata = MyLocal()
    >>> mydata.number = 42
    >>> mydata.color = 'red'

So, the separate thread:

    >>> thread = threading.Thread(target=f)
    >>> thread.start()
    >>> thread.join()

affects what we see:

    >>> mydata.number
    11

<div class="local()">

A class that represents thread-local data.

</div>

### Thread objects

The `Thread` class represents an activity that is run in a separate thread of control. There are two ways to specify the activity: by passing a callable object to the constructor, or by overriding the `~Thread.run` method in a subclass. No other methods (except for the constructor) should be overridden in a subclass. In other words, *only* override the `__init__()` and `~Thread.run` methods of this class.

Once a thread object is created, its activity must be started by calling the thread's `~Thread.start` method. This invokes the `~Thread.run` method in a separate thread of control.

Once the thread's activity is started, the thread is considered 'alive'. It stops being alive when its `~Thread.run` method terminates -- either normally, or by raising an unhandled exception. The `~Thread.is_alive` method tests whether the thread is alive.

Other threads can call a thread's `~Thread.join` method. This blocks the calling thread until the thread whose `~Thread.join` method is called is terminated.

A thread has a name. The name can be passed to the constructor, and read or changed through the `~Thread.name` attribute.

If the `~Thread.run` method raises an exception, `threading.excepthook` is called to handle it. By default, `threading.excepthook` ignores silently `SystemExit`.

A thread can be flagged as a "daemon thread". The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the `~Thread.daemon` property or the *daemon* constructor argument.

> [!NOTE]
> Daemon threads are abruptly stopped at shutdown. Their resources (such as open files, database transactions, etc.) may not be released properly. If you want your threads to stop gracefully, make them non-daemonic and use a suitable signalling mechanism such as an `Event`.

There is a "main thread" object; this corresponds to the initial thread of control in the Python program. It is not a daemon thread.

There is the possibility that "dummy thread objects" are created. These are thread objects corresponding to "alien threads", which are threads of control started outside the threading module, such as directly from C code. Dummy thread objects have limited functionality; they are always considered alive and daemonic, and cannot be `joined <meth-thread-join>`. They are never deleted, since it is impossible to detect the termination of alien threads.

<div class="Thread(group=None, target=None, name=None, args=(), kwargs={}, *, \ daemon=None, context=None)">

This constructor should always be called with keyword arguments. Arguments are:

*group* must be `None` as it is reserved for future extension when a `!ThreadGroup` class is implemented.

*target* is the callable object to be invoked by the `run` method. Defaults to `None`, meaning nothing is called.

*name* is the thread name. By default, a unique name is constructed of the form "Thread-*N*" where *N* is a small decimal number, or "Thread-*N* (target)" where "target" is `target.__name__` if the *target* argument is specified.

*args* is a list or tuple of arguments for the target invocation. Defaults to `()`.

*kwargs* is a dictionary of keyword arguments for the target invocation. Defaults to `{}`.

If not `None`, *daemon* explicitly sets whether the thread is daemonic. If `None` (the default), the daemonic property is inherited from the current thread.

*context* is the `~contextvars.Context` value to use when starting the thread. The default value is `None` which indicates that the `sys.flags.thread_inherit_context` flag controls the behaviour. If the flag is true, threads will start with a copy of the context of the caller of `~Thread.start`. If false, they will start with an empty context. To explicitly start with an empty context, pass a new instance of `~contextvars.Context()`. To explicitly start with a copy of the current context, pass the value from `~contextvars.copy_context`. The flag defaults true on free-threaded builds and false otherwise.

If the subclass overrides the constructor, it must make sure to invoke the base class constructor (`Thread.__init__()`) before doing anything else to the thread.

<div class="versionchanged">

3.3 Added the *daemon* parameter.

</div>

<div class="versionchanged">

3.10 Use the *target* name if *name* argument is omitted.

</div>

<div class="versionchanged">

3.14 Added the *context* parameter.

</div>

<div class="method">

start()

Start the thread's activity.

It must be called at most once per thread object. It arranges for the object's `~Thread.run` method to be invoked in a separate thread of control.

This method will raise a `RuntimeError` if called more than once on the same thread object.

If supported, set the operating system thread name to `threading.Thread.name`. The name can be truncated depending on the operating system thread name limits.

<div class="versionchanged">

3.14 Set the operating system thread name.

</div>

</div>

<div class="method">

run()

Method representing the thread's activity.

You may override this method in a subclass. The standard `run` method invokes the callable object passed to the object's constructor as the *target* argument, if any, with positional and keyword arguments taken from the *args* and *kwargs* arguments, respectively.

Using list or tuple as the *args* argument which passed to the `Thread` could achieve the same effect.

Example:

    >>> from threading import Thread
    >>> t = Thread(target=print, args=[1])
    >>> t.run()
    1
    >>> t = Thread(target=print, args=(1,))
    >>> t.run()
    1

</div>

<div id="meth-thread-join">

<div class="method">

join(timeout=None)

Wait until the thread terminates. This blocks the calling thread until the thread whose `~Thread.join` method is called terminates -- either normally or through an unhandled exception -- or until the optional timeout occurs.

When the *timeout* argument is present and not `None`, it should be a real number specifying a timeout for the operation in seconds (or fractions thereof). As `~Thread.join` always returns `None`, you must call `~Thread.is_alive` after `~Thread.join` to decide whether a timeout happened -- if the thread is still alive, the `~Thread.join` call timed out.

When the *timeout* argument is not present or `None`, the operation will block until the thread terminates.

A thread can be joined many times.

`~Thread.join` raises a `RuntimeError` if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to `~Thread.join` a thread before it has been started and attempts to do so raise the same exception.

If an attempt is made to join a running daemonic thread in late stages of `Python finalization <interpreter shutdown>` `!join` raises a `PythonFinalizationError`.

<div class="versionchanged">

3.14

May raise `PythonFinalizationError`.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float.

</div>

</div>

</div>

<div class="attribute">

name

A string used for identification purposes only. It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor.

On some platforms, the thread name is set at the operating system level when the thread starts, so that it is visible in task managers. This name may be truncated to fit in a system-specific limit (for example, 15 bytes on Linux or 63 bytes on macOS).

Changes to *name* are only reflected at the OS level when the currently running thread is renamed. (Setting the *name* attribute of a different thread only updates the Python Thread object.)

</div>

<div class="method">

getName() setName()

Deprecated getter/setter API for `~Thread.name`; use it directly as a property instead.

<div class="deprecated">

3.10

</div>

</div>

<div class="attribute">

ident

The 'thread identifier' of this thread or `None` if the thread has not been started. This is a nonzero integer. See the `get_ident` function. Thread identifiers may be recycled when a thread exits and another thread is created. The identifier is available even after the thread has exited.

</div>

<div class="attribute">

native_id

The Thread ID (`TID`) of this thread, as assigned by the OS (kernel). This is a non-negative integer, or `None` if the thread has not been started. See the `get_native_id` function. This value may be used to uniquely identify this particular thread system-wide (until the thread terminates, after which the value may be recycled by the OS).

> [!NOTE]
> Similar to Process IDs, Thread IDs are only valid (guaranteed unique system-wide) from the time the thread is created until the thread has been terminated.

<div class="availability">

Windows, FreeBSD, Linux, macOS, OpenBSD, NetBSD, AIX, DragonFlyBSD.

</div>

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

is_alive()

Return whether the thread is alive.

This method returns `True` just before the `~Thread.run` method starts until just after the `~Thread.run` method terminates. The module function `.enumerate` returns a list of all alive threads.

</div>

<div class="attribute">

daemon

A boolean value indicating whether this thread is a daemon thread (`True`) or not (`False`). This must be set before `~Thread.start` is called, otherwise `RuntimeError` is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to `~Thread.daemon` = `False`.

The entire Python program exits when no alive non-daemon threads are left.

</div>

<div class="method">

isDaemon() setDaemon()

Deprecated getter/setter API for `~Thread.daemon`; use it directly as a property instead.

<div class="deprecated">

3.10

</div>

</div>

</div>

### Lock objects

A primitive lock is a synchronization primitive that is not owned by a particular thread when locked. In Python, it is currently the lowest level synchronization primitive available, implemented directly by the `_thread` extension module.

A primitive lock is in one of two states, "locked" or "unlocked". It is created in the unlocked state. It has two basic methods, `~Lock.acquire` and `~Lock.release`. When the state is unlocked, `~Lock.acquire` changes the state to locked and returns immediately. When the state is locked, `~Lock.acquire` blocks until a call to `~Lock.release` in another thread changes it to unlocked, then the `~Lock.acquire` call resets it to locked and returns. The `~Lock.release` method should only be called in the locked state; it changes the state to unlocked and returns immediately. If an attempt is made to release an unlocked lock, a `RuntimeError` will be raised.

Locks also support the `context management protocol <with-locks>`.

When more than one thread is blocked in `~Lock.acquire` waiting for the state to turn to unlocked, only one thread proceeds when a `~Lock.release` call resets the state to unlocked; which one of the waiting threads proceeds is not defined, and may vary across implementations.

All methods are executed atomically.

<div class="Lock()">

The class implementing primitive lock objects. Once a thread has acquired a lock, subsequent attempts to acquire it block, until it is released; any thread may release it.

<div class="versionchanged">

3.13 `Lock` is now a class. In earlier Pythons, `Lock` was a factory function which returned an instance of the underlying private lock type.

</div>

<div class="method">

acquire(blocking=True, timeout=-1)

Acquire a lock, blocking or non-blocking.

When invoked with the *blocking* argument set to `True` (the default), block until the lock is unlocked, then set it to locked and return `True`.

When invoked with the *blocking* argument set to `False`, do not block. If a call with *blocking* set to `True` would block, return `False` immediately; otherwise, set the lock to locked and return `True`.

When invoked with the *timeout* argument set to a positive value, block for at most the number of seconds specified by *timeout* and as long as the lock cannot be acquired. A *timeout* argument of `-1` specifies an unbounded wait. It is forbidden to specify a *timeout* when *blocking* is `False`.

The return value is `True` if the lock is acquired successfully, `False` if not (for example if the *timeout* expired).

<div class="versionchanged">

3.2 The *timeout* parameter is new.

</div>

<div class="versionchanged">

3.2 Lock acquisition can now be interrupted by signals on POSIX if the underlying threading implementation supports it.

</div>

<div class="versionchanged">

3.14 Lock acquisition can now be interrupted by signals on Windows.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float.

</div>

</div>

<div class="method">

release()

Release a lock. This can be called from any thread, not only the thread which has acquired the lock.

When the lock is locked, reset it to unlocked, and return. If any other threads are blocked waiting for the lock to become unlocked, allow exactly one of them to proceed.

When invoked on an unlocked lock, a `RuntimeError` is raised.

There is no return value.

</div>

<div class="method">

locked()

Return `True` if the lock is acquired.

</div>

</div>

### RLock objects

A reentrant lock is a synchronization primitive that may be acquired multiple times by the same thread. Internally, it uses the concepts of "owning thread" and "recursion level" in addition to the locked/unlocked state used by primitive locks. In the locked state, some thread owns the lock; in the unlocked state, no thread owns it.

Threads call a lock's `~RLock.acquire` method to lock it, and its `~Lock.release` method to unlock it.

> [!NOTE]
> Reentrant locks support the `context management protocol <with-locks>`, so it is recommended to use `with` instead of manually calling `~RLock.acquire` and `~RLock.release` to handle acquiring and releasing the lock for a block of code.

RLock's `~RLock.acquire`/`~RLock.release` call pairs may be nested, unlike Lock's `~Lock.acquire`/`~Lock.release`. Only the final `~RLock.release` (the `~Lock.release` of the outermost pair) resets the lock to an unlocked state and allows another thread blocked in `~RLock.acquire` to proceed.

`~RLock.acquire`/`~RLock.release` must be used in pairs: each acquire must have a release in the thread that has acquired the lock. Failing to call release as many times the lock has been acquired can lead to deadlock.

<div class="RLock()">

This class implements reentrant lock objects. A reentrant lock must be released by the thread that acquired it. Once a thread has acquired a reentrant lock, the same thread may acquire it again without blocking; the thread must release it once for each time it has acquired it.

Note that `RLock` is actually a factory function which returns an instance of the most efficient version of the concrete RLock class that is supported by the platform.

<div class="method">

acquire(blocking=True, timeout=-1)

Acquire a lock, blocking or non-blocking.

<div class="seealso">

`Using RLock as a context manager <with-locks>`  
Recommended over manual `!acquire` and `release` calls whenever practical.

</div>

When invoked with the *blocking* argument set to `True` (the default):

> - If no thread owns the lock, acquire the lock and return immediately.
> - If another thread owns the lock, block until we are able to acquire lock, or *timeout*, if set to a positive value.
> - If the same thread owns the lock, acquire the lock again, and return immediately. This is the difference between `Lock` and `!RLock`; `Lock` handles this case the same as the previous, blocking until the lock can be acquired.

When invoked with the *blocking* argument set to `False`:

> - If no thread owns the lock, acquire the lock and return immediately.
> - If another thread owns the lock, return immediately.
> - If the same thread owns the lock, acquire the lock again and return immediately.

In all cases, if the thread was able to acquire the lock, return `True`. If the thread was unable to acquire the lock (i.e. if not blocking or the timeout was reached) return `False`.

If called multiple times, failing to call `~RLock.release` as many times may lead to deadlock. Consider using `!RLock` as a context manager rather than calling acquire/release directly.

<div class="versionchanged">

3.2 The *timeout* parameter is new.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float.

</div>

</div>

<div class="method">

release()

Release a lock, decrementing the recursion level. If after the decrement it is zero, reset the lock to unlocked (not owned by any thread), and if any other threads are blocked waiting for the lock to become unlocked, allow exactly one of them to proceed. If after the decrement the recursion level is still nonzero, the lock remains locked and owned by the calling thread.

Only call this method when the calling thread owns the lock. A `RuntimeError` is raised if this method is called when the lock is not acquired.

There is no return value.

</div>

<div class="method">

locked()

Return a boolean indicating whether this object is locked right now.

<div class="versionadded">

3.14

</div>

</div>

</div>

### Condition objects

A condition variable is always associated with some kind of lock; this can be passed in or one will be created by default. Passing one in is useful when several condition variables must share the same lock. The lock is part of the condition object: you don't have to track it separately.

A condition variable obeys the `context management protocol <with-locks>`: using the `with` statement acquires the associated lock for the duration of the enclosed block. The `~Condition.acquire` and `~Condition.release` methods also call the corresponding methods of the associated lock.

Other methods must be called with the associated lock held. The `~Condition.wait` method releases the lock, and then blocks until another thread awakens it by calling `~Condition.notify` or `~Condition.notify_all`. Once awakened, `~Condition.wait` re-acquires the lock and returns. It is also possible to specify a timeout.

The `~Condition.notify` method wakes up one of the threads waiting for the condition variable, if any are waiting. The `~Condition.notify_all` method wakes up all threads waiting for the condition variable.

Note: the `~Condition.notify` and `~Condition.notify_all` methods don't release the lock; this means that the thread or threads awakened will not return from their `~Condition.wait` call immediately, but only when the thread that called `~Condition.notify` or `~Condition.notify_all` finally relinquishes ownership of the lock.

The typical programming style using condition variables uses the lock to synchronize access to some shared state; threads that are interested in a particular change of state call `~Condition.wait` repeatedly until they see the desired state, while threads that modify the state call `~Condition.notify` or `~Condition.notify_all` when they change the state in such a way that it could possibly be a desired state for one of the waiters. For example, the following code is a generic producer-consumer situation with unlimited buffer capacity:

    # Consume one item
    with cv:
        while not an_item_is_available():
            cv.wait()
        get_an_available_item()

    # Produce one item
    with cv:
        make_an_item_available()
        cv.notify()

The `while` loop checking for the application's condition is necessary because `~Condition.wait` can return after an arbitrary long time, and the condition which prompted the `~Condition.notify` call may no longer hold true. This is inherent to multi-threaded programming. The `~Condition.wait_for` method can be used to automate the condition checking, and eases the computation of timeouts:

    # Consume an item
    with cv:
        cv.wait_for(an_item_is_available)
        get_an_available_item()

To choose between `~Condition.notify` and `~Condition.notify_all`, consider whether one state change can be interesting for only one or several waiting threads. E.g. in a typical producer-consumer situation, adding one item to the buffer only needs to wake up one consumer thread.

<div class="Condition(lock=None)">

This class implements condition variable objects. A condition variable allows one or more threads to wait until they are notified by another thread.

If the *lock* argument is given and not `None`, it must be a `Lock` or `RLock` object, and it is used as the underlying lock. Otherwise, a new `RLock` object is created and used as the underlying lock.

<div class="versionchanged">

3.3 changed from a factory function to a class.

</div>

<div class="method">

acquire(\*args)

Acquire the underlying lock. This method calls the corresponding method on the underlying lock; the return value is whatever that method returns.

</div>

<div class="method">

release()

Release the underlying lock. This method calls the corresponding method on the underlying lock; there is no return value.

</div>

<div class="method">

locked()

Return a boolean indicating whether this object is locked right now.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

wait(timeout=None)

Wait until notified or until a timeout occurs. If the calling thread has not acquired the lock when this method is called, a `RuntimeError` is raised.

This method releases the underlying lock, and then blocks until it is awakened by a `notify` or `notify_all` call for the same condition variable in another thread, or until the optional timeout occurs. Once awakened or timed out, it re-acquires the lock and returns.

When the *timeout* argument is present and not `None`, it should be a real number specifying a timeout for the operation in seconds (or fractions thereof).

When the underlying lock is an `RLock`, it is not released using its `release` method, since this may not actually unlock the lock when it was acquired multiple times recursively. Instead, an internal interface of the `RLock` class is used, which really unlocks it even when it has been recursively acquired several times. Another internal interface is then used to restore the recursion level when the lock is reacquired.

The return value is `True` unless a given *timeout* expired, in which case it is `False`.

<div class="versionchanged">

3.2 Previously, the method always returned `None`.

</div>

</div>

<div class="method">

wait_for(predicate, timeout=None)

Wait until a condition evaluates to true. *predicate* should be a callable which result will be interpreted as a boolean value. A *timeout* may be provided giving the maximum time to wait.

This utility method may call `wait` repeatedly until the predicate is satisfied, or until a timeout occurs. The return value is the last return value of the predicate and will evaluate to `False` if the method timed out.

Ignoring the timeout feature, calling this method is roughly equivalent to writing:

    while not predicate():
        cv.wait()

Therefore, the same rules apply as with `wait`: The lock must be held when called and is re-acquired on return. The predicate is evaluated with the lock held.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

notify(n=1)

By default, wake up one thread waiting on this condition, if any. If the calling thread has not acquired the lock when this method is called, a `RuntimeError` is raised.

This method wakes up at most *n* of the threads waiting for the condition variable; it is a no-op if no threads are waiting.

The current implementation wakes up exactly *n* threads, if at least *n* threads are waiting. However, it's not safe to rely on this behavior. A future, optimized implementation may occasionally wake up more than *n* threads.

Note: an awakened thread does not actually return from its `wait` call until it can reacquire the lock. Since `notify` does not release the lock, its caller should.

</div>

<div class="method">

notify_all()

Wake up all threads waiting on this condition. This method acts like `notify`, but wakes up all waiting threads instead of one. If the calling thread has not acquired the lock when this method is called, a `RuntimeError` is raised.

The method `notifyAll` is a deprecated alias for this method.

</div>

</div>

### Semaphore objects

This is one of the oldest synchronization primitives in the history of computer science, invented by the early Dutch computer scientist Edsger W. Dijkstra (he used the names `P()` and `V()` instead of `~Semaphore.acquire` and `~Semaphore.release`).

A semaphore manages an internal counter which is decremented by each `~Semaphore.acquire` call and incremented by each `~Semaphore.release` call. The counter can never go below zero; when `~Semaphore.acquire` finds that it is zero, it blocks, waiting until some other thread calls `~Semaphore.release`.

Semaphores also support the `context management protocol <with-locks>`.

<div class="Semaphore(value=1)">

This class implements semaphore objects. A semaphore manages an atomic counter representing the number of `release` calls minus the number of `acquire` calls, plus an initial value. The `acquire` method blocks if necessary until it can return without making the counter negative. If not given, *value* defaults to 1.

The optional argument gives the initial *value* for the internal counter; it defaults to `1`. If the *value* given is less than 0, `ValueError` is raised.

<div class="versionchanged">

3.3 changed from a factory function to a class.

</div>

<div class="method">

acquire(blocking=True, timeout=None)

Acquire a semaphore.

When invoked without arguments:

- If the internal counter is larger than zero on entry, decrement it by one and return `True` immediately.
- If the internal counter is zero on entry, block until awoken by a call to `~Semaphore.release`. Once awoken (and the counter is greater than 0), decrement the counter by 1 and return `True`. Exactly one thread will be awoken by each call to `~Semaphore.release`. The order in which threads are awoken should not be relied on.

When invoked with *blocking* set to `False`, do not block. If a call without an argument would block, return `False` immediately; otherwise, do the same thing as when called without arguments, and return `True`.

When invoked with a *timeout* other than `None`, it will block for at most *timeout* seconds. If acquire does not complete successfully in that interval, return `False`. Return `True` otherwise.

<div class="versionchanged">

3.2 The *timeout* parameter is new.

</div>

<div class="versionchanged">

3.15 Accepts any real number as *timeout*, not only integer or float.

</div>

</div>

<div class="method">

release(n=1)

Release a semaphore, incrementing the internal counter by *n*. When it was zero on entry and other threads are waiting for it to become larger than zero again, wake up *n* of those threads.

<div class="versionchanged">

3.9 Added the *n* parameter to release multiple waiting threads at once.

</div>

</div>

</div>

<div class="BoundedSemaphore(value=1)">

Class implementing bounded semaphore objects. A bounded semaphore checks to make sure its current value doesn't exceed its initial value. If it does, `ValueError` is raised. In most situations semaphores are used to guard resources with limited capacity. If the semaphore is released too many times it's a sign of a bug. If not given, *value* defaults to 1.

<div class="versionchanged">

3.3 changed from a factory function to a class.

</div>

</div>

### `Semaphore` example

Semaphores are often used to guard resources with limited capacity, for example, a database server. In any situation where the size of the resource is fixed, you should use a bounded semaphore. Before spawning any worker threads, your main thread would initialize the semaphore:

    maxconnections = 5
    # ...
    pool_sema = BoundedSemaphore(value=maxconnections)

Once spawned, worker threads call the semaphore's acquire and release methods when they need to connect to the server:

    with pool_sema:
        conn = connectdb()
        try:
            # ... use connection ...
        finally:
            conn.close()

The use of a bounded semaphore reduces the chance that a programming error which causes the semaphore to be released more than it's acquired will go undetected.

### Event objects

This is one of the simplest mechanisms for communication between threads: one thread signals an event and other threads wait for it.

An event object manages an internal flag that can be set to true with the `~Event.set` method and reset to false with the `~Event.clear` method. The `~Event.wait` method blocks until the flag is true.

<div class="Event()">

Class implementing event objects. An event manages a flag that can be set to true with the `~Event.set` method and reset to false with the `clear` method. The `wait` method blocks until the flag is true. The flag is initially false.

<div class="versionchanged">

3.3 changed from a factory function to a class.

</div>

<div class="method">

is_set()

Return `True` if and only if the internal flag is true.

The method `isSet` is a deprecated alias for this method.

</div>

<div class="method">

set()

Set the internal flag to true. All threads waiting for it to become true are awakened. Threads that call `wait` once the flag is true will not block at all.

</div>

<div class="method">

clear()

Reset the internal flag to false. Subsequently, threads calling `wait` will block until `.set` is called to set the internal flag to true again.

</div>

<div class="method">

wait(timeout=None)

Block as long as the internal flag is false and the timeout, if given, has not expired. The return value represents the reason that this blocking method returned; `True` if returning because the internal flag is set to true, or `False` if a timeout is given and the internal flag did not become true within the given wait time.

When the timeout argument is present and not `None`, it should be a real number specifying a timeout for the operation in seconds, or fractions thereof.

<div class="versionchanged">

3.1 Previously, the method always returned `None`.

</div>

</div>

</div>

### Timer objects

This class represents an action that should be run only after a certain amount of time has passed --- a timer. `Timer` is a subclass of `Thread` and as such also functions as an example of creating custom threads.

Timers are started, as with threads, by calling their `Timer.start <Thread.start>` method. The timer can be stopped (before its action has begun) by calling the `~Timer.cancel` method. The interval the timer will wait before executing its action may not be exactly the same as the interval specified by the user.

For example:

    def hello():
        print("hello, world")

    t = Timer(30.0, hello)
    t.start()  # after 30 seconds, "hello, world" will be printed

<div class="Timer(interval, function, args=None, kwargs=None)">

Create a timer that will run *function* with arguments *args* and keyword arguments *kwargs*, after *interval* seconds have passed. If *args* is `None` (the default) then an empty list will be used. If *kwargs* is `None` (the default) then an empty dict will be used.

<div class="versionchanged">

3.3 changed from a factory function to a class.

</div>

<div class="method">

cancel()

Stop the timer, and cancel the execution of the timer's action. This will only work if the timer is still in its waiting stage.

</div>

</div>

### Barrier objects

<div class="versionadded">

3.2

</div>

This class provides a simple synchronization primitive for use by a fixed number of threads that need to wait for each other. Each of the threads tries to pass the barrier by calling the `~Barrier.wait` method and will block until all of the threads have made their `~Barrier.wait` calls. At this point, the threads are released simultaneously.

The barrier can be reused any number of times for the same number of threads.

As an example, here is a simple way to synchronize a client and server thread:

    b = Barrier(2, timeout=5)

    def server():
        start_server()
        b.wait()
        while True:
            connection = accept_connection()
            process_server_connection(connection)

    def client():
        b.wait()
        while True:
            connection = make_connection()
            process_client_connection(connection)

<div class="Barrier(parties, action=None, timeout=None)">

Create a barrier object for *parties* number of threads. An *action*, when provided, is a callable to be called by one of the threads when they are released. *timeout* is the default timeout value if none is specified for the `wait` method.

<div class="method">

wait(timeout=None)

Pass the barrier. When all the threads party to the barrier have called this function, they are all released simultaneously. If a *timeout* is provided, it is used in preference to any that was supplied to the class constructor.

The return value is an integer in the range 0 to *parties* -- 1, different for each thread. This can be used to select a thread to do some special housekeeping, e.g.:

    i = barrier.wait()
    if i == 0:
        # Only one thread needs to print this
        print("passed the barrier")

If an *action* was provided to the constructor, one of the threads will have called it prior to being released. Should this call raise an error, the barrier is put into the broken state.

If the call times out, the barrier is put into the broken state.

This method may raise a `BrokenBarrierError` exception if the barrier is broken or reset while a thread is waiting.

</div>

<div class="method">

reset()

Return the barrier to the default, empty state. Any threads waiting on it will receive the `BrokenBarrierError` exception.

Note that using this function may require some external synchronization if there are other threads whose state is unknown. If a barrier is broken it may be better to just leave it and create a new one.

</div>

<div class="method">

abort()

Put the barrier into a broken state. This causes any active or future calls to `wait` to fail with the `BrokenBarrierError`. Use this for example if one of the threads needs to abort, to avoid deadlocking the application.

It may be preferable to simply create the barrier with a sensible *timeout* value to automatically guard against one of the threads going awry.

</div>

<div class="attribute">

parties

The number of threads required to pass the barrier.

</div>

<div class="attribute">

n_waiting

The number of threads currently waiting in the barrier.

</div>

<div class="attribute">

broken

A boolean that is `True` if the barrier is in the broken state.

</div>

</div>

<div class="exception">

BrokenBarrierError

This exception, a subclass of `RuntimeError`, is raised when the `Barrier` object is reset or broken.

</div>

## Using locks, conditions, and semaphores in the `!with` statement

All of the objects provided by this module that have `acquire` and `release` methods can be used as context managers for a `with` statement. The `acquire` method will be called when the block is entered, and `release` will be called when the block is exited. Hence, the following snippet:

    with some_lock:
        # do something...

is equivalent to:

    some_lock.acquire()
    try:
        # do something...
    finally:
        some_lock.release()

Currently, `Lock`, `RLock`, `Condition`, `Semaphore`, and `BoundedSemaphore` objects may be used as `with` statement context managers.

## Iterator synchronization

By default, Python iterators do not support concurrent access. Most iterators make no guarantees when accessed simultaneously from multiple threads. Generator iterators, for example, raise `ValueError` if one of their iterator methods is called while the generator is already executing. The tools in this section allow reliable concurrency support to be added to ordinary iterators and iterator-producing callables.

The `serialize_iterator` wrapper lets multiple threads share a single iterator and take turns consuming from it. While one thread is running `__next__()`, the others block until the iterator becomes available. Each value produced by the underlying iterator is delivered to exactly one caller.

The `concurrent_tee` function lets multiple threads each receive the full stream of values from one underlying iterator. It creates independent iterators that all draw from the same source. Values are buffered until consumed by all of the derived iterators.

<div class="serialize_iterator(iterable)">

Return an iterator wrapper that serializes concurrent calls to `~iterator.__next__` using a lock.

If the wrapped iterator also defines `~generator.send`, `~generator.throw`, or `~generator.close`, those calls are serialized as well.

This makes it possible to share a single iterator, including a generator iterator, between multiple threads. A lock ensures that calls are handled one at a time. No values are duplicated or skipped by the wrapper itself. Each item from the underlying iterator is given to exactly one caller.

This wrapper does not copy or buffer values. Threads that call `next` while another thread is already advancing the iterator will block until the active call completes.

Example:

``` python
import threading

def squares(n):
    for x in range(n):
        yield x * x

def consume(name, iterable):
    for item in iterable:
        print(name, item)

source = threading.serialize_iterator(squares(5))

t1 = threading.Thread(target=consume, args=("left", source))
t2 = threading.Thread(target=consume, args=("right", source))
t1.start()
t2.start()
t1.join()
t2.join()
```

In this example, each number is printed exactly once, but the work is shared between the two threads.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

synchronized_iterator(func)

Wrap an iterator-producing callable so that each iterator it returns is automatically passed through `serialize_iterator`.

This is especially useful as a `decorator` for generator functions, allowing their generator-iterators to be consumed from multiple threads.

Example:

``` python
import threading

@threading.synchronized_iterator
def squares(n):
    for x in range(n):
        yield x * x

def consume(name, iterable):
    for item in iterable:
        print(name, item)

source = squares(5)

t1 = threading.Thread(target=consume, args=("left", source))
t2 = threading.Thread(target=consume, args=("right", source))
t1.start()
t2.start()
t1.join()
t2.join()
```

The returned wrapper preserves the metadata of *func*, such as its name and wrapped function reference.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

concurrent_tee(iterable, n=2)

Return *n* independent iterators from a single input *iterable*, with guaranteed behavior when the derived iterators are consumed concurrently.

This function is similar to `itertools.tee`, but is intended for cases where the source iterator may feed consumers running in different threads. Each returned iterator yields every value from the underlying iterable, in the same order.

Internally, values are buffered until every derived iterator has consumed them.

The returned iterators share the same underlying synchronization lock. Each individual derived iterator is intended to be consumed by one thread at a time. If a single derived iterator must itself be shared by multiple threads, wrap it with `serialize_iterator`.

If *n* is `0`, return an empty tuple. If *n* is negative, raise `ValueError`.

Example:

``` python
import threading

def squares(n):
    for x in range(n):
        yield x * x

def consume(name, iterable):
    for item in iterable:
        print(name, item)

source = squares(5)
left, right = threading.concurrent_tee(source)

t1 = threading.Thread(target=consume, args=("left", left))
t2 = threading.Thread(target=consume, args=("right", right))
t1.start()
t2.start()
t1.join()
t2.join()
```

In this example, both consumer threads see the full sequence of squares from a single generator expression.

<div class="versionadded">

3.15

</div>

</div>

# `!time` --- Time access and conversions

<div class="module" data-synopsis="Time access and conversions.">

time

</div>

------------------------------------------------------------------------

This module provides various time-related functions. For related functionality, see also the `datetime` and `calendar` modules.

Although this module is always available, not all functions are available on all platforms. Most of the functions defined in this module call platform C library functions with the same name. It may sometimes be helpful to consult the platform documentation, because the semantics of these functions varies among platforms.

An explanation of some terminology and conventions is in order.

<div id="epoch">

<div class="index">

single: epoch

</div>

</div>

- The `epoch` is the point where the time starts, the return value of `time.gmtime(0)`. It is January 1, 1970, 00:00:00 (UTC) on all platforms.

<div class="index">

seconds since the epoch

</div>

- The term `seconds since the epoch` refers to the total number of elapsed seconds since the epoch, typically excluding [leap seconds](https://en.wikipedia.org/wiki/Leap_second). Leap seconds are excluded from this total on all POSIX-compliant platforms.

<div class="index">

single: Year 2038

</div>

- The functions in this module may not handle dates and times before the [epoch](#epoch) or far in the future. The cut-off point in the future is determined by the C library; for 32-bit systems, it is typically in 2038.

<div class="index">

single: 2-digit years

</div>

- Function `strptime` can parse 2-digit years when given `%y` format code. When 2-digit years are parsed, they are converted according to the POSIX and ISO C standards: values 69--99 are mapped to 1969--1999, and values 0--68 are mapped to 2000--2068.

<div class="index">

single: UTC single: Coordinated Universal Time single: Greenwich Mean Time

</div>

- UTC is [Coordinated Universal Time](https://en.wikipedia.org/wiki/Coordinated_Universal_Time) and superseded [Greenwich Mean Time](https://en.wikipedia.org/wiki/Greenwich_Mean_Time) or GMT as the basis of international timekeeping. The acronym UTC is not a mistake but conforms to an earlier, language-agnostic naming scheme for time standards such as UT0, UT1, and UT2.

<div class="index">

single: Daylight Saving Time

</div>

- DST is Daylight Saving Time, an adjustment of the timezone by (usually) one hour during part of the year. DST rules are magic (determined by local law) and can change from year to year. The C library has a table containing the local rules (often it is read from a system file for flexibility) and is the only source of True Wisdom in this respect.

- The precision of the various real-time functions may be less than suggested by the units in which their value or argument is expressed. E.g. on most Unix systems, the clock "ticks" only 50 or 100 times a second.

- On the other hand, the precision of `.time` and `sleep` is better than their Unix equivalents: times are expressed as floating-point numbers, `.time` returns the most accurate time available (using Unix `!gettimeofday` where available), and `sleep` will accept a time with a nonzero fraction (Unix `!select` is used to implement this, where available).

- The time value as returned by `gmtime`, `localtime`, and `strptime`, and accepted by `asctime`, `mktime` and `strftime`, is a sequence of 9 integers. The return values of `gmtime`, `localtime`, and `strptime` also offer attribute names for individual fields.

  See `struct_time` for a description of these objects.

  <div class="versionchanged">

  3.3 The `struct_time` type was extended to provide the `~struct_time.tm_gmtoff` and `~struct_time.tm_zone` attributes when platform supports corresponding `struct tm` members.

  </div>

  <div class="versionchanged">

  3.6 The `struct_time` attributes `~struct_time.tm_gmtoff` and `~struct_time.tm_zone` are now available on all platforms.

  </div>

- Use the following functions to convert between time representations:

  | From                        | To                          | Use               |
  |-----------------------------|-----------------------------|-------------------|
  | seconds since the epoch     | `struct_time` in UTC        | `gmtime`          |
  | seconds since the epoch     | `struct_time` in local time | `localtime`       |
  | `struct_time` in UTC        | seconds since the epoch     | `calendar.timegm` |
  | `struct_time` in local time | seconds since the epoch     | `mktime`          |

## Functions

<div class="function">

asctime(\[time_tuple\])

Convert a tuple or `struct_time` representing a time as returned by `gmtime` or `localtime` to a string of the following form: `'Sun Jun 20 23:21:05 1993'`. The day field is two characters long and is space padded if the day is a single digit, for example: `'Wed Jun  9 04:26:40 1993'`.

If *time_tuple* is not provided, the current time as returned by `localtime` is used. Locale information is not used by `asctime`.

> [!NOTE]
> Unlike the C function of the same name, `asctime` does not add a trailing newline.

</div>

<div class="function">

pthread_getcpuclockid(thread_id, /)

Return the *clk_id* of the thread-specific CPU-time clock for the specified *thread_id*.

Use `threading.get_ident` or the `~threading.Thread.ident` attribute of `threading.Thread` objects to get a suitable value for *thread_id*.

> [!WARNING]
> Passing an invalid or expired *thread_id* may result in undefined behavior, such as segmentation fault.

<div class="availability">

Unix

See the man page for `pthread_getcpuclockid(3)` for further information.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

clock_getres(clk_id, /)

Return the resolution (precision) of the specified clock *clk_id*. Refer to `time-clock-id-constants` for a list of accepted values for *clk_id*.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

clock_gettime(clk_id, /) -\> float

Return the time of the specified clock *clk_id*. Refer to `time-clock-id-constants` for a list of accepted values for *clk_id*.

Use `clock_gettime_ns` to avoid the precision loss caused by the `float` type.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

clock_gettime_ns(clk_id, /) -\> int

Similar to `clock_gettime` but return time as nanoseconds.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

clock_settime(clk_id, time, /)

Set the time of the specified clock *clk_id*. Currently, `CLOCK_REALTIME` is the only accepted value for *clk_id*.

Use `clock_settime_ns` to avoid the precision loss caused by the `float` type.

<div class="availability">

Unix, not Android, not iOS.

</div>

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.15 Accepts any real number as *time*, not only integer or float.

</div>

</div>

<div class="function">

clock_settime_ns(clk_id, time: int, /)

Similar to `clock_settime` but set time with nanoseconds.

<div class="availability">

Unix, not Android, not iOS.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

ctime(seconds=None, /)

Convert a time expressed in seconds since the [epoch](#epoch) to a string of a form: `'Sun Jun 20 23:21:05 1993'` representing local time. The day field is two characters long and is space padded if the day is a single digit, for example: `'Wed Jun  9 04:26:40 1993'`.

If *seconds* is not provided or `None`, the current time as returned by `.time` is used. `ctime(seconds)` is equivalent to `asctime(localtime(seconds))`. Locale information is not used by `ctime`.

<div class="versionchanged">

3.15 Accepts any real number, not only integer or float.

</div>

</div>

<div class="function">

get_clock_info(name, /)

Get information on the specified clock as a namespace object. Supported clock names and the corresponding functions to read their value are:

- `'monotonic'`: `time.monotonic`
- `'perf_counter'`: `time.perf_counter`
- `'process_time'`: `time.process_time`
- `'thread_time'`: `time.thread_time`
- `'time'`: `time.time`

The result has the following attributes:

- *adjustable*: `True` if the clock can be set to jump forward or backward in time, `False` otherwise. Does not refer to gradual NTP rate adjustments.
- *implementation*: The name of the underlying C function used to get the clock value. Refer to `time-clock-id-constants` for possible values.
- *monotonic*: `True` if the clock cannot go backward, `False` otherwise
- *resolution*: The resolution of the clock in seconds (`float`)

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

gmtime(seconds=None, /)

Convert a time expressed in seconds since the [epoch](#epoch) to a `struct_time` in UTC in which the dst flag is always zero. If *seconds* is not provided or `None`, the current time as returned by `.time` is used. Fractions of a second are ignored. See above for a description of the `struct_time` object. See `calendar.timegm` for the inverse of this function.

<div class="versionchanged">

3.15 Accepts any real number, not only integer or float.

</div>

</div>

<div class="function">

localtime(seconds=None, /)

Like `gmtime` but converts to local time. If *seconds* is not provided or `None`, the current time as returned by `.time` is used. The dst flag is set to `1` when DST applies to the given time.

`localtime` may raise `OverflowError`, if the timestamp is outside the range of values supported by the platform C `localtime` or `gmtime` functions, and `OSError` on `localtime` or `gmtime` failure. It's common for this to be restricted to years between 1970 and 2038.

<div class="versionchanged">

3.15 Accepts any real number, not only integer or float.

</div>

</div>

<div class="function">

mktime(time_tuple, /)

This is the inverse function of `localtime`. Its argument is the `struct_time` or full 9-tuple (since the dst flag is needed; use `-1` as the dst flag if it is unknown) which expresses the time in *local* time, not UTC. It returns a floating-point number, for compatibility with `.time`. If the input value cannot be represented as a valid time, either `OverflowError` or `ValueError` will be raised (which depends on whether the invalid value is caught by Python or the underlying C libraries). The earliest date for which it can generate a time is platform-dependent.

</div>

<div class="function">

monotonic() -\> float

Return the value (in fractional seconds) of a monotonic clock, i.e. a clock that cannot go backwards. The clock is not affected by system clock updates. The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid.

Clock:

- On Windows, call `QueryPerformanceCounter()` and `QueryPerformanceFrequency()`.
- On macOS, call `mach_absolute_time()` and `mach_timebase_info()`.
- On HP-UX, call `gethrtime()`.
- Call `clock_gettime(CLOCK_HIGHRES)` if available.
- Otherwise, call `clock_gettime(CLOCK_MONOTONIC)`.

Use `monotonic_ns` to avoid the precision loss caused by the `float` type.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.5 The function is now always available and the clock is now the same for all processes.

</div>

<div class="versionchanged">

3.10 On macOS, the clock is now the same for all processes.

</div>

</div>

<div class="function">

monotonic_ns() -\> int

Similar to `monotonic`, but return time as nanoseconds.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

perf_counter() -\> float

<div class="index">

single: benchmarking

</div>

Return the value (in fractional seconds) of a performance counter, i.e. a clock with the highest available resolution to measure a short duration. It does include time elapsed during sleep. The clock is the same for all processes. The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid.

<div class="impl-detail">

On CPython, use the same clock as `time.monotonic` and is a monotonic clock, i.e. a clock that cannot go backwards.

</div>

Use `perf_counter_ns` to avoid the precision loss caused by the `float` type.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.10 On Windows, the clock is now the same for all processes.

</div>

<div class="versionchanged">

3.13 Use the same clock as `time.monotonic`.

</div>

</div>

<div class="function">

perf_counter_ns() -\> int

Similar to `perf_counter`, but return time as nanoseconds.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

process_time() -\> float

<div class="index">

single: CPU time single: processor time single: benchmarking

</div>

Return the value (in fractional seconds) of the sum of the system and user CPU time of the current process. It does not include time elapsed during sleep. It is process-wide by definition. The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid.

Use `process_time_ns` to avoid the precision loss caused by the `float` type.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

process_time_ns() -\> int

Similar to `process_time` but return time as nanoseconds.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

sleep(seconds, /)

Suspend execution of the calling thread for the given number of seconds. The argument may be a non-integer to indicate a more precise sleep time.

If the sleep is interrupted by a signal and no exception is raised by the signal handler, the sleep is restarted with a recomputed timeout.

The suspension time may be longer than requested by an arbitrary amount, because of the scheduling of other activity in the system.

**Windows implementation**

On Windows, if *seconds* is zero, the thread relinquishes the remainder of its time slice to any other thread that is ready to run. If there are no other threads ready to run, the function returns immediately, and the thread continues execution. On Windows 10 and newer the implementation uses a [high-resolution timer](https://learn.microsoft.com/windows/win32/api/synchapi/nf-synchapi-createwaitabletimerexw) which provides resolution of 100 nanoseconds. If *seconds* is zero, `Sleep(0)` is used.

**Unix implementation**

- Use `clock_nanosleep()` if available (resolution: 1 nanosecond);
- Or use `nanosleep()` if available (resolution: 1 nanosecond);
- Or use `select()` (resolution: 1 microsecond).

> [!NOTE]
> To emulate a "no-op", use `pass` instead of `time.sleep(0)`.
>
> To voluntarily relinquish the CPU, specify a real-time `scheduling
> policy <os-scheduling-policy>` and use `os.sched_yield` instead.

<div class="audit-event">

time.sleep seconds

</div>

<div class="versionchanged">

3.5 The function now sleeps at least *seconds* even if the sleep is interrupted by a signal, except if the signal handler raises an exception (see `475` for the rationale).

</div>

<div class="versionchanged">

3.11 On Unix, the `clock_nanosleep()` and `nanosleep()` functions are now used if available. On Windows, a waitable timer is now used.

</div>

<div class="versionchanged">

3.13 Raises an auditing event.

</div>

<div class="versionchanged">

3.15 Accepts any real number, not only integer or float.

</div>

</div>

<div class="index">

single: % (percent); datetime format

</div>

<div class="function">

strftime(format\[, time_tuple\])

Convert a tuple or `struct_time` representing a time as returned by `gmtime` or `localtime` to a string as specified by the *format* argument. If *time_tuple* is not provided, the current time as returned by `localtime` is used. *format* must be a string. `ValueError` is raised if any field in *time_tuple* is outside of the allowed range.

0 is a legal argument for any position in the time tuple; if it is normally illegal the value is forced to a correct one.

The following directives can be embedded in the *format* string. They are shown without the optional field width and precision specification, and are replaced by the indicated characters in the `strftime` result:

| Directive | Meaning | Notes |
|----|----|----|
| `%a` | Locale's abbreviated weekday name. |  |
| `%A` | Locale's full weekday name. |  |
| `%b` | Locale's abbreviated month name. |  |
| `%B` | Locale's full month name. |  |
| `%c` | Locale's appropriate date and time representation. |  |
| `%d` | Day of the month as a decimal number \[01,31\]. |  |
| `%f` | Microseconds as a decimal number \[000000,999999\]. | \(1\) |
| `%H` | Hour (24-hour clock) as a decimal number \[00,23\]. |  |
| `%I` | Hour (12-hour clock) as a decimal number \[01,12\]. |  |
| `%j` | Day of the year as a decimal number \[001,366\]. |  |
| `%m` | Month as a decimal number \[01,12\]. |  |
| `%M` | Minute as a decimal number \[00,59\]. |  |
| `%p` | Locale's equivalent of either AM or PM. | \(2\) |
| `%S` | Second as a decimal number \[00,61\]. | \(3\) |
| `%U` | Week number of the year (Sunday as the first day of the week) as a decimal number \[00,53\]. All days in a new year preceding the first Sunday are considered to be in week 0. | \(4\) |
| `%u` | Day of the week (Monday is 1; Sunday is 7) as a decimal number \[1, 7\]. |  |
| `%w` | Weekday as a decimal number \[0(Sunday),6\]. |  |
| `%W` | Week number of the year (Monday as the first day of the week) as a decimal number \[00,53\]. All days in a new year preceding the first Monday are considered to be in week 0. | \(4\) |
| `%x` | Locale's appropriate date representation. |  |
| `%X` | Locale's appropriate time representation. |  |
| `%y` | Year without century as a decimal number \[00,99\]. |  |
| `%Y` | Year with century as a decimal number. |  |
| `%z` | Time zone offset indicating a positive or negative time difference from UTC/GMT of the form +HHMM or -HHMM, where H represents decimal hour digits and M represents decimal minute digits \[-23:59, +23:59\].[^1] |  |
| `%Z` | Time zone name (no characters if no time zone exists). Deprecated.[^2] |  |
| `%G` | ISO 8601 year (similar to `%Y` but follows the rules for the ISO 8601 calendar year). The year starts with the week that contains the first Thursday of the calendar year. |  |
| `%V` | ISO 8601 week number (as a decimal number \[01,53\]). The first week of the year is the one that contains the first Thursday of the year. Weeks start on Monday. |  |
| `%%` | A literal `'%'` character. |  |

Notes:

\(1\)  
The `%f` format directive only applies to `strptime`, not to `strftime`. However, see also `datetime.datetime.strptime` and `datetime.datetime.strftime` where the `%f` format directive `applies to microseconds <format-codes>`.

\(2\)  
When used with the `strptime` function, the `%p` directive only affects the output hour field if the `%I` directive is used to parse the hour.

<div id="leap-second">

\(3\)  
The range really is `0` to `61`; value `60` is valid in timestamps representing [leap seconds](https://en.wikipedia.org/wiki/Leap_second) and value `61` is supported for historical reasons.

\(4\)  
When used with the `strptime` function, `%U` and `%W` are only used in calculations when the day of the week and the year are specified.

</div>

Here is an example, a format for dates compatible with that specified in the `5322` Internet email standard.[^3] :

    >>> from time import gmtime, strftime
    >>> strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime())
    'Thu, 28 Jun 2001 14:17:15 +0000'

Additional directives may be supported on certain platforms, but only the ones listed here have a meaning standardized by ANSI C. To see the full set of format codes supported on your platform, consult the `strftime(3)` documentation.

On some platforms, an optional field width and precision specification can immediately follow the initial `'%'` of a directive in the following order; this is also not portable. The field width is normally 2 except for `%j` where it is 3.

</div>

<div class="index">

single: % (percent); datetime format

</div>

<div class="function">

strptime(string\[, format\])

Parse a string representing a time according to a format. The return value is a `struct_time` as returned by `gmtime` or `localtime`.

The *format* parameter uses the same directives as those used by `strftime`; it defaults to `"%a %b %d %H:%M:%S %Y"` which matches the formatting returned by `ctime`. If *string* cannot be parsed according to *format*, or if it has excess data after parsing, `ValueError` is raised. The default values used to fill in any missing data when more accurate values cannot be inferred are `(1900, 1, 1, 0, 0, 0, 0, 1, -1)`. Both *string* and *format* must be strings.

For example:

> \>\>\> import time \>\>\> time.strptime("30 Nov 00", "%d %b %y") \# doctest: +NORMALIZE_WHITESPACE time.struct_time(tm_year=2000, tm_mon=11, tm_mday=30, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=335, tm_isdst=-1)

Support for the `%Z` directive is based on the values contained in `tzname` and whether `daylight` is true. Because of this, it is platform-specific except for recognizing UTC and GMT which are always known (and are considered to be non-daylight savings timezones).

Only the directives specified in the documentation are supported. Because `strftime()` is implemented per platform it can sometimes offer more directives than those listed. But `strptime()` is independent of any platform and thus does not necessarily support all directives available that are not documented as supported.

</div>

<div class="struct_time">

The type of the time value sequence returned by `gmtime`, `localtime`, and `strptime`. It is an object with a `named
tuple` interface: values can be accessed by index and by attribute name. The following values are present:

<table>
<tbody>
<tr>
<td>Index</td>
<td>Attribute</td>
<td>Values</td>
</tr>
<tr>
<td>0</td>
<td><div class="attribute">
<p>tm_year</p>
</div></td>
<td>(for example, 1993)</td>
</tr>
<tr>
<td>1</td>
<td><div class="attribute">
<p>tm_mon</p>
</div></td>
<td>range [1, 12]</td>
</tr>
<tr>
<td>2</td>
<td><div class="attribute">
<p>tm_mday</p>
</div></td>
<td>range [1, 31]</td>
</tr>
<tr>
<td>3</td>
<td><div class="attribute">
<p>tm_hour</p>
</div></td>
<td>range [0, 23]</td>
</tr>
<tr>
<td>4</td>
<td><div class="attribute">
<p>tm_min</p>
</div></td>
<td>range [0, 59]</td>
</tr>
<tr>
<td>5</td>
<td><div class="attribute">
<p>tm_sec</p>
</div></td>
<td>range [0, 61]; see <code class="interpreted-text" role="ref">Note (2) &lt;leap-second&gt;</code> in <code class="interpreted-text" role="func">strftime</code></td>
</tr>
<tr>
<td>6</td>
<td><div class="attribute">
<p>tm_wday</p>
</div></td>
<td>range [0, 6]; Monday is 0</td>
</tr>
<tr>
<td>7</td>
<td><div class="attribute">
<p>tm_yday</p>
</div></td>
<td>range [1, 366]</td>
</tr>
<tr>
<td>8</td>
<td><div class="attribute">
<p>tm_isdst</p>
</div></td>
<td>0, 1 or -1; see below</td>
</tr>
<tr>
<td>N/A</td>
<td><div class="attribute">
<p>tm_zone</p>
</div></td>
<td>abbreviation of timezone name</td>
</tr>
<tr>
<td>N/A</td>
<td><div class="attribute">
<p>tm_gmtoff</p>
</div></td>
<td>offset east of UTC in seconds</td>
</tr>
</tbody>
</table>

Note that unlike the C structure, the month value is a range of \[1, 12\], not \[0, 11\].

In calls to `mktime`, `tm_isdst` may be set to 1 when daylight savings time is in effect, and 0 when it is not. A value of -1 indicates that this is not known, and will usually result in the correct state being filled in.

When a tuple with an incorrect length is passed to a function expecting a `struct_time`, or having elements of the wrong type, a `TypeError` is raised.

</div>

<div class="function">

time() -\> float

Return the time in seconds since the [epoch](#epoch) as a floating-point number. The handling of [leap seconds](https://en.wikipedia.org/wiki/Leap_second) is platform dependent. On Windows and most Unix systems, the leap seconds are not counted towards the time in seconds since the [epoch](#epoch). This is commonly referred to as [Unix time](https://en.wikipedia.org/wiki/Unix_time).

Note that even though the time is always returned as a floating-point number, not all systems provide time with a better precision than 1 second. While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between the two calls.

The number returned by `.time` may be converted into a more common time format (i.e. year, month, day, hour, etc...) in UTC by passing it to `gmtime` function or in local time by passing it to the `localtime` function. In both cases a `struct_time` object is returned, from which the components of the calendar date may be accessed as attributes.

Clock:

- On Windows, call `GetSystemTimePreciseAsFileTime()`.
- Call `clock_gettime(CLOCK_REALTIME)` if available.
- Otherwise, call `gettimeofday()`.

Use `time_ns` to avoid the precision loss caused by the `float` type.

</div>

<div class="versionchanged">

3.13

On Windows, calls `GetSystemTimePreciseAsFileTime()` instead of `GetSystemTimeAsFileTime()`.

</div>

<div class="function">

time_ns() -\> int

Similar to `~time.time` but returns time as an integer number of nanoseconds since the [epoch](#epoch).

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

thread_time() -\> float

<div class="index">

single: CPU time single: processor time single: benchmarking

</div>

Return the value (in fractional seconds) of the sum of the system and user CPU time of the current thread. It does not include time elapsed during sleep. It is thread-specific by definition. The reference point of the returned value is undefined, so that only the difference between the results of two calls in the same thread is valid.

Use `thread_time_ns` to avoid the precision loss caused by the `float` type.

<div class="availability">

Linux, Unix, Windows.

Unix systems supporting `CLOCK_THREAD_CPUTIME_ID`.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

thread_time_ns() -\> int

Similar to `thread_time` but return time as nanoseconds.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

tzset()

Reset the time conversion rules used by the library routines. The environment variable `TZ` specifies how this is done. It will also set the variables `tzname` (from the `TZ` environment variable), `timezone` (non-DST seconds West of UTC), `altzone` (DST seconds west of UTC) and `daylight` (to 0 if this timezone does not have any daylight saving time rules, or to nonzero if there is a time, past, present or future when daylight saving time applies).

<div class="availability">

Unix.

</div>

> [!NOTE]
> Although in many cases, changing the `TZ` environment variable may affect the output of functions like `localtime` without calling `tzset`, this behavior should not be relied on.
>
> The `TZ` environment variable should contain no whitespace.

The standard format of the `TZ` environment variable is (whitespace added for clarity):

    std offset [dst [offset [,start[/time], end[/time]]]]

Where the components are:

`std` and `dst`  
Three or more alphanumerics giving the timezone abbreviations. These will be propagated into time.tzname

`offset`  
The offset has the form: `± hh[:mm[:ss]]`. This indicates the value added the local time to arrive at UTC. If preceded by a '-', the timezone is east of the Prime Meridian; otherwise, it is west. If no offset follows dst, summer time is assumed to be one hour ahead of standard time.

`start[/time], end[/time]`  
Indicates when to change to and back from DST. The format of the start and end dates are one of the following:

`J{n}`  
The Julian day *n* (1 \<= *n* \<= 365). Leap days are not counted, so in all years February 28 is day 59 and March 1 is day 60.

`{n}`  
The zero-based Julian day (0 \<= *n* \<= 365). Leap days are counted, and it is possible to refer to February 29.

`M{m}.{n}.{d}`  
The *d*'th day (0 \<= *d* \<= 6) of week *n* of month *m* of the year (1 \<= *n* \<= 5, 1 \<= *m* \<= 12, where week 5 means "the last *d* day in month *m*" which may occur in either the fourth or the fifth week). Week 1 is the first week in which the *d*'th day occurs. Day zero is a Sunday.

`time` has the same format as `offset` except that no leading sign ('-' or '+') is allowed. The default, if time is not given, is 02:00:00.

<!-- -->

    >>> os.environ['TZ'] = 'EST+05EDT,M4.1.0,M10.5.0'
    >>> time.tzset()
    >>> time.strftime('%X %x %Z')
    '02:07:36 05/08/03 EDT'
    >>> os.environ['TZ'] = 'AEST-10AEDT-11,M10.5.0,M3.5.0'
    >>> time.tzset()
    >>> time.strftime('%X %x %Z')
    '16:08:12 05/08/03 AEST'

On many Unix systems (including \*BSD, Linux, Solaris, and Darwin), it is more convenient to use the system's zoneinfo (`tzfile(5)`) database to specify the timezone rules. To do this, set the `TZ` environment variable to the path of the required timezone datafile, relative to the root of the systems 'zoneinfo' timezone database, usually located at `/usr/share/zoneinfo`. For example, `'US/Eastern'`, `'Australia/Melbourne'`, `'Egypt'` or `'Europe/Amsterdam'`. :

    >>> os.environ['TZ'] = 'US/Eastern'
    >>> time.tzset()
    >>> time.tzname
    ('EST', 'EDT')
    >>> os.environ['TZ'] = 'Egypt'
    >>> time.tzset()
    >>> time.tzname
    ('EET', 'EEST')

</div>

## Clock ID Constants

These constants are used as parameters for `clock_getres` and `clock_gettime`.

<div class="data">

CLOCK_BOOTTIME

Identical to `CLOCK_MONOTONIC`, except it also includes any time that the system is suspended.

This allows applications to get a suspend-aware monotonic clock without having to deal with the complications of `CLOCK_REALTIME`, which may have discontinuities if the time is changed using `settimeofday()` or similar.

<div class="availability">

Linux \>= 2.6.39.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

CLOCK_HIGHRES

The Solaris OS has a `CLOCK_HIGHRES` timer that attempts to use an optimal hardware source, and may give close to nanosecond resolution. `CLOCK_HIGHRES` is the nonadjustable, high-resolution clock.

<div class="availability">

Solaris.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

CLOCK_MONOTONIC

Clock that cannot be set and represents monotonic time since some unspecified starting point.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

CLOCK_MONOTONIC_RAW

Similar to `CLOCK_MONOTONIC`, but provides access to a raw hardware-based time that is not subject to NTP adjustments.

<div class="availability">

Linux \>= 2.6.28, macOS \>= 10.12.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

CLOCK_MONOTONIC_RAW_APPROX

Similar to `CLOCK_MONOTONIC_RAW`, but reads a value cached by the system at context switch and hence has less accuracy.

<div class="availability">

macOS \>= 10.12.

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

CLOCK_PROCESS_CPUTIME_ID

High-resolution per-process timer from the CPU.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

CLOCK_PROF

High-resolution per-process timer from the CPU.

<div class="availability">

FreeBSD, NetBSD \>= 7, OpenBSD.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

CLOCK_TAI

[International Atomic Time](https://www.nist.gov/pml/time-and-frequency-division/how-utcnist-related-coordinated-universal-time-utc-international)

The system must have a current leap second table in order for this to give the correct answer. PTP or NTP software can maintain a leap second table.

<div class="availability">

Linux.

</div>

<div class="versionadded">

3.9

</div>

</div>

<div class="data">

CLOCK_THREAD_CPUTIME_ID

Thread-specific CPU-time clock.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

CLOCK_UPTIME

Time whose absolute value is the time the system has been running and not suspended, providing accurate uptime measurement, both absolute and interval.

<div class="availability">

FreeBSD, OpenBSD \>= 5.5.

</div>

<div class="versionadded">

3.7

</div>

</div>

<div class="data">

CLOCK_UPTIME_RAW

Clock that increments monotonically, tracking the time since an arbitrary point, unaffected by frequency or time adjustments and not incremented while the system is asleep.

<div class="availability">

macOS \>= 10.12.

</div>

<div class="versionadded">

3.8

</div>

</div>

<div class="data">

CLOCK_UPTIME_RAW_APPROX

Like `CLOCK_UPTIME_RAW`, but the value is cached by the system at context switches and therefore has less accuracy.

<div class="availability">

macOS \>= 10.12.

</div>

<div class="versionadded">

3.13

</div>

</div>

The following constant is the only parameter that can be sent to `clock_settime`.

<div class="data">

CLOCK_REALTIME

Real-time clock. Setting this clock requires appropriate privileges. The clock is the same for all processes.

<div class="availability">

Unix.

</div>

<div class="versionadded">

3.3

</div>

</div>

## Timezone Constants

<div class="data">

altzone

The offset of the local DST timezone, in seconds west of UTC, if one is defined. This is negative if the local DST timezone is east of UTC (as in Western Europe, including the UK). Only use this if `daylight` is nonzero. See note below.

</div>

<div class="data">

daylight

Nonzero if a DST timezone is defined. See note below.

</div>

<div class="data">

timezone

The offset of the local (non-DST) timezone, in seconds west of UTC (negative in most of Western Europe, positive in the US, zero in the UK). See note below.

</div>

<div class="data">

tzname

A tuple of two strings: the first is the name of the local non-DST timezone, the second is the name of the local DST timezone. If no DST timezone is defined, the second string should not be used. See note below.

</div>

> [!NOTE]
> For the above Timezone constants (`altzone`, `daylight`, `timezone`, and `tzname`), the value is determined by the timezone rules in effect at module load time or the last time `tzset` is called and may be incorrect for times in the past. It is recommended to use the `~struct_time.tm_gmtoff` and `~struct_time.tm_zone` results from `localtime` to obtain timezone information.

<div class="seealso">

Module `datetime`  
More object-oriented interface to dates and times.

Module `locale`  
Internationalization services. The locale setting affects the interpretation of many format specifiers in `strftime` and `strptime`.

Module `calendar`  
General calendar-related functions. `~calendar.timegm` is the inverse of `gmtime` from this module.

</div>

**Footnotes**

[^1]: The use of `%Z` is now deprecated, but the `%z` escape that expands to the preferred hour/minute offset is not supported by all ANSI C libraries. Also, a strict reading of the original 1982 `822` standard calls for a two-digit year (`%y` rather than `%Y`), but practice moved to 4-digit years long before the year 2000. After that, `822` became obsolete and the 4-digit year has been first recommended by `1123` and then mandated by `2822`, with `5322` continuing this requirement.

[^2]: The use of `%Z` is now deprecated, but the `%z` escape that expands to the preferred hour/minute offset is not supported by all ANSI C libraries. Also, a strict reading of the original 1982 `822` standard calls for a two-digit year (`%y` rather than `%Y`), but practice moved to 4-digit years long before the year 2000. After that, `822` became obsolete and the 4-digit year has been first recommended by `1123` and then mandated by `2822`, with `5322` continuing this requirement.

[^3]: The use of `%Z` is now deprecated, but the `%z` escape that expands to the preferred hour/minute offset is not supported by all ANSI C libraries. Also, a strict reading of the original 1982 `822` standard calls for a two-digit year (`%y` rather than `%Y`), but practice moved to 4-digit years long before the year 2000. After that, `822` became obsolete and the 4-digit year has been first recommended by `1123` and then mandated by `2822`, with `5322` continuing this requirement.

# `!timeit` --- Measure execution time of small code snippets

<div class="module" data-synopsis="Measure the execution time of small code snippets.">

timeit

</div>

**Source code:** `Lib/timeit.py`

<div class="index">

single: Benchmarking single: Performance

</div>

------------------------------------------------------------------------

This module provides a simple way to time small bits of Python code. It has both a `timeit-command-line-interface` as well as a `callable <python-interface>` one. It avoids a number of common traps for measuring execution times. See also Tim Peters' introduction to the "Algorithms" chapter in the second edition of *Python Cookbook*, published by O'Reilly.

## Basic examples

The following example shows how the `timeit-command-line-interface` can be used to compare three different expressions:

```
$ python -m timeit "'-'.join(str(n) for n in range(100))"
10000 loops, best of 5: 30.2 usec per loop
$ python -m timeit "'-'.join([str(n) for n in range(100)])"
10000 loops, best of 5: 27.5 usec per loop
$ python -m timeit "'-'.join(map(str, range(100)))"
10000 loops, best of 5: 23.2 usec per loop
```

This can be achieved from the `python-interface` with:

    >>> import timeit
    >>> timeit.timeit('"-".join(str(n) for n in range(100))', number=10000)
    0.3018611848820001
    >>> timeit.timeit('"-".join([str(n) for n in range(100)])', number=10000)
    0.2727368790656328
    >>> timeit.timeit('"-".join(map(str, range(100)))', number=10000)
    0.23702679807320237

A callable can also be passed from the `python-interface`:

    >>> timeit.timeit(lambda: "-".join(map(str, range(100))), number=10000)
    0.19665591977536678

Note however that `.timeit` will automatically determine the number of repetitions only when the command-line interface is used. In the `timeit-examples` section you can find more advanced examples.

## Python interface

The module defines three convenience functions and a public class:

<div class="function">

timeit(stmt='pass', setup='pass', timer=\<default timer\>, number=1000000, globals=None)

Create a `Timer` instance with the given statement, *setup* code and *timer* function and run its `.timeit` method with *number* executions. The optional *globals* argument specifies a namespace in which to execute the code.

<div class="versionchanged">

3.5 The optional *globals* parameter was added.

</div>

</div>

<div class="function">

repeat(stmt='pass', setup='pass', timer=\<default timer\>, repeat=5, number=1000000, globals=None)

Create a `Timer` instance with the given statement, *setup* code and *timer* function and run its `.repeat` method with the given *repeat* count and *number* executions. The optional *globals* argument specifies a namespace in which to execute the code.

<div class="versionchanged">

3.5 The optional *globals* parameter was added.

</div>

<div class="versionchanged">

3.7 Default value of *repeat* changed from 3 to 5.

</div>

</div>

<div class="function">

default_timer()

The default timer, which is always time.perf_counter(), returns float seconds. An alternative, time.perf_counter_ns, returns integer nanoseconds.

<div class="versionchanged">

3.3 `time.perf_counter` is now the default timer.

</div>

</div>

<div class="Timer(stmt='pass', setup='pass', timer=&lt;timer function&gt;, globals=None)">

Class for timing execution speed of small code snippets.

The constructor takes a statement to be timed, an additional statement used for setup, and a timer function. Both statements default to `'pass'`; the timer function is platform-dependent (see the module doc string). *stmt* and *setup* may also contain multiple statements separated by `;` or newlines, as long as they don't contain multi-line string literals. The statement will by default be executed within timeit's namespace; this behavior can be controlled by passing a namespace to *globals*.

To measure the execution time of the first statement, use the `.timeit` method. The `.repeat` and `.autorange` methods are convenience methods to call `.timeit` multiple times.

The execution time of *setup* is excluded from the overall timed execution run.

The *stmt* and *setup* parameters can also take objects that are callable without arguments. This will embed calls to them in a timer function that will then be executed by `.timeit`. Note that the timing overhead is a little larger in this case because of the extra function calls.

<div class="versionchanged">

3.5 The optional *globals* parameter was added.

</div>

<div class="method">

Timer.timeit(number=1000000)

Time *number* executions of the main statement. This executes the setup statement once, and then returns the time it takes to execute the main statement a number of times. The default timer returns seconds as a float. The argument is the number of times through the loop, defaulting to one million. The main statement, the setup statement and the timer function to be used are passed to the constructor.

> [!NOTE]
> By default, `.timeit` temporarily turns off `garbage
> collection` during the timing. The advantage of this approach is that it makes independent timings more comparable. The disadvantage is that GC may be an important component of the performance of the function being measured. If so, GC can be re-enabled as the first statement in the *setup* string. For example:
>
>     timeit.Timer('for i in range(10): oct(i)', 'gc.enable()').timeit()

</div>

<div class="method">

Timer.autorange(callback=None, target_time=None)

Automatically determine how many times to call `.timeit`.

This is a convenience function that calls `.timeit` repeatedly so that the total time \>= *Timer.target_time* seconds, returning the eventual (number of loops, time taken for that number of loops). It calls `.timeit` with increasing numbers from the sequence 1, 2, 5, 10, 20, 50, ... until the time taken is at least *target_time* seconds.

If *callback* is given and is not `None`, it will be called after each trial with two arguments: `callback(number, time_taken)`.

<div class="versionadded">

3.6

</div>

<div class="versionchanged">

3.15 The optional *target_time* parameter was added.

</div>

</div>

<div class="method">

Timer.repeat(repeat=5, number=1000000)

Call `.timeit` a few times.

This is a convenience function that calls the `.timeit` repeatedly, returning a list of results. The first argument specifies how many times to call `.timeit`. The second argument specifies the *number* argument for `.timeit`.

> [!NOTE]
> It's tempting to calculate mean and standard deviation from the result vector and report these. However, this is not very useful. In a typical case, the lowest value gives a lower bound for how fast your machine can run the given code snippet; higher values in the result vector are typically not caused by variability in Python's speed, but by other processes interfering with your timing accuracy. So the `min` of the result is probably the only number you should be interested in. After that, you should look at the entire vector and apply common sense rather than statistics.

<div class="versionchanged">

3.7 Default value of *repeat* changed from 3 to 5.

</div>

</div>

<div class="method">

Timer.print_exc(file=None)

Helper to print a traceback from the timed code.

Typical use:

    t = Timer(...)       # outside the try/except
    try:
        t.timeit(...)    # or t.repeat(...)
    except Exception:
        t.print_exc()

The advantage over the standard traceback is that source lines in the compiled template will be displayed. The optional *file* argument directs where the traceback is sent; it defaults to `sys.stderr`.

</div>

</div>

## Command-line interface

When called as a program from the command line, the following form is used:

    python -m timeit [-n N] [-r N] [-u U] [-s S] [-p] [-v] [-h] [statement ...]

Where the following options are understood:

<div class="program">

timeit

</div>

<div class="option">

`-n N, --number=N`  

how many times to execute 'statement'

</div>

<div class="option">

`-r N, --repeat=N`  

how many times to repeat the timer (default 5)

</div>

<div class="option">

`-s S, --setup=S`  

statement to be executed once initially (default `pass`)

</div>

<div class="option">

`-p, --process`  

measure process time, not wallclock time, using `time.process_time` instead of `time.perf_counter`, which is the default

<div class="versionadded">

3.3

</div>

</div>

<div class="option">

`-u, --unit=U`  

specify a time unit for timer output; can select `nsec`, `usec`, `msec`, or `sec`

<div class="versionadded">

3.5

</div>

</div>

<div class="option">

`-t, --target-time=T`  

if `--number` is 0, the code will run until it takes at least this many seconds (default: 0.2)

<div class="versionadded">

3.15

</div>

</div>

<div class="option">

`-v, --verbose`  

print raw timing results; repeat for more digits precision

</div>

<div class="option">

`-h, --help`  

print a short usage message and exit

</div>

A multi-line statement may be given by specifying each line as a separate statement argument; indented lines are possible by enclosing an argument in quotes and using leading spaces. Multiple `-s` options are treated similarly.

If `-n` is not given, a suitable number of loops is calculated by trying increasing numbers from the sequence 1, 2, 5, 10, 20, 50, ... until the total time is at least `--target-time` seconds (default: 0.2).

`default_timer` measurements can be affected by other programs running on the same machine, so the best thing to do when accurate timing is necessary is to repeat the timing a few times and use the best time. The `-r` option is good for this; the default of 5 repetitions is probably enough in most cases. You can use `time.process_time` to measure CPU time.

> [!NOTE]
> There is a certain baseline overhead associated with executing a pass statement. The code here doesn't try to hide it, but you should be aware of it. The baseline overhead can be measured by invoking the program without arguments, and it might differ between Python versions.

<div class="versionadded">

3.15 Output is in color by default and can be `controlled using environment variables <using-on-controlling-color>`.

</div>

## Examples

It is possible to provide a setup statement that is executed only once at the beginning:

```
$ python -m timeit -s "text = 'sample string'; char = 'g'" "char in text"
5000000 loops, best of 5: 0.0877 usec per loop
$ python -m timeit -s "text = 'sample string'; char = 'g'" "text.find(char)"
1000000 loops, best of 5: 0.342 usec per loop
```

In the output, there are three fields. The loop count, which tells you how many times the statement body was run per timing loop repetition. The repetition count ('best of 5') which tells you how many times the timing loop was repeated, and finally the time the statement body took on average within the best repetition of the timing loop. That is, the time the fastest repetition took divided by the loop count.

    >>> import timeit
    >>> timeit.timeit('char in text', setup='text = "sample string"; char = "g"')
    0.41440500499993504
    >>> timeit.timeit('text.find(char)', setup='text = "sample string"; char = "g"')
    1.7246671520006203

The same can be done using the `Timer` class and its methods:

    >>> import timeit
    >>> t = timeit.Timer('char in text', setup='text = "sample string"; char = "g"')
    >>> t.timeit()
    0.3955516149999312
    >>> t.repeat()
    [0.40183617287970225, 0.37027556854118704, 0.38344867356679524, 0.3712595970846668, 0.37866875250654886]

The following examples show how to time expressions that contain multiple lines. Here we compare the cost of using `hasattr` vs. `try`/`except` to test for missing and present object attributes:

```
$ python -m timeit "try:" "  str.__bool__" "except AttributeError:" "  pass"
20000 loops, best of 5: 15.7 usec per loop
$ python -m timeit "if hasattr(str, '__bool__'): pass"
50000 loops, best of 5: 4.26 usec per loop

$ python -m timeit "try:" "  int.__bool__" "except AttributeError:" "  pass"
200000 loops, best of 5: 1.43 usec per loop
$ python -m timeit "if hasattr(int, '__bool__'): pass"
100000 loops, best of 5: 2.23 usec per loop
```

    >>> import timeit
    >>> # attribute is missing
    >>> s = """\
    ... try:
    ...     str.__bool__
    ... except AttributeError:
    ...     pass
    ... """
    >>> timeit.timeit(stmt=s, number=100000)
    0.9138244460009446
    >>> s = "if hasattr(str, '__bool__'): pass"
    >>> timeit.timeit(stmt=s, number=100000)
    0.5829014980008651
    >>>
    >>> # attribute is present
    >>> s = """\
    ... try:
    ...     int.__bool__
    ... except AttributeError:
    ...     pass
    ... """
    >>> timeit.timeit(stmt=s, number=100000)
    0.04215312199994514
    >>> s = "if hasattr(int, '__bool__'): pass"
    >>> timeit.timeit(stmt=s, number=100000)
    0.08588060699912603

To give the `!timeit` module access to functions you define, you can pass a *setup* parameter which contains an import statement:

    def test():
        """Stupid test function"""
        L = [i for i in range(100)]

    if __name__ == '__main__':
        import timeit
        print(timeit.timeit("test()", setup="from __main__ import test"))

Another option is to pass `globals` to the *globals* parameter, which will cause the code to be executed within your current global namespace. This can be more convenient than individually specifying imports:

    def f(x):
        return x**2
    def g(x):
        return x**4
    def h(x):
        return x**8

    import timeit
    print(timeit.timeit('[func(42) for func in (f,g,h)]', globals=globals()))

# Graphical user interfaces with Tk

<div class="index">

single: GUI single: Graphical User Interface single: Tkinter single: Tk

</div>

Tk/Tcl has long been an integral part of Python. It provides a robust and platform independent windowing toolkit, that is available to Python programmers using the `tkinter` package, and its extension, the `tkinter.ttk` module.

The `tkinter` package is a thin object-oriented layer on top of Tcl/Tk. To use `tkinter`, you don't need to write Tcl code, but you will need to consult the Tk documentation, and occasionally the Tcl documentation. `tkinter` is a set of wrappers that implement the Tk widgets as Python classes.

`tkinter`'s chief virtues are that it is fast, and that it usually comes bundled with Python. Although its standard documentation is weak, good material is available, which includes: references, tutorials, a book and others. `tkinter` is also famous for having an outdated look and feel, which has been vastly improved in Tk 8.5. Nevertheless, there are many other GUI libraries that you could be interested in. The Python wiki lists several alternative [GUI frameworks and tools](https://wiki.python.org/moin/GuiProgramming).

<div class="toctree">

tkinter.rst tkinter.colorchooser.rst tkinter.font.rst tkinter.fontchooser.rst tkinter.dialogs.rst tkinter.messagebox.rst tkinter.scrolledtext.rst tkinter.systray.rst tkinter.dnd.rst tkinter.ttk.rst idle.rst turtle.rst

</div>

# `!tkinter.colorchooser` --- Color choosing dialog

<div class="module" data-synopsis="Color choosing dialog">

tkinter.colorchooser

</div>

**Source code:** `Lib/tkinter/colorchooser.py`

------------------------------------------------------------------------

The `!tkinter.colorchooser` module provides the `Chooser` class as an interface to the native color picker dialog. `Chooser` implements a modal color choosing dialog window. The `Chooser` class inherits from the `~tkinter.commondialog.Dialog` class.

<div class="Chooser(master=None, **options)">

The class implementing the modal color-choosing dialog. Most applications use the `askcolor` convenience function rather than instantiating this class directly.

</div>

<div class="function">

askcolor(color=None, \*\*options)

Show a modal color-choosing dialog and return the chosen color. *color* is the color selected when the dialog opens. The return value is a tuple `((r, g, b), hexstr)`, where `r`, `g` and `b` are the red, green and blue components as integers in the range 0–255 and *hexstr* is the equivalent Tk color string, such as `'#ff8000'`. If the user cancels the dialog, `(None, None)` is returned.

<div class="versionchanged">

3.10 The RGB values in the returned color are now integers in the range 0–255 instead of floats.

</div>

</div>

<div class="seealso">

Module `tkinter.commondialog`  
Tkinter standard dialog module

</div>

# Tkinter dialogs

## `!tkinter.simpledialog` --- Standard Tkinter input dialogs

<div class="module" data-synopsis="Simple dialog windows">

tkinter.simpledialog

</div>

**Source code:** `Lib/tkinter/simpledialog.py`

------------------------------------------------------------------------

The `!tkinter.simpledialog` module contains convenience classes and functions for creating simple modal dialogs to get a value from the user.

<div class="function">

askfloat(title, prompt, *, initialvalue=None, minvalue=None, maxvalue=None, parent=None, use_ttk=True) askinteger(title, prompt,*, initialvalue=None, minvalue=None, maxvalue=None, parent=None, use_ttk=True) askstring(title, prompt, \*, initialvalue=None, show=None, parent=None, use_ttk=True)

Prompt the user to enter a value of the desired type and return it, or `None` if the dialog is cancelled.

*title* is the dialog title and *prompt* the message shown above the entry. *initialvalue* is the value initially placed in the entry. *parent* is the window over which the dialog is shown. `askinteger` and `askfloat` also accept *minvalue* and *maxvalue*, which bound the accepted value. `askstring` also accepts *show*, a character used to mask the entered text, for example `'*'` to hide a password. They use the themed `tkinter.ttk` widgets; pass `use_ttk=False` for the classic widgets.

</div>

<div class="Dialog(parent, title=None, *, use_ttk=False)">

The base class for custom dialogs. Instantiating it shows the dialog modally and returns once the user closes it; the entered value is then available in the `!result` attribute. When *use_ttk* is false (the default), the dialog is built from the classic `tkinter` widgets, modelled on the classic `tk_dialog`; when true, from the themed `tkinter.ttk` widgets, modelled on the Tk message box. The default is classic for compatibility, since the themed widgets set a themed background that classic widgets added in `body` would not match.

<div class="versionchanged">

3.16 Added the *use_ttk* parameter.

</div>

<div class="attribute">

result

The value produced by `apply`, or `None` if the dialog was cancelled.

</div>

<div class="method">

body(master)

Override to construct the dialog's interface and return the widget that should have initial focus.

</div>

<div class="method">

buttonbox()

Default behaviour adds OK and Cancel buttons. Override for custom button layouts.

</div>

<div class="method">

validate()

Validate the data entered by the user. Return true if it is valid, in which case the dialog proceeds to `apply`; return false to keep the dialog open. The default implementation always returns true; override it to check the input.

</div>

<div class="method">

apply()

Process the data entered by the user, for example by storing it in the `!result` attribute. Called after `validate` succeeds and just before the dialog is destroyed. The default implementation does nothing; override it to act on or store the result.

</div>

<div class="method">

destroy()

Destroy the dialog window, clearing the reference to the widget that had the initial focus.

</div>

</div>

<div class="SimpleDialog(master, text='', buttons=[], default=None, cancel=None, title=None, class_=None, *, bitmap=None, detail='', use_ttk=True)">

A simple modal dialog that displays the message *text* above a row of push buttons given by *buttons*, and returns the index of the button the user presses. Each entry of *buttons* is either a button label, or a mapping of button options such as `{'text': 'OK', 'underline': 0}`; an `underline` option makes `Alt` plus the underlined character invoke the button. *default* is the index of the default button, activated by the Return key when no button has the focus, *cancel* the index returned when the window is closed through the window manager, *title* the window title, and *class\_* the Tk class name of the window. *bitmap* is the name of a bitmap displayed beside the message (for example `'warning'` or `'question'`); the standard names `'error'`, `'info'`, `'question'` and `'warning'` are shown as themed icons when *use_ttk* is true. *detail* is a secondary message displayed below *text*. When *use_ttk* is true (the default), the dialog is built from the themed `tkinter.ttk` widgets, modelled on the Tk message box; when false, from the classic `tkinter` widgets, modelled on `tk_dialog`.

<div class="versionchanged">

3.16 The dialog is now built from the themed `tkinter.ttk` widgets by default, instead of the classic `tkinter` widgets. Added the *bitmap*, *detail* and *use_ttk* parameters. Entries of *buttons* may be mappings of button options.

</div>

<div class="method">

go()

Display the dialog, wait until the user presses a button or closes the window, and return the index of the chosen button.

</div>

</div>

## `!tkinter.filedialog` --- File selection dialogs

<div class="module" data-synopsis="Dialog classes for file selection">

tkinter.filedialog

</div>

**Source code:** `Lib/tkinter/filedialog.py`

------------------------------------------------------------------------

The `!tkinter.filedialog` module provides classes and factory functions for creating file/directory selection windows.

### Native load/save dialogs

The following classes and functions provide file dialog windows that combine a native look-and-feel with configuration options to customize behaviour. The following keyword arguments are applicable to the classes and functions listed below:

> *parent* - the window to place the dialog on top of
>
> *title* - the title of the window
>
> *initialdir* - the directory that the dialog starts in
>
> *initialfile* - the file selected upon opening of the dialog
>
> *filetypes* - a sequence of (label, pattern) tuples, '\*' wildcard is allowed
>
> *defaultextension* - default extension to append to file (save dialogs)
>
> *multiple* - when true, selection of multiple items is allowed

**Static factory functions**

The below functions when called create a modal, native look-and-feel dialog, wait for the user's selection, and return it. The exact return value depends on the function (see below); when the dialog is cancelled it is the empty value documented for that function -- an empty string, an empty tuple, an empty list or `None`.

<div class="function">

askopenfile(mode="r", \*\*options)

Create an `Open` dialog and return the opened file object, or `None` if the dialog is cancelled. The file is opened in mode *mode* (read-only `'r'` by default).

</div>

<div class="function">

askopenfiles(mode="r", \*\*options)

Create an `Open` dialog and return a list of the opened file objects, or an empty list if cancelled. The files are opened in mode *mode* (read-only `'r'` by default).

<div class="deprecated-removed">

3.16 3.19 Opening several files at once is error-prone, and the returned list cannot be used in a `with` statement. Iterate over the names returned by `askopenfilenames` and open them one by one instead.

</div>

</div>

<div class="function">

asksaveasfile(mode="w", \*\*options)

Create a `SaveAs` dialog and return the opened file object, or `None` if the dialog is cancelled. The file is opened in mode *mode* (`'w'` by default).

</div>

<div class="function">

askopenfilename(**options) askopenfilenames(**options)

Create an `Open` dialog. `askopenfilename` returns the selected filename as a string, or an empty string if the dialog is cancelled. `askopenfilenames` returns a tuple of the selected filenames, or an empty tuple if cancelled.

</div>

<div class="function">

asksaveasfilename(\*\*options)

Create a `SaveAs` dialog and return the selected filename as a string, or an empty string if the dialog is cancelled.

</div>

<div class="function">

askdirectory(\*\*options)

Prompt the user to select a directory, and return its path as a string, or an empty string if the dialog is cancelled. Additional keyword option: *mustexist* - if true, the user may only select an existing directory (false by default).

</div>

<div class="Open(master=None, **options) SaveAs(master=None, **options) Directory(master=None, **options)">

The above three classes provide native dialog windows for loading and saving files and for selecting a directory.

</div>

**Convenience classes**

The below classes are used for creating file/directory windows from scratch. These do not emulate the native look-and-feel of the platform.

> [!NOTE]
> The *FileDialog* class should be subclassed for custom event handling and behaviour.

<div class="FileDialog(master, title=None, *, use_ttk=True)">

Create a basic file selection dialog. Its layout -- a filter entry, side-by-side directory and file lists, and a selection entry -- follows the classic Motif file selection dialog. When *use_ttk* is true (the default), the dialog is built from the themed `tkinter.ttk` widgets; when false, from the classic `tkinter` widgets.

<div class="versionchanged">

3.16 The dialog is now built from the themed `tkinter.ttk` widgets by default, instead of the classic `tkinter` widgets. Added the *use_ttk* parameter.

</div>

<div class="method">

cancel_command(event=None)

Trigger the termination of the dialog window.

</div>

<div class="method">

dirs_double_event(event)

Event handler for double-click event on directory.

</div>

<div class="method">

dirs_select_event(event)

Event handler for click event on directory.

</div>

<div class="method">

files_double_event(event)

Event handler for double-click event on file.

</div>

<div class="method">

files_select_event(event)

Event handler for single-click event on file.

</div>

<div class="method">

filter_command(event=None)

Filter the files by directory.

</div>

<div class="method">

get_filter()

Retrieve the file filter currently in use.

</div>

<div class="method">

get_selection()

Retrieve the currently selected item.

</div>

<div class="method">

go(dir_or_file=os.curdir, pattern="\*", default="", key=None)

Render dialog and start event loop.

</div>

<div class="method">

ok_event(event)

Exit dialog returning current selection.

</div>

<div class="method">

ok_command()

Called when the user confirms the current selection. The base implementation accepts the selection and closes the dialog; `LoadFileDialog` and `SaveFileDialog` override it to check the selection first.

</div>

<div class="method">

quit(how=None)

Exit dialog returning filename, if any.

</div>

<div class="method">

set_filter(dir, pat)

Set the file filter.

</div>

<div class="method">

set_selection(file)

Update the current file selection to *file*.

</div>

</div>

<div class="LoadFileDialog(master, title=None, *, use_ttk=True)">

A subclass of FileDialog that creates a dialog window for selecting an existing file.

<div class="versionchanged">

3.16 Added the *use_ttk* parameter.

</div>

<div class="method">

ok_command()

Test that a file is provided and that the selection indicates an already existing file.

</div>

</div>

<div class="SaveFileDialog(master, title=None, *, use_ttk=True)">

A subclass of FileDialog that creates a dialog window for selecting a destination file.

<div class="versionchanged">

3.16 Added the *use_ttk* parameter.

</div>

<div class="method">

ok_command()

Test whether or not the selection points to a valid file that is not a directory. Confirmation is required if an already existing file is selected.

</div>

</div>

## `!tkinter.commondialog` --- Dialog window templates

<div class="module" data-synopsis="Tkinter base class for dialogs">

tkinter.commondialog

</div>

**Source code:** `Lib/tkinter/commondialog.py`

------------------------------------------------------------------------

The `!tkinter.commondialog` module provides the `Dialog` class that is the base class for dialogs defined in other supporting modules.

<div class="Dialog(master=None, **options)">

<div class="method">

show(\*\*options)

Render the Dialog window.

</div>

</div>

## `!tkinter.dialog` --- Classic Tk dialog boxes

<div class="module" data-synopsis="A simple dialog box built on the classic Tk widgets.">

tkinter.dialog

</div>

**Source code:** `Lib/tkinter/dialog.py`

------------------------------------------------------------------------

The `!tkinter.dialog` module provides a simple modal dialog box built on the classic (non-themed) Tk widgets.

<div class="data">

DIALOG_ICON

The name of a bitmap (`'questhead'`) suitable for use as the *bitmap* of a `Dialog`.

</div>

<div class="Dialog(master=None, cnf={}, **kw)">

Display a modal dialog box built from the classic (non-themed) Tk widgets and wait for the user to press one of its buttons. The options, given through *cnf* or as keyword arguments, are all required: *title* (the window title), *text* (the message), *bitmap* (the name of a bitmap icon, such as `DIALOG_ICON`), *default* (the index of the default button) and *strings* (the sequence of button labels). After construction, the `!num` attribute holds the index of the button the user pressed.

<div class="method">

destroy()

Do nothing. The dialog window is destroyed automatically before the constructor returns, so there is nothing left for this method to do.

</div>

</div>

<div class="seealso">

Modules `tkinter.messagebox`, `tut-files`

</div>

# `!tkinter.dnd` --- Drag and drop support

<div class="module" data-synopsis="Tkinter drag-and-drop interface">

tkinter.dnd

</div>

**Source code:** `Lib/tkinter/dnd.py`

------------------------------------------------------------------------

> [!NOTE]
> This is experimental and due to be deprecated when it is replaced with the Tk DND.

The `!tkinter.dnd` module provides drag-and-drop support for objects within a single application, within the same window or between windows. To enable an object to be dragged, you must create an event binding for it that starts the drag-and-drop process. Typically, you bind a ButtonPress event to a callback function that you write (see `Bindings-and-Events`). The function should call `dnd_start`, where *source* is the object to be dragged, and *event* is the event that invoked the call (the argument to your callback function).

Selection of a target object occurs as follows:

1.  Top-down search of the area under the mouse for a target widget:
    - the target widget should have a callable *dnd_accept* attribute;
    - if *dnd_accept* is not present or returns `None`, the search moves to the parent widget;
    - if no target widget is found, the target object is `None`.
2.  Call to `<old_target>.dnd_leave(source, event)`.
3.  Call to `<new_target>.dnd_enter(source, event)`.
4.  Call to `<target>.dnd_commit(source, event)` to notify of the drop.
5.  Call to `<source>.dnd_end(target, event)` to signal the end of drag-and-drop.

<div class="DndHandler(source, event)">

The *DndHandler* class handles drag-and-drop events tracking Motion and ButtonRelease events on the root of the event widget.

<div class="method">

cancel(event=None)

Cancel the drag-and-drop process.

</div>

<div class="method">

finish(event, commit=0)

Execute end of drag-and-drop functions.

</div>

<div class="method">

on_motion(event)

Inspect area below mouse for target objects while a drag is performed.

</div>

<div class="method">

on_release(event)

Signal end of drag when the release pattern is triggered.

</div>

</div>

<div class="function">

dnd_start(source, event)

Factory function for the drag-and-drop process. Return the `DndHandler` instance managing the drag, or `None` if a drag could not be started.

</div>

<div class="seealso">

`Bindings-and-Events`

</div>

# `!tkinter.font` --- Tkinter font wrapper

<div class="module" data-synopsis="Tkinter font-wrapping class">

tkinter.font

</div>

**Source code:** `Lib/tkinter/font.py`

------------------------------------------------------------------------

The `!tkinter.font` module provides the `Font` class for creating and using named fonts.

The different font weights and slants are:

<div class="data">

NORMAL BOLD ITALIC ROMAN

</div>

<div class="Font(root=None, font=None, name=None, exists=False, **options)">

The `Font` class represents a font used by Tk widgets. It either creates a new *named font* or refers to an existing font. A named font is Tk's way of identifying a font as a single object that can be referred to by name and reconfigured in place, rather than respecifying its attributes at each use.

With *exists* false (the default), a new named font is created. Its attributes are taken from the font description *font* if it is given, overridden by any keyword *options*. The new font is named *name*, or a generated unique name if *name* is omitted.

With *exists* true, an existing font is referred to instead of being created. If *name* is given, it is the name of the font, which is reconfigured by *font* and *options* if either is given. If *name* is omitted, the font description *font* is wrapped as is, without creating a named font, so that it is used without loss of precision by `actual`, `measure` and `metrics`. In this case no keyword options are accepted, and the `!name` attribute is the description itself rather than a string.

The font description *font* is a tuple of the family name, the size and zero or more styles, or any other form accepted by Tk, such as the name of a named font.

The keyword *options* are:

> *family* - font family, for example, Courier, Times\
> *size* - font size\
>     If *size* is positive it is interpreted as size in points.\
>     If *size* is a negative number its absolute value is treated\
>     as size in pixels.\
> *weight* - font emphasis (NORMAL, BOLD)\
> *slant* - ROMAN, ITALIC\
> *underline* - font underlining (0 - none, 1 - underline)\
> *overstrike* - font strikeout (0 - none, 1 - strikeout)

<div class="versionchanged">

3.10 Two fonts now compare equal (`==`) only when both are `Font` instances with the same name belonging to the same Tcl interpreter.

</div>

<div class="versionchanged">

3.16 A font description can now be wrapped without creating a new named font, and keyword options now override the attributes of the specified *font*.

</div>

<div class="method">

actual(option=None, displayof=None)

Return the actual attributes of the font, which may differ from the requested ones because of platform limitations. With no *option*, return a dictionary of all the attributes; if *option* is given, return the value of that single attribute. The attributes are resolved on the display of the *displayof* widget, or the main application window if it is not specified.

</div>

<div class="method">

cget(option)

Retrieve an attribute of the font.

> [!NOTE]
> `!cget` and `configure` operate on a named font and raise `~tkinter.TclError` for a wrapped font description. Use `actual` to query the attributes of the latter.

</div>

<div class="method" data-no-typesetting="">

config(\*\*options)

</div>

<div class="method">

configure(\*\*options)

Modify one or more attributes of the font. With no arguments, return a dictionary of the current attributes.

`config` is an alias of `!configure`.

</div>

<div class="method">

copy()

Return a distinct copy of the current font: a new named font with the same attributes but a different name, which can be reconfigured independently of the original. If the current font wraps a font description, the copy is instead a named font with its resolved attributes.

</div>

<div class="method">

measure(text, displayof=None)

Return amount of space the text would occupy on the specified display when formatted in the current font, as an integer number of pixels. If no display is specified then the main application window is assumed.

</div>

<div class="method">

metrics(*options,*\*kw)

Return font-specific data. With no options, return a dictionary mapping each metric name to its integer value; if one option name is given, return that metric's value as an integer. Options include:

*ascent* - distance between baseline and highest point that a character of the font can occupy

*descent* - distance between baseline and lowest point that a character of the font can occupy

*linespace* - minimum vertical separation necessary between any two characters of the font that ensures no vertical overlap between lines.

*fixed* - 1 if font is fixed-width else 0

</div>

</div>

<div class="function">

families(root=None, displayof=None)

Return a tuple of the names of the available font families.

</div>

<div class="function">

names(root=None)

Return a tuple of the names of all the defined fonts.

</div>

<div class="function">

nametofont(name, root=None)

Return a `Font` representation of the existing named font *name*. *root* is the widget whose Tcl interpreter owns the font; if omitted, the default root window is used.

<div class="versionchanged">

3.10 The *root* parameter was added.

</div>

</div>

# `!tkinter.fontchooser` --- Font selection dialog

<div class="module" data-synopsis="Font selection dialog">

tkinter.fontchooser

</div>

<div class="versionadded">

3.16

</div>

**Source code:** `Lib/tkinter/fontchooser.py`

------------------------------------------------------------------------

The `!tkinter.fontchooser` module provides the `FontChooser` class as an interface to the native font selection dialog.

The font dialog is application-global: there is a single font dialog per Tcl interpreter, and all `FontChooser` instances configure the same dialog.

Depending on the platform, the dialog may be modal or modeless, so `~FontChooser.show` may return immediately. The selected font is not returned: it is passed as a `~tkinter.font.Font` object to the callback specified with the *command* option.

The dialog also generates two virtual events on the parent window (see the *parent* option):

`<<TkFontchooserVisibility>>`  
Generated when the dialog is shown or hidden. Query the *visible* option to tell which.

`<<TkFontchooserFontChanged>>`  
Generated when the selected font changes.

> [!NOTE]
> The *command* callback is the only reliable way to obtain the selected font. On some platforms the *font* option is not updated to the user's choice.

<div class="FontChooser(master=None, **options)">

The class implementing the font selection dialog.

*master* is the widget whose Tcl interpreter owns the dialog. If omitted, it defaults to *parent* if that is given, or to the default root window otherwise.

The supported configuration options are:

- *parent* --- the window to which the dialog and its virtual events are related. It defaults to the main window; on macOS the dialog is shown as a sheet attached to it, rather than as a free-standing panel.
- *title* --- the title of the dialog.
- *font* --- the font that is currently selected in the dialog.
- *command* --- a callback that is called with a `~tkinter.font.Font` object wrapping the selected font when the user selects a font.
- *visible* --- whether the dialog is currently displayed (read-only).

The *font* option accepts the forms supported by `tkinter.font.Font`.

<div class="method">

configure(**options) config(**options)

Query or modify the options of the font dialog. With no arguments, return a dict of all option values. With a string argument, return the value of that option. Otherwise, set the given options.

</div>

<div class="method">

cget(option)

Return the value of the given option of the font dialog.

</div>

<div class="method">

show()

Display the font dialog. Depending on the platform, this method may return immediately or only once the dialog has been withdrawn.

</div>

<div class="method">

hide()

Hide the font dialog if it is displayed.

</div>

</div>

<div class="seealso">

Module `tkinter.font`  
Tkinter font-handling utilities

</div>

# `!tkinter` --- Python interface to Tcl/Tk

<div class="module" data-synopsis="Interface to Tcl/Tk for graphical user interfaces">

tkinter

</div>

**Source code:** `Lib/tkinter/__init__.py`

------------------------------------------------------------------------

The `!tkinter` package ("Tk interface") is the standard Python interface to the Tcl/Tk GUI toolkit. Both Tk and `!tkinter` are available on most Unix platforms, including macOS, as well as on Windows systems.

Running `python -m tkinter` from the command line should open a window demonstrating a simple Tk interface, letting you know that `!tkinter` is properly installed on your system, and also showing what version of Tcl/Tk is installed, so you can read the Tcl/Tk documentation specific to that version.

Tkinter supports a range of Tcl/Tk versions, built either with or without thread support. Tcl/Tk 8.5.12 is the minimum supported version; the official Python binary release bundles Tcl/Tk 9.0. See the source code for the `_tkinter` module for more information about supported versions.

<div class="versionchanged">

3.11 Support for Tcl/Tk versions older than 8.5.12 was removed.

</div>

Tkinter is not a thin wrapper, but adds a fair amount of its own logic to make the experience more pythonic. This documentation will concentrate on these additions and changes, and refer to the official Tcl/Tk documentation for details that are unchanged.

> [!NOTE]
> Tcl/Tk 8.5 (2007) introduced a modern set of themed user interface components along with a new API to use them (see `tkinter.ttk`). Both old and new APIs are still available. Most documentation you will find online still uses the old API and can be woefully outdated.

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

<div class="seealso">

- [TkDocs](https://tkdocs.com/)  
  Extensive tutorial on creating user interfaces with Tkinter. Explains key concepts, and illustrates recommended approaches using the modern API.

- [Tkinter 8.5 reference: a GUI for Python](https://www.tkdocs.com/shipman/)  
  Reference documentation for Tkinter 8.5 detailing available classes, methods, and options.

Tcl/Tk Resources:

- [Tk commands](https://www.tcl-lang.org/man/tcl9.0/TkCmd/index.html)  
  Comprehensive reference to each of the underlying Tcl/Tk commands used by Tkinter.

- [Tcl/Tk Home Page](https://www.tcl.tk)  
  Additional documentation, and links to Tcl/Tk core development.

Books:

- [Modern Tkinter for Busy Python Developers](https://tkdocs.com/book.html)  
  By Mark Roseman. (ISBN 978-1999149567)

- [Python GUI programming with Tkinter](https://www.packtpub.com/en-us/product/python-gui-programming-with-tkinter-9781788835886)  
  By Alan D. Moore. (ISBN 978-1788835886)

- [Programming Python](https://learning-python.com/about-pp4e.html)  
  By Mark Lutz; has excellent coverage of Tkinter. (ISBN 978-0596158101)

- [Tcl and the Tk Toolkit (2nd edition)](https://www.amazon.com/exec/obidos/ASIN/032133633X)  
  By John Ousterhout, inventor of Tcl/Tk, and Ken Jones; does not cover Tkinter. (ISBN 978-0321336330)

</div>

## Architecture

Tcl/Tk is not a single library but rather consists of a few distinct modules, each with separate functionality and its own official documentation. Python's binary releases also ship an add-on module together with it.

Tcl  
Tcl is a dynamic interpreted programming language, just like Python. Though it can be used on its own as a general-purpose programming language, it is most commonly embedded into C applications as a scripting engine or an interface to the Tk toolkit. The Tcl library has a C interface to create and manage one or more instances of a Tcl interpreter, run Tcl commands and scripts in those instances, and add custom commands implemented in either Tcl or C. Each interpreter has an event queue, and there are facilities to send events to it and process them. Unlike Python, Tcl's execution model is designed around cooperative multitasking, and Tkinter bridges this difference (see [Threading model](#threading-model) for details).

Tk  
Tk is a [Tcl package](https://wiki.tcl-lang.org/37432) implemented in C that adds custom commands to create and manipulate GUI widgets. Each `Tk` object embeds its own Tcl interpreter instance with Tk loaded into it. Tk's widgets are very customizable, though at the cost of a dated appearance. Tk uses Tcl's event queue to generate and process GUI events.

Ttk  
Themed Tk (Ttk) is a newer family of Tk widgets that provide a much better appearance on different platforms than many of the classic Tk widgets. Ttk is distributed as part of Tk, starting with Tk version 8.5. Python bindings are provided in a separate module, `tkinter.ttk`.

Internally, Tk and Ttk use facilities of the underlying operating system, that is, Xlib on Unix/X11, Cocoa on macOS, GDI on Windows.

When your Python application uses a class in Tkinter, for example, to create a widget, the `!tkinter` module first assembles a Tcl/Tk command string. It passes that Tcl command string to an internal `_tkinter` binary module, which then calls the Tcl interpreter to evaluate it. The Tcl interpreter will then call into the Tk and/or Ttk packages, which will in turn make calls to Xlib, Cocoa, or GDI.

## Tkinter modules

Support for Tkinter is spread across several modules. Most applications will need the main `!tkinter` module, as well as the `tkinter.ttk` module, which provides the modern themed widget set and API:

    from tkinter import *
    from tkinter import ttk

The modules that provide Tk support include:

`!tkinter`  
Main Tkinter module.

`tkinter.colorchooser`  
Dialog to let the user choose a color.

`tkinter.commondialog`  
Base class for the dialogs defined in the other modules listed here.

`tkinter.filedialog`  
Common dialogs to allow the user to specify a file to open or save.

`tkinter.font`  
Utilities to help work with fonts.

`tkinter.fontchooser`  
Dialog to let the user choose a font.

`tkinter.messagebox`  
Access to standard Tk dialog boxes.

`tkinter.scrolledtext`  
Text widget with a vertical scroll bar built in.

`tkinter.simpledialog`  
Basic dialogs and convenience functions.

`tkinter.systray`  
System tray icon and desktop notifications.

`tkinter.ttk`  
Themed widget set introduced in Tk 8.5, providing modern alternatives for many of the classic widgets in the main `!tkinter` module.

Additional modules:

<div class="module" data-synopsis="A binary module that contains the low-level interface to Tcl/Tk.">

<span id="tkinter">tkinter</span>

</div>

`_tkinter`  
A binary module that contains the low-level interface to Tcl/Tk. It is automatically imported by the main `!tkinter` module, and should never be used directly by application programmers. It is usually a shared library (or DLL), but might in some cases be statically linked with the Python interpreter.

`idlelib`  
Python's Integrated Development and Learning Environment (IDLE). Based on `!tkinter`.

`!tkinter.constants`  
Symbolic constants that can be used in place of strings when passing various parameters to Tkinter calls. Automatically imported by the main `!tkinter` module.

`tkinter.dnd`  
(experimental) Drag-and-drop support for `!tkinter`. This will become deprecated when it is replaced with the Tk DND.

`turtle`  
Turtle graphics in a Tk window.

<div class="currentmodule">

tkinter

</div>

## Tkinter life preserver

This section is not designed to be an exhaustive tutorial on either Tk or Tkinter. For that, refer to one of the external resources noted earlier. Instead, this section provides a very quick orientation to what a Tkinter application looks like, identifies foundational Tk concepts, and explains how the Tkinter wrapper is structured.

The remainder of this section will help you to identify the classes, methods, and options you'll need in your Tkinter application, and where to find more detailed documentation on them, including in the official Tcl/Tk reference manual.

### A Hello World program

We'll start by walking through a "Hello World" application in Tkinter. This isn't the smallest one we could write, but has enough to illustrate some key concepts you'll need to know.

    from tkinter import *
    from tkinter import ttk
    root = Tk()
    frm = ttk.Frame(root, padding=10)
    frm.grid()
    ttk.Label(frm, text="Hello World!").grid(column=0, row=0)
    ttk.Button(frm, text="Quit", command=root.destroy).grid(column=1, row=0)
    root.mainloop()

After the imports, the next line creates an instance of the `Tk` class, which initializes Tk and creates its associated Tcl interpreter. It also creates a toplevel window, known as the root window, which serves as the main window of the application.

The following line creates a frame widget, which in this case will contain a label and a button we'll create next. The frame is fit inside the root window.

The next line creates a label widget holding a static text string. The `~Grid.grid` method is used to specify the relative layout (position) of the label within its containing frame widget, similar to how tables in HTML work.

A button widget is then created, and placed to the right of the label. When pressed, it will call the `~Misc.destroy` method of the root window.

Finally, the `mainloop` method puts everything on the display, and responds to user input until the program terminates.

### Important Tk concepts

Even this simple program illustrates the following key Tk concepts:

widgets  
A Tkinter user interface is made up of individual *widgets*. Each widget is represented as a Python object, instantiated from classes like `ttk.Frame`, `ttk.Label`, and `ttk.Button`.

widget hierarchy  
Widgets are arranged in a *hierarchy*. The label and button were contained within a frame, which in turn was contained within the root window. When creating each *child* widget, its *parent* widget is passed as the first argument to the widget constructor.

configuration options  
Widgets have *configuration options*, which modify their appearance and behavior, such as the text to display in a label or button. Different classes of widgets will have different sets of options.

geometry management  
Widgets aren't automatically added to the user interface when they are created. A *geometry manager* like `grid` controls where in the user interface they are placed.

event loop  
Tkinter reacts to user input, changes from your program, and even refreshes the display only when actively running an *event loop*. If your program isn't running the event loop, your user interface won't update.

### Understanding how Tkinter wraps Tcl/Tk

When your application uses Tkinter's classes and methods, internally Tkinter is assembling strings representing Tcl/Tk commands, and executing those commands in the Tcl interpreter attached to your application's `Tk` instance.

Whether it's trying to navigate reference documentation, trying to find the right method or option, adapting some existing code, or debugging your Tkinter application, there are times that it will be useful to understand what those underlying Tcl/Tk commands look like.

To illustrate, here is the Tcl/Tk equivalent of the main part of the Tkinter script above.

    ttk::frame .frm -padding 10
    grid .frm
    grid [ttk::label .frm.lbl -text "Hello World!"] -column 0 -row 0
    grid [ttk::button .frm.btn -text "Quit" -command "destroy ."] -column 1 -row 0

Tcl's syntax is similar to many shell languages, where the first word is the command to be executed, with arguments to that command following it, separated by spaces. Without getting into too many details, notice the following:

- The commands used to create widgets (like `ttk::frame`) correspond to widget classes in Tkinter.
- Tcl widget options (like `-text`) correspond to keyword arguments in Tkinter.
- Widgets are referred to by a *pathname* in Tcl (like `.frm.btn`), whereas Tkinter doesn't use names but object references.
- A widget's place in the widget hierarchy is encoded in its (hierarchical) pathname, which uses a `.` (dot) as a path separator. The pathname for the root window is just `.` (dot). In Tkinter, the hierarchy is defined not by pathname but by specifying the parent widget when creating each child widget.
- Operations which are implemented as separate *commands* in Tcl (like `grid` or `destroy`) are represented as *methods* on Tkinter widget objects. As you'll see shortly, at other times Tcl uses what appear to be method calls on widget objects, which more closely mirror what is used in Tkinter.

### How do I...? What option does...?

If you're not sure how to do something in Tkinter, and you can't immediately find it in the tutorial or reference documentation you're using, there are a few strategies that can be helpful.

First, remember that the details of how individual widgets work may vary across different versions of both Tkinter and Tcl/Tk. If you're searching documentation, make sure it corresponds to the Python and Tcl/Tk versions installed on your system.

When searching for how to use an API, it helps to know the exact name of the class, option, or method that you're using. Introspection, either in an interactive Python shell or with `print`, can help you identify what you need.

To find out what configuration options are available on any widget, call its `~Misc.configure` method, which returns a dictionary containing a variety of information about each object, including its default and current values. Use `~Misc.keys` to get just the names of each option.

    btn = ttk.Button(frm, ...)
    print(btn.configure().keys())

As most widgets have many configuration options in common, it can be useful to find out which are specific to a particular widget class. Comparing the list of options to that of a simpler widget, like a frame, is one way to do that.

    print(set(btn.configure().keys()) - set(frm.configure().keys()))

Similarly, you can find the available methods for a widget object using the standard `dir` function. If you try it, you'll see there are over 200 common widget methods, so again identifying those specific to a widget class is helpful.

    print(dir(btn))
    print(set(dir(btn)) - set(dir(frm)))

### Navigating the Tcl/Tk reference manual

As noted, the official [Tk commands](https://www.tcl-lang.org/man/tcl9.0/TkCmd/index.html) reference manual (man pages) is often the most accurate description of what specific operations on widgets do. Even when you know the name of the option or method that you need, you may still have a few places to look.

While all operations in Tkinter are implemented as method calls on widget objects, you've seen that many Tcl/Tk operations appear as commands that take a widget pathname as its first parameter, followed by optional parameters, for example

    destroy .
    grid .frm.btn -column 0 -row 0

Others, however, look more like methods called on a widget object (in fact, when you create a widget in Tcl/Tk, it creates a Tcl command with the name of the widget pathname, with the first parameter to that command being the name of a method to call).

    .frm.btn invoke
    .frm.lbl configure -text "Goodbye"

In the official Tcl/Tk reference documentation, you'll find most operations that look like method calls on the man page for a specific widget (for example, you'll find the `~tkinter.ttk.Button.invoke` method on the [ttk::button](https://www.tcl-lang.org/man/tcl9.0/TkCmd/ttk_button.html) man page), while functions that take a widget as a parameter often have their own man page (for example, [grid](https://www.tcl-lang.org/man/tcl9.0/TkCmd/grid.html)).

You'll find many common options and methods in the [options](https://www.tcl-lang.org/man/tcl9.0/TkCmd/options.html) or [ttk::widget](https://www.tcl-lang.org/man/tcl9.0/TkCmd/ttk_widget.html) man pages, while others are found in the man page for a specific widget class.

You'll also find that many Tkinter methods have compound names, for example, `~Misc.winfo_x`, `~Misc.winfo_height`, `~Misc.winfo_viewable`. You'd find documentation for all of these in the [winfo](https://www.tcl-lang.org/man/tcl9.0/TkCmd/winfo.html) man page.

> [!NOTE]
> Somewhat confusingly, there are also methods on all Tkinter widgets that don't actually operate on the widget, but operate at a global scope, independent of any widget. Examples are methods for accessing the clipboard or the system bell. (They happen to be implemented as methods in the base `Widget` class that all Tkinter widgets inherit from).

## Threading model

Python and Tcl/Tk have very different threading models, which `!tkinter` tries to bridge. If you use threads, you may need to be aware of this.

A Python interpreter may have many threads associated with it. In Tcl, multiple threads can be created, but each thread has a separate Tcl interpreter instance associated with it. Threads can also create more than one interpreter instance, though each interpreter instance can be used only by the one thread that created it.

Each `Tk` object created by `!tkinter` contains a Tcl interpreter. It also keeps track of which thread created that interpreter. Calls to `!tkinter` can be made from any Python thread. Internally, if a call comes from a thread other than the one that created the `Tk` object, an event is posted to the interpreter's event queue, and when executed, the result is returned to the calling Python thread.

Tcl/Tk applications are normally event-driven, meaning that after initialization, the interpreter runs an event loop (that is, `Tk.mainloop <Misc.mainloop>`) and responds to events. Because it is single-threaded, event handlers must respond quickly, otherwise they will block other events from being processed. To avoid this, any long-running computations should not run in an event handler, but are either broken into smaller pieces using timers, or run in another thread. This is different from many GUI toolkits where the GUI runs in a completely separate thread from all application code including event handlers.

If the Tcl interpreter is not running the event loop and processing events, any `!tkinter` calls made from threads other than the one running the Tcl interpreter will fail.

A number of special cases exist:

- Tcl/Tk libraries built without thread support are now rare: Tcl/Tk 9.0 (the bundled version) is always thread-aware, so this case only arises with some older 8.x builds. When the library is not thread-aware, `!tkinter` calls the library from the originating Python thread, even if this is different than the thread that created the Tcl interpreter. A global lock ensures only one call occurs at a time.
- While `!tkinter` allows you to create more than one instance of a `Tk` object (with its own interpreter), all interpreters that are part of the same thread share a common event queue, which gets ugly fast. In practice, don't create more than one instance of `Tk` at a time. Otherwise, it's best to create them in separate threads and ensure you're running a thread-aware Tcl/Tk build.
- Blocking event handlers are not the only way to prevent the Tcl interpreter from reentering the event loop. It is even possible to run multiple nested event loops or abandon the event loop entirely. If you're doing anything tricky when it comes to events or threads, be aware of these possibilities.
- There are a few select `!tkinter` functions that presently work only when called from the thread that created the Tcl interpreter.

## Handy reference

### Setting options

Options control things like the color and border width of a widget. Options can be set in three ways:

At object creation time, using keyword arguments  
    fred = Button(self, fg="red", bg="blue")

After object creation, treating the option name like a dictionary index  
    fred["fg"] = "red"
    fred["bg"] = "blue"

Use the config() method to update multiple attrs subsequent to object creation  
    fred.config(fg="red", bg="blue")

> [!NOTE]
> The `fg` and `bg` options used here, and other options that control a widget's appearance, belong to the classic `!tkinter` widgets. The themed `tkinter.ttk` widgets recommended in the introduction do not accept them; style a themed widget through the `ttk.Style <tkinter.ttk.Style>` class instead. The three ways of setting an option shown above apply to both widget sets.

For a complete explanation of a given option and its behavior, see the Tk man pages for the widget in question.

Note that the man pages list "STANDARD OPTIONS" and "WIDGET SPECIFIC OPTIONS" for each widget. The former is a list of options that are common to many widgets, the latter are the options that are idiosyncratic to that particular widget. The Standard Options are documented on the `options(3)` man page.

No distinction between standard and widget-specific options is made in this document. Some options don't apply to some kinds of widgets. Whether a given widget responds to a particular option depends on the class of the widget; buttons have a `command` option, labels do not.

The options supported by a given widget are listed in that widget's man page, or can be queried at runtime by calling the `~Misc.config` method without arguments, or by calling the `~Misc.keys` method on that widget. The return value of these calls is a dictionary whose key is the name of the option as a string (for example, `'relief'`) and whose values are 5-tuples.

Some options, like `bg`, are synonyms for common options with long names (`bg` is shorthand for "background").

| Index | Meaning                          | Example    |
|-------|----------------------------------|------------|
| 0     | option name                      | `'relief'` |
| 1     | option name for database lookup  | `'relief'` |
| 2     | option class for database lookup | `'Relief'` |
| 3     | default value                    | `'raised'` |
| 4     | current value                    | `'groove'` |

Example:

    >>> print(fred.config())
    {'relief': ('relief', 'relief', 'Relief', 'raised', 'groove')}

Of course, the dictionary printed will include all the options available and their values. This is meant only as an example.

### Geometry management<span id="the-packer"></span><span id="tkinter-geometry-management"></span><span id="pack-the-packer"></span>

<div class="index">

single: geometry management (widgets) single: packing (widgets)

</div>

Creating a widget does not display it. A widget appears only after it has been handed to a *geometry manager*, which works out its size and position inside its container and keeps the layout up to date as the container is resized or its content changes. Forgetting to call a geometry manager is a common early mistake: the widget is created, but nothing shows up.

Tk provides three geometry managers. Each is inherited by every widget, so any widget can be managed by any of them (but see the warning below about the incompatibility of grid and pack). The choice depends on the kind of layout you want.

`grid <Grid.grid_configure>`  
Arranges widgets in a two-dimensional table of rows and columns. It is the most flexible manager and the one to reach for by default: layouts that would otherwise need several nested frames can often be expressed as a single grid, and rows and columns can be told how to absorb extra space.

    ttk.Label(frm, text="Name:").grid(column=0, row=0, sticky="w")
    ttk.Entry(frm).grid(column=1, row=0)
    ttk.Button(frm, text="OK").grid(column=1, row=1, sticky="e")

`pack <Pack.pack_configure>`  
Stacks widgets against one side of their container -- `"top"` (the default), `"bottom"`, `"left"` or `"right"` -- and can make them fill or expand into the space that is left. It is convenient for simple arrangements, such as a single row or column of widgets or a content area framed by a toolbar and a status bar.

    toolbar.pack(side="top", fill="x")
    status.pack(side="bottom", fill="x")
    body.pack(side="left", expand=True, fill="both")

`place <Place.place_configure>`  
Positions each widget at an explicit spot, given either as absolute screen distances or as a fraction of the container's size. It offers the most control but the least automatic behavior, and is used the least; it suits special cases such as overlapping widgets or precise custom layouts.

    background.place(x=0, y=0, relwidth=1.0, relheight=1.0)
    badge.place(relx=1.0, rely=0.0, anchor="ne")

Layouts are built up by nesting: grid or pack widgets, including frames, inside a frame or toplevel. Toplevels are managed by the OS window manager. Classic and themed `tkinter.ttk` widgets can be managed interchangeably.

> [!WARNING]
> Do not apply `!pack` and `!grid` to two widgets that share the same container. The two managers negotiate sizes in incompatible ways, and the application can hang as they repeatedly resize the container against each other. To combine them, keep each manager's widgets in a separate frame.

The full set of options accepted by each manager, with their values and defaults, is documented under `Grid.grid_configure`, `Pack.pack_configure` and `Place.place_configure`; see also the `grid(3tk)`, `pack(3tk)` and `place(3tk)` man pages.

### Coupling widget variables

Some widgets can tie their current value directly to a program variable, so that the two stay in sync. Options such as `variable`, `textvariable`, `value`, `onvalue` and `offvalue` set up this connection: when the user changes the widget the variable is updated, and when the variable is set the widget redraws to match.

A widget can be linked only to a `Variable` object, not to an ordinary Python variable. This is not a limitation of `!tkinter` but a consequence of how the two languages differ: the link relies on Tcl being notified every time the value changes, and Python offers no way to react when a plain variable is reassigned. A `Variable` sidesteps this by keeping its value inside the Tcl interpreter and exposing it through explicit `~Variable.get` and `~Variable.set` methods.

Ready-made subclasses cover the common types: `StringVar`, `IntVar`, `DoubleVar` and `BooleanVar`. Pass one as a widget's `textvariable` (or `variable`) option, then read and update it with `~Variable.get` and `~Variable.set`; the widget tracks it with no further work on your part.

Keep a reference to the variable for as long as the widget uses it -- for example by storing it as an attribute. A `Variable` that is garbage collected removes its underlying Tcl variable, breaking the connection to the widget (see `Variable`).

For example:

    import tkinter as tk
    from tkinter import ttk

    root = tk.Tk()

    # Create the application variable and give it an initial value.
    contents = tk.StringVar(value="this is a variable")

    # Tell the entry widget to track the variable.
    entry = ttk.Entry(root, textvariable=contents)
    entry.pack()

    # Print the current value whenever the user presses Return.
    def print_contents(event):
        print("The current entry content is:", contents.get())

    entry.bind("<Return>", print_contents)

    # Setting the variable from the program updates the entry through the
    # same link.
    def clear():
        contents.set("")

    ttk.Button(root, text="Clear", command=clear).pack()

    root.mainloop()

### The window manager

<div class="index">

single: window manager (widgets)

</div>

The *window manager* is the part of the desktop responsible for the title bar, border and controls drawn around each top-level window, and for such things as its title, position, size and icon. Tk gives access to these through the `Wm` mixin, which is inherited by the `Tk` root window and by every `Toplevel`. You therefore call the window-manager methods directly on a top-level window. Each has a short name and an equivalent `wm_`-prefixed name, for example `~Wm.title` and `~Wm.wm_title`.

These methods act on the top-level window whether its content is built from the classic widgets or the themed `tkinter.ttk` widgets. To reach the top-level window containing an arbitrary widget, call its `~Misc.winfo_toplevel` method.

For example:

    import tkinter as tk
    from tkinter import ttk

    root = tk.Tk()
    root.title("My Application")
    root.geometry("640x480")
    root.minsize(320, 240)

    ttk.Label(root, text="Hello").pack(padx=20, pady=20)

    root.mainloop()

See `Wm` for the full set of window-manager methods.

### Tk option data types

<div class="index">

single: Tk Option Data Types

</div>

Many widget options documented in the reference accept values of a small number of common types, described here. The standard options and their types are described in the `options(3tk)` manual page.

anchor  
Legal values are points of the compass: `"n"`, `"ne"`, `"e"`, `"se"`, `"s"`, `"sw"`, `"w"`, `"nw"`, and also `"center"`. The *labelanchor* option of `LabelFrame` and `ttk.LabelFrame <tkinter.ttk.LabelFrame>` accepts also `"en"`, `"es"`, `"wn"` and `"ws"`. The *sticky* option of the grid geometry manager accepts a combination of the compass directions, such as `"nsew"`, or an empty string.

bitmap  
There are ten built-in, named bitmaps: `'error'`, `'gray12'`, `'gray25'`, `'gray50'`, `'gray75'`, `'hourglass'`, `'info'`, `'questhead'`, `'question'`, `'warning'`. On macOS the following bitmaps are available as well: `'application'`, `'accessory'`, `'folder'`, `'pfolder'`, `'trash'`, `'floppy'`, `'ramdisk'`, `'cdrom'`, `'preferences'`, `'querydoc'`, `'stop'`, `'note'` and `'caution'`. To specify an X bitmap file, give the full path to the file, preceded with an `@`, as in `"@/usr/contrib/bitmap/gumby.bit"`.

boolean  
You can pass the integers 0 or 1, `True` or `False`, or the strings `"0"`, `"1"`, `"yes"`, `"no"`, `"true"`, `"false"`, `"on"` or `"off"`. The strings are matched ignoring case, and an unambiguous prefix, such as `"t"`, is enough. Any non-zero integer means true.

callback  
This is any callable object. Most callbacks take no arguments. For example:

    def print_it():
        print("hi there")
    fred["command"] = print_it

Some options call it with arguments, such as the *command* option of `Scale`, which is called with the new value.

color  
Colors can be given as symbolic names, such as `"red"` or `"light blue"`, or as strings representing RGB values in 4 bit: `"#RGB"`, 8 bit: `"#RRGGBB"`, 12 bit: `"#RRRGGGBBB"`, or 16 bit: `"#RRRRGGGGBBBB"` ranges, where R,G,B here represent any legal hex digit. Tk provides a portable set of symbolic names available on all platforms; they are matched ignoring case and spaces, so `"light blue"`, `"Light Blue"` and `"lightblue"` are the same color. The full list of names, including the platform-specific names of the current system colors (for example `"systemButtonFace"` on Windows or `"systemWindowBody"` on macOS), is given in the `colors(3tk)` manual page. `~tkinter.Misc.winfo_rgb` returns the red, green and blue components of a color.

compound  
Determines how to combine an image and a text displayed in a widget. Legal values are `"none"` (display the image if there is one, otherwise the text), `"text"`, `"image"`, `"center"` (display the text in the center of the image) and `"top"`, `"bottom"`, `"left"`, `"right"` (display the image above, below, to the left of or to the right of the text). The themed widgets accept all these values, the classic widgets accept all except `"text"` and `"image"`.

cursor  
The name of the mouse cursor to display while the pointer is over the widget. Tk provides a portable set of cursor names available on all platforms (for example `"arrow"`, `"watch"`, `"cross"`, or `"hand2"`); the standard X cursor names from `cursorfont.h` may also be used, without the `XC_` prefix (so `XC_hand2` becomes `"hand2"`). The full list of names, including the platform-specific ones, is given in the `cursors(3tk)` manual page. You can also specify a bitmap and mask file of your own. On Windows a cursor file (`.cur` or `.ani`) may be used directly, giving its path preceded with an `@`, as in `"@C:/cursors/bart.ani"`.

distance  
Screen distances can be specified in either pixels or absolute distances. Pixels are given as numbers and absolute distances as strings, with the trailing character denoting units: `c` for centimetres, `i` for inches, `m` for millimetres, `p` for printer's points. For example, 3.5 inches is expressed as `"3.5i"`.

When a screen distance option is read back (for example with `!cget`), a distance with no unit suffix is returned as an `int` or a `float`. A distance with a unit suffix depends on the screen resolution and is returned as an opaque Tcl object that can be passed back to Tk.

<div class="versionchanged">

3.16 Screen distances with no unit suffix are returned as an `int` or a `float`.

</div>

font  
Tk uses a font description such as `{courier 10 bold}`; in `!tkinter` this is most naturally passed as a tuple of `(family, size, *styles)` (or as the equivalent string `"Courier 10 bold"`). Font sizes with positive numbers are measured in points; sizes with negative numbers are measured in pixels. The name of a named font, such as `"TkDefaultFont"`, a `tkinter.font.Font` instance, or, on X11, an XLFD font name can be used as well.

justify  
Legal values are the strings: `"left"`, `"center"`, and `"right"`.

orient  
Legal values are `"horizontal"` and `"vertical"`.

padding  
This is a legal distance (see above), applied on all sides, or a sequence of up to four distances specifying the left, top, right and bottom padding. If fewer than four are specified, the bottom padding defaults to the top one, the right one to the left one, and the top one to the left one. It is only supported by the themed widgets.

region  
Used for the *scrollregion* option of `Canvas`. This is a string with four space-delimited elements, each of which is a legal distance (see above), or a sequence of four such elements. For example: `"2 3 4 5"`, `(2, 3, 4, 5)`, `"3i 2i 4.5i 2i"` and `"3c 2c 4c 10.43c"` are all legal regions.

relief  
Determines what the border style of a widget will be. Legal values are: `"raised"`, `"sunken"`, `"flat"`, `"groove"`, `"ridge"`, and `"solid"`.

scrollcommand  
This is almost always the `!set` method of some scrollbar widget, but can be any widget method that takes a single argument.

state  
Legal values are `"normal"`, `"disabled"` and, depending on the widget, also `"active"`, `"hidden"` (for `Canvas` items) or `"readonly"` (for `Entry`, `Spinbox` and the themed widgets). The themed widgets have in addition a more general set of states, which is manipulated with `ttk.Widget.state <tkinter.ttk.Widget.state>`.

wrap  
Must be one of `"none"`, `"char"` or `"word"` for the `Text` widget and its tags; for a tag an empty string means to use the widget's value.

### Bindings and events

<div class="index">

single: bind (widgets) single: events (widgets)

</div>

The bind method from the widget command allows you to watch for certain events and to have a callback function trigger when that event type occurs. The form of the bind method is:

    def bind(self, sequence, func, add=''):

where:

sequence  
is a string that denotes the target kind of event. Physical events use the `<modifier-modifier-type-detail>` form (for example `"<Enter>"` or `"<Control-Button-1>"`); application-defined virtual events use double angle brackets, as in `"<<Paste>>"`. (See the `bind(3tk)` man page for details.)

func  
is a Python function, taking one argument, to be invoked when the event occurs. An Event instance will be passed as the argument. (Functions deployed this way are commonly known as *callbacks*.)

add  
is optional, either `''` or `'+'`. Passing an empty string denotes that this binding is to replace any other bindings that this event is associated with. Passing a `'+'` means that this function is to be added to the list of functions bound to this event type.

For example:

    def turn_red(self, event):
        event.widget["activeforeground"] = "red"

    self.button.bind("<Enter>", self.turn_red)

Notice how the widget field of the event is being accessed in the `turn_red()` callback. This field contains the widget that caught the X event. The following table lists the other event fields you can access, and how they are denoted in Tk, which can be useful when referring to the Tk man pages.

| Tk  | Tkinter Event Field | Tk  | Tkinter Event Field |
|-----|---------------------|-----|---------------------|
| %f  | focus               | %A  | char                |
| %h  | height              | %E  | send_event          |
| %k  | keycode             | %K  | keysym              |
| %s  | state               | %N  | keysym_num          |
| %t  | time                | %T  | type                |
| %w  | width               | %W  | widget              |
| %x  | x                   | %X  | x_root              |
| %y  | y                   | %Y  | y_root              |
| %#  | serial              | %b  | num                 |
| %d  | detail              | %D  | delta               |

The `add` parameter above only affects the bindings you make yourself. Every widget also inherits *class bindings* that implement its standard behavior -- for example a `Text` widget binds `Control-t` to transpose two characters. These are described in the bindings section of the widget's Tk man page (such as `text(3tk)` or `entry(3tk)`).

Class bindings are processed separately from your own, so binding an event yourself does not replace the default; both run. To suppress an unwanted default binding, bind the event on the widget and return the string `"break"` from your callback.

### The index parameter

A number of widgets require "index" parameters to be passed. These are used to point at a specific place in a Text widget, or to particular characters in an Entry widget, or to particular menu items in a Menu widget.

Entry widget indexes (index, view index, etc.)  
Entry widgets have methods and options that refer to character positions in the text being displayed. Anytime an index is needed, you may pass in:

- an integer which refers to the numeric position of a character, counted from the beginning of the text, starting with 0;
- the string `"anchor"`, which refers to the anchor point of the selection, set with the widget's selection methods;
- the string `"end"`, which refers to the position just after the last character;
- the string `"insert"`, which refers to the character just after the insertion cursor;
- the strings `"sel.first"` and `"sel.last"`, which refer to the first character in the selection and the position just after the last (it is an error to use these if there is no selection);
- a string consisting of `@` followed by an integer, as in `"@6"`, where the integer is interpreted as an x pixel coordinate in the entry's coordinate system, selecting the character spanning that point.

Text widget indexes  
The index notation for Text widgets is very rich and is best described in the Tk man pages.

Menu indexes (menu.invoke(), menu.entryconfig(), etc.)  
Some options and methods for menus manipulate specific menu entries. Anytime a menu index is needed for an option or a parameter, you may pass in:

- an integer which refers to the numeric position of the entry in the widget, counted from the top, starting with 0;
- the string `"active"`, which refers to the menu position that is currently under the cursor;
- the string `"last"` which refers to the last menu item;
- a string consisting of `@` followed by an integer, as in `"@6"`, where the integer is interpreted as a y pixel coordinate in the menu's coordinate system;
- the string `"none"`, which indicates no menu entry at all, most often used with menu.activate() to deactivate all entries, and finally,
- a text string that is pattern matched against the label of the menu entry, as scanned from the top of the menu to the bottom. Note that this index type is considered after all the others, which means that matches for menu items labelled `last`, `active`, or `none` may be interpreted as the above literals, instead.

### Images

Images of different formats can be created through the corresponding subclass of `tkinter.Image`:

- `BitmapImage` for images in XBM format.
- `PhotoImage` for images in PGM, PPM, GIF and PNG formats. The latter is supported starting with Tk 8.6.

Either type of image is created through either the `file` or the `data` option (other options are available as well).

<div class="versionchanged">

3.13 Added the `!PhotoImage` method `!copy_replace` to copy a region from one image to other image, possibly with pixel zooming and/or subsampling. Add *from_coords* parameter to `!PhotoImage` methods `!copy`, `!zoom` and `!subsample`. Add *zoom* and *subsample* parameters to `!PhotoImage` method `!copy`.

</div>

The image object can then be used wherever an `image` option is supported by some widget (for example, labels, buttons, menus). In these cases, Tk will not keep a reference to the image. When the last Python reference to the image object is deleted, the image data is deleted as well, and Tk will display an empty box wherever the image was used.

<div class="seealso">

The [Pillow](https://python-pillow.org/) package adds support for formats such as BMP, JPEG, TIFF, and WebP, among others.

</div>

## Reference

<div class="currentmodule">

tkinter

</div>

This section documents the classes, methods, functions and constants of the `!tkinter` module. Most of them wrap Tcl/Tk commands; consult the official Tcl/Tk manual pages for the full list of widget options and further details.

<div class="exception">

TclError

The exception raised when a call into the Tcl interpreter fails, for example when a widget is given an unknown option or an invalid value.

</div>

### Base and mixin classes

<div class="Misc()">

The `!Misc` class is a mix-in inherited by `Tk` and, through `BaseWidget`, by every widget. It provides the large set of methods common to all Tk objects: querying window information, managing event bindings and the event loop, controlling the keyboard focus and pointer grabs, accessing the selection, clipboard and option database, and assorted utility and introspection services. Because they are inherited, these methods are available on every widget and on the `Tk` application object, and are documented here once rather than repeated for each widget.

<div class="method">

cget(key)

Return the current value of the configuration option named *key* for this widget, as a string. The expression `widget[key]` is equivalent and may be used instead.

</div>

<div class="method" data-no-typesetting="">

config(cnf=None, \*\*kw)

</div>

<div class="method">

configure(cnf=None, \*\*kw)

Query or modify the configuration options of the widget. With no arguments, return a dictionary mapping every available option name to a tuple describing it (its name, X resource name, X resource class, default value and current value). If a single option name is given as a string, return the tuple for just that option. If one or more keyword arguments are given, or a dictionary is passed as *cnf*, set each named option to the corresponding value; the expression `widget[key] = value` sets a single option in the same way.

`config` is an alias of `!configure`.

</div>

<div class="method">

keys()

Return a list of the names of all configuration options of this widget.

</div>

<div class="method">

getboolean(s)

Interpret the string *s* as a Tcl boolean and return the corresponding `bool`. Tcl accepts values such as `'1'`, `'0'`, `'yes'`, `'no'`, `'true'`, `'false'`, `'on'` and `'off'`. Raise `ValueError` if *s* is not a valid boolean.

</div>

<div class="method">

getdouble(s)

Interpret the string *s* as a Tcl floating-point number and return it as a `float`. Raise `ValueError` if *s* is not a valid number.

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

getint(s)

Interpret the string *s* as a Tcl integer and return it as an `int`. Raise `ValueError` if *s* is not a valid integer.

</div>

<div class="method">

getvar(name)

Return the value of the Tcl global variable named *name*.

</div>

<div class="method">

setvar(name, value)

Set the Tcl global variable named *name* to *value*.

The `!getvar` and `!setvar` methods give direct access to Tcl variables. In most code you will instead use a `Variable` subclass such as `StringVar` or `IntVar`, which wraps a Tcl variable and converts its value to and from a Python type.

</div>

<div class="method">

register(func, subst=None, needcleanup=1)

Register the Python callable *func* as a Tcl command and return the name of the new command as a string. Whenever Tcl invokes that command, *func* is called; if *subst* is given, it is applied to the command's arguments first. This is the mechanism used internally to turn Python callbacks into the command names passed to Tk options such as *command*. Unless *needcleanup* is false, the command is deleted automatically when the widget is destroyed.

<div class="versionchanged">

3.13 The arguments passed to *func* are no longer converted to strings.

</div>

</div>

<div class="method">

deletecommand(name)

Delete the Tcl command named *name*, such as one previously returned by `register`.

</div>

<div class="method">

nametowidget(name)

Return the widget instance corresponding to the Tk pathname *name*.

</div>

<div class="method">

send(interp, cmd, \*args)

Send the Tcl command *cmd*, with the given *args*, to the Tcl interpreter registered under the name *interp*, and return its result. This is not available on all platforms.

</div>

<div class="method">

destroy()

Destroy this widget and all of its descendant widgets, and delete the Tcl commands associated with them.

</div>

<div class="method" data-no-typesetting="">

lift(aboveThis=None)

</div>

<div class="method">

tkraise(aboveThis=None)

Raise this widget in the stacking order so that it is drawn on top of its siblings. If *aboveThis* is given, the widget is moved to be just above it in the stacking order instead.

`lift` is an alias of `!tkraise`.

</div>

<div class="method">

lower(belowThis=None)

Lower this widget in the stacking order so that it is drawn beneath its siblings. If *belowThis* is given, the widget is moved to be just below it in the stacking order instead.

`tkraise`/`lift` and `lower` are overridden by the `Canvas` widget, where they restack canvas items instead.

</div>

<div class="method">

image_names()

Return the names of all images that currently exist in the Tcl interpreter.

This is overridden by the `Text` widget, where `!image_names` returns the names of its embedded images instead.

</div>

<div class="method">

image_types()

Return the available image types, such as `'photo'` and `'bitmap'`.

</div>

<div class="method" data-no-typesetting="">

anchor(anchor=None)

</div>

<div class="method">

grid_anchor(anchor=None)

Set the anchor that controls where the grid is placed inside this container when the container is larger than the grid and no row or column has a non-zero weight. *anchor* is one of the usual anchor strings, such as `'nw'` (the default) or `'center'`. Called with no argument, this method has no effect.

`anchor` is an alias of `!grid_anchor`.

<div class="versionadded">

3.3

</div>

</div>

<div class="method" data-no-typesetting="">

bbox(column=None, row=None, col2=None, row2=None)

</div>

<div class="method">

grid_bbox(column=None, row=None, col2=None, row2=None)

Return the bounding box, in pixels, of a region of the grid laid out in this container, as a 4-tuple `(xoffset, yoffset, width, height)`. With no arguments the bounding box of the whole grid is returned. If *column* and *row* are given, the box spans from the cell at row and column 0 to that cell; if *col2* and *row2* are also given, it spans from the cell (*column*, *row*) to the cell (*col2*, *row2*).

`bbox` is an alias of `!grid_bbox`, except on `Canvas`, `Listbox`, `Spinbox`, `Text`, `ttk.Entry <tkinter.ttk.Entry>` and `ttk.Treeview <tkinter.ttk.Treeview>`, which provide their own `!bbox` method.

</div>

<div class="method" data-no-typesetting="">

columnconfigure(index, cnf={}, \*\*kw)

</div>

<div class="method">

grid_columnconfigure(index, cnf={}, \*\*kw)

Query or set the properties of the column (or columns) *index* of the grid managed by this container. *index* may be a column number; when setting options it may also be a list of column numbers, the string `'all'` to affect every column, or a child widget whose occupied columns are affected. The supported options are:

*minsize*  
The column's minimum size, in pixels.

*weight*  
An integer setting how much of any extra space is apportioned to the column. A weight of `0` keeps the column at its requested size, and a column of weight two grows twice as fast as a column of weight one.

*uniform*  
The name of a uniform group. Columns sharing a non-empty group name are kept in sizes that are strictly proportional to their weights.

*pad*  
Extra space, in pixels, added to the largest widget in the column when computing the column's size.

With a single option name, return that option's value; with no options, return a dictionary of all of them.

`columnconfigure` is an alias of `!grid_columnconfigure`.

</div>

<div class="method" data-no-typesetting="">

rowconfigure(index, cnf={}, \*\*kw)

</div>

<div class="method">

grid_rowconfigure(index, cnf={}, \*\*kw)

Query or set the properties of the row (or rows) *index* of the grid managed by this container. *index* is interpreted as for `grid_columnconfigure`, and the supported options (*minsize*, *weight*, *uniform* and *pad*) are the same, applied to a row instead of a column.

`rowconfigure` is an alias of `!grid_rowconfigure`.

</div>

<div class="method">

grid_location(x, y)

Return the `(column, row)` of the grid cell that contains the pixel at position (*x*, *y*), given in pixels relative to this container. For locations above or to the left of the grid, `-1` is returned for the corresponding coordinate.

</div>

<div class="method">

grid_propagate() grid_propagate(flag)

Enable or disable geometry propagation for this container when it manages its children with the grid geometry manager. When *flag* is true, the container resizes itself to fit the requested sizes of its children; when it is false, its size is left under your control. Called with no argument, return the current setting as a boolean.

</div>

<div class="method" data-no-typesetting="">

size()

</div>

<div class="method">

grid_size()

Return the size of the grid managed by this container as a `(columns, rows)` tuple.

`size` is an alias of `!grid_size`, except on `Listbox` and `ttk.Treeview <tkinter.ttk.Treeview>`, which provide their own `!size` method.

</div>

<div class="method">

grid_slaves(row=None, column=None)

Return a list of the child widgets managed in this container's grid, most recently managed first. If *row* or *column* is given, only the children in that row or column are returned.

</div>

<div class="method">

grid_content(row=None, column=None)

Same as `grid_slaves`: return the child widgets managed in this container's grid, optionally restricted to a given *row* or *column*.

<div class="versionadded">

3.15

</div>

</div>

<div class="method" data-no-typesetting="">

propagate() propagate(flag)

</div>

<div class="method">

pack_propagate() pack_propagate(flag)

Enable or disable geometry propagation for this container when it manages its children with the pack geometry manager. When *flag* is true, the container resizes itself to fit the requested sizes of its children; when it is false, its size is left under your control. Called with no argument, return the current setting as a boolean.

`propagate` is an alias of `!pack_propagate`.

</div>

<div class="method" data-no-typesetting="">

slaves()

</div>

<div class="method">

pack_slaves()

Return a list of the child widgets managed by this container with the pack geometry manager, in packing order.

`slaves` is an alias of `!pack_slaves`.

</div>

<div class="method" data-no-typesetting="">

content()

</div>

<div class="method">

pack_content()

Same as `pack_slaves`: return the child widgets managed by this container with the pack geometry manager, in packing order.

`content` is an alias of `!pack_content`.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

place_slaves()

Return a list of the child widgets managed by this container with the place geometry manager.

</div>

<div class="method">

place_content()

Same as `place_slaves`: return the child widgets managed by this container with the place geometry manager.

<div class="versionadded">

3.15

</div>

</div>

The methods with the `bind` and `unbind` prefixes associate event patterns with callbacks and remove those associations.

<div class="method">

bind(sequence=None, func=None, add=None)

Bind the event pattern *sequence* on this widget to the callable *func*.

*sequence* is an event pattern, such as `'<Button-1>'` (a mouse click) or `'<KeyPress-a>'`, optionally a concatenation of several such patterns that must occur shortly after one another. When the event occurs, *func* is called with an `Event` instance describing it as its only argument; if *func* returns the string `'break'`, no further bindings for the event are invoked.

If *add* is true, *func* is added to any functions already bound to *sequence*; otherwise it replaces them. The binding applies only to this widget.

`!bind` returns a string identifier (a *funcid*) that can later be passed to `unbind` to remove the binding without leaking the associated Tcl command.

If *func* is omitted, return the binding currently associated with *sequence*; if *sequence* is also omitted, return a list of all the sequences for which bindings exist on this widget.

</div>

<div class="method">

bind_class(className, sequence=None, func=None, add=None)

Like `bind`, but bind *func* to the binding tag *className* rather than to a single widget, so that the binding applies to every widget having that tag. *className* is usually the name of a widget class, such as `'Button'`, in which case the binding affects all widgets of that class. The set of binding tags for a widget can be inspected and changed with `bindtags`.

The remaining arguments and the return value are as for `bind`.

</div>

<div class="method">

bind_all(sequence=None, func=None, add=None)

Like `bind`, but bind *func* to the special binding tag `'all'`, so that the binding applies to every widget in the application.

The remaining arguments and the return value are as for `bind`.

</div>

<div class="method">

unbind(sequence, funcid=None)

Remove bindings for the event pattern *sequence* on this widget.

If *funcid* is given, only the function identified by it (a value returned from a previous call to `bind`) is removed, and its associated Tcl command is deleted. Otherwise all bindings for *sequence* are destroyed, leaving it unbound.

<div class="versionchanged">

3.13 If *funcid* is given, only that callback is unbound; other callbacks bound to *sequence* are kept.

</div>

</div>

<div class="method">

unbind_class(className, sequence)

Remove all bindings for the event pattern *sequence* from the binding tag *className*. See `bind_class`.

</div>

<div class="method">

unbind_all(sequence)

Remove all bindings for the event pattern *sequence* from the special binding tag `'all'`. See `bind_all`.

</div>

<div class="method">

bindtags(tagList=None)

If *tagList* is omitted, return a tuple of the binding tags associated with this widget. When an event occurs in a widget, it is applied to each of the widget's binding tags in order, and for each tag the most specific matching binding is executed. By default a widget has four binding tags: its own pathname, its widget class, the pathname of its nearest toplevel ancestor, and `'all'`, in that order.

If *tagList* is given, it must be a sequence of strings; the widget's binding tags are set to its elements, which determines the order in which bindings are evaluated.

</div>

The methods with the `event_` prefix define virtual events and generate events programmatically.

<div class="method">

event_add(virtual, \*sequences)

Associate the virtual event *virtual*, whose name has the form `'<<Paste>>'`, with each of the physical event patterns given by *sequences*, so that the virtual event triggers whenever any of them occurs. If *virtual* is already defined, the new sequences are added to its existing ones.

</div>

<div class="method">

event_delete(virtual, \*sequences)

Remove each of *sequences* from those associated with the virtual event *virtual*. Sequences that are not currently associated with *virtual* are ignored. If no *sequences* are given, all physical event sequences are removed, so that *virtual* no longer triggers.

</div>

<div class="method">

event_generate(sequence, \*\*kw)

Generate the event *sequence* on this widget and arrange for it to be processed just as if it had come from the window system. *sequence* must be a single event pattern, such as `'<Button-1>'` or `'<<Paste>>'`, not a concatenation of several. Keyword arguments specify additional fields of the event, for example *x* and *y* for the pointer position, or *when* to control when the event is processed; refer to the Tk `event` manual page for the full list.

</div>

<div class="method">

event_info(virtual=None)

If *virtual* is omitted, return a tuple of all the virtual events that are currently defined. If *virtual* is given, return a tuple of the physical event sequences currently associated with it, or an empty tuple if it is not defined.

</div>

The methods with the `after` prefix schedule callbacks to run after a delay or when the application is idle.

<div class="method">

after(ms, func=None, *args,*\*kw)

Schedule the callable *func* to be called after *ms* milliseconds, with *args* and *kw* passed to it as positional and keyword arguments. Return an identifier that can be passed to `after_cancel` to cancel the call.

If *func* is omitted, sleep for *ms* milliseconds instead, processing no events during that time, and return `None`.

<div class="versionchanged">

3.10 *func* can now be any callable object, not only a function.

</div>

<div class="versionchanged">

3.14 Keyword arguments are now passed to *func*.

</div>

</div>

<div class="method">

after_cancel(id)

Cancel a callback previously scheduled with `after` or `after_idle`. *id* must be an identifier returned by one of those methods; passing a value that is not such an identifier raises `ValueError`. If the callback has already run or been cancelled, this has no effect.

<div class="versionchanged">

3.7 Passing `None` (or any false value) as *id* now raises `ValueError`.

</div>

</div>

<div class="method">

after_idle(func, *args,*\*kw)

Schedule the callable *func* to be called, with *args* and *kw* passed to it, when the Tk main loop next becomes idle, that is, when it has no other events to process. Return an identifier that can be passed to `after_cancel` to cancel the call.

<div class="versionchanged">

3.14 Keyword arguments are now passed to *func*.

</div>

</div>

<div class="method">

after_info(id=None)

If *id* is omitted, return a tuple of the identifiers of all callbacks currently scheduled with `after` and `after_idle` for this interpreter.

If *id* is given, it must identify a callback that has not yet run or been cancelled, and the return value is a tuple `(script, type)`, where *script* refers to the function to be called and *type* is either `'idle'` or `'timer'`. A `TclError` is raised if *id* does not exist.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

mainloop(n=0)

Enter the Tk event loop, which processes events until all windows are destroyed. This is normally called once, on the root window, to run the application.

</div>

<div class="method">

quit()

Quit the Tcl interpreter, causing `mainloop` to return.

</div>

<div class="method">

update()

Enter the event loop until all pending events, including idle callbacks, have been processed. This brings the display up to date and handles any events that are already queued, then returns.

</div>

<div class="method">

update_idletasks()

Enter the event loop until all pending idle callbacks have been called. This updates the display of windows, for example after geometry changes, but does not process events caused by the user.

</div>

<div class="method" data-no-typesetting="">

waitvar(name)

</div>

<div class="method">

wait_variable(name)

Wait until the Tcl variable *name* is modified, continuing to process events in the meantime so that the application stays responsive. *name* is usually a `Variable` instance, such as an `IntVar` or `StringVar`.

`waitvar` is an alias of `!wait_variable`.

</div>

<div class="method">

wait_window(window=None)

Wait until *window* is destroyed, continuing to process events in the meantime. If *window* is omitted, this widget is used. This is typically used to wait for the user to finish interacting with a dialog box.

</div>

<div class="method">

wait_visibility(window=None)

Wait until the visibility state of *window* changes, for example when it first appears on the screen, continuing to process events in the meantime. If *window* is omitted, this widget is used. This is typically used to wait for a newly created window to become visible before acting on it.

</div>

The methods with the `focus_` prefix manage the keyboard focus.

<div class="method" data-no-typesetting="">

focus_set()

</div>

<div class="method">

focus()

Direct the keyboard input focus for this widget's display to this widget. If the application does not currently have the input focus on this widget's display, the widget is remembered as the focus window for its top level, and the focus will be redirected to it the next time the window manager gives the focus to the top level. `focus` is an alias of `!focus_set`, except on the `Canvas` and `ttk.Treeview <tkinter.ttk.Treeview>` widgets, which provide their own `!focus` method.

</div>

<div class="method">

focus_force()

Direct the keyboard input focus to this widget even if the application does not currently have the input focus for the widget's display. This method should be used sparingly, if at all; normally an application should wait for the window manager to give it the focus rather than claiming it.

</div>

<div class="method">

focus_get()

Return the widget that currently has the keyboard focus in the application, or `None` if no widget in the application has the focus. Use `focus_displayof` to work correctly with several displays.

</div>

<div class="method">

focus_displayof()

Return the widget that currently has the keyboard focus on the display where this widget is located, or `None` if no widget in the application has the focus on that display.

</div>

<div class="method">

focus_lastfor()

Return the most recent widget to have had the keyboard focus among all the widgets in the same top level as this widget; this is the widget that will receive the focus the next time the window manager gives the focus to the top level. If no widget in that top level has ever had the focus, or if the most recent focus widget has been deleted, the top level itself is returned.

</div>

<div class="method">

tk_focusFollowsMouse()

Reconfigure Tk to use an implicit focus model in which the focus is set to a widget whenever the mouse pointer enters it. This cannot easily be disabled once enabled.

</div>

<div class="method">

tk_focusNext()

Return the next widget after this one in the keyboard traversal order, or `None` if there is none. The traversal order goes first to the next child, then recursively to the children of that child, and then to the next sibling higher in the stacking order. A widget is skipped if its `takefocus` option is set to `0`. This method is used in the default bindings for the `Tab` key.

</div>

<div class="method">

tk_focusPrev()

Return the previous widget before this one in the keyboard traversal order, or `None` if there is none. See `tk_focusNext` for how the order is defined. This method is used in the default bindings for the `Shift-Tab` key.

</div>

The methods with the `grab_` prefix set and query the input grab, which directs all input events to a single widget.

<div class="method">

grab_set()

Set a local grab on this widget. A grab confines pointer events to this widget and its descendants: while the pointer is outside the widget's subtree, button presses and releases and pointer motion are reported to the grab widget, and windows outside the subtree become insensitive until the grab is released. A local grab affects only the grabbing application. Any grab previously set by this application on the widget's display is automatically released. Setting a grab is the usual way to make a dialog modal: while the grab is in effect the user cannot interact with the other windows of the application.

</div>

<div class="method">

grab_set_global()

Set a global grab on this widget. A global grab is like the local grab set by `grab_set`, but it locks out all other applications on the screen, so that only this widget's subtree is sensitive to pointer events, and it also grabs the keyboard. Use with caution: it is easy to render a display unusable with a global grab, since other applications stop receiving events until it is released.

</div>

<div class="method">

grab_release()

Release the grab on this widget if there is one; otherwise do nothing.

</div>

<div class="method">

grab_current()

Return the widget that currently holds the grab in this application for this widget's display, or `None` if there is no such widget.

</div>

<div class="method">

grab_status()

Return `None` if no grab is currently set on this widget, `"local"` if a local grab is set, or `"global"` if a global grab is set.

</div>

The methods with the `selection_` prefix retrieve and manage the X selection.

<div class="method">

selection_clear(\*\*kw)

Clear the X selection, so that no window owns it anymore. The selection to clear is given by the keyword argument *selection*, an atom name such as `'PRIMARY'` or `'CLIPBOARD'`; it defaults to `PRIMARY`. The *displayof* keyword argument names a widget that determines the display on which to operate, and defaults to this widget.

This is overridden by the `Entry`, `Listbox` and `Spinbox` widgets, where `!selection_clear` clears the widget's own selection instead.

</div>

<div class="method">

selection_get(\*\*kw)

Return the contents of the current X selection. The keyword argument *selection* names the selection and defaults to `PRIMARY`. The keyword argument *type* specifies the form in which the data is to be returned (the desired conversion target), an atom name such as `'STRING'` or `'FILE_NAME'`; it defaults to `STRING`, except on X11, where `UTF8_STRING` is tried first and `STRING` is used as a fallback. The *displayof* keyword argument names a widget that determines the display from which to retrieve the selection, and defaults to this widget.

</div>

<div class="method">

selection_handle(command, \*\*kw)

Register *command* as a handler to supply the X selection owned by this widget when another application requests it. When the selection is retrieved, *command* is called with two arguments, the starting character offset and the maximum number of characters to return, and must return at most that many characters of the selection starting at that offset; for very long selections it is called repeatedly with increasing offsets. The keyword argument *selection* names the selection (default `PRIMARY`) and the keyword argument *type* gives the form of the selection that the handler supplies (such as `'STRING'` or `'FILE_NAME'`, default `STRING`).

</div>

<div class="method">

selection_own(\*\*kw)

Make this widget the owner of the X selection on its display. The previous owner, if any, is notified that it has lost the selection. The keyword argument *selection* names the selection and defaults to `PRIMARY`.

</div>

<div class="method">

selection_own_get(\*\*kw)

Return the widget in this application that owns the X selection on the display containing this widget, or `None` if no widget in this application owns the selection. The keyword argument *selection* names the selection and defaults to `PRIMARY`. The *displayof* keyword argument names a widget that determines the display to query, and defaults to this widget.

</div>

The methods with the `clipboard_` prefix manage the clipboard.

<div class="method">

clipboard_append(string, \*\*kw)

Append *string* to the Tk clipboard and claim ownership of the clipboard on this widget's display. Before appending, the clipboard should be emptied with `clipboard_clear`; all appends should be completed before returning to the event loop so that the clipboard is updated atomically. The keyword argument *type* specifies the form of the data, an atom name such as `'STRING'` or `'FILE_NAME'` (default `STRING`), and the keyword argument *format* specifies the representation used to transmit it (default `STRING`). The *displayof* keyword argument names a widget that determines the target display, and defaults to this widget. The contents can be retrieved with `clipboard_get` or `selection_get`.

</div>

<div class="method">

clipboard_clear(\*\*kw)

Claim ownership of the clipboard on this widget's display and remove any previous contents. The *displayof* keyword argument names a widget that determines the target display, and defaults to this widget.

</div>

<div class="method">

clipboard_get(\*\*kw)

Retrieve data from the clipboard on this widget's display. The keyword argument *type* specifies the form in which the data is to be returned, an atom name such as `'STRING'` or `'FILE_NAME'`; it defaults to `STRING`, except on X11, where `UTF8_STRING` is tried first and `STRING` is used as a fallback. The *displayof* keyword argument names a widget that determines the display, and defaults to the root window of the application. This is equivalent to `selection_get(selection='CLIPBOARD')`.

</div>

The methods with the `option_` prefix query and modify the Tk option database.

<div class="method">

option_add(pattern, value, priority=None)

Add an option to the Tk option database that associates *value* with *pattern*. *pattern* consists of names and/or classes separated by asterisks or dots, in the usual X format. *priority* is an integer between 0 and 100, or one of the symbolic names `'widgetDefault'` (20), `'startupFile'` (40), `'userDefault'` (60), or `'interactive'` (80); it defaults to `interactive`.

</div>

<div class="method">

option_clear()

Clear the Tk option database. Default options from the `!RESOURCE_MANAGER` property or the `.Xdefaults` file are reloaded automatically the next time an option is added to or removed from the database.

</div>

<div class="method">

option_get(name, className)

Return the value of the option matching this widget under *name* and *className* from the Tk option database, or an empty string if there is no matching entry. When several entries match, the one with the highest priority is returned, and among entries of equal priority the most recently added one.

</div>

<div class="method">

option_readfile(fileName, priority=None)

Read the file named *fileName*, which should have the standard format for an X resource database such as `.Xdefaults`, and add all the options it specifies to the Tk option database. *priority* is interpreted as for `option_add` and defaults to `interactive`.

</div>

<div class="method">

bell(displayof=0)

Ring the bell on the display for this widget, using the display's current bell-related settings, and reset the screen saver for the screen. If *displayof* is given as a widget, the bell is rung on that widget's display instead.

</div>

<div class="method">

tk_setPalette(background, /) tk_setPalette(*args,*\*kw)

Set a new color scheme for all Tk widget elements. Existing widgets are updated and the option database is changed so that future widgets use the new colors. A single color argument is taken as the normal background color, from which a complete palette is computed. Alternatively, the arguments may be given as keyword *name*/\*value\* pairs naming individual options in the option database. The recognized option names are `activeBackground`, `activeForeground`, `background`, `disabledForeground`, `foreground`, `highlightBackground`, `highlightColor`, `insertBackground`, `selectColor`, `selectBackground`, `selectForeground`, and `troughColor`; reasonable defaults are computed for any that are not specified.

</div>

<div class="method">

tk_bisque()

Restore the application's colors to the light brown (bisque) color scheme used in Tk 3.6 and earlier versions. Provided for backward compatibility.

</div>

<div class="method">

tk_strictMotif(boolean=None)

Query or set whether Tk's look and feel should strictly adhere to Motif. A true *boolean* value enables strict Motif compliance (for example, no color change when the mouse passes over a slider). Return the resulting setting.

</div>

<div class="method">

tk_appname(name=None)

Query or set the name used to communicate with this application through the `send` command. With no argument, return the current name; otherwise change it to *name* and return the actual name set, which may have a suffix appended to keep it unique among the applications on the display.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

tk_useinputmethods(boolean=None, \*, displayof=0)

Query or set whether Tk uses the X Input Methods (XIM) for filtering events, and return the resulting state. This is significant only on X11; if XIM support is not available it always returns `False`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

tk_caret(\*, x=None, y=None, height=None)

Set or query the caret location for the widget's display. The caret is the per-display insertion position used for global focus indication (for accessibility) and for placing the input method (XIM or IME) window. With no argument, return the current location as a dictionary with the keys `'x'`, `'y'` and `'height'`; otherwise update the given coordinates.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

tk_scaling(number=None, \*, displayof=0)

Query or set the scaling factor used by Tk to convert between physical units (such as points, inches or millimeters) and pixels, expressed as the number of pixels per point (where a point is 1/72 inch). With no argument, return the current factor; otherwise set it to the floating-point *number*.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

tk_inactive(reset=False, \*, displayof=0)

Return the number of milliseconds since the last time the user interacted with the system, or `-1` if the windowing system does not support this. If *reset* is true, reset the inactivity timer to zero instead and return `None`.

<div class="versionadded">

3.16

</div>

</div>

The methods with the `busy_` prefix manage the busy state of a window, which shows a busy cursor and ignores user input.

<div class="method" data-no-typesetting="">

busy(\*\*kw)

</div>

<div class="method" data-no-typesetting="">

busy_hold(\*\*kw)

</div>

<div class="method" data-no-typesetting="">

tk_busy(\*\*kw)

</div>

<div class="method">

tk_busy_hold(\*\*kw)

Make this widget appear busy. A transparent window is placed in front of the widget, so that it and all of its descendants in the widget hierarchy are blocked from pointer events and display a busy cursor. Normally `update` should be called immediately afterwards to ensure that the hold operation is in effect before the application starts its processing.

The only supported configuration option is *cursor*, the cursor to be displayed while the widget is busy; it may have any of the values accepted by `!configure`.

`busy_hold`, `busy` and `tk_busy` are aliases of `!tk_busy_hold`.

<div class="versionadded">

3.13

</div>

</div>

<div class="method" data-no-typesetting="">

busy_configure(cnf=None, \*\*kw)

</div>

<div class="method" data-no-typesetting="">

busy_config(cnf=None, \*\*kw)

</div>

<div class="method" data-no-typesetting="">

tk_busy_config(cnf=None, \*\*kw)

</div>

<div class="method">

tk_busy_configure(cnf=None, \*\*kw)

Query or modify the configuration options of the busy window. The widget must have been previously made busy by `tk_busy_hold`. With no arguments, return a dictionary describing all of the available options; if *cnf* is the name of an option, return a tuple describing that one option. Otherwise set the given options to the given values. Options may have any of the values accepted by `tk_busy_hold`.

The option database is referenced through the widget name or class. For example, if a `Frame` widget named `frame` is to be made busy, the busy cursor can be specified for it by either of the calls:

    w.option_add('*frame.busyCursor', 'gumby')
    w.option_add('*Frame.BusyCursor', 'gumby')

`busy_configure`, `busy_config` and `tk_busy_config` are aliases of `!tk_busy_configure`.

<div class="versionadded">

3.13

</div>

</div>

<div class="method" data-no-typesetting="">

busy_cget(option)

</div>

<div class="method">

tk_busy_cget(option)

Return the current value of the busy configuration *option*. The widget must have been previously made busy by `tk_busy_hold`, and *option* may have any of the values accepted by that method.

`busy_cget` is an alias of `!tk_busy_cget`.

<div class="versionadded">

3.13

</div>

</div>

<div class="method" data-no-typesetting="">

busy_forget()

</div>

<div class="method">

tk_busy_forget()

Make this widget no longer busy, releasing the resources (including the transparent window) allocated when it was made busy. User events will again be received by the widget. These resources are also released when the widget is destroyed.

`busy_forget` is an alias of `!tk_busy_forget`.

<div class="versionadded">

3.13

</div>

</div>

<div class="method" data-no-typesetting="">

busy_status()

</div>

<div class="method">

tk_busy_status()

Return `True` if the widget is currently busy, `False` otherwise.

`busy_status` is an alias of `!tk_busy_status`.

<div class="versionadded">

3.13

</div>

</div>

<div class="method" data-no-typesetting="">

busy_current(pattern=None)

</div>

<div class="method">

tk_busy_current(pattern=None)

Return a list of widgets that are currently busy. If *pattern* is given, only busy widgets whose path names match the pattern are returned.

`busy_current` is an alias of `!tk_busy_current`.

<div class="versionadded">

3.13

</div>

</div>

The methods with the `winfo_` prefix retrieve information about windows managed by Tk.

<div class="method">

winfo_atom(name, displayof=0)

Return the integer identifier for the atom whose name is *name*, creating a new atom if none exists. If *displayof* is given, the atom is looked up on the display of that window; otherwise it is looked up on the display of the application's main window.

</div>

<div class="method">

winfo_atomname(id, displayof=0)

Return the textual name for the atom whose integer identifier is *id*. This is the inverse of `winfo_atom`. If *displayof* is given, the identifier is looked up on the display of that window; otherwise it is looked up on the display of the application's main window.

</div>

<div class="method">

winfo_cells()

Return the number of cells in the colormap for the widget.

</div>

<div class="method">

winfo_children()

Return a list containing the widgets that are children of the widget, in stacking order from lowest to highest. Toplevel windows are returned as children of their logical parents.

</div>

<div class="method">

winfo_class()

Return the class name of the widget.

</div>

<div class="method">

winfo_colormapfull()

Return `True` if the colormap for the widget is known to be full, `False` otherwise.

</div>

<div class="method">

winfo_containing(rootX, rootY, displayof=0)

Return the widget containing the point given by *rootX* and *rootY*, or `None` if no window in this application contains the point. The coordinates are in screen units in the coordinate system of the root window. If *displayof* is given, the coordinates refer to the screen containing that window; otherwise they refer to the screen of the application's main window.

</div>

<div class="method">

winfo_depth()

Return the depth of the widget, that is, the number of bits per pixel.

</div>

<div class="method">

winfo_exists()

Return `True` if the widget exists, `False` otherwise.

</div>

<div class="method">

winfo_fpixels(number)

Return a floating-point value giving the number of pixels in the widget corresponding to the screen distance *number* (for example, `"2.0c"` or `"1i"`). The result may be fractional; for a rounded integer value use `winfo_pixels`.

</div>

<div class="method">

winfo_geometry()

Return the geometry of the widget, in the form `widthxheight+x+y`. All dimensions are in pixels. The position is relative to the parent widget; use `winfo_rootx` and `winfo_rooty` for the position on the screen. An offset can be negative; see `~Wm.geometry`.

</div>

<div class="method">

winfo_height()

Return the height of the widget in pixels. When a window is first created its height is 1 pixel; it is eventually changed by a geometry manager. See also `winfo_reqheight`.

</div>

<div class="method">

winfo_id()

Return a low-level platform-specific identifier for the widget. On Unix this is the X window identifier, and on Windows it is the window handle.

</div>

<div class="method">

winfo_interps(displayof=0)

Return a tuple of the names of all Tcl interpreters currently registered for a particular display. If *displayof* is given, the return value refers to the display of that window; otherwise it refers to the display of the application's main window.

</div>

<div class="method">

winfo_isdark()

On macOS and Windows, return `True` if the widget is in "dark mode", and `False` otherwise. Always return `False` on X11.

<div class="versionadded">

3.16

Requires Tk 9.1 or newer.

</div>

</div>

<div class="method">

winfo_ismapped()

Return `True` if the widget is currently mapped, `False` otherwise.

</div>

<div class="method">

winfo_manager()

Return the name of the geometry manager currently responsible for the widget, or an empty string if it is not managed by any geometry manager.

</div>

<div class="method">

winfo_name()

Return the widget's name within its parent, as opposed to its full path name.

</div>

<div class="method">

winfo_parent()

Return the path name of the widget's parent, or an empty string if the widget is the main window of the application.

</div>

<div class="method">

winfo_pathname(id, displayof=0)

Return the path name of the window whose identifier is *id*. If *displayof* is given, the identifier is looked up on the display of that window; otherwise it is looked up on the display of the application's main window.

</div>

<div class="method">

winfo_pixels(number)

Return the number of pixels in the widget corresponding to the screen distance *number* (for example, `"2.0c"` or `"1i"`). The result is rounded to the nearest integer; for a fractional result use `winfo_fpixels`.

</div>

<div class="method">

winfo_pointerx()

Return the pointer's *x* coordinate, in pixels, relative to the screen's root window (or virtual root, if one is in use). Return `-1` if the pointer is not on the same screen as the widget.

</div>

<div class="method">

winfo_pointerxy()

Return the pointer's coordinates as an `(x, y)` tuple, in pixels, relative to the screen's root window (or virtual root, if one is in use). Both coordinates are `-1` if the pointer is not on the same screen as the widget.

</div>

<div class="method">

winfo_pointery()

Return the pointer's *y* coordinate, in pixels, relative to the screen's root window (or virtual root, if one is in use). Return `-1` if the pointer is not on the same screen as the widget.

</div>

<div class="method">

winfo_reqheight()

Return the widget's requested height in pixels. This is the value used by the widget's geometry manager to compute its geometry.

</div>

<div class="method">

winfo_reqwidth()

Return the widget's requested width in pixels. This is the value used by the widget's geometry manager to compute its geometry.

</div>

<div class="method">

winfo_rgb(color)

Return an `(r, g, b)` tuple of the red, green, and blue intensities, in the range 0 to 65535, that correspond to *color* in the widget. *color* may be specified in any of the forms acceptable for a color option.

</div>

<div class="method">

winfo_rootx()

Return the *x* coordinate, in the root window of the screen, of the upper-left corner of the widget's border (or of the widget itself if it has no border).

</div>

<div class="method">

winfo_rooty()

Return the *y* coordinate, in the root window of the screen, of the upper-left corner of the widget's border (or of the widget itself if it has no border).

</div>

<div class="method">

winfo_screen()

Return the name of the screen associated with the widget, in the form `displayName.screenIndex`.

</div>

<div class="method">

winfo_screencells()

Return the number of cells in the default colormap for the widget's screen.

</div>

<div class="method">

winfo_screendepth()

Return the depth of the root window of the widget's screen, that is, the number of bits per pixel.

</div>

<div class="method">

winfo_screenheight()

Return the height of the widget's screen in pixels.

</div>

<div class="method">

winfo_screenmmheight()

Return the height of the widget's screen in millimeters.

</div>

<div class="method">

winfo_screenmmwidth()

Return the width of the widget's screen in millimeters.

</div>

<div class="method">

winfo_screenvisual()

Return the default visual class for the widget's screen, one of `"directcolor"`, `"grayscale"`, `"pseudocolor"`, `"staticcolor"`, `"staticgray"`, or `"truecolor"`.

</div>

<div class="method">

winfo_screenwidth()

Return the width of the widget's screen in pixels.

</div>

<div class="method">

winfo_server()

Return a string containing information about the server for the widget's display. The exact format of this string may vary from platform to platform.

</div>

<div class="method">

winfo_toplevel()

Return the top-of-hierarchy window containing the widget. In standard Tk this is always a `Toplevel` widget.

</div>

<div class="method">

winfo_viewable()

Return `True` if the widget and all of its ancestors up through the nearest toplevel window are mapped, `False` otherwise.

</div>

<div class="method">

winfo_visual()

Return the visual class for the widget, one of `"directcolor"`, `"grayscale"`, `"pseudocolor"`, `"staticcolor"`, `"staticgray"`, or `"truecolor"`.

</div>

<div class="method">

winfo_visualid()

Return the X identifier for the visual for the widget.

</div>

<div class="method">

winfo_visualsavailable(includeids=False)

Return a list describing the visuals available for the widget's screen. Each item consists of a visual class (see `winfo_visual`) followed by an integer depth. If *includeids* is true, the X identifier for the visual is also included.

</div>

<div class="method">

winfo_vrootheight()

Return the height of the virtual root window associated with the widget if there is one; otherwise return the height of the widget's screen.

</div>

<div class="method">

winfo_vrootwidth()

Return the width of the virtual root window associated with the widget if there is one; otherwise return the width of the widget's screen.

</div>

<div class="method">

winfo_vrootx()

Return the *x* offset of the virtual root window associated with the widget, relative to the root window of its screen. This is normally zero or negative, and is `0` if there is no virtual root window.

</div>

<div class="method">

winfo_vrooty()

Return the *y* offset of the virtual root window associated with the widget, relative to the root window of its screen. This is normally zero or negative, and is `0` if there is no virtual root window.

</div>

<div class="method">

winfo_width()

Return the width of the widget in pixels. When a window is first created its width is 1 pixel; it is eventually changed by a geometry manager. See also `winfo_reqwidth`.

</div>

<div class="method">

winfo_x()

Return the *x* coordinate, in the widget's parent, of the upper-left corner of the widget's border (or of the widget itself if it has no border).

</div>

<div class="method">

winfo_y()

Return the *y* coordinate, in the widget's parent, of the upper-left corner of the widget's border (or of the widget itself if it has no border).

</div>

<div class="method">

info_patchlevel()

Return the Tcl/Tk patch level as a named tuple with the same five fields as `sys.version_info`: *major*, *minor*, *micro*, *releaselevel* and *serial*. *releaselevel* is `'alpha'`, `'beta'` or `'final'`. Converting it to a string gives the version in the usual Tcl/Tk notation, for example `'9.0.3'` for a final release or `'9.1b2'` for a pre-release.

<div class="versionadded">

3.11

</div>

</div>

</div>

<div class="Wm()">

The `!Wm` mixin provides access to the window manager, allowing an application to control such things as the title, geometry and icon of a top-level window, the way it is resized, and how it responds to window manager protocols. It is mixed into `Tk` and `Toplevel`, so its methods are available on every top-level window. Each method has two equivalent spellings: a short name and a `wm_`-prefixed name (for example, `title` and `wm_title`). See also `tkinter-window-manager`.

<div class="method" data-no-typesetting="">

wm_aspect(minNumer=None, minDenom=None, maxNumer=None, maxDenom=None)

</div>

<div class="method">

aspect(minNumer=None, minDenom=None, maxNumer=None, maxDenom=None)

Constrain the aspect ratio (the ratio of width to height) of the window. If all four arguments are given, the window manager keeps the ratio between `minNumer/minDenom` and `maxNumer/maxDenom`; passing empty strings removes any existing restriction. With no arguments, return a tuple of the four current values, or `None` if no aspect restriction is in effect. `wm_aspect` is an alias of `!aspect`.

</div>

<div class="method" data-no-typesetting="">

wm_attributes(*args, return_python_dict=False,*\*kwargs)

</div>

<div class="method">

attributes(*args, return_python_dict=False,*\*kwargs)

Query or set platform-specific attributes of the window. With no arguments, return the platform-specific flags and their values; pass *return_python_dict* as true to get them as a dictionary. A single option name such as `'alpha'` returns the value of that option, and options are set using keyword arguments (`alpha=0.5`).

The available attributes differ by platform. All platforms support:

*alpha*  
The window's opacity, from `0.0` (fully transparent) to `1.0` (opaque). Where transparency is unsupported the value stays at `1.0`.

*appearance*  
Whether the window is rendered in dark mode on Windows and macOS: `'auto'`, `'light'` or `'dark'` (this has no effect on X11).

*fullscreen*  
Whether the window takes up the entire screen and has no borders.

*topmost*  
Whether the window is displayed above all other windows.

Windows additionally supports:

*disabled*  
Whether the window is in a disabled state.

*toolwindow*  
Whether the window uses the tool window style.

*transparentcolor*  
The color that is made fully transparent, or an empty string for none.

macOS additionally supports:

*class*  
Whether the underlying Aqua window is an `nswindow` or an `nspanel`; this can only be set before the window is created.

*modified*  
The modification state shown by the window's close button and proxy icon.

*notify*  
Whether the application's dock icon bounces to request attention.

*stylemask*  
The style mask of the underlying Aqua window, given as a list of bit names such as `titled` or `resizable`.

*tabbingid*  
The identifier of the tab group that the window belongs to.

*tabbingmode*  
Whether the window may be opened as a tab: `'auto'`, `'preferred'` or `'disallowed'`.

*titlepath*  
The path of the file represented by the window's proxy icon.

*transparent*  
Whether the content area is transparent and the window shadow is turned off.

X11 additionally supports:

*type*  
The window type, or a list of types in order of preference, that the window manager should use to interpret the window, such as `'dialog'` or `'splash'`.

*zoomed*  
Whether the window is maximized.

> [!NOTE]
> Tk 8.6 added the *type* attribute, and Tk 9.0 added the *appearance*, *class*, *stylemask*, *tabbingid* and *tabbingmode* attributes.

On X11 changes are applied asynchronously, so a queried value may not yet reflect the most recent request. `wm_attributes` is an alias of `!attributes`.

<div class="versionchanged">

3.13 A single attribute may now be queried by name without the leading `-`, and attributes may be set using keyword arguments. The *return_python_dict* parameter was added.

</div>

<div class="deprecated">

3.13 Setting an attribute by passing the option name (with a leading `-`) and its value as two positional arguments, as in `w.attributes('-alpha', 0.5)`, is deprecated; use keyword arguments instead.

</div>

</div>

<div class="method" data-no-typesetting="">

wm_client(name=None)

</div>

<div class="method">

client(name=None)

Store *name*, which should be the name of the host on which the application is running, in the window's `WM_CLIENT_MACHINE` property for use by the window or session manager. An empty string deletes the property. With no argument, return the last name set, or an empty string. `wm_client` is an alias of `!client`.

</div>

<div class="method" data-no-typesetting="">

wm_colormapwindows(\*wlist)

</div>

<div class="method">

colormapwindows(\*wlist)

Manipulate the `WM_COLORMAP_WINDOWS` property, which tells the window manager about windows that have private colormaps. If *wlist* is given, overwrite the property with those windows (their order is a priority order for installing colormaps). With no arguments, return the list of windows currently named in the property. `wm_colormapwindows` is an alias of `!colormapwindows`.

</div>

<div class="method" data-no-typesetting="">

wm_command(value=None)

</div>

<div class="method">

command(value=None)

Store *value* in the window's `WM_COMMAND` property for use by the window or session manager; it should be a list giving the words of the command used to invoke the application. An empty string deletes the property. With no argument, return the last value set, or an empty string. `wm_command` is an alias of `!command`.

</div>

<div class="method" data-no-typesetting="">

wm_deiconify()

</div>

<div class="method">

deiconify()

Display the window in normal (non-iconified) form by mapping it. If the window has never been mapped, this ensures it appears de-iconified when it is first mapped. On Windows the window is also raised and given the focus. `wm_deiconify` is an alias of `!deiconify`.

</div>

<div class="method" data-no-typesetting="">

wm_focusmodel(model=None)

</div>

<div class="method">

focusmodel(model=None)

Set or query the focus model for the window. *model* is either `'active'` (the window claims the input focus for itself or its descendants, even when the focus is in another application) or `'passive'` (the window relies on the window manager to give it the focus). With no argument, return the current model. The default is `'passive'`, which is what the `!focus` command assumes. `wm_focusmodel` is an alias of `!focusmodel`.

</div>

<div class="method" data-no-typesetting="">

wm_forget(window)

</div>

<div class="method">

forget(window)

Unmap *window* from the screen so that it is no longer managed by the window manager. A `Toplevel` is then treated like a `Frame`, although its `-menu` configuration is remembered and the menu reappears if the widget is managed again. `wm_forget` is an alias of `!forget`.

Not to be confused with `Pack.forget`.

<div class="versionadded">

3.3

</div>

</div>

<div class="method" data-no-typesetting="">

wm_frame()

</div>

<div class="method">

frame()

Return the platform-specific window identifier for the outermost decorative frame containing the window, if the window manager has reparented it into such a frame; otherwise return the identifier of the window itself. `wm_frame` is an alias of `!frame`.

</div>

<div class="method" data-no-typesetting="">

wm_geometry(newGeometry=None)

</div>

<div class="method">

geometry(newGeometry=None)

Set or query the geometry of the window. *newGeometry* has the form `=widthxheight+x+y`, where any of `=`, `widthxheight` and the `+x+y` position may be omitted. *width* and *height* are in pixels (or grid units for a gridded window); a position preceded by `+` is measured from the left or top edge of the screen and one preceded by `-` from the right or bottom edge. An offset can be negative, as in `'200x100+-9+-8'`, when the window edge is positioned beyond the corresponding screen edge. An empty string cancels any user-specified geometry, letting the window revert to its natural size. With no argument, return the current geometry as a string of the form `'200x200+10+10'`. `wm_geometry` is an alias of `!geometry`.

</div>

<div class="method" data-no-typesetting="">

wm_grid(baseWidth=None, baseHeight=None, widthInc=None, heightInc=None)

</div>

<div class="method">

grid(baseWidth=None, baseHeight=None, widthInc=None, heightInc=None)

Manage the window as a gridded window and define the relationship between grid units and pixels. *baseWidth* and *baseHeight* are the numbers of grid units for the window's internally requested size, and *widthInc* and *heightInc* are the pixel sizes of a horizontal and vertical grid unit. Empty strings turn off gridded management. With no arguments, return a tuple of the four current values, or `None` if the window is not gridded. `wm_grid` is an alias of `!grid`.

Not to be confused with the grid geometry manager `Grid.grid`.

</div>

<div class="method" data-no-typesetting="">

wm_group(pathName=None)

</div>

<div class="method">

group(pathName=None)

Set or query the leader of a group of related windows. *pathName* gives the path name of the group leader; the window manager may, for example, unmap all windows in the group when the leader is iconified. An empty string removes the window from any group. With no argument, return the path name of the current group leader, or an empty string. `wm_group` is an alias of `!group`.

</div>

<div class="method" data-no-typesetting="">

wm_iconbadge(badge)

</div>

<div class="method">

iconbadge(badge)

Set a badge for the window's icon, intended for display in the Dock (macOS), taskbar (Windows) or app panel (X11). *badge* may be a positive integer (for example a count of unread messages) or an exclamation point to denote that attention is needed; an empty string removes the badge. On X11 the variable `::tk::icons::base_icon(window)` must be set to the window's icon image for the badge to appear. `wm_iconbadge` is an alias of `!iconbadge`.

<div class="versionadded">

3.16

Requires Tk 9.0 or newer.

</div>

</div>

<div class="method" data-no-typesetting="">

wm_iconbitmap(bitmap=None, default=None)

</div>

<div class="method">

iconbitmap(bitmap=None, default=None)

Set or query the bitmap used by the window manager for the window's icon. *bitmap* names a bitmap in one of the standard forms accepted by Tk; an empty string cancels the current icon bitmap. With no argument, return the name of the current icon bitmap, or an empty string. On Windows the *default* argument names an icon (for example an `.ico` file) applied to all top-level windows that have no icon of their own. `wm_iconbitmap` is an alias of `!iconbitmap`.

</div>

<div class="method" data-no-typesetting="">

wm_iconify()

</div>

<div class="method">

iconify()

Iconify the window. If the window has not yet been mapped for the first time, arrange for it to appear in the iconified state when it is eventually mapped. `wm_iconify` is an alias of `!iconify`.

</div>

<div class="method" data-no-typesetting="">

wm_iconmask(bitmap=None)

</div>

<div class="method">

iconmask(bitmap=None)

Set or query the bitmap used as a mask for the icon (see `iconbitmap`). Where the mask is zero no icon is displayed; where it is one, the corresponding bits of the icon bitmap are shown. An empty string cancels the current mask. With no argument, return the name of the current icon mask, or an empty string. `wm_iconmask` is an alias of `!iconmask`.

</div>

<div class="method" data-no-typesetting="">

wm_iconname(newName=None)

</div>

<div class="method">

iconname(newName=None)

Set or query the name displayed by the window manager inside the window's icon. With no argument, return the current icon name, or an empty string if none has been set (in which case the window manager normally displays the window's title). `wm_iconname` is an alias of `!iconname`.

</div>

<div class="method" data-no-typesetting="">

wm_iconphoto(default=False, \*images)

</div>

<div class="method">

iconphoto(default=False, \*images)

Set the titlebar icon for the window from one or more `PhotoImage` objects given in *images*. Several images of different sizes (for example 16x16 and 32x32) may be supplied so that the window manager can choose an appropriate one. The image data is taken as a snapshot at the time of the call; later changes to the images are not reflected. If *default* is true, the icon is also applied to all top-level windows created in the future. On macOS only the first image is used. `wm_iconphoto` is an alias of `!iconphoto`.

<div class="versionadded">

3.3

</div>

</div>

<div class="method" data-no-typesetting="">

wm_iconposition(x=None, y=None)

</div>

<div class="method">

iconposition(x=None, y=None)

Set or query a hint to the window manager about where the window's icon should be positioned. Empty strings cancel an existing hint. With no arguments, return a tuple of the two current values, or `None` if no hint is in effect. `wm_iconposition` is an alias of `!iconposition`.

</div>

<div class="method" data-no-typesetting="">

wm_iconwindow(pathName=None)

</div>

<div class="method">

iconwindow(pathName=None)

Set or query the window used as the icon for the window. When the window is iconified, *pathName* is mapped to serve as its icon and unmapped again when it is de-iconified. An empty string cancels the association. With no argument, return the path name of the current icon window, or an empty string. Not all window managers support icon windows, and the concept is meaningless on non-X11 platforms. `wm_iconwindow` is an alias of `!iconwindow`.

</div>

<div class="method" data-no-typesetting="">

wm_manage(widget)

</div>

<div class="method">

manage(widget)

Make *widget* a stand-alone top-level window, decorated by the window manager with a title bar and so on. Only `Frame`, `LabelFrame` and `Toplevel` widgets may be used (the `tkinter.ttk` versions are **not** accepted); passing any other widget type raises an error. `wm_manage` is an alias of `!manage`.

<div class="versionadded">

3.3

</div>

</div>

<div class="method" data-no-typesetting="">

wm_maxsize(width=None, height=None)

</div>

<div class="method">

maxsize(width=None, height=None)

Set or query the maximum permissible dimensions of the window, in pixels (or grid units for a gridded window). The window manager restricts the window to be no larger than *width* and *height*. With no arguments, return a tuple of the current maximum width and height. The maximum size defaults to the size of the screen. `wm_maxsize` is an alias of `!maxsize`.

</div>

<div class="method" data-no-typesetting="">

wm_minsize(width=None, height=None)

</div>

<div class="method">

minsize(width=None, height=None)

Set or query the minimum permissible dimensions of the window, in pixels (or grid units for a gridded window). The window manager restricts the window to be no smaller than *width* and *height*. With no arguments, return a tuple of the current minimum width and height. The minimum size defaults to one pixel in each dimension. `wm_minsize` is an alias of `!minsize`.

</div>

<div class="method" data-no-typesetting="">

wm_overrideredirect(boolean=None)

</div>

<div class="method">

overrideredirect(boolean=None)

Set or query the override-redirect flag for the window. When this flag is set, the window is ignored by the window manager: it is not reparented into a decorative frame and the user cannot manipulate it through the usual window manager controls. With no argument, return a boolean indicating whether the flag is set, or `None` if it has not been set. The flag is reliably honored only when the window is first mapped or remapped from the withdrawn state. `wm_overrideredirect` is an alias of `!overrideredirect`.

</div>

<div class="method" data-no-typesetting="">

wm_positionfrom(who=None)

</div>

<div class="method">

positionfrom(who=None)

Set or query the source of the window's current position. *who* is either `'program'` or `'user'` and indicates whether the position was requested by the program or by the user; an empty string cancels the current source. With no argument, return the current source, or an empty string if none has been set. Tk automatically sets the source to `'user'` when `geometry` is called, unless it has been set explicitly to `'program'`. `wm_positionfrom` is an alias of `!positionfrom`.

</div>

<div class="method" data-no-typesetting="">

wm_protocol(name=None, func=None)

</div>

<div class="method">

protocol(name=None, func=None)

Register *func* as the handler for the window manager protocol *name*, an atom such as `'WM_DELETE_WINDOW'`, `'WM_SAVE_YOURSELF'` or `'WM_TAKE_FOCUS'`; *func* is then called whenever the window manager sends a message of that protocol. Tk installs a default `WM_DELETE_WINDOW` handler that destroys the window, which this method can replace. If *func* is an empty string, the handler is removed. With only *name*, return the name of its registered handler command, or an empty string if none is set (the default `WM_DELETE_WINDOW` handler is not reported); with no arguments, return a tuple of the protocols that currently have handlers. `wm_protocol` is an alias of `!protocol`.

</div>

<div class="method" data-no-typesetting="">

wm_resizable(width=None, height=None)

</div>

<div class="method">

resizable(width=None, height=None)

Control whether the user may interactively resize the window. *width* and *height* are boolean values that determine whether the window's width and height may be changed. With no arguments, return a tuple of two `0`/`1` values indicating whether each dimension is currently resizable. By default a window is resizable in both dimensions. `wm_resizable` is an alias of `!resizable`.

</div>

<div class="method" data-no-typesetting="">

wm_sizefrom(who=None)

</div>

<div class="method">

sizefrom(who=None)

Set or query the source of the window's current size. *who* is either `'program'` or `'user'` and indicates whether the size was requested by the program or by the user; an empty string cancels the current source. With no argument, return the current source, or an empty string if none has been set. `wm_sizefrom` is an alias of `!sizefrom`.

</div>

<div class="method" data-no-typesetting="">

wm_stackorder(relation=None, window=None)

</div>

<div class="method">

stackorder(relation=None, window=None)

Query the stacking order of top-level windows. With no arguments, return a list of the mapped top-level widgets in stacking order, from lowest to highest, recursively including this window's top-level children. If *relation* is `'isabove'` or `'isbelow'` and *window* is another top-level, return `True` if this window is respectively above or below *window* in the stacking order, and `False` otherwise. `wm_stackorder` is an alias of `!stackorder`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method" data-no-typesetting="">

wm_state(newstate=None)

</div>

<div class="method">

state(newstate=None)

Set or query the state of the window. With no argument, return the current state: one of `'normal'`, `'iconic'`, `'withdrawn'`, `'icon'` or, on Windows and macOS only, `'zoomed'`. `'iconic'` refers to a window that has been iconified, while `'icon'` refers to a window serving as the icon for another window (see `iconwindow`); the `'icon'` state cannot be set. `wm_state` is an alias of `!state`.

Not to be confused with `ttk.Widget.state
<tkinter.ttk.Widget.state>`.

</div>

<div class="method" data-no-typesetting="">

wm_title(string=None)

</div>

<div class="method">

title(string=None)

Set or query the title for the window, which the window manager should display in the window's title bar. With no argument, return the current title. The title defaults to the window's name. `wm_title` is an alias of `!title`.

</div>

<div class="method" data-no-typesetting="">

wm_transient(master=None)

</div>

<div class="method">

transient(master=None)

Mark the window as a transient window (such as a pull-down menu or dialog) working on behalf of *master*, the path name of another top-level window. An empty string clears the transient status. With no argument, return the path name of the current master, or an empty string. A transient window mirrors state changes in its master and may be decorated differently by the window manager; it is an error to make a window a transient of itself. `wm_transient` is an alias of `!transient`.

</div>

<div class="method" data-no-typesetting="">

wm_withdraw()

</div>

<div class="method">

withdraw()

Withdraw the window from the screen, unmapping it and causing the window manager to forget about it. If the window has never been mapped, it is instead mapped in the withdrawn state. It is sometimes necessary to withdraw a window and then re-map it (for example with `deiconify`) to make some window managers notice changes to window attributes. `wm_withdraw` is an alias of `!withdraw`.

</div>

</div>

<div class="Pack()">

Geometry manager that arranges widgets by packing them against the sides of their container. The `!Pack` mix-in is inherited by all widgets (through `Widget`) and provides the methods for managing a widget with the *pack* geometry manager. See also `tkinter-geometry-management`.

> [!NOTE]
> `Pack`, `Place` and `Grid` all define the short method names `!forget`, `!info`, `!slaves`, `!content` and `!propagate`. On a widget the bare names resolve to the *pack* manager's versions, since `Pack` and `Misc` precede `Place` and `Grid` in the method resolution order, whatever manager actually manages the widget; and `!configure`/`!config` configure the widget's options, not its geometry. Use the explicit `pack_*`, `grid_*` and `place_*` methods (and `pack`, `grid`, `place` for geometry configuration) to act on a specific geometry manager.

<div class="method" data-no-typesetting="">

configure(cnf={}, \*\*kw)

</div>

<div class="method" data-no-typesetting="">

config(cnf={}, \*\*kw)

</div>

<div class="method">

pack_configure(cnf={}, **kw) pack(cnf={},**kw)

Pack the widget inside its container, positioning it relative to the siblings already packed there. The supported options are:

*side*  
Which side of the container to pack the widget against: `'top'` (the default), `'bottom'`, `'left'` or `'right'`.

*fill*  
Whether to stretch the widget to fill its parcel: `'none'` (the default), `'x'`, `'y'` or `'both'`.

*expand*  
Whether the widget should expand to consume any extra space in its container (a boolean, default false).

*anchor*  
Where to position the widget in its parcel when the parcel is larger than the widget: an anchor such as `'n'` or `'sw'` (default `'center'`).

*ipadx*, *ipady*  
Internal padding added on the left and right (*ipadx*) or top and bottom (*ipady*) of the widget, as a screen distance (default `0`).

*padx*, *pady*  
External padding left on the left and right (*padx*) or top and bottom (*pady*) of the widget, as a screen distance or a pair of two distances for the two sides (default `0`).

*after*  
Pack the widget after the given widget in the packing order, using the same container.

*before*  
Pack the widget before the given widget in the packing order, using the same container.

*in\_*  
The container in which to pack the widget; it defaults to the parent widget.

`pack`, `configure` and `config` are aliases of `!pack_configure`.

</div>

<div class="method" data-no-typesetting="">

forget()

</div>

<div class="method">

pack_forget()

Unmap the widget and remove it from the packing order, forgetting its packing options. It can be packed again later with `pack_configure`. `forget` is an alias of `!pack_forget`, except on `PanedWindow`, `ttk.Notebook <tkinter.ttk.Notebook>` and `ttk.PanedWindow <tkinter.ttk.PanedWindow>`, which provide their own `!forget` method.

Not to be confused with `Wm.forget`.

</div>

<div class="method" data-no-typesetting="">

info()

</div>

<div class="method">

pack_info()

Return a dictionary of the widget's current packing options. `info` is an alias of `!pack_info`.

</div>

<div class="method" data-no-typesetting="">

propagate() propagate(flag)

</div>

<div class="method">

pack_propagate() pack_propagate(flag)

Same as `Misc.pack_propagate`, treating this widget as a container: enable or disable geometry propagation. `propagate` is an alias of `!pack_propagate`.

</div>

<div class="method" data-no-typesetting="">

slaves()

</div>

<div class="method">

pack_slaves()

Same as `Misc.pack_slaves`: return the list of widgets packed in this widget. `slaves` is an alias of `!pack_slaves`.

</div>

<div class="method" data-no-typesetting="">

content()

</div>

<div class="method">

pack_content()

Same as `Misc.pack_content`. `content` is an alias of `!pack_content`.

<div class="versionadded">

3.15

</div>

</div>

</div>

<div class="Place()">

Geometry manager that places widgets at explicit positions and sizes within their container. The `!Place` mix-in is inherited by all widgets (through `Widget`). See also `tkinter-geometry-management`.

<div class="method" data-no-typesetting="">

configure(cnf={}, \*\*kw)

</div>

<div class="method" data-no-typesetting="">

config(cnf={}, \*\*kw)

</div>

<div class="method">

place_configure(cnf={}, **kw) place(cnf={},**kw)

Place the widget inside its container at an absolute or relative position. The supported options are:

*x*, *y*  
The absolute horizontal and vertical position of the widget's anchor point, as a screen distance (default `0`).

*relx*, *rely*  
The horizontal and vertical position of the widget's anchor point as a fraction of the container's width and height, where `0.0` is the left or top edge and `1.0` is the right or bottom edge. If both the absolute and the relative option are given, their values are summed.

*anchor*  
Which point of the widget is placed at the given position: an anchor such as `'n'` or `'se'` (default `'nw'`).

*width*, *height*  
The absolute width and height of the widget, as a screen distance. By default the widget's requested size is used.

*relwidth*, *relheight*  
The width and height of the widget as a fraction of the container's width and height. If both the absolute and the relative option are given, their values are summed.

*bordermode*  
How the container's border affects placement: `'inside'` (the default) measures the area inside the border, `'outside'` measures the area including the border, and `'ignore'` uses the official X area.

*in\_*  
The container relative to which the widget is placed; it must be the widget's parent or a descendant of the parent, and defaults to the parent.

`place`, `configure` and `config` are aliases of `!place_configure`.

</div>

<div class="method" data-no-typesetting="">

forget()

</div>

<div class="method">

place_forget()

Unmap the widget and remove it from the placement, forgetting its place options.

</div>

<div class="method" data-no-typesetting="">

info()

</div>

<div class="method">

place_info()

Return a dictionary of the widget's current place options.

</div>

<div class="method" data-no-typesetting="">

slaves()

</div>

<div class="method">

place_slaves()

Same as `Misc.place_slaves`: return the list of widgets placed in this widget.

</div>

<div class="method" data-no-typesetting="">

content()

</div>

<div class="method">

place_content()

Same as `Misc.place_content`.

<div class="versionadded">

3.15

</div>

</div>

</div>

<div class="Grid()">

Geometry manager that arranges widgets in a two-dimensional grid of rows and columns within their container. The `!Grid` mix-in is inherited by all widgets (through `Widget`). See also `tkinter-geometry-management`.

<div class="method" data-no-typesetting="">

configure(cnf={}, \*\*kw)

</div>

<div class="method" data-no-typesetting="">

config(cnf={}, \*\*kw)

</div>

<div class="method">

grid_configure(cnf={}, **kw) grid(cnf={},**kw)

Position the widget in a cell of its container's grid.

Not to be confused with `Wm.grid`.

The supported options are:

*row*, *column*  
The row and column of the cell to place the widget in, counting from `0`. *column* defaults to the column after the previous widget placed in the same `!grid_configure` call (or `0`), and *row* defaults to the next empty row.

*rowspan*, *columnspan*  
The number of rows and columns the widget should span (default `1`).

*sticky*  
How to position or stretch the widget when its cell is larger than the widget: a string containing zero or more of the characters `'n'`, `'s'`, `'e'` and `'w'`, naming the cell sides the widget sticks to. Specifying both `'n'` and `'s'` (or `'e'` and `'w'`) stretches the widget to fill the height (or width) of the cell. The default is `''`, which centers the widget at its requested size.

*ipadx*, *ipady*  
Internal padding added on the left and right (*ipadx*) or top and bottom (*ipady*) of the widget, as a screen distance (default `0`).

*padx*, *pady*  
External padding left on the left and right (*padx*) or top and bottom (*pady*) of the widget, as a screen distance or a pair of two distances for the two sides (default `0`).

*in\_*  
The container in whose grid to place the widget; it defaults to the parent widget.

`grid`, `configure` and `config` are aliases of `!grid_configure`.

</div>

<div class="method" data-no-typesetting="">

forget()

</div>

<div class="method">

grid_forget()

Unmap the widget and remove it from the grid, forgetting its grid options.

</div>

<div class="method">

grid_remove()

Unmap the widget and remove it from the grid, but remember its grid options so that it is restored to the same cell if it is gridded again.

</div>

<div class="method" data-no-typesetting="">

info()

</div>

<div class="method">

grid_info()

Return a dictionary of the widget's current grid options.

</div>

<div class="method" data-no-typesetting="">

bbox(column=None, row=None, col2=None, row2=None)

</div>

<div class="method">

grid_bbox(column=None, row=None, col2=None, row2=None)

Same as `Misc.grid_bbox`. `bbox` is an alias of `!grid_bbox`, except on `Canvas`, `Listbox`, `Spinbox`, `Text`, `ttk.Entry <tkinter.ttk.Entry>` and `ttk.Treeview <tkinter.ttk.Treeview>`, which provide their own `!bbox` method.

</div>

<div class="method" data-no-typesetting="">

columnconfigure(index, cnf={}, \*\*kw)

</div>

<div class="method">

grid_columnconfigure(index, cnf={}, \*\*kw)

Same as `Misc.grid_columnconfigure`: query or set the options (such as *weight*, *minsize*, *pad* and *uniform*) of a grid column. `columnconfigure` is an alias of `!grid_columnconfigure`.

</div>

<div class="method" data-no-typesetting="">

rowconfigure(index, cnf={}, \*\*kw)

</div>

<div class="method">

grid_rowconfigure(index, cnf={}, \*\*kw)

Same as `Misc.grid_rowconfigure`: query or set the options of a grid row. `rowconfigure` is an alias of `!grid_rowconfigure`.

</div>

<div class="method" data-no-typesetting="">

location(x, y)

</div>

<div class="method">

grid_location(x, y)

Same as `Misc.grid_location`: return the `(column, row)` of the cell that covers the pixel at *x*, *y*. `location` is an alias of `!grid_location`.

</div>

<div class="method" data-no-typesetting="">

size()

</div>

<div class="method">

grid_size()

Same as `Misc.grid_size`: return a `(columns, rows)` tuple giving the size of the grid. `size` is an alias of `!grid_size`, except on `Listbox` and `ttk.Treeview <tkinter.ttk.Treeview>`, which provide their own `!size` method.

</div>

<div class="method" data-no-typesetting="">

propagate() propagate(flag)

</div>

<div class="method">

grid_propagate() grid_propagate(flag)

Same as `Misc.grid_propagate`.

</div>

<div class="method" data-no-typesetting="">

slaves(row=None, column=None)

</div>

<div class="method">

grid_slaves(row=None, column=None)

Same as `Misc.grid_slaves`: return the widgets managed in the grid, optionally restricted to a *row* and/or *column*.

</div>

<div class="method" data-no-typesetting="">

content(row=None, column=None)

</div>

<div class="method">

grid_content(row=None, column=None)

Same as `Misc.grid_content`.

<div class="versionadded">

3.15

</div>

</div>

</div>

<div class="XView()">

Mix-in providing the horizontal-scrolling interface shared by widgets such as `Entry`, `Canvas`, `Listbox`, `Text` and `Spinbox`. A widget's `xview` method is registered as the *command* of a horizontal `Scrollbar`.

<div class="method">

xview(\*args)

Query or change the horizontal position of the view. With no arguments, return a tuple `(first, last)` of two fractions between 0 and 1 giving the portion of the document that is currently visible. Otherwise the arguments are passed to the Tk `xview` widget command and are usually generated by a scrollbar; `xview_moveto` and `xview_scroll` provide a more convenient interface.

</div>

<div class="method">

xview_moveto(fraction)

Adjust the view so that *fraction* of the total width of the document is off-screen to the left. *fraction* is a number between 0 and 1.

</div>

<div class="method">

xview_scroll(number, what)

Shift the view left or right by *number* units. *what* is either `'units'` or `'pages'`; a negative *number* scrolls left and a positive one scrolls right. The `Text` widget accepts also `'pixels'`, and then *number* can be a screen distance, such as `'1i'`.

</div>

</div>

<div class="YView()">

Mix-in providing the vertical-scrolling interface shared by widgets such as `Canvas`, `Listbox` and `Text`. A widget's `yview` method is registered as the *command* of a vertical `Scrollbar`.

<div class="method">

yview(\*args)

Query or change the vertical position of the view. With no arguments, return a tuple `(first, last)` of two fractions between 0 and 1 giving the portion of the document that is currently visible. Otherwise the arguments are passed to the Tk `yview` widget command, usually generated by a scrollbar; `yview_moveto` and `yview_scroll` provide a more convenient interface.

</div>

<div class="method">

yview_moveto(fraction)

Adjust the view so that *fraction* of the total height of the document is off-screen above the top. *fraction* is a number between 0 and 1.

</div>

<div class="method">

yview_scroll(number, what)

Shift the view up or down by *number* units. *what* is either `'units'` or `'pages'`; a negative *number* scrolls up and a positive one scrolls down. The `Text` widget accepts also `'pixels'`, and then *number* can be a screen distance, such as `'1i'`.

</div>

</div>

<div class="BaseWidget(master, widgetName, cnf={}, kw={}, extra=())">

Internal base class for all widgets. It inherits from `Misc` and adds the machinery that creates the underlying Tk widget; application code normally uses `Widget` or a concrete widget class rather than instantiating `!BaseWidget` directly.

<div class="method">

destroy()

Destroy this widget and all of its children, removing the corresponding Tk widgets and deleting the associated Tcl commands.

</div>

</div>

<div class="Widget(master, widgetName, cnf={}, kw={}, extra=())">

Internal base class for the standard widgets. It combines `BaseWidget` with the geometry-manager mix-ins `Pack`, `Place` and `Grid`, so that every widget can be managed by any of the three geometry managers. The concrete widget classes (`Button`, `Label`, and so on) derive from `!Widget`.

</div>

### Toplevel widgets

<div class="Tk(screenName=None, baseName=None, className='Tk', useTk=True, sync=False, use=None)">

Construct a toplevel Tk widget, which is usually the main window of an application, and initialize a Tcl interpreter for this widget. Each instance has its own associated Tcl interpreter. Inherits from `Misc` and `Wm`.

To create a Tcl interpreter without initializing the Tk subsystem, use the `Tcl` factory function instead.

The `Tk` class is typically instantiated using all default values. However, the following keyword arguments are currently recognized:

*screenName*  
When given (as a string), sets the `DISPLAY` environment variable. (X11 only)

*baseName*  
Name of the profile file. By default, *baseName* is derived from the program name (`sys.argv[0]`).

*className*  
Name of the widget class. Used as a profile file and also as the name with which Tcl is invoked (*argv0* in *interp*).

*useTk*  
If `True`, initialize the Tk subsystem. The `tkinter.Tcl() <Tcl>` function sets this to `False`.

*sync*  
If `True`, execute all X server commands synchronously, so that errors are reported immediately. Can be used for debugging. (X11 only)

*use*  
Specifies the *id* of the window in which to embed the application, instead of it being created as an independent toplevel window. *id* must be specified in the same way as the value for the -use option for toplevel widgets (that is, it has a form like that returned by `~Misc.winfo_id`).

Note that on some platforms this will only work correctly if *id* refers to a Tk frame or toplevel that has its -container option enabled.

`Tk` reads and interprets profile files, named `.{className}.tcl` and `.{baseName}.tcl`, into the Tcl interpreter and calls `exec` on the contents of `.{className}.py` and `.{baseName}.py`. The path for the profile files is the `HOME` environment variable or, if that isn't defined, then `os.curdir`.

> [!NOTE]
> On Windows, creating a Tcl interpreter (by instantiating `Tk` or calling `Tcl`) sets the `HOME` environment variable for the process, if it is not already set, to `%HOMEDRIVE%%HOMEPATH%` (or `USERPROFILE`, or `c:\`). This is done by Tcl and can affect other code that reads `HOME`.

<div class="attribute">

tk

The Tk application object created by instantiating `Tk`. This provides access to the Tcl interpreter. Each widget that is attached the same instance of `Tk` has the same value for its `tk` attribute.

</div>

<div class="attribute">

master

The widget object that contains this widget. For `Tk`, the `!master` is `None` because it is the main window. The terms *master* and *parent* are similar and sometimes used interchangeably as argument names; however, calling `~Misc.winfo_parent` returns a string of the widget name whereas `!master` returns the object. *parent*/\*child\* reflects the tree-like relationship while *master* (or *container*)/\*content\* reflects the container structure.

</div>

<div class="attribute">

children

The immediate descendants of this widget as a `dict` with the child widget names as the keys and the child instance objects as the values.

</div>

<div class="method">

destroy()

Destroy this and all descendant widgets and, for the main window, end the connection to the underlying Tcl interpreter.

</div>

<div class="method">

loadtk()

Finish loading and initializing the Tk subsystem. This is needed only when the interpreter was created without Tk (for example through `Tcl`); it is called automatically when *useTk* is true.

</div>

<div class="method">

readprofile(baseName, className)

Read and source the user's profile files `.{className}.tcl` and `.{baseName}.tcl` into the Tcl interpreter, and execute the corresponding `.{className}.py` and `.{baseName}.py` files. This is called during initialization; see the description of the constructor above.

</div>

<div class="method">

report_callback_exception(exc, val, tb)

Report a callback exception. This is called when an exception propagates out of a Tkinter callback; *exc*, *val* and *tb* are the exception type, value and traceback as returned by `sys.exc_info`. The default implementation prints a traceback to `sys.stderr`. It can be overridden to customize error handling, for example to display the traceback in a dialog.

</div>

</div>

<div class="Toplevel(master=None, cnf={}, **kw)">

A `!Toplevel` widget is a top-level window, similar to a `Frame` except that its X parent is the root window of a screen rather than its logical parent. Its primary purpose is to serve as a container for dialog boxes and other collections of widgets; its only visible features are its background and an optional 3-D border. Notable options include *menu*, which installs a `Menu` as the window's menubar. Inherits from `BaseWidget` and `Wm`, so a toplevel is managed by the window manager. Refer to the Tk `toplevel` manual page for the full list of options.

</div>

### Widget classes

<div class="Button(master=None, cnf={}, **kw)">

A `!Button` widget displays a textual string, bitmap or image and invokes a command when the user presses it (by clicking mouse button 1 over the button or, when the button has focus, by pressing the space key). Inherits from `Widget`. In addition to the standard widget options, a button accepts the options documented in the Tk `button` manual page, such as *command* (the callback invoked when the button is pressed), *textvariable*, *state* and *default*.

<div class="method">

invoke()

Invoke the command associated with the button, if there is one, and return its result, or an empty string if no command is associated with the button. This is ignored if the button's state is `disabled`.

</div>

<div class="method">

flash()

Flash the button by redisplaying it several times, alternating between the active and normal colors. At the end of the flash the button is left in the same normal or active state as when the method was called. This is ignored if the button's state is `disabled`.

</div>

</div>

<div class="Canvas(master=None, cnf={}, **kw)">

A `!Canvas` widget implements structured graphics. It displays any number of *items*, such as arcs, lines, ovals, polygons, rectangles, text, bitmaps, images and embedded windows, which may be drawn, moved, re-colored and bound to events. Inherits from `Widget`, `XView` and `YView`, so the view can be scrolled horizontally and vertically with `~XView.xview` and `~YView.yview`. Refer to the Tk `canvas` manual page for the full list of widget and item options.

Each item has a unique integer *id*, assigned when it is created, and zero or more string *tags*. A tag is an arbitrary string that does not have the form of an integer; the same tag may be shared by many items, which makes tags convenient for grouping items. The special tag `'all'` matches every item in the canvas, and `'current'` matches the topmost item under the mouse pointer. Most methods take a *tagOrId* argument that may be an integer id naming a single item, or a tag naming zero or more items; as described in the Tk `canvas` manual page, a tag may also be a logical expression of tags combined with the operators `&&`, `||`, `^`, `!` and parentheses. When a method that operates on a single item is given a *tagOrId* matching several items, it normally uses the lowest matching item in the display list.

The items are kept in a *display list* that determines drawing order: items later in the list are drawn on top of earlier ones. A newly created item is placed at the top of the list; the order can be changed with `tag_raise` and `tag_lower`.

<div class="method">

tk_print()

Print the contents of the canvas using the native print dialog. Requires Tk 8.7/9.0 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

create_arc(*args,kw) create_bitmap(*args, **kw) create_image(\*args,**kw) create_line(*args,kw) create_oval(*args, **kw) create_polygon(\*args,**kw) create_rectangle(*args,kw) create_text(*args, **kw) create_window(\*args,**kw)

Create a new item of the corresponding type and return its integer id. Each method is called as `create_TYPE(coord..., **options)`: the leading positional arguments give the coordinates that define the item (as separate numbers, as a single sequence of numbers, or as coordinate pairs), and the keyword arguments set item-specific options. Coordinates and screen distances may be given as numbers (interpreted as pixels) or as strings with a unit suffix (`'m'`, `'c'`, `'i'` or `'p'` for millimetres, centimetres, inches or printer's points), but are always stored and returned in pixels.

The item types are: `arc` (an arc-shaped region that is a section of an oval, defined by two diagonally opposite corners `x1, y1, x2, y2` of the enclosing rectangle); `bitmap` (a two-color bitmap positioned at a point `x, y`); `image` (a Tk image positioned at a point `x, y`); `line` (a line or curve through the points `x1, y1, ..., xn, yn`); `oval` (a circle or ellipse inscribed in the rectangle `x1, y1, x2, y2`); `polygon` (a closed polygon through the points `x1, y1, ..., xn, yn`); `rectangle` (a rectangle with corners `x1, y1, x2, y2`); `text` (a string of text positioned at a point `x, y`); and `window` (a child widget embedded in the canvas at a point `x, y`, specified with the *window* option).

Most item types accept a common set of *standard item options*, plus a few options specific to each type. Option names are passed as keyword arguments, without the leading hyphen.

The standard item options are:

*fill*  
The color used to fill the item's interior, or to draw a *line* item or the characters of a *text* item. An empty string (the default for all types except *line* and *text*) leaves the item unfilled.

*outline*  
The color used to draw the item's outline. An empty string draws no outline.

*width*  
The width of the outline, defaulting to `1.0`. Has no effect if *outline* is empty.

*dash*  
A dash pattern for the outline, given either as a sequence of segment lengths in pixels or as a string of the characters `'.'`, `','`, `'-'`, `'_'` and space. An empty pattern (the default) draws a solid outline.

*dashoffset*  
The starting offset in pixels into the *dash* pattern. Ignored if there is no *dash* pattern.

*stipple*  
A bitmap used as a stipple pattern when filling the item. Only well supported on X11.

*outlinestipple*  
A bitmap used as a stipple pattern when drawing the outline. Has no effect if *outline* is empty.

*offset*, *outlineoffset*  
The offset of the fill and outline stipple patterns, given as `'x,y'` or as a side such as `'n'`, `'se'` or `'center'`. Stipple offsets are only supported on X11.

*state*  
Overrides the canvas state for this item; one of `'normal'`, `'disabled'` or `'hidden'`.

*tags*  
A single tag or a sequence of tags to associate with the item, replacing any existing tags.

Many of these options have *active...* and *disabled...* variants (such as *activefill*, *disabledfill*, *activewidth*, *disableddash*, *activeoutline*, *disabledstipple*) that override the base option when the item is the active item (under the mouse pointer) or is in the disabled state.

The following item types support additional options.

For `arc` items:

*start*  
The start of the arc's angular range, in degrees measured counter-clockwise from the 3-o'clock position.

*extent*  
The size of the angular range, in degrees counter-clockwise from *start*.

*style*  
How the arc is drawn: `'pieslice'` (the default), `'chord'` or `'arc'`.

For `line` items:

*arrow*  
Where to draw arrowheads: `'none'` (the default), `'first'`, `'last'` or `'both'`.

*arrowshape*  
A sequence of three distances describing the shape of the arrowheads.

*capstyle*  
How line ends are drawn: `'butt'` (the default), `'projecting'` or `'round'`.

*joinstyle*  
How line vertices are drawn: `'round'` (the default), `'bevel'` or `'miter'`.

*smooth*  
The smoothing method: a false value (the default) for no smoothing, or `'true'`/`'bezier'` or `'raw'` to draw the line as a curve.

*splinesteps*  
The number of line segments approximating each spline when *smooth* is enabled.

For `polygon` items:

*joinstyle*, *smooth*, *splinesteps*  
As for `line` items, applied to the polygon's outline.

For `text` items:

*text*  
The string to display; newline characters start new lines.

*font*  
The font used for the text.

*justify*  
How lines are justified: `'left'` (the default), `'right'` or `'center'`.

*anchor*  
How the text is positioned relative to its point, defaulting to `'center'`.

*width*  
The maximum line length; if non-zero, lines are wrapped at spaces.

*angle*  
How many degrees to rotate the text counter-clockwise about its positioning point, from `0.0` to `360.0` (default `0.0`).

*underline*  
The index of a character to underline, or `-1` for none.

For `bitmap` items:

*bitmap*  
The bitmap to display.

*anchor*  
How the bitmap is positioned relative to its point.

*background*, *foreground*  
The colors used for the bitmap's `0` and `1` pixels; an empty *background* makes the `0` pixels transparent. Both have *active...* and *disabled...* variants, and *bitmap* has *activebitmap* and *disabledbitmap* variants.

For `image` items:

*image*  
The Tk image to display, previously created with the image protocols.

*anchor*  
How the image is positioned relative to its point.

Both options have *active...* and *disabled...* variants (*activeimage*, *disabledimage*) used in the active and disabled states.

For `window` items:

*window*  
The widget to embed; it must be a child of the canvas or of one of its ancestors, and may not be a top-level window.

*anchor*  
How the window is positioned relative to its point.

*width*, *height*  
The size to assign to the window; if zero (the default), the window is given its requested size.

`oval` and `rectangle` items have no type-specific options; they use only the standard item options.

> [!NOTE]
> Tk 8.6 added the *angle* option for `text` items.

</div>

<div class="method">

coords(tagOrId) coords(tagOrId, coordList, /) coords(tagOrId, /, \*coordList)

Query or modify the coordinates of an item. With only *tagOrId*, return a list of the floating-point coordinates of the item given by *tagOrId* (the first matching item if it matches several). Given new coordinates, replace the coordinates of that item with them; like the `create_*` methods, the coordinates may be given as separate numbers, as a single sequence, or as coordinate pairs. The returned coordinates are always in pixels, regardless of the units used to specify them; for rectangles, ovals and arcs they are ordered left, top, right, bottom.

<div class="versionchanged">

3.12 The arguments are now flattened: the coordinates may be given as separate arguments, as a single sequence, or grouped in pairs, like the `create_*` methods.

</div>

</div>

<div class="method">

move(tagOrId, xAmount, yAmount, /)

Move each of the items given by *tagOrId* in the canvas coordinate space by adding *xAmount* to every x-coordinate and *yAmount* to every y-coordinate of the item.

</div>

<div class="method">

moveto(tagOrId, x='', y='')

Move the items given by *tagOrId* so that the first coordinate pair (the upper-left corner of the bounding box) of the lowest matching item is at position (*x*, *y*). *x* or *y* may be an empty string, in which case the corresponding coordinate is unchanged. All matching items keep their positions relative to each other.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

scale(tagOrId, xOrigin, yOrigin, xScale, yScale, /)

Rescale the coordinates of all items given by *tagOrId* in canvas coordinate space. Each x-coordinate is adjusted so that its distance from *xOrigin* changes by a factor of *xScale*, and each y-coordinate so that its distance from *yOrigin* changes by a factor of *yScale* (a factor of `1.0` leaves the coordinate unchanged).

</div>

<div class="method">

rotate(tagOrId, xOrigin, yOrigin, angle, /)

Rotate the coordinates of all items given by *tagOrId* in canvas coordinate space about the origin (*xOrigin*, *yOrigin*) by *angle* degrees anticlockwise. Negative values of *angle* rotate clockwise.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

delete(\*tagOrIds)

Delete each of the items given by the *tagOrIds* arguments.

</div>

<div class="method">

dchars(tagOrId, first, /) dchars(tagOrId, first, last, /)

Delete from each of the items given by *tagOrId* the characters (for text items) or coordinates (for line and polygon items) in the range from *first* to *last* inclusive; *last* defaults to *first*. Items that do not support indexing ignore this operation.

</div>

<div class="method">

insert(tagOrId, beforeThis, string, /)

Insert *string* into each of the items given by *tagOrId* just before the character or coordinate whose index is *beforeThis*. For line and polygon items *string* must be a valid sequence of coordinates.

</div>

<div class="method">

rchars(tagOrId, first, last, string, /)

Replace the characters (for text items) or coordinates (for line and polygon items) in the range from *first* to *last* inclusive of each of the items given by *tagOrId* with *string*. For line and polygon items *string* must be a valid sequence of coordinates. Items that do not support indexing ignore this operation.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

itemcget(tagOrId, option)

Return the current value of the configuration option *option* for the item given by *tagOrId* (the lowest matching item if it matches several). This is like `~Misc.cget` but applies to an individual item.

</div>

<div class="method" data-no-typesetting="">

itemconfig(tagOrId, cnf=None, \*\*kw)

</div>

<div class="method">

itemconfigure(tagOrId, cnf=None, \*\*kw)

Query or modify the configuration options of the items given by *tagOrId*. This mirrors `~Misc.configure`, except that it applies to individual items rather than to the canvas as a whole. With no options, it returns a dictionary describing the current options of the first matching item; otherwise it sets the given options on every matching item. The legal options are those accepted by the corresponding `create_*` method. `itemconfig` is an alias of `!itemconfigure`.

</div>

<div class="method">

type(tagOrId)

Return the type of the item given by *tagOrId* (the first matching item if it matches several), such as `'rectangle'` or `'text'`, or `None` if *tagOrId* does not match any item.

</div>

<div class="method">

gettags(tagOrId, /)

Return a tuple of the tags associated with the item given by *tagOrId* (the first matching item in display-list order if it matches several). Return an empty tuple if no item matches or the item has no tags.

</div>

<div class="method">

dtag(tagOrId, /) dtag(tagOrId, tagToDelete, /)

Remove the tag *tagToDelete* (which defaults to *tagOrId*) from each of the items given by *tagOrId*. Items that do not have that tag are unaffected.

</div>

<div class="method">

addtag(newtag, searchSpec, /, \*args)

Add the tag *newtag* to each item selected by the search specification *searchSpec* (and any further *args*). *searchSpec* is one of `'above'`, `'all'`, `'below'`, `'closest'`, `'enclosed'`, `'overlapping'` or `'withtag'`; the `addtag_*` methods below are convenient wrappers that supply each of these forms.

</div>

<div class="method">

addtag_above(newtag, tagOrId)

Add the tag *newtag* to the item just above (after) *tagOrId* in the display list.

</div>

<div class="method">

addtag_all(newtag)

Add the tag *newtag* to all items in the canvas.

</div>

<div class="method">

addtag_below(newtag, tagOrId)

Add the tag *newtag* to the item just below (before) *tagOrId* in the display list.

</div>

<div class="method">

addtag_closest(newtag, x, y, halo=None, start=None)

Add the tag *newtag* to the item closest to the point (*x*, *y*). If *halo* is given, any item within that distance of the point is treated as overlapping it. If *start* is given (a tag or id), select the topmost closest item that lies below *start* in the display list, which can be used to step through all the closest items.

</div>

<div class="method">

addtag_enclosed(newtag, x1, y1, x2, y2)

Add the tag *newtag* to every item completely enclosed within the rectangle (*x1*, *y1*, *x2*, *y2*), where *x1* \<= *x2* and *y1* \<= *y2*.

</div>

<div class="method">

addtag_overlapping(newtag, x1, y1, x2, y2)

Add the tag *newtag* to every item that overlaps or is enclosed within the rectangle (*x1*, *y1*, *x2*, *y2*), where *x1* \<= *x2* and *y1* \<= *y2*.

</div>

<div class="method">

addtag_withtag(newtag, tagOrId)

Add the tag *newtag* to every item given by *tagOrId*.

</div>

<div class="method">

find(searchSpec, /, \*args)

Return a tuple of the ids of all items selected by the search specification *searchSpec* (and any further *args*), in stacking order with the lowest item first. The search specification has any of the forms accepted by `addtag`. The `find_*` methods below are more convenient wrappers around it.

</div>

<div class="method">

find_above(tagOrId)

Return a tuple containing the id of the item just above *tagOrId* in the display list.

</div>

<div class="method">

find_all()

Return a tuple of the ids of all items in the canvas, in stacking order.

</div>

<div class="method">

find_below(tagOrId)

Return a tuple containing the id of the item just below *tagOrId* in the display list.

</div>

<div class="method">

find_closest(x, y, halo=None, start=None)

Return a tuple containing the id of the item closest to the point (*x*, *y*). *halo* and *start* are interpreted as for `addtag_closest`.

</div>

<div class="method">

find_enclosed(x1, y1, x2, y2)

Return a tuple of the ids of all items completely enclosed within the rectangle (*x1*, *y1*, *x2*, *y2*).

</div>

<div class="method">

find_overlapping(x1, y1, x2, y2)

Return a tuple of the ids of all items that overlap or are enclosed within the rectangle (*x1*, *y1*, *x2*, *y2*).

</div>

<div class="method">

find_withtag(tagOrId)

Return a tuple of the ids of all items given by *tagOrId*.

</div>

<div class="method" data-no-typesetting="">

lift(tagOrId, aboveThis=None, /)

</div>

<div class="method" data-no-typesetting="">

tkraise(tagOrId, aboveThis=None, /)

</div>

<div class="method">

tag_raise(tagOrId, aboveThis=None, /)

Move all items given by *tagOrId* to a new position in the display list just above the item given by *aboveThis*, or to the top of the display list if *aboveThis* is omitted. When several items are moved their relative order is preserved. This has no effect on embedded window items, whose stacking order is controlled by `Misc.tkraise` and `Misc.lower` instead. `lift` and `tkraise` are aliases of `!tag_raise`.

</div>

<div class="method" data-no-typesetting="">

lower(tagOrId, belowThis=None, /)

</div>

<div class="method">

tag_lower(tagOrId, belowThis=None, /)

Move all items given by *tagOrId* to a new position in the display list just below the item given by *belowThis*, or to the bottom of the display list if *belowThis* is omitted. When several items are moved their relative order is preserved. This has no effect on embedded window items. `lower` is an alias of `!tag_lower`.

> [!NOTE]
> On a `Canvas`, `tkraise`/`lift` and `lower` restack canvas items, shadowing the inherited `Misc.tkraise`/`Misc.lift` and `Misc.lower` methods that restack the widget itself, which are therefore not available.

</div>

<div class="method">

tag_bind(tagOrId, sequence=None, func=None, add=None)

Bind the callback *func* to the event *sequence* for all items given by *tagOrId*, so that *func* is invoked whenever that event occurs for one of the items. This is like `Widget.bind <Misc.bind>` but operates on canvas items rather than on whole widgets; only mouse, keyboard and virtual events may be bound. Mouse events are directed to the current item and keyboard events to the focus item (see `focus`). If *add* is true the new binding is added to any existing bindings for the same sequence, rather than replacing them. Return the identifier of the bound function, which can be passed to `tag_unbind`.

</div>

<div class="method">

tag_unbind(tagOrId, sequence, funcid=None)

Remove for all items given by *tagOrId* the binding for the event *sequence*. If *funcid* is given, only that callback (as returned by `tag_bind`) is unbound and deregistered.

<div class="versionchanged">

3.13 If *funcid* is given, only that callback is unbound.

</div>

</div>

<div class="method">

bbox(tagOrId, /, \*tagOrIds)

Return a 4-tuple `(x1, y1, x2, y2)` giving an approximate bounding box, in pixels, that encloses all the items given by *tagOrId* and any further *tagOrIds*. The result may overestimate the true bounding box by a few pixels. Return `None` if no item matches or the matching items have nothing to display.

This shadows the inherited `!Misc.bbox`; use `~Misc.grid_bbox` for the grid bounding box.

</div>

<div class="method">

canvasx(screenx, gridspacing=None)

Given a window x-coordinate *screenx*, return the canvas x-coordinate displayed at that location. If *gridspacing* is given, the result is rounded to the nearest multiple of *gridspacing* units.

</div>

<div class="method">

canvasy(screeny, gridspacing=None)

Given a window y-coordinate *screeny*, return the canvas y-coordinate displayed at that location. If *gridspacing* is given, the result is rounded to the nearest multiple of *gridspacing* units.

</div>

<div class="method">

focus() focus(tagOrId, /)

With *tagOrId*, set the keyboard focus for the canvas to the first item given by *tagOrId* that supports the insertion cursor; the focus is left unchanged if no such item exists. If *tagOrId* is an empty string, reset the focus so that no item has it. With no argument, return the id of the item that currently has the focus, or an empty string if none does. An item only displays the insertion cursor when both it is the focus item and its canvas has the input focus.

This shadows the inherited `!Misc.focus`; use `~Misc.focus_set` to focus the widget itself.

</div>

<div class="method">

icursor(tagOrId, index, /)

Set the insertion cursor of the items given by *tagOrId* to just before the character given by *index*. Items that do not support an insertion cursor are unaffected. The cursor is only displayed when the item has the focus, but its position may be set at any time.

</div>

<div class="method">

index(tagOrId, index, /)

Return as an integer the numerical index within *tagOrId* corresponding to *index*, which is a textual description of a position (for text items an index into the characters, for line and polygon items an index into the coordinates). If *tagOrId* matches several items, the first one that supports indexing is used.

</div>

<div class="method">

select_adjust(tagOrId, index)

Adjust the end of the selection in *tagOrId* nearest to *index* so that it is at *index*, and make the other end the anchor point for future `select_to` calls. If the selection is not currently in *tagOrId*, this behaves like `select_to`.

</div>

<div class="method">

select_clear()

Clear the selection if it is in this canvas; otherwise do nothing.

</div>

<div class="method">

select_from(tagOrId, index)

Set the selection anchor point to just before the character given by *index* in the item given by *tagOrId*. This does not change the selection itself; it sets the fixed end for future `select_to` calls.

</div>

<div class="method">

select_item()

Return the id of the item that holds the selection, or `None` if the selection is not in this canvas. Unlike `find` and the `find_*` methods, this returns the id as a string rather than an integer.

</div>

<div class="method">

select_to(tagOrId, index)

Set the selection to the characters of *tagOrId* between the selection anchor point and *index*, inclusive of *index*. The anchor point is the one set by the most recent `select_adjust` or `select_from` call.

</div>

<div class="method">

scan_mark(x, y)

Record *x*, *y* and the current view, for use with later `scan_dragto` calls. This is typically bound to a mouse button press in the widget.

</div>

<div class="method">

scan_dragto(x, y, gain=10)

Scroll the canvas by *gain* times the difference between *x*, *y* and the coordinates passed to the last `scan_mark` call. This is typically bound to mouse motion events in the widget, producing the effect of dragging the canvas at high speed through its window.

</div>

<div class="method">

postscript(cnf={}, \*\*kw)

Generate a PostScript (Encapsulated PostScript, version 3.0) representation of part or all of the canvas. If the *file* or *channel* option is given, the PostScript is written there and an empty string is returned; otherwise it is returned as a string. By default only the area currently visible in the window is generated, so it is usually necessary either to call `~Misc.update` first or to use the *width* and *height* options. Supported options include *colormap*, *colormode*, *file*, *fontmap*, *height*, *pageanchor*, *pageheight*, *pagewidth*, *pagex*, *pagey*, *rotate*, *width*, *x* and *y*.

</div>

</div>

<div class="Checkbutton(master=None, cnf={}, **kw)">

A `!Checkbutton` widget displays a textual string, bitmap or image together with a square indicator, and toggles a boolean selection when pressed. It has all the behavior of a simple button and, in addition, can be selected: when selected the indicator is drawn with a check mark and the associated variable is set to the `onvalue`, and when deselected the indicator is drawn empty and the variable is set to the `offvalue`. Inherits from `Widget`. In addition to the standard widget options, a checkbutton accepts the options documented in the Tk `checkbutton` manual page, such as *variable*, *onvalue*, *offvalue* and *command*.

<div class="method">

invoke()

Do just what would happen if the user pressed the checkbutton with the mouse: toggle the selection state of the button and invoke the associated command, if there is one. Return the result of the command, or an empty string if no command is associated with the checkbutton. This is ignored if the checkbutton's state is `disabled`.

</div>

<div class="method">

select()

Select the checkbutton and set the associated variable to its `onvalue`.

</div>

<div class="method">

deselect()

Deselect the checkbutton and set the associated variable to its `offvalue`.

</div>

<div class="method">

toggle()

Toggle the selection state of the button, redisplaying it and modifying its associated variable to reflect the new state.

</div>

<div class="method">

flash()

Flash the checkbutton by redisplaying it several times, alternating between the active and normal colors. At the end of the flash the checkbutton is left in the same normal or active state as when the method was called. This is ignored if the checkbutton's state is `disabled`.

</div>

</div>

<div class="Entry(master=None, cnf={}, **kw)">

An `!Entry` widget displays a single line of text and lets the user edit it. Inherits from `Widget` and `XView`; since entries can hold strings too long to fit in the window, they support horizontal scrolling through `~XView.xview`.

In addition to the standard widget options, an entry accepts the options documented in the Tk `entry` manual page. Notable ones are *textvariable* (the name of a variable kept in sync with the entry's contents), *show* (if set, each character is displayed as the given character rather than its true value, useful for password entry), *validate* and *validatecommand* (which together let a callback accept or reject edits), and *state* (one of `'normal'`, `'disabled'` or `'readonly'`).

Many of the methods below take an *index* argument that selects a character in the entry's string. As described in the Tk `entry` manual page, *index* may be a number (counting from 0), `'insert'` (the character just after the insertion cursor), `'end'` (just after the last character), `'anchor'` (the selection anchor point), `'sel.first'` and `'sel.last'` (the ends of the selection), or `@x` (the character covering pixel x-coordinate *x* in the window). Out-of-range indices are rounded to the nearest legal value.

<div class="method">

delete(first, last=None)

Delete the characters from index *first* up to but not including index *last*. If *last* is omitted, only the single character at *first* is deleted.

</div>

<div class="method">

get()

Return the entry's current string.

</div>

<div class="method">

insert(index, string)

Insert *string* just before the character given by *index*.

</div>

<div class="method">

icursor(index)

Arrange for the insertion cursor to be displayed just before the character given by *index*.

</div>

<div class="method">

index(index)

Return the numerical index corresponding to *index*.

</div>

<div class="method" data-no-typesetting="">

select_adjust(index)

</div>

<div class="method">

selection_adjust(index)

Locate the end of the selection nearest to the character given by *index*, and adjust that end to be at *index* (including but not going beyond it); the other end becomes the anchor point for future `selection_to` calls. If there is no selection in the entry, a new one is created between *index* and the most recent anchor point, inclusive. `select_adjust` is an alias of `!selection_adjust`.

</div>

<div class="method" data-no-typesetting="">

select_clear()

</div>

<div class="method">

selection_clear()

Clear the selection if it is currently in this widget. If the selection is not in this widget the method has no effect. `select_clear` is an alias of `!selection_clear`.

> [!NOTE]
> This shadows the inherited `Misc.selection_clear`, which clears the X selection; that method is not available on an `Entry`.

</div>

<div class="method" data-no-typesetting="">

select_from(index)

</div>

<div class="method">

selection_from(index)

Set the selection anchor point to just before the character given by *index*, without changing the selection. `select_from` is an alias of `!selection_from`.

</div>

<div class="method" data-no-typesetting="">

select_present()

</div>

<div class="method">

selection_present()

Return `True` if there are characters selected in the entry, `False` otherwise. `select_present` is an alias of `!selection_present`.

</div>

<div class="method" data-no-typesetting="">

select_range(start, end)

</div>

<div class="method">

selection_range(start, end)

Set the selection to include the characters starting with the one indexed by *start* and ending with the one just before *end*. If *end* refers to the same character as *start* or an earlier one, the selection is cleared. `select_range` is an alias of `!selection_range`.

</div>

<div class="method" data-no-typesetting="">

select_to(index)

</div>

<div class="method">

selection_to(index)

Set the selection between the anchor point and *index*: if *index* is before the anchor point, the selection runs from *index* up to but not including the anchor; if *index* is after it, from the anchor up to but not including *index*; if they coincide, nothing happens. The anchor point is the one set by the most recent `selection_from` or `selection_adjust` call. If there is no selection in the entry, a new one is created using the most recent anchor point. `select_to` is an alias of `!selection_to`.

</div>

<div class="method">

scan_mark(x)

Record *x* and the current view in the entry window, for use with later `scan_dragto` calls. Typically associated with a mouse button press in the widget.

</div>

<div class="method">

scan_dragto(x)

Compute the difference between *x* and the *x* given to the last `scan_mark` call, and adjust the view left or right by 10 times that difference. Typically associated with mouse motion events, to produce the effect of dragging the entry at high speed through the window.

</div>

<div class="method">

validate()

Force an evaluation of the command given by the *validatecommand* option, independently of the conditions specified by the *validate* option, and return whether the value is considered valid.

<div class="versionadded">

3.16

</div>

</div>

</div>

<div class="Frame(master=None, cnf={}, **kw)">

A `!Frame` widget is a simple container. Its primary purpose is to act as a spacer or container for complex window layouts; its only features are its background and an optional 3-D border to make the frame appear raised or sunken. Inherits from `Widget`. Refer to the Tk `frame` manual page for the full list of options.

</div>

<div class="Label(master=None, cnf={}, **kw)">

A `!Label` widget displays a non-interactive textual string, bitmap or image. The displayed text is set with the *text* option or linked to a variable through *textvariable*, and an image can be shown using the *image* option. Text must all be in a single font but may occupy multiple lines, and one character may be underlined with the *underline* option. Inherits from `Widget`. Refer to the Tk `label` manual page for the full list of options.

</div>

<div class="LabelFrame(master=None, cnf={}, **kw)">

A `!LabelFrame` widget is a container that has the features of a `Frame` plus the ability to display a label. The label text is set with the *text* option and positioned with *labelanchor*, or an arbitrary widget may be used as the label by giving it as the *labelwidget* option. Inherits from `Widget`. Refer to the Tk `labelframe` manual page for the full list of options.

</div>

<div class="Listbox(master=None, cnf={}, **kw)">

A `!Listbox` widget displays a list of single-line text items, one per line, of which the user can select one or more. The way the selection behaves is governed by the *selectmode* option, which is one of `browse` (the default; at most one item, which may be dragged with the mouse), `single` (at most one item), `multiple` (any number of items, toggled individually), or `extended` (any number of items, including discontiguous ranges, selected by clicking and dragging). Inherits from `Widget`, `XView` and `YView`, so the view can be scrolled horizontally and vertically with `~XView.xview` and `~YView.yview`. Refer to the Tk `listbox` manual page for the full list of options.

Many of the methods take an *index* argument identifying a particular item. As described in the Tk `listbox` manual page, *index* may be a numeric index (counting from 0 at the top), `'active'` (the item with the location cursor, set with `activate`), `'anchor'` (the selection anchor, set with `selection_anchor`), `'end'` (the last item, or for `index` and `insert` the position just after it), or `@x,y` (the item covering pixel coordinates *x*, *y* in the listbox window). Arguments named *first* and *last* are indices of the same forms.

<div class="method">

insert(index, \*elements)

Insert the given *elements* as new items just before the item given by *index*. If *index* is `'end'`, the new items are appended to the end of the list.

</div>

<div class="method">

delete(first, last=None)

Delete the items in the range from *first* to *last* inclusive. If *last* is omitted, it defaults to *first*, so that a single item is deleted.

</div>

<div class="method">

get(first, last=None)

If *last* is omitted, return the contents of the item given by *first*, or an empty string if *first* refers to a non-existent item. If *last* is given, return a tuple of all the items in the range from *first* to *last* inclusive.

</div>

<div class="method">

size()

Return the total number of items in the listbox.

This shadows the inherited `!Misc.size`; use `~Misc.grid_size` for the grid size.

</div>

<div class="method">

index(index)

Return the integer index value corresponding to *index*, or `None` if *index* is out of range. If *index* is `'end'`, the result is a count of the number of items in the listbox (not the index of the last item).

</div>

<div class="method">

bbox(index)

Return a tuple `(x, y, width, height)` describing the bounding box, in pixels relative to the widget, of the text of the item given by *index*. Return `None` if no part of that item is visible on the screen, or if *index* refers to a non-existent item; if the item is only partly visible, the result still gives the full area of the item, including the parts that are not visible.

This shadows the inherited `!Misc.bbox`; use `~Misc.grid_bbox` for the grid bounding box.

</div>

<div class="method">

nearest(y)

Given a y-coordinate within the listbox window, return the index of the visible item nearest to that y-coordinate.

</div>

<div class="method">

see(index)

Adjust the view so that the item given by *index* is visible. If the item is already visible the method has no effect; if it is near an edge of the window the listbox scrolls just enough to bring it into view at that edge, otherwise the listbox scrolls to center the item.

</div>

<div class="method">

activate(index)

Set the active item to the one given by *index*. If *index* is outside the range of items, the closest item is activated instead. The active item is drawn as specified by the *activestyle* option when the widget has the input focus, and its index may be retrieved with the `'active'` index.

</div>

<div class="method">

curselection()

Return a tuple containing the numerical indices of all of the items that are currently selected, or an empty tuple if no items are selected.

</div>

<div class="method" data-no-typesetting="">

select_anchor(index)

</div>

<div class="method">

selection_anchor(index)

Set the selection anchor to the item given by *index*. If *index* refers to a non-existent item, the closest item is used. The selection anchor is the end of the selection that is fixed while dragging out a selection with the mouse, and may afterwards be referred to with the `'anchor'` index. `select_anchor` is an alias of `!selection_anchor`.

</div>

<div class="method" data-no-typesetting="">

select_clear(first, last=None)

</div>

<div class="method">

selection_clear(first, last=None)

Deselect any of the items in the range from *first* to *last* inclusive that are selected. The selection state of items outside this range is not changed. `select_clear` is an alias of `!selection_clear`.

> [!NOTE]
> This shadows the inherited `Misc.selection_clear`, which clears the X selection; that method is not available on a `Listbox`.

</div>

<div class="method" data-no-typesetting="">

select_includes(index)

</div>

<div class="method">

selection_includes(index)

Return `True` if the item given by *index* is currently selected, `False` otherwise. `select_includes` is an alias of `!selection_includes`.

</div>

<div class="method" data-no-typesetting="">

select_set(first, last=None)

</div>

<div class="method">

selection_set(first, last=None)

Select all of the items in the range from *first* to *last* inclusive, without affecting the selection state of items outside that range. `select_set` is an alias of `!selection_set`.

</div>

<div class="method">

itemcget(index, option)

Return the current value of the configuration option *option* for the item given by *index*.

</div>

<div class="method" data-no-typesetting="">

itemconfig(index, cnf=None, \*\*kw)

</div>

<div class="method">

itemconfigure(index, cnf=None, \*\*kw)

Query or modify the configuration options of the item given by *index*. This mirrors `~Misc.configure`, except that it applies to an individual item rather than to the listbox as a whole. With no options, it returns a dictionary describing the current options of the item; otherwise it sets the given options. The supported item options are *background*, *foreground*, *selectbackground* and *selectforeground*. `itemconfig` is an alias of `!itemconfigure`.

</div>

<div class="method">

scan_mark(x, y)

Record *x*, *y* and the current view, for use with later `scan_dragto` calls. This is typically bound to a mouse button press in the widget.

</div>

<div class="method">

scan_dragto(x, y)

Scroll the listbox by 10 times the difference between *x*, *y* and the coordinates passed to the last `scan_mark` call. This is typically bound to mouse motion events in the widget, producing the effect of dragging the list at high speed through the window.

</div>

</div>

<div class="Menu(master=None, cnf={}, **kw)">

A `!Menu` widget displays a column of entries, each of which may be a command, a checkbutton, a radiobutton, a cascade (which posts an associated submenu) or a separator. Menus are used as the menubar of a toplevel window, as pulldown menus posted from a cascade entry or menubutton, and as popup menus. Inherits from `Widget`.

Many of the entry methods take an *index* argument that selects which entry to operate on. As described in the Tk `menu` manual page, *index* may be a numeric index (counting from 0 at the top), `'active'` (the currently active entry), `'end'` or `'last'` (the bottommost entry), `'none'` (no entry at all, written `{}` in Tcl), `@y` (the entry covering pixel y-coordinate *y* in the menu window), or a pattern matched against the labels of the entries from the top down.

<div class="method">

add(itemType, cnf={}, \*\*kw)

Add a new entry to the bottom of the menu. *itemType* is one of `'command'`, `'cascade'`, `'checkbutton'`, `'radiobutton'` or `'separator'` and determines the type of the new entry; the remaining options configure it. The `!add_command`, `!add_cascade`, `!add_checkbutton`, `!add_radiobutton` and `!add_separator` convenience methods call this method with the corresponding *itemType*.

The entry is configured by the following options, although not every option applies to every entry type (a separator accepts none of them):

*label*  
The text to display in the entry.

*command*  
The function to call when the entry is invoked (command, checkbutton and radiobutton entries).

*accelerator*  
A string displayed at the right of the entry to advertise an accelerator keystroke; it does not itself create the binding.

*underline*  
The index of a character in the label to underline for keyboard traversal.

*state*  
One of `'normal'`, `'active'` or `'disabled'`.

*image*  
An image to display instead of, or together with, the text label.

*compound*  
Where to show the image relative to the text: `'none'` (the default), `'text'`, `'image'`, `'top'`, `'bottom'`, `'left'` or `'right'`.

*bitmap*  
A bitmap to display instead of the text label.

*font*  
The font to use for the text.

*background*, *foreground*  
The entry's background and foreground colors in its normal state (ignored on macOS).

*activebackground*, *activeforeground*  
The background and foreground colors used when the entry is active (ignored on macOS).

*columnbreak*  
If true, the entry starts a new column instead of being placed below the previous entry.

*hidemargin*  
If true, the standard margin around the entry is omitted, which is useful when a menu is used as a palette.

*menu*  
The submenu posted by a cascade entry; it must be a child of this menu.

*variable*  
The variable associated with a checkbutton or radiobutton entry.

*onvalue*, *offvalue*  
The values stored in *variable* when a checkbutton entry is selected or cleared.

*value*  
The value stored in *variable* when a radiobutton entry is selected.

*indicatoron*  
Whether to display the indicator of a checkbutton or radiobutton entry.

*selectcolor*  
The color of the indicator of a checkbutton or radiobutton entry when it is selected.

*selectimage*  
The image displayed when a checkbutton or radiobutton entry is selected and *image* is also given.

</div>

<div class="method">

add_cascade(cnf={}, \*\*kw)

Add a new cascade entry to the bottom of the menu. A cascade entry has an associated submenu, given by its *menu* option, which must be a child of this menu; posting the entry posts the submenu next to it.

</div>

<div class="method">

add_checkbutton(cnf={}, \*\*kw)

Add a new checkbutton entry to the bottom of the menu. When invoked, a checkbutton entry toggles between its *onvalue* and *offvalue*, storing the result in its associated *variable*, and displays an indicator showing whether it is selected.

</div>

<div class="method">

add_command(cnf={}, \*\*kw)

Add a new command entry to the bottom of the menu. A command entry behaves much like a button: when it is invoked, the callback given by its *command* option is called.

</div>

<div class="method">

add_radiobutton(cnf={}, \*\*kw)

Add a new radiobutton entry to the bottom of the menu. Radiobutton entries sharing the same *variable* form a group of which only one may be selected at a time; selecting an entry stores its *value* in the variable.

</div>

<div class="method">

add_separator(cnf={}, \*\*kw)

Add a separator to the bottom of the menu. A separator is displayed as a horizontal dividing line and cannot be activated or invoked.

</div>

<div class="method">

insert(index, itemType, cnf={}, \*\*kw)

Same as `add`, except that the new entry is inserted just before the entry given by *index* instead of being appended to the end of the menu. *itemType* is one of `'command'`, `'cascade'`, `'checkbutton'`, `'radiobutton'` or `'separator'`. The `!insert_command`, `!insert_cascade`, `!insert_checkbutton`, `!insert_radiobutton` and `!insert_separator` convenience methods call this method with the corresponding *itemType*.

</div>

<div class="method">

insert_cascade(index, cnf={}, \*\*kw)

Insert a new cascade entry before the entry given by *index* (see `add_cascade`).

</div>

<div class="method">

insert_checkbutton(index, cnf={}, \*\*kw)

Insert a new checkbutton entry before the entry given by *index* (see `add_checkbutton`).

</div>

<div class="method">

insert_command(index, cnf={}, \*\*kw)

Insert a new command entry before the entry given by *index* (see `add_command`).

</div>

<div class="method">

insert_radiobutton(index, cnf={}, \*\*kw)

Insert a new radiobutton entry before the entry given by *index* (see `add_radiobutton`).

</div>

<div class="method">

insert_separator(index, cnf={}, \*\*kw)

Insert a separator before the entry given by *index* (see `add_separator`).

</div>

<div class="method">

delete(index1, index2=None)

Delete all of the menu entries between *index1* and *index2* inclusive. If *index2* is omitted, it defaults to *index1*, so that a single entry is deleted. Attempts to delete a tear-off entry are ignored; remove it by changing the *tearoff* option instead.

</div>

<div class="method">

entrycget(index, option)

Return the current value of the configuration option *option* for the entry given by *index*.

</div>

<div class="method" data-no-typesetting="">

entryconfig(index, cnf=None, \*\*kw)

</div>

<div class="method">

entryconfigure(index, cnf=None, \*\*kw)

Query or modify the configuration options of the entry given by *index*. This mirrors `~Misc.configure`, except that it applies to an individual entry rather than to the menu as a whole. With no options, it returns a dictionary describing the current options of the entry; otherwise it sets the given options. The supported options are those accepted by `add` for the entry's type. `entryconfig` is an alias of `!entryconfigure`.

</div>

<div class="method">

index(index)

Return the numerical index corresponding to *index*, or `None` if *index* selects no entry.

</div>

<div class="method">

type(index)

Return the type of the entry given by *index*: one of `'command'`, `'cascade'`, `'checkbutton'`, `'radiobutton'`, `'separator'` or `'tearoff'` (for the tear-off entry).

</div>

<div class="method">

activate(index)

Make the entry given by *index* the active entry, redisplaying it with its active colors, and deactivate any previously active entry. If *index* selects no entry, or the selected entry is disabled, the menu ends up with no active entry.

</div>

<div class="method">

invoke(index)

Invoke the action of the entry given by *index*, as if it had been clicked. Nothing happens if the entry is disabled. If the entry has a *command* associated with it, the result of that command is returned; otherwise the result is an empty string.

</div>

<div class="method">

post(x, y)

Display the menu on the screen at the root-window coordinates *x* and *y*, adjusting them if necessary so that the whole menu is visible. If the *postcommand* option has been specified, it is evaluated before the menu is posted.

</div>

<div class="method">

postcascade(index)

Post the submenu associated with the cascade entry given by *index*, unposting any previously posted submenu. This has no effect if *index* does not name a cascade entry or if the menu itself is not posted.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

tk_popup(x, y, entry='')

Post the menu as a popup at the root-window coordinates *x* and *y*. If *entry* is given, the menu is positioned so that this entry is displayed under the pointer.

</div>

<div class="method">

unpost()

Unmap the menu so that it is no longer displayed, also unposting any posted lower-level cascaded submenu. This has no effect on Windows and macOS, which manage the unposting of menus themselves.

</div>

<div class="method">

xposition(index)

Return the x-coordinate, within the menu window, of the leftmost pixel of the entry given by *index*.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

yposition(index)

Return the y-coordinate, within the menu window, of the topmost pixel of the entry given by *index*.

</div>

</div>

<div class="Menubutton(master=None, cnf={}, **kw)">

A `!Menubutton` widget displays a textual string, bitmap or image and posts an associated `Menu`, given by its *menu* option, when the user presses it. Like a `Label` it can show *text*, a *textvariable*, or an *image*, and the *direction* option controls where the menu appears relative to the button. Inherits from `Widget`. Refer to the Tk `menubutton` manual page for the full list of options.

</div>

<div class="Message(master=None, cnf={}, **kw)">

A `!Message` widget displays a non-interactive textual string, given by the *text* option or linked to a variable through *textvariable*. Unlike a `Label`, it breaks the string into multiple lines in order to produce a given aspect ratio, choosing line breaks at word boundaries, and it can justify the text left, centered or right. Inherits from `Widget`. Refer to the Tk `message` manual page for the full list of options.

</div>

<div class="OptionMenu(master, variable, value, *values, **kwargs)">

A helper subclass of `Menubutton` that displays a pop-up menu of mutually exclusive choices. *variable* is a `Variable` kept in sync with the selection, *value* is the initial choice, and *values* are the remaining menu entries. The keyword argument *command* may be given a callback that is invoked with the selected value, and the keyword argument *name* sets the Tk widget name. Other keyword arguments are passed to the underlying `Menubutton` and may override its default appearance.

<div class="method">

destroy()

Destroy the widget, also cleaning up the associated pop-up menu.

</div>

<div class="versionchanged">

3.14 Added support for the *name* keyword argument.

</div>

<div class="versionchanged">

3.16 Other `Menubutton` options can now be passed as keyword arguments.

</div>

</div>

<div class="PanedWindow(master=None, cnf={}, **kw)">

A `!PanedWindow` is a geometry-manager widget that arranges any number of child *panes* in a row (when *orient* is `'horizontal'`) or a column (when *orient* is `'vertical'`). Each pane holds one widget, and each pair of adjacent panes is separated by a movable *sash* that the user can drag with the mouse to resize the widgets on either side of it. Inherits from `Widget`.

The *orient* option selects the layout direction, *sashwidth* sets the width of each sash and *sashrelief* its relief. When *showhandle* is true a small handle is drawn on each sash that the user can grab to drag it. Refer to the Tk `panedwindow` manual page for the full list of options.

<div class="method">

add(child, \*\*kw)

Add *child* to the panedwindow as a new pane, placed after any existing panes. The keyword arguments specify per-pane management options for *child*; they may be any of the options accepted by `paneconfigure`.

</div>

<div class="method" data-no-typesetting="">

forget(child)

</div>

<div class="method">

remove(child)

Remove the pane containing *child* from the panedwindow. All geometry management options for *child* are forgotten. `forget` is an alias of `!remove`. This shadows the inherited geometry-manager `!forget`; use `~Pack.pack_forget`, `~Grid.grid_forget` or `~Place.place_forget` to remove the widget itself from its manager.

</div>

<div class="method">

panes()

Return a tuple of the widgets managed by the panedwindow, one per pane, in order.

</div>

<div class="method">

panecget(child, option)

Return the current value of the management option *option* for the pane containing *child*. *option* may be any value allowed by `paneconfigure`.

</div>

<div class="method" data-no-typesetting="">

paneconfig(child, cnf=None, \*\*kw)

</div>

<div class="method">

paneconfigure(child, cnf=None, \*\*kw)

Query or modify the management options of the pane containing the widget *child*. With no options, it returns a dictionary describing all of the available options for the pane; given a single option name as a string, it returns a description of that one option; otherwise it sets the given options. The supported options include *after* and *before* (insert the pane after or before another managed window), *height* and *width* (the outer dimensions of the window, including any border), *minsize* (the minimum size in the paned dimension), *padx* and *pady* (extra space to leave on each side of the window), *sticky* (position or stretch the window within an oversized pane, using a string of the characters `n`, `s`, `e` and `w`), *hide* (hide the pane while keeping it in the list of panes) and *stretch* (how extra space is allocated to the pane: one of `'always'`, `'first'`, `'last'`, `'middle'` or `'never'`). `paneconfig` is an alias of `!paneconfigure`.

<div class="deprecated-removed">

3.16 3.18 The first parameter was renamed from *tagOrId* to *child*. The old name is still accepted as a keyword argument.

</div>

</div>

<div class="method">

identify(x, y)

Identify the panedwindow component underneath the point given by *x* and *y*, in window coordinates. If the point is over a sash or a sash handle, the result is a two-element tuple containing the index of the sash or handle and a word indicating whether it is over a sash or a handle, such as `(0, 'sash')` or `(2, 'handle')`. If the point is over any other part of the panedwindow, the result is an empty string.

</div>

<div class="method">

sash(\*args)

Query or change the position of the sashes in the panedwindow. This is a thin wrapper around the Tk `sash` subcommand; the convenience methods `sash_coord`, `sash_mark` and `sash_place` should normally be used instead.

</div>

<div class="method">

sash_coord(index)

Return the current x and y coordinate pair for the sash given by *index*, which must be an integer between 0 and one less than the number of panes in the panedwindow. The coordinates returned are those of the top left corner of the region containing the sash.

</div>

<div class="method">

sash_mark(index)

Record the current mouse position for the sash given by *index*, for use together with later sash-drag operations to move the sash.

</div>

<div class="method">

sash_place(index, x, y)

Place the sash given by *index* at the coordinates *x* and *y*.

</div>

<div class="method">

proxy(\*args)

Query or change the position of the sash proxy, the "ghost" sash shown while a sash is being dragged with non-opaque resizing. This is a thin wrapper around the Tk `proxy` subcommand; the convenience methods `proxy_coord`, `proxy_forget` and `proxy_place` should normally be used instead.

</div>

<div class="method">

proxy_coord()

Return a tuple containing the x and y coordinates of the most recent proxy location.

</div>

<div class="method">

proxy_forget()

Remove the proxy from the display.

</div>

<div class="method">

proxy_place(x, y)

Place the proxy at the coordinates *x* and *y*.

</div>

</div>

<div class="Radiobutton(master=None, cnf={}, **kw)">

A `!Radiobutton` widget displays a textual string, bitmap or image together with a diamond or circular indicator, and selects one choice out of several. It has all the behavior of a simple button and, in addition, can be selected: typically several radiobuttons share a single *variable*, and selecting one sets that variable to the radiobutton's *value*; each radiobutton also monitors the variable and automatically selects or deselects itself when the variable changes. Inherits from `Widget`. In addition to the standard widget options, a radiobutton accepts the options documented in the Tk `radiobutton` manual page, such as *variable*, *value* and *command*.

<div class="method">

invoke()

Do just what would happen if the user pressed the radiobutton with the mouse: select the button and invoke the associated command, if there is one. Return the result of the command, or an empty string if no command is associated with the radiobutton. This is ignored if the radiobutton's state is `disabled`.

</div>

<div class="method">

select()

Select the radiobutton and set the associated variable to the value corresponding to this widget.

</div>

<div class="method">

deselect()

Deselect the radiobutton and set the associated variable to an empty string. If this radiobutton was not currently selected, this has no effect.

</div>

<div class="method">

flash()

Flash the radiobutton by redisplaying it several times, alternating between the active and normal colors. At the end of the flash the radiobutton is left in the same normal or active state as when the method was called. This is ignored if the radiobutton's state is `disabled`.

</div>

</div>

<div class="Scale(master=None, cnf={}, **kw)">

A `!Scale` widget lets the user select a numerical value by moving a slider along a trough. It can be oriented vertically or horizontally and can optionally display a label and the current value. Inherits from `Widget`.

In addition to the standard widget options, a scale accepts the options documented in the Tk `scale` manual page, such as *from\_*, *to*, *resolution*, *orient*, *tickinterval*, *variable* and *command*. As elsewhere in `!tkinter`, the leading `-` of the Tk option name is dropped; *from* is spelled `from_` because `from` is a Python keyword.

With a non-integer *resolution*, see `numeric values and the locale
<tkinter-numeric-locale>`.

<div class="method">

get()

Return the current value of the scale. The result is an integer if the scale's *resolution* yields whole numbers, and a float otherwise.

</div>

<div class="method">

set(value)

Set the scale to *value*, moving the slider accordingly. This has no effect if the scale is disabled.

</div>

<div class="method">

coords(value=None)

Return a tuple `(x, y)` giving the pixel coordinates, relative to the widget, of the point on the centerline of the trough that corresponds to *value*. If *value* is omitted, the scale's current value is used.

</div>

<div class="method">

identify(x, y)

Return a string describing the part of the scale at the pixel coordinates *x*, *y*: `'slider'`, `'trough1'` (the part of the trough above or to the left of the slider), `'trough2'` (below or to the right of the slider), or an empty string if the point is not over any of these elements.

</div>

</div>

<div class="Scrollbar(master=None, cnf={}, **kw)">

A `!Scrollbar` widget displays a slider and two arrows that let the user scroll an associated widget, such as a `Listbox`, `Text`, `Canvas` or `Entry`. It is connected to the scrolled widget by setting that widget's *xscrollcommand* or *yscrollcommand* option to the scrollbar's `set` method, and the scrollbar's *command* option to the scrolled widget's `~XView.xview` or `~YView.yview` method. Inherits from `Widget`.

<div class="method">

get()

Return the current scrollbar settings as a tuple `(first, last)` of two fractions between 0 and 1, describing the portion of the document that is currently visible, as last passed to `set`.

</div>

<div class="method">

set(first, last)

Set the scrollbar. *first* and *last* are fractions between 0 and 1 giving the positions of the start and end of the visible portion of the associated document. This method is normally registered as the scrolled widget's *xscrollcommand* or *yscrollcommand* and called by that widget.

</div>

<div class="method">

activate(index=None)

Mark the element *index* (one of `'arrow1'`, `'slider'` or `'arrow2'`) as active, displaying it according to the *activebackground* and *activerelief* options. If *index* is omitted, return the name of the currently active element, or `None` if no element is active.

<div class="versionchanged">

3.5 The *index* argument is now optional.

</div>

</div>

<div class="method">

delta(deltax, deltay)

Return a float indicating the fractional change in the scrollbar setting that corresponds to moving the slider by *deltax* pixels horizontally (for horizontal scrollbars) or *deltay* pixels vertically (for vertical scrollbars).

</div>

<div class="method">

fraction(x, y)

Return a float between 0 and 1 indicating where the point at pixel coordinates *x*, *y* lies in the trough: 0 corresponds to the top or left of the trough and 1 to the bottom or right.

</div>

<div class="method">

identify(x, y)

Return the name of the element under the pixel coordinates *x*, *y* (such as `'arrow1'`), or an empty string if the point does not lie in any element of the scrollbar.

</div>

</div>

<div class="Spinbox(master=None, cnf={}, **kw)">

A `!Spinbox` widget is an `Entry`-like widget with a pair of up/down arrow buttons that let the user step through a range of values in addition to editing the value directly. The set of values may be a numeric range given by the *from\_*, *to* and *increment* options, or an explicit list of strings given by the *values* option (which takes precedence over the range). Each time an arrow is invoked the *command* callback, if any, is called; the boolean *wrap* option controls whether stepping past either end of the range wraps around to the other end; the *format* option specifies how numeric values are formatted; and the *validate* option enables validation of the entered text. Inherits from `Widget` and `XView`.

With a non-integer *increment*, see `numeric values and the locale
<tkinter-numeric-locale>`.

Many of the methods take an *index* argument identifying a character in the spinbox's string. As described in the Tk `spinbox` manual page, *index* may be a numeric index (counting from 0), `'anchor'` (the selection anchor point), `'end'` (just after the last character), `'insert'` (the character just after the insertion cursor), `'sel.first'` or `'sel.last'` (the ends of the selection), or `@x` (the character covering pixel x-coordinate *x* in the window).

<div class="method">

get()

Return the spinbox's string.

</div>

<div class="method">

insert(index, s)

Insert the characters of the string *s* just before the character given by *index*.

</div>

<div class="method">

delete(first, last=None)

Delete one or more characters of the spinbox. *first* is the index of the first character to delete, and *last* is the index of the character just after the last one to delete. If *last* is omitted, a single character at *first* is deleted.

</div>

<div class="method">

icursor(index)

Arrange for the insertion cursor to be displayed just before the character given by *index*.

</div>

<div class="method">

index(index)

Return the numerical index corresponding to *index*, as a string.

</div>

<div class="method">

bbox(index)

Return a tuple of four integers `(x, y, width, height)` describing the bounding box of the character given by *index*. *x* and *y* are the pixel coordinates of the upper-left corner of the character relative to the widget, and *width* and *height* are its size in pixels. The bounding box may refer to a region outside the visible area of the window.

This shadows the inherited `!Misc.bbox`; use `~Misc.grid_bbox` for the grid bounding box.

</div>

<div class="method">

identify(x, y)

Return the name of the window element at the pixel coordinates *x*, *y*: one of `'buttondown'`, `'buttonup'`, `'entry'` or `'none'`.

</div>

<div class="method">

invoke(element)

Invoke the spin button given by *element*, either `'buttonup'` or `'buttondown'`, triggering the action associated with it.

</div>

<div class="method">

scan(\*args)

A thin wrapper around the Tk `scan` widget subcommand, used to implement fast dragging of the view: `scan('mark', x)` records *x* and the current view, and `scan('dragto', x)` adjusts the view relative to that mark. The `scan_mark` and `scan_dragto` methods wrap the two forms.

</div>

<div class="method">

scan_mark(x)

Record *x* and the current view in the spinbox window, for use with a later `scan_dragto` call. This is typically associated with a mouse button press in the widget.

</div>

<div class="method">

scan_dragto(x)

Adjust the view by 10 times the difference between *x* and the *x* passed to the last `scan_mark` call. This is typically associated with mouse motion events, producing the effect of dragging the spinbox at high speed through the window.

</div>

<div class="method">

selection(\*args)

A thin wrapper around the Tk `selection` widget subcommand, used to adjust the selection within the spinbox. It has several forms depending on the first argument, such as `selection('adjust', index)`, `selection('clear')`, `selection('element', ?elem?)`, `selection('from', index)`, `selection('present')`, `selection('range', start, end)` and `selection('to', index)`. The `selection_adjust`, `selection_clear`, `selection_element`, `selection_from`, `selection_present`, `selection_range` and `selection_to` methods wrap these forms.

</div>

<div class="method">

selection_adjust(index)

Locate the end of the selection nearest to the character given by *index* and adjust that end of the selection to be at *index* (including but not going beyond *index*). The other end becomes the anchor point for future `selection_to` calls. If the selection is not currently in the spinbox, a new selection is created to include the characters between *index* and the most recent anchor point, inclusive.

</div>

<div class="method">

selection_clear()

Clear the selection if it is currently in this widget. If the selection is not in this widget, the method has no effect.

> [!NOTE]
> This shadows the inherited `Misc.selection_clear`, which clears the X selection; that method is not available on a `Spinbox`.

</div>

<div class="method">

selection_element(element=None)

Set or get the currently selected element. If *element* (one of `'buttonup'`, `'buttondown'` or `'none'`) is given, that spin button is selected and displayed depressed; otherwise the name of the currently selected element is returned.

</div>

<div class="method">

selection_from(index)

Set the selection anchor point to just before the character given by *index*, without changing the selection itself.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

selection_present()

Return `True` if there are characters selected in the spinbox, `False` otherwise.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

selection_range(start, end)

Set the selection to include the characters starting with the one indexed by *start* and ending with the one just before *end*. If *end* refers to the same character as *start* or an earlier one, the selection is cleared.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

selection_to(index)

Set the selection between *index* and the anchor point. If *index* is before the anchor point, the selection runs from *index* up to but not including the anchor point; if it is after, the selection runs from the anchor point up to but not including *index*; if it is the same, nothing happens. The anchor point is the one set by the most recent `selection_from` or `selection_adjust` call. If the selection is not in this widget, a new selection is created using the most recent anchor point.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

validate()

Force an evaluation of the command given by the *validatecommand* option, independently of the conditions specified by the *validate* option, and return whether the value is considered valid.

<div class="versionadded">

3.16

</div>

</div>

</div>

<div class="Text(master=None, cnf={}, **kw)">

A `!Text` widget displays and edits multi-line text. Portions of the text may be styled with **tags**, particular positions may be annotated with floating **marks**, and arbitrary images and other widgets may be embedded in the text. The widget also provides an unlimited undo/redo mechanism and supports peer widgets that share the same underlying data. Inherits from `Widget`, `XView` and `YView`, so the view can be scrolled horizontally and vertically with `~XView.xview` and `~YView.yview`. Refer to the Tk `text` manual page for the full list of options.

Most of the methods take one or more *index* arguments that identify a position within the text. As described in the Tk `text` manual page, an index is a string consisting of a base, optionally followed by one or more modifiers. The base may be `'line.char'` (line *line*, character *char*, where lines are counted from 1 and characters within a line from 0; `'line.end'` refers to the newline ending the line), `'end'` (the position just after the last newline), the name of a mark, `'tag.first'` or `'tag.last'` (the first character tagged with *tag*, or the position just after the last such character), the name of an embedded image or window, or `@x,y` (the character covering pixel coordinates *x*, *y* in the widget). A modifier such as `'+5 chars'`, `'-3 lines'`, `'linestart'`, `'lineend'`, `'wordstart'` or `'wordend'` adjusts the index relative to its base; several modifiers may be combined and are applied from left to right, for example `'insert wordstart - 1 c'`.

<div class="method">

tk_print()

Print the contents of the text widget using the native print dialog. Requires Tk 8.7/9.0 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

insert(index, chars, \*args)

Insert the string *chars* just before the character at *index* (if *index* is `'end'`, just before the final newline). By default the new text inherits any tags present on both sides of the insertion point. If *args* is given, it consists of alternating *tagList*, *chars* values: the preceding *chars* receives exactly the tags listed (a tag list may be a single tag name or a sequence of names), overriding the surrounding tags.

</div>

<div class="method">

delete(index1, index2=None)

Delete the range of characters from *index1* up to but not including *index2*. If *index2* is omitted, the single character at *index1* is deleted. The widget always keeps a newline as its last character, so a deletion that would remove it is adjusted accordingly.

</div>

<div class="method">

replace(index1, index2, chars, \*args)

Replace the range of characters from *index1* up to but not including *index2* with *chars*. This is equivalent to a `delete` followed by an `insert` at *index1*; *args* is interpreted as for `insert`.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

get(index1, index2=None)

Return the text from *index1* up to but not including *index2* as a string. If *index2* is omitted, return the single character at *index1*. Embedded images and windows are omitted from the result.

</div>

<div class="method">

index(index)

Return the position corresponding to *index* in the canonical `'line.char'` form.

</div>

<div class="method">

compare(index1, op, index2)

Compare the positions of *index1* and *index2* using the relational operator *op*, which must be one of `'<'`, `'<='`, `'=='`, `'>='`, `'>'` or `'!='`, and return the boolean result.

</div>

<div class="method">

count(index1, index2, \*options, return_ints=False)

Count the number of items of the requested kinds between *index1* and *index2*; the count is negative if *index1* is after *index2*. Each of *options* names a kind of item to count: `'chars'`, `'displaychars'`, `'displayindices'`, `'displaylines'`, `'indices'`, `'lines'`, `'xpixels'` or `'ypixels'` (the default, used when no option is given, is `'indices'`). The pseudo-option `'update'` forces any out-of-date layout information to be recalculated before the following options are evaluated. When *return_ints* is true and a single counting option is given, return a plain integer; otherwise return a tuple with one integer per counting option (or `None` if the result is empty).

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.13 Added the *return_ints* parameter.

</div>

</div>

<div class="method">

see(index)

Adjust the view so that the character given by *index* is visible. If it is already visible the method has no effect; if it is a short distance out of view the widget scrolls just enough to bring it to the nearest edge, otherwise it scrolls to center *index* in the window.

</div>

<div class="method">

bbox(index)

Return a tuple `(x, y, width, height)` giving the bounding box, in pixels, of the visible part of the character at *index*, or `None` if that character is not visible on the screen.

This shadows the inherited `!Misc.bbox`; use `~Misc.grid_bbox` for the grid bounding box.

</div>

<div class="method">

dlineinfo(index)

Return a tuple `(x, y, width, height, baseline)` describing the display line that contains *index*: the first four values give the bounding box of the line in pixels and *baseline* gives the offset of the baseline measured down from the top of the area. Return `None` if that display line is not visible on the screen.

</div>

<div class="method">

mark_set(markName, index)

Set the mark named *markName* to the position just before the character at *index*, creating the mark if it does not already exist. A mark created this way has right gravity by default.

</div>

<div class="method">

mark_unset(\*markNames)

Remove each of the marks named in *markNames*. The special `insert` and `current` marks may not be removed.

</div>

<div class="method">

mark_names()

Return a tuple of the names of all marks currently set in the widget.

</div>

<div class="method">

mark_gravity(markName, direction=None)

If *direction* is omitted, return the gravity of mark *markName*, either `'left'` or `'right'`. Otherwise set its gravity to *direction*. The gravity determines on which side of the mark text inserted at the mark's position appears: a mark with right gravity (the default) stays to the right of such text.

</div>

<div class="method">

mark_next(index)

Return the name of the first mark at or after *index*, or `None` if there is none. When *index* is the name of a mark, the search starts just after that mark.

</div>

<div class="method">

mark_previous(index)

Return the name of the last mark at or before *index*, or `None` if there is none. When *index* is the name of a mark, the search starts just before that mark.

</div>

<div class="method">

tag_add(tagName, index1, \*args)

Add the tag *tagName* to the range of characters from *index1* up to but not including the next index in *args*. Further pairs of indices may follow in *args* to tag additional ranges; a trailing single index tags just the character at that index.

</div>

<div class="method">

tag_remove(tagName, index1, index2=None)

Remove the tag *tagName* from the characters from *index1* up to but not including *index2* (or from the single character at *index1* if *index2* is omitted). The tag itself continues to exist even if no characters carry it.

</div>

<div class="method">

tag_delete(\*tagNames)

Delete each of the tags named in *tagNames*, removing them from all characters and discarding their options and bindings.

</div>

<div class="method" data-no-typesetting="">

tag_config(tagName, cnf=None, \*\*kw)

</div>

<div class="method">

tag_configure(tagName, cnf=None, \*\*kw)

Query or modify the configuration options of the tag *tagName*. This mirrors `~Misc.configure`, except that it applies to a tag rather than to the widget as a whole: with no options it returns a dictionary describing the current options, otherwise it sets the given options. Defining a tag this way also gives it a priority higher than any existing tag.

The supported tag options, all controlling the appearance of the tagged text, are:

*font*  
The font to use for the text.

*foreground*  
The color to use for the text.

*background*  
The color to use for the area behind the text.

*fgstipple*, *bgstipple*  
Bitmaps used to stipple the foreground (text) and the background; only well supported on X11.

*borderwidth*  
The width of the border drawn around the text according to *relief* (default `0`).

*relief*  
The 3-D appearance of the text's border: `'flat'` (the default), `'raised'`, `'sunken'`, `'ridge'`, `'groove'` or `'solid'`.

*offset*  
How far the text is raised above (or, if negative, lowered below) the baseline, for superscripts and subscripts.

*underline*  
Whether to underline the text (a boolean).

*underlinefg*  
The color of the underline; it defaults to the text color.

*overstrike*  
Whether to draw a line through the middle of the text.

*overstrikefg*  
The color of the overstrike line; it defaults to the text color.

*elide*  
Whether the text is elided (hidden).

*justify*  
How to justify the first character of a display line: `'left'` (the default), `'right'` or `'center'`.

*wrap*  
How to wrap lines that are too long: `'char'`, `'word'` or `'none'`.

*lmargin1*, *lmargin2*  
The indentation, in pixels, of the first display line of a logical line and of the remaining display lines.

*lmargincolor*  
The color of the left margin area.

*rmargin*  
The right-hand margin, in pixels.

*rmargincolor*  
The color of the right margin area.

*spacing1*, *spacing2*, *spacing3*  
Extra space, in pixels, above the first display line of a logical line, between its display lines, and below its last display line.

*tabs*  
The set of tab stops, in the same form as the widget's *tabs* option.

*tabstyle*  
How tab stops are interpreted: `'tabular'` or `'wordprocessor'`.

*selectbackground*, *selectforeground*  
The background and foreground colors used for the text while it is selected.

> [!NOTE]
> Tk 8.6 added the *lmargincolor*, *overstrikefg*, *rmargincolor*, *selectbackground*, *selectforeground* and *underlinefg* options.

`tag_config` is an alias of `!tag_configure`.

</div>

<div class="method">

tag_cget(tagName, option)

Return the current value of the configuration option *option* for the tag *tagName*.

</div>

<div class="method">

tag_names(index=None)

If *index* is omitted, return a tuple of the names of all tags defined in the widget; otherwise return only the names of the tags applied to the character at *index*. The names are ordered from lowest to highest priority.

</div>

<div class="method">

tag_ranges(tagName)

Return a tuple of indices describing all ranges of text tagged with *tagName*. The result alternates start and end indices, so that elements `2*i` and `2*i+1` bound the *i*-th range.

</div>

<div class="method">

tag_nextrange(tagName, index1, index2=None)

Search forward from *index1* (up to *index2* if given) for the first range of characters tagged with *tagName*, and return a two-element tuple of its start and end indices, or an empty tuple if there is no such range.

</div>

<div class="method">

tag_prevrange(tagName, index1, index2=None)

Search backward from *index1* (down to *index2* if given) for the nearest preceding range of characters tagged with *tagName*, and return a two-element tuple of its start and end indices, or an empty tuple if there is no such range.

</div>

<div class="method">

tag_raise(tagName, aboveThis=None)

Raise the priority of tag *tagName* so that it is just above the priority of *aboveThis*, or to the highest priority of all tags if *aboveThis* is omitted. When the display options of overlapping tags conflict, the higher-priority tag wins.

</div>

<div class="method">

tag_lower(tagName, belowThis=None)

Lower the priority of tag *tagName* so that it is just below the priority of *belowThis*, or to the lowest priority of all tags if *belowThis* is omitted.

</div>

<div class="method">

tag_bind(tagName, sequence, func, add=None)

Bind the event *sequence* on characters tagged with *tagName* to the callback *func*, so that *func* is invoked when that event occurs over such a character. If *add* is true the binding is added alongside any existing bindings for *sequence*, otherwise it replaces them. Works like `~Misc.bind` and returns the identifier of the new binding.

</div>

<div class="method">

tag_unbind(tagName, sequence, funcid=None)

Remove the bindings of the event *sequence* on characters tagged with *tagName*. If *funcid* is given, only that binding (as returned by `tag_bind`) is removed and its callback is unregistered.

<div class="versionchanged">

3.13 If *funcid* is given, only that callback is unbound.

</div>

</div>

<div class="method">

image_create(index, cnf={}, \*\*kw)

Embed an image at *index* and return the name assigned to this image instance, which may then be used as an index or passed to the other `image_*` methods. The options, given in *cnf* and *kw*, include *image* (the Tk image to display), *name* (a base name for the instance), *align*, *padx* and *pady*.

</div>

<div class="method">

image_cget(index, option)

Return the current value of the configuration option *option* for the embedded image at *index*.

</div>

<div class="method">

image_configure(index, cnf=None, \*\*kw)

Query or modify the configuration options of the embedded image at *index*, like `~Misc.configure` but applied to that image.

</div>

<div class="method">

image_names()

Return a tuple of the names of all images embedded in the widget.

> [!NOTE]
> This shadows the inherited `Misc.image_names`, which returns the names of all images in the Tcl interpreter; that method is not available on a `Text`.

</div>

<div class="method">

window_create(index, cnf={}, \*\*kw)

Embed a window (any widget) at *index*. The options, given in *cnf* and *kw*, include *window* (the widget to embed), *create* (a callback that creates the widget on demand), *align*, the boolean *stretch* (whether the widget is stretched vertically to the height of the line), *padx* and *pady*. The embedded widget must be a descendant of the text widget's parent.

</div>

<div class="method">

window_cget(index, option)

Return the current value of the configuration option *option* for the embedded window at *index*.

</div>

<div class="method" data-no-typesetting="">

window_config(index, cnf=None, \*\*kw)

</div>

<div class="method">

window_configure(index, cnf=None, \*\*kw)

Query or modify the configuration options of the embedded window at *index*, like `~Misc.configure` but applied to that window.

`window_config` is an alias of `!window_configure`.

</div>

<div class="method">

window_names()

Return a tuple of the names of all windows embedded in the widget.

</div>

<div class="method">

edit(\*args)

Low-level wrapper around the Tk `edit` widget command that controls the undo/redo mechanism and the modified flag; *args* is the `edit` subcommand and its arguments. The `!edit_\*` methods below are thin wrappers around it and are usually more convenient.

</div>

<div class="method">

edit_modified(arg=None)

If *arg* is omitted, return the current state of the modified flag as a `bool`; the flag is set automatically whenever the text is inserted or deleted. Otherwise set the flag to the boolean *arg*.

</div>

<div class="method">

edit_canundo()

Return `True` if there is an edit action on the undo stack that can be undone, and `False` otherwise.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

edit_canredo()

Return `True` if there is an edit action on the redo stack that can be reapplied, and `False` otherwise.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

edit_undo()

Undo the most recent edit action, that is, all the inserts and deletes recorded on the undo stack since the previous separator, and move it to the redo stack. Raises `TclError` if the undo stack is empty. Has no effect unless the *undo* option is true. Since Tk 9.0, returns a tuple of indices delimiting the ranges of text that were changed.

</div>

<div class="method">

edit_redo()

Reapply the most recently undone edit action, provided no further edits have been made since, and move it back to the undo stack. Raises `TclError` if the redo stack is empty. Has no effect unless the *undo* option is true. Since Tk 9.0, returns a tuple of indices delimiting the ranges of text that were changed.

</div>

<div class="method">

edit_reset()

Clear the undo and redo stacks.

</div>

<div class="method">

edit_separator()

Push a separator onto the undo stack, marking a boundary between edit actions for undo and redo. Has no effect unless the *undo* option is true. Separators are inserted automatically when the *autoseparators* option is true.

</div>

<div class="method">

search(pattern, index, stopindex=None, forwards=None, backwards=None, exact=None, regexp=None, nocase=None, count=None, elide=None, \*, nolinestop=None, strictlimits=None)

Search for *pattern* starting at *index* and return the index of the first character of the first match, or an empty string if there is no match. Searching stops at *stopindex* if given; otherwise it wraps around the ends of the text until the starting position is reached again. The following boolean keyword flags control the search: *forwards* or *backwards* select the direction (forward is the default); *exact* (the default) or *regexp* select literal or regular-expression matching; *nocase* makes the match case-insensitive; *elide* causes hidden text to be searched as well; *nolinestop* (regexp only) lets `.` and `[^` match newlines; and *strictlimits* requires the whole match to lie within *index* and *stopindex*. If *count* is a `Variable`, the number of index positions in the match is stored in it.

<div class="versionchanged">

3.15 Added the *nolinestop* and *strictlimits* parameters.

</div>

</div>

<div class="method">

search_all(pattern, index, stopindex=None, \*, forwards=None, backwards=None, exact=None, regexp=None, nocase=None, count=None, elide=None, nolinestop=None, overlap=None, strictlimits=None)

Like `search`, but find every match in the searched range and return a tuple of the starting indices of all matches (empty if there are none). By default overlapping matches are not reported; passing a true *overlap* returns every match that is not wholly contained in another. If *count* is a `Variable`, it receives a list with one element per match.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

scan_mark(x, y)

Record *x*, *y* and the current view, for use with later `scan_dragto` calls. This is typically bound to a mouse button press in the widget.

</div>

<div class="method">

scan_dragto(x, y)

Scroll the widget by 10 times the difference between *x*, *y* and the coordinates passed to the last `scan_mark` call. This is typically bound to mouse motion events, producing the effect of dragging the text at high speed through the window.

</div>

<div class="method">

debug(boolean=None)

If *boolean* is omitted, return whether internal consistency checks of the B-tree data structure are enabled. Otherwise enable or disable them. The setting is shared by all text widgets and may noticeably slow down widgets holding large amounts of text.

</div>

<div class="method">

dump(index1, index2=None, command=None, \*\*kw)

Return the contents of the widget from *index1* up to but not including *index2* (or just the segment at *index1* if *index2* is omitted), including text and information about marks, tags, images and windows. The result is a list of `(key, value, index)` triples, where *key* is one of `'text'`, `'mark'`, `'tagon'`, `'tagoff'`, `'image'` or `'window'`. By default all kinds are reported; passing any of the keyword arguments *all*, *text*, *mark*, *tag*, *image* or *window* as true restricts the dump to the selected kinds. If *command* is given, it is called once per triple with the three values as arguments and nothing is returned.

</div>

<div class="method">

peer_create(newPathName, cnf={}, \*\*kw)

Create a peer text widget with the path name *newPathName* that shares this widget's underlying data (text, marks, tags, images and the undo stack). Changes made through any peer are reflected in all of them. By default the peer covers the same lines as this widget; standard text options, including *startline* and *endline*, may be given to override this.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

peer_names()

Return a tuple of the path names of this widget's peers, not including the widget itself.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

sync(command=None)

Control the synchronization of the displayed view with the underlying text, which may lag behind when line heights have not yet been computed (for example, for lines that have never been displayed). If *command* is omitted, bring the line metrics up to date immediately by forcing computation of any outdated line heights, and return once they are current. Otherwise schedule *command* to be called, with no arguments, exactly once as soon as all line heights are up to date; if there are no pending calculations, it is called immediately.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

pendingsync()

Return `True` if the line height calculations are not up to date, and `False` otherwise. The `<<WidgetViewSync>>` virtual event fires whenever this state changes, with the *detail* field set to the new value.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

yview_pickplace(\*what)

Adjust the view so that the location given by *what* is visible. This is an obsolete equivalent of `see`, which should be used instead.

</div>

</div>

### Variable classes

<div class="Variable(master=None, value=None, name=None)">

The base class for the Tk variable wrappers. A Tk variable is a value stored in the Tcl interpreter that can be linked to widgets through their *variable* or *textvariable* options (see `coupling-widget-variables`), so that changes propagate both ways: updating the variable updates every widget bound to it, and a user editing such a widget updates the variable.

*master* is the widget whose Tcl interpreter owns the variable; if omitted, the default root window is used. *value* is the initial value; if omitted, a type-specific default is used. *name* is the name of the variable in the Tcl interpreter; if omitted, a unique name of the form `'PY_VARnum'` is generated. If *name* matches an existing variable and *value* is omitted, the existing value is retained.

In most cases you should use one of the typed subclasses below -- `StringVar`, `IntVar`, `DoubleVar` or `BooleanVar` -- rather than `!Variable` directly.

> [!NOTE]
> When a `!Variable` is garbage collected, its Tcl variable is unset. Keep a reference to it for as long as a widget is linked to it, for example by storing it as an attribute rather than in a local variable. Otherwise Tk recreates the Tcl variable to keep the widget working, but it is never unset again, leaking one Tcl variable per dropped wrapper.

<div class="versionchanged">

3.10 Two variables now compare equal (`==`) only when they have the same name, are of the same class, and belong to the same Tcl interpreter.

</div>

<div class="method">

get()

Return the current value of the variable. For the base class the value is returned as a string; the typed subclasses convert it to the appropriate Python type.

</div>

<div class="method" data-no-typesetting="">

initialize(value)

</div>

<div class="method">

set(value)

Set the variable to *value*. `initialize` is an alias of `!set`.

<div class="versionadded">

3.3 The *initialize* spelling.

</div>

</div>

<div class="method">

trace_add(mode, callback)

Register *callback* to be called when the variable is accessed according to *mode*. *mode* is one of the strings `'array'`, `'read'`, `'write'` or `'unset'`, or a list or tuple of such strings.

When triggered, *callback* is called with three arguments: the name of the Tcl variable, an index (or an empty string if the variable is not an element of an array), and the *mode* that triggered the call.

Return the internal name of the registered callback, which can be passed to `trace_remove`.

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

trace_remove(mode, cbname)

Remove a trace callback from the variable. *mode* must match the *mode* that was passed to `trace_add`, and *cbname* is the callback name returned by `trace_add`.

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

trace_info()

Return a list of `(modes, cbname)` pairs describing all traces currently set on the variable, where *modes* is a tuple of mode strings and *cbname* is the internal callback name.

<div class="versionadded">

3.6

</div>

</div>

<div class="method" data-no-typesetting="">

trace(mode, callback)

</div>

<div class="method">

trace_variable(mode, callback)

Register *callback* to be called when the variable is accessed according to *mode*. *mode* is one of the strings `'r'`, `'w'` or `'u'`, for read, write or unset. Return the internal name of the registered callback. `trace` is an alias of `!trace_variable`.

<div class="deprecated">

3.6 Use `trace_add` instead. This method wraps a Tcl feature that was removed in Tcl 9.0.

</div>

</div>

<div class="method">

trace_vdelete(mode, cbname)

Remove the trace callback named *cbname* registered for *mode* with `trace_variable`.

<div class="deprecated">

3.6 Use `trace_remove` instead. This method wraps a Tcl feature that was removed in Tcl 9.0.

</div>

</div>

<div class="method">

trace_vinfo()

Return a list of `(mode, cbname)` pairs for all traces set on the variable with `trace_variable`.

<div class="deprecated">

3.6 Use `trace_info` instead. This method wraps a Tcl feature that was removed in Tcl 9.0.

</div>

</div>

</div>

<div class="StringVar(master=None, value=None, name=None)">

A `Variable` subclass that holds a string. The default value is `''`.

<div class="method">

get()

Return the value of the variable as a `str`.

</div>

</div>

<div class="IntVar(master=None, value=None, name=None)">

A `Variable` subclass that holds an integer. The default value is `0`.

<div class="method">

get()

Return the value of the variable as an `int`.

</div>

</div>

<div class="DoubleVar(master=None, value=None, name=None)">

A `Variable` subclass that holds a float. The default value is `0.0`.

<div class="method">

get()

Return the value of the variable as a `float`.

</div>

<div id="tkinter-numeric-locale">

> [!NOTE]
> A floating-point value is always parsed with a period (`.`) as the decimal separator, but `Spinbox`, `Scale` and `ttk.Spinbox <tkinter.ttk.Spinbox>` format it according to the `LC_NUMERIC` locale. Under a locale that uses a comma they produce a value that `get` cannot read, raising `TclError`. Set `LC_NUMERIC` to a locale that uses a period (such as `'C'`) to avoid this.

</div>

</div>

<div class="BooleanVar(master=None, value=None, name=None)">

A `Variable` subclass that holds a boolean. The default value is `False`.

<div class="method">

get()

Return the value of the variable as a `bool`. Raise a `ValueError` if the value cannot be interpreted as a boolean.

</div>

<div class="method" data-no-typesetting="">

initialize(value)

</div>

<div class="method">

set(value)

Set the variable to *value*, converting it to a boolean. `initialize` is an alias of `!set`.

<div class="versionadded">

3.3 The *initialize* spelling.

</div>

</div>

</div>

### Image classes

<div class="Image(imgtype, name=None, cnf={}, master=None, **kw)">

Base class for Tk images. *imgtype* is the Tk image type, one of `'photo'` or `'bitmap'`. An image is a named object that can be displayed by widgets through their *image* option; deleting all references to the `!Image` object deletes the underlying Tk image. Usually you create a `PhotoImage` or `BitmapImage` rather than an `!Image` directly.

The image's configuration options are given by *cnf* and *kw* and may be queried and changed later with the mapping protocol (using `image[key]`) or with the `configure` method.

<div class="method" data-no-typesetting="">

config(\*\*kw)

</div>

<div class="method">

configure(\*\*kw)

Modify one or more configuration options of the image. The valid options depend on the image type; see `PhotoImage` and `BitmapImage`. `config` is an alias of `!configure`.

</div>

<div class="method">

height()

Return the height of the image, in pixels.

</div>

<div class="method">

width()

Return the width of the image, in pixels.

</div>

<div class="method">

type()

Return the type of the image, that is the value of *imgtype* with which it was created (for example `'photo'` or `'bitmap'`).

</div>

</div>

<div class="PhotoImage(name=None, cnf={}, master=None, **kw)">

A full-color image (the Tk `photo` image type), stored internally with a varying degree of transparency per pixel. It can read and write GIF, PPM/PGM and (in Tk 8.6 and later) PNG files, read SVG files (in Tk 9.0 and later), and be drawn in widgets. Inherits from `Image`.

The configuration options include *data* (the image contents as a string), *file* (the name of a file to read the contents from), *format* (the name of the file format handler), *width* and *height* (the size of the image, used when building it up piece by piece), *gamma* and *palette*.

<div class="method">

blank()

Blank the image; that is, set the entire image to have no data, so that it is displayed as transparent and the background of whatever window it is displayed in shows through.

</div>

<div class="method">

redither()

Recalculate the dithered image in each window where it is displayed. This is useful when the image data was supplied in pieces, in which case the dithered image may not be exactly correct.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cget(option)

Return the current value of the configuration option *option*.

</div>

<div class="method">

copy(\*, from_coords=None, zoom=None, subsample=None)

Return a new `PhotoImage` with a copy of this image.

*from_coords* specifies a rectangular sub-region of the source image to be copied. It must be a tuple or a list of 1 to 4 integers `(x1, y1, x2, y2)`. `(x1, y1)` and `(x2, y2)` specify diagonally opposite corners of the rectangle. If *x2* and *y2* are not specified, they default to the bottom-right corner of the source image. The pixels copied include the left and top edges of the rectangle but not the bottom or right edges. If *from_coords* is not given, the whole source image is copied.

If *zoom* or *subsample* are specified, the image is transformed as in the `zoom` or `subsample` methods. The value must be a single integer or a pair of integers.

<div class="versionchanged">

3.13 Added the *from_coords*, *zoom* and *subsample* parameters.

</div>

</div>

<div class="method">

copy_replace(sourceImage, \*, from_coords=None, to=None, shrink=False, zoom=None, subsample=None, compositingrule=None)

Copy a region from *sourceImage* (which must be a `PhotoImage`) into this image, possibly with pixel zooming and/or subsampling. If no options are specified, the whole of *sourceImage* is copied into this image, starting at coordinates `(0, 0)`.

*from_coords* specifies a rectangular sub-region of the source image to be copied, as in the `copy` method.

*to* specifies a rectangular sub-region of the destination image to be affected. It must be a tuple or a list of 1 to 4 integers `(x1, y1, x2, y2)`. If *x2* and *y2* are not specified, they default to `(x1, y1)` plus the size of the source region (after subsampling and zooming, if specified). If *x2* and *y2* are specified, the source region is replicated if necessary to fill the destination region in a tiled fashion.

If *shrink* is true, the size of the destination image is reduced, if necessary, so that the region being copied into is at the bottom-right corner of the image.

If *zoom* or *subsample* are specified, the image is transformed as in the `zoom` or `subsample` methods. The value must be a single integer or a pair of integers.

*compositingrule* specifies how transparent pixels in the source image are combined with the destination image. With `'overlay'` (the default), the old contents of the destination image remain visible, as if the source image were printed on a piece of transparent film and placed over the top of the destination. With `'set'`, the old contents of the destination image are discarded and the source image is used as-is.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

data(format=None, \*, from_coords=None, background=None, grayscale=False, metadata=None)

Return the image data.

*format* specifies the name of the image file format handler to use. If it is not given, the data is returned as a tuple (one element per row) of strings containing space-separated (one element per pixel/column) colors in `#RRGGBB` format.

*from_coords* specifies a rectangular region of the image to be returned. It must be a tuple or a list of 1 to 4 integers `(x1, y1, x2, y2)`. If only *x1* and *y1* are specified, the region extends from `(x1, y1)` to the bottom-right corner of the image. If all four coordinates are given, they specify diagonally opposite corners of the region, including `(x1, y1)` and excluding `(x2, y2)`. If *from_coords* is not given, the whole image is returned.

If *background* is specified, the data does not contain any transparency information; in all transparent pixels the color is replaced by the specified color.

If *grayscale* is true, the data does not contain color information; all pixel data is transformed into grayscale.

*metadata* is a dictionary passed to the image format driver. It requires Tcl/Tk 9.0 or newer.

<div class="versionadded">

3.13

</div>

<div class="versionchanged">

3.16 Added the *metadata* parameter.

</div>

</div>

<div class="method">

get(x, y, \*, withalpha=False)

Return the color of the pixel at coordinates (*x*, *y*) as an `(r, g, b)` tuple of three integers between 0 and 255, representing the red, green and blue components respectively. If *withalpha* is true, the returned tuple has a fourth element giving the alpha (opacity) value of the pixel.

<div class="versionchanged">

3.16 Added the *withalpha* parameter, which requires Tcl/Tk 9.0 or newer.

</div>

</div>

<div class="method">

put(data, to=None, \*, format=None, metadata=None)

Set pixels of the image to the colors given in *data*, which must be a string or a nested sequence of horizontal rows of pixel colors (for example `"{red green} {blue yellow}"`).

*to* specifies the coordinates of the region of the image into which the data are copied. It must be a tuple or a list of 2 or 4 integers `(x1, y1)` or `(x1, y1, x2, y2)` giving the top-left corner, and optionally the bottom-right corner, of the region. The default position is `(0, 0)`.

*format* specifies the format of the image *data*, so that only image file format handlers whose names begin with it are tried.

*metadata* is a dictionary passed to the image format driver. It requires Tcl/Tk 9.0 or newer.

<div class="versionchanged">

3.16 Added the *format* and *metadata* parameters.

</div>

</div>

<div class="method">

read(filename, format=None, \*, from_coords=None, to=None, shrink=False, metadata=None)

Read image data from the file named *filename* into the image.

*format* specifies the format of the image data in the file.

*metadata* is a dictionary passed to the image format driver. It requires Tcl/Tk 9.0 or newer.

*from_coords* specifies a rectangular sub-region of the image file data to be copied to the destination image. It must be a tuple or a list of 1 to 4 integers `(x1, y1, x2, y2)`. If only *x1* and *y1* are specified, the region extends from `(x1, y1)` to the bottom-right corner of the image in the file. If all four coordinates are given, they specify diagonally opposite corners of the region. If *from_coords* is not given, the whole of the image in the file is read.

*to* specifies the coordinates of the top-left corner of the region of the image into which the data are read. The default is `(0, 0)`.

If *shrink* is true, the size of the image is reduced, if necessary, so that the region into which the file data are read is at the bottom-right corner of the image.

<div class="versionadded">

3.13

</div>

<div class="versionchanged">

3.16 Added the *metadata* parameter.

</div>

</div>

<div class="method">

subsample(x, y='', \*, from_coords=None)

Return a new `PhotoImage` based on this image but using only every *x*-th pixel in the X direction and every *y*-th pixel in the Y direction. If *y* is not given, it defaults to the same value as *x*.

*from_coords* specifies a rectangular sub-region of the source image to be copied, as in the `copy` method.

<div class="versionchanged">

3.13 Added the *from_coords* parameter.

</div>

</div>

<div class="method">

transparency_get(x, y)

Return `True` if the pixel at coordinates (*x*, *y*) is fully transparent, `False` otherwise.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

transparency_set(x, y, boolean)

Make the pixel at coordinates (*x*, *y*) fully transparent if *boolean* is true, fully opaque otherwise.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

write(filename, format=None, from_coords=None, \*, background=None, grayscale=False, metadata=None)

Write image data from the image to the file named *filename*.

*format* specifies the name of the image file format handler to use. If it is not given, the format is guessed from the file extension.

*from_coords* specifies a rectangular region of the image to be written. It must be a tuple or a list of 1 to 4 integers `(x1, y1, x2, y2)`. If only *x1* and *y1* are specified, the region extends from `(x1, y1)` to the bottom-right corner of the image. If all four coordinates are given, they specify diagonally opposite corners of the region. If *from_coords* is not given, the whole image is written.

If *background* is specified, the data does not contain any transparency information; in all transparent pixels the color is replaced by the specified color.

If *grayscale* is true, the data does not contain color information; all pixel data is transformed into grayscale.

*metadata* is a dictionary passed to the image format driver. It requires Tcl/Tk 9.0 or newer.

<div class="versionchanged">

3.13 Added the *background* and *grayscale* parameters.

</div>

<div class="versionchanged">

3.16 Added the *metadata* parameter.

</div>

</div>

<div class="method">

zoom(x, y='', \*, from_coords=None)

Return a new `PhotoImage` with this image magnified by a factor of *x* in the X direction and *y* in the Y direction. If *y* is not given, it defaults to the same value as *x*.

*from_coords* specifies a rectangular sub-region of the source image to be copied, as in the `copy` method.

<div class="versionchanged">

3.13 Added the *from_coords* parameter.

</div>

</div>

</div>

<div class="BitmapImage(name=None, cnf={}, master=None, **kw)">

A two-color image (the Tk `bitmap` image type) created from an X11 bitmap. Each pixel displays a foreground color, a background color, or nothing (producing a transparent effect). Inherits from `Image`.

The configuration options are *data* or *file* (the source bitmap, given as a string in X11 bitmap format or as the name of a file in that format), *maskdata* or *maskfile* (the mask bitmap, in the same forms), and *foreground* and *background* (the two colors). For pixels where the mask is zero the image displays nothing; for other pixels it displays the foreground color where the source is one and the background color where the source is zero. If *background* is set to an empty string, the background pixels are transparent.

`!BitmapImage` has no methods of its own beyond those inherited from `Image`.

</div>

### Other classes

<div class="Event()">

A container for the attributes of an event passed to a callback bound with `Misc.bind`. An `!Event` instance has the following attributes, each corresponding to a field of the underlying Tk event; depending on the event type, some attributes may be set to the string `'??'` to indicate that they are not meaningful. See `bindings-and-events`.

<div class="attribute">

serial

The serial number of the event.

</div>

<div class="attribute">

num

The mouse button that was pressed or released (for button events).

</div>

<div class="attribute">

focus

Whether the window has the focus (for `Enter` and `Leave` events).

</div>

<div class="attribute">

height width

The new height and width of the window (for `Configure` and `Expose` events).

</div>

<div class="attribute">

keycode

The keycode of the key that was pressed or released.

</div>

<div class="attribute">

state

The state of the event, as a number (for most events) or a string (for `Visibility` events).

</div>

<div class="attribute">

time

The timestamp of the event, in milliseconds.

</div>

<div class="attribute">

x y

The pointer position relative to the widget, in pixels.

</div>

<div class="attribute">

x_root y_root

The pointer position relative to the top-left corner of the screen, in pixels.

</div>

<div class="attribute">

char

The character typed, as a string (for key events).

</div>

<div class="attribute">

send_event

`True` if the event was sent by another application.

</div>

<div class="attribute">

keysym

The symbolic name of the key that was pressed or released.

</div>

<div class="attribute">

keysym_num

The numeric value of `keysym`.

</div>

<div class="attribute">

type

The `EventType` of the event.

</div>

<div class="attribute">

widget

The widget on which the event occurred.

</div>

<div class="attribute">

delta

The amount the mouse wheel was rotated (for `MouseWheel` events).

</div>

<div class="attribute">

user_data

The data string of a virtual event, as passed to the *data* option of `Misc.event_generate`. It is `'??'` for non-virtual events.

<div class="versionadded">

3.15

</div>

</div>

<div class="attribute">

detail

A fixed detail string for `Enter`, `Leave`, `FocusIn`, `FocusOut` and `ConfigureRequest` events (see the Tcl/Tk documentation). It is `'??'` for other events.

<div class="versionadded">

3.15

</div>

</div>

</div>

<div class="EventType(*values)">

An `enum.StrEnum` enumerating the Tk event types, used as the value of `Event.type`. Its members include, among others, `KeyPress`, `KeyRelease`, `ButtonPress`, `ButtonRelease`, `Motion`, `Enter`, `Leave`, `FocusIn`, `FocusOut`, `Configure`, `Map`, `Unmap`, `Expose`, `Destroy` and `MouseWheel`.

<div class="versionadded">

3.6

</div>

</div>

<div class="CallWrapper(func, subst, widget)">

Internal helper that wraps a Python callback so that it can be invoked from Tcl. *func* is the Python function, *subst* is an optional function that pre-processes the Tcl arguments, and *widget* is the widget used for error reporting. Instances are created automatically by `Misc.register`; this class is not normally used directly.

</div>

### Module-level functions

<div class="function">

Tcl(screenName=None, baseName=None, className='Tk', useTk=False)

The `Tcl` function is a factory function which creates an object much like that created by the `Tk` class, except that it does not initialize the Tk subsystem. This is most often useful when driving the Tcl interpreter in an environment where one doesn't want to create extraneous toplevel windows, or where one cannot (such as Unix/Linux systems without an X server). An object created by the `Tcl` object can have a Toplevel window created (and the Tk subsystem initialized) by calling its `~Tk.loadtk` method.

</div>

<div class="function">

NoDefaultRoot()

Inhibit the creation of an implicit default root window. Afterwards `!tkinter` no longer creates a shared default root automatically, and operations that rely on one --- such as constructing a widget without an explicit *master* --- raise a `RuntimeError`. Call this early in larger applications to make the root window explicit.

</div>

<div class="function">

mainloop(n=0)

Run the Tk main event loop on the default root window until all windows are destroyed. Equivalent to calling `Misc.mainloop` on the default root.

</div>

<div class="function">

getboolean(s)

Convert the Tcl boolean string *s* (one of `'1'`, `'true'`, `'yes'`, `'on'` and similar, or their false counterparts) to a Python `bool`. Raise `ValueError` for an invalid value.

</div>

<div class="function">

getdouble(s)

Convert *s* to a floating-point number. This is the built-in `float`.

</div>

<div class="function">

getint(s)

Convert *s* to an integer. This is the built-in `int`.

</div>

<div class="function">

image_names()

Return the names of all existing images in the default root's interpreter.

</div>

<div class="function">

image_types()

Return the available image types (such as `'photo'` and `'bitmap'`) in the default root's interpreter.

</div>

### File handlers

Tk allows you to register and unregister a callback function which will be called from the Tk mainloop when I/O is possible on a file descriptor. Only one handler may be registered per file descriptor. Example code:

    import tkinter
    widget = tkinter.Tk()
    mask = tkinter.READABLE | tkinter.WRITABLE
    widget.tk.createfilehandler(file, mask, callback)
    ...
    widget.tk.deletefilehandler(file)

This feature is not available on Windows.

Since you don't know how many bytes are available for reading, you may not want to use the `~io.BufferedIOBase` or `~io.TextIOBase` `~io.BufferedIOBase.read` or `~io.IOBase.readline` methods, since these will insist on reading a predefined number of bytes. For sockets, the `~socket.socket.recv` or `~socket.socket.recvfrom` methods will work fine; for other files, use raw reads or `os.read(file.fileno(), maxbytecount)`.

<div class="method">

Widget.tk.createfilehandler(file, mask, func)

Registers the file handler callback function *func*. The *file* argument may either be an object with a `~io.IOBase.fileno` method (such as a file or socket object), or an integer file descriptor. The *mask* argument is an ORed combination of any of the three constants below. The callback is called as follows:

    callback(file, mask)

</div>

<div class="method">

Widget.tk.deletefilehandler(file)

Unregisters a file handler.

</div>

<div class="data">

READABLE WRITABLE EXCEPTION

Constants used in the *mask* arguments.

</div>

### Constants

The following symbolic constants are available in both the `!tkinter` and `!tkinter.constants` namespaces.

<div class="data">

TRUE YES ON

Truthy values, all equal to the integer `1`.

</div>

<div class="data">

FALSE NO OFF

Falsy values, all equal to the integer `0`.

</div>

<div class="data">

N S E W NE NW SE SW NS EW NSEW CENTER

Compass directions (`'n'`, `'s'`, `'e'`, `'w'` and the diagonals and edges) plus `CENTER` (`'center'`), used as values for the *anchor* and *sticky* options and by methods such as `Misc.grid_anchor`.

</div>

<div class="data">

LEFT RIGHT TOP BOTTOM

Sides for the *side* option of the packer (see `Pack.pack_configure`).

</div>

<div class="data">

X Y BOTH NONE

Values for the *fill* option of the packer: `'x'`, `'y'`, `'both'` or `'none'`.

</div>

<div class="data">

RAISED SUNKEN FLAT RIDGE GROOVE SOLID

Values for the *relief* option, which controls a widget's 3-D border.

</div>

<div class="data">

HORIZONTAL VERTICAL

Values for the *orient* option of widgets such as `Scale`, `Scrollbar` and `PanedWindow`.

</div>

<div class="data">

CHAR WORD

Values for the *wrap* option of the `Text` widget, selecting line wrapping on character or word boundaries.

</div>

<div class="data">

BASELINE

The text-alignment value `'baseline'`.

</div>

<div class="data">

INSIDE OUTSIDE

Values for the *bordermode* option of the placer (see `Place.place_configure`).

</div>

<div class="data">

INSERT CURRENT END ANCHOR SEL SEL_FIRST SEL_LAST

Symbolic indices used by the `Text`, `Entry`, `Listbox` and `Canvas` widgets, such as `'insert'` (the insertion cursor), `'current'`, `'end'`, `'anchor'` and the bounds of the selection (`'sel.first'` and `'sel.last'`).

</div>

<div class="data">

ALL

The special tag `'all'`, which matches every item of a `Canvas` or every character of a `Text` (for example `canvas.delete(ALL)`).

</div>

<div class="data">

NORMAL DISABLED ACTIVE HIDDEN

Values for the *state* option of various widgets and items.

</div>

<div class="data">

CASCADE CHECKBUTTON COMMAND RADIOBUTTON SEPARATOR

Menu entry types, used as the *itemType* argument of `Menu.add` and `Menu.insert`.

</div>

<div class="data">

SINGLE BROWSE MULTIPLE EXTENDED

Values for the *selectmode* option of the `Listbox` widget.

</div>

<div class="data">

PIESLICE CHORD ARC

Values for the *style* option of `Canvas` arc items.

</div>

<div class="data">

BUTT PROJECTING ROUND BEVEL MITER

Values for the *capstyle* (`'butt'`, `'projecting'`, `'round'`) and *joinstyle* (`'round'`, `'bevel'`, `'miter'`) options of `Canvas` line items.

</div>

<div class="data">

FIRST LAST

Values for the *arrow* option of `Canvas` line items, indicating which ends have arrowheads.

</div>

<div class="data">

MOVETO SCROLL

The first argument passed by a `Scrollbar` to the `XView.xview` or `YView.yview` method of the scrolled widget.

</div>

<div class="data">

UNITS PAGES

Values for the *what* argument of `XView.xview_scroll` and `YView.yview_scroll`.

</div>

<div class="data">

UNDERLINE NUMERIC DOTBOX

Other option values: `'underline'`, `'numeric'` and `'dotbox'`.

</div>

### Version information

<div class="data">

TCL_VERSION_INFO TK_VERSION_INFO

The versions of the Tcl and Tk libraries that were used for building the `!_tkinter` module, as named tuples with the same five fields as `sys.version_info`: *major*, *minor*, *micro*, *releaselevel* and *serial*. *releaselevel* is `'alpha'`, `'beta'` or `'final'`. Converting them to a string gives the version in the usual Tcl/Tk notation, for example `'9.0.3'` for a final release or `'9.1b2'` for a pre-release. These may be different from the libraries actually used at runtime, which are available as `Misc.info_patchlevel` (the Tcl version) and the `tk_patchLevel` Tcl variable.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

TkVersion TclVersion

The major and minor version of the Tk and Tcl libraries that were used for building the `!_tkinter` module, as floats, for example `8.6`. Prefer `TK_VERSION_INFO` and `TCL_VERSION_INFO`, which are more precise and compare correctly.

</div>

# `!tkinter.messagebox` --- Tkinter message prompts

<div class="module" data-synopsis="Various types of alert dialogs">

tkinter.messagebox

</div>

**Source code:** `Lib/tkinter/messagebox.py`

------------------------------------------------------------------------

The `!tkinter.messagebox` module provides a template base class as well as a variety of convenience methods for commonly used configurations. The message boxes are modal: each blocks until the user responds, then returns a value that depends on the function. The `show*` functions and `Message.show` return the symbolic name of the button the user pressed, as a string (such as `OK` or `YES`). Common message box styles and layouts include but are not limited to:

<figure>
<img src="tk_msg.png" />
</figure>

<div class="Message(master=None, **options)">

Create a message window with an application-specified message, an icon and a set of buttons. Each of the buttons in the message window is identified by a unique symbolic name (see the *type* options).

The following options are supported:

> *command*  
> Specifies the function to invoke when the user closes the dialog. The name of the button clicked by the user to close the dialog is passed as argument. This is only available on macOS.
>
> *default*  
> Gives the `symbolic name <messagebox-buttons>` of the default button for this message window (`OK`, `CANCEL`, and so on). If this option is not specified, the first button in the dialog will be made the default.
>
> *detail*  
> Specifies an auxiliary message to the main message given by the *message* option. The message detail will be presented beneath the main message and, where supported by the OS, in a less emphasized font than the main message.
>
> *icon*  
> Specifies an `icon <messagebox-icons>` to display. If this option is not specified, then the `INFO` icon will be displayed.
>
> *message*  
> Specifies the message to display in this message box. The default value is an empty string.
>
> *parent*  
> Makes the specified window the logical parent of the message box. The message box is displayed on top of its parent window.
>
> *title*  
> Specifies a string to display as the title of the message box. This option is ignored on macOS, where platform guidelines forbid the use of a title on this kind of dialog.
>
> *type*  
> Arranges for a `predefined set of buttons <messagebox-types>` to be displayed.

> [!NOTE]
> Tk 8.6 added the *command* option.

<div class="method">

show(\*\*options)

Display a message window and wait for the user to select one of the buttons. Then return the symbolic name of the selected button. Keyword arguments can override options specified in the constructor.

</div>

</div>

**Information message box**

<div class="function">

showinfo(title=None, message=None, \*\*options)

Creates and displays an information message box with the specified title and message.

</div>

**Warning message boxes**

<div class="function">

showwarning(title=None, message=None, \*\*options)

Creates and displays a warning message box with the specified title and message.

</div>

<div class="function">

showerror(title=None, message=None, \*\*options)

Creates and displays an error message box with the specified title and message.

</div>

**Question message boxes**

<div class="function">

askquestion(title=None, message=None, *, type=YESNO,*\*options)

Ask a question. By default shows buttons `YES` and `NO`. Returns the symbolic name of the selected button.

</div>

<div class="function">

askokcancel(title=None, message=None, \*\*options)

Ask if operation should proceed. Shows buttons `OK` and `CANCEL`. Returns `True` if the answer is ok and `False` otherwise.

</div>

<div class="function">

askretrycancel(title=None, message=None, \*\*options)

Ask if operation should be retried. Shows buttons `RETRY` and `CANCEL`. Return `True` if the answer is retry and `False` otherwise.

</div>

<div class="function">

askyesno(title=None, message=None, \*\*options)

Ask a question. Shows buttons `YES` and `NO`. Returns `True` if the answer is yes and `False` otherwise.

</div>

<div class="function">

askyesnocancel(title=None, message=None, \*\*options)

Ask a question. Shows buttons `YES`, `NO` and `CANCEL`. Return `True` if the answer is yes, `None` if cancelled, and `False` otherwise.

</div>

<div id="messagebox-buttons">

Symbolic names of buttons:

</div>

<div class="data" value="'abort'">

ABORT

</div>

<div class="data" value="'retry'">

RETRY

</div>

<div class="data" value="'ignore'">

IGNORE

</div>

<div class="data" value="'ok'">

OK

</div>

<div class="data" value="'cancel'">

CANCEL

</div>

<div class="data" value="'yes'">

YES

</div>

<div class="data" value="'no'">

NO

</div>

<div id="messagebox-types">

Predefined sets of buttons:

</div>

<div class="data" value="'abortretryignore'">

ABORTRETRYIGNORE

Displays three buttons whose symbolic names are `ABORT`, `RETRY` and `IGNORE`.

</div>

<div class="data" value="'ok'" data-noindex="">

OK

Displays one button whose symbolic name is `OK`.

</div>

<div class="data" value="'okcancel'">

OKCANCEL

Displays two buttons whose symbolic names are `OK` and `CANCEL`.

</div>

<div class="data" value="'retrycancel'">

RETRYCANCEL

Displays two buttons whose symbolic names are `RETRY` and `CANCEL`.

</div>

<div class="data" value="'yesno'">

YESNO

Displays two buttons whose symbolic names are `YES` and `NO`.

</div>

<div class="data" value="'yesnocancel'">

YESNOCANCEL

Displays three buttons whose symbolic names are `YES`, `NO` and `CANCEL`.

</div>

<div id="messagebox-icons">

Icon images:

</div>

<div class="data" value="'error'">

ERROR

</div>

<div class="data" value="'info'">

INFO

</div>

<div class="data" value="'question'">

QUESTION

</div>

<div class="data" value="'warning'">

WARNING

</div>

# `!tkinter.scrolledtext` --- Scrolled text widget

<div class="module" data-synopsis="Text widget with a vertical scroll bar.">

tkinter.scrolledtext

</div>

**Source code:** `Lib/tkinter/scrolledtext.py`

------------------------------------------------------------------------

The `!tkinter.scrolledtext` module provides a class of the same name which implements a basic text widget which has a vertical scroll bar configured to do the "right thing." Using the `ScrolledText` class is a lot easier than setting up a text widget and scroll bar directly.

The text widget and scrollbar are packed together in a `~tkinter.Frame`, and the methods of the `~tkinter.Pack`, `~tkinter.Grid` and `~tkinter.Place` geometry managers are acquired from the `~tkinter.Frame` object. This allows the `ScrolledText` widget to be used directly to achieve most normal geometry management behavior.

Should more specific control be necessary, the following attributes are available:

<div class="ScrolledText(master=None, *, use_ttk=False, **kw)">

The keyword arguments are passed to the `~tkinter.Text` widget.

When *use_ttk* is true, the surrounding frame and the scroll bar are the themed `tkinter.ttk` widgets; the default is the classic `tkinter` widgets.

<div class="versionchanged">

3.16 Added the *use_ttk* parameter.

</div>

<div class="attribute">

frame

The frame which surrounds the text and scroll bar widgets.

</div>

<div class="attribute">

vbar

The scroll bar widget.

</div>

</div>

# `!tkinter.systray` --- System tray icon and notifications

<div class="module" data-synopsis="System tray icon and desktop notifications">

tkinter.systray

</div>

**Source code:** `Lib/tkinter/systray.py`

<div class="versionadded">

3.16

</div>

------------------------------------------------------------------------

The `!tkinter.systray` module provides the `SysTrayIcon` class as an interface to the system tray (or taskbar) icon, and the `notify` function which sends a desktop notification. They require Tk 8.7/9.0 or newer.

Only one system tray icon is supported per Tcl interpreter.

<div class="SysTrayIcon(master=None, *, exists=False, **options)">

The class implementing the system tray icon.

With *exists* false (the default), a new icon is created; creating a second one raises `~tkinter.TclError`. With *exists* true, the instance refers to the already-existing icon instead of creating one, reconfiguring it with any given options.

The supported configuration options are:

- *image* --- the image displayed in the system tray (required when creating an icon). On Windows, it must be a `!PhotoImage`.
- *text* --- the text displayed in the tooltip of the icon.
- *button1* --- a callback that is called without arguments when the icon is clicked with the left mouse button.
- *button3* --- a callback that is called without arguments when the icon is clicked with the right mouse button.

<div class="method">

configure(**options) config(**options)

Query or modify the options of the system tray icon. With no arguments, return a dict of all option values. With a string argument, return the value of that option. Otherwise, set the given options.

</div>

<div class="method">

cget(option)

Return the value of the given option of the system tray icon.

</div>

<div class="method">

exists()

Return whether the system tray icon exists.

</div>

<div class="method">

destroy()

Destroy the system tray icon. A new icon can be created afterwards.

</div>

<div class="method">

notify(title, message)

Send a desktop notification with the given title and message.

</div>

</div>

<div class="function">

notify(title, message, \*, master=None)

Send a desktop notification with the given title and message without creating a system tray icon first. On Windows, sending a notification requires an existing system tray icon, which is also displayed in the notification; use the `SysTrayIcon.notify` method instead.

</div>

# `!tkinter.ttk` --- Tk themed widgets

<div class="module" data-synopsis="Tk themed widget set">

tkinter.ttk

</div>

**Source code:** `Lib/tkinter/ttk.py`

<div class="index">

single: ttk

</div>

------------------------------------------------------------------------

The `!tkinter.ttk` module provides access to the Tk themed widget set, introduced in Tk 8.5. Its widgets adapt their appearance to the platform's native theme, giving an application a better and more consistent look and feel than the classic `tkinter` widgets, whose appearance is fixed.

The basic idea for `!tkinter.ttk` is to separate, to the extent possible, the code implementing a widget's behavior from the code implementing its appearance.

Ttk widgets are used just like the classic `tkinter` widgets and share the same machinery: the widget hierarchy, the geometry managers, variable coupling and event binding. Those foundational concepts are covered in the `tkinter` documentation and are not repeated here.

<div class="versionadded">

3.1

</div>

<div class="seealso">

[Tk Widget Styling Support (TIP \#48)](https://tip.tcl-lang.org/48.html)  
The Tcl Improvement Proposal that introduced the themed widget styling engine.

</div>

## Using Ttk

To start using Ttk, import its module:

    from tkinter import ttk

To override the basic Tk widgets, the import should follow the Tk import:

    from tkinter import *
    from tkinter.ttk import *

That code causes several `!tkinter.ttk` widgets (`Button`, `Checkbutton`, `Entry`, `Frame`, `Label`, `LabelFrame`, `Menubutton`, `OptionMenu`, `PanedWindow`, `Radiobutton`, `Scale`, `Scrollbar` and `Spinbox`) to automatically replace the Tk widgets.

> [!NOTE]
> Overriding the classic widgets with `from tkinter.ttk import *` is convenient for adapting existing code, but new code is usually clearer if it imports the module as `from tkinter import ttk` and refers to the themed widgets explicitly, such as `ttk.Button`.

This has the direct benefit of using the new widgets which gives a better look and feel across platforms; however, the replacement widgets are not completely compatible. The main difference is that widget options such as `fg`, `bg` and others related to widget styling are no longer present in Ttk widgets. Instead, use the `ttk.Style <Style>` class for improved styling effects.

## Ttk widgets

Ttk comes with 18 widgets, twelve of which already existed in tkinter: `Button`, `Checkbutton`, `Entry`, `Frame`, `Label`, `LabelFrame`, `Menubutton`, `PanedWindow`, `Radiobutton`, `Scale`, `Scrollbar`, and `Spinbox`. The other six are new: `Combobox`, `Notebook`, `Progressbar`, `Separator`, `Sizegrip` and `Treeview`. All of them are subclasses of `Widget`.

Using the Ttk widgets gives the application an improved look and feel. As discussed above, there are differences in how the styling is coded.

Tk code:

    l1 = tkinter.Label(text="Test", fg="black", bg="white")
    l2 = tkinter.Label(text="Test", fg="black", bg="white")

Ttk code:

    style = ttk.Style()
    style.configure("BW.TLabel", foreground="black", background="white")

    l1 = ttk.Label(text="Test", style="BW.TLabel")
    l2 = ttk.Label(text="Test", style="BW.TLabel")

For more information about [TtkStyling](#TtkStyling), see the `Style` class documentation.

## Widget

`ttk.Widget <Widget>` defines standard options and methods supported by Tk themed widgets and is not supposed to be directly instantiated.

### Standard options

All the `!ttk` Widgets accept the following options:

<div class="tabularcolumns">

\|l\|L\|

</div>

| Option | Description |
|----|----|
| class | Specifies the window class. The class is used when querying the option database for the window's other options, to determine the default bindtags for the window, and to select the widget's default layout and style. This option is read-only, and may only be specified when the window is created. |
| cursor | Specifies the mouse cursor to be used for the widget. See the *cursor* option type under `Tk-option-data-types`. If set to the empty string (the default), the cursor is inherited from the parent widget. |
| takefocus | Determines whether the window accepts the focus during keyboard traversal. 0, 1 or an empty string is returned. If 0 is returned, it means that the window should be skipped entirely during keyboard traversal. If 1, it means that the window should receive the input focus as long as it is viewable. And an empty string means that the traversal scripts make the decision about whether or not to focus on the window. |
| style | May be used to specify a custom widget style. |

### Scrollable widget options

The following options are supported by widgets that are controlled by a scrollbar.

<div class="tabularcolumns">

\|l\|L\|

</div>

<table>
<colgroup>
<col style="width: 22%" />
<col style="width: 77%" />
</colgroup>
<thead>
<tr>
<th><p>Option</p></th>
<th><p>Description</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p>xscrollcommand</p></td>
<td><p>Used to communicate with horizontal scrollbars.</p>
<p>When the view in the widget's window changes, the widget calls the <em>xscrollcommand</em> callback.</p>
<p>Usually this option consists of the method <code class="interpreted-text" role="meth">Scrollbar.set &lt;tkinter.Scrollbar.set&gt;</code> of some scrollbar. This will cause the scrollbar to be updated whenever the view in the window changes.</p></td>
</tr>
<tr>
<td><p>yscrollcommand</p></td>
<td><p>Used to communicate with vertical scrollbars. For some more information, see above.</p></td>
</tr>
</tbody>
</table>

### Label options

The following options are supported by labels, buttons and other button-like widgets.

<div class="tabularcolumns">

\|l\|p{0.7linewidth}\|

</div>

<table>
<colgroup>
<col style="width: 20%" />
<col style="width: 80%" />
</colgroup>
<thead>
<tr>
<th><p>Option</p></th>
<th><p>Description</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p>text</p></td>
<td><p>Specifies a text string to be displayed inside the widget.</p></td>
</tr>
<tr>
<td><p>textvariable</p></td>
<td><p>Specifies a name whose value will be used in place of the text option resource.</p></td>
</tr>
<tr>
<td><p>underline</p></td>
<td><p>If set, specifies the index (0-based) of a character to underline in the text string. The underline character is used for mnemonic activation.</p></td>
</tr>
<tr>
<td><p>image</p></td>
<td><p>Specifies an image to display. This is a list of 1 or more elements. The first element is the default image name. The rest of the list if a sequence of statespec/value pairs as defined by <code class="interpreted-text" role="meth">Style.map</code>, specifying different images to use when the widget is in a particular state or a combination of states. All images in the list should have the same size.</p></td>
</tr>
<tr>
<td><p>compound</p></td>
<td><p>Specifies how to display the image relative to the text, in the case both text and images options are present. Valid values are:</p>
<ul>
<li>text: display text only</li>
<li>image: display image only</li>
<li>top, bottom, left, right: display image above, below, left of, or right of the text, respectively.</li>
<li>none: the default. display the image if present, otherwise the text.</li>
</ul></td>
</tr>
<tr>
<td><p>width</p></td>
<td><p>If greater than zero, specifies how much space, in character widths, to allocate for the text label, if less than zero, specifies a minimum width. If zero or unspecified, the natural width of the text label is used.</p></td>
</tr>
</tbody>
</table>

### Compatibility options

<div class="tabularcolumns">

\|l\|L\|

</div>

| Option | Description |
|----|----|
| state | May be set to "normal" or "disabled" to control the "disabled" state bit. This is a write-only option: setting it changes the widget state, but the `Widget.state` method does not affect this option. |

### Widget states

The widget state is a bitmap of independent state flags.

<div class="tabularcolumns">

\|l\|L\|

</div>

| Flag | Description |
|----|----|
| active | The mouse cursor is over the widget and pressing a mouse button will cause some action to occur |
| disabled | Widget is disabled under program control |
| focus | Widget has keyboard focus |
| pressed | Widget is being pressed |
| selected | "On", "true", or "current" for things like Checkbuttons and radiobuttons |
| background | Windows and Mac have a notion of an "active" or foreground window. The *background* state is set for widgets in a background window, and cleared for those in the foreground window |
| readonly | Widget should not allow user modification |
| alternate | A widget-specific alternate display format |
| invalid | The widget's value is invalid |

A state specification is a sequence of state names, optionally prefixed with an exclamation point indicating that the bit is off.

### ttk.Widget

Besides the methods described below, the `ttk.Widget <Widget>` supports the methods `tkinter.Widget.cget <tkinter.Misc.cget>` and `tkinter.Widget.configure <tkinter.Misc.configure>`.

<div class="Widget">

<div class="method">

identify(x, y)

Returns the name of the element at position *x* *y*, or the empty string if the point does not lie within any element.

*x* and *y* are pixel coordinates relative to the widget.

</div>

<div class="method">

instate(statespec, callback=None, *args,*\*kw)

Test the widget's state. If a callback is not specified, returns `True` if the widget state matches *statespec* and `False` otherwise. If callback is specified then it is called with args if widget state matches *statespec*.

</div>

<div class="method">

state(statespec=None)

Modify or inquire widget state. If *statespec* is specified, sets the widget state according to it and return a new *statespec* indicating which flags were changed. If *statespec* is not specified, returns the currently enabled state flags.

Not to be confused with `Wm.state <tkinter.Wm.state>`.

</div>

*statespec* will usually be a list or a tuple.

</div>

## Combobox

The `ttk.Combobox <Combobox>` widget combines a text field with a pop-down list of values. This widget is a subclass of `Entry`.

Besides the methods inherited from `Widget`: `~tkinter.Misc.cget`, `~tkinter.Misc.configure`, `~Widget.identify`, `~Widget.instate` and `~Widget.state`, and the following inherited from `Entry`: `~Entry.bbox`, `~tkinter.Entry.delete`, `~tkinter.Entry.icursor`, `~tkinter.Entry.index`, `~tkinter.Entry.insert`, `selection* <tkinter.Entry.selection_adjust>`, `xview* <tkinter.XView.xview>`, it has some other methods, described at `ttk.Combobox <Combobox>`.

### Options

This widget accepts the following specific options:

<div class="tabularcolumns">

\|l\|L\|

</div>

| Option | Description |
|----|----|
| exportselection | Boolean value. If set, the widget selection is linked to the X selection (which can be returned by invoking Misc.selection_get, for example). |
| justify | Specifies how the text is aligned within the widget. One of "left", "center", or "right". |
| height | Specifies the height of the pop-down listbox, in rows. |
| postcommand | A script (possibly registered with Misc.register) that is called immediately before displaying the values. It may specify which values to display. |
| state | One of "normal", "readonly", or "disabled". In the "readonly" state, the value may not be edited directly, and the user can only select one of the values from the dropdown list. In the "normal" state, the text field is directly editable. In the "disabled" state, no interaction is possible. |
| textvariable | Specifies a name whose value is linked to the widget value. Whenever the value associated with that name changes, the widget value is updated, and vice versa. See `tkinter.StringVar`. |
| values | Specifies the list of values to display in the drop-down listbox. |
| width | Specifies an integer value indicating the desired width of the entry window, in average-size characters of the widget's font. |

> [!NOTE]
> Tk 9.1 added the *locale* option, which selects the locale used to determine word and character boundaries within the text (`"C"` by default).

### Virtual events

The combobox widgets generates a **\<\<ComboboxSelected\>\>** virtual event when the user selects an element from the list of values.

### ttk.Combobox

<div class="Combobox">

<div class="method">

current(newindex=None)

If *newindex* is specified, sets the combobox value to the element position *newindex*. Otherwise, returns the index of the current value or -1 if the current value is not in the values list.

</div>

<div class="method">

get()

Returns the current value of the combobox.

</div>

<div class="method">

set(value)

Sets the value of the combobox to *value*.

</div>

</div>

## Spinbox

The `ttk.Spinbox <Spinbox>` widget is a `ttk.Entry <Entry>` enhanced with increment and decrement arrows. It can be used for numbers or lists of string values. This widget is a subclass of `Entry`. Besides the methods inherited from `Widget`: `~tkinter.Misc.cget`, `~tkinter.Misc.configure`, `~Widget.identify`, `~Widget.instate` and `~Widget.state`, and the following inherited from `Entry`: `~Entry.bbox`, `~tkinter.Entry.delete`, `~tkinter.Entry.icursor`, `~tkinter.Entry.index`, `~tkinter.Entry.insert`, `xview* <tkinter.XView.xview>`, it has some other methods, described at `ttk.Spinbox <Spinbox>`.

### Options

This widget accepts the following specific options:

<div class="tabularcolumns">

\|l\|L\|

</div>

| Option | Description |
|----|----|
| from | Float value. If set, this is the minimum value to which the decrement button will decrement. Must be spelled as `from_` when used as an argument, since `from` is a Python keyword. |
| to | Float value. If set, this is the maximum value to which the increment button will increment. |
| increment | Float value. Specifies the amount which the increment/decrement buttons change the value. Defaults to 1.0. |
| values | Sequence of string or float values. If specified, the increment/decrement buttons will cycle through the items in this sequence rather than incrementing or decrementing numbers. |
| wrap | Boolean value. If `True`, increment and decrement buttons will cycle from the `to` value to the `from` value or the `from` value to the `to` value, respectively. |
| format | String value. This specifies the format of numbers set by the increment/decrement buttons. It must be in the form "%W.Pf", where W is the padded width of the value, P is the precision, and '%' and 'f' are literal. |
| command | Python callable. Will be called with no arguments whenever either of the increment or decrement buttons are pressed. |

### Virtual events

The spinbox widget generates an **\<\<Increment\>\>** virtual event when the user presses \<Up\>, and a **\<\<Decrement\>\>** virtual event when the user presses \<Down\>.

### ttk.Spinbox

<div class="Spinbox">

With a non-integer increment, see `numeric values and the locale
<tkinter-numeric-locale>`.

<div class="versionadded">

3.8

</div>

<div class="method">

get()

Returns the current value of the spinbox.

</div>

<div class="method">

set(value)

Sets the value of the spinbox to *value*.

</div>

</div>

## Notebook

Ttk Notebook widget manages a collection of windows and displays a single one at a time. Each child window is associated with a tab, which the user may select to change the currently displayed window.

### Options

This widget accepts the following specific options:

<div class="tabularcolumns">

\|l\|L\|

</div>

| Option | Description |
|----|----|
| height | If present and greater than zero, specifies the desired height of the pane area (not including internal padding or tabs). Otherwise, the maximum height of all panes is used. |
| padding | Specifies the amount of extra space to add around the outside of the notebook. The padding is a list up to four length specifications left top right bottom. If fewer than four elements are specified, bottom defaults to top, right defaults to left, and top defaults to left. |
| width | If present and greater than zero, specified the desired width of the pane area (not including internal padding). Otherwise, the maximum width of all panes is used. |

### Tab options

There are also specific options for tabs:

<div class="tabularcolumns">

\|l\|L\|

</div>

| Option | Description |
|----|----|
| state | Either "normal", "disabled" or "hidden". If "disabled", then the tab is not selectable. If "hidden", then the tab is not shown. |
| sticky | Specifies how the child window is positioned within the pane area. Value is a string containing zero or more of the characters "n", "s", "e" or "w". Each letter refers to a side (north, south, east or west) that the child window will stick to, as per the `grid <tkinter.Grid.grid>` geometry manager. |
| padding | Specifies the amount of extra space to add between the notebook and this pane. Syntax is the same as for the option padding used by this widget. |
| text | Specifies a text to be displayed in the tab. |
| image | Specifies an image to display in the tab. See the option image described in `Widget`. |
| compound | Specifies how to display the image relative to the text, in the case both options text and image are present. See [Label Options](#label-options) for legal values. |
| underline | Specifies the index (0-based) of a character to underline in the text string. The underlined character is used for mnemonic activation if `Notebook.enable_traversal` is called. |

### Tab identifiers

The tab_id present in several methods of `ttk.Notebook <Notebook>` may take any of the following forms:

- An integer between zero and the number of tabs
- The name of a child window
- A positional specification of the form "@x,y", which identifies the tab
- The literal string "current", which identifies the currently selected tab
- The literal string "end", which returns the number of tabs (only valid for `Notebook.index`)

### Virtual events

This widget generates a **\<\<NotebookTabChanged\>\>** virtual event after a new tab is selected.

### ttk.Notebook

<div class="Notebook">

<div class="method">

add(child, \*\*kw)

Adds a new tab to the notebook.

If window is currently managed by the notebook but hidden, it is restored to its previous position.

See [Tab Options](#tab-options) for the list of available options.

</div>

<div class="method">

forget(tab_id)

Removes the tab specified by *tab_id*, unmaps and unmanages the associated window.

This shadows the inherited geometry-manager `!forget`; use `~tkinter.Pack.pack_forget`, `~tkinter.Grid.grid_forget` or `~tkinter.Place.place_forget` to remove the widget itself from its manager.

</div>

<div class="method">

hide(tab_id)

Hides the tab specified by *tab_id*.

The tab will not be displayed, but the associated window remains managed by the notebook and its configuration remembered. Hidden tabs may be restored with the `add` command.

</div>

<div class="method">

identify(x, y)

Returns the name of the tab element at position *x*, *y*, or the empty string if none.

</div>

<div class="method">

index(tab_id)

Returns the numeric index of the tab specified by *tab_id*, or the total number of tabs if *tab_id* is the string "end".

</div>

<div class="method">

insert(pos, child, \*\*kw)

Inserts a pane at the specified position.

*pos* is either the string "end", an integer index, or the name of a managed child. If *child* is already managed by the notebook, moves it to the specified position.

See [Tab Options](#tab-options) for the list of available options.

</div>

<div class="method">

select(tab_id=None)

Selects the specified *tab_id*.

The associated child window will be displayed, and the previously selected window (if different) is unmapped. If *tab_id* is omitted, returns the widget name of the currently selected pane.

</div>

<div class="method">

tab(tab_id, option=None, \*\*kw)

Query or modify the options of the specific *tab_id*.

If *kw* is not given, returns a dictionary of the tab option values. If *option* is specified, returns the value of that *option*. Otherwise, sets the options to the corresponding values.

</div>

<div class="method">

tabs()

Returns a tuple of windows managed by the notebook.

</div>

<div class="method">

enable_traversal()

Enable keyboard traversal for a toplevel window containing this notebook.

This will extend the bindings for the toplevel window containing the notebook as follows:

- `Control-Tab`: selects the tab following the currently selected one.
- `Shift-Control-Tab`: selects the tab preceding the currently selected one.
- `Alt-K`: where *K* is the mnemonic (underlined) character of any tab, will select that tab.

Multiple notebooks in a single toplevel may be enabled for traversal, including nested notebooks. However, notebook traversal only works properly if all panes are direct children of the notebook.

</div>

</div>

## Progressbar

The `ttk.Progressbar <Progressbar>` widget shows the status of a long-running operation. It can operate in two modes: 1) the determinate mode which shows the amount completed relative to the total amount of work to be done and 2) the indeterminate mode which provides an animated display to let the user know that work is progressing.

### Options

This widget accepts the following specific options:

<div class="tabularcolumns">

\|l\|L\|

</div>

| Option | Description |
|----|----|
| orient | One of "horizontal" or "vertical". Specifies the orientation of the progress bar. |
| length | Specifies the length of the long axis of the progress bar (width if horizontal, height if vertical). |
| mode | One of "determinate" or "indeterminate". |
| maximum | A number specifying the maximum value. Defaults to 100. |
| value | The current value of the progress bar. In "determinate" mode, this represents the amount of work completed. In "indeterminate" mode, it is interpreted as modulo *maximum*; that is, the progress bar completes one "cycle" when its value increases by *maximum*. |
| variable | A name which is linked to the option value. If specified, the value of the progress bar is automatically set to the value of this name whenever the latter is modified. |
| phase | Read-only option. The widget periodically increments the value of this option whenever its value is greater than 0 and, in determinate mode, less than maximum. This option may be used by the current theme to provide additional animation effects. |

### ttk.Progressbar

<div class="Progressbar">

<div class="method">

start(interval=None)

Begin autoincrement mode: schedules a recurring timer event that calls `Progressbar.step` every *interval* milliseconds. If omitted, *interval* defaults to 50 milliseconds.

</div>

<div class="method">

step(amount=None)

Increments the progress bar's value by *amount*.

*amount* defaults to 1.0 if omitted.

</div>

<div class="method">

stop()

Stop autoincrement mode: cancels any recurring timer event initiated by `Progressbar.start` for this progress bar.

</div>

</div>

## Separator

The `ttk.Separator <Separator>` widget displays a horizontal or vertical separator bar.

It has no other methods besides the ones inherited from `ttk.Widget <Widget>`.

### Options

This widget accepts the following specific option:

<div class="tabularcolumns">

\|l\|L\|

</div>

| Option | Description |
|----|----|
| orient | One of "horizontal" or "vertical". Specifies the orientation of the separator. |

## Sizegrip

The `ttk.Sizegrip <Sizegrip>` widget (also known as a grow box) allows the user to resize the containing toplevel window by pressing and dragging the grip.

This widget has neither specific options nor specific methods, besides the ones inherited from `ttk.Widget <Widget>`.

### Platform-specific notes

- On macOS, toplevel windows automatically include a built-in size grip by default. Adding a `Sizegrip` is harmless, since the built-in grip will just mask the widget.

### Bugs

- If the containing toplevel's position was specified relative to the right or bottom of the screen (for example, ....), the `Sizegrip` widget will not resize the window.
- This widget supports only "southeast" resizing.

## Treeview

The `ttk.Treeview <Treeview>` widget displays a hierarchical collection of items. Each item has a textual label, an optional image, and an optional list of data values. The data values are displayed in successive columns after the tree label.

The order in which data values are displayed may be controlled by setting the widget option `displaycolumns`. The tree widget can also display column headings. Columns may be accessed by number or symbolic names listed in the widget option columns. See [Column Identifiers](#column-identifiers).

Each item is identified by a unique name. The widget will generate item IDs if they are not supplied by the caller. There is a distinguished root item, named `{}`. The root item itself is not displayed; its children appear at the top level of the hierarchy.

Each item also has a list of tags, which can be used to associate event bindings with individual items and control the appearance of the item.

The Treeview widget supports horizontal and vertical scrolling, according to the options described in [Scrollable Widget Options](#scrollable-widget-options) and the methods `Treeview.xview` and `Treeview.yview`.

### Options

This widget accepts the following specific options:

<div class="tabularcolumns">

\|l\|p{0.7linewidth}\|

</div>

<table>
<colgroup>
<col style="width: 22%" />
<col style="width: 77%" />
</colgroup>
<thead>
<tr>
<th><p>Option</p></th>
<th><p>Description</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p>columns</p></td>
<td><p>A list of column identifiers, specifying the number of columns and their names.</p></td>
</tr>
<tr>
<td><p>displaycolumns</p></td>
<td><p>A list of column identifiers (either symbolic or integer indices) specifying which data columns are displayed and the order in which they appear, or the string "#all".</p></td>
</tr>
<tr>
<td><p>height</p></td>
<td><p>Specifies the number of rows which should be visible. Note: the requested width is determined from the sum of the column widths.</p></td>
</tr>
<tr>
<td><p>padding</p></td>
<td><p>Specifies the internal padding for the widget. The padding is a list of up to four length specifications.</p></td>
</tr>
<tr>
<td><p>selectmode</p></td>
<td><p>Controls how the built-in class bindings manage the selection. One of "extended", "browse" or "none". If set to "extended" (the default), multiple items may be selected. If "browse", only a single item will be selected at a time. If "none", the selection will not be changed.</p>
<p>Note that the application code and tag bindings can set the selection however they wish, regardless of the value of this option.</p></td>
</tr>
<tr>
<td><p>show</p></td>
<td><p>A list containing zero or more of the following values, specifying which elements of the tree to display.</p>
<ul>
<li>tree: display tree labels in column #0.</li>
<li>headings: display the heading row.</li>
</ul>
<p>The default is "tree headings", that is, show all elements.</p>
<p><strong>Note</strong>: Column #0 always refers to the tree column, even if show="tree" is not specified.</p></td>
</tr>
</tbody>
</table>

> [!NOTE]
> Tk 9.0 added several `Treeview` features. The *selectmode* option gained the values `"single"` and `"multiple"`; the new widget options *selecttype* (`"item"` or `"cell"` selection), *striped* (zebra-striped rows), and *titlecolumns* / *titleitems* (columns or rows frozen against scrolling) were introduced; the column *separator* option was added; and items gained a *hidden* option. Tk 9.1 added the *rowheight* and *headingheight* options.

### Item options

The following item options may be specified for items in the insert and item widget commands.

<div class="tabularcolumns">

\|l\|L\|

</div>

<table>
<colgroup>
<col style="width: 12%" />
<col style="width: 87%" />
</colgroup>
<thead>
<tr>
<th><p>Option</p></th>
<th><p>Description</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p>text</p></td>
<td><p>The textual label to display for the item.</p></td>
</tr>
<tr>
<td><p>image</p></td>
<td><p>A Tk Image, displayed to the left of the label.</p></td>
</tr>
<tr>
<td><p>values</p></td>
<td><p>The list of values associated with the item.</p>
<p>Each item should have the same number of values as the widget option columns. If there are fewer values than columns, the remaining values are assumed empty. If there are more values than columns, the extra values are ignored.</p></td>
</tr>
<tr>
<td><p>open</p></td>
<td><p><code>True</code>/<code>False</code> value indicating whether the item's children should be displayed or hidden.</p></td>
</tr>
<tr>
<td><p>tags</p></td>
<td><p>A list of tags associated with this item.</p></td>
</tr>
</tbody>
</table>

### Tag options

The following options may be specified on tags:

<div class="tabularcolumns">

\|l\|L\|

</div>

| Option | Description |
|----|----|
| foreground | Specifies the text foreground color. |
| background | Specifies the cell or item background color. |
| font | Specifies the font to use when drawing text. |
| image | Specifies the item image, in case the item's image option is empty. |

### Column identifiers

Column identifiers take any of the following forms:

- A symbolic name from the list of columns option.
- An integer n, specifying the nth data column.
- A string of the form \#n, where n is an integer, specifying the nth display column.

Notes:

- Item's option values may be displayed in a different order than the order in which they are stored.
- Column \#0 always refers to the tree column, even if show="tree" is not specified.

A data column number is an index into an item's option values list; a display column number is the column number in the tree where the values are displayed. Tree labels are displayed in column \#0. If option displaycolumns is not set, then data column n is displayed in column \#n+1. Again, **column \#0 always refers to the tree column**.

### Virtual events

The Treeview widget generates the following virtual events.

<div class="tabularcolumns">

\|l\|L\|

</div>

| Event | Description |
|----|----|
| \<\<TreeviewSelect\>\> | Generated whenever the selection changes. |
| \<\<TreeviewOpen\>\> | Generated just before settings the focus item to open=True. |
| \<\<TreeviewClose\>\> | Generated just after setting the focus item to open=False. |

The `Treeview.focus` and `Treeview.selection` methods can be used to determine the affected item or items.

### ttk.Treeview

<div class="Treeview">

<div class="method">

bbox(item, column=None)

Returns the bounding box (relative to the treeview widget's window) of the specified *item* in the form (x, y, width, height).

If *column* is specified, returns the bounding box of that cell. If the *item* is not visible (that is, if it is a descendant of a closed item or is scrolled offscreen), returns an empty string.

This shadows the inherited `!Misc.bbox`; use `~tkinter.Misc.grid_bbox` for the grid bounding box.

</div>

<div class="method">

get_children(item=None)

Returns a tuple of children belonging to *item*.

If *item* is not specified, returns root children.

</div>

<div class="method">

set_children(item, \*newchildren)

Replaces *item*'s children with *newchildren*.

Children present in *item* that are not present in *newchildren* are detached from the tree. No items in *newchildren* may be an ancestor of *item*. Note that not specifying *newchildren* results in detaching *item*'s children.

</div>

<div class="method">

column(column, option=None, \*\*kw)

Query or modify the options for the specified *column*.

If *kw* is not given, returns a dict of the column option values. If *option* is specified then the value for that *option* is returned. Otherwise, sets the options to the corresponding values.

The valid options/values are:

*id*  
Returns the column name. This is a read-only option.

*anchor*: One of the standard Tk anchor values.  
Specifies how the text in this column should be aligned with respect to the cell.

*minwidth*: width  
The minimum width of the column in pixels. The treeview widget will not make the column any smaller than specified by this option when the widget is resized or the user drags a column.

*separator*: `True`/`False`  
Specifies whether a column separator should be drawn to the right of the column.

*stretch*: `True`/`False`  
Specifies whether the column's width should be adjusted when the widget is resized.

*width*: width  
The width of the column in pixels.

To configure the tree column, call this with column = "#0"

</div>

<div class="method">

delete(\*items)

Delete all specified *items* and all their descendants.

The root item may not be deleted.

</div>

<div class="method">

detach(\*items)

Unlinks all of the specified *items* from the tree.

The items and all of their descendants are still present, and may be reinserted at another point in the tree, but will not be displayed.

The root item may not be detached.

</div>

<div class="method">

exists(item)

Returns `True` if the specified *item* is present in the tree, `False` otherwise.

</div>

<div class="method">

focus(item=None)

If *item* is specified, sets the focus item to *item*. Otherwise, returns the current focus item, or '' if there is none.

This shadows the inherited `!Misc.focus`; use `~tkinter.Misc.focus_set` to focus the widget itself.

</div>

<div class="method">

heading(column, option=None, \*\*kw)

Query or modify the heading options for the specified *column*.

If *kw* is not given, returns a dict of the heading option values. If *option* is specified then the value for that *option* is returned. Otherwise, sets the options to the corresponding values.

The valid options/values are:

*text*: text  
The text to display in the column heading.

*image*: imageName  
Specifies an image to display to the right of the column heading.

*anchor*: anchor  
Specifies how the heading text should be aligned. One of the standard Tk anchor values.

*command*: callback  
A callback to be invoked when the heading label is pressed.

To configure the tree column heading, call this with column = "#0".

</div>

<div class="method">

identify(component, x, y)

Returns a description of the specified *component* under the point given by *x* and *y*, or the empty string if no such *component* is present at that position.

</div>

<div class="method">

identify_row(y)

Returns the item ID of the item at position *y*.

</div>

<div class="method">

identify_column(x)

Returns the display column identifier of the cell at position *x*.

The tree column has ID \#0.

</div>

<div class="method">

identify_region(x, y)

Returns one of:

| region    | meaning                             |
|-----------|-------------------------------------|
| heading   | Tree heading area.                  |
| separator | Space between two columns headings. |
| tree      | The tree area.                      |
| cell      | A data cell.                        |

Availability: Tk 8.6.

</div>

<div class="method">

identify_element(x, y)

Returns the element at position *x*, *y*.

Availability: Tk 8.6.

</div>

<div class="method">

index(item)

Returns the integer index of *item* within its parent's list of children.

</div>

<div class="method">

insert(parent, index, iid=None, \*\*kw)

Creates a new item and returns the item identifier of the newly created item.

*parent* is the item ID of the parent item, or the empty string to create a new top-level item. *index* is an integer, or the value "end", specifying where in the list of parent's children to insert the new item. If *index* is less than or equal to zero, the new node is inserted at the beginning; if *index* is greater than or equal to the current number of children, it is inserted at the end. If *iid* is specified, it is used as the item identifier; *iid* must not already exist in the tree. Otherwise, a new unique identifier is generated.

See [Item Options](#item-options) for the list of available options.

</div>

<div class="method">

item(item, option=None, \*\*kw)

Query or modify the options for the specified *item*.

If no options are given, a dict with options/values for the item is returned. If *option* is specified then the value for that option is returned. Otherwise, sets the options to the corresponding values as given by *kw*.

</div>

<div class="method" data-no-typesetting="">

reattach(item, parent, index)

</div>

<div class="method">

move(item, parent, index)

Moves *item* to position *index* in *parent*'s list of children.

It is illegal to move an item under one of its descendants. If *index* is less than or equal to zero, *item* is moved to the beginning; if greater than or equal to the number of children, it is moved to the end. If *item* was detached it is reattached.

`reattach` is an alias of `!move`.

</div>

<div class="method">

next(item)

Returns the identifier of *item*'s next sibling, or '' if *item* is the last child of its parent. Equivalent to `after_item(item, hidden=True, recurse=False)`.

</div>

<div class="method">

parent(item)

Returns the ID of the parent of *item*, or '' if *item* is at the top level of the hierarchy.

</div>

<div class="method">

prev(item)

Returns the identifier of *item*'s previous sibling, or '' if *item* is the first child of its parent. Equivalent to `before_item(item, hidden=True, recurse=False)`.

</div>

<div class="method">

after_item(item, \*, hidden=False, recurse=True)

Returns the identifier of the item after *item* (the first child, a next sibling, or a next sibling of an ancestor), or '' if there is none. By default only visible items are considered; if *hidden* is true, hidden items are included too. If *recurse* is false, only siblings of *item* are considered (see `next`).

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

before_item(item, \*, hidden=False, recurse=True)

Returns the identifier of the item before *item* (a previous sibling or the parent of *item*), or '' if there is none. By default only visible items are considered; if *hidden* is true, hidden items are included too. If *recurse* is false, only siblings of *item* are considered (see `prev`).

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

depth(item)

Returns the number of levels between *item* and the root item.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

haschildren(item)

Returns `True` if *item* has children, `False` otherwise.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

visible(item)

Returns `True` if *item* is visible, `False` otherwise. An item is visible if it is not detached, not hidden, and all of its ancestors are open and not hidden.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

size(item, \*, hidden=False, recurse=False)

Returns the number of children of *item*. If *hidden* is true, hidden items are included. If *recurse* is true, all descendants of *item* are included. Use `''` for the root item. `size(item, hidden=True)` equals `len(get_children(item))` (which always includes hidden items).

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

range(first, last, \*, hidden=False, recurse=True)

Returns a tuple of items from *first* through *last*, inclusive. If *hidden* is true, hidden items are included. If *recurse* is false, descendants and ancestors are excluded.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

identifier(item, index)

Returns the identifier of the item at *index* within *item*'s list of children.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

current()

Returns the current item id and column id as a 2-tuple, or an empty tuple if there is none. The current item is the item under the mouse pointer.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

expand(\*items, recurse=False)

Set all of the specified items to the open state. If *recurse* is true, also open all of their descendants; this requires Tk 9.1. Use `''` for the root item. `expand(item)` is equivalent to `item(item, open=True)`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

collapse(\*items, recurse=False)

Set all of the specified items to the closed state. If *recurse* is true, also close all of their descendants; this requires Tk 9.1. Use `''` for the root item. `collapse(item)` is equivalent to `item(item, open=False)`.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

hide(\*items, recurse=False)

Hide all of the specified items and all of their child items. If *recurse* is true, also hide all of their descendants. Use `''` for the root item. `hide(item)` is equivalent to `item(item, hidden=True)`.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

unhide(\*items, recurse=False)

Unhide all of the specified items. If *recurse* is true, also unhide all of their descendants. Use `''` for the root item. `unhide(item)` is equivalent to `item(item, hidden=False)`.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

detached(item=None)

Returns information about detached items (see `detach`). Without arguments, returns a tuple of all detached items, but not their descendants (see `detached_all`). With *item*, returns whether *item* is detached; since Tk 9.1, also returns `True` if an ancestor of *item* is detached.

Requires Tk 9.0 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

detached_all()

Returns a tuple of all detached items and all of their descendants (see `detach`).

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellfocus(cell=None)

Get or set the focus cell. Without *cell*, returns the focus cell as an `(item, column)` 2-tuple, or an empty tuple if there is none. With *cell*, sets the focus cell; use `''` to clear it. A cell is specified as an `(item, column)` pair.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

sort(parent, \*, column=None, command=None, dictionary=False, integer=False, real=False, nocase=False, decreasing=False, ignoreempty=False, recurse=False)

Sort the children of *parent*. By default the children are sorted by the value of the first display column, as Unicode strings, in increasing order. *column* selects the column to sort on. *dictionary*, *integer* and *real* select the comparison type; *nocase* makes string comparison case-insensitive. *command* is a function of two values returning a negative, zero or positive number. *decreasing* reverses the order. *ignoreempty* skips empty values (with *integer* or *real*). *recurse* also sorts all descendants.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

search(parent, pattern, \*, columns=None, start=None, stop=None, dictionary=False, integer=False, real=False, nocase=False, glob=False, regexp=False, backwards=False, hidden=False, recurse=False, wraparound=False)

Search *parent*'s children for *pattern* and return the identifier of the first matching item, or `''` if there is no match. By default *pattern* is matched for exact equality against the value of each displayed column, as Unicode strings, searching forwards through the direct children of *parent*. *glob* or *regexp* select glob-style or regular expression matching; *dictionary*, *integer* and *real* select the comparison type; *nocase* makes it case-insensitive. *columns* limits the search to the given columns. *start* and *stop* bound the search; *backwards* reverses its direction; *wraparound* continues from the other end. *hidden* also searches hidden and closed items; *recurse* searches all descendants.

See `search_all`, `search_cell` and `search_all_cells`.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

search_all(parent, pattern, \*\*kwargs)

Like `search`, but returns a tuple of the identifiers of all matching items.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

search_cell(parent, pattern, \*\*kwargs)

Like `search`, but returns the first matching cell as an `(item, column)` 2-tuple, or an empty tuple if there is no match.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

search_all_cells(parent, pattern, \*\*kwargs)

Like `search`, but returns a tuple of all matching cells, each an `(item, column)` 2-tuple.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection()

Returns a tuple of the selected cells, each an `(item, column)` 2-tuple. The cell selection is independent from the item selection (see `selection`).

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection_set(\*cells)

The specified cells become the new cell selection. Each cell is an `(item, column)` pair. Call without arguments to clear the cell selection.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection_add(\*cells)

Add the specified cells to the cell selection.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection_remove(\*cells)

Remove the specified cells from the cell selection.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection_set_range(first, last, \*, hidden=True, recurse=True)

Set the cell selection to the rectangle of cells from *first* to *last*. *first* and *last* are the opposite corner cells, each an `(item, column)` pair, and must be in displayed columns. All other cells are unselected. If *hidden* is false, hidden cells are excluded; if *recurse* is false, cells in descendant items are excluded.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection_add_range(first, last, \*, hidden=True, recurse=True)

Like `cellselection_set_range`, but adds the rectangle of cells to the cell selection instead of replacing it.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection_remove_range(first, last, \*, hidden=True, recurse=True)

Like `cellselection_set_range`, but removes the rectangle of cells from the cell selection.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection_anchor(cell=None)

Get or set the cell selection anchor. Without *cell*, returns the anchor as an `(item, column)` 2-tuple, or an empty tuple if it is unset. With *cell*, sets the anchor; use `''` to unset it.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection_includes(\*cells)

Returns whether all of the specified cells are selected.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

cellselection_present()

Returns whether any cell is selected.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

tag_cell_add(tagname, \*cells)

Add the given tag to each of the specified cells. Each cell is an `(item, column)` pair. Cell tags are independent from item tags.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

tag_cell_remove(tagname, \*cells)

Remove the given tag from each of the specified cells. If no cell is specified, the tag is removed from all cells.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

tag_cell_has(tagname, cell=None)

Test for a cell tag, or list the cells that have it. If *cell* is specified, returns whether that cell has the given tag. Otherwise returns a tuple of all cells (as `(item, column)` 2-tuples) that have the tag.

Requires Tk 9.1 or newer.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

see(item)

Ensure that *item* is visible.

Sets all of *item*'s ancestors open option to `True`, and scrolls the widget if necessary so that *item* is within the visible portion of the tree.

</div>

<div class="method">

selection()

Returns a tuple of selected items.

<div class="versionchanged">

3.8 `selection()` no longer takes arguments. For changing the selection state use the following selection methods.

</div>

</div>

<div class="method">

selection_set(\*items)

*items* becomes the new selection.

<div class="versionchanged">

3.6 *items* can be passed as separate arguments, not just as a single tuple.

</div>

</div>

<div class="method">

selection_add(\*items)

Add *items* to the selection.

<div class="versionchanged">

3.6 *items* can be passed as separate arguments, not just as a single tuple.

</div>

</div>

<div class="method">

selection_remove(\*items)

Remove *items* from the selection.

<div class="versionchanged">

3.6 *items* can be passed as separate arguments, not just as a single tuple.

</div>

</div>

<div class="method">

selection_toggle(\*items)

Toggle the selection state of each item in *items*.

<div class="versionchanged">

3.6 *items* can be passed as separate arguments, not just as a single tuple.

</div>

</div>

<div class="method">

set(item, column=None, value=None)

With one argument, returns a dictionary of column/value pairs for the specified *item*. With two arguments, returns the current value of the specified *column*. With three arguments, sets the value of given *column* in given *item* to the specified *value*.

</div>

<div class="method">

tag_bind(tagname, sequence=None, callback=None)

Bind a callback for the given event *sequence* to the tag *tagname*. When an event is delivered to an item, the callbacks for each of the item's tags option are called.

</div>

<div class="method">

tag_configure(tagname, option=None, \*\*kw)

Query or modify the options for the specified *tagname*.

If *kw* is not given, returns a dict of the option settings for *tagname*. If *option* is specified, returns the value for that *option* for the specified *tagname*. Otherwise, sets the options to the corresponding values for the given *tagname*.

</div>

<div class="method">

tag_has(tagname, item=None)

If *item* is specified, returns `True` if the specified *item* has the given *tagname* and `False` otherwise. Otherwise, returns a tuple of all items that have the specified tag.

Availability: Tk 8.6

</div>

<div class="method">

xview(\*args)

Query or modify horizontal position of the treeview.

</div>

<div class="method">

yview(\*args)

Query or modify vertical position of the treeview.

</div>

</div>

## Ttk styling

Each widget in `!ttk` is assigned a style, which specifies the set of elements making up the widget and how they are arranged, along with dynamic and default settings for element options. By default the style name is the same as the widget's class name, but it may be overridden by the widget's style option. If you don't know the class name of a widget, use the method `Misc.winfo_class <tkinter.Misc.winfo_class>` (somewidget.winfo_class()).

<div class="seealso">

[Introduction to the Tk theme engine](https://www.tcl-lang.org/man/tcl9.0/TkCmd/ttk_intro.html)  
The `ttk::intro` man page explains how the theme engine works.

[The Tile Widget Set](https://tktable.sourceforge.net/tile/tile-tcl2004.pdf)  
Joe English's 2004 paper introducing the theme engine (then the separate *Tile* extension), with diagrams of how elements and layouts make up a widget's appearance.

</div>

<div class="Style">

This class is used to manipulate the style database.

<div class="method">

configure(style, query_opt=None, \*\*kw)

Query or set the default value of the specified option(s) in *style*.

Each key in *kw* is an option and each value is a string identifying the value for that option.

For example, to change every default button to be a flat button with some padding and a different background color:

    from tkinter import ttk
    import tkinter

    root = tkinter.Tk()

    ttk.Style().configure("TButton", padding=6, relief="flat",
       background="#ccc")

    btn = ttk.Button(text="Sample")
    btn.pack()

    root.mainloop()

</div>

<div class="method">

map(style, query_opt=None, \*\*kw)

Query or sets dynamic values of the specified option(s) in *style*.

Each key in *kw* is an option and each value should be a list or a tuple (usually) containing statespecs grouped in tuples, lists, or some other preference. A statespec is a compound of one or more states and then a value.

An example may make it more understandable:

    import tkinter
    from tkinter import ttk

    root = tkinter.Tk()

    style = ttk.Style()
    style.map("C.TButton",
        foreground=[('pressed', 'red'), ('active', 'blue')],
        background=[('pressed', '!disabled', 'black'), ('active', 'white')]
        )

    colored_btn = ttk.Button(text="Test", style="C.TButton").pack()

    root.mainloop()

Note that the order of the (states, value) sequences for an option does matter, if the order is changed to `[('active', 'blue'), ('pressed', 'red')]` in the foreground option, for example, the result would be a blue foreground when the widget were in active or pressed states.

When called to query the map (without specifying values to set), it returns a dictionary mapping each option to its list of statespecs.

<div class="versionchanged">

3.10 The value returned when querying the map was corrected.

</div>

</div>

<div class="method">

lookup(style, option, state=None, default=None)

Returns the value specified for *option* in *style*.

If *state* is specified, it is expected to be a sequence of one or more states. If the *default* argument is set, it is used as a fallback value in case no specification for option is found.

To check what font a Button uses by default:

    from tkinter import ttk

    print(ttk.Style().lookup("TButton", "font"))

</div>

<div class="method">

layout(style, layoutspec=None)

Define the widget layout for given *style*. If *layoutspec* is omitted, return the layout specification for given style.

*layoutspec*, if specified, is expected to be a list or some other sequence type (excluding strings), where each item should be a tuple and the first item is the layout name and the second item should have the format described in [Layouts](#layouts).

To understand the format, see the following example (it is not intended to do anything useful):

    from tkinter import ttk
    import tkinter

    root = tkinter.Tk()

    style = ttk.Style()
    style.layout("TMenubutton", [
       ("Menubutton.background", None),
       ("Menubutton.button", {"children":
           [("Menubutton.focus", {"children":
               [("Menubutton.padding", {"children":
                   [("Menubutton.label", {"side": "left", "expand": 1})]
               })]
           })]
       }),
    ])

    mbtn = ttk.Menubutton(text='Text')
    mbtn.pack()
    root.mainloop()

</div>

<div class="method">

element_create(elementname, etype, *args,*\*kw)

Create a new element in the current theme, of the given *etype* which is expected to be either "image", "from" or "vsapi". The latter is only available in Tk 8.6 on Windows.

If "image" is used, *args* should contain the default image name followed by statespec/value pairs (this is the imagespec), and *kw* may have the following options:

border=padding  
padding is a list of up to four integers, specifying the left, top, right, and bottom borders, respectively.

height=height  
Specifies a minimum height for the element. If less than zero, the base image's height is used as a default.

padding=padding  
Specifies the element's interior padding. Defaults to border's value if not specified.

sticky=spec  
Specifies how the image is placed within the final parcel. spec contains zero or more characters "n", "s", "w", or "e".

width=width  
Specifies a minimum width for the element. If less than zero, the base image's width is used as a default.

Example:

    img1 = tkinter.PhotoImage(master=root, file='button.png')
    img1 = tkinter.PhotoImage(master=root, file='button-pressed.png')
    img1 = tkinter.PhotoImage(master=root, file='button-active.png')
    style = ttk.Style(root)
    style.element_create('Button.button', 'image',
                         img1, ('pressed', img2), ('active', img3),
                         border=(2, 4), sticky='we')

If "from" is used as the value of *etype*, `element_create` will clone an existing element. *args* is expected to contain a themename, from which the element will be cloned, and optionally an element to clone from. If this element to clone from is not specified, an empty element will be used. *kw* is discarded.

Example:

    style = ttk.Style(root)
    style.element_create('plain.background', 'from', 'default')

If "vsapi" is used as the value of *etype*, `element_create` will create a new element in the current theme whose visual appearance is drawn using the Microsoft Visual Styles API which is responsible for the themed styles on Windows XP and Vista. *args* is expected to contain the Visual Styles class and part as given in the Microsoft documentation followed by an optional sequence of tuples of ttk states and the corresponding Visual Styles API state value. *kw* may have the following options:

padding=padding  
Specify the element's interior padding. *padding* is a list of up to four integers specifying the left, top, right and bottom padding quantities respectively. If fewer than four elements are specified, bottom defaults to top, right defaults to left, and top defaults to left. In other words, a list of three numbers specify the left, vertical, and right padding; a list of two numbers specify the horizontal and the vertical padding; a single number specifies the same padding all the way around the widget. This option may not be mixed with any other options.

margins=padding  
Specifies the elements exterior padding. *padding* is a list of up to four integers specifying the left, top, right and bottom padding quantities respectively. This option may not be mixed with any other options.

width=width  
Specifies the width for the element. If this option is set then the Visual Styles API will not be queried for the recommended size or the part. If this option is set then *height* should also be set. The *width* and *height* options cannot be mixed with the *padding* or *margins* options.

height=height  
Specifies the height of the element. See the comments for *width*.

Example:

    style = ttk.Style(root)
    style.element_create('pin', 'vsapi', 'EXPLORERBAR', 3, [
                         ('pressed', '!selected', 3),
                         ('active', '!selected', 2),
                         ('pressed', 'selected', 6),
                         ('active', 'selected', 5),
                         ('selected', 4),
                         ('', 1)])
    style.layout('Explorer.Pin',
                 [('Explorer.Pin.pin', {'sticky': 'news'})])
    pin = ttk.Checkbutton(style='Explorer.Pin')
    pin.pack(expand=True, fill='both')

<div class="versionchanged">

3.13 Added support of the "vsapi" element factory.

</div>

</div>

<div class="method">

element_names()

Returns a tuple of elements defined in the current theme.

</div>

<div class="method">

element_options(elementname)

Returns a tuple of *elementname*'s options.

</div>

<div class="method">

theme_create(themename, parent=None, settings=None)

Create a new theme.

It is an error if *themename* already exists. If *parent* is specified, the new theme will inherit styles, elements and layouts from the parent theme. If *settings* are present they are expected to have the same syntax used for `theme_settings`.

</div>

<div class="method">

theme_settings(themename, settings)

Temporarily sets the current theme to *themename*, apply specified *settings* and then restore the previous theme.

Each key in *settings* is a style and each value may contain the keys 'configure', 'map', 'layout' and 'element create' and they are expected to have the same format as specified by the methods `Style.configure`, `Style.map`, `Style.layout` and `Style.element_create` respectively.

As an example, let's change the Combobox for the default theme a bit:

    from tkinter import ttk
    import tkinter

    root = tkinter.Tk()

    style = ttk.Style()
    style.theme_settings("default", {
       "TCombobox": {
           "configure": {"padding": 5},
           "map": {
               "background": [("active", "green2"),
                              ("!disabled", "green4")],
               "fieldbackground": [("!disabled", "green3")],
               "foreground": [("focus", "OliveDrab1"),
                              ("!disabled", "OliveDrab2")]
           }
       }
    })

    combo = ttk.Combobox().pack()

    root.mainloop()

</div>

<div class="method">

theme_names()

Returns a tuple of all known themes.

</div>

<div class="method">

theme_styles(themename=None)

Returns a tuple of all styles in *themename*. If *themename* is not given, the current theme is used.

<div class="versionadded">

3.16

</div>

Availability: Tk 9.0.

</div>

<div class="method">

theme_use(themename=None)

If *themename* is not given, returns the theme in use. Otherwise, sets the current theme to *themename*, refreshes all widgets and emits a \<\<ThemeChanged\>\> event.

</div>

</div>

### Layouts

A layout can be just `None`, if it takes no options, or a dict of options specifying how to arrange the element. The layout mechanism uses a simplified version of the pack geometry manager: given an initial cavity, each element is allocated a parcel.

The valid options/values are:

*side*: whichside  
Specifies which side of the cavity to place the element; one of top, right, bottom or left. If omitted, the element occupies the entire cavity.

*sticky*: nswe  
Specifies where the element is placed inside its allocated parcel.

*unit*: 0 or 1  
If set to 1, causes the element and all of its descendants to be treated as a single element for the purposes of `Widget.identify` et al. It's used for things like scrollbar thumbs with grips.

*children*: \[sublayout... \]  
Specifies a list of elements to place inside the element. Each element is a tuple (or other sequence type) where the first item is the layout name, and the other is a [Layout](#layouts).

## Additional widgets

The following themed widgets complete the `tkinter.ttk` widget set. Each is the themed counterpart of the like-named classic `tkinter` widget and inherits the common methods of `Widget`.

<div class="Button(master=None, **kw)">

Ttk `Button` widget, displays a textual label and/or image, and evaluates a command when pressed. It is the themed counterpart of `tkinter.Button` and inherits the common widget methods from `Widget`.

<div class="method">

invoke()

Invoke the command associated with the button and return its result.

</div>

</div>

<div class="Checkbutton(master=None, **kw)">

Ttk `Checkbutton` widget, used to control a boolean variable that is toggled on and off. It is the themed counterpart of `tkinter.Checkbutton` and inherits the common widget methods from `Widget`.

<div class="method">

invoke()

Toggle the button between its selected and deselected states, invoke the command associated with the button, and return its result.

</div>

</div>

<div class="Entry(master=None, widget=None, **kw)">

Ttk `Entry` widget, displays a one-line text string and allows the user to edit it. It is the themed counterpart of `tkinter.Entry` and inherits the common widget methods from `Widget` as well as the editing methods from `tkinter.Entry`.

<div class="method">

bbox(index)

Return a tuple `(x, y, width, height)` giving the bounding box of the character at the given *index*.

This shadows the inherited `!Misc.bbox`; use `~tkinter.Misc.grid_bbox` for the grid bounding box.

</div>

<div class="method">

identify(x, y)

Return the name of the element under the point given by *x* and *y*, or the empty string if no element is present at that location.

</div>

<div class="method">

validate()

Force validation of the entry and return `True` if validation succeeded, and `False` otherwise.

</div>

</div>

<div class="Frame(master=None, **kw)">

Ttk `Frame` widget, a container used to group and lay out other widgets. It is the themed counterpart of `tkinter.Frame` and inherits the common widget methods from `Widget`.

</div>

<div class="Label(master=None, **kw)">

Ttk `Label` widget, displays a textual label and/or image. It is the themed counterpart of `tkinter.Label` and inherits the common widget methods from `Widget`.

</div>

<div class="Labelframe(master=None, **kw)">

Ttk `Labelframe` widget, a container that draws a border and a title label around its contents. It is the themed counterpart of `tkinter.LabelFrame` and inherits the common widget methods from `Widget`.

</div>

<div class="Menubutton(master=None, **kw)">

Ttk `Menubutton` widget, displays a textual label and/or image, and pops up a menu when pressed. It is the themed counterpart of `tkinter.Menubutton` and inherits the common widget methods from `Widget`.

</div>

<div class="OptionMenu(master, variable, default=None, *values, **kwargs)">

Ttk `OptionMenu` widget, a `Menubutton` that pops up a menu of mutually exclusive choices. *variable* is the variable that tracks the currently selected value, *default* is the value to set initially, and *values* are the entries to display in the menu. A *command* keyword argument may be given to specify a callable that is invoked with the selected value whenever the selection changes; the *style* keyword argument sets the style used by the underlying menubutton; the *direction* keyword argument sets where the menu is posted relative to the menubutton (one of `'above'`, `'below'` (the default), `'left'`, `'right'` or `'flush'`); and the *name* keyword argument sets the Tk widget name.

<div class="method">

set_menu(default=None, \*values)

Replace the entries of the menu with *values*. If *default* is given, also set it as the current value of the *variable*.

</div>

<div class="method">

destroy()

Destroy this widget and its associated menu.

</div>

<div class="versionchanged">

3.14 Added support for the *name* keyword argument.

</div>

</div>

<div class="Panedwindow(master=None, **kw)">

Ttk `Panedwindow` widget, displays a number of subwindows stacked either vertically or horizontally. The user may adjust the relative sizes of the subwindows by dragging the sash between panes. It is the themed counterpart of `tkinter.PanedWindow` and inherits the common widget methods from `Widget`, as well as the `!add` and `!panes` methods from `tkinter.PanedWindow`.

<div class="method">

insert(pos, child, \*\*kw)

Insert a pane containing *child* at the position *pos*. *pos* is either the string `'end'`, an integer index, or the name of a managed subwindow. If *child* is already managed by the paned window, move it to the specified position. Any keyword arguments set pane options.

</div>

<div class="method">

forget(child)

Remove *child*, which may be either an integer index or the name of a managed subwindow, from the panes.

This shadows the inherited geometry-manager `!forget`; use `~tkinter.Pack.pack_forget`, `~tkinter.Grid.grid_forget` or `~tkinter.Place.place_forget` to remove the widget itself from its manager.

</div>

<div class="method">

pane(pane, option=None, \*\*kw)

Query or modify the options of the specified *pane*, where *pane* is either an integer index or the name of a managed subwindow. If no arguments are given, return a dictionary of the pane option values. If *option* is specified, return the value of that option. Otherwise, set the options given as keyword arguments to their corresponding values.

</div>

<div class="method">

sashpos(index, newpos=None)

If *newpos* is specified, set the position of sash number *index* and return its new position. This may adjust the positions of adjacent sashes to ensure that positions are monotonically increasing; positions are also constrained to be between 0 and the total size of the widget. If *newpos* is omitted, return the current position of the sash.

</div>

</div>

<div class="Radiobutton(master=None, **kw)">

Ttk `Radiobutton` widget, used as part of a group to control a single shared variable by selecting one of several mutually exclusive values. It is the themed counterpart of `tkinter.Radiobutton` and inherits the common widget methods from `Widget`.

<div class="method">

invoke()

Set the option variable to the button's value, select the button, invoke the command associated with the button, and return its result.

</div>

</div>

<div class="Scale(master=None, **kw)">

Ttk `Scale` widget, displays a slider that lets the user select a numeric value from a range by moving the slider along a trough. It is the themed counterpart of `tkinter.Scale` and inherits the common widget methods from `Widget`.

<div class="method">

configure(cnf=None, \*\*kw)

Modify or query the widget options, like `Widget.configure <tkinter.Misc.configure>`. In addition, this method clips the `from` and `to` values so that the current value stays within the range defined by them.

<div class="versionchanged">

3.9 Now returns the configuration value, like `Widget.configure <tkinter.Misc.configure>`.

</div>

</div>

<div class="method">

get(x=None, y=None)

Return the current value of the scale. If *x* and *y* are given, return the value corresponding to the pixel coordinate *x*, *y* instead.

</div>

</div>

<div class="Scrollbar(master=None, **kw)">

Ttk `Scrollbar` widget, controls the viewport of an associated scrollable widget such as a `Treeview`, `Entry` or `tkinter.Text`. It is the themed counterpart of `tkinter.Scrollbar` and inherits the common widget methods from `Widget`, as well as the `!set` and `!get` methods from `tkinter.Scrollbar`.

</div>

<div class="Separator(master=None, **kw)">

Ttk `Separator` widget, displays a horizontal or vertical separator line. It has no direct counterpart in `tkinter` and inherits the common widget methods from `Widget`.

</div>

<div class="Sizegrip(master=None, **kw)">

Ttk `Sizegrip` widget, displays a grip that allows the user to resize the containing toplevel window by pressing and dragging the grip, typically placed in the bottom-right corner. It has no direct counterpart in `tkinter` and inherits the common widget methods from `Widget`.

</div>

<div class="LabeledScale(master=None, variable=None, from_=0, to=10, **kw)">

A `Frame` containing a `Scale` and a `Label` that shows the scale's current value. *variable* is the `~tkinter.IntVar` tracked by the scale (one is created if it is not given), and *from\_* and *to* define the range of the scale.

<div class="method">

destroy()

Destroy this widget and remove the trace callback registered on the associated variable.

</div>

</div>

<div class="LabelFrame(master=None, **kw)">

Alias of `Labelframe`, kept for naming compatibility with `tkinter.LabelFrame`.

</div>

<div class="PanedWindow(master=None, **kw)">

Alias of `Panedwindow`, kept for naming compatibility with `tkinter.PanedWindow`.

</div>

# `!token` --- Constants used with Python parse trees

<div class="module" data-synopsis="Constants representing terminal nodes of the parse tree.">

token

</div>

**Source code:** `Lib/token.py`

------------------------------------------------------------------------

This module provides constants which represent the numeric values of leaf nodes of the parse tree (terminal tokens). Refer to the file `Grammar/Tokens` in the Python distribution for the definitions of the names in the context of the language grammar. The specific numeric values which the names map to may change between Python versions.

The module also provides a mapping from numeric codes to names and some functions. The functions mirror definitions in the Python C header files.

Note that a token's value may depend on tokenizer options. For example, a `"+"` token may be reported as either `PLUS` or `OP`, or a `"match"` token may be either `NAME` or `SOFT_KEYWORD`.

<div class="data">

tok_name

Dictionary mapping the numeric values of the constants defined in this module back to name strings, allowing more human-readable representation of parse trees to be generated.

</div>

<div class="function">

ISTERMINAL(x)

Return `True` for terminal token values.

</div>

<div class="function">

ISNONTERMINAL(x)

Return `True` for non-terminal token values.

</div>

<div class="function">

ISEOF(x)

Return `True` if *x* is the marker indicating the end of input.

</div>

The token constants are:

<div class="data">

NAME

Token value that indicates an `identifier or keyword <identifiers>`.

</div>

<div class="data">

NUMBER

Token value that indicates a `numeric literal <numbers>`

</div>

<div class="data">

STRING

Token value that indicates a `string or byte literal <strings>`, excluding `formatted string literals <f-strings>`. The token string is not interpreted: it includes the surrounding quotation marks and the prefix (if given); backslashes are included literally, without processing escape sequences.

</div>

<div class="data">

OP

A generic token value that indicates an `operator <operators>` or `delimiter <delimiters>`.

<div class="impl-detail">

This value is only reported by the `tokenize` module. Internally, the tokenizer uses `exact token types <token_operators_delimiters>` instead.

</div>

</div>

<div class="data">

COMMENT

Token value used to indicate a comment. The parser ignores `!COMMENT` tokens.

</div>

<div class="data">

NEWLINE

Token value that indicates the end of a `logical line <logical-lines>`.

</div>

<div class="data">

NL

Token value used to indicate a non-terminating newline. `!NL` tokens are generated when a logical line of code is continued over multiple physical lines. The parser ignores `!NL` tokens.

</div>

<div class="data">

INDENT

Token value used at the beginning of a `logical line <logical-lines>` to indicate the start of an `indented block <indentation>`.

</div>

<div class="data">

DEDENT

Token value used at the beginning of a `logical line <logical-lines>` to indicate the end of an `indented block <indentation>`.

</div>

<div class="data">

FSTRING_START

Token value used to indicate the beginning of an `f-string literal <f-strings>`.

<div class="impl-detail">

The token string includes the prefix and the opening quote(s), but none of the contents of the literal.

</div>

</div>

<div class="data">

FSTRING_MIDDLE

Token value used for literal text inside an `f-string literal <f-strings>`, including format specifications.

<div class="impl-detail">

Replacement fields (that is, the non-literal parts of f-strings) use the same tokens as other expressions, and are delimited by `LBRACE`, `RBRACE`, `EXCLAMATION` and `COLON` tokens.

</div>

</div>

<div class="data">

FSTRING_END

Token value used to indicate the end of a `f-string <f-strings>`.

<div class="impl-detail">

The token string contains the closing quote(s).

</div>

</div>

<div class="data">

TSTRING_START

Token value used to indicate the beginning of a template string literal.

<div class="impl-detail">

The token string includes the prefix and the opening quote(s), but none of the contents of the literal.

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

TSTRING_MIDDLE

Token value used for literal text inside a template string literal including format specifications.

<div class="impl-detail">

Replacement fields (that is, the non-literal parts of t-strings) use the same tokens as other expressions, and are delimited by `LBRACE`, `RBRACE`, `EXCLAMATION` and `COLON` tokens.

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

TSTRING_END

Token value used to indicate the end of a template string literal.

<div class="impl-detail">

The token string contains the closing quote(s).

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

ENDMARKER

Token value that indicates the end of input. Used in `top-level grammar rules <top-level>`.

</div>

<div class="data">

ENCODING

Token value that indicates the encoding used to decode the source bytes into text. The first token returned by `tokenize.tokenize` will always be an `ENCODING` token.

<div class="impl-detail">

This token type isn't used by the C tokenizer but is needed for the `tokenize` module.

</div>

</div>

The following token types are not produced by the `tokenize` module, and are defined for special uses in the tokenizer or parser:

<div class="data">

TYPE_IGNORE

Token value indicating that a `type: ignore` comment was recognized. Such tokens are produced instead of regular `COMMENT` tokens only with the `~ast.PyCF_TYPE_COMMENTS` flag.

</div>

<div class="data">

TYPE_COMMENT

Token value indicating that a type comment was recognized. Such tokens are produced instead of regular `COMMENT` tokens only with the `~ast.PyCF_TYPE_COMMENTS` flag.

</div>

<div class="data">

SOFT_KEYWORD

Token value indicating a `soft keyword <soft-keywords>`.

The tokenizer never produces this value. To check for a soft keyword, pass a `NAME` token's string to `keyword.issoftkeyword`.

</div>

<div class="data">

ERRORTOKEN

Token value used to indicate wrong input.

The `tokenize` module generally indicates errors by raising exceptions instead of emitting this token. It can also emit tokens such as `OP` or `NAME` with strings that are later rejected by the parser.

</div>

<div id="token_operators_delimiters">

The remaining tokens represent specific `operators <operators>` and `delimiters <delimiters>`. (The `tokenize` module reports these as `OP`; see `exact_type` in the `tokenize` documentation for details.)

</div>

The following non-token constants are provided:

<div class="data">

N_TOKENS

The number of token types defined in this module.

</div>

<div class="data">

EXACT_TOKEN_TYPES

A dictionary mapping the string representation of a token to its numeric code.

<div class="versionadded">

3.8

</div>

</div>

<div class="versionchanged">

3.5 Added `!AWAIT` and `!ASYNC` tokens.

</div>

<div class="versionchanged">

3.7 Added `COMMENT`, `NL` and `ENCODING` tokens.

</div>

<div class="versionchanged">

3.7 Removed `!AWAIT` and `!ASYNC` tokens. "async" and "await" are now tokenized as `NAME` tokens.

</div>

<div class="versionchanged">

3.8 Added `TYPE_COMMENT`, `TYPE_IGNORE`, `COLONEQUAL`. Added `!AWAIT` and `!ASYNC` tokens back (they're needed to support parsing older Python versions for `ast.parse` with `feature_version` set to 6 or lower).

</div>

<div class="versionchanged">

3.12 Added `EXCLAMATION`.

</div>

<div class="versionchanged">

3.13 Removed `!AWAIT` and `!ASYNC` tokens again.

</div>

# `!tokenize` --- Tokenizer for Python source

<div class="module" data-synopsis="Lexical scanner for Python source code.">

tokenize

</div>

**Source code:** `Lib/tokenize.py`

------------------------------------------------------------------------

The `!tokenize` module provides a lexical scanner for Python source code, implemented in Python. The scanner in this module returns comments as tokens as well, making it useful for implementing "pretty-printers", including colorizers for on-screen displays.

To simplify token stream handling, all `operator <operators>` and `delimiter <delimiters>` tokens and `Ellipsis` are returned using the generic `~token.OP` token type. The `TokenInfo.exact_type` property gives the exact token type.

> [!WARNING]
> Note that the functions in this module are only designed to parse syntactically valid Python code (code that does not raise when parsed using `ast.parse`). The behavior of the functions in this module is **undefined** when providing invalid Python code and it can change at any point.

## Tokenizing input

The primary entry point is a `generator`:

<div class="function">

tokenize(readline)

The `.tokenize` generator requires one argument, *readline*, which must be a callable object which provides the same interface as the `io.IOBase.readline` method of file objects. Each call to the function should return one line of input as bytes.

The generator produces `TokenInfo` objects.

<div class="versionchanged">

3.1 Changed to yield `TokenInfo` objects instead of plain tuples.

</div>

`.tokenize` determines the source encoding of the file by looking for a UTF-8 BOM or encoding cookie, according to `263`.

</div>

<div class="function">

generate_tokens(readline)

Tokenize a source reading unicode strings instead of bytes.

Like `.tokenize`, the *readline* argument is a callable returning a single line of input. However, `generate_tokens` expects *readline* to return a str object rather than bytes.

The result is an iterator yielding `TokenInfo` objects, exactly like `.tokenize`. It does not yield an `~token.ENCODING` token.

</div>

<div class="TokenInfo(type, string, start, end, line)">

A `named tuple` describing a token. It has five fields and an additional `exact_type` property.

<div class="versionadded">

3.1

</div>

<div class="attribute">

type

An integer identifying the token type, as defined by the constants in `token`, such as `~token.NAME` and `~token.STRING`. Operators and delimiters have type `~token.OP`; use `exact_type` for their specific token type.

</div>

<div class="attribute">

string

The token text as a `str`. For `~token.STRING` tokens, this includes the quotes, any prefix, and escape sequences, without evaluating them.

For an `~token.ENCODING` token, this is the encoding name. Tokens inserted without corresponding source text can have an empty string. In `~token.FSTRING_MIDDLE` and `~token.TSTRING_MIDDLE` tokens, doubled braces are replaced by single braces.

</div>

<div class="attribute">

start

A 2-tuple `(srow, scol)` of ints specifying the inclusive start position of the token in the source. Rows are numbered from 1 and columns from 0. Columns count Unicode code points; a tab counts as one column, without expanding it to a tab stop.

For an `~token.ENCODING` token, this is `(0, 0)`.

</div>

<div class="attribute">

end

A 2-tuple `(erow, ecol)` of ints specifying the exclusive end position of the token. Rows are numbered from 1 and columns from 0. For a token spanning multiple lines, *erow* is the final row and *ecol* is the column on that row.

For example, the name `spam` at the beginning of the first line has `start` equal to `(1, 0)` and `end` equal to `(1, 4)`.

For an `~token.ENCODING` token, this is `(0, 0)`. A `~token.NEWLINE` or `~token.NL` token inserted at the end of input without a final newline can have an end column one beyond the source text.

</div>

<div class="attribute">

line

A `str` containing the full source line on which the token was found, including surrounding text and the line terminator if present. For a token spanning multiple lines, this can contain several source lines. It is empty for `~token.ENCODING` and `~token.ENDMARKER` tokens.

</div>

<div class="attribute">

exact_type

An integer identifying the specific operator type for `~token.OP` tokens, as defined by the constants in `token`. For all other token types, this equals `type`. For example, a `+` token has `type` equal to `~token.OP` and `exact_type` equal to `~token.PLUS`.

<div class="versionadded">

3.3

</div>

</div>

</div>

All constants from the `token` module are also exported from `!tokenize`.

Another function is provided to reverse the tokenization process. This is useful for creating tools that tokenize a script, modify the token stream, and write back the modified script.

<div class="function">

untokenize(iterable)

Converts tokens back into Python source code. The *iterable* must return sequences with at least two elements, the token type and the token string. Any additional sequence elements are ignored.

The result is guaranteed to tokenize back to match the input so that the conversion is lossless and round-trips are assured. The guarantee applies only to the token type and token string as the spacing between tokens (column positions) may change.

It returns bytes, encoded using the `~token.ENCODING` token, which is the first token sequence output by `.tokenize`. If there is no encoding token in the input, it returns a str instead.

</div>

`.tokenize` needs to detect the encoding of source files it tokenizes. The function it uses to do this is available:

<div class="function">

detect_encoding(readline)

The `detect_encoding` function is used to detect the encoding that should be used to decode a Python source file. It requires one argument, readline, in the same way as the `.tokenize` generator.

It will call readline a maximum of twice, and return the encoding used (as a string) and a list of any lines (not decoded from bytes) it has read in.

It detects the encoding from the presence of a UTF-8 BOM or an encoding cookie as specified in `263`. If both a BOM and a cookie are present, but disagree, a `SyntaxError` will be raised. Note that if the BOM is found, `'utf-8-sig'` will be returned as an encoding.

If no encoding is specified, then the default of `'utf-8'` will be returned.

Use `.open` to open Python source files: it uses `detect_encoding` to detect the file encoding.

</div>

<div class="function">

open(filename)

Open a file in read only mode using the encoding detected by `detect_encoding`.

<div class="versionadded">

3.2

</div>

</div>

<div class="exception">

TokenError

Raised when either a docstring or expression that may be split over several lines is not completed anywhere in the file, for example:

    """Beginning of
    docstring

or:

    [1,
     2,
     3

</div>

## Command-line usage

<div class="versionadded">

3.3

</div>

The `!tokenize` module can be executed as a script from the command line. It is as simple as:

``` sh
python -m tokenize [-e] [filename.py]
```

The following options are accepted:

<div class="program">

tokenize

</div>

<div class="option">

`-h, --help`  

show this help message and exit

</div>

<div class="option">

`-e, --exact`  

display token names using the exact type

</div>

If `filename.py` is specified its contents are tokenized to stdout. Otherwise, tokenization is performed on stdin.

<div class="versionadded">

3.15 Output is in color by default and can be `controlled using environment variables <using-on-controlling-color>`.

</div>

## Examples

Example of a script rewriter that transforms float literals into Decimal objects:

    from tokenize import tokenize, untokenize, NUMBER, STRING, NAME, OP
    from io import BytesIO

    def decistmt(s):
        """Substitute Decimals for floats in a string of statements.

        >>> from decimal import Decimal
        >>> s = 'print(+21.3e-5*-.1234/81.7)'
        >>> decistmt(s)
        "print (+Decimal ('21.3e-5')*-Decimal ('.1234')/Decimal ('81.7'))"

        The format of the exponent is inherited from the platform C library.
        Known cases are "e-007" (Windows) and "e-07" (not Windows).  Since
        we're only showing 12 digits, and the 13th isn't close to 5, the
        rest of the output should be platform-independent.

        >>> exec(s)  #doctest: +ELLIPSIS
        -3.21716034272e-0...7

        Output from calculations with Decimal should be identical across all
        platforms.

        >>> exec(decistmt(s))
        -3.217160342717258261933904529E-7
        """
        result = []
        g = tokenize(BytesIO(s.encode('utf-8')).readline)  # tokenize the string
        for toknum, tokval, _, _, _ in g:
            if toknum == NUMBER and '.' in tokval:  # replace NUMBER tokens
                result.extend([
                    (NAME, 'Decimal'),
                    (OP, '('),
                    (STRING, repr(tokval)),
                    (OP, ')')
                ])
            else:
                result.append((toknum, tokval))
        return untokenize(result).decode('utf-8')

Example of tokenizing from the command line. The script:

    def say_hello():
        print("Hello, World!")

    say_hello()

will be tokenized to the following output where the first column is the range of the line/column coordinates where the token is found, the second column is the name of the token, and the final column is the value of the token (if any):

```
$ python -m tokenize hello.py
0,0-0,0:            ENCODING       'utf-8'
1,0-1,3:            NAME           'def'
1,4-1,13:           NAME           'say_hello'
1,13-1,14:          OP             '('
1,14-1,15:          OP             ')'
1,15-1,16:          OP             ':'
1,16-1,17:          NEWLINE        '\n'
2,0-2,4:            INDENT         '    '
2,4-2,9:            NAME           'print'
2,9-2,10:           OP             '('
2,10-2,25:          STRING         '"Hello, World!"'
2,25-2,26:          OP             ')'
2,26-2,27:          NEWLINE        '\n'
3,0-3,1:            NL             '\n'
4,0-4,0:            DEDENT         ''
4,0-4,9:            NAME           'say_hello'
4,9-4,10:           OP             '('
4,10-4,11:          OP             ')'
4,11-4,12:          NEWLINE        '\n'
5,0-5,0:            ENDMARKER      ''
```

The exact token type names can be displayed using the `-e` option:

```
$ python -m tokenize -e hello.py
0,0-0,0:            ENCODING       'utf-8'
1,0-1,3:            NAME           'def'
1,4-1,13:           NAME           'say_hello'
1,13-1,14:          LPAR           '('
1,14-1,15:          RPAR           ')'
1,15-1,16:          COLON          ':'
1,16-1,17:          NEWLINE        '\n'
2,0-2,4:            INDENT         '    '
2,4-2,9:            NAME           'print'
2,9-2,10:           LPAR           '('
2,10-2,25:          STRING         '"Hello, World!"'
2,25-2,26:          RPAR           ')'
2,26-2,27:          NEWLINE        '\n'
3,0-3,1:            NL             '\n'
4,0-4,0:            DEDENT         ''
4,0-4,9:            NAME           'say_hello'
4,9-4,10:           LPAR           '('
4,10-4,11:          RPAR           ')'
4,11-4,12:          NEWLINE        '\n'
5,0-5,0:            ENDMARKER      ''
```

Example of tokenizing a file programmatically, reading unicode strings instead of bytes with `generate_tokens`:

    import tokenize

    with tokenize.open('hello.py') as f:
        tokens = tokenize.generate_tokens(f.readline)
        for token in tokens:
            print(token)

Or reading bytes directly with `.tokenize`:

    import tokenize

    with open('hello.py', 'rb') as f:
        tokens = tokenize.tokenize(f.readline)
        for token in tokens:
            print(token)

# `!tomllib` --- Parse TOML files

<div class="module" data-synopsis="Parse TOML files.">

tomllib

</div>

**Source code:** `Lib/tomllib`

------------------------------------------------------------------------

This module provides an interface for parsing TOML 1.1.0 (Tom's Obvious Minimal Language, [https://toml.io](https://toml.io/en/)). This module does not support writing TOML.

<div class="versionadded">

3.11 The module was added with support for TOML 1.0.0.

</div>

<div class="versionchanged">

3.15 Added TOML 1.1.0 support. See the `What's New <whatsnew315-tomllib-1-1-0>` for details.

</div>

> [!WARNING]
> Be cautious when parsing data from untrusted sources. A malicious TOML string may cause the decoder to consume considerable CPU and memory resources. Limiting the size of data to be parsed is recommended.

<div class="seealso">

The `Tomli-W package <tomli-w>` is a TOML writer that can be used in conjunction with this module, providing a write API familiar to users of the standard library `marshal` and `pickle` modules.

</div>

<div class="seealso">

The `TOML Kit package <tomlkit>` is a style-preserving TOML library with both read and write capability. It is a recommended replacement for this module for editing already existing TOML files.

</div>

This module defines the following functions:

<div class="function">

load(fp, /, \*, parse_float=float)

Read a TOML file. The first argument should be a readable and binary file object. Return a `dict`. Convert TOML types to Python using this `conversion table <toml-to-py-table>`.

*parse_float* will be called with the string of every TOML float to be decoded. By default, this is equivalent to `float(num_str)`. This can be used to use another datatype or parser for TOML floats (e.g. `decimal.Decimal`). The callable must not return a `dict` or a `list`, else a `ValueError` is raised.

A `TOMLDecodeError` will be raised on an invalid TOML document.

</div>

<div class="function">

loads(s, /, \*, parse_float=float)

Load TOML from a `str` object. Return a `dict`. Convert TOML types to Python using this `conversion table <toml-to-py-table>`. The *parse_float* argument has the same meaning as in `load`.

A `TOMLDecodeError` will be raised on an invalid TOML document.

</div>

The following exceptions are available:

<div class="exception">

TOMLDecodeError(msg, doc, pos)

Subclass of `ValueError` with the following additional attributes:

<div class="attribute">

msg

The unformatted error message.

</div>

<div class="attribute">

doc

The TOML document being parsed.

</div>

<div class="attribute">

pos

The index of *doc* where parsing failed.

</div>

<div class="attribute">

lineno

The line corresponding to *pos*.

</div>

<div class="attribute">

colno

The column corresponding to *pos*.

</div>

<div class="versionchanged">

3.14 Added the *msg*, *doc* and *pos* parameters. Added the `msg`, `doc`, `pos`, `lineno` and `colno` attributes.

</div>

<div class="deprecated">

3.14 Passing free-form positional arguments is deprecated.

</div>

</div>

## Examples

Parsing a TOML file:

    import tomllib

    with open("pyproject.toml", "rb") as f:
        data = tomllib.load(f)

Parsing a TOML string:

    import tomllib

    toml_str = """
    python-version = "3.11.0"
    python-implementation = "CPython"
    """

    data = tomllib.loads(toml_str)

## Conversion Table

<div id="toml-to-py-table">

| TOML | Python |
|----|----|
| TOML document | dict |
| string | str |
| integer | int |
| float | float (configurable with *parse_float*) |
| boolean | bool |
| offset date-time | datetime.datetime (`tzinfo` attribute set to an instance of `datetime.timezone`) |
| local date-time | datetime.datetime (`tzinfo` attribute set to `None`) |
| local date | datetime.date |
| local time | datetime.time |
| array | list |
| table | dict |
| inline table | dict |
| array of tables | list of dicts |

</div>

## Limits and interoperability considerations

`!tomllib` places some limits on the documents it can handle, and it preserves details that other TOML parsers are allowed to ignore. When writing portable TOML files, only use features that are guaranteed or recommended by the standard.

The implementation details listed here may change in future versions of Python.

Tables/dicts  
The TOML spec does not guarantee key/value pairs in TOML documents and tables to be in any specific order.

<div class="impl-detail">

`!tomllib` loads dictionary entries in the order they appear in the source.

</div>

Integers  
TOML recommends supporting integers in `range(−2**63, 2**63)`.

<div class="impl-detail">

`!tomllib` uses `Python's limit on integer string conversion
<int_max_str_digits>` (4300 digits by default).

</div>

Floats  
TOML recommends supporting at least IEEE 754 binary64 values, which means that numbers with more than 15 significant decimal digits are likely to be rounded.

<div class="impl-detail">

`!tomllib` uses Python `float` by default; on many common platforms this is the recommended binary64. See `sys.float_info` for details.

</div>

Nesting limit  
TOML 1.1.0 does not recommend a limit on how deeply arrays and tables may be nested inside one another. (A limit of 100 has been proposed for a future version of TOML.)

<div class="impl-detail">

In `!tomllib`, the nesting level is mainly limited by Python's `recursion limit <sys.getrecursionlimit>`. Note that code that calls `!tomllib` may contribute to the limit.

</div>

# `!trace` --- Trace or track Python statement execution

<div class="module" data-synopsis="Trace or track Python statement execution.">

trace

</div>

**Source code:** `Lib/trace.py`

------------------------------------------------------------------------

The `!trace` module allows you to trace program execution, generate annotated statement coverage listings, print caller/callee relationships and list functions executed during a program run. It can be used in another program or from the command line.

<div class="seealso">

[Coverage.py](https://coverage.readthedocs.io/)  
A popular third-party coverage tool that provides HTML output along with advanced features such as branch coverage.

</div>

## Command-Line Usage

The `!trace` module can be invoked from the command line. It can be as simple as :

    python -m trace --count -C . somefile.py ...

The above will execute `somefile.py` and generate annotated listings of all Python modules imported during the execution into the current directory.

<div class="program">

trace

</div>

<div class="option">

`--help`  

Display usage and exit.

</div>

<div class="option">

`--version`  

Display the version of the module and exit.

</div>

<div class="versionadded">

3.8 Added `--module` option that allows running an executable module.

</div>

### Main options

At least one of the following options must be specified when invoking `!trace`. The `--listfuncs <-l>` option is mutually exclusive with the `--trace <-t>` and `--count <-c>` options. When `--listfuncs <-l>` is provided, neither `--count <-c>` nor `--trace <-t>` are accepted, and vice versa.

<div class="program">

trace

</div>

<div class="option">

`-c, --count`  

Produce a set of annotated listing files upon program completion that shows how many times each statement was executed. See also `--coverdir <-C>`, `--file <-f>` and `--no-report <-R>` below.

</div>

<div class="option">

`-t, --trace`  

Display lines as they are executed.

</div>

<div class="option">

`-l, --listfuncs`  

Display the functions executed by running the program.

</div>

<div class="option">

`-r, --report`  

Produce an annotated list from an earlier program run that used the `--count <-c>` and `--file <-f>` option. This does not execute any code.

</div>

<div class="option">

`-T, --trackcalls`  

Display the calling relationships exposed by running the program.

</div>

### Modifiers

<div class="program">

trace

</div>

<div class="option">

`-f, --file=<file>`  

Name of a file to accumulate counts over several tracing runs. Should be used with the `--count <-c>` option.

</div>

<div class="option">

`-C, --coverdir=<dir>`  

Directory where the report files go. The coverage report for `package.module` is written to file `{dir}/{package}/{module}.cover`.

</div>

<div class="option">

`-m, --missing`  

When generating annotated listings, mark lines which were not executed with `>>>>>>`.

</div>

<div class="option">

`-s, --summary`  

When using `--count <-c>` or `--report <-r>`, write a brief summary to stdout for each file processed.

</div>

<div class="option">

`-R, --no-report`  

Do not generate annotated listings. This is useful if you intend to make several runs with `--count <-c>`, and then produce a single set of annotated listings at the end.

</div>

<div class="option">

`-g, --timing`  

Prefix each line with the time since the program started. Only used while tracing.

</div>

### Filters

These options may be repeated multiple times.

<div class="program">

trace

</div>

<div class="option">

`--ignore-module=<mod>`  

Ignore each of the given module names and its submodules (if it is a package). The argument can be a list of names separated by a comma.

</div>

<div class="option">

`--ignore-dir=<dir>`  

Ignore all modules and packages in the named directory and subdirectories. The argument can be a list of directories separated by `os.pathsep`.

</div>

## Programmatic Interface

<div class="Trace(count=1, trace=1, countfuncs=0, countcallers=0, ignoremods=(),\ ignoredirs=(), infile=None, outfile=None, timing=False)">

Create an object to trace execution of a single statement or expression. All parameters are optional. *count* enables counting of line numbers. *trace* enables line execution tracing. *countfuncs* enables listing of the functions called during the run. *countcallers* enables call relationship tracking. *ignoremods* is a list of modules or packages to ignore. *ignoredirs* is a list of directories whose modules or packages should be ignored. *infile* is the name of the file from which to read stored count information. *outfile* is the name of the file in which to write updated count information. *timing* enables a timestamp relative to when tracing was started to be displayed.

<div class="method">

run(cmd)

Execute the command and gather statistics from the execution with the current tracing parameters. *cmd* must be a string or code object, suitable for passing into `exec`.

</div>

<div class="method">

runctx(cmd, globals=None, locals=None)

Execute the command and gather statistics from the execution with the current tracing parameters, in the defined global and local environments. If not defined, *globals* and *locals* default to empty dictionaries.

</div>

<div class="method">

runfunc(func, /, *args,*\*kwds)

Call *func* with the given arguments under control of the `Trace` object with the current tracing parameters.

</div>

<div class="method">

results()

Return a `CoverageResults` object that contains the cumulative results of all previous calls to `run`, `runctx` and `runfunc` for the given `Trace` instance. Does not reset the accumulated trace results.

</div>

</div>

<div class="CoverageResults">

A container for coverage results, created by `Trace.results`. Should not be created directly by the user.

<div class="method">

update(other)

Merge in data from another `CoverageResults` object.

</div>

<div class="method">

write_results(show_missing=True, summary=False, coverdir=None,\*, ignore_missing_files=False)

Write coverage results. Set *show_missing* to show lines that had no hits. Set *summary* to include in the output the coverage summary per module. *coverdir* specifies the directory into which the coverage result files will be output. If `None`, the results for each source file are placed in its directory.

If *ignore_missing_files* is `True`, coverage counts for files that no longer exist are silently ignored. Otherwise, a missing file will raise a `FileNotFoundError`.

<div class="versionchanged">

3.13 Added *ignore_missing_files* parameter.

</div>

</div>

</div>

A simple example demonstrating the use of the programmatic interface:

    import sys
    import trace

    # create a Trace object, telling it what to ignore, and whether to
    # do tracing or line-counting or both.
    tracer = trace.Trace(
        ignoredirs=[sys.prefix, sys.exec_prefix],
        trace=0,
        count=1)

    # run the new command using the given tracer
    tracer.run('main()')

    # make a report, placing output in the current directory
    r = tracer.results()
    r.write_results(show_missing=True, coverdir=".")

# `!traceback` --- Print or retrieve a stack traceback

<div class="module" data-synopsis="Print or retrieve a stack traceback.">

traceback

</div>

**Source code:** `Lib/traceback.py`

------------------------------------------------------------------------

This module provides a standard interface to extract, format and print stack traces of Python programs. It is more flexible than the interpreter's default traceback display, and therefore makes it possible to configure certain aspects of the output. Finally, it contains a utility for capturing enough information about an exception to print it later, without the need to save a reference to the actual exception. Since exceptions can be the roots of large objects graph, this utility can significantly improve memory management.

<div class="index">

pair: object; traceback

</div>

The module uses `traceback objects <traceback-objects>` --- these are objects of type `types.TracebackType`, which are assigned to the `~BaseException.__traceback__` field of `BaseException` instances.

<div class="seealso">

Module `faulthandler`  
Used to dump Python tracebacks explicitly, on a fault, after a timeout, or on a user signal.

Module `pdb`  
Interactive source code debugger for Python programs.

</div>

The module's API can be divided into two parts:

- Module-level functions offering basic functionality, which are useful for interactive inspection of exceptions and tracebacks.
- `TracebackException` class and its helper classes `StackSummary` and `FrameSummary`. These offer both more flexibility in the output generated and the ability to store the information necessary for later formatting without holding references to actual exception and traceback objects.

## Module-Level Functions

<div class="function">

print_tb(tb, limit=None, file=None)

Print up to *limit* stack trace entries from `traceback object <traceback-objects>` *tb* (starting from the caller's frame) if *limit* is positive. Otherwise, print the last `abs(limit)` entries. If *limit* is omitted or `None`, all entries are printed. If *file* is omitted or `None`, the output goes to `sys.stderr`; otherwise it should be an open `file <file object>` or `file-like object` to receive the output.

> [!NOTE]
> The meaning of the *limit* parameter is different than the meaning of `sys.tracebacklimit`. A negative *limit* value corresponds to a positive value of `!sys.tracebacklimit`, whereas the behaviour of a positive *limit* value cannot be achieved with `!sys.tracebacklimit`.

<div class="versionchanged">

3.5 Added negative *limit* support.

</div>

</div>

<div class="function">

print_exception(exc, /\[, value, tb\], limit=None, file=None, chain=True)

Print exception information and stack trace entries from `traceback object <traceback-objects>` *tb* to *file*. This differs from `print_tb` in the following ways:

- if *tb* is not `None`, it prints a header `Traceback (most recent call last):`
- it prints the exception type and *value* after the stack trace

<div class="index">

single: ^ (caret); marker

</div>

- if *type(value)* is `SyntaxError` and *value* has the appropriate format, it prints the line where the syntax error occurred with a caret indicating the approximate position of the error.

Since Python 3.10, instead of passing *value* and *tb*, an exception object can be passed as the first argument. If *value* and *tb* are provided, the first argument is ignored in order to provide backwards compatibility.

The optional *limit* argument has the same meaning as for `print_tb`. If *chain* is true (the default), then chained exceptions (the `~BaseException.__cause__` or `~BaseException.__context__` attributes of the exception) will be printed as well, like the interpreter itself does when printing an unhandled exception.

<div class="versionchanged">

3.5 The *etype* argument is ignored and inferred from the type of *value*.

</div>

<div class="versionchanged">

3.10 The *etype* parameter has been renamed to *exc* and is now positional-only.

</div>

</div>

<div class="function">

print_exc(limit=None, file=None, chain=True)

This is a shorthand for `print_exception(sys.exception(), limit=limit, file=file, chain=chain)`.

</div>

<div class="function">

print_last(limit=None, file=None, chain=True)

This is a shorthand for `print_exception(sys.last_exc, limit=limit, file=file, chain=chain)`. In general it will work only after an exception has reached an interactive prompt (see `sys.last_exc`).

</div>

<div class="function">

print_stack(f=None, limit=None, file=None)

Print up to *limit* stack trace entries (starting from the invocation point) if *limit* is positive. Otherwise, print the last `abs(limit)` entries. If *limit* is omitted or `None`, all entries are printed. The optional *f* argument can be used to specify an alternate `stack frame <frame-objects>` to start. The optional *file* argument has the same meaning as for `print_tb`.

<div class="versionchanged">

3.5 Added negative *limit* support.

</div>

</div>

<div class="function">

extract_tb(tb, limit=None)

Return a `StackSummary` object representing a list of "pre-processed" stack trace entries extracted from the `traceback object <traceback-objects>` *tb*. It is useful for alternate formatting of stack traces. The optional *limit* argument has the same meaning as for `print_tb`. A "pre-processed" stack trace entry is a `FrameSummary` object with attributes representing the information that is usually printed for a stack trace.

</div>

<div class="function">

extract_stack(f=None, limit=None)

Extract the raw traceback from the current `stack frame <frame-objects>`. The return value has the same format as for `extract_tb`. The optional *f* and *limit* arguments have the same meaning as for `print_stack`.

</div>

<div class="function">

print_list(extracted_list, file=None)

Print the list of tuples as returned by `extract_tb` or `extract_stack` as a formatted stack trace to the given file. If *file* is `None`, the output is written to `sys.stderr`.

</div>

<div class="function">

format_list(extracted_list)

Given a list of tuples or `FrameSummary` objects as returned by `extract_tb` or `extract_stack`, return a list of strings ready for printing. Each string in the resulting list corresponds to the item with the same index in the argument list. Each string ends in a newline; the strings may contain internal newlines as well, for those items whose source text line is not `None`.

</div>

<div class="function">

format_exception_only(exc, /\[, value\], \*, show_group=False)

Format the exception part of a traceback using an exception value such as given by `sys.last_exc`. The return value is a list of strings, each ending in a newline. The list contains the exception's message, which is normally a single string; however, for `SyntaxError` exceptions, it contains several lines that (when printed) display detailed information about where the syntax error occurred. Following the message, the list contains the exception's `notes <BaseException.__notes__>`.

Since Python 3.10, instead of passing *value*, an exception object can be passed as the first argument. If *value* is provided, the first argument is ignored in order to provide backwards compatibility.

When *show_group* is `True`, and the exception is an instance of `BaseExceptionGroup`, the nested exceptions are included as well, recursively, with indentation relative to their nesting depth.

<div class="versionchanged">

3.10 The *etype* parameter has been renamed to *exc* and is now positional-only.

</div>

<div class="versionchanged">

3.11 The returned list now includes any `notes <BaseException.__notes__>` attached to the exception.

</div>

<div class="versionchanged">

3.13 *show_group* parameter was added.

</div>

</div>

<div class="function">

format_exception(exc, /\[, value, tb\], limit=None, chain=True)

Format a stack trace and the exception information. The arguments have the same meaning as the corresponding arguments to `print_exception`. The return value is a list of strings, each ending in a newline and some containing internal newlines. When these lines are concatenated and printed, exactly the same text is printed as does `print_exception`.

<div class="versionchanged">

3.5 The *etype* argument is ignored and inferred from the type of *value*.

</div>

<div class="versionchanged">

3.10 This function's behavior and signature were modified to match `print_exception`.

</div>

</div>

<div class="function">

format_exc(limit=None, chain=True)

This is like `print_exc(limit)` but returns a string instead of printing to a file.

</div>

<div class="function">

format_tb(tb, limit=None)

A shorthand for `format_list(extract_tb(tb, limit))`.

</div>

<div class="function">

format_stack(f=None, limit=None)

A shorthand for `format_list(extract_stack(f, limit))`.

</div>

<div class="function">

clear_frames(tb)

Clears the local variables of all the stack frames in a `traceback <traceback-objects>` *tb* by calling the `~frame.clear` method of each `frame object <frame-objects>`.

<div class="versionadded">

3.4

</div>

</div>

<div class="function">

walk_stack(f)

Walk a stack following `f.f_back <frame.f_back>` from the given frame, yielding the frame and line number for each frame. If *f* is `None`, the current stack is used. This helper is used with `StackSummary.extract`.

<div class="versionadded">

3.5

</div>

<div class="versionchanged">

3.14 This function previously returned a generator that would walk the stack when first iterated over. The generator returned now is the state of the stack when `walk_stack` is called.

</div>

</div>

<div class="function">

walk_tb(tb)

Walk a traceback following `~traceback.tb_next` yielding the frame and line number for each frame. This helper is used with `StackSummary.extract`.

<div class="versionadded">

3.5

</div>

</div>

## `!TracebackException` Objects

<div class="versionadded">

3.5

</div>

`!TracebackException` objects are created from actual exceptions to capture data for later printing. They offer a more lightweight method of storing this information by avoiding holding references to `traceback<traceback-objects>` and `frame<frame-objects>` objects. In addition, they expose more options to configure the output compared to the module-level functions described above.

<div class="TracebackException(exc_type, exc_value, exc_traceback, *, limit=None, lookup_lines=True, capture_locals=False, compact=False, max_group_width=15, max_group_depth=10)">

Capture an exception for later rendering. The meaning of *limit*, *lookup_lines* and *capture_locals* are as for the `StackSummary` class.

If *compact* is true, only data that is required by `!TracebackException`'s `format` method is saved in the class attributes. In particular, the `__context__` field is calculated only if `__cause__` is `None` and `__suppress_context__` is false.

Note that when locals are captured, they are also shown in the traceback.

*max_group_width* and *max_group_depth* control the formatting of exception groups (see `BaseExceptionGroup`). The depth refers to the nesting level of the group, and the width refers to the size of a single exception group's exceptions array. The formatted output is truncated when either limit is exceeded.

<div class="versionchanged">

3.10 Added the *compact* parameter.

</div>

<div class="versionchanged">

3.11 Added the *max_group_width* and *max_group_depth* parameters.

</div>

<div class="attribute">

\_\_cause\_\_

A `!TracebackException` of the original `~BaseException.__cause__`.

</div>

<div class="attribute">

\_\_context\_\_

A `!TracebackException` of the original `~BaseException.__context__`.

</div>

<div class="attribute">

exceptions

If `self` represents an `ExceptionGroup`, this field holds a list of `!TracebackException` instances representing the nested exceptions. Otherwise it is `None`.

<div class="versionadded">

3.11

</div>

</div>

<div class="attribute">

\_\_suppress_context\_\_

The `~BaseException.__suppress_context__` value from the original exception.

</div>

<div class="attribute">

\_\_notes\_\_

The `~BaseException.__notes__` value from the original exception, or `None` if the exception does not have any notes. If it is not `None` is it formatted in the traceback after the exception string.

<div class="versionadded">

3.11

</div>

</div>

<div class="attribute">

stack

A `StackSummary` representing the traceback.

</div>

<div class="attribute">

exc_type

The class of the original exception.

<div class="deprecated">

3.13

</div>

</div>

<div class="attribute">

exc_type_str

String display of the class of the original exception.

<div class="versionadded">

3.13

</div>

</div>

<div class="attribute">

filename

For syntax errors - the file name where the error occurred.

</div>

<div class="attribute">

lineno

For syntax errors - the line number where the error occurred.

</div>

<div class="attribute">

end_lineno

For syntax errors - the end line number where the error occurred. Can be `None` if not present.

<div class="versionadded">

3.10

</div>

</div>

<div class="attribute">

text

For syntax errors - the text where the error occurred.

</div>

<div class="attribute">

offset

For syntax errors - the offset into the text where the error occurred.

</div>

<div class="attribute">

end_offset

For syntax errors - the end offset into the text where the error occurred. Can be `None` if not present.

<div class="versionadded">

3.10

</div>

</div>

<div class="attribute">

msg

For syntax errors - the compiler error message.

</div>

<div class="classmethod">

from_exception(exc, \*, limit=None, lookup_lines=True, capture_locals=False, compact=False, max_group_width=15, max_group_depth=10)

Capture an exception for later rendering. *limit*, *lookup_lines* and *capture_locals* are as for the `StackSummary` class.

Note that when locals are captured, they are also shown in the traceback.

</div>

<div class="method">

print(\*, file=None, chain=True)

Print to *file* (default `sys.stderr`) the exception information returned by `format`.

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

format(\*, chain=True)

Format the exception.

If *chain* is not `True`, `__cause__` and `__context__` will not be formatted.

The return value is a generator of strings, each ending in a newline and some containing internal newlines. `~traceback.print_exception` is a wrapper around this method which just prints the lines to a file.

</div>

<div class="method">

format_exception_only(\*, show_group=False)

Format the exception part of the traceback.

The return value is a generator of strings, each ending in a newline.

When *show_group* is `False`, the generator emits the exception's message followed by its notes (if it has any). The exception message is normally a single string; however, for `SyntaxError` exceptions, it consists of several lines that (when printed) display detailed information about where the syntax error occurred.

When *show_group* is `True`, and the exception is an instance of `BaseExceptionGroup`, the nested exceptions are included as well, recursively, with indentation relative to their nesting depth.

<div class="versionchanged">

3.11 The exception's `notes <BaseException.__notes__>` are now included in the output.

</div>

<div class="versionchanged">

3.13 Added the *show_group* parameter.

</div>

</div>

</div>

## `!StackSummary` Objects

<div class="versionadded">

3.5

</div>

`!StackSummary` objects represent a call stack ready for formatting.

<div class="StackSummary">

<div class="classmethod">

extract(frame_gen, \*, limit=None, lookup_lines=True, capture_locals=False)

Construct a `!StackSummary` object from a frame generator (such as is returned by `~traceback.walk_stack` or `~traceback.walk_tb`).

If *limit* is supplied, only this many frames are taken from *frame_gen*. If *lookup_lines* is `False`, the returned `FrameSummary` objects will not have read their lines in yet, making the cost of creating the `!StackSummary` cheaper (which may be valuable if it may not actually get formatted). If *capture_locals* is `True` the local variables in each `!FrameSummary` are captured as object representations.

<div class="versionchanged">

3.12 Exceptions raised from `repr` on a local variable (when *capture_locals* is `True`) are no longer propagated to the caller.

</div>

</div>

<div class="classmethod">

from_list(a_list)

Construct a `!StackSummary` object from a supplied list of `FrameSummary` objects or old-style list of tuples. Each tuple should be a 4-tuple with *filename*, *lineno*, *name*, *line* as the elements.

</div>

<div class="method">

format()

Returns a list of strings ready for printing. Each string in the resulting list corresponds to a single `frame <frame-objects>` from the stack. Each string ends in a newline; the strings may contain internal newlines as well, for those items with source text lines.

For long sequences of the same frame and line, the first few repetitions are shown, followed by a summary line stating the exact number of further repetitions.

<div class="versionchanged">

3.6 Long sequences of repeated frames are now abbreviated.

</div>

</div>

<div class="method">

format_frame_summary(frame_summary)

Returns a string for printing one of the `frames <frame-objects>` involved in the stack. This method is called for each `FrameSummary` object to be printed by `StackSummary.format`. If it returns `None`, the frame is omitted from the output.

<div class="versionadded">

3.11

</div>

</div>

</div>

## `!FrameSummary` Objects

<div class="versionadded">

3.5

</div>

A `!FrameSummary` object represents a single `frame <frame-objects>` in a `traceback <traceback-objects>`.

<div class="FrameSummary(filename, lineno, name, *,\ lookup_line=True, locals=None,\ line=None, end_lineno=None, colno=None, end_colno=None)">

Represents a single `frame <frame-objects>` in the `traceback <traceback-objects>` or stack that is being formatted or printed. It may optionally have a stringified version of the frame's locals included in it. If *lookup_line* is `False`, the source code is not looked up until the `!FrameSummary` has the `~FrameSummary.line` attribute accessed (which also happens when casting it to a `tuple`). `~FrameSummary.line` may be directly provided, and will prevent line lookups happening at all. *locals* is an optional local variable mapping, and if supplied the variable representations are stored in the summary for later display.

`!FrameSummary` instances have the following attributes:

<div class="attribute">

FrameSummary.filename

The filename of the source code for this frame. Equivalent to accessing `f.f_code.co_filename <codeobject.co_filename>` on a `frame object <frame-objects>` *f*.

</div>

<div class="attribute">

FrameSummary.lineno

The line number of the source code for this frame.

</div>

<div class="attribute">

FrameSummary.name

Equivalent to accessing `f.f_code.co_name <codeobject.co_name>` on a `frame object <frame-objects>` *f*.

</div>

<div class="attribute">

FrameSummary.line

A string representing the source code for this frame, with leading and trailing whitespace stripped. If the source is not available, it is `None`.

</div>

<div class="attribute">

FrameSummary.end_lineno

The last line number of the source code for this frame. By default, it is set to `lineno` and indexation starts from 1.

<div class="versionchanged">

3.13 The default value changed from `None` to `lineno`.

</div>

</div>

<div class="attribute">

FrameSummary.colno

The column number of the source code for this frame. By default, it is `None` and indexation starts from 0.

</div>

<div class="attribute">

FrameSummary.end_colno

The last column number of the source code for this frame. By default, it is `None` and indexation starts from 0.

</div>

</div>

## Examples of Using the Module-Level Functions

This simple example implements a basic read-eval-print loop, similar to (but less useful than) the standard Python interactive interpreter loop. For a more complete implementation of the interpreter loop, refer to the `code` module. :

    import sys, traceback

    def run_user_code(envdir):
        source = input(">>> ")
        try:
            exec(source, envdir)
        except Exception:
            print("Exception in user code:")
            print("-"*60)
            traceback.print_exc(file=sys.stdout)
            print("-"*60)

    envdir = {}
    while True:
        run_user_code(envdir)

The following example demonstrates the different ways to print and format the exception and traceback:

<div class="testcode">

import sys, traceback

def lumberjack():  
bright_side_of_life()

def bright_side_of_life():  
return tuple()\[0\]

try:  
lumberjack()

except IndexError as exc:  
print("\*\*\* print_tb:") traceback.print_tb(exc.\_\_traceback\_\_, limit=1, file=sys.stdout) print("\*\*\* print_exception:") traceback.print_exception(exc, limit=2, file=sys.stdout) print("\*\*\* print_exc:") traceback.print_exc(limit=2, file=sys.stdout) print("\*\*\* format_exc, first and last line:") formatted_lines = traceback.format_exc().splitlines() print(formatted_lines\[0\]) print(formatted_lines\[-1\]) print("\*\*\* format_exception:") print(repr(traceback.format_exception(exc))) print("\*\*\* extract_tb:") print(repr(traceback.extract_tb(exc.\_\_traceback\_\_))) print("\*\*\* format_tb:") print(repr(traceback.format_tb(exc.\_\_traceback\_\_))) print("\*\*\* tb_lineno:", exc.\_\_traceback\_\_.tb_lineno)

</div>

The output for the example would look similar to this:

<div class="testoutput" data-options="+NORMALIZE_WHITESPACE">

**\* print_tb: File "\<doctest...\>", line 10, in \<module\> lumberjack() \~\~\~\~\~\~\~\~\~~^^**\* print_exception: Traceback (most recent call last): File "\<doctest...\>", line 10, in \<module\> lumberjack() \~\~\~\~\~\~\~\~\~~^^ File "\<doctest...\>", line 4, in lumberjack bright_side_of_life() \~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~~^^ IndexError: tuple index out of range **\* print_exc: Traceback (most recent call last): File "\<doctest...\>", line 10, in \<module\> lumberjack() \~\~\~\~\~\~\~\~\~~^^ File "\<doctest...\>", line 4, in lumberjack bright_side_of_life() \~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~~^^ IndexError: tuple index out of range**\* format_exc, first and last line: Traceback (most recent call last): IndexError: tuple index out of range **\* format_exception: \['Traceback (most recent call last):n', ' File "\<doctest default\[0\]\>", line 10, in \<module\>n lumberjack()n \~\~\~\~\~\~\~\~\~~^^n', ' File "\<doctest default\[0\]\>", line 4, in lumberjackn bright_side_of_life()n \~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~~^^n', ' File "\<doctest default\[0\]\>", line 7, in bright_side_of_lifen return tuple()\[0\]n \~\~\~\~\~\~~^^^n', 'IndexError: tuple index out of rangen'\]**\* extract_tb: \[\<FrameSummary file \<doctest...\>, line 10 in \<module\>\>, \<FrameSummary file \<doctest...\>, line 4 in lumberjack\>, \<FrameSummary file \<doctest...\>, line 7 in bright_side_of_life\>\] **\* format_tb: \[' File "\<doctest default\[0\]\>", line 10, in \<module\>n lumberjack()n \~\~\~\~\~\~\~\~\~~^^n', ' File "\<doctest default\[0\]\>", line 4, in lumberjackn bright_side_of_life()n \~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~\~~^^n', ' File "\<doctest default\[0\]\>", line 7, in bright_side_of_lifen return tuple()\[0\]n \~\~\~\~\~\~~^^^n'\]**\* tb_lineno: 10

</div>

The following example shows the different ways to print and format the stack:

    >>> import traceback
    >>> def another_function():
    ...     lumberstack()
    ...
    >>> def lumberstack():
    ...     traceback.print_stack()
    ...     print(repr(traceback.extract_stack()))
    ...     print(repr(traceback.format_stack()))
    ...
    >>> another_function()
      File "<doctest>", line 10, in <module>
        another_function()
      File "<doctest>", line 3, in another_function
        lumberstack()
      File "<doctest>", line 6, in lumberstack
        traceback.print_stack()
    [('<doctest>', 10, '<module>', 'another_function()'),
     ('<doctest>', 3, 'another_function', 'lumberstack()'),
     ('<doctest>', 7, 'lumberstack', 'print(repr(traceback.extract_stack()))')]
    ['  File "<doctest>", line 10, in <module>\n    another_function()\n',
     '  File "<doctest>", line 3, in another_function\n    lumberstack()\n',
     '  File "<doctest>", line 8, in lumberstack\n    print(repr(traceback.format_stack()))\n']

This last example demonstrates the final few formatting functions:

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> import traceback \>\>\> traceback.format_list(\[('spam.py', 3, '\<module\>', 'spam.eggs()'), ... ('eggs.py', 42, 'eggs', 'return "bacon"')\]) \[' File "spam.py", line 3, in \<module\>n spam.eggs()n', ' File "eggs.py", line 42, in eggsn return "bacon"n'\] \>\>\> an_error = IndexError('tuple index out of range') \>\>\> traceback.format_exception_only(an_error) \['IndexError: tuple index out of rangen'\]

</div>

## Examples of Using `TracebackException`

With the helper class, we have more options:

    >>> import sys
    >>> from traceback import TracebackException
    >>>
    >>> def lumberjack():
    ...     bright_side_of_life()
    ...
    >>> def bright_side_of_life():
    ...     t = "bright", "side", "of", "life"
    ...     return t[5]
    ...
    >>> try:
    ...     lumberjack()
    ... except IndexError as e:
    ...     exc = e
    ...
    >>> try:
    ...     try:
    ...         lumberjack()
    ...     except:
    ...         1/0
    ... except Exception as e:
    ...     chained_exc = e
    ...
    >>> # limit works as with the module-level functions
    >>> TracebackException.from_exception(exc, limit=-2).print()
    Traceback (most recent call last):
      File "<python-input-1>", line 6, in lumberjack
        bright_side_of_life()
        ~~~~~~~~~~~~~~~~~~~^^
      File "<python-input-1>", line 10, in bright_side_of_life
        return t[5]
               ~^^^
    IndexError: tuple index out of range

    >>> # capture_locals adds local variables in frames
    >>> TracebackException.from_exception(exc, limit=-2, capture_locals=True).print()
    Traceback (most recent call last):
      File "<python-input-1>", line 6, in lumberjack
        bright_side_of_life()
        ~~~~~~~~~~~~~~~~~~~^^
      File "<python-input-1>", line 10, in bright_side_of_life
        return t[5]
               ~^^^
        t = ("bright", "side", "of", "life")
    IndexError: tuple index out of range

    >>> # The *chain* kwarg to print() controls whether chained
    >>> # exceptions are displayed
    >>> TracebackException.from_exception(chained_exc).print()
    Traceback (most recent call last):
      File "<python-input-19>", line 4, in <module>
        lumberjack()
        ~~~~~~~~~~^^
      File "<python-input-8>", line 7, in lumberjack
        bright_side_of_life()
        ~~~~~~~~~~~~~~~~~~~^^
      File "<python-input-8>", line 11, in bright_side_of_life
        return t[5]
               ~^^^
    IndexError: tuple index out of range

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<python-input-19>", line 6, in <module>
        1/0
        ~^~
    ZeroDivisionError: division by zero

    >>> TracebackException.from_exception(chained_exc).print(chain=False)
    Traceback (most recent call last):
      File "<python-input-19>", line 6, in <module>
        1/0
        ~^~
    ZeroDivisionError: division by zero

# `!tracemalloc` --- Trace memory allocations

<div class="module" data-synopsis="Trace memory allocations.">

tracemalloc

</div>

<div class="versionadded">

3.4

</div>

**Source code:** `Lib/tracemalloc.py`

------------------------------------------------------------------------

The tracemalloc module is a debug tool to trace memory blocks allocated by Python. It provides the following information:

- Traceback where an object was allocated
- Statistics on allocated memory blocks per filename and per line number: total size, number and average size of allocated memory blocks
- Compute the differences between two snapshots to detect memory leaks

To trace most memory blocks allocated by Python, the module should be started as early as possible by setting the `PYTHONTRACEMALLOC` environment variable to `1`, or by using `-X` `tracemalloc` command line option. The `tracemalloc.start` function can be called at runtime to start tracing Python memory allocations.

By default, a trace of an allocated memory block only stores the most recent frame (1 frame). To store 25 frames at startup: set the `PYTHONTRACEMALLOC` environment variable to `25`, or use the `-X` `tracemalloc=25` command line option.

## Examples

### Display the top 10

Display the 10 files allocating the most memory:

    import tracemalloc

    tracemalloc.start()

    # ... run your application ...

    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics('lineno')

    print("[ Top 10 ]")
    for stat in top_stats[:10]:
        print(stat)

Example of output of the Python test suite:

    [ Top 10 ]
    <frozen importlib._bootstrap>:716: size=4855 KiB, count=39328, average=126 B
    <frozen importlib._bootstrap>:284: size=521 KiB, count=3199, average=167 B
    /usr/lib/python3.4/collections/__init__.py:368: size=244 KiB, count=2315, average=108 B
    /usr/lib/python3.4/unittest/case.py:381: size=185 KiB, count=779, average=243 B
    /usr/lib/python3.4/unittest/case.py:402: size=154 KiB, count=378, average=416 B
    /usr/lib/python3.4/abc.py:133: size=88.7 KiB, count=347, average=262 B
    <frozen importlib._bootstrap>:1446: size=70.4 KiB, count=911, average=79 B
    <frozen importlib._bootstrap>:1454: size=52.0 KiB, count=25, average=2131 B
    <string>:5: size=49.7 KiB, count=148, average=344 B
    /usr/lib/python3.4/sysconfig.py:411: size=48.0 KiB, count=1, average=48.0 KiB

We can see that Python loaded `4855 KiB` data (bytecode and constants) from modules and that the `collections` module allocated `244 KiB` to build `~collections.namedtuple` types.

See `Snapshot.statistics` for more options.

### Compute differences

Take two snapshots and display the differences:

    import tracemalloc
    tracemalloc.start()
    # ... start your application ...

    snapshot1 = tracemalloc.take_snapshot()
    # ... call the function leaking memory ...
    snapshot2 = tracemalloc.take_snapshot()

    top_stats = snapshot2.compare_to(snapshot1, 'lineno')

    print("[ Top 10 differences ]")
    for stat in top_stats[:10]:
        print(stat)

Example of output before/after running some tests of the Python test suite:

    [ Top 10 differences ]
    <frozen importlib._bootstrap>:716: size=8173 KiB (+4428 KiB), count=71332 (+39369), average=117 B
    /usr/lib/python3.4/linecache.py:127: size=940 KiB (+940 KiB), count=8106 (+8106), average=119 B
    /usr/lib/python3.4/unittest/case.py:571: size=298 KiB (+298 KiB), count=589 (+589), average=519 B
    <frozen importlib._bootstrap>:284: size=1005 KiB (+166 KiB), count=7423 (+1526), average=139 B
    /usr/lib/python3.4/mimetypes.py:217: size=112 KiB (+112 KiB), count=1334 (+1334), average=86 B
    /usr/lib/python3.4/http/server.py:848: size=96.0 KiB (+96.0 KiB), count=1 (+1), average=96.0 KiB
    /usr/lib/python3.4/inspect.py:1465: size=83.5 KiB (+83.5 KiB), count=109 (+109), average=784 B
    /usr/lib/python3.4/unittest/mock.py:491: size=77.7 KiB (+77.7 KiB), count=143 (+143), average=557 B
    /usr/lib/python3.4/urllib/parse.py:476: size=71.8 KiB (+71.8 KiB), count=969 (+969), average=76 B
    /usr/lib/python3.4/contextlib.py:38: size=67.2 KiB (+67.2 KiB), count=126 (+126), average=546 B

We can see that Python has loaded `8173 KiB` of module data (bytecode and constants), and that this is `4428 KiB` more than had been loaded before the tests, when the previous snapshot was taken. Similarly, the `linecache` module has cached `940 KiB` of Python source code to format tracebacks, all of it since the previous snapshot.

If the system has little free memory, snapshots can be written on disk using the `Snapshot.dump` method to analyze the snapshot offline. Then use the `Snapshot.load` method reload the snapshot.

### Get the traceback of a memory block

Code to display the traceback of the biggest memory block:

    import tracemalloc

    # Store 25 frames
    tracemalloc.start(25)

    # ... run your application ...

    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics('traceback')

    # pick the biggest memory block
    stat = top_stats[0]
    print("%s memory blocks: %.1f KiB" % (stat.count, stat.size / 1024))
    for line in stat.traceback.format():
        print(line)

Example of output of the Python test suite (traceback limited to 25 frames):

    903 memory blocks: 870.1 KiB
      File "<frozen importlib._bootstrap>", line 716
      File "<frozen importlib._bootstrap>", line 1036
      File "<frozen importlib._bootstrap>", line 934
      File "<frozen importlib._bootstrap>", line 1068
      File "<frozen importlib._bootstrap>", line 619
      File "<frozen importlib._bootstrap>", line 1581
      File "<frozen importlib._bootstrap>", line 1614
      File "/usr/lib/python3.4/doctest.py", line 101
        import pdb
      File "<frozen importlib._bootstrap>", line 284
      File "<frozen importlib._bootstrap>", line 938
      File "<frozen importlib._bootstrap>", line 1068
      File "<frozen importlib._bootstrap>", line 619
      File "<frozen importlib._bootstrap>", line 1581
      File "<frozen importlib._bootstrap>", line 1614
      File "/usr/lib/python3.4/test/support/__init__.py", line 1728
        import doctest
      File "/usr/lib/python3.4/test/test_pickletools.py", line 21
        support.run_doctest(pickletools)
      File "/usr/lib/python3.4/test/regrtest.py", line 1276
        test_runner()
      File "/usr/lib/python3.4/test/regrtest.py", line 976
        display_failure=not verbose)
      File "/usr/lib/python3.4/test/regrtest.py", line 761
        match_tests=ns.match_tests)
      File "/usr/lib/python3.4/test/regrtest.py", line 1563
        main()
      File "/usr/lib/python3.4/test/__main__.py", line 3
        regrtest.main_in_temp_cwd()
      File "/usr/lib/python3.4/runpy.py", line 73
        exec(code, run_globals)
      File "/usr/lib/python3.4/runpy.py", line 160
        "__main__", fname, loader, pkg_name)

We can see that the most memory was allocated in the `importlib` module to load data (bytecode and constants) from modules: `870.1 KiB`. The traceback is where the `importlib` loaded data most recently: on the `import pdb` line of the `doctest` module. The traceback may change if a new module is loaded.

### Pretty top

Code to display the 10 lines allocating the most memory with a pretty output, ignoring `<frozen importlib._bootstrap>` and `<unknown>` files:

    import linecache
    import os
    import tracemalloc

    def display_top(snapshot, key_type='lineno', limit=10):
        snapshot = snapshot.filter_traces((
            tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
            tracemalloc.Filter(False, "<unknown>"),
        ))
        top_stats = snapshot.statistics(key_type)

        print("Top %s lines" % limit)
        for index, stat in enumerate(top_stats[:limit], 1):
            frame = stat.traceback[0]
            print("#%s: %s:%s: %.1f KiB"
                  % (index, frame.filename, frame.lineno, stat.size / 1024))
            line = linecache.getline(frame.filename, frame.lineno).strip()
            if line:
                print('    %s' % line)

        other = top_stats[limit:]
        if other:
            size = sum(stat.size for stat in other)
            print("%s other: %.1f KiB" % (len(other), size / 1024))
        total = sum(stat.size for stat in top_stats)
        print("Total allocated size: %.1f KiB" % (total / 1024))

    tracemalloc.start()

    # ... run your application ...

    snapshot = tracemalloc.take_snapshot()
    display_top(snapshot)

Example of output of the Python test suite:

    Top 10 lines
    #1: Lib/base64.py:414: 419.8 KiB
        _b85chars2 = [(a + b) for a in _b85chars for b in _b85chars]
    #2: Lib/base64.py:306: 419.8 KiB
        _a85chars2 = [(a + b) for a in _a85chars for b in _a85chars]
    #3: collections/__init__.py:368: 293.6 KiB
        exec(class_definition, namespace)
    #4: Lib/abc.py:133: 115.2 KiB
        cls = super().__new__(mcls, name, bases, namespace)
    #5: unittest/case.py:574: 103.1 KiB
        testMethod()
    #6: Lib/linecache.py:127: 95.4 KiB
        lines = fp.readlines()
    #7: urllib/parse.py:476: 71.8 KiB
        for a in _hexdig for b in _hexdig}
    #8: <string>:5: 62.0 KiB
    #9: Lib/_weakrefset.py:37: 60.0 KiB
        self.data = set()
    #10: Lib/base64.py:142: 59.8 KiB
        _b32tab2 = [a + b for a in _b32tab for b in _b32tab]
    6220 other: 3602.8 KiB
    Total allocated size: 5303.1 KiB

See `Snapshot.statistics` for more options.

#### Record the current and peak size of all traced memory blocks

The following code computes two sums like `0 + 1 + 2 + ...` inefficiently, by creating a list of those numbers. This list consumes a lot of memory temporarily. We can use `get_traced_memory` and `reset_peak` to observe the small memory usage after the sum is computed as well as the peak memory usage during the computations:

    import tracemalloc

    tracemalloc.start()

    # Example code: compute a sum with a large temporary list
    large_sum = sum(list(range(100000)))

    first_size, first_peak = tracemalloc.get_traced_memory()

    tracemalloc.reset_peak()

    # Example code: compute a sum with a small temporary list
    small_sum = sum(list(range(1000)))

    second_size, second_peak = tracemalloc.get_traced_memory()

    print(f"{first_size=}, {first_peak=}")
    print(f"{second_size=}, {second_peak=}")

Output:

    first_size=664, first_peak=3592984
    second_size=804, second_peak=29704

Using `reset_peak` ensured we could accurately record the peak during the computation of `small_sum`, even though it is much smaller than the overall peak size of memory blocks since the `start` call. Without the call to `reset_peak`, `second_peak` would still be the peak from the computation `large_sum` (that is, equal to `first_peak`). In this case, both peaks are much higher than the final memory usage, and which suggests we could optimise (by removing the unnecessary call to `list`, and writing `sum(range(...))`).

## API

### Functions

<div class="function">

clear_traces()

Clear traces of memory blocks allocated by Python.

See also `stop`.

</div>

<div class="function">

get_object_traceback(obj)

Get the traceback where the Python object *obj* was allocated. Return a `Traceback` instance, or `None` if the `!tracemalloc` module is not tracing memory allocations or did not trace the allocation of the object.

See also `gc.get_referrers` and `sys.getsizeof` functions.

</div>

<div class="function">

get_traceback_limit()

Get the maximum number of frames stored in the traceback of a trace.

The `!tracemalloc` module must be tracing memory allocations to get the limit, otherwise an exception is raised.

The limit is set by the `start` function.

</div>

<div class="function">

get_traced_memory()

Get the current size and peak size of memory blocks traced by the `!tracemalloc` module as a tuple: `(current: int, peak: int)`.

</div>

<div class="function">

reset_peak()

Set the peak size of memory blocks traced by the `!tracemalloc` module to the current size.

Do nothing if the `!tracemalloc` module is not tracing memory allocations.

This function only modifies the recorded peak size, and does not modify or clear any traces, unlike `clear_traces`. Snapshots taken with `take_snapshot` before a call to `reset_peak` can be meaningfully compared to snapshots taken after the call.

See also `get_traced_memory`.

<div class="versionadded">

3.9

</div>

</div>

<div class="function">

get_tracemalloc_memory()

Get the memory usage in bytes of the `!tracemalloc` module used to store traces of memory blocks. Return an `int`.

</div>

<div class="function">

is_tracing()

`True` if the `!tracemalloc` module is tracing Python memory allocations, `False` otherwise.

See also `start` and `stop` functions.

</div>

<div class="function">

start(nframe: int=1)

Start tracing Python memory allocations: install hooks on Python memory allocators. Collected tracebacks of traces will be limited to *nframe* frames. By default, a trace of a memory block only stores the most recent frame: the limit is `1`. *nframe* must be greater or equal to `1`.

You can still read the original number of total frames that composed the traceback by looking at the `Traceback.total_nframe` attribute.

Storing more than `1` frame is only useful to compute statistics grouped by `'traceback'` or to compute cumulative statistics: see the `Snapshot.compare_to` and `Snapshot.statistics` methods.

Storing more frames increases the memory and CPU overhead of the `!tracemalloc` module. Use the `get_tracemalloc_memory` function to measure how much memory is used by the `!tracemalloc` module.

The `PYTHONTRACEMALLOC` environment variable (`PYTHONTRACEMALLOC=NFRAME`) and the `-X` `tracemalloc=NFRAME` command line option can be used to start tracing at startup.

See also `stop`, `is_tracing` and `get_traceback_limit` functions.

</div>

<div class="function">

stop()

Stop tracing Python memory allocations: uninstall hooks on Python memory allocators. Also clears all previously collected traces of memory blocks allocated by Python.

Call `take_snapshot` function to take a snapshot of traces before clearing them.

See also `start`, `is_tracing` and `clear_traces` functions.

</div>

<div class="function">

take_snapshot()

Take a snapshot of traces of memory blocks allocated by Python. Return a new `Snapshot` instance.

The snapshot does not include memory blocks allocated before the `!tracemalloc` module started to trace memory allocations.

Tracebacks of traces are limited to `get_traceback_limit` frames. Use the *nframe* parameter of the `start` function to store more frames.

The `!tracemalloc` module must be tracing memory allocations to take a snapshot, see the `start` function.

See also the `get_object_traceback` function.

</div>

### DomainFilter

<div class="DomainFilter(inclusive: bool, domain: int)">

Filter traces of memory blocks by their address space (domain).

<div class="versionadded">

3.6

</div>

<div class="attribute">

inclusive

If *inclusive* is `True` (include), match memory blocks allocated in the address space `domain`.

If *inclusive* is `False` (exclude), match memory blocks not allocated in the address space `domain`.

</div>

<div class="attribute">

domain

Address space of a memory block (`int`). Read-only property.

</div>

</div>

### Filter

<div class="Filter(inclusive: bool, filename_pattern: str, lineno: int=None, all_frames: bool=False, domain: int=None)">

Filter on traces of memory blocks.

See the `fnmatch.fnmatch` function for the syntax of *filename_pattern*. The `'.pyc'` file extension is replaced with `'.py'`.

Examples:

- `Filter(True, subprocess.__file__)` only includes traces of the `subprocess` module
- `Filter(False, tracemalloc.__file__)` excludes traces of the `!tracemalloc` module
- `Filter(False, "<unknown>")` excludes empty tracebacks

<div class="versionchanged">

3.5 The `'.pyo'` file extension is no longer replaced with `'.py'`.

</div>

<div class="versionchanged">

3.6 Added the `domain` attribute.

</div>

<div class="attribute">

domain

Address space of a memory block (`int` or `None`).

tracemalloc uses the domain `0` to trace memory allocations made by Python. C extensions can use other domains to trace other resources.

</div>

<div class="attribute">

inclusive

If *inclusive* is `True` (include), only match memory blocks allocated in a file with a name matching `filename_pattern` at line number `lineno`.

If *inclusive* is `False` (exclude), ignore memory blocks allocated in a file with a name matching `filename_pattern` at line number `lineno`.

</div>

<div class="attribute">

lineno

Line number (`int`) of the filter. If *lineno* is `None`, the filter matches any line number.

</div>

<div class="attribute">

filename_pattern

Filename pattern of the filter (`str`). Read-only property.

</div>

<div class="attribute">

all_frames

If *all_frames* is `True`, all frames of the traceback are checked. If *all_frames* is `False`, only the most recent frame is checked.

This attribute has no effect if the traceback limit is `1`. See the `get_traceback_limit` function and `Snapshot.traceback_limit` attribute.

</div>

</div>

### Frame

<div class="Frame">

Frame of a traceback.

The `Traceback` class is a sequence of `Frame` instances.

<div class="attribute">

filename

Filename (`str`).

</div>

<div class="attribute">

lineno

Line number (`int`).

</div>

</div>

### Snapshot

<div class="Snapshot">

Snapshot of traces of memory blocks allocated by Python.

The `take_snapshot` function creates a snapshot instance.

<div class="method">

compare_to(old_snapshot: Snapshot, key_type: str, cumulative: bool=False)

Compute the differences with an old snapshot. Get statistics as a sorted list of `StatisticDiff` instances grouped by *key_type*.

See the `Snapshot.statistics` method for *key_type* and *cumulative* parameters.

The result is sorted from the biggest to the smallest by: absolute value of `StatisticDiff.size_diff`, `StatisticDiff.size`, absolute value of `StatisticDiff.count_diff`, `Statistic.count` and then by `StatisticDiff.traceback`.

</div>

<div class="method">

dump(filename)

Write the snapshot into a file.

Use `load` to reload the snapshot.

</div>

<div class="method">

filter_traces(filters)

Create a new `Snapshot` instance with a filtered `traces` sequence, *filters* is a list of `DomainFilter` and `Filter` instances. If *filters* is an empty list, return a new `Snapshot` instance with a copy of the traces.

All inclusive filters are applied at once, a trace is ignored if no inclusive filters match it. A trace is ignored if at least one exclusive filter matches it.

<div class="versionchanged">

3.6 `DomainFilter` instances are now also accepted in *filters*.

</div>

</div>

<div class="classmethod">

load(filename)

Load a snapshot from a file.

See also `dump`.

</div>

<div class="method">

statistics(key_type: str, cumulative: bool=False)

Get statistics as a sorted list of `Statistic` instances grouped by *key_type*:

| key_type      | description              |
|---------------|--------------------------|
| `'filename'`  | filename                 |
| `'lineno'`    | filename and line number |
| `'traceback'` | traceback                |

If *cumulative* is `True`, cumulate size and count of memory blocks of all frames of the traceback of a trace, not only the most recent frame. The cumulative mode can only be used with *key_type* equal to `'filename'` and `'lineno'`.

The result is sorted from the biggest to the smallest by: `Statistic.size`, `Statistic.count` and then by `Statistic.traceback`.

</div>

<div class="attribute">

traceback_limit

Maximum number of frames stored in the traceback of `traces`: result of the `get_traceback_limit` when the snapshot was taken.

</div>

<div class="attribute">

traces

Traces of all memory blocks allocated by Python: sequence of `Trace` instances.

The sequence has an undefined order. Use the `Snapshot.statistics` method to get a sorted list of statistics.

</div>

</div>

### Statistic

<div class="Statistic">

Statistic on memory allocations.

`Snapshot.statistics` returns a list of `Statistic` instances.

See also the `StatisticDiff` class.

<div class="attribute">

count

Number of memory blocks (`int`).

</div>

<div class="attribute">

size

Total size of memory blocks in bytes (`int`).

</div>

<div class="attribute">

traceback

Traceback where the memory block was allocated, `Traceback` instance.

</div>

</div>

### StatisticDiff

<div class="StatisticDiff">

Statistic difference on memory allocations between an old and a new `Snapshot` instance.

`Snapshot.compare_to` returns a list of `StatisticDiff` instances. See also the `Statistic` class.

<div class="attribute">

count

Number of memory blocks in the new snapshot (`int`): `0` if the memory blocks have been released in the new snapshot.

</div>

<div class="attribute">

count_diff

Difference of number of memory blocks between the old and the new snapshots (`int`).

</div>

<div class="attribute">

size

Total size of memory blocks in bytes in the new snapshot (`int`): `0` if the memory blocks have been released in the new snapshot.

</div>

<div class="attribute">

size_diff

Difference of total size of memory blocks in bytes between the old and the new snapshots (`int`).

</div>

<div class="attribute">

traceback

Traceback where the memory blocks were allocated, `Traceback` instance.

</div>

</div>

### Trace

<div class="Trace">

Trace of a memory block.

The `Snapshot.traces` attribute is a sequence of `Trace` instances.

<div class="versionchanged">

3.6 Added the `domain` attribute.

</div>

<div class="attribute">

domain

Address space of a memory block (`int`). Read-only property.

tracemalloc uses the domain `0` to trace memory allocations made by Python. C extensions can use other domains to trace other resources.

</div>

<div class="attribute">

size

Size of the memory block in bytes (`int`).

</div>

<div class="attribute">

traceback

Traceback where the memory block was allocated, `Traceback` instance.

</div>

</div>

### Traceback

<div class="Traceback">

Sequence of `Frame` instances sorted from the oldest frame to the most recent frame.

A traceback contains at least `1` frame. If the `tracemalloc` module failed to get a frame, the filename `"<unknown>"` at line number `0` is used.

When a snapshot is taken, tracebacks of traces are limited to `get_traceback_limit` frames. See the `take_snapshot` function. The original number of frames of the traceback is stored in the `Traceback.total_nframe` attribute. That allows one to know if a traceback has been truncated by the traceback limit.

The `Trace.traceback` attribute is a `Traceback` instance.

<div class="versionchanged">

3.7 Frames are now sorted from the oldest to the most recent, instead of most recent to oldest.

</div>

<div class="attribute">

total_nframe

Total number of frames that composed the traceback before truncation. This attribute can be set to `None` if the information is not available.

</div>

<div class="versionchanged">

3.9 The `Traceback.total_nframe` attribute was added.

</div>

<div class="method">

format(limit=None, most_recent_first=False)

Format the traceback as a list of lines. Use the `linecache` module to retrieve lines from the source code. If *limit* is set, format the *limit* most recent frames if *limit* is positive. Otherwise, format the `abs(limit)` oldest frames. If *most_recent_first* is `True`, the order of the formatted frames is reversed, returning the most recent frame first instead of last.

Similar to the `traceback.format_tb` function, except that `.format` does not include newlines.

Example:

    print("Traceback (most recent call first):")
    for line in traceback:
        print(line)

Output:

    Traceback (most recent call first):
      File "test.py", line 9
        obj = Object()
      File "test.py", line 12
        tb = tracemalloc.get_object_traceback(f())

</div>

</div>

# `!tty` --- Terminal control functions

<div class="module" data-synopsis="Utility functions that perform common terminal control operations.">

tty

</div>

**Source code:** `Lib/tty.py`

------------------------------------------------------------------------

The `!tty` module defines functions for putting the tty into cbreak and raw modes.

<div class="availability">

Unix.

</div>

Because it requires the `termios` module, it will work only on Unix.

The `!tty` module defines the following functions:

<div class="function">

cfmakeraw(mode)

Convert the tty attribute list *mode*, which is a list like the one returned by `termios.tcgetattr`, to that of a tty in raw mode.

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

cfmakecbreak(mode)

Convert the tty attribute list *mode*, which is a list like the one returned by `termios.tcgetattr`, to that of a tty in cbreak mode.

This clears the `ECHO` and `ICANON` local mode flags in *mode* as well as setting the minimum input to 1 byte with no delay.

<div class="versionadded">

3.12

</div>

<div class="versionchanged">

3.12.2 The `ICRNL` flag is no longer cleared. This matches Linux and macOS `stty cbreak` behavior and what `setcbreak` historically did.

</div>

</div>

<div class="function">

setraw(fd, when=termios.TCSAFLUSH)

Change the mode of the file descriptor *fd* to raw. If *when* is omitted, it defaults to `termios.TCSAFLUSH`, and is passed to `termios.tcsetattr`. The return value of `termios.tcgetattr` is saved before setting *fd* to raw mode; this value is returned.

<div class="versionchanged">

3.12 The return value is now the original tty attributes, instead of `None`.

</div>

</div>

<div class="function">

setcbreak(fd, when=termios.TCSAFLUSH)

Change the mode of file descriptor *fd* to cbreak. If *when* is omitted, it defaults to `termios.TCSAFLUSH`, and is passed to `termios.tcsetattr`. The return value of `termios.tcgetattr` is saved before setting *fd* to cbreak mode; this value is returned.

This clears the `ECHO` and `ICANON` local mode flags as well as setting the minimum input to 1 byte with no delay.

<div class="versionchanged">

3.12 The return value is now the original tty attributes, instead of `None`.

</div>

<div class="versionchanged">

3.12.2 The `ICRNL` flag is no longer cleared. This restores the behavior of Python 3.11 and earlier as well as matching what Linux, macOS, & BSDs describe in their `stty(1)` man pages regarding cbreak mode.

</div>

</div>

<div class="seealso">

Module `termios`  
Low-level terminal control interface.

</div>

# `!turtle` --- Turtle graphics

<div class="module" data-synopsis="An educational framework for simple graphics applications">

turtle

</div>

**Source code:** `Lib/turtle.py`

<div class="testsetup" data-skipif="_tkinter is None">

default

from turtle import \* turtle = Turtle()

</div>

<div class="testcleanup" data-skipif="_tkinter is None">

import os os.remove("my_drawing.ps") \# Destroy the turtle window after tests are complete \# Imported via star import in testsetup bye()

</div>

------------------------------------------------------------------------

<div class="sidebar">

**Turtle star**

Turtle can draw intricate shapes using programs that repeat simple moves.

<img src="turtle-star.png" class="align-center" alt="A yellow starburst of thin spikes with a red outline, drawn by turtle." />

</div>

Imagine a robotic turtle starting at (0, 0) in the x-y plane. After an `import turtle`, give it the command `turtle.forward(15)`, and it moves (on-screen!) 15 pixels in the direction it is facing, drawing a line as it moves. Give it the command `turtle.right(25)`, and it rotates in-place 25 degrees clockwise.

Turtle graphics is an implementation of [the drawing tools introduced in Logo](https://en.wikipedia.org/wiki/Turtle_(robot)) in 1967. It was created as an educational tool, and its instant, visible feedback makes it an effective way for learners to encounter programming concepts. It is also a convenient way to produce simple graphical output without bringing in external libraries.

This document includes four main sections:

- `turtle-tutorial` teaches the basics of turtle drawing.
- `turtle-reference` describes the functions, methods and classes this module defines.
- `turtle-howtos` detail how to handle specific tasks.
- `turtle-explanation` provides background on the object-oriented interface.

> [!NOTE]
> Turtle graphics requires the `tkinter` `optional module`. The python.org installers for Windows and macOS include it, but some Linux distributions and other platforms may package it separately. If `import turtle` fails with an error mentioning `_tkinter`, look for documentation from your distributor (that is, whoever provided Python to you). Check this in advance if you're planning to use turtle graphics with a learner.

## Tutorial<span id="get-started"></span><span id="turtle-tutorial"></span>

New users should start here. In this tutorial we'll explore some of the basics of turtle drawing.

### Starting a turtle environment

In a Python shell, import all the objects of the `turtle` module:

    from turtle import *

If you run into a `Standard library module '_tkinter' was not found` error, you'll have to install the `Tk interface package <tkinter>` on your system.

Nothing appears yet. To open the turtle window, make the turtle visible:

    showturtle()

A new window opens, with the turtle in the middle, shown as an arrow heading East.

### Basic drawing

Send the turtle forward 100 steps:

    forward(100)

You should see a line drawn by the turtle, heading East. Change the direction of the turtle, so that it turns 120 degrees left (anti-clockwise):

    left(120)

Let's continue by drawing a triangle:

    forward(100)
    left(120)
    forward(100)

Notice how the turtle, represented by an arrow, points in different directions as you steer it.

Experiment with those commands, and also with `backward()` and `right()`. Many commands also have shorter aliases, such as `fd()` for `forward`.

#### Pen control

Try changing the color - for example, `color('blue')` - and width of the line - for example, `width(3)` - and then drawing again.

You can also move the turtle around without drawing, by lifting up the pen: `up()` before moving. To start drawing again, use `down()`.

#### The turtle's position

Send your turtle back to its starting-point (useful if it has disappeared off-screen):

    home()

The home position is at the center of the turtle's screen. If you ever need to know them, get the turtle's x-y coordinates with:

    pos()

Home is at `(0, 0)`.

And after a while, it will probably help to clear the window so we can start anew:

    clearscreen()

### Making algorithmic patterns

Using loops, it's possible to build up geometric patterns:

    for steps in range(100):
        for c in ('blue', 'red', 'green'):
            color(c)
            forward(steps)
            right(30)

\- which of course, are limited only by the imagination!

Let's draw the star shape at the top of this page. We want red lines, filled in with yellow:

    color('red')
    fillcolor('yellow')

Just as `up()` and `down()` determine whether lines will be drawn, filling can be turned on and off:

    begin_fill()

Next we'll create a loop:

    start = pos()

    while True:
        forward(200)
        left(170)
        if distance(start) < 1:
            break

`distance(start) < 1` is a good way to know when the turtle is back at its start position.

Finally, complete the filling:

    end_fill()

(Note that filling only actually takes place when you give the `end_fill()` command.)

## Reference<span id="turtle-reference"></span>

### Turtle methods and functions<span id="turtle-methods"></span>

Most of the examples in this section refer to a Turtle instance called `turtle`.

#### Move and draw

<div class="function">

forward(distance) fd(distance)

param distance  
a number

Move the turtle forward by the specified *distance*, in the direction the turtle is headed.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.position() (0.00,0.00) \>\>\> turtle.forward(25) \>\>\> turtle.position() (25.00,0.00) \>\>\> turtle.forward(-75) \>\>\> turtle.position() (-50.00,0.00)

</div>

</div>

<div class="function">

back(distance) bk(distance) backward(distance)

param distance  
a number

Move the turtle backward by *distance*, opposite to the direction the turtle is headed. The turtle's heading does not change.

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.goto(0, 0)

</div>

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.position() (0.00,0.00) \>\>\> turtle.backward(30) \>\>\> turtle.position() (-30.00,0.00)

</div>

</div>

<div class="function">

right(angle) rt(angle)

param angle  
a number

Turn the turtle right by the specified *angle*. The angle is measured in degrees by default; the unit can be changed with `degrees` or `radians`. How the heading is measured depends on the turtle mode, see `mode`.

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.setheading(22)

</div>

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.heading() 22.0 \>\>\> turtle.right(45) \>\>\> turtle.heading() 337.0

</div>

</div>

<div class="function">

left(angle) lt(angle)

param angle  
a number

Turn the turtle left by the specified *angle*. The angle is measured in degrees by default; the unit can be changed with `degrees` or `radians`. How the heading is measured depends on the turtle mode, see `mode`.

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.setheading(22)

</div>

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.heading() 22.0 \>\>\> turtle.left(45) \>\>\> turtle.heading() 67.0

</div>

</div>

<div class="function">

goto(x, y=None) setpos(x, y=None) setposition(x, y=None)

param x  
a number or a pair/vector of numbers

param y  
a number or `None`

Move the turtle to an absolute position. If *y* is `None`, *x* must be a pair of coordinates or a `Vec2D`, for example as returned by `pos`. If the pen is down, a line is drawn. The turtle's heading does not change.

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.goto(0, 0)

</div>

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> tp = turtle.pos() \>\>\> tp (0.00,0.00) \>\>\> turtle.goto(60,30) \>\>\> turtle.pos() (60.00,30.00) \>\>\> turtle.goto((20,80)) \>\>\> turtle.pos() (20.00,80.00) \>\>\> turtle.goto(tp) \>\>\> turtle.pos() (0.00,0.00)

</div>

</div>

<div class="function">

teleport(x=None, y=None, \*, fill_gap=False)

param x  
a number or `None`

param y  
a number or `None`

param fill_gap  
a boolean

Teleport, or instantly move, the turtle to an absolute position. Unlike `goto`, no line is drawn. If *x* or *y* is not given, that coordinate is kept as it is. The turtle's heading does not change.

If the turtle is filling a shape, the shape drawn so far is finished and filled before teleporting, and filling starts again at the new position. Pass `fill_gap=True` to keep filling one shape instead. The teleport then counts as an edge of the shape, as it would with `goto`.

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.goto(0, 0)

</div>

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> tp = turtle.pos() \>\>\> tp (0.00,0.00) \>\>\> turtle.teleport(60) \>\>\> turtle.pos() (60.00,0.00) \>\>\> turtle.teleport(y=10) \>\>\> turtle.pos() (60.00,10.00) \>\>\> turtle.teleport(20, 30) \>\>\> turtle.pos() (20.00,30.00)

</div>

<div class="versionadded">

3.12

</div>

</div>

<div class="function">

setx(x)

param x  
a number

Set the turtle's x coordinate to *x*. The y coordinate is unchanged.

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.goto(0, 240)

</div>

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.position() (0.00,240.00) \>\>\> turtle.setx(10) \>\>\> turtle.position() (10.00,240.00)

</div>

</div>

<div class="function">

sety(y)

param y  
a number

Set the turtle's y coordinate to *y*. The x coordinate is unchanged.

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.goto(0, 40)

</div>

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.position() (0.00,40.00) \>\>\> turtle.sety(-10) \>\>\> turtle.position() (0.00,-10.00)

</div>

</div>

<div class="function">

setheading(to_angle) seth(to_angle)

param to_angle  
a number

Set the turtle's heading to *to_angle*. Here are some common directions in degrees:

| standard mode | logo mode   |
|---------------|-------------|
| 0 - east      | 0 - north   |
| 90 - north    | 90 - east   |
| 180 - west    | 180 - south |
| 270 - south   | 270 - west  |

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.setheading(90) \>\>\> turtle.heading() 90.0

</div>

</div>

<div class="function">

home()

Move the turtle to the origin, coordinates (0,0). The turtle's heading is set to its start orientation, which depends on the turtle mode, see `mode`.

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.setheading(90) \>\>\> turtle.goto(0, -10)

</div>

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.heading() 90.0 \>\>\> turtle.position() (0.00,-10.00) \>\>\> turtle.home() \>\>\> turtle.position() (0.00,0.00) \>\>\> turtle.heading() 0.0

</div>

</div>

<div class="function">

circle(radius, extent=None, steps=None)

param radius  
a number

param extent  
a number or `None`

param steps  
an integer or `None`

Draw a circle with the given *radius*. The center is *radius* units left of the turtle; *extent*, an angle, determines which part of the circle is drawn. If *extent* is not given, draw the entire circle. If *extent* is not a full circle, one endpoint of the arc is the current pen position. Draw the arc in counterclockwise direction if *radius* is positive, otherwise in clockwise direction. Finally, the turtle's heading is changed by *extent*.

As the circle is approximated by an inscribed regular polygon, *steps* determines the number of steps to use. If not given, it will be calculated automatically. May be used to draw regular polygons.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.home() \>\>\> turtle.position() (0.00,0.00) \>\>\> turtle.heading() 0.0 \>\>\> turtle.circle(50) \>\>\> turtle.position() (-0.00,0.00) \>\>\> turtle.heading() 0.0 \>\>\> turtle.circle(120, 180) \# draw a semicircle \>\>\> turtle.position() (0.00,240.00) \>\>\> turtle.heading() 180.0

</div>

</div>

<div class="function">

dot() dot(size) dot(color, /) dot(size, color, /) dot(size, r, g, b, /)

param size  
a positive integer or `None`

param color  
a colorstring or a numeric color tuple

Draw a circular dot with diameter *size*, using *color*. If *size* is not given, the larger of `pensize + 4` and `2 * pensize` is used. If *color* is not given, the current pencolor is used.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.home() \>\>\> turtle.dot() \>\>\> turtle.fd(50); turtle.dot(20, "blue"); turtle.fd(50) \>\>\> turtle.position() (100.00,-0.00) \>\>\> turtle.heading() 0.0

</div>

</div>

<div class="function">

stamp()

Stamp a copy of the turtle shape onto the canvas at the current turtle position. Return a stamp_id for that stamp, which can be used to delete it by calling `clearstamp(stamp_id)`.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.color("blue") \>\>\> stamp_id = turtle.stamp() \>\>\> turtle.fd(50)

</div>

</div>

<div class="function">

clearstamp(stampid)

param stampid  
an integer, must be return value of previous `stamp` call

Delete stamp with given *stampid*.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.position() (150.00,-0.00) \>\>\> turtle.color("blue") \>\>\> astamp = turtle.stamp() \>\>\> turtle.fd(50) \>\>\> turtle.position() (200.00,-0.00) \>\>\> turtle.clearstamp(astamp) \>\>\> turtle.position() (200.00,-0.00)

</div>

</div>

<div class="function">

clearstamps(n=None)

param n  
an integer (or `None`)

Delete all or first/last *n* of turtle's stamps. If *n* is `None`, delete all stamps, if *n* \> 0 delete first *n* stamps, else if *n* \< 0 delete last *n* stamps.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> for i in range(8): ... unused_stamp_id = turtle.stamp() ... turtle.fd(30) \>\>\> turtle.clearstamps(2) \>\>\> turtle.clearstamps(-2) \>\>\> turtle.clearstamps()

</div>

</div>

<div class="function">

undo()

Undo (repeatedly) the last turtle action(s). Number of available undo actions is determined by the size of the undobuffer.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> for i in range(4): ... turtle.fd(50); turtle.lt(80) ... \>\>\> for i in range(8): ... turtle.undo()

</div>

</div>

<div class="function">

speed(speed=None)

param speed  
an integer, a string or `None`

Return or set the turtle's speed. If *speed* is not given, return the current speed. Otherwise set the turtle's speed to an integer from 0 to 10. If *speed* is a number greater than 10 or smaller than 0.5, the speed is set to 0. You can also use these names instead of numbers:

- "fastest": 0
- "fast": 10
- "normal": 6
- "slow": 3
- "slowest": 1

Speeds from 1 to 10 give increasingly faster animation of line drawing and turtle turning. A speed of 0 means that no animation takes place: `forward` and `back` make the turtle jump, and likewise `left` and `right` make the turtle turn instantly.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.speed() 3 \>\>\> turtle.speed('normal') \>\>\> turtle.speed() 6 \>\>\> turtle.speed(9) \>\>\> turtle.speed() 9

</div>

</div>

#### Tell Turtle's state

<div class="function">

position() pos()

Return the turtle's current location (x,y) as a `Vec2D` vector.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.pos() (440.00,-0.00)

</div>

</div>

<div class="function">

towards(x, y=None)

param x  
a number or a pair/vector of numbers or a turtle instance

param y  
a number if *x* is a number, else `None`

Return the angle of the line from the turtle's position to (x,y). If *y* is `None`, *x* must be a pair of coordinates, a `Vec2D`, for example as returned by `pos`, or another turtle. The angle is measured from the turtle's start orientation, which depends on the turtle mode, see `mode`.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.goto(10, 10) \>\>\> turtle.towards(0,0) 225.0

</div>

</div>

<div class="function">

xcor()

Return the turtle's x coordinate.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.home() \>\>\> turtle.left(50) \>\>\> turtle.forward(100) \>\>\> turtle.pos() (64.28,76.60) \>\>\> print(round(turtle.xcor(), 5)) 64.27876

</div>

</div>

<div class="function">

ycor()

Return the turtle's y coordinate.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.home() \>\>\> turtle.left(60) \>\>\> turtle.forward(100) \>\>\> print(turtle.pos()) (50.00,86.60) \>\>\> print(round(turtle.ycor(), 5)) 86.60254

</div>

</div>

<div class="function">

heading()

Return the turtle's current heading. The value depends on the turtle mode, see `mode`.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.home() \>\>\> turtle.left(67) \>\>\> turtle.heading() 67.0

</div>

</div>

<div class="function">

distance(x, y=None)

param x  
a number or a pair/vector of numbers or a turtle instance

param y  
a number if *x* is a number, else `None`

Return the distance from the turtle to (x,y) in turtle step units. If *y* is `None`, *x* must be a pair of coordinates, a `Vec2D`, for example as returned by `pos`, or another turtle.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.home() \>\>\> turtle.distance(30,40) 50.0 \>\>\> turtle.distance((30,40)) 50.0 \>\>\> joe = Turtle() \>\>\> joe.forward(77) \>\>\> turtle.distance(joe) 77.0

</div>

</div>

#### Settings for measurement

<div class="function">

degrees(fullcircle=360.0)

param fullcircle  
a number

Set the angle measurement units to degrees. The number of degrees in a full circle is set to *fullcircle*, which defaults to 360.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.home() \>\>\> turtle.left(90) \>\>\> turtle.heading() 90.0

\>\>\> \# Change angle measurement unit to grad (also known as gon, \>\>\> \# grade, or gradian and equals 1/100-th of the right angle.) \>\>\> turtle.degrees(400.0) \>\>\> turtle.heading() 100.0 \>\>\> turtle.degrees(360) \>\>\> turtle.heading() 90.0

</div>

</div>

<div class="function">

radians()

Set the angle measurement units to radians. Equivalent to `degrees(2 * math.pi)`.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.home() \>\>\> turtle.left(90) \>\>\> turtle.heading() 90.0 \>\>\> turtle.radians() \>\>\> turtle.heading() 1.5707963267948966

</div>

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.degrees(360)

</div>

</div>

#### Pen control

##### Drawing state

<div class="function">

pendown() pd() down()

Lower the pen. The turtle draws a line when it moves.

</div>

<div class="function">

penup() pu() up()

Lift the pen. The turtle does not draw when it moves.

</div>

<div class="function">

pensize(width=None) width(width=None)

param width  
a positive number or `None`

Return or set the line thickness. If *width* is not given, return the current pensize. If resizemode is set to "auto" and the turtle shape is a polygon, that polygon is drawn with the same line thickness.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.pensize() 1 \>\>\> turtle.pensize(10) \# from here on lines of width 10 are drawn

</div>

</div>

<div class="function">

pen(pen=None, \*\*pendict)

param pen  
a dictionary with some or all of the below listed keys

param pendict  
one or more keyword-arguments with the below listed keys as keywords

Return or set the pen's attributes in a "pen-dictionary" with the following key/value pairs:

- "shown": True/False
- "pendown": True/False
- "pencolor": color-string or color-tuple
- "fillcolor": color-string or color-tuple
- "pensize": positive number
- "speed": number in range 0..10
- "resizemode": "auto" or "user" or "noresize"
- "stretchfactor": (positive number, positive number)
- "outline": positive number
- "tilt": number

This dictionary can be used as argument for a subsequent call to `pen` to restore the former pen-state. Moreover one or more of these attributes can be provided as keyword-arguments. This can be used to set several pen attributes in one statement.

<div class="doctest" data-skipif="_tkinter is None" data-options="+NORMALIZE_WHITESPACE">

\>\>\> turtle.pen(fillcolor="black", pencolor="red", pensize=10) \>\>\> sorted(turtle.pen().items()) \[('fillcolor', 'black'), ('outline', 1), ('pencolor', 'red'), ('pendown', True), ('pensize', 10), ('resizemode', 'noresize'), ('shearfactor', 0.0), ('shown', True), ('speed', 9), ('stretchfactor', (1.0, 1.0)), ('tilt', 0.0)\] \>\>\> penstate=turtle.pen() \>\>\> turtle.color("yellow", "") \>\>\> turtle.penup() \>\>\> sorted(turtle.pen().items())\[:3\] \[('fillcolor', ''), ('outline', 1), ('pencolor', 'yellow')\] \>\>\> turtle.pen(penstate, fillcolor="green") \>\>\> sorted(turtle.pen().items())\[:3\] \[('fillcolor', 'green'), ('outline', 1), ('pencolor', 'red')\]

</div>

</div>

<div class="function">

isdown()

Return `True` if the pen is down, `False` if it's up.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.penup() \>\>\> turtle.isdown() False \>\>\> turtle.pendown() \>\>\> turtle.isdown() True

</div>

</div>

##### Color control

<div class="function">

pencolor() pencolor(color, /) pencolor(r, g, b, /)

Return or set the pencolor.

Four input formats are allowed:

`pencolor()`  
Return the current pencolor as color specification string or as a tuple (see example). May be used as input to another `color`, `pencolor`, `fillcolor` or `bgcolor` call.

`pencolor(colorstring)`  
Set the pencolor to *colorstring*, which is a Tk color specification string, such as `"red"`, `"yellow"`, or `"#33cc8c"`.

`pencolor((r, g, b))`  
Set the pencolor to the RGB color represented by the tuple of *r*, *g*, and *b*. Each of *r*, *g*, and *b* must be between 0 and colormode, where colormode is either 1.0 or 255 (see `colormode`).

`pencolor(r, g, b)`  
Set the pencolor to the RGB color represented by *r*, *g*, and *b*. Each of *r*, *g*, and *b* must be between 0 and colormode.

If the turtle shape is a polygon, the outline of that polygon is drawn with the newly set pencolor.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> colormode() 1.0 \>\>\> turtle.pencolor() 'red' \>\>\> turtle.pencolor("brown") \>\>\> turtle.pencolor() 'brown' \>\>\> tup = (0.2, 0.8, 0.55) \>\>\> turtle.pencolor(tup) \>\>\> turtle.pencolor() (0.2, 0.8, 0.5490196078431373) \>\>\> colormode(255) \>\>\> turtle.pencolor() (51.0, 204.0, 140.0) \>\>\> turtle.pencolor('#32c18f') \>\>\> turtle.pencolor() (50.0, 193.0, 143.0)

</div>

</div>

<div class="function">

fillcolor() fillcolor(color, /) fillcolor(r, g, b, /)

Return or set the fillcolor.

Four input formats are allowed:

`fillcolor()`  
Return the current fillcolor as color specification string, possibly in tuple format (see example). May be used as input to another `color`, `pencolor`, `fillcolor` or `bgcolor` call.

`fillcolor(colorstring)`  
Set the fillcolor to *colorstring*, which is a Tk color specification string, such as `"red"`, `"yellow"`, or `"#33cc8c"`.

`fillcolor((r, g, b))`  
Set the fillcolor to the RGB color represented by the tuple of *r*, *g*, and *b*. Each of *r*, *g*, and *b* must be between 0 and colormode, where colormode is either 1.0 or 255 (see `colormode`).

`fillcolor(r, g, b)`  
Set the fillcolor to the RGB color represented by *r*, *g*, and *b*. Each of *r*, *g*, and *b* must be between 0 and colormode.

If the turtle shape is a polygon, the interior of that polygon is drawn with the newly set fillcolor.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.fillcolor("violet") \>\>\> turtle.fillcolor() 'violet' \>\>\> turtle.pencolor() (50.0, 193.0, 143.0) \>\>\> turtle.fillcolor((50, 193, 143)) \# Integers, not floats \>\>\> turtle.fillcolor() (50.0, 193.0, 143.0) \>\>\> turtle.fillcolor('#ffffff') \>\>\> turtle.fillcolor() (255.0, 255.0, 255.0)

</div>

</div>

<div class="function">

color() color(color, /) color(r, g, b, /) color(pencolor, fillcolor, /)

Return or set the pencolor and fillcolor.

Several input formats are allowed. They use 0 to 3 arguments as follows:

`color()`  
Return the current pencolor and the current fillcolor as a pair of color specification strings or tuples as returned by `pencolor` and `fillcolor`.

`color(colorstring)`, `color((r,g,b))`, `color(r,g,b)`  
Inputs as in `pencolor`. Set both the pencolor and the fillcolor to the given value.

`color(colorstring1, colorstring2)`, `color((r1,g1,b1), (r2,g2,b2))`  
Equivalent to `pencolor(colorstring1)` and `fillcolor(colorstring2)` and analogously if the other input format is used.

If the turtle shape is a polygon, the outline and interior of that polygon are drawn with the newly set colors.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.color("red", "green") \>\>\> turtle.color() ('red', 'green') \>\>\> color("#285078", "#a0c8f0") \>\>\> color() ((40.0, 80.0, 120.0), (160.0, 200.0, 240.0))

</div>

</div>

See also: Screen method `colormode`.

##### Filling

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> turtle.home()

</div>

<div class="function">

filling()

Return `True` if the turtle is filling, `False` otherwise.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.begin_fill() \>\>\> if turtle.filling(): ... turtle.pensize(5) ... else: ... turtle.pensize(3)

</div>

</div>

<div class="function">

fill()

Fill the shape drawn in the `with turtle.fill():` block.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.color("black", "red") \>\>\> with turtle.fill(): ... turtle.circle(80)

</div>

Using `!fill` is equivalent to adding the `begin_fill` before the fill-block and `end_fill` after the fill-block:

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.color("black", "red") \>\>\> turtle.begin_fill() \>\>\> turtle.circle(80) \>\>\> turtle.end_fill()

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

begin_fill()

To be called just before drawing a shape to be filled.

</div>

<div class="function">

end_fill()

Fill the shape drawn after the last call to `begin_fill`.

Whether or not overlap regions for self-intersecting polygons or multiple shapes are filled depends on the operating system graphics, type of overlap, and number of overlaps. For example, the Turtle star above may be either all yellow or have some white regions.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.color("black", "red") \>\>\> turtle.begin_fill() \>\>\> turtle.circle(80) \>\>\> turtle.end_fill()

</div>

</div>

##### More drawing control

<div class="function">

reset()

Delete the turtle's drawings and re-center the turtle. The turtle's settings are restored to their default values.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.goto(0,-22) \>\>\> turtle.left(100) \>\>\> turtle.position() (0.00,-22.00) \>\>\> turtle.heading() 100.0 \>\>\> turtle.reset() \>\>\> turtle.position() (0.00,0.00) \>\>\> turtle.heading() 0.0

</div>

</div>

<div class="function">

clear()

Delete the turtle's drawings from the screen. The state and position of the turtle as well as the drawings of other turtles are not affected.

</div>

<div class="function">

write(arg, move=False, align="left", font=("Arial", 8, "normal"))

param arg  
object to be written to the TurtleScreen

param move  
True/False

param align  
one of the strings "left", "center" or right"

param font  
a triple (fontname, fontsize, fonttype)

Write text - the string representation of *arg* - at the current turtle position according to *align* ("left", "center" or "right") and with the given font. If *move* is true, the pen is moved to the bottom-right corner of the text. By default, *move* is `False`.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.write("Home = ", True, align="center") \>\>\> turtle.write((0,0), True)

</div>

</div>

#### Turtle state

##### Visibility

<div class="function">

hideturtle() ht()

Make the turtle invisible. If the pen is down, movement still draws.

</div>

<div class="function">

showturtle() st()

Make the turtle visible.

</div>

<div class="function">

isvisible()

Return `True` if the turtle is visible, `False` if it's hidden.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.hideturtle() \>\>\> turtle.isvisible() False \>\>\> turtle.showturtle() \>\>\> turtle.isvisible() True

</div>

</div>

##### Appearance

<div class="function">

shape(name=None)

param name  
a string or `None`

Set the turtle shape to the named shape. If *name* is not given, return the name of the current shape. The shape must exist in the `TurtleScreen`'s shape dictionary. Initially there are the following shapes: "arrow", "turtle", "circle", "square", "triangle" and "classic". To add shapes, see the `Screen` method `register_shape`.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.shape() 'classic' \>\>\> turtle.shape("turtle") \>\>\> turtle.shape() 'turtle'

</div>

</div>

<div class="function">

resizemode(rmode=None)

param rmode  
a string or `None`

Set the resizemode to "auto", "user" or "noresize". If *rmode* is not given, return the current resizemode. The different resizemodes have the following effects:

- "auto": the turtle is resized according to the value of `pensize`.
- "user": the turtle is resized according to the stretch factors and outline width, which are set by `shapesize`.
- "noresize": the turtle is not resized.

`resizemode("user")` is called by `shapesize` when used with arguments.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.resizemode() 'noresize' \>\>\> turtle.resizemode("auto") \>\>\> turtle.resizemode() 'auto'

</div>

</div>

<div class="function">

shapesize(stretch_wid=None, stretch_len=None, outline=None) turtlesize(stretch_wid=None, stretch_len=None, outline=None)

param stretch_wid  
a positive number or `None`

param stretch_len  
a positive number or `None`

param outline  
a positive number or `None`

Return or set the turtle's stretch factors and outline width. If no arguments are given, return the current values. Otherwise set resizemode to "user", so that the turtle is displayed stretched according to its stretch factors:

- *stretch_wid*: the stretch factor perpendicular to the turtle's heading
- *stretch_len*: the stretch factor in the direction of the turtle's heading
- *outline*: the width of the shape's outline

For example, `shapesize(2, 1)` makes the turtle twice as wide without changing its length. If only *stretch_wid* is given, *stretch_len* is set to the same value.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.shapesize() (1.0, 1.0, 1) \>\>\> turtle.resizemode("user") \>\>\> turtle.shapesize(5, 5, 12) \>\>\> turtle.shapesize() (5, 5, 12) \>\>\> turtle.shapesize(outline=8) \>\>\> turtle.shapesize() (5, 5, 8)

</div>

</div>

<div class="function">

shearfactor(shear=None)

param shear  
number (optional)

Set or return the current shearfactor. Shear the turtleshape according to the given shearfactor shear, which is the tangent of the shear angle. Do *not* change the turtle's heading (direction of movement). If shear is not given: return the current shearfactor, i. e. the tangent of the shear angle, by which lines parallel to the heading of the turtle are sheared.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.shape("circle") \>\>\> turtle.shapesize(5,2) \>\>\> turtle.shearfactor(0.5) \>\>\> turtle.shearfactor() 0.5

</div>

</div>

<div class="function">

tilt(angle)

param angle  
a number

Rotate the turtleshape by *angle* from its current tilt-angle, but do *not* change the turtle's heading (direction of movement).

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.reset() \>\>\> turtle.shape("circle") \>\>\> turtle.shapesize(5,2) \>\>\> turtle.tilt(30) \>\>\> turtle.fd(50) \>\>\> turtle.tilt(30) \>\>\> turtle.fd(50)

</div>

</div>

<div class="function">

tiltangle(angle=None)

param angle  
a number (optional)

Set or return the current tilt-angle. If angle is given, rotate the turtleshape to point in the direction specified by angle, regardless of its current tilt-angle. Do *not* change the turtle's heading (direction of movement). If angle is not given: return the current tilt-angle, i. e. the angle between the orientation of the turtleshape and the heading of the turtle (its direction of movement).

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.reset() \>\>\> turtle.shape("circle") \>\>\> turtle.shapesize(5,2) \>\>\> turtle.tilt(45) \>\>\> turtle.tiltangle() 45.0

</div>

</div>

<div class="function">

shapetransform(t11=None, t12=None, t21=None, t22=None)

param t11  
a number (optional)

param t12  
a number (optional)

param t21  
a number (optional)

param t12  
a number (optional)

Set or return the current transformation matrix of the turtle shape.

If none of the matrix elements are given, return the transformation matrix as a tuple of 4 elements. Otherwise set the given elements and transform the turtleshape according to the matrix consisting of first row t11, t12 and second row t21, t22. The determinant t11 \* t22 - t12 \* t21 must not be zero, otherwise an error is raised. Modify stretchfactor, shearfactor and tiltangle according to the given matrix.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle = Turtle() \>\>\> turtle.shape("square") \>\>\> turtle.shapesize(4,2) \>\>\> turtle.shearfactor(-0.5) \>\>\> turtle.shapetransform() (4.0, -1.0, -0.0, 2.0)

</div>

</div>

<div class="function">

get_shapepoly()

Return the current shape polygon as tuple of coordinate pairs. This can be used to define a new shape or components of a compound shape.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.shape("square") \>\>\> turtle.shapetransform(4, -1, 0, 2) \>\>\> turtle.get_shapepoly() ((50, -20), (30, 20), (-50, 20), (-30, -20))

</div>

</div>

#### Using events

<div class="function" data-noindex="">

onclick(fun, btn=1, add=None)

param fun  
a function with two arguments or `None`

param btn  
an integer

param add  
a boolean or `None`

Bind *fun* to mouse-click events on this turtle. *fun* is called with the coordinates of the clicked point on the canvas. If *fun* is `None`, existing bindings are removed. *btn* is the number of the mouse button and defaults to 1, the left mouse button. If *add* is `True`, a new binding is added, otherwise it replaces a former binding. Example for the anonymous turtle, that is, the procedural way:

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> def turn(x, y): ... left(180) ... \>\>\> onclick(turn) \# Now clicking into the turtle will turn it. \>\>\> onclick(None) \# event-binding will be removed

</div>

</div>

<div class="function">

onrelease(fun, btn=1, add=None)

param fun  
a function with two arguments which will be called with the coordinates of the clicked point on the canvas

param btn  
number of the mouse-button, defaults to 1 (left mouse button)

param add  
`True` or `False` -- if `True`, a new binding will be added, otherwise it will replace a former binding

Bind *fun* to mouse-button-release events on this turtle. If *fun* is `None`, existing bindings are removed.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> class MyTurtle(Turtle): ... def glow(self,x,y): ... self.fillcolor("red") ... def unglow(self,x,y): ... self.fillcolor("") ... \>\>\> turtle = MyTurtle() \>\>\> turtle.onclick(turtle.glow) \# clicking on turtle turns fillcolor red, \>\>\> turtle.onrelease(turtle.unglow) \# releasing turns it to transparent.

</div>

</div>

<div class="function">

ondrag(fun, btn=1, add=None)

param fun  
a function with two arguments which will be called with the coordinates of the clicked point on the canvas

param btn  
number of the mouse-button, defaults to 1 (left mouse button)

param add  
`True` or `False` -- if `True`, a new binding will be added, otherwise it will replace a former binding

Bind *fun* to mouse-move events on this turtle. If *fun* is `None`, existing bindings are removed.

Remark: Every sequence of mouse-move-events on a turtle is preceded by a mouse-click event on that turtle.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.ondrag(turtle.goto)

</div>

Subsequently, clicking and dragging the Turtle will move it across the screen thereby producing handdrawings (if pen is down).

</div>

#### Special Turtle methods

<div class="function">

poly()

Record the vertices of a polygon drawn in the `with turtle.poly():` block. The first and last vertices will be connected.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> with turtle.poly(): ... turtle.forward(100) ... turtle.right(60) ... turtle.forward(100)

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

begin_poly()

Start recording the vertices of a polygon. The current turtle position is the first vertex of the polygon.

</div>

<div class="function">

end_poly()

Stop recording the vertices of a polygon. The current turtle position is the last vertex of the polygon. This will be connected with the first vertex.

</div>

<div class="function">

get_poly()

Return the last recorded polygon.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.home() \>\>\> turtle.begin_poly() \>\>\> turtle.fd(100) \>\>\> turtle.left(20) \>\>\> turtle.fd(30) \>\>\> turtle.left(60) \>\>\> turtle.fd(50) \>\>\> turtle.end_poly() \>\>\> p = turtle.get_poly() \>\>\> register_shape("myFavouriteShape", p)

</div>

</div>

<div class="function">

clone()

Create and return a clone of the turtle. The clone has the same position, heading and turtle properties.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> leonel = Turtle() \>\>\> marco = leonel.clone()

</div>

</div>

<div class="function">

getturtle() getpen()

Return the Turtle object itself.

Call `getturtle()` to get the turtle used by functions such as `forward` and `left`:

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> pet = getturtle() \>\>\> pet.fd(50) \>\>\> pet \<turtle.Turtle object at 0x...\>

</div>

</div>

<div class="function">

getscreen()

Return the `TurtleScreen` object the turtle is drawing on. TurtleScreen methods can then be called for that object.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> ts = turtle.getscreen() \>\>\> ts \<turtle.\_Screen object at 0x...\> \>\>\> ts.bgcolor("pink")

</div>

</div>

<div class="function">

setundobuffer(size)

param size  
an integer or `None`

Set or disable the undo buffer. If *size* is an integer, an empty undo buffer of the given size is installed. *size* is the maximum number of turtle actions that can be undone by `undo`. If *size* is `None`, the undo buffer is disabled.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.setundobuffer(42)

</div>

</div>

<div class="function">

undobufferentries()

Return the number of entries in the undo buffer.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> while undobufferentries(): ... undo()

</div>

</div>

### Screen methods and functions<span id="methods-of-turtlescreen-screen"></span>

Most of the examples in this section refer to a TurtleScreen instance called `screen`.

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> screen = Screen()

</div>

#### Window control

<div class="function">

bgcolor() bgcolor(color, /) bgcolor(r, g, b, /)

Return or set the background color of the TurtleScreen.

Four input formats are allowed:

`bgcolor()`  
Return the current background color as a color specification string or as a tuple (see example). May be used as input to another `color`, `pencolor`, `fillcolor` or `bgcolor` call.

`bgcolor(colorstring)`  
Set the background color to *colorstring*, which is a Tk color specification string, such as `"red"`, `"yellow"`, or `"#33cc8c"`.

`bgcolor((r, g, b))`  
Set the background color to the RGB color represented by the tuple of *r*, *g*, and *b*. Each of *r*, *g*, and *b* must be between 0 and colormode, where colormode is either 1.0 or 255 (see `colormode`).

`bgcolor(r, g, b)`  
Set the background color to the RGB color represented by *r*, *g*, and *b*. Each of *r*, *g*, and *b* must be between 0 and colormode.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.bgcolor("orange") \>\>\> screen.bgcolor() 'orange' \>\>\> screen.bgcolor("#800080") \>\>\> screen.bgcolor() (128.0, 0.0, 128.0)

</div>

</div>

<div class="function">

bgpic(picname=None)

param picname  
a string, name of an image file (PNG, GIF, PGM, and PPM) or `"nopic"`, or `None`

Set background image or return name of current backgroundimage. If *picname* is a filename, set the corresponding image as background. If *picname* is `"nopic"`, delete background image, if present. If *picname* is `None`, return the filename of the current backgroundimage. :

    >>> screen.bgpic()
    'nopic'
    >>> screen.bgpic("landscape.gif")
    >>> screen.bgpic()
    "landscape.gif"

</div>

<div class="function" data-noindex="">

clear()

> [!NOTE]
> This TurtleScreen method is available as a global function only under the name `clearscreen`. The global function `clear` is a different one derived from the Turtle method `clear`.

</div>

<div class="function">

clearscreen()

Delete all drawings and all turtles from the TurtleScreen. Reset the now empty TurtleScreen to its initial state: white background, no background image, no event bindings and tracing on.

</div>

<div class="function" data-noindex="">

reset()

> [!NOTE]
> This TurtleScreen method is available as a global function only under the name `resetscreen`. The global function `reset` is another one derived from the Turtle method `reset`.

</div>

<div class="function">

resetscreen()

Reset all turtles on the screen to their initial state.

</div>

<div class="function">

screensize(canvwidth=None, canvheight=None, bg=None)

param canvwidth  
positive integer, new width of canvas in pixels

param canvheight  
positive integer, new height of canvas in pixels

param bg  
colorstring or color-tuple, new background color

If no arguments are given, return current (canvaswidth, canvasheight). Else resize the canvas the turtles are drawing on. Do not alter the drawing window. To observe hidden parts of the canvas, use the scrollbars. With this method, one can make visible those parts of a drawing which were outside the canvas before.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.screensize() (400, 300) \>\>\> screen.screensize(2000,1500) \>\>\> screen.screensize() (2000, 1500)

</div>

e.g. to search for an erroneously escaped turtle ;-)

</div>

<div class="function">

setworldcoordinates(llx, lly, urx, ury)

param llx  
a number

param lly  
a number

param urx  
a number

param ury  
a number

Set up a user-defined coordinate system. *llx* and *lly* are the x and y coordinates of the lower left corner of the canvas. *urx* and *ury* are the x and y coordinates of the upper right corner. Switch to mode "world" if necessary, which performs a reset. If mode "world" is already active, all drawings are redrawn according to the new coordinates. In user-defined coordinate systems, angles may appear distorted, see `mode`.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.reset() \>\>\> screen.setworldcoordinates(-50,-7.5,50,7.5) \>\>\> for \_ in range(72): ... left(10) ... \>\>\> for \_ in range(8): ... left(45); fd(2) \# a regular octagon

</div>

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> screen.reset() \>\>\> for t in turtles(): ... t.reset()

</div>

</div>

#### Animation control

<div class="function">

no_animation()

Temporarily turn off turtle animation. The code inside the `with` block is not animated. Once the block is exited, the drawing appears. This is useful for drawing complex shapes where even the fastest speed setting is too slow.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> with screen.no_animation(): ... for dist in range(2, 400, 2): ... fd(dist) ... rt(90)

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

delay(delay=None)

param delay  
a nonnegative integer or `None`

Return or set the drawing delay in milliseconds. This is approximately the time interval between two consecutive canvas updates. The longer the drawing delay, the slower the animation.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.delay() 10 \>\>\> screen.delay(5) \>\>\> screen.delay() 5

</div>

</div>

<div class="function">

tracer(n=None, delay=None)

param n  
a nonnegative integer or `None`

param delay  
a nonnegative integer or `None`

Turn turtle animation on or off and set the drawing delay. If *n* is given, only every n-th regular screen update is performed, which can be used to speed up the drawing of complex graphics. If *n* is 0, animation is turned off and the screen is only updated by calling `update`. If *delay* is given, it sets the drawing delay, see `delay`. When called without arguments, return the current value of *n*.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.tracer(8, 25) \>\>\> dist = 2 \>\>\> for i in range(200): ... fd(dist) ... rt(90) ... dist += 2

</div>

</div>

<div class="function">

update()

Perform a TurtleScreen update. To be used when tracer is turned off.

</div>

See also the RawTurtle/Turtle method `speed`.

#### Using screen events

<div class="function">

listen(xdummy=None, ydummy=None)

Set focus on TurtleScreen (in order to collect key-events). Dummy arguments are provided in order to be able to pass `listen` to the onclick method.

</div>

<div class="function">

onkey(fun, key) onkeyrelease(fun, key)

param fun  
a function with no arguments or `None`

param key  
a string: key (e.g. "a") or key-symbol (e.g. "space")

Bind *fun* to key-release event of key. If *fun* is `None`, event bindings are removed. Remark: in order to be able to register key-events, TurtleScreen must have the focus. (See method `listen`.)

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> def f(): ... fd(50) ... lt(60) ... \>\>\> screen.onkey(f, "Up") \>\>\> screen.listen()

</div>

</div>

<div class="function">

onkeypress(fun, key=None)

param fun  
a function with no arguments or `None`

param key  
a string: key (e.g. "a") or key-symbol (e.g. "space")

Bind *fun* to key-press event of key if key is given, or to any key-press-event if no key is given. Remark: in order to be able to register key-events, TurtleScreen must have focus. (See method `listen`.)

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> def f(): ... fd(50) ... \>\>\> screen.onkey(f, "Up") \>\>\> screen.listen()

</div>

</div>

<div class="function">

onclick(fun, btn=1, add=None) onscreenclick(fun, btn=1, add=None)

param fun  
a function with two arguments or `None`

param btn  
an integer

param add  
a boolean or `None`

Bind *fun* to mouse-click events on this screen. *fun* is called with the coordinates of the clicked point on the canvas. If *fun* is `None`, existing bindings are removed. *btn* is the number of the mouse button and defaults to 1, the left mouse button. If *add* is `True`, a new binding is added, otherwise it replaces a former binding.

Example for a TurtleScreen instance named `screen` and a Turtle instance named `turtle`:

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.onclick(turtle.goto) \# Subsequently clicking into the TurtleScreen will \>\>\> \# make the turtle move to the clicked point. \>\>\> screen.onclick(None) \# remove event binding again

</div>

> [!NOTE]
> This TurtleScreen method is available as a global function only under the name `onscreenclick`. The global function `onclick` is another one derived from the Turtle method `onclick`.

</div>

<div class="function">

ontimer(fun, t=0)

param fun  
a function with no arguments

param t  
a number \>= 0

Install a timer that calls *fun* after *t* milliseconds.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> running = True \>\>\> def f(): ... if running: ... fd(50) ... lt(60) ... screen.ontimer(f, 250) \>\>\> f() \### makes the turtle march around \>\>\> running = False

</div>

</div>

<div class="function">

mainloop() done()

Start the event loop by calling Tkinter's `~tkinter.Misc.mainloop` method. Must be the last statement in a turtle graphics program. Must *not* be used if a script is run from within IDLE in -n mode (see `idle-no-subprocess`).

</div>

#### Input methods

<div class="function">

textinput(title, prompt)

param title  
a string

param prompt  
a string

Pop up a dialog window for input of a string. *title* is the title of the dialog window and *prompt* is a text describing what information to input. Return the string entered. If the dialog is canceled, return `None`. :

    >>> screen.textinput("NIM", "Name of first player:")

</div>

<div class="function">

numinput(title, prompt, default=None, minval=None, maxval=None)

param title  
a string

param prompt  
a string

param default  
a number or `None`

param minval  
a number or `None`

param maxval  
a number or `None`

Pop up a dialog window for input of a number. *title* is the title of the dialog window and *prompt* is a text describing what numerical information to input. *default* is the default value, *minval* and *maxval* are the minimum and maximum values for input. If these are given, the number entered must be between *minval* and *maxval*. If not, a hint is issued and the dialog remains open for correction. Return the number entered. If the dialog is canceled, return `None`. :

    >>> screen.numinput("Poker", "Your stakes:", 1000, minval=10, maxval=10000)

</div>

#### Settings and special methods

<div class="function">

mode(mode=None)

param mode  
a string or `None`

Set the turtle mode and perform a reset. *mode* is one of the strings "standard", "logo" or "world". If *mode* is not given, return the current mode.

Mode "standard" is compatible with the old `!turtle` module. Mode "logo" is compatible with most Logo turtle graphics. Mode "world" uses user-defined world coordinates, see `setworldcoordinates`. In this mode, angles appear distorted if the x and y units are not of equal length.

| Mode       | Initial turtle heading | positive angles  |
|------------|------------------------|------------------|
| "standard" | to the right (east)    | counterclockwise |
| "logo"     | upward (north)         | clockwise        |

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> mode("logo") \# resets turtle heading to north \>\>\> mode() 'logo'

</div>

</div>

<div class="function">

colormode(cmode=None)

param cmode  
a number or `None`

Return the colormode or set it to 1.0 or 255. After setting it, the *r*, *g* and *b* values of color triples have to be between 0 and *cmode*.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.colormode(1) \>\>\> turtle.pencolor(240, 160, 80) Traceback (most recent call last): ... TurtleGraphicsError: bad color sequence: (240, 160, 80) \>\>\> screen.colormode() 1.0 \>\>\> screen.colormode(255) \>\>\> screen.colormode() 255 \>\>\> turtle.pencolor(240,160,80)

</div>

</div>

<div class="function">

getcanvas()

Return the Canvas of this TurtleScreen. Useful for insiders who know what to do with a Tkinter Canvas.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> cv = screen.getcanvas() \>\>\> cv \<turtle.ScrolledCanvas object ...\>

</div>

</div>

<div class="function">

getshapes()

Return a list of names of all currently available turtle shapes.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.getshapes() \['arrow', 'blank', 'circle', ..., 'turtle'\]

</div>

</div>

<div class="function">

register_shape(name, shape=None) addshape(name, shape=None)

param name  
a string

param shape  
a string, a tuple of pairs of numbers, a `Shape` object or `None`

Add a turtle shape to the TurtleScreen's shape list. It can be called in four different ways:

1)  *name* is the name of an image file (PNG, GIF, PGM, and PPM) and *shape* is `None`: Install the corresponding image shape. :

        >>> screen.register_shape("turtle.gif")

2)  *name* is an arbitrary string and *shape* is the name of an image file (PNG, GIF, PGM, and PPM): Install the corresponding image shape. :

        >>> screen.register_shape("turtle", "turtle.gif")

3)  *name* is an arbitrary string and *shape* is a tuple of pairs of coordinates: Install the corresponding polygon shape.

    <div class="doctest" data-skipif="_tkinter is None">

    \>\>\> screen.register_shape("triangle", ((5,-3), (0,5), (-5,-3)))

    </div>

4)  *name* is an arbitrary string and *shape* is a (compound) `Shape` object: Install the corresponding compound shape.

Image shapes do not rotate when turning the turtle, so they do not display the heading of the turtle. To use a registered shape, call `shape` with its name.

<div class="versionchanged">

3.14 Added support for PNG, PGM, and PPM image formats. Both a shape name and an image file name can be specified.

</div>

</div>

<div class="function">

turtles()

Return the list of turtles on the screen.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> for turtle in screen.turtles(): ... turtle.color("red")

</div>

</div>

<div class="function">

window_height()

Return the height of the turtle window. :

    >>> screen.window_height()
    480

</div>

<div class="function">

window_width()

Return the width of the turtle window. :

    >>> screen.window_width()
    640

</div>

#### Screen-only methods<span id="screenspecific"></span>

<div class="function">

bye()

Close the turtle graphics window.

</div>

<div class="function">

exitonclick()

Bind `bye()` method to mouse clicks on the Screen.

If the value "using_IDLE" in the configuration dictionary is `False` (default value), also enter mainloop. Remark: If IDLE with the `-n` switch (no subprocess) is used, this value should be set to `True` in `turtle.cfg`. In this case IDLE's own mainloop is active also for the client script.

</div>

<div class="function">

save(filename, overwrite=False)

Save the current turtle drawing (and turtles) as a PostScript file.

param filename  
the path of the saved PostScript file

param overwrite  
if `False` and there already exists a file with the given filename, then the function will raise a `FileExistsError`. If it is `True`, the file will be overwritten.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.save("my_drawing.ps") \>\>\> screen.save("my_drawing.ps", overwrite=True)

</div>

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

setup(width=\_CFG\["width"\], height=\_CFG\["height"\], startx=\_CFG\["leftright"\], starty=\_CFG\["topbottom"\])

Set the size and position of the main window. Default values of arguments are stored in the configuration dictionary and can be changed via a `turtle.cfg` file.

param width  
if an integer, a size in pixels, if a float, a fraction of the screen; default is 50% of screen

param height  
if an integer, the height in pixels, if a float, a fraction of the screen; default is 75% of screen

param startx  
if positive, starting position in pixels from the left edge of the screen, if negative from the right edge, if `None`, center window horizontally

param starty  
if positive, starting position in pixels from the top edge of the screen, if negative from the bottom edge, if `None`, center window vertically

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.setup (width=200, height=200, startx=0, starty=0) \>\>\> \# sets window to 200x200 pixels, in upper left of screen \>\>\> screen.setup(width=.75, height=0.5, startx=None, starty=None) \>\>\> \# sets window to 75% of screen by 50% of screen and centers

</div>

</div>

<div class="function">

title(titlestring)

param titlestring  
a string

Set the title of the turtle window to *titlestring*. The title is shown in the title bar of the turtle graphics window.

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> screen.title("Welcome to the turtle zoo!")

</div>

</div>

### Public classes

<div class="RawTurtle(canvas) RawPen(canvas)">

param canvas  
a `!tkinter.Canvas`, a `ScrolledCanvas` or a `TurtleScreen`

Create a turtle. The turtle has all methods described above as "methods of Turtle/RawTurtle".

</div>

<div class="Turtle()">

Subclass of RawTurtle, has the same interface but draws on a default `Screen` object created automatically when needed for the first time.

</div>

<div class="TurtleScreen(cv)">

param cv  
a `!tkinter.Canvas`

Provides screen oriented methods like `bgcolor` etc. that are described above.

</div>

<div class="Screen()">

Subclass of `TurtleScreen`, with `four methods added
<screenspecific>`. Calling `Screen()` returns the single screen object. If it does not exist yet, it is created. Otherwise the existing one is returned.

</div>

<div class="ScrolledCanvas(master)">

param master  
some Tkinter widget to contain the ScrolledCanvas, i.e. a Tkinter-canvas with scrollbars added

Used by class Screen, which thus automatically provides a ScrolledCanvas as playground for the turtles.

</div>

<div class="Shape(type_, data)">

param type\_  
a string

param data  
depends on *type\_*

Data structure modeling shapes. *type\_* is one of the strings "polygon", "image" or "compound". What *data* must be depends on *type\_*: for "polygon" a tuple of pairs of coordinates, for "image" an image (only used internally), and for "compound" `None`, as a compound shape is built up using the `addcomponent` method.

<div class="method">

addcomponent(poly, fill, outline=None)

param poly  
a tuple of pairs of numbers

param fill  
a color

param outline  
a color or `None`

Add a component to a shape of type "compound". *poly* is a polygon, a tuple of pairs of numbers. *fill* is the color the polygon is filled with and *outline* is the color of its outline. If *outline* is not given, *fill* is used for it as well.

Example:

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> poly = ((0,0), (10,-5), (0,10), (-10,-5)) \>\>\> s = Shape("compound") \>\>\> s.addcomponent(poly, "red", "blue") \>\>\> \# ... add more components and then use register_shape()

</div>

See `compoundshapes`.

</div>

</div>

<div class="Vec2D(x, y)">

A two-dimensional vector class, used as a helper class for implementing turtle graphics. May be useful for turtle graphics programs too. Derived from tuple, so a vector is a tuple!

Provides (for *a*, *b* vectors, *k* number):

- `a + b` vector addition
- `a - b` vector subtraction
- `a * b` inner product
- `k * a` and `a * k` multiplication with scalar
- `abs(a)` absolute value of a
- `a.rotate(angle)` rotation

</div>

### Exceptions

The `!turtle` module defines the following exception:

<div class="exception">

TurtleGraphicsError

Raised for invalid arguments or operations. For example, a malformed color string:

<div class="doctest" data-skipif="_tkinter is None">

\>\>\> turtle.color("blau") Traceback (most recent call last): ... turtle.TurtleGraphicsError: bad color string: blau

</div>

</div>

## How-to guides<span id="turtle-how-to"></span><span id="turtle-howtos"></span>

This section covers some typical turtle use-cases and approaches.

### How to automatically begin and end filling

Starting with Python 3.14, you can use the `fill` `context manager` instead of `begin_fill` and `end_fill` to automatically begin and end fill. Here is an example:

    with fill():
        for i in range(4):
            forward(100)
            right(90)

    forward(200)

The code above is equivalent to:

    begin_fill()
    for i in range(4):
        forward(100)
        right(90)
    end_fill()

    forward(200)

### How to use compound shapes<span id="compoundshapes"></span>

To use compound turtle shapes, which consist of several polygons of different color, you must use the helper class `Shape` explicitly as described below:

1.  Create an empty `Shape` object of type "compound".

2.  Add as many components to this object as desired, using the `~Shape.addcomponent` method.

    For example:

    <div class="doctest" data-skipif="_tkinter is None">

    \>\>\> s = Shape("compound") \>\>\> poly1 = ((0,0), (10,-5), (0,10), (-10,-5)) \>\>\> s.addcomponent(poly1, "red", "blue") \>\>\> poly2 = ((0,0), (10,-5), (-10,-5)) \>\>\> s.addcomponent(poly2, "blue", "red")

    </div>

3.  Now add the Shape to the Screen's shapelist and use it:

    <div class="doctest" data-skipif="_tkinter is None">

    \>\>\> register_shape("myshape", s) \>\>\> shape("myshape")

    </div>

### How to use the `turtle` module namespace

Using `from turtle import *` is convenient - but be warned that it imports a rather large collection of objects, and if you're doing anything but turtle graphics you run the risk of a name conflict (this becomes even more an issue if you're using turtle graphics in a script where other modules might be imported).

The solution is to use `import turtle` - `fd()` becomes `turtle.fd()`, `width()` becomes `turtle.width()` and so on. (If typing "turtle" over and over again becomes tedious, use for example `import turtle as t` instead.)

### How to use turtle graphics in a script

It's recommended to use the `turtle` module namespace as described immediately above, for example:

    import turtle as t
    from random import random

    for i in range(100):
        steps = int(random() * 100)
        angle = int(random() * 360)
        t.right(angle)
        t.fd(steps)

Another step is also required though - as soon as the script ends, Python will also close the turtle's window. Add:

    t.mainloop()

to the end of the script. The script will now wait to be dismissed and will not exit until it is terminated, for example by closing the turtle graphics window.

### How to use object-oriented turtle graphics

<div class="seealso">

`Explanation of the object-oriented interface <turtle-explanation>`

</div>

Other than for very basic introductory purposes, or for trying things out as quickly as possible, it's more usual and much more powerful to use the object-oriented approach to turtle graphics. For example, this allows multiple turtles on screen at once.

In this approach, the various turtle commands are methods of objects (mostly of `Turtle` objects). You *can* use the object-oriented approach in the shell, but it would be more typical in a Python script.

The example above then becomes:

    from turtle import Turtle
    from random import random

    t = Turtle()
    for i in range(100):
        steps = int(random() * 100)
        angle = int(random() * 360)
        t.right(angle)
        t.fd(steps)

    t.screen.mainloop()

Note the last line. `t.screen` is an instance of the `Screen` that a Turtle instance exists on; it's created automatically along with the turtle.

The turtle's screen can be customised, for example:

    t.screen.title('Object-oriented turtle demo')
    t.screen.bgcolor("orange")

### How to use `!help`

The public methods of the Screen and Turtle classes are documented extensively via docstrings. So these can be used as online-help via the Python help facilities:

- When using IDLE, tooltips show the signatures and first lines of the docstrings of typed in function-/method calls.

- Calling `help` on methods or functions displays the docstrings:

      >>> help(Screen.bgcolor)
      Help on method bgcolor in module turtle:

      bgcolor(self, *args) unbound turtle.Screen method
          Set or return backgroundcolor of the TurtleScreen.

          Arguments (if given): a color string or three numbers
          in the range 0..colormode or a 3-tuple of such numbers.


          >>> screen.bgcolor("orange")
          >>> screen.bgcolor()
          "orange"
          >>> screen.bgcolor(0.5,0,0.5)
          >>> screen.bgcolor()
          "#800080"

      >>> help(Turtle.penup)
      Help on method penup in module turtle:

      penup(self) unbound turtle.Turtle method
          Pull the pen up -- no drawing when moving.

          Aliases: penup | pu | up

          No argument

          >>> turtle.penup()

- The docstrings of the functions which are derived from methods have a modified form:

      >>> help(bgcolor)
      Help on function bgcolor in module turtle:

      bgcolor(*args)
          Set or return backgroundcolor of the TurtleScreen.

          Arguments (if given): a color string or three numbers
          in the range 0..colormode or a 3-tuple of such numbers.

          Example::

            >>> bgcolor("orange")
            >>> bgcolor()
            "orange"
            >>> bgcolor(0.5,0,0.5)
            >>> bgcolor()
            "#800080"

      >>> help(penup)
      Help on function penup in module turtle:

      penup()
          Pull the pen up -- no drawing when moving.

          Aliases: penup | pu | up

          No argument

          Example:
          >>> penup()

These modified docstrings are created automatically together with the function definitions that are derived from the methods at import time.

### Translation of docstrings into different languages

The docstrings of the public methods of the Screen and Turtle classes, and of the corresponding functions, can be replaced with translations, so that `help` and IDE tooltips are shown in another language. However, only the help text is translated, the names of the functions and methods stay the same.

Translations are distributed on PyPI in the `turtle-translations` package. To use them, install the package with `pip` and select the language with the `PYTHON_TURTLE_LANG` environment variable. For example, to show the help text in Spanish:

```
$ python -m pip install turtle-translations
$ PYTHON_TURTLE_LANG=es python
>>> import turtle
>>> help(turtle.forward)
```

The language can also be set permanently with the *language* entry of the `turtle.cfg` file (see `turtle-configuration`). If no translation is found for the selected language, the English docstrings are kept.

To add a new language or improve an existing translation, see the contribution instructions in the `turtle-translations` project.

A translation is a docstring dictionary. It is a top-level module named `turtle_docstringdict_{language}.py` on `sys.path` defining a dictionary named `docsdict`, the keys of which are method names such as `Turtle.forward` and the values of which are the translated docstrings. It is read in at import time. Entries naming a method which does not exist in the running version are ignored.

<div class="versionchanged">

3.16 Entries naming an unknown method are ignored instead of reported.

</div>

<div class="envvar">

PYTHON_TURTLE_LANG

The name of the language to read the translation for. It takes precedence over the *language* entry of the `turtle.cfg` file.

<div class="versionadded">

3.16

</div>

</div>

<div class="function">

write_docstringdict(filename="turtle_docstringdict")

param filename  
a string, used as filename

Create and write docstring-dictionary to a Python script with the given filename. This function has to be called explicitly (it is not used by the turtle graphics classes). The docstring dictionary will be written to the Python script `{filename}.py`. It is intended to serve as a template for translation of the docstrings into different languages.

</div>

### How to configure Screen and Turtles

The built-in default configuration mimics the appearance and behaviour of the old turtle module in order to retain best possible compatibility with it.

If you want to use a different configuration which better reflects the features of this module or which better fits to your needs, e.g. for use in a classroom, you can prepare a configuration file `turtle.cfg` which will be read at import time and modify the configuration according to its settings.

The built in configuration would correspond to the following `turtle.cfg`:

``` ini
width = 0.5
height = 0.75
leftright = None
topbottom = None
canvwidth = 400
canvheight = 300
mode = standard
colormode = 1.0
delay = 10
undobuffersize = 1000
shape = classic
pencolor = black
fillcolor = black
resizemode = noresize
visible = True
language = english
exampleturtle = turtle
examplescreen = screen
title = Python Turtle Graphics
using_IDLE = False
```

Short explanation of selected entries:

- The first four lines correspond to the arguments of the `Screen.setup <setup>` method.
- Line 5 and 6 correspond to the arguments of the method `Screen.screensize <screensize>`.
- *shape* can be any of the built-in shapes, e.g: arrow, turtle, etc. For more info try `help(shape)`.
- If you want to use no fill color (i.e. make the turtle transparent), you have to write `fillcolor = ""` (but all nonempty strings must not have quotes in the cfg file).
- If you want to reflect the turtle its state, you have to use `resizemode = auto`.
- The *language* entry selects the language of the docstrings, unless the `PYTHON_TURTLE_LANG` environment variable is set. See `turtle-docstring-translation` for more information.
- The entries *exampleturtle* and *examplescreen* define the names of these objects as they occur in the docstrings. The transformation of method-docstrings to function-docstrings will delete these names from the docstrings.
- *using_IDLE*: Set this to `True` if you regularly work with IDLE and its `-n` switch ("no subprocess"). This will prevent `exitonclick` to enter the mainloop.

There can be a `turtle.cfg` file in the directory where `!turtle` is stored and an additional one in the current working directory. The latter will override the settings of the first one.

The `Lib/turtledemo` directory contains a `turtle.cfg` file. You can study it as an example and see its effects when running the demos (preferably not from within the demo-viewer).

## Explanation

A turtle object draws on a screen object, and there a number of key classes in the turtle object-oriented interface that can be used to create them and relate them to each other.

A `Turtle` instance will automatically create a `Screen` instance if one is not already present.

`Turtle` is a subclass of `RawTurtle`, which *doesn't* automatically create a drawing surface - a *canvas* will need to be provided or created for it. The *canvas* can be a `!tkinter.Canvas`, `ScrolledCanvas` or `TurtleScreen`.

`TurtleScreen` is the basic drawing surface for a turtle. `Screen` is a subclass of `TurtleScreen`, and includes `some additional methods <screenspecific>` for managing its appearance (including size and title) and behaviour. `TurtleScreen`'s constructor needs a `!tkinter.Canvas` or a `ScrolledCanvas` as an argument.

The functional interface for turtle graphics uses the various methods of `Turtle` and `TurtleScreen`/`Screen`. Behind the scenes, a screen object is automatically created whenever a function derived from a `Screen` method is called. Similarly, a turtle object is automatically created whenever any of the functions derived from a Turtle method is called.

To use multiple turtles on a screen, the object-oriented interface must be used.

## `!turtledemo` --- Demo scripts

<div class="module" data-synopsis="A viewer for example turtle scripts">

turtledemo

</div>

The `!turtledemo` package includes a set of demo scripts. These scripts can be run and viewed using the supplied demo viewer as follows:

    python -m turtledemo

Alternatively, you can run the demo scripts individually. For example, :

    python -m turtledemo.bytedesign

The `!turtledemo` package directory contains:

- A demo viewer `__main__.py` which can be used to view the sourcecode of the scripts and run them at the same time.
- Multiple scripts demonstrating different features of the `!turtle` module. Examples can be accessed via the Examples menu. They can also be run standalone.
- A `turtle.cfg` file which serves as an example of how to write and use such files.

The demo scripts are:

<div class="currentmodule">

turtle

</div>

<div class="tabularcolumns">

\|l\|L\|L\|

</div>

| Name | Description | Features |
|----|----|----|
| `bytedesign` | complex classical turtle graphics pattern | `tracer`, `delay`, `update` |
| `chaos` | graphs Verhulst dynamics, shows that computer's computations can generate results sometimes against the common sense expectations | world coordinates |
| `clock` | analog clock showing time of your computer | turtles as clock's hands, `ontimer` |
| `colormixer` | experiment with r, g, b | `ondrag` |
| `forest` | 3 breadth-first trees | randomization |
| `fractalcurves` | Hilbert & Koch curves | recursion |
| `lindenmayer` | ethnomathematics (indian kolams) | L-System |
| `minimal_hanoi` | Towers of Hanoi | Rectangular Turtles as Hanoi discs (`shape`, `shapesize`) |
| `nim` | play the classical nim game with three heaps of sticks against the computer. | turtles as nimsticks, event driven (mouse, keyboard) |
| `paint` | super minimalistic drawing program | `onclick` |
| `peace` | elementary | turtle: appearance and animation |
| `penrose` | aperiodic tiling with kites and darts | `stamp` |
| `planet_and_moon` | simulation of gravitational system | compound shapes, `Vec2D` |
| `rosette` | a pattern from the wikipedia article on turtle graphics | `clone`, `undo` |
| `round_dance` | dancing turtles rotating pairwise in opposite direction | compound shapes, `clone` `shapesize`, `tilt`, `get_shapepoly`, `update` |
| `sorting_animate` | visual demonstration of different sorting methods | simple alignment, randomization |
| `tree` | a (graphical) breadth first tree (using generators) | `clone` |
| `two_canvases` | simple design | turtles on two canvases |
| `yinyang` | another elementary example | `circle` |

Have fun!

<div class="doctest" data-skipif="_tkinter is None" data-hide="">

\>\>\> for turtle in turtles(): ... turtle.reset() \>\>\> turtle.penup() \>\>\> turtle.goto(-200,25) \>\>\> turtle.pendown() \>\>\> turtle.write("No one expects the Spanish Inquisition!", ... font=("Arial", 20, "normal")) \>\>\> turtle.penup() \>\>\> turtle.goto(-100,-50) \>\>\> turtle.pendown() \>\>\> turtle.write("Our two chief Turtles are...", ... font=("Arial", 16, "normal")) \>\>\> turtle.penup() \>\>\> turtle.goto(-450,-75) \>\>\> turtle.write(str(turtles()))

</div>

# `!types` --- Dynamic type creation and names for built-in types

<div class="module" data-synopsis="Names for built-in types.">

types

</div>

**Source code:** `Lib/types.py`

------------------------------------------------------------------------

This module defines utility functions to assist in dynamic creation of new types.

It also defines names for some object types that are used by the standard Python interpreter, but not exposed as builtins like `int` or `str` are.

Finally, it provides some additional type-related utility classes and functions that are not fundamental enough to be builtins.

## Dynamic Type Creation

<div class="function">

new_class(name, bases=(), kwds=None, exec_body=None)

Creates a class object dynamically using the appropriate metaclass.

The first three arguments are the components that make up a class definition header: the class name, the base classes (in order), the keyword arguments (such as `metaclass`).

The *exec_body* argument is a callback that is used to populate the freshly created class namespace. It should accept the class namespace as its sole argument and update the namespace directly with the class contents. If no callback is provided, it has the same effect as passing in `lambda ns: None`.

<div class="versionadded">

3.3

</div>

</div>

<div class="function">

prepare_class(name, bases=(), kwds=None)

Calculates the appropriate metaclass and creates the class namespace.

The arguments are the components that make up a class definition header: the class name, the base classes (in order) and the keyword arguments (such as `metaclass`).

The return value is a 3-tuple: `metaclass, namespace, kwds`

*metaclass* is the appropriate metaclass, *namespace* is the prepared class namespace and *kwds* is an updated copy of the passed in *kwds* argument with any `'metaclass'` entry removed. If no *kwds* argument is passed in, this will be an empty dict.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.6

The default value for the `namespace` element of the returned tuple has changed. Now an insertion-order-preserving mapping is used when the metaclass does not have a `__prepare__` method.

</div>

</div>

<div class="seealso">

`metaclasses`  
Full details of the class creation process supported by these functions

`3115` - Metaclasses in Python 3000  
Introduced the `__prepare__` namespace hook

</div>

<div class="function">

resolve_bases(bases)

Resolve MRO entries dynamically as specified by `560`.

This function looks for items in *bases* that are not instances of `type`, and returns a tuple where each such object that has an `~object.__mro_entries__` method is replaced with an unpacked result of calling this method. If a *bases* item is an instance of `type`, or it doesn't have an `!__mro_entries__` method, then it is included in the return tuple unchanged.

<div class="versionadded">

3.7

</div>

</div>

<div class="function">

get_original_bases(cls, /)

Return the tuple of objects originally given as the bases of *cls* before the `~object.__mro_entries__` method has been called on any bases (following the mechanisms laid out in `560`). This is useful for introspecting `Generics <user-defined-generics>`.

For classes that have an `__orig_bases__` attribute, this function returns the value of `cls.__orig_bases__`. For classes without the `__orig_bases__` attribute, `cls.__bases__ <type.__bases__>` is returned.

Examples:

    from typing import TypeVar, Generic, NamedTuple, TypedDict

    T = TypeVar("T")
    class Foo(Generic[T]): ...
    class Bar(Foo[int], float): ...
    class Baz(list[str]): ...
    Eggs = NamedTuple("Eggs", [("a", int), ("b", str)])
    Spam = TypedDict("Spam", {"a": int, "b": str})

    assert Bar.__bases__ == (Foo, float)
    assert get_original_bases(Bar) == (Foo[int], float)

    assert Baz.__bases__ == (list,)
    assert get_original_bases(Baz) == (list[str],)

    assert Eggs.__bases__ == (tuple,)
    assert get_original_bases(Eggs) == (NamedTuple,)

    assert Spam.__bases__ == (dict,)
    assert get_original_bases(Spam) == (TypedDict,)

    assert int.__bases__ == (object,)
    assert get_original_bases(int) == (object,)

<div class="versionadded">

3.12

</div>

</div>

<div class="seealso">

`560` - Core support for typing module and generic types

</div>

## Standard Interpreter Types

This module provides names for many of the types that are required to implement a Python interpreter. It deliberately avoids including some of the types that arise only incidentally during processing such as the `listiterator` type.

Typical use of these names is for `isinstance` or `issubclass` checks.

If you instantiate any of these types, note that signatures may vary between Python versions.

Standard names are defined for the following types:

<div class="NoneType">

The type of `None`.

<div class="versionadded">

3.10

</div>

</div>

<div class="FunctionType LambdaType">

The type of user-defined functions and functions created by `lambda` expressions.

<div class="audit-event">

function.\_\_new\_\_ code types.FunctionType

</div>

The audit event only occurs for direct instantiation of function objects, and is not raised for normal compilation.

</div>

<div class="GeneratorType">

The type of `generator`-iterator objects, created by generator functions.

</div>

<div class="CoroutineType">

The type of `coroutine` objects, created by `async def` functions.

<div class="versionadded">

3.5

</div>

</div>

<div class="AsyncGeneratorType">

The type of `asynchronous generator`-iterator objects, created by asynchronous generator functions.

<div class="versionadded">

3.6

</div>

</div>

<div class="CodeType(**kwargs)">

<div class="index">

pair: built-in function; compile

</div>

The type of `code objects <code-objects>` such as returned by `compile`.

<div class="audit-event">

code.\_\_new\_\_ code,filename,name,argcount,posonlyargcount,kwonlyargcount,nlocals,stacksize,flags types.CodeType

</div>

Note that the audited arguments may not match the names or positions required by the initializer. The audit event only occurs for direct instantiation of code objects, and is not raised for normal compilation.

</div>

<div class="CellType">

The type for cell objects: such objects are used as containers for a function's `closure variables <closure variable>`.

<div class="versionadded">

3.8

</div>

</div>

<div class="MethodType">

The type of methods of user-defined class instances.

</div>

<div class="BuiltinFunctionType BuiltinMethodType">

The type of built-in functions like `len` or `sys.exit`, and methods of built-in classes. (Here, the term "built-in" means "written in C".)

</div>

<div class="WrapperDescriptorType">

The type of methods of some built-in data types and base classes such as `object.__init__` or `object.__lt__`.

<div class="versionadded">

3.7

</div>

</div>

<div class="MethodWrapperType">

The type of *bound* methods of some built-in data types and base classes. For example it is the type of `object().__str__`.

<div class="versionadded">

3.7

</div>

</div>

<div class="NotImplementedType">

The type of `NotImplemented`.

<div class="versionadded">

3.10

</div>

</div>

<div class="MethodDescriptorType">

The type of methods of some built-in data types such as `str.join`.

<div class="versionadded">

3.7

</div>

</div>

<div class="ClassMethodDescriptorType">

The type of *unbound* class methods of some built-in data types such as `dict.__dict__['fromkeys']`.

<div class="versionadded">

3.7

</div>

</div>

<div class="ModuleType(name, doc=None)">

The type of `modules <module>`. The constructor takes the name of the module to be created and optionally its `docstring`.

<div class="seealso">

`Documentation on module objects <module-objects>`  
Provides details on the special attributes that can be found on instances of `!ModuleType`.

`importlib.util.module_from_spec`  
Modules created using the `!ModuleType` constructor are created with many of their special attributes unset or set to default values. `!module_from_spec` provides a more robust way of creating `!ModuleType` instances which ensures the various attributes are set appropriately.

</div>

</div>

<div class="EllipsisType">

The type of `Ellipsis`.

<div class="versionadded">

3.10

</div>

</div>

<div class="GenericAlias(t_origin, t_args)">

The type of `parameterized generics <types-genericalias>` such as `list[int]`.

`t_origin` should be a non-parameterized generic class, such as `list`, `tuple` or `dict`. `t_args` should be a `tuple` (possibly of length 1) of types which parameterize `t_origin`:

    >>> from types import GenericAlias

    >>> list[int] == GenericAlias(list, (int,))
    True
    >>> dict[str, int] == GenericAlias(dict, (str, int))
    True

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.9.2 This type can now be subclassed.

</div>

<div class="seealso">

`Generic Alias Types<types-genericalias>`  
In-depth documentation on instances of `!types.GenericAlias`

`585` - Type Hinting Generics In Standard Collections  
Introducing the `!types.GenericAlias` class

</div>

</div>

<div class="UnionType">

The type of `union type expressions<types-union>`.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.14

This is now an alias for `typing.Union`.

</div>

</div>

<div class="TracebackType(tb_next, tb_frame, tb_lasti, tb_lineno)">

The type of traceback objects such as found in `sys.exception().__traceback__`.

See `the language reference <traceback-objects>` for details of the available attributes and operations, and guidance on creating tracebacks dynamically.

</div>

<div class="FrameType">

The type of `frame objects <frame-objects>` such as found in `tb.tb_frame <traceback.tb_frame>` if `tb` is a traceback object.

</div>

<div class="FrameLocalsProxyType">

The type of frame locals proxy objects, as found on the `frame.f_locals` attribute.

<div class="versionadded">

3.15

</div>

<div class="seealso">

`667`

</div>

</div>

<div class="LazyImportType">

The type of lazy import proxy objects. These objects are created when a module is lazily imported and serve as placeholders until the module is actually accessed. This type can be used to detect lazy imports programmatically.

<div class="versionadded">

3.15

</div>

<div class="seealso">

`810`

</div>

<div class="method">

resolve()

Reify the lazy import and return the "real" object being imported.

</div>

</div>

<div class="GetSetDescriptorType">

The type of objects defined in extension modules with `PyGetSetDef`, such as `FrameType.f_locals <frame.f_locals>` or `array.array.typecode`. This type is used as descriptor for object attributes; it has the same purpose as the `property` type, but for classes defined in extension modules.

</div>

<div class="MemberDescriptorType">

The type of objects defined in extension modules with `PyMemberDef`, such as `datetime.timedelta.days`. This type is used as descriptor for simple C data members which use standard conversion functions; it has the same purpose as the `property` type, but for classes defined in extension modules.

In addition, when a class is defined with a `~object.__slots__` attribute, then for each slot, an instance of `!MemberDescriptorType` will be added as an attribute on the class. This allows the slot to appear in the class's `~type.__dict__`.

<div class="impl-detail">

In other implementations of Python, this type may be identical to `GetSetDescriptorType`.

</div>

</div>

<div class="MappingProxyType(mapping)">

Read-only proxy of a mapping. It provides a dynamic view on the mapping's entries, which means that when the mapping changes, the view reflects these changes.

`!MappingProxyType`s are `generic <generics>` over two types, signifying (respectively) the types of the underlying mapping's keys and values.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.9

Updated to support the new union (`|`) operator from `584`, which simply delegates to the underlying mapping.

</div>

<div class="describe">

key in proxy

Return `True` if the underlying mapping has a key *key*, else `False`.

</div>

<div class="describe">

proxy\[key\]

Return the item of the underlying mapping with key *key*. Raises a `KeyError` if *key* is not in the underlying mapping.

</div>

<div class="describe">

iter(proxy)

Return an iterator over the keys of the underlying mapping. This is a shortcut for `iter(proxy.keys())`.

</div>

<div class="describe">

len(proxy)

Return the number of items in the underlying mapping.

</div>

<div class="method">

copy()

Return a shallow copy of the underlying mapping.

</div>

<div class="method">

get(key\[, default\])

Return the value for *key* if *key* is in the underlying mapping, else *default*. If *default* is not given, it defaults to `None`, so that this method never raises a `KeyError`.

</div>

<div class="method">

items()

Return a new view of the underlying mapping's items (`(key, value)` pairs).

</div>

<div class="method">

keys()

Return a new view of the underlying mapping's keys.

</div>

<div class="method">

values()

Return a new view of the underlying mapping's values.

</div>

<div class="describe">

reversed(proxy)

Return a reverse iterator over the keys of the underlying mapping.

<div class="versionadded">

3.9

</div>

</div>

<div class="describe">

hash(proxy)

Return a hash of the underlying mapping.

<div class="versionadded">

3.12

</div>

</div>

</div>

<div class="CapsuleType">

The type of `capsule objects <capsules>`.

<div class="versionadded">

3.13

</div>

</div>

## Additional Utility Classes and Functions

<div class="SimpleNamespace">

A simple `object` subclass that provides attribute access to its namespace, as well as a meaningful repr.

Unlike `object`, with `!SimpleNamespace` you can add and remove attributes.

`SimpleNamespace` objects may be initialized in the same way as `dict`: either with keyword arguments, with a single positional argument, or with both. When initialized with keyword arguments, those are directly added to the underlying namespace. Alternatively, when initialized with a positional argument, the underlying namespace will be updated with key-value pairs from that argument (either a mapping object or an `iterable` object producing key-value pairs). All such keys must be strings.

The type is roughly equivalent to the following code:

    class SimpleNamespace:
        def __init__(self, mapping_or_iterable=(), /, **kwargs):
            self.__dict__.update(mapping_or_iterable)
            self.__dict__.update(kwargs)

        def __repr__(self):
            items = (f"{k}={v!r}" for k, v in self.__dict__.items())
            return "{}({})".format(type(self).__name__, ", ".join(items))

        def __eq__(self, other):
            if isinstance(self, SimpleNamespace) and isinstance(other, SimpleNamespace):
               return self.__dict__ == other.__dict__
            return NotImplemented

`SimpleNamespace` may be useful as a replacement for `class NS: pass`. However, for a structured record type use `~collections.namedtuple` instead.

`!SimpleNamespace` objects are supported by `copy.replace`.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.9 Attribute order in the repr changed from alphabetical to insertion (like `dict`).

</div>

<div class="versionchanged">

3.13 Added support for an optional positional argument.

</div>

</div>

<div class="function">

DynamicClassAttribute(fget=None, fset=None, fdel=None, doc=None)

Route attribute access on a class to \_\_getattr\_\_.

This is a descriptor, used to define attributes that act differently when accessed through an instance and through a class. Instance access remains normal, but access to an attribute through a class will be routed to the class's \_\_getattr\_\_ method; this is done by raising AttributeError.

This allows one to have properties active on an instance, and have virtual attributes on the class with the same name (see `enum.Enum` for an example).

<div class="versionadded">

3.4

</div>

</div>

## Coroutine Utility Functions

<div class="function">

coroutine(gen_func)

This function transforms a `generator` function into a `coroutine function` which returns a generator-based coroutine. The generator-based coroutine is still a `generator iterator`, but is also considered to be a `coroutine` object and is `awaitable`. However, it may not necessarily implement the `~object.__await__` method.

If *gen_func* is a generator function, it will be modified in-place.

If *gen_func* is not a generator function, it will be wrapped. If it returns an instance of `collections.abc.Generator`, the instance will be wrapped in an *awaitable* proxy object. All other types of objects will be returned as is.

<div class="versionadded">

3.5

</div>

</div>

# `!typing` --- Support for type hints

<div class="testsetup">

- 

import typing from dataclasses import dataclass from typing import \*

</div>

<div class="module" data-synopsis="Support for type hints (see :pep:`484`).">

typing

</div>

<div class="versionadded">

3.5

</div>

**Source code:** `Lib/typing.py`

> [!NOTE]
> The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as `type checkers <static type checker>`, IDEs, linters, etc.

------------------------------------------------------------------------

This module provides runtime support for type hints.

Consider the function below:

    def surface_area_of_cube(edge_length: float) -> str:
        return f"The surface area of the cube is {6 * edge_length ** 2}."

The function `surface_area_of_cube` takes an argument expected to be an instance of `float`, as indicated by the `type hint` `edge_length: float`. The function is expected to return an instance of `str`, as indicated by the `-> str` hint.

While type hints can be simple classes like `float` or `str`, they can also be more complex. The `typing` module provides a vocabulary of more advanced type hints.

New features are frequently added to the `typing` module. The `typing_extensions` package provides backports of these new features to older versions of Python.

<div class="seealso">

[Typing cheat sheet](https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html)  
A quick overview of type hints (hosted at the mypy docs)

Type System Reference section of [the mypy docs](https://mypy.readthedocs.io/en/stable/index.html)  
The Python typing system is standardised via PEPs, so this reference should broadly apply to most Python type checkers. (Some parts may still be specific to mypy.)

[Static Typing with Python](https://typing.python.org/en/latest/)  
Type-checker-agnostic documentation written by the community detailing type system features, useful typing related tools and typing best practices.

</div>

## Specification for the Python Type System

The canonical, up-to-date specification of the Python type system can be found at [Specification for the Python type system](https://typing.python.org/en/latest/spec/index.html).

## Type aliases

A type alias is defined using the `type` statement, which creates an instance of `TypeAliasType`. In this example, `Vector` and `list[float]` will be treated equivalently by static type checkers:

    type Vector = list[float]

    def scale(scalar: float, vector: Vector) -> Vector:
        return [scalar * num for num in vector]

    # passes type checking; a list of floats qualifies as a Vector.
    new_vector = scale(2.0, [1.0, -4.2, 5.4])

Type aliases are useful for simplifying complex type signatures. For example:

    from collections.abc import Sequence

    type ConnectionOptions = dict[str, str]
    type Address = tuple[str, int]
    type Server = tuple[Address, ConnectionOptions]

    def broadcast_message(message: str, servers: Sequence[Server]) -> None:
        ...

    # The static type checker will treat the previous type signature as
    # being exactly equivalent to this one.
    def broadcast_message(
        message: str,
        servers: Sequence[tuple[tuple[str, int], dict[str, str]]]
    ) -> None:
        ...

The `type` statement is new in Python 3.12. For backwards compatibility, type aliases can also be created through simple assignment:

    Vector = list[float]

Or marked with `TypeAlias` to make it explicit that this is a type alias, not a normal variable assignment:

    from typing import TypeAlias

    Vector: TypeAlias = list[float]

## NewType

Use the `NewType` helper to create distinct types:

    from typing import NewType

    UserId = NewType('UserId', int)
    some_id = UserId(524313)

The static type checker will treat the new type as if it were a subclass of the original type. This is useful in helping catch logical errors:

    def get_user_name(user_id: UserId) -> str:
        ...

    # passes type checking
    user_a = get_user_name(UserId(42351))

    # fails type checking; an int is not a UserId
    user_b = get_user_name(-1)

You may still perform all `int` operations on a variable of type `UserId`, but the result will always be of type `int`. This lets you pass in a `UserId` wherever an `int` might be expected, but will prevent you from accidentally creating a `UserId` in an invalid way:

    # 'output' is of type 'int', not 'UserId'
    output = UserId(23413) + UserId(54341)

Note that these checks are enforced only by the static type checker. At runtime, the statement `Derived = NewType('Derived', Base)` will make `Derived` a callable that immediately returns whatever parameter you pass it. That means the expression `Derived(some_value)` does not create a new class or introduce much overhead beyond that of a regular function call.

More precisely, the expression `some_value is Derived(some_value)` is always true at runtime.

It is invalid to create a subtype of `Derived`:

    from typing import NewType

    UserId = NewType('UserId', int)

    # Fails at runtime and does not pass type checking
    class AdminUserId(UserId): pass

However, it is possible to create a `NewType` based on a 'derived' `NewType`:

    from typing import NewType

    UserId = NewType('UserId', int)

    ProUserId = NewType('ProUserId', UserId)

and typechecking for `ProUserId` will work as expected.

See `484` for more details.

> [!NOTE]
> Recall that the use of a type alias declares two types to be *equivalent* to one another. Doing `type Alias = Original` will make the static type checker treat `Alias` as being *exactly equivalent* to `Original` in all cases. This is useful when you want to simplify complex type signatures.
>
> In contrast, `NewType` declares one type to be a *subtype* of another. Doing `Derived = NewType('Derived', Original)` will make the static type checker treat `Derived` as a *subclass* of `Original`, which means a value of type `Original` cannot be used in places where a value of type `Derived` is expected. This is useful when you want to prevent logic errors with minimal runtime cost.

<div class="versionadded">

3.5.2

</div>

<div class="versionchanged">

3.10 `NewType` is now a class rather than a function. As a result, there is some additional runtime cost when calling `NewType` over a regular function.

</div>

<div class="versionchanged">

3.11 The performance of calling `NewType` has been restored to its level in Python 3.9.

</div>

## Annotating callable objects

Functions -- or other `callable` objects -- can be annotated using `collections.abc.Callable` or deprecated `typing.Callable`. `Callable[[int], str]` signifies a function that takes a single parameter of type `int` and returns a `str`.

For example:

<div class="testcode">

from collections.abc import Callable, Awaitable

def feeder(get_next_item: Callable\[\[\], str\]) -\> None:  
... \# Body

def async_query(on_success: Callable\[\[int\], None\],  
on_error: Callable\[\[int, Exception\], None\]) -\> None: ... \# Body

async def on_update(value: str) -\> None:  
... \# Body

callback: Callable\[\[str\], Awaitable\[None\]\] = on_update

</div>

<div class="index">

single: ...; ellipsis literal

</div>

The subscription syntax must always be used with exactly two values: the argument list and the return type. The argument list must be a list of types, a `ParamSpec`, `Concatenate`, or an ellipsis (`...`). The return type must be a single type.

If a literal ellipsis `...` is given as the argument list, it indicates that a callable with any arbitrary parameter list would be acceptable:

<div class="testcode">

def concat(x: str, y: str) -\> str:  
return x + y

x: Callable\[..., str\] x = str \# OK x = concat \# Also OK

</div>

`Callable` cannot express complex signatures such as functions that take a variadic number of arguments, `overloaded functions <overload>`, or functions that have keyword-only parameters. However, these signatures can be expressed by defining a `Protocol` class with a `~object.__call__` method:

<div class="testcode">

from collections.abc import Iterable from typing import Protocol

class Combiner(Protocol):  
def \_\_call\_\_(self, \*vals: bytes, maxlen: int \| None = None) -\> list\[bytes\]: ...

def batch_proc(data: Iterable\[bytes\], cb_results: Combiner) -\> bytes:  
for item in data:  
...

def good_cb(*vals: bytes, maxlen: int \| None = None) -\> list\[bytes\]: ... def bad_cb(*vals: bytes, maxitems: int \| None) -\> list\[bytes\]:  
...

batch_proc(\[\], good_cb) \# OK batch_proc(\[\], bad_cb) \# Error! Argument 2 has incompatible type because of \# different name and kind in the callback

</div>

Callables which take other callables as arguments may indicate that their parameter types are dependent on each other using `ParamSpec`. Additionally, if that callable adds or removes arguments from other callables, the `Concatenate` operator may be used. They take the form `Callable[ParamSpecVariable, ReturnType]` and `Callable[Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable], ReturnType]` respectively.

<div class="versionchanged">

3.10 `Callable` now supports `ParamSpec` and `Concatenate`. See `612` for more details.

</div>

<div class="seealso">

The documentation for `ParamSpec` and `Concatenate` provides examples of usage in `Callable`.

</div>

## Generics

Since type information about objects kept in containers cannot be statically inferred in a generic way, many container classes in the standard library support subscription to denote the expected types of container elements.

<div class="testcode">

from collections.abc import Mapping, Sequence

class Employee: ...

\# Sequence\[Employee\] indicates that all elements in the sequence \# must be instances of "Employee". \# Mapping\[str, str\] indicates that all keys and all values in the mapping \# must be strings. def notify_by_email(employees: Sequence\[Employee\], overrides: Mapping\[str, str\]) -\> None: ...

</div>

Generic functions and classes can be parameterized by using `type parameter syntax <type-params>`:

    from collections.abc import Sequence

    def first[T](l: Sequence[T]) -> T:  # Function is generic over the TypeVar "T"
        return l[0]

Or by using the `TypeVar` factory directly:

    from collections.abc import Sequence
    from typing import TypeVar

    U = TypeVar('U')                  # Declare type variable "U"

    def second(l: Sequence[U]) -> U:  # Function is generic over the TypeVar "U"
        return l[1]

<div class="versionchanged">

3.12 Syntactic support for generics is new in Python 3.12.

</div>

## Annotating tuples

For most containers in Python, the typing system assumes that all elements in the container will be of the same type. For example:

    from collections.abc import Mapping

    # Type checker will infer that all elements in ``x`` are meant to be ints
    x: list[int] = []

    # Type checker error: ``list`` only accepts a single type argument:
    y: list[int, str] = [1, 'foo']

    # Type checker will infer that all keys in ``z`` are meant to be strings,
    # and that all values in ``z`` are meant to be either strings or ints
    z: Mapping[str, str | int] = {}

`list` only accepts one type argument, so a type checker would emit an error on the `y` assignment above. Similarly, `~collections.abc.Mapping` only accepts two type arguments: the first indicates the type of the keys, and the second indicates the type of the values.

Unlike most other Python containers, however, it is common in idiomatic Python code for tuples to have elements which are not all of the same type. For this reason, tuples are special-cased in Python's typing system. `tuple` accepts *any number* of type arguments:

    # OK: ``x`` is assigned to a tuple of length 1 where the sole element is an int
    x: tuple[int] = (5,)

    # OK: ``y`` is assigned to a tuple of length 2;
    # element 1 is an int, element 2 is a str
    y: tuple[int, str] = (5, "foo")

    # Error: the type annotation indicates a tuple of length 1,
    # but ``z`` has been assigned to a tuple of length 3
    z: tuple[int] = (1, 2, 3)

<div class="index">

single: ...; ellipsis literal

</div>

To denote a tuple which could be of *any* length, and in which all elements are of the same type `T`, use the literal ellipsis `...`: `tuple[T, ...]`. To denote an empty tuple, use `tuple[()]`. Using plain `tuple` as an annotation is equivalent to using `tuple[Any, ...]`:

    x: tuple[int, ...] = (1, 2)
    # These reassignments are OK: ``tuple[int, ...]`` indicates x can be of any length
    x = (1, 2, 3)
    x = ()
    # This reassignment is an error: all elements in ``x`` must be ints
    x = ("foo", "bar")

    # ``y`` can only ever be assigned to an empty tuple
    y: tuple[()] = ()

    z: tuple = ("foo", "bar")
    # These reassignments are OK: plain ``tuple`` is equivalent to ``tuple[Any, ...]``
    z = (1, 2, 3)
    z = ()

## The type of class objects

A variable annotated with `C` may accept a value of type `C`. In contrast, a variable annotated with `type[C]` (or deprecated `typing.Type[C] <Type>`) may accept values that are classes themselves -- specifically, it will accept the *class object* of `C`. For example:

    a = 3         # Has type ``int``
    b = int       # Has type ``type[int]``
    c = type(a)   # Also has type ``type[int]``

Note that `type[C]` is covariant:

    class User: ...
    class ProUser(User): ...
    class TeamUser(User): ...

    def make_new_user(user_class: type[User]) -> User:
        # ...
        return user_class()

    make_new_user(User)      # OK
    make_new_user(ProUser)   # Also OK: ``type[ProUser]`` is a subtype of ``type[User]``
    make_new_user(TeamUser)  # Still fine
    make_new_user(User())    # Error: expected ``type[User]`` but got ``User``
    make_new_user(int)       # Error: ``type[int]`` is not a subtype of ``type[User]``

The only legal parameters for `type` are classes, `Any`, `type variables <generics>`, and unions of any of these types. For example:

    def new_non_team_user(user_class: type[BasicUser | ProUser]): ...

    new_non_team_user(BasicUser)  # OK
    new_non_team_user(ProUser)    # OK
    new_non_team_user(TeamUser)   # Error: ``type[TeamUser]`` is not a subtype
                                  # of ``type[BasicUser | ProUser]``
    new_non_team_user(User)       # Also an error

`type[Any]` is equivalent to `type`, which is the root of Python's `metaclass hierarchy <metaclasses>`.

## Annotating generators and coroutines

A generator can be annotated using the generic type `Generator[YieldType, SendType, ReturnType] <collections.abc.Generator>`. For example:

    def echo_round() -> Generator[int, float, str]:
        sent = yield 0
        while sent >= 0:
            sent = yield round(sent)
        return 'Done'

Note that unlike many other generic classes in the standard library, the `SendType` of `~collections.abc.Generator` behaves contravariantly, not covariantly or invariantly.

The `SendType` and `ReturnType` parameters default to `!None`:

    def infinite_stream(start: int) -> Generator[int]:
        while True:
            yield start
            start += 1

It is also possible to set these types explicitly:

    def infinite_stream(start: int) -> Generator[int, None, None]:
        while True:
            yield start
            start += 1

Simple generators that only ever yield values can also be annotated as having a return type of either `Iterable[YieldType] <collections.abc.Iterable>` or `Iterator[YieldType] <collections.abc.Iterator>`:

    def infinite_stream(start: int) -> Iterator[int]:
        while True:
            yield start
            start += 1

Async generators are handled in a similar fashion, but don't expect a `ReturnType` type argument (`AsyncGenerator[YieldType, SendType] <collections.abc.AsyncGenerator>`). The `SendType` argument defaults to `!None`, so the following definitions are equivalent:

    async def infinite_stream(start: int) -> AsyncGenerator[int]:
        while True:
            yield start
            start = await increment(start)

    async def infinite_stream(start: int) -> AsyncGenerator[int, None]:
        while True:
            yield start
            start = await increment(start)

As in the synchronous case, `AsyncIterable[YieldType] <collections.abc.AsyncIterable>` and `AsyncIterator[YieldType] <collections.abc.AsyncIterator>` are available as well:

    async def infinite_stream(start: int) -> AsyncIterator[int]:
        while True:
            yield start
            start = await increment(start)

Coroutines can be annotated using `Coroutine[YieldType, SendType, ReturnType] <collections.abc.Coroutine>`. Generic arguments correspond to those of `~collections.abc.Generator`, for example:

    from collections.abc import Coroutine
    c: Coroutine[list[str], str, int]  # Some coroutine defined elsewhere
    x = c.send('hi')                   # Inferred type of 'x' is list[str]
    async def bar() -> None:
        y = await c                    # Inferred type of 'y' is int

## User-defined generic types

A user-defined class can be defined as a generic class.

    from logging import Logger

    class LoggedVar[T]:
        def __init__(self, value: T, name: str, logger: Logger) -> None:
            self.name = name
            self.logger = logger
            self.value = value

        def set(self, new: T) -> None:
            self.log('Set ' + repr(self.value))
            self.value = new

        def get(self) -> T:
            self.log('Get ' + repr(self.value))
            return self.value

        def log(self, message: str) -> None:
            self.logger.info('%s: %s', self.name, message)

This syntax indicates that the class `LoggedVar` is parameterised around a single `type variable <typevar>` `T` . This also makes `T` valid as a type within the class body.

Generic classes implicitly inherit from `Generic`. For compatibility with Python 3.11 and lower, it is also possible to inherit explicitly from `Generic` to indicate a generic class:

    from typing import TypeVar, Generic

    T = TypeVar('T')

    class LoggedVar(Generic[T]):
        ...

Generic classes have `~object.__class_getitem__` methods, meaning they can be parameterised at runtime (e.g. `LoggedVar[int]` below):

    from collections.abc import Iterable

    def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None:
        for var in vars:
            var.set(0)

A generic type can have any number of type variables. All varieties of `TypeVar` are permissible as parameters for a generic type:

    from typing import TypeVar, Generic, Sequence

    class WeirdTrio[T, B: Sequence[bytes], S: (int, str)]:
        ...

    OldT = TypeVar('OldT', contravariant=True)
    OldB = TypeVar('OldB', bound=Sequence[bytes], covariant=True)
    OldS = TypeVar('OldS', int, str)

    class OldWeirdTrio(Generic[OldT, OldB, OldS]):
        ...

Each type variable argument to `Generic` must be distinct. This is thus invalid:

    from typing import TypeVar, Generic
    ...

    class Pair[M, M]:  # SyntaxError
        ...

    T = TypeVar('T')

    class Pair(Generic[T, T]):   # INVALID
        ...

Generic classes can also inherit from other classes:

    from collections.abc import Sized

    class LinkedList[T](Sized):
        ...

When inheriting from generic classes, some type parameters could be fixed:

    from collections.abc import Mapping

    class MyDict[T](Mapping[str, T]):
        ...

In this case `MyDict` has a single parameter, `T`.

Using a generic class without specifying type parameters assumes `Any` for each position. In the following example, `MyIterable` is not generic but implicitly inherits from `Iterable[Any]`:

<div class="testcode">

from collections.abc import Iterable

class MyIterable(Iterable): \# Same as Iterable\[Any\]  
...

</div>

User-defined generic type aliases are also supported. Examples:

    from collections.abc import Iterable

    type Response[S] = Iterable[S] | int

    # Return type here is same as Iterable[str] | int
    def response(query: str) -> Response[str]:
        ...

    type Vec[T] = Iterable[tuple[T, T]]

    def inproduct[T: (int, float, complex)](v: Vec[T]) -> T: # Same as Iterable[tuple[T, T]]
        return sum(x*y for x, y in v)

For backward compatibility, generic type aliases can also be created through a simple assignment:

    from collections.abc import Iterable
    from typing import TypeVar

    S = TypeVar("S")
    Response = Iterable[S] | int

<div class="versionchanged">

3.7 `Generic` no longer has a custom metaclass.

</div>

<div class="versionchanged">

3.12 Syntactic support for generics and type aliases is new in version 3.12. Previously, generic classes had to explicitly inherit from `Generic` or contain a type variable in one of their bases.

</div>

User-defined generics for parameter expressions are also supported via parameter specification variables in the form `[**P]`. The behavior is consistent with type variables' described above as parameter specification variables are treated by the `!typing` module as a specialized type variable. The one exception to this is that a list of types can be used to substitute a `ParamSpec`:

    >>> class Z[T, **P]: ...  # T is a TypeVar; P is a ParamSpec
    ...
    >>> Z[int, [dict, float]]
    __main__.Z[int, [dict, float]]

Classes generic over a `ParamSpec` can also be created using explicit inheritance from `Generic`. In this case, `**` is not used:

    from typing import ParamSpec, Generic

    P = ParamSpec('P')

    class Z(Generic[P]):
        ...

Another difference between `TypeVar` and `ParamSpec` is that a generic with only one parameter specification variable will accept parameter lists in the forms `X[[Type1, Type2, ...]]` and also `X[Type1, Type2, ...]` for aesthetic reasons. Internally, the latter is converted to the former, so the following are equivalent:

    >>> class X[**P]: ...
    ...
    >>> X[int, str]
    __main__.X[[int, str]]
    >>> X[[int, str]]
    __main__.X[[int, str]]

Note that generics with `ParamSpec` may not have correct `__parameters__` after substitution in some cases because they are intended primarily for static type checking.

<div class="versionchanged">

3.10 `Generic` can now be parameterized over parameter expressions. See `ParamSpec` and `612` for more details.

</div>

A user-defined generic class can have ABCs as base classes without a metaclass conflict. Generic metaclasses are not supported. The outcome of parameterizing generics is cached, and most types in the `!typing` module are `hashable` and comparable for equality.

## The `Any` type

A special kind of type is `Any`. A static type checker will treat every type as assignable to `Any` and `Any` as assignable to every type.

This means that it is possible to perform any operation or method call on a value of type `Any` and assign it to any variable:

    from typing import Any

    a: Any = None
    a = []          # OK
    a = 2           # OK

    s: str = ''
    s = a           # OK

    def foo(item: Any) -> int:
        # Passes type checking; 'item' could be any type,
        # and that type might have a 'bar' method
        item.bar()
        ...

Notice that no type checking is performed when assigning a value of type `Any` to a more precise type. For example, the static type checker did not report an error when assigning `a` to `s` even though `s` was declared to be of type `str` and receives an `int` value at runtime!

Furthermore, all functions without a return type or parameter types will implicitly default to using `Any`:

    def legacy_parser(text):
        ...
        return data

    # A static type checker will treat the above
    # as having the same signature as:
    def legacy_parser(text: Any) -> Any:
        ...
        return data

This behavior allows `Any` to be used as an *escape hatch* when you need to mix dynamically and statically typed code.

Contrast the behavior of `Any` with the behavior of `object`. Similar to `Any`, every type is a subtype of `object`. However, unlike `Any`, the reverse is not true: `object` is *not* a subtype of every other type.

That means when the type of a value is `object`, a type checker will reject almost all operations on it, and assigning it to a variable (or using it as a return value) of a more specialized type is a type error. For example:

    def hash_a(item: object) -> int:
        # Fails type checking; an object does not have a 'magic' method.
        item.magic()
        ...

    def hash_b(item: Any) -> int:
        # Passes type checking
        item.magic()
        ...

    # Passes type checking, since ints and strs are subclasses of object
    hash_a(42)
    hash_a("foo")

    # Passes type checking, since Any is assignable to all types
    hash_b(42)
    hash_b("foo")

Use `object` to indicate that a value could be any type in a typesafe manner. Use `Any` to indicate that a value is dynamically typed.

## Nominal vs structural subtyping

Initially `484` defined the Python static type system as using *nominal subtyping*. This means that a class `A` is allowed where a class `B` is expected if and only if `A` is a subclass of `B`.

This requirement previously also applied to abstract base classes, such as `~collections.abc.Iterable`. The problem with this approach is that a class had to be explicitly marked to support them, which is unpythonic and unlike what one would normally do in idiomatic dynamically typed Python code. For example, this conforms to `484`:

    from collections.abc import Sized, Iterable, Iterator

    class Bucket(Sized, Iterable[int]):
        ...
        def __len__(self) -> int: ...
        def __iter__(self) -> Iterator[int]: ...

`544` solves this problem by allowing users to write the above code without explicit base classes in the class definition, allowing `Bucket` to be implicitly considered a subtype of both `Sized` and `Iterable[int]` by static type checkers. This is known as *structural subtyping* (or static duck-typing):

    from collections.abc import Iterator, Iterable

    class Bucket:  # Note: no base classes
        ...
        def __len__(self) -> int: ...
        def __iter__(self) -> Iterator[int]: ...

    def collect(items: Iterable[int]) -> int: ...
    result = collect(Bucket())  # Passes type check

Moreover, by subclassing a special class `Protocol`, a user can define new custom protocols to fully enjoy structural subtyping (see examples below).

## Module contents

The `typing` module defines the following classes, functions and decorators.

### Special typing primitives

#### Special types

These can be used as types in annotations. They do not support subscription using `[]`.

<div class="data">

Any

Special type indicating an unconstrained type.

- Every type is assignable to `Any`.
- `Any` is assignable to every type.

<div class="versionchanged">

3.11 `Any` can now be used as a base class. This can be useful for avoiding type checker errors with classes that can duck type anywhere or are highly dynamic.

</div>

</div>

<div class="data">

AnyStr

A `constrained type variable <typing-constrained-typevar>`.

Definition:

    AnyStr = TypeVar('AnyStr', str, bytes)

`AnyStr` is meant to be used for functions that may accept `str` or `bytes` arguments but cannot allow the two to mix.

For example:

    def concat(a: AnyStr, b: AnyStr) -> AnyStr:
        return a + b

    concat("foo", "bar")    # OK, output has type 'str'
    concat(b"foo", b"bar")  # OK, output has type 'bytes'
    concat("foo", b"bar")   # Error, cannot mix str and bytes

Note that, despite its name, `AnyStr` has nothing to do with the `Any` type, nor does it mean "any string". In particular, `AnyStr` and `str | bytes` are different from each other and have different use cases:

    # Invalid use of AnyStr:
    # The type variable is used only once in the function signature,
    # so cannot be "solved" by the type checker
    def greet_bad(cond: bool) -> AnyStr:
        return "hi there!" if cond else b"greetings!"

    # The better way of annotating this function:
    def greet_proper(cond: bool) -> str | bytes:
        return "hi there!" if cond else b"greetings!"

<div class="deprecated-removed">

3.13 3.18 Deprecated in favor of the new `type parameter syntax <type-params>`. Use `class A[T: (str, bytes)]: ...` instead of importing `AnyStr`. See `695` for more details.

In Python 3.16, `AnyStr` will be removed from `typing.__all__`, and deprecation warnings will be emitted at runtime when it is accessed or imported from `typing`. `AnyStr` will be removed from `typing` in Python 3.18.

</div>

</div>

<div class="data">

LiteralString

Special type that includes only literal strings.

Any string literal is compatible with `LiteralString`, as is another `LiteralString`. However, an object typed as just `str` is not. A string created by composing `LiteralString`-typed objects is also acceptable as a `LiteralString`.

Example:

<div class="testcode">

def run_query(sql: LiteralString) -\> None:  
...

def caller(arbitrary_string: str, literal_string: LiteralString) -\> None:  
run_query("SELECT \* FROM students") \# OK run_query(literal_string) \# OK run_query("SELECT \* FROM " + literal_string) \# OK run_query(arbitrary_string) \# type checker error run_query( \# type checker error f"SELECT \* FROM students WHERE name = {arbitrary_string}" )

</div>

`LiteralString` is useful for sensitive APIs where arbitrary user-generated strings could generate problems. For example, the two cases above that generate type checker errors could be vulnerable to an SQL injection attack.

See `675` for more details.

<div class="versionadded">

3.11

</div>

</div>

<div class="data">

Never NoReturn

`!Never` and `!NoReturn` represent the [bottom type](https://en.wikipedia.org/wiki/Bottom_type), a type that has no members.

They can be used to indicate that a function never returns, such as `sys.exit`:

    from typing import Never  # or NoReturn

    def stop() -> Never:
        raise RuntimeError('no way')

Or to define a function that should never be called, as there are no valid arguments, such as `assert_never`:

    from typing import Never  # or NoReturn

    def never_call_me(arg: Never) -> None:
        pass

    def int_or_str(arg: int | str) -> None:
        never_call_me(arg)  # type checker error
        match arg:
            case int():
                print("It's an int")
            case str():
                print("It's a str")
            case _:
                never_call_me(arg)  # OK, arg is of type Never (or NoReturn)

`!Never` and `!NoReturn` have the same meaning in the type system and static type checkers treat both equivalently.

<div class="versionadded">

3.6.2

Added `NoReturn`.

</div>

<div class="versionadded">

3.11

Added `Never`.

</div>

</div>

<div class="data">

Self

Special type to represent the current enclosed class.

For example:

    from typing import Self, reveal_type

    class Foo:
        def return_self(self) -> Self:
            ...
            return self

    class SubclassOfFoo(Foo): pass

    reveal_type(Foo().return_self())  # Revealed type is "Foo"
    reveal_type(SubclassOfFoo().return_self())  # Revealed type is "SubclassOfFoo"

This annotation is semantically equivalent to the following, albeit in a more succinct fashion:

    from typing import TypeVar

    Self = TypeVar("Self", bound="Foo")

    class Foo:
        def return_self(self: Self) -> Self:
            ...
            return self

In general, if something returns `self`, as in the above examples, you should use `Self` as the return annotation. If `Foo.return_self` was annotated as returning `"Foo"`, then the type checker would infer the object returned from `SubclassOfFoo.return_self` as being of type `Foo` rather than `SubclassOfFoo`.

Other common use cases include:

- `classmethod`s that are used as alternative constructors and return instances of the `cls` parameter.
- Annotating an `~object.__enter__` method which returns self.

You should not use `Self` as the return annotation if the method is not guaranteed to return an instance of a subclass when the class is subclassed:

    class Eggs:
        # Self would be an incorrect return annotation here,
        # as the object returned is always an instance of Eggs,
        # even in subclasses
        def returns_eggs(self) -> "Eggs":
            return Eggs()

See `673` for more details.

<div class="versionadded">

3.11

</div>

</div>

<div class="data">

TypeAlias

Special annotation for explicitly declaring a `type alias <type-aliases>`.

For example:

    from typing import TypeAlias

    Factors: TypeAlias = list[int]

`TypeAlias` is particularly useful on older Python versions for annotating aliases that make use of forward references, as it can be hard for type checkers to distinguish these from normal variable assignments:

<div class="testcode">

from typing import Generic, TypeAlias, TypeVar

T = TypeVar("T")

\# "Box" does not exist yet, \# so we have to use quotes for the forward reference on Python \<3.12. \# Using `TypeAlias` tells the type checker that this is a type alias declaration, \# not a variable assignment to a string. BoxOfStrings: TypeAlias = "Box\[str\]"

class Box(Generic\[T\]):  
@classmethod def make_box_of_strings(cls) -\> BoxOfStrings: ...

</div>

See `613` for more details.

<div class="versionadded">

3.10

</div>

<div class="deprecated">

3.12 `TypeAlias` is deprecated in favor of the `type` statement, which creates instances of `TypeAliasType` and which natively supports forward references. Note that while `TypeAlias` and `TypeAliasType` serve similar purposes and have similar names, they are distinct and the latter is not the type of the former. Removal of `TypeAlias` is not currently planned, but users are encouraged to migrate to `type` statements.

</div>

</div>

#### Special forms

These can be used as types in annotations. They all support subscription using `[]`, but each has a unique syntax.

<div class="Union">

Union type; `Union[X, Y]` is equivalent to `X | Y` and means either X or Y.

To define a union, use e.g. `Union[int, str]` or the shorthand `int | str`. Using that shorthand is recommended. Details:

- The arguments must be types and there must be at least one.

- Unions of unions are flattened, e.g.:

      Union[Union[int, str], float] == Union[int, str, float]

  However, this does not apply to unions referenced through a type alias, to avoid forcing evaluation of the underlying `TypeAliasType`:

      type A = Union[int, str]
      Union[A, float] != Union[int, str, float]

- Unions of a single argument vanish, e.g.:

      Union[int] == int  # The constructor actually returns int

- Redundant arguments are skipped, e.g.:

      Union[int, str, int] == Union[int, str] == int | str

- When comparing unions, the argument order is ignored, e.g.:

      Union[int, str] == Union[str, int]

- You cannot subclass or instantiate a `Union`.

- You cannot write `Union[X][Y]`.

<div class="versionchanged">

3.7 Don't remove explicit subclasses from unions at runtime.

</div>

<div class="versionchanged">

3.10 Unions can now be written as `X | Y`. See `union type expressions<types-union>`.

</div>

<div class="versionchanged">

3.14 `types.UnionType` is now an alias for `Union`, and both `Union[int, str]` and `int | str` create instances of the same class. To check whether an object is a `Union` at runtime, use `isinstance(obj, Union)`. For compatibility with earlier versions of Python, use `get_origin(obj) is typing.Union or get_origin(obj) is types.UnionType`.

</div>

</div>

<div class="data">

Optional

`Optional[X]` is equivalent to `X | None` (or `Union[X, None]`).

Note that this is not the same concept as an optional argument, which is one that has a default. An optional argument with a default does not require the `Optional` qualifier on its type annotation just because it is optional. For example:

    def foo(arg: int = 0) -> None:
        ...

On the other hand, if an explicit value of `None` is allowed, the use of `Optional` is appropriate, whether the argument is optional or not. For example:

    def foo(arg: Optional[int] = None) -> None:
        ...

<div class="versionchanged">

3.10 Optional can now be written as `X | None`. See `union type expressions<types-union>`.

</div>

</div>

<div class="data">

Concatenate

Special form for annotating higher-order functions.

<div class="index">

single: ...; ellipsis literal

</div>

`Concatenate` can be used in conjunction with `Callable <annotating-callables>` and `ParamSpec` to annotate a higher-order callable which adds, removes, or transforms parameters of another callable. Usage is in the form `Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable]`. `Concatenate` is valid when used in `Callable <annotating-callables>` type hints and when instantiating user-defined generic classes with `ParamSpec` parameters. The last parameter to `Concatenate` must be a `ParamSpec` or ellipsis (`...`).

For example, to annotate a decorator `with_lock` which provides a `threading.Lock` to the decorated function, `Concatenate` can be used to indicate that `with_lock` expects a callable which takes in a `Lock` as the first argument, and returns a callable with a different type signature. In this case, the `ParamSpec` indicates that the returned callable's parameter types are dependent on the parameter types of the callable being passed in:

    from collections.abc import Callable
    from threading import Lock
    from typing import Concatenate

    # Use this lock to ensure that only one thread is executing a function
    # at any time.
    my_lock = Lock()

    def with_lock[**P, R](f: Callable[Concatenate[Lock, P], R]) -> Callable[P, R]:
        '''A type-safe decorator which provides a lock.'''
        def inner(*args: P.args, **kwargs: P.kwargs) -> R:
            # Provide the lock as the first argument.
            return f(my_lock, *args, **kwargs)
        return inner

    @with_lock
    def sum_threadsafe(lock: Lock, numbers: list[float]) -> float:
        '''Add a list of numbers together in a thread-safe manner.'''
        with lock:
            return sum(numbers)

    # We don't need to pass in the lock ourselves thanks to the decorator.
    sum_threadsafe([1.1, 2.2, 3.3])

<div class="versionadded">

3.10

</div>

<div class="seealso">

- `612` -- Parameter Specification Variables (the PEP which introduced `ParamSpec` and `Concatenate`)
- `ParamSpec`
- `annotating-callables`

</div>

</div>

<div class="data">

Literal

Special typing form to define "literal types".

`Literal` can be used to indicate to type checkers that the annotated object has a value equivalent to one of the provided literals.

For example:

    def validate_simple(data: Any) -> Literal[True]:  # always returns True
        ...

    type Mode = Literal['r', 'rb', 'w', 'wb']
    def open_helper(file: str, mode: Mode) -> str:
        ...

    open_helper('/some/path', 'r')      # Passes type check
    open_helper('/other/path', 'typo')  # Error in type checker

`Literal[...]` cannot be subclassed. At runtime, an arbitrary value is allowed as type argument to `Literal[...]`, but type checkers may impose restrictions. See `586` for more details about literal types.

Additional details:

- The arguments must be literal values and there must be at least one.

- Nested `Literal` types are flattened, e.g.:

      assert Literal[Literal[1, 2], 3] == Literal[1, 2, 3]

  However, this does not apply to `Literal` types referenced through a type alias, to avoid forcing evaluation of the underlying `TypeAliasType`:

      type A = Literal[1, 2]
      assert Literal[A, 3] != Literal[1, 2, 3]

- Redundant arguments are skipped, e.g.:

      assert Literal[1, 2, 1] == Literal[1, 2]

- When comparing literals, the argument order is ignored, e.g.:

      assert Literal[1, 2] == Literal[2, 1]

- You cannot subclass or instantiate a `Literal`.

- You cannot write `Literal[X][Y]`.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.9.1 `Literal` now de-duplicates parameters. Equality comparisons of `Literal` objects are no longer order dependent. `Literal` objects will now raise a `TypeError` exception during equality comparisons if one of their parameters are not `hashable`.

</div>

</div>

<div class="data">

ClassVar

Special type construct to mark class variables.

As introduced in `526`, a variable annotation wrapped in ClassVar indicates that a given attribute is intended to be used as a class variable and should not be set on instances of that class. Usage:

    class Starship:
        stats: ClassVar[dict[str, int]] = {} # class variable
        damage: int = 10                     # instance variable

`ClassVar` accepts only types and cannot be further subscribed.

`ClassVar` is not a class itself, and cannot be used with `isinstance` or `issubclass`. `ClassVar` does not change Python runtime behavior, but it can be used by static type checkers. For example, a type checker might flag the following code as an error:

    enterprise_d = Starship(3000)
    enterprise_d.stats = {} # Error, setting class variable on instance
    Starship.stats = {}     # This is OK

<div class="versionadded">

3.5.3

</div>

<div class="versionchanged">

3.13

`ClassVar` can now be nested in `Final` and vice versa.

</div>

</div>

<div class="data">

Final

Special typing construct to indicate final names to type checkers.

Final names cannot be reassigned in any scope. Final names declared in class scopes cannot be overridden in subclasses.

For example:

    MAX_SIZE: Final = 9000
    MAX_SIZE += 1  # Error reported by type checker

    class Connection:
        TIMEOUT: Final[int] = 10

    class FastConnector(Connection):
        TIMEOUT = 1  # Error reported by type checker

There is no runtime checking of these properties. See `591` for more details.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.13

`Final` can now be nested in `ClassVar` and vice versa.

</div>

</div>

<div class="data">

Required

Special typing construct to mark a `TypedDict` key as required.

This is mainly useful for `total=False` TypedDicts. See `TypedDict` and `655` for more details.

<div class="versionadded">

3.11

</div>

</div>

<div class="data">

NotRequired

Special typing construct to mark a `TypedDict` key as potentially missing.

See `TypedDict` and `655` for more details.

<div class="versionadded">

3.11

</div>

</div>

<div class="data">

ReadOnly

A special typing construct to mark an item of a `TypedDict` as read-only.

For example:

    class Movie(TypedDict):
       title: ReadOnly[str]
       year: int

    def mutate_movie(m: Movie) -> None:
       m["year"] = 1999  # allowed
       m["title"] = "The Matrix"  # type checker error

There is no runtime checking for this property.

See `TypedDict` and `705` for more details.

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

Annotated

Special typing form to add context-specific metadata to an annotation.

Add metadata `x` to a given type `T` by using the annotation `Annotated[T, x]`. Metadata added using `Annotated` can be used by static analysis tools or at runtime. At runtime, the metadata is stored in a `!__metadata__` attribute.

If a library or tool encounters an annotation `Annotated[T, x]` and has no special logic for the metadata, it should ignore the metadata and simply treat the annotation as `T`. As such, `Annotated` can be useful for code that wants to use annotations for purposes outside Python's static typing system.

Using `Annotated[T, x]` as an annotation still allows for static typechecking of `T`, as type checkers will simply ignore the metadata `x`. In this way, `Annotated` differs from the `no_type_check` decorator, which can also be used for adding annotations outside the scope of the typing system, but completely disables typechecking for a function or class.

The responsibility of how to interpret the metadata lies with the tool or library encountering an `Annotated` annotation. A tool or library encountering an `Annotated` type can scan through the metadata elements to determine if they are of interest (e.g., using `isinstance`).

<div class="describe">

Annotated\[\<type\>, \<metadata\>\]

</div>

Here is an example of how you might use `Annotated` to add metadata to type annotations if you were doing range analysis:

<div class="testcode">

@dataclass class ValueRange: lo: int hi: int

T1 = Annotated\[int, ValueRange(-10, 5)\] T2 = Annotated\[T1, ValueRange(-20, 3)\]

</div>

The first argument to `Annotated` must be a valid type. Multiple metadata elements can be supplied as `Annotated` supports variadic arguments. The order of the metadata elements is preserved and matters for equality checks:

    @dataclass
    class ctype:
         kind: str

    a1 = Annotated[int, ValueRange(3, 10), ctype("char")]
    a2 = Annotated[int, ctype("char"), ValueRange(3, 10)]

    assert a1 != a2  # Order matters

It is up to the tool consuming the annotations to decide whether the client is allowed to add multiple metadata elements to one annotation and how to merge those annotations.

Nested `Annotated` types are flattened. The order of the metadata elements starts with the innermost annotation:

    assert Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[
        int, ValueRange(3, 10), ctype("char")
    ]

However, this does not apply to `Annotated` types referenced through a type alias, to avoid forcing evaluation of the underlying `TypeAliasType`:

    type From3To10[T] = Annotated[T, ValueRange(3, 10)]
    assert Annotated[From3To10[int], ctype("char")] != Annotated[
       int, ValueRange(3, 10), ctype("char")
    ]

Duplicated metadata elements are not removed:

    assert Annotated[int, ValueRange(3, 10)] != Annotated[
        int, ValueRange(3, 10), ValueRange(3, 10)
    ]

`Annotated` can be used with nested and generic aliases:

> <div class="testcode">
>
> @dataclass class MaxLen: value: int
>
> type Vec\[T\] = Annotated\[list\[tuple\[T, T\]\], MaxLen(10)\]
>
> \# When used in a type annotation, a type checker will treat "V" the same as \# `Annotated[list[tuple[int, int]], MaxLen(10)]`: type V = Vec\[int\]
>
> </div>

`Annotated` cannot be used with an unpacked `TypeVarTuple`:

    type Variadic[*Ts] = Annotated[*Ts, Ann1] = Annotated[T1, T2, T3, ..., Ann1]  # NOT valid

where `T1`, `T2`, ... are `TypeVars <TypeVar>`. This is invalid as only one type should be passed to Annotated.

By default, `get_type_hints` strips the metadata from annotations. Pass `include_extras=True` to have the metadata preserved:

> <div class="doctest">
>
> \>\>\> from typing import Annotated, get_type_hints \>\>\> def func(x: Annotated\[int, "metadata"\]) -\> None: pass ... \>\>\> get_type_hints(func) {'x': \<class 'int'\>, 'return': \<class 'NoneType'\>} \>\>\> get_type_hints(func, include_extras=True) {'x': typing.Annotated\[int, 'metadata'\], 'return': \<class 'NoneType'\>}
>
> </div>

At runtime, the metadata associated with an `Annotated` type can be retrieved via the `!__metadata__` attribute:

> <div class="doctest">
>
> \>\>\> from typing import Annotated \>\>\> X = Annotated\[int, "very", "important", "metadata"\] \>\>\> X typing.Annotated\[int, 'very', 'important', 'metadata'\] \>\>\> X.\_\_metadata\_\_ ('very', 'important', 'metadata')
>
> </div>

If you want to retrieve the original type wrapped by `Annotated`, use the `!__origin__` attribute:

> <div class="doctest">
>
> \>\>\> from typing import Annotated, get_origin \>\>\> Password = Annotated\[str, "secret"\] \>\>\> Password.\_\_origin\_\_ \<class 'str'\>
>
> </div>

Note that using `get_origin` will return `Annotated` itself:

> <div class="doctest">
>
> \>\>\> get_origin(Password) typing.Annotated
>
> </div>

<div class="seealso">

`593` - Flexible function and variable annotations  
The PEP introducing `Annotated` to the standard library.

</div>

<div class="versionadded">

3.9

</div>

</div>

<div class="data">

TypeForm

A special form representing the value that results from evaluating a type expression.

This value encodes the information supplied in the type expression, and it represents the type described by that type expression.

When used in a type expression, `TypeForm` describes a set of type form objects. It accepts a single type argument, which must be a valid type expression. `TypeForm[T]` describes the set of all type form objects that represent the type `T` or types assignable to `T`.

`TypeForm(obj)` simply returns `obj` unchanged. This is useful for explicitly marking a value as a type form for static type checkers.

Example:

    from typing import Any, TypeForm

    def cast[T](typ: TypeForm[T], value: Any) -> T: ...

    reveal_type(cast(int, "x"))  # Revealed type is "int"

See `747` for details.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

TypeIs

Special typing construct for marking user-defined type predicate functions.

`TypeIs` can be used to annotate the return type of a user-defined type predicate function. `TypeIs` only accepts a single type argument. At runtime, functions marked this way should return a boolean and take at least one positional argument.

`TypeIs` aims to benefit *type narrowing* -- a technique used by static type checkers to determine a more precise type of an expression within a program's code flow. Usually type narrowing is done by analyzing conditional code flow and applying the narrowing to a block of code. The conditional expression here is sometimes referred to as a "type predicate":

    def is_str(val: str | float):
        # "isinstance" type predicate
        if isinstance(val, str):
            # Type of ``val`` is narrowed to ``str``
            ...
        else:
            # Else, type of ``val`` is narrowed to ``float``.
            ...

Sometimes it would be convenient to use a user-defined boolean function as a type predicate. Such a function should use `TypeIs[...]` or `TypeGuard` as its return type to alert static type checkers to this intention. `TypeIs` usually has more intuitive behavior than `TypeGuard`, but it cannot be used when the input and output types are incompatible (e.g., `list[object]` to `list[int]`) or when the function does not return `True` for all instances of the narrowed type.

Using `-> TypeIs[NarrowedType]` tells the static type checker that for a given function:

1.  The return value is a boolean.
2.  If the return value is `True`, the type of its argument is the intersection of the argument's original type and `NarrowedType`.
3.  If the return value is `False`, the type of its argument is narrowed to exclude `NarrowedType`.

For example:

    from typing import assert_type, final, TypeIs

    class Parent: pass
    class Child(Parent): pass
    @final
    class Unrelated: pass

    def is_parent(val: object) -> TypeIs[Parent]:
        return isinstance(val, Parent)

    def run(arg: Child | Unrelated):
        if is_parent(arg):
            # Type of ``arg`` is narrowed to the intersection
            # of ``Parent`` and ``Child``, which is equivalent to
            # ``Child``.
            assert_type(arg, Child)
        else:
            # Type of ``arg`` is narrowed to exclude ``Parent``,
            # so only ``Unrelated`` is left.
            assert_type(arg, Unrelated)

The type inside `TypeIs` must be consistent with the type of the function's argument; if it is not, static type checkers will raise an error. An incorrectly written `TypeIs` function can lead to unsound behavior in the type system; it is the user's responsibility to write such functions in a type-safe manner.

If a `TypeIs` function is a class or instance method, then the type in `TypeIs` maps to the type of the second parameter (after `cls` or `self`).

In short, the form `def foo(arg: TypeA) -> TypeIs[TypeB]: ...`, means that if `foo(arg)` returns `True`, then `arg` is an instance of `TypeB`, and if it returns `False`, it is not an instance of `TypeB`.

`TypeIs` also works with type variables. For more information, see `742` (Narrowing types with `TypeIs`).

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

TypeGuard

Special typing construct for marking user-defined type predicate functions.

Type predicate functions are user-defined functions that return whether their argument is an instance of a particular type. `TypeGuard` works similarly to `TypeIs`, but has subtly different effects on type checking behavior (see below).

Using `-> TypeGuard` tells the static type checker that for a given function:

1.  The return value is a boolean.
2.  If the return value is `True`, the type of its argument is the type inside `TypeGuard`.

`TypeGuard` also works with type variables. See `647` for more details.

For example:

    def is_str_list(val: list[object]) -> TypeGuard[list[str]]:
        '''Determines whether all objects in the list are strings'''
        return all(isinstance(x, str) for x in val)

    def func1(val: list[object]):
        if is_str_list(val):
            # Type of ``val`` is narrowed to ``list[str]``.
            print(" ".join(val))
        else:
            # Type of ``val`` remains as ``list[object]``.
            print("Not a list of strings!")

`TypeIs` and `TypeGuard` differ in the following ways:

- `TypeIs` requires the narrowed type to be a subtype of the input type, while `TypeGuard` does not. The main reason is to allow for things like narrowing `list[object]` to `list[str]` even though the latter is not a subtype of the former, since `list` is invariant.
- When a `TypeGuard` function returns `True`, type checkers narrow the type of the variable to exactly the `TypeGuard` type. When a `TypeIs` function returns `True`, type checkers can infer a more precise type combining the previously known type of the variable with the `TypeIs` type. (Technically, this is known as an intersection type.)
- When a `TypeGuard` function returns `False`, type checkers cannot narrow the type of the variable at all. When a `TypeIs` function returns `False`, type checkers can narrow the type of the variable to exclude the `TypeIs` type.

<div class="versionadded">

3.10

</div>

</div>

<div class="data">

Unpack

Typing operator to conceptually mark an object as having been unpacked.

For example, using the unpack operator `*` on a `type variable tuple <typevartuple>` is equivalent to using `Unpack` to mark the type variable tuple as having been unpacked:

    Ts = TypeVarTuple('Ts')
    tup: tuple[*Ts]
    # Effectively does:
    tup: tuple[Unpack[Ts]]

In fact, `Unpack` can be used interchangeably with `*` in the context of `typing.TypeVarTuple <TypeVarTuple>` and `builtins.tuple <tuple>` types. You might see `Unpack` being used explicitly in older versions of Python, where `*` couldn't be used in certain places:

    # In older versions of Python, TypeVarTuple and Unpack
    # are located in the `typing_extensions` backports package.
    from typing_extensions import TypeVarTuple, Unpack

    Ts = TypeVarTuple('Ts')
    tup: tuple[*Ts]         # Syntax error on Python <= 3.10!
    tup: tuple[Unpack[Ts]]  # Semantically equivalent, and backwards-compatible

`Unpack` can also be used along with `typing.TypedDict` for typing `**kwargs` in a function signature:

    from typing import TypedDict, Unpack

    class Movie(TypedDict):
        name: str
        year: int

    # This function expects two keyword arguments - `name` of type `str`
    # and `year` of type `int`.
    def foo(**kwargs: Unpack[Movie]): ...

See `692` for more details on using `Unpack` for `**kwargs` typing.

<div class="versionadded">

3.11

</div>

</div>

#### Building generic types and type aliases

The following classes should not be used directly as annotations. Their intended purpose is to be building blocks for creating generic types and type aliases.

These objects can be created through special syntax (`type parameter lists <type-params>` and the `type` statement). For compatibility with Python 3.11 and earlier, they can also be created without the dedicated syntax, as documented below.

<div class="Generic">

Abstract base class for generic types.

A generic type is typically declared by adding a list of type parameters after the class name:

    class Mapping[KT, VT]:
        def __getitem__(self, key: KT) -> VT:
            ...
            # Etc.

Such a class implicitly inherits from `Generic`. The runtime semantics of this syntax are discussed in the `Language Reference <generic-classes>`.

This class can then be used as follows:

    def lookup_name[X, Y](mapping: Mapping[X, Y], key: X, default: Y) -> Y:
        try:
            return mapping[key]
        except KeyError:
            return default

Here the brackets after the function name indicate a `generic function <generic-functions>`.

For backwards compatibility, generic classes can also be declared by explicitly inheriting from `Generic`. In this case, the type parameters must be declared separately:

    KT = TypeVar('KT')
    VT = TypeVar('VT')

    class Mapping(Generic[KT, VT]):
        def __getitem__(self, key: KT) -> VT:
            ...
            # Etc.

</div>

<div id="typevar">

<div class="TypeVar(name, *constraints, bound=None, covariant=False, contravariant=False, infer_variance=False, default=typing.NoDefault)">

Type variable.

The preferred way to construct a type variable is via the dedicated syntax for `generic functions <generic-functions>`, `generic classes <generic-classes>`, and `generic type aliases <generic-type-aliases>`:

    class Sequence[T]:  # T is a TypeVar
        ...

This syntax can also be used to create bounded and constrained type variables:

    class StrSequence[S: str]:  # S is a TypeVar with a `str` upper bound;
        ...                     # we can say that S is "bounded by `str`"


    class StrOrBytesSequence[A: (str, bytes)]:  # A is a TypeVar constrained to str or bytes
        ...

However, if desired, reusable type variables can also be constructed manually, like so:

    T = TypeVar('T')  # Can be anything
    S = TypeVar('S', bound=str)  # Can be any subtype of str
    A = TypeVar('A', str, bytes)  # Must be exactly str or bytes

Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function and type alias definitions. See `Generic` for more information on generic types. Generic functions work as follows:

    def repeat[T](x: T, n: int) -> Sequence[T]:
        """Return a list containing n references to x."""
        return [x]*n


    def print_capitalized[S: str](x: S) -> S:
        """Print x capitalized, and return x."""
        print(x.capitalize())
        return x


    def concatenate[A: (str, bytes)](x: A, y: A) -> A:
        """Add two strings or bytes objects together."""
        return x + y

Note that type variables can be *bounded*, *constrained*, or neither, but cannot be both bounded *and* constrained.

The variance of type variables is inferred by type checkers when they are created through the `type parameter syntax <type-params>` or when `infer_variance=True` is passed. Manually created type variables may be explicitly marked covariant or contravariant by passing `covariant=True` or `contravariant=True`. By default, manually created type variables are invariant. See `484` and `695` for more details.

Bounded type variables and constrained type variables have different semantics in several important ways. Using a *bounded* type variable means that the `TypeVar` will be solved using the most specific type possible:

    x = print_capitalized('a string')
    reveal_type(x)  # revealed type is str

    class StringSubclass(str):
        pass

    y = print_capitalized(StringSubclass('another string'))
    reveal_type(y)  # revealed type is StringSubclass

    z = print_capitalized(45)  # error: int is not a subtype of str

The upper bound of a type variable can be a concrete type, abstract type (ABC or Protocol), or even a union of types:

    # Can be anything with an __abs__ method
    def print_abs[T: SupportsAbs](arg: T) -> None:
        print("Absolute value:", abs(arg))

    U = TypeVar('U', bound=str|bytes)  # Can be any subtype of the union str|bytes
    V = TypeVar('V', bound=SupportsAbs)  # Can be anything with an __abs__ method

<div id="typing-constrained-typevar">

Using a *constrained* type variable, however, means that the `TypeVar` can only ever be solved as being exactly one of the constraints given:

    a = concatenate('one', 'two')
    reveal_type(a)  # revealed type is str

    b = concatenate(StringSubclass('one'), StringSubclass('two'))
    reveal_type(b)  # revealed type is str, despite StringSubclass being passed in

    c = concatenate('one', b'two')  # error: type variable 'A' can be either str or bytes in a function call, but not both

</div>

At runtime, `isinstance(x, T)` will raise `TypeError`.

<div class="attribute">

\_\_name\_\_

The name of the type variable.

</div>

<div class="attribute">

\_\_covariant\_\_

Whether the type var has been explicitly marked as covariant.

</div>

<div class="attribute">

\_\_contravariant\_\_

Whether the type var has been explicitly marked as contravariant.

</div>

<div class="attribute">

\_\_infer_variance\_\_

Whether the type variable's variance should be inferred by type checkers.

<div class="versionadded">

3.12

</div>

</div>

<div class="attribute">

\_\_bound\_\_

The upper bound of the type variable, if any.

<div class="versionchanged">

3.12

For type variables created through `type parameter syntax <type-params>`, the bound is evaluated only when the attribute is accessed, not when the type variable is created (see `lazy-evaluation`).

</div>

</div>

<div class="method">

evaluate_bound

An `evaluate function` corresponding to the `~TypeVar.__bound__` attribute. When called directly, this method supports only the `~annotationlib.Format.VALUE` format, which is equivalent to accessing the `~TypeVar.__bound__` attribute directly, but the method object can be passed to `annotationlib.call_evaluate_function` to evaluate the value in a different format.

<div class="versionadded">

3.14

</div>

</div>

<div class="attribute">

\_\_constraints\_\_

A tuple containing the constraints of the type variable, if any.

<div class="versionchanged">

3.12

For type variables created through `type parameter syntax <type-params>`, the constraints are evaluated only when the attribute is accessed, not when the type variable is created (see `lazy-evaluation`).

</div>

</div>

<div class="method">

evaluate_constraints

An `evaluate function` corresponding to the `~TypeVar.__constraints__` attribute. When called directly, this method supports only the `~annotationlib.Format.VALUE` format, which is equivalent to accessing the `~TypeVar.__constraints__` attribute directly, but the method object can be passed to `annotationlib.call_evaluate_function` to evaluate the value in a different format.

<div class="versionadded">

3.14

</div>

</div>

<div class="attribute">

\_\_default\_\_

The default value of the type variable, or `typing.NoDefault` if it has no default.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

evaluate_default

An `evaluate function` corresponding to the `~TypeVar.__default__` attribute. When called directly, this method supports only the `~annotationlib.Format.VALUE` format, which is equivalent to accessing the `~TypeVar.__default__` attribute directly, but the method object can be passed to `annotationlib.call_evaluate_function` to evaluate the value in a different format.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

has_default()

Return whether or not the type variable has a default value. This is equivalent to checking whether `__default__` is not the `typing.NoDefault` singleton, except that it does not force evaluation of the `lazily evaluated <lazy-evaluation>` default value.

<div class="versionadded">

3.13

</div>

</div>

<div class="versionchanged">

3.12

Type variables can now be declared using the `type parameter <type-params>` syntax introduced by `695`. The `infer_variance` parameter was added.

</div>

<div class="versionchanged">

3.13

Support for default values was added.

</div>

</div>

</div>

<div id="typevartuple">

<div class="TypeVarTuple(name, *, bound=None, covariant=False, contravariant=False, infer_variance=False, default=typing.NoDefault)">

Type variable tuple. A specialized form of `type variable <typevar>` that enables *variadic* generics.

Type variable tuples can be declared in `type parameter lists <type-params>` using a single asterisk (`*`) before the name:

    def move_first_element_to_last[T, *Ts](tup: tuple[T, *Ts]) -> tuple[*Ts, T]:
        return (*tup[1:], tup[0])

Or by explicitly invoking the `TypeVarTuple` constructor:

    T = TypeVar("T")
    Ts = TypeVarTuple("Ts")

    def move_first_element_to_last(tup: tuple[T, *Ts]) -> tuple[*Ts, T]:
        return (*tup[1:], tup[0])

A normal type variable enables parameterization with a single type. A type variable tuple, in contrast, allows parameterization with an *arbitrary* number of types by acting like an *arbitrary* number of type variables wrapped in a tuple. For example:

    # T is bound to int, Ts is bound to ()
    # Return value is (1,), which has type tuple[int]
    move_first_element_to_last(tup=(1,))

    # T is bound to int, Ts is bound to (str,)
    # Return value is ('spam', 1), which has type tuple[str, int]
    move_first_element_to_last(tup=(1, 'spam'))

    # T is bound to int, Ts is bound to (str, float)
    # Return value is ('spam', 3.0, 1), which has type tuple[str, float, int]
    move_first_element_to_last(tup=(1, 'spam', 3.0))

    # This fails to type check (and fails at runtime)
    # because tuple[()] is not compatible with tuple[T, *Ts]
    # (at least one element is required)
    move_first_element_to_last(tup=())

Note the use of the unpacking operator `*` in `tuple[T, *Ts]`. Conceptually, you can think of `Ts` as a tuple of type variables `(T1, T2, ...)`. `tuple[T, *Ts]` would then become `tuple[T, *(T1, T2, ...)]`, which is equivalent to `tuple[T, T1, T2, ...]`. (Note that in older versions of Python, you might see this written using `Unpack <Unpack>` instead, as `Unpack[Ts]`.)

Type variable tuples must *always* be unpacked. This helps distinguish type variable tuples from normal type variables:

    x: Ts          # Not valid
    x: tuple[Ts]   # Not valid
    x: tuple[*Ts]  # The correct way to do it

Type variable tuples can be used in the same contexts as normal type variables. For example, in class definitions, arguments, and return types:

    class Array[*Shape]:
        def __getitem__(self, key: tuple[*Shape]) -> float: ...
        def __abs__(self) -> "Array[*Shape]": ...
        def get_shape(self) -> tuple[*Shape]: ...

Type variable tuples can be happily combined with normal type variables:

<div class="testcode">

class Array\[DType, \*Shape\]: \# This is fine  
pass

class Array2\[\*Shape, DType\]: \# This would also be fine  
pass

class Height: ... class Width: ...

float_array_1d: Array\[float, Height\] = Array() \# Totally fine int_array_2d: Array\[int, Height, Width\] = Array() \# Yup, fine too

</div>

However, note that at most one type variable tuple may appear in a single list of type arguments or type parameters:

    x: tuple[*Ts, *Ts]            # Not valid
    class Array[*Shape, *Shape]:  # Not valid
        pass

Finally, an unpacked type variable tuple can be used as the type annotation of `*args`:

    def call_soon[*Ts](
        callback: Callable[[*Ts], None],
        *args: *Ts
    ) -> None:
        ...
        callback(*args)

In contrast to non-unpacked annotations of `*args` - e.g. `*args: int`, which would specify that *all* arguments are `int` - `*args: *Ts` enables reference to the types of the *individual* arguments in `*args`. Here, this allows us to ensure the types of the `*args` passed to `call_soon` match the types of the (positional) arguments of `callback`.

See `646` for more details on type variable tuples.

<div class="attribute">

\_\_name\_\_

The name of the type variable tuple.

</div>

<div class="attribute">

\_\_covariant\_\_

Whether the type variable tuple has been explicitly marked as covariant.

<div class="versionadded">

3.15

</div>

</div>

<div class="attribute">

\_\_contravariant\_\_

Whether the type variable tuple has been explicitly marked as contravariant.

<div class="versionadded">

3.15

</div>

</div>

<div class="attribute">

\_\_infer_variance\_\_

Whether the type variable tuple's variance should be inferred by type checkers.

<div class="versionadded">

3.15

</div>

</div>

<div class="attribute">

\_\_default\_\_

The default value of the type variable tuple, or `typing.NoDefault` if it has no default.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

evaluate_default

An `evaluate function` corresponding to the `~TypeVarTuple.__default__` attribute. When called directly, this method supports only the `~annotationlib.Format.VALUE` format, which is equivalent to accessing the `~TypeVarTuple.__default__` attribute directly, but the method object can be passed to `annotationlib.call_evaluate_function` to evaluate the value in a different format.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

has_default()

Return whether or not the type variable tuple has a default value. This is equivalent to checking whether `__default__` is not the `typing.NoDefault` singleton, except that it does not force evaluation of the `lazily evaluated <lazy-evaluation>` default value.

<div class="versionadded">

3.13

</div>

</div>

Type variable tuples created with `covariant=True` or `contravariant=True` can be used to declare covariant or contravariant generic types. The `bound` argument is also accepted, similar to `TypeVar`, but its actual semantics are yet to be decided.

<div class="versionadded">

3.11

</div>

<div class="versionchanged">

3.12

Type variable tuples can now be declared using the `type parameter <type-params>` syntax introduced by `695`.

</div>

<div class="versionchanged">

3.13

Support for default values was added.

</div>

<div class="versionchanged">

3.15

Added support for the `bound`, `covariant`, `contravariant`, and `infer_variance` parameters.

</div>

</div>

</div>

<div class="ParamSpec(name, *, bound=None, covariant=False, contravariant=False, infer_variance=False, default=typing.NoDefault)">

Parameter specification variable. A specialized version of `type variables <typevar>`.

In `type parameter lists <type-params>`, parameter specifications can be declared with two asterisks (`**`):

    type IntFunc[**P] = Callable[P, int]

For compatibility with Python 3.11 and earlier, `ParamSpec` objects can also be created as follows:

    P = ParamSpec('P')

Parameter specification variables exist primarily for the benefit of static type checkers. They are used to forward the parameter types of one callable to another callable -- a pattern commonly found in higher order functions and decorators. They are only valid when used in `Concatenate`, or as the first argument to `Callable`, or as parameters for user-defined Generics. See `Generic` for more information on generic types.

For example, to add basic logging to a function, one can create a decorator `add_logging` to log function calls. The parameter specification variable tells the type checker that the callable passed into the decorator and the new callable returned by it have inter-dependent type parameters:

    from collections.abc import Callable
    import logging

    def add_logging[T, **P](f: Callable[P, T]) -> Callable[P, T]:
        '''A type-safe decorator to add logging to a function.'''
        def inner(*args: P.args, **kwargs: P.kwargs) -> T:
            logging.info(f'{f.__name__} was called')
            return f(*args, **kwargs)
        return inner

    @add_logging
    def add_two(x: float, y: float) -> float:
        '''Add two numbers together.'''
        return x + y

Without `ParamSpec`, the simplest way to annotate this previously was to use a `TypeVar` with upper bound `Callable[..., Any]`. However this causes two problems:

1.  The type checker can't type check the `inner` function because `*args` and `**kwargs` have to be typed `Any`.
2.  `~cast` may be required in the body of the `add_logging` decorator when returning the `inner` function, or the static type checker must be told to ignore the `return inner`.

<div class="attribute">

args

</div>

<div class="attribute">

kwargs

Since `ParamSpec` captures both positional and keyword parameters, `P.args` and `P.kwargs` can be used to split a `ParamSpec` into its components. `P.args` represents the tuple of positional parameters in a given call and should only be used to annotate `*args`. `P.kwargs` represents the mapping of keyword parameters to their values in a given call, and should be only be used to annotate `**kwargs`. Both attributes require the annotated parameter to be in scope. At runtime, `P.args` and `P.kwargs` are instances respectively of `ParamSpecArgs` and `ParamSpecKwargs`.

</div>

<div class="attribute">

\_\_name\_\_

The name of the parameter specification.

</div>

<div class="attribute">

\_\_covariant\_\_

Whether the parameter specification has been explicitly marked as covariant.

</div>

<div class="attribute">

\_\_contravariant\_\_

Whether the parameter specification has been explicitly marked as contravariant.

</div>

<div class="attribute">

\_\_infer_variance\_\_

Whether the parameter specification's variance should be inferred by type checkers.

<div class="versionadded">

3.12

</div>

</div>

<div class="attribute">

\_\_default\_\_

The default value of the parameter specification, or `typing.NoDefault` if it has no default.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

evaluate_default

An `evaluate function` corresponding to the `~ParamSpec.__default__` attribute. When called directly, this method supports only the `~annotationlib.Format.VALUE` format, which is equivalent to accessing the `~ParamSpec.__default__` attribute directly, but the method object can be passed to `annotationlib.call_evaluate_function` to evaluate the value in a different format.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

has_default()

Return whether or not the parameter specification has a default value. This is equivalent to checking whether `__default__` is not the `typing.NoDefault` singleton, except that it does not force evaluation of the `lazily evaluated <lazy-evaluation>` default value.

<div class="versionadded">

3.13

</div>

</div>

Parameter specification variables created with `covariant=True` or `contravariant=True` can be used to declare covariant or contravariant generic types. The `bound` argument is also accepted, similar to `TypeVar`. However the actual semantics of these keywords are yet to be decided.

<div class="versionadded">

3.10

</div>

<div class="versionchanged">

3.12

Parameter specifications can now be declared using the `type parameter <type-params>` syntax introduced by `695`.

</div>

<div class="versionchanged">

3.13

Support for default values was added.

</div>

> [!NOTE]
> Only parameter specification variables defined in global scope can be pickled.

<div class="seealso">

\* `612` -- Parameter Specification Variables (the PEP which introduced `ParamSpec` and `Concatenate`) \* `Concatenate` \* `annotating-callables`

</div>

</div>

<div class="ParamSpecArgs ParamSpecKwargs">

Arguments and keyword arguments attributes of a `ParamSpec`. The `P.args` attribute of a `ParamSpec` is an instance of `ParamSpecArgs`, and `P.kwargs` is an instance of `ParamSpecKwargs`. They are intended for runtime introspection and have no special meaning to static type checkers.

Calling `get_origin` on either of these objects will return the original `ParamSpec`:

<div class="doctest">

\>\>\> from typing import ParamSpec, get_origin \>\>\> P = ParamSpec("P") \>\>\> get_origin(P.args) is P True \>\>\> get_origin(P.kwargs) is P True

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="TypeAliasType(name, value, *, type_params=(), qualname=None)">

The type of type aliases created through the `type` statement.

Example:

<div class="doctest">

\>\>\> type Alias = int \>\>\> type(Alias) \<class 'typing.TypeAliasType'\>

</div>

<div class="versionadded">

3.12

</div>

<div class="attribute">

\_\_name\_\_

The name of the type alias:

<div class="doctest">

\>\>\> type Alias = int \>\>\> Alias.\_\_name\_\_ 'Alias'

</div>

</div>

<div class="attribute">

\_\_qualname\_\_

The `qualified name` of the type alias:

<div class="doctest">

\>\>\> class Class: ... type Alias = int ... \>\>\> Class.Alias.\_\_qualname\_\_ 'Class.Alias'

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="attribute">

\_\_module\_\_

The name of the module in which the type alias was defined:

    >>> type Alias = int
    >>> Alias.__module__
    '__main__'

This attribute is writable.

<div class="versionchanged">

3.15

The attribute is now writable.

</div>

</div>

<div class="attribute">

\_\_type_params\_\_

The type parameters of the type alias, or an empty tuple if the alias is not generic:

<div class="doctest">

\>\>\> type ListOrSet\[T\] = list\[T\] \| set\[T\] \>\>\> ListOrSet.\_\_type_params\_\_ (T,) \>\>\> type NotGeneric = int \>\>\> NotGeneric.\_\_type_params\_\_ ()

</div>

</div>

<div class="attribute">

\_\_value\_\_

The type alias's value. This is `lazily evaluated <lazy-evaluation>`, so names used in the definition of the alias are not resolved until the `__value__` attribute is accessed:

<div class="doctest">

\>\>\> type Mutually = Recursive \>\>\> type Recursive = Mutually \>\>\> Mutually Mutually \>\>\> Recursive Recursive \>\>\> Mutually.\_\_value\_\_ Recursive \>\>\> Recursive.\_\_value\_\_ Mutually

</div>

</div>

<div class="method">

evaluate_value

An `evaluate function` corresponding to the `__value__` attribute. When called directly, this method supports only the `~annotationlib.Format.VALUE` format, which is equivalent to accessing the `__value__` attribute directly, but the method object can be passed to `annotationlib.call_evaluate_function` to evaluate the value in a different format:

<div class="doctest">

\>\>\> type Alias = undefined \>\>\> Alias.\_\_value\_\_ Traceback (most recent call last): ... NameError: name 'undefined' is not defined \>\>\> from annotationlib import Format, call_evaluate_function \>\>\> Alias.evaluate_value(Format.VALUE) Traceback (most recent call last): ... NameError: name 'undefined' is not defined \>\>\> call_evaluate_function(Alias.evaluate_value, Format.FORWARDREF) ForwardRef('undefined')

</div>

<div class="versionadded">

3.14

</div>

</div>

**Unpacking**

Type aliases support star unpacking using the `*Alias` syntax. This is equivalent to using `Unpack[Alias]` directly:

<div class="doctest">

\>\>\> type Alias = tuple\[int, str\] \>\>\> type Unpacked = tuple\[bool, \*Alias\] \>\>\> Unpacked.\_\_value\_\_ tuple\[bool, typing.Unpack\[Alias\]\]

</div>

<div class="versionadded">

3.14

</div>

</div>

#### Other special directives

These functions and classes should not be used directly as annotations. Their intended purpose is to be building blocks for creating and declaring types.

<div class="NamedTuple">

Typed version of `collections.namedtuple`.

Usage:

    class Employee(NamedTuple):
        name: str
        id: int

This is equivalent to:

    Employee = collections.namedtuple('Employee', ['name', 'id'])

To give a field a default value, you can assign to it in the class body:

    class Employee(NamedTuple):
        name: str
        id: int = 3

    employee = Employee('Guido')
    assert employee.id == 3

Fields with a default value must come after any fields without a default.

The types for each field name can be retrieved by calling `annotationlib.get_annotations` on the resulting class. (The field names are in the `_fields` attribute and the default values are in the `_field_defaults` attribute, both of which are part of the `~collections.namedtuple` API.)

`NamedTuple` subclasses can also have docstrings and methods:

    class Employee(NamedTuple):
        """Represents an employee."""
        name: str
        id: int = 3

        def __repr__(self) -> str:
            return f'<Employee {self.name}, id={self.id}>'

`NamedTuple` subclasses can be generic:

    class Group[T](NamedTuple):
        key: T
        group: list[T]

Backward-compatible usage:

    # For creating a generic NamedTuple on Python 3.11
    T = TypeVar("T")

    class Group(NamedTuple, Generic[T]):
        key: T
        group: list[T]

    # A functional syntax is also supported
    Employee = NamedTuple('Employee', [('name', str), ('id', int)])

<div class="versionchanged">

3.6 Added support for `526` variable annotation syntax.

</div>

<div class="versionchanged">

3.6.1 Added support for default values, methods, and docstrings.

</div>

<div class="versionchanged">

3.8 The `_field_types` and `__annotations__` attributes are now regular dictionaries instead of instances of `OrderedDict`.

</div>

<div class="versionchanged">

3.9 Removed the `_field_types` attribute in favor of the more standard `__annotations__` attribute which has the same information.

</div>

<div class="versionchanged">

3.9 `NamedTuple` is now a function rather than a class. It can still be used as a class base, as described above.

</div>

<div class="versionchanged">

3.11 Added support for generic namedtuples.

</div>

<div class="versionchanged">

3.14 Using `super` (and the `__class__` `closure variable`) in methods of `NamedTuple` subclasses is unsupported and causes a `TypeError`.

</div>

</div>

<div class="NewType(name, tp)">

Helper class to create low-overhead `distinct types <distinct>`.

A `NewType` is considered a distinct type by a type checker. At runtime, however, calling a `NewType` returns its argument unchanged.

Usage:

    UserId = NewType('UserId', int)  # Declare the NewType "UserId"
    first_user = UserId(1)  # "UserId" returns the argument unchanged at runtime

<div class="attribute">

\_\_module\_\_

The name of the module in which the new type is defined.

</div>

<div class="attribute">

\_\_name\_\_

The name of the new type.

</div>

<div class="attribute">

\_\_supertype\_\_

The type that the new type is based on.

</div>

<div class="versionadded">

3.5.2

</div>

<div class="versionchanged">

3.10 `NewType` is now a class rather than a function.

</div>

</div>

<div class="Protocol(Generic)">

Base class for protocol classes.

Protocol classes are defined like this:

    class Proto(Protocol):
        def meth(self) -> int:
            ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing), for example:

    class C:
        def meth(self) -> int:
            return 0

    def func(x: Proto) -> int:
        return x.meth()

    func(C())  # Passes static type check

See `544` for more details. Protocol classes decorated with `runtime_checkable` (described later) act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes without this decorator cannot be used as the second argument to `isinstance` or `issubclass`.

Protocol classes can be generic, for example:

    class GenProto[T](Protocol):
        def meth(self) -> T:
            ...

In code that needs to be compatible with Python 3.11 or older, generic Protocols can be written as follows:

    T = TypeVar("T")

    class GenProto(Protocol[T]):
        def meth(self) -> T:
            ...

<div class="versionadded">

3.8

</div>

<div class="deprecated-removed">

3.15 3.20 It is deprecated to call `isinstance` and `issubclass` checks on protocol classes that were not explicitly decorated with `!runtime_checkable` but that inherit from a runtime-checkable protocol class. This will throw a `TypeError` in Python 3.20.

</div>

</div>

<div class="decorator">

runtime_checkable

Mark a protocol class as a runtime protocol.

Such a protocol can be used with `isinstance` and `issubclass`. This allows a simple-minded structural check, very similar to "one-trick ponies" in `collections.abc` such as `~collections.abc.Iterable`. For example:

    @runtime_checkable
    class Closable(Protocol):
        def close(self): ...

    assert isinstance(open('/some/file'), Closable)

    @runtime_checkable
    class Named(Protocol):
        name: str

    import threading
    assert isinstance(threading.Thread(name='Bob'), Named)

Runtime checkability of protocols is not inherited. A subclass of a runtime-checkable protocol is only runtime-checkable if it is explicitly marked as such, regardless of class hierarchy:

    @runtime_checkable
    class Iterable(Protocol):
        def __iter__(self): ...

    # Without @runtime_checkable, Reversible would no longer be runtime-checkable.
    @runtime_checkable
    class Reversible(Iterable, Protocol):
        def __reversed__(self): ...

This decorator raises `TypeError` when applied to a non-protocol class.

> [!NOTE]
> `!runtime_checkable` will check only the presence of the required methods or attributes, not their type signatures or types. For example, `ssl.SSLObject` is a class, therefore it passes an `issubclass` check against `Callable <annotating-callables>`. However, the `ssl.SSLObject.__init__` method exists only to raise a `TypeError` with a more informative message, therefore making it impossible to call (instantiate) `ssl.SSLObject`.

> [!NOTE]
> An `isinstance` check against a runtime-checkable protocol can be surprisingly slow compared to an `isinstance()` check against a non-protocol class. Consider using alternative idioms such as `hasattr` calls for structural checks in performance-sensitive code.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.12 The internal implementation of `isinstance` checks against runtime-checkable protocols now uses `inspect.getattr_static` to look up attributes (previously, `hasattr` was used). As a result, some objects which used to be considered instances of a runtime-checkable protocol may no longer be considered instances of that protocol on Python 3.12+, and vice versa. Most users are unlikely to be affected by this change.

</div>

<div class="versionchanged">

3.12 The members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on `isinstance` checks comparing objects to the protocol. See `What's new in Python 3.12 <whatsnew-typing-py312>` for more details.

</div>

<div class="deprecated-removed">

3.15 3.20 It is deprecated to call `isinstance` and `issubclass` checks on protocol classes that were not explicitly decorated with `!runtime_checkable` but that inherit from a runtime-checkable protocol class. This will throw a `TypeError` in Python 3.20.

</div>

</div>

<div class="TypedDict(dict)">

Special construct to add type hints to a dictionary. At runtime "`!TypedDict` instances" are simply `dicts <dict>`.

`TypedDict` declares a dictionary type that expects all of its instances to have a certain set of keys, where each key is associated with a value of a consistent type. This expectation is not checked at runtime but is only enforced by type checkers. Usage:

    class Point2D(TypedDict):
        x: int
        y: int
        label: str

    a: Point2D = {'x': 1, 'y': 2, 'label': 'good'}  # OK
    b: Point2D = {'z': 3, 'label': 'bad'}           # Fails type check

    assert Point2D(x=1, y=2, label='first') == dict(x=1, y=2, label='first')

An alternative way to create a `TypedDict` is by using function-call syntax. The second argument must be a literal `dict`:

    Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str})

This functional syntax allows defining keys which are not valid `identifiers <identifiers>`, for example because they are keywords or contain hyphens, or when key names must not be `mangled <private-name-mangling>` like regular private names:

    # raises SyntaxError
    class Point2D(TypedDict):
        in: int  # 'in' is a keyword
        x-y: int  # name with hyphens

    class Definition(TypedDict):
        __schema: str  # mangled to `_Definition__schema`

    # OK, functional syntax
    Point2D = TypedDict('Point2D', {'in': int, 'x-y': int})
    Definition = TypedDict('Definition', {'__schema': str})  # not mangled

By default, all keys must be present in a `TypedDict`. It is possible to mark individual keys as non-required using `NotRequired`:

    class Point2D(TypedDict):
        x: int
        y: int
        label: NotRequired[str]

    # Alternative syntax
    Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': NotRequired[str]})

This means that a `Point2D` `TypedDict` can have the `label` key omitted.

It is also possible to mark all keys as non-required by default by specifying a totality of `False`:

    class Point2D(TypedDict, total=False):
        x: int
        y: int

    # Alternative syntax
    Point2D = TypedDict('Point2D', {'x': int, 'y': int}, total=False)

This means that a `Point2D` `TypedDict` can have any of the keys omitted. A type checker is only expected to support a literal `False` or `True` as the value of the `total` argument. `True` is the default, and makes all items defined in the class body required.

Individual keys of a `total=False` `TypedDict` can be marked as required using `Required`:

    class Point2D(TypedDict, total=False):
        x: Required[int]
        y: Required[int]
        label: str

    # Alternative syntax
    Point2D = TypedDict('Point2D', {
        'x': Required[int],
        'y': Required[int],
        'label': str
    }, total=False)

It is possible for a `TypedDict` type to inherit from one or more other `TypedDict` types using the class-based syntax. Usage:

    class Point3D(Point2D):
        z: int

`Point3D` has three items: `x`, `y` and `z`. It is equivalent to this definition:

    class Point3D(TypedDict):
        x: int
        y: int
        z: int

By default, a `TypedDict` is open, meaning that it may contain additional keys at runtime beyond those defined in the class body. The *closed* class argument can be used to control this; if `closed=True`, the `TypedDict` cannot contain additional keys.

    class ClosedPoint(TypedDict, closed=True):
        x: int
        y: int

    class ClosedPoint3D(ClosedPoint):  # type checker error: cannot add keys to a closed TypedDict
        z: int

Setting `closed=False` explicitly requests the default open behavior. If the argument is not passed, this state is inherited from the parent class.

In addition to being open or closed, a `TypedDict` can also be configured to have extra items. If the *extra_items* class argument is set to a type, the `TypedDict` can contain arbitrary additional keys, but the values of those keys must be of the specified type.

    class ExtraItemsPoint(TypedDict, extra_items=int):
        x: int
        y: int

    point: ExtraItemsPoint = {'x': 1, 'y': 2, 'anything': 3}  # OK

The *extra_items* argument is also inherited through subclassing. It is unset by default, and it may not be used together with the *closed* argument.

A `TypedDict` cannot inherit from a non-`TypedDict` class, except for `Generic`. For example:

    class X(TypedDict):
        x: int

    class Y(TypedDict):
        y: int

    class Z(object): pass  # A non-TypedDict class

    class XY(X, Y): pass  # OK

    class XZ(X, Z): pass  # raises TypeError

A `TypedDict` can be generic:

    class Group[T](TypedDict):
        key: T
        group: list[T]

To create a generic `TypedDict` that is compatible with Python 3.11 or lower, inherit from `Generic` explicitly:

<div class="testcode">

T = TypeVar("T")

class Group(TypedDict, Generic\[T\]):  
key: T group: list\[T\]

</div>

A `TypedDict` can be introspected via `annotationlib.get_annotations` (see `annotations-howto` for more information on annotations best practices) and the following attributes:

<div class="attribute">

\_\_total\_\_

`Point2D.__total__` gives the value of the `total` argument. Example:

<div class="doctest">

\>\>\> from typing import TypedDict \>\>\> class Point2D(TypedDict): pass \>\>\> Point2D.\_\_total\_\_ True \>\>\> class Point2D(TypedDict, total=False): pass \>\>\> Point2D.\_\_total\_\_ False \>\>\> class Point3D(Point2D): pass \>\>\> Point3D.\_\_total\_\_ True

</div>

This attribute reflects *only* the value of the `total` argument to the current `TypedDict` class, not whether the class is semantically total. For example, a `TypedDict` with `__total__` set to `True` may have keys marked with `NotRequired`, or it may inherit from another `TypedDict` with `total=False`. Therefore, it is generally better to use `__required_keys__` and `__optional_keys__` for introspection.

</div>

<div class="attribute">

\_\_required_keys\_\_

<div class="versionadded">

3.9

</div>

</div>

<div class="attribute">

\_\_optional_keys\_\_

`Point2D.__required_keys__` and `Point2D.__optional_keys__` return `frozenset` objects containing required and non-required keys, respectively.

Keys marked with `Required` will always appear in `__required_keys__` and keys marked with `NotRequired` will always appear in `__optional_keys__`.

For backwards compatibility with Python 3.10 and below, it is also possible to use inheritance to declare both required and non-required keys in the same `TypedDict`. This is done by declaring a `TypedDict` with one value for the `total` argument and then inheriting from it in another `TypedDict` with a different value for `total`:

<div class="doctest">

\>\>\> class Point2D(TypedDict, total=False): ... x: int ... y: int ... \>\>\> class Point3D(Point2D): ... z: int ... \>\>\> Point3D.\_\_required_keys\_\_ == frozenset({'z'}) True \>\>\> Point3D.\_\_optional_keys\_\_ == frozenset({'x', 'y'}) True

</div>

<div class="versionadded">

3.9

</div>

> [!NOTE]
> If `from __future__ import annotations` is used or if annotations are given as strings, annotations are not evaluated when the `TypedDict` is defined. Therefore, the runtime introspection that `__required_keys__` and `__optional_keys__` rely on may not work properly, and the values of the attributes may be incorrect.

</div>

<div class="attribute">

\_\_readonly_keys\_\_

A `frozenset` containing the names of all read-only keys. Keys are read-only if they carry the `ReadOnly` qualifier.

<div class="versionadded">

3.13

</div>

</div>

<div class="attribute">

\_\_mutable_keys\_\_

A `frozenset` containing the names of all mutable keys. Keys are mutable if they do not carry the `ReadOnly` qualifier.

<div class="versionadded">

3.13

</div>

</div>

<div class="attribute">

\_\_closed\_\_

The value of the *closed* class argument. It can be `True`, `False`, or `None`.

</div>

<div class="attribute">

\_\_extra_items\_\_

The value of the *extra_items* class argument. It can be a valid type or `NoExtraItems`.

</div>

See the [TypedDict](https://typing.python.org/en/latest/spec/typeddict.html#typeddict) section in the typing documentation for more examples and detailed rules.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.9 `TypedDict` is now a function rather than a class. It can still be used as a class base, as described above.

</div>

<div class="versionchanged">

3.11 Added support for marking individual keys as `Required` or `NotRequired`. See `655`.

</div>

<div class="versionchanged">

3.11 Added support for generic `TypedDict`s.

</div>

<div class="versionchanged">

3.13 Removed support for the keyword-argument method of creating `TypedDict`s.

</div>

<div class="versionchanged">

3.13 Support for the `ReadOnly` qualifier was added. See `705`.

</div>

<div class="versionchanged">

3.15 Support for the *closed* and *extra_items* class arguments was added. See `728`.

</div>

</div>

### Protocols

The following protocols are provided by the `!typing` module. All are decorated with `runtime_checkable`.

<div class="SupportsAbs">

A protocol with one abstract method `__abs__` that is covariant in its return type.

</div>

<div class="SupportsBytes">

A protocol with one abstract method `__bytes__`.

</div>

<div class="SupportsComplex">

A protocol with one abstract method `__complex__`.

</div>

<div class="SupportsFloat">

A protocol with one abstract method `__float__`.

</div>

<div class="SupportsIndex">

A protocol with one abstract method `__index__`.

<div class="versionadded">

3.8

</div>

</div>

<div class="SupportsInt">

A protocol with one abstract method `__int__`.

</div>

<div class="SupportsRound">

A protocol with one abstract method `__round__` that is covariant in its return type.

</div>

### ABCs and Protocols for working with I/O

<div class="IO[AnyStr] TextIO BinaryIO">

Generic class `IO[AnyStr]` and its subclasses `TextIO(IO[str])` and `BinaryIO(IO[bytes])` represent the types of I/O streams such as returned by `open`. Please note that these classes are not protocols, and their interface is fairly broad.

</div>

The protocols `io.Reader` and `io.Writer` offer a simpler alternative for argument types, when only the `read()` or `write()` methods are accessed, respectively:

    def read_and_write(reader: Reader[str], writer: Writer[bytes]):
        data = reader.read()
        writer.write(data.encode())

Also consider using `collections.abc.Iterable` for iterating over the lines of an input stream:

    def read_config(stream: Iterable[str]):
        for line in stream:
            ...

### Functions and decorators

<div class="function">

cast(typ, val)

Cast a value to a type.

This returns the value unchanged. To the type checker this signals that the return value has the designated type, but at runtime we intentionally don't check anything (we want this to be as fast as possible).

</div>

<div class="function">

assert_type(val, typ, /)

Ask a static type checker to confirm that *val* has an inferred type of *typ*.

At runtime this does nothing: it returns the first argument unchanged with no checks or side effects, no matter the actual type of the argument.

When a static type checker encounters a call to `assert_type()`, it emits an error if the value is not of the specified type:

    def greet(name: str) -> None:
        assert_type(name, str)  # OK, inferred type of `name` is `str`
        assert_type(name, int)  # type checker error

This function is useful for ensuring the type checker's understanding of a script is in line with the developer's intentions:

    def complex_function(arg: object):
        # Do some complex type-narrowing logic,
        # after which we hope the inferred type will be `int`
        ...
        # Test whether the type checker correctly understands our function
        assert_type(arg, int)

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

assert_never(arg, /)

Ask a static type checker to confirm that a line of code is unreachable.

Example:

    def int_or_str(arg: int | str) -> None:
        match arg:
            case int():
                print("It's an int")
            case str():
                print("It's a str")
            case _ as unreachable:
                assert_never(unreachable)

Here, the annotations allow the type checker to infer that the last case can never execute, because `arg` is either an `int` or a `str`, and both options are covered by earlier cases.

If a type checker finds that a call to `assert_never()` is reachable, it will emit an error. For example, if the type annotation for `arg` was instead `int | str | float`, the type checker would emit an error pointing out that `unreachable` is of type `float`. For a call to `assert_never` to pass type checking, the inferred type of the argument passed in must be the bottom type, `Never`, and nothing else.

At runtime, this throws an exception when called.

<div class="seealso">

[Unreachable Code and Exhaustiveness Checking](https://typing.python.org/en/latest/guides/unreachable.html) has more information about exhaustiveness checking with static typing.

</div>

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

reveal_type(obj, /)

Ask a static type checker to reveal the inferred type of an expression.

When a static type checker encounters a call to this function, it emits a diagnostic with the inferred type of the argument. For example:

    x: int = 1
    reveal_type(x)  # Revealed type is "builtins.int"

This can be useful when you want to debug how your type checker handles a particular piece of code.

At runtime, this function prints the runtime type of its argument to `sys.stderr` and returns the argument unchanged (allowing the call to be used within an expression):

    x = reveal_type(1)  # prints "Runtime type is int"
    print(x)  # prints "1"

Note that the runtime type may be different from (more or less specific than) the type statically inferred by a type checker.

Most type checkers support `reveal_type()` anywhere, even if the name is not imported from `typing`. Importing the name from `typing`, however, allows your code to run without runtime errors and communicates intent more clearly.

<div class="versionadded">

3.11

</div>

</div>

<div class="decorator">

dataclass_transform(*, eq_default=True, order_default=False, kw_only_default=False, frozen_default=False, field_specifiers=(),*\*kwargs)

Decorator to mark an object as providing `dataclass <dataclasses.dataclass>`-like behavior.

`@dataclass_transform` may be used to decorate a class, metaclass, or a function that is itself a decorator. The presence of `@dataclass_transform()` tells a static type checker that the decorated object performs runtime "magic" that transforms a class in a similar way to `dataclasses.dataclass`.

Example usage with a decorator function:

<div class="testcode">

@dataclass_transform() def create_model\[T\](cls: type\[T\]) -\> type\[T\]: ... return cls

@create_model class CustomerModel: id: int name: str

</div>

On a base class:

    @dataclass_transform()
    class ModelBase: ...

    class CustomerModel(ModelBase):
        id: int
        name: str

On a metaclass:

    @dataclass_transform()
    class ModelMeta(type): ...

    class ModelBase(metaclass=ModelMeta): ...

    class CustomerModel(ModelBase):
        id: int
        name: str

The `CustomerModel` classes defined above will be treated by type checkers similarly to classes created with `dataclasses.dataclass`. For example, type checkers will assume these classes have `__init__` methods that accept `id` and `name`.

The decorated class, metaclass, or function may accept the following bool arguments which type checkers will assume have the same effect as they would have on the `dataclasses.dataclass` decorator: `init`, `eq`, `order`, `unsafe_hash`, `frozen`, `match_args`, `kw_only`, and `slots`. It must be possible for the value of these arguments (`True` or `False`) to be statically evaluated.

The arguments to the `@dataclass_transform` decorator can be used to customize the default behaviors of the decorated class, metaclass, or function:

param bool eq_default  
Indicates whether the `eq` parameter is assumed to be `True` or `False` if it is omitted by the caller. Defaults to `True`.

param bool order_default  
Indicates whether the `order` parameter is assumed to be `True` or `False` if it is omitted by the caller. Defaults to `False`.

param bool kw_only_default  
Indicates whether the `kw_only` parameter is assumed to be `True` or `False` if it is omitted by the caller. Defaults to `False`.

param bool frozen_default  
Indicates whether the `frozen` parameter is assumed to be `True` or `False` if it is omitted by the caller. Defaults to `False`.

<div class="versionadded">

3.12

</div>

param field_specifiers  
Specifies a static list of supported classes or functions that describe fields, similar to `dataclasses.field`. Defaults to `()`.

type field_specifiers  
tuple\[Callable\[..., Any\], ...\]

param Any \*\*kwargs  
Arbitrary other keyword arguments are accepted in order to allow for possible future extensions.

Type checkers recognize the following optional parameters on field specifiers:

| Parameter name | Description |
|----|----|
| `init` | Indicates whether the field should be included in the synthesized `__init__` method. If unspecified, `init` defaults to `True`. |
| `default` | Provides the default value for the field. |
| `default_factory` | Provides a runtime callback that returns the default value for the field. If neither `default` nor `default_factory` are specified, the field is assumed to have no default value and must be provided a value when the class is instantiated. |
| `factory` | An alias for the `default_factory` parameter on field specifiers. |
| `kw_only` | Indicates whether the field should be marked as keyword-only. If `True`, the field will be keyword-only. If `False`, it will not be keyword-only. If unspecified, the value of the `kw_only` parameter on the object decorated with `@dataclass_transform` will be used, or if that is unspecified, the value of `kw_only_default` on `@dataclass_transform` will be used. |
| `alias` | Provides an alternative name for the field. This alternative name is used in the synthesized `__init__` method. |

**Recognised parameters for field specifiers**

At runtime, this decorator records its arguments in the `__dataclass_transform__` attribute on the decorated object. It has no other runtime effect.

See `681` for more details.

<div class="versionadded">

3.11

</div>

</div>

<div id="overload">

<div class="decorator">

overload

Decorator for creating overloaded functions and methods.

The `@overload` decorator allows describing functions and methods that support multiple different combinations of argument types. A series of `@overload`-decorated definitions must be followed by exactly one non-`@overload`-decorated definition (for the same function/method).

`@overload`-decorated definitions are for the benefit of the type checker only, since they will be overwritten by the non-`@overload`-decorated definition. The non-`@overload`-decorated definition, meanwhile, will be used at runtime but should be ignored by a type checker. At runtime, calling an `@overload`-decorated function directly will raise `NotImplementedError`.

An example of overload that gives a more precise type than can be expressed using a union or a type variable:

<div class="testcode">

@overload def process(response: None) -\> None: ... @overload def process(response: int) -\> tuple\[int, str\]: ... @overload def process(response: bytes) -\> str: ... def process(response): ... \# actual implementation goes here

</div>

See `484` for more details and comparison with other typing semantics.

<div class="versionchanged">

3.11 Overloaded functions can now be introspected at runtime using `get_overloads`.

</div>

</div>

</div>

<div class="function">

get_overloads(func)

Return a sequence of `overload`-decorated definitions for *func*.

*func* is the function object for the implementation of the overloaded function. For example, given the definition of `process` in the documentation for `overload`, `get_overloads(process)` will return a sequence of three function objects for the three defined overloads. If called on a function with no overloads, `get_overloads()` returns an empty sequence.

`get_overloads()` can be used for introspecting an overloaded function at runtime.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

clear_overloads()

Clear all registered overloads in the internal registry.

This can be used to reclaim the memory used by the registry.

<div class="versionadded">

3.11

</div>

</div>

<div class="decorator">

final

Decorator to indicate final methods and final classes.

Decorating a method with `@final` indicates to a type checker that the method cannot be overridden in a subclass. Decorating a class with `@final` indicates that it cannot be subclassed.

For example:

    class Base:
        @final
        def done(self) -> None:
            ...
    class Sub(Base):
        def done(self) -> None:  # Error reported by type checker
            ...

    @final
    class Leaf:
        ...
    class Other(Leaf):  # Error reported by type checker
        ...

There is no runtime checking of these properties. See `591` for more details.

<div class="versionadded">

3.8

</div>

<div class="versionchanged">

3.11 The decorator will now attempt to set a `__final__` attribute to `True` on the decorated object. Thus, a check like `if getattr(obj, "__final__", False)` can be used at runtime to determine whether an object `obj` has been marked as final. If the decorated object does not support setting attributes, the decorator returns the object unchanged without raising an exception.

</div>

</div>

<div class="decorator">

no_type_check

Decorator to indicate that annotations are not type hints.

This works as a class or function `decorator`. With a class, it applies recursively to all methods and classes defined in that class (but not to methods defined in its superclasses or subclasses). Type checkers will ignore all annotations in a function or class with this decorator.

`@no_type_check` mutates the decorated object in place.

</div>

<div class="decorator">

override

Decorator to indicate that a method in a subclass is intended to override a method or attribute in a superclass.

Type checkers should emit an error if a method decorated with `@override` does not, in fact, override anything. This helps prevent bugs that may occur when a base class is changed without an equivalent change to a child class.

For example:

<div class="testcode">

class Base:  
def log_status(self) -\> None:  
...

class Sub(Base):  
@override def log_status(self) -\> None: \# Okay: overrides Base.log_status ...

@override def done(self) -\> None: \# Error reported by type checker ...

</div>

There is no runtime checking of this property.

The decorator will attempt to set an `__override__` attribute to `True` on the decorated object. Thus, a check like `if getattr(obj, "__override__", False)` can be used at runtime to determine whether an object `obj` has been marked as an override. If the decorated object does not support setting attributes, the decorator returns the object unchanged without raising an exception.

See `698` for more details.

<div class="versionadded">

3.12

</div>

</div>

<div class="decorator">

disjoint_base

Decorator to mark a class as a disjoint base.

Type checkers do not allow child classes of a disjoint base `C` to inherit from other disjoint bases that are not parent or child classes of `C`.

For example:

    @disjoint_base
    class Disjoint1: pass

    @disjoint_base
    class Disjoint2: pass

    class Disjoint3(Disjoint1, Disjoint2): pass  # Type checker error

Type checkers can use knowledge of disjoint bases to detect unreachable code and determine when two types can overlap.

The corresponding runtime concept is a solid base (see `multiple-inheritance`). Classes that are solid bases at runtime can be marked with `@disjoint_base` in stub files. Users may also mark other classes as disjoint bases to indicate to type checkers that multiple inheritance with other disjoint bases should not be allowed.

Note that the concept of a solid base is a CPython implementation detail, and the exact set of standard library classes that are disjoint bases at runtime may change in future versions of Python.

<div class="versionadded">

3.15

</div>

</div>

<div class="decorator">

type_check_only

Decorator to mark a class or function as unavailable at runtime.

This decorator is itself not available at runtime. It is mainly intended to mark classes that are defined in type stub files if an implementation returns an instance of a private class:

    @type_check_only
    class Response:  # private or not available at runtime
        code: int
        def get_header(self, name: str) -> str: ...

    def fetch_response() -> Response: ...

Note that returning instances of private classes is not recommended. It is usually preferable to make such classes public.

</div>

### Introspection helpers

<div class="function">

get_type_hints(obj, globalns=None, localns=None, include_extras=False, \*, format=Format.VALUE)

Return a dictionary containing type hints for a function, method, module, class object, or other callable object.

This is often the same as `annotationlib.get_annotations`, but this function makes the following changes to the annotations dictionary:

- Forward references encoded as string literals or `ForwardRef` objects are handled by evaluating them in *globalns*, *localns*, and (where applicable) *obj*'s `type parameter <type-params>` namespace. If *globalns* or *localns* is not given, appropriate namespace dictionaries are inferred from *obj*.
- `None` is replaced with `types.NoneType`.
- If `no_type_check` has been applied to *obj*, an empty dictionary is returned.
- If *obj* is a class `C`, the function returns a dictionary that merges annotations from `C`'s base classes with those on `C` directly. This is done by traversing `C.__mro__ <type.__mro__>` and iteratively combining `annotations <variable annotation>` of each base class. Annotations on classes appearing earlier in the `method resolution order` always take precedence over annotations on classes appearing later in the method resolution order.
- The function recursively replaces all occurrences of `Annotated[T, ...]`, `Required[T]`, `NotRequired[T]`, and `ReadOnly[T]` with `T`, unless *include_extras* is set to `True` (see `Annotated` for more information).

> [!CAUTION]
> This function may execute arbitrary code contained in annotations. See `annotationlib-security` for more information.

> [!NOTE]
> If `Format.VALUE <annotationlib.Format.VALUE>` is used and any forward references in the annotations of *obj* are not resolvable, a `NameError` exception is raised. For example, this can happen with names imported under `if TYPE_CHECKING <TYPE_CHECKING>`. More generally, any kind of exception can be raised if an annotation contains invalid Python code.

> [!NOTE]
> Calling `get_type_hints` on an instance is not supported. To retrieve annotations for an instance, call `get_type_hints` on the instance's class instead (for example, `get_type_hints(type(obj))`).

<div class="versionchanged">

3.9 Added `include_extras` parameter as part of `593`. See the documentation on `Annotated` for more information.

</div>

<div class="versionchanged">

3.11 Previously, `Optional[t]` was added for function and method annotations if a default value equal to `None` was set. Now the annotation is returned unchanged.

</div>

<div class="versionchanged">

3.14 Added the `format` parameter. See the documentation on `annotationlib.get_annotations` for more information.

</div>

<div class="versionchanged">

3.14 Calling `get_type_hints` on instances is no longer supported. Some instances were accepted in earlier versions as an undocumented implementation detail.

</div>

</div>

<div class="function">

get_origin(tp)

Get the unsubscripted version of a type: for a typing object of the form `X[Y, Z, ...]` return `X`.

If `X` is a typing-module alias for a builtin or `collections` class, it will be normalized to the original class. If `X` is an instance of `ParamSpecArgs` or `ParamSpecKwargs`, return the underlying `ParamSpec`. Return `None` for unsupported objects.

Examples:

<div class="testcode">

assert get_origin(str) is None assert get_origin(Dict\[str, int\]) is dict assert get_origin(Union\[int, str\]) is Union assert get_origin(Annotated\[str, "metadata"\]) is Annotated P = ParamSpec('P') assert get_origin(P.args) is P assert get_origin(P.kwargs) is P

</div>

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

get_args(tp)

Get type arguments with all substitutions performed: for a typing object of the form `X[Y, Z, ...]` return `(Y, Z, ...)`.

If `X` is a union or `Literal` contained in another generic type, the order of `(Y, Z, ...)` may be different from the order of the original arguments `[Y, Z, ...]` due to type caching. Return `()` for unsupported objects.

Examples:

<div class="testcode">

assert get_args(int) == () assert get_args(Dict\[int, str\]) == (int, str) assert get_args(Union\[int, str\]) == (int, str)

</div>

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

get_protocol_members(tp)

Return the set of members defined in a `Protocol`.

<div class="doctest">

\>\>\> from typing import Protocol, get_protocol_members \>\>\> class P(Protocol): ... def a(self) -\> str: ... ... b: int \>\>\> get_protocol_members(P) == frozenset({'a', 'b'}) True

</div>

Raise `TypeError` for arguments that are not Protocols.

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

is_protocol(tp)

Determine if a type is a `Protocol`.

For example:

<div class="testcode">

class P(Protocol):  
def a(self) -\> str: ... b: int

assert is_protocol(P) assert not is_protocol(int)

</div>

This function only returns true for `Protocol` classes, not for `generic aliases <types-genericalias>` of them:

<div class="testcode">

class GenericP\[T\](Protocol):  
def a(self) -\> T: ... b: int

assert not is_protocol(GenericP\[int\])

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="function">

is_typeddict(tp)

Check if a type is a `TypedDict`.

For example:

<div class="testcode">

class Film(TypedDict):  
title: str year: int

assert is_typeddict(Film) assert not is_typeddict(list \| str)

\# TypedDict is a factory for creating typed dicts, \# not a typed dict itself assert not is_typeddict(TypedDict)

</div>

This function only returns true for `TypedDict` classes, not for `generic aliases <types-genericalias>` of them:

<div class="testcode">

class GenericFilm\[T\](TypedDict):  
title: str year: T

assert not is_typeddict(GenericFilm\[int\])

</div>

<div class="versionadded">

3.10

</div>

</div>

<div class="ForwardRef">

Class used for internal typing representation of string forward references.

For example, `List["SomeClass"]` is implicitly transformed into `List[ForwardRef("SomeClass")]`. `!ForwardRef` should not be instantiated by a user, but may be used by introspection tools.

> [!NOTE]
> `585` generic types such as `list["SomeClass"]` will not be implicitly transformed into `list[ForwardRef("SomeClass")]` and thus will not automatically resolve to `list[SomeClass]`.

<div class="versionadded">

3.7.4

</div>

<div class="versionchanged">

3.14 This is now an alias for `annotationlib.ForwardRef`. Several undocumented behaviors of this class have been changed; for example, after a `ForwardRef` has been evaluated, the evaluated value is no longer cached.

</div>

</div>

<div class="function">

evaluate_forward_ref(forward_ref, \*, owner=None, globals=None, locals=None, type_params=None, format=annotationlib.Format.VALUE)

Evaluate an `annotationlib.ForwardRef` as a `type hint`.

This is similar to calling `annotationlib.ForwardRef.evaluate`, but unlike that method, `!evaluate_forward_ref` also recursively evaluates forward references nested within the type hint.

See the documentation for `annotationlib.ForwardRef.evaluate` for the meaning of the *owner*, *globals*, *locals*, *type_params*, and *format* parameters.

> [!CAUTION]
> This function may execute arbitrary code contained in annotations. See `annotationlib-security` for more information.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

NoDefault

A sentinel object used to indicate that a type parameter has no default value. For example:

<div class="doctest">

\>\>\> T = TypeVar("T") \>\>\> T.\_\_default\_\_ is typing.NoDefault True \>\>\> S = TypeVar("S", default=None) \>\>\> S.\_\_default\_\_ is None True

</div>

<div class="versionadded">

3.13

</div>

</div>

<div class="data">

NoExtraItems

A `sentinel` object used to indicate that a `TypedDict` does not have the *extra_items* class argument.

<div class="doctest">

\>\>\> from typing import TypedDict, NoExtraItems \>\>\> class Point(TypedDict): ... x: int ... y: int ... \>\>\> Point.\_\_extra_items\_\_ is NoExtraItems True

</div>

</div>

### Constant

<div class="data">

TYPE_CHECKING

A special constant that is assumed to be `True` by static type checkers. It's `False` at runtime.

A module which is expensive to import, and which only contain types used for typing annotations, can be safely imported inside an `if TYPE_CHECKING:` block. This prevents the module from actually being imported at runtime; annotations aren't eagerly evaluated (see `649`) so using undefined symbols in annotations is harmless--as long as you don't later examine them. Your static type analysis tool will set `TYPE_CHECKING` to `True` during static type analysis, which means the module will be imported and the types will be checked properly during such analysis.

Usage:

    if TYPE_CHECKING:
        import expensive_mod

    def fun(arg: expensive_mod.SomeType) -> None:
        local_var: expensive_mod.AnotherType = other_fun()

If you occasionally need to examine type annotations at runtime which may contain undefined symbols, use `annotationlib.get_annotations` with a `format` parameter of `annotationlib.Format.STRING` or `annotationlib.Format.FORWARDREF` to safely retrieve the annotations without raising `NameError`.

<div class="versionadded">

3.5.2

</div>

</div>

### Deprecated aliases<span id="generic-concrete-collections"></span>

This module defines several deprecated aliases to pre-existing standard library classes. These were originally included in the `!typing` module in order to support parameterizing these generic classes using `[]`. However, the aliases became redundant in Python 3.9 when the corresponding pre-existing classes were enhanced to support `[]` (see `585`).

The redundant types are deprecated as of Python 3.9. However, while the aliases may be removed at some point, removal of these aliases is not currently planned. As such, no deprecation warnings are currently issued by the interpreter for these aliases.

If at some point it is decided to remove these deprecated aliases, a deprecation warning will be issued by the interpreter for at least two releases prior to removal. The aliases are guaranteed to remain in the `!typing` module without deprecation warnings until at least Python 3.14.

Type checkers are encouraged to flag uses of the deprecated types if the program they are checking targets a minimum Python version of 3.9 or newer.

#### Aliases to built-in types

<div class="Dict(dict, MutableMapping[KT, VT])">

Deprecated alias to `dict`.

Note that to annotate arguments, it is preferred to use an abstract collection type such as `~collections.abc.Mapping` rather than to use `dict` or `!typing.Dict`.

<div class="deprecated">

3.9 `builtins.dict <dict>` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="List(list, MutableSequence[T])">

Deprecated alias to `list`.

Note that to annotate arguments, it is preferred to use an abstract collection type such as `~collections.abc.Sequence` or `~collections.abc.Iterable` rather than to use `list` or `!typing.List`.

<div class="deprecated">

3.9 `builtins.list <list>` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Set(set, MutableSet[T])">

Deprecated alias to `builtins.set <set>`.

Note that to annotate arguments, it is preferred to use an abstract collection type such as `collections.abc.Set` rather than to use `set` or `typing.Set`.

<div class="deprecated">

3.9 `builtins.set <set>` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="FrozenSet(frozenset, AbstractSet[T_co])">

Deprecated alias to `builtins.frozenset <frozenset>`.

<div class="deprecated">

3.9 `builtins.frozenset <frozenset>` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="data">

Tuple

Deprecated alias for `tuple`.

`tuple` and `Tuple` are special-cased in the type system; see `annotating-tuples` for more details.

<div class="deprecated">

3.9 `builtins.tuple <tuple>` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Type(Generic[CT_co])">

Deprecated alias to `type`.

See `type-of-class-objects` for details on using `type` or `typing.Type` in type annotations.

<div class="versionadded">

3.5.2

</div>

<div class="deprecated">

3.9 `builtins.type <type>` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

#### Aliases to types in `collections`

<div class="DefaultDict(collections.defaultdict, MutableMapping[KT, VT])">

Deprecated alias to `collections.defaultdict`.

<div class="versionadded">

3.5.2

</div>

<div class="deprecated">

3.9 `collections.defaultdict` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="OrderedDict(collections.OrderedDict, MutableMapping[KT, VT])">

Deprecated alias to `collections.OrderedDict`.

<div class="versionadded">

3.7.2

</div>

<div class="deprecated">

3.9 `collections.OrderedDict` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="ChainMap(collections.ChainMap, MutableMapping[KT, VT])">

Deprecated alias to `collections.ChainMap`.

<div class="versionadded">

3.6.1

</div>

<div class="deprecated">

3.9 `collections.ChainMap` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Counter(collections.Counter, Dict[T, int])">

Deprecated alias to `collections.Counter`.

<div class="versionadded">

3.6.1

</div>

<div class="deprecated">

3.9 `collections.Counter` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Deque(deque, MutableSequence[T])">

Deprecated alias to `collections.deque`.

<div class="versionadded">

3.6.1

</div>

<div class="deprecated">

3.9 `collections.deque` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

#### Aliases to other concrete types

<div class="Pattern Match">

Deprecated aliases corresponding to the return types from `re.compile` and `re.search`.

These types (and the corresponding functions) are generic over `AnyStr`. `Pattern` can be specialised as `Pattern[str]` or `Pattern[bytes]`; `Match` can be specialised as `Match[str]` or `Match[bytes]`.

<div class="deprecated">

3.9 Classes `Pattern` and `Match` from `re` now support `[]`. See `585` and `types-genericalias`.

</div>

</div>

<div class="Text">

Deprecated alias for `str`.

`Text` is provided to supply a forward compatible path for Python 2 code: in Python 2, `Text` is an alias for `unicode`.

Use `Text` to indicate that a value must contain a unicode string in a manner that is compatible with both Python 2 and Python 3:

    def add_unicode_checkmark(text: Text) -> Text:
        return text + u' \u2713'

<div class="versionadded">

3.5.2

</div>

<div class="deprecated">

3.11 Python 2 is no longer supported, and most type checkers also no longer support type checking Python 2 code. Removal of the alias is not currently planned, but users are encouraged to use `str` instead of `Text`.

</div>

</div>

#### Aliases to container ABCs in `collections.abc`<span id="abstract-base-classes"></span>

<div class="AbstractSet(Collection[T_co])">

Deprecated alias to `collections.abc.Set`.

<div class="deprecated">

3.9 `collections.abc.Set` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="ByteString(Sequence[int])">

Deprecated alias to `collections.abc.ByteString`.

Use `isinstance(obj, collections.abc.Buffer)` to test if `obj` implements the `buffer protocol <bufferobjects>` at runtime. For use in type annotations, either use `~collections.abc.Buffer` or a union that explicitly specifies the types your code supports (e.g., `bytes | bytearray | memoryview`).

`!ByteString` was originally intended to be an abstract class that would serve as a supertype of both `bytes` and `bytearray`. However, since the ABC never had any methods, knowing that an object was an instance of `!ByteString` never actually told you anything useful about the object. Other common buffer types such as `memoryview` were also never understood as subtypes of `!ByteString` (either at runtime or by static type checkers).

See `PEP 688 <688#current-options>` for more details.

<div class="deprecated-removed">

3.9 3.17

</div>

</div>

<div class="Collection(Sized, Iterable[T_co], Container[T_co])">

Deprecated alias to `collections.abc.Collection`.

<div class="versionadded">

3.6

</div>

<div class="deprecated">

3.9 `collections.abc.Collection` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Container(Generic[T_co])">

Deprecated alias to `collections.abc.Container`.

<div class="deprecated">

3.9 `collections.abc.Container` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="ItemsView(MappingView, AbstractSet[tuple[KT_co, VT_co]])">

Deprecated alias to `collections.abc.ItemsView`.

<div class="deprecated">

3.9 `collections.abc.ItemsView` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="KeysView(MappingView, AbstractSet[KT_co])">

Deprecated alias to `collections.abc.KeysView`.

<div class="deprecated">

3.9 `collections.abc.KeysView` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Mapping(Collection[KT], Generic[KT, VT_co])">

Deprecated alias to `collections.abc.Mapping`.

<div class="deprecated">

3.9 `collections.abc.Mapping` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="MappingView(Sized)">

Deprecated alias to `collections.abc.MappingView`.

<div class="deprecated">

3.9 `collections.abc.MappingView` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="MutableMapping(Mapping[KT, VT])">

Deprecated alias to `collections.abc.MutableMapping`.

<div class="deprecated">

3.9 `collections.abc.MutableMapping` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="MutableSequence(Sequence[T])">

Deprecated alias to `collections.abc.MutableSequence`.

<div class="deprecated">

3.9 `collections.abc.MutableSequence` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="MutableSet(AbstractSet[T])">

Deprecated alias to `collections.abc.MutableSet`.

<div class="deprecated">

3.9 `collections.abc.MutableSet` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Sequence(Reversible[T_co], Collection[T_co])">

Deprecated alias to `collections.abc.Sequence`.

<div class="deprecated">

3.9 `collections.abc.Sequence` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="ValuesView(MappingView, Collection[_VT_co])">

Deprecated alias to `collections.abc.ValuesView`.

<div class="deprecated">

3.9 `collections.abc.ValuesView` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

#### Aliases to asynchronous ABCs in `collections.abc`

<div class="Coroutine(Awaitable[ReturnType], Generic[YieldType, SendType, ReturnType])">

Deprecated alias to `collections.abc.Coroutine`.

See `annotating-generators-and-coroutines` for details on using `collections.abc.Coroutine` and `typing.Coroutine` in type annotations.

<div class="versionadded">

3.5.3

</div>

<div class="deprecated">

3.9 `collections.abc.Coroutine` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="AsyncGenerator(AsyncIterator[YieldType], Generic[YieldType, SendType])">

Deprecated alias to `collections.abc.AsyncGenerator`.

See `annotating-generators-and-coroutines` for details on using `collections.abc.AsyncGenerator` and `typing.AsyncGenerator` in type annotations.

<div class="versionadded">

3.6.1

</div>

<div class="deprecated">

3.9 `collections.abc.AsyncGenerator` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

<div class="versionchanged">

3.13 The `SendType` parameter now has a default.

</div>

</div>

<div class="AsyncIterable(Generic[T_co])">

Deprecated alias to `collections.abc.AsyncIterable`.

<div class="versionadded">

3.5.2

</div>

<div class="deprecated">

3.9 `collections.abc.AsyncIterable` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="AsyncIterator(AsyncIterable[T_co])">

Deprecated alias to `collections.abc.AsyncIterator`.

<div class="versionadded">

3.5.2

</div>

<div class="deprecated">

3.9 `collections.abc.AsyncIterator` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Awaitable(Generic[T_co])">

Deprecated alias to `collections.abc.Awaitable`.

<div class="versionadded">

3.5.2

</div>

<div class="deprecated">

3.9 `collections.abc.Awaitable` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

#### Aliases to other ABCs in `collections.abc`

<div class="Iterable(Generic[T_co])">

Deprecated alias to `collections.abc.Iterable`.

<div class="deprecated">

3.9 `collections.abc.Iterable` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Iterator(Iterable[T_co])">

Deprecated alias to `collections.abc.Iterator`.

<div class="deprecated">

3.9 `collections.abc.Iterator` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="data">

Callable

Deprecated alias to `collections.abc.Callable`.

See `annotating-callables` for details on how to use `collections.abc.Callable` and `typing.Callable` in type annotations.

<div class="deprecated">

3.9 `collections.abc.Callable` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

<div class="versionchanged">

3.10 `Callable` now supports `ParamSpec` and `Concatenate`. See `612` for more details.

</div>

</div>

<div class="Generator(Iterator[YieldType], Generic[YieldType, SendType, ReturnType])">

Deprecated alias to `collections.abc.Generator`.

See `annotating-generators-and-coroutines` for details on using `collections.abc.Generator` and `typing.Generator` in type annotations.

<div class="deprecated">

3.9 `collections.abc.Generator` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

<div class="versionchanged">

3.13 Default values for the send and return types were added.

</div>

</div>

<div class="Hashable">

Deprecated alias to `collections.abc.Hashable`.

<div class="deprecated">

3.12 Use `collections.abc.Hashable` directly instead.

</div>

</div>

<div class="Reversible(Iterable[T_co])">

Deprecated alias to `collections.abc.Reversible`.

<div class="deprecated">

3.9 `collections.abc.Reversible` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

</div>

<div class="Sized">

Deprecated alias to `collections.abc.Sized`.

<div class="deprecated">

3.12 Use `collections.abc.Sized` directly instead.

</div>

</div>

#### Aliases to `contextlib` ABCs

<div class="ContextManager(Generic[T_co, ExitT_co])">

Deprecated alias to `contextlib.AbstractContextManager`.

The first type parameter, `T_co`, represents the type returned by the `~object.__enter__` method. The optional second type parameter, `ExitT_co`, which defaults to `bool | None`, represents the type returned by the `~object.__exit__` method.

<div class="versionadded">

3.5.4

</div>

<div class="deprecated">

3.9 `contextlib.AbstractContextManager` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

<div class="versionchanged">

3.13 Added the optional second type parameter, `ExitT_co`.

</div>

</div>

<div class="AsyncContextManager(Generic[T_co, AExitT_co])">

Deprecated alias to `contextlib.AbstractAsyncContextManager`.

The first type parameter, `T_co`, represents the type returned by the `~object.__aenter__` method. The optional second type parameter, `AExitT_co`, which defaults to `bool | None`, represents the type returned by the `~object.__aexit__` method.

<div class="versionadded">

3.6.2

</div>

<div class="deprecated">

3.9 `contextlib.AbstractAsyncContextManager` now supports subscripting (`[]`). See `585` and `types-genericalias`.

</div>

<div class="versionchanged">

3.13 Added the optional second type parameter, `AExitT_co`.

</div>

</div>

## Deprecation Timeline of Major Features

Certain features in `typing` are deprecated and may be removed in a future version of Python. The following table summarizes major deprecations for your convenience. This is subject to change, and not all deprecations are listed.

| Feature | Deprecated in | Projected removal | PEP/issue |
|----|----|----|----|
| `typing` versions of standard collections | 3.9 | Undecided (see `deprecated-aliases` for more information) | `585` |
| `typing.ByteString` | 3.9 | 3.17 | `91896` |
| `typing.Text` | 3.11 | Undecided | `92332` |
| `typing.Hashable` and `typing.Sized` | 3.12 | Undecided | `94309` |
| `typing.TypeAlias` | 3.12 | Undecided | `695` |
| `typing.AnyStr` | 3.13 | 3.18 | `105578` |

# `!unicodedata` --- Unicode Database

<div class="module" data-synopsis="Access the Unicode Database.">

unicodedata

</div>

<div class="index">

single: Unicode single: character pair: Unicode; database

</div>

------------------------------------------------------------------------

This module provides access to the Unicode Character Database (UCD) which defines character properties for all Unicode characters. The data contained in this database is compiled from the [UCD version 18.0.0](https://www.unicode.org/Public/18.0.0/ucd).

The module uses the same names and symbols as defined by Unicode Standard Annex \#44, ["Unicode Character Database"](https://www.unicode.org/reports/tr44/).

<div class="seealso">

The `unicode-howto` for more information about Unicode and how to use this module.

</div>

<table>
<tbody>
<tr>
<td colspan="2"><strong>Lookup</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">lookup(name) &lt;lookup&gt;</code></td>
<td>Look up character by name</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">name(chr) &lt;name&gt;</code></td>
<td>Return the name assigned to a character</td>
</tr>
<tr>
<td colspan="2"><strong>Numeric values</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">decimal(chr) &lt;unicodedata.decimal&gt;</code></td>
<td>Decimal value of a character</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">digit(chr) &lt;digit&gt;</code></td>
<td>Digit value of a character</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">numeric(chr) &lt;numeric&gt;</code></td>
<td>Numeric value of a character</td>
</tr>
<tr>
<td colspan="2"><strong>Properties</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">bidirectional(chr) &lt;bidirectional&gt;</code></td>
<td>Bidirectional class of a character</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">block(chr) &lt;block&gt;</code></td>
<td>Unicode block of a character</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">category(chr) &lt;category&gt;</code></td>
<td>General category of a character</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">combining(chr) &lt;combining&gt;</code></td>
<td>Canonical combining class of a character</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">decomposition(chr) &lt;decomposition&gt;</code></td>
<td>Character decomposition mapping</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">east_asian_width(chr) &lt;east_asian_width&gt;</code></td>
<td>East Asian width of a character</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">extended_pictographic(chr) &lt;extended_pictographic&gt;</code></td>
<td>Check if a character has the Extended_Pictographic property</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">grapheme_cluster_break(chr) &lt;grapheme_cluster_break&gt;</code></td>
<td>Grapheme_Cluster_Break property of a character</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">indic_conjunct_break(chr) &lt;indic_conjunct_break&gt;</code></td>
<td>Indic_Conjunct_Break property of a character</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isxidcontinue(chr) &lt;isxidcontinue&gt;</code></td>
<td>Check if a character is a valid identifier continuation</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">isxidstart(chr) &lt;isxidstart&gt;</code></td>
<td>Check if a character is a valid identifier start</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">mirrored(chr) &lt;mirrored&gt;</code></td>
<td>Mirrored property of a character</td>
</tr>
<tr>
<td colspan="2"><strong>Normalization</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">normalize(form, unistr) &lt;normalize&gt;</code></td>
<td>Return the normalized form of a string</td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">is_normalized(form, unistr) &lt;is_normalized&gt;</code></td>
<td>Check if a Unicode string is normalized</td>
</tr>
<tr>
<td colspan="2"><strong>Text segmentation</strong></td>
</tr>
<tr>
<td><code class="interpreted-text" role="func">iter_graphemes(unistr) &lt;iter_graphemes&gt;</code></td>
<td>Iterate over grapheme clusters in a string</td>
</tr>
</tbody>
</table>

<div class="function">

lookup(name, /)

Look up character by name. If a character with the given name is found, return the corresponding character. If not found, `KeyError` is raised. For example:

    >>> unicodedata.lookup('LEFT CURLY BRACKET')
    '{'

The characters returned by this function are the same as those produced by `\N` escape sequence in string literals. For example:

    >>> unicodedata.lookup('MIDDLE DOT') == '\N{MIDDLE DOT}'
    True

<div class="versionchanged">

3.3 Support for name aliases[^1] and named sequences[^2] has been added.

</div>

</div>

<div class="function">

name(chr, default=None, /)

Returns the name assigned to the character *chr* as a string. If no name is defined, *default* is returned, or, if not given, `ValueError` is raised. For example:

    >>> unicodedata.name('½')
    'VULGAR FRACTION ONE HALF'
    >>> unicodedata.name('\uFFFF', 'fallback')
    'fallback'

</div>

<div class="function">

decimal(chr, default=None, /)

Returns the decimal value assigned to the character *chr* as integer. If no such value is defined, *default* is returned, or, if not given, `ValueError` is raised. For example:

    >>> unicodedata.decimal('\N{ARABIC-INDIC DIGIT NINE}')
    9
    >>> unicodedata.decimal('\N{SUPERSCRIPT NINE}', -1)
    -1

</div>

<div class="function">

digit(chr, default=None, /)

Returns the digit value assigned to the character *chr* as integer. If no such value is defined, *default* is returned, or, if not given, `ValueError` is raised:

    >>> unicodedata.digit('\N{SUPERSCRIPT NINE}')
    9

</div>

<div class="function">

numeric(chr, default=None, /)

Returns the numeric value assigned to the character *chr* as float. If no such value is defined, *default* is returned, or, if not given, `ValueError` is raised:

    >>> unicodedata.numeric('½')
    0.5

</div>

<div class="function">

category(chr, /)

Returns the general category assigned to the character *chr* as string. General category names consist of two letters. See the [General Category Values section of the Unicode Character Database documentation](https://www.unicode.org/reports/tr44/#General_Category_Values) for a list of category codes. For example:

    >>> unicodedata.category('A')  # 'L'etter, 'u'ppercase
    'Lu'

</div>

<div class="function">

bidirectional(chr, /)

Returns the bidirectional class assigned to the character *chr* as string. If no such value is defined, an empty string is returned. See the [Bidirectional Class Values section of the Unicode Character Database](https://www.unicode.org/reports/tr44/#Bidi_Class_Values) documentation for a list of bidirectional codes. For example:

    >>> unicodedata.bidirectional('\N{ARABIC-INDIC DIGIT SEVEN}') # 'A'rabic, 'N'umber
    'AN'

</div>

<div class="function">

combining(chr, /)

Returns the canonical combining class assigned to the character *chr* as integer. Returns `0` if no combining class is defined. See the [Canonical Combining Class Values section of the Unicode Character Database](https://www.unicode.org/reports/tr44/#Canonical_Combining_Class_Values) for more information.

</div>

<div class="function">

east_asian_width(chr, /)

Returns the east asian width assigned to the character *chr* as string. For a list of widths and or more information, see the [Unicode Standard Annex \#11](https://www.unicode.org/reports/tr11/).

</div>

<div class="function">

block(chr, /)

Returns the [block](https://www.unicode.org/versions/Unicode18.0.0/core-spec/chapter-3/#G64189) assigned to the character *chr*. For example:

    >>> unicodedata.block('S')
    'Basic Latin'

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

mirrored(chr, /)

Returns the mirrored property assigned to the character *chr* as integer. Returns `1` if the character has been identified as a "mirrored" character in bidirectional text, `0` otherwise. For example:

    >>> unicodedata.mirrored('>')
    1

</div>

<div class="function">

isxidstart(chr, /)

Return `True` if *chr* is a valid identifier start per the [Unicode Standard Annex \#31](https://www.unicode.org/reports/tr31/), that is, it has the `XID_Start` property. Return `False` otherwise. For example:

    >>> unicodedata.isxidstart('S')
    True
    >>> unicodedata.isxidstart('0')
    False

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

isxidcontinue(chr, /)

Return `True` if *chr* is a valid identifier character per the [Unicode Standard Annex \#31](https://www.unicode.org/reports/tr31/), that is, it has the `XID_Continue` property. Return `False` otherwise. For example:

    >>> unicodedata.isxidcontinue('S')
    True
    >>> unicodedata.isxidcontinue(' ')
    False

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

decomposition(chr, /)

Returns the character decomposition mapping assigned to the character *chr* as string. An empty string is returned in case no such mapping is defined. For example:

    >>> unicodedata.decomposition('Ã')
    '0041 0303'

</div>

<div class="function">

grapheme_cluster_break(chr, /)

Returns the Grapheme_Cluster_Break property assigned to the character.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

indic_conjunct_break(chr, /)

Returns the Indic_Conjunct_Break property assigned to the character.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

extended_pictographic(chr, /)

Returns `True` if the character has the Extended_Pictographic property, `False` otherwise.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

normalize(form, unistr, /)

Return the normal form *form* for the Unicode string *unistr*. Valid values for *form* are 'NFC', 'NFKC', 'NFD', and 'NFKD'.

The Unicode standard defines various normalization forms of a Unicode string, based on the definition of canonical equivalence and compatibility equivalence. In Unicode, several characters can be expressed in various way. For example, the character U+00C7 (LATIN CAPITAL LETTER C WITH CEDILLA) can also be expressed as the sequence U+0043 (LATIN CAPITAL LETTER C) U+0327 (COMBINING CEDILLA).

For each character, there are two normal forms: normal form C and normal form D. Normal form D (NFD) is also known as canonical decomposition, and translates each character into its decomposed form. Normal form C (NFC) first applies a canonical decomposition, then composes pre-combined characters again.

In addition to these two forms, there are two additional normal forms based on compatibility equivalence. In Unicode, certain characters are supported which normally would be unified with other characters. For example, U+2160 (ROMAN NUMERAL ONE) is really the same thing as U+0049 (LATIN CAPITAL LETTER I). However, it is supported in Unicode for compatibility with existing character sets (for example, gb2312).

The normal form KD (NFKD) will apply the compatibility decomposition, that is, replace all compatibility characters with their equivalents. The normal form KC (NFKC) first applies the compatibility decomposition, followed by the canonical composition.

Even if two unicode strings are normalized and look the same to a human reader, if one has combining characters and the other doesn't, they may not compare equal.

</div>

<div class="function">

is_normalized(form, unistr, /)

Return whether the Unicode string *unistr* is in the normal form *form*. Valid values for *form* are 'NFC', 'NFKC', 'NFD', and 'NFKD'.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

iter_graphemes(unistr, start=0, end=sys.maxsize, /)

Returns an iterator to iterate over grapheme clusters. With optional *start*, iteration begins at that position. With optional *end*, iteration stops at that position.

Converting an emitted item to string returns a substring corresponding to the grapheme cluster. Its `start` and `end` attributes denote the start and end of the grapheme cluster.

It uses extended grapheme cluster rules defined by Unicode Standard Annex \#29, ["Unicode Text Segmentation"](https://www.unicode.org/reports/tr29/).

<div class="versionadded">

3.15

</div>

</div>

In addition, the module exposes the following constants:

<div class="data">

unidata_version

The version of the Unicode database used in this module.

</div>

<div class="data">

ucd_3_2_0

This is an object that has most of the methods of the entire module, but uses the Unicode database version 3.2 instead, for applications that require this specific version of the Unicode database (such as IDNA).

</div>

**Footnotes**

[^1]: <https://www.unicode.org/Public/18.0.0/ucd/NameAliases.txt>

[^2]: <https://www.unicode.org/Public/18.0.0/ucd/NamedSequences.txt>

# `!unittest` --- Unit testing framework

<div class="module" data-synopsis="Unit testing framework for Python.">

unittest

</div>

**Source code:** `Lib/unittest/__init__.py`

------------------------------------------------------------------------

(If you are already familiar with the basic concepts of testing, you might want to skip to `the list of assert methods <assert-methods>`.)

The `!unittest` unit testing framework was originally inspired by JUnit and has a similar flavor as major unit testing frameworks in other languages. It supports test automation, sharing of setup and shutdown code for tests, aggregation of tests into collections, and independence of the tests from the reporting framework.

To achieve this, `!unittest` supports some important concepts in an object-oriented way:

test fixture  
A `test fixture` represents the preparation needed to perform one or more tests, and any associated cleanup actions. This may involve, for example, creating temporary or proxy databases, directories, or starting a server process.

test case  
A `test case` is the individual unit of testing. It checks for a specific response to a particular set of inputs. `!unittest` provides a base class, `TestCase`, which may be used to create new test cases.

test suite  
A `test suite` is a collection of test cases, test suites, or both. It is used to aggregate tests that should be executed together.

test runner  
A `test runner` is a component which orchestrates the execution of tests and provides the outcome to the user. The runner may use a graphical interface, a textual interface, or return a special value to indicate the results of executing the tests.

<div class="seealso">

Module `doctest`  
Another test-support module with a very different flavor.

[Simple Smalltalk Testing: With Patterns](https://web.archive.org/web/20150315073817/http://www.xprogramming.com/testfram.htm)  
Kent Beck's original paper on testing frameworks using the pattern shared by `!unittest`.

[pytest](https://docs.pytest.org/)  
Third-party unittest framework with a lighter-weight syntax for writing tests. For example, `assert func(10) == 42`.

[The Python Testing Tools Taxonomy](https://wiki.python.org/moin/PythonTestingToolsTaxonomy)  
An extensive list of Python testing tools including functional testing frameworks and mock object libraries.

[Testing in Python Mailing List](http://lists.idyll.org/listinfo/testing-in-python)  
A special-interest-group for discussion of testing, and testing tools, in Python.

The script `Tools/unittestgui/unittestgui.py` in the Python source distribution is a GUI tool for test discovery and execution. This is intended largely for ease of use for those new to unit testing. For production environments it is recommended that tests be driven by a continuous integration system such as [Buildbot](https://buildbot.net/), [Jenkins](https://www.jenkins.io/), [GitHub Actions](https://github.com/features/actions), or [AppVeyor](https://www.appveyor.com/).

</div>

## Basic example

The `!unittest` module provides a rich set of tools for constructing and running tests. This section demonstrates that a small subset of the tools suffice to meet the needs of most users.

Here is a short script to test three string methods:

    import unittest

    class TestStringMethods(unittest.TestCase):

        def test_upper(self):
            self.assertEqual('foo'.upper(), 'FOO')

        def test_isupper(self):
            self.assertTrue('FOO'.isupper())
            self.assertFalse('Foo'.isupper())

        def test_split(self):
            s = 'hello world'
            self.assertEqual(s.split(), ['hello', 'world'])
            # check that s.split fails when the separator is not a string
            with self.assertRaises(TypeError):
                s.split(2)

    if __name__ == '__main__':
        unittest.main()

A test case is created by subclassing `unittest.TestCase`. The three individual tests are defined with methods whose names start with the letters `test`. This naming convention informs the test runner about which methods represent tests.

The crux of each test is a call to `~TestCase.assertEqual` to check for an expected result; `~TestCase.assertTrue` or `~TestCase.assertFalse` to verify a condition; or `~TestCase.assertRaises` to verify that a specific exception gets raised. These methods are used instead of the `assert` statement so the test runner can accumulate all test results and produce a report.

The `~TestCase.setUp` and `~TestCase.tearDown` methods allow you to define instructions that will be executed before and after each test method. They are covered in more detail in the section `organizing-tests`.

The final block shows a simple way to run the tests. `unittest.main` provides a command-line interface to the test script. When run from the command line, the above script produces an output that looks like this:

    ...
    ----------------------------------------------------------------------
    Ran 3 tests in 0.000s

    OK

Passing the `-v` option to your test script will instruct `unittest.main` to enable a higher level of verbosity, and produce the following output:

    test_isupper (__main__.TestStringMethods.test_isupper) ... ok
    test_split (__main__.TestStringMethods.test_split) ... ok
    test_upper (__main__.TestStringMethods.test_upper) ... ok

    ----------------------------------------------------------------------
    Ran 3 tests in 0.001s

    OK

The above examples show the most commonly used `!unittest` features which are sufficient to meet many everyday testing needs. The remainder of the documentation explores the full feature set from first principles.

<div class="versionchanged">

3.11 The behavior of returning a value from a test method (other than the default `None` value), is now deprecated.

</div>

## Command-Line Interface

The unittest module can be used from the command line to run tests from modules, classes or even individual test methods:

    python -m unittest test_module1 test_module2
    python -m unittest test_module.TestClass
    python -m unittest test_module.TestClass.test_method

You can pass in a list with any combination of module names, and fully qualified class or method names.

Test modules can be specified by file path as well:

    python -m unittest tests/test_something.py

This allows you to use the shell filename completion to specify the test module. The file specified must still be importable as a module. The path is converted to a module name by removing the '.py' and converting path separators into '.'. If you want to execute a test file that isn't importable as a module you should execute the file directly instead.

You can run tests with more detail (higher verbosity) by passing in the -v flag:

    python -m unittest -v test_module

Repeat it for even more detail: `-vv` reports also the individual examples of a `doctest`.

When executed without arguments `unittest-test-discovery` is started:

    python -m unittest

For a list of all the command-line options:

    python -m unittest -h

<div class="versionchanged">

3.2 In earlier versions it was only possible to run individual test methods and not modules or classes.

</div>

<div class="versionadded">

3.14 Output is colorized by default and can be `controlled using environment variables <using-on-controlling-color>`.

</div>

### Command-line options

`unittest` supports these command-line options:

<div class="program">

unittest

</div>

<div class="option">

`-b, --buffer`  

The standard output and standard error streams are buffered during the test run. Output during a passing test is discarded. Output is echoed normally on test fail or error and is added to the failure messages.

</div>

<div class="option">

`-c, --catch`  

`Control-C` during the test run waits for the current test to end and then reports all the results so far. A second `Control-C` raises the normal `KeyboardInterrupt` exception.

See [Signal Handling](#signal-handling) for the functions that provide this functionality.

</div>

<div class="option">

`-f, --failfast`  

Stop the test run on the first error or failure.

</div>

<div class="option">

`-k`  

Only run test methods and classes that match the pattern or substring. This option may be used multiple times, in which case all test cases that match any of the given patterns are included.

Patterns that contain a wildcard character (`*`) are matched against the test name using `fnmatch.fnmatchcase`; otherwise simple case-sensitive substring matching is used.

Patterns are matched against the fully qualified test method name as imported by the test loader.

For example, `-k foo` matches `foo_tests.SomeTest.test_something`, `bar_tests.SomeTest.test_foo`, but not `bar_tests.FooTest.test_something`.

</div>

<div class="option">

`--locals`  

Show local variables in tracebacks.

</div>

<div class="option">

`--durations N`  

Show the N slowest test cases (N=0 for all).

</div>

<div class="versionadded">

3.2 The command-line options `-b`, `-c` and `-f` were added.

</div>

<div class="versionadded">

3.5 The command-line option `--locals`.

</div>

<div class="versionadded">

3.7 The command-line option `-k`.

</div>

<div class="versionadded">

3.12 The command-line option `--durations`.

</div>

The command line can also be used for test discovery, for running all of the tests in a project or just a subset.

## Test Discovery

<div class="versionadded">

3.2

</div>

Unittest supports simple test discovery. In order to be compatible with test discovery, all of the test files must be `modules <tut-modules>` or `packages <tut-packages>` importable from the top-level directory of the project (this means that their filenames must be valid `identifiers
<identifiers>`).

Test discovery is implemented in `TestLoader.discover`, but can also be used from the command line. The basic command-line usage is:

    cd project_directory
    python -m unittest discover

> [!NOTE]
> As a shortcut, `python -m unittest` is the equivalent of `python -m unittest discover`. If you want to pass arguments to test discovery the `discover` sub-command must be used explicitly.

The `discover` sub-command has the following options:

<div class="program">

unittest discover

</div>

<div class="option">

`-v, --verbose`  

Verbose output. May be repeated: `-vv` reports also the individual examples of a `doctest`.

<div class="versionchanged">

3.16 The option can be repeated.

</div>

</div>

<div class="option">

`-s, --start-directory directory`  

Directory to start discovery (`.` default)

</div>

<div class="option">

`-p, --pattern pattern`  

Pattern to match test files (`test*.py` default)

</div>

<div class="option">

`-t, --top-level-directory directory`  

Top level directory of project (defaults to start directory)

</div>

The `-s`, `-p`, and `-t` options can be passed in as positional arguments in that order. The following two command lines are equivalent:

    python -m unittest discover -s project_directory -p "*_test.py"
    python -m unittest discover project_directory "*_test.py"

As well as being a path it is possible to pass a package name, for example `myproject.subpackage.test`, as the start directory. The package name you supply will then be imported and its location on the filesystem will be used as the start directory.

> [!CAUTION]
> Test discovery loads tests by importing them. Once test discovery has found all the test files from the start directory you specify it turns the paths into package names to import. For example `foo/bar/baz.py` will be imported as `foo.bar.baz`.
>
> If you have a package installed globally and attempt test discovery on a different copy of the package then the import *could* happen from the wrong place. If this happens test discovery will warn you and exit.
>
> If you supply the start directory as a package name rather than a path to a directory then discover assumes that whichever location it imports from is the location you intended, so you will not get the warning.

Test modules and packages can customize test loading and discovery by through the [load_tests protocol](#load_tests-protocol).

<div class="versionchanged">

3.4 Test discovery supports `namespace packages <namespace package>`.

</div>

<div class="versionchanged">

3.11 Test discovery dropped the `namespace packages <namespace package>` support. It has been broken since Python 3.7. Start directory and its subdirectories containing tests must be regular package that have `__init__.py` file.

If the start directory is the dotted name of the package, the ancestor packages can be namespace packages.

</div>

<div class="versionchanged">

3.14 Test discovery supports `namespace package` as start directory again. To avoid scanning directories unrelated to Python, tests are not searched in subdirectories that do not contain `__init__.py`.

</div>

## Organizing test code

The basic building blocks of unit testing are `test cases` --- single scenarios that must be set up and checked for correctness. In `!unittest`, test cases are represented by `unittest.TestCase` instances. To make your own test cases you must write subclasses of `TestCase` or use `FunctionTestCase`.

The testing code of a `TestCase` instance should be entirely self contained, such that it can be run either in isolation or in arbitrary combination with any number of other test cases.

The simplest `TestCase` subclass will simply implement a test method (i.e. a method whose name starts with `test`) in order to perform specific testing code:

    import unittest

    class DefaultWidgetSizeTestCase(unittest.TestCase):
        def test_default_widget_size(self):
            widget = Widget('The widget')
            self.assertEqual(widget.size(), (50, 50))

Note that in order to test something, we use one of the `assert\* methods <assert-methods>` provided by the `TestCase` base class. If the test fails, an exception will be raised with an explanatory message, and `!unittest` will identify the test case as a `failure`. Any other exceptions will be treated as `errors`.

Tests can be numerous, and their set-up can be repetitive. Luckily, we can factor out set-up code by implementing a method called `~TestCase.setUp`, which the testing framework will automatically call for every single test we run:

    import unittest

    class WidgetTestCase(unittest.TestCase):
        def setUp(self):
            self.widget = Widget('The widget')

        def test_default_widget_size(self):
            self.assertEqual(self.widget.size(), (50,50),
                             'incorrect default size')

        def test_widget_resize(self):
            self.widget.resize(100,150)
            self.assertEqual(self.widget.size(), (100,150),
                             'wrong size after resize')

> [!NOTE]
> The order in which the various tests will be run is determined by sorting the test method names with respect to the built-in ordering for strings.

If the `~TestCase.setUp` method raises an exception while the test is running, the framework will consider the test to have suffered an error, and the test method will not be executed.

Similarly, we can provide a `~TestCase.tearDown` method that tidies up after the test method has been run:

    import unittest

    class WidgetTestCase(unittest.TestCase):
        def setUp(self):
            self.widget = Widget('The widget')

        def tearDown(self):
            self.widget.dispose()

If `~TestCase.setUp` succeeded, `~TestCase.tearDown` will be run whether the test method succeeded or not.

Such a working environment for the testing code is called a `test fixture`. A new TestCase instance is created as a unique test fixture used to execute each individual test method. Thus `~TestCase.setUp`, `~TestCase.tearDown`, and `!TestCase.__init__` will be called once per test.

It is recommended that you use TestCase implementations to group tests together according to the features they test. `!unittest` provides a mechanism for this: the `test suite`, represented by `!unittest`'s `TestSuite` class. In most cases, calling `unittest.main` will do the right thing and collect all the module's test cases for you and execute them.

However, should you want to customize the building of your test suite, you can do it yourself:

    def suite():
        suite = unittest.TestSuite()
        suite.addTest(WidgetTestCase('test_default_widget_size'))
        suite.addTest(WidgetTestCase('test_widget_resize'))
        return suite

    if __name__ == '__main__':
        runner = unittest.TextTestRunner()
        runner.run(suite())

You can place the definitions of test cases and test suites in the same modules as the code they are to test (such as `widget.py`), but there are several advantages to placing the test code in a separate module, such as `test_widget.py`:

- The test module can be run standalone from the command line.
- The test code can more easily be separated from shipped code.
- There is less temptation to change test code to fit the code it tests without a good reason.
- Test code should be modified much less frequently than the code it tests.
- Tested code can be refactored more easily.
- Tests for modules written in C must be in separate modules anyway, so why not be consistent?
- If the testing strategy changes, there is no need to change the source code.

## Re-using old test code

Some users will find that they have existing test code that they would like to run from `!unittest`, without converting every old test function to a `TestCase` subclass.

For this reason, `!unittest` provides a `FunctionTestCase` class. This subclass of `TestCase` can be used to wrap an existing test function. Set-up and tear-down functions can also be provided.

Given the following test function:

    def testSomething():
        something = makeSomething()
        assert something.name is not None
        # ...

one can create an equivalent test case instance as follows, with optional set-up and tear-down methods:

    testcase = unittest.FunctionTestCase(testSomething,
                                         setUp=makeSomethingDB,
                                         tearDown=deleteSomethingDB)

> [!NOTE]
> Even though `FunctionTestCase` can be used to quickly convert an existing test base over to a `!unittest`-based system, this approach is not recommended. Taking the time to set up proper `TestCase` subclasses will make future test refactorings infinitely easier.

In some cases, the existing tests may have been written using the `doctest` module. If so, `doctest` provides a `~doctest.DocTestSuite` class that can automatically build `unittest.TestSuite` instances from the existing `doctest`-based tests.

## Skipping tests and expected failures

<div class="versionadded">

3.1

</div>

Unittest supports skipping individual test methods and even whole classes of tests. In addition, it supports marking a test as an "expected failure," a test that is broken and will fail, but shouldn't be counted as a failure on a `TestResult`.

Skipping a test is simply a matter of using the `skip` `decorator` or one of its conditional variants, calling `TestCase.skipTest` within a `~TestCase.setUp` or test method, or raising `SkipTest` directly.

Basic skipping looks like this:

    class MyTestCase(unittest.TestCase):

        @unittest.skip("demonstrating skipping")
        def test_nothing(self):
            self.fail("shouldn't happen")

        @unittest.skipIf(mylib.__version__ < (1, 3),
                         "not supported in this library version")
        def test_format(self):
            # Tests that work for only a certain version of the library.
            pass

        @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
        def test_windows_support(self):
            # windows specific testing code
            pass

        def test_maybe_skipped(self):
            if not external_resource_available():
                self.skipTest("external resource not available")
            # test code that depends on the external resource
            pass

This is the output of running the example above in verbose mode:

    test_format (__main__.MyTestCase.test_format) ... skipped 'not supported in this library version'
    test_nothing (__main__.MyTestCase.test_nothing) ... skipped 'demonstrating skipping'
    test_maybe_skipped (__main__.MyTestCase.test_maybe_skipped) ... skipped 'external resource not available'
    test_windows_support (__main__.MyTestCase.test_windows_support) ... skipped 'requires Windows'

    ----------------------------------------------------------------------
    Ran 4 tests in 0.005s

    OK (skipped=4)

Classes can be skipped just like methods:

    @unittest.skip("showing class skipping")
    class MySkippedTestCase(unittest.TestCase):
        def test_not_run(self):
            pass

`TestCase.setUp` can also skip the test. This is useful when a resource that needs to be set up is not available.

Expected failures use the `expectedFailure` decorator. :

    class ExpectedFailureTestCase(unittest.TestCase):
        @unittest.expectedFailure
        def test_fail(self):
            self.assertEqual(1, 0, "broken")

It's easy to roll your own skipping decorators by making a decorator that calls `skip` on the test when it wants it to be skipped. This decorator skips the test unless the passed object has a certain attribute:

    def skipUnlessHasattr(obj, attr):
        if hasattr(obj, attr):
            return lambda func: func
        return unittest.skip("{!r} doesn't have {!r}".format(obj, attr))

The following decorators and exception implement test skipping and expected failures:

<div class="decorator">

skip(reason)

Unconditionally skip the decorated test. *reason* should describe why the test is being skipped.

</div>

<div class="decorator">

skipIf(condition, reason)

Skip the decorated test if *condition* is true.

</div>

<div class="decorator">

skipUnless(condition, reason)

Skip the decorated test unless *condition* is true.

</div>

<div class="decorator">

expectedFailure

Mark the test as an expected failure or error. If the test fails or errors in the test function itself (rather than in one of the `test fixture` methods) then it will be considered a success. If the test passes, it will be considered a failure.

</div>

<div class="exception">

SkipTest(reason)

This exception is raised to skip a test.

Usually you can use `TestCase.skipTest` or one of the skipping decorators instead of raising this directly.

</div>

Skipped tests will not have `~TestCase.setUp` or `~TestCase.tearDown` run around them. Skipped classes will not have `~TestCase.setUpClass` or `~TestCase.tearDownClass` run. Skipped modules will not have `setUpModule` or `tearDownModule` run.

## Distinguishing test iterations using subtests

<div class="versionadded">

3.4

</div>

When there are very small differences among your tests, for instance some parameters, unittest allows you to distinguish them inside the body of a test method using the `~TestCase.subTest` context manager.

For example, the following test:

    class NumbersTest(unittest.TestCase):

        def test_even(self):
            """
            Test that numbers between 0 and 5 are all even.
            """
            for i in range(0, 6):
                with self.subTest(i=i):
                    self.assertEqual(i % 2, 0)

will produce the following output:

    ======================================================================
    FAIL: test_even (__main__.NumbersTest.test_even) (i=1)
    Test that numbers between 0 and 5 are all even.
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "subtests.py", line 11, in test_even
        self.assertEqual(i % 2, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: 1 != 0

    ======================================================================
    FAIL: test_even (__main__.NumbersTest.test_even) (i=3)
    Test that numbers between 0 and 5 are all even.
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "subtests.py", line 11, in test_even
        self.assertEqual(i % 2, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: 1 != 0

    ======================================================================
    FAIL: test_even (__main__.NumbersTest.test_even) (i=5)
    Test that numbers between 0 and 5 are all even.
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "subtests.py", line 11, in test_even
        self.assertEqual(i % 2, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: 1 != 0

Without using a subtest, execution would stop after the first failure, and the error would be less easy to diagnose because the value of `i` wouldn't be displayed:

    ======================================================================
    FAIL: test_even (__main__.NumbersTest.test_even)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "subtests.py", line 32, in test_even
        self.assertEqual(i % 2, 0)
    AssertionError: 1 != 0

## Classes and functions

This section describes in depth the API of `!unittest`.

### Test cases

<div class="TestCase(methodName='runTest')">

Instances of the `TestCase` class represent the logical test units in the `!unittest` universe. This class is intended to be used as a base class, with specific tests being implemented by concrete subclasses. This class implements the interface needed by the test runner to allow it to drive the tests, and methods that the test code can use to check for and report various kinds of failure.

Each instance of `TestCase` will run a single base method: the method named *methodName*. In most uses of `TestCase`, you will neither change the *methodName* nor reimplement the default `runTest()` method.

<div class="versionchanged">

3.2 `TestCase` can be instantiated successfully without providing a *methodName*. This makes it easier to experiment with `TestCase` from the interactive interpreter.

</div>

`TestCase` instances provide three groups of methods: one group used to run the test, another used by the test implementation to check conditions and report failures, and some inquiry methods allowing information about the test itself to be gathered.

Methods in the first group (running the test) are:

<div class="method">

setUp()

Method called to prepare the test fixture. This is called immediately before calling the test method; other than `AssertionError` or `SkipTest`, any exception raised by this method will be considered an error rather than a test failure. The default implementation does nothing.

</div>

<div class="method">

tearDown()

Method called immediately after the test method has been called and the result recorded. This is called even if the test method raised an exception, so the implementation in subclasses may need to be particularly careful about checking internal state. Any exception, other than `AssertionError` or `SkipTest`, raised by this method will be considered an additional error rather than a test failure (thus increasing the total number of reported errors). This method will only be called if the `setUp` succeeds, regardless of the outcome of the test method. The default implementation does nothing.

</div>

<div class="method">

setUpClass()

A class method called before tests in an individual class are run. `setUpClass` is called with the class as the only argument and must be decorated as a `classmethod`:

    @classmethod
    def setUpClass(cls):
        ...

See [Class and Module Fixtures](#class-and-module-fixtures) for more details.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

tearDownClass()

A class method called after tests in an individual class have run. `tearDownClass` is called with the class as the only argument and must be decorated as a `classmethod`:

    @classmethod
    def tearDownClass(cls):
        ...

See [Class and Module Fixtures](#class-and-module-fixtures) for more details.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

run(result=None)

Run the test, collecting the result into the `TestResult` object passed as *result*. If *result* is omitted or `None`, a temporary result object is created (by calling the `defaultTestResult` method) and used. The result object is returned to `run`'s caller.

The same effect may be had by simply calling the `TestCase` instance.

<div class="versionchanged">

3.3 Previous versions of `run` did not return the result. Neither did calling an instance.

</div>

</div>

<div class="method">

skipTest(reason)

Calling this during a test method or `setUp` skips the current test. See `unittest-skipping` for more information.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

subTest(msg=None, \*\*params)

Return a context manager which executes the enclosed code block as a subtest. *msg* and *params* are optional, arbitrary values which are displayed whenever a subtest fails, allowing you to identify them clearly.

A test case can contain any number of subtest declarations, and they can be arbitrarily nested.

See `subtests` for more information.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

debug()

Run the test without collecting the result. This allows exceptions raised by the test to be propagated to the caller, and can be used to support running tests under a debugger.

</div>

<div id="assert-methods">

The `TestCase` class provides several assert methods to check for and report failures. The following table lists the most commonly used methods (see the tables below for more assert methods):

</div>

| Method                          | Checks that            | New in |
|---------------------------------|------------------------|--------|
| `assertEqual(a, b)              
 <TestCase.assertEqual>`          | `a == b`               |        |
| `assertNotEqual(a, b)           
 <TestCase.assertNotEqual>`       | `a != b`               |        |
| `assertTrue(x)                  
 <TestCase.assertTrue>`           | `bool(x) is True`      |        |
| `assertFalse(x)                 
 <TestCase.assertFalse>`          | `bool(x) is False`     |        |
| `assertIs(a, b)                 
 <TestCase.assertIs>`             | `a is b`               | 3.1    |
| `assertIsNot(a, b)              
 <TestCase.assertIsNot>`          | `a is not b`           | 3.1    |
| `assertIsNone(x)                
 <TestCase.assertIsNone>`         | `x is None`            | 3.1    |
| `assertIsNotNone(x)             
 <TestCase.assertIsNotNone>`      | `x is not None`        | 3.1    |
| `assertIn(a, b)                 
 <TestCase.assertIn>`             | `a in b`               | 3.1    |
| `assertNotIn(a, b)              
 <TestCase.assertNotIn>`          | `a not in b`           | 3.1    |
| `assertIsInstance(a, b)         
 <TestCase.assertIsInstance>`     | `isinstance(a, b)`     | 3.2    |
| `assertNotIsInstance(a, b)      
 <TestCase.assertNotIsInstance>`  | `not isinstance(a, b)` | 3.2    |
| `assertIsSubclass(a, b)         
 <TestCase.assertIsSubclass>`     | `issubclass(a, b)`     | 3.14   |
| `assertNotIsSubclass(a, b)      
 <TestCase.assertNotIsSubclass>`  | `not issubclass(a, b)` | 3.14   |

All the assert methods accept a *msg* argument that, if specified, is used as the error message on failure (see also `longMessage`). Note that the *msg* keyword argument can be passed to `assertRaises`, `assertRaisesRegex`, `assertWarns`, `assertWarnsRegex` only when they are used as a context manager.

<div class="method">

assertEqual(first, second, msg=None)

Test that *first* and *second* are equal. If the values do not compare equal, the test will fail.

In addition, if *first* and *second* are the exact same type and one of list, tuple, dict, set, frozenset or str or any type that a subclass registers with `addTypeEqualityFunc` the type-specific equality function will be called in order to generate a more useful default error message (see also the `list of type-specific methods
<type-specific-methods>`).

<div class="versionchanged">

3.1 Added the automatic calling of type-specific equality function.

</div>

<div class="versionchanged">

3.2 `assertMultiLineEqual` added as the default type equality function for comparing strings.

</div>

</div>

<div class="method">

assertNotEqual(first, second, msg=None)

Test that *first* and *second* are not equal. If the values do compare equal, the test will fail.

</div>

<div class="method">

assertTrue(expr, msg=None) assertFalse(expr, msg=None)

Test that *expr* is true (or false).

Note that this is equivalent to `bool(expr) is True` and not to `expr is True` (use `assertIs(expr, True)` for the latter). This method should also be avoided when more specific methods are available (e.g. `assertEqual(a, b)` instead of `assertTrue(a == b)`), because they provide a better error message in case of failure.

</div>

<div class="method">

assertIs(first, second, msg=None) assertIsNot(first, second, msg=None)

Test that *first* and *second* are (or are not) the same object.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

assertIsNone(expr, msg=None) assertIsNotNone(expr, msg=None)

Test that *expr* is (or is not) `None`.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

assertIn(member, container, msg=None) assertNotIn(member, container, msg=None)

Test that *member* is (or is not) in *container*.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

assertIsInstance(obj, cls, msg=None) assertNotIsInstance(obj, cls, msg=None)

Test that *obj* is (or is not) an instance of *cls* (which can be a class or a tuple of classes, as supported by `isinstance`). To check for the exact type, use `assertIs(type(obj), cls) <assertIs>`.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

assertIsSubclass(cls, superclass, msg=None) assertNotIsSubclass(cls, superclass, msg=None)

Test that *cls* is (or is not) a subclass of *superclass* (which can be a class or a tuple of classes, as supported by `issubclass`). To check for the exact type, use `assertIs(cls, superclass) <assertIs>`.

<div class="versionadded">

3.14

</div>

</div>

It is also possible to check the production of exceptions, warnings, and log messages using the following methods:

| Method | Checks that | New in |
|----|----|----|
| `assertRaises(exc, fun, *args, **kwds)
<TestCase.assertRaises>` | `fun(*args, **kwds)` raises *exc* |  |
| `assertRaisesRegex(exc, r, fun, *args, **kwds)
<TestCase.assertRaisesRegex>` | `fun(*args, **kwds)` raises *exc* and the message matches regex *r* | 3.1 |
| `assertWarns(warn, fun, *args, **kwds)
<TestCase.assertWarns>` | `fun(*args, **kwds)` raises *warn* | 3.2 |
| `assertWarnsRegex(warn, r, fun, *args, **kwds)
<TestCase.assertWarnsRegex>` | `fun(*args, **kwds)` raises *warn* and the message matches regex *r* | 3.2 |
| `assertLogs(logger, level)
<TestCase.assertLogs>` | The `with` block logs on *logger* with minimum *level* | 3.4 |
| `assertNoLogs(logger, level)
<TestCase.assertNoLogs>` | The `with` block does not log on *logger* with minimum *level* | 3.10 |

<div class="method">

assertRaises(exception, callable, *args,kwds) assertRaises(exception,*, msg=None)

Test that an exception is raised when *callable* is called with any positional or keyword arguments that are also passed to `assertRaises`. The test passes if *exception* is raised, is an error if another exception is raised, or fails if no exception is raised. To catch any of a group of exceptions, a tuple containing the exception classes may be passed as *exception*.

If only the *exception* and possibly the *msg* arguments are given, return a context manager so that the code under test can be written inline rather than as a function:

    with self.assertRaises(SomeException):
        do_something()

When used as a context manager, `assertRaises` accepts the additional keyword argument *msg*.

The context manager will store the caught exception object in its `!exception` attribute. This can be useful if the intention is to perform additional checks on the exception raised:

    with self.assertRaises(SomeException) as cm:
        do_something()

    the_exception = cm.exception
    self.assertEqual(the_exception.error_code, 3)

<div class="versionchanged">

3.1 Added the ability to use `assertRaises` as a context manager.

</div>

<div class="versionchanged">

3.2 Added the `!exception` attribute.

</div>

<div class="versionchanged">

3.3 Added the *msg* keyword argument when used as a context manager.

</div>

</div>

<div class="method">

assertRaisesRegex(exception, regex, callable, *args,kwds) assertRaisesRegex(exception, regex,*, msg=None)

Like `assertRaises` but also tests that *regex* matches on the string representation of the raised exception. *regex* may be a regular expression object or a string containing a regular expression suitable for use by `re.search`. Examples:

    self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$",
                           int, 'XYZ')

or:

    with self.assertRaisesRegex(ValueError, 'literal'):
       int('XYZ')

<div class="versionadded">

3.1 Added under the name `assertRaisesRegexp`.

</div>

<div class="versionchanged">

3.2 Renamed to `assertRaisesRegex`.

</div>

<div class="versionchanged">

3.3 Added the *msg* keyword argument when used as a context manager.

</div>

</div>

<div class="method">

assertWarns(warning, callable, *args,kwds) assertWarns(warning,*, msg=None)

Test that a warning is triggered when *callable* is called with any positional or keyword arguments that are also passed to `assertWarns`. The test passes if *warning* is triggered and fails if it isn't. Any exception is an error. To catch any of a group of warnings, a tuple containing the warning classes may be passed as *warnings*.

If only the *warning* and possibly the *msg* arguments are given, return a context manager so that the code under test can be written inline rather than as a function:

    with self.assertWarns(SomeWarning):
        do_something()

When used as a context manager, `assertWarns` accepts the additional keyword argument *msg*.

The context manager will store the caught warning object in its `!warning` attribute, and the source line which triggered the warnings in the `!filename` and `!lineno` attributes. This can be useful if the intention is to perform additional checks on the warning caught:

    with self.assertWarns(SomeWarning) as cm:
        do_something()

    self.assertIn('myfile.py', cm.filename)
    self.assertEqual(320, cm.lineno)

The context managers can be nested to test that multiple different warnings are emitted:

    with (self.assertWarns(SomeWarning),
          self.assertWarns(OtherWarning)):
        do_something()

This method works regardless of the warning filters in place when it is called.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Added the *msg* keyword argument when used as a context manager.

</div>

<div class="versionchanged">

3.15 Warnings that do not match the specified category are no longer swallowed. Nested context managers are now supported.

</div>

</div>

<div class="method">

assertWarnsRegex(warning, regex, callable, *args,kwds) assertWarnsRegex(warning, regex,*, msg=None)

Like `assertWarns` but also tests that *regex* matches on the message of the triggered warning. *regex* may be a regular expression object or a string containing a regular expression suitable for use by `re.search`. Example:

    self.assertWarnsRegex(DeprecationWarning,
                          r'legacy_function\(\) is deprecated',
                          legacy_function, 'XYZ')

or:

    with self.assertWarnsRegex(RuntimeWarning, 'unsafe frobnicating'):
        frobnicate('/etc/passwd')

The context managers can be nested to test that multiple different warnings are emitted:

    with (self.assertWarns(SomeWarning, regex1),
          self.assertWarns(OtherWarning, regex2)):
        do_something()

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 Added the *msg* keyword argument when used as a context manager.

</div>

<div class="versionchanged">

3.15 Warnings that do not match the specified category or regex are no longer swallowed. Nested context managers are now supported.

</div>

</div>

<div class="method">

assertLogs(logger=None, level=None, formatter=None)

A context manager to test that at least one message is logged on the *logger* or one of its children, with at least the given *level*.

If given, *logger* should be a `logging.Logger` object or a `str` giving the name of a logger. The default is the root logger, which will catch all messages that were not blocked by a non-propagating descendent logger.

If given, *level* should be either a numeric logging level or its string equivalent (for example either `"ERROR"` or `logging.ERROR`). The default is `logging.INFO`.

If given, *formatter* should be a `logging.Formatter` object. The default is a formatter with format string `"%(levelname)s:%(name)s:%(message)s"`

The test passes if at least one message emitted inside the `with` block matches the *logger* and *level* conditions, otherwise it fails.

The object returned by the context manager is a recording helper which keeps tracks of the matching log messages. It has two attributes:

<div class="attribute">

records

A list of `logging.LogRecord` objects of the matching log messages.

</div>

<div class="attribute">

output

A list of `str` objects with the formatted output of matching messages.

</div>

Example:

    with self.assertLogs('foo', level='INFO') as cm:
        logging.getLogger('foo').info('first message')
        logging.getLogger('foo.bar').error('second message')
    self.assertEqual(cm.output, ['INFO:foo:first message',
                                 'ERROR:foo.bar:second message'])

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.15 Now accepts a *formatter* to control how messages are formatted.

</div>

</div>

<div class="method">

assertNoLogs(logger=None, level=None)

A context manager to test that no messages are logged on the *logger* or one of its children, with at least the given *level*.

If given, *logger* should be a `logging.Logger` object or a `str` giving the name of a logger. The default is the root logger, which will catch all messages.

If given, *level* should be either a numeric logging level or its string equivalent (for example either `"ERROR"` or `logging.ERROR`). The default is `logging.INFO`.

Unlike `assertLogs`, nothing will be returned by the context manager.

<div class="versionadded">

3.10

</div>

</div>

There are also other methods used to perform more specific checks, such as:

| Method | Checks that | New in |
|----|----|----|
| `assertAlmostEqual(a, b)
<TestCase.assertAlmostEqual>` | `round(a-b, 7) == 0` |  |
| `assertNotAlmostEqual(a, b)
<TestCase.assertNotAlmostEqual>` | `round(a-b, 7) != 0` |  |
| `assertGreater(a, b)
<TestCase.assertGreater>` | `a > b` | 3.1 |
| `assertGreaterEqual(a, b)
<TestCase.assertGreaterEqual>` | `a >= b` | 3.1 |
| `assertLess(a, b)
<TestCase.assertLess>` | `a < b` | 3.1 |
| `assertLessEqual(a, b)
<TestCase.assertLessEqual>` | `a <= b` | 3.1 |
| `assertRegex(s, r)
<TestCase.assertRegex>` | `r.search(s)` | 3.1 |
| `assertNotRegex(s, r)
<TestCase.assertNotRegex>` | `not r.search(s)` | 3.2 |
| `assertCountEqual(a, b)
<TestCase.assertCountEqual>` | *a* contains the same elements as *b*, regardless of their order. | 3.2 |
| `assertStartsWith(a, b)
<TestCase.assertStartsWith>` | `a.startswith(b)` | 3.14 |
| `assertNotStartsWith(a, b)
<TestCase.assertNotStartsWith>` | `not a.startswith(b)` | 3.14 |
| `assertEndsWith(a, b)
<TestCase.assertEndsWith>` | `a.endswith(b)` | 3.14 |
| `assertNotEndsWith(a, b)
<TestCase.assertNotEndsWith>` | `not a.endswith(b)` | 3.14 |
| `assertHasAttr(a, b)
<TestCase.assertHasAttr>` | `hasattr(a, b)` | 3.14 |
| `assertNotHasAttr(a, b)
<TestCase.assertNotHasAttr>` | `not hasattr(a, b)` | 3.14 |

<div class="method">

assertAlmostEqual(first, second, places=7, msg=None, delta=None) assertNotAlmostEqual(first, second, places=7, msg=None, delta=None)

Test that *first* and *second* are approximately (or not approximately) equal by computing the difference, rounding to the given number of decimal *places* (default 7), and comparing to zero. Note that these methods round the values to the given number of *decimal places* (i.e. like the `round` function) and not *significant digits*.

If *delta* is supplied instead of *places* then the difference between *first* and *second* must be less or equal to (or greater than) *delta*.

Supplying both *delta* and *places* raises a `TypeError`.

<div class="versionchanged">

3.2 `assertAlmostEqual` automatically considers almost equal objects that compare equal. `assertNotAlmostEqual` automatically fails if the objects compare equal. Added the *delta* keyword argument.

</div>

</div>

<div class="method">

assertGreater(first, second, msg=None) assertGreaterEqual(first, second, msg=None) assertLess(first, second, msg=None) assertLessEqual(first, second, msg=None)

Test that *first* is respectively \>, \>=, \< or \<= than *second* depending on the method name. If not, the test will fail:

    >>> self.assertGreaterEqual(3, 4)
    AssertionError: "3" unexpectedly not greater than or equal to "4"

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

assertRegex(text, regex, msg=None) assertNotRegex(text, regex, msg=None)

Test that a *regex* search matches (or does not match) *text*. In case of failure, the error message will include the pattern and the *text* (or the pattern and the part of *text* that unexpectedly matched). *regex* may be a regular expression object or a string containing a regular expression suitable for use by `re.search`.

<div class="versionadded">

3.1 Added under the name `assertRegexpMatches`.

</div>

<div class="versionchanged">

3.2 The method `assertRegexpMatches()` has been renamed to `.assertRegex`.

</div>

<div class="versionadded">

3.2 `.assertNotRegex`.

</div>

</div>

<div class="method">

assertCountEqual(first, second, msg=None)

Test that sequence *first* contains the same elements as *second*, regardless of their order. When they don't, an error message listing the differences between the sequences will be generated.

Duplicate elements are *not* ignored when comparing *first* and *second*. It verifies whether each element has the same count in both sequences. Equivalent to: `assertEqual(Counter(list(first)), Counter(list(second)))` but works with sequences of unhashable objects as well.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

assertStartsWith(s, prefix, msg=None)

</div>

<div class="method">

assertNotStartsWith(s, prefix, msg=None)

Test that the Unicode or byte string *s* starts (or does not start) with a *prefix*. *prefix* can also be a tuple of strings to try.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

assertEndsWith(s, suffix, msg=None)

</div>

<div class="method">

assertNotEndsWith(s, suffix, msg=None)

Test that the Unicode or byte string *s* ends (or does not end) with a *suffix*. *suffix* can also be a tuple of strings to try.

<div class="versionadded">

3.14

</div>

</div>

<div class="method">

assertHasAttr(obj, name, msg=None)

</div>

<div class="method">

assertNotHasAttr(obj, name, msg=None)

Test that the object *obj* has (or has not) an attribute *name*.

<div class="versionadded">

3.14

</div>

</div>

<div id="type-specific-methods">

The `assertEqual` method dispatches the equality check for objects of the same type to different type-specific methods. These methods are already implemented for most of the built-in types, but it's also possible to register new methods using `addTypeEqualityFunc`:

</div>

<div class="method">

addTypeEqualityFunc(typeobj, function)

Registers a type-specific method called by `assertEqual` to check if two objects of exactly the same *typeobj* (not subclasses) compare equal. *function* must take two positional arguments and a third msg=None keyword argument just as `assertEqual` does. It must raise `self.failureException(msg) <failureException>` when inequality between the first two parameters is detected -- possibly providing useful information and explaining the inequalities in details in the error message.

<div class="versionadded">

3.1

</div>

</div>

The list of type-specific methods automatically used by `~TestCase.assertEqual` are summarized in the following table. Note that it's usually not necessary to invoke these methods directly.

| Method                           | Used to compare    | New in |
|----------------------------------|--------------------|--------|
| `assertMultiLineEqual(a, b)      
 <TestCase.assertMultiLineEqual>`  | strings            | 3.1    |
| `assertSequenceEqual(a, b)       
 <TestCase.assertSequenceEqual>`   | sequences          | 3.1    |
| `assertListEqual(a, b)           
 <TestCase.assertListEqual>`       | lists              | 3.1    |
| `assertTupleEqual(a, b)          
 <TestCase.assertTupleEqual>`      | tuples             | 3.1    |
| `assertSetEqual(a, b)            
 <TestCase.assertSetEqual>`        | sets or frozensets | 3.1    |
| `assertDictEqual(a, b)           
 <TestCase.assertDictEqual>`       | dicts              | 3.1    |

<div class="method">

assertMultiLineEqual(first, second, msg=None)

Test that the multiline string *first* is equal to the string *second*. When not equal a diff of the two strings highlighting the differences will be included in the error message. This method is used by default when comparing strings with `assertEqual`.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

assertSequenceEqual(first, second, msg=None, seq_type=None)

Tests that two sequences are equal. If a *seq_type* is supplied, both *first* and *second* must be instances of *seq_type* or a failure will be raised. If the sequences are different an error message is constructed that shows the difference between the two.

This method is not called directly by `assertEqual`, but it's used to implement `assertListEqual` and `assertTupleEqual`.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

assertListEqual(first, second, msg=None) assertTupleEqual(first, second, msg=None)

Tests that two lists or tuples are equal. If not, an error message is constructed that shows only the differences between the two. An error is also raised if either of the parameters are of the wrong type. These methods are used by default when comparing lists or tuples with `assertEqual`.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

assertSetEqual(first, second, msg=None)

Tests that two sets are equal. If not, an error message is constructed that lists the differences between the sets. This method is used by default when comparing sets or frozensets with `assertEqual`.

Fails if either of *first* or *second* does not have a `~frozenset.difference` method.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

assertDictEqual(first, second, msg=None)

Test that two dictionaries are equal. If not, an error message is constructed that shows the differences in the dictionaries. This method will be used by default to compare dictionaries in calls to `assertEqual`.

<div class="versionadded">

3.1

</div>

</div>

<div id="other-methods-and-attrs">

Finally the `TestCase` provides the following methods and attributes:

</div>

<div class="method">

fail(msg=None)

Signals a test failure unconditionally, with *msg* or `None` for the error message.

</div>

<div class="attribute">

failureException

This class attribute gives the exception raised by the test method. If a test framework needs to use a specialized exception, possibly to carry additional information, it must subclass this exception in order to "play fair" with the framework. The initial value of this attribute is `AssertionError`.

</div>

<div class="attribute">

longMessage

This class attribute determines what happens when a custom failure message is passed as the msg argument to an assertXYY call that fails. `True` is the default value. In this case, the custom message is appended to the end of the standard failure message. When set to `False`, the custom message replaces the standard message.

The class setting can be overridden in individual test methods by assigning an instance attribute, self.longMessage, to `True` or `False` before calling the assert methods.

The class setting gets reset before each test call.

<div class="versionadded">

3.1

</div>

</div>

<div class="attribute">

maxDiff

This attribute controls the maximum length of diffs output by assert methods that report diffs on failure. It defaults to 80\*8 characters. Assert methods affected by this attribute are `assertSequenceEqual` (including all the sequence comparison methods that delegate to it), `assertDictEqual` and `assertMultiLineEqual`.

Setting `maxDiff` to `None` means that there is no maximum length of diffs.

<div class="versionadded">

3.2

</div>

</div>

Testing frameworks can use the following methods to collect information on the test:

<div class="method">

countTestCases()

Return the number of tests represented by this test object. For `TestCase` instances, this will always be `1`.

</div>

<div class="method">

defaultTestResult()

Return an instance of the test result class that should be used for this test case class (if no other result instance is provided to the `run` method).

For `TestCase` instances, this will always be an instance of `TestResult`; subclasses of `TestCase` should override this as necessary.

</div>

<div class="method">

id()

Return a string identifying the specific test case. This is usually the full name of the test method, including the module and class name.

</div>

<div class="method">

shortDescription()

Returns a description of the test, or `None` if no description has been provided. The default implementation of this method returns the first line of the test method's docstring, if available, or `None`.

<div class="versionchanged">

3.1 In 3.1 this was changed to add the test name to the short description even in the presence of a docstring. This caused compatibility issues with unittest extensions and adding the test name was moved to the `TextTestResult` in Python 3.2.

</div>

</div>

<div class="method">

addCleanup(function, /, *args,*\*kwargs)

Add a function to be called after `tearDown` to cleanup resources used during the test. Functions will be called in reverse order to the order they are added (`LIFO (last-in, first-out)`). They are called with any arguments and keyword arguments passed into `addCleanup` when they are added.

If `setUp` fails, meaning that `tearDown` is not called, then any cleanup functions added will still be called.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

enterContext(cm)

Enter the supplied `context manager`. If successful, also add its `~object.__exit__` method as a cleanup function by `addCleanup` and return the result of the `~object.__enter__` method.

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

doCleanups()

This method is called unconditionally after `tearDown`, or after `setUp` if `setUp` raises an exception.

It is responsible for calling all the cleanup functions added by `addCleanup`. If you need cleanup functions to be called *prior* to `tearDown` then you can call `doCleanups` yourself.

`doCleanups` pops methods off the stack of cleanup functions one at a time, so it can be called at any time.

<div class="versionadded">

3.1

</div>

</div>

<div class="classmethod">

addClassCleanup(function, /, *args,*\*kwargs)

Add a function to be called after `tearDownClass` to cleanup resources used during the test class. Functions will be called in reverse order to the order they are added (`LIFO (last-in, first-out)`). They are called with any arguments and keyword arguments passed into `addClassCleanup` when they are added.

If `setUpClass` fails, meaning that `tearDownClass` is not called, then any cleanup functions added will still be called.

<div class="versionadded">

3.8

</div>

</div>

<div class="classmethod">

enterClassContext(cm)

Enter the supplied `context manager`. If successful, also add its `~object.__exit__` method as a cleanup function by `addClassCleanup` and return the result of the `~object.__enter__` method.

<div class="versionadded">

3.11

</div>

</div>

<div class="classmethod">

doClassCleanups()

This method is called unconditionally after `tearDownClass`, or after `setUpClass` if `setUpClass` raises an exception.

It is responsible for calling all the cleanup functions added by `addClassCleanup`. If you need cleanup functions to be called *prior* to `tearDownClass` then you can call `doClassCleanups` yourself.

`doClassCleanups` pops methods off the stack of cleanup functions one at a time, so it can be called at any time.

<div class="versionadded">

3.8

</div>

</div>

</div>

<div class="IsolatedAsyncioTestCase(methodName='runTest')">

This class provides an API similar to `TestCase` and also accepts coroutines as test functions.

<div class="versionadded">

3.8

</div>

<div class="attribute">

loop_factory

The *loop_factory* passed to `asyncio.Runner`. Override in subclasses with `asyncio.EventLoop` to avoid using the asyncio policy system.

<div class="versionadded">

3.13

</div>

</div>

<div class="method" async="">

asyncSetUp()

Method called to prepare the test fixture. This is called after `TestCase.setUp`. This is called immediately before calling the test method; other than `AssertionError` or `SkipTest`, any exception raised by this method will be considered an error rather than a test failure. The default implementation does nothing.

</div>

<div class="method" async="">

asyncTearDown()

Method called immediately after the test method has been called and the result recorded. This is called before `~TestCase.tearDown`. This is called even if the test method raised an exception, so the implementation in subclasses may need to be particularly careful about checking internal state. Any exception, other than `AssertionError` or `SkipTest`, raised by this method will be considered an additional error rather than a test failure (thus increasing the total number of reported errors). This method will only be called if the `asyncSetUp` succeeds, regardless of the outcome of the test method. The default implementation does nothing.

</div>

<div class="method">

addAsyncCleanup(function, /, *args,*\*kwargs)

This method accepts a coroutine that can be used as a cleanup function.

</div>

<div class="method" async="">

enterAsyncContext(cm)

Enter the supplied `asynchronous context manager`. If successful, also add its `~object.__aexit__` method as a cleanup function by `addAsyncCleanup` and return the result of the `~object.__aenter__` method.

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

run(result=None)

Sets up a new event loop to run the test, collecting the result into the `TestResult` object passed as *result*. If *result* is omitted or `None`, a temporary result object is created (by calling the `~TestCase.defaultTestResult` method) and used. The result object is returned to `run`'s caller. At the end of the test all the tasks in the event loop are cancelled.

</div>

An example illustrating the order:

    from unittest import IsolatedAsyncioTestCase

    events = []


    class Test(IsolatedAsyncioTestCase):


        def setUp(self):
            events.append("setUp")

        async def asyncSetUp(self):
            self._async_connection = await AsyncConnection()
            events.append("asyncSetUp")

        async def test_response(self):
            events.append("test_response")
            response = await self._async_connection.get("https://example.com")
            self.assertEqual(response.status_code, 200)
            self.addAsyncCleanup(self.on_cleanup)

        def tearDown(self):
            events.append("tearDown")

        async def asyncTearDown(self):
            await self._async_connection.close()
            events.append("asyncTearDown")

        async def on_cleanup(self):
            events.append("cleanup")

    if __name__ == "__main__":
        unittest.main()

After running the test, `events` would contain `["setUp", "asyncSetUp", "test_response", "asyncTearDown", "tearDown", "cleanup"]`.

</div>

<div class="FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None)">

This class implements the portion of the `TestCase` interface which allows the test runner to drive the test, but does not provide the methods which test code can use to check and report errors. This is used to create test cases using legacy test code, allowing it to be integrated into a `!unittest`-based test framework.

</div>

### Grouping tests

<div class="TestSuite(tests=())">

This class represents an aggregation of individual test cases and test suites. The class presents the interface needed by the test runner to allow it to be run as any other test case. Running a `TestSuite` instance is the same as iterating over the suite, running each test individually.

If *tests* is given, it must be an iterable of individual test cases or other test suites that will be used to build the suite initially. Additional methods are provided to add test cases and suites to the collection later on.

`TestSuite` objects behave much like `TestCase` objects, except they do not actually implement a test. Instead, they are used to aggregate tests into groups of tests that should be run together. Some additional methods are available to add tests to `TestSuite` instances:

<div class="method">

TestSuite.addTest(test)

Add a `TestCase` or `TestSuite` to the suite.

</div>

<div class="method">

TestSuite.addTests(tests)

Add all the tests from an iterable of `TestCase` and `TestSuite` instances to this test suite.

This is equivalent to iterating over *tests*, calling `addTest` for each element.

</div>

`TestSuite` shares the following methods with `TestCase`:

<div class="method">

run(result)

Run the tests associated with this suite, collecting the result into the test result object passed as *result*. Note that unlike `TestCase.run`, `TestSuite.run` requires the result object to be passed in.

</div>

<div class="method">

debug()

Run the tests associated with this suite without collecting the result. This allows exceptions raised by the test to be propagated to the caller and can be used to support running tests under a debugger.

</div>

<div class="method">

countTestCases()

Return the number of tests represented by this test object, including all individual tests and sub-suites.

</div>

<div class="method">

\_\_iter\_\_()

Tests grouped by a `TestSuite` are always accessed by iteration. Subclasses can lazily provide tests by overriding `!__iter__`. Note that this method may be called several times on a single suite (for example when counting tests or comparing for equality) so the tests returned by repeated iterations before `TestSuite.run` must be the same for each call iteration. After `TestSuite.run`, callers should not rely on the tests returned by this method unless the caller uses a subclass that overrides `!TestSuite._removeTestAtIndex` to preserve test references.

<div class="versionchanged">

3.2 In earlier versions the `TestSuite` accessed tests directly rather than through iteration, so overriding `!__iter__` wasn't sufficient for providing tests.

</div>

<div class="versionchanged">

3.4 In earlier versions the `TestSuite` held references to each `TestCase` after `TestSuite.run`. Subclasses can restore that behavior by overriding `!TestSuite._removeTestAtIndex`.

</div>

</div>

In the typical usage of a `TestSuite` object, the `run` method is invoked by a `!TestRunner` rather than by the end-user test harness.

</div>

### Loading and running tests

<div class="TestLoader()">

The `TestLoader` class is used to create test suites from classes and modules. Normally, there is no need to create an instance of this class; the `!unittest` module provides an instance that can be shared as `unittest.defaultTestLoader`. Using a subclass or instance, however, allows customization of some configurable properties.

`TestLoader` objects have the following attributes:

<div class="attribute">

errors

A list of the non-fatal errors encountered while loading tests. Not reset by the loader at any point. Fatal errors are signalled by the relevant method raising an exception to the caller. Non-fatal errors are also indicated by a synthetic test that will raise the original error when run.

<div class="versionadded">

3.5

</div>

</div>

`TestLoader` objects have the following methods:

<div class="method">

loadTestsFromTestCase(testCaseClass)

Return a suite of all test cases contained in the `TestCase`-derived `!testCaseClass`.

A test case instance is created for each method named by `getTestCaseNames`. By default these are the method names beginning with `test`. If `getTestCaseNames` returns no methods, but the `!runTest` method is implemented, a single test case is created for that method instead.

</div>

<div class="method">

loadTestsFromModule(module, \*, pattern=None)

Return a suite of all test cases contained in the given module. This method searches *module* for classes derived from `TestCase` and creates an instance of the class for each test method defined for the class.

> [!NOTE]
> While using a hierarchy of `TestCase`-derived classes can be convenient in sharing fixtures and helper functions, defining test methods on base classes that are not intended to be instantiated directly does not play well with this method. Doing so, however, can be useful when the fixtures are different and defined in subclasses.

If a module provides a `load_tests` function it will be called to load the tests. This allows modules to customize test loading. This is the [load_tests protocol](#load_tests-protocol). The *pattern* argument is passed as the third argument to `load_tests`.

<div class="versionchanged">

3.2 Support for `load_tests` added.

</div>

<div class="versionchanged">

3.5 Support for a keyword-only argument *pattern* has been added.

</div>

<div class="versionchanged">

3.12 The undocumented and unofficial *use_load_tests* parameter has been removed.

</div>

</div>

<div class="method">

loadTestsFromName(name, module=None)

Return a suite of all test cases given a string specifier.

The specifier *name* is a "dotted name" that may resolve either to a module, a test case class, a test method within a test case class, a `TestSuite` instance, or a callable object which returns a `TestCase` or `TestSuite` instance. These checks are applied in the order listed here; that is, a method on a possible test case class will be picked up as "a test method within a test case class", rather than "a callable object".

For example, if you have a module `!SampleTests` containing a `TestCase`-derived class `!SampleTestCase` with three test methods (`!test_one`, `!test_two`, and `!test_three`), the specifier `'SampleTests.SampleTestCase'` would cause this method to return a suite which will run all three test methods. Using the specifier `'SampleTests.SampleTestCase.test_two'` would cause it to return a test suite which will run only the `!test_two` test method. The specifier can refer to modules and packages which have not been imported; they will be imported as a side-effect.

The method optionally resolves *name* relative to the given *module*.

<div class="versionchanged">

3.5 If an `ImportError` or `AttributeError` occurs while traversing *name* then a synthetic test that raises that error when run will be returned. These errors are included in the errors accumulated by self.errors.

</div>

</div>

<div class="method">

loadTestsFromNames(names, module=None)

Similar to `loadTestsFromName`, but takes a sequence of names rather than a single name. The return value is a test suite which supports all the tests defined for each name.

</div>

<div class="method">

getTestCaseNames(testCaseClass)

Return a sorted sequence of method names found within *testCaseClass*; this should be a subclass of `TestCase`.

</div>

<div class="method">

discover(start_dir, pattern='test\*.py', top_level_dir=None)

Find all the test modules by recursing into subdirectories from the specified start directory, and return a TestSuite object containing them. Only test files that match *pattern* will be loaded. (Using shell style pattern matching.) Only module names that are importable (i.e. are valid Python identifiers) will be loaded.

All test modules must be importable from the top level of the project. If the start directory is not the top level directory then *top_level_dir* must be specified separately.

If importing a module fails, for example due to a syntax error, then this will be recorded as a single error and discovery will continue. If the import failure is due to `SkipTest` being raised, it will be recorded as a skip instead of an error.

If a package (a directory containing a file named `__init__.py`) is found, the package will be checked for a `load_tests` function. If this exists then it will be called `package.load_tests(loader, tests, pattern)`. Test discovery takes care to ensure that a package is only checked for tests once during an invocation, even if the load_tests function itself calls `loader.discover`.

If `load_tests` exists then discovery does *not* recurse into the package, `load_tests` is responsible for loading all tests in the package.

The pattern is deliberately not stored as a loader attribute so that packages can continue discovery themselves.

*top_level_dir* is stored internally, and used as a default to any nested calls to `discover()`. That is, if a package's `load_tests` calls `loader.discover()`, it does not need to pass this argument.

*start_dir* can be a dotted module name as well as a directory.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.4 Modules that raise `SkipTest` on import are recorded as skips, not errors.

*start_dir* can be a `namespace packages <namespace package>`.

Paths are sorted before being imported so that execution order is the same even if the underlying file system's ordering is not dependent on file name.

</div>

<div class="versionchanged">

3.5 Found packages are now checked for `load_tests` regardless of whether their path matches *pattern*, because it is impossible for a package name to match the default pattern.

</div>

<div class="versionchanged">

3.11 *start_dir* can not be a `namespace packages <namespace package>`. It has been broken since Python 3.7, and Python 3.11 officially removes it.

</div>

<div class="versionchanged">

3.13 *top_level_dir* is only stored for the duration of *discover* call.

</div>

<div class="versionchanged">

3.14 *start_dir* can once again be a `namespace package`.

</div>

</div>

The following attributes of a `TestLoader` can be configured either by subclassing or assignment on an instance:

<div class="attribute">

testMethodPrefix

String giving the prefix of method names which will be interpreted as test methods. The default value is `'test'`.

This affects `getTestCaseNames` and all the `loadTestsFrom*` methods.

</div>

<div class="attribute">

sortTestMethodsUsing

Function to be used to compare method names when sorting them in `getTestCaseNames` and all the `loadTestsFrom*` methods.

</div>

<div class="attribute">

suiteClass

Callable object that constructs a test suite from a list of tests. No methods on the resulting object are needed. The default value is the `TestSuite` class.

This affects all the `loadTestsFrom*` methods.

</div>

<div class="attribute">

testNamePatterns

List of Unix shell-style wildcard test name patterns that test methods have to match to be included in test suites (see `-k` option).

If this attribute is not `None` (the default), all test methods to be included in test suites must match one of the patterns in this list. Note that matches are always performed using `fnmatch.fnmatchcase`, so unlike patterns passed to the `-k` option, simple substring patterns will have to be converted using `*` wildcards.

This affects all the `loadTestsFrom*` methods.

<div class="versionadded">

3.7

</div>

</div>

</div>

<div class="TestResult">

This class is used to compile information about which tests have succeeded and which have failed.

A `TestResult` object stores the results of a set of tests. The `TestCase` and `TestSuite` classes ensure that results are properly recorded; test authors do not need to worry about recording the outcome of tests.

Testing frameworks built on top of `!unittest` may want access to the `TestResult` object generated by running a set of tests for reporting purposes; a `TestResult` instance is returned by the `!TestRunner.run` method for this purpose.

`TestResult` instances have the following attributes that will be of interest when inspecting the results of running a set of tests:

<div class="attribute">

errors

A list containing 2-tuples of `TestCase` instances and strings holding formatted tracebacks. Each tuple represents a test which raised an unexpected exception.

</div>

<div class="attribute">

failures

A list containing 2-tuples of `TestCase` instances and strings holding formatted tracebacks. Each tuple represents a test where a failure was explicitly signalled using the `assert\* methods <assert-methods>`.

</div>

<div class="attribute">

skipped

A list containing 2-tuples of `TestCase` instances and strings holding the reason for skipping the test.

<div class="versionadded">

3.1

</div>

</div>

<div class="attribute">

expectedFailures

A list containing 2-tuples of `TestCase` instances and strings holding formatted tracebacks. Each tuple represents an expected failure or error of the test case.

</div>

<div class="attribute">

unexpectedSuccesses

A list containing `TestCase` instances that were marked as expected failures, but succeeded.

</div>

<div class="attribute">

collectedDurations

A list containing 2-tuples of test case names and floats representing the elapsed time of each test which was run.

<div class="versionadded">

3.12

</div>

</div>

<div class="attribute">

shouldStop

Set to `True` when the execution of tests should stop by `stop`.

</div>

<div class="attribute">

testsRun

The total number of tests run so far.

</div>

<div class="attribute">

buffer

If set to true, `sys.stdout` and `sys.stderr` will be buffered in between `startTest` and `stopTest` being called. Collected output will only be echoed onto the real `sys.stdout` and `sys.stderr` if the test fails or errors. Any output is also attached to the failure / error message.

<div class="versionadded">

3.2

</div>

</div>

<div class="attribute">

failfast

If set to true `stop` will be called on the first failure or error, halting the test run.

<div class="versionadded">

3.2

</div>

</div>

<div class="attribute">

tb_locals

If set to true then local variables will be shown in tracebacks.

<div class="versionadded">

3.5

</div>

</div>

<div class="attribute">

verbosity

The level of details which the test runner reports: `0` -- quiet, `1` -- progress dots, `2` -- test names, `3` -- also the individual examples of a `doctest`. A test runner is expected to set it to its own verbosity.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

wasSuccessful()

Return `True` if all tests run so far have passed, otherwise returns `False`.

<div class="versionchanged">

3.4 Returns `False` if there were any `unexpectedSuccesses` from tests marked with the `expectedFailure` decorator.

</div>

</div>

<div class="method">

stop()

This method can be called to signal that the set of tests being run should be aborted by setting the `shouldStop` attribute to `True`. `!TestRunner` objects should respect this flag and return without running any additional tests.

For example, this feature is used by the `TextTestRunner` class to stop the test framework when the user signals an interrupt from the keyboard. Interactive tools which provide `!TestRunner` implementations can use this in a similar manner.

</div>

The following methods of the `TestResult` class are used to maintain the internal data structures, and may be extended in subclasses to support additional reporting requirements. This is particularly useful in building tools which support interactive reporting while tests are being run.

<div class="method">

startTest(test)

Called when the test case *test* is about to be run.

</div>

<div class="method">

stopTest(test)

Called after the test case *test* has been executed, regardless of the outcome.

</div>

<div class="method">

startTestRun()

Called once before any tests are executed.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

stopTestRun()

Called once after all tests are executed.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

addError(test, err)

Called when the test case *test* raises an unexpected exception. *err* is a tuple of the form returned by `sys.exc_info`: `(type, value, traceback)`.

The default implementation appends a tuple `(test, formatted_err)` to the instance's `errors` attribute, where *formatted_err* is a formatted traceback derived from *err*.

</div>

<div class="method">

addFailure(test, err)

Called when the test case *test* signals a failure. *err* is a tuple of the form returned by `sys.exc_info`: `(type, value, traceback)`.

The default implementation appends a tuple `(test, formatted_err)` to the instance's `failures` attribute, where *formatted_err* is a formatted traceback derived from *err*.

</div>

<div class="method">

addSuccess(test)

Called when the test case *test* succeeds.

The default implementation does nothing.

</div>

<div class="method">

addSkip(test, reason)

Called when the test case *test* is skipped. *reason* is the reason the test gave for skipping.

The default implementation appends a tuple `(test, reason)` to the instance's `skipped` attribute.

</div>

<div class="method">

addExpectedFailure(test, err)

Called when the test case *test* fails or errors, but was marked with the `expectedFailure` decorator.

The default implementation appends a tuple `(test, formatted_err)` to the instance's `expectedFailures` attribute, where *formatted_err* is a formatted traceback derived from *err*.

</div>

<div class="method">

addUnexpectedSuccess(test)

Called when the test case *test* was marked with the `expectedFailure` decorator, but succeeded.

The default implementation appends the test to the instance's `unexpectedSuccesses` attribute.

</div>

<div class="method">

addSubTest(test, subtest, outcome)

Called when a subtest finishes. *test* is the test case corresponding to the test method. *subtest* is a custom `TestCase` instance describing the subtest.

If *outcome* is `None`, the subtest succeeded. Otherwise, it failed with an exception where *outcome* is a tuple of the form returned by `sys.exc_info`: `(type, value, traceback)`.

The default implementation does nothing when the outcome is a success, and records subtest failures as normal failures.

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

addDuration(test, elapsed)

Called when the test case finishes. *elapsed* is the time represented in seconds, and it includes the execution of cleanup functions.

<div class="versionadded">

3.12

</div>

</div>

</div>

<div class="TextTestResult(stream, descriptions, verbosity, *, durations=None)">

A concrete implementation of `TestResult` used by the `TextTestRunner`. Subclasses should accept `**kwargs` to ensure compatibility as the interface changes.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.12 Added the *durations* keyword parameter.

</div>

</div>

<div class="data">

defaultTestLoader

Instance of the `TestLoader` class intended to be shared. If no customization of the `TestLoader` is needed, this instance can be used instead of repeatedly creating new instances.

</div>

<div class="TextTestRunner(stream=None, descriptions=True, verbosity=1, failfast=False, \ buffer=False, resultclass=None, warnings=None, *, \ tb_locals=False, durations=None)">

A basic test runner implementation that outputs results to a stream. If *stream* is `None`, the default, `sys.stderr` is used as the output stream. This class has a few configurable parameters, but is essentially very simple. Graphical applications which run test suites should provide alternate implementations. Such implementations should accept `**kwargs` as the interface to construct runners changes when features are added to unittest.

By default this runner shows `DeprecationWarning`, `PendingDeprecationWarning`, `ResourceWarning` and `ImportWarning` even if they are `ignored by default
<warning-ignored>`. This behavior can be overridden using Python's `!-Wd` or `!-Wa` options (see `Warning control <using-on-warnings>`) and leaving *warnings* to `None`.

<div class="versionchanged">

3.2 Added the *warnings* parameter.

</div>

<div class="versionchanged">

3.2 The default stream is set to `sys.stderr` at instantiation time rather than import time.

</div>

<div class="versionchanged">

3.5 Added the *tb_locals* parameter.

</div>

<div class="versionchanged">

3.12 Added the *durations* parameter.

</div>

<div class="method">

<span id="makeresult">makeResult</span>()

This method returns the instance of `TestResult` used by `run`. It is not intended to be called directly, but can be overridden in subclasses to provide a custom `TestResult`.

`_makeResult()` instantiates the class or callable passed in the `TextTestRunner` constructor as the `resultclass` argument. It defaults to `TextTestResult` if no `resultclass` is provided. The result class is instantiated with the following arguments:

    stream, descriptions, verbosity

</div>

<div class="method">

run(test)

This method is the main public interface to the `TextTestRunner`. This method takes a `TestSuite` or `TestCase` instance. A `TestResult` is created by calling `_makeResult` and the test(s) are run and the results printed to stdout.

</div>

</div>

<div class="function">

main(module='\_\_main\_\_', defaultTest=None, argv=None, testRunner=None, testLoader=unittest.defaultTestLoader, exit=True, verbosity=1, failfast=None, catchbreak=None, buffer=None, warnings=None)

A command-line program that loads a set of tests from *module* and runs them; this is primarily for making test modules conveniently executable. The simplest use for this function is to include the following line at the end of a test script:

    if __name__ == '__main__':
        unittest.main()

You can run tests with more detailed information by passing in the verbosity argument:

    if __name__ == '__main__':
        unittest.main(verbosity=2)

The *defaultTest* argument is either the name of a single test or an iterable of test names to run if no test names are specified via *argv*. If not specified or `None` and no test names are provided via *argv*, all tests found in *module* are run.

The *argv* argument can be a list of options passed to the program, with the first element being the program name. If not specified or `None`, the values of `sys.argv` are used.

The *testRunner* argument can either be a test runner class or an already created instance of it. By default `main` calls `sys.exit` with an exit code indicating success (0) or failure (1) of the tests run. An exit code of 5 indicates that no tests were run or skipped.

The *testLoader* argument has to be a `TestLoader` instance, and defaults to `defaultTestLoader`.

`main` supports being used from the interactive interpreter by passing in the argument `exit=False`. This displays the result on standard output without calling `sys.exit`:

    >>> from unittest import main
    >>> main(module='test_module', exit=False)

The *failfast*, *catchbreak* and *buffer* parameters have the same effect as the same-name [command-line options](#command-line-options).

The *warnings* argument specifies the `warning filter <warning-filter>` that should be used while running the tests. If it's not specified, it will remain `None` if a `!-W` option is passed to `python` (see `Warning control <using-on-warnings>`), otherwise it will be set to `'default'`.

Calling `main` returns an object with the `result` attribute that contains the result of the tests run as a `unittest.TestResult`.

<div class="versionchanged">

3.1 The *exit* parameter was added.

</div>

<div class="versionchanged">

3.2 The *verbosity*, *failfast*, *catchbreak*, *buffer* and *warnings* parameters were added.

</div>

<div class="versionchanged">

3.4 The *defaultTest* parameter was changed to also accept an iterable of test names.

</div>

</div>

#### load_tests Protocol

<div class="versionadded">

3.2

</div>

Modules or packages can customize how tests are loaded from them during normal test runs or test discovery by implementing a function called `load_tests`.

If a test module defines `load_tests` it will be called by `TestLoader.loadTestsFromModule` with the following arguments:

    load_tests(loader, standard_tests, pattern)

where *pattern* is passed straight through from `loadTestsFromModule`. It defaults to `None`.

It should return a `TestSuite`.

*loader* is the instance of `TestLoader` doing the loading. *standard_tests* are the tests that would be loaded by default from the module. It is common for test modules to only want to add or remove tests from the standard set of tests. The third argument is used when loading packages as part of test discovery.

A typical `load_tests` function that loads tests from a specific set of `TestCase` classes may look like:

    test_cases = (TestCase1, TestCase2, TestCase3)

    def load_tests(loader, tests, pattern):
        suite = TestSuite()
        for test_class in test_cases:
            tests = loader.loadTestsFromTestCase(test_class)
            suite.addTests(tests)
        return suite

If discovery is started in a directory containing a package, either from the command line or by calling `TestLoader.discover`, then the package `__init__.py` will be checked for `load_tests`. If that function does not exist, discovery will recurse into the package as though it were just another directory. Otherwise, discovery of the package's tests will be left up to `load_tests` which is called with the following arguments:

    load_tests(loader, standard_tests, pattern)

This should return a `TestSuite` representing all the tests from the package. (`standard_tests` will only contain tests collected from `__init__.py`.)

Because the pattern is passed into `load_tests` the package is free to continue (and potentially modify) test discovery. A 'do nothing' `load_tests` function for a test package would look like:

    def load_tests(loader, standard_tests, pattern):
        # top level directory cached on loader instance
        this_dir = os.path.dirname(__file__)
        package_tests = loader.discover(start_dir=this_dir, pattern=pattern)
        standard_tests.addTests(package_tests)
        return standard_tests

<div class="versionchanged">

3.5 Discovery no longer checks package names for matching *pattern* due to the impossibility of package names matching the default pattern.

</div>

## Class and Module Fixtures

Class and module level fixtures are implemented in `TestSuite`. When the test suite encounters a test from a new class then `~TestCase.tearDownClass` from the previous class (if there is one) is called, followed by `~TestCase.setUpClass` from the new class.

Similarly if a test is from a different module from the previous test then `tearDownModule` from the previous module is run, followed by `setUpModule` from the new module.

After all the tests have run the final `tearDownClass` and `tearDownModule` are run.

Note that shared fixtures do not play well with \[potential\] features like test parallelization and they break test isolation. They should be used with care.

The default ordering of tests created by the unittest test loaders is to group all tests from the same modules and classes together. This will lead to `setUpClass` / `setUpModule` (etc) being called exactly once per class and module. If you randomize the order, so that tests from different modules and classes are adjacent to each other, then these shared fixture functions may be called multiple times in a single test run.

Shared fixtures are not intended to work with suites with non-standard ordering. A `BaseTestSuite` still exists for frameworks that don't want to support shared fixtures.

If there are any exceptions raised during one of the shared fixture functions the test is reported as an error. Because there is no corresponding test instance an `_ErrorHolder` object (that has the same interface as a `TestCase`) is created to represent the error. If you are just using the standard unittest test runner then this detail doesn't matter, but if you are a framework author it may be relevant.

### setUpClass and tearDownClass

These must be implemented as class methods:

    import unittest

    class Test(unittest.TestCase):
        @classmethod
        def setUpClass(cls):
            cls._connection = createExpensiveConnectionObject()

        @classmethod
        def tearDownClass(cls):
            cls._connection.destroy()

If you want the `setUpClass` and `tearDownClass` on base classes called then you must call up to them yourself. The implementations in `TestCase` are empty.

If an exception is raised during a `setUpClass` then the tests in the class are not run and the `tearDownClass` is not run. Skipped classes will not have `setUpClass` or `tearDownClass` run. If the exception is a `SkipTest` exception then the class will be reported as having been skipped instead of as an error.

### setUpModule and tearDownModule

<div class="function" data-no-typesetting="">

setUpModule tearDownModule

</div>

These should be implemented as functions:

    def setUpModule():
        createConnection()

    def tearDownModule():
        closeConnection()

If an exception is raised in a `setUpModule` then none of the tests in the module will be run and the `tearDownModule` will not be run. If the exception is a `SkipTest` exception then the module will be reported as having been skipped instead of as an error.

To add cleanup code that must be run even in the case of an exception, use `addModuleCleanup`:

<div class="function">

addModuleCleanup(function, /, *args,*\*kwargs)

Add a function to be called after `tearDownModule` to cleanup resources used during the test class. Functions will be called in reverse order to the order they are added (`LIFO (last-in, first-out)`). They are called with any arguments and keyword arguments passed into `addModuleCleanup` when they are added.

If `setUpModule` fails, meaning that `tearDownModule` is not called, then any cleanup functions added will still be called.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

enterModuleContext(cm)

Enter the supplied `context manager`. If successful, also add its `~object.__exit__` method as a cleanup function by `addModuleCleanup` and return the result of the `~object.__enter__` method.

<div class="versionadded">

3.11

</div>

</div>

<div class="function">

doModuleCleanups()

This function is called unconditionally after `tearDownModule`, or after `setUpModule` if `setUpModule` raises an exception.

It is responsible for calling all the cleanup functions added by `addModuleCleanup`. If you need cleanup functions to be called *prior* to `tearDownModule` then you can call `doModuleCleanups` yourself.

`doModuleCleanups` pops methods off the stack of cleanup functions one at a time, so it can be called at any time.

<div class="versionadded">

3.8

</div>

</div>

## Signal Handling

<div class="versionadded">

3.2

</div>

The `-c/--catch <unittest -c>` command-line option to unittest, along with the `catchbreak` parameter to `unittest.main`, provide more friendly handling of control-C during a test run. With catch break behavior enabled control-C will allow the currently running test to complete, and the test run will then end and report all the results so far. A second control-c will raise a `KeyboardInterrupt` in the usual way.

The control-c handling signal handler attempts to remain compatible with code or tests that install their own `signal.SIGINT` handler. If the `unittest` handler is called but *isn't* the installed `signal.SIGINT` handler, i.e. it has been replaced by the system under test and delegated to, then it calls the default handler. This will normally be the expected behavior by code that replaces an installed handler and delegates to it. For individual tests that need `unittest` control-c handling disabled the `removeHandler` decorator can be used.

There are a few utility functions for framework authors to enable control-c handling functionality within test frameworks.

<div class="function">

installHandler()

Install the control-c handler. When a `signal.SIGINT` is received (usually in response to the user pressing control-c) all registered results have `~TestResult.stop` called.

</div>

<div class="function">

registerResult(result)

Register a `TestResult` object for control-c handling. Registering a result stores a weak reference to it, so it doesn't prevent the result from being garbage collected.

Registering a `TestResult` object has no side-effects if control-c handling is not enabled, so test frameworks can unconditionally register all results they create independently of whether or not handling is enabled.

</div>

<div class="function">

removeResult(result)

Remove a registered result. Once a result has been removed then `~TestResult.stop` will no longer be called on that result object in response to a control-c.

</div>

<div class="function">

removeHandler(function=None)

When called without arguments this function removes the control-c handler if it has been installed. This function can also be used as a test decorator to temporarily remove the handler while the test is being executed:

    @unittest.removeHandler
    def test_signal_handling(self):
        ...

</div>

# `!unittest.mock` --- getting started

<div class="currentmodule">

unittest.mock

</div>

<div class="versionadded">

3.3

</div>

<div id="getting-started">

<div class="testsetup">

import asyncio import unittest from unittest.mock import Mock, MagicMock, AsyncMock, patch, call, sentinel

class SomeClass:  
attribute = 'this is a doctest'

@staticmethod def static_method(): pass

</div>

</div>

## Using Mock

### Mock patching methods

Common uses for `Mock` objects include:

- Patching methods
- Recording method calls on objects

You might want to replace a method on an object to check that it is called with the correct arguments by another part of the system:

> \>\>\> real = SomeClass() \>\>\> real.method = MagicMock(name='method') \>\>\> real.method(3, 4, 5, key='value') \<MagicMock name='method()' id='...'\>

Once our mock has been used (`real.method` in this example) it has methods and attributes that allow you to make assertions about how it has been used.

> [!NOTE]
> In most of these examples the `Mock` and `MagicMock` classes are interchangeable. As the `MagicMock` is the more capable class it makes a sensible one to use by default.

Once the mock has been called its `~Mock.called` attribute is set to `True`. More importantly we can use the `~Mock.assert_called_with` or `~Mock.assert_called_once_with` method to check that it was called with the correct arguments.

This example tests that calling `ProductionClass().method` results in a call to the `something` method:

> \>\>\> class ProductionClass: ... def method(self): ... self.something(1, 2, 3) ... def something(self, a, b, c): ... pass ... \>\>\> real = ProductionClass() \>\>\> real.something = MagicMock() \>\>\> real.method() \>\>\> real.something.assert_called_once_with(1, 2, 3)

### Mock for method calls on an object

In the last example we patched a method directly on an object to check that it was called correctly. Another common use case is to pass an object into a method (or some part of the system under test) and then check that it is used in the correct way.

The simple `ProductionClass` below has a `closer` method. If it is called with an object then it calls `close` on it.

> \>\>\> class ProductionClass: ... def closer(self, something): ... something.close() ...

So to test it we need to pass in an object with a `close` method and check that it was called correctly.

> \>\>\> real = ProductionClass() \>\>\> mock = Mock() \>\>\> real.closer(mock) \>\>\> mock.close.assert_called_with()

We don't have to do any work to provide the 'close' method on our mock. Accessing close creates it. So, if 'close' hasn't already been called then accessing it in the test will create it, but `~Mock.assert_called_with` will raise a failure exception.

### Mocking classes

A common use case is to mock out classes instantiated by your code under test. When you patch a class, then that class is replaced with a mock. Instances are created by *calling the class*. This means you access the "mock instance" by looking at the return value of the mocked class.

In the example below we have a function `some_function` that instantiates `Foo` and calls a method on it. The call to `patch` replaces the class `Foo` with a mock. The `Foo` instance is the result of calling the mock, so it is configured by modifying the mock `~Mock.return_value`. :

    >>> def some_function():
    ...     instance = module.Foo()
    ...     return instance.method()
    ...
    >>> with patch('module.Foo') as mock:
    ...     instance = mock.return_value
    ...     instance.method.return_value = 'the result'
    ...     result = some_function()
    ...     assert result == 'the result'

### Naming your mocks

It can be useful to give your mocks a name. The name is shown in the repr of the mock and can be helpful when the mock appears in test failure messages. The name is also propagated to attributes or methods of the mock:

> \>\>\> mock = MagicMock(name='foo') \>\>\> mock \<MagicMock name='foo' id='...'\> \>\>\> mock.method \<MagicMock name='foo.method' id='...'\>

### Tracking all calls

Often you want to track more than a single call to a method. The `~Mock.mock_calls` attribute records all calls to child attributes of the mock - and also to their children.

> \>\>\> mock = MagicMock() \>\>\> mock.method() \<MagicMock name='mock.method()' id='...'\> \>\>\> mock.attribute.method(10, x=53) \<MagicMock name='mock.attribute.method()' id='...'\> \>\>\> mock.mock_calls \[call.method(), call.attribute.method(10, x=53)\]

If you make an assertion about `mock_calls` and any unexpected methods have been called, then the assertion will fail. This is useful because as well as asserting that the calls you expected have been made, you are also checking that they were made in the right order and with no additional calls:

You use the `call` object to construct lists for comparing with `mock_calls`:

> \>\>\> expected = \[call.method(), call.attribute.method(10, x=53)\] \>\>\> mock.mock_calls == expected True

However, parameters to calls that return mocks are not recorded, which means it is not possible to track nested calls where the parameters used to create ancestors are important:

> \>\>\> m = Mock() \>\>\> m.factory(important=True).deliver() \<Mock name='mock.factory().deliver()' id='...'\> \>\>\> m.mock_calls\[-1\] == call.factory(important=False).deliver() True

### Setting return values and attributes

Setting the return values on a mock object is trivially easy:

> \>\>\> mock = Mock() \>\>\> mock.return_value = 3 \>\>\> mock() 3

Of course you can do the same for methods on the mock:

> \>\>\> mock = Mock() \>\>\> mock.method.return_value = 3 \>\>\> mock.method() 3

The return value can also be set in the constructor:

> \>\>\> mock = Mock(return_value=3) \>\>\> mock() 3

If you need an attribute setting on your mock, just do it:

> \>\>\> mock = Mock() \>\>\> mock.x = 3 \>\>\> mock.x 3

Sometimes you want to mock up a more complex situation, like for example `mock.connection.cursor().execute("SELECT 1")`. If we wanted this call to return a list, then we have to configure the result of the nested call.

We can use `call` to construct the set of calls in a "chained call" like this for easy assertion afterwards:

> \>\>\> mock = Mock() \>\>\> cursor = mock.connection.cursor.return_value \>\>\> cursor.execute.return_value = \['foo'\] \>\>\> mock.connection.cursor().execute("SELECT 1") \['foo'\] \>\>\> expected = call.connection.cursor().execute("SELECT 1").call_list() \>\>\> mock.mock_calls \[call.connection.cursor(), call.connection.cursor().execute('SELECT 1')\] \>\>\> mock.mock_calls == expected True

It is the call to `.call_list()` that turns our call object into a list of calls representing the chained calls.

### Raising exceptions with mocks

A useful attribute is `~Mock.side_effect`. If you set this to an exception class or instance then the exception will be raised when the mock is called.

> \>\>\> mock = Mock(side_effect=Exception('Boom!')) \>\>\> mock() Traceback (most recent call last): ... Exception: Boom!

### Side effect functions and iterables

`side_effect` can also be set to a function or an iterable. The use case for `side_effect` as an iterable is where your mock is going to be called several times, and you want each call to return a different value. When you set `side_effect` to an iterable every call to the mock returns the next value from the iterable:

> \>\>\> mock = MagicMock(side_effect=\[4, 5, 6\]) \>\>\> mock() 4 \>\>\> mock() 5 \>\>\> mock() 6

For more advanced use cases, like dynamically varying the return values depending on what the mock is called with, `side_effect` can be a function. The function will be called with the same arguments as the mock. Whatever the function returns is what the call returns:

> \>\>\> vals = {(1, 2): 1, (2, 3): 2} \>\>\> def side_effect(\*args): ... return vals\[args\] ... \>\>\> mock = MagicMock(side_effect=side_effect) \>\>\> mock(1, 2) 1 \>\>\> mock(2, 3) 2

### Mocking asynchronous iterators

Since Python 3.8, `AsyncMock` and `MagicMock` have support to mock `async-iterators` through `__aiter__`. The `~Mock.return_value` attribute of `__aiter__` can be used to set the return values to be used for iteration.

> \>\>\> mock = MagicMock() \# AsyncMock also works here \>\>\> mock.\_\_aiter\_\_.return_value = \[1, 2, 3\] \>\>\> async def main(): ... return \[i async for i in mock\] ... \>\>\> asyncio.run(main()) \[1, 2, 3\]

### Mocking asynchronous context manager

Since Python 3.8, `AsyncMock` and `MagicMock` have support to mock `async-context-managers` through `__aenter__` and `__aexit__`. By default, `__aenter__` and `__aexit__` are `AsyncMock` instances that return an async function.

> \>\>\> class AsyncContextManager: ... async def \_\_aenter\_\_(self): ... return self ... async def \_\_aexit\_\_(self, exc_type, exc, tb): ... pass ... \>\>\> mock_instance = MagicMock(AsyncContextManager()) \# AsyncMock also works here \>\>\> async def main(): ... async with mock_instance as result: ... pass ... \>\>\> asyncio.run(main()) \>\>\> mock_instance.\_\_aenter\_\_.assert_awaited_once() \>\>\> mock_instance.\_\_aexit\_\_.assert_awaited_once()

### Creating a mock from an existing object

One problem with over use of mocking is that it couples your tests to the implementation of your mocks rather than your real code. Suppose you have a class that implements `some_method`. In a test for another class, you provide a mock of this object that *also* provides `some_method`. If later you refactor the first class, so that it no longer has `some_method` - then your tests will continue to pass even though your code is now broken!

`Mock` allows you to provide an object as a specification for the mock, using the *spec* keyword argument. Accessing methods / attributes on the mock that don't exist on your specification object will immediately raise an attribute error. If you change the implementation of your specification, then tests that use that class will start failing immediately without you having to instantiate the class in those tests.

> \>\>\> mock = Mock(spec=SomeClass) \>\>\> mock.old_method() Traceback (most recent call last): ... AttributeError: Mock object has no attribute 'old_method'. Did you mean: 'class_method'?

Using a specification also enables a smarter matching of calls made to the mock, regardless of whether some parameters were passed as positional or named arguments:

    >>> def f(a, b, c): pass
    ...
    >>> mock = Mock(spec=f)
    >>> mock(1, 2, 3)
    <Mock name='mock()' id='140161580456576'>
    >>> mock.assert_called_with(a=1, b=2, c=3)

If you want this smarter matching to also work with method calls on the mock, you can use `auto-speccing <auto-speccing>`.

If you want a stronger form of specification that prevents the setting of arbitrary attributes as well as the getting of them then you can use *spec_set* instead of *spec*.

### Using side_effect to return per file content

`mock_open` is used to patch `open` method. `~Mock.side_effect` can be used to return a new Mock object per call. This can be used to return different contents per file stored in a dictionary:

    DEFAULT = "default"
    data_dict = {"file1": "data1",
                 "file2": "data2"}

    def open_side_effect(name):
        return mock_open(read_data=data_dict.get(name, DEFAULT))()

    with patch("builtins.open", side_effect=open_side_effect):
        with open("file1") as file1:
            assert file1.read() == "data1"

        with open("file2") as file2:
            assert file2.read() == "data2"

        with open("file3") as file2:
            assert file2.read() == "default"

## Patch decorators

> [!NOTE]
> With `patch` it matters that you patch objects in the namespace where they are looked up. This is normally straightforward, but for a quick guide read `where to patch <where-to-patch>`.

A common need in tests is to patch a class attribute or a module attribute, for example patching a builtin or patching a class in a module to test that it is instantiated. Modules and classes are effectively global, so patching on them has to be undone after the test or the patch will persist into other tests and cause hard to diagnose problems.

mock provides three convenient decorators for this: `patch`, `patch.object` and `patch.dict`. `patch` takes a single string, of the form `package.module.Class.attribute` to specify the attribute you are patching. It also optionally takes a value that you want the attribute (or class or whatever) to be replaced with. 'patch.object' takes an object and the name of the attribute you would like patched, plus optionally the value to patch it with.

`patch.object`:

    >>> original = SomeClass.attribute
    >>> @patch.object(SomeClass, 'attribute', sentinel.attribute)
    ... def test():
    ...     assert SomeClass.attribute == sentinel.attribute
    ...
    >>> test()
    >>> assert SomeClass.attribute == original

    >>> @patch('package.module.attribute', sentinel.attribute)
    ... def test():
    ...     from package.module import attribute
    ...     assert attribute is sentinel.attribute
    ...
    >>> test()

If you are patching a module (including `builtins`) then use `patch` instead of `patch.object`:

> \>\>\> mock = MagicMock(return_value=sentinel.file_handle) \>\>\> with patch('builtins.open', mock): ... handle = open('filename', 'r') ... \>\>\> mock.assert_called_with('filename', 'r') \>\>\> assert handle == sentinel.file_handle, "incorrect file handle returned"

The module name can be 'dotted', in the form `package.module` if needed:

    >>> @patch('package.module.ClassName.attribute', sentinel.attribute)
    ... def test():
    ...     from package.module import ClassName
    ...     assert ClassName.attribute == sentinel.attribute
    ...
    >>> test()

A nice pattern is to actually decorate test methods themselves:

> \>\>\> class MyTest(unittest.TestCase): ... @patch.object(SomeClass, 'attribute', sentinel.attribute) ... def test_something(self): ... self.assertEqual(SomeClass.attribute, sentinel.attribute) ... \>\>\> original = SomeClass.attribute \>\>\> MyTest('test_something').test_something() \>\>\> assert SomeClass.attribute == original

If you want to patch with a Mock, you can use `patch` with only one argument (or `patch.object` with two arguments). The mock will be created for you and passed into the test function / method:

> \>\>\> class MyTest(unittest.TestCase): ... @patch.object(SomeClass, 'static_method') ... def test_something(self, mock_method): ... SomeClass.static_method() ... mock_method.assert_called_with() ... \>\>\> MyTest('test_something').test_something()

You can stack up multiple patch decorators using this pattern:

    >>> class MyTest(unittest.TestCase):
    ...     @patch('package.module.ClassName1')
    ...     @patch('package.module.ClassName2')
    ...     def test_something(self, MockClass2, MockClass1):
    ...         self.assertIs(package.module.ClassName1, MockClass1)
    ...         self.assertIs(package.module.ClassName2, MockClass2)
    ...
    >>> MyTest('test_something').test_something()

When you nest patch decorators the mocks are passed in to the decorated function in the same order they applied (the normal *Python* order that decorators are applied). This means from the bottom up, so in the example above the mock for `test_module.ClassName2` is passed in first.

There is also `patch.dict` for setting values in a dictionary just during a scope and restoring the dictionary to its original state when the test ends:

> \>\>\> foo = {'key': 'value'} \>\>\> original = foo.copy() \>\>\> with patch.dict(foo, {'newkey': 'newvalue'}, clear=True): ... assert foo == {'newkey': 'newvalue'} ... \>\>\> assert foo == original

`patch`, `patch.object` and `patch.dict` can all be used as context managers.

Where you use `patch` to create a mock for you, you can get a reference to the mock using the "as" form of the with statement:

> \>\>\> class ProductionClass: ... def method(self): ... pass ... \>\>\> with patch.object(ProductionClass, 'method') as mock_method: ... mock_method.return_value = None ... real = ProductionClass() ... real.method(1, 2, 3) ... \>\>\> mock_method.assert_called_with(1, 2, 3)

As an alternative `patch`, `patch.object` and `patch.dict` can be used as class decorators. When used in this way it is the same as applying the decorator individually to every method whose name starts with "test".

## Further examples

Here are some more examples for some slightly more advanced scenarios.

### Mocking chained calls

Mocking chained calls is actually straightforward with mock once you understand the `~Mock.return_value` attribute. When a mock is called for the first time, or you fetch its `return_value` before it has been called, a new `Mock` is created.

This means that you can see how the object returned from a call to a mocked object has been used by interrogating the `return_value` mock:

> \>\>\> mock = Mock() \>\>\> mock().foo(a=2, b=3) \<Mock name='mock().foo()' id='...'\> \>\>\> mock.return_value.foo.assert_called_with(a=2, b=3)

From here it is a simple step to configure and then make assertions about chained calls. Of course another alternative is writing your code in a more testable way in the first place...

So, suppose we have some code that looks a little bit like this:

> \>\>\> class Something: ... def \_\_init\_\_(self): ... self.backend = BackendProvider() ... def method(self): ... response = self.backend.get_endpoint('foobar').create_call('spam', 'eggs').start_call() ... \# more code

Assuming that `BackendProvider` is already well tested, how do we test `method()`? Specifically, we want to test that the code section `# more code` uses the response object in the correct way.

As this chain of calls is made from an instance attribute we can monkey patch the `backend` attribute on a `Something` instance. In this particular case we are only interested in the return value from the final call to `start_call` so we don't have much configuration to do. Let's assume the object it returns is 'file-like', so we'll ensure that our response object uses the builtin `open` as its `spec`.

To do this we create a mock instance as our mock backend and create a mock response object for it. To set the response as the return value for that final `start_call` we could do this:

    mock_backend.get_endpoint.return_value.create_call.return_value.start_call.return_value = mock_response

We can do that in a slightly nicer way using the `~Mock.configure_mock` method to directly set the return value for us:

    >>> something = Something()
    >>> mock_response = Mock(spec=open)
    >>> mock_backend = Mock()
    >>> config = {'get_endpoint.return_value.create_call.return_value.start_call.return_value': mock_response}
    >>> mock_backend.configure_mock(**config)

With these we monkey patch the "mock backend" in place and can make the real call:

    >>> something.backend = mock_backend
    >>> something.method()

Using `~Mock.mock_calls` we can check the chained call with a single assert. A chained call is several calls in one line of code, so there will be several entries in `mock_calls`. We can use `call.call_list` to create this list of calls for us:

    >>> chained = call.get_endpoint('foobar').create_call('spam', 'eggs').start_call()
    >>> call_list = chained.call_list()
    >>> assert mock_backend.mock_calls == call_list

### Partial mocking

For some tests, you may want to mock out a call to `datetime.date.today` to return a known date, but don't want to prevent the code under test from creating new date objects. Unfortunately `datetime.date` is written in C, so you cannot just monkey-patch out the static `datetime.date.today` method.

Instead, you can effectively wrap the date class with a mock, while passing through calls to the constructor to the real class (and returning real instances).

The `patch decorator <patch>` is used here to mock out the `date` class in the module under test. The `~Mock.side_effect` attribute on the mock date class is then set to a lambda function that returns a real date. When the mock date class is called a real date will be constructed and returned by `side_effect`. :

    >>> import datetime as dt
    >>> with patch('mymodule.date') as mock_date:
    ...     mock_date.today.return_value = dt.date(2010, 10, 8)
    ...     mock_date.side_effect = lambda *args, **kw: dt.date(*args, **kw)
    ...
    ...     assert mymodule.date.today() == dt.date(2010, 10, 8)
    ...     assert mymodule.date(2009, 6, 8) == dt.date(2009, 6, 8)

Note that we don't patch `datetime.date` globally, we patch `date` in the module that *uses* it. See `where to patch <where-to-patch>`.

When `date.today()` is called a known date is returned, but calls to the `date(...)` constructor still return normal dates. Without this you can find yourself having to calculate an expected result using exactly the same algorithm as the code under test, which is a classic testing anti-pattern.

Calls to the date constructor are recorded in the `mock_date` attributes (`call_count` and friends) which may also be useful for your tests.

An alternative way of dealing with mocking dates, or other builtin classes, is discussed in [this blog entry](https://williambert.online/2011/07/how-to-unit-testing-in-django-with-mocking-and-patching/).

### Mocking a generator method

A Python generator is a function or method that uses the `yield` statement to return a series of values when iterated over[^1].

A generator method / function is called to return the generator object. It is the generator object that is then iterated over. The protocol method for iteration is `~container.__iter__`, so we can mock this using a `MagicMock`.

Here's an example class with an "iter" method implemented as a generator:

> \>\>\> class Foo: ... def iter(self): ... for i in \[1, 2, 3\]: ... yield i ... \>\>\> foo = Foo() \>\>\> list(foo.iter()) \[1, 2, 3\]

How would we mock this class, and in particular its "iter" method?

To configure the values returned from the iteration (implicit in the call to `list`), we need to configure the object returned by the call to `foo.iter()`.

> \>\>\> mock_foo = MagicMock() \>\>\> mock_foo.iter.return_value = iter(\[1, 2, 3\]) \>\>\> list(mock_foo.iter()) \[1, 2, 3\]

### Applying the same patch to every test method

If you want several patches in place for multiple test methods the obvious way is to apply the patch decorators to every method. This can feel like unnecessary repetition. Instead, you can use `patch` (in all its various forms) as a class decorator. This applies the patches to all test methods on the class. A test method is identified by methods whose names start with `test`:

    >>> @patch('mymodule.SomeClass')
    ... class MyTest(unittest.TestCase):
    ...
    ...     def test_one(self, MockSomeClass):
    ...         self.assertIs(mymodule.SomeClass, MockSomeClass)
    ...
    ...     def test_two(self, MockSomeClass):
    ...         self.assertIs(mymodule.SomeClass, MockSomeClass)
    ...
    ...     def not_a_test(self):
    ...         return 'something'
    ...
    >>> MyTest('test_one').test_one()
    >>> MyTest('test_two').test_two()
    >>> MyTest('test_two').not_a_test()
    'something'

An alternative way of managing patches is to use the `start-and-stop`. These allow you to move the patching into your `setUp` and `tearDown` methods. :

    >>> class MyTest(unittest.TestCase):
    ...     def setUp(self):
    ...         self.patcher = patch('mymodule.foo')
    ...         self.mock_foo = self.patcher.start()
    ...
    ...     def test_foo(self):
    ...         self.assertIs(mymodule.foo, self.mock_foo)
    ...
    ...     def tearDown(self):
    ...         self.patcher.stop()
    ...
    >>> MyTest('test_foo').run()

If you use this technique you must ensure that the patching is "undone" by calling `stop`. This can be fiddlier than you might think, because if an exception is raised in the setUp then tearDown is not called. `unittest.TestCase.addCleanup` makes this easier:

    >>> class MyTest(unittest.TestCase):
    ...     def setUp(self):
    ...         patcher = patch('mymodule.foo')
    ...         self.addCleanup(patcher.stop)
    ...         self.mock_foo = patcher.start()
    ...
    ...     def test_foo(self):
    ...         self.assertIs(mymodule.foo, self.mock_foo)
    ...
    >>> MyTest('test_foo').run()

### Mocking unbound methods

Sometimes a test needs to patch an *unbound method*, which means patching the method on the class rather than on the instance. In order to make assertions about which objects were calling this particular method, you need to pass `self` as the first argument. The issue is that you can't patch with a mock for this, because if you replace an unbound method with a mock it doesn't become a bound method when fetched from the instance, and so it doesn't get `self` passed in. The workaround is to patch the unbound method with a real function instead. The `patch` decorator makes it so simple to patch out methods with a mock that having to create a real function becomes a nuisance.

If you pass `autospec=True` to patch then it does the patching with a *real* function object. This function object has the same signature as the one it is replacing, but delegates to a mock under the hood. You still get your mock auto-created in exactly the same way as before. What it means though, is that if you use it to patch out an unbound method on a class the mocked function will be turned into a bound method if it is fetched from an instance. It will have `self` passed in as the first argument, which is exactly what was needed:

> \>\>\> class Foo: ... def foo(self): ... pass ... \>\>\> with patch.object(Foo, 'foo', autospec=True) as mock_foo: ... mock_foo.return_value = 'foo' ... foo = Foo() ... foo.foo() ... 'foo' \>\>\> mock_foo.assert_called_once_with(foo)

If we don't use `autospec=True` then the unbound method is patched out with a Mock instance instead, and isn't called with `self`.

### Checking multiple calls with mock

mock has a nice API for making assertions about how your mock objects are used.

> \>\>\> mock = Mock() \>\>\> mock.foo_bar.return_value = None \>\>\> mock.foo_bar('baz', spam='eggs') \>\>\> mock.foo_bar.assert_called_with('baz', spam='eggs')

If your mock is only being called once you can use the `~Mock.assert_called_once_with` method that also asserts that the `~Mock.call_count` is one.

> \>\>\> mock.foo_bar.assert_called_once_with('baz', spam='eggs') \>\>\> mock.foo_bar() \>\>\> mock.foo_bar.assert_called_once_with('baz', spam='eggs') Traceback (most recent call last): ... AssertionError: Expected 'foo_bar' to be called once. Called 2 times. Calls: \[call('baz', spam='eggs'), call()\].

Both `assert_called_with` and `assert_called_once_with` make assertions about the *most recent* call. If your mock is going to be called several times, and you want to make assertions about *all* those calls you can use `~Mock.call_args_list`:

> \>\>\> mock = Mock(return_value=None) \>\>\> mock(1, 2, 3) \>\>\> mock(4, 5, 6) \>\>\> mock() \>\>\> mock.call_args_list \[call(1, 2, 3), call(4, 5, 6), call()\]

The `call` helper makes it easy to make assertions about these calls. You can build up a list of expected calls and compare it to `call_args_list`. This looks remarkably similar to the repr of the `call_args_list`:

> \>\>\> expected = \[call(1, 2, 3), call(4, 5, 6), call()\] \>\>\> mock.call_args_list == expected True

### Coping with mutable arguments

Another situation is rare, but can bite you, is when your mock is called with mutable arguments. `call_args` and `call_args_list` store *references* to the arguments. If the arguments are mutated by the code under test then you can no longer make assertions about what the values were when the mock was called.

Here's some example code that shows the problem. Imagine the following functions defined in 'mymodule':

    def frob(val):
        pass

    def grob(val):
        "First frob and then clear val"
        frob(val)
        val.clear()

When we try to test that `grob` calls `frob` with the correct argument look what happens:

    >>> with patch('mymodule.frob') as mock_frob:
    ...     val = {6}
    ...     mymodule.grob(val)
    ...
    >>> val
    set()
    >>> mock_frob.assert_called_with({6})
    Traceback (most recent call last):
        ...
    AssertionError: Expected: (({6},), {})
    Called with: ((set(),), {})

One possibility would be for mock to copy the arguments you pass in. This could then cause problems if you do assertions that rely on object identity for equality.

Here's one solution that uses the `~Mock.side_effect` functionality. If you provide a `side_effect` function for a mock then `side_effect` will be called with the same args as the mock. This gives us an opportunity to copy the arguments and store them for later assertions. In this example we're using *another* mock to store the arguments so that we can use the mock methods for doing the assertion. Again a helper function sets this up for us. :

    >>> from copy import deepcopy
    >>> from unittest.mock import Mock, patch, DEFAULT
    >>> def copy_call_args(mock):
    ...     new_mock = Mock()
    ...     def side_effect(*args, **kwargs):
    ...         args = deepcopy(args)
    ...         kwargs = deepcopy(kwargs)
    ...         new_mock(*args, **kwargs)
    ...         return DEFAULT
    ...     mock.side_effect = side_effect
    ...     return new_mock
    ...
    >>> with patch('mymodule.frob') as mock_frob:
    ...     new_mock = copy_call_args(mock_frob)
    ...     val = {6}
    ...     mymodule.grob(val)
    ...
    >>> new_mock.assert_called_with({6})
    >>> new_mock.call_args
    call({6})

`copy_call_args` is called with the mock that will be called. It returns a new mock that we do the assertion on. The `side_effect` function makes a copy of the args and calls our `new_mock` with the copy.

> [!NOTE]
> If your mock is only going to be used once there is an easier way of checking arguments at the point they are called. You can simply do the checking inside a `side_effect` function.
>
> > \>\>\> def side_effect(arg): ... assert arg == {6} ... \>\>\> mock = Mock(side_effect=side_effect) \>\>\> mock({6}) \>\>\> mock(set()) Traceback (most recent call last): ... AssertionError

An alternative approach is to create a subclass of `Mock` or `MagicMock` that copies (using `copy.deepcopy`) the arguments. Here's an example implementation:

> \>\>\> from copy import deepcopy \>\>\> class CopyingMock(MagicMock): ... def \_\_call\_\_(self, /, *args,kwargs): ... args = deepcopy(args) ... kwargs = deepcopy(kwargs) ... return super().\_\_call\_\_(*args, \*\*kwargs) ... \>\>\> c = CopyingMock(return_value=None) \>\>\> arg = set() \>\>\> c(arg) \>\>\> arg.add(1) \>\>\> c.assert_called_with(set()) \>\>\> c.assert_called_with(arg) Traceback (most recent call last): ... AssertionError: expected call not found. Expected: mock({1}) Actual: mock(set()) \>\>\> c.foo \<CopyingMock name='mock.foo' id='...'\>

When you subclass `Mock` or `MagicMock` all dynamically created attributes, and the `return_value` will use your subclass automatically. That means all children of a `CopyingMock` will also have the type `CopyingMock`.

### Nesting patches

Using patch as a context manager is nice, but if you do multiple patches you can end up with nested with statements indenting further and further to the right:

    >>> class MyTest(unittest.TestCase):
    ...
    ...     def test_foo(self):
    ...         with patch('mymodule.Foo') as mock_foo:
    ...             with patch('mymodule.Bar') as mock_bar:
    ...                 with patch('mymodule.Spam') as mock_spam:
    ...                     assert mymodule.Foo is mock_foo
    ...                     assert mymodule.Bar is mock_bar
    ...                     assert mymodule.Spam is mock_spam
    ...
    >>> original = mymodule.Foo
    >>> MyTest('test_foo').test_foo()
    >>> assert mymodule.Foo is original

With unittest `cleanup` functions and the `start-and-stop` we can achieve the same effect without the nested indentation. A simple helper method, `create_patch`, puts the patch in place and returns the created mock for us:

    >>> class MyTest(unittest.TestCase):
    ...
    ...     def create_patch(self, name):
    ...         patcher = patch(name)
    ...         thing = patcher.start()
    ...         self.addCleanup(patcher.stop)
    ...         return thing
    ...
    ...     def test_foo(self):
    ...         mock_foo = self.create_patch('mymodule.Foo')
    ...         mock_bar = self.create_patch('mymodule.Bar')
    ...         mock_spam = self.create_patch('mymodule.Spam')
    ...
    ...         assert mymodule.Foo is mock_foo
    ...         assert mymodule.Bar is mock_bar
    ...         assert mymodule.Spam is mock_spam
    ...
    >>> original = mymodule.Foo
    >>> MyTest('test_foo').run()
    >>> assert mymodule.Foo is original

### Mocking a dictionary with MagicMock

You may want to mock a dictionary, or other container object, recording all access to it whilst having it still behave like a dictionary.

We can do this with `MagicMock`, which will behave like a dictionary, and using `~Mock.side_effect` to delegate dictionary access to a real underlying dictionary that is under our control.

When the `~object.__getitem__` and `~object.__setitem__` methods of our `MagicMock` are called (normal dictionary access) then `side_effect` is called with the key (and in the case of `__setitem__` the value too). We can also control what is returned.

After the `MagicMock` has been used we can use attributes like `~Mock.call_args_list` to assert about how the dictionary was used:

> \>\>\> my_dict = {'a': 1, 'b': 2, 'c': 3} \>\>\> def getitem(name): ... return my_dict\[name\] ... \>\>\> def setitem(name, val): ... my_dict\[name\] = val ... \>\>\> mock = MagicMock() \>\>\> mock.\_\_getitem\_\_.side_effect = getitem \>\>\> mock.\_\_setitem\_\_.side_effect = setitem

> [!NOTE]
> An alternative to using `MagicMock` is to use `Mock` and *only* provide the magic methods you specifically want:
>
> > \>\>\> mock = Mock() \>\>\> mock.\_\_getitem\_\_ = Mock(side_effect=getitem) \>\>\> mock.\_\_setitem\_\_ = Mock(side_effect=setitem)
>
> A *third* option is to use `MagicMock` but passing in `dict` as the *spec* (or *spec_set*) argument so that the `MagicMock` created only has dictionary magic methods available:
>
> > \>\>\> mock = MagicMock(spec_set=dict) \>\>\> mock.\_\_getitem\_\_.side_effect = getitem \>\>\> mock.\_\_setitem\_\_.side_effect = setitem

With these side effect functions in place, the `mock` will behave like a normal dictionary but recording the access. It even raises a `KeyError` if you try to access a key that doesn't exist.

> \>\>\> mock\['a'\] 1 \>\>\> mock\['c'\] 3 \>\>\> mock\['d'\] Traceback (most recent call last): ... KeyError: 'd' \>\>\> mock\['b'\] = 'fish' \>\>\> mock\['d'\] = 'eggs' \>\>\> mock\['b'\] 'fish' \>\>\> mock\['d'\] 'eggs'

After it has been used you can make assertions about the access using the normal mock methods and attributes:

> \>\>\> mock.\_\_getitem\_\_.call_args_list \[call('a'), call('c'), call('d'), call('b'), call('d')\] \>\>\> mock.\_\_setitem\_\_.call_args_list \[call('b', 'fish'), call('d', 'eggs')\] \>\>\> my_dict {'a': 1, 'b': 'fish', 'c': 3, 'd': 'eggs'}

### Mock subclasses and their attributes

There are various reasons why you might want to subclass `Mock`. One reason might be to add helper methods. Here's a silly example:

> \>\>\> class MyMock(MagicMock): ... def has_been_called(self): ... return self.called ... \>\>\> mymock = MyMock(return_value=None) \>\>\> mymock \<MyMock id='...'\> \>\>\> mymock.has_been_called() False \>\>\> mymock() \>\>\> mymock.has_been_called() True

The standard behaviour for `Mock` instances is that attributes and the return value mocks are of the same type as the mock they are accessed on. This ensures that `Mock` attributes are `Mocks` and `MagicMock` attributes are `MagicMocks` [^2]. So if you're subclassing to add helper methods then they'll also be available on the attributes and return value mock of instances of your subclass.

> \>\>\> mymock.foo \<MyMock name='mock.foo' id='...'\> \>\>\> mymock.foo.has_been_called() False \>\>\> mymock.foo() \<MyMock name='mock.foo()' id='...'\> \>\>\> mymock.foo.has_been_called() True

Sometimes this is inconvenient. For example, [one user](https://code.google.com/archive/p/mock/issues/105) is subclassing mock to created a [Twisted adaptor](https://twisted.org/documents/11.0.0/api/twisted.python.components.html). Having this applied to attributes too actually causes errors.

`Mock` (in all its flavours) uses a method called `_get_child_mock` to create these "sub-mocks" for attributes and return values. You can prevent your subclass being used for attributes by overriding this method. The signature is that it takes arbitrary keyword arguments (`**kwargs`) which are then passed onto the mock constructor:

> \>\>\> class Subclass(MagicMock): ... def <span id="get_child_mock">get_child_mock</span>(self, /, **kwargs): ... return MagicMock(**kwargs) ... \>\>\> mymock = Subclass() \>\>\> mymock.foo \<MagicMock name='mock.foo' id='...'\> \>\>\> assert isinstance(mymock, Subclass) \>\>\> assert not isinstance(mymock.foo, Subclass) \>\>\> assert not isinstance(mymock(), Subclass)

### Mocking imports with patch.dict

One situation where mocking can be hard is where you have a local import inside a function. These are harder to mock because they aren't using an object from the module namespace that we can patch out.

Generally local imports are to be avoided. They are sometimes done to prevent circular dependencies, for which there is *usually* a much better way to solve the problem (refactor the code) or to prevent "up front costs" by delaying the import. This can also be solved in better ways than an unconditional local import (store the module as a class or module attribute and only do the import on first use).

That aside there is a way to use `mock` to affect the results of an import. Importing fetches an *object* from the `sys.modules` dictionary. Note that it fetches an *object*, which need not be a module. Importing a module for the first time results in a module object being put in `sys.modules`, so usually when you import something you get a module back. This need not be the case however.

This means you can use `patch.dict` to *temporarily* put a mock in place in `sys.modules`. Any imports whilst this patch is active will fetch the mock. When the patch is complete (the decorated function exits, the with statement body is complete or `patcher.stop()` is called) then whatever was there previously will be restored safely.

Here's an example that mocks out the 'fooble' module.

> \>\>\> import sys \>\>\> mock = Mock() \>\>\> with patch.dict('sys.modules', {'fooble': mock}): ... import fooble ... fooble.blob() ... \<Mock name='mock.blob()' id='...'\> \>\>\> assert 'fooble' not in sys.modules \>\>\> mock.blob.assert_called_once_with()

As you can see the `import fooble` succeeds, but on exit there is no 'fooble' left in `sys.modules`.

This also works for the `from module import name` form:

> \>\>\> mock = Mock() \>\>\> with patch.dict('sys.modules', {'fooble': mock}): ... from fooble import blob ... blob.blip() ... \<Mock name='mock.blob.blip()' id='...'\> \>\>\> mock.blob.blip.assert_called_once_with()

With slightly more work you can also mock package imports:

> \>\>\> mock = Mock() \>\>\> modules = {'package': mock, 'package.module': mock.module} \>\>\> with patch.dict('sys.modules', modules): ... from package.module import fooble ... fooble() ... \<Mock name='mock.module.fooble()' id='...'\> \>\>\> mock.module.fooble.assert_called_once_with()

### Tracking order of calls and less verbose call assertions

The `Mock` class allows you to track the *order* of method calls on your mock objects through the `~Mock.method_calls` attribute. This doesn't allow you to track the order of calls between separate mock objects, however we can use `~Mock.mock_calls` to achieve the same effect.

Because mocks track calls to child mocks in `mock_calls`, and accessing an arbitrary attribute of a mock creates a child mock, we can create our separate mocks from a parent one. Calls to those child mock will then all be recorded, in order, in the `mock_calls` of the parent:

> \>\>\> manager = Mock() \>\>\> mock_foo = manager.foo \>\>\> mock_bar = manager.bar
>
> \>\>\> mock_foo.something() \<Mock name='mock.foo.something()' id='...'\> \>\>\> mock_bar.other.thing() \<Mock name='mock.bar.other.thing()' id='...'\>
>
> \>\>\> manager.mock_calls \[call.foo.something(), call.bar.other.thing()\]

We can then assert about the calls, including the order, by comparing with the `mock_calls` attribute on the manager mock:

> \>\>\> expected_calls = \[call.foo.something(), call.bar.other.thing()\] \>\>\> manager.mock_calls == expected_calls True

If `patch` is creating, and putting in place, your mocks then you can attach them to a manager mock using the `~Mock.attach_mock` method. After attaching calls will be recorded in `mock_calls` of the manager. :

    >>> manager = MagicMock()
    >>> with patch('mymodule.Class1') as MockClass1:
    ...     with patch('mymodule.Class2') as MockClass2:
    ...         manager.attach_mock(MockClass1, 'MockClass1')
    ...         manager.attach_mock(MockClass2, 'MockClass2')
    ...         MockClass1().foo()
    ...         MockClass2().bar()
    <MagicMock name='mock.MockClass1().foo()' id='...'>
    <MagicMock name='mock.MockClass2().bar()' id='...'>
    >>> manager.mock_calls
    [call.MockClass1(),
    call.MockClass1().foo(),
    call.MockClass2(),
    call.MockClass2().bar()]

If many calls have been made, but you're only interested in a particular sequence of them then an alternative is to use the `~Mock.assert_has_calls` method. This takes a list of calls (constructed with the `call` object). If that sequence of calls are in `~Mock.mock_calls` then the assert succeeds.

> \>\>\> m = MagicMock() \>\>\> m().foo().bar().baz() \<MagicMock name='mock().foo().bar().baz()' id='...'\> \>\>\> m.one().two().three() \<MagicMock name='mock.one().two().three()' id='...'\> \>\>\> calls = call.one().two().three().call_list() \>\>\> m.assert_has_calls(calls)

Even though the chained call `m.one().two().three()` aren't the only calls that have been made to the mock, the assert still succeeds.

Sometimes a mock may have several calls made to it, and you are only interested in asserting about *some* of those calls. You may not even care about the order. In this case you can pass `any_order=True` to `assert_has_calls`:

> \>\>\> m = MagicMock() \>\>\> m(1), m.two(2, 3), m.seven(7), m.fifty('50') (...) \>\>\> calls = \[call.fifty('50'), call(1), call.seven(7)\] \>\>\> m.assert_has_calls(calls, any_order=True)

### More complex argument matching

Using the same basic concept as `ANY` we can implement matchers to do more complex assertions on objects used as arguments to mocks.

Suppose we expect some object to be passed to a mock that by default compares equal based on object identity (which is the Python default for user defined classes). To use `~Mock.assert_called_with` we would need to pass in the exact same object. If we are only interested in some of the attributes of this object then we can create a matcher that will check these attributes for us.

You can see in this example how a 'standard' call to `assert_called_with` isn't sufficient:

> \>\>\> class Foo: ... def \_\_init\_\_(self, a, b): ... self.a, self.b = a, b ... \>\>\> mock = Mock(return_value=None) \>\>\> mock(Foo(1, 2)) \>\>\> mock.assert_called_with(Foo(1, 2)) Traceback (most recent call last): ... AssertionError: expected call not found. Expected: mock(\<\_\_main\_\_.Foo object at 0x...\>) Actual: mock(\<\_\_main\_\_.Foo object at 0x...\>)

A comparison function for our `Foo` class might look something like this:

> \>\>\> def compare(self, other): ... if not type(self) == type(other): ... return False ... if self.a != other.a: ... return False ... if self.b != other.b: ... return False ... return True ...

And a matcher object that can use comparison functions like this for its equality operation would look something like this:

> \>\>\> class Matcher: ... def \_\_init\_\_(self, compare, some_obj): ... self.compare = compare ... self.some_obj = some_obj ... def \_\_eq\_\_(self, other): ... return self.compare(self.some_obj, other) ...

Putting all this together:

> \>\>\> match_foo = Matcher(compare, Foo(1, 2)) \>\>\> mock.assert_called_with(match_foo)

The `Matcher` is instantiated with our compare function and the `Foo` object we want to compare against. In `assert_called_with` the `Matcher` equality method will be called, which compares the object the mock was called with against the one we created our matcher with. If they match then `assert_called_with` passes, and if they don't an `AssertionError` is raised:

> \>\>\> match_wrong = Matcher(compare, Foo(3, 4)) \>\>\> mock.assert_called_with(match_wrong) Traceback (most recent call last): ... AssertionError: Expected: ((\<Matcher object at 0x...\>,), {}) Called with: ((\<Foo object at 0x...\>,), {})

With a bit of tweaking you could have the comparison function raise the `AssertionError` directly and provide a more useful failure message.

As of version 1.5, the Python testing library [PyHamcrest](https://pyhamcrest.readthedocs.io/) provides similar functionality, that may be useful here, in the form of its equality matcher ([hamcrest.library.integration.match_equality](https://pyhamcrest.readthedocs.io/en/release-1.8/integration/#module-hamcrest.library.integration.match_equality)).

[^1]: There are also generator expressions and more [advanced uses](http://www.dabeaz.com/coroutines/index.html) of generators, but we aren't concerned about them here. A very good introduction to generators and how powerful they are is: [Generator Tricks for Systems Programmers](http://www.dabeaz.com/generators/).

[^2]: An exception to this rule are the non-callable mocks. Attributes use the callable variant because otherwise non-callable mocks couldn't have callable methods.

# `!unittest.mock` --- mock object library

<div class="module" data-synopsis="Mock object library.">

unittest.mock

</div>

<div class="currentmodule">

unittest.mock

</div>

<div class="versionadded">

3.3

</div>

**Source code:** `Lib/unittest/mock.py`

------------------------------------------------------------------------

`!unittest.mock` is a library for testing in Python. It allows you to replace parts of your system under test with mock objects and make assertions about how they have been used.

`!unittest.mock` provides a core `Mock` class removing the need to create a host of stubs throughout your test suite. After performing an action, you can make assertions about which methods / attributes were used and arguments they were called with. You can also specify return values and set needed attributes in the normal way.

Additionally, mock provides a `patch` decorator that handles patching module and class level attributes within the scope of a test, along with `sentinel` for creating unique objects. See the [quick guide](#quick-guide) for some examples of how to use `Mock`, `MagicMock` and `patch`.

Mock is designed for use with `unittest` and is based on the 'action -\> assertion' pattern instead of 'record -\> replay' used by many mocking frameworks.

There is a backport of `!unittest.mock` for earlier versions of Python, available as `mock` on PyPI.

## Quick Guide

<div class="testsetup">

class ProductionClass:  
def method(self, a, b, c):  
pass

class SomeClass:  
@staticmethod def static_method(args): return args

@classmethod def class_method(cls, args): return args

</div>

`Mock` and `MagicMock` objects create all attributes and methods as you access them and store details of how they have been used. You can configure them, to specify return values or limit what attributes are available, and then make assertions about how they have been used:

> \>\>\> from unittest.mock import MagicMock \>\>\> thing = ProductionClass() \>\>\> thing.method = MagicMock(return_value=3) \>\>\> thing.method(3, 4, 5, key='value') 3 \>\>\> thing.method.assert_called_with(3, 4, 5, key='value')

`~Mock.side_effect` allows you to perform side effects, including raising an exception when a mock is called:

> \>\>\> from unittest.mock import Mock \>\>\> mock = Mock(side_effect=KeyError('foo')) \>\>\> mock() Traceback (most recent call last): ... KeyError: 'foo'
>
> \>\>\> values = {'a': 1, 'b': 2, 'c': 3} \>\>\> def side_effect(arg): ... return values\[arg\] ... \>\>\> mock.side_effect = side_effect \>\>\> mock('a'), mock('b'), mock('c') (1, 2, 3) \>\>\> mock.side_effect = \[5, 4, 3, 2, 1\] \>\>\> mock(), mock(), mock() (5, 4, 3)

Mock has many other ways you can configure it and control its behaviour. For example the *spec* argument configures the mock to take its specification from another object. Attempting to access attributes or methods on the mock that don't exist on the spec will fail with an `AttributeError`.

The `patch` decorator / context manager makes it easy to mock classes or objects in a module under test. The object you specify will be replaced with a mock (or other object) during the test and restored when the test ends:

    >>> from unittest.mock import patch
    >>> @patch('module.ClassName2')
    ... @patch('module.ClassName1')
    ... def test(MockClass1, MockClass2):
    ...     module.ClassName1()
    ...     module.ClassName2()
    ...     assert MockClass1 is module.ClassName1
    ...     assert MockClass2 is module.ClassName2
    ...     assert MockClass1.called
    ...     assert MockClass2.called
    ...
    >>> test()

> [!NOTE]
> When you nest patch decorators the mocks are passed in to the decorated function in the same order they applied (the normal *Python* order that decorators are applied). This means from the bottom up, so in the example above the mock for `module.ClassName1` is passed in first.
>
> With `patch` it matters that you patch objects in the namespace where they are looked up. This is normally straightforward, but for a quick guide read `where to patch <where-to-patch>`.

As well as a decorator `patch` can be used as a context manager in a with statement:

> \>\>\> with patch.object(ProductionClass, 'method', return_value=None) as mock_method: ... thing = ProductionClass() ... thing.method(1, 2, 3) ... \>\>\> mock_method.assert_called_once_with(1, 2, 3)

There is also `patch.dict` for setting values in a dictionary just during a scope and restoring the dictionary to its original state when the test ends:

> \>\>\> foo = {'key': 'value'} \>\>\> original = foo.copy() \>\>\> with patch.dict(foo, {'newkey': 'newvalue'}, clear=True): ... assert foo == {'newkey': 'newvalue'} ... \>\>\> assert foo == original

Mock supports the mocking of Python `magic methods <magic-methods>`. The easiest way of using magic methods is with the `MagicMock` class. It allows you to do things like:

> \>\>\> mock = MagicMock() \>\>\> mock.\_\_str\_\_.return_value = 'foobarbaz' \>\>\> str(mock) 'foobarbaz' \>\>\> mock.\_\_str\_\_.assert_called_with()

Mock allows you to assign functions (or other Mock instances) to magic methods and they will be called appropriately. The `MagicMock` class is just a Mock variant that has all of the magic methods pre-created for you (well, all the useful ones anyway).

The following is an example of using magic methods with the ordinary Mock class:

> \>\>\> mock = Mock() \>\>\> mock.\_\_str\_\_ = Mock(return_value='wheeeeee') \>\>\> str(mock) 'wheeeeee'

For ensuring that the mock objects in your tests have the same api as the objects they are replacing, you can use `auto-speccing <auto-speccing>`. Auto-speccing can be done through the *autospec* argument to patch, or the `create_autospec` function. Auto-speccing creates mock objects that have the same attributes and methods as the objects they are replacing, and any functions and methods (including constructors) have the same call signature as the real object.

This ensures that your mocks will fail in the same way as your production code if they are used incorrectly:

> \>\>\> from unittest.mock import create_autospec \>\>\> def function(a, b, c): ... pass ... \>\>\> mock_function = create_autospec(function, return_value='fishy') \>\>\> mock_function(1, 2, 3) 'fishy' \>\>\> mock_function.assert_called_once_with(1, 2, 3) \>\>\> mock_function('wrong arguments') Traceback (most recent call last): ... TypeError: missing a required argument: 'b'

`create_autospec` can also be used on classes, where it copies the signature of the `__init__` method, and on callable objects where it copies the signature of the `__call__` method.

## The Mock Class

<div class="testsetup">

import asyncio import inspect import unittest import threading from unittest.mock import sentinel, DEFAULT, ANY from unittest.mock import patch, call, Mock, MagicMock, PropertyMock, AsyncMock from unittest.mock import ThreadingMock from unittest.mock import mock_open

</div>

`Mock` is a flexible mock object intended to replace the use of stubs and test doubles throughout your code. Mocks are callable and create attributes as new mocks when you access them[^1]. Accessing the same attribute will always return the same mock. Mocks record how you use them, allowing you to make assertions about what your code has done to them.

`MagicMock` is a subclass of `Mock` with all the magic methods pre-created and ready to use. There are also non-callable variants, useful when you are mocking out objects that aren't callable: `NonCallableMock` and `NonCallableMagicMock`

The `patch` decorators makes it easy to temporarily replace classes in a particular module with a `Mock` object. By default `patch` will create a `MagicMock` for you. You can specify an alternative class of `Mock` using the *new_callable* argument to `patch`.

<div class="Mock(spec=None, side_effect=None, return_value=DEFAULT, wraps=None, name=None, spec_set=None, unsafe=False, **kwargs)">

Create a new `Mock` object. `Mock` takes several optional arguments that specify the behaviour of the Mock object:

- *spec*: This can be either a list or tuple of strings, or an existing object (a class or instance) that acts as the specification for the mock object. If you pass in an object then a list of strings is formed by calling dir on the object (excluding unsupported magic attributes and methods). Accessing any attribute not in this list will raise an `AttributeError`.

  If *spec* is an object (rather than a list of strings) then `~object.__class__` returns the class of the spec object. This allows mocks to pass `isinstance` tests.

  <div class="versionchanged">

  3.16 `dir` now works for a mock created with a tuple *spec*.

  </div>

- *spec_set*: A stricter variant of *spec*. If used, attempting to *set* or get an attribute on the mock that isn't on the object passed as *spec_set* will raise an `AttributeError`.

- *side_effect*: A function to be called whenever the Mock is called. See the `~Mock.side_effect` attribute. Useful for raising exceptions or dynamically changing return values. The function is called with the same arguments as the mock, and unless it returns `DEFAULT`, the return value of this function is used as the return value.

  Alternatively *side_effect* can be an exception class or instance. In this case the exception will be raised when the mock is called.

  If *side_effect* is an iterable then each call to the mock will return the next value from the iterable.

  A *side_effect* can be cleared by setting it to `None`.

- *return_value*: The value returned when the mock is called. By default this is a new Mock (created on first access). See the `return_value` attribute.

- *unsafe*: By default, accessing any attribute whose name starts with *assert*, *assret*, *asert*, *aseert* or *assrt* will raise an `AttributeError`. Passing `unsafe=True` will allow access to these attributes.

  <div class="versionadded">

  3.5

  </div>

- *wraps*: Item for the mock object to wrap. If *wraps* is not `None` then calling the Mock will pass the call through to the wrapped object (returning the real result). Attribute access on the mock will return a Mock object that wraps the corresponding attribute of the wrapped object (so attempting to access an attribute that doesn't exist will raise an `AttributeError`).

  If the mock has an explicit *return_value* set then calls are not passed to the wrapped object and the *return_value* is returned instead.

- *name*: If the mock has a name then it will be used in the repr of the mock. This can be useful for debugging. The name is propagated to child mocks.

Mocks can also be called with arbitrary keyword arguments. These will be used to set attributes on the mock after it is created. See the `configure_mock` method for details.

<div class="method">

assert_called()

Assert that the mock was called at least once.

> \>\>\> mock = Mock() \>\>\> mock.method() \<Mock name='mock.method()' id='...'\> \>\>\> mock.method.assert_called()

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

assert_called_once()

Assert that the mock was called exactly once.

> \>\>\> mock = Mock() \>\>\> mock.method() \<Mock name='mock.method()' id='...'\> \>\>\> mock.method.assert_called_once() \>\>\> mock.method() \<Mock name='mock.method()' id='...'\> \>\>\> mock.method.assert_called_once() Traceback (most recent call last): ... AssertionError: Expected 'method' to have been called once. Called 2 times. Calls: \[call(), call()\].

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

assert_called_with(*args,*\*kwargs)

This method is a convenient way of asserting that the last call has been made in a particular way:

> \>\>\> mock = Mock() \>\>\> mock.method(1, 2, 3, test='wow') \<Mock name='mock.method()' id='...'\> \>\>\> mock.method.assert_called_with(1, 2, 3, test='wow')

</div>

<div class="method">

assert_called_once_with(*args,*\*kwargs)

Assert that the mock was called exactly once and that call was with the specified arguments.

> \>\>\> mock = Mock(return_value=None) \>\>\> mock('foo', bar='baz') \>\>\> mock.assert_called_once_with('foo', bar='baz') \>\>\> mock('other', bar='values') \>\>\> mock.assert_called_once_with('other', bar='values') Traceback (most recent call last): ... AssertionError: Expected 'mock' to be called once. Called 2 times. Calls: \[call('foo', bar='baz'), call('other', bar='values')\].

</div>

<div class="method">

assert_any_call(*args,*\*kwargs)

Assert the mock has been called with the specified arguments.

The assert passes if the mock has *ever* been called, unlike `assert_called_with` and `assert_called_once_with` that only pass if the call is the most recent one, and in the case of `assert_called_once_with` it must also be the only call.

> \>\>\> mock = Mock(return_value=None) \>\>\> mock(1, 2, arg='thing') \>\>\> mock('some', 'thing', 'else') \>\>\> mock.assert_any_call(1, 2, arg='thing')

</div>

<div class="method">

assert_has_calls(calls, any_order=False)

Assert the mock has been called with the specified calls. The `mock_calls` list is checked for the calls.

If *any_order* is false then the calls must be sequential. There can be extra calls before or after the specified calls.

If *any_order* is true then the calls can be in any order, but they must all appear in `mock_calls`.

> \>\>\> mock = Mock(return_value=None) \>\>\> mock(1) \>\>\> mock(2) \>\>\> mock(3) \>\>\> mock(4) \>\>\> calls = \[call(2), call(3)\] \>\>\> mock.assert_has_calls(calls) \>\>\> calls = \[call(4), call(2), call(3)\] \>\>\> mock.assert_has_calls(calls, any_order=True)

</div>

<div class="method">

assert_not_called()

Assert the mock was never called.

> \>\>\> m = Mock() \>\>\> m.hello.assert_not_called() \>\>\> obj = m.hello() \>\>\> m.hello.assert_not_called() Traceback (most recent call last): ... AssertionError: Expected 'hello' to not have been called. Called 1 times. Calls: \[call()\].

<div class="versionadded">

3.5

</div>

</div>

<div class="method">

reset_mock(\*, return_value=False, side_effect=False)

The reset_mock method resets all the call attributes on a mock object:

<div class="doctest">

\>\>\> mock = Mock(return_value=None) \>\>\> mock('hello') \>\>\> mock.called True \>\>\> mock.reset_mock() \>\>\> mock.called False

</div>

This can be useful where you want to make a series of assertions that reuse the same object.

*return_value* parameter when set to `True` resets `return_value`:

<div class="doctest">

\>\>\> mock = Mock(return_value=5) \>\>\> mock('hello') 5 \>\>\> mock.reset_mock(return_value=True) \>\>\> mock('hello') \# doctest: +ELLIPSIS \<Mock name='mock()' id='...'\>

</div>

*side_effect* parameter when set to `True` resets `side_effect`:

<div class="doctest">

\>\>\> mock = Mock(side_effect=ValueError) \>\>\> mock('hello') Traceback (most recent call last): ... ValueError \>\>\> mock.reset_mock(side_effect=True) \>\>\> mock('hello') \# doctest: +ELLIPSIS \<Mock name='mock()' id='...'\>

</div>

Note that `reset_mock` *doesn't* clear the `return_value`, `side_effect` or any child attributes you have set using normal assignment by default.

Child mocks are reset as well.

<div class="versionchanged">

3.6 Added two keyword-only arguments to the reset_mock function.

</div>

</div>

<div class="method">

mock_add_spec(spec, spec_set=False)

Add a spec to a mock. *spec* can either be an object or a list or tuple of strings. Only attributes on the *spec* can be fetched as attributes from the mock.

If *spec_set* is true then only attributes on the spec can be set.

</div>

<div class="method">

attach_mock(mock, attribute)

Attach a mock as an attribute of this one, replacing its name and parent. Calls to the attached mock will be recorded in the `method_calls` and `mock_calls` attributes of this one.

</div>

<div class="method">

configure_mock(\*\*kwargs)

Set attributes on the mock through keyword arguments.

Attributes plus return values and side effects can be set on child mocks using standard dot notation and unpacking a dictionary in the method call:

> \>\>\> mock = Mock() \>\>\> attrs = {'method.return_value': 3, 'other.side_effect': KeyError} \>\>\> mock.configure_mock(\*\*attrs) \>\>\> mock.method() 3 \>\>\> mock.other() Traceback (most recent call last): ... KeyError

The same thing can be achieved in the constructor call to mocks:

> \>\>\> attrs = {'method.return_value': 3, 'other.side_effect': KeyError} \>\>\> mock = Mock(some_attribute='eggs', \*\*attrs) \>\>\> mock.some_attribute 'eggs' \>\>\> mock.method() 3 \>\>\> mock.other() Traceback (most recent call last): ... KeyError

`configure_mock` exists to make it easier to do configuration after the mock has been created.

</div>

<div class="method">

\_\_dir\_\_()

`Mock` objects limit the results of `dir(some_mock)` to useful results. For mocks with a *spec* this includes all the permitted attributes for the mock.

See `FILTER_DIR` for what this filtering does, and how to switch it off.

</div>

<div class="method">

<span id="get_child_mock">get_child_mock</span>(\*\*kw)

Create the child mocks for attributes and return value. By default child mocks will be the same type as the parent. Subclasses of Mock may want to override this to customize the way child mocks are made.

For non-callable mocks the callable variant will be used (rather than any custom subclass).

</div>

<div class="attribute">

called

A boolean representing whether or not the mock object has been called:

> \>\>\> mock = Mock(return_value=None) \>\>\> mock.called False \>\>\> mock() \>\>\> mock.called True

</div>

<div class="attribute">

call_count

An integer telling you how many times the mock object has been called:

> \>\>\> mock = Mock(return_value=None) \>\>\> mock.call_count 0 \>\>\> mock() \>\>\> mock() \>\>\> mock.call_count 2

</div>

<div class="attribute">

return_value

Set this to configure the value returned by calling the mock:

> \>\>\> mock = Mock() \>\>\> mock.return_value = 'fish' \>\>\> mock() 'fish'

The default return value is a mock object and you can configure it in the normal way:

> \>\>\> mock = Mock() \>\>\> mock.return_value.attribute = sentinel.Attribute \>\>\> mock.return_value() \<Mock name='mock()()' id='...'\> \>\>\> mock.return_value.assert_called_with()

`return_value` can also be set in the constructor:

> \>\>\> mock = Mock(return_value=3) \>\>\> mock.return_value 3 \>\>\> mock() 3

</div>

<div class="attribute">

side_effect

This can either be a function to be called when the mock is called, an iterable or an exception (class or instance) to be raised.

If you pass in a function it will be called with same arguments as the mock and unless the function returns the `DEFAULT` singleton the call to the mock will then return whatever the function returns. If the function returns `DEFAULT` then the mock will return its normal value (from the `return_value`).

If you pass in an iterable, it is used to retrieve an iterator which must yield a value on every call. This value can either be an exception instance to be raised, or a value to be returned from the call to the mock (`DEFAULT` handling is identical to the function case).

An example of a mock that raises an exception (to test exception handling of an API):

> \>\>\> mock = Mock() \>\>\> mock.side_effect = Exception('Boom!') \>\>\> mock() Traceback (most recent call last): ... Exception: Boom!

Using `side_effect` to return a sequence of values:

> \>\>\> mock = Mock() \>\>\> mock.side_effect = \[3, 2, 1\] \>\>\> mock(), mock(), mock() (3, 2, 1)

Using a callable:

> \>\>\> mock = Mock(return_value=3) \>\>\> def side_effect(*args,*\*kwargs): ... return DEFAULT ... \>\>\> mock.side_effect = side_effect \>\>\> mock() 3

`side_effect` can be set in the constructor. Here's an example that adds one to the value the mock is called with and returns it:

> \>\>\> side_effect = lambda value: value + 1 \>\>\> mock = Mock(side_effect=side_effect) \>\>\> mock(3) 4 \>\>\> mock(-8) -7

Setting `side_effect` to `None` clears it:

> \>\>\> m = Mock(side_effect=KeyError, return_value=3) \>\>\> m() Traceback (most recent call last): ... KeyError \>\>\> m.side_effect = None \>\>\> m() 3

</div>

<div class="attribute">

call_args

This is either `None` (if the mock hasn't been called), or the arguments that the mock was last called with. This will be in the form of a tuple: the first member, which can also be accessed through the `args` property, is any positional arguments the mock was called with (or an empty tuple) and the second member, which can also be accessed through the `kwargs` property, is any keyword arguments (or an empty dictionary).

> \>\>\> mock = Mock(return_value=None) \>\>\> print(mock.call_args) None \>\>\> mock() \>\>\> mock.call_args call() \>\>\> mock.call_args == () True \>\>\> mock(3, 4) \>\>\> mock.call_args call(3, 4) \>\>\> mock.call_args == ((3, 4),) True \>\>\> mock.call_args.args (3, 4) \>\>\> mock.call_args.kwargs {} \>\>\> mock(3, 4, 5, key='fish', next='w00t!') \>\>\> mock.call_args call(3, 4, 5, key='fish', next='w00t!') \>\>\> mock.call_args.args (3, 4, 5) \>\>\> mock.call_args.kwargs {'key': 'fish', 'next': 'w00t!'}

`call_args`, along with members of the lists `call_args_list`, `method_calls` and `mock_calls` are `call` objects. These are tuples, so they can be unpacked to get at the individual arguments and make more complex assertions. See `calls as tuples <calls-as-tuples>`.

<div class="versionchanged">

3.8 Added `args` and `kwargs` properties.

</div>

</div>

<div class="attribute">

call_args_list

This is a list of all the calls made to the mock object in sequence (so the length of the list is the number of times it has been called). Before any calls have been made it is an empty list. The `call` object can be used for conveniently constructing lists of calls to compare with `call_args_list`.

> \>\>\> mock = Mock(return_value=None) \>\>\> mock() \>\>\> mock(3, 4) \>\>\> mock(key='fish', next='w00t!') \>\>\> mock.call_args_list \[call(), call(3, 4), call(key='fish', next='w00t!')\] \>\>\> expected = \[(), ((3, 4),), ({'key': 'fish', 'next': 'w00t!'},)\] \>\>\> mock.call_args_list == expected True

Members of `call_args_list` are `call` objects. These can be unpacked as tuples to get at the individual arguments. See `calls as tuples <calls-as-tuples>`.

</div>

<div class="attribute">

method_calls

As well as tracking calls to themselves, mocks also track calls to methods and attributes, and *their* methods and attributes:

> \>\>\> mock = Mock() \>\>\> mock.method() \<Mock name='mock.method()' id='...'\> \>\>\> mock.property.method.attribute() \<Mock name='mock.property.method.attribute()' id='...'\> \>\>\> mock.method_calls \[call.method(), call.property.method.attribute()\]

Members of `method_calls` are `call` objects. These can be unpacked as tuples to get at the individual arguments. See `calls as tuples <calls-as-tuples>`.

</div>

<div class="attribute">

mock_calls

`mock_calls` records *all* calls to the mock object, its methods, magic methods *and* return value mocks.

> \>\>\> mock = MagicMock() \>\>\> result = mock(1, 2, 3) \>\>\> mock.first(a=3) \<MagicMock name='mock.first()' id='...'\> \>\>\> mock.second() \<MagicMock name='mock.second()' id='...'\> \>\>\> int(mock) 1 \>\>\> result(1) \<MagicMock name='mock()()' id='...'\> \>\>\> expected = \[call(1, 2, 3), call.first(a=3), call.second(), ... call.\_\_int\_\_(), call()(1)\] \>\>\> mock.mock_calls == expected True

Members of `mock_calls` are `call` objects. These can be unpacked as tuples to get at the individual arguments. See `calls as tuples <calls-as-tuples>`.

> [!NOTE]
> The way `mock_calls` are recorded means that where nested calls are made, the parameters of ancestor calls are not recorded and so will always compare equal:
>
> > \>\>\> mock = MagicMock() \>\>\> mock.top(a=3).bottom() \<MagicMock name='mock.top().bottom()' id='...'\> \>\>\> mock.mock_calls \[call.top(a=3), call.top().bottom()\] \>\>\> mock.mock_calls\[-1\] == call.top(a=-1).bottom() True

</div>

<div class="attribute">

\_\_class\_\_

Normally the `!__class__` attribute of an object will return its type. For a mock object with a `!spec`, `!__class__` returns the spec class instead. This allows mock objects to pass `isinstance` tests for the object they are replacing / masquerading as:

> \>\>\> mock = Mock(spec=3) \>\>\> isinstance(mock, int) True

`!__class__` is assignable to, this allows a mock to pass an `isinstance` check without forcing you to use a spec:

> \>\>\> mock = Mock() \>\>\> mock.\_\_class\_\_ = dict \>\>\> isinstance(mock, dict) True

</div>

</div>

<div class="NonCallableMock(spec=None, wraps=None, name=None, spec_set=None, **kwargs)">

A non-callable version of `Mock`. The constructor parameters have the same meaning of `Mock`, with the exception of *return_value* and *side_effect* which have no meaning on a non-callable mock.

</div>

Mock objects that use a class or an instance as a `!spec` or `!spec_set` are able to pass `isinstance` tests:

> \>\>\> mock = Mock(spec=SomeClass) \>\>\> isinstance(mock, SomeClass) True \>\>\> mock = Mock(spec_set=SomeClass()) \>\>\> isinstance(mock, SomeClass) True

The `Mock` classes have support for mocking magic methods. See `magic
methods <magic-methods>` for the full details.

The mock classes and the `patch` decorators all take arbitrary keyword arguments for configuration. For the `patch` decorators the keywords are passed to the constructor of the mock being created. The keyword arguments are for configuring attributes of the mock:

> \>\>\> m = MagicMock(attribute=3, other='fish') \>\>\> m.attribute 3 \>\>\> m.other 'fish'

The return value and side effect of child mocks can be set in the same way, using dotted notation. As you can't use dotted names directly in a call you have to create a dictionary and unpack it using `**`:

> \>\>\> attrs = {'method.return_value': 3, 'other.side_effect': KeyError} \>\>\> mock = Mock(some_attribute='eggs', \*\*attrs) \>\>\> mock.some_attribute 'eggs' \>\>\> mock.method() 3 \>\>\> mock.other() Traceback (most recent call last): ... KeyError

A callable mock which was created with a *spec* (or a *spec_set*) will introspect the specification object's signature when matching calls to the mock. Therefore, it can match the actual call's arguments regardless of whether they were passed positionally or by name:

    >>> def f(a, b, c): pass
    ...
    >>> mock = Mock(spec=f)
    >>> mock(1, 2, c=3)
    <Mock name='mock()' id='140161580456576'>
    >>> mock.assert_called_with(1, 2, 3)
    >>> mock.assert_called_with(a=1, b=2, c=3)

This applies to `~Mock.assert_called_with`, `~Mock.assert_called_once_with`, `~Mock.assert_has_calls` and `~Mock.assert_any_call`. When `auto-speccing`, it will also apply to method calls on the mock object.

<div class="versionchanged">

3.4 Added signature introspection on specced and autospecced mock objects.

</div>

<div class="PropertyMock(*args, **kwargs)">

A mock intended to be used as a `property`, or other `descriptor`, on a class. `PropertyMock` provides `~object.__get__` and `~object.__set__` methods so you can specify a return value when it is fetched.

Fetching a `PropertyMock` instance from an object calls the mock, with no args. Setting it calls the mock with the value being set. :

    >>> class Foo:
    ...     @property
    ...     def foo(self):
    ...         return 'something'
    ...     @foo.setter
    ...     def foo(self, value):
    ...         pass
    ...
    >>> with patch('__main__.Foo.foo', new_callable=PropertyMock) as mock_foo:
    ...     mock_foo.return_value = 'mockity-mock'
    ...     this_foo = Foo()
    ...     print(this_foo.foo)
    ...     this_foo.foo = 6
    ...
    mockity-mock
    >>> mock_foo.mock_calls
    [call(), call(6)]

</div>

Because of the way mock attributes are stored you can't directly attach a `PropertyMock` to a mock object. Instead you can attach it to the mock type object:

    >>> m = MagicMock()
    >>> p = PropertyMock(return_value=3)
    >>> type(m).foo = p
    >>> m.foo
    3
    >>> p.assert_called_once_with()

> [!CAUTION]
> If an `AttributeError` is raised by `PropertyMock`, it will be interpreted as a missing descriptor and `~object.__getattr__` will be called on the parent mock:
>
>     >>> m = MagicMock()
>     >>> no_attribute = PropertyMock(side_effect=AttributeError)
>     >>> type(m).my_property = no_attribute
>     >>> m.my_property
>     <MagicMock name='mock.my_property' id='140165240345424'>
>
> See `~object.__getattr__` for details.

<div class="AsyncMock(spec=None, side_effect=None, return_value=DEFAULT, wraps=None, name=None, spec_set=None, unsafe=False, **kwargs)">

An asynchronous version of `MagicMock`. The `AsyncMock` object will behave so the object is recognized as an async function, and the result of a call is an awaitable.

> \>\>\> mock = AsyncMock() \>\>\> inspect.iscoroutinefunction(mock) True \>\>\> inspect.isawaitable(mock()) \# doctest: +SKIP True

The result of `mock()` is an async function which will have the outcome of `side_effect` or `return_value` after it has been awaited:

- if `side_effect` is a function, the async function will return the result of that function,
- if `side_effect` is an exception, the async function will raise the exception,
- if `side_effect` is an iterable, the async function will return the next value of the iterable, however, if the sequence of result is `exhausted`, `StopAsyncIteration` is raised immediately,
- if `side_effect` is not defined, the async function will return the value defined by `return_value`, hence, by default, the async function returns a new `AsyncMock` object.

Setting the *spec* of a `Mock` or `MagicMock` to an async function will result in a coroutine object being returned after calling.

> \>\>\> async def async_func(): pass ... \>\>\> mock = MagicMock(async_func) \>\>\> mock \<MagicMock spec='function' id='...'\> \>\>\> mock() \# doctest: +SKIP \<coroutine object AsyncMockMixin.\_mock_call at ...\>

Setting the *spec* of a `Mock`, `MagicMock`, or `AsyncMock` to a class with asynchronous and synchronous functions will automatically detect the synchronous functions and set them as `MagicMock` (if the parent mock is `AsyncMock` or `MagicMock`) or `Mock` (if the parent mock is `Mock`). All asynchronous functions will be `AsyncMock`.

\>\>\> class ExampleClass: ... def sync_foo(): ... pass ... async def async_foo(): ... pass ... \>\>\> a_mock = AsyncMock(ExampleClass) \>\>\> a_mock.sync_foo \<MagicMock name='mock.sync_foo' id='...'\> \>\>\> a_mock.async_foo \<AsyncMock name='mock.async_foo' id='...'\> \>\>\> mock = Mock(ExampleClass) \>\>\> mock.sync_foo \<Mock name='mock.sync_foo' id='...'\> \>\>\> mock.async_foo \<AsyncMock name='mock.async_foo' id='...'\>

<div class="versionadded">

3.8

</div>

<div class="method">

assert_awaited()

Assert that the mock was awaited at least once. Note that this is separate from the object having been called, the `await` keyword must be used:

> \>\>\> mock = AsyncMock() \>\>\> async def main(coroutine_mock): ... await coroutine_mock ... \>\>\> coroutine_mock = mock() \>\>\> mock.called True \>\>\> mock.assert_awaited() Traceback (most recent call last): ... AssertionError: Expected mock to have been awaited. \>\>\> asyncio.run(main(coroutine_mock)) \>\>\> mock.assert_awaited()

</div>

<div class="method">

assert_awaited_once()

Assert that the mock was awaited exactly once.

> \>\>\> mock = AsyncMock() \>\>\> async def main(): ... await mock() ... \>\>\> asyncio.run(main()) \>\>\> mock.assert_awaited_once() \>\>\> asyncio.run(main()) \>\>\> mock.assert_awaited_once() Traceback (most recent call last): ... AssertionError: Expected mock to have been awaited once. Awaited 2 times.

</div>

<div class="method">

assert_awaited_with(*args,*\*kwargs)

Assert that the last await was with the specified arguments.

> \>\>\> mock = AsyncMock() \>\>\> async def main(*args,kwargs): ... await mock(*args, \*\*kwargs) ... \>\>\> asyncio.run(main('foo', bar='bar')) \>\>\> mock.assert_awaited_with('foo', bar='bar') \>\>\> mock.assert_awaited_with('other') Traceback (most recent call last): ... AssertionError: expected await not found. Expected: mock('other') Actual: mock('foo', bar='bar')

</div>

<div class="method">

assert_awaited_once_with(*args,*\*kwargs)

Assert that the mock was awaited exactly once and with the specified arguments.

> \>\>\> mock = AsyncMock() \>\>\> async def main(*args,kwargs): ... await mock(*args, \*\*kwargs) ... \>\>\> asyncio.run(main('foo', bar='bar')) \>\>\> mock.assert_awaited_once_with('foo', bar='bar') \>\>\> asyncio.run(main('foo', bar='bar')) \>\>\> mock.assert_awaited_once_with('foo', bar='bar') Traceback (most recent call last): ... AssertionError: Expected mock to have been awaited once. Awaited 2 times.

</div>

<div class="method">

assert_any_await(*args,*\*kwargs)

Assert the mock has ever been awaited with the specified arguments.

> \>\>\> mock = AsyncMock() \>\>\> async def main(*args,kwargs): ... await mock(*args, \*\*kwargs) ... \>\>\> asyncio.run(main('foo', bar='bar')) \>\>\> asyncio.run(main('hello')) \>\>\> mock.assert_any_await('foo', bar='bar') \>\>\> mock.assert_any_await('other') Traceback (most recent call last): ... AssertionError: mock('other') await not found

</div>

<div class="method">

assert_has_awaits(calls, any_order=False)

Assert the mock has been awaited with the specified calls. The `await_args_list` list is checked for the awaits.

If *any_order* is false then the awaits must be sequential. There can be extra calls before or after the specified awaits.

If *any_order* is true then the awaits can be in any order, but they must all appear in `await_args_list`.

> \>\>\> mock = AsyncMock() \>\>\> async def main(*args,kwargs): ... await mock(*args, \*\*kwargs) ... \>\>\> calls = \[call("foo"), call("bar")\] \>\>\> mock.assert_has_awaits(calls) Traceback (most recent call last): ... AssertionError: Awaits not found. Expected: \[call('foo'), call('bar')\] Actual: \[\] \>\>\> asyncio.run(main('foo')) \>\>\> asyncio.run(main('bar')) \>\>\> mock.assert_has_awaits(calls)

</div>

<div class="method">

assert_not_awaited()

Assert that the mock was never awaited.

> \>\>\> mock = AsyncMock() \>\>\> mock.assert_not_awaited()

</div>

<div class="method">

reset_mock(*args,*\*kwargs)

See `Mock.reset_mock`. Also sets `await_count` to 0, `await_args` to None, and clears the `await_args_list`.

</div>

<div class="attribute">

await_count

An integer keeping track of how many times the mock object has been awaited.

> \>\>\> mock = AsyncMock() \>\>\> async def main(): ... await mock() ... \>\>\> asyncio.run(main()) \>\>\> mock.await_count 1 \>\>\> asyncio.run(main()) \>\>\> mock.await_count 2

</div>

<div class="attribute">

await_args

This is either `None` (if the mock hasn’t been awaited), or the arguments that the mock was last awaited with. Functions the same as `Mock.call_args`.

> \>\>\> mock = AsyncMock() \>\>\> async def main(*args): ... await mock(*args) ... \>\>\> mock.await_args \>\>\> asyncio.run(main('foo')) \>\>\> mock.await_args call('foo') \>\>\> asyncio.run(main('bar')) \>\>\> mock.await_args call('bar')

</div>

<div class="attribute">

await_args_list

This is a list of all the awaits made to the mock object in sequence (so the length of the list is the number of times it has been awaited). Before any awaits have been made it is an empty list.

> \>\>\> mock = AsyncMock() \>\>\> async def main(*args): ... await mock(*args) ... \>\>\> mock.await_args_list \[\] \>\>\> asyncio.run(main('foo')) \>\>\> mock.await_args_list \[call('foo')\] \>\>\> asyncio.run(main('bar')) \>\>\> mock.await_args_list \[call('foo'), call('bar')\]

</div>

</div>

<div class="ThreadingMock(spec=None, side_effect=None, return_value=DEFAULT, wraps=None, name=None, spec_set=None, unsafe=False, *, timeout=UNSET, **kwargs)">

A version of `MagicMock` for multithreading tests. The `ThreadingMock` object provides extra methods to wait for a call to be invoked, rather than assert on it immediately.

The default timeout is specified by the `timeout` argument, or if unset by the `ThreadingMock.DEFAULT_TIMEOUT` attribute, which defaults to blocking (`None`).

You can configure the global default timeout by setting `ThreadingMock.DEFAULT_TIMEOUT`.

<div class="method">

wait_until_called(\*, timeout=UNSET)

Waits until the mock is called.

If a timeout was passed at the creation of the mock or if a timeout argument is passed to this function, the function raises an `AssertionError` if the call is not performed in time.

> \>\>\> mock = ThreadingMock() \>\>\> thread = threading.Thread(target=mock) \>\>\> thread.start() \>\>\> mock.wait_until_called(timeout=1) \>\>\> thread.join()

</div>

<div class="method">

wait_until_any_call_with(*args,*\*kwargs)

Waits until the mock is called with the specified arguments.

If a timeout was passed at the creation of the mock the function raises an `AssertionError` if the call is not performed in time.

> \>\>\> mock = ThreadingMock() \>\>\> thread = threading.Thread(target=mock, args=("arg1", "arg2",), kwargs={"arg": "thing"}) \>\>\> thread.start() \>\>\> mock.wait_until_any_call_with("arg1", "arg2", arg="thing") \>\>\> thread.join()

</div>

<div class="attribute">

DEFAULT_TIMEOUT

Global default timeout in seconds to create instances of `ThreadingMock`.

</div>

<div class="versionadded">

3.13

</div>

</div>

### Calling

Mock objects are callable. The call will return the value set as the `~Mock.return_value` attribute. The default return value is a new Mock object; it is created the first time the return value is accessed (either explicitly or by calling the Mock) - but it is stored and the same one returned each time.

Calls made to the object will be recorded in the attributes like `~Mock.call_args` and `~Mock.call_args_list`.

If `~Mock.side_effect` is set then it will be called after the call has been recorded, so if `!side_effect` raises an exception the call is still recorded.

The simplest way to make a mock raise an exception when called is to make `~Mock.side_effect` an exception class or instance:

> \>\>\> m = MagicMock(side_effect=IndexError) \>\>\> m(1, 2, 3) Traceback (most recent call last): ... IndexError \>\>\> m.mock_calls \[call(1, 2, 3)\] \>\>\> m.side_effect = KeyError('Bang!') \>\>\> m('two', 'three', 'four') Traceback (most recent call last): ... KeyError: 'Bang!' \>\>\> m.mock_calls \[call(1, 2, 3), call('two', 'three', 'four')\]

If `~Mock.side_effect` is a function then whatever that function returns is what calls to the mock return. The `!side_effect` function is called with the same arguments as the mock. This allows you to vary the return value of the call dynamically, based on the input:

> \>\>\> def side_effect(value): ... return value + 1 ... \>\>\> m = MagicMock(side_effect=side_effect) \>\>\> m(1) 2 \>\>\> m(2) 3 \>\>\> m.mock_calls \[call(1), call(2)\]

If you want the mock to still return the default return value (a new mock), or any set return value, then there are two ways of doing this. Either return `~Mock.return_value` from inside `~Mock.side_effect`, or return `DEFAULT`:

> \>\>\> m = MagicMock() \>\>\> def side_effect(*args,kwargs): ... return m.return_value ... \>\>\> m.side_effect = side_effect \>\>\> m.return_value = 3 \>\>\> m() 3 \>\>\> def side_effect(*args, \*\*kwargs): ... return DEFAULT ... \>\>\> m.side_effect = side_effect \>\>\> m() 3

To remove a `~Mock.side_effect`, and return to the default behaviour, set the `!side_effect` to `None`:

> \>\>\> m = MagicMock(return_value=6) \>\>\> def side_effect(*args,*\*kwargs): ... return 3 ... \>\>\> m.side_effect = side_effect \>\>\> m() 3 \>\>\> m.side_effect = None \>\>\> m() 6

The `~Mock.side_effect` can also be any iterable object. Repeated calls to the mock will return values from the iterable (until the iterable is `exhausted` and a `StopIteration` is raised):

> \>\>\> m = MagicMock(side_effect=\[1, 2, 3\]) \>\>\> m() 1 \>\>\> m() 2 \>\>\> m() 3 \>\>\> m() Traceback (most recent call last): ... StopIteration

If any members of the iterable are exceptions they will be raised instead of returned:

    >>> iterable = (33, ValueError, 66)
    >>> m = MagicMock(side_effect=iterable)
    >>> m()
    33
    >>> m()
    Traceback (most recent call last):
     ...
    ValueError
    >>> m()
    66

### Deleting Attributes

Mock objects create attributes on demand. This allows them to pretend to be objects of any type.

You may want a mock object to return `False` to a `hasattr` call, or raise an `AttributeError` when an attribute is fetched. You can do this by providing an object as a `!spec` for a mock, but that isn't always convenient.

You "block" attributes by deleting them. Once deleted, accessing an attribute will raise an `AttributeError`.

> \>\>\> mock = MagicMock() \>\>\> hasattr(mock, 'm') True \>\>\> del mock.m \>\>\> hasattr(mock, 'm') False \>\>\> del mock.f \>\>\> mock.f Traceback (most recent call last): ... AttributeError: f

### Mock names and the name attribute

Since "name" is an argument to the `Mock` constructor, if you want your mock object to have a "name" attribute you can't just pass it in at creation time. There are two alternatives. One option is to use `~Mock.configure_mock`:

    >>> mock = MagicMock()
    >>> mock.configure_mock(name='my_name')
    >>> mock.name
    'my_name'

A simpler option is to simply set the "name" attribute after mock creation:

    >>> mock = MagicMock()
    >>> mock.name = "foo"

### Attaching Mocks as Attributes

When you attach a mock as an attribute of another mock (or as the return value) it becomes a "child" of that mock. Calls to the child are recorded in the `~Mock.method_calls` and `~Mock.mock_calls` attributes of the parent. This is useful for configuring child mocks and then attaching them to the parent, or for attaching mocks to a parent that records all calls to the children and allows you to make assertions about the order of calls between mocks:

> \>\>\> parent = MagicMock() \>\>\> child1 = MagicMock(return_value=None) \>\>\> child2 = MagicMock(return_value=None) \>\>\> parent.child1 = child1 \>\>\> parent.child2 = child2 \>\>\> child1(1) \>\>\> child2(2) \>\>\> parent.mock_calls \[call.child1(1), call.child2(2)\]

The exception to this is if the mock has a name. This allows you to prevent the "parenting" if for some reason you don't want it to happen.

> \>\>\> mock = MagicMock() \>\>\> not_a_child = MagicMock(name='not-a-child') \>\>\> mock.attribute = not_a_child \>\>\> mock.attribute() \<MagicMock name='not-a-child()' id='...'\> \>\>\> mock.mock_calls \[\]

Mocks created for you by `patch` are automatically given names. To attach mocks that have names to a parent you use the `~Mock.attach_mock` method:

    >>> thing1 = object()
    >>> thing2 = object()
    >>> parent = MagicMock()
    >>> with patch('__main__.thing1', return_value=None) as child1:
    ...     with patch('__main__.thing2', return_value=None) as child2:
    ...         parent.attach_mock(child1, 'child1')
    ...         parent.attach_mock(child2, 'child2')
    ...         child1('one')
    ...         child2('two')
    ...
    >>> parent.mock_calls
    [call.child1('one'), call.child2('two')]

## The patchers

The patch decorators are used for patching objects only within the scope of the function they decorate. They automatically handle the unpatching for you, even if exceptions are raised. All of these functions can also be used in with statements or as class decorators.

### patch

> [!NOTE]
> The key is to do the patching in the right namespace. See the section [where to patch](#where-to-patch).

<div class="function">

patch(target, new=DEFAULT, spec=None, create=False, spec_set=None, autospec=None, new_callable=None, \*\*kwargs)

`patch` acts as a function decorator, class decorator or a context manager. Inside the body of the function or with statement, the *target* is patched with a *new* object. When the function/with statement exits the patch is undone.

If *new* is omitted, then the target is replaced with an `AsyncMock` if the patched object is an async function or a `MagicMock` otherwise. If `patch` is used as a decorator and *new* is omitted, the created mock is passed in as an extra argument to the decorated function. If `patch` is used as a context manager the created mock is returned by the context manager.

*target* should be a string in the form `'package.module.ClassName'`. The *target* is imported and the specified object replaced with the *new* object, so the *target* must be importable from the environment you are calling `patch` from. The target is imported when the decorated function is executed, not at decoration time.

The *spec* and *spec_set* keyword arguments are passed to the `MagicMock` if patch is creating one for you.

In addition you can pass `spec=True` or `spec_set=True`, which causes patch to pass in the object being mocked as the spec/spec_set object.

*new_callable* allows you to specify a different class, or callable object, that will be called to create the *new* object. By default `AsyncMock` is used for async functions and `MagicMock` for the rest.

A more powerful form of *spec* is *autospec*. If you set `autospec=True` then the mock will be created with a spec from the object being replaced. All attributes of the mock will also have the spec of the corresponding attribute of the object being replaced. Methods and functions being mocked will have their arguments checked and will raise a `TypeError` if they are called with the wrong signature. For mocks replacing a class, their return value (the 'instance') will have the same spec as the class. See the `create_autospec` function and `auto-speccing`.

Instead of `autospec=True` you can pass `autospec=some_object` to use an arbitrary object as the spec instead of the one being replaced.

By default `patch` will fail to replace attributes that don't exist. If you pass in `create=True`, and the attribute doesn't exist, patch will create the attribute for you when the patched function is called, and delete it again after the patched function has exited. This is useful for writing tests against attributes that your production code creates at runtime. It is off by default because it can be dangerous. With it switched on you can write passing tests against APIs that don't actually exist!

> [!NOTE]
> <div class="versionchanged">
>
> 3.5 If you are patching builtins in a module then you don't need to pass `create=True`, it will be added by default.
>
> </div>

Patch can be used as a `~unittest.TestCase` class decorator. It works by decorating each test method in the class. This reduces the boilerplate code when your test methods share a common patchings set. `patch` finds tests by looking for method names that start with `patch.TEST_PREFIX`. By default this is `'test'`, which matches the way `unittest` finds tests. You can specify an alternative prefix by setting `patch.TEST_PREFIX`.

Patch can be used as a context manager, with the with statement. Here the patching applies to the indented block after the with statement. If you use "as" then the patched object will be bound to the name after the "as"; very useful if `patch` is creating a mock object for you.

`patch` takes arbitrary keyword arguments. These will be passed to `AsyncMock` if the patched object is asynchronous, to `MagicMock` otherwise or to *new_callable* if specified.

`patch.dict(...)`, `patch.multiple(...)` and `patch.object(...)` are available for alternate use-cases.

</div>

`patch` as function decorator, creating the mock for you and passing it into the decorated function:

    >>> @patch('__main__.SomeClass')
    ... def function(normal_argument, mock_class):
    ...     print(mock_class is SomeClass)
    ...
    >>> function(None)
    True

Patching a class replaces the class with a `MagicMock` *instance*. If the class is instantiated in the code under test then it will be the `~Mock.return_value` of the mock that will be used.

If the class is instantiated multiple times you could use `~Mock.side_effect` to return a new mock each time. Alternatively you can set the *return_value* to be anything you want.

To configure return values on methods of *instances* on the patched class you must do this on the `~Mock.return_value`. For example:

    >>> class Class:
    ...     def method(self):
    ...         pass
    ...
    >>> with patch('__main__.Class') as MockClass:
    ...     instance = MockClass.return_value
    ...     instance.method.return_value = 'foo'
    ...     assert Class() is instance
    ...     assert Class().method() == 'foo'
    ...

If you use *spec* or *spec_set* and `patch` is replacing a *class*, then the return value of the created mock will have the same spec. :

    >>> Original = Class
    >>> patcher = patch('__main__.Class', spec=True)
    >>> MockClass = patcher.start()
    >>> instance = MockClass()
    >>> assert isinstance(instance, Original)
    >>> patcher.stop()

The *new_callable* argument is useful where you want to use an alternative class to the default `MagicMock` for the created mock. For example, if you wanted a `NonCallableMock` to be used:

    >>> thing = object()
    >>> with patch('__main__.thing', new_callable=NonCallableMock) as mock_thing:
    ...     assert thing is mock_thing
    ...     thing()
    ...
    Traceback (most recent call last):
      ...
    TypeError: 'NonCallableMock' object is not callable

Another use case might be to replace an object with an `io.StringIO` instance:

    >>> from io import StringIO
    >>> def foo():
    ...     print('Something')
    ...
    >>> @patch('sys.stdout', new_callable=StringIO)
    ... def test(mock_stdout):
    ...     foo()
    ...     assert mock_stdout.getvalue() == 'Something\n'
    ...
    >>> test()

When `patch` is creating a mock for you, it is common that the first thing you need to do is to configure the mock. Some of that configuration can be done in the call to patch. Any arbitrary keywords you pass into the call will be used to set attributes on the created mock:

    >>> patcher = patch('__main__.thing', first='one', second='two')
    >>> mock_thing = patcher.start()
    >>> mock_thing.first
    'one'
    >>> mock_thing.second
    'two'

As well as attributes on the created mock attributes, like the `~Mock.return_value` and `~Mock.side_effect`, of child mocks can also be configured. These aren't syntactically valid to pass in directly as keyword arguments, but a dictionary with these as keys can still be expanded into a `patch` call using `**`:

    >>> config = {'method.return_value': 3, 'other.side_effect': KeyError}
    >>> patcher = patch('__main__.thing', **config)
    >>> mock_thing = patcher.start()
    >>> mock_thing.method()
    3
    >>> mock_thing.other()
    Traceback (most recent call last):
      ...
    KeyError

By default, attempting to patch a function in a module (or a method or an attribute in a class) that does not exist will fail with `AttributeError`:

    >>> @patch('sys.non_existing_attribute', 42)
    ... def test():
    ...     assert sys.non_existing_attribute == 42
    ...
    >>> test()
    Traceback (most recent call last):
      ...
    AttributeError: <module 'sys' (built-in)> does not have the attribute 'non_existing_attribute'

but adding `create=True` in the call to `patch` will make the previous example work as expected:

    >>> @patch('sys.non_existing_attribute', 42, create=True)
    ... def test(mock_stdout):
    ...     assert sys.non_existing_attribute == 42
    ...
    >>> test()

<div class="versionchanged">

3.8

`patch` now returns an `AsyncMock` if the target is an async function.

</div>

### patch.object

<div class="function">

patch.object(target, attribute, new=DEFAULT, spec=None, create=False, spec_set=None, autospec=None, new_callable=None, \*\*kwargs)

patch the named member (*attribute*) on an object (*target*) with a mock object.

`patch.object` can be used as a decorator, class decorator or a context manager. Arguments *new*, *spec*, *create*, *spec_set*, *autospec* and *new_callable* have the same meaning as for `patch`. Like `patch`, `patch.object` takes arbitrary keyword arguments for configuring the mock object it creates.

When used as a class decorator `patch.object` honours `patch.TEST_PREFIX` for choosing which methods to wrap.

</div>

You can either call `patch.object` with three arguments or two arguments. The three argument form takes the object to be patched, the attribute name and the object to replace the attribute with.

When calling with the two argument form you omit the replacement object, and a mock is created for you and passed in as an extra argument to the decorated function:

> \>\>\> @patch.object(SomeClass, 'class_method') ... def test(mock_method): ... SomeClass.class_method(3) ... mock_method.assert_called_with(3) ... \>\>\> test()

*spec*, *create* and the other arguments to `patch.object` have the same meaning as they do for `patch`.

### patch.dict

<div class="function">

patch.dict(in_dict, values=(), clear=False, \*\*kwargs)

Patch a dictionary, or dictionary like object, and restore the dictionary to its original state after the test, where the restored dictionary is a copy of the dictionary as it was before the test.

*in_dict* can be a dictionary or a mapping like container. If it is a mapping then it must at least support getting, setting and deleting items plus iterating over keys.

*in_dict* can also be a string specifying the name of the dictionary, which will then be fetched by importing it.

*values* can be a dictionary of values to set in the dictionary. *values* can also be an iterable of `(key, value)` pairs.

If *clear* is true then the dictionary will be cleared before the new values are set.

`patch.dict` can also be called with arbitrary keyword arguments to set values in the dictionary.

<div class="versionchanged">

3.8

`patch.dict` now returns the patched dictionary when used as a context manager.

</div>

</div>

`patch.dict` can be used as a context manager, decorator or class decorator:

> \>\>\> foo = {} \>\>\> @patch.dict(foo, {'newkey': 'newvalue'}) ... def test(): ... assert foo == {'newkey': 'newvalue'} ... \>\>\> test() \>\>\> assert foo == {}

When used as a class decorator `patch.dict` honours `patch.TEST_PREFIX` (default to `'test'`) for choosing which methods to wrap:

> \>\>\> import os \>\>\> import unittest \>\>\> from unittest.mock import patch \>\>\> @patch.dict('os.environ', {'newkey': 'newvalue'}) ... class TestSample(unittest.TestCase): ... def test_sample(self): ... self.assertEqual(os.environ\['newkey'\], 'newvalue')

If you want to use a different prefix for your test, you can inform the patchers of the different prefix by setting `patch.TEST_PREFIX`. For more details about how to change the value of see `test-prefix`.

`patch.dict` can be used to add members to a dictionary, or simply let a test change a dictionary, and ensure the dictionary is restored when the test ends.

> \>\>\> foo = {} \>\>\> with patch.dict(foo, {'newkey': 'newvalue'}) as patched_foo: ... assert foo == {'newkey': 'newvalue'} ... assert patched_foo == {'newkey': 'newvalue'} ... \# You can add, update or delete keys of foo (or patched_foo, it's the same dict) ... patched_foo\['spam'\] = 'eggs' ... \>\>\> assert foo == {} \>\>\> assert patched_foo == {}
>
> \>\>\> import os \>\>\> with patch.dict('os.environ', {'newkey': 'newvalue'}): ... print(os.environ\['newkey'\]) ... newvalue \>\>\> assert 'newkey' not in os.environ

Keywords can be used in the `patch.dict` call to set values in the dictionary:

> \>\>\> mymodule = MagicMock() \>\>\> mymodule.function.return_value = 'fish' \>\>\> with patch.dict('sys.modules', mymodule=mymodule): ... import mymodule ... mymodule.function('some', 'args') ... 'fish'

`patch.dict` can be used with dictionary like objects that aren't actually dictionaries. At the very minimum they must support item getting, setting, deleting and either iteration or membership test. This corresponds to the magic methods `~object.__getitem__`, `~object.__setitem__`, `~object.__delitem__` and either `~container.__iter__` or `~object.__contains__`.

> \>\>\> class Container: ... def \_\_init\_\_(self): ... self.values = {} ... def \_\_getitem\_\_(self, name): ... return self.values\[name\] ... def \_\_setitem\_\_(self, name, value): ... self.values\[name\] = value ... def \_\_delitem\_\_(self, name): ... del self.values\[name\] ... def \_\_iter\_\_(self): ... return iter(self.values) ... \>\>\> thing = Container() \>\>\> thing\['one'\] = 1 \>\>\> with patch.dict(thing, one=2, two=3): ... assert thing\['one'\] == 2 ... assert thing\['two'\] == 3 ... \>\>\> assert thing\['one'\] == 1 \>\>\> assert list(thing) == \['one'\]

### patch.multiple

<div class="function">

patch.multiple(target, spec=None, create=False, spec_set=None, autospec=None, new_callable=None, \*\*kwargs)

Perform multiple patches in a single call. It takes the object to be patched (either as an object or a string to fetch the object by importing) and keyword arguments for the patches:

    with patch.multiple(settings, FIRST_PATCH='one', SECOND_PATCH='two'):
        ...

Use `DEFAULT` as the value if you want `patch.multiple` to create mocks for you. In this case the created mocks are passed into a decorated function by keyword, and a dictionary is returned when `patch.multiple` is used as a context manager.

`patch.multiple` can be used as a decorator, class decorator or a context manager. The arguments *spec*, *spec_set*, *create*, *autospec* and *new_callable* have the same meaning as for `patch`. These arguments will be applied to *all* patches done by `patch.multiple`.

When used as a class decorator `patch.multiple` honours `patch.TEST_PREFIX` for choosing which methods to wrap.

</div>

If you want `patch.multiple` to create mocks for you, then you can use `DEFAULT` as the value. If you use `patch.multiple` as a decorator then the created mocks are passed into the decorated function by keyword. :

    >>> thing = object()
    >>> other = object()

    >>> @patch.multiple('__main__', thing=DEFAULT, other=DEFAULT)
    ... def test_function(thing, other):
    ...     assert isinstance(thing, MagicMock)
    ...     assert isinstance(other, MagicMock)
    ...
    >>> test_function()

`patch.multiple` can be nested with other `patch` decorators, but put arguments passed by keyword *after* any of the standard arguments created by `patch`:

    >>> @patch('sys.exit')
    ... @patch.multiple('__main__', thing=DEFAULT, other=DEFAULT)
    ... def test_function(mock_exit, other, thing):
    ...     assert 'other' in repr(other)
    ...     assert 'thing' in repr(thing)
    ...     assert 'exit' in repr(mock_exit)
    ...
    >>> test_function()

If `patch.multiple` is used as a context manager, the value returned by the context manager is a dictionary where created mocks are keyed by name:

    >>> with patch.multiple('__main__', thing=DEFAULT, other=DEFAULT) as values:
    ...     assert 'other' in repr(values['other'])
    ...     assert 'thing' in repr(values['thing'])
    ...     assert values['thing'] is thing
    ...     assert values['other'] is other
    ...

### patch methods: start and stop

All the patchers have `!start` and `!stop` methods. These make it simpler to do patching in `setUp` methods or where you want to do multiple patches without nesting decorators or with statements.

To use them call `patch`, `patch.object` or `patch.dict` as normal and keep a reference to the returned `patcher` object. You can then call `!start` to put the patch in place and `!stop` to undo it.

If you are using `patch` to create a mock for you then it will be returned by the call to `patcher.start`. :

    >>> patcher = patch('package.module.ClassName')
    >>> from package import module
    >>> original = module.ClassName
    >>> new_mock = patcher.start()
    >>> assert module.ClassName is not original
    >>> assert module.ClassName is new_mock
    >>> patcher.stop()
    >>> assert module.ClassName is original
    >>> assert module.ClassName is not new_mock

A typical use case for this might be for doing multiple patches in the `setUp` method of a `~unittest.TestCase`:

    >>> class MyTest(unittest.TestCase):
    ...     def setUp(self):
    ...         self.patcher1 = patch('package.module.Class1')
    ...         self.patcher2 = patch('package.module.Class2')
    ...         self.MockClass1 = self.patcher1.start()
    ...         self.MockClass2 = self.patcher2.start()
    ...
    ...     def tearDown(self):
    ...         self.patcher1.stop()
    ...         self.patcher2.stop()
    ...
    ...     def test_something(self):
    ...         assert package.module.Class1 is self.MockClass1
    ...         assert package.module.Class2 is self.MockClass2
    ...
    >>> MyTest('test_something').run()

> [!CAUTION]
> If you use this technique you must ensure that the patching is "undone" by calling `stop`. This can be fiddlier than you might think, because if an exception is raised in the `setUp` then `tearDown` is not called. `unittest.TestCase.addCleanup` makes this easier:
>
>     >>> class MyTest(unittest.TestCase):
>     ...     def setUp(self):
>     ...         patcher = patch('package.module.Class')
>     ...         self.MockClass = patcher.start()
>     ...         self.addCleanup(patcher.stop)
>     ...
>     ...     def test_something(self):
>     ...         assert package.module.Class is self.MockClass
>     ...
>
> As an added bonus you no longer need to keep a reference to the `patcher` object.

It is also possible to stop all patches which have been started by using `patch.stopall`.

<div class="function">

patch.stopall

Stop all active patches. Only stops patches started with `start`.

</div>

### patch builtins

You can patch any builtins within a module. The following example patches builtin `ord`:

    >>> @patch('__main__.ord')
    ... def test(mock_ord):
    ...     mock_ord.return_value = 101
    ...     print(ord('c'))
    ...
    >>> test()
    101

### TEST_PREFIX

All of the patchers can be used as class decorators. When used in this way they wrap every test method on the class. The patchers recognise methods that start with `'test'` as being test methods. This is the same way that the `unittest.TestLoader` finds test methods by default.

It is possible that you want to use a different prefix for your tests. You can inform the patchers of the different prefix by setting `patch.TEST_PREFIX`:

    >>> patch.TEST_PREFIX = 'foo'
    >>> value = 3
    >>>
    >>> @patch('__main__.value', 'not three')
    ... class Thing:
    ...     def foo_one(self):
    ...         print(value)
    ...     def foo_two(self):
    ...         print(value)
    ...
    >>>
    >>> Thing().foo_one()
    not three
    >>> Thing().foo_two()
    not three
    >>> value
    3

### Nesting Patch Decorators

If you want to perform multiple patches then you can simply stack up the decorators.

You can stack up multiple patch decorators using this pattern:

> \>\>\> @patch.object(SomeClass, 'class_method') ... @patch.object(SomeClass, 'static_method') ... def test(mock1, mock2): ... assert SomeClass.static_method is mock1 ... assert SomeClass.class_method is mock2 ... SomeClass.static_method('foo') ... SomeClass.class_method('bar') ... return mock1, mock2 ... \>\>\> mock1, mock2 = test() \>\>\> mock1.assert_called_once_with('foo') \>\>\> mock2.assert_called_once_with('bar')

Note that the decorators are applied from the bottom upwards. This is the standard way that Python applies decorators. The order of the created mocks passed into your test function matches this order.

### Where to patch

`patch` works by (temporarily) changing the object that a *name* points to with another one. There can be many names pointing to any individual object, so for patching to work you must ensure that you patch the name used by the system under test.

The basic principle is that you patch where an object is *looked up*, which is not necessarily the same place as where it is defined. A couple of examples will help to clarify this.

Imagine we have a project that we want to test with the following structure:

    a.py
        -> Defines SomeClass

    b.py
        -> from a import SomeClass
        -> some_function instantiates SomeClass

Now we want to test `some_function` but we want to mock out `SomeClass` using `patch`. The problem is that when we import module b, which we will have to do when it imports `SomeClass` from module a. If we use `patch` to mock out `a.SomeClass` then it will have no effect on our test; module b already has a reference to the *real* `SomeClass` and it looks like our patching had no effect.

The key is to patch out `SomeClass` where it is used (or where it is looked up). In this case `some_function` will actually look up `SomeClass` in module b, where we have imported it. The patching should look like:

    @patch('b.SomeClass')

However, consider the alternative scenario where instead of `from a import SomeClass` module b does `import a` and `some_function` uses `a.SomeClass`. Both of these import forms are common. In this case the class we want to patch is being looked up in the module and so we have to patch `a.SomeClass` instead:

    @patch('a.SomeClass')

### Patching Descriptors and Proxy Objects

Both [patch](#patch) and [patch.object](#patch.object) correctly patch and restore descriptors: class methods, static methods and properties. You should patch these on the *class* rather than an instance. They also work with *some* objects that proxy attribute access, like the [django settings object](https://web.archive.org/web/20200603181648/http://www.voidspace.org.uk/python/weblog/arch_d7_2010_12_04.shtml#e1198).

## MagicMock and magic method support

### Mocking Magic Methods

`Mock` supports mocking the Python protocol methods, also known as `"magic methods" <magic method>`. This allows mock objects to replace containers or other objects that implement Python protocols.

Because magic methods are looked up differently from normal methods[^2], this support has been specially implemented. This means that only specific magic methods are supported. The supported list includes *almost* all of them. If there are any missing that you need please let us know.

You mock magic methods by setting the method you are interested in to a function or a mock instance. If you are using a function then it *must* take `self` as the first argument[^3].

> \>\>\> def \_\_str\_\_(self): ... return 'fooble' ... \>\>\> mock = Mock() \>\>\> mock.\_\_str\_\_ = \_\_str\_\_ \>\>\> str(mock) 'fooble'
>
> \>\>\> mock = Mock() \>\>\> mock.\_\_str\_\_ = Mock() \>\>\> mock.\_\_str\_\_.return_value = 'fooble' \>\>\> str(mock) 'fooble'
>
> \>\>\> mock = Mock() \>\>\> mock.\_\_iter\_\_ = Mock(return_value=iter(\[\])) \>\>\> list(mock) \[\]

One use case for this is for mocking objects used as context managers in a `with` statement:

> \>\>\> mock = Mock() \>\>\> mock.\_\_enter\_\_ = Mock(return_value='foo') \>\>\> mock.\_\_exit\_\_ = Mock(return_value=False) \>\>\> with mock as m: ... assert m == 'foo' ... \>\>\> mock.\_\_enter\_\_.assert_called_with() \>\>\> mock.\_\_exit\_\_.assert_called_with(None, None, None)

Calls to magic methods do not appear in `~Mock.method_calls`, but they are recorded in `~Mock.mock_calls`.

> [!NOTE]
> If you use the *spec* keyword argument to create a mock then attempting to set a magic method that isn't in the spec will raise an `AttributeError`.

The full list of supported magic methods is:

- `__hash__`, `__sizeof__`, `__repr__` and `__str__`
- `__dir__`, `__format__` and `__subclasses__`
- `__round__`, `__floor__`, `__trunc__` and `__ceil__`
- Comparisons: `__lt__`, `__gt__`, `__le__`, `__ge__`, `__eq__` and `__ne__`
- Container methods: `__getitem__`, `__setitem__`, `__delitem__`, `__contains__`, `__len__`, `__iter__`, `__reversed__` and `__missing__`
- Context manager: `__enter__`, `__exit__`, `__aenter__` and `__aexit__`
- Unary numeric methods: `__neg__`, `__pos__` and `__invert__`
- The numeric methods (including right hand and in-place variants): `__add__`, `__sub__`, `__mul__`, `__matmul__`, `__truediv__`, `__floordiv__`, `__mod__`, `__divmod__`, `__lshift__`, `__rshift__`, `__and__`, `__xor__`, `__or__`, and `__pow__`
- Numeric conversion methods: `__complex__`, `__int__`, `__float__` and `__index__`
- Descriptor methods: `__get__`, `__set__` and `__delete__`
- Pickling: `__reduce__`, `__reduce_ex__`, `__getinitargs__`, `__getnewargs__`, `__getstate__` and `__setstate__`
- File system path representation: `__fspath__`
- Asynchronous iteration methods: `__aiter__` and `__anext__`

<div class="versionchanged">

3.8 Added support for `os.PathLike.__fspath__`.

</div>

<div class="versionchanged">

3.8 Added support for `__aenter__`, `__aexit__`, `__aiter__` and `__anext__`.

</div>

The following methods exist but are *not* supported as they are either in use by mock, can't be set dynamically, or can cause problems:

- `__getattr__`, `__setattr__`, `__init__` and `__new__`
- `__prepare__`, `__instancecheck__`, `__subclasscheck__`, `__del__`

### Magic Mock

There are two `MagicMock` variants: `MagicMock` and `NonCallableMagicMock`.

<div class="MagicMock(*args, **kw)">

`MagicMock` is a subclass of `Mock` with default implementations of most of the `magic methods <magic method>`. You can use `MagicMock` without having to configure the magic methods yourself.

The constructor parameters have the same meaning as for `Mock`.

If you use the *spec* or *spec_set* arguments then *only* magic methods that exist in the spec will be created.

</div>

<div class="NonCallableMagicMock(*args, **kw)">

A non-callable version of `MagicMock`.

The constructor parameters have the same meaning as for `MagicMock`, with the exception of *return_value* and *side_effect* which have no meaning on a non-callable mock.

</div>

The magic methods are setup with `MagicMock` objects, so you can configure them and use them in the usual way:

> \>\>\> mock = MagicMock() \>\>\> mock\[3\] = 'fish' \>\>\> mock.\_\_setitem\_\_.assert_called_with(3, 'fish') \>\>\> mock.\_\_getitem\_\_.return_value = 'result' \>\>\> mock\[2\] 'result'

By default many of the protocol methods are required to return objects of a specific type. These methods are preconfigured with a default return value, so that they can be used without you having to do anything if you aren't interested in the return value. You can still *set* the return value manually if you want to change the default.

Methods and their defaults:

- `__lt__`: `NotImplemented`
- `__gt__`: `!NotImplemented`
- `__le__`: `!NotImplemented`
- `__ge__`: `!NotImplemented`
- `__int__`: `1`
- `__contains__`: `False`
- `__len__`: `0`
- `__iter__`: `iter([])`
- `__exit__`: `False`
- `__aexit__`: `False`
- `__complex__`: `1j`
- `__float__`: `1.0`
- `__bool__`: `True`
- `__index__`: `1`
- `__hash__`: default hash for the mock
- `__str__`: default str for the mock
- `__sizeof__`: default sizeof for the mock

For example:

> \>\>\> mock = MagicMock() \>\>\> int(mock) 1 \>\>\> len(mock) 0 \>\>\> list(mock) \[\] \>\>\> object() in mock False

The two equality methods, `!__eq__` and `!__ne__`, are special. They do the default equality comparison on identity, using the `~Mock.side_effect` attribute, unless you change their return value to return something else:

    >>> MagicMock() == 3
    False
    >>> MagicMock() != 3
    True
    >>> mock = MagicMock()
    >>> mock.__eq__.return_value = True
    >>> mock == 3
    True

The return value of `!__iter__` can be any iterable object and isn't required to be an iterator:

> \>\>\> mock = MagicMock() \>\>\> mock.\_\_iter\_\_.return_value = \['a', 'b', 'c'\] \>\>\> list(mock) \['a', 'b', 'c'\] \>\>\> list(mock) \['a', 'b', 'c'\]

If the return value *is* an iterator, then iterating over it once will consume it and subsequent iterations will result in an empty list:

> \>\>\> mock.\_\_iter\_\_.return_value = iter(\['a', 'b', 'c'\]) \>\>\> list(mock) \['a', 'b', 'c'\] \>\>\> list(mock) \[\]

`MagicMock` has all of the supported magic methods configured except for some of the obscure and obsolete ones. You can still set these up if you want.

Magic methods that are supported but not setup by default in `MagicMock` are:

- `__subclasses__`
- `__dir__`
- `__format__`
- `__get__`, `__set__` and `__delete__`
- `__reversed__` and `__missing__`
- `__reduce__`, `__reduce_ex__`, `__getinitargs__`, `__getnewargs__`, `__getstate__` and `__setstate__`
- `__getformat__`

## Helpers

### sentinel

<div class="data">

sentinel

The `sentinel` object provides a convenient way of providing unique objects for your tests.

Attributes are created on demand when you access them by name. Accessing the same attribute will always return the same object. The objects returned have a sensible repr so that test failure messages are readable.

<div class="versionchanged">

3.7 The `sentinel` attributes now preserve their identity when they are `copied <copy>` or `pickled <pickle>`.

</div>

</div>

Sometimes when testing you need to test that a specific object is passed as an argument to another method, or returned. It can be common to create named sentinel objects to test this. `sentinel` provides a convenient way of creating and testing the identity of objects like this.

In this example we monkey patch `method` to return `sentinel.some_object`:

> \>\>\> real = ProductionClass() \>\>\> real.method = Mock(name="method") \>\>\> real.method.return_value = sentinel.some_object \>\>\> result = real.method() \>\>\> assert result is sentinel.some_object \>\>\> result sentinel.some_object

### DEFAULT

<div class="data">

DEFAULT

The `DEFAULT` object is a pre-created sentinel (actually `sentinel.DEFAULT`). It can be used by `~Mock.side_effect` functions to indicate that the normal return value should be used.

</div>

### call

<div class="function">

call(*args,*\*kwargs)

`call` is a helper object for making simpler assertions, for comparing with `~Mock.call_args`, `~Mock.call_args_list`, `~Mock.mock_calls` and `~Mock.method_calls`. `call` can also be used with `~Mock.assert_has_calls`.

> \>\>\> m = MagicMock(return_value=None) \>\>\> m(1, 2, a='foo', b='bar') \>\>\> m() \>\>\> m.call_args_list == \[call(1, 2, a='foo', b='bar'), call()\] True

</div>

<div class="method">

call.call_list()

For a call object that represents multiple calls, `call_list` returns a list of all the intermediate calls as well as the final call.

</div>

`call_list` is particularly useful for making assertions on "chained calls". A chained call is multiple calls on a single line of code. This results in multiple entries in `~Mock.mock_calls` on a mock. Manually constructing the sequence of calls can be tedious.

`~call.call_list` can construct the sequence of calls from the same chained call:

> \>\>\> m = MagicMock() \>\>\> m(1).method(arg='foo').other('bar')(2.0) \<MagicMock name='mock().method().other()()' id='...'\> \>\>\> kall = call(1).method(arg='foo').other('bar')(2.0) \>\>\> kall.call_list() \[call(1), call().method(arg='foo'), call().method().other('bar'), call().method().other()(2.0)\] \>\>\> m.mock_calls == kall.call_list() True

<div id="calls-as-tuples">

A `call` object is either a tuple of (positional args, keyword args) or (name, positional args, keyword args) depending on how it was constructed. When you construct them yourself this isn't particularly interesting, but the `call` objects that are in the `Mock.call_args`, `Mock.call_args_list` and `Mock.mock_calls` attributes can be introspected to get at the individual arguments they contain.

</div>

The `call` objects in `Mock.call_args` and `Mock.call_args_list` are two-tuples of (positional args, keyword args) whereas the `call` objects in `Mock.mock_calls`, along with ones you construct yourself, are three-tuples of (name, positional args, keyword args).

You can use their "tupleness" to pull out the individual arguments for more complex introspection and assertions. The positional arguments are a tuple (an empty tuple if there are no positional arguments) and the keyword arguments are a dictionary:

> \>\>\> m = MagicMock(return_value=None) \>\>\> m(1, 2, 3, arg='one', arg2='two') \>\>\> kall = m.call_args \>\>\> kall.args (1, 2, 3) \>\>\> kall.kwargs {'arg': 'one', 'arg2': 'two'} \>\>\> kall.args is kall\[0\] True \>\>\> kall.kwargs is kall\[1\] True
>
> \>\>\> m = MagicMock() \>\>\> m.foo(4, 5, 6, arg='two', arg2='three') \<MagicMock name='mock.foo()' id='...'\> \>\>\> kall = m.mock_calls\[0\] \>\>\> name, args, kwargs = kall \>\>\> name 'foo' \>\>\> args (4, 5, 6) \>\>\> kwargs {'arg': 'two', 'arg2': 'three'} \>\>\> name is m.mock_calls\[0\]\[0\] True

### create_autospec

<div class="function">

create_autospec(spec, spec_set=False, instance=False, \*\*kwargs)

Create a mock object using another object as a spec. Attributes on the mock will use the corresponding attribute on the *spec* object as their spec.

Functions or methods being mocked will have their arguments checked to ensure that they are called with the correct signature.

If *spec_set* is `True` then attempting to set attributes that don't exist on the spec object will raise an `AttributeError`.

If a class is used as a spec then the return value of the mock (the instance of the class) will have the same spec. You can use a class as the spec for an instance object by passing `instance=True`. The returned mock will only be callable if instances of the mock are callable.

`create_autospec` also takes arbitrary keyword arguments that are passed to the constructor of the created mock.

</div>

See `auto-speccing` for examples of how to use auto-speccing with `create_autospec` and the *autospec* argument to `patch`.

<div class="versionchanged">

3.8

`create_autospec` now returns an `AsyncMock` if the target is an async function.

</div>

### ANY

<div class="data">

ANY

</div>

Sometimes you may need to make assertions about *some* of the arguments in a call to mock, but either not care about some of the arguments or want to pull them individually out of `~Mock.call_args` and make more complex assertions on them.

To ignore certain arguments you can pass in objects that compare equal to *everything*. Calls to `~Mock.assert_called_with` and `~Mock.assert_called_once_with` will then succeed no matter what was passed in.

> \>\>\> mock = Mock(return_value=None) \>\>\> mock('foo', bar=object()) \>\>\> mock.assert_called_once_with('foo', bar=ANY)

`ANY` can also be used in comparisons with call lists like `~Mock.mock_calls`:

> \>\>\> m = MagicMock(return_value=None) \>\>\> m(1) \>\>\> m(1, 2) \>\>\> m(object()) \>\>\> m.mock_calls == \[call(1), call(1, 2), ANY\] True

`ANY` is not limited to comparisons with call objects and so can also be used in test assertions:

    class TestStringMethods(unittest.TestCase):

        def test_split(self):
            s = 'hello world'
            self.assertEqual(s.split(), ['hello', ANY])

### FILTER_DIR

<div class="data">

FILTER_DIR

</div>

`FILTER_DIR` is a module level variable that controls the way mock objects respond to `dir`. The default is `True`, which uses the filtering described below, to only show useful members. If you dislike this filtering, or need to switch it off for diagnostic purposes, then set `mock.FILTER_DIR = False`.

With filtering on, `dir(some_mock)` shows only useful attributes and will include any dynamically created attributes that wouldn't normally be shown. If the mock was created with a *spec* (or *autospec* of course) then all the attributes from the original are shown, even if they haven't been accessed yet:

<div class="doctest" data-options="+ELLIPSIS,+NORMALIZE_WHITESPACE">

\>\>\> dir(Mock()) \['assert_any_call', 'assert_called', 'assert_called_once', 'assert_called_once_with', 'assert_called_with', 'assert_has_calls', 'assert_not_called', 'attach_mock', ... \>\>\> from urllib import request \>\>\> dir(Mock(spec=request)) \['AbstractBasicAuthHandler', 'AbstractDigestAuthHandler', 'AbstractHTTPHandler', 'BaseHandler', ...

</div>

Many of the not-very-useful (private to `Mock` rather than the thing being mocked) underscore and double underscore prefixed attributes have been filtered from the result of calling `dir` on a `Mock`. If you dislike this behaviour you can switch it off by setting the module level switch `FILTER_DIR`:

<div class="doctest" data-options="+ELLIPSIS,+NORMALIZE_WHITESPACE">

\>\>\> from unittest import mock \>\>\> mock.FILTER_DIR = False \>\>\> dir(mock.Mock()) \['\_NonCallableMock\_\_get_return_value', '\_NonCallableMock\_\_get_side_effect', '\_NonCallableMock\_\_return_value_doc', '\_NonCallableMock\_\_set_return_value', '\_NonCallableMock\_\_set_side_effect', '\_\_call\_\_', '\_\_class\_\_', ...

</div>

Alternatively you can just use `vars(my_mock)` (instance members) and `dir(type(my_mock))` (type members) to bypass the filtering irrespective of `FILTER_DIR`.

### mock_open

<div class="function">

mock_open(mock=None, read_data='')

A helper function to create a mock to replace the use of `open`. It works for `open` called directly or used as a context manager.

The *mock* argument is the mock object to configure. If `None` (the default) then a `MagicMock` will be created for you, with the API limited to methods or attributes available on standard file handles.

*read_data* is a string for the `~io.RawIOBase.read`, `~io.IOBase.readline`, and `~io.IOBase.readlines` methods of the file handle to return. Calls to those methods will take data from *read_data* until it is depleted. The mock of these methods is pretty simplistic: every time the *mock* is called, the *read_data* is rewound to the start. If you need more control over the data that you are feeding to the tested code you will need to customize this mock for yourself. When that is insufficient, one of the in-memory filesystem packages on [PyPI](https://pypi.org) can offer a realistic filesystem for testing.

<div class="versionchanged">

3.4 Added `~io.IOBase.readline` and `~io.IOBase.readlines` support. The mock of `~io.RawIOBase.read` changed to consume *read_data* rather than returning it on each call.

</div>

<div class="versionchanged">

3.5 *read_data* is now reset on each call to the *mock*.

</div>

<div class="versionchanged">

3.8 Added `~container.__iter__` to implementation so that iteration (such as in for loops) correctly consumes *read_data*.

</div>

</div>

Using `open` as a context manager is a great way to ensure your file handles are closed properly and is becoming common:

    with open('/some/path', 'w') as f:
        f.write('something')

The issue is that even if you mock out the call to `open` it is the *returned object* that is used as a context manager (and has `~object.__enter__` and `~object.__exit__` called).

Mocking context managers with a `MagicMock` is common enough and fiddly enough that a helper function is useful. :

    >>> m = mock_open()
    >>> with patch('__main__.open', m):
    ...     with open('foo', 'w') as h:
    ...         h.write('some stuff')
    ...
    >>> m.mock_calls
    [call('foo', 'w'),
     call().__enter__(),
     call().write('some stuff'),
     call().__exit__(None, None, None)]
    >>> m.assert_called_once_with('foo', 'w')
    >>> handle = m()
    >>> handle.write.assert_called_once_with('some stuff')

And for reading files:

    >>> with patch('__main__.open', mock_open(read_data='bibble')) as m:
    ...     with open('foo') as h:
    ...         result = h.read()
    ...
    >>> m.assert_called_once_with('foo')
    >>> assert result == 'bibble'

### Autospeccing

Autospeccing is based on the existing `!spec` feature of mock. It limits the api of mocks to the api of an original object (the spec), but it is recursive (implemented lazily) so that attributes of mocks only have the same api as the attributes of the spec. In addition mocked functions / methods have the same call signature as the original so they raise a `TypeError` if they are called incorrectly.

Before I explain how auto-speccing works, here's why it is needed.

`Mock` is a very powerful and flexible object, but it suffers from a flaw which is general to mocking. If you refactor some of your code, rename members and so on, any tests for code that is still using the *old api* but uses mocks instead of the real objects will still pass. This means your tests can all pass even though your code is broken.

<div class="versionchanged">

3.5

Before 3.5, tests with a typo in the word assert would silently pass when they should raise an error. You can still achieve this behavior by passing `unsafe=True` to Mock.

</div>

Note that this is another reason why you need integration tests as well as unit tests. Testing everything in isolation is all fine and dandy, but if you don't test how your units are "wired together" there is still lots of room for bugs that tests might have caught.

`!unittest.mock` already provides a feature to help with this, called speccing. If you use a class or instance as the `!spec` for a mock then you can only access attributes on the mock that exist on the real class:

> \>\>\> from urllib import request \>\>\> mock = Mock(spec=request.Request) \>\>\> mock.assret_called_with \# Intentional typo! Traceback (most recent call last): ... AttributeError: Mock object has no attribute 'assret_called_with'

The spec only applies to the mock itself, so we still have the same issue with any methods on the mock:

```
>>> mock.header_items()
<mock.Mock object at 0x...>
>>> mock.header_items.assret_called_with()  # Intentional typo!
```

Auto-speccing solves this problem. You can either pass `autospec=True` to `patch` / `patch.object` or use the `create_autospec` function to create a mock with a spec. If you use the `autospec=True` argument to `patch` then the object that is being replaced will be used as the spec object. Because the speccing is done "lazily" (the spec is created as attributes on the mock are accessed) you can use it with very complex or deeply nested objects (like modules that import modules that import modules) without a big performance hit.

Here's an example of it in use:

    >>> from urllib import request
    >>> patcher = patch('__main__.request', autospec=True)
    >>> mock_request = patcher.start()
    >>> request is mock_request
    True
    >>> mock_request.Request
    <MagicMock name='request.Request' spec='Request' id='...'>

You can see that `!request.Request` has a spec. `!request.Request` takes two arguments in the constructor (one of which is *self*). Here's what happens if we try to call it incorrectly:

    >>> req = request.Request()
    Traceback (most recent call last):
     ...
    TypeError: <lambda>() takes at least 2 arguments (1 given)

The spec also applies to instantiated classes (i.e. the return value of specced mocks):

    >>> req = request.Request('foo')
    >>> req
    <NonCallableMagicMock name='request.Request()' spec='Request' id='...'>

`!Request` objects are not callable, so the return value of instantiating our mocked out `!request.Request` is a non-callable mock. With the spec in place any typos in our asserts will raise the correct error:

    >>> req.add_header('spam', 'eggs')
    <MagicMock name='request.Request().add_header()' id='...'>
    >>> req.add_header.assret_called_with  # Intentional typo!
    Traceback (most recent call last):
     ...
    AttributeError: Mock object has no attribute 'assret_called_with'
    >>> req.add_header.assert_called_with('spam', 'eggs')

In many cases you will just be able to add `autospec=True` to your existing `patch` calls and then be protected against bugs due to typos and api changes.

As well as using *autospec* through `patch` there is a `create_autospec` for creating autospecced mocks directly:

> \>\>\> from urllib import request \>\>\> mock_request = create_autospec(request) \>\>\> mock_request.Request('foo', 'bar') \<NonCallableMagicMock name='mock.Request()' spec='Request' id='...'\>

This isn't without caveats and limitations however, which is why it is not the default behaviour. In order to know what attributes are available on the spec object, autospec has to introspect (access attributes) the spec. As you traverse attributes on the mock a corresponding traversal of the original object is happening under the hood. If any of your specced objects have properties or descriptors that can trigger code execution then you may not be able to use autospec. On the other hand it is much better to design your objects so that introspection is safe[^4].

A more serious problem is that it is common for instance attributes to be created in the `~object.__init__` method and not to exist on the class at all. *autospec* can't know about any dynamically created attributes and restricts the api to visible attributes. :

    >>> class Something:
    ...   def __init__(self):
    ...     self.a = 33
    ...
    >>> with patch('__main__.Something', autospec=True):
    ...   thing = Something()
    ...   thing.a
    ...
    Traceback (most recent call last):
      ...
    AttributeError: Mock object has no attribute 'a'

There are a few different ways of resolving this problem. The easiest, but not necessarily the least annoying, way is to simply set the required attributes on the mock after creation. Just because *autospec* doesn't allow you to fetch attributes that don't exist on the spec it doesn't prevent you setting them:

    >>> with patch('__main__.Something', autospec=True):
    ...   thing = Something()
    ...   thing.a = 33
    ...

There is a more aggressive version of both *spec* and *autospec* that *does* prevent you setting non-existent attributes. This is useful if you want to ensure your code only *sets* valid attributes too, but obviously it prevents this particular scenario:

> \>\>\> with patch('\_\_main\_\_.Something', autospec=True, spec_set=True): ... thing = Something() ... thing.a = 33 ... Traceback (most recent call last): ... AttributeError: Mock object has no attribute 'a'

Probably the best way of solving the problem is to add class attributes as default values for instance members initialised in `~object.__init__`. Note that if you are only setting default attributes in `!__init__` then providing them via class attributes (shared between instances of course) is faster too. e.g.

``` python
class Something:
    a = 33
```

This brings up another issue. It is relatively common to provide a default value of `None` for members that will later be an object of a different type. `None` would be useless as a spec because it wouldn't let you access *any* attributes or methods on it. As `None` is *never* going to be useful as a spec, and probably indicates a member that will normally of some other type, autospec doesn't use a spec for members that are set to `None`. These will just be ordinary mocks (well - MagicMocks):

> \>\>\> class Something: ... member = None ... \>\>\> mock = create_autospec(Something) \>\>\> mock.member.foo.bar.baz() \<MagicMock name='mock.member.foo.bar.baz()' id='...'\>

If modifying your production classes to add defaults isn't to your liking then there are more options. One of these is simply to use an instance as the spec rather than the class. The other is to create a subclass of the production class and add the defaults to the subclass without affecting the production class. Both of these require you to use an alternative object as the spec. Thankfully `patch` supports this - you can simply pass the alternative object as the *autospec* argument:

    >>> class Something:
    ...   def __init__(self):
    ...     self.a = 33
    ...
    >>> class SomethingForTest(Something):
    ...   a = 33
    ...
    >>> p = patch('__main__.Something', autospec=SomethingForTest)
    >>> mock = p.start()
    >>> mock.a
    <NonCallableMagicMock name='Something.a' spec='int' id='...'>

### Sealing mocks

<div class="testsetup">

from unittest.mock import seal

</div>

<div class="function">

seal(mock)

Seal will disable the automatic creation of mocks when accessing an attribute of the mock being sealed or any of its attributes that are already mocks recursively.

If a mock instance with a name or a spec is assigned to an attribute it won't be considered in the sealing chain. This allows one to prevent seal from fixing part of the mock object. :

    >>> mock = Mock()
    >>> mock.submock.attribute1 = 2
    >>> mock.not_submock = mock.Mock(name="sample_name")
    >>> seal(mock)
    >>> mock.new_attribute  # This will raise AttributeError.
    >>> mock.submock.attribute2  # This will raise AttributeError.
    >>> mock.not_submock.attribute2  # This won't raise.

<div class="versionadded">

3.7

</div>

</div>

## Order of precedence of `!side_effect`, `!return_value` and *wraps*

The order of their precedence is:

1.  `~Mock.side_effect`
2.  `~Mock.return_value`
3.  *wraps*

If all three are set, mock will return the value from `~Mock.side_effect`, ignoring `~Mock.return_value` and the wrapped object altogether. If any two are set, the one with the higher precedence will return the value. Regardless of the order of which was set first, the order of precedence remains unchanged.

> \>\>\> from unittest.mock import Mock \>\>\> class Order: ... @staticmethod ... def get_value(): ... return "third" ... \>\>\> order_mock = Mock(spec=Order, wraps=Order) \>\>\> order_mock.get_value.side_effect = \["first"\] \>\>\> order_mock.get_value.return_value = "second" \>\>\> order_mock.get_value() 'first'

As `None` is the default value of `~Mock.side_effect`, if you reassign its value back to `None`, the order of precedence will be checked between `~Mock.return_value` and the wrapped object, ignoring `~Mock.side_effect`.

> \>\>\> order_mock.get_value.side_effect = None \>\>\> order_mock.get_value() 'second'

If the value being returned by `~Mock.side_effect` is `DEFAULT`, it is ignored and the order of precedence moves to the successor to obtain the value to return.

> \>\>\> from unittest.mock import DEFAULT \>\>\> order_mock.get_value.side_effect = \[DEFAULT\] \>\>\> order_mock.get_value() 'second'

When `Mock` wraps an object, the default value of `~Mock.return_value` will be `DEFAULT`.

> \>\>\> order_mock = Mock(spec=Order, wraps=Order) \>\>\> order_mock.return_value sentinel.DEFAULT \>\>\> order_mock.get_value.return_value sentinel.DEFAULT

The order of precedence will ignore this value and it will move to the last successor which is the wrapped object.

As the real call is being made to the wrapped object, creating an instance of this mock will return the real instance of the class. The positional arguments, if any, required by the wrapped object must be passed.

> \>\>\> order_mock_instance = order_mock() \>\>\> isinstance(order_mock_instance, Order) True \>\>\> order_mock_instance.get_value() 'third'
>
> \>\>\> order_mock.get_value.return_value = DEFAULT \>\>\> order_mock.get_value() 'third'
>
> \>\>\> order_mock.get_value.return_value = "second" \>\>\> order_mock.get_value() 'second'

But if you assign `None` to it, this will not be ignored as it is an explicit assignment. So, the order of precedence will not move to the wrapped object.

> \>\>\> order_mock.get_value.return_value = None \>\>\> order_mock.get_value() is None True

Even if you set all three at once when initializing the mock, the order of precedence remains the same:

> \>\>\> order_mock = Mock(spec=Order, wraps=Order, ... \*\*{"get_value.side_effect": \["first"\], ... "get_value.return_value": "second"} ... ) ... \>\>\> order_mock.get_value() 'first' \>\>\> order_mock.get_value.side_effect = None \>\>\> order_mock.get_value() 'second' \>\>\> order_mock.get_value.return_value = DEFAULT \>\>\> order_mock.get_value() 'third'

If `~Mock.side_effect` is `exhausted`, the order of precedence will not cause a value to be obtained from the successors. Instead, `StopIteration` exception is raised.

> \>\>\> order_mock = Mock(spec=Order, wraps=Order) \>\>\> order_mock.get_value.side_effect = \["first side effect value", ... "another side effect value"\] \>\>\> order_mock.get_value.return_value = "second"
>
> \>\>\> order_mock.get_value() 'first side effect value' \>\>\> order_mock.get_value() 'another side effect value'
>
> \>\>\> order_mock.get_value() Traceback (most recent call last): ... StopIteration

[^1]: The only exceptions are magic methods and attributes (those that have leading and trailing double underscores). Mock doesn't create these but instead raises an `AttributeError`. This is because the interpreter will often implicitly request these methods, and gets *very* confused to get a new Mock object when it expects a magic method. If you need magic method support see `magic methods <magic-methods>`.

[^2]: Magic methods *should* be looked up on the class rather than the instance. Different versions of Python are inconsistent about applying this rule. The supported protocol methods should work with all supported versions of Python.

[^3]: The function is basically hooked up to the class, but each `Mock` instance is kept isolated from the others.

[^4]: This only applies to classes or already instantiated objects. Calling a mocked class to create a mock instance *does not* create a real instance. It is only attribute lookups - along with calls to `dir` - that are done.

# Unix-specific services

The modules described in this chapter provide interfaces to features that are unique to the Unix operating system, or in some cases to some or many variants of it. Here's an overview:

<div class="toctree">

shlex.rst posix.rst pwd.rst grp.rst termios.rst tty.rst pty.rst fcntl.rst resource.rst syslog.rst

</div>

# `!urllib.error` --- Exception classes raised by urllib.request

<div class="module" data-synopsis="Exception classes raised by urllib.request.">

urllib.error

</div>

**Source code:** `Lib/urllib/error.py`

------------------------------------------------------------------------

The `!urllib.error` module defines the exception classes for exceptions raised by `urllib.request`. The base exception class is `URLError`.

The following exceptions are raised by `!urllib.error` as appropriate:

<div class="exception">

URLError

The handlers raise this exception (or derived exceptions) when they run into a problem. It is a subclass of `OSError`.

<div class="attribute">

reason

The reason for this error. It can be a message string or another exception instance.

</div>

<div class="versionchanged">

3.3 `URLError` used to be a subtype of `IOError`, which is now an alias of `OSError`.

</div>

</div>

<div class="exception">

HTTPError(url, code, msg, hdrs, fp)

Though being an exception (a subclass of `URLError`), an `HTTPError` can also function as a non-exceptional file-like return value (the same thing that `~urllib.request.urlopen` returns). This is useful when handling exotic HTTP errors, such as requests for authentication.

<div class="attribute">

url

Contains the request URL. An alias for *filename* attribute.

</div>

<div class="attribute">

code

An HTTP status code as defined in `2616`. This numeric value corresponds to a value found in the dictionary of codes as found in `http.server.BaseHTTPRequestHandler.responses`.

</div>

<div class="attribute">

reason

This is usually a string explaining the reason for this error. An alias for *msg* attribute.

</div>

<div class="attribute">

headers

The HTTP response headers for the HTTP request that caused the `HTTPError`. An alias for *hdrs* attribute.

<div class="versionadded">

3.4

</div>

</div>

<div class="attribute">

fp

A file-like object where the HTTP error body can be read from.

</div>

</div>

<div class="exception">

ContentTooShortError(msg, content)

This exception is raised when the `~urllib.request.urlretrieve` function detects that the amount of the downloaded data is less than the expected amount (given by the *Content-Length* header).

<div class="attribute">

content

The downloaded (and supposedly truncated) data.

</div>

</div>

# `!urllib` --- URL handling modules

<div class="module">

urllib

</div>

**Source code:** `Lib/urllib/`

------------------------------------------------------------------------

`urllib` is a package that collects several modules for working with URLs:

- `urllib.request` for opening and reading URLs
- `urllib.error` containing the exceptions raised by `urllib.request`
- `urllib.parse` for parsing URLs
- `urllib.robotparser` for parsing `robots.txt` files

# `!urllib.parse` --- Parse URLs into components

<div class="module" data-synopsis="Parse URLs into or assemble them from components.">

urllib.parse

</div>

**Source code:** `Lib/urllib/parse.py`

<div class="index">

single: WWW single: World Wide Web single: URL pair: URL; parsing pair: relative; URL

</div>

------------------------------------------------------------------------

This module defines a standard interface to break Uniform Resource Locator (URL) strings up in components (addressing scheme, network location, path etc.), to combine the components back into a URL string, and to convert a "relative URL" to an absolute URL given a "base URL."

The module has been designed to match the internet RFC on Relative Uniform Resource Locators. It supports the following URL schemes: `file`, `ftp`, `gopher`, `hdl`, `http`, `https`, `imap`, `itms-services`, `mailto`, `mms`, `news`, `nntp`, `prospero`, `rsync`, `rtsp`, `rtsps`, `rtspu`, `sftp`, `shttp`, `sip`, `sips`, `snews`, `svn`, `svn+ssh`, `telnet`, `wais`, `ws`, `wss`.

<div class="impl-detail">

The inclusion of the `itms-services` URL scheme can prevent an app from passing Apple's App Store review process for the macOS and iOS App Stores. Handling for the `itms-services` scheme is always removed on iOS; on macOS, it *may* be removed if CPython has been built with the `--with-app-store-compliance` option.

</div>

The `!urllib.parse` module defines functions that fall into two broad categories: URL parsing and URL quoting. These are covered in detail in the following sections.

This module's functions use the deprecated term `netloc` (or `net_loc`), which was introduced in `1808`. However, this term has been obsoleted by `3986`, which introduced the term `authority` as its replacement. The use of `netloc` is continued for backward compatibility.

## URL Parsing

The URL parsing functions focus on splitting a URL string into its components, or on combining URL components into a URL string.

<div class="function">

urlsplit(urlstring, scheme=None, allow_fragments=True, \*, missing_as_none=False)

Parse a URL into five components, returning a 5-item `named tuple` `SplitResult` or `SplitResultBytes`. This corresponds to the general structure of a URL: `scheme://netloc/path?query#fragment`. Each tuple item is a string, possibly empty, or `None` if *missing_as_none* is true. Not defined component are represented an empty string (by default) or `None` if *missing_as_none* is true. The delimiters as shown above are not part of the result, except for a leading slash in the *path* component, which is retained if present.

Additionally, the netloc property is broken down into these additional attributes added to the returned object: username, password, hostname, and port.

Percent-encoded sequences are not decoded.

For example:

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> from urllib.parse import urlsplit \>\>\> urlsplit("scheme://netloc/path?query#fragment") SplitResult(scheme='scheme', netloc='netloc', path='/path', query='query', fragment='fragment') \>\>\> o = urlsplit("http://docs.python.org:80/3/library/urllib.parse.html?" ... "highlight=params#url-parsing") \>\>\> o SplitResult(scheme='http', netloc='docs.python.org:80', path='/3/library/urllib.parse.html', query='highlight=params', fragment='url-parsing') \>\>\> o.scheme 'http' \>\>\> o.netloc 'docs.python.org:80' \>\>\> o.hostname 'docs.python.org' \>\>\> o.port 80 \>\>\> o.\_replace(fragment="").geturl() 'http://docs.python.org:80/3/library/urllib.parse.html?highlight=params' \>\>\> urlsplit("http://docs.python.org?") SplitResult(scheme='http', netloc='docs.python.org', path='', query='', fragment='') \>\>\> urlsplit("http://docs.python.org?", missing_as_none=True) SplitResult(scheme='http', netloc='docs.python.org', path='', query='', fragment=None)

</div>

Following the syntax specifications in `1808`, `!urlsplit` recognizes a netloc only if it is properly introduced by '//'. Otherwise the input is presumed to be a relative URL and thus to start with a path component.

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> from urllib.parse import urlsplit \>\>\> urlsplit('//www.cwi.nl:80/%7Eguido/Python.html') SplitResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html', query='', fragment='') \>\>\> urlsplit('www.cwi.nl/%7Eguido/Python.html') SplitResult(scheme='', netloc='', path='www.cwi.nl/%7Eguido/Python.html', query='', fragment='') \>\>\> urlsplit('help/Python.html') SplitResult(scheme='', netloc='', path='help/Python.html', query='', fragment='') \>\>\> urlsplit('help/Python.html', missing_as_none=True) SplitResult(scheme=None, netloc=None, path='help/Python.html', query=None, fragment=None)

</div>

The *scheme* argument gives the default addressing scheme, to be used only if the URL does not specify one. It should be the same type (text or bytes) as *urlstring* or `None`, except that the `''` is always allowed, and is automatically converted to `b''` if appropriate.

If the *allow_fragments* argument is false, fragment identifiers are not recognized. Instead, they are parsed as part of the path or query component, and `fragment` is set to `None` or the empty string (depending on the value of *missing_as_none*) in the return value.

The return value is a `named tuple`, which means that its items can be accessed by index or as named attributes, which are:

| Attribute | Index | Value | Value if not present |
|----|----|----|----|
| `scheme` | 0 | URL scheme specifier | *scheme* parameter or empty string[^1] |
| `netloc` | 1 | Network location part | `None` or empty string[^2] |
| `path` | 2 | Hierarchical path | empty string |
| `query` | 3 | Query component | `None` or empty string[^3] |
| `fragment` | 4 | Fragment identifier | `None` or empty string[^4] |
| `username` |  | User name | `None` |
| `password` |  | Password | `None` |
| `hostname` |  | Host name (lower case) | `None` |
| `port` |  | Port number as integer, if present | `None` |

Reading the `port` attribute will raise a `ValueError` if an invalid port is specified in the URL. See section `urlparse-result-object` for more information on the result object.

Unmatched square brackets in the `netloc` attribute will raise a `ValueError`.

Characters in the `netloc` attribute that decompose under NFKC normalization (as used by the IDNA encoding) into any of `/`, `?`, `#`, `@`, or `:` will raise a `ValueError`. If the URL is decomposed before parsing, no error will be raised.

Following some of the [WHATWG spec](https://url.spec.whatwg.org/#concept-basic-url-parser) that updates `3986`, leading C0 control and space characters are stripped from the URL. `\n`, `\r` and tab `\t` characters are removed from the URL at any position.

As is the case with all named tuples, the subclass has a few additional methods and attributes that are particularly useful. One such method is `_replace`. The `_replace` method will return a new `SplitResult` object replacing specified fields with new values.

<div class="doctest" data-options="+NORMALIZE_WHITESPACE">

\>\>\> from urllib.parse import urlsplit \>\>\> u = urlsplit('//www.cwi.nl:80/%7Eguido/Python.html') \>\>\> u SplitResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html', query='', fragment='') \>\>\> u.\_replace(scheme='http') SplitResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html', query='', fragment='')

</div>

> [!WARNING]
> `urlsplit` does not perform validation. See `URL parsing
> security <url-parsing-security>` for details.

<div class="versionchanged">

3.2 Added IPv6 URL parsing capabilities.

</div>

<div class="versionchanged">

3.3 The fragment is now parsed for all URL schemes (unless *allow_fragments* is false), in accordance with `3986`. Previously, an allowlist of schemes that support fragments existed.

</div>

<div class="versionchanged">

3.6 Out-of-range port numbers now raise `ValueError`, instead of returning `None`.

</div>

<div class="versionchanged">

3.8 Characters that affect netloc parsing under NFKC normalization will now raise `ValueError`.

</div>

<div class="versionchanged">

3.10 ASCII newline and tab characters are stripped from the URL.

</div>

<div class="versionchanged">

3.12 Leading WHATWG C0 control and space characters are stripped from the URL.

</div>

<div class="versionchanged">

3.15 Added the *missing_as_none* parameter.

</div>

</div>

<div class="function">

parse_qs(qs, keep_blank_values=False, strict_parsing=False, encoding='utf-8', errors='replace', max_num_fields=None, separator='&')

Parse a query string given as a string argument (data of type `application/x-www-form-urlencoded`). Data are returned as a dictionary. The dictionary keys are the unique query variable names and the values are lists of values for each name.

The optional argument *keep_blank_values* is a flag indicating whether blank values in percent-encoded queries should be treated as blank strings. A true value indicates that blanks should be retained as blank strings. The default false value indicates that blank values are to be ignored and treated as if they were not included.

The optional argument *strict_parsing* is a flag indicating what to do with parsing errors. If false (the default), errors are silently ignored. If true, errors raise a `ValueError` exception.

The optional *encoding* and *errors* parameters specify how to decode percent-encoded sequences into Unicode characters, as accepted by the `bytes.decode` method.

The optional argument *max_num_fields* is the maximum number of fields to read. If set, then throws a `ValueError` if there are more than *max_num_fields* fields read.

The optional argument *separator* is the symbol to use for separating the query arguments. It defaults to `&`.

Use the `urllib.parse.urlencode` function (with the `doseq` parameter set to `True`) to convert such dictionaries into query strings.

<div class="versionchanged">

3.2 Add *encoding* and *errors* parameters.

</div>

<div class="versionchanged">

3.8 Added *max_num_fields* parameter.

</div>

<div class="versionchanged">

3.10 Added *separator* parameter with the default value of `&`. Python versions earlier than Python 3.10 allowed using both `;` and `&` as query parameter separator. This has been changed to allow only a single separator key, with `&` as the default separator.

</div>

<div class="deprecated">

3.14 Accepting objects with false values (like `0` and `[]`) except empty strings and byte-like objects and `None` is now deprecated.

</div>

</div>

<div class="function">

parse_qsl(qs, keep_blank_values=False, strict_parsing=False, encoding='utf-8', errors='replace', max_num_fields=None, separator='&')

Parse a query string given as a string argument (data of type `application/x-www-form-urlencoded`). Data are returned as a list of name, value pairs.

The optional argument *keep_blank_values* is a flag indicating whether blank values in percent-encoded queries should be treated as blank strings. A true value indicates that blanks should be retained as blank strings. The default false value indicates that blank values are to be ignored and treated as if they were not included.

The optional argument *strict_parsing* is a flag indicating what to do with parsing errors. If false (the default), errors are silently ignored. If true, errors raise a `ValueError` exception.

The optional *encoding* and *errors* parameters specify how to decode percent-encoded sequences into Unicode characters, as accepted by the `bytes.decode` method.

The optional argument *max_num_fields* is the maximum number of fields to read. If set, then throws a `ValueError` if there are more than *max_num_fields* fields read.

The optional argument *separator* is the symbol to use for separating the query arguments. It defaults to `&`.

Use the `urllib.parse.urlencode` function to convert such lists of pairs into query strings.

<div class="versionchanged">

3.2 Add *encoding* and *errors* parameters.

</div>

<div class="versionchanged">

3.8 Added *max_num_fields* parameter.

</div>

<div class="versionchanged">

3.10 Added *separator* parameter with the default value of `&`. Python versions earlier than Python 3.10 allowed using both `;` and `&` as query parameter separator. This has been changed to allow only a single separator key, with `&` as the default separator.

</div>

</div>

<div class="function">

urlunsplit(parts) urlunsplit(parts, \*, keep_empty)

Construct a URL from a tuple as returned by `urlsplit`. The *parts* argument can be any five-item iterable.

This may result in a slightly different, but equivalent URL, if the URL that was parsed originally had unnecessary delimiters (for example, a `?` with an empty query; the RFC states that these are equivalent).

If *keep_empty* is true, empty strings are kept in the result (for example, a `?` for an empty query), only `None` components are omitted. This allows rebuilding a URL that was parsed with option `missing_as_none=True`. By default, *keep_empty* is true if *parts* is the result of the `urlsplit` call with `missing_as_none=True`.

<div class="versionchanged">

3.15 Added the *keep_empty* parameter.

</div>

</div>

<div class="function">

urlparse(urlstring, scheme=None, allow_fragments=True, \*, missing_as_none=False)

This is similar to `urlsplit`, but additionally splits the *path* component on *path* and *params*. This function returns a 6-item `named tuple` `ParseResult` or `ParseResultBytes`. Its items are the same as for the `!urlsplit` result, except that *params* is inserted at index 3, between *path* and *query*.

This function is based on obsoleted `1738` and `1808`, which listed *params* as the main URL component. The more recent URL syntax allows parameters to be applied to each segment of the *path* portion of the URL (see `3986`). `urlsplit` should generally be used instead of `urlparse`. A separate function is needed to separate the path segments and parameters.

</div>

<div class="function">

urlunparse(parts) urlunparse(parts, \*, keep_empty)

Combine the elements of a tuple as returned by `urlparse` into a complete URL as a string. The *parts* argument can be any six-item iterable.

This may result in a slightly different, but equivalent URL, if the URL that was parsed originally had unnecessary delimiters (for example, a `?` with an empty query; the RFC states that these are equivalent).

If *keep_empty* is true, empty strings are kept in the result (for example, a `?` for an empty query), only `None` components are omitted. This allows rebuilding a URL that was parsed with option `missing_as_none=True`. By default, *keep_empty* is true if *parts* is the result of the `urlparse` call with `missing_as_none=True`.

<div class="versionchanged">

3.15 Added the *keep_empty* parameter.

</div>

</div>

<div class="function">

urljoin(base, url, allow_fragments=True)

Construct a full ("absolute") URL by combining a "base URL" (*base*) with another URL (*url*). Informally, this uses components of the base URL, in particular the addressing scheme, the network location and (part of) the path, to provide missing components in the relative URL. For example:

> \>\>\> from urllib.parse import urljoin \>\>\> urljoin('http://www.cwi.nl/%7Eguido/Python.html', 'FAQ.html') 'http://www.cwi.nl/%7Eguido/FAQ.html'

The *allow_fragments* argument has the same meaning and default as for `urlsplit`.

> [!NOTE]
> If *url* is an absolute URL (that is, it starts with `//` or `scheme://`), the *url*'s hostname and/or scheme will be present in the result. For example:
>
> <div class="doctest">
>
> \>\>\> urljoin('http://www.cwi.nl/%7Eguido/Python.html', ... '//www.python.org/%7Eguido') 'http://www.python.org/%7Eguido'
>
> </div>
>
> If you do not want that behavior, preprocess the *url* with `urlsplit` and `urlunsplit`, removing possible *scheme* and *netloc* parts.

> [!WARNING]
> Because an absolute URL may be passed as the `url` parameter, it is generally **not secure** to use `urljoin` with an attacker-controlled `url`. For example in, `urljoin("https://website.com/users/", username)`, if `username` can contain an absolute URL, the result of `urljoin` will be the absolute URL.

<div class="versionchanged">

3.5

Behavior updated to match the semantics defined in `3986`.

</div>

</div>

<div class="function">

urldefrag(url, \*, missing_as_none=False)

If *url* contains a fragment identifier, return a modified version of *url* with no fragment identifier, and the fragment identifier as a separate string. If there is no fragment identifier in *url*, return *url* unmodified and an empty string (by default) or `None` if *missing_as_none* is true.

The return value is a `named tuple`, its items can be accessed by index or as named attributes:

| Attribute  | Index | Value                | Value if not present       |
|------------|-------|----------------------|----------------------------|
| `url`      | 0     | URL with no fragment | empty string               |
| `fragment` | 1     | Fragment identifier  | `None` or empty string[^5] |

See section `urlparse-result-object` for more information on the result object.

<div class="versionchanged">

3.2 Result is a structured object rather than a simple 2-tuple.

</div>

<div class="versionchanged">

3.15 Added the *missing_as_none* parameter.

</div>

</div>

<div class="function">

unwrap(url)

Extract the url from a wrapped URL (that is, a string formatted as `<URL:scheme://host/path>`, `<scheme://host/path>`, `URL:scheme://host/path` or `scheme://host/path`). If *url* is not a wrapped URL, it is returned without changes.

</div>

## URL parsing security

The `urlsplit` and `urlparse` APIs do not perform **validation** of inputs. They may not raise errors on inputs that other applications consider invalid. They may also succeed on some inputs that might not be considered URLs elsewhere. Their purpose is for practical functionality rather than purity.

Instead of raising an exception on unusual input, they may instead return some component parts as empty strings or `None` (depending on the value of the *missing_as_none* argument). Or components may contain more than perhaps they should.

We recommend that users of these APIs where the values may be used anywhere with security implications code defensively. Do some verification within your code before trusting a returned component part. Does that `scheme` make sense? Is that a sensible `path`? Is there anything strange about that `hostname`? etc.

What constitutes a URL is not universally well defined. Different applications have different needs and desired constraints. For instance the living [WHATWG spec](https://url.spec.whatwg.org/#concept-basic-url-parser) describes what user facing web clients such as a web browser require. While `3986` is more general. These functions incorporate some aspects of both, but cannot be claimed compliant with either. The APIs and existing user code with expectations on specific behaviors predate both standards leading us to be very cautious about making API behavior changes.

## Parsing ASCII Encoded Bytes

The URL parsing functions were originally designed to operate on character strings only. In practice, it is useful to be able to manipulate properly quoted and encoded URLs as sequences of ASCII bytes. Accordingly, the URL parsing functions in this module all operate on `bytes` and `bytearray` objects in addition to `str` objects.

If `str` data is passed in, the result will also contain only `str` data. If `bytes` or `bytearray` data is passed in, the result will contain only `bytes` data.

Attempting to mix `str` data with `bytes` or `bytearray` in a single function call will result in a `TypeError` being raised, while attempting to pass in non-ASCII byte values will trigger `UnicodeDecodeError`.

To support easier conversion of result objects between `str` and `bytes`, all return values from URL parsing functions provide either an `encode` method (when the result contains `str` data) or a `decode` method (when the result contains `bytes` data). The signatures of these methods match those of the corresponding `str` and `bytes` methods (except that the default encoding is `'ascii'` rather than `'utf-8'`). Each produces a value of a corresponding type that contains either `bytes` data (for `encode` methods) or `str` data (for `decode` methods).

Applications that need to operate on potentially improperly quoted URLs that may contain non-ASCII data will need to do their own decoding from bytes to characters before invoking the URL parsing methods.

The behaviour described in this section applies only to the URL parsing functions. The URL quoting functions use their own rules when producing or consuming byte sequences as detailed in the documentation of the individual URL quoting functions.

<div class="versionchanged">

3.2 URL parsing functions now accept ASCII encoded byte sequences

</div>

## Structured Parse Results

The result objects from the `urlsplit`, `urlparse` and `urldefrag` functions are subclasses of the `tuple` type. These subclasses add the attributes listed in the documentation for those functions, the encoding and decoding support described in the previous section, as well as an additional method:

<div class="method">

urllib.parse.SplitResult.geturl()

Return the re-combined version of the original URL as a string. This may differ from the original URL in that the scheme may be normalized to lower case and empty components may be dropped. Specifically, empty parameters, queries, and fragment identifiers will be removed unless the URL was parsed with `missing_as_none=True`.

For `urldefrag` results, only empty fragment identifiers will be removed. For `urlsplit` and `urlparse` results, all noted changes will be made to the URL returned by this method.

The result of this method remains unchanged if passed back through the original parsing function:

> \>\>\> from urllib.parse import urlsplit \>\>\> url = 'HTTP://www.Python.org/doc/#' \>\>\> r1 = urlsplit(url) \>\>\> r1.geturl() 'http://www.Python.org/doc/' \>\>\> r2 = urlsplit(r1.geturl()) \>\>\> r2.geturl() 'http://www.Python.org/doc/' \>\>\> r3 = urlsplit(url, missing_as_none=True) \>\>\> r3.geturl() 'http://www.Python.org/doc/#'

</div>

The following classes provide the implementations of the structured parse results when operating on `str` objects:

<div class="DefragResult(url, fragment)">

Concrete class for `urldefrag` results containing `str` data. The `encode` method returns a `DefragResultBytes` instance.

<div class="versionadded">

3.2

</div>

</div>

<div class="ParseResult(scheme, netloc, path, params, query, fragment)">

Concrete class for `urlparse` results containing `str` data. The `encode` method returns a `ParseResultBytes` instance.

</div>

<div class="SplitResult(scheme, netloc, path, query, fragment)">

Concrete class for `urlsplit` results containing `str` data. The `encode` method returns a `SplitResultBytes` instance.

</div>

The following classes provide the implementations of the parse results when operating on `bytes` or `bytearray` objects:

<div class="DefragResultBytes(url, fragment)">

Concrete class for `urldefrag` results containing `bytes` data. The `decode` method returns a `DefragResult` instance.

<div class="versionadded">

3.2

</div>

</div>

<div class="ParseResultBytes(scheme, netloc, path, params, query, fragment)">

Concrete class for `urlparse` results containing `bytes` data. The `decode` method returns a `ParseResult` instance.

<div class="versionadded">

3.2

</div>

</div>

<div class="SplitResultBytes(scheme, netloc, path, query, fragment)">

Concrete class for `urlsplit` results containing `bytes` data. The `decode` method returns a `SplitResult` instance.

<div class="versionadded">

3.2

</div>

</div>

## URL Quoting

The URL quoting functions focus on taking program data and making it safe for use as URL components by quoting special characters and appropriately encoding non-ASCII text. They also support reversing these operations to recreate the original data from the contents of a URL component if that task isn't already covered by the URL parsing functions above.

<div class="function">

quote(string, safe='/', encoding=None, errors=None)

Replace special characters in *string* using the `%{xx}` escape. Letters, digits, and the characters `'_.-~'` are never quoted. By default, this function is intended for quoting the path section of a URL. The optional *safe* parameter specifies additional ASCII characters that should not be quoted --- its default value is `'/'`.

*string* may be either a `str` or a `bytes` object.

<div class="versionchanged">

3.7 Moved from `2396` to `3986` for quoting URL strings. "~" is now included in the set of unreserved characters.

</div>

The optional *encoding* and *errors* parameters specify how to deal with non-ASCII characters, as accepted by the `str.encode` method. Although these parameters default to `None` in the function signature, when processing `str` inputs, *encoding* effectively defaults to `'utf-8'` and *errors* to `'strict'`, meaning unsupported characters raise a `UnicodeEncodeError`. *encoding* and *errors* must not be supplied if *string* is a `bytes`, or a `TypeError` is raised.

Note that `quote(string, safe, encoding, errors)` is equivalent to `quote_from_bytes(string.encode(encoding, errors), safe)`.

Example: `quote('/El Niño/')` yields `'/El%20Ni%C3%B1o/'`.

</div>

<div class="function">

quote_plus(string, safe='', encoding=None, errors=None)

Like `quote`, but also replace spaces with plus signs, as required for quoting HTML form values when building up a query string to go into a URL. Plus signs in the original string are escaped unless they are included in *safe*. It also does not have *safe* default to `'/'`.

Example: `quote_plus('/El Niño/')` yields `'%2FEl+Ni%C3%B1o%2F'`.

</div>

<div class="function">

quote_from_bytes(bytes, safe='/')

Like `quote`, but accepts a `bytes` object rather than a `str`, and does not perform string-to-bytes encoding.

Example: `quote_from_bytes(b'a&\xef')` yields `'a%26%EF'`.

</div>

<div class="function">

unquote(string, encoding='utf-8', errors='replace')

Replace `%{xx}` escapes with their single-character equivalent. The optional *encoding* and *errors* parameters specify how to decode percent-encoded sequences into Unicode characters, as accepted by the `bytes.decode` method.

*string* may be either a `str` or a `bytes` object.

*encoding* defaults to `'utf-8'`. *errors* defaults to `'replace'`, meaning invalid sequences are replaced by a placeholder character.

Example: `unquote('/El%20Ni%C3%B1o/')` yields `'/El Niño/'`.

<div class="versionchanged">

3.9 *string* parameter supports bytes and str objects (previously only str).

</div>

</div>

<div class="function">

unquote_plus(string, encoding='utf-8', errors='replace')

Like `unquote`, but also replace plus signs with spaces, as required for unquoting HTML form values.

*string* must be a `str`.

Example: `unquote_plus('/El+Ni%C3%B1o/')` yields `'/El Niño/'`.

</div>

<div class="function">

unquote_to_bytes(string)

Replace `%{xx}` escapes with their single-octet equivalent, and return a `bytes` object.

*string* may be either a `str` or a `bytes` object.

If it is a `str`, unescaped non-ASCII characters in *string* are encoded into UTF-8 bytes.

Example: `unquote_to_bytes('a%26%EF')` yields `b'a&\xef'`.

</div>

<div class="function">

urlencode(query, doseq=False, safe='', encoding=None, errors=None, quote_via=quote_plus)

Convert a mapping object or a sequence of two-element tuples, which may contain `str` or `bytes` objects, to a percent-encoded ASCII text string. If the resultant string is to be used as a *data* for POST operation with the `~urllib.request.urlopen` function, then it should be encoded to bytes, otherwise it would result in a `TypeError`.

The resulting string is a series of `key=value` pairs separated by `'&'` characters, where both *key* and *value* are quoted using the *quote_via* function. By default, `quote_plus` is used to quote the values, which means spaces are quoted as a `'+'` character and '/' characters are encoded as `%2F`, which follows the standard for GET requests (`application/x-www-form-urlencoded`). An alternate function that can be passed as *quote_via* is `quote`, which will encode spaces as `%20` and not encode '/' characters. For maximum control of what is quoted, use `quote` and specify a value for *safe*.

When a sequence of two-element tuples is used as the *query* argument, the first element of each tuple is a key and the second is a value. The value element in itself can be a sequence and in that case, if the optional parameter *doseq* evaluates to `True`, individual `key=value` pairs separated by `'&'` are generated for each element of the value sequence for the key. The order of parameters in the encoded string will match the order of parameter tuples in the sequence.

The *safe*, *encoding*, and *errors* parameters are passed down to *quote_via* (the *encoding* and *errors* parameters are only passed when a query element is a `str`).

To reverse this encoding process, `parse_qs` and `parse_qsl` are provided in this module to parse query strings into Python data structures.

Refer to `urllib examples <urllib-examples>` to find out how the `urllib.parse.urlencode` method can be used for generating the query string of a URL or data for a POST request.

<div class="versionchanged">

3.2 *query* supports bytes and string objects.

</div>

<div class="versionchanged">

3.5 Added the *quote_via* parameter.

</div>

<div class="deprecated">

3.14 Accepting objects with false values (like `0` and `[]`) except empty strings and byte-like objects and `None` is now deprecated.

</div>

</div>

<div class="seealso">

[WHATWG](https://url.spec.whatwg.org/) - URL Living standard  
Working Group for the URL Standard that defines URLs, domains, IP addresses, the application/x-www-form-urlencoded format, and their API.

`3986` - Uniform Resource Identifiers  
This is the current standard (STD66). Any changes to urllib.parse module should conform to this. Certain deviations could be observed, which are mostly for backward compatibility purposes and for certain de-facto parsing requirements as commonly observed in major browsers.

`2732` - Format for Literal IPv6 Addresses in URL's.  
This specifies the parsing requirements of IPv6 URLs.

`2396` - Uniform Resource Identifiers (URI): Generic Syntax  
Document describing the generic syntactic requirements for both Uniform Resource Names (URNs) and Uniform Resource Locators (URLs).

`2368` - The mailto URL scheme.  
Parsing requirements for mailto URL schemes.

`1808` - Relative Uniform Resource Locators  
This Request For Comments includes the rules for joining an absolute and a relative URL, including a fair number of "Abnormal Examples" which govern the treatment of border cases.

`1738` - Uniform Resource Locators (URL)  
This specifies the formal syntax and semantics of absolute URLs.

</div>

[^1]: Depending on the value of the *missing_as_none* argument.

[^2]: Depending on the value of the *missing_as_none* argument.

[^3]: Depending on the value of the *missing_as_none* argument.

[^4]: Depending on the value of the *missing_as_none* argument.

[^5]: Depending on the value of the *missing_as_none* argument.

# `!urllib.request` --- Extensible library for opening URLs

<div class="module" data-synopsis="Extensible library for opening URLs.">

urllib.request

</div>

**Source code:** `Lib/urllib/request.py`

------------------------------------------------------------------------

The `!urllib.request` module defines functions and classes which help in opening URLs (mostly HTTP) in a complex world --- basic and digest authentication, redirections, cookies and more.

<div class="seealso">

The [Requests package](https://requests.readthedocs.io/en/master/) is recommended for a higher-level HTTP client interface.

</div>

> [!WARNING]
> On macOS it is unsafe to use this module in programs using `os.fork` because the `getproxies` implementation for macOS uses a higher-level system API. Set the environment variable `no_proxy` to `*` to avoid this problem (e.g. `os.environ["no_proxy"] = "*"`).

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

The `!urllib.request` module defines the following functions:

<div class="function">

urlopen(url, data=None\[, timeout\], \*, context=None)

Open *url*, which can be either a string containing a valid, properly encoded URL, or a `Request` object.

*data* must be an object specifying additional data to be sent to the server, or `None` if no such data is needed. See `Request` for details.

urllib.request module uses HTTP/1.1 and includes `Connection:close` header in its HTTP requests.

The optional *timeout* parameter specifies a timeout in seconds for blocking operations like the connection attempt (if not specified, the global default timeout setting will be used). This actually only works for HTTP, HTTPS and FTP connections.

If *context* is specified, it must be a `ssl.SSLContext` instance describing the various SSL options. See `~http.client.HTTPSConnection` for more details.

This function always returns an object which can work as a `context manager` and has the properties *url*, *headers*, and *status*. See `urllib.response.addinfourl` for more detail on these properties.

For HTTP and HTTPS URLs, this function returns a `http.client.HTTPResponse` object slightly modified. In addition to the three new methods above, the msg attribute contains the same information as the `~http.client.HTTPResponse.reason` attribute --- the reason phrase returned by server --- instead of the response headers as it is specified in the documentation for `~http.client.HTTPResponse`.

For FTP, file, and data URLs, this function returns a `urllib.response.addinfourl` object.

Raises `~urllib.error.URLError` on protocol errors.

Exceptions raised while reading or parsing an HTTP response are not wrapped in `~urllib.error.URLError` and propagate unchanged. These include subclasses of `http.client.HTTPException`, such as `~http.client.BadStatusLine` for a malformed status line, and subclasses of `OSError`, such as `TimeoutError` if the server does not respond within *timeout* seconds, or `~http.client.RemoteDisconnected` if the server closes the connection without sending a response. `ValueError` is raised if *url* is not a valid URL.

Note that `None` may be returned if no handler handles the request (though the default installed global `OpenerDirector` uses `UnknownHandler` to ensure this never happens).

In addition, if proxy settings are detected (for example, when a `*_proxy` environment variable like `!http_proxy` is set), `ProxyHandler` is default installed and makes sure the requests are handled through the proxy.

The legacy `urllib.urlopen` function from Python 2.6 and earlier has been discontinued; `urllib.request.urlopen` corresponds to the old `urllib2.urlopen`. Proxy handling, which was done by passing a dictionary parameter to `urllib.urlopen`, can be obtained by using `ProxyHandler` objects.

<div class="audit-event">

urllib.Request fullurl,data,headers,method urllib.request.urlopen

The default opener raises an `auditing event <auditing>` `urllib.Request` with arguments `fullurl`, `data`, `headers`, `method` taken from the request object.

</div>

<div class="versionchanged">

3.2 *cafile* and *capath* were added.

HTTPS virtual hosts are now supported if possible (that is, if `ssl.HAS_SNI` is true).

*data* can be an iterable object.

</div>

<div class="versionchanged">

3.3 *cadefault* was added.

</div>

<div class="versionchanged">

3.4.3 *context* was added.

</div>

<div class="versionchanged">

3.10 HTTPS connection now send an ALPN extension with protocol indicator `http/1.1` when no *context* is given. Custom *context* should set ALPN protocols with `~ssl.SSLContext.set_alpn_protocols`.

</div>

<div class="versionchanged">

3.13 Remove *cafile*, *capath* and *cadefault* parameters: use the *context* parameter instead.

</div>

</div>

<div class="function">

install_opener(opener)

Install an `OpenerDirector` instance as the default global opener. Installing an opener is only necessary if you want urlopen to use that opener; otherwise, simply call `OpenerDirector.open` instead of `~urllib.request.urlopen`. The code does not check for a real `OpenerDirector`, and any class with the appropriate interface will work.

</div>

<div class="function">

build_opener(\[handler, ...\])

Return an `OpenerDirector` instance, which chains the handlers in the order given. *handler*s can be either instances of `BaseHandler`, or subclasses of `BaseHandler` (in which case it must be possible to call the constructor without any parameters). Instances of the following classes will be in front of the *handler*s, unless the *handler*s contain them, instances of them or subclasses of them: `ProxyHandler` (if proxy settings are detected), `UnknownHandler`, `HTTPHandler`, `HTTPDefaultErrorHandler`, `HTTPRedirectHandler`, `FTPHandler`, `FileHandler`, `HTTPErrorProcessor`.

If the Python installation has SSL support (i.e., if the `ssl` module can be imported), `HTTPSHandler` will also be added.

A `BaseHandler` subclass may also change its `handler_order` attribute to modify its position in the handlers list.

</div>

<div class="function">

pathname2url(path, \*, add_scheme=False)

Convert the given local path to a `file:` URL. This function uses `~urllib.parse.quote` function to encode the path.

If *add_scheme* is false (the default), the return value omits the `file:` scheme prefix. Set *add_scheme* to true to return a complete URL.

This example shows the function being used on Windows:

    >>> from urllib.request import pathname2url
    >>> path = 'C:\\Program Files'
    >>> pathname2url(path, add_scheme=True)
    'file:///C:/Program%20Files'

<div class="versionchanged">

3.14 Windows drive letters are no longer converted to uppercase, and `:` characters not following a drive letter no longer cause an `OSError` exception to be raised on Windows.

</div>

<div class="versionchanged">

3.14 Paths beginning with a slash are converted to URLs with authority sections. For example, the path `/etc/hosts` is converted to the URL `///etc/hosts`.

</div>

<div class="versionchanged">

3.14 The *add_scheme* parameter was added.

</div>

</div>

<div class="function">

url2pathname(url, \*, require_scheme=False, resolve_host=False)

Convert the given `file:` URL to a local path. This function uses `~urllib.parse.unquote` to decode the URL.

If *require_scheme* is false (the default), the given value should omit a `file:` scheme prefix. If *require_scheme* is set to true, the given value should include the prefix; a `~urllib.error.URLError` is raised if it doesn't.

The URL authority is discarded if it is empty, `localhost`, or the local hostname. Otherwise, if *resolve_host* is set to true, the authority is resolved using `socket.gethostbyname` and discarded if it matches a local IP address (as per `RFC 8089 §3 <8089#section-3>`). If the authority is still unhandled, then on Windows a UNC path is returned, and on other platforms a `~urllib.error.URLError` is raised.

This example shows the function being used on Windows:

    >>> from urllib.request import url2pathname
    >>> url = 'file:///C:/Program%20Files'
    >>> url2pathname(url, require_scheme=True)
    'C:\\Program Files'

<div class="versionchanged">

3.14 Windows drive letters are no longer converted to uppercase, and `:` characters not following a drive letter no longer cause an `OSError` exception to be raised on Windows.

</div>

<div class="versionchanged">

3.14 The URL authority is discarded if it matches the local hostname. Otherwise, if the authority isn't empty or `localhost`, then on Windows a UNC path is returned (as before), and on other platforms a `~urllib.error.URLError` is raised.

</div>

<div class="versionchanged">

3.14 The URL query and fragment components are discarded if present.

</div>

<div class="versionchanged">

3.14 The *require_scheme* and *resolve_host* parameters were added.

</div>

</div>

<div class="function">

getproxies()

This helper function returns a dictionary of scheme to proxy server URL mappings. It scans the environment for variables named `<scheme>_proxy`, in a case insensitive approach, for all operating systems first, and when it cannot find it, looks for proxy information from System Configuration for macOS and Windows Systems Registry for Windows. If both lowercase and uppercase environment variables exist (and disagree), lowercase is preferred.

> [!NOTE]
> If the environment variable `REQUEST_METHOD` is set, which usually indicates your script is running in a CGI environment, the environment variable `HTTP_PROXY` (uppercase `_PROXY`) will be ignored. This is because that variable can be injected by a client using the "Proxy:" HTTP header. If you need to use an HTTP proxy in a CGI environment, either use `ProxyHandler` explicitly, or make sure the variable name is in lowercase (or at least the `_proxy` suffix).

</div>

The following classes are provided:

<div class="Request(url, data=None, headers={}, origin_req_host=None, unverifiable=False, method=None)">

This class is an abstraction of a URL request.

*url* should be a string containing a valid, properly encoded URL.

*data* must be an object specifying additional data to send to the server, or `None` if no such data is needed. Currently HTTP requests are the only ones that use *data*. The supported object types include bytes, file-like objects, and iterables of bytes-like objects. If no `Content-Length` nor `Transfer-Encoding` header field has been provided, `HTTPHandler` will set these headers according to the type of *data*. `Content-Length` will be used to send bytes objects, while `Transfer-Encoding: chunked` as specified in `7230`, Section 3.3.1 will be used to send files and other iterables.

For an HTTP POST request method, *data* should be a buffer in the standard `application/x-www-form-urlencoded` format. The `urllib.parse.urlencode` function takes a mapping or sequence of 2-tuples and returns an ASCII string in this format. It should be encoded to bytes before being used as the *data* parameter.

*headers* should be a dictionary, and will be treated as if `add_header` was called with each key and value as arguments. This is often used to "spoof" the `User-Agent` header value, which is used by a browser to identify itself -- some HTTP servers only allow requests coming from common browsers as opposed to scripts. For example, Mozilla Firefox may identify itself as `"Mozilla/5.0 (X11; U; Linux i686) Gecko/20071127 Firefox/2.0.0.11"`, while `urllib`'s default user agent string is `"Python-urllib/2.6"` (on Python 2.6). All header keys are sent in camel case.

An appropriate `Content-Type` header should be included if the *data* argument is present. If this header has not been provided and *data* is not `None`, `Content-Type: application/x-www-form-urlencoded` will be added as a default.

The next two arguments are only of interest for correct handling of third-party HTTP cookies:

*origin_req_host* should be the request-host of the origin transaction, as defined by `2965`. It defaults to `http.cookiejar.request_host(self)`. This is the host name or IP address of the original request that was initiated by the user. For example, if the request is for an image in an HTML document, this should be the request-host of the request for the page containing the image.

*unverifiable* should indicate whether the request is unverifiable, as defined by `2965`. It defaults to `False`. An unverifiable request is one whose URL the user did not have the option to approve. For example, if the request is for an image in an HTML document, and the user had no option to approve the automatic fetching of the image, this should be true.

*method* should be a string that indicates the HTTP request method that will be used (e.g. `'HEAD'`). If provided, its value is stored in the `~Request.method` attribute and is used by `get_method`. The default is `'GET'` if *data* is `None` or `'POST'` otherwise. Subclasses may indicate a different default method by setting the `~Request.method` attribute in the class itself.

> [!NOTE]
> The request will not work as expected if the data object is unable to deliver its content more than once (e.g. a file or an iterable that can produce the content only once) and the request is retried for HTTP redirects or authentication. The *data* is sent to the HTTP server right away after the headers. There is no support for a 100-continue expectation in the library.

<div class="versionchanged">

3.3 `Request.method` argument is added to the Request class.

</div>

<div class="versionchanged">

3.4 Default `Request.method` may be indicated at the class level.

</div>

<div class="versionchanged">

3.6 Do not raise an error if the `Content-Length` has not been provided and *data* is neither `None` nor a bytes object. Fall back to use chunked transfer encoding instead.

</div>

</div>

<div class="OpenerDirector()">

The `OpenerDirector` class opens URLs via `BaseHandler`s chained together. It manages the chaining of handlers, and recovery from errors.

</div>

<div class="BaseHandler()">

This is the base class for all registered handlers --- and handles only the simple mechanics of registration.

</div>

<div class="HTTPDefaultErrorHandler()">

A class which defines a default handler for HTTP error responses; all responses are turned into `~urllib.error.HTTPError` exceptions.

</div>

<div class="HTTPRedirectHandler()">

A class to handle redirections.

</div>

<div class="HTTPCookieProcessor(cookiejar=None)">

A class to handle HTTP Cookies.

</div>

<div class="ProxyHandler(proxies=None)">

Cause requests to go through a proxy. If *proxies* is given, it must be a dictionary mapping protocol names to URLs of proxies. The default is to read the list of proxies from the environment variables `<protocol>_proxy`. If no proxy environment variables are set, then in a Windows environment proxy settings are obtained from the registry's Internet Settings section, and in a macOS environment proxy information is retrieved from the System Configuration Framework.

To disable autodetected proxy pass an empty dictionary.

The `no_proxy` environment variable can be used to specify hosts which shouldn't be reached via proxy; if set, it should be a comma-separated list of hostname suffixes, optionally with `:port` appended, for example `cern.ch,ncsa.uiuc.edu,some.host:8080`.

> [!NOTE]
> `HTTP_PROXY` will be ignored if a variable `REQUEST_METHOD` is set; see the documentation on `~urllib.request.getproxies`.

</div>

<div class="HTTPPasswordMgr()">

Keep a database of `(realm, uri) -> (user, password)` mappings.

</div>

<div class="HTTPPasswordMgrWithDefaultRealm()">

Keep a database of `(realm, uri) -> (user, password)` mappings. A realm of `None` is considered a catch-all realm, which is searched if no other realm fits.

</div>

<div class="HTTPPasswordMgrWithPriorAuth()">

A variant of `HTTPPasswordMgrWithDefaultRealm` that also has a database of `uri -> is_authenticated` mappings. Can be used by a BasicAuth handler to determine when to send authentication credentials immediately instead of waiting for a `401` response first.

<div class="versionadded">

3.5

</div>

</div>

<div class="AbstractBasicAuthHandler(password_mgr=None)">

This is a mixin class that helps with HTTP authentication, both to the remote host and to a proxy. *password_mgr*, if given, should be something that is compatible with `HTTPPasswordMgr`; refer to section `http-password-mgr` for information on the interface that must be supported. If *passwd_mgr* also provides `is_authenticated` and `update_authenticated` methods (see `http-password-mgr-with-prior-auth`), then the handler will use the `is_authenticated` result for a given URI to determine whether or not to send authentication credentials with the request. If `is_authenticated` returns `True` for the URI, credentials are sent. If `is_authenticated` is `False`, credentials are not sent, and then if a `401` response is received the request is re-sent with the authentication credentials. If authentication succeeds, `update_authenticated` is called to set `is_authenticated` `True` for the URI, so that subsequent requests to the URI or any of its super-URIs will automatically include the authentication credentials.

<div class="versionadded">

3.5 Added `is_authenticated` support.

</div>

</div>

<div class="HTTPBasicAuthHandler(password_mgr=None)">

Handle authentication with the remote host. *password_mgr*, if given, should be something that is compatible with `HTTPPasswordMgr`; refer to section `http-password-mgr` for information on the interface that must be supported. HTTPBasicAuthHandler will raise a `ValueError` when presented with a wrong Authentication scheme.

</div>

<div class="ProxyBasicAuthHandler(password_mgr=None)">

Handle authentication with the proxy. *password_mgr*, if given, should be something that is compatible with `HTTPPasswordMgr`; refer to section `http-password-mgr` for information on the interface that must be supported.

</div>

<div class="AbstractDigestAuthHandler(password_mgr=None)">

This is a mixin class that helps with HTTP authentication, both to the remote host and to a proxy. *password_mgr*, if given, should be something that is compatible with `HTTPPasswordMgr`; refer to section `http-password-mgr` for information on the interface that must be supported.

<div class="versionchanged">

3.14 Added support for HTTP digest authentication algorithm `SHA-256`.

</div>

</div>

<div class="HTTPDigestAuthHandler(password_mgr=None)">

Handle authentication with the remote host. *password_mgr*, if given, should be something that is compatible with `HTTPPasswordMgr`; refer to section `http-password-mgr` for information on the interface that must be supported. When both Digest Authentication Handler and Basic Authentication Handler are both added, Digest Authentication is always tried first. If the Digest Authentication returns a 40x response again, it is sent to Basic Authentication handler to Handle. This Handler method will raise a `ValueError` when presented with an authentication scheme other than Digest or Basic.

<div class="versionchanged">

3.3 Raise `ValueError` on unsupported Authentication Scheme.

</div>

</div>

<div class="ProxyDigestAuthHandler(password_mgr=None)">

Handle authentication with the proxy. *password_mgr*, if given, should be something that is compatible with `HTTPPasswordMgr`; refer to section `http-password-mgr` for information on the interface that must be supported.

</div>

<div class="HTTPHandler()">

A class to handle opening of HTTP URLs.

</div>

<div class="HTTPSHandler(debuglevel=0, context=None, check_hostname=None)">

A class to handle opening of HTTPS URLs. *context* and *check_hostname* have the same meaning as in `http.client.HTTPSConnection`.

<div class="versionchanged">

3.2 *context* and *check_hostname* were added.

</div>

</div>

<div class="FileHandler()">

Open local files.

</div>

<div class="DataHandler()">

Open data URLs.

<div class="versionadded">

3.4

</div>

</div>

<div class="FTPHandler()">

Open FTP URLs.

</div>

<div class="CacheFTPHandler()">

Open FTP URLs, keeping a cache of open FTP connections to minimize delays.

</div>

<div class="UnknownHandler()">

A catch-all class to handle unknown URLs.

</div>

<div class="HTTPErrorProcessor()">

Process HTTP error responses.

</div>

## Request Objects

The following methods describe `Request`'s public interface, and so all may be overridden in subclasses. It also defines several public attributes that can be used by clients to inspect the parsed request.

<div class="attribute">

Request.full_url

The original URL passed to the constructor.

<div class="versionchanged">

3.4

</div>

Request.full_url is a property with setter, getter and a deleter. Getting `~Request.full_url` returns the original request URL with the fragment, if it was present.

</div>

<div class="attribute">

Request.type

The URI scheme.

</div>

<div class="attribute">

Request.host

The URI authority, typically a host, but may also contain a port separated by a colon.

</div>

<div class="attribute">

Request.origin_req_host

The original host for the request, without port.

</div>

<div class="attribute">

Request.selector

The URI path. If the `Request` uses a proxy, then selector will be the full URL that is passed to the proxy.

</div>

<div class="attribute">

Request.data

The entity body for the request, or `None` if not specified.

<div class="versionchanged">

3.4 Changing value of `Request.data` now deletes "Content-Length" header if it was previously set or calculated.

</div>

</div>

<div class="attribute">

Request.unverifiable

boolean, indicates whether the request is unverifiable as defined by `2965`.

</div>

<div class="attribute">

Request.method

The HTTP request method to use. By default its value is `None`, which means that `~Request.get_method` will do its normal computation of the method to be used. Its value can be set (thus overriding the default computation in `~Request.get_method`) either by providing a default value by setting it at the class level in a `Request` subclass, or by passing a value in to the `Request` constructor via the *method* argument.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.4 A default value can now be set in subclasses; previously it could only be set via the constructor argument.

</div>

</div>

<div class="method">

Request.get_method()

Return a string indicating the HTTP request method. If `Request.method` is not `None`, return its value, otherwise return `'GET'` if `Request.data` is `None`, or `'POST'` if it's not. This is only meaningful for HTTP requests.

<div class="versionchanged">

3.3 get_method now looks at the value of `Request.method`.

</div>

</div>

<div class="method">

Request.add_header(key, val)

Add another header to the request. Headers are currently ignored by all handlers except HTTP handlers, where they are added to the list of headers sent to the server. Note that there cannot be more than one header with the same name, and later calls will overwrite previous calls in case the *key* collides. Currently, this is no loss of HTTP functionality, since all headers which have meaning when used more than once have a (header-specific) way of gaining the same functionality using only one header. Note that headers added using this method are also added to redirected requests.

</div>

<div class="method">

Request.add_unredirected_header(key, header)

Add a header that will not be added to a redirected request.

</div>

<div class="method">

Request.has_header(header)

Return whether the instance has the named header (checks both regular and unredirected).

</div>

<div class="method">

Request.remove_header(header)

Remove named header from the request instance (both from regular and unredirected headers).

<div class="versionadded">

3.4

</div>

</div>

<div class="method">

Request.get_full_url()

Return the URL given in the constructor.

<div class="versionchanged">

3.4

</div>

Returns `Request.full_url`

</div>

<div class="method">

Request.set_proxy(host, type)

Prepare the request by connecting to a proxy server. The *host* and *type* will replace those of the instance, and the instance's selector will be the original URL given in the constructor.

</div>

<div class="method">

Request.get_header(header_name, default=None)

Return the value of the given header. If the header is not present, return the default value.

</div>

<div class="method">

Request.header_items()

Return a list of tuples (header_name, header_value) of the Request headers.

</div>

<div class="versionchanged">

3.4 The request methods add_data, has_data, get_data, get_type, get_host, get_selector, get_origin_req_host and is_unverifiable that were deprecated since 3.3 have been removed.

</div>

## OpenerDirector Objects

`OpenerDirector` instances have the following methods:

<div class="method">

OpenerDirector.add_handler(handler)

*handler* should be an instance of `BaseHandler`. The following methods are searched, and added to the possible chains (note that HTTP errors are a special case). Note that, in the following, *protocol* should be replaced with the actual protocol to handle, for example `http_response` would be the HTTP protocol response handler. Also *type* should be replaced with the actual HTTP code, for example `http_error_404` would handle HTTP 404 errors.

- `!<protocol>_open` --- signal that the handler knows how to open *protocol* URLs.

  See [`BaseHandler.<protocol>_open`](..%20method::%20BaseHandler.%3Cprotocol%3E_open(req):noindex:) for more information.

- `!http_error_\<type\>` --- signal that the handler knows how to handle HTTP errors with HTTP error code *type*.

  See [`BaseHandler.http_error_\<nnn\>`](..%20method::%20BaseHandler.http_error_%3Cnnn%3E(req,%20fp,%20code,%20msg,%20hdrs)) for more information.

- `!<protocol>_error` --- signal that the handler knows how to handle errors from (non-`http`) *protocol*.

- `!<protocol>_request` --- signal that the handler knows how to pre-process *protocol* requests.

  See [`BaseHandler.<protocol>_request`](..%20method::%20BaseHandler.%3Cprotocol%3E_request(req):noindex:) for more information.

- `!<protocol>_response` --- signal that the handler knows how to post-process *protocol* responses.

  See [`BaseHandler.<protocol>_response`](..%20method::%20BaseHandler.%3Cprotocol%3E_response(req,%20response):noindex:) for more information.

</div>

<div class="method">

OpenerDirector.open(url, data=None\[, timeout\])

Open the given *url* (which can be a request object or a string), optionally passing the given *data*. Arguments, return values and exceptions raised are the same as those of `urlopen` (which simply calls the `open` method on the currently installed global `OpenerDirector`). The optional *timeout* parameter specifies a timeout in seconds for blocking operations like the connection attempt (if not specified, the global default timeout setting will be used). The timeout feature actually works only for HTTP, HTTPS and FTP connections.

</div>

<div class="method">

OpenerDirector.error(proto, \*args)

Handle an error of the given protocol. This will call the registered error handlers for the given protocol with the given arguments (which are protocol specific). The HTTP protocol is a special case which uses the HTTP response code to determine the specific error handler; refer to the `!http_error_\<type\>` methods of the handler classes.

Return values and exceptions raised are the same as those of `urlopen`.

</div>

OpenerDirector objects open URLs in three stages:

The order in which these methods are called within each stage is determined by sorting the handler instances.

1.  Every handler with a method named like `!<protocol>_request` has that method called to pre-process the request.

2.  Handlers with a method named like `!<protocol>_open` are called to handle the request. This stage ends when a handler either returns a non-`None` value (ie. a response), or raises an exception (usually `~urllib.error.URLError`). Exceptions are allowed to propagate.

    In fact, the above algorithm is first tried for methods named `~BaseHandler.default_open`. If all such methods return `None`, the algorithm is repeated for methods named like `!<protocol>_open`. If all such methods return `None`, the algorithm is repeated for methods named `~BaseHandler.unknown_open`.

    Note that the implementation of these methods may involve calls of the parent `OpenerDirector` instance's `~OpenerDirector.open` and `~OpenerDirector.error` methods.

3.  Every handler with a method named like `!<protocol>_response` has that method called to post-process the response.

## BaseHandler Objects

`BaseHandler` objects provide a couple of methods that are directly useful, and others that are meant to be used by derived classes. These are intended for direct use:

<div class="method">

BaseHandler.add_parent(director)

Add a director as parent.

</div>

<div class="method">

BaseHandler.close()

Remove any parents.

</div>

The following attribute and methods should only be used by classes derived from `BaseHandler`.

> [!NOTE]
> The convention has been adopted that subclasses defining `!<protocol>_request` or `!<protocol>_response` methods are named `!\*Processor`; all others are named `!\*Handler`.

<div class="attribute">

BaseHandler.parent

A valid `OpenerDirector`, which can be used to open using a different protocol, or handle errors.

</div>

<div class="method">

BaseHandler.default_open(req)

This method is *not* defined in `BaseHandler`, but subclasses should define it if they want to catch all URLs.

This method, if implemented, will be called by the parent `OpenerDirector`. It should return a file-like object as described in the return value of the `~OpenerDirector.open` method of `OpenerDirector`, or `None`. It should raise `~urllib.error.URLError`, unless a truly exceptional thing happens (for example, `MemoryError` should not be mapped to `~urllib.error.URLError`).

This method will be called before any protocol-specific open method.

</div>

> This method is *not* defined in `BaseHandler`, but subclasses should define it if they want to handle URLs with the given protocol.
>
> This method, if defined, will be called by the parent `OpenerDirector`. Return values should be the same as for `~BaseHandler.default_open`.

<div class="method">

BaseHandler.unknown_open(req)

This method is *not* defined in `BaseHandler`, but subclasses should define it if they want to catch all URLs with no specific registered handler to open it.

This method, if implemented, will be called by the `parent` `OpenerDirector`. Return values should be the same as for `default_open`.

</div>

<div class="method">

BaseHandler.http_error_default(req, fp, code, msg, hdrs)

This method is *not* defined in `BaseHandler`, but subclasses should override it if they intend to provide a catch-all for otherwise unhandled HTTP errors. It will be called automatically by the `OpenerDirector` getting the error, and should not normally be called in other circumstances.

`OpenerDirector` will call this method with five positional arguments:

1.  a `Request` object,
2.  a file-like object with the HTTP error body,
3.  the three-digit code of the error, as a string,
4.  the user-visible explanation of the code, as a string, and
5.  the headers of the error, as a mapping object.

Return values and exceptions raised should be the same as those of `urlopen`.

</div>

> *nnn* should be a three-digit HTTP error code. This method is also not defined in `BaseHandler`, but will be called, if it exists, on an instance of a subclass, when an HTTP error with code *nnn* occurs.
>
> Subclasses should override this method to handle specific HTTP errors.
>
> Arguments, return values and exceptions raised should be the same as for `~BaseHandler.http_error_default`.

> This method is *not* defined in `BaseHandler`, but subclasses should define it if they want to pre-process requests of the given protocol.
>
> This method, if defined, will be called by the parent `OpenerDirector`. *req* will be a `Request` object. The return value should be a `Request` object.

> This method is *not* defined in `BaseHandler`, but subclasses should define it if they want to post-process responses of the given protocol.
>
> This method, if defined, will be called by the parent `OpenerDirector`. *req* will be a `Request` object. *response* will be an object implementing the same interface as the return value of `urlopen`. The return value should implement the same interface as the return value of `urlopen`.

## HTTPRedirectHandler Objects

> [!NOTE]
> Some HTTP redirections require action from this module's client code. If this is the case, `~urllib.error.HTTPError` is raised. See `2616` for details of the precise meanings of the various redirection codes.
>
> An `~urllib.error.HTTPError` exception raised as a security consideration if the HTTPRedirectHandler is presented with a redirected URL which is not an HTTP, HTTPS or FTP URL.

<div class="method">

HTTPRedirectHandler.redirect_request(req, fp, code, msg, hdrs, newurl)

Return a `Request` or `None` in response to a redirect. This is called by the default implementations of the `!http_error_30\*` methods when a redirection is received from the server. If a redirection should take place, return a new `Request` to allow `!http_error_30\*` to perform the redirect to *newurl*. Otherwise, raise `~urllib.error.HTTPError` if no other handler should try to handle this URL, or return `None` if you can't but another handler might.

> [!NOTE]
> The default implementation of this method does not strictly follow `2616`, which says that 301 and 302 responses to `POST` requests must not be automatically redirected without confirmation by the user. In reality, browsers do allow automatic redirection of these responses, changing the POST to a `GET`, and the default implementation reproduces this behavior.

</div>

<div class="method">

HTTPRedirectHandler.http_error_301(req, fp, code, msg, hdrs)

Redirect to the `Location:` or `URI:` URL. This method is called by the parent `OpenerDirector` when getting an HTTP 'moved permanently' response.

</div>

<div class="method">

HTTPRedirectHandler.http_error_302(req, fp, code, msg, hdrs)

The same as `http_error_301`, but called for the 'found' response.

</div>

<div class="method">

HTTPRedirectHandler.http_error_303(req, fp, code, msg, hdrs)

The same as `http_error_301`, but called for the 'see other' response.

</div>

<div class="method">

HTTPRedirectHandler.http_error_307(req, fp, code, msg, hdrs)

The same as `http_error_301`, but called for the 'temporary redirect' response. It does not allow changing the request method from `POST` to `GET`.

</div>

<div class="method">

HTTPRedirectHandler.http_error_308(req, fp, code, msg, hdrs)

The same as `http_error_301`, but called for the 'permanent redirect' response. It does not allow changing the request method from `POST` to `GET`.

<div class="versionadded">

3.11

</div>

</div>

## HTTPCookieProcessor Objects

`HTTPCookieProcessor` instances have one attribute:

<div class="attribute">

HTTPCookieProcessor.cookiejar

The `http.cookiejar.CookieJar` in which cookies are stored.

</div>

## ProxyHandler Objects

<div class="method" data-noindex="">

ProxyHandler.\<protocol\>\_open(request)

The `ProxyHandler` will have a method `!<protocol>_open` for every *protocol* which has a proxy in the *proxies* dictionary given in the constructor. The method will modify requests to go through the proxy, by calling `request.set_proxy()`, and call the next handler in the chain to actually execute the protocol.

</div>

## HTTPPasswordMgr Objects

These methods are available on `HTTPPasswordMgr` and `HTTPPasswordMgrWithDefaultRealm` objects.

<div class="method">

HTTPPasswordMgr.add_password(realm, uri, user, passwd)

*uri* can be either a single URI, or a sequence of URIs. *realm*, *user* and *passwd* must be strings. This causes `(user, passwd)` to be used as authentication tokens when authentication for *realm* and a super-URI of any of the given URIs is given. If a URI includes a scheme, its credentials only match authentication URIs with the same scheme or no scheme. A URI without a scheme matches authentication URIs with any scheme.

<div class="versionchanged">

3.16 Authentication credentials for URIs with a scheme are now scoped by that scheme.

</div>

</div>

<div class="method">

HTTPPasswordMgr.find_user_password(realm, authuri)

Get user/password for given realm and URI, if any. This method will return `(None, None)` if there is no matching user/password.

For `HTTPPasswordMgrWithDefaultRealm` objects, the realm `None` will be searched if the given *realm* has no matching user/password.

</div>

## HTTPPasswordMgrWithPriorAuth Objects

This password manager extends `HTTPPasswordMgrWithDefaultRealm` to support tracking URIs for which authentication credentials should always be sent.

<div class="method">

HTTPPasswordMgrWithPriorAuth.add_password(realm, uri, user, passwd, is_authenticated=False)

*realm*, *uri*, *user*, *passwd* are as for `HTTPPasswordMgr.add_password`. *is_authenticated* sets the initial value of the `is_authenticated` flag for the given URI or list of URIs. If *is_authenticated* is specified as `True`, *realm* is ignored.

</div>

<div class="method">

HTTPPasswordMgrWithPriorAuth.find_user_password(realm, authuri)

Same as for `HTTPPasswordMgrWithDefaultRealm` objects

</div>

<div class="method">

HTTPPasswordMgrWithPriorAuth.update_authenticated(self, uri, is_authenticated=False)

Update the `is_authenticated` flag for the given *uri* or list of URIs.

</div>

<div class="method">

HTTPPasswordMgrWithPriorAuth.is_authenticated(self, authuri)

Returns the current state of the `is_authenticated` flag for the given URI.

</div>

## AbstractBasicAuthHandler Objects

<div class="method">

AbstractBasicAuthHandler.http_error_auth_reqed(authreq, host, req, headers)

Handle an authentication request by getting a user/password pair, and re-trying the request. *authreq* should be the name of the header where the information about the realm is included in the request, *host* specifies the URL and path to authenticate for, *req* should be the (failed) `Request` object, and *headers* should be the error headers.

*headers* must be a mapping-like object with case-insensitive lookup that implements the `get_all()` method, such as `email.message.Message` or `wsgiref.headers.Headers`.

*host* is either an authority (e.g. `"python.org"`) or a URL containing an authority component (e.g. `"https://python.org/"`). In either case, the authority must not contain a userinfo component (so, `"python.org"` and `"python.org:80"` are fine, `"joe:password@python.org"` is not).

</div>

## HTTPBasicAuthHandler Objects

<div class="method">

HTTPBasicAuthHandler.http_error_401(req, fp, code, msg, hdrs)

Retry the request with authentication information, if available.

</div>

## ProxyBasicAuthHandler Objects

<div class="method">

ProxyBasicAuthHandler.http_error_407(req, fp, code, msg, hdrs)

Retry the request with authentication information, if available.

</div>

## AbstractDigestAuthHandler Objects

<div class="method">

AbstractDigestAuthHandler.http_error_auth_reqed(authreq, host, req, headers)

*authreq* should be the name of the header where the information about the realm is included in the request, *host* should be the host to authenticate to, *req* should be the (failed) `Request` object, and *headers* should be the error headers.

*headers* must be a mapping-like object with case-insensitive lookup, such as `email.message.Message` or `wsgiref.headers.Headers`.

</div>

## HTTPDigestAuthHandler Objects

<div class="method">

HTTPDigestAuthHandler.http_error_401(req, fp, code, msg, hdrs)

Retry the request with authentication information, if available.

</div>

## ProxyDigestAuthHandler Objects

<div class="method">

ProxyDigestAuthHandler.http_error_407(req, fp, code, msg, hdrs)

Retry the request with authentication information, if available.

</div>

## HTTPHandler Objects

<div class="method">

HTTPHandler.http_open(req)

Send an HTTP request, which can be either GET or POST, depending on `req.data`.

</div>

## HTTPSHandler Objects

<div class="method">

HTTPSHandler.https_open(req)

Send an HTTPS request, which can be either GET or POST, depending on `req.data`.

</div>

## FileHandler Objects

<div class="method">

FileHandler.file_open(req)

Open the file locally, if there is no host name, or the host name is `'localhost'`.

<div class="versionchanged">

3.2 This method is applicable only for local hostnames. When a remote hostname is given, a `~urllib.error.URLError` is raised.

</div>

</div>

## DataHandler Objects

<div class="method">

DataHandler.data_open(req)

Read a data URL. This kind of URL contains the content encoded in the URL itself. The data URL syntax is specified in `2397`. This implementation ignores white spaces in base64 encoded data URLs so the URL may be wrapped in whatever source file it comes from. But even though some browsers don't mind about a missing padding at the end of a base64 encoded data URL, this implementation will raise a `ValueError` in that case.

</div>

## FTPHandler Objects

<div class="method">

FTPHandler.ftp_open(req)

Open the FTP file indicated by *req*. The login is always done with empty username and password.

</div>

## CacheFTPHandler Objects

`CacheFTPHandler` objects are `FTPHandler` objects with the following additional methods:

<div class="method">

CacheFTPHandler.setTimeout(t)

Set timeout of connections to *t* seconds.

</div>

<div class="method">

CacheFTPHandler.setMaxConns(m)

Set maximum number of cached connections to *m*.

</div>

## UnknownHandler Objects

<div class="method">

UnknownHandler.unknown_open()

Raise a `~urllib.error.URLError` exception.

</div>

## HTTPErrorProcessor Objects

<div class="method">

HTTPErrorProcessor.http_response(request, response)

Process HTTP error responses.

For 200 error codes, the response object is returned immediately.

For non-200 error codes, this simply passes the job on to the `!http_error_\<type\>` handler methods, via `OpenerDirector.error`. Eventually, `HTTPDefaultErrorHandler` will raise an `~urllib.error.HTTPError` if no other handler handles the error.

</div>

<div class="method">

HTTPErrorProcessor.https_response(request, response)

Process HTTPS error responses.

The behavior is same as `http_response`.

</div>

## Examples

In addition to the examples below, more examples are given in `urllib-howto`.

This example gets the python.org main page and displays the first 300 bytes of it:

    >>> import urllib.request
    >>> with urllib.request.urlopen('https://www.python.org/') as f:
    ...     # The response may be compressed (for example, 'gzip').
    ...     print(f.headers.get('Content-Encoding'))
    ...     data = f.read()
    ...     if f.headers.get('Content-Encoding') == 'gzip':
    ...         import gzip
    ...         data = gzip.decompress(data)
    ...     print(data[:300].decode('utf-8', errors='replace'))

Note that urlopen returns a bytes object. This is because there is no way for urlopen to automatically determine the encoding of the byte stream it receives from the HTTP server. In general, a program will decode the returned bytes object to string once it determines or guesses the appropriate encoding.

The following HTML spec document, <https://html.spec.whatwg.org/#charset>, lists the various ways in which an HTML or an XML document could have specified its encoding information.

For additional information, see the W3C document: <https://www.w3.org/International/questions/qa-html-encoding-declarations>.

As the python.org website uses *utf-8* encoding as specified in its meta tag, we will use the same for decoding the bytes object:

    >>> with urllib.request.urlopen('https://www.python.org/') as f:
    ...     # Check for compression and decode appropriately.
    ...     enc = f.headers.get('Content-Encoding')
    ...     data = f.read()
    ...     if enc == 'gzip':
    ...         import gzip
    ...         data = gzip.decompress(data)
    ...     print(data[:100].decode('utf-8', errors='replace'))
    ...

It is also possible to achieve the same result without using the `context manager` approach:

    >>> import urllib.request
    >>> f = urllib.request.urlopen('https://www.python.org/')
    >>> try:
    ...     enc = f.headers.get('Content-Encoding')
    ...     data = f.read()
    ...     if enc == 'gzip':
    ...         import gzip
    ...         data = gzip.decompress(data)
    ...     print(data[:100].decode('utf-8', errors='replace'))
    ... finally:
    ...     f.close()

In the following example, we are sending a data-stream to the stdin of a CGI and reading the data it returns to us. Note that this example will only work when the Python installation supports SSL. :

    >>> import urllib.request
    >>> req = urllib.request.Request(url='https://localhost/cgi-bin/test.cgi',
    ...                       data=b'This data is passed to stdin of the CGI')
    >>> with urllib.request.urlopen(req) as f:
    ...     print(f.read().decode('utf-8'))
    ...
    Got Data: "This data is passed to stdin of the CGI"

The code for the sample CGI used in the above example is:

    #!/usr/bin/env python
    import sys
    data = sys.stdin.read()
    print('Content-type: text/plain\n\nGot Data: "%s"' % data)

Here is an example of doing a `PUT` request using `Request`:

    import urllib.request
    DATA = b'some data'
    req = urllib.request.Request(url='http://localhost:8080', data=DATA, method='PUT')
    with urllib.request.urlopen(req) as f:
        pass
    print(f.status)
    print(f.reason)

Use of Basic HTTP Authentication:

    import urllib.request
    # Create an OpenerDirector with support for Basic HTTP Authentication...
    auth_handler = urllib.request.HTTPBasicAuthHandler()
    auth_handler.add_password(realm='PDQ Application',
                              uri='https://mahler:8092/site-updates.py',
                              user='klem',
                              passwd='kadidd!ehopper')
    opener = urllib.request.build_opener(auth_handler)
    # ...and install it globally so it can be used with urlopen.
    urllib.request.install_opener(opener)
    with urllib.request.urlopen('http://www.example.com/login.html') as f:
        print(f.read().decode('utf-8'))

`build_opener` provides many handlers by default, including a `ProxyHandler`. By default, `ProxyHandler` uses the environment variables named `<scheme>_proxy`, where `<scheme>` is the URL scheme involved. For example, the `!http_proxy` environment variable is read to obtain the HTTP proxy's URL.

This example replaces the default `ProxyHandler` with one that uses programmatically supplied proxy URLs, and adds proxy authorization support with `ProxyBasicAuthHandler`. :

    proxy_handler = urllib.request.ProxyHandler({'http': 'http://www.example.com:3128/'})
    proxy_auth_handler = urllib.request.ProxyBasicAuthHandler()
    proxy_auth_handler.add_password('realm', 'host', 'username', 'password')

    opener = urllib.request.build_opener(proxy_handler, proxy_auth_handler)
    # This time, rather than install the OpenerDirector, we use it directly:
    with opener.open('http://www.example.com/login.html') as f:
       print(f.read().decode('utf-8'))

Adding HTTP headers:

Use the *headers* argument to the `Request` constructor, or:

    import urllib.request
    req = urllib.request.Request('http://www.example.com/')
    req.add_header('Referer', 'https://www.python.org/')
    # Customize the default User-Agent header value:
    req.add_header('User-Agent', 'urllib-example/0.1 (Contact: . . .)')
    with urllib.request.urlopen(req) as f:
        print(f.read().decode('utf-8'))

`OpenerDirector` automatically adds a `User-Agent` header to every `Request`. To change this:

    import urllib.request
    opener = urllib.request.build_opener()
    opener.addheaders = [('User-agent', 'Mozilla/5.0')]
    with opener.open('http://www.example.com/') as f:
       print(f.read().decode('utf-8'))

Also, remember that a few standard headers (`Content-Length`, `Content-Type` and `Host`) are added when the `Request` is passed to `urlopen` (or `OpenerDirector.open`).

<div id="urllib-examples">

Here is an example session that uses the `GET` method to retrieve a URL containing parameters:

    >>> import urllib.request
    >>> import urllib.parse
    >>> params = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
    >>> url = "https://www.python.org/?%s" % params
    >>> with urllib.request.urlopen(url) as f:
    ...     print(f.read().decode('utf-8'))
    ...

</div>

The following example uses the `POST` method instead. Note that params output from urlencode is encoded to bytes before it is sent to urlopen as data:

    >>> import urllib.request
    >>> import urllib.parse
    >>> data = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
    >>> data = data.encode('ascii')
    >>> with urllib.request.urlopen("https://httpbin.org/post", data) as f:
    ...     print(f.read().decode('utf-8'))
    ...

The following example uses an explicitly specified HTTP proxy, overriding environment settings:

    >>> import urllib.request
    >>> proxies = {'http': 'http://proxy.example.com:8080/'}
    >>> opener = urllib.request.build_opener(urllib.request.ProxyHandler(proxies))
    >>> with opener.open("https://www.python.org") as f:
    ...     f.read().decode('utf-8')
    ...

The following example uses no proxies at all, overriding environment settings:

    >>> import urllib.request
    >>> opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
    >>> with opener.open("https://www.python.org/") as f:
    ...     f.read().decode('utf-8')
    ...

## Legacy interface

The following functions and classes are ported from the Python 2 module `urllib` (as opposed to `urllib2`). They might become deprecated at some point in the future.

<div class="function">

urlretrieve(url, filename=None, reporthook=None, data=None)

Copy a network object denoted by a URL to a local file. If the URL points to a local file, the object will not be copied unless filename is supplied. Return a tuple `(filename, headers)` where *filename* is the local file name under which the object can be found, and *headers* is whatever the `!info` method of the object returned by `urlopen` returned (for a remote object). Exceptions are the same as for `urlopen`.

The second argument, if present, specifies the file location to copy to (if absent, the location will be a tempfile with a generated name). The third argument, if present, is a callable that will be called once on establishment of the network connection and once after each block read thereafter. The callable will be passed three arguments; a count of blocks transferred so far, a block size in bytes, and the total size of the file. The third argument may be `-1` on older FTP servers which do not return a file size in response to a retrieval request.

The following example illustrates the most common usage scenario:

    >>> import urllib.request
    >>> local_filename, headers = urllib.request.urlretrieve('https://python.org/')
    >>> html = open(local_filename)
    >>> html.close()

If the *url* uses the `http:` scheme identifier, the optional *data* argument may be given to specify a `POST` request (normally the request type is `GET`). The *data* argument must be a bytes object in standard `application/x-www-form-urlencoded` format; see the `urllib.parse.urlencode` function.

`urlretrieve` will raise `~urllib.error.ContentTooShortError` when it detects that the amount of data available was less than the expected amount (which is the size reported by a *Content-Length* header). This can occur, for example, when the download is interrupted.

The *Content-Length* is treated as a lower bound: if there's more data to read, urlretrieve reads more data, but if less data is available, it raises the exception.

You can still retrieve the downloaded data in this case, it is stored in the `!content` attribute of the exception instance.

If no *Content-Length* header was supplied, urlretrieve can not check the size of the data it has downloaded, and just returns it. In this case you just have to assume that the download was successful.

</div>

<div class="function">

urlcleanup()

Cleans up temporary files that may have been left behind by previous calls to `urlretrieve`. It also resets the default global opener installed by `install_opener`.

</div>

## `!urllib.request` Restrictions

<div class="index">

pair: HTTP; protocol pair: FTP; protocol

</div>

- Currently, only the following protocols are supported: HTTP (versions 0.9 and 1.0), FTP, local files, and data URLs.

  <div class="versionchanged">

  3.4 Added support for data URLs.

  </div>

- The caching feature of `urlretrieve` has been disabled until someone finds the time to hack proper processing of Expiration time headers.

- There should be a function to query whether a particular URL is in the cache.

- For backward compatibility, if a URL appears to point to a local file but the file can't be opened, the URL is re-interpreted using the FTP protocol. This can sometimes cause confusing error messages.

- The `urlopen` and `urlretrieve` functions can cause arbitrarily long delays while waiting for a network connection to be set up. This means that it is difficult to build an interactive web client using these functions without using threads.

  <div class="index">

  single: HTML pair: HTTP; protocol

  </div>

- The data returned by `urlopen` or `urlretrieve` is the raw data returned by the server. This may be binary data (such as an image), plain text or (for example) HTML. The HTTP protocol provides type information in the reply header, which can be inspected by looking at the `Content-Type` header. If the returned data is HTML, you can use the module `html.parser` to parse it.

  <div class="index">

  single: FTP

  </div>

- The code handling the FTP protocol cannot differentiate between a file and a directory. This can lead to unexpected behavior when attempting to read a URL that points to a file that is not accessible. If the URL ends in a `/`, it is assumed to refer to a directory and will be handled accordingly. But if an attempt to read a file leads to a 550 error (meaning the URL cannot be found or is not accessible, often for permission reasons), then the path is treated as a directory in order to handle the case when a directory is specified by a URL but the trailing `/` has been left off. This can cause misleading results when you try to fetch a file whose read permissions make it inaccessible; the FTP code will try to read it, fail with a 550 error, and then perform a directory listing for the unreadable file. If fine-grained control is needed, consider using the `ftplib` module.

# `!urllib.response` --- Response classes used by urllib

<div class="module" data-synopsis="Response classes used by urllib.">

urllib.response

</div>

The `!urllib.response` module defines functions and classes which define a minimal file-like interface, including `read()` and `readline()`. Functions defined by this module are used internally by the `!urllib.request` module. The typical response object is a `urllib.response.addinfourl` instance:

<div class="addinfourl">

<div class="attribute">

url

URL of the resource retrieved, commonly used to determine if a redirect was followed.

</div>

<div class="attribute">

headers

Returns the headers of the response in the form of an `~email.message.EmailMessage` instance.

</div>

<div class="attribute">

status

<div class="versionadded">

3.9

</div>

Status code returned by server.

</div>

<div class="method">

geturl()

<div class="deprecated">

3.9 Deprecated in favor of `~addinfourl.url`.

</div>

</div>

<div class="method">

info()

<div class="deprecated">

3.9 Deprecated in favor of `~addinfourl.headers`.

</div>

</div>

<div class="attribute">

code

<div class="deprecated">

3.9 Deprecated in favor of `~addinfourl.status`.

</div>

</div>

<div class="method">

getcode()

<div class="deprecated">

3.9 Deprecated in favor of `~addinfourl.status`.

</div>

</div>

</div>

# `!urllib.robotparser` --- Parser for robots.txt

<div class="module" data-synopsis="Load a robots.txt file and answer questions about
fetchability of other URLs.">

urllib.robotparser

</div>

**Source code:** `Lib/urllib/robotparser.py`

<div class="index">

single: WWW single: World Wide Web single: URL single: robots.txt

</div>

------------------------------------------------------------------------

This module provides a single class, `RobotFileParser`, which answers questions about whether or not a particular user agent can fetch a URL on the website that published the `robots.txt` file. For more details on the structure of `robots.txt` files, see `9309`.

<div class="RobotFileParser(url='')">

This class provides methods to read, parse and answer questions about the `robots.txt` file at *url* or a `urllib.request.Request` object.

<div class="versionchanged">

3.16 *url* parameter can be a `urllib.request.Request` object.

</div>

<div class="method">

set_url(url)

Sets the URL referring to a `robots.txt` file or a `urllib.request.Request` object.

<div class="versionchanged">

3.16 *url* parameter can be a `urllib.request.Request` object.

</div>

</div>

<div class="method">

read()

Reads the `robots.txt` URL and feeds it to the parser.

</div>

<div class="method">

parse(lines)

Parses the lines argument.

</div>

<div class="method">

can_fetch(useragent, url)

Returns `True` if the *useragent* is allowed to fetch the *url* according to the rules contained in the parsed `robots.txt` file.

</div>

<div class="method">

mtime()

Returns the time the `robots.txt` file was last fetched. This is useful for long-running web spiders that need to check for new `robots.txt` files periodically.

</div>

<div class="method">

modified()

Sets the time the `robots.txt` file was last fetched to the current time.

</div>

<div class="method">

crawl_delay(useragent)

Returns the value of the `Crawl-delay` parameter from `robots.txt` for the *useragent* in question. If there is no such parameter or it doesn't apply to the *useragent* specified or the `robots.txt` entry for this parameter has invalid syntax, return `None`.

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

request_rate(useragent)

Returns the contents of the `Request-rate` parameter from `robots.txt` as a `named tuple` `RequestRate(requests, seconds)`. If there is no such parameter or it doesn't apply to the *useragent* specified or the `robots.txt` entry for this parameter has invalid syntax, return `None`.

<div class="versionadded">

3.6

</div>

</div>

<div class="method">

site_maps()

Returns the contents of the `Sitemap` parameter from `robots.txt` in the form of a `list`. If there is no such parameter or the `robots.txt` entry for this parameter has invalid syntax, return `None`.

<div class="versionadded">

3.8

</div>

</div>

</div>

The following example demonstrates basic use of the `RobotFileParser` class:

    >>> import urllib.robotparser
    >>> rp = urllib.robotparser.RobotFileParser()
    >>> rp.set_url("http://www.pythontest.net/robots.txt")
    >>> rp.read()
    >>> rrate = rp.request_rate("*")
    >>> rrate.requests
    1
    >>> rrate.seconds
    1
    >>> rp.crawl_delay("*")
    6
    >>> rp.can_fetch("*", "http://www.pythontest.net/")
    True
    >>> rp.can_fetch("*", "http://www.pythontest.net/no-robots-here/")
    False

The following example demonstrates use of a `urllib.request.Request` object with additional user-agent headers populated:

    >>> import urllib.robotparser
    >>> import urllib.request
    >>> rp = urllib.robotparser.RobotFileParser()
    >>> rp.set_url(urllib.request.Request("http://www.pythontest.net/robots.txt", headers={"User-Agent": "IsraBot"}))
    >>> rp.read()
    >>> rp.can_fetch("*", "http://www.pythontest.net/")
    True
    >>> rp.can_fetch("*", "http://www.pythontest.net/no-robots-here/")
    False

# `!uu` --- Encode and decode uuencode files

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

uu

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

Encoding and decoding in the uu format can instead be achieved using `codecs.encode` and `codecs.decode`, specifying `"uu"` as the encoding.

The last version of Python that provided the `!uu` module was [Python 3.12](https://docs.python.org/3.12/library/uu.html).

# `!uuid` --- UUID objects according to `9562`

<div class="module" data-synopsis="UUID objects (universally unique identifiers) according to RFC 9562">

uuid

</div>

**Source code:** `Lib/uuid.py`

------------------------------------------------------------------------

This module provides immutable `UUID` objects (the `UUID` class) and `functions <uuid-factory-functions>` for generating UUIDs corresponding to a specific UUID version as specified in `9562` (which supersedes `4122`), for example, `uuid1` for UUID version 1, `uuid3` for UUID version 3, and so on. Note that UUID version 2 is deliberately omitted as it is outside the scope of the RFC.

If all you want is a unique ID, you should probably call `uuid1` or `uuid4`. Note that `uuid1` may compromise privacy since it creates a UUID containing the computer's network address. `uuid4` creates a random UUID.

Depending on support from the underlying platform, `uuid1` may or may not return a "safe" UUID. A safe UUID is one which is generated using synchronization methods that ensure no two processes can obtain the same UUID. All instances of `UUID` have an `~UUID.is_safe` attribute which relays any information about the UUID's safety, using this enumeration:

<div class="SafeUUID">

<div class="versionadded">

3.7

</div>

<div class="attribute">

SafeUUID.safe

The UUID was generated by the platform in a multiprocessing-safe way.

</div>

<div class="attribute">

SafeUUID.unsafe

The UUID was not generated in a multiprocessing-safe way.

</div>

<div class="attribute">

SafeUUID.unknown

The platform does not provide information on whether the UUID was generated safely or not.

</div>

</div>

<div class="UUID(hex=None, bytes=None, bytes_le=None, fields=None, int=None, version=None, *, is_safe=SafeUUID.unknown)">

Create a UUID from either a string of 32 hexadecimal digits, a 16-byte `bytes` object in big-endian order as the *bytes* argument, a 16-byte `bytes` object in little-endian order as the *bytes_le* argument, a tuple of six integers (32-bit *time_low*, 16-bit *time_mid*, 16-bit *time_hi_version*, 8-bit *clock_seq_hi_variant*, 8-bit *clock_seq_low*, 48-bit *node*) as the *fields* argument, or a single 128-bit integer as the *int* argument. When a string of hex digits is given, curly braces, hyphens, and a URN prefix are all optional. For example, these expressions all yield the same UUID:

    UUID('{12345678-1234-5678-1234-567812345678}')
    UUID('12345678123456781234567812345678')
    UUID('urn:uuid:12345678-1234-5678-1234-567812345678')
    UUID(bytes=b'\x12\x34\x56\x78'*4)
    UUID(bytes_le=b'\x78\x56\x34\x12\x34\x12\x78\x56' +
                  b'\x12\x34\x56\x78\x12\x34\x56\x78')
    UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678))
    UUID(int=0x12345678123456781234567812345678)

Exactly one of *hex*, *bytes*, *bytes_le*, *fields*, or *int* must be given. The *version* argument is optional; if given, the resulting UUID will have its variant and version number set according to `9562`, overriding bits in the given *hex*, *bytes*, *bytes_le*, *fields*, or *int*.

<div class="versionchanged">

next Passing an object other than `bytes` as *bytes* or *bytes_le* now consistently raises `TypeError`. Previously, some invalid inputs raised `AssertionError` or `ValueError`.

</div>

Comparison of UUID objects are made by way of comparing their `UUID.int` attributes. Comparison with a non-UUID object raises a `TypeError`.

`str(uuid)` returns a string in the form `12345678-1234-5678-1234-567812345678` where the 32 hexadecimal digits represent the UUID.

</div>

`UUID` instances have these read-only attributes:

<div class="attribute">

UUID.bytes

The UUID as a 16-byte `bytes` object (containing the six integer fields in big-endian byte order).

</div>

<div class="attribute">

UUID.bytes_le

The UUID as a 16-byte `bytes` object (with *time_low*, *time_mid*, and *time_hi_version* in little-endian byte order).

</div>

<div class="attribute">

UUID.fields

A tuple of the six integer fields of the UUID, which are also available as six individual attributes and two derived attributes:

</div>

<table>
<tbody>
<tr>
<td>Field</td>
<td>Meaning</td>
</tr>
<tr>
<td><div class="attribute">
<p>UUID.time_low</p>
</div></td>
<td>The first 32 bits of the UUID. Only relevant to version 1.</td>
</tr>
<tr>
<td><div class="attribute">
<p>UUID.time_mid</p>
</div></td>
<td>The next 16 bits of the UUID. Only relevant to version 1.</td>
</tr>
<tr>
<td><div class="attribute">
<p>UUID.time_hi_version</p>
</div></td>
<td>The next 16 bits of the UUID. Only relevant to version 1.</td>
</tr>
<tr>
<td><div class="attribute">
<p>UUID.clock_seq_hi_variant</p>
</div></td>
<td>The next 8 bits of the UUID. Only relevant to versions 1 and 6.</td>
</tr>
<tr>
<td><div class="attribute">
<p>UUID.clock_seq_low</p>
</div></td>
<td>The next 8 bits of the UUID. Only relevant to versions 1 and 6.</td>
</tr>
<tr>
<td><div class="attribute">
<p>UUID.node</p>
</div></td>
<td>The last 48 bits of the UUID. Only relevant to version 1.</td>
</tr>
<tr>
<td><div class="attribute">
<p>UUID.time</p>
</div></td>
<td>The 60-bit timestamp as a count of 100-nanosecond intervals since Gregorian epoch (1582-10-15 00:00:00) for versions 1 and 6, or the 48-bit timestamp in milliseconds since Unix epoch (1970-01-01 00:00:00) for version 7.</td>
</tr>
<tr>
<td><div class="attribute">
<p>UUID.clock_seq</p>
</div></td>
<td>The 14-bit sequence number. Only relevant to versions 1 and 6.</td>
</tr>
</tbody>
</table>

<div class="attribute">

UUID.hex

The UUID as a 32-character lowercase hexadecimal string.

</div>

<div class="attribute">

UUID.int

The UUID as a 128-bit integer.

</div>

<div class="attribute">

UUID.urn

The UUID as a URN as specified in `9562`.

</div>

<div class="attribute">

UUID.variant

The UUID variant, which determines the internal layout of the UUID. This will be one of the constants `RESERVED_NCS`, `RFC_4122`, `RESERVED_MICROSOFT`, or `RESERVED_FUTURE`.

</div>

<div class="attribute">

UUID.version

The UUID version number (1 through 8, meaningful only when the variant is `RFC_4122`).

<div class="versionchanged">

3.14 Added UUID versions 6, 7 and 8.

</div>

</div>

<div class="attribute">

UUID.is_safe

An enumeration of `SafeUUID` which indicates whether the platform generated the UUID in a multiprocessing-safe way.

<div class="versionadded">

3.7

</div>

</div>

The `!uuid` module defines the following functions:

<div class="function">

getnode()

Get the hardware address as a 48-bit positive integer. The first time this runs, it may launch a separate program, which could be quite slow. If all attempts to obtain the hardware address fail, we choose a random 48-bit number with the multicast bit (least significant bit of the first octet) set to 1 as recommended in `4122`. "Hardware address" means the MAC address of a network interface. On a machine with multiple network interfaces, universally administered MAC addresses (i.e. where the second least significant bit of the first octet is *unset*) will be preferred over locally administered MAC addresses, but with no other ordering guarantees.

<div class="versionchanged">

3.7 Universally administered MAC addresses are preferred over locally administered MAC addresses, since the former are guaranteed to be globally unique, while the latter are not.

</div>

</div>

<div id="uuid-factory-functions">

<div class="function">

uuid1(node=None, clock_seq=None)

Generate a UUID from a host ID, sequence number, and the current time according to `RFC 9562, §5.1 <9562#section-5.1>`.

When *node* is not specified, `getnode` is used to obtain the hardware address as a 48-bit positive integer. When a sequence number *clock_seq* is not specified, a pseudo-random 14-bit positive integer is generated.

If *node* or *clock_seq* exceed their expected bit count, only their least significant bits are kept.

</div>

</div>

<div class="function">

uuid3(namespace, name)

Generate a UUID based on the MD5 hash of a namespace identifier (which is a UUID) and a name (which is a `bytes` object or a string that will be encoded using UTF-8) according to `RFC 9562, §5.3 <9562#section-5.3>`.

</div>

<div class="function">

uuid4()

Generate a random UUID in a cryptographically-secure method according to `RFC 9562, §5.4 <9562#section-5.4>`.

</div>

<div class="function">

uuid5(namespace, name)

Generate a UUID based on the SHA-1 hash of a namespace identifier (which is a UUID) and a name (which is a `bytes` object or a string that will be encoded using UTF-8) according to `RFC 9562, §5.5 <9562#section-5.5>`.

</div>

<div class="function">

uuid6(node=None, clock_seq=None)

Generate a UUID from a sequence number and the current time according to `RFC 9562, §5.6 <9562#section-5.6>`.

This is an alternative to `uuid1` to improve database locality.

When *node* is not specified, `getnode` is used to obtain the hardware address as a 48-bit positive integer. When a sequence number *clock_seq* is not specified, a pseudo-random 14-bit positive integer is generated.

If *node* or *clock_seq* exceed their expected bit count, only their least significant bits are kept.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

uuid7()

Generate a time-based UUID according to `RFC 9562, §5.7 <9562#section-5.7>`.

For portability across platforms lacking sub-millisecond precision, UUIDs produced by this function embed a 48-bit timestamp and use a 42-bit counter to guarantee monotonicity within a millisecond.

<div class="versionadded">

3.14

</div>

</div>

<div class="function">

uuid8(a=None, b=None, c=None)

Generate a pseudo-random UUID according to `RFC 9562, §5.8 <9562#section-5.8>`.

When specified, the parameters *a*, *b* and *c* are expected to be positive integers of 48, 12 and 62 bits respectively. If they exceed their expected bit count, only their least significant bits are kept; non-specified arguments are substituted for a pseudo-random integer of appropriate size.

By default, *a*, *b* and *c* are not generated by a cryptographically secure pseudo-random number generator (CSPRNG). Use `uuid4` when a UUID needs to be used in a security-sensitive context.

<div class="versionadded">

3.14

</div>

</div>

The `!uuid` module defines the following namespace identifiers for use with `uuid3` or `uuid5`.

<div class="data">

NAMESPACE_DNS

When this namespace is specified, the *name* string is a fully qualified domain name.

</div>

<div class="data">

NAMESPACE_URL

When this namespace is specified, the *name* string is a URL.

</div>

<div class="data">

NAMESPACE_OID

When this namespace is specified, the *name* string is an ISO OID.

</div>

<div class="data">

NAMESPACE_X500

When this namespace is specified, the *name* string is an X.500 DN in DER or a text output format.

</div>

The `!uuid` module defines the following constants for the possible values of the `~UUID.variant` attribute:

<div class="data">

RESERVED_NCS

Reserved for NCS compatibility.

</div>

<div class="data">

RFC_4122

Specifies the UUID layout given in `4122`. This constant is kept for backward compatibility even though `4122` has been superseded by `9562`.

</div>

<div class="data">

RESERVED_MICROSOFT

Reserved for Microsoft compatibility.

</div>

<div class="data">

RESERVED_FUTURE

Reserved for future definition.

</div>

The `!uuid` module defines the special Nil and Max UUID values:

<div class="data">

NIL

A special form of UUID that is specified to have all 128 bits set to zero according to `RFC 9562, §5.9 <9562#section-5.9>`.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

MAX

A special form of UUID that is specified to have all 128 bits set to one according to `RFC 9562, §5.10 <9562#section-5.10>`.

<div class="versionadded">

3.14

</div>

</div>

<div class="seealso">

`9562` - A Universally Unique IDentifier (UUID) URN Namespace  
This specification defines a Uniform Resource Name namespace for UUIDs, the internal format of UUIDs, and methods of generating UUIDs.

</div>

## Command-Line Usage

<div class="versionadded">

3.12

</div>

The `!uuid` module can be executed as a script from the command line.

``` sh
python -m uuid [-h] [-u {uuid1,uuid3,uuid4,uuid5,uuid6,uuid7,uuid8}] [-n NAMESPACE] [-N NAME]
```

The following options are accepted:

<div class="program">

uuid

</div>

<div class="option">

`-h, --help`  

Show the help message and exit.

</div>

<div class="option">

`-u <uuid>`  

`--uuid <uuid>`  

Specify the function name to use to generate the uuid. By default `uuid4` is used.

<div class="versionchanged">

3.14 Allow generating UUID versions 6, 7 and 8.

</div>

</div>

<div class="option">

`-n <namespace>`  

`--namespace <namespace>`  

The namespace is a `UUID`, or `@ns` where `ns` is a well-known predefined UUID addressed by namespace name. Such as `@dns`, `@url`, `@oid`, and `@x500`. Only required for `uuid3` / `uuid5` functions.

</div>

<div class="option">

`-N <name>`  

`--name <name>`  

The name used as part of generating the uuid. Only required for `uuid3` / `uuid5` functions.

</div>

<div class="option">

`-C <num>`  

`--count <num>`  

Generate *num* fresh UUIDs.

<div class="versionadded">

3.14

</div>

</div>

## Example

Here are some examples of typical usage of the `!uuid` module:

    >>> import uuid

    >>> # make a UUID based on the host ID and current time
    >>> uuid.uuid1()  # doctest: +SKIP
    UUID('a8098c1a-f86e-11da-bd1a-00112444be1e')

    >>> # make a UUID using an MD5 hash of a namespace UUID and a name
    >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org')
    UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e')

    >>> # make a random UUID
    >>> uuid.uuid4()
    UUID('16fd2706-8baf-433b-82eb-8c7fada847da')

    >>> # make a UUID using a SHA-1 hash of a namespace UUID and a name
    >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org')
    UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d')

    >>> # make a UUID from a string of hex digits (braces and hyphens ignored)
    >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}')

    >>> # convert a UUID to a string of hex digits in standard form
    >>> str(x)
    '00010203-0405-0607-0809-0a0b0c0d0e0f'

    >>> # get the raw 16 bytes of the UUID
    >>> x.bytes
    b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f'

    >>> # make a UUID from a 16-byte bytes object
    >>> uuid.UUID(bytes=x.bytes)
    UUID('00010203-0405-0607-0809-0a0b0c0d0e0f')

    >>> # get the Nil UUID
    >>> uuid.NIL
    UUID('00000000-0000-0000-0000-000000000000')

    >>> # get the Max UUID
    >>> uuid.MAX
    UUID('ffffffff-ffff-ffff-ffff-ffffffffffff')

    >>> # same as UUIDv1 but with fields reordered to improve DB locality
    >>> uuid.uuid6()  # doctest: +SKIP
    UUID('1f0799c0-98b9-62db-92c6-a0d365b91053')

    >>> # get UUIDv7 creation (local) time as a timestamp in milliseconds
    >>> u = uuid.uuid7()
    >>> u.time  # doctest: +SKIP
    1743936859822

    >>> # get UUIDv7 creation (local) time as a datetime object
    >>> import datetime as dt
    >>> dt.datetime.fromtimestamp(u.time / 1000)  # doctest: +SKIP
    datetime.datetime(...)

    >>> # make a UUID with custom blocks
    >>> uuid.uuid8(0x12345678, 0x9abcdef0, 0x11223344)
    UUID('00001234-5678-8ef0-8000-000011223344')

## Command-Line Example

Here are some examples of typical usage of the `!uuid` command-line interface:

```
# generate a random UUID - by default uuid4() is used
$ python -m uuid

# generate a UUID using uuid1()
$ python -m uuid -u uuid1

# generate a UUID using uuid5
$ python -m uuid -u uuid5 -n @url -N example.com

# generate 42 random UUIDs
$ python -m uuid -C 42
```

# `!venv` --- Creation of virtual environments

<div class="module" data-synopsis="Creation of virtual environments.">

venv

</div>

<div class="versionadded">

3.3

</div>

**Source code:** `Lib/venv/`

<div class="index">

pair: Environments; virtual

</div>

------------------------------------------------------------------------

<div id="venv-def">

<div id="venv-intro">

The `!venv` module supports creating lightweight "virtual environments", each with their own independent set of Python packages installed in their `site` directories. A virtual environment is created on top of an existing Python installation, known as the virtual environment's "base" Python, and by default is isolated from the packages in the base environment, so that only those explicitly installed in the virtual environment are available. See `sys-path-init-virtual-environments` and `site`'s `virtual environments documentation <site-virtual-environments-configuration>` for more information.

</div>

</div>

When used from within a virtual environment, common installation tools such as `pip` will install Python packages into a virtual environment without needing to be told to do so explicitly.

A virtual environment is (amongst other things):

- Used to contain a specific Python interpreter and software libraries and binaries which are needed to support a project (library or application). These are by default isolated from software in other virtual environments and Python interpreters and libraries installed in the operating system.
- Contained in a directory, conventionally named `.venv` or `venv` in the project directory, or under a container directory for lots of virtual environments, such as `~/.virtualenvs`.
- Not checked into source control systems such as Git.
- Considered as disposable -- it should be simple to delete and recreate it from scratch. You don't place any project code in the environment.
- Not considered as movable or copyable -- you just recreate the same environment in the target location.

See `405` for more background on Python virtual environments.

<div class="seealso">

[Python Packaging User Guide: Creating and using virtual environments](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#create-and-use-virtual-environments)

</div>

<div class="availability">

not Android, not iOS, not WASI.

This module is not supported on `mobile platforms <mobile-availability>` or `WebAssembly platforms <wasm-availability>`.

</div>

## Creating virtual environments

`Virtual environments <venv-def>` are created by executing the `venv` module:

```
python -m venv /path/to/new/virtual/environment
```

This creates the target directory (including parent directories as needed) and places a `pyvenv.cfg` file in it with a `home` key pointing to the Python installation from which the command was run. It also creates a `bin` (or `Scripts` on Windows) subdirectory containing a copy or symlink of the Python executable (as appropriate for the platform or arguments used at environment creation time). It also creates a `lib/pythonX.Y/site-packages` subdirectory (on Windows, this is `Lib\\site-packages`). If an existing directory is specified, it will be re-used. Reusing an existing directory does not leave it unchanged: `venv` may create, update, or replace files in the target directory. Use a dedicated directory for the virtual environment, and avoid placing project files directly inside it.

<div class="versionchanged">

3.5 The use of `venv` is now recommended for creating virtual environments.

</div>

<div class="deprecated-removed">

3.6 3.8 `pyvenv` was the recommended tool for creating virtual environments for Python 3.3 and 3.4, and replaced in 3.5 by executing `venv` directly.

</div>

On Windows, invoke the `venv` command as follows:

```
PS> python -m venv C:\path\to\new\virtual\environment
```

The command, if run with `-h`, will show the available options:

```
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
            [--upgrade] [--without-pip] [--prompt PROMPT] [--upgrade-deps]
            [--without-scm-ignore-files]
            ENV_DIR [ENV_DIR ...]

Creates virtual Python environments in one or more target directories.

Once an environment has been created, you may wish to activate it, e.g. by
sourcing an activate script in its bin directory.
```

<div class="option">

ENV_DIR

A required argument specifying the directory to create the environment in.

</div>

<div class="option">

`--system-site-packages`  

Give the virtual environment access to the system site-packages directory.

</div>

<div class="option">

`--symlinks`  

Try to use symlinks rather than copies, when symlinks are not the default for the platform.

</div>

<div class="option">

`--copies`  

Try to use copies rather than symlinks, even when symlinks are the default for the platform.

</div>

<div class="option">

`--clear`  

Delete all contents of the environment directory if it already exists, including files that were not created by `venv`, before environment creation.

</div>

<div class="option">

`--upgrade`  

Upgrade the environment directory to use this version of Python, assuming Python has been upgraded in-place.

</div>

<div class="option">

`--without-pip`  

Skips installing or upgrading pip in the virtual environment (pip is bootstrapped by default).

</div>

<div class="option">

`--prompt <PROMPT>`  

Provides an alternative prompt prefix for this environment.

</div>

<div class="option">

`--upgrade-deps`  

Upgrade core dependencies (pip) to the latest version in PyPI.

</div>

<div class="option">

`--without-scm-ignore-files`  

Skips adding SCM ignore files to the environment directory (Git is supported by default).

</div>

<div class="versionchanged">

3.4 Installs pip by default, added the `--without-pip` and `--copies` options.

</div>

<div class="versionchanged">

3.4 In earlier versions, if the target directory already existed, an error was raised, unless the `--clear` or `--upgrade` option was provided.

</div>

<div class="versionchanged">

3.9 Add `--upgrade-deps` option to upgrade pip + setuptools to the latest on PyPI.

</div>

<div class="versionchanged">

3.12

`setuptools` is no longer a core venv dependency.

</div>

<div class="versionchanged">

3.13

Added the `--without-scm-ignore-files` option.

</div>

<div class="versionchanged">

3.13 `venv` now creates a `.gitignore` file for Git by default.

</div>

> [!NOTE]
> While symlinks are supported on Windows, they are not recommended. Of particular note is that double-clicking `python.exe` in File Explorer will resolve the symlink eagerly and ignore the virtual environment.

> [!NOTE]
> On Microsoft Windows, it may be required to enable the `Activate.ps1` script by setting the execution policy for the user. You can do this by issuing the following PowerShell command:
>
> ``` powershell
> PS C:\> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
> ```
>
> See [About Execution Policies](https://go.microsoft.com/fwlink/?LinkID=135170) for more information.

The created `pyvenv.cfg` file also includes the `include-system-site-packages` key, set to `true` if `venv` is run with the `--system-site-packages` option, `false` otherwise.

Unless the `--without-pip` option is given, `ensurepip` will be invoked to bootstrap `pip` into the virtual environment.

Multiple paths can be given to `venv`, in which case an identical virtual environment will be created, according to the given options, at each provided path.

## How venvs work

When a Python interpreter is running from a virtual environment, `sys.prefix` and `sys.exec_prefix` point to the directories of the virtual environment, whereas `sys.base_prefix` and `sys.base_exec_prefix` point to those of the base Python used to create the environment. It is sufficient to check `sys.prefix != sys.base_prefix` to determine if the current interpreter is running from a virtual environment.

A virtual environment may be "activated" using a script in its binary directory (`bin` on POSIX; `Scripts` on Windows). This will prepend that directory to your `PATH`, so that running `python` will invoke the environment's Python interpreter and you can run installed scripts without having to use their full path. The invocation of the activation script is platform-specific (`{<venv>}` must be replaced by the path to the directory containing the virtual environment):

<table style="width:98%;">
<colgroup>
<col style="width: 17%" />
<col style="width: 16%" />
<col style="width: 64%" />
</colgroup>
<thead>
<tr>
<th>Platform</th>
<th>Shell</th>
<th>Command to activate virtual environment</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="4">POSIX</td>
<td>bash/zsh</td>
<td><code class="interpreted-text" role="samp">$ source {&lt;venv&gt;}/bin/activate</code></td>
</tr>
<tr>
<td>fish</td>
<td><code class="interpreted-text" role="samp">$ source {&lt;venv&gt;}/bin/activate.fish</code></td>
</tr>
<tr>
<td>csh/tcsh</td>
<td><code class="interpreted-text" role="samp">$ source {&lt;venv&gt;}/bin/activate.csh</code></td>
</tr>
<tr>
<td>pwsh</td>
<td><code class="interpreted-text" role="samp">$ {&lt;venv&gt;}/bin/Activate.ps1</code></td>
</tr>
<tr>
<td rowspan="2">Windows</td>
<td>cmd.exe</td>
<td><code class="interpreted-text" role="samp">C:\\&gt; {&lt;venv&gt;}\\Scripts\\activate.bat</code></td>
</tr>
<tr>
<td>PowerShell</td>
<td><code class="interpreted-text" role="samp">PS C:\\&gt; {&lt;venv&gt;}\\Scripts\\Activate.ps1</code></td>
</tr>
</tbody>
</table>

<div class="versionadded">

3.4 `fish` and `csh` activation scripts.

</div>

<div class="versionadded">

3.8 PowerShell activation scripts installed under POSIX for PowerShell Core support.

</div>

You don't specifically *need* to activate a virtual environment, as you can just specify the full path to that environment's Python interpreter when invoking Python. Furthermore, all scripts installed in the environment should be runnable without activating it.

In order to achieve this, scripts installed into virtual environments have a "shebang" line which points to the environment's Python interpreter, `#!/{<path-to-venv>}/bin/python`. This means that the script will run with that interpreter regardless of the value of `PATH`. On Windows, "shebang" line processing is supported if you have the `launcher` installed. Thus, double-clicking an installed script in a Windows Explorer window should run it with the correct interpreter without the environment needing to be activated or on the `PATH`.

When a virtual environment has been activated, the `!VIRTUAL_ENV` environment variable is set to the path of the environment. Since explicitly activating a virtual environment is not required to use it, `!VIRTUAL_ENV` cannot be relied upon to determine whether a virtual environment is being used.

> [!WARNING]
> Because scripts installed in environments should not expect the environment to be activated, their shebang lines contain the absolute paths to their environment's interpreters. Because of this, environments are inherently non-portable, in the general case. You should always have a simple means of recreating an environment (for example, if you have a requirements file `requirements.txt`, you can invoke `pip install -r requirements.txt` using the environment's `pip` to install all of the packages needed by the environment). If for any reason you need to move the environment to a new location, you should recreate it at the desired location and delete the one at the old location. If you move an environment because you moved a parent directory of it, you should recreate the environment in its new location. Otherwise, software installed into the environment may not work as expected.

You can deactivate a virtual environment by typing `deactivate` in your shell. The exact mechanism is platform-specific and is an internal implementation detail (typically, a script or shell function will be used).

## API

The high-level method described above makes use of a simple API which provides mechanisms for third-party virtual environment creators to customize environment creation according to their needs, the `EnvBuilder` class.

<div class="EnvBuilder(system_site_packages=False, clear=False, \ symlinks=None, upgrade=False, with_pip=False, \ prompt=None, upgrade_deps=False, \ *, scm_ignore_files=frozenset())">

The `EnvBuilder` class accepts the following keyword arguments on instantiation:

- *system_site_packages* -- a boolean value indicating that the system Python site-packages should be available to the environment (defaults to `False`).
- *clear* -- a boolean value which, if true, will delete the contents of any existing target directory, before creating the environment.
- *symlinks* -- a boolean value indicating whether to attempt to symlink the Python binary rather than copying. If `None`, the default is `False` on Windows and `True` on other platforms, matching the `CLI <venv-cli>`.
- *upgrade* -- a boolean value which, if true, will upgrade an existing environment with the running Python - for use when that Python has been upgraded in-place (defaults to `False`).
- *with_pip* -- a boolean value which, if true, ensures pip is installed in the virtual environment. This uses `ensurepip` with the `--default-pip` option.
- *prompt* -- a string to be used after virtual environment is activated (defaults to `None` which means directory name of the environment would be used). If the special string `"."` is provided, the basename of the current directory is used as the prompt.
- *upgrade_deps* -- Update the base venv modules to the latest on PyPI
- *scm_ignore_files* -- Create ignore files based for the specified source control managers (SCM) in the iterable. Support is defined by having a method named `create_{scm}_ignore_file`. The only value supported by default is `"git"` via `create_git_ignore_file`.

<div class="versionchanged">

3.4 Added the `with_pip` parameter

</div>

<div class="versionchanged">

3.6 Added the `prompt` parameter

</div>

<div class="versionchanged">

3.9 Added the `upgrade_deps` parameter

</div>

<div class="versionchanged">

3.13 Added the `scm_ignore_files` parameter

</div>

<div class="versionchanged">

3.16 The default value of *symlinks* is now platform-dependent.

</div>

`EnvBuilder` may be used as a base class.

<div class="method">

create(env_dir)

Create a virtual environment by specifying the target directory (absolute or relative to the current directory) which is to contain the virtual environment. The `create` method will either create the environment in the specified directory, or raise an appropriate exception.

The `create` method of the `EnvBuilder` class illustrates the hooks available for subclass customization:

``` python
def create(self, env_dir):
    """
    Create a virtualized Python environment in a directory.
    env_dir is the target directory to create an environment in.
    """
    env_dir = os.path.abspath(env_dir)
    context = self.ensure_directories(env_dir)
    self.create_configuration(context)
    self.setup_python(context)
    self.setup_scripts(context)
    self.post_setup(context)
```

Each of the methods `ensure_directories`, `create_configuration`, `setup_python`, `setup_scripts` and `post_setup` can be overridden.

</div>

<div class="method">

ensure_directories(env_dir)

Creates the environment directory and all necessary subdirectories that don't already exist, and returns a context object. This context object is just a holder for attributes (such as paths) for use by the other methods. If the `EnvBuilder` is created with the arg `clear=True`, contents of the environment directory will be cleared and then all necessary subdirectories will be recreated.

The returned context object is a `types.SimpleNamespace` with the following attributes:

- `env_dir` - The location of the virtual environment. Used for `__VENV_DIR__` in activation scripts (see `install_scripts`).
- `env_name` - The name of the virtual environment. Used for `__VENV_NAME__` in activation scripts (see `install_scripts`).
- `prompt` - The prompt to be used by the activation scripts. Used for `__VENV_PROMPT__` in activation scripts (see `install_scripts`).
- `executable` - The underlying Python executable used by the virtual environment. This takes into account the case where a virtual environment is created from another virtual environment.
- `inc_path` - The include path for the virtual environment.
- `lib_path` - The purelib path for the virtual environment.
- `platlib_path` - The platlib path for the virtual environment.
- `bin_path` - The script path for the virtual environment.
- `bin_name` - The name of the script path relative to the virtual environment location. Used for `__VENV_BIN_NAME__` in activation scripts (see `install_scripts`).
- `env_exe` - The name of the Python interpreter in the virtual environment. Used for `__VENV_PYTHON__` in activation scripts (see `install_scripts`).
- `env_exec_cmd` - The name of the Python interpreter, taking into account filesystem redirections. This can be used to run Python in the virtual environment.

<div class="versionchanged">

3.11 The *venv* `sysconfig installation scheme <installation_paths>` is used to construct the paths of the created directories.

</div>

<div class="versionchanged">

3.12 The attribute `lib_path` was added to the context, and the context object was documented.

</div>

<div class="versionchanged">

3.15 The attribute `platlib_path` was added to the context.

</div>

</div>

<div class="method">

create_configuration(context)

Creates the `pyvenv.cfg` configuration file in the environment.

</div>

<div class="method">

setup_python(context)

Creates a copy or symlink to the Python executable in the environment. On POSIX systems, if a specific executable `python3.x` was used, symlinks to `python` and `python3` will be created pointing to that executable, unless files with those names already exist. On POSIX systems, a broken symlink at a destination path is removed before the copy or symlink is created.

<div class="versionchanged">

3.16 A broken symlink at a destination path is now removed and replaced. Previously it was left in place, or it made the copy fail.

</div>

</div>

<div class="method">

setup_scripts(context)

Installs activation scripts appropriate to the platform into the virtual environment.

</div>

<div class="method">

upgrade_dependencies(context)

Upgrades the core venv dependency packages (currently `pip`) in the environment. This is done by shelling out to the `pip` executable in the environment.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.12

`setuptools` is no longer a core venv dependency.

</div>

</div>

<div class="method">

post_setup(context)

A placeholder method which can be overridden in third party implementations to pre-install packages in the virtual environment or perform other post-creation steps.

</div>

<div class="method">

install_scripts(context, path)

This method can be called from `setup_scripts` or `post_setup` in subclasses to assist in installing custom scripts into the virtual environment.

*path* is the path to a directory that should contain subdirectories `common`, `posix`, `nt`; each containing scripts destined for the `bin` directory in the environment. The contents of `common` and the directory corresponding to `os.name` are copied after some text replacement of placeholders:

- `__VENV_DIR__` is replaced with the absolute path of the environment directory.
- `__VENV_NAME__` is replaced with the environment name (final path segment of environment directory).
- `__VENV_PROMPT__` is replaced with the prompt (the environment name surrounded by parentheses and with a following space)
- `__VENV_BIN_NAME__` is replaced with the name of the bin directory (either `bin` or `Scripts`).
- `__VENV_PYTHON__` is replaced with the absolute path of the environment's executable.

The directories are allowed to exist (for when an existing environment is being upgraded).

</div>

<div class="method">

create_git_ignore_file(context)

Creates a `.gitignore` file within the virtual environment that causes the entire directory to be ignored by the Git source control manager.

<div class="versionadded">

3.13

</div>

</div>

<div class="versionchanged">

3.7.2 Windows now uses redirector scripts for `python[w].exe` instead of copying the actual binaries. In 3.7.2 only `setup_python` does nothing unless running from a build in the source tree.

</div>

<div class="versionchanged">

3.7.3 Windows copies the redirector scripts as part of `setup_python` instead of `setup_scripts`. This was not the case in 3.7.2. When using symlinks, the original executables will be linked.

</div>

</div>

There is also a module-level convenience function:

<div class="function">

create(env_dir, system_site_packages=False, clear=False, symlinks=None, with_pip=False, prompt=None, upgrade_deps=False, \*, scm_ignore_files=frozenset())

Create an `EnvBuilder` with the given keyword arguments, and call its `~EnvBuilder.create` method with the *env_dir* argument.

<div class="versionadded">

3.3

</div>

<div class="versionchanged">

3.4 Added the *with_pip* parameter

</div>

<div class="versionchanged">

3.6 Added the *prompt* parameter

</div>

<div class="versionchanged">

3.9 Added the *upgrade_deps* parameter

</div>

<div class="versionchanged">

3.13 Added the *scm_ignore_files* parameter

</div>

<div class="versionchanged">

3.16 The default value of *symlinks* is now platform-dependent.

</div>

</div>

## An example of extending `EnvBuilder`

The following script shows how to extend `EnvBuilder` by implementing a subclass which installs setuptools and pip into a created virtual environment:

``` python
import os
import os.path
from subprocess import Popen, PIPE
import sys
from threading import Thread
from urllib.parse import urlsplit
from urllib.request import urlretrieve
import venv

class ExtendedEnvBuilder(venv.EnvBuilder):
    """
    This builder installs setuptools and pip so that you can pip or
    easy_install other packages into the created virtual environment.

    :param nodist: If true, setuptools and pip are not installed into the
                   created virtual environment.
    :param nopip: If true, pip is not installed into the created
                  virtual environment.
    :param progress: If setuptools or pip are installed, the progress of the
                     installation can be monitored by passing a progress
                     callable. If specified, it is called with two
                     arguments: a string indicating some progress, and a
                     context indicating where the string is coming from.
                     The context argument can have one of three values:
                     'main', indicating that it is called from virtualize()
                     itself, and 'stdout' and 'stderr', which are obtained
                     by reading lines from the output streams of a subprocess
                     which is used to install the app.

                     If a callable is not specified, default progress
                     information is output to sys.stderr.
    """

    def __init__(self, *args, **kwargs):
        self.nodist = kwargs.pop('nodist', False)
        self.nopip = kwargs.pop('nopip', False)
        self.progress = kwargs.pop('progress', None)
        self.verbose = kwargs.pop('verbose', False)
        super().__init__(*args, **kwargs)

    def post_setup(self, context):
        """
        Set up any packages which need to be pre-installed into the
        virtual environment being created.

        :param context: The information for the virtual environment
                        creation request being processed.
        """
        os.environ['VIRTUAL_ENV'] = context.env_dir
        if not self.nodist:
            self.install_setuptools(context)
        # Can't install pip without setuptools
        if not self.nopip and not self.nodist:
            self.install_pip(context)

    def reader(self, stream, context):
        """
        Read lines from a subprocess' output stream and either pass to a progress
        callable (if specified) or write progress information to sys.stderr.
        """
        progress = self.progress
        while True:
            s = stream.readline()
            if not s:
                break
            if progress is not None:
                progress(s, context)
            else:
                if not self.verbose:
                    sys.stderr.write('.')
                else:
                    sys.stderr.write(s.decode('utf-8'))
                sys.stderr.flush()
        stream.close()

    def install_script(self, context, name, url):
        _, _, path, _, _ = urlsplit(url)
        fn = os.path.split(path)[-1]
        binpath = context.bin_path
        distpath = os.path.join(binpath, fn)
        # Download script into the virtual environment's binaries folder
        urlretrieve(url, distpath)
        progress = self.progress
        if self.verbose:
            term = '\n'
        else:
            term = ''
        if progress is not None:
            progress('Installing %s ...%s' % (name, term), 'main')
        else:
            sys.stderr.write('Installing %s ...%s' % (name, term))
            sys.stderr.flush()
        # Install in the virtual environment
        args = [context.env_exe, fn]
        p = Popen(args, stdout=PIPE, stderr=PIPE, cwd=binpath)
        t1 = Thread(target=self.reader, args=(p.stdout, 'stdout'))
        t1.start()
        t2 = Thread(target=self.reader, args=(p.stderr, 'stderr'))
        t2.start()
        p.wait()
        t1.join()
        t2.join()
        if progress is not None:
            progress('done.', 'main')
        else:
            sys.stderr.write('done.\n')
        # Clean up - no longer needed
        os.unlink(distpath)

    def install_setuptools(self, context):
        """
        Install setuptools in the virtual environment.

        :param context: The information for the virtual environment
                        creation request being processed.
        """
        url = "https://bootstrap.pypa.io/ez_setup.py"
        self.install_script(context, 'setuptools', url)
        # clear up the setuptools archive which gets downloaded
        pred = lambda o: o.startswith('setuptools-') and o.endswith('.tar.gz')
        files = filter(pred, os.listdir(context.bin_path))
        for f in files:
            f = os.path.join(context.bin_path, f)
            os.unlink(f)

    def install_pip(self, context):
        """
        Install pip in the virtual environment.

        :param context: The information for the virtual environment
                        creation request being processed.
        """
        url = 'https://bootstrap.pypa.io/get-pip.py'
        self.install_script(context, 'pip', url)


def main(args=None):
    import argparse

    parser = argparse.ArgumentParser(prog=__name__,
                                     description='Creates virtual Python '
                                                 'environments in one or '
                                                 'more target '
                                                 'directories.')
    parser.add_argument('dirs', metavar='ENV_DIR', nargs='+',
                        help='A directory in which to create the '
                             'virtual environment.')
    parser.add_argument('--no-setuptools', default=False,
                        action='store_true', dest='nodist',
                        help="Don't install setuptools or pip in the "
                             "virtual environment.")
    parser.add_argument('--no-pip', default=False,
                        action='store_true', dest='nopip',
                        help="Don't install pip in the virtual "
                             "environment.")
    parser.add_argument('--system-site-packages', default=False,
                        action='store_true', dest='system_site',
                        help='Give the virtual environment access to the '
                             'system site-packages dir.')
    if os.name == 'nt':
        use_symlinks = False
    else:
        use_symlinks = True
    parser.add_argument('--symlinks', default=use_symlinks,
                        action='store_true', dest='symlinks',
                        help='Try to use symlinks rather than copies, '
                             'when symlinks are not the default for '
                             'the platform.')
    parser.add_argument('--clear', default=False, action='store_true',
                        dest='clear', help='Delete the contents of the '
                                           'virtual environment '
                                           'directory if it already '
                                           'exists, before virtual '
                                           'environment creation.')
    parser.add_argument('--upgrade', default=False, action='store_true',
                        dest='upgrade', help='Upgrade the virtual '
                                             'environment directory to '
                                             'use this version of '
                                             'Python, assuming Python '
                                             'has been upgraded '
                                             'in-place.')
    parser.add_argument('--verbose', default=False, action='store_true',
                        dest='verbose', help='Display the output '
                                             'from the scripts which '
                                             'install setuptools and pip.')
    options = parser.parse_args(args)
    if options.upgrade and options.clear:
        raise ValueError('you cannot supply --upgrade and --clear together.')
    builder = ExtendedEnvBuilder(system_site_packages=options.system_site,
                                   clear=options.clear,
                                   symlinks=options.symlinks,
                                   upgrade=options.upgrade,
                                   nodist=options.nodist,
                                   nopip=options.nopip,
                                   verbose=options.verbose)
    for d in options.dirs:
        builder.create(d)

if __name__ == '__main__':
    rc = 1
    try:
        main()
        rc = 0
    except Exception as e:
        print('Error: %s' % e, file=sys.stderr)
    sys.exit(rc)
```

This script is also available for download [online](https://gist.github.com/vsajip/4673395).

# `!warnings` --- Warning control

<div class="module" data-synopsis="Issue warning messages and control their disposition.">

warnings

</div>

**Source code:** `Lib/warnings.py`

<div class="index">

single: warnings

</div>

------------------------------------------------------------------------

Warning messages are typically issued in situations where it is useful to alert the user of some condition in a program, where that condition (normally) doesn't warrant raising an exception and terminating the program. For example, one might want to issue a warning when a program uses an obsolete module.

Python programmers issue warnings by calling the `warn` function defined in this module. (C programmers use `PyErr_WarnEx`; see `exceptionhandling` for details).

Warning messages are normally written to `sys.stderr`, but their disposition can be changed flexibly, from ignoring all warnings to turning them into exceptions. The disposition of warnings can vary based on the `warning category
<warning-categories>`, the text of the warning message, and the source location where it is issued. Repetitions of a particular warning for the same source location are typically suppressed.

There are two stages in warning control: first, each time a warning is issued, a determination is made whether a message should be issued or not; next, if a message is to be issued, it is formatted and printed using a user-settable hook.

The determination whether to issue a warning message is controlled by the `warning filter <warning-filter>`, which is a sequence of matching rules and actions. Rules can be added to the filter by calling `filterwarnings` and reset to its default state by calling `resetwarnings`.

The printing of warning messages is done by calling `showwarning`, which may be overridden; the default implementation of this function formats the message by calling `formatwarning`, which is also available for use by custom implementations.

<div class="seealso">

`logging.captureWarnings` allows you to handle all warnings with the standard logging infrastructure.

</div>

## Warning Categories

There are a number of built-in exceptions that represent warning categories. This categorization is useful to be able to filter out groups of warnings.

While these are technically `built-in exceptions <warning-categories-as-exceptions>`, they are documented here, because conceptually they belong to the warnings mechanism.

User code can define additional warning categories by subclassing one of the standard warning categories. A warning category must always be a subclass of the `Warning` class.

The following warnings category classes are currently defined:

<div class="tabularcolumns">

\|l\|p{0.6linewidth}\|

</div>

| Class | Description |
|----|----|
| `Warning` | Base class for warning categories. It is a subclass of `Exception`. |
| `UserWarning` | Base class for warnings generated by user code. The default category for `warn`. |
| `DeprecationWarning` | Base class for warnings about deprecated features when those warnings are intended for other Python developers (ignored by default, unless triggered by code in `__main__`). |
| `PendingDeprecationWarning` | Base class for warnings about features that will be deprecated in the future (ignored by default). |
| `SyntaxWarning` | Base class for warnings about dubious syntax (typically emitted when compiling Python source code, and hence may not be suppressed by runtime filters). |
| `RuntimeWarning` | Base class for warnings about dubious runtime behavior. |
| `FutureWarning` | Base class for warnings about deprecated features when those warnings are intended for end users of applications that are written in Python. |
| `ImportWarning` | Base class for warnings triggered during the process of importing a module (ignored by default). |
| `UnicodeWarning` | Base class for warnings related to Unicode. |
| `EncodingWarning` | Base class for warnings related to encodings. See `io-encoding-warning` for details. |
| `BytesWarning` | Base class for warnings related to `bytes` and `bytearray`. |
| `ResourceWarning` | Base class for warnings related to resource usage (ignored by default). |

<div class="versionchanged">

3.7 Previously `DeprecationWarning` and `FutureWarning` were distinguished based on whether a feature was being removed entirely or changing its behaviour. They are now distinguished based on their intended audience and the way they're handled by the default warnings filters.

</div>

## The Warnings Filter

The warnings filter controls whether warnings are ignored, displayed, or turned into errors (raising an exception).

Conceptually, the warnings filter maintains an ordered list of filter specifications; any specific warning is matched against each filter specification in the list in turn until a match is found; the filter determines the disposition of the match. Each entry is a tuple of the form (*action*, *message*, *category*, *module*, *lineno*), where:

- *action* is one of the following strings:

  | Value | Disposition |
  |----|----|
  | `"default"` | print the first occurrence of matching warnings for each location (module + line number) where the warning is issued |
  | `"error"` | turn matching warnings into exceptions |
  | `"ignore"` | never print matching warnings |
  | `"always"` | always print matching warnings |
  | `"all"` | alias to "always" |
  | `"module"` | print the first occurrence of matching warnings for each module where the warning is issued (regardless of line number) |
  | `"once"` | print only the first occurrence of matching warnings, regardless of location |

- *message* is a string containing a regular expression that the start of the warning message must match, case-insensitively. In `-W` and `PYTHONWARNINGS`, if *message* starts and ends with a forward slash (`/`), it specifies a regular expression as above; otherwise it is a literal string that the start of the warning message must match (case-insensitively), ignoring any whitespace at the start or end of *message*.

- *category* is a class (a subclass of `Warning`) of which the warning category must be a subclass in order to match.

- *module* is a string containing a regular expression that the start of the fully qualified module name must match, case-sensitively. In `-W` and `PYTHONWARNINGS`, if *module* starts and ends with a forward slash (`/`), it specifies a regular expression as above; otherwise it is a literal string that the fully qualified module name must be equal to (case-sensitively), ignoring any whitespace at the start or end of *module*.

- *lineno* is an integer that the line number where the warning occurred must match, or `0` to match all line numbers.

Since the `Warning` class is derived from the built-in `Exception` class, to turn a warning into an error we simply raise `category(message)`.

If a warning is reported and doesn't match any registered filter then the "default" action is applied (hence its name).

### Repeated Warning Suppression Criteria

The filters that suppress repeated warnings apply the following criteria to determine if a warning is considered a repeat:

- `"default"`: A warning is considered a repeat only if the (*message*, *category*, *module*, *lineno*) are all the same.
- `"module"`: A warning is considered a repeat if the (*message*, *category*, *module*) are the same, ignoring the line number.
- `"once"`: A warning is considered a repeat if the (*message*, *category*) are the same, ignoring the module and line number.

### Describing Warning Filters

The warnings filter is initialized by `-W` options passed to the Python interpreter command line and the `PYTHONWARNINGS` environment variable. The interpreter saves the arguments for all supplied entries without interpretation in `sys.warnoptions`; the `!warnings` module parses these when it is first imported (invalid options are ignored, after printing a message to `sys.stderr`).

Individual warnings filters are specified as a sequence of fields separated by colons:

    action:message:category:module:line

The meaning of each of these fields is as described in `warning-filter`. When listing multiple filters on a single line (as for `PYTHONWARNINGS`), the individual filters are separated by commas and the filters listed later take precedence over those listed before them (as they're applied left-to-right, and the most recently applied filters take precedence over earlier ones).

Commonly used warning filters apply to either all warnings, warnings in a particular category, or warnings raised by particular modules or packages. Some examples:

    default                      # Show all warnings (even those ignored by default)
    ignore                       # Ignore all warnings
    error                        # Convert all warnings to errors
    error::ResourceWarning       # Treat ResourceWarning messages as errors
    default::DeprecationWarning  # Show DeprecationWarning messages
    ignore,default:::mymodule    # Only report warnings triggered by "mymodule"
    error:::mymodule             # Convert warnings to errors in "mymodule"

### Default Warning Filter

By default, Python installs several warning filters, which can be overridden by the `-W` command-line option, the `PYTHONWARNINGS` environment variable and calls to `filterwarnings`.

In regular release builds, the default warning filter has the following entries (in order of precedence):

    default::DeprecationWarning:__main__
    ignore::DeprecationWarning
    ignore::PendingDeprecationWarning
    ignore::ImportWarning
    ignore::ResourceWarning

In a `debug build <debug-build>`, the list of default warning filters is empty.

<div class="versionchanged">

3.2 `DeprecationWarning` is now ignored by default in addition to `PendingDeprecationWarning`.

</div>

<div class="versionchanged">

3.7 `DeprecationWarning` is once again shown by default when triggered directly by code in `__main__`.

</div>

<div class="versionchanged">

3.7 `BytesWarning` no longer appears in the default filter list and is instead configured via `sys.warnoptions` when `-b` is specified twice.

</div>

### Overriding the default filter

Developers of applications written in Python may wish to hide *all* Python level warnings from their users by default, and only display them when running tests or otherwise working on the application. The `sys.warnoptions` attribute used to pass filter configurations to the interpreter can be used as a marker to indicate whether or not warnings should be disabled:

    import sys

    if not sys.warnoptions:
        import warnings
        warnings.simplefilter("ignore")

Developers of test runners for Python code are advised to instead ensure that *all* warnings are displayed by default for the code under test, using code like:

    import sys

    if not sys.warnoptions:
        import os, warnings
        warnings.simplefilter("default") # Change the filter in this process
        os.environ["PYTHONWARNINGS"] = "default" # Also affect subprocesses

Finally, developers of interactive shells that run user code in a namespace other than `__main__` are advised to ensure that `DeprecationWarning` messages are made visible by default, using code like the following (where `user_ns` is the module used to execute code entered interactively):

    import warnings
    warnings.filterwarnings("default", category=DeprecationWarning,
                                       module=user_ns.get("__name__"))

## Temporarily Suppressing Warnings

If you are using code that you know will raise a warning, such as a deprecated function, but do not want to see the warning (even when warnings have been explicitly configured via the command line), then it is possible to suppress the warning using the `catch_warnings` context manager:

    import warnings

    def fxn():
        warnings.warn("deprecated", DeprecationWarning)

    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        fxn()

While within the context manager all warnings will simply be ignored. This allows you to use known-deprecated code without having to see the warning while not suppressing the warning for other code that might not be aware of its use of deprecated code.

> > [!NOTE]
> > See `warning-concurrent-safe` for details on the concurrency-safety of the `catch_warnings` context manager when used in programs using multiple threads or async functions.

## Testing Warnings

To test warnings raised by code, use the `catch_warnings` context manager. With it you can temporarily mutate the warnings filter to facilitate your testing. For instance, do the following to capture all raised warnings to check:

    import warnings

    def fxn():
        warnings.warn("deprecated", DeprecationWarning)

    with warnings.catch_warnings(record=True) as w:
        # Cause all warnings to always be triggered.
        warnings.simplefilter("always")
        # Trigger a warning.
        fxn()
        # Verify some things
        assert len(w) == 1
        assert issubclass(w[-1].category, DeprecationWarning)
        assert "deprecated" in str(w[-1].message)

One can also cause all warnings to be exceptions by using `error` instead of `always`. One thing to be aware of is that if a warning has already been raised because of a `once`/`default` rule, then no matter what filters are set the warning will not be seen again unless the warnings registry related to the warning has been cleared.

Once the context manager exits, the warnings filter is restored to its state when the context was entered. This prevents tests from changing the warnings filter in unexpected ways between tests and leading to indeterminate test results.

> > [!NOTE]
> > See `warning-concurrent-safe` for details on the concurrency-safety of the `catch_warnings` context manager when used in programs using multiple threads or async functions.

When testing multiple operations that raise the same kind of warning, it is important to test them in a manner that confirms each operation is raising a new warning (e.g. set warnings to be raised as exceptions and check the operations raise exceptions, check that the length of the warning list continues to increase after each operation, or else delete the previous entries from the warnings list before each new operation).

## Updating Code For New Versions of Dependencies

Warning categories that are primarily of interest to Python developers (rather than end users of applications written in Python) are ignored by default.

Notably, this "ignored by default" list includes `DeprecationWarning` (for every module except `__main__`), which means developers should make sure to test their code with typically ignored warnings made visible in order to receive timely notifications of future breaking API changes (whether in the standard library or third party packages).

In the ideal case, the code will have a suitable test suite, and the test runner will take care of implicitly enabling all warnings when running tests (the test runner provided by the `unittest` module does this).

In less ideal cases, applications can be checked for use of deprecated interfaces by passing `-Wd <-W>` to the Python interpreter (this is shorthand for `!-W default`) or setting `PYTHONWARNINGS=default` in the environment. This enables default handling for all warnings, including those that are ignored by default. To change what action is taken for encountered warnings you can change what argument is passed to `-W` (e.g. `!-W error`). See the `-W` flag for more details on what is possible.

## Available Functions

<div class="function">

warn(message, category=None, stacklevel=1, source=None, \*, skip_file_prefixes=())

Issue a warning, or maybe ignore it or raise an exception. The *category* argument, if given, must be a `warning category class <warning-categories>`; it defaults to `UserWarning`. Alternatively, *message* can be a `Warning` instance, in which case *category* will be ignored and `message.__class__` will be used. In this case, the message text will be `str(message)`. This function raises an exception if the particular warning issued is changed into an error by the `warnings filter <warning-filter>`. The *stacklevel* argument can be used by wrapper functions written in Python, like this:

    def deprecated_api(message):
        warnings.warn(message, DeprecationWarning, stacklevel=2)

This makes the warning refer to `deprecated_api`'s caller, rather than to the source of `deprecated_api` itself (since the latter would defeat the purpose of the warning message).

The *skip_file_prefixes* keyword argument can be used to indicate which stack frames are ignored when counting stack levels. This can be useful when you want the warning to always appear at call sites outside of a package when a constant *stacklevel* does not fit all call paths or is otherwise challenging to maintain. If supplied, it must be a tuple of strings. When prefixes are supplied, stacklevel is implicitly overridden to be `max(2, stacklevel)`. To cause a warning to be attributed to the caller from outside of the current package you might write:

    # example/lower.py
    _warn_skips = (os.path.dirname(__file__),)

    def one_way(r_luxury_yacht=None, t_wobbler_mangrove=None):
        if r_luxury_yacht:
            warnings.warn("Please migrate to t_wobbler_mangrove=.",
                          skip_file_prefixes=_warn_skips)

    # example/higher.py
    from . import lower

    def another_way(**kw):
        lower.one_way(**kw)

This makes the warning refer to both the `example.lower.one_way()` and `example.higher.another_way()` call sites only from calling code living outside of `example` package.

*source*, if supplied, is the destroyed object which emitted a `ResourceWarning`.

<div class="versionchanged">

3.6 Added *source* parameter.

</div>

<div class="versionchanged">

3.12 Added *skip_file_prefixes*.

</div>

</div>

<div class="function">

warn_explicit(message, category, filename, lineno, module=None, registry=None, module_globals=None, source=None)

This is a low-level interface to the functionality of `warn`, passing in explicitly the message, category, filename and line number, and optionally other arguments. *message* must be a string and *category* a subclass of `Warning` or *message* may be a `Warning` instance, in which case *category* will be ignored.

*module*, if supplied, should be the module name. If no module is passed, the module regular expression in `warnings filter <warning-filter>` will be tested against the module names constructed from the path components starting from all parent directories (with `/__init__.py`, `.py` and, on Windows, `.pyw` stripped) and against the filename with `.py` stripped. For example, when the filename is `'/path/to/package/module.py'`, it will be tested against `'path.to.package.module'`, `'to.package.module'` `'package.module'`, `'module'`, and `'/path/to/package/module'`.

*registry*, if supplied, should be the `__warningregistry__` dictionary of the module. If no registry is passed, each warning is treated as the first occurrence, that is, filter actions `"default"`, `"module"` and `"once"` are handled as `"always"`.

*module_globals*, if supplied, should be the global namespace in use by the code for which the warning is issued. (This argument is used to support displaying source for modules found in zipfiles or other non-filesystem import sources).

*source*, if supplied, is the destroyed object which emitted a `ResourceWarning`.

<div class="versionchanged">

3.6 Add the *source* parameter.

</div>

<div class="versionchanged">

3.15 If no module is passed, test the filter regular expression against module names created from the path, not only the path itself.

</div>

</div>

<div class="function">

showwarning(message, category, filename, lineno, file=None, line=None)

Write a warning to a file. The default implementation calls `formatwarning(message, category, filename, lineno, line)` and writes the resulting string to *file*, which defaults to `sys.stderr`. You may replace this function with any callable by assigning to `warnings.showwarning`. *line* is a line of source code to be included in the warning message; if *line* is not supplied, `showwarning` will try to read the line specified by *filename* and *lineno*.

</div>

<div class="function">

formatwarning(message, category, filename, lineno, line=None)

Format a warning the standard way. This returns a string which may contain embedded newlines and ends in a newline. *line* is a line of source code to be included in the warning message; if *line* is not supplied, `formatwarning` will try to read the line specified by *filename* and *lineno*.

</div>

<div class="function">

filterwarnings(action, message='', category=Warning, module='', lineno=0, append=False)

Insert an entry into the list of `warnings filter specifications
<warning-filter>`. The entry is inserted at the front by default; if *append* is true, it is inserted at the end. This checks the types of the arguments, compiles the *message* and *module* regular expressions, and inserts them as a tuple in the list of warnings filters. Entries closer to the front of the list override entries later in the list, if both match a particular warning. Omitted arguments default to a value that matches everything.

</div>

<div class="function">

simplefilter(action, category=Warning, lineno=0, append=False)

Insert a simple entry into the list of `warnings filter specifications
<warning-filter>`. The meaning of the function parameters is as for `filterwarnings`, but regular expressions are not needed as the filter inserted always matches any message in any module as long as the category and line number match.

</div>

<div class="function">

resetwarnings()

Reset the warnings filter. This discards the effect of all previous calls to `filterwarnings`, including that of the `-W` command line options and calls to `simplefilter`.

</div>

<div class="decorator">

deprecated(message, /, \*, category=DeprecationWarning, stacklevel=1)

Decorator to indicate that a class, function or overload is deprecated.

When this decorator is applied to an object, deprecation warnings may be emitted at runtime when the object is used. `static type checkers <static type checker>` will also generate a diagnostic on usage of the deprecated object.

Usage:

    from warnings import deprecated
    from typing import overload

    @deprecated("Use B instead")
    class A:
        pass

    @deprecated("Use g instead")
    def f():
        pass

    @overload
    @deprecated("int support is deprecated")
    def g(x: int) -> int: ...
    @overload
    def g(x: str) -> int: ...

The warning specified by *category* will be emitted at runtime on use of deprecated objects. For functions, that happens on calls; for classes, on instantiation and on creation of subclasses. If the *category* is `None`, no warning is emitted at runtime. The *stacklevel* determines where the warning is emitted. If it is `1` (the default), the warning is emitted at the direct caller of the deprecated object; if it is higher, it is emitted further up the stack. Static type checker behavior is not affected by the *category* and *stacklevel* arguments.

The deprecation message passed to the decorator is saved in the `__deprecated__` attribute on the decorated object. If applied to an overload, the decorator must be after the `~typing.overload` decorator for the attribute to exist on the overload as returned by `typing.get_overloads`.

<div class="versionadded">

3.13 See `702`.

</div>

</div>

## Available Context Managers

<div class="catch_warnings(*, record=False, module=None, action=None, category=Warning, lineno=0, append=False)">

A context manager that copies and, upon exit, restores the warnings filter and the `showwarning` function. If the *record* argument is `False` (the default) the context manager returns `None` on entry. If *record* is `True`, a list is returned that is progressively populated with objects as seen by a custom `showwarning` function (which also suppresses output to `sys.stderr`). Each object in the list is guaranteed to have the following attributes:

> - `message`: the warning message (an instance of `Warning`)
> - `category`: the warning category (a subclass of `Warning`)
> - `filename`: the file name where the warning occurred (`str`)
> - `lineno`: the line number in the file (`int`)
> - `file`: the file object used for output (if any), or `None`
> - `line`: the line of source code (if available), or `None`
> - `source`: the original object that generated the warning (if available), or `None`
> - `module`: the module name where the warning occurred (`str`), or `None`

<div class="versionchanged">

3.6 The `source` attribute was added.

</div>

<div class="versionchanged">

3.15 The `module` attribute was added.

</div>

The type of these objects is not specified and may change; only the presence of these attributes is guaranteed.

The *module* argument takes a module that will be used instead of the module returned when you import `!warnings` whose filter will be protected. This argument exists primarily for testing the `!warnings` module itself.

If the *action* argument is not `None`, the remaining arguments are passed to `simplefilter` as if it were called immediately on entering the context.

See `warning-filter` for the meaning of the *category* and *lineno* parameters.

> [!NOTE]
> See `warning-concurrent-safe` for details on the concurrency-safety of the `catch_warnings` context manager when used in programs using multiple threads or async functions.

<div class="versionchanged">

3.11

Added the *action*, *category*, *lineno*, and *append* parameters.

</div>

</div>

## Concurrent safety of Context Managers

The behavior of `catch_warnings` context manager depends on the `sys.flags.context_aware_warnings` flag. If the flag is true, the context manager behaves in a concurrent-safe fashion and otherwise not. Concurrent-safe means that it is both thread-safe and safe to use within `asyncio coroutines <coroutine>` and tasks. Being thread-safe means that behavior is predictable in a multi-threaded program. The flag defaults to true for free-threaded builds and false otherwise.

If the `~sys.flags.context_aware_warnings` flag is false, then `catch_warnings` will modify the global attributes of the `!warnings` module. This is not safe if used within a concurrent program (using multiple threads or using asyncio coroutines). For example, if two or more threads use the `catch_warnings` class at the same time, the behavior is undefined.

If the flag is true, `catch_warnings` will not modify global attributes and will instead use a `~contextvars.ContextVar` to store the newly established warning filtering state. A context variable provides thread-local storage and it makes the use of `catch_warnings` thread-safe.

The *record* parameter of the context handler also behaves differently depending on the value of the flag. When *record* is true and the flag is false, the context manager works by replacing and then later restoring the module's `showwarning` function. That is not concurrent-safe.

When *record* is true and the flag is true, the `showwarning` function is not replaced. Instead, the recording status is indicated by an internal property in the context variable. In this case, the `showwarning` function will not be restored when exiting the context handler.

The `~sys.flags.context_aware_warnings` flag can be set the `-X
context_aware_warnings<-X>` command-line option or by the `PYTHON_CONTEXT_AWARE_WARNINGS` environment variable.

> > [!NOTE]
> > It is likely that most programs that desire thread-safe behaviour of the warnings module will also want to set the `~sys.flags.thread_inherit_context` flag to true. That flag causes threads created by `threading.Thread` to start with a copy of the context variables from the thread starting it. When true, the context established by `catch_warnings` in one thread will also apply to new threads started by it. If false, new threads will start with an empty warnings context variable, meaning that any filtering that was established by a `catch_warnings` context manager will no longer be active.

<div class="versionchanged">

3.14

Added the `sys.flags.context_aware_warnings` flag and the use of a context variable for `catch_warnings` if the flag is true. Previous versions of Python acted as if the flag was always set to false.

</div>

# `!wave` --- Read and write WAV files

<div class="module" data-synopsis="Provide an interface to the WAV sound format.">

wave

</div>

**Source code:** `Lib/wave.py`

------------------------------------------------------------------------

The `!wave` module provides a convenient interface to the Waveform Audio "WAVE" (or "WAV") file format.

The module supports uncompressed PCM and IEEE floating-point WAV formats.

<div class="versionchanged">

3.12

Support for `WAVE_FORMAT_EXTENSIBLE` headers was added, provided that the extended format is `KSDATAFORMAT_SUBTYPE_PCM`.

</div>

<div class="versionchanged">

3.15

Support for reading and writing `WAVE_FORMAT_IEEE_FLOAT` files was added.

</div>

The `!wave` module defines the following function and exception:

<div class="function">

open(file, mode=None)

If *file* is a string, a `path-like object` or a `bytes-like object` open the file by that name, otherwise treat it as a file-like object. *mode* can be:

`'rb'`  
Read only mode.

`'wb'`  
Write only mode.

Note that it does not allow read/write WAV files.

A *mode* of `'rb'` returns a `Wave_read` object, while a *mode* of `'wb'` returns a `Wave_write` object. If *mode* is omitted and a file-like object is passed as *file*, `file.mode` is used as the default value for *mode*.

If you pass in a file-like object, the wave object will not close it when its `close()` method is called; it is the caller's responsibility to close the file object.

The `.open` function may be used in a `with` statement. When the `!with` block completes, the `Wave_read.close` or `Wave_write.close` method is called.

<div class="versionchanged">

3.4 Added support for unseekable files.

</div>

<div class="versionchanged">

3.15 Added support for `path-like objects <path-like object>` and `bytes-like objects <bytes-like object>`.

</div>

</div>

<div class="exception">

Error

An error raised when something is impossible because it violates the WAV specification or hits an implementation deficiency.

</div>

<div class="data">

WAVE_FORMAT_PCM

Format code for uncompressed PCM audio.

</div>

<div class="data">

WAVE_FORMAT_IEEE_FLOAT

Format code for IEEE floating-point audio.

<div class="versionadded">

3.15

</div>

</div>

<div class="data">

WAVE_FORMAT_EXTENSIBLE

Format code for WAVE extensible headers.

<div class="versionadded">

3.12

</div>

</div>

## Wave_read Objects

<div class="Wave_read">

Read a WAV file.

Wave_read objects, as returned by `.open`, have the following methods:

<div class="method">

close()

Close the stream if it was opened by `!wave`, and make the instance unusable. This is called automatically on object collection.

</div>

<div class="method">

getnchannels()

Returns number of audio channels (`1` for mono, `2` for stereo).

</div>

<div class="method">

getsampwidth()

Returns sample width in bytes.

</div>

<div class="method">

getframerate()

Returns sampling frequency.

</div>

<div class="method">

getnframes()

Returns number of audio frames.

</div>

<div class="method">

getformat()

Returns the frame format code.

This is one of `WAVE_FORMAT_PCM`, `WAVE_FORMAT_IEEE_FLOAT`, or `WAVE_FORMAT_EXTENSIBLE`.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

getcomptype()

Returns compression type (`'NONE'` is the only supported type).

</div>

<div class="method">

getcompname()

Human-readable version of `getcomptype`. Usually `'not compressed'` parallels `'NONE'`.

</div>

<div class="method">

getparams()

Returns a `~collections.namedtuple` `(nchannels, sampwidth, framerate, nframes, comptype, compname)`, equivalent to output of the `get*()` methods.

</div>

<div class="method">

readframes(n)

Reads and returns at most *n* frames of audio, as a `bytes` object.

</div>

<div class="method">

rewind()

Rewind the file pointer to the beginning of the audio stream.

</div>

The following two methods define a term "position" which is compatible between them, and is otherwise implementation dependent.

<div class="method">

setpos(pos)

Set the file pointer to the specified position.

</div>

<div class="method">

tell()

Return current file pointer position.

</div>

</div>

## Wave_write Objects

<div class="Wave_write">

Write a WAV file.

Wave_write objects, as returned by `.open`.

For seekable output streams, the `wave` header will automatically be updated to reflect the number of frames actually written. For unseekable streams, the *nframes* value must be accurate when the first frame data is written. An accurate *nframes* value can be achieved either by calling `setnframes` or `setparams` with the number of frames that will be written before `close` is called and then using `writeframesraw` to write the frame data, or by calling `writeframes` with all of the frame data to be written. In the latter case `writeframes` will calculate the number of frames in the data and set *nframes* accordingly before writing the frame data.

<div class="versionchanged">

3.4 Added support for unseekable files.

</div>

Wave_write objects have the following methods:

<div class="method">

close()

Make sure *nframes* is correct, and close the file if it was opened by `!wave`. This method is called upon object collection. It will raise an exception if the output stream is not seekable and *nframes* does not match the number of frames actually written.

</div>

<div class="method">

setnchannels(n)

Set the number of channels.

</div>

<div class="method">

getnchannels()

Return the number of channels.

</div>

<div class="method">

setsampwidth(n)

Set the sample width to *n* bytes.

For `WAVE_FORMAT_IEEE_FLOAT`, only 4-byte (32-bit) and 8-byte (64-bit) sample widths are supported.

</div>

<div class="method">

getsampwidth()

Return the sample width in bytes.

</div>

<div class="method">

setframerate(n)

Set the frame rate to *n*.

<div class="versionchanged">

3.2 A non-integral input to this method is rounded to the nearest integer.

</div>

</div>

<div class="method">

getframerate()

Return the frame rate.

</div>

<div class="method">

setnframes(n)

Set the number of frames to *n*. This will be changed later if the number of frames actually written is different (this update attempt will raise an error if the output stream is not seekable).

</div>

<div class="method">

getnframes()

Return the number of audio frames written so far.

</div>

<div class="method">

setcomptype(type, name)

Set the compression type and description. At the moment, only compression type `NONE` is supported, meaning no compression.

</div>

<div class="method">

getcomptype()

Return the compression type (`'NONE'`).

</div>

<div class="method">

getcompname()

Return the human-readable compression type name.

</div>

<div class="method">

setformat(format)

Set the frame format code.

Supported values are `WAVE_FORMAT_PCM` and `WAVE_FORMAT_IEEE_FLOAT`.

When setting `WAVE_FORMAT_IEEE_FLOAT`, the sample width must be 4 or 8 bytes.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

getformat()

Return the current frame format code.

<div class="versionadded">

3.15

</div>

</div>

<div class="method">

setparams(tuple)

The *tuple* should be `(nchannels, sampwidth, framerate, nframes, comptype, compname, format)`, with values valid for the `set*()` methods. Sets all parameters.

For backwards compatibility, a 6-item tuple without *format* is also accepted and defaults to `WAVE_FORMAT_PCM`.

For `format=WAVE_FORMAT_IEEE_FLOAT`, *sampwidth* must be 4 or 8.

<div class="versionchanged">

3.15 Added optional item *format* to the *tuple*.

</div>

</div>

<div class="method">

getparams()

Return a `~collections.namedtuple` `(nchannels, sampwidth, framerate, nframes, comptype, compname)` containing the current output parameters.

</div>

<div class="method">

tell()

Return current position in the file, with the same disclaimer for the `Wave_read.tell` and `Wave_read.setpos` methods.

</div>

<div class="method">

writeframesraw(data)

Write audio frames, without correcting *nframes*.

<div class="versionchanged">

3.4 Any `bytes-like object` is now accepted.

</div>

</div>

<div class="method">

writeframes(data)

Write audio frames and make sure *nframes* is correct. It will raise an error if the output stream is not seekable and the total number of frames that have been written after *data* has been written does not match the previously set value for *nframes*.

<div class="versionchanged">

3.4 Any `bytes-like object` is now accepted.

</div>

Note that it is invalid to set any parameters after calling `writeframes` or `writeframesraw`, and any attempt to do so will raise `wave.Error`.

For `WAVE_FORMAT_IEEE_FLOAT` output, a `fact` chunk is written as required by the WAVE specification for non-PCM formats.

</div>

</div>

# `!weakref` --- Weak references

<div class="module" data-synopsis="Support for weak references and weak dictionaries.">

weakref

</div>

**Source code:** `Lib/weakref.py`

------------------------------------------------------------------------

The `!weakref` module allows the Python programmer to create `weak
references` to objects.

In the following, the term `referent` means the object which is referred to by a weak reference.

A weak reference to an object is not enough to keep the object alive: when the only remaining references to a referent are weak references, `garbage collection` is free to destroy the referent and reuse its memory for something else. However, until the object is actually destroyed the weak reference may return the object even if there are no strong references to it.

A primary use for weak references is to implement caches or mappings holding large objects, where it's desired that a large object not be kept alive solely because it appears in a cache or mapping.

For example, if you have a number of large binary image objects, you may wish to associate a name with each. If you used a Python dictionary to map names to images, or images to names, the image objects would remain alive just because they appeared as values or keys in the dictionaries. The `WeakKeyDictionary` and `WeakValueDictionary` classes supplied by the `!weakref` module are an alternative, using weak references to construct mappings that don't keep objects alive solely because they appear in the mapping objects. If, for example, an image object is a value in a `WeakValueDictionary`, then when the last remaining references to that image object are the weak references held by weak mappings, garbage collection can reclaim the object, and its corresponding entries in weak mappings are simply deleted.

`WeakKeyDictionary` and `WeakValueDictionary` use weak references in their implementation, setting up callback functions on the weak references that notify the weak dictionaries when a key or value has been reclaimed by garbage collection. `WeakSet` implements the `set` interface, but keeps weak references to its elements, just like a `WeakKeyDictionary` does.

`finalize` provides a straight forward way to register a cleanup function to be called when an object is garbage collected. This is simpler to use than setting up a callback function on a raw weak reference, since the module automatically ensures that the finalizer remains alive until the object is collected.

Most programs should find that using one of these weak container types or `finalize` is all they need -- it's not usually necessary to create your own weak references directly. The low-level machinery is exposed by the `!weakref` module for the benefit of advanced uses.

Not all objects can be weakly referenced. Objects which support weak references include class instances, functions written in Python (but not in C), instance methods, sets, frozensets, some `file objects <file object>`, `generators <generator>`, type objects, sockets, arrays, deques, regular expression pattern objects, code objects, and frame objects.

<div class="versionchanged">

3.2 Added support for thread.lock, threading.Lock, and code objects.

</div>

<div class="versionchanged">

3.16 Added support for frame objects.

</div>

Several built-in types such as `list` and `dict` do not directly support weak references but can add support through subclassing:

    class Dict(dict):
        pass

    obj = Dict(red=1, green=2, blue=3)   # this object is weak referenceable

<div class="impl-detail">

Other built-in types such as `tuple` and `int` do not support weak references even when subclassed.

</div>

Extension types can easily be made to support weak references; see `weakref-support`.

When `__slots__` are defined for a given type, weak reference support is disabled unless a `'__weakref__'` string is also present in the sequence of strings in the `__slots__` declaration. See `__slots__ documentation <slots>` for details.

<div class="ref(object[, callback])">

Return a weak reference to *object*. The original object can be retrieved by calling the reference object if the referent is still alive; if the referent is no longer alive, calling the reference object will cause `None` to be returned. If *callback* is provided and not `None`, and the returned weakref object is still alive, the callback will be called when the object is about to be finalized; the weak reference object will be passed as the only parameter to the callback; the referent will no longer be available.

It is allowable for many weak references to be constructed for the same object. Callbacks registered for each weak reference will be called from the most recently registered callback to the oldest registered callback.

Exceptions raised by the callback will be noted on the standard error output, but cannot be propagated; they are handled in exactly the same way as exceptions raised from an object's `~object.__del__` method.

Weak references are `hashable` if the *object* is hashable. They will maintain their hash value even after the *object* was deleted. If `hash` is called the first time only after the *object* was deleted, the call will raise `TypeError`.

Weak references support tests for equality, but not ordering. If the referents are still alive, two references have the same equality relationship as their referents (regardless of the *callback*). If either referent has been deleted, the references are equal only if the reference objects are the same object.

This is a subclassable type rather than a factory function.

Weak references are `generic <generics>` over the type of the object they reference.

A weak reference can be pickled if the referent is strongly referenced elsewhere in the same pickle; otherwise pickling raises `pickle.PicklingError`. The unpickled weak reference refers to the unpickled referent. A reference to a dead object unpickles as a reference to a dead object. The *callback* is not preserved.

Weak references are atomic for the `copy` module: `copy.copy` and `copy.deepcopy` return the original weak reference without copying the referent.

<div class="attribute">

\_\_callback\_\_

This read-only attribute returns the callback currently associated to the weakref. If there is no callback or if the referent of the weakref is no longer alive then this attribute will have value `None`.

</div>

<div class="versionchanged">

3.4 Added the `__callback__` attribute.

</div>

<div class="versionchanged">

3.16 Raise `!TypeError` if *callback* is not callable or `None`.

</div>

<div class="versionchanged">

next Weak references can now be pickled.

</div>

</div>

<div class="function">

proxy(object\[, callback\])

Return a proxy to *object* which uses a weak reference. This supports use of the proxy in most contexts instead of requiring the explicit dereferencing used with weak reference objects. The returned object will have a type of either `ProxyType` or `CallableProxyType`, depending on whether *object* is callable. Proxy objects are not `hashable` regardless of the referent; this avoids a number of problems related to their fundamentally mutable nature, and prevents their use as dictionary keys. *callback* is the same as the parameter of the same name to the `ref` function.

Accessing an attribute of the proxy object after the referent is garbage collected raises `ReferenceError`.

<div class="versionchanged">

3.8 Extended the operator support on proxy objects to include the matrix multiplication operators `@` and `@=`.

</div>

<div class="versionchanged">

3.16 Raise `!TypeError` if *callback* is not callable or `None`.

</div>

</div>

<div class="function">

getweakrefcount(object)

Return the number of weak references and proxies which refer to *object*.

</div>

<div class="function">

getweakrefs(object)

Return a list of all weak reference and proxy objects which refer to *object*.

</div>

<div class="WeakKeyDictionary([dict])">

Mapping class that references keys weakly. Entries in the dictionary will be discarded when there is no longer a strong reference to the key. This can be used to associate additional data with an object owned by other parts of an application without adding attributes to those objects. This can be especially useful with objects that override attribute accesses.

Note that when a key with equal value to an existing key (but not equal identity) is inserted into the dictionary, it replaces the value but does not replace the existing key. Due to this, when the reference to the original key is deleted, it also deletes the entry in the dictionary:

    >>> class T(str): pass
    ...
    >>> k1, k2 = T(), T()
    >>> d = weakref.WeakKeyDictionary()
    >>> d[k1] = 1   # d = {k1: 1}
    >>> d[k2] = 2   # d = {k1: 2}
    >>> del k1      # d = {}

A workaround would be to remove the key prior to reassignment:

    >>> class T(str): pass
    ...
    >>> k1, k2 = T(), T()
    >>> d = weakref.WeakKeyDictionary()
    >>> d[k1] = 1   # d = {k1: 1}
    >>> del d[k1]
    >>> d[k2] = 2   # d = {k2: 2}
    >>> del k1      # d = {k2: 2}

A `!WeakKeyDictionary` can be copied, and pickled if the keys are strongly referenced elsewhere in the same pickle (see `ref`).

<div class="versionchanged">

3.9 Added support for `|` and `|=` operators, as specified in `584`.

</div>

<div class="versionchanged">

next `!WeakKeyDictionary` can now be pickled.

</div>

</div>

`WeakKeyDictionary` objects have an additional method that exposes the internal references directly. The references are not guaranteed to be "live" at the time they are used, so the result of calling the references needs to be checked before being used. This can be used to avoid creating references that will cause the garbage collector to keep the keys around longer than needed.

<div class="method">

WeakKeyDictionary.keyrefs()

Return an iterable of the weak references to the keys.

</div>

<div class="WeakValueDictionary([dict])">

Mapping class that references values weakly. Entries in the dictionary will be discarded when no strong reference to the value exists any more.

A `!WeakValueDictionary` can be copied, and pickled if the values are strongly referenced elsewhere in the same pickle (see `ref`).

<div class="versionchanged">

3.9 Added support for `|` and `|=` operators, as specified in `584`.

</div>

<div class="versionchanged">

next `!WeakValueDictionary` can now be pickled.

</div>

</div>

`WeakValueDictionary` objects have the following additional methods. These have the same issues as the `WeakKeyDictionary.keyrefs` method.

<div class="method">

WeakValueDictionary.itervaluerefs()

Return an `iterator` that yields the weak references to the values.

</div>

<div class="method">

WeakValueDictionary.valuerefs()

Return an iterable of the weak references to the values.

</div>

<div class="WeakSet([elements])">

Set class that keeps weak references to its elements. An element will be discarded when no strong reference to it exists any more.

A `!WeakSet` can be pickled if the elements are strongly referenced elsewhere in the same pickle (see `ref`). A copy or a deep copy shares the elements with the original; user-defined attributes are copied or deep-copied.

<div class="versionchanged">

next `!WeakSet` can now be pickled and copied.

</div>

</div>

<div class="WeakMethod(method[, callback])">

A custom `ref` subclass which simulates a weak reference to a bound method (i.e., a method defined on a class and looked up on an instance). Since a bound method is ephemeral, a standard weak reference cannot keep hold of it. `WeakMethod` has special code to recreate the bound method until either the object or the original function dies:

    >>> class C:
    ...     def method(self):
    ...         print("method called!")
    ...
    >>> c = C()
    >>> r = weakref.ref(c.method)
    >>> r()
    >>> r = weakref.WeakMethod(c.method)
    >>> r()
    <bound method C.method of <__main__.C object at 0x7fc859830220>>
    >>> r()()
    method called!
    >>> del c
    >>> gc.collect()
    0
    >>> r()
    >>>

*callback* is the same as the parameter of the same name to the `ref` function.

A `!WeakMethod` can be copied, and pickled if the object the method is bound to is strongly referenced elsewhere in the same pickle (see `ref`). A copy or a deep copy is an equal weak method to the same object.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

next `!WeakMethod` can now be pickled and copied.

</div>

</div>

<div class="finalize(obj, func, /, *args, **kwargs)">

Return a callable finalizer object which will be called when *obj* is garbage collected. Unlike an ordinary weak reference, a finalizer will always survive until the reference object is collected, greatly simplifying lifecycle management.

A finalizer is considered *alive* until it is called (either explicitly or at garbage collection), and after that it is *dead*. Calling a live finalizer returns the result of evaluating `func(*arg, **kwargs)`, whereas calling a dead finalizer returns `None`.

Exceptions raised by finalizer callbacks during garbage collection will be shown on the standard error output, but cannot be propagated. They are handled in the same way as exceptions raised from an object's `~object.__del__` method or a weak reference's callback.

When the program exits (or more generally, at `interpreter shutdown`), each remaining live finalizer is called unless its `atexit` attribute has been set to false. They are called in reverse order of creation.

A finalizer will never invoke its callback during the later part of the `interpreter shutdown` when module globals are liable to have been replaced by `None`.

<div class="method">

\_\_call\_\_()

If *self* is alive then mark it as dead and return the result of calling `func(*args, **kwargs)`. If *self* is dead then return `None`.

</div>

<div class="method">

detach()

If *self* is alive then mark it as dead and return the tuple `(obj, func, args, kwargs)`. If *self* is dead then return `None`.

</div>

<div class="method">

peek()

If *self* is alive then return the tuple `(obj, func, args, kwargs)`. If *self* is dead then return `None`.

</div>

<div class="attribute">

alive

Property which is true if the finalizer is alive, false otherwise.

</div>

<div class="attribute">

atexit

A writable boolean property which by default is true. At `interpreter shutdown`, all remaining live finalizers for which `.atexit` is true are called in reverse order of creation.

</div>

> [!NOTE]
> It is important to ensure that *func*, *args* and *kwargs* do not own any references to *obj*, either directly or indirectly, since otherwise *obj* will never be garbage collected. In particular, *func* should not be a bound method of *obj*.

<div class="versionadded">

3.4

</div>

</div>

<div class="ReferenceType">

The type object for weak references objects.

</div>

<div class="ProxyType">

The type object for proxies of objects which are not callable.

</div>

<div class="CallableProxyType">

The type object for proxies of callable objects.

</div>

<div class="data">

ProxyTypes

Sequence containing all the type objects for proxies. This can make it simpler to test if an object is a proxy without being dependent on naming both proxy types.

</div>

<div class="seealso">

`205` - Weak References  
The proposal and rationale for this feature, including links to earlier implementations and information about similar features in other languages.

</div>

## Weak Reference Objects

Weak reference objects have no methods and no attributes besides `ref.__callback__`. A weak reference object allows the referent to be obtained, if it still exists, by calling it:

> \>\>\> import weakref \>\>\> class Object: ... pass ... \>\>\> o = Object() \>\>\> r = weakref.ref(o) \>\>\> o2 = r() \>\>\> o is o2 True

If the referent no longer exists, calling the reference object returns `None`:

> \>\>\> del o, o2 \>\>\> print(r()) None

Testing that a weak reference object is still live should be done using the expression `ref() is not None`. Normally, application code that needs to use a reference object should follow this pattern:

    # r is a weak reference object
    o = r()
    if o is None:
        # referent has been garbage collected
        print("Object has been deallocated; can't frobnicate.")
    else:
        print("Object is still live!")
        o.do_something_useful()

Using a separate test for "liveness" creates race conditions in threaded applications; another thread can cause a weak reference to become invalidated before the weak reference is called; the idiom shown above is safe in threaded applications as well as single-threaded applications.

Specialized versions of `ref` objects can be created through subclassing. This is used in the implementation of the `WeakValueDictionary` to reduce the memory overhead for each entry in the mapping. This may be most useful to associate additional information with a reference, but could also be used to insert additional processing on calls to retrieve the referent.

This example shows how a subclass of `ref` can be used to store additional information about an object and affect the value that's returned when the referent is accessed:

    import weakref

    class ExtendedRef(weakref.ref):
        def __init__(self, ob, callback=None, /, **annotations):
            super().__init__(ob, callback)
            self.__counter = 0
            for k, v in annotations.items():
                setattr(self, k, v)

        def __call__(self):
            """Return a pair containing the referent and the number of
            times the reference has been called.
            """
            ob = super().__call__()
            if ob is not None:
                self.__counter += 1
                ob = (ob, self.__counter)
            return ob

## Example

This simple example shows how an application can use object IDs to retrieve objects that it has seen before. The IDs of the objects can then be used in other data structures without forcing the objects to remain alive, but the objects can still be retrieved by ID if they do.

    import weakref

    _id2obj_dict = weakref.WeakValueDictionary()

    def remember(obj):
        oid = id(obj)
        _id2obj_dict[oid] = obj
        return oid

    def id2obj(oid):
        return _id2obj_dict[oid]

## Finalizer Objects

The main benefit of using `finalize` is that it makes it simple to register a callback without needing to preserve the returned finalizer object. For instance

> \>\>\> import weakref \>\>\> class Object: ... pass ... \>\>\> kenny = Object() \>\>\> weakref.finalize(kenny, print, "You killed Kenny!") \#doctest:+ELLIPSIS \<finalize object at ...; for 'Object' at ...\> \>\>\> del kenny You killed Kenny!

The finalizer can be called directly as well. However the finalizer will invoke the callback at most once.

> \>\>\> def callback(x, y, z): ... print("CALLBACK") ... return x + y + z ... \>\>\> obj = Object() \>\>\> f = weakref.finalize(obj, callback, 1, 2, z=3) \>\>\> assert f.alive \>\>\> assert f() == 6 CALLBACK \>\>\> assert not f.alive \>\>\> f() \# callback not called because finalizer dead \>\>\> del obj \# callback not called because finalizer dead

You can unregister a finalizer using its `~finalize.detach` method. This kills the finalizer and returns the arguments passed to the constructor when it was created.

> \>\>\> obj = Object() \>\>\> f = weakref.finalize(obj, callback, 1, 2, z=3) \>\>\> f.detach() \#doctest:+ELLIPSIS (\<...Object object ...\>, \<function callback ...\>, (1, 2), {'z': 3}) \>\>\> newobj, func, args, kwargs = \_ \>\>\> assert not f.alive \>\>\> assert newobj is obj \>\>\> assert func(*args,*\*kwargs) == 6 CALLBACK

Unless you set the `~finalize.atexit` attribute to `False`, a finalizer will be called when the program exits if it is still alive. For instance

<div class="doctest" data-options="+SKIP">

\>\>\> obj = Object() \>\>\> weakref.finalize(obj, print, "obj dead or exiting") \<finalize object at ...; for 'Object' at ...\> \>\>\> exit() obj dead or exiting

</div>

## Comparing finalizers with `~object.__del__` methods

Suppose we want to create a class whose instances represent temporary directories. The directories should be deleted with their contents when the first of the following events occurs:

- the object is garbage collected,
- the object's `!remove` method is called, or
- the program exits.

We might try to implement the class using a `~object.__del__` method as follows:

    class TempDir:
        def __init__(self):
            self.name = tempfile.mkdtemp()

        def remove(self):
            if self.name is not None:
                shutil.rmtree(self.name)
                self.name = None

        @property
        def removed(self):
            return self.name is None

        def __del__(self):
            self.remove()

Starting with Python 3.4, `~object.__del__` methods no longer prevent reference cycles from being garbage collected, and module globals are no longer forced to `None` during `interpreter shutdown`. So this code should work without any issues on CPython.

However, handling of `~object.__del__` methods is notoriously implementation specific, since it depends on internal details of the interpreter's garbage collector implementation.

A more robust alternative can be to define a finalizer which only references the specific functions and objects that it needs, rather than having access to the full state of the object:

    class TempDir:
        def __init__(self):
            self.name = tempfile.mkdtemp()
            self._finalizer = weakref.finalize(self, shutil.rmtree, self.name)

        def remove(self):
            self._finalizer()

        @property
        def removed(self):
            return not self._finalizer.alive

Defined like this, our finalizer only receives a reference to the details it needs to clean up the directory appropriately. If the object never gets garbage collected the finalizer will still be called at exit.

The other advantage of weakref based finalizers is that they can be used to register finalizers for classes where the definition is controlled by a third party, such as running code when a module is unloaded:

    import weakref, sys
    def unloading_module():
        # implicit reference to the module globals from the function body
    weakref.finalize(sys.modules[__name__], unloading_module)

> [!NOTE]
> If you create a finalizer object in a daemonic thread just as the program exits then there is the possibility that the finalizer does not get called at exit. However, in a daemonic thread `atexit.register`, `try: ... finally: ...` and `with: ...` do not guarantee that cleanup occurs either.

# `!webbrowser` --- Convenient web-browser controller

<div class="module" data-synopsis="Easy-to-use controller for web browsers.">

webbrowser

</div>

**Source code:** `Lib/webbrowser.py`

------------------------------------------------------------------------

The `!webbrowser` module provides a high-level interface to allow displaying web-based documents to users. Under most circumstances, simply calling the `.open` function from this module will do the right thing.

Under Unix, graphical browsers are preferred under X11, but text-mode browsers will be used if graphical browsers are not available or an X11 display isn't available. If text-mode browsers are used, the calling process will block until the user exits the browser.

If the environment variable `BROWSER` exists, it is interpreted as the `os.pathsep`-separated list of browsers to try ahead of the platform defaults. When the value of a list part contains the string `%s`, then it is interpreted as a literal browser command line to be used with the argument URL substituted for `%s`; if the value is a single word that refers to one of the already registered browsers this browser is added to the front of the search list; if the part does not contain `%s`, it is simply interpreted as the name of the browser to launch.[^1]

<div class="versionchanged">

3.14

The `BROWSER` variable can now also be used to reorder the list of platform defaults. This is particularly useful on macOS where the platform defaults do not refer to command-line tools on `PATH`.

</div>

For non-Unix platforms, or when a remote browser is available on Unix, the controlling process will not wait for the user to finish with the browser, but allow the remote browser to maintain its own windows on the display. If remote browsers are not available on Unix, the controlling process will launch a new browser and wait.

On iOS, the `BROWSER` environment variable, as well as any arguments controlling autoraise, browser preference, and new tab/window creation will be ignored. Web pages will *always* be opened in the user's preferred browser, in a new tab, with the browser being brought to the foreground. The use of the `!webbrowser` module on iOS requires the `ctypes` module. If `ctypes` isn't available, calls to `.open` will fail.

## Command-line interface

<div class="program">

webbrowser

</div>

The script `webbrowser` can be used as a command-line interface for the module. It accepts a URL as the argument. It accepts the following optional parameters:

<div class="option">

`-n, --new-window`  

Opens the URL in a new browser window, if possible.

</div>

<div class="option">

`-t, --new-tab`  

Opens the URL in a new browser tab.

</div>

The options are, naturally, mutually exclusive. Usage example:

``` bash
python -m webbrowser -t "https://www.python.org"
```

<div class="availability">

not WASI, not Android.

</div>

The following exception is defined:

<div class="exception">

Error

Exception raised when a browser control error occurs.

</div>

The following functions are defined:

<div class="function">

open(url, new=0, autoraise=True)

Display *url* using the default browser. If *new* is 0, the *url* is opened in the same browser window if possible. If *new* is 1, a new browser window is opened if possible. If *new* is 2, a new browser page ("tab") is opened if possible. If *autoraise* is `True`, the window is raised if possible (note that under many window managers this will occur regardless of the setting of this variable).

Returns `True` if a browser was successfully launched, `False` otherwise.

Note that on some platforms, trying to open a filename using this function, may work and start the operating system's associated program. However, this is neither supported nor portable.

<div class="audit-event">

webbrowser.open url webbrowser.open

</div>

</div>

<div class="function">

open_new(url)

Open *url* in a new window of the default browser, if possible, otherwise, open *url* in the only browser window.

Returns `True` if a browser was successfully launched, `False` otherwise.

</div>

<div class="function">

open_new_tab(url)

Open *url* in a new page ("tab") of the default browser, if possible, otherwise equivalent to `open_new`.

Returns `True` if a browser was successfully launched, `False` otherwise.

</div>

<div class="function">

get(using=None)

Return a controller object for the browser type *using*. If *using* is `None`, return a controller for a default browser appropriate to the caller's environment.

</div>

<div class="function">

register(name, constructor, instance=None, \*, preferred=False)

Register the browser type *name*. Once a browser type is registered, the `get` function can return a controller for that browser type. If *instance* is not provided, or is `None`, *constructor* will be called without parameters to create an instance when needed. If *instance* is provided, *constructor* will never be called, and may be `None`.

Setting *preferred* to `True` makes this browser a preferred result for a `get` call with no argument. Otherwise, this entry point is only useful if you plan to either set the `BROWSER` variable or call `get` with a nonempty argument matching the name of a handler you declare.

<div class="versionchanged">

3.7 *preferred* keyword-only parameter was added.

</div>

</div>

A number of browser types are predefined. This table gives the type names that may be passed to the `get` function and the corresponding instantiations for the controller classes, all defined in this module.

| Type Name            | Class Name                     | Notes |
|----------------------|--------------------------------|-------|
| `'mozilla'`          | `Mozilla('mozilla')`           |       |
| `'firefox'`          | `Mozilla('mozilla')`           |       |
| `'epiphany'`         | `Epiphany('epiphany')`         |       |
| `'kfmclient'`        | `Konqueror()`                  | \(1\) |
| `'konqueror'`        | `Konqueror()`                  | \(1\) |
| `'kfm'`              | `Konqueror()`                  | \(1\) |
| `'opera'`            | `Opera()`                      |       |
| `'links'`            | `GenericBrowser('links')`      |       |
| `'elinks'`           | `Elinks('elinks')`             |       |
| `'lynx'`             | `GenericBrowser('lynx')`       |       |
| `'w3m'`              | `GenericBrowser('w3m')`        |       |
| `'windows-default'`  | `WindowsDefault`               | \(2\) |
| `'macos'`            | `MacOS('default')`             | \(3\) |
| `'safari'`           | `MacOS('safari')`              | \(3\) |
| `'chrome'`           | `MacOS('google chrome')`       | \(3\) |
| `'firefox'`          | `MacOS('firefox')`             | \(3\) |
| `'google-chrome'`    | `Chrome('google-chrome')`      |       |
| `'chromium'`         | `Chromium('chromium')`         |       |
| `'chromium-browser'` | `Chromium('chromium-browser')` |       |
| `'iosbrowser'`       | `IOSBrowser`                   | \(4\) |

Notes:

\(1\)  
"Konqueror" is the file manager for the KDE desktop environment for Unix, and only makes sense to use if KDE is running. Some way of reliably detecting KDE would be nice; the `!KDEDIR` variable is not sufficient. Note also that the name "kfm" is used even when using the `konqueror` command with KDE 2 --- the implementation selects the best strategy for running Konqueror.

\(2\)  
Only on Windows platforms.

\(3\)  
Only on macOS.

\(4\)  
Only on iOS.

<div class="versionadded">

3.2 A new `!MacOSXOSAScript` class has been added and is used on Mac instead of the previous `!MacOSX` class. This adds support for opening browsers not currently set as the OS default.

</div>

<div class="versionadded">

3.3 Support for Chrome/Chromium has been added.

</div>

<div class="versionchanged">

3.12 Support for several obsolete browsers has been removed. Removed browsers include Grail, Mosaic, Netscape, Galeon, Skipstone, Iceape, and Firefox versions 35 and below.

</div>

<div class="versionchanged">

3.13 Support for iOS has been added.

</div>

<div class="versionadded">

3.15 `!MacOS` has been added as a replacement for `!MacOSXOSAScript`, opening browsers via `/usr/bin/open` instead of `osascript`.

</div>

<div class="deprecated-removed">

3.15 3.17 `!MacOSXOSAScript` is deprecated in favour of `!MacOS`. Using `/usr/bin/open` instead of `osascript` is a security and usability improvement: `osascript` may be blocked on managed systems due to its abuse potential as a general-purpose scripting interpreter.

</div>

Here are some simple examples:

    url = 'https://docs.python.org/'

    # Open URL in a new tab, if a browser window is already open.
    webbrowser.open_new_tab(url)

    # Open URL in new window, raising the window if possible.
    webbrowser.open_new(url)

## Browser controller objects

Browser controllers provide the `~controller.name` attribute, and the following three methods which parallel module-level convenience functions:

<div class="attribute">

controller.name

System-dependent name for the browser.

</div>

<div class="method">

controller.open(url, new=0, autoraise=True)

Display *url* using the browser handled by this controller. If *new* is 1, a new browser window is opened if possible. If *new* is 2, a new browser page ("tab") is opened if possible.

</div>

<div class="method">

controller.open_new(url)

Open *url* in a new window of the browser handled by this controller, if possible, otherwise, open *url* in the only browser window. Alias `open_new`.

</div>

<div class="method">

controller.open_new_tab(url)

Open *url* in a new page ("tab") of the browser handled by this controller, if possible, otherwise equivalent to `open_new`.

</div>

**Footnotes**

[^1]: Executables named here without a full path will be searched in the directories given in the `PATH` environment variable.

# MS Windows Specific Services

This chapter describes modules that are only available on MS Windows platforms.

<div class="toctree">

msvcrt.rst winreg.rst winsound.rst

</div>

# `!winreg` --- Windows registry access

<div class="module" data-synopsis="Routines and objects for manipulating the Windows registry.">

winreg

</div>

**Source code:** `PC/winreg.c`

------------------------------------------------------------------------

These functions expose the Windows registry API to Python. Instead of using an integer as the registry handle, a `handle object <handle-object>` is used to ensure that the handles are closed correctly, even if the programmer neglects to explicitly close them.

<div class="availability">

Windows.

</div>

<div id="exception-changed">

<div class="versionchanged">

3.3 Several functions in this module used to raise a `WindowsError`, which is now an alias of `OSError`.

</div>

</div>

## Functions

This module offers the following functions:

<div class="function">

CloseKey(hkey)

Closes a previously opened registry key. The *hkey* argument specifies a previously opened key.

> [!NOTE]
> If *hkey* is not closed using this method (or via `hkey.Close()
> <PyHKEY.Close>`), it is closed when the *hkey* object is destroyed by Python.

</div>

<div class="function">

ConnectRegistry(computer_name, key)

Establishes a connection to a predefined registry handle on another computer, and returns a `handle object <handle-object>`.

*computer_name* is the name of the remote computer, of the form `r"\\computername"`. If `None`, the local computer is used.

*key* is the predefined handle to connect to.

The return value is the handle of the opened key. If the function fails, an `OSError` exception is raised.

<div class="audit-event">

winreg.ConnectRegistry computer_name,key winreg.ConnectRegistry

</div>

<div class="versionchanged">

3.3 See `above <exception-changed>`.

</div>

</div>

<div class="function">

CreateKey(key, sub_key)

Creates or opens the specified key, returning a `handle object <handle-object>`.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*sub_key* is a string that names the key this method opens or creates.

If *key* is one of the predefined keys, *sub_key* may be `None`. In that case, the handle returned is the same key handle passed in to the function.

If the key already exists, this function opens the existing key.

The return value is the handle of the opened key. If the function fails, an `OSError` exception is raised.

<div class="audit-event">

winreg.CreateKey key,sub_key,access winreg.CreateKey

</div>

<div class="audit-event">

winreg.OpenKey/result key winreg.CreateKey

</div>

<div class="versionchanged">

3.3 See `above <exception-changed>`.

</div>

</div>

<div class="function">

CreateKeyEx(key, sub_key, reserved=0, access=KEY_WRITE)

Creates or opens the specified key, returning a `handle object <handle-object>`.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*sub_key* is a string that names the key this method opens or creates.

*reserved* is a reserved integer, and must be zero. The default is zero.

*access* is an integer that specifies an access mask that describes the desired security access for the key. Default is `KEY_WRITE`. See `Access Rights <access-rights>` for other allowed values.

If *key* is one of the predefined keys, *sub_key* may be `None`. In that case, the handle returned is the same key handle passed in to the function.

If the key already exists, this function opens the existing key.

The return value is the handle of the opened key. If the function fails, an `OSError` exception is raised.

<div class="audit-event">

winreg.CreateKey key,sub_key,access winreg.CreateKeyEx

</div>

<div class="audit-event">

winreg.OpenKey/result key winreg.CreateKeyEx

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 See `above <exception-changed>`.

</div>

</div>

<div class="function">

DeleteKey(key, sub_key)

Deletes the specified key.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*sub_key* is a string that must be a subkey of the key identified by the *key* parameter. This value must not be `None`, and the key may not have subkeys.

*This method can not delete keys with subkeys.*

If the method succeeds, the entire key, including all of its values, is removed. If the method fails, an `OSError` exception is raised.

<div class="audit-event">

winreg.DeleteKey key,sub_key,access winreg.DeleteKey

</div>

<div class="versionchanged">

3.3 See `above <exception-changed>`.

</div>

</div>

<div class="function">

DeleteKeyEx(key, sub_key, access=KEY_WOW64_64KEY, reserved=0)

Deletes the specified key.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*sub_key* is a string that must be a subkey of the key identified by the *key* parameter. This value must not be `None`, and the key may not have subkeys.

*reserved* is a reserved integer, and must be zero. The default is zero.

*access* is an integer that specifies an access mask that describes the desired security access for the key. Default is `KEY_WOW64_64KEY`. On 32-bit Windows, the WOW64 constants are ignored. See `Access Rights <access-rights>` for other allowed values.

*This method can not delete keys with subkeys.*

If the method succeeds, the entire key, including all of its values, is removed. If the method fails, an `OSError` exception is raised.

On unsupported Windows versions, `NotImplementedError` is raised.

<div class="audit-event">

winreg.DeleteKey key,sub_key,access winreg.DeleteKeyEx

</div>

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.3 See `above <exception-changed>`.

</div>

</div>

<div class="function">

DeleteTree(key, sub_key=None)

Deletes the specified key and all its subkeys and values recursively.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*sub_key* is a string that names the subkey to delete. If `None`, deletes all subkeys and values of the specified key.

This function deletes a key and all its descendants. If *sub_key* is `None`, all subkeys and values of the specified key are deleted.

<div class="audit-event">

winreg.DeleteTree key,sub_key winreg.DeleteTree

</div>

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

DeleteValue(key, value)

Removes a named value from a registry key.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*value* is a string that identifies the value to remove.

<div class="audit-event">

winreg.DeleteValue key,value winreg.DeleteValue

</div>

</div>

<div class="function">

EnumKey(key, index)

Enumerates subkeys of an open registry key, returning a string.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*index* is an integer that identifies the index of the key to retrieve.

The function retrieves the name of one subkey each time it is called. It is typically called repeatedly until an `OSError` exception is raised, indicating, no more values are available.

<div class="audit-event">

winreg.EnumKey key,index winreg.EnumKey

</div>

<div class="versionchanged">

3.3 See `above <exception-changed>`.

</div>

</div>

<div class="function">

EnumValue(key, index)

Enumerates values of an open registry key, returning a tuple.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*index* is an integer that identifies the index of the value to retrieve.

The function retrieves the name of one subkey each time it is called. It is typically called repeatedly, until an `OSError` exception is raised, indicating no more values.

The result is a tuple of 3 items:

| Index | Meaning |
|----|----|
| `0` | A string that identifies the value name |
| `1` | An object that holds the value data, and whose type depends on the underlying registry type |
| `2` | An integer that identifies the type of the value data (see table in docs for `SetValueEx`) |

<div class="audit-event">

winreg.EnumValue key,index winreg.EnumValue

</div>

<div class="versionchanged">

3.3 See `above <exception-changed>`.

</div>

</div>

<div class="index">

single: % (percent); environment variables expansion (Windows)

</div>

<div class="function">

ExpandEnvironmentStrings(str)

Expands environment variable placeholders `%NAME%` in strings like `REG_EXPAND_SZ`:

    >>> ExpandEnvironmentStrings('%windir%')
    'C:\\Windows'

<div class="audit-event">

winreg.ExpandEnvironmentStrings str winreg.ExpandEnvironmentStrings

</div>

</div>

<div class="function">

FlushKey(key)

Writes all the attributes of a key to the registry.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

It is not necessary to call `FlushKey` to change a key. Registry changes are flushed to disk by the registry using its lazy flusher. Registry changes are also flushed to disk at system shutdown. Unlike `CloseKey`, the `FlushKey` method returns only when all the data has been written to the registry. An application should only call `FlushKey` if it requires absolute certainty that registry changes are on disk.

> [!NOTE]
> If you don't know whether a `FlushKey` call is required, it probably isn't.

</div>

<div class="function">

LoadKey(key, sub_key, file_name)

Creates a subkey under the specified key and stores registration information from a specified file into that subkey.

*key* is a handle returned by `ConnectRegistry` or one of the constants `HKEY_USERS` or `HKEY_LOCAL_MACHINE`.

*sub_key* is a string that identifies the subkey to load.

*file_name* is the name of the file to load registry data from. This file must have been created with the `SaveKey` function. Under the file allocation table (FAT) file system, the filename may not have an extension.

A call to `LoadKey` fails if the calling process does not have the `!SE_RESTORE_PRIVILEGE` privilege. Note that privileges are different from permissions -- see the [RegLoadKey documentation](https://msdn.microsoft.com/en-us/library/ms724889%28v=VS.85%29.aspx) for more details.

If *key* is a handle returned by `ConnectRegistry`, then the path specified in *file_name* is relative to the remote computer.

<div class="audit-event">

winreg.LoadKey key,sub_key,file_name winreg.LoadKey

</div>

</div>

<div class="function">

OpenKey(key, sub_key, reserved=0, access=KEY_READ) OpenKeyEx(key, sub_key, reserved=0, access=KEY_READ)

Opens the specified key, returning a `handle object <handle-object>`.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*sub_key* is a string that identifies the sub_key to open.

*reserved* is a reserved integer, and must be zero. The default is zero.

*access* is an integer that specifies an access mask that describes the desired security access for the key. Default is `KEY_READ`. See `Access
Rights <access-rights>` for other allowed values.

The result is a new handle to the specified key.

If the function fails, `OSError` is raised.

<div class="audit-event">

winreg.OpenKey key,sub_key,access winreg.OpenKey

</div>

<div class="audit-event">

winreg.OpenKey/result key winreg.OpenKey

</div>

<div class="versionchanged">

3.2 Allow the use of named arguments.

</div>

<div class="versionchanged">

3.3 See `above <exception-changed>`.

</div>

</div>

<div class="function">

QueryInfoKey(key)

Returns information about a key, as a tuple.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

The result is a tuple of 3 items:

| Index | Meaning |
|----|----|
| `0` | An integer giving the number of sub keys this key has. |
| `1` | An integer giving the number of values this key has. |
| `2` | An integer giving when the key was last modified (if available) as 100's of nanoseconds since Jan 1, 1601. |

<div class="audit-event">

winreg.QueryInfoKey key winreg.QueryInfoKey

</div>

</div>

<div class="function">

QueryValue(key, sub_key)

Retrieves the unnamed value for a key, as a string.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*sub_key* is a string that holds the name of the subkey with which the value is associated. If this parameter is `None` or empty, the function retrieves the value set by the `SetValue` method for the key identified by *key*.

Values in the registry have name, type, and data components. This method retrieves the data for a key's first value that has a `NULL` name. But the underlying API call doesn't return the type, so always use `QueryValueEx` if possible.

<div class="audit-event">

winreg.QueryValue key,sub_key,value_name winreg.QueryValue

</div>

</div>

<div class="function">

QueryValueEx(key, value_name)

Retrieves the type and data for a specified value name associated with an open registry key.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*value_name* is a string indicating the value to query.

The result is a tuple of 2 items:

| Index | Meaning |
|----|----|
| `0` | The value of the registry item. |
| `1` | An integer giving the registry type for this value (see table in docs for `SetValueEx`) |

<div class="audit-event">

winreg.QueryValue key,sub_key,value_name winreg.QueryValueEx

</div>

</div>

<div class="function">

SaveKey(key, file_name)

Saves the specified key, and all its subkeys to the specified file.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*file_name* is the name of the file to save registry data to. This file cannot already exist. If this filename includes an extension, it cannot be used on file allocation table (FAT) file systems by the `LoadKey` method.

If *key* represents a key on a remote computer, the path described by *file_name* is relative to the remote computer. The caller of this method must possess the **SeBackupPrivilege** security privilege. Note that privileges are different than permissions -- see the [Conflicts Between User Rights and Permissions documentation](https://msdn.microsoft.com/en-us/library/ms724878%28v=VS.85%29.aspx) for more details.

This function passes `NULL` for *security_attributes* to the API.

<div class="audit-event">

winreg.SaveKey key,file_name winreg.SaveKey

</div>

</div>

<div class="function">

SetValue(key, sub_key, type, value)

Associates a value with a specified key.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*sub_key* is a string that names the subkey with which the value is associated.

*type* is an integer that specifies the type of the data. Currently this must be `REG_SZ`, meaning only strings are supported. Use the `SetValueEx` function for support for other data types.

*value* is a string that specifies the new value.

If the key specified by the *sub_key* parameter does not exist, the SetValue function creates it.

Value lengths are limited by available memory. Long values (more than 2048 bytes) should be stored as files with the filenames stored in the configuration registry. This helps the registry perform efficiently.

The key identified by the *key* parameter must have been opened with `KEY_SET_VALUE` access.

<div class="audit-event">

winreg.SetValue key,sub_key,type,value winreg.SetValue

</div>

</div>

<div class="function">

SetValueEx(key, value_name, reserved, type, value)

Stores data in the value field of an open registry key.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

*value_name* is a string that names the subkey with which the value is associated.

*reserved* can be anything -- zero is always passed to the API.

*type* is an integer that specifies the type of the data. See `Value Types <value-types>` for the available types.

*value* is a string that specifies the new value.

This method can also set additional value and type information for the specified key. The key identified by the key parameter must have been opened with `KEY_SET_VALUE` access.

To open the key, use the `CreateKey` or `OpenKey` methods.

Value lengths are limited by available memory. Long values (more than 2048 bytes) should be stored as files with the filenames stored in the configuration registry. This helps the registry perform efficiently.

<div class="audit-event">

winreg.SetValue key,sub_key,type,value winreg.SetValueEx

</div>

</div>

<div class="function">

DisableReflectionKey(key)

Disables registry reflection for 32-bit processes running on a 64-bit operating system.

*key* is an already open key, or one of the predefined `HKEY_* constants
<hkey-constants>`.

Will generally raise `NotImplementedError` if executed on a 32-bit operating system.

If the key is not on the reflection list, the function succeeds but has no effect. Disabling reflection for a key does not affect reflection of any subkeys.

<div class="audit-event">

winreg.DisableReflectionKey key winreg.DisableReflectionKey

</div>

</div>

<div class="function">

EnableReflectionKey(key)

Restores registry reflection for the specified disabled key.

*key* is an already open key, or one of the predefined `HKEY_* constants
<hkey-constants>`.

Will generally raise `NotImplementedError` if executed on a 32-bit operating system.

Restoring reflection for a key does not affect reflection of any subkeys.

<div class="audit-event">

winreg.EnableReflectionKey key winreg.EnableReflectionKey

</div>

</div>

<div class="function">

QueryReflectionKey(key)

Determines the reflection state for the specified key.

*key* is an already open key, or one of the predefined `HKEY_* constants <hkey-constants>`.

Returns `True` if reflection is disabled.

Will generally raise `NotImplementedError` if executed on a 32-bit operating system.

<div class="audit-event">

winreg.QueryReflectionKey key winreg.QueryReflectionKey

</div>

</div>

## Constants

The following constants are defined for use in many `!winreg` functions.

### [HKEY]()\* Constants

<div class="data">

HKEY_CLASSES_ROOT

Registry entries subordinate to this key define types (or classes) of documents and the properties associated with those types. Shell and COM applications use the information stored under this key.

</div>

<div class="data">

HKEY_CURRENT_USER

Registry entries subordinate to this key define the preferences of the current user. These preferences include the settings of environment variables, data about program groups, colors, printers, network connections, and application preferences.

</div>

<div class="data">

HKEY_LOCAL_MACHINE

Registry entries subordinate to this key define the physical state of the computer, including data about the bus type, system memory, and installed hardware and software.

</div>

<div class="data">

HKEY_USERS

Registry entries subordinate to this key define the default user configuration for new users on the local computer and the user configuration for the current user.

</div>

<div class="data">

HKEY_PERFORMANCE_DATA

Registry entries subordinate to this key allow you to access performance data. The data is not actually stored in the registry; the registry functions cause the system to collect the data from its source.

</div>

<div class="data">

HKEY_CURRENT_CONFIG

Contains information about the current hardware profile of the local computer system.

</div>

<div class="data">

HKEY_DYN_DATA

This key is not used in versions of Windows after 98.

</div>

### Access Rights

For more information, see [Registry Key Security and Access](https://msdn.microsoft.com/en-us/library/ms724878%28v=VS.85%29.aspx).

<div class="data">

KEY_ALL_ACCESS

Combines the STANDARD_RIGHTS_REQUIRED, `KEY_QUERY_VALUE`, `KEY_SET_VALUE`, `KEY_CREATE_SUB_KEY`, `KEY_ENUMERATE_SUB_KEYS`, `KEY_NOTIFY`, and `KEY_CREATE_LINK` access rights.

</div>

<div class="data">

KEY_WRITE

Combines the STANDARD_RIGHTS_WRITE, `KEY_SET_VALUE`, and `KEY_CREATE_SUB_KEY` access rights.

</div>

<div class="data">

KEY_READ

Combines the STANDARD_RIGHTS_READ, `KEY_QUERY_VALUE`, `KEY_ENUMERATE_SUB_KEYS`, and `KEY_NOTIFY` values.

</div>

<div class="data">

KEY_EXECUTE

Equivalent to `KEY_READ`.

</div>

<div class="data">

KEY_QUERY_VALUE

Required to query the values of a registry key.

</div>

<div class="data">

KEY_SET_VALUE

Required to create, delete, or set a registry value.

</div>

<div class="data">

KEY_CREATE_SUB_KEY

Required to create a subkey of a registry key.

</div>

<div class="data">

KEY_ENUMERATE_SUB_KEYS

Required to enumerate the subkeys of a registry key.

</div>

<div class="data">

KEY_NOTIFY

Required to request change notifications for a registry key or for subkeys of a registry key.

</div>

<div class="data">

KEY_CREATE_LINK

Reserved for system use.

</div>

#### 64-bit Specific

For more information, see [Accessing an Alternate Registry View](https://msdn.microsoft.com/en-us/library/aa384129(v=VS.85).aspx).

<div class="data">

KEY_WOW64_64KEY

Indicates that an application on 64-bit Windows should operate on the 64-bit registry view. On 32-bit Windows, this constant is ignored.

</div>

<div class="data">

KEY_WOW64_32KEY

Indicates that an application on 64-bit Windows should operate on the 32-bit registry view. On 32-bit Windows, this constant is ignored.

</div>

### Value Types

For more information, see [Registry Value Types](https://msdn.microsoft.com/en-us/library/ms724884%28v=VS.85%29.aspx).

<div class="data">

REG_BINARY

Binary data in any form.

</div>

<div class="data">

REG_DWORD

32-bit number.

</div>

<div class="data">

REG_DWORD_LITTLE_ENDIAN

A 32-bit number in little-endian format. Equivalent to `REG_DWORD`.

</div>

<div class="data">

REG_DWORD_BIG_ENDIAN

A 32-bit number in big-endian format.

</div>

<div class="data">

REG_EXPAND_SZ

Null-terminated string containing references to environment variables (`%PATH%`).

</div>

<div class="data">

REG_LINK

A Unicode symbolic link.

</div>

<div class="data">

REG_MULTI_SZ

A sequence of null-terminated strings, terminated by two null characters. (Python handles this termination automatically.)

</div>

<div class="data">

REG_NONE

No defined value type.

</div>

<div class="data">

REG_QWORD

A 64-bit number.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

REG_QWORD_LITTLE_ENDIAN

A 64-bit number in little-endian format. Equivalent to `REG_QWORD`.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

REG_RESOURCE_LIST

A device-driver resource list.

</div>

<div class="data">

REG_FULL_RESOURCE_DESCRIPTOR

A hardware setting.

</div>

<div class="data">

REG_RESOURCE_REQUIREMENTS_LIST

A hardware resource list.

</div>

<div class="data">

REG_SZ

A null-terminated string.

</div>

## Registry Handle Objects

This object wraps a Windows HKEY object, automatically closing it when the object is destroyed. To guarantee cleanup, you can call either the `~PyHKEY.Close` method on the object, or the `CloseKey` function.

All registry functions in this module return one of these objects.

All registry functions in this module which accept a handle object also accept an integer, however, use of the handle object is encouraged.

Handle objects provide semantics for `~object.__bool__` -- thus :

    if handle:
        print("Yes")

will print `Yes` if the handle is currently valid (has not been closed or detached).

The object also support equality comparison semantics, so handle objects will compare equal if they both reference the same underlying Windows handle value. Closed handle objects (those with a handle value of zero) always compare equal.

Handle objects can be converted to an integer (e.g., using the built-in `int` function), in which case the underlying Windows handle value is returned. You can also use the `~PyHKEY.Detach` method to return the integer handle, and also disconnect the Windows handle from the handle object.

<div class="method">

PyHKEY.Close()

Closes the underlying Windows handle.

If the handle is already closed, no error is raised.

</div>

<div class="method">

PyHKEY.Detach()

Detaches the Windows handle from the handle object.

The result is an integer that holds the value of the handle before it is detached. If the handle is already detached or closed, this will return zero.

After calling this function, the handle is effectively invalidated, but the handle is not closed. You would call this function when you need the underlying Win32 handle to exist beyond the lifetime of the handle object.

<div class="audit-event">

winreg.PyHKEY.Detach key winreg.PyHKEY.Detach

</div>

</div>

<div class="method">

PyHKEY.\_\_enter\_\_() PyHKEY.\_\_exit\_\_(\*exc_info)

The HKEY object implements `~object.__enter__` and `~object.__exit__` and thus supports the context protocol for the `with` statement:

    with OpenKey(HKEY_LOCAL_MACHINE, "foo") as key:
        ...  # work with key

will automatically close *key* when control leaves the `with` block.

</div>

<div class="versionchanged">

3.15 Handle objects are now compared by their underlying Windows handle value instead of object identity for equality comparisons.

</div>

# `!winsound` --- Sound-playing interface for Windows

<div class="module" data-synopsis="Access to the sound-playing machinery for Windows.">

winsound

</div>

**Source code:** `PC/winsound.c`

------------------------------------------------------------------------

The `!winsound` module provides access to the basic sound-playing machinery provided by Windows platforms. It includes functions and several constants.

<div class="availability">

Windows.

</div>

## Functions

<div class="function">

Beep(frequency, duration)

Beep the PC's speaker. The *frequency* parameter specifies frequency, in hertz, of the sound, and must be in the range 37 through 32,767. The *duration* parameter specifies the number of milliseconds the sound should last. If the system is not able to beep the speaker, `RuntimeError` is raised.

</div>

<div class="function">

PlaySound(sound, flags)

Call the underlying `!PlaySound` function from the Platform API. The *sound* parameter may be a filename, a system sound alias, audio data as a `bytes-like object`, or `None`. Its interpretation depends on the value of *flags*, which can be a bitwise ORed combination of the constants described below. If the *sound* parameter is `None`, any currently playing waveform sound is stopped. If the system indicates an error, `RuntimeError` is raised.

</div>

<div class="function">

MessageBeep(type=MB_OK)

Call the underlying `!MessageBeep` function from the Platform API. This plays a sound as specified in the registry. The *type* argument specifies which sound to play; possible values are `-1`, `MB_ICONASTERISK`, `MB_ICONEXCLAMATION`, `MB_ICONHAND`, `MB_ICONQUESTION`, and `MB_OK`, all described below. The value `-1` produces a "simple beep"; this is the final fallback if a sound cannot be played otherwise. If the system indicates an error, `RuntimeError` is raised.

</div>

## Constants

<div class="data">

SND_FILENAME

The *sound* parameter is the name of a WAV file. Do not use with `SND_ALIAS`.

</div>

<div class="data">

SND_ALIAS

The *sound* parameter is a sound association name from the registry. If the registry contains no such name, play the system default sound unless `SND_NODEFAULT` is also specified. If no default sound is registered, raise `RuntimeError`. Do not use with `SND_FILENAME`.

All Win32 systems support at least the following; most systems support many more:

| `PlaySound` *name*    | Corresponding Control Panel Sound name |
|-----------------------|----------------------------------------|
| `'SystemAsterisk'`    | Asterisk                               |
| `'SystemExclamation'` | Exclamation                            |
| `'SystemExit'`        | Exit Windows                           |
| `'SystemHand'`        | Critical Stop                          |
| `'SystemQuestion'`    | Question                               |

For example:

    import winsound
    # Play Windows exit sound.
    winsound.PlaySound("SystemExit", winsound.SND_ALIAS)

    # Probably play Windows default sound, if any is registered (because
    # "*" probably isn't the registered name of any sound).
    winsound.PlaySound("*", winsound.SND_ALIAS)

</div>

<div class="data">

SND_LOOP

Play the sound repeatedly. The `SND_ASYNC` flag must also be used to avoid blocking. Cannot be used with `SND_MEMORY`.

</div>

<div class="data">

SND_MEMORY

The *sound* parameter to `PlaySound` is a memory image of a WAV file, as a `bytes-like object`.

> [!NOTE]
> This module does not support playing from a memory image asynchronously, so a combination of this flag and `SND_ASYNC` will raise `RuntimeError`.

</div>

<div class="data">

SND_PURGE

Stop playing all instances of the specified sound.

> [!NOTE]
> This flag is not supported on modern Windows platforms.

</div>

<div class="data">

SND_ASYNC

Return immediately, allowing sounds to play asynchronously.

</div>

<div class="data">

SND_NODEFAULT

If the specified sound cannot be found, do not play the system default sound.

</div>

<div class="data">

SND_NOSTOP

Do not interrupt sounds currently playing.

</div>

<div class="data">

SND_NOWAIT

Return immediately if the sound driver is busy.

> [!NOTE]
> This flag is not supported on modern Windows platforms.

</div>

<div class="data">

SND_APPLICATION

The *sound* parameter is an application-specific alias in the registry. This flag can be combined with the `SND_ALIAS` flag to specify an application-defined sound alias.

</div>

<div class="data">

SND_SENTRY

Triggers a SoundSentry event when the sound is played.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

SND_SYNC

The sound is played synchronously. This is the default behavior.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

SND_SYSTEM

Assign the sound to the audio session for system notification sounds.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

MB_ICONASTERISK

Play the `SystemDefault` sound.

</div>

<div class="data">

MB_ICONEXCLAMATION

Play the `SystemExclamation` sound.

</div>

<div class="data">

MB_ICONHAND

Play the `SystemHand` sound.

</div>

<div class="data">

MB_ICONQUESTION

Play the `SystemQuestion` sound.

</div>

<div class="data">

MB_OK

Play the `SystemDefault` sound.

</div>

<div class="data">

MB_ICONERROR

Play the `SystemHand` sound.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

MB_ICONINFORMATION

Play the `SystemDefault` sound.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

MB_ICONSTOP

Play the `SystemHand` sound.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

MB_ICONWARNING

Play the `SystemExclamation` sound.

<div class="versionadded">

3.14

</div>

</div>

# `!wsgiref` --- WSGI Utilities and Reference Implementation

<div class="module" data-synopsis="WSGI Utilities and Reference Implementation.">

wsgiref

</div>

**Source code:** `Lib/wsgiref`

------------------------------------------------------------------------

> [!WARNING]
> `!wsgiref` is a reference implementation and is not recommended for production. The module only implements basic security checks.

The Web Server Gateway Interface (WSGI) is a standard interface between web server software and web applications written in Python. Having a standard interface makes it easy to use an application that supports WSGI with a number of different web servers.

Only authors of web servers and programming frameworks need to know every detail and corner case of the WSGI design. You don't need to understand every detail of WSGI just to install a WSGI application or to write a web application using an existing framework.

`!wsgiref` is a reference implementation of the WSGI specification that can be used to add WSGI support to a web server or framework. It provides utilities for manipulating WSGI environment variables and response headers, base classes for implementing WSGI servers, a demo HTTP server that serves WSGI applications, types for static type checking, and a validation tool that checks WSGI servers and applications for conformance to the WSGI specification (`3333`).

See [wsgi.readthedocs.io](https://wsgi.readthedocs.io/) for more information about WSGI, and links to tutorials and other resources.

## `!wsgiref.util` -- WSGI environment utilities

<div class="module" data-synopsis="WSGI environment utilities.">

wsgiref.util

</div>

This module provides a variety of utility functions for working with WSGI environments. A WSGI environment is a dictionary containing HTTP request variables as described in `3333`. All of the functions taking an *environ* parameter expect a WSGI-compliant dictionary to be supplied; please see `3333` for a detailed specification and `~wsgiref.types.WSGIEnvironment` for a type alias that can be used in type annotations.

<div class="function">

guess_scheme(environ)

Return a guess for whether `wsgi.url_scheme` should be "http" or "https", by checking for a `HTTPS` environment variable in the *environ* dictionary. The return value is a string.

This function is useful when creating a gateway that wraps CGI or a CGI-like protocol such as FastCGI. Typically, servers providing such protocols will include a `HTTPS` variable with a value of "1", "yes", or "on" when a request is received via SSL. So, this function returns "https" if such a value is found, and "http" otherwise.

</div>

<div class="function">

request_uri(environ, include_query=True)

Return the full request URI, optionally including the query string, using the algorithm found in the "URL Reconstruction" section of `3333`. If *include_query* is false, the query string is not included in the resulting URI.

</div>

<div class="function">

application_uri(environ)

Similar to `request_uri`, except that the `PATH_INFO` and `QUERY_STRING` variables are ignored. The result is the base URI of the application object addressed by the request.

</div>

<div class="function">

shift_path_info(environ)

Shift a single name from `PATH_INFO` to `SCRIPT_NAME` and return the name. The *environ* dictionary is *modified* in-place; use a copy if you need to keep the original `PATH_INFO` or `SCRIPT_NAME` intact.

If there are no remaining path segments in `PATH_INFO`, `None` is returned.

Typically, this routine is used to process each portion of a request URI path, for example to treat the path as a series of dictionary keys. This routine modifies the passed-in environment to make it suitable for invoking another WSGI application that is located at the target URI. For example, if there is a WSGI application at `/foo`, and the request URI path is `/foo/bar/baz`, and the WSGI application at `/foo` calls `shift_path_info`, it will receive the string "bar", and the environment will be updated to be suitable for passing to a WSGI application at `/foo/bar`. That is, `SCRIPT_NAME` will change from `/foo` to `/foo/bar`, and `PATH_INFO` will change from `/bar/baz` to `/baz`.

When `PATH_INFO` is just a "/", this routine returns an empty string and appends a trailing slash to `SCRIPT_NAME`, even though empty path segments are normally ignored, and `SCRIPT_NAME` doesn't normally end in a slash. This is intentional behavior, to ensure that an application can tell the difference between URIs ending in `/x` from ones ending in `/x/` when using this routine to do object traversal.

</div>

<div class="function">

setup_testing_defaults(environ)

Update *environ* with trivial defaults for testing purposes.

This routine adds various parameters required for WSGI, including `HTTP_HOST`, `SERVER_NAME`, `SERVER_PORT`, `REQUEST_METHOD`, `SCRIPT_NAME`, `PATH_INFO`, and all of the `3333`-defined `wsgi.*` variables. It only supplies default values, and does not replace any existing settings for these variables.

This routine is intended to make it easier for unit tests of WSGI servers and applications to set up dummy environments. It should NOT be used by actual WSGI servers or applications, since the data is fake!

Example usage (see also `~wsgiref.simple_server.demo_app` for another example):

    from wsgiref.util import setup_testing_defaults
    from wsgiref.simple_server import make_server

    # A relatively simple WSGI application. It's going to print out the
    # environment dictionary after being updated by setup_testing_defaults
    def simple_app(environ, start_response):
        setup_testing_defaults(environ)

        status = '200 OK'
        headers = [('Content-type', 'text/plain; charset=utf-8')]

        start_response(status, headers)

        ret = [("%s: %s\n" % (key, value)).encode("utf-8")
               for key, value in environ.items()]
        return ret

    with make_server('', 8000, simple_app) as httpd:
        print("Serving on port 8000...")
        httpd.serve_forever()

</div>

In addition to the environment functions above, the `!wsgiref.util` module also provides these miscellaneous utilities:

<div class="function">

is_hop_by_hop(header_name)

Return `True` if 'header_name' is an HTTP/1.1 "Hop-by-Hop" header, as defined by `2616`.

</div>

<div class="FileWrapper(filelike, blksize=8192)">

A concrete implementation of the `wsgiref.types.FileWrapper` protocol used to convert a file-like object to an `iterator`. The resulting objects are `iterable`s. As the object is iterated over, the optional *blksize* parameter will be repeatedly passed to the *filelike* object's `read` method to obtain bytestrings to yield. When `read` returns an empty bytestring, iteration is ended and is not resumable.

If *filelike* has a `close` method, the returned object will also have a `close` method, and it will invoke the *filelike* object's `close` method when called.

Example usage:

    from io import StringIO
    from wsgiref.util import FileWrapper

    # We're using a StringIO-buffer for as the file-like object
    filelike = StringIO("This is an example file-like object"*10)
    wrapper = FileWrapper(filelike, blksize=5)

    for chunk in wrapper:
        print(chunk)

<div class="versionchanged">

3.11 Support for `~object.__getitem__` method has been removed.

</div>

</div>

## `!wsgiref.headers` -- WSGI response header tools

<div class="module" data-synopsis="WSGI response header tools.">

wsgiref.headers

</div>

This module provides a single class, `Headers`, for convenient manipulation of WSGI response headers using a mapping-like interface.

<div class="Headers([headers])">

Create a mapping-like object wrapping *headers*, which must be a list of header name/value tuples as described in `3333`. The default value of *headers* is an empty list.

`Headers` objects support typical mapping operations including `~object.__getitem__`, `~dict.get`, `~object.__setitem__`, `~dict.setdefault`, `~object.__delitem__` and `~object.__contains__`. For each of these methods, the key is the header name (treated case-insensitively), and the value is the first value associated with that header name. Setting a header deletes any existing values for that header, then adds a new value at the end of the wrapped header list. Headers' existing order is generally maintained, with new headers added to the end of the wrapped list.

Unlike a dictionary, `Headers` objects do not raise an error when you try to get or delete a key that isn't in the wrapped header list. Getting a nonexistent header just returns `None`, and deleting a nonexistent header does nothing.

`Headers` objects also support `keys`, `values`, and `items` methods. The lists returned by `keys` and `items` can include the same key more than once if there is a multi-valued header. The `len()` of a `Headers` object is the same as the length of its `items`, which is the same as the length of the wrapped header list. In fact, the `items` method just returns a copy of the wrapped header list.

Calling `bytes()` on a `Headers` object returns a formatted bytestring suitable for transmission as HTTP response headers. Each header is placed on a line with its value, separated by a colon and a space. Each line is terminated by a carriage return and line feed, and the bytestring is terminated with a blank line.

In addition to their mapping interface and formatting features, `Headers` objects also have the following methods for querying and adding multi-valued headers, and for adding headers with MIME parameters:

<div class="method">

Headers.get_all(name)

Return a list of all the values for the named header.

The returned list will be sorted in the order they appeared in the original header list or were added to this instance, and may contain duplicates. Any fields deleted and re-inserted are always appended to the header list. If no fields exist with the given name, returns an empty list.

</div>

<div class="method">

Headers.add_header(name, value, \*\*\_params)

Add a (possibly multi-valued) header, with optional MIME parameters specified via keyword arguments.

*name* is the header field to add. Keyword arguments can be used to set MIME parameters for the header field. Each parameter must be a string or `None`. Underscores in parameter names are converted to dashes, since dashes are illegal in Python identifiers, but many MIME parameter names include dashes. If the parameter value is a string, it is added to the header value parameters in the form `name="value"`. If it is `None`, only the parameter name is added. (This is used for MIME parameters without a value.) Example usage:

    h.add_header('content-disposition', 'attachment', filename='bud.gif')

The above will add a header that looks like this:

    Content-Disposition: attachment; filename="bud.gif"

</div>

<div class="versionchanged">

3.5 *headers* parameter is optional.

</div>

</div>

## `!wsgiref.simple_server` -- a simple WSGI HTTP server

<div class="module" data-synopsis="A simple WSGI HTTP server.">

wsgiref.simple_server

</div>

This module implements a simple HTTP server (based on `http.server`) that serves WSGI applications. Each server instance serves a single WSGI application on a given host and port. If you want to serve multiple applications on a single host and port, you should create a WSGI application that parses `PATH_INFO` to select which application to invoke for each request. (E.g., using the `shift_path_info` function from `wsgiref.util`.)

<div class="function">

make_server(host, port, app, server_class=WSGIServer, handler_class=WSGIRequestHandler)

Create a new WSGI server listening on *host* and *port*, accepting connections for *app*. The return value is an instance of the supplied *server_class*, and will process requests using the specified *handler_class*. *app* must be a WSGI application object, as defined by `3333`.

Example usage:

    from wsgiref.simple_server import make_server, demo_app

    with make_server('', 8000, demo_app) as httpd:
        print("Serving HTTP on port 8000...")

        # Respond to requests until process is killed
        httpd.serve_forever()

        # Alternative: serve one request, then exit
        httpd.handle_request()

</div>

<div class="function">

demo_app(environ, start_response)

This function is a small but complete WSGI application that returns a text page containing the message "Hello world!" and a list of the key/value pairs provided in the *environ* parameter. It's useful for verifying that a WSGI server (such as `!wsgiref.simple_server`) is able to run a simple WSGI application correctly.

The *start_response* callable should follow the `.StartResponse` protocol.

</div>

<div class="WSGIServer(server_address, RequestHandlerClass)">

Create a `WSGIServer` instance. *server_address* should be a `(host,port)` tuple, and *RequestHandlerClass* should be the subclass of `http.server.BaseHTTPRequestHandler` that will be used to process requests.

You do not normally need to call this constructor, as the `make_server` function can handle all the details for you.

`WSGIServer` is a subclass of `http.server.HTTPServer`, so all of its methods (such as `serve_forever` and `handle_request`) are available. `WSGIServer` also provides these WSGI-specific methods:

<div class="method">

WSGIServer.set_app(application)

Sets the callable *application* as the WSGI application that will receive requests.

</div>

<div class="method">

WSGIServer.get_app()

Returns the currently set application callable.

</div>

Normally, however, you do not need to use these additional methods, as `set_app` is normally called by `make_server`, and the `get_app` exists mainly for the benefit of request handler instances.

</div>

<div class="WSGIRequestHandler(request, client_address, server)">

Create an HTTP handler for the given *request* (i.e. a socket), *client_address* (a `(host,port)` tuple), and *server* (`WSGIServer` instance).

You do not need to create instances of this class directly; they are automatically created as needed by `WSGIServer` objects. You can, however, subclass this class and supply it as a *handler_class* to the `make_server` function. Some possibly relevant methods for overriding in subclasses:

<div class="method">

WSGIRequestHandler.get_environ()

Return a `~wsgiref.types.WSGIEnvironment` dictionary for a request. The default implementation copies the contents of the `WSGIServer` object's `base_environ` dictionary attribute and then adds various headers derived from the HTTP request. Each call to this method should return a new dictionary containing all of the relevant CGI environment variables as specified in `3333`.

</div>

<div class="method">

WSGIRequestHandler.get_stderr()

Return the object that should be used as the `wsgi.errors` stream. The default implementation just returns `sys.stderr`.

</div>

<div class="method">

WSGIRequestHandler.handle()

Process the HTTP request. The default implementation creates a handler instance using a `wsgiref.handlers` class to implement the actual WSGI application interface.

</div>

</div>

## `!wsgiref.validate` --- WSGI conformance checker

<div class="module" data-synopsis="WSGI conformance checker.">

wsgiref.validate

</div>

When creating new WSGI application objects, frameworks, servers, or middleware, it can be useful to validate the new code's conformance using `!wsgiref.validate`. This module provides a function that creates WSGI application objects that validate communications between a WSGI server or gateway and a WSGI application object, to check both sides for protocol conformance.

Note that this utility does not guarantee complete `3333` compliance; an absence of errors from this module does not necessarily mean that errors do not exist. However, if this module does produce an error, then it is virtually certain that either the server or application is not 100% compliant.

This module is based on the `paste.lint` module from Ian Bicking's "Python Paste" library.

<div class="function">

validator(application)

Wrap *application* and return a new WSGI application object. The returned application will forward all requests to the original *application*, and will check that both the *application* and the server invoking it are conforming to the WSGI specification and to `2616`.

Any detected nonconformance results in an `AssertionError` being raised; note, however, that how these errors are handled is server-dependent. For example, `wsgiref.simple_server` and other servers based on `wsgiref.handlers` (that don't override the error handling methods to do something else) will simply output a message that an error has occurred, and dump the traceback to `sys.stderr` or some other error stream.

This wrapper may also generate output using the `warnings` module to indicate behaviors that are questionable but which may not actually be prohibited by `3333`. Unless they are suppressed using Python command-line options or the `warnings` API, any such warnings will be written to `sys.stderr` (*not* `wsgi.errors`, unless they happen to be the same object).

Example usage:

    from wsgiref.validate import validator
    from wsgiref.simple_server import make_server

    # Our callable object which is intentionally not compliant to the
    # standard, so the validator is going to break
    def simple_app(environ, start_response):
        status = '200 OK'  # HTTP Status
        headers = [('Content-type', 'text/plain')]  # HTTP Headers
        start_response(status, headers)

        # This is going to break because we need to return a list, and
        # the validator is going to inform us
        return b"Hello World"

    # This is the application wrapped in a validator
    validator_app = validator(simple_app)

    with make_server('', 8000, validator_app) as httpd:
        print("Listening on port 8000....")
        httpd.serve_forever()

</div>

## `!wsgiref.handlers` -- server/gateway base classes

<div class="module" data-synopsis="WSGI server/gateway base classes.">

wsgiref.handlers

</div>

This module provides base handler classes for implementing WSGI servers and gateways. These base classes handle most of the work of communicating with a WSGI application, as long as they are given a CGI-like environment, along with input, output, and error streams.

<div class="CGIHandler()">

CGI-based invocation via `sys.stdin`, `sys.stdout`, `sys.stderr` and `os.environ`. This is useful when you have a WSGI application and want to run it as a CGI script. Simply invoke `CGIHandler().run(app)`, where `app` is the WSGI application object you wish to invoke.

This class is a subclass of `BaseCGIHandler` that sets `wsgi.run_once` to true, `wsgi.multithread` to false, and `wsgi.multiprocess` to true, and always uses `sys` and `os` to obtain the necessary CGI streams and environment.

</div>

<div class="IISCGIHandler()">

A specialized alternative to `CGIHandler`, for use when deploying on Microsoft's IIS web server, without having set the config allowPathInfo option (IIS\>=7) or metabase allowPathInfoForScriptMappings (IIS\<7).

By default, IIS gives a `PATH_INFO` that duplicates the `SCRIPT_NAME` at the front, causing problems for WSGI applications that wish to implement routing. This handler strips any such duplicated path.

IIS can be configured to pass the correct `PATH_INFO`, but this causes another bug where `PATH_TRANSLATED` is wrong. Luckily this variable is rarely used and is not guaranteed by WSGI. On IIS\<7, though, the setting can only be made on a vhost level, affecting all other script mappings, many of which break when exposed to the `PATH_TRANSLATED` bug. For this reason IIS\<7 is almost never deployed with the fix (Even IIS7 rarely uses it because there is still no UI for it.).

There is no way for CGI code to tell whether the option was set, so a separate handler class is provided. It is used in the same way as `CGIHandler`, i.e., by calling `IISCGIHandler().run(app)`, where `app` is the WSGI application object you wish to invoke.

<div class="versionadded">

3.2

</div>

</div>

<div class="BaseCGIHandler(stdin, stdout, stderr, environ, multithread=True, multiprocess=False)">

Similar to `CGIHandler`, but instead of using the `sys` and `os` modules, the CGI environment and I/O streams are specified explicitly. The *multithread* and *multiprocess* values are used to set the `wsgi.multithread` and `wsgi.multiprocess` flags for any applications run by the handler instance.

This class is a subclass of `SimpleHandler` intended for use with software other than HTTP "origin servers". If you are writing a gateway protocol implementation (such as CGI, FastCGI, SCGI, etc.) that uses a `Status:` header to send an HTTP status, you probably want to subclass this instead of `SimpleHandler`.

</div>

<div class="SimpleHandler(stdin, stdout, stderr, environ, multithread=True, multiprocess=False)">

Similar to `BaseCGIHandler`, but designed for use with HTTP origin servers. If you are writing an HTTP server implementation, you will probably want to subclass this instead of `BaseCGIHandler`.

This class is a subclass of `BaseHandler`. It overrides the `!__init__`, `~BaseHandler.get_stdin`, `~BaseHandler.get_stderr`, `~BaseHandler.add_cgi_vars`, `~BaseHandler._write`, and `~BaseHandler._flush` methods to support explicitly setting the environment and streams via the constructor. The supplied environment and streams are stored in the `stdin`, `stdout`, `stderr`, and `environ` attributes.

The `~io.BufferedIOBase.write` method of *stdout* should write each chunk in full, like `io.BufferedIOBase`.

</div>

<div class="BaseHandler()">

This is an abstract base class for running WSGI applications. Each instance will handle a single HTTP request, although in principle you could create a subclass that was reusable for multiple requests.

`BaseHandler` instances have only one method intended for external use:

<div class="method">

BaseHandler.run(app)

Run the specified WSGI application, *app*.

</div>

All of the other `BaseHandler` methods are invoked by this method in the process of running the application, and thus exist primarily to allow customizing the process.

The following methods MUST be overridden in a subclass:

<div class="method">

BaseHandler.\_write(data)

Buffer the bytes *data* for transmission to the client. It's okay if this method actually transmits the data; `BaseHandler` just separates write and flush operations for greater efficiency when the underlying system actually has such a distinction.

</div>

<div class="method">

BaseHandler.\_flush()

Force buffered data to be transmitted to the client. It's okay if this method is a no-op (i.e., if `_write` actually sends the data).

</div>

<div class="method">

BaseHandler.get_stdin()

Return an object compatible with `~wsgiref.types.InputStream` suitable for use as the `wsgi.input` of the request currently being processed.

</div>

<div class="method">

BaseHandler.get_stderr()

Return an object compatible with `~wsgiref.types.ErrorStream` suitable for use as the `wsgi.errors` of the request currently being processed.

</div>

<div class="method">

BaseHandler.add_cgi_vars()

Insert CGI variables for the current request into the `environ` attribute.

</div>

Here are some other methods and attributes you may wish to override. This list is only a summary, however, and does not include every method that can be overridden. You should consult the docstrings and source code for additional information before attempting to create a customized `BaseHandler` subclass.

Attributes and methods for customizing the WSGI environment:

<div class="attribute">

BaseHandler.wsgi_multithread

The value to be used for the `wsgi.multithread` environment variable. It defaults to true in `BaseHandler`, but may have a different default (or be set by the constructor) in the other subclasses.

</div>

<div class="attribute">

BaseHandler.wsgi_multiprocess

The value to be used for the `wsgi.multiprocess` environment variable. It defaults to true in `BaseHandler`, but may have a different default (or be set by the constructor) in the other subclasses.

</div>

<div class="attribute">

BaseHandler.wsgi_run_once

The value to be used for the `wsgi.run_once` environment variable. It defaults to false in `BaseHandler`, but `CGIHandler` sets it to true by default.

</div>

<div class="attribute">

BaseHandler.os_environ

The default environment variables to be included in every request's WSGI environment. By default, this is a copy of `os.environ` at the time that `!wsgiref.handlers` was imported, but subclasses can either create their own at the class or instance level. Note that the dictionary should be considered read-only, since the default value is shared between multiple classes and instances.

</div>

<div class="attribute">

BaseHandler.server_software

If the `origin_server` attribute is set, this attribute's value is used to set the default `SERVER_SOFTWARE` WSGI environment variable, and also to set a default `Server:` header in HTTP responses. It is ignored for handlers (such as `BaseCGIHandler` and `CGIHandler`) that are not HTTP origin servers.

<div class="versionchanged">

3.3 The term "Python" is replaced with implementation specific term like "CPython", "Jython" etc.

</div>

</div>

<div class="method">

BaseHandler.get_scheme()

Return the URL scheme being used for the current request. The default implementation uses the `guess_scheme` function from `wsgiref.util` to guess whether the scheme should be "http" or "https", based on the current request's `environ` variables.

</div>

<div class="method">

BaseHandler.setup_environ()

Set the `environ` attribute to a fully populated WSGI environment. The default implementation uses all of the above methods and attributes, plus the `get_stdin`, `get_stderr`, and `add_cgi_vars` methods and the `wsgi_file_wrapper` attribute. It also inserts a `SERVER_SOFTWARE` key if not present, as long as the `origin_server` attribute is a true value and the `server_software` attribute is set.

</div>

Methods and attributes for customizing exception handling:

<div class="method">

BaseHandler.log_exception(exc_info)

Log the *exc_info* tuple in the server log. *exc_info* is a `(type, value, traceback)` tuple. The default implementation simply writes the traceback to the request's `wsgi.errors` stream and flushes it. Subclasses can override this method to change the format or retarget the output, mail the traceback to an administrator, or whatever other action may be deemed suitable.

</div>

<div class="attribute">

BaseHandler.traceback_limit

The maximum number of frames to include in tracebacks output by the default `log_exception` method. If `None`, all frames are included.

</div>

<div class="method">

BaseHandler.error_output(environ, start_response)

This method is a WSGI application to generate an error page for the user. It is only invoked if an error occurs before headers are sent to the client.

This method can access the current error using `sys.exception()`, and should pass that information to *start_response* when calling it (as described in the "Error Handling" section of `3333`). In particular, the *start_response* callable should follow the `.StartResponse` protocol.

The default implementation just uses the `error_status`, `error_headers`, and `error_body` attributes to generate an output page. Subclasses can override this to produce more dynamic error output.

Note, however, that it's not recommended from a security perspective to spit out diagnostics to any old user; ideally, you should have to do something special to enable diagnostic output, which is why the default implementation doesn't include any.

</div>

<div class="attribute">

BaseHandler.error_status

The HTTP status used for error responses. This should be a status string as defined in `3333`; it defaults to a 500 code and message.

</div>

<div class="attribute">

BaseHandler.error_headers

The HTTP headers used for error responses. This should be a list of WSGI response headers (`(name, value)` tuples), as described in `3333`. The default list just sets the content type to `text/plain`.

</div>

<div class="attribute">

BaseHandler.error_body

The error response body. This should be an HTTP response body bytestring. It defaults to the plain text, "A server error occurred. Please contact the administrator."

</div>

Methods and attributes for `3333`'s "Optional Platform-Specific File Handling" feature:

<div class="attribute">

BaseHandler.wsgi_file_wrapper

A `wsgi.file_wrapper` factory, compatible with `wsgiref.types.FileWrapper`, or `None`. The default value of this attribute is the `wsgiref.util.FileWrapper` class.

</div>

<div class="method">

BaseHandler.sendfile()

Override to implement platform-specific file transmission. This method is called only if the application's return value is an instance of the class specified by the `wsgi_file_wrapper` attribute. It should return a true value if it was able to successfully transmit the file, so that the default transmission code will not be executed. The default implementation of this method just returns a false value.

</div>

Miscellaneous methods and attributes:

<div class="attribute">

BaseHandler.origin_server

This attribute should be set to a true value if the handler's `_write` and `_flush` are being used to communicate directly to the client, rather than via a CGI-like gateway protocol that wants the HTTP status in a special `Status:` header.

This attribute's default value is true in `BaseHandler`, but false in `BaseCGIHandler` and `CGIHandler`.

</div>

<div class="attribute">

BaseHandler.http_version

If `origin_server` is true, this string attribute is used to set the HTTP version of the response set to the client. It defaults to `"1.0"`.

</div>

</div>

<div class="function">

read_environ()

Transcode CGI variables from `os.environ` to `3333` "bytes in unicode" strings, returning a new dictionary. This function is used by `CGIHandler` and `IISCGIHandler` in place of directly using `os.environ`, which is not necessarily WSGI-compliant on all platforms and web servers using Python 3 -- specifically, ones where the OS's actual environment is Unicode (i.e. Windows), or ones where the environment is bytes, but the system encoding used by Python to decode it is anything other than ISO-8859-1 (e.g. Unix systems using UTF-8).

If you are implementing a CGI-based handler of your own, you probably want to use this routine instead of just copying values out of `os.environ` directly.

<div class="versionadded">

3.2

</div>

</div>

## `!wsgiref.types` -- WSGI types for static type checking

<div class="module" data-synopsis="WSGI types for static type checking">

wsgiref.types

</div>

This module provides various types for static type checking as described in `3333`.

<div class="versionadded">

3.11

</div>

<div class="StartResponse">

A `typing.Protocol` describing `start_response()
<3333#the-start-response-callable>` callables (`3333`).

</div>

<div class="data">

WSGIEnvironment

A type alias describing a WSGI environment dictionary.

</div>

<div class="data">

WSGIApplication

A type alias describing a WSGI application callable.

</div>

<div class="InputStream()">

A `typing.Protocol` describing a `WSGI Input Stream
<3333#input-and-error-streams>`.

</div>

<div class="ErrorStream()">

A `typing.Protocol` describing a `WSGI Error Stream
<3333#input-and-error-streams>`.

</div>

<div class="FileWrapper()">

A `typing.Protocol` describing a `file wrapper
<3333#optional-platform-specific-file-handling>`. See `wsgiref.util.FileWrapper` for a concrete implementation of this protocol.

</div>

## Examples

This is a working "Hello World" WSGI application, where the *start_response* callable should follow the `.StartResponse` protocol:

    """
    Every WSGI application must have an application object - a callable
    object that accepts two arguments. For that purpose, we're going to
    use a function (note that you're not limited to a function, you can
    use a class for example). The first argument passed to the function
    is a dictionary containing CGI-style environment variables and the
    second variable is the callable object.
    """
    from wsgiref.simple_server import make_server


    def hello_world_app(environ, start_response):
        status = "200 OK"  # HTTP Status
        headers = [("Content-type", "text/plain; charset=utf-8")]  # HTTP Headers
        start_response(status, headers)

        # The returned object is going to be printed
        return [b"Hello World"]

    with make_server("", 8000, hello_world_app) as httpd:
        print("Serving on port 8000...")

        # Serve until process is killed
        httpd.serve_forever()

Example of a WSGI application serving the current directory, accept optional directory and port number (default: 8000) on the command line:

    """
    Small wsgiref based web server. Takes a path to serve from and an
    optional port number (defaults to 8000), then tries to serve files.
    MIME types are guessed from the file names, 404 errors are raised
    if the file is not found.
    """
    import mimetypes
    import os
    import sys
    from wsgiref import simple_server, util


    def app(environ, respond):
        # Get the file name and MIME type
        fn = os.path.join(path, environ["PATH_INFO"][1:])
        if "." not in fn.split(os.path.sep)[-1]:
            fn = os.path.join(fn, "index.html")
        mime_type = mimetypes.guess_file_type(fn)[0]

        # Return 200 OK if file exists, otherwise 404 Not Found
        if os.path.exists(fn):
            respond("200 OK", [("Content-Type", mime_type)])
            return util.FileWrapper(open(fn, "rb"))
        else:
            respond("404 Not Found", [("Content-Type", "text/plain")])
            return [b"not found"]


    if __name__ == "__main__":
        # Get the path and port from command-line arguments
        path = sys.argv[1] if len(sys.argv) > 1 else os.getcwd()
        port = int(sys.argv[2]) if len(sys.argv) > 2 else 8000

        # Make and start the server until control-c
        httpd = simple_server.make_server("", port, app)
        print(f"Serving {path} on port {port}, control-C to stop")
        try:
            httpd.serve_forever()
        except KeyboardInterrupt:
            print("Shutting down.")
            httpd.server_close()

# `!xdrlib` --- Encode and decode XDR data

<div class="module" data-synopsis="Removed in 3.13." data-deprecated="">

xdrlib

</div>

<div class="deprecated-removed">

3.11 3.13

</div>

This module is no longer part of the Python standard library. It was `removed in Python 3.13 <whatsnew313-pep594>` after being deprecated in Python 3.11. The removal was decided in `594`.

The last version of Python that provided the `!xdrlib` module was [Python 3.12](https://docs.python.org/3.12/library/xdrlib.html).

# `!xml.dom` --- The Document Object Model API

<div class="module" data-synopsis="Document Object Model API for Python.">

xml.dom

</div>

**Source code:** `Lib/xml/dom/__init__.py`

------------------------------------------------------------------------

The Document Object Model, or "DOM," is a cross-language API from the World Wide Web Consortium (W3C) for accessing and modifying XML documents. A DOM implementation presents an XML document as a tree structure, or allows client code to build such a structure from scratch. It then gives access to the structure through a set of objects which provided well-known interfaces.

The DOM is extremely useful for random-access applications. SAX only allows you a view of one bit of the document at a time. If you are looking at one SAX element, you have no access to another. If you are looking at a text node, you have no access to a containing element. When you write a SAX application, you need to keep track of your program's position in the document somewhere in your own code. SAX does not do it for you. Also, if you need to look ahead in the XML document, you are just out of luck.

Some applications are simply impossible in an event driven model with no access to a tree. Of course you could build some sort of tree yourself in SAX events, but the DOM allows you to avoid writing that code. The DOM is a standard tree representation for XML data.

The Document Object Model is being defined by the W3C in stages, or "levels" in their terminology. The Python mapping of the API is substantially based on the DOM Level 2 recommendation.

DOM applications typically start by parsing some XML into a DOM. How this is accomplished is not covered at all by DOM Level 1, and Level 2 provides only limited improvements: There is a `DOMImplementation` object class which provides access to `Document` creation methods, but no way to access an XML reader/parser/Document builder in an implementation-independent way. There is also no well-defined way to access these methods without an existing `Document` object. In Python, each DOM implementation will provide a function `getDOMImplementation`. DOM Level 3 adds a Load/Store specification, which defines an interface to the reader, but this is not yet available in the Python standard library.

Once you have a DOM document object, you can access the parts of your XML document through its properties and methods. These properties are defined in the DOM specification; this portion of the reference manual describes the interpretation of the specification in Python.

The specification provided by the W3C defines the DOM API for Java, ECMAScript, and OMG IDL. The Python mapping defined here is based in large part on the IDL version of the specification, but strict compliance is not required (though implementations are free to support the strict mapping from IDL). See section `dom-conformance` for a detailed discussion of mapping requirements.

<div class="seealso">

[Document Object Model (DOM) Level 2 Specification](https://www.w3.org/TR/2000/REC-DOM-Level-2-Core-20001113/)  
The W3C recommendation upon which the Python DOM API is based.

[Document Object Model (DOM) Level 1 Specification](https://www.w3.org/TR/REC-DOM-Level-1/)  
The W3C recommendation for the DOM supported by `xml.dom.minidom`.

[Python Language Mapping Specification](https://www.omg.org/spec/PYTH/1.2/PDF)  
This specifies the mapping from OMG IDL to Python.

</div>

## Module Contents

The `!xml.dom` contains the following functions:

<div class="function">

registerDOMImplementation(name, factory)

Register the *factory* function with the name *name*. The factory function should return an object which implements the `DOMImplementation` interface. The factory function can return the same object every time, or a new one for each call, as appropriate for the specific implementation (e.g. if that implementation supports some customization).

</div>

<div class="function">

getDOMImplementation(name=None, features=())

Return a suitable DOM implementation. The *name* is either well-known, the module name of a DOM implementation, or `None`. If it is not `None`, imports the corresponding module and returns a `DOMImplementation` object if the import succeeds. If no name is given, and if the environment variable `!PYTHON_DOM` is set, this variable is used to find the implementation. The only well-known name in the standard library is `'minidom'`, for `xml.dom.minidom`.

If name is not given, this examines the available implementations to find one with the required feature set. If no implementation can be found, raise an `ImportError`. The features list must be a sequence of `(feature, version)` pairs which are passed to the `~DOMImplementation.hasFeature` method on available `DOMImplementation` objects.

</div>

Some convenience constants are also provided:

<div class="data">

EMPTY_NAMESPACE

The value used to indicate that no namespace is associated with a node in the DOM. This is typically found as the `~Node.namespaceURI` of a node, or used as the *namespaceURI* parameter to a namespaces-specific method.

</div>

<div class="data">

XML_NAMESPACE

The namespace URI associated with the reserved prefix `xml`, as defined by [Namespaces in XML](https://www.w3.org/TR/REC-xml-names/) (section 4).

</div>

<div class="data">

XMLNS_NAMESPACE

The namespace URI for namespace declarations, as defined by [Document Object Model (DOM) Level 2 Core Specification](https://www.w3.org/TR/DOM-Level-2-Core/core.html) (section 1.1.8).

</div>

<div class="data">

XHTML_NAMESPACE

The URI of the XHTML namespace as defined by [XHTML 1.0: The Extensible HyperText Markup Language](https://www.w3.org/TR/xhtml1/) (section 3.1.1).

</div>

In addition, `!xml.dom` contains a base `Node` class and the DOM exception classes. The `Node` class provided by this module does not implement any of the methods or attributes defined by the DOM specification; concrete DOM implementations must provide those. The `Node` class provided as part of this module does provide the constants used for the `~Node.nodeType` attribute on concrete `Node` objects; they are located within the class rather than at the module level to conform with the DOM specifications.

## Objects in the DOM

The definitive documentation for the DOM is the DOM specification from the W3C.

The names documented in this section are DOM interfaces. With the exception of `Node` and the exception classes, they are not provided by the `!xml.dom` module itself, but by concrete DOM implementations, such as `xml.dom.minidom`.

Note that DOM attributes may also be manipulated as nodes instead of as simple strings. It is fairly rare that you must do this, however, so this usage is not yet documented.

| Interface | Section | Purpose |
|----|----|----|
| `DOMImplementation` | `dom-implementation-objects` | Interface to the underlying implementation. |
| `Node` | `dom-node-objects` | Base interface for most objects in a document. |
| `NodeList` | `dom-nodelist-objects` | Interface for a sequence of nodes. |
| `DocumentType` | `dom-documenttype-objects` | Information about the declarations needed to process a document. |
| `Document` | `dom-document-objects` | Object which represents an entire document. |
| `Element` | `dom-element-objects` | Element nodes in the document hierarchy. |
| `Attr` | `dom-attr-objects` | Attribute value nodes on element nodes. |
| `Comment` | `dom-comment-objects` | Representation of comments in the source document. |
| `Text` | `dom-text-objects` | Nodes containing textual content from the document. |
| `ProcessingInstruction` | `dom-pi-objects` | Processing instruction representation. |

An additional section describes the exceptions defined for working with the DOM in Python.

### DOMImplementation Objects

<div class="DOMImplementation" data-no-typesetting="">

The `DOMImplementation` interface provides a way for applications to determine the availability of particular features in the DOM they are using. DOM Level 2 added the ability to create new `Document` and `DocumentType` objects using the `DOMImplementation` as well.

</div>

<div class="method">

DOMImplementation.hasFeature(feature, version)

Return `True` if the feature identified by the pair of strings *feature* and *version* is implemented.

</div>

<div class="method">

DOMImplementation.createDocument(namespaceUri, qualifiedName, doctype)

Return a new `Document` object (the root of the DOM), with a child `Element` object having the given *namespaceUri* and *qualifiedName*. The *doctype* must be a `DocumentType` object created by `createDocumentType`, or `None`. In the Python DOM API, the first two arguments can also be `None` in order to indicate that no `Element` child is to be created.

</div>

<div class="method">

DOMImplementation.createDocumentType(qualifiedName, publicId, systemId)

Return a new `DocumentType` object that encapsulates the given *qualifiedName*, *publicId*, and *systemId* strings, representing the information contained in an XML document type declaration.

Raise `InvalidCharacterErr` if the name is not a valid XML name.

</div>

### Node Objects

<div class="Node" data-no-typesetting="">

All of the components of an XML document are subclasses of `Node`.

</div>

Only nodes of the following types can have children, and only children of the listed types:

`Document`  
at most one `Element`, at most one `DocumentType`, `ProcessingInstruction` and `Comment`

`DocumentFragment` and `Element`  
`Element`, `Text`, `CDATASection`, `ProcessingInstruction` and `Comment`

`Attr`  
`Text`

Nodes of other types cannot have children. Inserting a child of a not allowed type raises `HierarchyRequestErr`.

<div class="attribute">

Node.nodeType

An integer representing the node type. Symbolic constants for the types are on the `Node` object. This is a read-only attribute.

</div>

<div class="data">

Node.ELEMENT_NODE Node.ATTRIBUTE_NODE Node.TEXT_NODE Node.CDATA_SECTION_NODE Node.ENTITY_REFERENCE_NODE Node.ENTITY_NODE Node.PROCESSING_INSTRUCTION_NODE Node.COMMENT_NODE Node.DOCUMENT_NODE Node.DOCUMENT_TYPE_NODE Node.DOCUMENT_FRAGMENT_NODE Node.NOTATION_NODE

Integer constants for the possible values of the `~Node.nodeType` attribute.

</div>

<div class="attribute">

Node.parentNode

The parent of the current node, or `None` for the document node. The value is always a `Node` object or `None`. For `Element` nodes, this will be the parent element, except for the root element, in which case it will be the `Document` object. For `Attr` nodes, this is always `None`. This is a read-only attribute.

</div>

<div class="attribute">

Node.attributes

A `NamedNodeMap` of attribute objects. Only elements have actual values for this; others provide `None` for this attribute. This is a read-only attribute.

</div>

<div class="attribute">

Node.previousSibling

The node that immediately precedes this one with the same parent. For instance the element with an end-tag that comes just before the *self* element's start-tag. Of course, XML documents are made up of more than just elements so the previous sibling could be text, a comment, or something else. If this node is the first child of the parent, this attribute will be `None`. This is a read-only attribute.

</div>

<div class="attribute">

Node.nextSibling

The node that immediately follows this one with the same parent. See also `previousSibling`. If this is the last child of the parent, this attribute will be `None`. This is a read-only attribute.

</div>

<div class="attribute">

Node.childNodes

A `NodeList` of the children of this node. If the node has no children, the list is empty. This is a read-only attribute.

</div>

<div class="attribute">

Node.firstChild

The first child of the node, if there are any, or `None`. This is a read-only attribute.

</div>

<div class="attribute">

Node.lastChild

The last child of the node, if there are any, or `None`. This is a read-only attribute.

</div>

<div class="attribute">

Node.localName

The part of the `~Element.tagName` following the colon if there is one, else the entire `~Element.tagName`. The value is a string.

</div>

<div class="attribute">

Node.prefix

The part of the `~Element.tagName` preceding the colon if there is one, else the empty string. The value is a string, or `None`.

</div>

<div class="attribute">

Node.namespaceURI

The namespace associated with the element name. This will be a string or `None`. This is a read-only attribute.

</div>

<div class="attribute">

Node.ownerDocument

The `Document` object to which this node belongs, or `None` for a document itself. This is a read-only attribute.

</div>

<div class="method">

Node.isSupported(feature, version)

Return whether the DOM implementation supports a particular *feature*, as `DOMImplementation.hasFeature` does.

</div>

<div class="method">

Node.setUserData(key, data, handler)

Associate *data* with *key* on this node and return the data previously associated with *key*, or `None`. If *data* is `None`, the association is removed. *handler* is called when the node is cloned, imported, renamed or deleted; pass `None` if no notification is needed.

</div>

<div class="method">

Node.getUserData(key)

Return the data associated with *key* on this node by `~Node.setUserData`, or `None`.

</div>

<div class="attribute">

Node.nodeName

The name of this node, depending on its type; see the table below. You can always get the information you would get here from another property such as the `~Element.tagName` property for elements or the `~Attr.name` property for attributes. This is a read-only attribute.

</div>

<div class="attribute">

Node.nodeValue

The value of this node, depending on its type; see the table below. The value is a string or `None`.

</div>

The values of `~Node.nodeName` and `~Node.nodeValue` for each node type are:

| Node type | nodeName | nodeValue |
|----|----|----|
| `Attr` | `~Attr.name` | `~Attr.value` |
| `CDATASection` | `'#cdata-section'` | the content |
| `Comment` | `'#comment'` | the content |
| `Document` | `'#document'` | `None` |
| `DocumentFragment` | `'#document-fragment'` | `None` |
| `DocumentType` | `~DocumentType.name` | `None` |
| `Element` | `~Element.tagName` | `None` |
| `Entity` | the name of the entity | `None` |
| `Notation` | the name of the notation | `None` |
| `ProcessingInstruction` | `~ProcessingInstruction.target` | `~ProcessingInstruction.data` |
| `Text` | `'#text'` | the content |

<div class="method">

Node.hasAttributes()

Return `True` if the node has any attributes.

</div>

<div class="method">

Node.hasChildNodes()

Return `True` if the node has any child nodes.

</div>

<div class="method">

Node.isSameNode(other)

Return `True` if *other* refers to the same node as this node. This is especially useful for DOM implementations which use any sort of proxy architecture (because more than one object can refer to the same node).

> [!NOTE]
> This is based on a proposed DOM Level 3 API which is still in the "working draft" stage, but this particular interface appears uncontroversial. Changes from the W3C will not necessarily affect this method in the Python DOM interface (though any new W3C API for this would also be supported).

</div>

<div class="method">

Node.appendChild(newChild)

Add a new child node to this node at the end of the list of children, returning *newChild*. If the node was already in the tree, it is removed first.

Raise `WrongDocumentErr` if *newChild* was created by another document, and `HierarchyRequestErr` if it is this node itself or its ancestor.

</div>

<div class="method">

Node.insertBefore(newChild, refChild)

Insert a new child node before an existing child. It must be the case that *refChild* is a child of this node; if not, `NotFoundErr` is raised. *newChild* is returned. If *refChild* is `None`, it inserts *newChild* at the end of the children's list.

Raise `WrongDocumentErr` if *newChild* was created by another document, and `HierarchyRequestErr` if it is this node itself or its ancestor.

</div>

<div class="method">

Node.removeChild(oldChild)

Remove a child node. *oldChild* must be a child of this node; if not, `NotFoundErr` is raised. *oldChild* is returned on success. If *oldChild* will not be used further, its `~xml.dom.minidom.Node.unlink` method should be called.

</div>

<div class="method">

Node.replaceChild(newChild, oldChild)

Replace an existing node with a new node. It must be the case that *oldChild* is a child of this node; if not, `NotFoundErr` is raised.

Raise `WrongDocumentErr` if *newChild* was created by another document, and `HierarchyRequestErr` if it is this node itself or its ancestor.

</div>

<div class="method">

Node.normalize()

Join adjacent text nodes so that all stretches of text are stored as single `Text` instances. This simplifies processing text from a DOM tree for many applications.

</div>

<div class="method">

Node.cloneNode(deep)

Clone this node. Setting *deep* means to clone all child nodes as well. This returns the clone.

</div>

### NodeList Objects

<div class="NodeList" data-no-typesetting="">

A `NodeList` represents a sequence of nodes. These objects are used in two ways in the DOM Core recommendation: an `Element` object provides one as its list of child nodes, and the `~Element.getElementsByTagName` and `~Element.getElementsByTagNameNS` methods of `Node` return objects with this interface to represent query results.

</div>

`NodeList` does *not* inherit from `Node`.

The DOM Level 2 recommendation defines one method and one attribute for these objects:

<div class="method">

NodeList.item(i)

Return the *i*'th item from the sequence, or `None` if *i* is out of range. Negative indices are not supported.

</div>

<div class="attribute">

NodeList.length

The number of nodes in the sequence.

</div>

In addition, the Python DOM interface requires that some additional support is provided to allow `NodeList` objects to be used as Python sequences. All `NodeList` implementations must include support for `~object.__len__` and `~object.__getitem__`; this allows iteration over the `NodeList` in `for` statements and proper support for the `len` built-in function.

If a DOM implementation supports modification of the document, the `NodeList` implementation must also support the `~object.__setitem__` and `~object.__delitem__` methods.

### DocumentType Objects

<div class="DocumentType" data-no-typesetting="">

Information about the notations and entities declared by a document (including the external subset if the parser uses it and can provide the information) is available from a `DocumentType` object. The `DocumentType` for a document is available from the `Document` object's `~Document.doctype` attribute; if there is no `DOCTYPE` declaration for the document, the document's `~Document.doctype` attribute will be set to `None` instead of an instance of this interface.

</div>

`DocumentType` is a specialization of `Node`, and adds the following attributes:

<div class="attribute">

DocumentType.publicId

The public identifier for the external subset of the document type definition, or `None` if the `DOCTYPE` declaration does not specify it.

</div>

<div class="attribute">

DocumentType.systemId

The system identifier, a URI, for the external subset of the document type definition, or `None` if the `DOCTYPE` declaration does not specify it.

</div>

<div class="attribute">

DocumentType.internalSubset

A string giving the complete internal subset from the document. This does not include the brackets which enclose the subset. If the document has no internal subset, this should be `None`.

</div>

<div class="attribute">

DocumentType.name

The name of the root element as given in the `DOCTYPE` declaration, if present.

</div>

<div class="attribute">

DocumentType.entities

This is a `NamedNodeMap` of `Entity` nodes giving the definitions of external entities. For entity names defined more than once, only the first definition is provided (others are ignored as required by the XML recommendation). This may be `None` if the information is not provided by the parser, or if no entities are defined.

</div>

<div class="attribute">

DocumentType.notations

This is a `NamedNodeMap` of `Notation` nodes giving the definitions of notations. For notation names defined more than once, only the first definition is provided (others are ignored as required by the XML recommendation). This may be `None` if the information is not provided by the parser, or if no notations are defined.

</div>

### Document Objects

<div class="Document" data-no-typesetting="">

A `Document` represents an entire XML document, including its constituent elements, attributes, processing instructions, comments etc. Remember that it inherits properties from `Node`.

</div>

<div class="attribute">

Document.documentElement

The one and only root element of the document.

</div>

<div class="attribute">

Document.doctype

The `DocumentType` node of the document, or `None`. This is a read-only attribute.

</div>

<div class="attribute">

Document.implementation

The `DOMImplementation` object which created this document. This is a read-only attribute.

</div>

<div class="attribute">

Document.strictErrorChecking

Whether error checking is enforced.

</div>

<div class="attribute">

Document.documentURI

The location of the document, or `None` if it is unknown.

</div>

<div class="method">

Document.createDocumentFragment()

Create and return an empty `DocumentFragment` node.

</div>

<div class="method">

Document.createCDATASection(data)

Create and return a `CDATASection` node containing *data*.

</div>

<div class="method">

Document.importNode(importedNode, deep)

Return a copy of *importedNode* which belongs to this document. The original node is not removed from its document. If *deep* is true, the descendants of the node are copied too.

</div>

<div class="method">

Document.createElement(tagName)

Create and return a new element node. The element is not inserted into the document when it is created. You need to explicitly insert it with one of the other methods such as `~Node.insertBefore` or `~Node.appendChild`.

Raise `InvalidCharacterErr` if the name is not a valid XML name.

</div>

<div class="method">

Document.createElementNS(namespaceURI, tagName)

Create and return a new element with a namespace. The *tagName* may have a prefix. The element is not inserted into the document when it is created. You need to explicitly insert it with one of the other methods such as `~Node.insertBefore` or `~Node.appendChild`.

Raise `InvalidCharacterErr` if the name is not a valid XML name. Raise `NamespaceErr` if the qualified name is malformed, if it has a prefix and the namespace URI is empty, or if the prefix is `'xml'` and the namespace URI is not the XML namespace.

</div>

<div class="method">

Document.createTextNode(data)

Create and return a text node containing the data passed as a parameter. As with the other creation methods, this one does not insert the node into the tree.

</div>

<div class="method">

Document.createEntityReference(name)

Create and return a new entity reference node. The node is not inserted into the document when it is created.

Raise `InvalidCharacterErr` if the name is not a valid XML name.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

Document.createComment(data)

Create and return a comment node containing the data passed as a parameter. As with the other creation methods, this one does not insert the node into the tree.

</div>

<div class="method">

Document.createProcessingInstruction(target, data)

Create and return a processing instruction node containing the *target* and *data* passed as parameters. As with the other creation methods, this one does not insert the node into the tree.

Raise `InvalidCharacterErr` if the name is not a valid XML name.

</div>

<div class="method">

Document.createAttribute(name)

Create and return an attribute node. This method does not associate the attribute node with any particular element. You must use `~Element.setAttributeNode` on the appropriate `Element` object to use the newly created attribute instance.

Raise `InvalidCharacterErr` if the name is not a valid XML name.

</div>

<div class="method">

Document.createAttributeNS(namespaceURI, qualifiedName)

Create and return an attribute node with a namespace. The *tagName* may have a prefix. This method does not associate the attribute node with any particular element. You must use `~Element.setAttributeNode` on the appropriate `Element` object to use the newly created attribute instance.

Raise `InvalidCharacterErr` if the name is not a valid XML name. Raise `NamespaceErr` if the qualified name is malformed, if it has a prefix and the namespace URI is empty, if the prefix is `'xml'` and the namespace URI is not the XML namespace, or if the name or the prefix is `'xmlns'` and the namespace URI is not the XMLNS namespace, or vice versa.

</div>

<div class="method">

Document.getElementById(id)

Return the element with the given ID, or `None`. Only attributes declared as being of type ID in the DTD or by `Element.setIdAttribute` are searched.

</div>

<div class="method">

Document.getElementsByTagName(tagName)

Search for all descendants (direct children, children's children, etc.) with a particular element type name.

</div>

<div class="method">

Document.getElementsByTagNameNS(namespaceURI, localName)

Search for all descendants (direct children, children's children, etc.) with a particular namespace URI and localname. The localname is the part of the namespace after the prefix.

</div>

<div class="method">

Document.renameNode(n, namespaceURI, name)

Rename the element or attribute node *n* and return it. *namespaceURI* is the new namespace URI, or `~xml.dom.EMPTY_NAMESPACE` if the node does not belong to a namespace. *name* is the new qualified name.

Raise `WrongDocumentErr` if *n* was created by another document, and `NotSupportedErr` if it is neither an element nor an attribute.

</div>

### Element Objects

<div class="Element" data-no-typesetting="">

`Element` is a subclass of `Node`, so inherits all the attributes of that class.

</div>

<div class="attribute">

Element.tagName

The element type name. In a namespace-using document it may have colons in it. The value is a string.

</div>

<div class="method">

Element.setIdAttribute(name)

Declare that the attribute *name* is of type ID, so that the element is found by `Document.getElementById`. Raise `NotFoundErr` if the element has no such attribute.

</div>

<div class="method">

Element.setIdAttributeNS(namespaceURI, localName)

The same as `~Element.setIdAttribute`, but for an attribute specified by its namespace URI and local name.

</div>

<div class="method">

Element.setIdAttributeNode(idAttr)

The same as `~Element.setIdAttribute`, but for an already retrieved attribute node.

</div>

<div class="method">

Element.getElementsByTagName(tagName)

Same as equivalent method in the `Document` class.

</div>

<div class="method">

Element.getElementsByTagNameNS(namespaceURI, localName)

Same as equivalent method in the `Document` class.

</div>

<div class="method">

Element.hasAttribute(name)

Return `True` if the element has an attribute named by *name*.

</div>

<div class="method">

Element.hasAttributeNS(namespaceURI, localName)

Return `True` if the element has an attribute named by *namespaceURI* and *localName*.

</div>

<div class="method">

Element.getAttribute(name)

Return the value of the attribute named by *name* as a string. If no such attribute exists, an empty string is returned, as if the attribute had no value.

</div>

<div class="method">

Element.getAttributeNode(attrname)

Return the `Attr` node for the attribute named by *attrname*.

</div>

<div class="method">

Element.getAttributeNS(namespaceURI, localName)

Return the value of the attribute named by *namespaceURI* and *localName* as a string. If no such attribute exists, an empty string is returned, as if the attribute had no value.

</div>

<div class="method">

Element.getAttributeNodeNS(namespaceURI, localName)

Return an attribute value as a node, given a *namespaceURI* and *localName*.

</div>

<div class="method">

Element.removeAttribute(name)

Remove an attribute by name.

</div>

<div class="method">

Element.removeAttributeNode(oldAttr)

Remove and return *oldAttr* from the attribute list, if present. If *oldAttr* is not present, `NotFoundErr` is raised.

</div>

<div class="method">

Element.removeAttributeNS(namespaceURI, localName)

Remove an attribute by name. Note that it uses a localName, not a qname.

</div>

<div class="method">

Element.setAttribute(name, value)

Set an attribute value from a string.

Raise `InvalidCharacterErr` if the name is not a valid XML name.

</div>

<div class="method">

Element.setAttributeNode(newAttr)

Add a new attribute node to the element, replacing an existing attribute if necessary if the `~Attr.name` attribute matches. If a replacement occurs, the old attribute node will be returned. If *newAttr* is already in use, `InuseAttributeErr` will be raised.

Raise `WrongDocumentErr` if *newAttr* was created by another document.

</div>

<div class="method">

Element.setAttributeNodeNS(newAttr)

Add a new attribute node to the element, replacing an existing attribute if necessary if the `~Node.namespaceURI` and `~Attr.localName` attributes match. If a replacement occurs, the old attribute node will be returned. If *newAttr* is already in use, `InuseAttributeErr` will be raised.

Raise `WrongDocumentErr` if *newAttr* was created by another document.

</div>

<div class="method">

Element.setAttributeNS(namespaceURI, qname, value)

Set an attribute value from a string, given a *namespaceURI* and a *qname*. Note that a qname is the whole attribute name. This is different than above.

Raise `InvalidCharacterErr` if the name is not a valid XML name. Raise `NamespaceErr` if the qualified name is malformed, if it has a prefix and the namespace URI is empty, if the prefix is `'xml'` and the namespace URI is not the XML namespace, or if the name or the prefix is `'xmlns'` and the namespace URI is not the XMLNS namespace, or vice versa.

</div>

### Attr Objects

<div class="Attr" data-no-typesetting="">

`Attr` inherits from `Node`, so inherits all its attributes.

</div>

Attribute nodes are not part of the document tree. They are contained in the `~Node.attributes` map of an element, not in its children, and their `~Node.parentNode`, `~Node.previousSibling` and `~Node.nextSibling` are always `None`.

<div class="attribute">

Attr.name

The attribute name. In a namespace-using document it may include a colon.

</div>

<div class="attribute">

Attr.localName

The part of the name following the colon if there is one, else the entire name. This is a read-only attribute.

</div>

<div class="attribute">

Attr.prefix

The part of the name preceding the colon if there is one, else the empty string.

</div>

<div class="attribute">

Attr.isId

Whether this attribute is of type ID, either because it is declared as such in the DTD or because `Element.setIdAttribute` was used. This is a read-only attribute.

</div>

<div class="attribute">

Attr.ownerElement

The `Element` node to which this attribute belongs, or `None` if it is not used. This is a read-only attribute.

</div>

<div class="attribute">

Attr.specified

Whether the value of the attribute was explicitly set in the document, as opposed to being defaulted from the DTD. This is a read-only attribute.

</div>

<div class="attribute">

Attr.value

The text value of the attribute. This is a synonym for the `~Node.nodeValue` attribute.

</div>

### NamedNodeMap Objects

<div class="NamedNodeMap" data-no-typesetting="">

`NamedNodeMap` does *not* inherit from `Node`.

</div>

<div class="attribute">

NamedNodeMap.length

The length of the attribute list.

</div>

<div class="method">

NamedNodeMap.item(index)

Return an attribute with a particular index. The order you get the attributes in is arbitrary but will be consistent for the life of a DOM. Each item is an attribute node. Get its value with the `~Attr.value` attribute.

</div>

<div class="method">

NamedNodeMap.getNamedItem(name)

Return the node with the given `~Attr.name`, or `None` if there is no such node.

</div>

<div class="method">

NamedNodeMap.getNamedItemNS(namespaceURI, localName)

Return the node with the given namespace URI and local name, or `None` if there is no such node.

</div>

<div class="method">

NamedNodeMap.setNamedItem(node)

Add *node* to the map, using its `~Attr.name` as the key. Return the node which it replaces, or `None` if it replaces no node.

Raise `WrongDocumentErr` if *node* was created by another document, and `InuseAttributeErr` if it belongs to another element.

</div>

<div class="method">

NamedNodeMap.setNamedItemNS(node)

Add *node* to the map, using its namespace URI and local name as the key. Return the node which it replaces, or `None` if it replaces no node.

Raise `WrongDocumentErr` if *node* was created by another document, and `InuseAttributeErr` if it belongs to another element.

</div>

<div class="method">

NamedNodeMap.removeNamedItem(name)

Remove and return the node with the given `~Attr.name`. Raise `NotFoundErr` if there is no such node.

</div>

<div class="method">

NamedNodeMap.removeNamedItemNS(namespaceURI, localName)

Remove and return the node with the given namespace URI and local name. Raise `NotFoundErr` if there is no such node.

</div>

You can also use the standardized `!getAttribute\*` family of methods on the `Element` objects.

### DocumentFragment Objects

<div class="DocumentFragment" data-no-typesetting="">

`DocumentFragment` is a lightweight container of nodes. It is a subclass of `Node`. When it is inserted into the document tree, its children are inserted instead of it, and it becomes empty.

</div>

### CharacterData Objects

<div class="CharacterData" data-no-typesetting="">

`CharacterData` represents text-like data in the XML document. It is a subclass of `Node`, and the base class of `Text`, `CDATASection` and `Comment`. Such nodes cannot have child nodes.

</div>

<div class="attribute">

CharacterData.data

The content of the node as a string.

</div>

<div class="attribute">

CharacterData.length

The number of characters in `~CharacterData.data`. This is a read-only attribute.

</div>

<div class="method">

CharacterData.substringData(offset, count)

Return the substring of `~CharacterData.data` of *count* characters starting at *offset*.

</div>

<div class="method">

CharacterData.appendData(arg)

Append the string *arg* to `~CharacterData.data`.

</div>

<div class="method">

CharacterData.insertData(offset, arg)

Insert the string *arg* into `~CharacterData.data` at *offset*.

</div>

<div class="method">

CharacterData.deleteData(offset, count)

Remove *count* characters from `~CharacterData.data` starting at *offset*.

</div>

<div class="method">

CharacterData.replaceData(offset, count, arg)

Replace *count* characters of `~CharacterData.data` starting at *offset* with the string *arg*.

</div>

### Comment Objects

<div class="Comment" data-no-typesetting="">

`Comment` represents a comment in the XML document. It is a subclass of `CharacterData`.

</div>

<div class="attribute">

Comment.data

The content of the comment as a string. The attribute contains all characters between the leading `<!-``-` and trailing `-``->`, but does not include them.

</div>

### Text and CDATASection Objects

<div class="Text" data-no-typesetting="">

<div class="CDATASection" data-no-typesetting="">

The `Text` interface represents text in the XML document. If the parser and DOM implementation support the DOM's XML extension, portions of the text enclosed in CDATA marked sections are stored in `CDATASection` objects. These two interfaces are identical, but provide different values for the `~Node.nodeType` attribute.

</div>

</div>

`Text` extends the `CharacterData` interface, and `CDATASection` extends `Text`.

<div class="attribute">

Text.data

The content of the text node as a string.

</div>

<div class="attribute">

Text.wholeText

The text of all `Text` nodes logically adjacent to this node, concatenated in document order. This is a read-only attribute.

</div>

<div class="method">

Text.replaceWholeText(content)

Replace the text of all `Text` nodes logically adjacent to this node with *content*, removing the other nodes. Return this node, or `None` if *content* is empty.

</div>

<div class="method">

Text.splitText(offset)

Split this node into two nodes at *offset*, keeping the first part in this node and returning a new sibling node with the rest.

</div>

> [!NOTE]
> The use of a `CDATASection` node does not indicate that the node represents a complete CDATA marked section, only that the content of the node was part of a CDATA section. A single CDATA section may be represented by more than one node in the document tree. There is no way to determine whether two adjacent `CDATASection` nodes represent different CDATA marked sections.

### ProcessingInstruction Objects

<div class="ProcessingInstruction" data-no-typesetting="">

Represents a processing instruction in the XML document; this inherits from the `Node` interface and cannot have child nodes.

</div>

<div class="attribute">

ProcessingInstruction.target

The content of the processing instruction up to the first whitespace character. This is a read-only attribute.

</div>

<div class="attribute">

ProcessingInstruction.data

The content of the processing instruction following the first whitespace character.

</div>

### Entity Objects

<div class="Entity" data-no-typesetting="">

`Entity` represents a parsed or unparsed entity declared in the DTD. It is a subclass of `Node`. Entity nodes are contained in `DocumentType.entities` and cannot be inserted into the document tree. The name of the entity is its `~Node.nodeName`.

</div>

<div class="attribute">

Entity.publicId

The public identifier of the entity, or `None` if it is not specified. This is a read-only attribute.

</div>

<div class="attribute">

Entity.systemId

The system identifier of the entity, or `None` if it is not specified. This is a read-only attribute.

</div>

<div class="attribute">

Entity.notationName

The name of the notation for an unparsed entity, or `None` for a parsed entity. This is a read-only attribute.

</div>

### Notation Objects

<div class="Notation" data-no-typesetting="">

`Notation` represents a notation declared in the DTD. It is a subclass of `Node` and cannot have child nodes. Notation nodes are contained in `DocumentType.notations` and cannot be inserted into the document tree. The name of the notation is its `~Node.nodeName`.

</div>

<div class="attribute">

Notation.publicId

The public identifier of the notation, or `None` if it is not specified. This is a read-only attribute.

</div>

<div class="attribute">

Notation.systemId

The system identifier of the notation, or `None` if it is not specified. This is a read-only attribute.

</div>

### EntityReference Objects

<div class="EntityReference" data-no-typesetting="">

`EntityReference` represents an entity reference in the XML document. It is a subclass of `Node`. The name of the referenced entity is its `~Node.nodeName`. Its children are the replacement text of the entity, and are read-only.

</div>

Parsers may expand entity references, so such a node only occurs in a document if it was created explicitly.

<div class="versionadded">

3.16

</div>

### Exceptions

The DOM Level 2 recommendation defines a single exception, `DOMException`, and a number of constants that allow applications to determine what sort of error occurred. `DOMException` instances carry a `code` attribute that provides the appropriate value for the specific exception.

The Python DOM interface provides the constants, but also expands the set of exceptions so that a specific exception exists for each of the exception codes defined by the DOM. The implementations must raise the appropriate specific exception, each of which carries the appropriate value for the `code` attribute.

<div class="exception">

DOMException

Base exception class used for all specific DOM exceptions. This exception class cannot be directly instantiated.

</div>

<div class="exception">

DomstringSizeErr

Raised when a specified range of text does not fit into a string. This is not known to be used in the Python DOM implementations, but may be received from DOM implementations not written in Python.

</div>

<div class="exception">

HierarchyRequestErr

Raised when an attempt is made to insert a node where the node type is not allowed.

</div>

<div class="exception">

IndexSizeErr

Raised when an index or size parameter to a method is negative or exceeds the allowed values.

</div>

<div class="exception">

InuseAttributeErr

Raised when an attempt is made to insert an `Attr` node that is already present elsewhere in the document.

</div>

<div class="exception">

InvalidAccessErr

Raised if a parameter or an operation is not supported on the underlying object.

</div>

<div class="exception">

InvalidCharacterErr

This exception is raised when a string parameter contains a character that is not permitted in the context it's being used in by the XML 1.0 recommendation. For example, attempting to create an `Element` node with a space in the element type name will cause this error to be raised.

</div>

<div class="exception">

InvalidModificationErr

Raised when an attempt is made to modify the type of a node.

</div>

<div class="exception">

InvalidStateErr

Raised when an attempt is made to use an object that is not defined or is no longer usable.

</div>

<div class="exception">

NamespaceErr

If an attempt is made to change any object in a way that is not permitted with regard to the [Namespaces in XML](https://www.w3.org/TR/REC-xml-names/) recommendation, this exception is raised.

</div>

<div class="exception">

NotFoundErr

Exception when a node does not exist in the referenced context. For example, `NamedNodeMap.removeNamedItem` will raise this if the node passed in does not exist in the map.

</div>

<div class="exception">

NotSupportedErr

Raised when the implementation does not support the requested type of object or operation.

</div>

<div class="exception">

NoDataAllowedErr

This is raised if data is specified for a node which does not support data.

</div>

<div class="exception">

NoModificationAllowedErr

Raised on attempts to modify an object where modifications are not allowed (such as for read-only nodes).

</div>

<div class="exception">

SyntaxErr

Raised when an invalid or illegal string is specified.

</div>

<div class="exception">

ValidationErr

Raised when an operation would make the document invalid with respect to partial validity. This is not known to be used in the Python DOM implementations, but may be received from DOM implementations not written in Python.

</div>

<div class="exception">

WrongDocumentErr

Raised when a node is inserted in a different document than it currently belongs to, and the implementation does not support migrating the node from one document to the other.

</div>

The exception codes defined in the DOM recommendation map to the exceptions described above according to this table:

<table>
<colgroup>
<col style="width: 54%" />
<col style="width: 45%" />
</colgroup>
<thead>
<tr>
<th><p>Constant</p></th>
<th><p>Exception</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><div class="data">
<p>DOMSTRING_SIZE_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">DomstringSizeErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>HIERARCHY_REQUEST_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">HierarchyRequestErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>INDEX_SIZE_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">IndexSizeErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>INUSE_ATTRIBUTE_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">InuseAttributeErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>INVALID_ACCESS_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">InvalidAccessErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>INVALID_CHARACTER_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">InvalidCharacterErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>INVALID_MODIFICATION_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">InvalidModificationErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>INVALID_STATE_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">InvalidStateErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>NAMESPACE_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">NamespaceErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>NOT_FOUND_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">NotFoundErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>NOT_SUPPORTED_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">NotSupportedErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>NO_DATA_ALLOWED_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">NoDataAllowedErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>NO_MODIFICATION_ALLOWED_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">NoModificationAllowedErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>SYNTAX_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">SyntaxErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>VALIDATION_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">ValidationErr</code></p></td>
</tr>
<tr>
<td><div class="data">
<p>WRONG_DOCUMENT_ERR</p>
</div></td>
<td><p><code class="interpreted-text" role="exc">WrongDocumentErr</code></p></td>
</tr>
</tbody>
</table>

## Conformance

This section describes the conformance requirements and relationships between the Python DOM API, the W3C DOM recommendations, and the OMG IDL mapping for Python.

### Type Mapping

The IDL types used in the DOM specification are mapped to Python types according to the following table.

| IDL Type       | Python Type      |
|----------------|------------------|
| `boolean`      | `bool` or `int`  |
| `int`          | `int`            |
| `long int`     | `int`            |
| `unsigned int` | `int`            |
| `DOMString`    | `str` or `bytes` |
| `null`         | `None`           |

### Accessor Methods

The mapping from OMG IDL to Python defines accessor functions for IDL `attribute` declarations in much the way the Java mapping does. Mapping the IDL declarations :

    readonly attribute string someValue;
             attribute string anotherValue;

yields three accessor functions: a "get" method for `!someValue` (`!_get_someValue`), and "get" and "set" methods for `!anotherValue` (`!_get_anotherValue` and `!_set_anotherValue`). The mapping, in particular, does not require that the IDL attributes are accessible as normal Python attributes: `object.someValue` is *not* required to work, and may raise an `AttributeError`.

The Python DOM API, however, *does* require that normal attribute access work. This means that the typical surrogates generated by Python IDL compilers are not likely to work, and wrapper objects may be needed on the client if the DOM objects are accessed via CORBA. While this does require some additional consideration for CORBA DOM clients, the implementers with experience using DOM over CORBA from Python do not consider this a problem. Attributes that are declared `readonly` may not restrict write access in all DOM implementations.

In the Python DOM API, accessor functions are not required. If provided, they should take the form defined by the Python IDL mapping, but these methods are considered unnecessary since the attributes are accessible directly from Python. "Set" accessors should never be provided for `readonly` attributes.

The IDL definitions do not fully embody the requirements of the W3C DOM API, such as the notion of certain objects, such as the return value of `~Element.getElementsByTagName`, being "live". The Python DOM API does not require implementations to enforce such requirements.

# `!xml.dom.minidom` --- Minimal DOM implementation

<div class="module" data-synopsis="Minimal Document Object Model (DOM) implementation.">

xml.dom.minidom

</div>

**Source code:** `Lib/xml/dom/minidom.py`

------------------------------------------------------------------------

`!xml.dom.minidom` is a minimal implementation of the Document Object Model interface, with an API similar to that in other languages. It is intended to be simpler than the full DOM and also significantly smaller. Users who are not already proficient with the DOM should consider using the `xml.etree.ElementTree` module for their XML processing instead.

> [!NOTE]
> If you need to parse untrusted or unauthenticated data, see `xml-security`.

DOM applications typically start by parsing some XML into a DOM. With `!xml.dom.minidom`, this is done through the parse functions:

    from xml.dom.minidom import parse, parseString

    dom1 = parse('c:\\temp\\mydata.xml')  # parse an XML file by name

    datasource = open('c:\\temp\\mydata.xml')
    dom2 = parse(datasource)  # parse an open file

    dom3 = parseString('<myxml>Some data<empty/> some more data</myxml>')

The `parse` function can take either a filename or an open file object.

<div class="function">

parse(filename_or_file, parser=None, bufsize=None)

Return a `Document` from the given input. *filename_or_file* may be either a file name, or a file-like object. *parser*, if given, must be a SAX2 parser object. This function will change the document handler of the parser and activate namespace support; other parser configuration (like setting an entity resolver) must have been done in advance.

</div>

If you have XML in a string, you can use the `parseString` function instead:

<div class="function">

parseString(string, parser=None)

Return a `Document` that represents the *string*. This method creates an `io.StringIO` object for the string and passes that on to `parse`.

</div>

Both functions return a `Document` object representing the content of the document.

What the `parse` and `parseString` functions do is connect an XML parser with a "DOM builder" that can accept parse events from any SAX parser and convert them into a DOM tree. The names of the functions are perhaps misleading, but are easy to grasp when learning the interfaces. The parsing of the document will be completed before these functions return; it's simply that these functions do not provide a parser implementation themselves.

You can also create a `Document` by calling a method on a "DOM Implementation" object. You can get this object either by calling the `getDOMImplementation` function in the `xml.dom` package or the `!xml.dom.minidom` module. Once you have a `Document`, you can add child nodes to it to populate the DOM:

    from xml.dom.minidom import getDOMImplementation

    impl = getDOMImplementation()

    newdoc = impl.createDocument(None, "some_tag", None)
    top_element = newdoc.documentElement
    text = newdoc.createTextNode('Some textual content.')
    top_element.appendChild(text)

Once you have a DOM document object, you can access the parts of your XML document through its properties and methods. These properties are defined in the DOM specification. The main property of the document object is the `documentElement` property. It gives you the main element in the XML document: the one that holds all others. Here is an example program:

    dom3 = parseString("<myxml>Some data</myxml>")
    assert dom3.documentElement.tagName == "myxml"

When you are finished with a DOM tree, you may optionally call the `unlink` method to encourage early cleanup of the now-unneeded objects. `unlink` is an `!xml.dom.minidom`-specific extension to the DOM API that renders the node and its descendants essentially useless. Otherwise, Python's garbage collector will eventually take care of the objects in the tree.

<div class="seealso">

[Document Object Model (DOM) Level 1 Specification](https://www.w3.org/TR/REC-DOM-Level-1/)  
The W3C recommendation for the DOM supported by `!xml.dom.minidom`.

</div>

## DOM Objects

The definition of the DOM API for Python is given as part of the `xml.dom` module documentation. This section lists the differences between the API and `!xml.dom.minidom`.

<div class="method">

Node.unlink()

Break internal references within the DOM so that it will be garbage collected on versions of Python without cyclic GC. Even when cyclic GC is available, using this can make large amounts of memory available sooner, so calling this on DOM objects as soon as they are no longer needed is good practice. This only needs to be called on the `Document` object, but may be called on child nodes to discard children of that node.

You can avoid calling this method explicitly by using the `with` statement. The following code will automatically unlink *dom* when the `!with` block is exited:

    with xml.dom.minidom.parse(datasource) as dom:
        ... # Work with dom.

</div>

<div class="method">

Node.writexml(writer, indent="", addindent="", newl="", encoding=None, standalone=None)

Write XML to the writer object. The writer receives texts but not bytes as input, it should have a `write` method which matches that of the file object interface. The *indent* parameter is the indentation of the current node. The *addindent* parameter is the incremental indentation to use for subnodes of the current one. The *newl* parameter specifies the string to use to terminate newlines.

For the `Document` node, an additional keyword argument *encoding* can be used to specify the encoding field of the XML header.

Similarly, explicitly stating the *standalone* argument causes the standalone document declarations to be added to the prologue of the XML document. If the value is set to `True`, `standalone="yes"` is added, otherwise it is set to `"no"`. Not stating the argument will omit the declaration from the document.

<div class="versionchanged">

3.8 The `writexml` method now preserves the attribute order specified by the user.

</div>

<div class="versionchanged">

3.9 The *standalone* parameter was added.

</div>

<div class="versionchanged">

3.16 Namespace declarations missing for the serialized element and its attributes are now written. It now works for `!DocumentFragment` nodes.

</div>

</div>

<div class="method">

Node.toxml(encoding=None, standalone=None)

Return a string or byte string containing the XML represented by the DOM node.

With an explicit *encoding*[^1] argument, the result is a byte string in the specified encoding. With no *encoding* argument, the result is a Unicode string, and the XML declaration in the resulting string does not specify an encoding. Encoding this string in an encoding other than UTF-8 is likely incorrect, since UTF-8 is the default encoding of XML.

The *standalone* argument behaves exactly as in `writexml`.

<div class="versionchanged">

3.8 The `toxml` method now preserves the attribute order specified by the user.

</div>

<div class="versionchanged">

3.9 The *standalone* parameter was added.

</div>

<div class="versionchanged">

3.16 It now works for `!DocumentFragment` nodes.

</div>

</div>

<div class="method">

Node.toprettyxml(indent="t", newl="n", encoding=None, standalone=None)

Return a pretty-printed version of the document. *indent* specifies the indentation string and defaults to a tabulator; *newl* specifies the string emitted at the end of each line and defaults to `\n`.

The *encoding* argument behaves like the corresponding argument of `toxml`.

The *standalone* argument behaves exactly as in `writexml`.

No indentation is added inside an element which is marked with `xml:space="preserve"`, which is declared in the DTD as not having element content, or, in absence of such declaration, which contains text, because this would change its content.

<div class="versionchanged">

3.8 The `toprettyxml` method now preserves the attribute order specified by the user.

</div>

<div class="versionchanged">

3.9 The *standalone* parameter was added.

</div>

<div class="versionchanged">

3.16 Whitespace is no longer added inside an element with mixed content or marked with `xml:space="preserve"`. It now works for `!DocumentFragment` nodes.

</div>

</div>

## DOM Example

This example program is a fairly realistic example of a simple program. In this particular case, we do not take much advantage of the flexibility of the DOM.

<div class="literalinclude">

../includes/minidom-example.py

</div>

## minidom and the DOM standard

The `!xml.dom.minidom` module is essentially a DOM 1.0-compatible DOM with some DOM 2 features (primarily namespace features).

Usage of the DOM interface in Python is straight-forward. The following mapping rules apply:

- Interfaces are accessed through instance objects. Applications should not instantiate the classes themselves; they should use the creator functions available on the `Document` object. Derived interfaces support all operations (and attributes) from the base interfaces, plus any new operations.

- Operations are used as methods. Since the DOM uses only `in` parameters, the arguments are passed in normal order (from left to right). There are no optional arguments. `void` operations return `None`.

- IDL attributes map to instance attributes. For compatibility with the OMG IDL language mapping for Python, an attribute `foo` can also be accessed through accessor methods `_get_foo` and `_set_foo`. `readonly` attributes must not be changed; this is not enforced at runtime.

- The types `short int`, `unsigned int`, `unsigned long long`, and `boolean` all map to Python integer objects.

- The type `DOMString` maps to Python strings. `!xml.dom.minidom` supports either bytes or strings, but will normally produce strings. Values of type `DOMString` may also be `None` where allowed to have the IDL `null` value by the DOM specification from the W3C.

- `const` declarations map to variables in their respective scope (e.g. `xml.dom.minidom.Node.PROCESSING_INSTRUCTION_NODE`); they must not be changed.

- `DOMException` is currently not supported in `!xml.dom.minidom`. Instead, `!xml.dom.minidom` uses standard Python exceptions such as `TypeError` and `AttributeError`.

- Each of the `~xml.dom.NodeList` and `~xml.dom.NamedNodeMap` interfaces has two implementations, which provide additional methods and operations.

  `~xml.dom.Node.childNodes` is a subclass of `list`, or, for nodes which cannot have children, a subclass of `tuple`. It supports iteration, concatenation, indexing and slicing.

  `~xml.dom.Node.attributes` supports `len()`, the `in` operator, subscription by a name or by a `(namespaceURI, localName)` tuple, assignment and deletion, and the methods `!get`, `!keys`, `!keysNS`, `!values`, `!items` and `!itemsNS`. `~xml.dom.DocumentType.entities` and `~xml.dom.DocumentType.notations` are read-only and support only `len()` and subscription by a name.

- `~xml.dom.Document.strictErrorChecking` is always `False`.

  <div class="versionchanged">

  3.16 Previously, `~xml.dom.Attr.specified` was always `False`.

  </div>

- The constraints of the DOM are now enforced, and the corresponding exceptions are raised.

  <div class="versionchanged">

  3.16 Previously, many invalid operations silently succeeded and produced an invalid document, but removing an absent attribute raised `~xml.dom.NotFoundErr`.

  </div>

  <div class="versionchanged">

  3.16 Namespaces are now validated in the factory methods and when setting `~xml.dom.Node.prefix` of an attribute.

  </div>

The following interfaces have no implementation in `!xml.dom.minidom`:

- `DOMTimeStamp`

This reflects information in the XML document that is not of general utility to most DOM users.

<div class="versionchanged">

3.16 `~xml.dom.EntityReference` is now implemented. Note that the parser expands entity references, so they only occur in a document if created explicitly.

</div>

**Footnotes**

[^1]: The encoding name included in the XML output should conform to the appropriate standards. For example, "UTF-8" is valid, but "UTF8" is not valid in an XML document's declaration, even though Python accepts it as an encoding name. See <https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl> and <https://www.iana.org/assignments/character-sets/character-sets.xhtml>.

# `!xml.dom.pulldom` --- Support for building partial DOM trees

<div class="module" data-synopsis="Support for building partial DOM trees from SAX events.">

xml.dom.pulldom

</div>

**Source code:** `Lib/xml/dom/pulldom.py`

------------------------------------------------------------------------

The `!xml.dom.pulldom` module provides a "pull parser" which can also be asked to produce DOM-accessible fragments of the document where necessary. The basic concept involves pulling "events" from a stream of incoming XML and processing them. In contrast to SAX which also employs an event-driven processing model together with callbacks, the user of a pull parser is responsible for explicitly pulling events from the stream, looping over those events until either processing is finished or an error condition occurs.

> [!NOTE]
> If you need to parse untrusted or unauthenticated data, see `xml-security`.

<div class="versionchanged">

3.7.1

The SAX parser no longer processes general external entities by default to increase security by default. To enable processing of external entities, pass a custom parser instance in:

    from xml.dom.pulldom import parse
    from xml.sax import make_parser
    from xml.sax.handler import feature_external_ges

    parser = make_parser()
    parser.setFeature(feature_external_ges, True)
    parse(filename, parser=parser)

</div>

Example:

    from xml.dom import pulldom

    doc = pulldom.parse('sales_items.xml')
    for event, node in doc:
        if event == pulldom.START_ELEMENT and node.tagName == 'item':
            if int(node.getAttribute('price')) > 50:
                doc.expandNode(node)
                print(node.toxml())

`event` is one of the following constants, and `node` is the node which the event is about. The nodes implement the `xml.dom` interfaces; they are created by the DOM implementation given to `PullDOM`, which is `xml.dom.minidom` by default.

<div class="data">

START_DOCUMENT END_DOCUMENT

The start and the end of the document. *node* is the `~xml.dom.Document`.

</div>

<div class="data">

START_ELEMENT END_ELEMENT

The start tag and the end tag of an element. *node* is the `~xml.dom.Element`.

</div>

<div class="data">

CHARACTERS

Character data. *node* is the `~xml.dom.Text` node.

</div>

<div class="data">

IGNORABLE_WHITESPACE

White space in element content, as declared in the DTD. *node* is the `~xml.dom.Text` node.

</div>

<div class="data">

COMMENT

A comment. *node* is the `~xml.dom.Comment` node.

</div>

<div class="data">

PROCESSING_INSTRUCTION

A processing instruction. *node* is the `~xml.dom.ProcessingInstruction` node.

</div>

Since the document is treated as a "flat" stream of events, the document "tree" is implicitly traversed and the desired elements are found regardless of their depth in the tree. In other words, one does not need to consider hierarchical issues such as recursive searching of the document nodes, although if the context of elements were important, one would either need to maintain some context-related state (i.e. remembering where one is in the document at any given point) or to make use of the `DOMEventStream.expandNode` method and switch to DOM-related processing.

<div class="PullDOM(documentFactory=None)">

Subclass of `xml.sax.handler.ContentHandler` which turns SAX events into the events of the pull parser. The nodes are created, but they are not added to the tree, unless `~DOMEventStream.expandNode` is called. *documentFactory*, if given, is a DOM implementation used to create the document; by default the implementation of `xml.dom.minidom` is used.

</div>

<div class="SAX2DOM(documentFactory=None)">

Subclass of `PullDOM` which also adds every created node to the tree, so that the complete document is built.

</div>

<div class="function">

parse(stream_or_string, parser=None, bufsize=None)

Return a `DOMEventStream` from the given input. *stream_or_string* may be either a file name, or a file-like object. *parser*, if given, must be an `~xml.sax.xmlreader.XMLReader` object. This function will change the document handler of the parser and activate namespace support; other parser configuration (like setting an entity resolver) must have been done in advance.

</div>

If you have XML in a string, you can use the `parseString` function instead:

<div class="function">

parseString(string, parser=None)

Return a `DOMEventStream` that represents the *string*. *string* must be a `str` instance; to parse bytes, pass a binary file object to `parse`.

</div>

<div class="data">

default_bufsize

Default value for the *bufsize* parameter to `parse`.

The value of this variable can be changed before calling `parse` and the new value will take effect.

</div>

## DOMEventStream Objects

<div class="DOMEventStream(stream, parser, bufsize)">

Produce the events for the data read from the file object *stream* by the `~xml.sax.xmlreader.XMLReader` *parser*. The data is read by *bufsize* bytes, or characters for a text stream, at a time.

<div class="versionchanged">

3.11 Support for `~object.__getitem__` method has been removed.

</div>

<div class="method">

getEvent()

Return the next `(event, node)` tuple, or `None` at the end of the document. See above for the events and the corresponding nodes. The current node does not contain information about its children, unless `expandNode` is called.

</div>

<div class="method">

expandNode(node)

Expands all children of *node* into *node*. Example:

    from xml.dom import pulldom

    xml = '<html><title>Foo</title> <p>Some text <div>and more</div></p> </html>'
    doc = pulldom.parseString(xml)
    for event, node in doc:
        if event == pulldom.START_ELEMENT and node.tagName == 'p':
            # Following statement only prints '<p/>'
            print(node.toxml())
            doc.expandNode(node)
            # Following statement prints node with all its children '<p>Some text <div>and more</div></p>'
            print(node.toxml())

</div>

<div class="method">

reset()

Discard the events which are not read yet and prepare the object for parsing a new document.

</div>

<div class="method">

clear()

Release the parser and the document. The stream is not closed, and the object can no longer be used.

</div>

</div>

# `!xml.etree.ElementTree` --- The ElementTree XML API

<div class="module" data-synopsis="Implementation of the ElementTree API.">

xml.etree.ElementTree

</div>

**Source code:** `Lib/xml/etree/ElementTree.py`

------------------------------------------------------------------------

The `!xml.etree.ElementTree` module implements a simple and efficient API for parsing and creating XML data.

<div class="versionchanged">

3.3 This module will use a fast implementation whenever available.

</div>

<div class="deprecated">

3.3 The `!xml.etree.cElementTree` alias of this module is deprecated.

</div>

> [!NOTE]
> If you need to parse untrusted or unauthenticated data, see `xml-security`.

## Tutorial

This is a short tutorial for using `!xml.etree.ElementTree` (`ET` in short). The goal is to demonstrate some of the building blocks and basic concepts of the module.

### XML tree and elements

XML is an inherently hierarchical data format, and the most natural way to represent it is with a tree. `ET` has two classes for this purpose - `ElementTree` represents the whole XML document as a tree, and `Element` represents a single node in this tree. Interactions with the whole document (reading and writing to/from files) are usually done on the `ElementTree` level. Interactions with a single XML element and its sub-elements are done on the `Element` level.

### Parsing XML

We'll be using the fictive `country_data.xml` XML document as the sample data for this section:

``` xml
<?xml version="1.0"?>
<data>
    <country name="Liechtenstein">
        <rank>1</rank>
        <year>2008</year>
        <gdppc>141100</gdppc>
        <neighbor name="Austria" direction="E"/>
        <neighbor name="Switzerland" direction="W"/>
    </country>
    <country name="Singapore">
        <rank>4</rank>
        <year>2011</year>
        <gdppc>59900</gdppc>
        <neighbor name="Malaysia" direction="N"/>
    </country>
    <country name="Panama">
        <rank>68</rank>
        <year>2011</year>
        <gdppc>13600</gdppc>
        <neighbor name="Costa Rica" direction="W"/>
        <neighbor name="Colombia" direction="E"/>
    </country>
</data>
```

We can import this data by reading from a file:

    import xml.etree.ElementTree as ET
    tree = ET.parse('country_data.xml')
    root = tree.getroot()

Or directly from a string:

    root = ET.fromstring(country_data_as_string)

`fromstring` parses XML from a string directly into an `Element`, which is the root element of the parsed tree. Other parsing functions may create an `ElementTree`. Check the documentation to be sure.

As an `Element`, `root` has a tag and a dictionary of attributes:

    >>> root.tag
    'data'
    >>> root.attrib
    {}

It also has children nodes over which we can iterate:

    >>> for child in root:
    ...     print(child.tag, child.attrib)
    ...
    country {'name': 'Liechtenstein'}
    country {'name': 'Singapore'}
    country {'name': 'Panama'}

Children are nested, and we can access specific child nodes by index:

    >>> root[0][1].text
    '2008'

> [!NOTE]
> Not all elements of the XML input will end up as elements of the parsed tree. Currently, this module skips over any XML comments, processing instructions, and document type declarations in the input. Nevertheless, trees built using this module's API rather than parsing from XML text can have comments and processing instructions in them; they will be included when generating XML output. A document type declaration may be accessed by passing a custom `TreeBuilder` instance to the `XMLParser` constructor.

### Pull API for non-blocking parsing

Most parsing functions provided by this module require the whole document to be read at once before returning any result. It is possible to use an `XMLParser` and feed data into it incrementally, but it is a push API that calls methods on a callback target, which is too low-level and inconvenient for most needs. Sometimes what the user really wants is to be able to parse XML incrementally, without blocking operations, while enjoying the convenience of fully constructed `Element` objects.

The most powerful tool for doing this is `XMLPullParser`. It does not require a blocking read to obtain the XML data, and is instead fed with data incrementally with `XMLPullParser.feed` calls. To get the parsed XML elements, call `XMLPullParser.read_events`. Here is an example:

    >>> parser = ET.XMLPullParser(['start', 'end'])
    >>> parser.feed('<mytag>sometext')
    >>> list(parser.read_events())
    [('start', <Element 'mytag' at 0x7fa66db2be58>)]
    >>> parser.feed(' more text</mytag>')
    >>> for event, elem in parser.read_events():
    ...     print(event)
    ...     print(elem.tag, 'text=', elem.text)
    ...
    end
    mytag text= sometext more text

The obvious use case is applications that operate in a non-blocking fashion where the XML data is being received from a socket or read incrementally from some storage device. In such cases, blocking reads are unacceptable.

Because it's so flexible, `XMLPullParser` can be inconvenient to use for simpler use-cases. If you don't mind your application blocking on reading XML data but would still like to have incremental parsing capabilities, take a look at `iterparse`.

Note that both parsers build the tree incrementally: it is not freed incrementally, so every parsed element is kept until the whole document is read. To keep the memory usage low, get rid of the data which is not needed any more.

If the processed elements are large, it is enough to clear them. This works wherever they are in the tree, but the emptied elements are left in it:

    for event, elem in ET.iterparse(source):
        if elem.tag == 'record':
            process(elem)
            elem.clear()

If an element has a large number of children, remove the processed children from it:

    for event, elem in ET.iterparse(source, events=('start', 'end')):
        if event == 'start' and elem.tag == 'parent':
            parent = elem
        elif event == 'end' and elem.tag == 'child':
            process(elem)
            parent.remove(elem)

These examples are not universal, they only give an idea for two common cases. If you do not need a tree at all, parse with `XMLParser` and a custom target instead; it is not built then, and nothing has to be removed.

Where *immediate* feedback through events is wanted, calling method `XMLPullParser.flush` can help reduce delay; please make sure to study the related security notes.

### Finding interesting elements

`Element` has some useful methods that help iterate recursively over all the sub-tree below it (its children, their children, and so on). For example, `Element.iter`:

    >>> for neighbor in root.iter('neighbor'):
    ...     print(neighbor.attrib)
    ...
    {'name': 'Austria', 'direction': 'E'}
    {'name': 'Switzerland', 'direction': 'W'}
    {'name': 'Malaysia', 'direction': 'N'}
    {'name': 'Costa Rica', 'direction': 'W'}
    {'name': 'Colombia', 'direction': 'E'}

`Element.findall` finds only elements with a tag which are direct children of the current element. `Element.find` finds the *first* child with a particular tag, and `Element.text` accesses the element's text content. `Element.get` accesses the element's attributes:

    >>> for country in root.findall('country'):
    ...     rank = country.find('rank').text
    ...     name = country.get('name')
    ...     print(name, rank)
    ...
    Liechtenstein 1
    Singapore 4
    Panama 68

More sophisticated specification of which elements to look for is possible by using `XPath <elementtree-xpath>`.

### Modifying an XML File

`ElementTree` provides a simple way to build XML documents and write them to files. The `ElementTree.write` method serves this purpose.

Once created, an `Element` object may be manipulated by directly changing its fields (such as `Element.text`), adding and modifying attributes (`Element.set` method), as well as adding new children (for example with `Element.append`).

Let's say we want to add one to each country's rank, and add an `updated` attribute to the rank element:

    >>> for rank in root.iter('rank'):
    ...     new_rank = int(rank.text) + 1
    ...     rank.text = str(new_rank)
    ...     rank.set('updated', 'yes')
    ...
    >>> tree.write('output.xml')

Our XML now looks like this:

``` xml
<?xml version="1.0"?>
<data>
    <country name="Liechtenstein">
        <rank updated="yes">2</rank>
        <year>2008</year>
        <gdppc>141100</gdppc>
        <neighbor name="Austria" direction="E"/>
        <neighbor name="Switzerland" direction="W"/>
    </country>
    <country name="Singapore">
        <rank updated="yes">5</rank>
        <year>2011</year>
        <gdppc>59900</gdppc>
        <neighbor name="Malaysia" direction="N"/>
    </country>
    <country name="Panama">
        <rank updated="yes">69</rank>
        <year>2011</year>
        <gdppc>13600</gdppc>
        <neighbor name="Costa Rica" direction="W"/>
        <neighbor name="Colombia" direction="E"/>
    </country>
</data>
```

We can remove elements using `Element.remove`. Let's say we want to remove all countries with a rank higher than 50:

    >>> for country in root.findall('country'):
    ...     # using root.findall() to avoid removal during traversal
    ...     rank = int(country.find('rank').text)
    ...     if rank > 50:
    ...         root.remove(country)
    ...
    >>> tree.write('output.xml')

Note that concurrent modification while iterating can lead to problems, just like when iterating and modifying Python lists or dicts. Therefore, the example first collects all matching elements with `root.findall()`, and only then iterates over the list of matches.

Our XML now looks like this:

``` xml
<?xml version="1.0"?>
<data>
    <country name="Liechtenstein">
        <rank updated="yes">2</rank>
        <year>2008</year>
        <gdppc>141100</gdppc>
        <neighbor name="Austria" direction="E"/>
        <neighbor name="Switzerland" direction="W"/>
    </country>
    <country name="Singapore">
        <rank updated="yes">5</rank>
        <year>2011</year>
        <gdppc>59900</gdppc>
        <neighbor name="Malaysia" direction="N"/>
    </country>
</data>
```

### Building XML documents

The `SubElement` function also provides a convenient way to create new sub-elements for a given element:

    >>> a = ET.Element('a')
    >>> b = ET.SubElement(a, 'b')
    >>> c = ET.SubElement(a, 'c')
    >>> d = ET.SubElement(c, 'd')
    >>> ET.dump(a)
    <a><b /><c><d /></c></a>

### Parsing XML with Namespaces

If the XML input has [namespaces](https://en.wikipedia.org/wiki/XML_namespace), tags and attributes with prefixes in the form `prefix:sometag` get expanded to `{uri}sometag` where the *prefix* is replaced by the full *URI*. Also, if there is a [default namespace](https://www.w3.org/TR/xml-names/#defaulting), that full URI gets prepended to all of the non-prefixed tags.

Here is an XML example that incorporates two namespaces, one with the prefix "fictional" and the other serving as the default namespace:

``` xml
<?xml version="1.0"?>
<actors xmlns:fictional="http://characters.example.com"
        xmlns="http://people.example.com">
    <actor>
        <name>John Cleese</name>
        <fictional:character>Lancelot</fictional:character>
        <fictional:character>Archie Leach</fictional:character>
    </actor>
    <actor>
        <name>Eric Idle</name>
        <fictional:character>Sir Robin</fictional:character>
        <fictional:character>Gunther</fictional:character>
        <fictional:character>Commander Clement</fictional:character>
    </actor>
</actors>
```

One way to search and explore this XML example is to manually add the URI to every tag or attribute in the xpath of a `~Element.find` or `~Element.findall`:

    root = fromstring(xml_text)
    for actor in root.findall('{http://people.example.com}actor'):
        name = actor.find('{http://people.example.com}name')
        print(name.text)
        for char in actor.findall('{http://characters.example.com}character'):
            print(' |-->', char.text)

A better way to search the namespaced XML example is to create a dictionary with your own prefixes and use those in the search functions:

    ns = {'real_person': 'http://people.example.com',
          'role': 'http://characters.example.com'}

    for actor in root.findall('real_person:actor', ns):
        name = actor.find('real_person:name', ns)
        print(name.text)
        for char in actor.findall('role:character', ns):
            print(' |-->', char.text)

These two approaches both output:

    John Cleese
     |--> Lancelot
     |--> Archie Leach
    Eric Idle
     |--> Sir Robin
     |--> Gunther
     |--> Commander Clement

## XPath support

This module provides limited support for [XPath expressions](https://www.w3.org/TR/xpath) for locating elements in a tree. The goal is to support a small subset of the abbreviated syntax; a full XPath engine is outside the scope of the module.

### Example

Here's an example that demonstrates some of the XPath capabilities of the module. We'll be using the `countrydata` XML document from the `Parsing XML <elementtree-parsing-xml>` section:

    import xml.etree.ElementTree as ET

    root = ET.fromstring(countrydata)

    # Top-level elements
    root.findall(".")

    # All 'neighbor' grand-children of 'country' children of the top-level
    # elements
    root.findall("./country/neighbor")

    # Nodes with name='Singapore' that have a 'year' child
    root.findall(".//year/..[@name='Singapore']")

    # 'year' nodes that are children of nodes with name='Singapore'
    root.findall(".//*[@name='Singapore']/year")

    # All 'neighbor' nodes that are the second child of their parent
    root.findall(".//neighbor[2]")

For XML with namespaces, use the usual qualified `{namespace}tag` notation:

    # All dublin-core "title" tags in the document
    root.findall(".//{http://purl.org/dc/elements/1.1/}title")

### Supported XPath syntax

<div class="tabularcolumns">

\|l\|L\|

</div>

<table>
<colgroup>
<col style="width: 30%" />
<col style="width: 69%" />
</colgroup>
<thead>
<tr>
<th><p>Syntax</p></th>
<th><p>Meaning</p></th>
</tr>
</thead>
<tbody>
<tr>
<td><p><code>tag</code></p></td>
<td><p>Selects all child elements with the given tag. For example, <code>spam</code> selects all child elements named <code>spam</code>, and <code>spam/egg</code> selects all grandchildren named <code>egg</code> in all children named <code>spam</code>. <code>{namespace}*</code> selects all tags in the given namespace, <code>{*}spam</code> selects tags named <code>spam</code> in any (or no) namespace, and <code>{}*</code> only selects tags that are not in a namespace.</p>
<div class="versionchanged">
<p>3.8 Support for star-wildcards was added.</p>
</div></td>
</tr>
<tr>
<td><p><code>*</code></p></td>
<td><p>Selects all child elements, including comments and processing instructions. For example, <code>*/egg</code> selects all grandchildren named <code>egg</code>.</p></td>
</tr>
<tr>
<td><p><code>.</code></p></td>
<td><p>Selects the current node. This is mostly useful at the beginning of the path, to indicate that it's a relative path.</p></td>
</tr>
<tr>
<td><p><code>//</code></p></td>
<td><p>Selects all subelements, on all levels beneath the current element. For example, <code>.//egg</code> selects all <code>egg</code> elements in the entire tree.</p></td>
</tr>
<tr>
<td><p><code>..</code></p></td>
<td><p>Selects the parent element. Returns <code>None</code> if the path attempts to reach the ancestors of the start element (the element <code>find</code> was called on).</p></td>
</tr>
<tr>
<td><p><code>[@attrib]</code></p></td>
<td><p>Selects all elements that have the given attribute.</p></td>
</tr>
<tr>
<td><p><code>[@attrib='value']</code></p></td>
<td><p>Selects all elements for which the given attribute has the given value. The value cannot contain quotes.</p></td>
</tr>
<tr>
<td><p><code>[@attrib!='value']</code></p></td>
<td><p>Selects all elements for which the given attribute does not have the given value. The value cannot contain quotes.</p>
<div class="versionadded">
<p>3.10</p>
</div></td>
</tr>
<tr>
<td><p><code>[tag]</code></p></td>
<td><p>Selects all elements that have a child named <code>tag</code>. Only immediate children are supported.</p></td>
</tr>
<tr>
<td><p><code>[.='text']</code></p></td>
<td><p>Selects all elements whose complete text content, including descendants, equals the given <code>text</code>.</p>
<div class="versionadded">
<p>3.7</p>
</div></td>
</tr>
<tr>
<td><p><code>[.!='text']</code></p></td>
<td><p>Selects all elements whose complete text content, including descendants, does not equal the given <code>text</code>.</p>
<div class="versionadded">
<p>3.10</p>
</div></td>
</tr>
<tr>
<td><p><code>[tag='text']</code></p></td>
<td><p>Selects all elements that have a child named <code>tag</code> whose complete text content, including descendants, equals the given <code>text</code>.</p></td>
</tr>
<tr>
<td><p><code>[tag!='text']</code></p></td>
<td><p>Selects all elements that have a child named <code>tag</code> whose complete text content, including descendants, does not equal the given <code>text</code>.</p>
<div class="versionadded">
<p>3.10</p>
</div></td>
</tr>
<tr>
<td><p><code>[position]</code></p></td>
<td><p>Selects all elements that are located at the given position. The position can be either an integer (1 is the first position), the expression <code>last()</code> (for the last position), or a position relative to the last position (e.g. <code>last()-1</code>).</p></td>
</tr>
</tbody>
</table>

Predicates (expressions within square brackets) must be preceded by a tag name, an asterisk, or another predicate. `position` predicates must be preceded by a tag name.

## Reference

### Functions

<div class="function">

canonicalize(xml_data=None, *, out=None, from_file=None,*\*options)

[C14N 2.0](https://www.w3.org/TR/xml-c14n2/) transformation function.

Canonicalization is a way to normalise XML output in a way that allows byte-by-byte comparisons and digital signatures. It reduces the freedom that XML serializers have and instead generates a more constrained XML representation. The main restrictions regard the placement of namespace declarations, the ordering of attributes, and ignorable whitespace.

This function takes an XML data string (*xml_data*) or a file path or file-like object (*from_file*) as input, converts it to the canonical form, and writes it out using the *out* file(-like) object, if provided, or returns it as a text string if not. The output file receives text, not bytes. It should therefore be opened in text mode with `utf-8` encoding.

Typical uses:

    xml_data = "<root>...</root>"
    print(canonicalize(xml_data))

    with open("c14n_output.xml", mode='w', encoding='utf-8') as out_file:
        canonicalize(xml_data, out=out_file)

    with open("c14n_output.xml", mode='w', encoding='utf-8') as out_file:
        canonicalize(from_file="inputfile.xml", out=out_file)

The configuration *options* are as follows:

- *with_comments*: set to true to include comments (default: false)
- *strip_text*: set to true to strip whitespace before and after text content (default: false)
- *rewrite_prefixes*: set to true to replace namespace prefixes by "n{number}" (default: false)
- *qname_aware_tags*: a set of qname aware tag names in which prefixes should be replaced in text content (default: empty)
- *qname_aware_attrs*: a set of qname aware attribute names in which prefixes should be replaced in text content (default: empty)
- *exclude_attrs*: a set of attribute names that should not be serialised
- *exclude_tags*: a set of tag names that should not be serialised

In the option list above, "a set" refers to any collection or iterable of strings, no ordering is expected.

<div class="versionadded">

3.8

</div>

</div>

<div class="function">

Comment(text=None)

Comment element factory. This factory function creates a special element that will be serialized as an XML comment by the standard serializer. *text* is a string containing the comment string. Returns an element instance representing a comment.

Note that `XMLParser` skips over comments in the input instead of creating comment objects for them. An `ElementTree` will only contain comment nodes if they have been inserted into to the tree using one of the `Element` methods.

</div>

<div class="function">

dump(elem)

Writes an element tree or element structure to sys.stdout. This function should be used for debugging only.

The exact output format is implementation dependent. In this version, it's written as an ordinary XML file.

*elem* is an element tree or an individual element.

<div class="versionchanged">

3.8 The `dump` function now preserves the attribute order specified by the user.

</div>

</div>

<div class="function">

fromstring(text, parser=None)

Parses an XML section from a string constant. Same as `XML`. *text* is a string containing XML data. *parser* is an optional parser instance. If not given, the standard `XMLParser` parser is used. Returns an `Element` instance.

</div>

<div class="function">

fromstringlist(sequence, parser=None)

Parses an XML document from a sequence of string fragments. *sequence* is a list or other sequence containing XML data fragments. *parser* is an optional parser instance. If not given, the standard `XMLParser` parser is used. Returns an `Element` instance.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

indent(tree, space=" ", level=0)

Appends whitespace to the subtree to indent the tree visually. This can be used to generate pretty-printed XML output. *tree* can be an Element or ElementTree. *space* is the whitespace string that will be inserted for each indentation level, two space characters by default. For indenting partial subtrees inside of an already indented tree, pass the initial indentation level as *level*.

No whitespace is added inside an element which is marked with `xml:space="preserve"` or which contains text, because this would change its content.

<div class="versionadded">

3.9

</div>

<div class="versionchanged">

3.16 Whitespace is no longer added inside an element with mixed content or marked with `xml:space="preserve"`.

</div>

</div>

<div class="function">

iselement(element)

Check if an object appears to be a valid element object. *element* is an element instance. Return `True` if this is an element object.

</div>

<div class="function">

iterparse(source, events=None, parser=None, \*, target=None)

Parses an XML section incrementally, and reports what's going on to the user. Unless a custom target is used, an element tree is built. *source* is a filename or `file object` containing XML data. *events* is a sequence of events to report back. The supported events are the strings `"start"`, `"end"`, `"comment"`, `"pi"`, `"start-ns"` and `"end-ns"` (the "ns" events are used to get detailed namespace information). If *events* is omitted, only `"end"` events are reported. *parser* is an optional parser instance. If not given, the standard `XMLParser` parser is used. *parser* must be an instance of `XMLParser` or its subclass. *target* is the target of the standard parser, as for `XMLPullParser`; it cannot be used together with *parser*. Returns an `iterator` providing `(event, obj)` pairs, as described for `XMLPullParser.read_events`; it has a `root` attribute that references the root element of the resulting XML tree, or the value returned by the `close()` method of a custom target, once *source* is fully read. If a custom target is used, it is set to the value returned by the `!close` method of the target.

The iterator has the `!close` method that closes the internal file object if *source* is a filename.

Note that while `iterparse` builds the tree incrementally, it issues blocking reads on *source* (or the file it names). As such, it's unsuitable for applications where blocking reads can't be made. For fully non-blocking parsing, see `XMLPullParser`.

The tree is only built incrementally, it is not freed incrementally: every parsed element is kept until the whole document is read. See `elementtree-pull-parsing` for how to keep the memory usage low.

> [!NOTE]
> `iterparse` only guarantees that it has seen the "\>" character of a starting tag when it emits a "start" event, so the attributes are defined, but the contents of the text and tail attributes are undefined at that point. The same applies to the element children; they may or may not be present.
>
> If you need a fully populated element, look for "end" events instead.

<div class="deprecated">

3.4 The *parser* argument.

</div>

<div class="versionchanged">

3.8 The `comment` and `pi` events were added.

</div>

<div class="versionchanged">

3.13 Added the `!close` method.

</div>

<div class="versionchanged">

3.15 A `ResourceWarning` is now emitted if the iterator opened a file and is not explicitly closed.

</div>

<div class="versionchanged">

3.16 Added the *target* parameter.

</div>

</div>

<div class="function">

parse(source, parser=None)

Parses an XML section into an element tree. *source* is a filename or file object containing XML data. *parser* is an optional parser instance. If not given, the standard `XMLParser` parser is used. Returns an `ElementTree` instance.

</div>

<div class="function">

ProcessingInstruction(target, text=None)

PI element factory. This factory function creates a special element that will be serialized as an XML processing instruction. *target* is a string containing the PI target. *text* is a string containing the PI contents, if given. Returns an element instance, representing a processing instruction.

Note that `XMLParser` skips over processing instructions in the input instead of creating PI objects for them. An `ElementTree` will only contain processing instruction nodes if they have been inserted into to the tree using one of the `Element` methods.

</div>

<div class="function">

register_namespace(prefix, uri)

Registers a namespace prefix. The registry is global, and any existing mapping for either the given prefix or the namespace URI will be removed. *prefix* is a namespace prefix. *uri* is a namespace uri. Tags and attributes in this namespace will be serialized with the given prefix, if at all possible.

<div class="versionadded">

3.2

</div>

</div>

<div class="function">

SubElement(parent, tag, /, attrib={}, \*\*extra)

Subelement factory. This function creates an element instance, and appends it to an existing element.

*parent* is the parent element. *tag* is the subelement name. *attrib* is an optional dictionary, containing element attributes. *extra* contains additional attributes, given as keyword arguments. Returns an element instance.

<div class="versionchanged">

3.15 *attrib* can now be a `frozendict`.

</div>

<div class="versionchanged">

3.15 *parent* and *tag* are now positional-only parameters.

</div>

</div>

<div class="function">

tostring(element, encoding="us-ascii", method="xml", \*, xml_declaration=None, default_namespace=None, short_empty_elements=True, standalone=None)

Generates a string representation of an XML element, including all subelements. *element* is an `Element` instance. *encoding*[^1] is the output encoding (default is US-ASCII). Use `encoding="unicode"` to generate a Unicode string (otherwise, a bytestring is generated). *method* is either `"xml"`, `"html"` or `"text"` (default is `"xml"`). *xml_declaration*, *default_namespace*, *short_empty_elements* and *standalone* has the same meaning as in `ElementTree.write`. Returns an (optionally) encoded string containing the XML data.

<div class="versionchanged">

3.4 Added the *short_empty_elements* parameter.

</div>

<div class="versionchanged">

3.8 Added the *xml_declaration* and *default_namespace* parameters.

</div>

<div class="versionchanged">

3.8 The `tostring` function now preserves the attribute order specified by the user.

</div>

<div class="versionchanged">

3.16 Added the *standalone* parameter.

</div>

</div>

<div class="function">

tostringlist(element, encoding="us-ascii", method="xml", \*, xml_declaration=None, default_namespace=None, short_empty_elements=True, standalone=None)

Generates a string representation of an XML element, including all subelements. *element* is an `Element` instance. *encoding*[^2] is the output encoding (default is US-ASCII). Use `encoding="unicode"` to generate a Unicode string (otherwise, a bytestring is generated). *method* is either `"xml"`, `"html"` or `"text"` (default is `"xml"`). *xml_declaration*, *default_namespace*, *short_empty_elements* and *standalone* has the same meaning as in `ElementTree.write`. Returns a list of (optionally) encoded strings containing the XML data. It does not guarantee any specific sequence, except that `b"".join(tostringlist(element)) == tostring(element)`.

<div class="versionadded">

3.2

</div>

<div class="versionchanged">

3.4 Added the *short_empty_elements* parameter.

</div>

<div class="versionchanged">

3.8 Added the *xml_declaration* and *default_namespace* parameters.

</div>

<div class="versionchanged">

3.8 The `tostringlist` function now preserves the attribute order specified by the user.

</div>

<div class="versionchanged">

3.16 Added the *standalone* parameter.

</div>

</div>

<div class="function">

XML(text, parser=None)

Parses an XML section from a string constant. This function can be used to embed "XML literals" in Python code. *text* is a string containing XML data. *parser* is an optional parser instance. If not given, the standard `XMLParser` parser is used. Returns an `Element` instance.

</div>

<div class="function">

XMLID(text, parser=None)

Parses an XML section from a string constant, and also returns a dictionary which maps from element id:s to elements. *text* is a string containing XML data. *parser* is an optional parser instance. If not given, the standard `XMLParser` parser is used. Returns a tuple containing an `Element` instance and a dictionary.

</div>

## XInclude support

This module provides limited support for [XInclude directives](https://www.w3.org/TR/xinclude/), via the `xml.etree.ElementInclude` helper module. This module can be used to insert subtrees and text strings into element trees, based on information in the tree.

### Example

Here's an example that demonstrates use of the XInclude module. To include an XML document in the current document, use the `{http://www.w3.org/2001/XInclude}include` element and set the **parse** attribute to `"xml"`, and use the **href** attribute to specify the document to include.

``` xml
<?xml version="1.0"?>
<document xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:include href="source.xml" parse="xml" />
</document>
```

By default, the **href** attribute is treated as a file name. You can use custom loaders to override this behaviour. Also note that the standard helper does not support XPointer syntax.

To process this file, load it as usual, and pass the root element to the `!xml.etree.ElementTree` module:

``` python
from xml.etree import ElementTree, ElementInclude

tree = ElementTree.parse("document.xml")
root = tree.getroot()

ElementInclude.include(root)
```

The ElementInclude module replaces the `{http://www.w3.org/2001/XInclude}include` element with the root element from the **source.xml** document. The result might look something like this:

``` xml
<document xmlns:xi="http://www.w3.org/2001/XInclude">
  <para>This is a paragraph.</para>
</document>
```

If the **parse** attribute is omitted, it defaults to "xml". The href attribute is required.

To include a text document, use the `{http://www.w3.org/2001/XInclude}include` element, and set the **parse** attribute to "text":

``` xml
<?xml version="1.0"?>
<document xmlns:xi="http://www.w3.org/2001/XInclude">
  Copyright (c) <xi:include href="year.txt" parse="text" />.
</document>
```

The result might look something like:

``` xml
<document xmlns:xi="http://www.w3.org/2001/XInclude">
  Copyright (c) 2003.
</document>
```

## Reference

### Functions

<div class="module">

xml.etree.ElementInclude

</div>

<div class="function">

default_loader(href, parse, encoding=None)

Default loader. This default loader reads an included resource from disk. *href* is a URL. *parse* is for parse mode either "xml" or "text". *encoding* is an optional text encoding. If not given, encoding is `utf-8`. Returns the expanded resource. If the parse mode is `"xml"`, this is an `~xml.etree.ElementTree.Element` instance. If the parse mode is `"text"`, this is a string. If the loader fails, it can return `None` or raise an exception.

</div>

<div class="function">

include(elem, loader=None, base_url=None, max_depth=6)

This function expands XInclude directives in-place in tree pointed by *elem*. *elem* is either the root `~xml.etree.ElementTree.Element` or an `~xml.etree.ElementTree.ElementTree` instance to find such element. *loader* is an optional resource loader. If omitted, it defaults to `default_loader`. If given, it should be a callable that implements the same interface as `default_loader`. *base_url* is base URL of the original file, to resolve relative include file references. *max_depth* is the maximum number of recursive inclusions. Limited to reduce the risk of malicious content explosion. Pass `None` to disable the limitation.

<div class="versionchanged">

3.9 Added the *base_url* and *max_depth* parameters.

</div>

</div>

### Element Objects

<div class="module" data-noindex="" data-no-index="">

xml.etree.ElementTree

</div>

<div class="Element(tag, /, attrib={}, **extra)">

Element class. This class defines the Element interface, and provides a reference implementation of this interface.

*tag* is the element name. *attrib* is an optional dictionary, containing element attributes. *extra* contains additional attributes, given as keyword arguments.

The element name and the attribute names and values are strings or `QName` instances, and the text and the tail are strings or `None`. The element name can also be `Comment` or `ProcessingInstruction`, which are used for special elements. If it is `None`, the element itself is not serialized: only its text and its children are written, and its attributes are ignored. This can be used for a fragment which contains several elements. With `method="html"` the attribute value can also be `None`, which produces an empty attribute (such as `checked`). Other objects can be stored in the tree, but they cannot be serialized.

<div class="versionchanged">

3.15 *attrib* can now be a `frozendict`.

</div>

<div class="versionchanged">

3.15 *tag* is now a positional-only parameter.

</div>

<div class="attribute">

tag

A string identifying what kind of data this element represents (the element type, in other words).

</div>

<div class="attribute">

text tail

These attributes can be used to hold additional data associated with the element. Their values are usually strings but may be any application-specific object. If the element is created from an XML file, the *text* attribute holds either the text between the element's start tag and its first child or end tag, or `None`, and the *tail* attribute holds either the text between the element's end tag and the next tag, or `None`. For the XML data

``` xml
<a><b>1<c>2<d/>3</c></b>4</a>
```

the *a* element has `None` for both *text* and *tail* attributes, the *b* element has *text* `"1"` and *tail* `"4"`, the *c* element has *text* `"2"` and *tail* `None`, and the *d* element has *text* `None` and *tail* `"3"`.

To collect the inner text of an element, see `itertext`, for example `"".join(element.itertext())`.

Applications may store arbitrary objects in these attributes.

</div>

<div class="attribute">

attrib

A dictionary containing the element's attributes. Note that while the *attrib* value is always a real mutable Python dictionary, an ElementTree implementation may choose to use another internal representation, and create the dictionary only if someone asks for it. To take advantage of such implementations, use the dictionary methods below whenever possible.

</div>

The following dictionary-like methods work on the element attributes.

<div class="method">

clear()

Resets an element. This function removes all subelements, clears all attributes, and sets the text and tail attributes to `None`.

</div>

<div class="method">

get(key, default=None)

Gets the element attribute named *key*.

Returns the attribute value, or *default* if the attribute was not found.

</div>

<div class="method">

items()

Returns the element attributes as (name, value) pairs.

</div>

<div class="method">

keys()

Returns the element attribute names.

</div>

<div class="method">

set(key, value)

Set the attribute *key* on the element to *value*.

</div>

The following methods work on the element's children (subelements).

<div class="method">

append(subelement)

Adds the element *subelement* to the end of this element's internal list of subelements. Raises `TypeError` if *subelement* is not an `Element`.

</div>

<div class="method">

extend(subelements)

Appends *subelements* from an iterable of elements. Raises `TypeError` if a subelement is not an `Element`.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

find(match, namespaces=None)

Finds the first subelement matching *match*. *match* may be a tag name or a `path <elementtree-xpath>`. Returns an element instance or `None`. *namespaces* is an optional mapping from namespace prefix to full name. Pass `''` as prefix to move all unprefixed tag names in the expression into the given namespace.

</div>

<div class="method">

findall(match, namespaces=None)

Finds all matching subelements, by tag name or `path <elementtree-xpath>`. Returns a list containing all matching elements in document order. *namespaces* is an optional mapping from namespace prefix to full name. Pass `''` as prefix to move all unprefixed tag names in the expression into the given namespace.

</div>

<div class="method">

findtext(match, default=None, namespaces=None)

Finds text for the first subelement matching *match*. *match* may be a tag name or a `path <elementtree-xpath>`. Returns the text content of the first matching element, or *default* if no element was found. Note that if the matching element has no text content an empty string is returned. *namespaces* is an optional mapping from namespace prefix to full name. Pass `''` as prefix to move all unprefixed tag names in the expression into the given namespace.

</div>

<div class="method">

insert(index, subelement)

Inserts *subelement* at the given position in this element. Raises `TypeError` if *subelement* is not an `Element`.

</div>

<div class="method">

iter(tag=None)

Creates a tree `iterator` with the current element as the root. The iterator iterates over this element and all elements below it, in document (depth first) order. If *tag* is not `None` or `'*'`, only elements whose tag equals *tag* are returned from the iterator. If the tree structure is modified during iteration, the result is undefined.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

iterfind(match, namespaces=None)

Finds all matching subelements, by tag name or `path <elementtree-xpath>`. Returns an iterable yielding all matching elements in document order. *namespaces* is an optional mapping from namespace prefix to full name.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

itertext()

Creates a text iterator. The iterator loops over this element and all subelements, in document order, and returns all inner text.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

makeelement(tag, attrib)

Creates a new element object of the same type as this element. Do not call this method, use the `SubElement` factory function instead.

</div>

<div class="method">

remove(subelement)

Removes *subelement* from the element. Unlike the find\* methods this method compares elements based on the instance identity, not on tag value or contents.

</div>

`Element` objects also support the following sequence type methods for working with subelements: `~object.__delitem__`, `~object.__getitem__`, `~object.__setitem__`, `~object.__len__`.

Caution: Elements with no subelements will test as `False`. In a future release of Python, all elements will test as `True` regardless of whether subelements exist. Instead, prefer explicit `len(elem)` or `elem is not None` tests.:

    element = root.find('foo')

    if not element:  # careful!
        print("element not found, or element has no subelements")

    if element is None:
        print("element not found")

<div class="versionchanged">

3.12 Testing the truth value of an Element emits `DeprecationWarning`.

</div>

Prior to Python 3.8, the serialisation order of the XML attributes of elements was artificially made predictable by sorting the attributes by their name. Based on the now guaranteed ordering of dicts, this arbitrary reordering was removed in Python 3.8 to preserve the order in which attributes were originally parsed or created by user code.

In general, user code should try not to depend on a specific ordering of attributes, given that the [XML Information Set](https://www.w3.org/TR/xml-infoset/) explicitly excludes the attribute order from conveying information. Code should be prepared to deal with any ordering on input. In cases where deterministic XML output is required, e.g. for cryptographic signing or test data sets, canonical serialisation is available with the `canonicalize` function.

In cases where canonical output is not applicable but a specific attribute order is still desirable on output, code should aim for creating the attributes directly in the desired order, to avoid perceptual mismatches for readers of the code. In cases where this is difficult to achieve, a recipe like the following can be applied prior to serialisation to enforce an order independently from the Element creation:

    def reorder_attributes(root):
        for el in root.iter():
            attrib = el.attrib
            if len(attrib) > 1:
                # adjust attribute order, e.g. by sorting
                attribs = sorted(attrib.items())
                attrib.clear()
                attrib.update(attribs)

</div>

### ElementTree Objects

<div class="ElementTree(element=None, file=None)">

ElementTree wrapper class. This class represents an entire element hierarchy, and adds some extra support for serialization to and from standard XML.

*element* is the root element. The tree is initialized with the contents of the XML *file* if given.

<div class="method">

<span id="setroot">setroot</span>(element)

Replaces the root element for this tree. This discards the current contents of the tree, and replaces it with the given element. Use with care. *element* is an element instance.

</div>

<div class="method">

find(match, namespaces=None)

Same as `Element.find`, starting at the root of the tree.

</div>

<div class="method">

findall(match, namespaces=None)

Same as `Element.findall`, starting at the root of the tree.

</div>

<div class="method">

findtext(match, default=None, namespaces=None)

Same as `Element.findtext`, starting at the root of the tree.

</div>

<div class="method">

getroot()

Returns the root element for this tree.

</div>

<div class="method">

iter(tag=None)

Creates and returns a tree iterator for the root element. The iterator loops over all elements in this tree, in section order. *tag* is the tag to look for (default is to return all elements).

</div>

<div class="method">

iterfind(match, namespaces=None)

Same as `Element.iterfind`, starting at the root of the tree.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

parse(source, parser=None)

Loads an external XML section into this element tree. *source* is a file name or `file object`. *parser* is an optional parser instance. If not given, the standard `XMLParser` parser is used. Returns the section root element.

</div>

<div class="method">

write(file, encoding="us-ascii", xml_declaration=None, default_namespace=None, method="xml", \*, short_empty_elements=True, standalone=None)

Writes the element tree to a file, as XML. *file* is a file name, or a `file object` opened for writing. *encoding*[^3] is the output encoding (default is US-ASCII). *xml_declaration* controls if an XML declaration should be added to the file. Use `False` for never, `True` for always, `None` for only if not US-ASCII or UTF-8 or Unicode (default is `None`). *default_namespace* sets the default XML namespace (for "xmlns"). *method* is either `"xml"`, `"html"` or `"text"` (default is `"xml"`). The keyword-only *short_empty_elements* parameter controls the formatting of elements that contain no content. If `True` (the default), they are emitted as a single self-closed tag, otherwise they are emitted as a pair of start/end tags.

The keyword-only *standalone* parameter is the value of the standalone document declaration in the XML declaration. Use `True` for `standalone="yes"`, `False` for `standalone="no"`, and `None` (the default) to omit it. An XML declaration is written if *standalone* is not `None`; combining it with `xml_declaration=False` raises a `ValueError`.

The output is either a string (`str`) or binary (`bytes`). This is controlled by the *encoding* argument. If *encoding* is `"unicode"`, the output is a string; otherwise, it's binary. Note that this may conflict with the type of *file* if it's an open `file object`; make sure you do not try to write a string to a binary stream and vice versa.

<div class="versionchanged">

3.4 Added the *short_empty_elements* parameter.

</div>

<div class="versionchanged">

3.8 The `write` method now preserves the attribute order specified by the user.

</div>

<div class="versionchanged">

3.16 Added the *standalone* parameter.

</div>

</div>

</div>

This is the XML file that is going to be manipulated:

    <html>
        <head>
            <title>Example page</title>
        </head>
        <body>
            <p>Moved to <a href="http://example.org/">example.org</a>
            or <a href="http://example.com/">example.com</a>.</p>
        </body>
    </html>

Example of changing the attribute "target" of every link in first paragraph:

    >>> from xml.etree.ElementTree import ElementTree
    >>> tree = ElementTree()
    >>> tree.parse("index.xhtml")
    <Element 'html' at 0xb77e6fac>
    >>> p = tree.find("body/p")     # Finds first occurrence of tag p in body
    >>> p
    <Element 'p' at 0xb77ec26c>
    >>> links = list(p.iter("a"))   # Returns list of all links
    >>> links
    [<Element 'a' at 0xb77ec2ac>, <Element 'a' at 0xb77ec1cc>]
    >>> for i in links:             # Iterates through all found links
    ...     i.attrib["target"] = "blank"
    ...
    >>> tree.write("output.xhtml")

### QName Objects

<div class="QName(text_or_uri, tag=None)">

QName wrapper. This can be used to wrap a QName attribute value, in order to get proper namespace handling on output. *text_or_uri* is a string containing the QName value, in the form {uri}local, or, if the tag argument is given, the URI part of a QName. If *tag* is given, the first argument is interpreted as a URI, and this argument is interpreted as a local name. `QName` instances are opaque.

</div>

### TreeBuilder Objects

<div class="TreeBuilder(element_factory=None, *, comment_factory=None, \ pi_factory=None, insert_comments=False, insert_pis=False)">

Generic element structure builder. This builder converts a sequence of start, data, end, comment and pi method calls to a well-formed element structure. You can use this class to build an element structure using a custom XML parser, or a parser for some other XML-like format.

*element_factory*, when given, must be a callable accepting two positional arguments: a tag and a dict of attributes. It is expected to return a new element instance.

The *comment_factory* and *pi_factory* functions, when given, should behave like the `Comment` and `ProcessingInstruction` functions to create comments and processing instructions. When not given, the default factories will be used. When *insert_comments* and/or *insert_pis* is true, comments/pis will be inserted into the tree if they appear within the root element (but not outside of it).

<div class="method">

close()

Flushes the builder buffers, and returns the toplevel document element. Returns an `Element` instance.

</div>

<div class="method">

data(data)

Adds text to the current element. *data* is a string.

</div>

<div class="method">

end(tag)

Closes the current element. *tag* is the element name. Returns the closed element.

</div>

<div class="method">

start(tag, attrs)

Opens a new element. *tag* is the element name. *attrs* is a dictionary containing element attributes. Returns the opened element.

</div>

<div class="method">

comment(text)

Creates a comment with the given *text*. If `insert_comments` is true, this will also add it to the tree.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

pi(target, text)

Creates a process instruction with the given *target* name and *text*. If `insert_pis` is true, this will also add it to the tree.

<div class="versionadded">

3.8

</div>

</div>

In addition, a custom `TreeBuilder` object can provide the following methods:

<div class="method">

doctype(name, pubid, system)

Handles a doctype declaration. *name* is the doctype name. *pubid* is the public identifier. *system* is the system identifier. This method does not exist on the default `TreeBuilder` class.

<div class="versionadded">

3.2

</div>

</div>

<div class="method">

start_ns(prefix, uri)

Is called whenever the parser encounters a new namespace declaration, before the `start()` callback for the opening element that defines it. *prefix* is `''` for the default namespace and the declared namespace prefix name otherwise. *uri* is the namespace URI.

<div class="versionadded">

3.8

</div>

</div>

<div class="method">

end_ns(prefix)

Is called after the `end()` callback of an element that declared a namespace prefix mapping, with the name of the *prefix* that went out of scope.

<div class="versionadded">

3.8

</div>

</div>

</div>

<div class="C14NWriterTarget(write, *, \ with_comments=False, strip_text=False, rewrite_prefixes=False, \ qname_aware_tags=None, qname_aware_attrs=None, \ exclude_attrs=None, exclude_tags=None)">

A [C14N 2.0](https://www.w3.org/TR/xml-c14n2/) writer. Arguments are the same as for the `canonicalize` function. This class does not build a tree but translates the callback events directly into a serialised form using the *write* function.

<div class="versionadded">

3.8

</div>

</div>

### XMLParser Objects

<div class="XMLParser(*, target=None, encoding=None)">

This class is the low-level building block of the module. It uses `xml.parsers.expat` for efficient, event-based parsing of XML. It can be fed XML data incrementally with the `feed` method, and parsing events are translated to a push API - by invoking callbacks on the *target* object. If *target* is omitted, the standard `TreeBuilder` is used. If *encoding*[^4] is given, the value overrides the encoding specified in the XML file.

<div class="versionchanged">

3.8 Parameters are now `keyword-only <keyword-only_parameter>`. The *html* argument is no longer supported.

</div>

<div class="method">

close()

Finishes feeding data to the parser. Returns the result of calling the `close()` method of the *target* passed during construction; by default, this is the toplevel document element.

</div>

<div class="method">

feed(data)

Feeds data to the parser. *data* is a string or encoded data (`bytes` or a `bytes-like object`).

</div>

<div class="method">

flush()

Triggers parsing of any previously fed unparsed data, which can be used to ensure more immediate feedback, in particular with Expat \>=2.6.0. The implementation of `flush` temporarily disables reparse deferral with Expat (if currently enabled) and triggers a reparse. Disabling reparse deferral has security consequences; please see `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` for details.

`!flush` has been backported to some prior releases of CPython as a security fix. Check for availability using `hasattr` if used in code running across a variety of Python versions.

<div class="versionadded">

3.13

</div>

</div>

`XMLParser.feed` calls *target*'s `start(tag, attrs_dict)` method for each opening tag, its `end(tag)` method for each closing tag, and data is processed by method `data(data)`. For further supported callback methods, see the `TreeBuilder` class. `XMLParser.close` calls *target*'s method `close()`. `XMLParser` can be used not only for building a tree structure. This is an example of counting the maximum depth of an XML file:

    >>> from xml.etree.ElementTree import XMLParser
    >>> class MaxDepth:                     # The target object of the parser
    ...     maxDepth = 0
    ...     depth = 0
    ...     def start(self, tag, attrib):   # Called for each opening tag.
    ...         self.depth += 1
    ...         if self.depth > self.maxDepth:
    ...             self.maxDepth = self.depth
    ...     def end(self, tag):             # Called for each closing tag.
    ...         self.depth -= 1
    ...     def data(self, data):
    ...         pass            # We do not need to do anything with data.
    ...     def close(self):    # Called when all data has been parsed.
    ...         return self.maxDepth
    ...
    >>> target = MaxDepth()
    >>> parser = XMLParser(target=target)
    >>> exampleXml = """
    ... <a>
    ...   <b>
    ...   </b>
    ...   <b>
    ...     <c>
    ...       <d>
    ...       </d>
    ...     </c>
    ...   </b>
    ... </a>"""
    >>> parser.feed(exampleXml)
    >>> parser.close()
    4

</div>

### XMLPullParser Objects

<div class="XMLPullParser(events=None, *, target=None)">

A pull parser suitable for non-blocking applications. Its input-side API is similar to that of `XMLParser`, but instead of pushing calls to a callback target, `XMLPullParser` collects an internal list of parsing events and lets the user read from it. *events* is a sequence of events to report back. The supported events are the strings `"start"`, `"end"`, `"comment"`, `"pi"`, `"start-ns"` and `"end-ns"` (the "ns" events are used to get detailed namespace information). If *events* is omitted, only `"end"` events are reported.

*target* is the target object of the underlying `XMLParser`. If omitted, the standard `TreeBuilder` is used, and the reported objects are `Element` instances. With other targets the reported object is the value returned by the corresponding method of the target, so no tree is built if the target does not build one. The target must implement the methods for all requested events, except `!start_ns` and `!end_ns`: if they are not implemented, a `(prefix, uri)` tuple and `None` are reported for the `"start-ns"` and `"end-ns"` events.

<div class="versionchanged">

3.16 Added the *target* parameter.

</div>

<div class="method">

feed(data)

Feed the given data to the parser. *data* is a string or encoded data (`bytes` or a `bytes-like object`).

</div>

<div class="method">

flush()

Triggers parsing of any previously fed unparsed data, which can be used to ensure more immediate feedback, in particular with Expat \>=2.6.0. The implementation of `flush` temporarily disables reparse deferral with Expat (if currently enabled) and triggers a reparse. Disabling reparse deferral has security consequences; please see `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` for details.

`!flush` has been backported to some prior releases of CPython as a security fix. Check for availability using `hasattr` if used in code running across a variety of Python versions.

<div class="versionadded">

3.13

</div>

</div>

<div class="method">

close()

Signal the parser that the data stream is terminated. Unlike `XMLParser.close`, this method always returns `None`. Any events not yet retrieved when the parser is closed can still be read with `read_events`.

</div>

<div class="method">

read_events()

Return an iterator over the events which have been encountered in the data fed to the parser. The iterator yields `(event, obj)` pairs, where *event* is a string representing the type of event (e.g. `"end"`) and *obj* is the object returned by the corresponding method of the target. With the standard `TreeBuilder` it is as follows.

- `start`, `end`: the current Element.
- `comment`, `pi`: the current comment / processing instruction
- `start-ns`: a tuple `(prefix, uri)` naming the declared namespace mapping.
- `end-ns`: `None` (this may change in a future version)

Events provided in a previous call to `read_events` will not be yielded again. Events are consumed from the internal queue only when they are retrieved from the iterator, so multiple readers iterating in parallel over iterators obtained from `read_events` will have unpredictable results.

</div>

> [!NOTE]
> `XMLPullParser` only guarantees that it has seen the "\>" character of a starting tag when it emits a "start" event, so the attributes are defined, but the contents of the text and tail attributes are undefined at that point. The same applies to the element children; they may or may not be present.
>
> If you need a fully populated element, look for "end" events instead.

<div class="versionadded">

3.4

</div>

<div class="versionchanged">

3.8 The `comment` and `pi` events were added.

</div>

</div>

### Exceptions

<div class="ParseError">

XML parse error, raised by the various parsing methods in this module when parsing fails. The string representation of an instance of this exception will contain a user-friendly error message. In addition, it will have the following attributes available:

<div class="attribute">

code

A numeric error code from the expat parser. See the documentation of `xml.parsers.expat` for the list of error codes and their meanings.

</div>

<div class="attribute">

position

A tuple of *line*, *column* numbers, specifying where the error occurred.

</div>

</div>

**Footnotes**

[^1]: The encoding string included in XML output should conform to the appropriate standards. For example, "UTF-8" is valid, but "UTF8" is not. See <https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl> and <https://www.iana.org/assignments/character-sets/character-sets.xhtml>.

[^2]: The encoding string included in XML output should conform to the appropriate standards. For example, "UTF-8" is valid, but "UTF8" is not. See <https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl> and <https://www.iana.org/assignments/character-sets/character-sets.xhtml>.

[^3]: The encoding string included in XML output should conform to the appropriate standards. For example, "UTF-8" is valid, but "UTF8" is not. See <https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl> and <https://www.iana.org/assignments/character-sets/character-sets.xhtml>.

[^4]: The encoding string included in XML output should conform to the appropriate standards. For example, "UTF-8" is valid, but "UTF8" is not. See <https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl> and <https://www.iana.org/assignments/character-sets/character-sets.xhtml>.

# XML Processing Modules

<div class="module" data-synopsis="Package containing XML processing modules">

xml

</div>

**Source code:** `Lib/xml/`

------------------------------------------------------------------------

Python's interfaces for processing XML are grouped in the `xml` package.

> [!NOTE]
> If you need to parse untrusted or unauthenticated data, see `xml-security`.

It is important to note that modules in the `!xml` package require that there be at least one SAX-compliant XML parser available. The Expat parser is included with Python, so the `xml.parsers.expat` module will always be available.

The documentation for the `xml.dom` and `xml.sax` packages are the definition of the Python bindings for the DOM and SAX interfaces.

The XML handling submodules are:

- `xml.etree.ElementTree`: the ElementTree API, a simple and lightweight XML processor

<!-- -->

- `xml.dom`: the DOM API definition
- `xml.dom.minidom`: a minimal DOM implementation
- `xml.dom.pulldom`: support for building partial DOM trees

<!-- -->

- `xml.sax`: SAX2 base classes and convenience functions
- `xml.parsers.expat`: the Expat parser binding

This module also defines utility functions.

<div class="function">

is_valid_name(name)

Return `True` if the string is a valid element or attribute name, `False` otherwise.

Almost all characters are permitted in names, except control characters and those which either are or reasonably could be used as delimiters. Characters like ":", "-", ".", "\_", and "·" are permitted, but "\<", "/", "!", "?", and "=" are forbidden. The name cannot start with a digit or a character like "-", ".", and "·".

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

is_valid_text(data)

Return `True` if the string is a sequence of legal XML 1.0 characters, `False` otherwise.

Almost all characters are permitted in XML 1.0 documents, except C0 control characters (excluding TAB, CR and LF), surrogate characters and special Unicode characters U+FFFE and U+FFFF.

<div class="versionadded">

3.15

</div>

</div>

## XML security<span id="xml-security"></span>

An attacker can abuse XML features to carry out denial of service attacks, access local files, generate network connections to other machines, or circumvent firewalls when attacker-controlled XML is being parsed, in Python or elsewhere.

The built-in XML parsers of Python rely on the library [libexpat](https://github.com/libexpat/libexpat), commonly called Expat, for parsing XML.

By default, Expat itself does not access local files or create network connections.

Expat versions lower than 2.7.2 may be vulnerable to the "billion laughs", "quadratic blowup" and "large tokens" vulnerabilities, or to disproportional use of dynamic memory. Python bundles a copy of Expat, and whether Python uses the bundled or a system-wide Expat, depends on how the Python interpreter `has been configured <--with-system-expat>` in your environment. Python may be vulnerable if it uses such older versions of Expat. Check `!pyexpat.EXPAT_VERSION`.

`xmlrpc` is **vulnerable** to the "decompression bomb" attack.

billion laughs / exponential entity expansion  
The [Billion Laughs](https://en.wikipedia.org/wiki/Billion_laughs) attack -- also known as exponential entity expansion -- uses multiple levels of nested entities. Each entity refers to another entity several times, and the final entity definition contains a small string. The exponential expansion results in several gigabytes of text and consumes lots of memory and CPU time.

quadratic blowup entity expansion  
A quadratic blowup attack is similar to a [Billion Laughs](https://en.wikipedia.org/wiki/Billion_laughs) attack; it abuses entity expansion, too. Instead of nested entities it repeats one large entity with a couple of thousand chars over and over again. The attack isn't as efficient as the exponential case but it avoids triggering parser countermeasures that forbid deeply nested entities.

decompression bomb  
Decompression bombs (aka [ZIP bomb](https://en.wikipedia.org/wiki/Zip_bomb)) apply to all XML libraries that can parse compressed XML streams such as gzipped HTTP streams or LZMA-compressed files. For an attacker it can reduce the amount of transmitted data by three magnitudes or more.

large tokens  
Expat needs to re-parse unfinished tokens; without the protection introduced in Expat 2.6.0, this can lead to quadratic runtime that can be used to cause denial of service in the application parsing XML. The issue is known as `2023-52425`.

# `!xml.sax.handler` --- Base classes for SAX handlers

<div class="module" data-synopsis="Base classes for SAX event handlers.">

xml.sax.handler

</div>

**Source code:** `Lib/xml/sax/handler.py`

------------------------------------------------------------------------

The SAX API defines five kinds of handlers: content handlers, DTD handlers, error handlers, entity resolvers and lexical handlers. Applications normally only need to implement those interfaces whose events they are interested in; they can implement the interfaces in a single object or in multiple objects. Handler implementations should inherit from the base classes provided in the module `!xml.sax.handler`, so that all methods get default implementations.

<div class="ContentHandler">

This is the main callback interface in SAX, and the one most important to applications. The order of events in this interface mirrors the order of the information in the document.

</div>

<div class="DTDHandler">

Handle DTD events.

This interface specifies only those DTD events required for basic parsing (unparsed entities and attributes).

</div>

<div class="EntityResolver">

Basic interface for resolving entities. If you create an object implementing this interface, then register the object with your Parser, the parser will call the method in your object to resolve all external entities.

</div>

<div class="ErrorHandler">

Interface used by the parser to present error and warning messages to the application. The methods of this object control whether errors are immediately converted to exceptions or are handled in some other way.

</div>

<div class="LexicalHandler">

Interface used by the parser to represent low frequency events which may not be of interest to many applications.

</div>

In addition to these classes, `!xml.sax.handler` provides symbolic constants for the feature and property names.

<div class="data">

feature_namespaces

value: `"http://xml.org/sax/features/namespaces"`\
true: Perform Namespace processing.\
false: Optionally do not perform Namespace processing (implies namespace-prefixes; default).\
access: (parsing) read-only; (not parsing) read/write

</div>

<div class="data">

feature_namespace_prefixes

value: `"http://xml.org/sax/features/namespace-prefixes"`\
true: Report the original prefixed names and attributes used for Namespace declarations.\
false: Do not report attributes used for Namespace declarations, and optionally do not report original prefixed names (default).\
access: (parsing) read-only; (not parsing) read/write

The parser based on `xml.parsers.expat` does not support this feature.

</div>

<div class="data">

feature_string_interning

value: `"http://xml.org/sax/features/string-interning"`\
true: All element names, prefixes, attribute names, Namespace URIs, and local names are interned in a dictionary (see `property_interning_dict`).\
false: Names are not necessarily interned, although they may be (default).\
access: (parsing) read-only; (not parsing) read/write

</div>

<div class="data">

feature_validation

value: `"http://xml.org/sax/features/validation"`\
true: Report all validation errors (implies external-general-entities and external-parameter-entities).\
false: Do not report validation errors.\
access: (parsing) read-only; (not parsing) read/write

The parser based on `xml.parsers.expat` does not support this feature, because Expat is a non-validating parser.

</div>

<div class="data">

feature_external_ges

> [!WARNING]
> Enabling opens a vulnerability to [external entity attacks](https://en.wikipedia.org/wiki/XML_external_entity_attack) if the parser is used with user-provided XML content. Please reflect on your [threat model](https://en.wikipedia.org/wiki/Threat_model) before enabling this feature.

value: `"http://xml.org/sax/features/external-general-entities"`\
true: Include all external general (text) entities.\
false: Do not include external general entities.\
access: (parsing) read-only; (not parsing) read/write

</div>

<div class="data">

feature_external_pes

value: `"http://xml.org/sax/features/external-parameter-entities"`\
true: Include all external parameter entities, including the external DTD subset.\
false: Do not include any external parameter entities, even the external DTD subset.\
access: (parsing) read-only; (not parsing) read/write

The parser based on `xml.parsers.expat` does not support this feature.

</div>

<div class="data">

all_features

List of all features.

</div>

<div class="data">

property_lexical_handler

value: `"http://xml.org/sax/properties/lexical-handler"`\
data type: `~xml.sax.handler.LexicalHandler`\
description: An optional extension handler for lexical events like comments.\
access: read/write

</div>

<div class="data">

property_declaration_handler

value: `"http://xml.org/sax/properties/declaration-handler"`\
data type: an object implementing the SAX2 `DeclHandler` interface\
description: An optional extension handler for DTD-related events other than notations and unparsed entities.\
access: read/write

No parser in the standard library supports this property, and the standard library provides no such handler.

</div>

<div class="data">

property_dom_node

value: `"http://xml.org/sax/properties/dom-node"`\
data type: `xml.dom.Node`\
description: When parsing, the current DOM node being visited if this is a DOM iterator; when not parsing, the root DOM node for iteration.\
access: (parsing) read-only; (not parsing) read/write

No parser in the standard library supports this property.

</div>

<div class="data">

property_xml_string

value: `"http://xml.org/sax/properties/xml-string"`\
data type: Bytes\
description: The literal string of characters that was the source for the current event.\
access: read-only, and only during a handler callback

</div>

<div class="data">

property_encoding

value: `"http://www.python.org/sax/properties/encoding"`\
data type: String\
description: The name of the encoding to assume for input data.\
access: read/write

No parser in the standard library supports this property.

</div>

<div class="data">

property_interning_dict

value: `"http://www.python.org/sax/properties/interning-dict"`\
data type: Dictionary\
description: The dictionary used to intern names, or `None` if names are not interned. Setting it enables interning, as does the `feature_string_interning` feature.\
access: read/write

</div>

<div class="data">

all_properties

List of all known property names.

</div>

## ContentHandler Objects

Users are expected to subclass `ContentHandler` to support their application. The following methods are called by the parser on the appropriate events in the input document:

<div class="method">

ContentHandler.setDocumentLocator(locator)

Called by the parser to give the application a locator for locating the origin of document events.

SAX parsers are strongly encouraged (though not absolutely required) to supply a locator: if it does so, it must supply the locator to the application by invoking this method before invoking any of the other methods in the DocumentHandler interface.

The locator allows the application to determine the end position of any document-related event, even if the parser is not reporting an error. Typically, the application will use this information for reporting its own errors (such as character content that does not match an application's business rules). The information returned by the locator is probably not sufficient for use with a search engine.

Note that the locator will return correct information only during the invocation of the events in this interface. The application should not attempt to use it at any other time.

</div>

<div class="method">

ContentHandler.startDocument()

Receive notification of the beginning of a document.

The SAX parser will invoke this method only once, before any other methods in this interface or in DTDHandler (except for `setDocumentLocator`).

</div>

<div class="method">

ContentHandler.endDocument()

Receive notification of the end of a document.

The SAX parser will invoke this method only once, and it will be the last method invoked during the parse. The parser shall not invoke this method until it has either abandoned parsing (because of an unrecoverable error) or reached the end of input.

</div>

<div class="method">

ContentHandler.startPrefixMapping(prefix, uri)

Begin the scope of a prefix-URI Namespace mapping.

The information from this event is not necessary for normal Namespace processing: the SAX XML reader will automatically replace prefixes for element and attribute names when the `feature_namespaces` feature is enabled (the default).

There are cases, however, when applications need to use prefixes in character data or in attribute values, where they cannot safely be expanded automatically; the `startPrefixMapping` and `endPrefixMapping` events supply the information to the application to expand prefixes in those contexts itself, if necessary.

Note that `startPrefixMapping` and `endPrefixMapping` events are not guaranteed to be properly nested relative to each-other: all `startPrefixMapping` events will occur before the corresponding `startElement` event, and all `endPrefixMapping` events will occur after the corresponding `endElement` event, but their order is not guaranteed.

</div>

<div class="method">

ContentHandler.endPrefixMapping(prefix)

End the scope of a prefix-URI mapping.

See `startPrefixMapping` for details. This event will always occur after the corresponding `endElement` event, but the order of `endPrefixMapping` events is not otherwise guaranteed.

</div>

<div class="method">

ContentHandler.startElement(name, attrs)

Signals the start of an element in non-namespace mode.

The *name* parameter contains the raw XML 1.0 name of the element type as a string and the *attrs* parameter holds an object of the `Attributes <attributes-objects>` interface containing the attributes of the element. The object passed as *attrs* may be re-used by the parser; holding on to a reference to it is not a reliable way to keep a copy of the attributes. To keep a copy of the attributes, use the `copy` method of the *attrs* object.

</div>

<div class="method">

ContentHandler.endElement(name)

Signals the end of an element in non-namespace mode.

The *name* parameter contains the name of the element type, just as with the `startElement` event.

</div>

<div class="method">

ContentHandler.startElementNS(name, qname, attrs)

Signals the start of an element in namespace mode.

The *name* parameter contains the name of the element type as a `(uri, localname)` tuple, the *qname* parameter contains the raw XML 1.0 name used in the source document, and the *attrs* parameter holds an instance of the `AttributesNS <attributes-ns-objects>` interface containing the attributes of the element. If no namespace is associated with the element, the *uri* component of *name* will be `None`. The object passed as *attrs* may be re-used by the parser; holding on to a reference to it is not a reliable way to keep a copy of the attributes. To keep a copy of the attributes, use the `copy` method of the *attrs* object.

Parsers may set the *qname* parameter to `None`, unless the `feature_namespace_prefixes` feature is activated.

</div>

<div class="method">

ContentHandler.endElementNS(name, qname)

Signals the end of an element in namespace mode.

The *name* parameter contains the name of the element type, just as with the `startElementNS` method, likewise the *qname* parameter.

</div>

<div class="method">

ContentHandler.characters(content)

Receive notification of character data.

The Parser will call this method to report each chunk of character data. SAX parsers may return all contiguous character data in a single chunk, or they may split it into several chunks; however, all of the characters in any single event must come from the same external entity so that the Locator provides useful information.

*content* may be a string or bytes instance; the `expat` reader module always produces strings.

> [!NOTE]
> The earlier SAX 1 interface provided by the Python XML Special Interest Group used a more Java-like interface for this method. Since most parsers used from Python did not take advantage of the older interface, the simpler signature was chosen to replace it. To convert old code to the new interface, use *content* instead of slicing content with the old *offset* and *length* parameters.

</div>

<div class="method">

ContentHandler.ignorableWhitespace(whitespace)

Receive notification of ignorable whitespace in element content.

Validating Parsers must use this method to report each chunk of ignorable whitespace (see the W3C XML 1.0 recommendation, section 2.10): non-validating parsers may also use this method if they are capable of parsing and using content models.

SAX parsers may return all contiguous whitespace in a single chunk, or they may split it into several chunks; however, all of the characters in any single event must come from the same external entity, so that the Locator provides useful information.

</div>

<div class="method">

ContentHandler.processingInstruction(target, data)

Receive notification of a processing instruction.

The Parser will invoke this method once for each processing instruction found: note that processing instructions may occur before or after the main document element.

A SAX parser should never report an XML declaration (XML 1.0, section 2.8) or a text declaration (XML 1.0, section 4.3.1) using this method.

</div>

<div class="method">

ContentHandler.skippedEntity(name)

Receive notification of a skipped entity.

The Parser will invoke this method once for each entity skipped. Non-validating processors may skip entities if they have not seen the declarations (because, for example, the entity was declared in an external DTD subset). All processors may skip external entities, depending on the values of the `feature_external_ges` and the `feature_external_pes` properties.

</div>

## DTDHandler Objects

`DTDHandler` instances provide the following methods:

<div class="method">

DTDHandler.notationDecl(name, publicId, systemId)

Handle a notation declaration event.

</div>

<div class="method">

DTDHandler.unparsedEntityDecl(name, publicId, systemId, ndata)

Handle an unparsed entity declaration event.

</div>

## EntityResolver Objects

<div class="method">

EntityResolver.resolveEntity(publicId, systemId)

Resolve the system identifier of an entity and return either the system identifier to read from as a string, or an InputSource to read from. The default implementation returns *systemId*.

</div>

## ErrorHandler Objects

Objects with this interface are used to receive error and warning information from the `~xml.sax.xmlreader.XMLReader`. If you create an object that implements this interface, then register the object with your `~xml.sax.xmlreader.XMLReader`, the parser will call the methods in your object to report all warnings and errors. There are three levels of errors available: warnings, (possibly) recoverable errors, and unrecoverable errors. All methods take a `~xml.sax.SAXParseException` as the only parameter. Errors and warnings may be converted to an exception by raising the passed-in exception object.

<div class="method">

ErrorHandler.error(exception)

Called when the parser encounters a recoverable error. If this method does not raise an exception, parsing may continue, but further document information should not be expected by the application. Allowing the parser to continue may allow additional errors to be discovered in the input document.

</div>

<div class="method">

ErrorHandler.fatalError(exception)

Called when the parser encounters an error it cannot recover from; parsing is expected to terminate when this method returns.

</div>

<div class="method">

ErrorHandler.warning(exception)

Called when the parser presents minor warning information to the application. Parsing is expected to continue when this method returns, and document information will continue to be passed to the application. Raising an exception in this method will cause parsing to end.

</div>

## LexicalHandler Objects

Optional SAX2 handler for lexical events.

This handler is used to obtain lexical information about an XML document. Lexical information includes information describing the document encoding used and XML comments embedded in the document, as well as section boundaries for the DTD and for any CDATA sections. The lexical handlers are used in the same manner as content handlers.

Set the LexicalHandler of an XMLReader by using the setProperty method with the property identifier `'http://xml.org/sax/properties/lexical-handler'`.

<div class="method">

LexicalHandler.comment(content)

Reports a comment anywhere in the document (including the DTD and outside the document element).

</div>

<div class="method">

LexicalHandler.startDTD(name, public_id, system_id)

Reports the start of the DTD declarations if the document has an associated DTD.

</div>

<div class="method">

LexicalHandler.endDTD()

Reports the end of DTD declaration.

</div>

<div class="method">

LexicalHandler.startCDATA()

Reports the start of a CDATA marked section.

The contents of the CDATA marked section will be reported through the characters handler.

</div>

<div class="method">

LexicalHandler.endCDATA()

Reports the end of a CDATA marked section.

</div>

# `!xml.sax` --- Support for SAX2 parsers

<div class="module" data-synopsis="Package containing SAX2 base classes and convenience functions.">

xml.sax

</div>

**Source code:** `Lib/xml/sax/__init__.py`

------------------------------------------------------------------------

The `!xml.sax` package provides a number of modules which implement the Simple API for XML (SAX) interface for Python. The package itself provides the SAX exceptions and the convenience functions which will be most used by users of the SAX API.

> [!NOTE]
> If you need to parse untrusted or unauthenticated data, see `xml-security`.

<div class="versionchanged">

3.7.1

The SAX parser no longer processes general external entities by default to increase security. Before, the parser created network connections to fetch remote files or loaded local files from the file system for DTD and entities. The feature can be enabled again with method `~xml.sax.xmlreader.XMLReader.setFeature` on the parser object and argument `~xml.sax.handler.feature_external_ges`.

</div>

The convenience functions and data are:

<div class="function">

make_parser(parser_list=())

Create and return a SAX `~xml.sax.xmlreader.XMLReader` object. The first parser found will be used. If *parser_list* is provided, it must be an iterable of strings which name modules that have a function named `create_parser`. Modules listed in *parser_list* will be used before modules in the default list of parsers.

<div class="versionchanged">

3.8 The *parser_list* argument can be any iterable, not just a list.

</div>

</div>

<div class="function">

parse(filename_or_stream, handler, errorHandler=handler.ErrorHandler())

Create a SAX parser and use it to parse a document. The document, passed in as *filename_or_stream*, can be a system identifier (a string identifying the input source -- typically a file name or a URL), a `path-like <path-like object>` object, or a file object. A system identifier which does not refer to an existing file is opened with `urllib.request.urlopen`. The *handler* parameter needs to be a SAX `~handler.ContentHandler` instance. If *errorHandler* is given, it must be a SAX `~handler.ErrorHandler` instance; if omitted, `SAXParseException` will be raised on all errors. There is no return value; all work must be done by the *handler* passed in.

</div>

<div class="function">

parseString(string, handler, errorHandler=handler.ErrorHandler())

Similar to `parse`, but parses from a buffer *string* received as a parameter. *string* must be a `str` instance or a `bytes-like object`.

<div class="versionchanged">

3.5 Added support of `str` instances.

</div>

</div>

<div class="data">

default_parser_list

The list of the names of modules which are tried by `make_parser` after the modules named in its *parser_list* argument. It contains `'xml.sax.expatreader'`, or, if the `!PY_SAX_PARSER` environment variable is set and the environment is not ignored, the comma-separated list of module names taken from it.

</div>

A typical SAX application uses three kinds of objects: readers, handlers and input sources. "Reader" in this context is another term for parser, i.e. some piece of code that reads the bytes or characters from the input source, and produces a sequence of events. The events then get distributed to the handler objects, i.e. the reader invokes a method on the handler. A SAX application must therefore obtain a reader object, create or open the input sources, create the handlers, and connect these objects all together. As the final step of preparation, the reader is called to parse the input. During parsing, methods on the handler objects are called based on structural and syntactic events from the input data.

For these objects, only the interfaces are relevant; they are normally not instantiated by the application itself. Since Python does not have an explicit notion of interface, they are formally introduced as classes, but applications may use implementations which do not inherit from the provided classes. The `~xml.sax.xmlreader.InputSource`, `~xml.sax.xmlreader.Locator`, `~xml.sax.xmlreader.Attributes`, `~xml.sax.xmlreader.AttributesNS`, and `~xml.sax.xmlreader.XMLReader` interfaces are defined in the module `xml.sax.xmlreader`. The handler interfaces are defined in `xml.sax.handler`. For convenience, `~xml.sax.xmlreader.InputSource` (which is often instantiated directly) and the handler classes are also available from `!xml.sax`. These interfaces are described below.

In addition to these classes, `!xml.sax` provides the following exception classes.

<div class="exception">

SAXException(msg, exception=None)

Encapsulate an XML error or warning. This class can contain basic error or warning information from either the XML parser or the application: it can be subclassed to provide additional functionality or to add localization. Note that although the handlers defined in the `~xml.sax.handler.ErrorHandler` interface receive instances of this exception, it is not required to actually raise the exception --- it is also useful as a container for information.

When instantiated, *msg* should be a human-readable description of the error. The optional *exception* parameter, if given, should be `None` or an exception that was caught by the parsing code and is being passed along as information.

This is the base class for the other SAX exception classes.

</div>

<div class="exception">

SAXParseException(msg, exception, locator)

Subclass of `SAXException` raised on parse errors. Instances of this class are passed to the methods of the SAX `~xml.sax.handler.ErrorHandler` interface to provide information about the parse error. This class supports the SAX `~xml.sax.xmlreader.Locator` interface as well as the `SAXException` interface.

</div>

<div class="exception">

SAXNotRecognizedException(msg, exception=None)

Subclass of `SAXException` raised when a SAX `~xml.sax.xmlreader.XMLReader` is confronted with an unrecognized feature or property. SAX applications and extensions may use this class for similar purposes.

</div>

<div class="exception">

SAXNotSupportedException(msg, exception=None)

Subclass of `SAXException` raised when a SAX `~xml.sax.xmlreader.XMLReader` is asked to enable a feature that is not supported, or to set a property to a value that the implementation does not support. SAX applications and extensions may use this class for similar purposes.

</div>

<div class="exception">

SAXReaderNotAvailable(msg, exception=None)

Subclass of `SAXNotSupportedException` raised when no parser is available. A parser module raises it when it is imported or during parsing if the parser it provides cannot be used, and `make_parser` raises it if no module from the tried ones provides a usable parser.

</div>

<div class="seealso">

[SAX: The Simple API for XML](http://www.saxproject.org/)  
This site is the focal point for the definition of the SAX API. It provides a Java implementation and online documentation. Links to implementations and historical information are also available.

Module `xml.sax.handler`  
Definitions of the interfaces for application-provided objects.

Module `xml.sax.saxutils`  
Convenience functions for use in SAX applications.

Module `xml.sax.xmlreader`  
Definitions of the interfaces for parser-provided objects.

</div>

## SAXException Objects

The `SAXException` exception class supports the following methods:

<div class="method">

SAXException.getMessage()

Return a human-readable message describing the error condition.

</div>

<div class="method">

SAXException.getException()

Return an encapsulated exception object, or `None`.

</div>

# `!xml.sax.xmlreader` --- Interface for XML parsers

<div class="module" data-synopsis="Interface which SAX-compliant XML parsers must implement.">

xml.sax.xmlreader

</div>

**Source code:** `Lib/xml/sax/xmlreader.py`

------------------------------------------------------------------------

SAX parsers implement the `XMLReader` interface. They are implemented in a Python module, which must provide a function `create_parser`. This function is invoked by `xml.sax.make_parser` with no arguments to create a new parser object.

<div class="XMLReader()">

Base class which can be inherited by SAX parsers.

</div>

<div class="IncrementalParser()">

In some cases, it is desirable not to parse an input source at once, but to feed chunks of the document as they get available. Note that the reader will normally not read the entire file, but read it in chunks as well; still `parse` won't return until the entire document is processed. So these interfaces should be used if the blocking behaviour of `parse` is not desirable.

When the parser is instantiated it is ready to begin accepting data from the feed method immediately. After parsing has been finished with a call to close the reset method must be called to make the parser ready to accept new data, either from feed or using the parse method.

Note that these methods must *not* be called during parsing, that is, after parse has been called and before it returns.

By default, the class also implements the parse method of the XMLReader interface using the feed, close and reset methods of the IncrementalParser interface as a convenience to SAX 2.0 driver writers.

</div>

<div class="Locator()">

Interface for associating a SAX event with a document location. A locator object will return valid results only during calls to DocumentHandler methods; at any other time, the results are unpredictable. If information is not available, methods may return `None`.

</div>

<div class="InputSource(system_id=None)">

Encapsulation of the information needed by the `XMLReader` to read entities.

This class may include information about the public identifier, system identifier, byte stream (possibly with character encoding information) and/or the character stream of an entity.

Applications will create objects of this class for use in the `XMLReader.parse` method and for returning from EntityResolver.resolveEntity.

An `InputSource` belongs to the application, the `XMLReader` is not allowed to modify `InputSource` objects passed to it from the application, although it may make copies and modify those.

</div>

<div class="AttributesImpl(attrs)">

This is an implementation of the `Attributes` interface (see section `attributes-objects`). This is a dictionary-like object which represents the element attributes in a `startElement` call. In addition to the most useful dictionary operations, it supports a number of other methods as described by the interface. Objects of this class should be instantiated by readers; *attrs* must be a dictionary-like object containing a mapping from attribute names to attribute values.

</div>

<div class="AttributesNSImpl(attrs, qnames)">

Namespace-aware variant of `AttributesImpl`, which will be passed to `startElementNS`. It is derived from `AttributesImpl`, but understands attribute names as two-tuples of *namespaceURI* and *localname*. In addition, it provides a number of methods expecting qualified names as they appear in the original document. This class implements the `AttributesNS` interface (see section `attributes-ns-objects`).

</div>

## XMLReader Objects

The `XMLReader` interface supports the following methods:

<div class="method">

XMLReader.parse(source)

Process an input source, producing SAX events. The *source* object can be a system identifier (a string identifying the input source -- typically a file name or a URL), a `pathlib.Path` or `path-like <path-like object>` object, or an `InputSource` object. When `parse` returns, the input is completely processed, and the parser object can be discarded or reset.

<div class="versionchanged">

3.5 Added support of character streams.

</div>

<div class="versionchanged">

3.8 Added support of path-like objects.

</div>

</div>

<div class="method">

XMLReader.getContentHandler()

Return the current `~xml.sax.handler.ContentHandler`.

</div>

<div class="method">

XMLReader.setContentHandler(handler)

Set the current `~xml.sax.handler.ContentHandler`. If no `~xml.sax.handler.ContentHandler` is set, content events will be discarded.

</div>

<div class="method">

XMLReader.getDTDHandler()

Return the current `~xml.sax.handler.DTDHandler`.

</div>

<div class="method">

XMLReader.setDTDHandler(handler)

Set the current `~xml.sax.handler.DTDHandler`. If no `~xml.sax.handler.DTDHandler` is set, DTD events will be discarded.

</div>

<div class="method">

XMLReader.getEntityResolver()

Return the current `~xml.sax.handler.EntityResolver`.

</div>

<div class="method">

XMLReader.setEntityResolver(handler)

Set the current `~xml.sax.handler.EntityResolver`. If no `~xml.sax.handler.EntityResolver` is set, attempts to resolve an external entity will result in opening the system identifier for the entity, and fail if it is not available.

</div>

<div class="method">

XMLReader.getErrorHandler()

Return the current `~xml.sax.handler.ErrorHandler`.

</div>

<div class="method">

XMLReader.setErrorHandler(handler)

Set the current error handler. If no `~xml.sax.handler.ErrorHandler` is set, errors will be raised as exceptions, and warnings will be printed.

</div>

<div class="method">

XMLReader.setLocale(locale)

Allow an application to set the locale for errors and warnings.

SAX parsers are not required to provide localization for errors and warnings; if they cannot support the requested locale, however, they must raise a SAX exception. Applications may request a locale change in the middle of a parse.

</div>

<div class="method">

XMLReader.getFeature(featurename)

Return the current setting for feature *featurename*. If the feature is not recognized, `SAXNotRecognizedException` is raised. The well-known featurenames are listed in the module `xml.sax.handler`.

</div>

<div class="method">

XMLReader.setFeature(featurename, value)

Set the *featurename* to *value*. If the feature is not recognized, `SAXNotRecognizedException` is raised. If the feature or its setting is not supported by the parser, *SAXNotSupportedException* is raised.

</div>

<div class="method">

XMLReader.getProperty(propertyname)

Return the current setting for property *propertyname*. If the property is not recognized, a `SAXNotRecognizedException` is raised. The well-known propertynames are listed in the module `xml.sax.handler`.

</div>

<div class="method">

XMLReader.setProperty(propertyname, value)

Set the *propertyname* to *value*. If the property is not recognized, `SAXNotRecognizedException` is raised. If the property or its setting is not supported by the parser, *SAXNotSupportedException* is raised.

</div>

## IncrementalParser Objects

Instances of `IncrementalParser` offer the following additional methods:

<div class="method">

IncrementalParser.feed(data)

Process a chunk of *data*.

</div>

<div class="method">

IncrementalParser.close()

Assume the end of the document. That will check well-formedness conditions that can be checked only at the end, invoke handlers, and may clean up resources allocated during parsing.

</div>

<div class="method">

IncrementalParser.prepareParser(source)

Prepare the parser for parsing *source*, an `InputSource` instance. It is called by `~XMLReader.parse` before feeding the data. The parser implementation must override this method; the default implementation raises `NotImplementedError`.

</div>

<div class="method">

IncrementalParser.reset()

This method is called after close has been called to reset the parser so that it is ready to parse new documents. The results of calling parse or feed after close without calling reset are undefined.

</div>

## Locator Objects

Instances of `Locator` provide these methods:

<div class="method">

Locator.getColumnNumber()

Return the column number where the current event begins.

</div>

<div class="method">

Locator.getLineNumber()

Return the line number where the current event begins.

</div>

<div class="method">

Locator.getPublicId()

Return the public identifier for the current event.

</div>

<div class="method">

Locator.getSystemId()

Return the system identifier for the current event.

</div>

## InputSource Objects

<div class="method">

InputSource.setPublicId(id)

Sets the public identifier of this `InputSource`.

</div>

<div class="method">

InputSource.getPublicId()

Returns the public identifier of this `InputSource`.

</div>

<div class="method">

InputSource.setSystemId(id)

Sets the system identifier of this `InputSource`.

</div>

<div class="method">

InputSource.getSystemId()

Returns the system identifier of this `InputSource`.

</div>

<div class="method">

InputSource.setEncoding(encoding)

Sets the character encoding of this `InputSource`.

The encoding must be a string acceptable for an XML encoding declaration (see section 4.3.3 of the XML recommendation).

The encoding attribute of the `InputSource` is ignored if the `InputSource` also contains a character stream.

</div>

<div class="method">

InputSource.getEncoding()

Get the character encoding of this InputSource.

</div>

<div class="method">

InputSource.setByteStream(bytefile)

Set the byte stream (a `binary file`) for this input source.

The SAX parser will ignore this if there is also a character stream specified, but it will use a byte stream in preference to opening a URI connection itself.

If the application knows the character encoding of the byte stream, it should set it with the setEncoding method.

</div>

<div class="method">

InputSource.getByteStream()

Get the byte stream for this input source.

The getEncoding method will return the character encoding for this byte stream, or `None` if unknown.

</div>

<div class="method">

InputSource.setCharacterStream(charfile)

Set the character stream (a `text file`) for this input source.

If there is a character stream specified, the SAX parser will ignore any byte stream and will not attempt to open a URI connection to the system identifier.

</div>

<div class="method">

InputSource.getCharacterStream()

Get the character stream for this input source.

</div>

## The `Attributes` Interface

`Attributes` objects implement a portion of the `mapping protocol
<mapping>`, including the methods `~collections.abc.Mapping.copy`, `~collections.abc.Mapping.get`, `~object.__contains__`, `~collections.abc.Mapping.items`, `~collections.abc.Mapping.keys`, and `~collections.abc.Mapping.values`. The following methods are also provided:

<div class="method">

Attributes.getLength()

Return the number of attributes.

</div>

<div class="method">

Attributes.getNames()

Return the names of the attributes.

</div>

<div class="method">

Attributes.getType(name)

Returns the type of the attribute *name*, which is normally `'CDATA'`.

</div>

<div class="method">

Attributes.getValue(name)

Return the value of attribute *name*.

</div>

## The `AttributesNS` Interface

This interface is a subtype of the `Attributes` interface (see section `attributes-objects`). All methods supported by that interface are also available on `AttributesNS` objects.

The following methods are also available:

<div class="method">

AttributesNS.getValueByQName(name)

Return the value for a qualified name.

</div>

<div class="method">

AttributesNS.getNameByQName(name)

Return the `(namespace, localname)` pair for a qualified *name*.

</div>

<div class="method">

AttributesNS.getQNameByName(name)

Return the qualified name for a `(namespace, localname)` pair.

</div>

<div class="method">

AttributesNS.getQNames()

Return the qualified names of all attributes.

</div>

# `!xml.sax.saxutils` --- SAX Utilities

<div class="module" data-synopsis="Convenience functions and classes for use with SAX.">

xml.sax.saxutils

</div>

**Source code:** `Lib/xml/sax/saxutils.py`

------------------------------------------------------------------------

The module `!xml.sax.saxutils` contains a number of classes and functions that are commonly useful when creating SAX applications, either in direct use, or as base classes.

<div class="function">

escape(data, entities={})

Escape `'&'`, `'<'`, and `'>'` in a string of data.

You can escape other strings of data by passing a dictionary as the optional *entities* parameter. The keys and values must all be strings; each key will be replaced with its corresponding value. The characters `'&'`, `'<'` and `'>'` are always escaped, even if *entities* is provided.

> [!NOTE]
> This function should only be used to escape characters that can't be used directly in XML. Do not use this function as a general string translation function.

</div>

<div class="function">

unescape(data, entities={})

Unescape `'&amp;'`, `'&lt;'`, and `'&gt;'` in a string of data.

You can unescape other strings of data by passing a dictionary as the optional *entities* parameter. The keys and values must all be strings; each key will be replaced with its corresponding value. `'&amp;'`, `'&lt;'`, and `'&gt;'` are always unescaped, even if *entities* is provided.

</div>

<div class="function">

quoteattr(data, entities={})

Similar to `escape`, but also prepares *data* to be used as an attribute value. The return value is a quoted version of *data* with any additional required replacements. `quoteattr` will select a quote character based on the content of *data*, attempting to avoid encoding any quote characters in the string. If both single- and double-quote characters are already in *data*, the double-quote characters will be encoded and *data* will be wrapped in double-quotes. The resulting string can be used directly as an attribute value:

    >>> print("<element attr=%s>" % quoteattr("ab ' cd \" ef"))
    <element attr="ab ' cd &quot; ef">

This function is useful when generating attribute values for HTML or any SGML using the reference concrete syntax.

</div>

<div class="XMLGenerator(out=None, encoding='iso-8859-1', short_empty_elements=False)">

This class implements the `~xml.sax.handler.ContentHandler` interface by writing SAX events back into an XML document. In other words, using an `XMLGenerator` as the content handler will reproduce the original document being parsed. *out* should be a file-like object which will default to *sys.stdout*. *encoding* is the encoding of the output stream which defaults to `'iso-8859-1'`. *short_empty_elements* controls the formatting of elements that contain no content: if `False` (the default) they are emitted as a pair of start/end tags, if set to `True` they are emitted as a single self-closed tag.

<div class="versionchanged">

3.2 Added the *short_empty_elements* parameter.

</div>

</div>

<div class="XMLFilterBase(base)">

This class is designed to sit between an `~xml.sax.xmlreader.XMLReader` and the client application's event handlers. By default, it does nothing but pass requests up to the reader and events on to the handlers unmodified, but subclasses can override specific methods to modify the event stream or the configuration requests as they pass through.

<div class="method">

getParent()

Return the parent reader, or `None` if it is not set.

</div>

<div class="method">

setParent(parent)

Set the parent reader, which the events are read from.

</div>

</div>

<div class="function">

prepare_input_source(source, base='')

This function takes an input source and an optional base URL and returns a fully resolved `~xml.sax.xmlreader.InputSource` object ready for reading. The input source can be given as a string, a file-like object, or an `~xml.sax.xmlreader.InputSource` object; parsers will use this function to implement the polymorphic *source* argument to their `~xml.sax.xmlreader.XMLReader.parse` method.

</div>

# `!xmlrpc.client` --- XML-RPC client access

<div class="module" data-synopsis="XML-RPC client access.">

xmlrpc.client

</div>

**Source code:** `Lib/xmlrpc/client.py`

------------------------------------------------------------------------

XML-RPC is a Remote Procedure Call method that uses XML passed via HTTP(S) as a transport. With it, a client can call methods with parameters on a remote server (the server is named by a URI) and get back structured data. This module supports writing XML-RPC client code; it handles all the details of translating between conformable Python objects and XML on the wire.

> [!WARNING]
> The `!xmlrpc.client` module is not secure against maliciously constructed data. If you need to parse untrusted or unauthenticated data, see `xml-security`.

<div class="versionchanged">

3.5

For HTTPS URIs, `!xmlrpc.client` now performs all the necessary certificate and hostname checks by default.

</div>

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

<div class="ServerProxy(uri, transport=None, encoding=None, verbose=False, \ allow_none=False, use_datetime=False, \ use_builtin_types=False, *, headers=(), context=None)">

A `ServerProxy` instance is an object that manages communication with a remote XML-RPC server. The required first argument is a URI (Uniform Resource Indicator), and will normally be the URL of the server. The optional second argument is a transport factory instance; by default it is an internal `SafeTransport` instance for https: URLs and an internal HTTP `Transport` instance otherwise. The optional third argument is an encoding, by default UTF-8. The optional fourth argument is a debugging flag.

The following parameters govern the use of the returned proxy instance. If *allow_none* is true, the Python constant `None` will be translated into XML; the default behaviour is for `None` to raise a `TypeError`. This is a commonly used extension to the XML-RPC specification, but isn't supported by all clients and servers; see [http://ontosys.com/xml-rpc/extensions.php](https://web.archive.org/web/20130120074804/http://ontosys.com/xml-rpc/extensions.php) for a description. The *use_builtin_types* flag can be used to cause date/time values to be presented as `datetime.datetime` objects and binary data to be presented as `bytes` objects; this flag is false by default. `datetime.datetime`, `bytes` and `bytearray` objects may be passed to calls. The *headers* parameter is an optional sequence of HTTP headers to send with each request, expressed as a sequence of 2-tuples representing the header name and value. (e.g. `[('Header-Name', 'value')]`). If an HTTPS URL is provided, *context* may be `ssl.SSLContext` and configures the SSL settings of the underlying HTTPS connection. The obsolete *use_datetime* flag is similar to *use_builtin_types* but it applies only to date/time values.

<div class="versionchanged">

3.3 The *use_builtin_types* flag was added.

</div>

<div class="versionchanged">

3.8 The *headers* parameter was added.

</div>

Both the HTTP and HTTPS transports support the URL syntax extension for HTTP Basic Authentication: `http://user:pass@host:port/path`. The `user:pass` portion will be base64-encoded as an HTTP 'Authorization' header, and sent to the remote server as part of the connection process when invoking an XML-RPC method. You only need to use this if the remote server requires a Basic Authentication user and password.

The returned instance is a proxy object with methods that can be used to invoke corresponding RPC calls on the remote server. If the remote server supports the introspection API, the proxy can also be used to query the remote server for the methods it supports (service discovery) and fetch other server-associated metadata.

Types that are conformable (e.g. that can be marshalled through XML), include the following (and except where noted, they are unmarshalled as the same Python type):

<div class="tabularcolumns">

\|l\|L\|

</div>

| XML-RPC type | Python type |
|----|----|
| `boolean` | `bool` |
| `int`, `i1`, `i2`, `i4`, `i8` or `biginteger` | `int` in range from -2147483648 to 2147483647. Values get the `<int>` tag. |
| `double` or `float` | `float`. Values get the `<double>` tag. |
| `string` | `str` |
| `array` | `list` or `tuple` containing conformable elements. Arrays are returned as `lists <list>`. |
| `struct` | `dict`. Keys must be strings, values may be any conformable type. Objects of user-defined classes can be passed in; only their `~object.__dict__` attribute is transmitted. |
| `dateTime.iso8601` | `DateTime` or `datetime.datetime`. Returned type depends on values of *use_builtin_types* and *use_datetime* flags. |
| `base64` | `Binary`, `bytes` or `bytearray`. Returned type depends on the value of the *use_builtin_types* flag. |
| `nil` | The `None` constant. Passing is allowed only if *allow_none* is true. |
| `bigdecimal` | `decimal.Decimal`. Returned type only. |

This is the full set of data types supported by XML-RPC. Method calls may also raise a special `Fault` instance, used to signal XML-RPC server errors, or `ProtocolError` used to signal an error in the HTTP/HTTPS transport layer. Both `Fault` and `ProtocolError` derive from a base class called `Error`. Note that the xmlrpc client module currently does not marshal instances of subclasses of built-in types.

When passing strings, characters special to XML such as `<`, `>`, and `&` will be automatically escaped. However, it's the caller's responsibility to ensure that the string is free of characters that aren't allowed in XML, such as the control characters with ASCII values between 0 and 31 (except, of course, tab, newline and carriage return); failing to do this will result in an XML-RPC request that isn't well-formed XML. If you have to pass arbitrary bytes via XML-RPC, use `bytes` or `bytearray` classes or the `Binary` wrapper class described below.

`Server` is retained as an alias for `ServerProxy` for backwards compatibility. New code should use `ServerProxy`.

<div class="versionchanged">

3.5 Added the *context* argument.

</div>

<div class="versionchanged">

3.6 Added support of type tags with prefixes (e.g. `ex:nil`). Added support of unmarshalling additional types used by Apache XML-RPC implementation for numerics: `i1`, `i2`, `i8`, `biginteger`, `float` and `bigdecimal`. See <https://ws.apache.org/xmlrpc/types.html> for a description.

</div>

</div>

<div class="seealso">

[XML-RPC HOWTO](https://tldp.org/HOWTO/XML-RPC-HOWTO/index.html)  
A good description of XML-RPC operation and client software in several languages. Contains pretty much everything an XML-RPC client developer needs to know.

[XML-RPC Introspection](https://xmlrpc-c.sourceforge.io/introspection.html)  
Describes the XML-RPC protocol extension for introspection.

[XML-RPC Specification](http://xmlrpc.scripting.com/spec.html)  
The official specification.

</div>

## ServerProxy Objects

A `ServerProxy` instance has a method corresponding to each remote procedure call accepted by the XML-RPC server. Calling the method performs an RPC, dispatched by both name and argument signature (e.g. the same method name can be overloaded with multiple argument signatures). The RPC finishes either by returning data in a conformant type or by raising a `Fault` or `ProtocolError` exception indicating an error.

Servers that support the XML introspection API support some common methods grouped under the reserved `~ServerProxy.system` attribute:

<div class="method">

ServerProxy.system.listMethods()

This method returns a list of strings, one for each (non-system) method supported by the XML-RPC server.

</div>

<div class="method">

ServerProxy.system.methodSignature(name)

This method takes one parameter, the name of a method implemented by the XML-RPC server. It returns an array of possible signatures for this method. A signature is an array of types. The first of these types is the return type of the method, the rest are parameters.

Because multiple signatures (ie. overloading) is permitted, this method returns a list of signatures rather than a singleton.

Signatures themselves are restricted to the top level parameters expected by a method. For instance if a method expects one array of structs as a parameter, and it returns a string, its signature is simply "string, array". If it expects three integers and returns a string, its signature is "string, int, int, int".

If no signature is defined for the method, a non-array value is returned. In Python this means that the type of the returned value will be something other than list.

</div>

<div class="method">

ServerProxy.system.methodHelp(name)

This method takes one parameter, the name of a method implemented by the XML-RPC server. It returns a documentation string describing the use of that method. If no such string is available, an empty string is returned. The documentation string may contain HTML markup.

</div>

<div class="versionchanged">

3.5

Instances of `ServerProxy` support the `context manager` protocol for closing the underlying transport.

</div>

A working example follows. The server code:

    from xmlrpc.server import SimpleXMLRPCServer

    def is_even(n):
        return n % 2 == 0

    server = SimpleXMLRPCServer(("localhost", 8000))
    print("Listening on port 8000...")
    server.register_function(is_even, "is_even")
    server.serve_forever()

The client code for the preceding server:

    import xmlrpc.client

    with xmlrpc.client.ServerProxy("http://localhost:8000/") as proxy:
        print("3 is even: %s" % str(proxy.is_even(3)))
        print("100 is even: %s" % str(proxy.is_even(100)))

## DateTime Objects

<div class="DateTime">

This class may be initialized with seconds since the epoch, a time tuple, an ISO 8601 time/date string, or a `datetime.datetime` instance. It has the following methods, supported mainly for internal use by the marshalling/unmarshalling code:

<div class="method">

decode(string)

Accept a string as the instance's new time value.

</div>

<div class="method">

encode(out)

Write the XML-RPC encoding of this `DateTime` item to the *out* stream object.

</div>

It also supports certain of Python's built-in operators through `rich comparison <object.__lt__>` and `~object.__repr__` methods.

</div>

A working example follows. The server code:

    import datetime as dt
    from xmlrpc.server import SimpleXMLRPCServer
    import xmlrpc.client

    def today():
        today = dt.datetime.today()
        return xmlrpc.client.DateTime(today)

    server = SimpleXMLRPCServer(("localhost", 8000))
    print("Listening on port 8000...")
    server.register_function(today, "today")
    server.serve_forever()

The client code for the preceding server:

    import xmlrpc.client
    import datetime as dt

    proxy = xmlrpc.client.ServerProxy("http://localhost:8000/")

    today = proxy.today()
    # convert the ISO 8601 string to a datetime object
    converted = dt.datetime.strptime(today.value, "%Y%m%dT%H:%M:%S")
    print(f"Today: {converted.strftime('%d.%m.%Y, %H:%M')}")

## Binary Objects

<div class="Binary">

This class may be initialized from bytes data (which may include NULs). The primary access to the content of a `Binary` object is provided by an attribute:

<div class="attribute">

data

The binary data encapsulated by the `Binary` instance. The data is provided as a `bytes` object.

</div>

`Binary` objects have the following methods, supported mainly for internal use by the marshalling/unmarshalling code:

<div class="method">

decode(bytes)

Accept a base64 `bytes` object and decode it as the instance's new data.

</div>

<div class="method">

encode(out)

Write the XML-RPC base 64 encoding of this binary item to the *out* stream object.

The encoded data will have newlines every 76 characters as per `RFC 2045 section 6.8 <2045#section-6.8>`, which was the de facto standard base64 specification when the XML-RPC spec was written.

</div>

It also supports certain of Python's built-in operators through `~object.__eq__` and `~object.__ne__` methods.

</div>

Example usage of the binary objects. We're going to transfer an image over XMLRPC:

    from xmlrpc.server import SimpleXMLRPCServer
    import xmlrpc.client

    def python_logo():
        with open("python_logo.jpg", "rb") as handle:
            return xmlrpc.client.Binary(handle.read())

    server = SimpleXMLRPCServer(("localhost", 8000))
    print("Listening on port 8000...")
    server.register_function(python_logo, 'python_logo')

    server.serve_forever()

The client gets the image and saves it to a file:

    import xmlrpc.client

    proxy = xmlrpc.client.ServerProxy("http://localhost:8000/")
    with open("fetched_python_logo.jpg", "wb") as handle:
        handle.write(proxy.python_logo().data)

## Fault Objects

<div class="Fault">

A `Fault` object encapsulates the content of an XML-RPC fault tag. Fault objects have the following attributes:

<div class="attribute">

faultCode

An int indicating the fault type.

</div>

<div class="attribute">

faultString

A string containing a diagnostic message associated with the fault.

</div>

</div>

In the following example we're going to intentionally cause a `Fault` by returning a complex type object. The server code:

    from xmlrpc.server import SimpleXMLRPCServer

    # A marshalling error is going to occur because we're returning a
    # complex number
    def add(x, y):
        return x+y+0j

    server = SimpleXMLRPCServer(("localhost", 8000))
    print("Listening on port 8000...")
    server.register_function(add, 'add')

    server.serve_forever()

The client code for the preceding server:

    import xmlrpc.client

    proxy = xmlrpc.client.ServerProxy("http://localhost:8000/")
    try:
        proxy.add(2, 5)
    except xmlrpc.client.Fault as err:
        print("A fault occurred")
        print("Fault code: %d" % err.faultCode)
        print("Fault string: %s" % err.faultString)

## ProtocolError Objects

<div class="ProtocolError">

A `ProtocolError` object describes a protocol error in the underlying transport layer (such as a 404 'not found' error if the server named by the URI does not exist). It has the following attributes:

<div class="attribute">

url

The URI or URL that triggered the error.

</div>

<div class="attribute">

errcode

The error code.

</div>

<div class="attribute">

errmsg

The error message or diagnostic string.

</div>

<div class="attribute">

headers

A dict containing the headers of the HTTP/HTTPS request that triggered the error.

</div>

</div>

In the following example we're going to intentionally cause a `ProtocolError` by providing an invalid URI:

    import xmlrpc.client

    # create a ServerProxy with a URI that doesn't respond to XMLRPC requests
    proxy = xmlrpc.client.ServerProxy("http://google.com/")

    try:
        proxy.some_method()
    except xmlrpc.client.ProtocolError as err:
        print("A protocol error occurred")
        print("URL: %s" % err.url)
        print("HTTP/HTTPS headers: %s" % err.headers)
        print("Error code: %d" % err.errcode)
        print("Error message: %s" % err.errmsg)

## MultiCall Objects

The `MultiCall` object provides a way to encapsulate multiple calls to a remote server into a single request[^1].

<div class="MultiCall(server)">

Create an object used to boxcar method calls. *server* is the eventual target of the call. Calls can be made to the result object, but they will immediately return `None`, and only store the call name and arguments in the `MultiCall` object. Calling the object itself causes all stored calls to be transmitted as a single `system.multicall` request. The result of this call is a `generator`; iterating over this generator yields the individual results.

</div>

A usage example of this class follows. The server code:

    from xmlrpc.server import SimpleXMLRPCServer

    def add(x, y):
        return x + y

    def subtract(x, y):
        return x - y

    def multiply(x, y):
        return x * y

    def divide(x, y):
        return x // y

    # A simple server with simple arithmetic functions
    server = SimpleXMLRPCServer(("localhost", 8000))
    print("Listening on port 8000...")
    server.register_multicall_functions()
    server.register_function(add, 'add')
    server.register_function(subtract, 'subtract')
    server.register_function(multiply, 'multiply')
    server.register_function(divide, 'divide')
    server.serve_forever()

The client code for the preceding server:

    import xmlrpc.client

    proxy = xmlrpc.client.ServerProxy("http://localhost:8000/")
    multicall = xmlrpc.client.MultiCall(proxy)
    multicall.add(7, 3)
    multicall.subtract(7, 3)
    multicall.multiply(7, 3)
    multicall.divide(7, 3)
    result = multicall()

    print("7+3=%d, 7-3=%d, 7*3=%d, 7//3=%d" % tuple(result))

## Convenience Functions

<div class="function">

dumps(params, methodname=None, methodresponse=None, encoding=None, allow_none=False)

Convert *params* into an XML-RPC request, or into a response if *methodresponse* is true. *params* can be either a tuple of arguments or an instance of the `Fault` exception class. If *methodresponse* is true, only a single value can be returned, meaning that *params* must be of length 1. *encoding*, if supplied, is the encoding to use in the generated XML; the default is UTF-8. Python's `None` value cannot be used in standard XML-RPC; to allow using it via an extension, provide a true value for *allow_none*.

</div>

<div class="function">

loads(data, use_datetime=False, use_builtin_types=False)

Convert an XML-RPC request or response into Python objects, a `(params, methodname)`. *params* is a tuple of argument; *methodname* is a string, or `None` if no method name is present in the packet. If the XML-RPC packet represents a fault condition, this function will raise a `Fault` exception. The *use_builtin_types* flag can be used to cause date/time values to be presented as `datetime.datetime` objects and binary data to be presented as `bytes` objects; this flag is false by default.

The obsolete *use_datetime* flag is similar to *use_builtin_types* but it applies only to date/time values.

<div class="versionchanged">

3.3 The *use_builtin_types* flag was added.

</div>

</div>

## Example of Client Usage

    # simple test program (from the XML-RPC specification)
    from xmlrpc.client import ServerProxy, Error

    # server = ServerProxy("http://localhost:8000") # local server
    with ServerProxy("http://betty.userland.com") as proxy:

        print(proxy)

        try:
            print(proxy.examples.getStateName(41))
        except Error as v:
            print("ERROR", v)

To access an XML-RPC server through a HTTP proxy, you need to define a custom transport. The following example shows how:

    import http.client
    import xmlrpc.client

    class ProxiedTransport(xmlrpc.client.Transport):

        def set_proxy(self, host, port=None, headers=None):
            self.proxy = host, port
            self.proxy_headers = headers

        def make_connection(self, host):
            connection = http.client.HTTPConnection(*self.proxy)
            connection.set_tunnel(host, headers=self.proxy_headers)
            self._connection = host, connection
            return connection

    transport = ProxiedTransport()
    transport.set_proxy('proxy-server', 8080)
    server = xmlrpc.client.ServerProxy('http://betty.userland.com', transport=transport)
    print(server.examples.getStateName(41))

## Example of Client and Server Usage

See `simplexmlrpcserver-example`.

**Footnotes**

[^1]: This approach has been first presented in [a discussion on xmlrpc.com](https://web.archive.org/web/20060624230303/http://www.xmlrpc.com/discuss/msgReader$1208?mode=topic).

# `!xmlrpc` --- XMLRPC server and client modules

<div class="module" data-synopsis="Server and client modules implementing XML-RPC.">

xmlrpc

</div>

XML-RPC is a Remote Procedure Call method that uses XML passed via HTTP as a transport. With it, a client can call methods with parameters on a remote server (the server is named by a URI) and get back structured data.

`xmlrpc` is a package that collects server and client modules implementing XML-RPC. The modules are:

- `xmlrpc.client`
- `xmlrpc.server`

# `!xmlrpc.server` --- Basic XML-RPC servers

<div class="module" data-synopsis="Basic XML-RPC server implementations.">

xmlrpc.server

</div>

**Source code:** `Lib/xmlrpc/server.py`

------------------------------------------------------------------------

The `!xmlrpc.server` module provides a basic server framework for XML-RPC servers written in Python. Servers can either be free standing, using `SimpleXMLRPCServer`, or embedded in a CGI environment, using `CGIXMLRPCRequestHandler`.

> [!WARNING]
> The `!xmlrpc.server` module is not secure against maliciously constructed data. If you need to parse untrusted or unauthenticated data, see `xml-security`.

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

<div class="SimpleXMLRPCServer(addr, requestHandler=SimpleXMLRPCRequestHandler,\ logRequests=True, allow_none=False, encoding=None,\ bind_and_activate=True, use_builtin_types=False)">

Create a new server instance. This class provides methods for registration of functions that can be called by the XML-RPC protocol. The *requestHandler* parameter should be a factory for request handler instances; it defaults to `SimpleXMLRPCRequestHandler`. The *addr* and *requestHandler* parameters are passed to the `socketserver.TCPServer` constructor. If *logRequests* is true (the default), requests will be logged; setting this parameter to false will turn off logging. The *allow_none* and *encoding* parameters are passed on to `xmlrpc.client` and control the XML-RPC responses that will be returned from the server. The *bind_and_activate* parameter controls whether `server_bind` and `server_activate` are called immediately by the constructor; it defaults to true. Setting it to false allows code to manipulate the *allow_reuse_address* class variable before the address is bound. The *use_builtin_types* parameter is passed to the `~xmlrpc.client.loads` function and controls which types are processed when date/times values or binary data are received; it defaults to false.

<div class="versionchanged">

3.3 The *use_builtin_types* flag was added.

</div>

</div>

<div class="CGIXMLRPCRequestHandler(allow_none=False, encoding=None,\ use_builtin_types=False)">

Create a new instance to handle XML-RPC requests in a CGI environment. The *allow_none* and *encoding* parameters are passed on to `xmlrpc.client` and control the XML-RPC responses that will be returned from the server. The *use_builtin_types* parameter is passed to the `~xmlrpc.client.loads` function and controls which types are processed when date/times values or binary data are received; it defaults to false.

<div class="versionchanged">

3.3 The *use_builtin_types* flag was added.

</div>

</div>

<div class="SimpleXMLRPCRequestHandler()">

Create a new request handler instance. This request handler supports `POST` requests and modifies logging so that the *logRequests* parameter to the `SimpleXMLRPCServer` constructor parameter is honored.

</div>

## SimpleXMLRPCServer objects

The `SimpleXMLRPCServer` class is based on `socketserver.TCPServer` and provides a means of creating simple, stand alone XML-RPC servers.

<div class="method">

SimpleXMLRPCServer.register_function(function=None, name=None)

Register a function that can respond to XML-RPC requests. If *name* is given, it will be the method name associated with *function*, otherwise `function.__name__` will be used. *name* is a string, and may contain characters not legal in Python identifiers, including the period character.

This method can also be used as a decorator. When used as a decorator, *name* can only be given as a keyword argument to register *function* under *name*. If no *name* is given, `function.__name__` will be used.

<div class="versionchanged">

3.7 `register_function` can be used as a decorator.

</div>

</div>

<div class="method">

SimpleXMLRPCServer.register_instance(instance, allow_dotted_names=False)

Register an object which is used to expose method names which have not been registered using `register_function`. If *instance* contains a `_dispatch` method, it is called with the requested method name and the parameters from the request. Its API is `def _dispatch(self, method, params)` (note that *params* does not represent a variable argument list). If it calls an underlying function to perform its task, that function is called as `func(*params)`, expanding the parameter list. The return value from `_dispatch` is returned to the client as the result. If *instance* does not have a `_dispatch` method, it is searched for an attribute matching the name of the requested method.

If the optional *allow_dotted_names* argument is true and the instance does not have a `_dispatch` method, then if the requested method name contains periods, each component of the method name is searched for individually, with the effect that a simple hierarchical search is performed. The value found from this search is then called with the parameters from the request, and the return value is passed back to the client.

> [!WARNING]
> Enabling the *allow_dotted_names* option allows intruders to access your module's global variables and may allow intruders to execute arbitrary code on your machine. Only use this option on a secure, closed network.

</div>

<div class="method">

SimpleXMLRPCServer.register_introspection_functions()

Registers the XML-RPC introspection functions `system.listMethods`, `system.methodHelp` and `system.methodSignature`.

</div>

<div class="method">

SimpleXMLRPCServer.register_multicall_functions()

Registers the XML-RPC multicall function system.multicall.

</div>

<div class="attribute">

SimpleXMLRPCRequestHandler.rpc_paths

An attribute value that must be a tuple listing valid path portions of the URL for receiving XML-RPC requests. Requests posted to other paths will result in a 404 "no such page" HTTP error. If this tuple is empty, all paths will be considered valid. The default value is `('/', '/RPC2')`.

</div>

### SimpleXMLRPCServer example

Server code:

    from xmlrpc.server import SimpleXMLRPCServer
    from xmlrpc.server import SimpleXMLRPCRequestHandler

    # Restrict to a particular path.
    class RequestHandler(SimpleXMLRPCRequestHandler):
        rpc_paths = ('/RPC2',)

    # Create server
    with SimpleXMLRPCServer(('localhost', 8000),
                            requestHandler=RequestHandler) as server:
        server.register_introspection_functions()

        # Register pow() function; this will use the value of
        # pow.__name__ as the name, which is just 'pow'.
        server.register_function(pow)

        # Register a function under a different name
        def adder_function(x, y):
            return x + y
        server.register_function(adder_function, 'add')

        # Register an instance; all the methods of the instance are
        # published as XML-RPC methods (in this case, just 'mul').
        class MyFuncs:
            def mul(self, x, y):
                return x * y

        server.register_instance(MyFuncs())

        # Run the server's main loop
        server.serve_forever()

The following client code will call the methods made available by the preceding server:

    import xmlrpc.client

    s = xmlrpc.client.ServerProxy('http://localhost:8000')
    print(s.pow(2,3))  # Returns 2**3 = 8
    print(s.add(2,3))  # Returns 5
    print(s.mul(5,2))  # Returns 5*2 = 10

    # Print list of available methods
    print(s.system.listMethods())

`register_function` can also be used as a decorator. The previous server example can register functions in a decorator way:

    from xmlrpc.server import SimpleXMLRPCServer
    from xmlrpc.server import SimpleXMLRPCRequestHandler

    class RequestHandler(SimpleXMLRPCRequestHandler):
        rpc_paths = ('/RPC2',)

    with SimpleXMLRPCServer(('localhost', 8000),
                            requestHandler=RequestHandler) as server:
        server.register_introspection_functions()

        # Register pow() function; this will use the value of
        # pow.__name__ as the name, which is just 'pow'.
        server.register_function(pow)

        # Register a function under a different name, using
        # register_function as a decorator. *name* can only be given
        # as a keyword argument.
        @server.register_function(name='add')
        def adder_function(x, y):
            return x + y

        # Register a function under function.__name__.
        @server.register_function
        def mul(x, y):
            return x * y

        server.serve_forever()

The following example included in the `Lib/xmlrpc/server.py` module shows a server allowing dotted names and registering a multicall function.

> [!WARNING]
> Enabling the *allow_dotted_names* option allows intruders to access your module's global variables and may allow intruders to execute arbitrary code on your machine. Only use this example within a secure, closed network.

    import datetime as dt

    class ExampleService:
        def getData(self):
            return '42'

        class currentTime:
            @staticmethod
            def getCurrentTime():
                return dt.datetime.now()

    with SimpleXMLRPCServer(("localhost", 8000)) as server:
        server.register_function(pow)
        server.register_function(lambda x,y: x+y, 'add')
        server.register_instance(ExampleService(), allow_dotted_names=True)
        server.register_multicall_functions()
        print('Serving XML-RPC on localhost port 8000')
        try:
            server.serve_forever()
        except KeyboardInterrupt:
            print("\nKeyboard interrupt received, exiting.")
            sys.exit(0)

This ExampleService demo can be invoked from the command line:

    python -m xmlrpc.server

The client that interacts with the above server is included in `Lib/xmlrpc/client.py`:

    server = ServerProxy("http://localhost:8000")

    try:
        print(server.currentTime.getCurrentTime())
    except Error as v:
        print("ERROR", v)

    multi = MultiCall(server)
    multi.getData()
    multi.pow(2,9)
    multi.add(1,2)
    try:
        for response in multi():
            print(response)
    except Error as v:
        print("ERROR", v)

This client which interacts with the demo XMLRPC server can be invoked as:

    python -m xmlrpc.client

## CGIXMLRPCRequestHandler

The `CGIXMLRPCRequestHandler` class can be used to handle XML-RPC requests sent to Python CGI scripts.

<div class="method">

CGIXMLRPCRequestHandler.register_function(function=None, name=None)

Register a function that can respond to XML-RPC requests. If *name* is given, it will be the method name associated with *function*, otherwise `function.__name__` will be used. *name* is a string, and may contain characters not legal in Python identifiers, including the period character.

This method can also be used as a decorator. When used as a decorator, *name* can only be given as a keyword argument to register *function* under *name*. If no *name* is given, `function.__name__` will be used.

<div class="versionchanged">

3.7 `register_function` can be used as a decorator.

</div>

</div>

<div class="method">

CGIXMLRPCRequestHandler.register_instance(instance)

Register an object which is used to expose method names which have not been registered using `register_function`. If instance contains a `_dispatch` method, it is called with the requested method name and the parameters from the request; the return value is returned to the client as the result. If instance does not have a `_dispatch` method, it is searched for an attribute matching the name of the requested method; if the requested method name contains periods, each component of the method name is searched for individually, with the effect that a simple hierarchical search is performed. The value found from this search is then called with the parameters from the request, and the return value is passed back to the client.

</div>

<div class="method">

CGIXMLRPCRequestHandler.register_introspection_functions()

Register the XML-RPC introspection functions `system.listMethods`, `system.methodHelp` and `system.methodSignature`.

</div>

<div class="method">

CGIXMLRPCRequestHandler.register_multicall_functions()

Register the XML-RPC multicall function `system.multicall`.

</div>

<div class="method">

CGIXMLRPCRequestHandler.handle_request(request_text=None)

Handle an XML-RPC request. If *request_text* is given, it should be the POST data provided by the HTTP server, otherwise the contents of stdin will be used.

</div>

Example:

    class MyFuncs:
        def mul(self, x, y):
            return x * y


    handler = CGIXMLRPCRequestHandler()
    handler.register_function(pow)
    handler.register_function(lambda x,y: x+y, 'add')
    handler.register_introspection_functions()
    handler.register_instance(MyFuncs())
    handler.handle_request()

## Documenting XMLRPC server

These classes extend the above classes to serve HTML documentation in response to HTTP GET requests. Servers can either be free standing, using `DocXMLRPCServer`, or embedded in a CGI environment, using `DocCGIXMLRPCRequestHandler`.

<div class="DocXMLRPCServer(addr, requestHandler=DocXMLRPCRequestHandler,\ logRequests=True, allow_none=False, encoding=None,\ bind_and_activate=True, use_builtin_types=True)">

Create a new server instance. All parameters have the same meaning as for `SimpleXMLRPCServer`; *requestHandler* defaults to `DocXMLRPCRequestHandler`.

<div class="versionchanged">

3.3 The *use_builtin_types* flag was added.

</div>

</div>

<div class="DocCGIXMLRPCRequestHandler()">

Create a new instance to handle XML-RPC requests in a CGI environment.

</div>

<div class="DocXMLRPCRequestHandler()">

Create a new request handler instance. This request handler supports XML-RPC POST requests, documentation GET requests, and modifies logging so that the *logRequests* parameter to the `DocXMLRPCServer` constructor parameter is honored.

</div>

## DocXMLRPCServer objects

The `DocXMLRPCServer` class is derived from `SimpleXMLRPCServer` and provides a means of creating self-documenting, stand alone XML-RPC servers. HTTP POST requests are handled as XML-RPC method calls. HTTP GET requests are handled by generating pydoc-style HTML documentation. This allows a server to provide its own web-based documentation.

<div class="method">

DocXMLRPCServer.set_server_title(server_title)

Set the title used in the generated HTML documentation. This title will be used inside the HTML "title" element.

</div>

<div class="method">

DocXMLRPCServer.set_server_name(server_name)

Set the name used in the generated HTML documentation. This name will appear at the top of the generated documentation inside a "h1" element.

</div>

<div class="method">

DocXMLRPCServer.set_server_documentation(server_documentation)

Set the description used in the generated HTML documentation. This description will appear as a paragraph, below the server name, in the documentation.

</div>

## DocCGIXMLRPCRequestHandler

The `DocCGIXMLRPCRequestHandler` class is derived from `CGIXMLRPCRequestHandler` and provides a means of creating self-documenting, XML-RPC CGI scripts. HTTP POST requests are handled as XML-RPC method calls. HTTP GET requests are handled by generating pydoc-style HTML documentation. This allows a server to provide its own web-based documentation.

<div class="method">

DocCGIXMLRPCRequestHandler.set_server_title(server_title)

Set the title used in the generated HTML documentation. This title will be used inside the HTML "title" element.

</div>

<div class="method">

DocCGIXMLRPCRequestHandler.set_server_name(server_name)

Set the name used in the generated HTML documentation. This name will appear at the top of the generated documentation inside a "h1" element.

</div>

<div class="method">

DocCGIXMLRPCRequestHandler.set_server_documentation(server_documentation)

Set the description used in the generated HTML documentation. This description will appear as a paragraph, below the server name, in the documentation.

</div>

# `!zipapp` --- Manage executable Python zip archives

<div class="module" data-synopsis="Manage executable Python zip archives">

zipapp

</div>

<div class="versionadded">

3.5

</div>

**Source code:** `Lib/zipapp.py`

<div class="index">

single: Executable Zip Files

</div>

------------------------------------------------------------------------

This module provides tools to manage the creation of zip files containing Python code, which can be `executed directly by the Python interpreter
<using-on-interface-options>`. The module provides both a `zipapp-command-line-interface` and a `zipapp-python-api`.

## Basic Example

The following example shows how the `zipapp-command-line-interface` can be used to create an executable archive from a directory containing Python code. When run, the archive will execute the `main` function from the module `myapp` in the archive.

```
$ python -m zipapp myapp -m "myapp:main"
$ python myapp.pyz
<output from myapp>
```

## Command-Line Interface

When called as a program from the command line, the following form is used:

```
$ python -m zipapp source [options]
```

If *source* is a directory, this will create an archive from the contents of *source*. If *source* is a file, it should be an archive, and it will be copied to the target archive (or the contents of its shebang line will be displayed if the --info option is specified).

The following options are understood:

<div class="program">

zipapp

</div>

<div class="option">

`-o <output>, --output=<output>`  

Write the output to a file named *output*. If this option is not specified, the output filename will be the same as the input *source*, with the extension `.pyz` added. If an explicit filename is given, it is used as is (so a `.pyz` extension should be included if required).

An output filename must be specified if the *source* is an archive (and in that case, *output* must not be the same as *source*).

</div>

<div class="option">

`-p <interpreter>, --python=<interpreter>`  

Add a `#!` line to the archive specifying *interpreter* as the command to run. Also, on POSIX, make the archive executable. The default is to write no `#!` line, and not make the file executable.

</div>

<div class="option">

`-m <mainfn>, --main=<mainfn>`  

Write a `__main__.py` file to the archive that executes *mainfn*. The *mainfn* argument should have the form "pkg.mod:fn", where "pkg.mod" is a package/module in the archive, and "fn" is a callable in the given module. The `__main__.py` file will execute that callable.

`--main` cannot be specified when copying an archive.

</div>

<div class="option">

`-c, --compress`  

Compress files with the deflate method, reducing the size of the output file. By default, files are stored uncompressed in the archive.

`--compress` has no effect when copying an archive.

<div class="versionadded">

3.7

</div>

</div>

<div class="option">

`--info`  

Display the interpreter embedded in the archive, for diagnostic purposes. In this case, any other options are ignored and SOURCE must be an archive, not a directory.

</div>

<div class="option">

`-h, --help`  

Print a short usage message and exit.

</div>

## Python API

The module defines two convenience functions:

<div class="function">

create_archive(source, target=None, interpreter=None, main=None, filter=None, compressed=False)

Create an application archive from *source*. The source can be any of the following:

- The name of a directory, or a `path-like object` referring to a directory, in which case a new application archive will be created from the content of that directory.
- The name of an existing application archive file, or a `path-like object` referring to such a file, in which case the file is copied to the target (modifying it to reflect the value given for the *interpreter* argument). The file name should include the `.pyz` extension, if required.
- A file object open for reading in bytes mode. The content of the file should be an application archive, and the file object is assumed to be positioned at the start of the archive.

The *target* argument determines where the resulting archive will be written:

- If it is the name of a file, or a `path-like object`, the archive will be written to that file.
- If it is an open file object, the archive will be written to that file object, which must be open for writing in bytes mode.
- If the target is omitted (or `None`), the source must be a directory and the target will be a file with the same name as the source, with a `.pyz` extension added.

The *interpreter* argument specifies the name of the Python interpreter with which the archive will be executed. It is written as a "shebang" line at the start of the archive. On POSIX, this will be interpreted by the OS, and on Windows it will be handled by the Python launcher. Omitting the *interpreter* results in no shebang line being written. If an interpreter is specified, and the target is a filename, the executable bit of the target file will be set.

The *main* argument specifies the name of a callable which will be used as the main program for the archive. It can only be specified if the source is a directory, and the source does not already contain a `__main__.py` file. The *main* argument should take the form "pkg.module:callable" and the archive will be run by importing "pkg.module" and executing the given callable with no arguments. It is an error to omit *main* if the source is a directory and does not contain a `__main__.py` file, as otherwise the resulting archive would not be executable.

The optional *filter* argument specifies a callback function that is passed a Path object representing the path to the file being added (relative to the source directory). It should return `True` if the file is to be added.

The optional *compressed* argument determines whether files are compressed. If set to `True`, files in the archive are compressed with the deflate method; otherwise, files are stored uncompressed. This argument has no effect when copying an existing archive.

If a file object is specified for *source* or *target*, it is the caller's responsibility to close it after calling create_archive.

When copying an existing archive, file objects supplied only need `read` and `readline`, or `write` methods. When creating an archive from a directory, if the target is a file object it will be passed to the `zipfile.ZipFile` class, and must supply the methods needed by that class.

<div class="versionchanged">

3.7 Added the *filter* and *compressed* parameters.

</div>

</div>

<div class="function">

get_interpreter(archive)

Return the interpreter specified in the `#!` line at the start of the archive. If there is no `#!` line, return `None`. The *archive* argument can be a filename or a file-like object open for reading in bytes mode. It is assumed to be at the start of the archive.

</div>

## Examples

Pack up a directory into an archive, and run it.

```
$ python -m zipapp myapp
$ python myapp.pyz
<output from myapp>
```

The same can be done using the `create_archive` function:

    >>> import zipapp
    >>> zipapp.create_archive('myapp', 'myapp.pyz')

To make the application directly executable on POSIX, specify an interpreter to use.

```
$ python -m zipapp myapp -p "/usr/bin/env python"
$ ./myapp.pyz
<output from myapp>
```

To replace the shebang line on an existing archive, create a modified archive using the `create_archive` function:

    >>> import zipapp
    >>> zipapp.create_archive('old_archive.pyz', 'new_archive.pyz', '/usr/bin/python3')

To update the file in place, do the replacement in memory using a `~io.BytesIO` object, and then overwrite the source afterwards. Note that there is a risk when overwriting a file in place that an error will result in the loss of the original file. This code does not protect against such errors, but production code should do so. Also, this method will only work if the archive fits in memory:

    >>> import zipapp
    >>> import io
    >>> temp = io.BytesIO()
    >>> zipapp.create_archive('myapp.pyz', temp, '/usr/bin/python2')
    >>> with open('myapp.pyz', 'wb') as f:
    ...     f.write(temp.getvalue())

## Specifying the Interpreter

Note that if you specify an interpreter and then distribute your application archive, you need to ensure that the interpreter used is portable. The Python launcher for Windows supports most common forms of POSIX `#!` line, but there are other issues to consider:

- If you use "/usr/bin/env python" (or other forms of the "python" command, such as "/usr/bin/python"), you need to consider that your users may have either Python 2 or Python 3 as their default, and write your code to work under both versions.
- If you use an explicit version, for example "/usr/bin/env python3" your application will not work for users who do not have that version. (This may be what you want if you have not made your code Python 2 compatible).
- There is no way to say "python X.Y or later", so be careful of using an exact version like "/usr/bin/env python3.4" as you will need to change your shebang line for users of Python 3.5, for example.

Typically, you should use an "/usr/bin/env python2" or "/usr/bin/env python3", depending on whether your code is written for Python 2 or 3.

## Creating Standalone Applications with zipapp

Using the `!zipapp` module, it is possible to create self-contained Python programs, which can be distributed to end users who only need to have a suitable version of Python installed on their system. The key to doing this is to bundle all of the application's dependencies into the archive, along with the application code.

The steps to create a standalone archive are as follows:

1.  Create your application in a directory as normal, so you have a `myapp` directory containing a `__main__.py` file, and any supporting application code.

2.  Install all of your application's dependencies into the `myapp` directory, using pip:

    ```
    $ python -m pip install -r requirements.txt --target myapp
    ```

    (this assumes you have your project requirements in a `requirements.txt` file - if not, you can just list the dependencies manually on the pip command line).

3.  Package the application using:

    ```
    $ python -m zipapp -p "interpreter" myapp
    ```

This will produce a standalone executable, which can be run on any machine with the appropriate interpreter available. See `zipapp-specifying-the-interpreter` for details. It can be shipped to users as a single file.

On Unix, the `myapp.pyz` file is executable as it stands. You can rename the file to remove the `.pyz` extension if you prefer a "plain" command name. On Windows, the `myapp.pyz[w]` file is executable by virtue of the fact that the Python interpreter registers the `.pyz` and `.pyzw` file extensions when installed.

### Caveats

If your application depends on a package that includes a C extension, that package cannot be run from a zip file (this is an OS limitation, as executable code must be present in the filesystem for the OS loader to load it). In this case, you can exclude that dependency from the zipfile, and either require your users to have it installed, or ship it alongside your zipfile and add code to your `__main__.py` to include the directory containing the unzipped module in `sys.path`. In this case, you will need to make sure to ship appropriate binaries for your target architecture(s) (and potentially pick the correct version to add to `sys.path` at runtime, based on the user's machine).

## The Python Zip Application Archive Format

Python has been able to execute zip files which contain a `__main__.py` file since version 2.6. In order to be executed by Python, an application archive simply has to be a standard zip file containing a `__main__.py` file which will be run as the entry point for the application. As usual for any Python script, the parent of the script (in this case the zip file) will be placed on `sys.path` and thus further modules can be imported from the zip file.

The zip file format allows arbitrary data to be prepended to a zip file. The zip application format uses this ability to prepend a standard POSIX "shebang" line to the file (`#!/path/to/interpreter`).

Formally, the Python zip application format is therefore:

1.  An optional shebang line, containing the characters `b'#!'` followed by an interpreter name, and then a newline (`b'\n'`) character. The interpreter name can be anything acceptable to the OS "shebang" processing, or the Python launcher on Windows. The interpreter should be encoded in UTF-8 on Windows, and in `sys.getfilesystemencoding` on POSIX.
2.  Standard zipfile data, as generated by the `zipfile` module. The zipfile content *must* include a file called `__main__.py` (which must be in the "root" of the zipfile - i.e., it cannot be in a subdirectory). The zipfile data can be compressed or uncompressed.

If an application archive has a shebang line, it may have the executable bit set on POSIX systems, to allow it to be executed directly.

There is no requirement that the tools in this module are used to create application archives - the module is a convenience, but archives in the above format created by any means are acceptable to Python.

# `!zipfile` --- Work with ZIP archives

<div class="module" data-synopsis="Read and write ZIP-format archive files.">

zipfile

</div>

**Source code:** `Lib/zipfile/`

------------------------------------------------------------------------

The ZIP file format is a common archive and compression standard. This module provides tools to create, read, write, append, and list a ZIP file. Any advanced use of this module will require an understanding of the format, as defined in [PKZIP Application Note](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT).

This module does not handle multipart ZIP files. It can handle ZIP files that use the ZIP64 extensions (that is ZIP files that are more than 4 GiB in size). It supports decryption of encrypted files in ZIP archives, but it cannot create an encrypted file. Decryption is extremely slow as it is implemented in native Python rather than C.

Handling compressed archives requires `optional modules <optional module>` such as `zlib`, `bz2`, `lzma`, and `compression.zstd`. If any of them are missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

The module defines the following items:

<div class="exception">

BadZipFile

The error raised for bad ZIP files.

<div class="versionadded">

3.2

</div>

</div>

<div class="exception">

BadZipfile

Alias of `BadZipFile`, for compatibility with older Python versions.

<div class="deprecated">

3.2

</div>

</div>

<div class="exception">

LargeZipFile

The error raised when a ZIP file would require ZIP64 functionality but that has not been enabled.

</div>

<div class="ZipFile" data-noindex="">

The class for reading and writing ZIP files. See section `zipfile-objects` for constructor details.

</div>

<div class="Path" data-noindex="">

Class that implements a subset of the interface provided by `pathlib.Path`, including the full `importlib.resources.abc.Traversable` interface.

<div class="versionadded">

3.8

</div>

</div>

<div class="PyZipFile" data-noindex="">

Class for creating ZIP archives containing Python libraries.

</div>

<div class="ZipInfo(filename='NoName', date_time=(1980,1,1,0,0,0))">

Class used to represent information about a member of an archive. Instances of this class are returned by the `.getinfo` and `.infolist` methods of `ZipFile` objects. Most users of the `!zipfile` module will not need to create these, but only use those created by this module. *filename* should be the full name of the archive member, and *date_time* should be a tuple containing six fields which describe the time of the last modification to the file; the fields are described in section `zipinfo-objects`.

<div class="versionchanged">

3.13 A public `!compress_level` attribute has been added to expose the formerly protected `!_compresslevel`. The older protected name continues to work as a property for backwards compatibility.

</div>

<div class="method">

<span id="for_archive">for_archive</span>(archive)

Resolve the date_time, compression attributes, and external attributes to suitable defaults as used by `ZipFile.writestr`.

Returns self for chaining.

<div class="versionadded">

3.14

</div>

</div>

</div>

<div class="function">

is_zipfile(filename)

Returns `True` if *filename* is a valid ZIP file based on its magic number, otherwise returns `False`. *filename* may be a file or file-like object too.

<div class="versionchanged">

3.1 Support for file and file-like objects.

</div>

</div>

<div class="data">

ZIP_STORED

The numeric constant for an uncompressed archive member.

</div>

<div class="data">

ZIP_DEFLATED

The numeric constant for the usual ZIP compression method. This requires the `zlib` module.

</div>

<div class="data">

ZIP_BZIP2

The numeric constant for the BZIP2 compression method. This requires the `bz2` module.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

ZIP_LZMA

The numeric constant for the LZMA compression method. This requires the `lzma` module.

<div class="versionadded">

3.3

</div>

</div>

<div class="data">

ZIP_ZSTANDARD

The numeric constant for Zstandard compression. This requires the `compression.zstd` module.

> [!NOTE]
> In APPNOTE 6.3.7, the method ID `20` was assigned to Zstandard compression. This was changed in APPNOTE 6.3.8 to method ID `93` to avoid conflicts, with method ID `20` being deprecated. For compatibility, the `!zipfile` module reads both method IDs but will only write data with method ID `93`.

<div class="versionadded">

3.14

</div>

</div>

> [!NOTE]
> The ZIP file format specification has included support for bzip2 compression since 2001, for LZMA compression since 2006, and Zstandard compression since 2020. However, some tools (including older Python releases) do not support these compression methods, and may either refuse to process the ZIP file altogether, or fail to extract individual files.

<div class="seealso">

[PKZIP Application Note](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT)  
Documentation on the ZIP file format by Phil Katz, the creator of the format and algorithms used.

[Info-ZIP Home Page](https://infozip.sourceforge.net/)  
Information about the Info-ZIP project's ZIP archive programs and development libraries.

</div>

## ZipFile objects

<div class="ZipFile(file, mode='r', compression=ZIP_STORED, allowZip64=True, \ compresslevel=None, *, strict_timestamps=True, \ metadata_encoding=None)">

Open a ZIP file, where *file* can be a path to a file (a string), a file-like object or a `path-like object`.

The *mode* parameter should be `'r'` to read an existing file, `'w'` to truncate and write a new file, `'a'` to append to an existing file, or `'x'` to exclusively create and write a new file. If *mode* is `'x'` and *file* refers to an existing file, a `FileExistsError` will be raised. If *mode* is `'a'` and *file* refers to an existing ZIP file, then additional files are added to it. If *file* does not refer to a ZIP file, then a new ZIP archive is appended to the file. This is meant for adding a ZIP archive to another file (such as `python.exe`). If *mode* is `'a'` and the file does not exist at all, it is created. If *mode* is `'r'` or `'a'`, the file should be seekable.

*compression* is the ZIP compression method to use when writing the archive, and should be `ZIP_STORED`, `ZIP_DEFLATED`, `ZIP_BZIP2`, `ZIP_LZMA`, or `ZIP_ZSTANDARD`; unrecognized values will cause `NotImplementedError` to be raised. If `ZIP_DEFLATED`, `ZIP_BZIP2`, `ZIP_LZMA`, or `ZIP_ZSTANDARD` is specified but the corresponding module (`zlib`, `bz2`, `lzma`, or `compression.zstd`) is not available, `RuntimeError` is raised. The default is `ZIP_STORED`.

If *allowZip64* is `True` (the default) zipfile will create ZIP files that use the ZIP64 extensions when the zipfile is larger than 4 GiB. If it is `false` `!zipfile` will raise an exception when the ZIP file would require ZIP64 extensions.

The *compresslevel* parameter controls the compression level to use when writing files to the archive. When using `ZIP_STORED` or `ZIP_LZMA` it has no effect. When using `ZIP_DEFLATED` integers `0` through `9` are accepted (see `zlib <zlib.compressobj>` for more information). When using `ZIP_BZIP2` integers `1` through `9` are accepted (see `bz2 <bz2.BZ2File>` for more information). When using `ZIP_ZSTANDARD` integers `-131072` through `22` are commonly accepted (see `CompressionParameter.compression_level <compression.zstd.CompressionParameter.compression_level>` for more on retrieving valid values and their meaning).

The *strict_timestamps* argument, when set to `False`, allows to zip files older than 1980-01-01 at the cost of setting the timestamp to 1980-01-01. Similar behavior occurs with files newer than 2107-12-31, the timestamp is also set to the limit.

When mode is `'r'`, *metadata_encoding* may be set to the name of a codec, which will be used to decode metadata such as the names of members and ZIP comments.

If the file is created with mode `'w'`, `'x'` or `'a'` and then `closed <close>` without adding any files to the archive, the appropriate ZIP structures for an empty archive will be written to the file.

ZipFile is also a context manager and therefore supports the `with` statement. In the example, *myzip* is closed after the `!with` statement's suite is finished---even if an exception occurs:

    with ZipFile('spam.zip', 'w') as myzip:
        myzip.write('eggs.txt')

> [!NOTE]
> *metadata_encoding* is an instance-wide setting for the ZipFile. It is not possible to set this on a per-member basis.
>
> This attribute is a workaround for legacy implementations which produce archives with names in the current locale encoding or code page (mostly on Windows). According to the .ZIP standard, the encoding of metadata may be specified to be either IBM code page (default) or UTF-8 by a flag in the archive header. That flag takes precedence over *metadata_encoding*, which is a Python-specific extension.

<div class="versionchanged">

3.2 Added the ability to use `ZipFile` as a context manager.

</div>

<div class="versionchanged">

3.3 Added support for `bzip2 <bz2>` and `lzma` compression.

</div>

<div class="versionchanged">

3.4 ZIP64 extensions are enabled by default.

</div>

<div class="versionchanged">

3.5 Added support for writing to unseekable streams. Added support for the `'x'` mode.

</div>

<div class="versionchanged">

3.6 Previously, a plain `RuntimeError` was raised for unrecognized compression values.

</div>

<div class="versionchanged">

3.6.2 The *file* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.7 Add the *compresslevel* parameter.

</div>

<div class="versionchanged">

3.8 The *strict_timestamps* keyword-only parameter.

</div>

<div class="versionchanged">

3.11 Added support for specifying member name encoding for reading metadata in the zipfile's directory and file headers.

</div>

<div class="versionchanged">

3.16 Deleting a writable, open `zipfile.ZipFile` now emits a `ResourceWarning`. Use as a `context manager` or call `~zipfile.ZipFile.close` explicitly.

</div>

</div>

<div class="method">

ZipFile.close()

Close the archive file. You must call `close` before exiting your program or essential records will not be written.

</div>

<div class="method">

ZipFile.getinfo(name)

Return a `ZipInfo` object with information about the archive member *name*. Calling `getinfo` for a name not currently contained in the archive will raise a `KeyError`.

</div>

<div class="method">

ZipFile.infolist()

Return a list containing a `ZipInfo` object for each member of the archive. The objects are in the same order as their entries in the actual ZIP file on disk if an existing archive was opened.

</div>

<div class="method">

ZipFile.namelist()

Return a list of archive members by name.

</div>

<div class="method">

ZipFile.open(name, mode='r', pwd=None, \*, force_zip64=False)

Access a member of the archive as a binary file-like object. *name* can be either the name of a file within the archive or a `ZipInfo` object. The *mode* parameter, if included, must be `'r'` (the default) or `'w'`. *pwd* is the password used to decrypt encrypted ZIP files as a `bytes` object.

`~ZipFile.open` is also a context manager and therefore supports the `with` statement:

    with ZipFile('spam.zip') as myzip:
        with myzip.open('eggs.txt') as myfile:
            print(myfile.read())

With *mode* `'r'` the file-like object (`ZipExtFile`) is read-only and provides the following methods: `~io.BufferedIOBase.read`, `~io.IOBase.readline`, `~io.IOBase.readlines`, `~io.IOBase.seek`, `~io.IOBase.tell`, `~container.__iter__`, `~iterator.__next__`. These objects can operate independently of the ZipFile.

With `mode='w'`, a writable file handle is returned, which supports the `~io.BufferedIOBase.write` method. While a writable file handle is open, attempting to read or write other files in the ZIP file will raise a `ValueError`.

In both cases the file-like object has also attributes `!name`, which is equivalent to the name of a file within the archive, and `!mode`, which is `'rb'` or `'wb'` depending on the input mode.

When writing a file, if the file size is not known in advance but may exceed 2 GiB, pass `force_zip64=True` to ensure that the header format is capable of supporting large files. If the file size is known in advance, construct a `ZipInfo` object with `~ZipInfo.file_size` set, and use that as the *name* parameter.

> [!NOTE]
> The `.open`, `read` and `extract` methods can take a filename or a `ZipInfo` object. You will appreciate this when trying to read a ZIP file that contains members with duplicate names.

<div class="versionchanged">

3.6 Removed support of `mode='U'`. Use `io.TextIOWrapper` for reading compressed text files in `universal newlines` mode.

</div>

<div class="versionchanged">

3.6 `ZipFile.open` can now be used to write files into the archive with the `mode='w'` option.

</div>

<div class="versionchanged">

3.6 Calling `.open` on a closed ZipFile will raise a `ValueError`. Previously, a `RuntimeError` was raised.

</div>

<div class="versionchanged">

3.13 Added attributes `!name` and `!mode` for the writeable file-like object. The value of the `!mode` attribute for the readable file-like object was changed from `'r'` to `'rb'`.

</div>

</div>

<div class="method">

ZipFile.extract(member, path=None, pwd=None)

Extract a member from the archive to the current working directory; *member* must be its full name or a `ZipInfo` object. Its file information is extracted as accurately as possible. *path* specifies a different directory to extract to. *member* can be a filename or a `ZipInfo` object. *pwd* is the password used for encrypted files as a `bytes` object.

Returns the normalized path created (a directory or new file).

> [!NOTE]
> If a member filename is an absolute path, a drive/UNC sharepoint and leading (back)slashes will be stripped, e.g.: `///foo/bar` becomes `foo/bar` on Unix, and `C:\foo\bar` becomes `foo\bar` on Windows. And all `".."` components in a member filename will be removed, e.g.: `../../foo../../ba..r` becomes `foo../ba..r`. On Windows illegal characters (`:`, `<`, `>`, `|`, `"`, `?`, and `*`) replaced by underscore (`_`).

<div class="versionchanged">

3.6 Calling `extract` on a closed ZipFile will raise a `ValueError`. Previously, a `RuntimeError` was raised.

</div>

<div class="versionchanged">

3.6.2 The *path* parameter accepts a `path-like object`.

</div>

</div>

<div class="method">

ZipFile.extractall(path=None, members=None, pwd=None)

Extract all members from the archive to the current working directory. *path* specifies a different directory to extract to. *members* is optional and must be a subset of the list returned by `namelist`. *pwd* is the password used for encrypted files as a `bytes` object.

> [!WARNING]
> Never extract archives from untrusted sources without prior inspection. It is possible that files are created outside of *path*, for example, members that have absolute filenames or filenames with ".." components. This module attempts to prevent that. See `extract` note.

<div class="versionchanged">

3.6 Calling `extractall` on a closed ZipFile will raise a `ValueError`. Previously, a `RuntimeError` was raised.

</div>

<div class="versionchanged">

3.6.2 The *path* parameter accepts a `path-like object`.

</div>

</div>

<div class="method">

ZipFile.printdir()

Print a table of contents for the archive to `sys.stdout`.

</div>

<div class="method">

ZipFile.setpassword(pwd)

Set *pwd* (a `bytes` object) as default password to extract encrypted files.

</div>

<div class="method">

ZipFile.read(name, pwd=None)

Return the bytes of the file *name* in the archive. *name* is the name of the file in the archive, or a `ZipInfo` object. The archive must be open for read or append. *pwd* is the password used for encrypted files as a `bytes` object and, if specified, overrides the default password set with `setpassword`. Calling `read` on a ZipFile that uses a compression method other than `ZIP_STORED`, `ZIP_DEFLATED`, `ZIP_BZIP2`, `ZIP_LZMA`, or `ZIP_ZSTANDARD` will raise a `NotImplementedError`. An error will also be raised if the corresponding compression module is not available.

<div class="versionchanged">

3.6 Calling `read` on a closed ZipFile will raise a `ValueError`. Previously, a `RuntimeError` was raised.

</div>

</div>

<div class="method">

ZipFile.testzip()

Read all the files in the archive and check their CRC's and file headers. Return the name of the first bad file, or else return `None`.

<div class="versionchanged">

3.6 Calling `testzip` on a closed ZipFile will raise a `ValueError`. Previously, a `RuntimeError` was raised.

</div>

</div>

<div class="method">

ZipFile.write(filename, arcname=None, compress_type=None, compresslevel=None)

Write the file named *filename* to the archive, giving it the archive name *arcname* (by default, this will be the same as *filename*, but without a drive letter and with leading path separators removed). If given, *compress_type* overrides the value given for the *compression* parameter to the constructor for the new entry. Similarly, *compresslevel* will override the constructor if given. The archive must be open with mode `'w'`, `'x'` or `'a'`.

> [!NOTE]
> The ZIP file standard historically did not specify a metadata encoding, but strongly recommended CP437 (the original IBM PC encoding) for interoperability. Recent versions allow use of UTF-8 (only). In this module, UTF-8 will automatically be used to write the member names if they contain any non-ASCII characters. It is not possible to write member names in any encoding other than ASCII or UTF-8.

> [!NOTE]
> Archive names should be relative to the archive root, that is, they should not start with a path separator.

> [!NOTE]
> If `arcname` (or `filename`, if `arcname` is not given) contains a null byte, the name of the file in the archive will be truncated at the null byte.

> [!NOTE]
> A leading slash in the filename may lead to the archive being impossible to open in some zip programs on Windows systems.

<div class="versionchanged">

3.6 Calling `write` on a ZipFile created with mode `'r'` or a closed ZipFile will raise a `ValueError`. Previously, a `RuntimeError` was raised.

</div>

</div>

<div class="method">

ZipFile.writestr(zinfo_or_arcname, data, compress_type=None, compresslevel=None)

Write a file into the archive. The contents is *data*, which may be either a `str` or a `bytes` instance; if it is a `str`, it is encoded as UTF-8 first. *zinfo_or_arcname* is either the file name it will be given in the archive, or a `ZipInfo` instance. If it's an instance, at least the filename, date, and time must be given. If it's a name, the date and time is set to the current date and time. The archive must be opened with mode `'w'`, `'x'` or `'a'`.

If given, *compress_type* overrides the value given for the *compression* parameter to the constructor for the new entry, or in the *zinfo_or_arcname* (if that is a `ZipInfo` instance). Similarly, *compresslevel* will override the constructor if given.

> [!NOTE]
> When passing a `ZipInfo` instance as the *zinfo_or_arcname* parameter, the compression method used will be that specified in the *compress_type* member of the given `ZipInfo` instance. By default, the `ZipInfo` constructor sets this member to `ZIP_STORED`.

<div class="versionchanged">

3.2 The *compress_type* argument.

</div>

<div class="versionchanged">

3.6 Calling `writestr` on a ZipFile created with mode `'r'` or a closed ZipFile will raise a `ValueError`. Previously, a `RuntimeError` was raised.

</div>

<div class="versionchanged">

3.14 Now respects the `SOURCE_DATE_EPOCH` environment variable. If set, it uses this value as the modification timestamp for the file written into the ZIP archive, instead of using the current time.

</div>

</div>

<div class="method">

ZipFile.mkdir(zinfo_or_directory, mode=511)

Create a directory inside the archive. If *zinfo_or_directory* is a string, a directory is created inside the archive with the mode that is specified in the *mode* argument. If, however, *zinfo_or_directory* is a `ZipInfo` instance then the *mode* argument is ignored.

The archive must be opened with mode `'w'`, `'x'` or `'a'`.

<div class="versionadded">

3.11

</div>

</div>

<div class="method">

ZipFile.remove(zinfo_or_arcname)

Removes a member entry from the archive's central directory. *zinfo_or_arcname* may be the full path of the member or a `ZipInfo` instance. If multiple members share the same full path and the path is given as a string, only one of them is removed and which one is unspecified; it should not be relied upon. Pass the specific `ZipInfo` instance to remove a particular member.

The archive must be opened with mode `'w'`, `'x'` or `'a'`.

Returns the removed `ZipInfo` instance.

Calling `remove` on a closed ZipFile will raise a `ValueError`.

> [!NOTE]
> This method only removes the member's entry from the central directory, making it inaccessible to most tools. The member's local file entry, including content and metadata, remains in the archive and is still recoverable using forensic tools. Call `repack` afterwards to remove the local file entry and reclaim space; pass the returned `ZipInfo` to `repack` to ensure the data is removed regardless of how the entry was written.

<div class="versionadded">

3.16

</div>

</div>

<div class="method">

ZipFile.repack(removed=None, \*, strict_descriptor=True\[, chunk_size\])

Rewrites the archive to remove unreferenced local file entries, shrinking its file size. The archive must be opened with mode `'a'`, and any file object returned by `ZipFile.open` must be closed first, since repacking moves the member data such objects refer to.

If *removed* is provided, it must be a sequence of `ZipInfo` objects representing the recently removed members, and only their corresponding local file entries will be removed. Otherwise, the archive is scanned to locate and remove local file entries that are no longer referenced in the central directory.

Passing *removed* is the most reliable way to reclaim space: the corresponding local file entries are located directly from the central directory and removed regardless of how they were written, whereas the scan used when *removed* is omitted may leave some entries in place (see *strict_descriptor* below). To remove members and reclaim their space in a single step:

    with ZipFile('spam.zip', 'a') as myzip:
        removed = [myzip.remove(name) for name in ('ham.txt', 'eggs.txt')]
        myzip.repack(removed)

When scanning, *strict_descriptor* controls how entries written with an unsigned *data descriptor* are handled. A data descriptor is an optional record holding an entry's CRC and sizes, stored just after the entry's data; it is used when the archive is written to a non-seekable stream, and is *signed* when it begins with a marker signature or *unsigned* otherwise. Unsigned descriptors have been deprecated by the [PKZIP Application Note](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT) since version 6.3.0 (released in 2006) and are written only by some legacy tools; signed descriptors—written by Python and other modern tools—are always detected. When *strict_descriptor* is true (the default), only signed data descriptors are detected, so an unreferenced entry written with an unsigned descriptor is not located and its space is not reclaimed by the scan. Setting `strict_descriptor=False` additionally detects unsigned descriptors, at the cost of a significantly slower scan—around 100 to 1000 times in the worst case—which may be exploitable as a denial-of-service vector on untrusted input. This does not affect entries without a data descriptor, and is not needed when *removed* is provided.

*chunk_size* may be specified to control the buffer size when moving entry data (default is 1 MiB).

Calling `repack` on a closed ZipFile will raise a `ValueError`.

> [!NOTE]
> The scanning algorithm is heuristic-based and assumes that the ZIP file is normally structured—for example, with local file entries stored consecutively, without overlap or interleaved binary data. Prepended binary data, such as a self-extractor stub, is recognized and preserved unless it happens to contain bytes that coincidentally resemble a valid local file entry in multiple respects—an extremely rare case. Embedded ZIP payloads are also handled correctly, as long as they follow normal structure. However, the algorithm does not guarantee correctness or safety on untrusted or intentionally crafted input. It is generally recommended to provide the *removed* argument for better reliability and performance.

<div class="versionadded">

3.16

</div>

</div>

The following data attributes are also available:

<div class="attribute">

ZipFile.filename

Name of the ZIP file.

</div>

<div class="attribute">

ZipFile.debug

The level of debug output to use. This may be set from `0` (the default, no output) to `3` (the most output). Debugging information is written to `sys.stdout`.

</div>

<div class="attribute">

ZipFile.comment

The comment associated with the ZIP file as a `bytes` object. If assigning a comment to a `ZipFile` instance created with mode `'w'`, `'x'` or `'a'`, it should be no longer than 65535 bytes. Comments longer than this will be truncated.

</div>

## Path objects

<div class="Path(root, at='')">

Construct a Path object from a `root` zipfile (which may be a `ZipFile` instance or `file` suitable for passing to the `ZipFile` constructor).

`at` specifies the location of this Path within the zipfile, e.g. 'dir/file.txt', 'dir/', or ''. Defaults to the empty string, indicating the root.

> [!NOTE]
> The `Path` class does not sanitize filenames within the ZIP archive. Unlike the `ZipFile.extract` and `ZipFile.extractall` methods, it is the caller's responsibility to validate or sanitize filenames to prevent path traversal vulnerabilities (for example, absolute paths or paths with ".." components). When handling untrusted archives, consider resolving filenames using `os.path.abspath` and checking against the target directory with `os.path.commonpath`.

</div>

Path objects expose the following features of `pathlib.Path` objects:

Path objects are traversable using the `/` operator or `joinpath`.

<div class="attribute">

Path.name

The final path component.

</div>

<div class="method">

Path.open(mode='r', *, pwd,*\*)

Invoke `ZipFile.open` on the current path. Allows opening for read or write, text or binary through supported modes: 'r', 'w', 'rb', 'wb'. Positional and keyword arguments are passed through to `io.TextIOWrapper` when opened as text and ignored otherwise. `pwd` is the `pwd` parameter to `ZipFile.open`.

<div class="versionchanged">

3.9 Added support for text and binary modes for open. Default mode is now text.

</div>

<div class="versionchanged">

3.11.2 The `encoding` parameter can be supplied as a positional argument without causing a `TypeError`. As it could in 3.9. Code needing to be compatible with unpatched 3.10 and 3.11 versions must pass all `io.TextIOWrapper` arguments, `encoding` included, as keywords.

</div>

</div>

<div class="method">

Path.iterdir()

Enumerate the children of the current directory.

<div class="versionchanged">

3.16 Raises `NotADirectoryError` when the current context does not reference a directory. Previously, it raised `ValueError`.

</div>

</div>

<div class="method">

Path.is_dir()

Return `True` if the current context references a directory.

</div>

<div class="method">

Path.is_file()

Return `True` if the current context references a file.

</div>

<div class="method">

Path.is_symlink()

Return `True` if the current context references a symbolic link.

<div class="versionadded">

3.12

</div>

<div class="versionchanged">

3.13 Previously, `is_symlink` would unconditionally return `False`.

</div>

</div>

<div class="method">

Path.exists()

Return `True` if the current context references a file or directory in the zip file.

</div>

<div class="data">

Path.suffix

The last dot-separated portion of the final component, if any. This is commonly called the file extension.

<div class="versionadded">

3.11 Added `Path.suffix` property.

</div>

</div>

<div class="data">

Path.stem

The final path component, without its suffix.

<div class="versionadded">

3.11 Added `Path.stem` property.

</div>

</div>

<div class="data">

Path.suffixes

A list of the path’s suffixes, commonly called file extensions.

<div class="versionadded">

3.11 Added `Path.suffixes` property.

</div>

</div>

<div class="method">

Path.read_text(*,*\*)

Read the current file as unicode text. Positional and keyword arguments are passed through to `io.TextIOWrapper` (except `buffer`, which is implied by the context).

<div class="versionchanged">

3.11.2 The `encoding` parameter can be supplied as a positional argument without causing a `TypeError`. As it could in 3.9. Code needing to be compatible with unpatched 3.10 and 3.11 versions must pass all `io.TextIOWrapper` arguments, `encoding` included, as keywords.

</div>

</div>

<div class="method">

Path.read_bytes()

Read the current file as bytes.

</div>

<div class="method">

Path.joinpath(\*other)

Return a new Path object with each of the *other* arguments joined. The following are equivalent:

    >>> Path(...).joinpath('child').joinpath('grandchild')
    >>> Path(...).joinpath('child', 'grandchild')
    >>> Path(...) / 'child' / 'grandchild'

<div class="versionchanged">

3.10 Prior to 3.10, `joinpath` was undocumented and accepted exactly one parameter.

</div>

</div>

The `zipp` project provides backports of the latest path object functionality to older Pythons. Use `zipp.Path` in place of `zipfile.Path` for early access to changes.

## PyZipFile objects

The `PyZipFile` constructor takes the same parameters as the `ZipFile` constructor, and one additional parameter, *optimize*.

<div class="PyZipFile(file, mode='r', compression=ZIP_STORED, allowZip64=True, \ optimize=-1)">

<div class="versionchanged">

3.2 Added the *optimize* parameter.

</div>

<div class="versionchanged">

3.4 ZIP64 extensions are enabled by default.

</div>

Instances have one method in addition to those of `ZipFile` objects:

<div class="method">

PyZipFile.writepy(pathname, basename='', filterfunc=None)

Search for files `\*.py` and add the corresponding file to the archive.

If the *optimize* parameter to `PyZipFile` was not given or `-1`, the corresponding file is a `\*.pyc` file, compiling if necessary.

If the *optimize* parameter to `PyZipFile` was `0`, `1` or `2`, only files with that optimization level (see `compile`) are added to the archive, compiling if necessary.

If *pathname* is a file, the filename must end with `.py`, and just the (corresponding `\*.pyc`) file is added at the top level (no path information). If *pathname* is a file that does not end with `.py`, a `RuntimeError` will be raised. If it is a directory, and the directory is not a package directory, then all the files `\*.pyc` are added at the top level. If the directory is a package directory, then all `\*.pyc` are added under the package name as a file path, and if any subdirectories are package directories, all of these are added recursively in sorted order.

*basename* is intended for internal use only.

*filterfunc*, if given, must be a function taking a single string argument. It will be passed each path (including each individual full file path) before it is added to the archive. If *filterfunc* returns a false value, the path will not be added, and if it is a directory its contents will be ignored. For example, if our test files are all either in `test` directories or start with the string `test_`, we can use a *filterfunc* to exclude them:

    >>> zf = PyZipFile('myprog.zip', 'w')
    >>> def notests(s):
    ...     fn = os.path.basename(s)
    ...     return (not (fn == 'test' or fn.startswith('test_')))
    ...
    >>> zf.writepy('myprog', filterfunc=notests)

The `writepy` method makes archives with file names like this:

    string.pyc                   # Top level name
    test/__init__.pyc            # Package directory
    test/testall.pyc             # Module test.testall
    test/bogus/__init__.pyc      # Subpackage directory
    test/bogus/myfile.pyc        # Submodule test.bogus.myfile

<div class="versionchanged">

3.4 Added the *filterfunc* parameter.

</div>

<div class="versionchanged">

3.6.2 The *pathname* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.7 Recursion sorts directory entries.

</div>

</div>

</div>

## ZipInfo objects

Instances of the `ZipInfo` class are returned by the `.getinfo` and `.infolist` methods of `ZipFile` objects. Each object stores information about a single member of the ZIP archive.

There is one classmethod to make a `ZipInfo` instance for a filesystem file:

<div class="classmethod">

ZipInfo.from_file(filename, arcname=None, \*, strict_timestamps=True)

Construct a `ZipInfo` instance for a file on the filesystem, in preparation for adding it to a zip file.

*filename* should be the path to a file or directory on the filesystem.

If *arcname* is specified, it is used as the name within the archive. If *arcname* is not specified, the name will be the same as *filename*, but with any drive letter and leading path separators removed.

The *strict_timestamps* argument, when set to `False`, allows to zip files older than 1980-01-01 at the cost of setting the timestamp to 1980-01-01. Similar behavior occurs with files newer than 2107-12-31, the timestamp is also set to the limit.

<div class="versionadded">

3.6

</div>

<div class="versionchanged">

3.6.2 The *filename* parameter accepts a `path-like object`.

</div>

<div class="versionchanged">

3.8 Added the *strict_timestamps* keyword-only parameter.

</div>

</div>

Instances have the following methods and attributes:

<div class="method">

ZipInfo.is_dir()

Return `True` if this archive member is a directory.

This uses the entry's name: directories should always end with `/`.

<div class="versionadded">

3.6

</div>

</div>

<div class="attribute">

ZipInfo.filename

Name of the file in the archive.

</div>

<div class="attribute">

ZipInfo.date_time

The time and date of the last modification to the archive member. This is a tuple of six values representing the "last \[modified\] file time" and "last \[modified\] file date" fields from the ZIP file's central directory.

The tuple contains:

| Index | Value                    |
|-------|--------------------------|
| `0`   | Year (\>= 1980)          |
| `1`   | Month (one-based)        |
| `2`   | Day of month (one-based) |
| `3`   | Hours (zero-based)       |
| `4`   | Minutes (zero-based)     |
| `5`   | Seconds (zero-based)     |

> [!NOTE]
> The ZIP format supports multiple timestamp fields in different locations (central directory, extra fields for NTFS/UNIX systems, etc.). This attribute specifically returns the timestamp from the central directory. The central directory timestamp format in ZIP files does not support timestamps before 1980. While some extra field formats (such as UNIX timestamps) can represent earlier dates, this attribute only returns the central directory timestamp.
>
> The central directory timestamp is interpreted as representing local time, rather than UTC time, to match the behavior of other zip tools.

</div>

<div class="attribute">

ZipInfo.compress_type

Type of compression for the archive member.

</div>

<div class="attribute">

ZipInfo.comment

Comment for the individual archive member as a `bytes` object.

</div>

<div class="attribute">

ZipInfo.extra

Expansion field data. The [PKZIP Application Note](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT) contains some comments on the internal structure of the data contained in this `bytes` object.

</div>

<div class="attribute">

ZipInfo.create_system

System which created ZIP archive.

</div>

<div class="attribute">

ZipInfo.create_version

PKZIP version which created ZIP archive.

</div>

<div class="attribute">

ZipInfo.extract_version

PKZIP version needed to extract archive.

</div>

<div class="attribute">

ZipInfo.reserved

Must be zero.

</div>

<div class="attribute">

ZipInfo.flag_bits

ZIP flag bits.

</div>

<div class="attribute">

ZipInfo.volume

Volume number of file header.

</div>

<div class="attribute">

ZipInfo.internal_attr

Internal attributes.

</div>

<div class="attribute">

ZipInfo.external_attr

External file attributes.

</div>

<div class="attribute">

ZipInfo.header_offset

Byte offset to the file header.

</div>

<div class="attribute">

ZipInfo.CRC

CRC-32 of the uncompressed file.

</div>

<div class="attribute">

ZipInfo.compress_size

Size of the compressed data.

</div>

<div class="attribute">

ZipInfo.file_size

Size of the uncompressed file.

</div>

## Command-line interface

The `!zipfile` module provides a simple command-line interface to interact with ZIP archives.

If you want to create a new ZIP archive, specify its name after the `-c` option and then list the filename(s) that should be included:

```
$ python -m zipfile -c monty.zip spam.txt eggs.txt
```

Passing a directory is also acceptable:

```
$ python -m zipfile -c monty.zip life-of-brian_1979/
```

If you want to extract a ZIP archive into the specified directory, use the `-e` option:

```
$ python -m zipfile -e monty.zip target-dir/
```

For a list of the files in a ZIP archive, use the `-l` option:

```
$ python -m zipfile -l monty.zip
```

### Command-line options

<div class="option">

`-l <zipfile>`  

`--list <zipfile>`  

List files in a zipfile.

</div>

<div class="option">

-c \<zipfile\> \<source1\> ... \<sourceN\> --create \<zipfile\> \<source1\> ... \<sourceN\>

Create zipfile from source files.

</div>

<div class="option">

-e \<zipfile\> \<output_dir\> --extract \<zipfile\> \<output_dir\>

Extract zipfile into target directory.

</div>

<div class="option">

`-t <zipfile>`  

`--test <zipfile>`  

Test whether the zipfile is valid or not.

</div>

<div class="option">

`--metadata-encoding <encoding>`  

Specify encoding of member names for `-l`, `-e` and `-t`.

<div class="versionadded">

3.11

</div>

</div>

## Decompression pitfalls

The extraction in zipfile module might fail due to some pitfalls listed below.

### From file itself

Decompression may fail due to incorrect password / CRC checksum / ZIP format or unsupported compression method / decryption.

### File system limitations

Exceeding limitations on different file systems can cause decompression failed. Such as allowable characters in the directory entries, length of the file name, length of the pathname, size of a single file, and number of files, etc.

### Resources limitations

The lack of memory or disk volume would lead to decompression failed. For example, decompression bombs (aka [ZIP bomb](https://en.wikipedia.org/wiki/Zip_bomb)) apply to zipfile library that can cause disk volume exhaustion.

### Interruption

Interruption during the decompression, such as pressing control-C or killing the decompression process may result in incomplete decompression of the archive.

### Default behaviors of extraction

Not knowing the default extraction behaviors can cause unexpected decompression results. For example, when extracting the same archive twice, it overwrites files without asking.

# `!zipimport` --- Import modules from Zip archives

<div class="module" data-synopsis="Support for importing Python modules from ZIP archives.">

zipimport

</div>

**Source code:** `Lib/zipimport.py`

------------------------------------------------------------------------

This module adds the ability to import Python modules (`\*.py`, `\*.pyc`) and packages from ZIP-format archives. It is usually not needed to use the `!zipimport` module explicitly; it is automatically used by the built-in `import` mechanism for `sys.path` items that are paths to ZIP archives.

Typically, `sys.path` is a list of directory names as strings. This module also allows an item of `sys.path` to be a string naming a ZIP file archive. The ZIP archive can contain a subdirectory structure to support package imports, and a path within the archive can be specified to only import from a subdirectory. For example, the path `example.zip/lib/` would only import from the `lib/` subdirectory within the archive.

Any files may be present in the ZIP archive, but importers are only invoked for `.py` and `.pyc` files. ZIP import of dynamic modules (`.pyd`, `.so`) is disallowed. Note that if an archive only contains `.py` files, Python will not attempt to modify the archive by adding the corresponding `.pyc` file, meaning that if a ZIP archive doesn't contain `.pyc` files, importing may be rather slow.

<div class="versionchanged">

3.15 Zstandard (*zstd*) compressed zip file entries are supported.

</div>

<div class="versionchanged">

3.13 ZIP64 is supported

</div>

<div class="versionchanged">

3.8 Previously, ZIP archives with an archive comment were not supported.

</div>

<div class="seealso">

[PKZIP Application Note](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT)  
Documentation on the ZIP file format by Phil Katz, the creator of the format and algorithms used.

`273` - Import Modules from Zip Archives  
Written by James C. Ahlstrom, who also provided an implementation. Python 2.3 follows the specification in `273`, but uses an implementation written by Just van Rossum that uses the import hooks described in `302`.

`importlib` - The implementation of the import machinery  
Package providing the relevant protocols for all importers to implement.

</div>

This module defines an exception:

<div class="exception">

ZipImportError

Exception raised by zipimporter objects. It's a subclass of `ImportError`, so it can be caught as `ImportError`, too.

</div>

## zipimporter Objects

`zipimporter` is the class for importing ZIP files.

<div class="zipimporter(archivepath)">

Create a new zipimporter instance. *archivepath* must be a path to a ZIP file, or to a specific path within a ZIP file. For example, an *archivepath* of `foo/bar.zip/lib` will look for modules in the `lib` directory inside the ZIP file `foo/bar.zip` (provided that it exists).

`ZipImportError` is raised if *archivepath* doesn't point to a valid ZIP archive.

<div class="versionchanged">

3.12

Methods `find_loader()` and `find_module()`, deprecated in 3.10 are now removed. Use `find_spec` instead.

</div>

<div class="method">

create_module(spec)

Implementation of `importlib.abc.Loader.create_module` that returns `None` to explicitly request the default semantics.

<div class="versionadded">

3.10

</div>

</div>

<div class="method">

exec_module(module)

Implementation of `importlib.abc.Loader.exec_module`.

<div class="versionadded">

3.10

</div>

</div>

<div class="method">

find_spec(fullname, target=None)

An implementation of `importlib.abc.PathEntryFinder.find_spec`.

<div class="versionadded">

3.10

</div>

</div>

<div class="method">

get_code(fullname)

Return the code object for the specified module. Raise `ZipImportError` if the module couldn't be imported.

</div>

<div class="method">

get_data(pathname)

Return the data associated with *pathname*. Raise `OSError` if the file wasn't found.

<div class="versionchanged">

3.3 `IOError` used to be raised, it is now an alias of `OSError`.

</div>

</div>

<div class="method">

get_filename(fullname)

Return the value `__file__` would be set to if the specified module was imported. Raise `ZipImportError` if the module couldn't be found.

<div class="versionadded">

3.1

</div>

</div>

<div class="method">

get_source(fullname)

Return the source code for the specified module. Raise `ZipImportError` if the module couldn't be found, return `None` if the archive does contain the module, but has no source for it.

</div>

<div class="method">

is_package(fullname)

Return `True` if the module specified by *fullname* is a package. Raise `ZipImportError` if the module couldn't be found.

</div>

<div class="method">

invalidate_caches()

Clear out the internal cache of information about files found within the ZIP archive.

<div class="versionadded">

3.10

</div>

</div>

<div class="attribute">

archive

The file name of the importer's associated ZIP file, without a possible subpath.

</div>

<div class="attribute">

prefix

The subpath within the ZIP file where modules are searched. This is the empty string for zipimporter objects which point to the root of the ZIP file.

</div>

The `archive` and `prefix` attributes, when combined with a slash, equal the original *archivepath* argument given to the `zipimporter` constructor.

</div>

## Examples

Here is an example that imports a module from a ZIP archive - note that the `!zipimport` module is not explicitly used.

```
$ unzip -l example_archive.zip
Archive:  example_archive.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
     8467  01-01-00 12:30   example.py
 --------                   -------
     8467                   1 file
```

```
>>> import sys
>>> # Add the archive to the front of the module search path
>>> sys.path.insert(0, 'example_archive.zip')
>>> import example
>>> example.__file__
'example_archive.zip/example.py'
```

# `!zlib` --- Compression compatible with `gzip`

<div class="module" data-synopsis="Low-level interface to compression and decompression routines
compatible with gzip.">

zlib

</div>

------------------------------------------------------------------------

For applications that require data compression, the functions in this module allow compression and decompression, using the [zlib library](https://www.zlib.net).

This is an `optional module`. If it is missing from your copy of CPython, look for documentation from your distributor (that is, whoever provided Python to you). If you are the distributor, see `optional-module-requirements`.

zlib's functions have many options and often need to be used in a particular order. This documentation doesn't attempt to cover all of the permutations; consult the [zlib manual](https://www.zlib.net/manual.html) for authoritative information.

For reading and writing `.gz` files see the `gzip` module.

The available exception and functions in this module are:

<div class="exception">

error

Exception raised on compression and decompression errors.

</div>

<div class="function">

adler32(data, value=1, /)

Computes an Adler-32 checksum of *data*. (An Adler-32 checksum is almost as reliable as a CRC32 but can be computed much more quickly.) The result is an unsigned 32-bit integer. If *value* is present, it is used as the starting value of the checksum; otherwise, a default value of 1 is used. Passing in *value* allows computing a running checksum over the concatenation of several inputs. The algorithm is not cryptographically strong, and should not be used for authentication or digital signatures. Since the algorithm is designed for use as a checksum algorithm, it is not suitable for use as a general hash algorithm.

<div class="versionchanged">

3.0 The result is always unsigned.

</div>

</div>

<div class="function">

adler32_combine(adler1, adler2, len2, /)

Combine two Adler-32 checksums into one.

Given the Adler-32 checksum *adler1* of a sequence `A` and the Adler-32 checksum *adler2* of a sequence `B` of length *len2*, return the Adler-32 checksum of `A` and `B` concatenated.

This function is typically useful to combine Adler-32 checksums that were concurrently computed. To compute checksums sequentially, use `adler32` with the running checksum as the `value` argument.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

compress(data, /, level=Z_DEFAULT_COMPRESSION, wbits=MAX_WBITS)

Compresses the bytes in *data*, returning a bytes object containing compressed data. *level* is an integer from `0` to `9` or `-1` controlling the level of compression; See `Z_BEST_SPEED` (`1`), `Z_BEST_COMPRESSION` (`9`), `Z_NO_COMPRESSION` (`0`), and the default, `Z_DEFAULT_COMPRESSION` (`-1`) for more information about these values.

<div id="compress-wbits">

The *wbits* argument controls the size of the history buffer (or the "window size") used when compressing data, and whether a header and trailer is included in the output. It can take several ranges of values, defaulting to `15` (`MAX_WBITS`):

</div>

- +9 to +15: The base-two logarithm of the window size, which therefore ranges between 512 and 32768. Larger values produce better compression at the expense of greater memory usage. The resulting output will include a zlib-specific header and trailer.
- −9 to −15: Uses the absolute value of *wbits* as the window size logarithm, while producing a raw output stream with no header or trailing checksum.
- +25 to +31 = 16 + (9 to 15): Uses the low 4 bits of the value as the window size logarithm, while including a basic `gzip` header and trailing checksum in the output.

Raises the `error` exception if any error occurs.

<div class="versionchanged">

3.6 *level* can now be used as a keyword parameter.

</div>

<div class="versionchanged">

3.11 The *wbits* parameter is now available to set window bits and compression type.

</div>

</div>

<div class="function">

compressobj(level=Z_DEFAULT_COMPRESSION, method=DEFLATED, wbits=MAX_WBITS, memLevel=DEF_MEM_LEVEL, strategy=Z_DEFAULT_STRATEGY\[, zdict\])

Returns a compression object, to be used for compressing data streams that won't fit into memory at once.

*level* is the compression level -- an integer from `0` to `9` or `-1`. See `Z_BEST_SPEED` (`1`), `Z_BEST_COMPRESSION` (`9`), `Z_NO_COMPRESSION` (`0`), and the default, `Z_DEFAULT_COMPRESSION` (`-1`) for more information about these values.

*method* is the compression algorithm. Currently, the only supported value is `DEFLATED`.

The *wbits* parameter controls the size of the history buffer (or the "window size"), and what header and trailer format will be used. It has the same meaning as [described for compress()](#compress-wbits).

The *memLevel* argument controls the amount of memory used for the internal compression state. Valid values range from `1` to `9`. Higher values use more memory, but are faster and produce smaller output.

*strategy* is used to tune the compression algorithm. Possible values are `Z_DEFAULT_STRATEGY`, `Z_FILTERED`, `Z_HUFFMAN_ONLY`, `Z_RLE` and `Z_FIXED`.

*zdict* is a predefined compression dictionary. This is a sequence of bytes (such as a `bytes` object) containing subsequences that are expected to occur frequently in the data that is to be compressed. Those subsequences that are expected to be most common should come at the end of the dictionary.

<div class="versionchanged">

3.3 Added the *zdict* parameter and keyword argument support.

</div>

</div>

<div class="function">

crc32(data, value=0, /)

<div class="index">

single: Cyclic Redundancy Check single: checksum; Cyclic Redundancy Check

</div>

Computes a CRC (Cyclic Redundancy Check) checksum of *data*. The result is an unsigned 32-bit integer. If *value* is present, it is used as the starting value of the checksum; otherwise, a default value of 0 is used. Passing in *value* allows computing a running checksum over the concatenation of several inputs. The algorithm is not cryptographically strong, and should not be used for authentication or digital signatures. Since the algorithm is designed for use as a checksum algorithm, it is not suitable for use as a general hash algorithm.

<div class="versionchanged">

3.0 The result is always unsigned.

</div>

</div>

<div class="function">

crc32_combine(crc1, crc2, len2, /)

Combine two CRC-32 checksums into one.

Given the CRC-32 checksum *crc1* of a sequence `A` and the CRC-32 checksum *crc2* of a sequence `B` of length *len2*, return the CRC-32 checksum of `A` and `B` concatenated.

This function is typically useful to combine CRC-32 checksums that were concurrently computed. To compute checksums sequentially, use `crc32` with the running checksum as the `value` argument.

<div class="versionadded">

3.15

</div>

</div>

<div class="function">

decompress(data, /, wbits=MAX_WBITS, bufsize=DEF_BUF_SIZE)

Decompresses the bytes in *data*, returning a bytes object containing the uncompressed data. The *wbits* parameter depends on the format of *data*, and is discussed further below. If *bufsize* is given, it is used as the initial size of the output buffer. Raises the `error` exception if any error occurs.

<div id="decompress-wbits">

The *wbits* parameter controls the size of the history buffer (or "window size"), and what header and trailer format is expected. It is similar to the parameter for `compressobj`, but accepts more ranges of values:

</div>

- +8 to +15: The base-two logarithm of the window size. The input must include a zlib header and trailer.
- 0: Automatically determine the window size from the zlib header. Only supported since zlib 1.2.3.5.
- −8 to −15: Uses the absolute value of *wbits* as the window size logarithm. The input must be a raw stream with no header or trailer.
- +24 to +31 = 16 + (8 to 15): Uses the low 4 bits of the value as the window size logarithm. The input must include a gzip header and trailer.
- +40 to +47 = 32 + (8 to 15): Uses the low 4 bits of the value as the window size logarithm, and automatically accepts either the zlib or gzip format.

When decompressing a stream, the window size must not be smaller than the size originally used to compress the stream; using a too-small value may result in an `error` exception. The default *wbits* value corresponds to the largest window size and requires a zlib header and trailer to be included.

*bufsize* is the initial size of the buffer used to hold decompressed data. If more space is required, the buffer size will be increased as needed, so you don't have to get this value exactly right; tuning it will only save a few calls to `malloc`.

<div class="versionchanged">

3.6 *wbits* and *bufsize* can be used as keyword arguments.

</div>

</div>

<div class="function">

decompressobj(wbits=MAX_WBITS, zdict=b'')

Returns a decompression object, to be used for decompressing data streams that won't fit into memory at once.

The *wbits* parameter controls the size of the history buffer (or the "window size"), and what header and trailer format is expected. It has the same meaning as [described for decompress()](#decompress-wbits).

The *zdict* parameter specifies a predefined compression dictionary. If provided, this must be the same dictionary as was used by the compressor that produced the data that is to be decompressed.

> [!NOTE]
> If *zdict* is a mutable object (such as a `bytearray`), you must not modify its contents between the call to `decompressobj` and the first call to the decompressor's `decompress()` method.

<div class="versionchanged">

3.3 Added the *zdict* parameter.

</div>

</div>

Compression objects support the following methods:

<div class="method">

Compress.compress(data, /)

Compress *data*, returning a bytes object containing compressed data for at least part of the data in *data*. This data should be concatenated to the output produced by any preceding calls to the `compress` method. Some input may be kept in internal buffers for later processing.

</div>

<div class="method">

Compress.flush(mode=Z_FINISH, /)

All pending input is processed, and a bytes object containing the remaining compressed output is returned. *mode* can be selected from the constants `Z_NO_FLUSH`, `Z_PARTIAL_FLUSH`, `Z_SYNC_FLUSH`, `Z_FULL_FLUSH`, `Z_BLOCK`, or `Z_FINISH`, defaulting to `Z_FINISH`. Except `Z_FINISH`, all constants allow compressing further bytestrings of data, while `Z_FINISH` finishes the compressed stream and prevents compressing any more data. After calling `flush` with *mode* set to `Z_FINISH`, the `compress` method cannot be called again; the only realistic action is to delete the object.

</div>

<div class="method">

Compress.copy()

Returns a copy of the compression object. This can be used to efficiently compress a set of data that share a common initial prefix.

</div>

<div class="versionchanged">

3.8 Added `copy.copy` and `copy.deepcopy` support to compression objects.

</div>

Decompression objects support the following methods and attributes:

<div class="attribute">

Decompress.unused_data

A bytes object which contains any bytes past the end of the compressed data. That is, this remains `b""` until the last byte that contains compression data is available. If the whole bytestring turned out to contain compressed data, this is `b""`, an empty bytes object.

</div>

<div class="attribute">

Decompress.unconsumed_tail

A bytes object that contains any data that was not consumed by the last `decompress` call because it exceeded the limit for the uncompressed data buffer. This data has not yet been seen by the zlib machinery, so you must feed it (possibly with further data concatenated to it) back to a subsequent `decompress` method call in order to get correct output.

</div>

<div class="attribute">

Decompress.eof

A boolean indicating whether the end of the compressed data stream has been reached.

This makes it possible to distinguish between a properly formed compressed stream, and an incomplete or truncated one.

<div class="versionadded">

3.3

</div>

</div>

<div class="method">

Decompress.decompress(data, /, max_length=0)

Decompress *data*, returning a bytes object containing the uncompressed data corresponding to at least part of the data in *string*. This data should be concatenated to the output produced by any preceding calls to the `decompress` method. Some of the input data may be preserved in internal buffers for later processing.

If the optional parameter *max_length* is non-zero then the return value will be no longer than *max_length*. This may mean that not all of the compressed input can be processed; and unconsumed data will be stored in the attribute `unconsumed_tail`. This bytestring must be passed to a subsequent call to `decompress` if decompression is to continue. If *max_length* is zero then the whole input is decompressed, and `unconsumed_tail` is empty. For example, the full content could be read like:

    process_output(d.decompress(data, max_length))
    while chunk := d.decompress(d.unconsumed_tail, max_length):
        process_output(chunk)

<div class="versionchanged">

3.6 *max_length* can be used as a keyword argument.

</div>

</div>

<div class="method">

Decompress.flush(length=DEF_BUF_SIZE, /)

All pending input is processed, and a bytes object containing the remaining uncompressed output is returned. After calling `flush`, the `decompress` method cannot be called again; the only realistic action is to delete the object.

The optional parameter *length* sets the initial size of the output buffer.

</div>

<div class="method">

Decompress.copy()

Returns a copy of the decompression object. This can be used to save the state of the decompressor midway through the data stream in order to speed up random seeks into the stream at a future point.

</div>

<div class="versionchanged">

3.8 Added `copy.copy` and `copy.deepcopy` support to decompression objects.

</div>

The following constants are available to configure compression and decompression behavior:

<div class="data">

DEFLATED

The deflate compression method.

</div>

<div class="data">

MAX_WBITS

The maximum window size, expressed as a power of 2. For example, if `!MAX_WBITS` is `15` it results in a window size of `32 KiB`.

</div>

<div class="data">

DEF_MEM_LEVEL

The default memory level for compression objects.

</div>

<div class="data">

DEF_BUF_SIZE

The default buffer size for decompression operations.

</div>

<div class="data">

Z_NO_COMPRESSION

Compression level `0`; no compression.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

Z_BEST_SPEED

Compression level `1`; fastest and produces the least compression.

</div>

<div class="data">

Z_BEST_COMPRESSION

Compression level `9`; slowest and produces the most compression.

</div>

<div class="data">

Z_DEFAULT_COMPRESSION

Default compression level (`-1`); a compromise between speed and compression. Currently equivalent to compression level `6`.

</div>

<div class="data">

Z_DEFAULT_STRATEGY

Default compression strategy, for normal data.

</div>

<div class="data">

Z_FILTERED

Compression strategy for data produced by a filter (or predictor).

</div>

<div class="data">

Z_HUFFMAN_ONLY

Compression strategy that forces Huffman coding only.

</div>

<div class="data">

Z_RLE

Compression strategy that limits match distances to one (run-length encoding).

This constant is only available if Python was compiled with zlib 1.2.0.1 or greater.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

Z_FIXED

Compression strategy that prevents the use of dynamic Huffman codes.

This constant is only available if Python was compiled with zlib 1.2.2.2 or greater.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

Z_NO_FLUSH

Flush mode `0`. No special flushing behavior.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

Z_PARTIAL_FLUSH

Flush mode `1`. Flush as much output as possible.

</div>

<div class="data">

Z_SYNC_FLUSH

Flush mode `2`. All output is flushed and the output is aligned to a byte boundary.

</div>

<div class="data">

Z_FULL_FLUSH

Flush mode `3`. All output is flushed and the compression state is reset.

</div>

<div class="data">

Z_FINISH

Flush mode `4`. All pending input is processed, no more input is expected.

</div>

<div class="data">

Z_BLOCK

Flush mode `5`. A deflate block is completed and emitted.

This constant is only available if Python was compiled with zlib 1.2.2.2 or greater.

<div class="versionadded">

3.6

</div>

</div>

<div class="data">

Z_TREES

Flush mode `6`, for inflate operations. Instructs inflate to return when it gets to the next deflate block boundary.

This constant is only available if Python was compiled with zlib 1.2.3.4 or greater.

<div class="versionadded">

3.6

</div>

</div>

Information about the version of the zlib library in use is available through the following constants:

<div class="data">

ZLIB_VERSION

The version string of the zlib library that was used for building the module. This may be different from the zlib library actually used at runtime, which is available as `zlib_version`.

</div>

<div class="data">

ZLIB_RUNTIME_VERSION zlib_version

The version string of the zlib library actually loaded by the interpreter.

Prefer `!zlib_version` over its `!ZLIB_RUNTIME_VERSION` alias.

<div class="versionadded">

3.3 The `!ZLIB_RUNTIME_VERSION` constant.

</div>

<div class="versionadded">

3.16 The `!zlib_version` constant; `!ZLIB_RUNTIME_VERSION` is now an alias for it.

</div>

</div>

<div class="data">

ZLIB_VERSION_INFO

A named tuple containing the four components of the zlib library version that was used for building the module: *major*, *minor*, *revision*, and *subversion*. All values are integers. The components can also be accessed by name, so `zlib.ZLIB_VERSION_INFO[0]` is equivalent to `zlib.ZLIB_VERSION_INFO.major` and so on. This may be different from the zlib library actually used at runtime, which is available as `zlib_version_info`.

<div class="versionadded">

3.16

</div>

</div>

<div class="data">

zlib_version_info

A named tuple containing the version of the zlib library actually loaded by the interpreter, with the same fields as `ZLIB_VERSION_INFO`.

<div class="versionadded">

3.16

</div>

</div>

The following constants are only present if zlib-ng was used to build the module:

<div class="data">

ZLIBNG_VERSION

The version string of the zlib-ng library that was used for building the module if zlib-ng was used. When present, the `ZLIB_VERSION` and `zlib_version` constants reflect the version of the zlib API provided by zlib-ng.

<div class="versionadded">

3.14

</div>

</div>

<div class="data">

ZLIBNG_VERSION_INFO

A named tuple containing the three components of the zlib-ng library version that was used for building the module: *major*, *minor*, and *revision*. All values are integers.

<div class="versionadded">

3.16

</div>

</div>

<div class="seealso">

Module `gzip`  
Reading and writing `gzip`-format files.

<https://www.zlib.net>  
The zlib library home page.

<https://www.zlib.net/manual.html>  
The zlib manual explains the semantics and usage of the library's many functions.

In case gzip (de)compression is a bottleneck, the [python-isal](https://github.com/pycompression/python-isal) package speeds up (de)compression with a mostly compatible API.

</div>

# `!zoneinfo` --- IANA time zone support

<div class="module" data-synopsis="IANA time zone support">

zoneinfo

</div>

<div class="versionadded">

3.9

</div>

**Source code:** `Lib/zoneinfo`

------------------------------------------------------------------------

The `!zoneinfo` module provides a concrete time zone implementation to support the IANA time zone database as originally specified in `615`. By default, `!zoneinfo` uses the system's time zone data if available; if no system time zone data is available, the library will fall back to using the first-party `tzdata` package available on PyPI.

<div class="seealso">

Module: `datetime`  
Provides the `~datetime.time` and `~datetime.datetime` types with which the `ZoneInfo` class is designed to be used.

Package `tzdata`  
First-party package maintained by the CPython core developers to supply time zone data via PyPI.

</div>

<div class="availability">

not WASI.

This module does not work or is not available on WebAssembly. See `wasm-availability` for more information.

</div>

## Using `ZoneInfo`

`ZoneInfo` is a concrete implementation of the `datetime.tzinfo` abstract base class, and is intended to be attached to `tzinfo`, either via the constructor, the `datetime.replace <datetime.datetime.replace>` method or `datetime.astimezone <datetime.datetime.astimezone>`:

    >>> from zoneinfo import ZoneInfo
    >>> import datetime as dt

    >>> when = dt.datetime(2020, 10, 31, 12, tzinfo=ZoneInfo("America/Los_Angeles"))
    >>> print(when)
    2020-10-31 12:00:00-07:00

    >>> when.tzname()
    'PDT'

Datetimes constructed in this way are compatible with datetime arithmetic and handle daylight saving time transitions with no further intervention:

    >>> when_add = when + dt.timedelta(days=1)

    >>> print(when_add)
    2020-11-01 12:00:00-08:00

    >>> when_add.tzname()
    'PST'

These time zones also support the `~datetime.datetime.fold` attribute introduced in `495`. During offset transitions which induce ambiguous times (such as a daylight saving time to standard time transition), the offset from *before* the transition is used when `fold=0`, and the offset *after* the transition is used when `fold=1`, for example:

    >>> when = dt.datetime(2020, 11, 1, 1, tzinfo=ZoneInfo("America/Los_Angeles"))
    >>> print(when)
    2020-11-01 01:00:00-07:00

    >>> print(when.replace(fold=1))
    2020-11-01 01:00:00-08:00

When converting from another time zone, the fold will be set to the correct value:

    >>> LOS_ANGELES = ZoneInfo("America/Los_Angeles")
    >>> when_utc = dt.datetime(2020, 11, 1, 8, tzinfo=dt.timezone.utc)

    >>> # Before the PDT -> PST transition
    >>> print(when_utc.astimezone(LOS_ANGELES))
    2020-11-01 01:00:00-07:00

    >>> # After the PDT -> PST transition
    >>> print((when_utc + dt.timedelta(hours=1)).astimezone(LOS_ANGELES))
    2020-11-01 01:00:00-08:00

## Data sources

The `zoneinfo` module does not directly provide time zone data, and instead pulls time zone information from the system time zone database or the first-party PyPI package `tzdata`, if available. Some systems, including notably Windows systems, do not have an IANA database available, and so for projects targeting cross-platform compatibility that require time zone data, it is recommended to declare a dependency on tzdata. If neither system data nor tzdata are available, all calls to `ZoneInfo` will raise `ZoneInfoNotFoundError`.

### Configuring the data sources

When `ZoneInfo(key)` is called, the constructor first searches the directories specified in `TZPATH` for a file matching `key`, and on failure looks for a match in the tzdata package. This behavior can be configured in three ways:

1.  The default `TZPATH` when not otherwise specified can be configured at `compile time <zoneinfo_data_compile_time_config>`.
2.  `TZPATH` can be configured using `an environment variable
    <zoneinfo_data_environment_var>`.
3.  At `runtime <zoneinfo_data_runtime_config>`, the search path can be manipulated using the `reset_tzpath` function.

#### Compile-time configuration

The default `TZPATH` includes several common deployment locations for the time zone database (except on Windows, where there are no "well-known" locations for time zone data). On POSIX systems, downstream distributors and those building Python from source who know where their system time zone data is deployed may change the default time zone path by specifying the compile-time option `TZPATH` (or, more likely, the `configure
flag --with-tzpath <--with-tzpath>`), which should be a string delimited by `os.pathsep`.

On all platforms, the configured value is available as the `TZPATH` key in `sysconfig.get_config_var`.

#### Environment configuration

When initializing `TZPATH` (either at import time or whenever `reset_tzpath` is called with no arguments), the `zoneinfo` module will use the environment variable `PYTHONTZPATH`, if it exists, to set the search path.

<div class="envvar">

PYTHONTZPATH

This is an `os.pathsep`-separated string containing the time zone search path to use. It must consist of only absolute rather than relative paths. Relative components specified in `PYTHONTZPATH` will not be used, but otherwise the behavior when a relative path is specified is implementation-defined; CPython will raise `InvalidTZPathWarning`, but other implementations are free to silently ignore the erroneous component or raise an exception.

</div>

To set the system to ignore the system data and use the tzdata package instead, set `PYTHONTZPATH=""`.

#### Runtime configuration

The TZ search path can also be configured at runtime using the `reset_tzpath` function. This is generally not an advisable operation, though it is reasonable to use it in test functions that require the use of a specific time zone path (or require disabling access to the system time zones).

## The `ZoneInfo` class

<div class="ZoneInfo(key)">

A concrete `datetime.tzinfo` subclass that represents an IANA time zone specified by the string `key`. Calls to the primary constructor will always return objects that compare identically; put another way, barring cache invalidation via `ZoneInfo.clear_cache`, for all values of `key`, the following assertion will always be true:

``` python
a = ZoneInfo(key)
b = ZoneInfo(key)
assert a is b
```

`key` must be in the form of a relative, normalized POSIX path, with no up-level references. The constructor will raise `ValueError` if a non-conforming key is passed.

If no file matching `key` is found, the constructor will raise `ZoneInfoNotFoundError`.

</div>

The `ZoneInfo` class has two alternate constructors:

<div class="classmethod">

ZoneInfo.from_file(file_obj, /, key=None)

Constructs a `ZoneInfo` object from a file-like object returning bytes (e.g. a file opened in binary mode or an `io.BytesIO` object). Unlike the primary constructor, this always constructs a new object.

The `key` parameter sets the name of the zone for the purposes of `~object.__str__` and `~object.__repr__`.

Objects created via this constructor cannot be pickled (see [pickling](#pickling)).

`ValueError` is raised if the data read from *file_obj* is not a valid TZif file.

</div>

<div class="classmethod">

ZoneInfo.no_cache(key)

An alternate constructor that bypasses the constructor's cache. It is identical to the primary constructor, but returns a new object on each call. This is most likely to be useful for testing or demonstration purposes, but it can also be used to create a system with a different cache invalidation strategy.

Objects created via this constructor will also bypass the cache of a deserializing process when unpickled.

> [!CAUTION]
> Using this constructor may change the semantics of your datetimes in surprising ways, only use it if you know that you need to.

</div>

The following class methods are also available:

<div class="classmethod">

ZoneInfo.clear_cache(\*, only_keys=None)

A method for invalidating the cache on the `ZoneInfo` class. If no arguments are passed, all caches are invalidated and the next call to the primary constructor for each key will return a new instance.

If an iterable of key names is passed to the `only_keys` parameter, only the specified keys will be removed from the cache. Keys passed to `only_keys` but not found in the cache are ignored.

> [!WARNING]
> Invoking this function may change the semantics of datetimes using `ZoneInfo` in surprising ways; this modifies module state and thus may have wide-ranging effects. Only use it if you know that you need to.

</div>

The class has one attribute:

<div class="attribute">

ZoneInfo.key

This is a read-only `attribute` that returns the value of `key` passed to the constructor, which should be a lookup key in the IANA time zone database (e.g. `America/New_York`, `Europe/Paris` or `Asia/Tokyo`).

For zones constructed from file without specifying a `key` parameter, this will be set to `None`.

> [!NOTE]
> Although it is a somewhat common practice to expose these to end users, these values are designed to be primary keys for representing the relevant zones and not necessarily user-facing elements. Projects like CLDR (the Unicode Common Locale Data Repository) can be used to get more user-friendly strings from these keys.

</div>

### String representations

The string representation returned when calling `str` on a `ZoneInfo` object defaults to using the `ZoneInfo.key` attribute (see the note on usage in the attribute documentation):

    >>> zone = ZoneInfo("Pacific/Kwajalein")
    >>> str(zone)
    'Pacific/Kwajalein'

    >>> when = dt.datetime(2020, 4, 1, 3, 15, tzinfo=zone)
    >>> f"{when.isoformat()} [{when.tzinfo}]"
    '2020-04-01T03:15:00+12:00 [Pacific/Kwajalein]'

For objects constructed from a file without specifying a `key` parameter, `str` falls back to calling `repr`. `ZoneInfo`'s `repr` is implementation-defined and not necessarily stable between versions, but it is guaranteed not to be a valid `ZoneInfo` key.

### Pickle serialization

Rather than serializing all transition data, `ZoneInfo` objects are serialized by key, and `ZoneInfo` objects constructed from files (even those with a value for `key` specified) cannot be pickled.

The behavior of a `ZoneInfo` file depends on how it was constructed:

1.  `ZoneInfo(key)`: When constructed with the primary constructor, a `ZoneInfo` object is serialized by key, and when deserialized, the deserializing process uses the primary and thus it is expected that these are the same object as other references to the same time zone. For example, if `europe_berlin_pkl` is a string containing a pickle constructed from `ZoneInfo("Europe/Berlin")`, one would expect the following behavior:

    ```
    >>> a = ZoneInfo("Europe/Berlin")
    >>> b = pickle.loads(europe_berlin_pkl)
    >>> a is b
    True
    ```

2.  `ZoneInfo.no_cache(key)`: When constructed from the cache-bypassing constructor, the `ZoneInfo` object is also serialized by key, but when deserialized, the deserializing process uses the cache bypassing constructor. If `europe_berlin_pkl_nc` is a string containing a pickle constructed from `ZoneInfo.no_cache("Europe/Berlin")`, one would expect the following behavior:

    ```
    >>> a = ZoneInfo("Europe/Berlin")
    >>> b = pickle.loads(europe_berlin_pkl_nc)
    >>> a is b
    False
    ```

3.  `ZoneInfo.from_file(file_obj, /, key=None)`: When constructed from a file, the `ZoneInfo` object raises an exception on pickling. If an end user wants to pickle a `ZoneInfo` constructed from a file, it is recommended that they use a wrapper type or a custom serialization function: either serializing by key or storing the contents of the file object and serializing that.

This method of serialization requires that the time zone data for the required key be available on both the serializing and deserializing side, similar to the way that references to classes and functions are expected to exist in both the serializing and deserializing environments. It also means that no guarantees are made about the consistency of results when unpickling a `ZoneInfo` pickled in an environment with a different version of the time zone data.

## Functions

<div class="function">

available_timezones()

Get a set containing all the valid keys for IANA time zones available anywhere on the time zone path. This is recalculated on every call to the function.

This function only includes canonical zone names and does not include "special" zones such as those under the `posix/` and `right/` directories, the `posixrules` or the `localtime` zone.

> [!CAUTION]
> This function may open a large number of files, as the best way to determine if a file on the time zone path is a valid time zone is to read the "magic string" at the beginning.

> [!NOTE]
> These values are not designed to be exposed to end-users; for user facing elements, applications should use something like CLDR (the Unicode Common Locale Data Repository) to get more user-friendly strings. See also the cautionary note on `ZoneInfo.key`.

</div>

<div class="function">

reset_tzpath(to=None)

Sets or resets the time zone search path (`TZPATH`) for the module. When called with no arguments, `TZPATH` is set to the default value.

Calling `reset_tzpath` will not invalidate the `ZoneInfo` cache, and so calls to the primary `ZoneInfo` constructor will only use the new `TZPATH` in the case of a cache miss.

The `to` parameter must be a `sequence` of strings or `os.PathLike` and not a string, all of which must be absolute paths. `ValueError` will be raised if something other than an absolute path is passed.

</div>

## Globals

<div class="data">

TZPATH

A read-only sequence representing the time zone search path -- when constructing a `ZoneInfo` from a key, the key is joined to each entry in the `TZPATH`, and the first file found is used.

`TZPATH` may contain only absolute paths, never relative paths, regardless of how it is configured.

The object that `zoneinfo.TZPATH` points to may change in response to a call to `reset_tzpath`, so it is recommended to use `zoneinfo.TZPATH` rather than importing `TZPATH` from `zoneinfo` or assigning a long-lived variable to `zoneinfo.TZPATH`.

For more information on configuring the time zone search path, see `zoneinfo_data_configuration`.

</div>

## Exceptions and warnings

<div class="exception">

ZoneInfoNotFoundError

Raised when construction of a `ZoneInfo` object fails because the specified key could not be found on the system. This is a subclass of `KeyError`.

</div>

<div class="exception">

InvalidTZPathWarning

Raised when `PYTHONTZPATH` contains an invalid component that will be filtered out, such as a relative path.

</div>