`!importlib.metadata` -- Accessing package metadata

Source code: Lib/importlib/metadata/__init__.py

importlib.metadata is a library that provides access to the metadata of an installed Distribution Package, such as its entry points or its top-level names (Import Packages, modules, if any). Built in part on Python's import system, this library provides the entry point and metadata APIs that were previously exposed by the now-removed pkg_resources package. Along with importlib.resources, it supersedes pkg_resources.

importlib.metadata operates on third-party distribution packages installed into Python's site-packages directory via tools such as pip. Specifically, it works with distributions with discoverable dist-info or egg-info directories, and metadata defined by the Core metadata specifications.

[!IMPORTANT] These are not necessarily equivalent to or correspond 1:1 with the top-level import package names that can be imported inside Python code. One distribution package can contain multiple import packages (and single modules), and one top-level import package may map to multiple distribution packages if it is a namespace package. You can use packages_distributions() <package-distributions> to get a mapping between them.

By default, distribution metadata can live on the file system or in zip archives on sys.path. Through an extension mechanism, the metadata can live almost anywhere.

Overview

Let's say you wanted to get the version string for a Distribution Package you've installed using pip. We start by creating a virtual environment and installing something into it:

$ python -m venv example
$ source example/bin/activate
(example) $ python -m pip install wheel

You can get the version string for wheel by running the following:

(example) $ python
>>> from importlib.metadata import version  # doctest: +SKIP
>>> version('wheel')  # doctest: +SKIP
'0.32.3'

You can also get a collection of entry points selectable by properties of the EntryPoint (typically 'group' or 'name'), such as console_scripts, distutils.commands and others. Each group contains a collection of EntryPoint <entry-points> objects.

You can get the metadata for a distribution <metadata>:

>>> from importlib.metadata import metadata  # doctest: +SKIP
>>> list(metadata('wheel'))  # doctest: +SKIP
['Metadata-Version', 'Name', 'Version', 'Summary', 'Home-page', 'Author', 'Author-email', 'Maintainer', 'Maintainer-email', 'License', 'Project-URL', 'Project-URL', 'Project-URL', 'Keywords', 'Platform', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Requires-Python', 'Provides-Extra', 'Requires-Dist', 'Requires-Dist']

You can also get a distribution's version number <version>, list its constituent files <files>, and get a list of the distribution's requirements.

Functional API

This package provides the following functionality via its public API.

Entry points

Query all entry points:

>>> eps = entry_points()  # doctest: +SKIP

The !entry_points function returns a !EntryPoints object, a collection of all !EntryPoint objects with names and groups attributes for convenience:

>>> sorted(eps.groups)  # doctest: +SKIP
['console_scripts', 'distutils.commands', 'distutils.setup_keywords', 'egg_info.writers', 'setuptools.installation']

!EntryPoints has a !select method to select entry points matching specific properties. Select entry points in the console_scripts group:

>>> scripts = eps.select(group='console_scripts')  # doctest: +SKIP

Equivalently, since !entry_points passes keyword arguments through to select:

>>> scripts = entry_points(group='console_scripts')  # doctest: +SKIP

Pick out a specific script named "wheel" (found in the wheel project):

>>> 'wheel' in scripts.names  # doctest: +SKIP
True
>>> wheel = scripts['wheel']  # doctest: +SKIP

Equivalently, query for that entry point during selection:

>>> (wheel,) = entry_points(group='console_scripts', name='wheel')  # doctest: +SKIP
>>> (wheel,) = entry_points().select(group='console_scripts', name='wheel')  # doctest: +SKIP

Inspect the resolved entry point:

>>> wheel  # doctest: +SKIP
EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')
>>> wheel.module  # doctest: +SKIP
'wheel.cli'
>>> wheel.attr  # doctest: +SKIP
'main'
>>> wheel.extras  # doctest: +SKIP
[]
>>> main = wheel.load()  # doctest: +SKIP
>>> main  # doctest: +SKIP
<function main at 0x103528488>

The group and name are arbitrary values defined by the package author and usually a client will wish to resolve all entry points for a particular group. Read the setuptools docs for more information on entry points, their definition, and usage.

Distribution metadata

Every Distribution Package includes some metadata, which you can extract using the !metadata function:

>>> wheel_metadata = metadata('wheel')  # doctest: +SKIP

The keys of the returned data structure name the metadata keywords, and the values are returned unparsed from the distribution metadata:

>>> wheel_metadata['Requires-Python']  # doctest: +SKIP
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'

PackageMetadata also presents a !json attribute that returns all the metadata in a JSON-compatible form per PEP 566:

>>> wheel_metadata.json['requires_python']
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'

The full set of available metadata is not described here. See the PyPA Core metadata specification for additional details.

Distribution versions

The !version function is the quickest way to get a Distribution Package's version number, as a string:

>>> version('wheel')  # doctest: +SKIP
'0.32.3'

Distribution files

The !files function takes a Distribution Package name and returns all of the files installed by this distribution. For example:

>>> util = [p for p in files('wheel') if 'util.py' in str(p)][0]  # doctest: +SKIP
>>> util  # doctest: +SKIP
PackagePath('wheel/util.py')
>>> util.size  # doctest: +SKIP
859
>>> util.dist  # doctest: +SKIP
<importlib.metadata._hooks.PathDistribution object at 0x101e0cef0>
>>> util.hash  # doctest: +SKIP
<FileHash mode: sha256 value: bYkw5oMccfazVCoYQwKkkemoVyMAFoR34mmKBx8R1NI>

Once you have the file, you can also read its contents:

>>> print(util.read_text())  # doctest: +SKIP
import base64
import sys
...
def as_bytes(s):
    if isinstance(s, text_type):
        return s.encode('utf-8')
    return s

You can also use the !locate method to get the absolute path to the file:

>>> util.locate()  # doctest: +SKIP
PosixPath('/home/gustav/example/lib/site-packages/wheel/util.py')

In the case where the metadata file listing files (RECORD or SOURCES.txt) is missing, !files will return None. The caller may wish to wrap calls to !files in always_iterable or otherwise guard against this condition if the target distribution is not known to have the metadata present.

Distribution requirements

To get the full set of requirements for a Distribution Package, use the !requires function:

>>> requires('wheel')  # doctest: +SKIP
["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"]

Mapping import to distribution packages

A convenience method to resolve the Distribution Package name (or names, in the case of a namespace package) that provide each importable top-level Python module or Import Package:

>>> packages_distributions()
{'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'], 'jaraco': ['jaraco.classes', 'jaraco.functools'], ...}

Some editable installs, do not supply top-level names, and thus this function is not reliable with such installs.

Distributions

While the module level API described above is the most common and convenient usage, all that information is accessible from the Distribution class. !Distribution is an abstract object that represents the metadata for a Python Distribution Package. Get the concrete !Distribution subclass instance for an installed distribution package by calling the distribution function:

>>> from importlib.metadata import distribution  # doctest: +SKIP
>>> dist = distribution('wheel')  # doctest: +SKIP
>>> type(dist)  # doctest: +SKIP
<class 'importlib.metadata.PathDistribution'>

Thus, an alternative way to get e.g. the version number is through the Distribution.version attribute:

>>> dist.version  # doctest: +SKIP
'0.32.3'

The same applies for entry_points and files.

>>> dist.metadata['Requires-Python']  # doctest: +SKIP
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
>>> dist.metadata['License']  # doctest: +SKIP
'MIT'

>>> dist.origin.url
'file:///path/to/wheel-0.32.3.editable-py3-none-any.whl'

Distribution Discovery

By default, this package provides built-in support for discovery of metadata for file system and zip file Distribution Packages. This metadata finder search defaults to sys.path, but varies slightly in how it interprets those values from how other import machinery does. In particular:

• importlib.metadata does not honor bytes objects on sys.path.
• importlib.metadata will incidentally honor pathlib.Path objects on sys.path even though such values will be ignored for imports.

Implementing Custom Providers

importlib.metadata address two API surfaces, one for consumers and another for providers. Most users are consumers, consuming metadata provided by the packages. There are other use-cases, however, where users wish to expose metadata through some other mechanism, such as alongside a custom importer. Such a use case calls for a custom provider.

Because Distribution Package metadata is not available through sys.path searches, or package loaders directly, the metadata for a distribution is found through import system finders <finders-and-loaders>. To find a distribution package's metadata, importlib.metadata queries the list of meta path finders <meta path finder> on sys.meta_path.

The implementation has hooks integrated into the PathFinder, serving metadata for distribution packages found on the file system.

The abstract class importlib.abc.MetaPathFinder defines the interface expected of finders by Python's import system. importlib.metadata extends this protocol by looking for an optional find_distributions callable on the finders from sys.meta_path and presents this extended interface as the DistributionFinder abstract base class, which defines this abstract method:

@abc.abstractmethod
def find_distributions(context=DistributionFinder.Context()) -> Iterable[Distribution]:
    """Return an iterable of all Distribution instances capable of
    loading the metadata for packages for the indicated ``context``.
    """

The DistributionFinder.Context object provides ~DistributionFinder.Context.path and ~DistributionFinder.Context.name properties indicating the path to search and name to match and may supply other relevant context sought by the consumer.

In practice, to support finding distribution package metadata in locations other than the file system, subclass !Distribution and implement the abstract methods. Then from a custom finder, return instances of this derived !Distribution in the find_distributions() method.

Example

Imagine a custom finder that loads Python modules from a database:

class DatabaseImporter(importlib.abc.MetaPathFinder):
    def __init__(self, db):
        self.db = db

    def find_spec(self, fullname, target=None) -> ModuleSpec:
        return self.db.spec_from_name(fullname)

sys.meta_path.append(DatabaseImporter(connect_db(...)))

That importer now presumably provides importable modules from a database, but it provides no metadata or entry points. For this custom importer to provide metadata, it would also need to implement DistributionFinder:

from importlib.metadata import DistributionFinder

class DatabaseImporter(DistributionFinder):
    ...

    def find_distributions(self, context=DistributionFinder.Context()):
        query = dict(name=context.name) if context.name else {}
        for dist_record in self.db.query_distributions(query):
            yield DatabaseDistribution(dist_record)

In this way, query_distributions would return records for each distribution served by the database matching the query. For example, if requests-1.0 is in the database, find_distributions would yield a DatabaseDistribution for Context(name='requests') or Context(name=None).

For the sake of simplicity, this example ignores context.path. The path attribute defaults to sys.path and is the set of import paths to be considered in the search. A DatabaseImporter could potentially function without any concern for a search path. Assuming the importer does no partitioning, the "path" would be irrelevant. In order to illustrate the purpose of path, the example would need to illustrate a more complex DatabaseImporter whose behavior varied depending on sys.path/PYTHONPATH. In that case, the find_distributions should honor the context.path and only yield Distributions pertinent to that path.

DatabaseDistribution, then, would look something like:

class DatabaseDistribution(importlib.metadata.Distribution):
    def __init__(self, record):
        self.record = record

    def read_text(self, filename):
        """
        Read a file like "METADATA" for the current distribution.
        """
        if filename == "METADATA":
            return f"""Name: {self.record.name}
Version: {self.record.version}
"""
        if filename == "entry_points.txt":
            return "\n".join(
              f"""[{ep.group}]\n{ep.name}={ep.value}"""
              for ep in self.record.entry_points)

    def locate_file(self, path):
        raise RuntimeError("This distribution has no file system")

This basic implementation should provide metadata and entry points for packages served by the DatabaseImporter, assuming that the record supplies suitable .name, .version, and .entry_points attributes.

The DatabaseDistribution may also provide other metadata files, like RECORD (required for !Distribution.files) or override the implementation of !Distribution.files. See the source for more inspiration.