Dev Guide

Developer Guide

Making Open Source More Inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. We ask that you open an issue or pull request if you come upon a term that we have missed. For more details, see our CTO Chris Wright's message.

Welcome to the Ansible Developer Guide!

Who should use this guide?

If you want to extend Ansible by using a custom module or plugin locally, creating a module or plugin, adding functionality to an existing module, or expanding test coverage, this guide is for you. We've included detailed information for developers on how to test and document modules, as well as the prerequisites for getting your module or plugin accepted into the main Ansible repository.

Find the task that best describes what you want to do:

If you prefer to read the entire guide, here's a list of the pages in order.

developing_locally developing_modules developing_modules_general developing_modules_checklist developing_modules_best_practices developing_python_3 debugging developing_modules_documenting ansible_markup sidecar developing_modules_general_windows developing_modules_in_groups testing module_lifecycle developing_plugins developing_inventory developing_core developing_program_flow_modules developing_api developing_rebasing developing_module_utilities developing_secret_masking developing_collections_path developing_collections migrating_roles collections_galaxy_meta overview_architecture

Ansible markup

Ansible markup allows you to format and structure documentation for Ansible modules, plugins, and roles. It lets you add basic formatting to text, such as bold, italics, code, and horizontal lines, as well as create various references, such as URLs, hyperlinks, Ansible module references, and RST references. The Ansible markup language was extended in 2023, starting with ansible-core 2.15. It now lets you apply semantic markup for values, module/plugin options, return values, environment variables, and for referencing plugins.

This page documents the currently supported Ansible markup.

Semantic markup within module documentation

Use the semantic markup to highlight option names, option values, and environment variables. The markup processor formats these highlighted terms in a uniform way. With semantic markup, we can modify how the output looks without changing underlying code.

The correct formats for semantic markup are as follows:

The parameters for these formatting functions can use escaping with backslashes: V(foo(bar="a\\b"\), baz) results in the formatted value foo(bar="a\b"), baz).

Rules for using O() and RV()

Rules for using O() and RV() are very strict. You must follow syntax rules so that documentation renderers can create hyperlinks for the options and return values, respectively.

The allowed syntaxes are as follows:

Option names can refer to suboptions by listing the path to the option separated by dots. For example, if you have an option foo with suboption bar, then you must use O(foo.bar) to reference that suboption. You can add array indications like O(foo[].bar) or even O(foo[-1].bar) to indicate specific list elements. Everything between [ and ] pairs will be ignored to determine the real name of the option. For example, O(foo[foo | length - 1].bar[]) results in the same link as O(foo.bar), but the text foo[foo | length - 1].bar[] displays instead of foo.bar.

The same syntaxes can be used for RV(), except that these will refer to return value names instead of option names; for example RV(ansible.builtin.service_facts#module:ansible_facts.services) refers to the ansible.builtin.service_facts#module:ansible_facts.services fact returned by the ansible.builtin.service_facts module <ansible.builtin.service_facts#module>.

Linking within module documentation

If you are creating your own documentation site, you will need to use the intersphinx extension to convert R(), M(), P(), O(), and RV() to the correct links.

Format macros within module documentation

While it is possible to use standard Ansible formatting macros to control the look of other terms in module documentation, you should do so sparingly.

Possible macros include the following:

Note that C(), B(), and I() do not allow escaping, and thus cannot contain the value ) as it always ends the formatting sequence. If you need to use ) inside C(), we recommend to use V() instead; see the above section on semantic markup.

`ansible-core` project branches and tags

devel branch

All new development on the next version of ansible-core occurs exclusively in the devel branch, and all bugfixes to prior releases must first be merged to devel before being backported to one or more stable branches for inclusion in servicing releases. Around the Beta 1 milestone, a new stable-X.Y branch is cut from devel, which is then updated to host development of the X.Y+1 release. External automated testing of Ansible content from devel is not generally recommended.

stable-X.Y branches

All ansible-core X.Y.Z releases are created from a corresponding stable-X.Y branch. A release's stable branch is typically cut from devel around X.Y.0 beta 1 (when the release is feature complete). All further bugfixes (no enhancements!) must be made against devel and backported to applicable stable branches.

vX.Y.Z tags

Each ansible-core vX.Y.Z release is tagged from the release commit in the corresponding stable-X.Y branch, allowing access to the exact source used to create the release. As of ansible-core 2.13, the auto-generated GitHub tarball of the tag contents is considered the official canonical release artifact.

milestone branch

A milestone branch is a slow-moving stream of the devel branch, intended for external testing of ansible-core features under active development. As described in the ansible_core_roadmaps for a given release, development is typically split into three phases of decreasing duration, with larger and more invasive changes targeted to be merged to devel in earlier phases. The milestone branch is updated to the contents of devel at the end of each development phase. This allows testing of semi-stable unreleased features on a predictable schedule without the exposure to the potential instability of the daily commit "fire hose" from devel. When a release reaches the Beta 1 milestone, the milestone branch will be updated to the first devel commit after the version number has been increased. Further testing of the same release should be done from the new stable-X.Y branch that was created. If a severe issue that significantly affects community testing or stability is discovered in the milestone branch, the branch contents may require unscheduled adjustment, but not in a way that prevents fast-forward updates (for example, milestone-only commits will not be created or cherry-picked from devel).

The following example is for illustrative purposes only. See the ansible_core_roadmaps for accurate dates. For example, the milestone branch in 2.13 ansible-core roadmap updated as follows:

Developer Guide

Making Open Source More Inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. We ask that you open an issue or pull request if you come upon a term that we have missed. For more details, see our CTO Chris Wright's message.

Welcome to the Ansible Developer Guide!

Who should use this guide?

If you want to extend Ansible by using a custom module or plugin locally, creating a module or plugin, adding functionality to an existing module, or expanding test coverage, this guide is for you. We've included detailed information for developers on how to test and document modules, as well as the prerequisites for getting your module or plugin accepted into the main Ansible repository.

Find the task that best describes what you want to do:

If you prefer to read the entire guide, here's a list of the pages in order.

developing_locally developing_modules developing_modules_general developing_modules_checklist developing_modules_best_practices developing_python_3 debugging developing_modules_documenting ansible_markup sidecar developing_modules_general_windows developing_modules_in_groups testing module_lifecycle developing_plugins developing_inventory developing_core developing_program_flow_modules developing_api developing_rebasing developing_module_utilities developing_secret_masking developing_collections_path developing_collections migrating_roles collections_galaxy_meta overview_architecture

Debugging modules

Detailed debugging steps

Ansible modules are put together as a zip file consisting of the module file and the various Python module boilerplate inside of a wrapper script. To see what is actually happening in the module, you need to extract the file from the wrapper. The wrapper script provides helper methods that let you do that.

The following steps use localhost as the target host, but you can use the same steps to debug against remote hosts as well. For a simpler approach to debugging without using the temporary files, see simple debugging <simple_debugging>.

  1. Set ANSIBLE_KEEP_REMOTE_FILES to 1 on the control host so Ansible will keep the remote module files instead of deleting them after the module finishes executing. Use the -vvv option to make Ansible more verbose. This will display the file name of the temporary module file.

    $ ANSIBLE_KEEP_REMOTE_FILES=1 ansible localhost -m ping -a 'data=debugging_session' -vvv
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: badger
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" && echo "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" )'
<127.0.0.1> PUT /var/tmp/tmpjdbJ1w TO /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/AnsiballZ_ping.py
<127.0.0.1> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/AnsiballZ_ping.py && sleep 0'
localhost | SUCCESS => {
    "changed": false,
    "invocation": {
        "module_args": {
            "data": "debugging_session"
        },
        "module_name": "ping"
    },
    "ping": "debugging_session"
}

  2. Navigate to the temporary directory from the previous step. If the previous command was run against a remote host, connect to that host first before trying to navigate to the temporary directory.

    $ ssh remotehost  # only if not debugging against localhost
$ cd /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595

  3. Run the wrapper's explode command to turn the string into some Python files that you can work with.

    $ python AnsiballZ_ping.py explode
Module expanded into:
/home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/debug_dir

    If you want to examine the wrapper file you can. It will show a small Python script with a large base64 encoded string. The string contains the module to execute.

  4. When you look into the temporary directory you'll see a structure like this:

    ├── AnsiballZ_ping.py
└── debug_dir
    ├── ansible
    │   ├── __init__.py
    │   ├── module_utils
    │   │   ├── __init__.py
    │   │   ├── _text.py
    │   │   ├── basic.py
    │   │   ├── common
    │   │   ├── compat
    │   │   ├── distro
    │   │   ├── parsing
    │   │   ├── pycompat24.py
    │   │   └── six
    │   └── modules
    │       ├── __init__.py
    │       └── ping.py
    └── args
    • AnsiballZ_ping.py is the Python script with the module code stored in a base64 encoded string. It contains various helper functions for executing the module.
    • ping.py is the code for the module itself. You can modify this code to see what effect it would have on your module, or for debugging purposes.
    • The args file contains a JSON string. The string is a dictionary containing the module arguments and other variables that Ansible passes into the module to change its behavior. Modify this file to change the parameters passed to the module.
    • The ansible directory contains the module code in modules as well as code from ansible.module_utils that is used by the module. Ansible includes files for any ansible.module_utils imports in the module but not any files from any other module. If your module uses ansible.module_utils.url Ansible will include it for you. But if your module includes requests, then you'll have to make sure that the Python requests library is installed on the system before running the module.

    You can modify files in this directory if you suspect that the module is having a problem in some of this boilerplate code rather than in the module code you have written.

  5. Once you edit the code or arguments in the exploded tree, use the execute subcommand to run it:

    $ python AnsiballZ_ping.py execute
{"invocation": {"module_args": {"data": "debugging_session"}}, "changed": false, "ping": "debugging_session"}

    This subcommand inserts the absolute path to debug_dir as the first item in sys.path and invokes the script using the arguments in the args file. You can continue to run the module like this until you understand the problem. Then you can copy the changes back into your real module file and test that the real module works by using the ansible or ansible-playbook.

Simple debugging

The easiest way to run a debugger in a module, either local or remote, is to use madbg. Add import madbg; madbg.set_trace() in the module code on the control node at the desired break point. To connect to the debugger, run madbg connect. See the madbg documentation for how to specify the host and port. If connecting to a remote node, make sure to use a port that is allowed by any firewall between the control node and the remote node.

This technique should work with any remote debugger, but we do not guarantee any particular remote debugging tool will work.

The q library is another very useful debugging tool.

Since print() statements do not work inside modules, raising an exception is a good approach if you just want to see some specific data. Put raise Exception(some_value) somewhere in the module and run it normally. Ansible will handle this exception, pass the message back to the control node, and display it.

Python API

Topics

Attention

The Ansible API is intended for internal Ansible use. Ansible may make changes to this API at any time that could break backward compatibility with older versions of the API. Because of this, external use is not supported by Ansible. If you want to use Python API only for executing playbooks or modules, consider ansible-runner first.

If you would like to use Ansible programmatically from a language other than Python, trigger events asynchronously, or have access control and logging demands, please see the AWX project.

developing_inventory
Developing dynamic inventory integrations

developing_modules_general
Getting started on developing a module

developing_plugins
How to develop plugins

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Developing collections

Collections are a distribution format for Ansible content. You can package and distribute playbooks, roles, modules, and plugins using collections. A typical collection addresses a set of related use cases. For example, the cisco.ios collection automates management of Cisco IOS devices.

You can create a collection and publish it to Ansible Galaxy or to a private Automation Hub instance. You can publish certified collections to the Red Hat Automation Hub, part of the Red Hat Ansible Automation Platform.

Examine the developing_collections_path to understand how to go from creating a collection to having it included in the Ansible package distribution.

developing_collections_creating developing_collections_shared developing_collections_testing developing_collections_resource_reporting developing_collections_distributing developing_collections_documenting

developing_collections_migrating developing_collections_contributing developing_collections_changelogs

developing_collections_structure collections_galaxy_meta

For instructions on developing modules, see developing_modules_general.

collections
Learn how to install and use collections in playbooks and roles

contributing_maintained_collections
Guidelines for contributing to selected collections

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Generating changelogs and porting guide entries in a collection

You can create and share changelog and porting guide entries for your collection. If your collection is part of the Ansible Community package, we recommend that you use the antsibull-changelog tool to generate Ansible-compatible changelogs. The Ansible changelog uses the output of this tool to collate all the collections included in an Ansible release into one combined changelog for the release.

Ansible here refers to the Ansible 2.10 or later release that includes a curated set of collections.

Understanding antsibull-changelog

The antsibull-changelog tool allows you to create and update changelogs for Ansible collections that are compatible with the combined Ansible changelogs. This is an update to the changelog generator used in prior Ansible releases. The tool adds three new changelog fragment categories: breaking_changes, security_fixes and trivial. The tool also generates the changelog.yaml file that Ansible uses to create the combined CHANGELOG.rst file and Porting Guide for the release.

See changelogs_how_to and the antsibull-changelog documentation for complete details.

The collection maintainers set the changelog policy for their collections. See the individual collection contributing guidelines for complete details.

Generating changelogs

To initialize changelog generation:

  1. Install antsibull-changelog: pip install antsibull-changelog.
  2. Initialize changelogs for your repository: antsibull-changelog init <path/to/your/collection>.
  3. Optionally, edit the changelogs/config.yaml file to customize the location of the generated changelog .rst file or other options. See Bootstrapping changelogs for collections for details.

To generate changelogs from the changelog fragments you created:

  1. Optionally, validate your changelog fragments: antsibull-changelog lint.
  2. Generate the changelog for your release: antsibull-changelog release [--version version_number].

Add the --reload-plugins option if you ran the antsibull-changelog release command previously and the version of the collection has not changed. antsibull-changelog caches the information on all plugins and does not update its cache until the collection version changes.

Porting Guide entries from changelog fragments

The Ansible changelog generator automatically adds several changelog fragment categories to the Ansible Porting Guide:

Including collection changelogs into Ansible

If your collection is part of Ansible, use one of the following three options to include your changelog into the Ansible release changelog:

For the first two options, Ansible pulls the changelog details from Galaxy so your changelogs must be included in the collection version on Galaxy that is included in the upcoming Ansible release.

collection_changelogs
Learn how to create good changelog fragments.

collections
Learn how to install and use collections.

contributing_maintained_collections
Guidelines for contributing to selected collections

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Contributing to collections

If you want to add functionality to an existing collection, modify a collection you are using to fix a bug, or change the behavior of a module in a collection, clone the Git repository for that collection and make changes on a branch. You can combine changes to a collection with a local checkout of Ansible (source hacking/env-setup). You should first check the collection repository to see if it has specific contribution guidelines. These are typically listed in the README.md or CONTRIBUTING.md files within the repository. See collection_quickstart for more general guidelines and testing_running_locally for testing guidelines.

Contributing to a collection: community.general

These instructions apply to collections hosted in the ansible_collections GitHub organization. For other collections, especially for collections not hosted on GitHub, check the README.md of the collection for information on contributing to it.

This example uses the community.general collection. To contribute to other collections in the same GitHub org, replace the folder names community and general with the namespace and collection name of a different collection.

Prerequisites

Creating a PR

mkdir -p ~/dev/ansible/collections/ansible_collections/community

cd ~/dev/ansible/collections/ansible_collections/community
git clone git@github.com:ansible-collections/community.general.git general

cd ~/dev/ansible/collections/ansible_collections/community/general
git remote add upstream git@github.com:ansible-collections/community.general.git

You can test your changes by using this checkout of community.general in playbooks and roles with whichever version of Ansible you have installed locally, including a local checkout of ansible/ansible's devel branch.

collections
Learn how to install and use collections.

contributing_maintained_collections
Guidelines for contributing to selected collections

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Creating collections

To create a collection:

  1. Create a new collection<creating_new_collections>, optionally using a custom collection template<creating_collection_from_custom_template>, with the ansible-galaxy collection init command.
  2. Add modules and other content to the collection.
  3. Build the collection into a collection artifact with ansible-galaxy collection build<building_collections>.
  4. Publish the collection artifact to Galaxy with ansible-galaxy collection publish<publishing_collections>.

A user can then install your collection on their systems.

Naming your collection

Collection names consist of a namespace and a name, separated by a period (.). Both namespace and name should be valid Python identifiers. This means that they should consist of ASCII letters, digits, and underscores.

Usually namespaces and names use lower-case letters, digits, and underscores, but no upper-case letters.

You should check Ansible Galaxy's namespace list to ensure that your desired namespace is not already registered. If you use a namespace or full collection name that conflicts with another collection on Galaxy, then the ansible-galaxy collection install can install the wrong collection.

There are a few special namespaces:

ansible

The ansible namespace is owned by Red Hat and reserved for official Ansible collections. Two special members are the synthetic ansible.builtin and ansible.legacy collections. These cannot be found on Ansible Galaxy, but are built-in into ansible-core.

community

The community namespace is owned by the Ansible community. Collections from this namespace generally live in the GitHub ansible-collection organization. If you want to create a collection in this namespace, request<request_coll_repo> it on the forum.

local

The local namespace does not contain any collection on Ansible Galaxy, and the intention is that this will never change. You can use the local namespace for collections that are locally on your machine or locally in your Git repositories, without having to fear collisions with actually existing collections on Ansible Galaxy.

To request a new namespace in Galaxy, do the following:

  1. Log in to the Ansible forum.
  2. Submit a namespace request.

The Red Hat Community and Partner Engineering team will be notified of your namespace request and create it for you.

Creating a new collection

Create your collection skeleton in a path that includes ansible_collections, for example collections/ansible_collections/.

To start a new collection, run the following command in your collections directory:

ansible_collections#> ansible-galaxy collection init my_namespace.my_collection

Both the namespace and collection names use the same strict set of requirements. Both are limited to alphanumeric characters and underscores, must have a minimum length of three characters, and cannot start with an underscore.

It will create the structure [my_namespace]/[my_collection]/[collection skeleton].

Hint

If Git is used for version control, the corresponding repository should be initialized in the collection directory.

Once the collection exists, you can populate the directories with the content you want inside the collection. See ansible-collections GitHub Org to get a better idea of what you can place inside a collection.

Reference: the ansible-galaxy collection command

Currently the ansible-galaxy collection command implements the following sub commands:

To learn more about the ansible-galaxy command-line tool, see the ansible-galaxy man page.

Creating a collection from a custom template

The built-in collection template is a simple example of a collection that works with ansible-core, but if you want to simplify your development process you may want to create a custom collection template to pass to ansible-galaxy collection init.

A collection skeleton is a directory that looks like a collection directory but any .j2 files (excluding those in templates/ and roles/*/templates/) will be templated by ansible-galaxy collection init. The skeleton's galaxy.yml.j2 file should use the variables namespace and collection_name which are derived from ansible-galaxy init namespace.collection_name, and will populate the metadata in the initialized collection's galaxy.yml file. There are a few additional variables available by default (for example, version is 1.0.0), and these can be supplemented/overridden using --extra-vars.

An example galaxy.yml.j2 file that accepts an optional dictionary variable dependencies could look like this:

namespace: {{ namespace }}
name: {{ collection_name }}
version: {{ (version|quote) is version('0.0.0', operator='gt', version_type='semver')|ternary(version, undef('version must be a valid semantic version greater than 0.0.0')) }}
dependencies: {{ dependencies | default({}, true) }}

To initialize a collection using the new template, pass the path to the skeleton with ansible-galaxy collection init:

ansible_collections#> ansible-galaxy collection init --collection-skeleton /path/to/my/namespace/skeleton --extra-vars "@my_vars_file.json" my_namespace.my_collection

You can configure the collection skeleton using collection_skeleton in ansible.cfg. To ignore files and directories in the collection skeleton, use the collection_skeleton_ignore option in ansible.cfg.

[galaxy]
collection_skeleton = /path/to/collection_skeleton
collection_skeleton_ignore = ^.git$,^.*/.git_keep$,^\./CLAUDE\.md$

The collection_skeleton_ignore option is a list of regular expressions to match files and directories to ignore. The regex are matched against the relative path of the file or directory from the skeleton directory.

Before ansible-core 2.17, collection skeleton templating is limited to the few hardcoded variables including namespace, collection_name, and version.

The default collection skeleton uses an internal filter comment_ify that isn't accessibly to --collection-skeleton. Use ansible-doc -t filter|test --list to see available plugins.

Creating collections with ansible-creator

ansible-creator is designed to quickly scaffold an Ansible collection project.

The Ansible Development Tools package offers a convenient way to install ansible-creator along with a curated set of tools for developing automation content.

After installing ansible-creator you can initialize a project in one of the following ways:

collections
Learn how to install and use collections.

collection_structure
Directories and files included in the collection skeleton

Ansible Development Tools (ADT)
Python package of tools to create and test Ansible content.

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Distributing collections

A collection is a distribution format for Ansible content. A typical collection contains modules and other plugins that address a set of related use cases. For example, a collection might automate administering a particular database. A collection can also contain roles and playbooks.

Before distributing your collection, ensure you have updated the galaxy.yml file. See collection_structure for details.

To distribute your collection and allow others to use it, you can publish your collection on one or more distribution server. Distribution servers include:

Distribution server Collections accepted
Ansible Galaxy All collections
Pulp 3 Galaxy All collections, supports signed collections
Red Hat Automation Hub Only collections certified by Red Hat, supports signed collections
Privately hosted Automation Hub Collections authorized by the owners

Distributing collections involves four major steps:

  1. Initial configuration of your distribution server or servers
  2. Building your collection tarball
  3. Preparing to publish your collection
  4. Publishing your collection

Initial configuration of your distribution server or servers

Configure a connection to one or more distribution servers so you can publish collections there. You only need to configure each distribution server once. You must repeat the other steps (building your collection tarball, preparing to publish, and publishing your collection) every time you publish a new collection or a new version of an existing collection.

  1. Create a namespace on each distribution server you want to use.
  2. Get an API token for each distribution server you want to use.
  3. Specify the API token for each distribution server you want to use.

Creating a namespace

You must upload your collection into a namespace on each distribution server. If you have a login for Ansible Galaxy, your Ansible Galaxy username is usually also an Ansible Galaxy namespace.

Namespaces on Ansible Galaxy cannot include hyphens. If you have a login for Ansible Galaxy that includes a hyphen, your Galaxy username is not also a Galaxy namespace. For example, awesome-user is a valid username for Ansible Galaxy, but it is not a valid namespace.

You can create additional namespaces on Ansible Galaxy if you choose. For Red Hat Automation Hub and private Automation Hub you must create a namespace before you can upload your collection. To create a namespace:

Specify the namespace in the galaxy.yml file for each collection. For more information on the galaxy.yml file, see collections_galaxy_meta.

Getting your API token

An API token authenticates your connection to each distribution server. You need a separate API token for each distribution server. Use the correct API token to connect to each distribution server securely and protect your content.

To get your API token:

Specifying your API token and distribution server

Each time you publish a collection, you must specify the API token and the distribution server to create a secure connection. You have two options for specifying the token and distribution server:

Specifying the token and distribution server in configuration

By default, Ansible Galaxy is configured as the only distribution server. You can add other distribution servers and specify your API token or tokens in configuration by editing the galaxy_server_list section of your ansible.cfg file. This is the most secure way to manage authentication for distribution servers. Specify a URL and token for each server. For example:

[galaxy]
server_list = release_galaxy

[galaxy_server.release_galaxy]
url=https://galaxy.ansible.com/
token=abcdefghijklmnopqrtuvwxyz

You cannot use apt-key with any servers defined in your galaxy_server_list <galaxy_server_config>. See galaxy_server_config for complete details.

Specifying the token at the command line

You can specify the API token at the command line using the --token argument of the ansible-galaxy command. There are three ways to specify the distribution server when passing the token at the command line:

For example:

ansible-galaxy collection publish path/to/my_namespace-my_collection-1.0.0.tar.gz --token abcdefghijklmnopqrtuvwxyz

Using the --token argument is insecure. Passing secrets at the command line may expose them to others on the system.

Building your collection tarball

After configuring one or more distribution servers, build a collection tarball. The collection tarball is the published artifact, the object that you upload and other users download to install your collection. To build a collection tarball:

  1. Review all settings in your galaxy.yml file. See collections_galaxy_meta for details. Ensure you have updated the version number. Each time you publish your collection, it must have a new version number. You cannot make changes to existing versions of your collection on a distribution server. If you try to upload the same collection version more than once, the distribution server returns the error Code: conflict.collection_exists. Collections follow semantic versioning rules. For more information on versions, see collection_versions. For more information on the galaxy.yml file, see collections_galaxy_meta.
  2. Run ansible-galaxy collection build from inside the top-level directory of the collection. For example:
collection_dir#> ansible-galaxy collection build

This command builds a tarball of the collection in the current directory, which you can upload to your selected distribution server:

my_collection/
├── galaxy.yml
├── ...
├── my_namespace-my_collection-1.0.0.tar.gz
└── ...

* To reduce the size of collections, certain files and folders are excluded from the collection tarball by default. See ignoring_files_and_folders_collections if your collection directory contains other files you want to exclude. * The current Galaxy maximum tarball size is 20 MB.

You can upload your tarball to one or more distribution servers. You can also distribute your collection locally by copying the tarball to install your collection directly on target systems.

Ignoring files and folders

You can exclude files from your collection with either build_ignore <build_ignore> or manifest_directives. For more information on the galaxy.yml file, see collections_galaxy_meta.

Include all, with explicit ignores

By default, the build step includes all the files in the collection directory in the tarball except for the following:

To exclude other files and folders from your collection tarball, set a list of file glob-like patterns in the build_ignore key in the collection's galaxy.yml file. These patterns use the following special characters for wildcard matching:

For example, to exclude the sensitive folder within the playbooks folder as well any .tar.gz archives, set the following in your galaxy.yml file:

build_ignore:
- playbooks/sensitive
- '*.tar.gz'

The build_ignore feature is only supported with ansible-galaxy collection build in Ansible 2.10 or newer.

Manifest Directives

2.14

The galaxy.yml file supports manifest directives that are historically used in Python packaging, as described in MANIFEST.in commands.

The use of manifest requires installing the optional distlib Python dependency.

The manifest feature is only supported with ansible-galaxy collection build in ansible-core 2.14 or newer, and is mutually exclusive with build_ignore.

For example, to exclude the sensitive folder within the playbooks folder as well as any .tar.gz archives, set the following in your galaxy.yml file:

manifest:
  directives:
    - recursive-exclude playbooks/sensitive **
    - global-exclude *.tar.gz

By default, the MANIFEST.in style directives would exclude all files by default, but there are default directives in place. Those default directives are described below. To see the directives in use during build, pass -vvv with the ansible-galaxy collection build command.

include meta/*.yml
include *.txt *.md *.rst COPYING LICENSE
recursive-include tests **
recursive-include docs **.rst **.yml **.yaml **.json **.j2 **.txt
recursive-include roles **.yml **.yaml **.json **.j2
recursive-include playbooks **.yml **.yaml **.json
recursive-include changelogs **.yml **.yaml
recursive-include plugins */**.py
recursive-include plugins/become **.yml **.yaml
recursive-include plugins/cache **.yml **.yaml
recursive-include plugins/callback **.yml **.yaml
recursive-include plugins/cliconf **.yml **.yaml
recursive-include plugins/connection **.yml **.yaml
recursive-include plugins/filter **.yml **.yaml
recursive-include plugins/httpapi **.yml **.yaml
recursive-include plugins/inventory **.yml **.yaml
recursive-include plugins/lookup **.yml **.yaml
recursive-include plugins/netconf **.yml **.yaml
recursive-include plugins/shell **.yml **.yaml
recursive-include plugins/strategy **.yml **.yaml
recursive-include plugins/test **.yml **.yaml
recursive-include plugins/vars **.yml **.yaml
recursive-include plugins/modules **.ps1 **.yml **.yaml
recursive-include plugins/module_utils **.ps1 **.psm1 **.cs
# manifest.directives from galaxy.yml inserted here
exclude galaxy.yml galaxy.yaml MANIFEST.json FILES.json <namespace>-<name>-*.tar.gz
recursive-exclude tests/output **
global-exclude /.* /__pycache__

<namespace>-<name>-*.tar.gz is expanded with the actual namespace and name.

The manifest.directives supplied in galaxy.yml are inserted after the default includes and before the default excludes.

To enable the use of manifest directives without supplying your own, insert either manifest: {} or manifest: null in the galaxy.yml file and remove any use of build_ignore.

If the default manifest directives do not meet your needs, you can set manifest.omit_default_directives to a value of true in galaxy.yml. You then must specify a full compliment of manifest directives in galaxy.yml. The defaults documented above are a good starting point.

Below is an example where the default directives are not included.

manifest:
  directives:
    - include meta/runtime.yml
    - include README.md LICENSE
    - recursive-include plugins */**.py
    - exclude galaxy.yml MANIFEST.json FILES.json <namespace>-<name>-*.tar.gz
    - recursive-exclude tests/output **
  omit_default_directives: true

Signing a collection

You can include a GnuPG signature with your collection on a Pulp 3 Galaxy server. See Enabling collection signing for details.

You can manually generate detached signatures for a collection using the gpg CLI using the following step. This step assume you have generated a GPG private key, but do not cover this process.

ansible-galaxy collection build
tar -Oxzf namespace-name-1.0.0.tar.gz MANIFEST.json | gpg --output namespace-name-1.0.0.asc --detach-sign --armor --local-user email@example.com -

Preparing to publish your collection

Each time you publish your collection, you must create a new version <collection_versions> on the distribution server. After you publish a version of a collection, you cannot delete or modify that version. To avoid unnecessary extra versions, check your collection for bugs, typos, and other issues locally before publishing:

  1. Install the collection locally.
  2. Review the locally installed collection before publishing a new version.

Installing your collection locally

You have two options for installing your collection locally:

Installing your collection locally from the tarball

To install your collection locally from the tarball, run ansible-galaxy collection install and specify the collection tarball. You can optionally specify a location using the -p flag. For example:

collection_dir#> ansible-galaxy collection install my_namespace-my_collection-1.0.0.tar.gz -p ./collections

Install the tarball into a directory configured in COLLECTIONS_PATHS so Ansible can easily find and load the collection. If you do not specify a path value, ansible-galaxy collection install installs the collection in the first path defined in COLLECTIONS_PATHS.

Installing your collection locally from a Git repository

To install your collection locally from a Git repository, specify the repository and the branch you want to install:

collection_dir#> ansible-galaxy collection install git+https://github.com/org/repo.git,devel

You can install a collection from a git repository instead of from Galaxy or Automation Hub. As a developer, installing from a git repository lets you review your collection before you create the tarball and publish the collection. As a user, installing from a git repository lets you use collections or versions that are not in Galaxy or Automation Hub yet. This functionality is meant as a minimal shortcut for developers of content as previously described, and git repositories may not support the full set of features from the ansible-galaxy CLI. In complex cases, a more flexible option may be to git clone the repository into the correct file structure of the collection installation directory.

The repository must contain a galaxy.yml or MANIFEST.json file. This file provides metadata such as the version number and namespace of the collection.

Installing a collection from a git repository at the command line

To install a collection from a git repository at the command line, use the URI of the repository instead of a collection name or path to a tar.gz file. Use the prefix git+, unless you're using SSH authentication with the user git (for example, git@github.com:ansible-collections/ansible.windows.git). You can specify a branch, commit, or tag using the comma-separated git commit-ish syntax.

For example:

# Install a collection in a repository using the latest commit on the branch 'devel'
ansible-galaxy collection install git+https://github.com/organization/repo_name.git,devel

# Install a collection from a private GitHub repository
ansible-galaxy collection install git@github.com:organization/repo_name.git

# Install a collection from a local git repository
ansible-galaxy collection install git+file:///home/user/path/to/repo_name.git

Embedding credentials into a git URI is not secure. Use safe authentication options to prevent your credentials from being exposed in logs or elsewhere.

Specifying the collection location within the git repository

When you install a collection from a git repository, Ansible uses the collection galaxy.yml or MANIFEST.json metadata file to build the collection. By default, Ansible searches two paths for collection galaxy.yml or MANIFEST.json metadata files:

If a galaxy.yml or MANIFEST.json file exists in the top level of the repository, Ansible uses the collection metadata in that file to install an individual collection.

├── galaxy.yml
├── plugins/
│   ├── lookup/
│   ├── modules/
│   └── module_utils/
└─── README.md

If a galaxy.yml or MANIFEST.json file exists in one or more directories in the repository path (one level deep), Ansible installs each directory with a metadata file as a collection. For example, Ansible installs both collection1 and collection2 from this repository structure by default:

├── collection1
│   ├── docs/
│   ├── galaxy.yml
│   └── plugins/
│       ├── inventory/
│       └── modules/
└── collection2
    ├── docs/
    ├── galaxy.yml
    ├── plugins/
    |   ├── filter/
    |   └── modules/
    └── roles/

If you have a different repository structure or only want to install a subset of collections, you can add a fragment to the end of your URI (before the optional comma-separated version) to indicate the location of the metadata file or files. The path should be a directory, not the metadata file itself. For example, to install only collection2 from the example repository with two collections:

ansible-galaxy collection install git+https://github.com/organization/repo_name.git#/collection2/

In some repositories, the main directory corresponds to the namespace:

namespace/
├── collectionA/
|   ├── docs/
|   ├── galaxy.yml
|   ├── plugins/
|   │   ├── README.md
|   │   └── modules/
|   ├── README.md
|   └── roles/
└── collectionB/
    ├── docs/
    ├── galaxy.yml
    ├── plugins/
    │   ├── connection/
    │   └── modules/
    ├── README.md
    └── roles/

You can install all collections in this repository, or install one collection from a specific commit:

# Install all collections in the namespace
ansible-galaxy collection install git+https://github.com/organization/repo_name.git#/namespace/

# Install an individual collection using a specific commit
ansible-galaxy collection install git+https://github.com/organization/repo_name.git#/namespace/collectionA/,7b60ddc245bc416b72d8ea6ed7b799885110f5e5

Reviewing your collection

Review the collection:

Understanding collection versioning

The only way to change a collection is to release a new version. The latest version of a collection (by highest version number) is the version displayed everywhere in Galaxy and Automation Hub. Users can still download older versions.

Follow semantic versioning when setting the version for your collection. In summary:

Read the official Semantic Versioning documentation for details and examples.

Publishing your collection

The last step in distributing your collection is publishing the tarball to Ansible Galaxy, Red Hat Automation Hub, or a privately hosted Automation Hub instance. You can publish your collection in two ways:

Publishing a collection from the command line

To upload the collection tarball from the command line using ansible-galaxy:

ansible-galaxy collection publish path/to/my_namespace-my_collection-1.0.0.tar.gz

This ansible-galaxy command assumes you have retrieved and stored your API token in configuration. See galaxy_specify_token for details.

The ansible-galaxy collection publish command triggers an import process, just as if you uploaded the collection through the Galaxy website. The command waits until the import process completes before reporting the status back. If you want to continue without waiting for the import result, use the --no-wait argument and manually look at the import progress in your My Imports page.

Publishing a collection from the website

See the Galaxy documentation to learn how to publish your collection directly on the Galaxy website.

collections
Learn how to install and use collections.

collections_galaxy_meta
Table of fields used in the galaxy.yml file

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Documenting collections

Documenting modules and plugins

Documenting modules is thoroughly documented in module_documenting. Plugins can be documented the same way as modules, that is with DOCUMENTATION, EXAMPLES, and RETURN blocks.

Documenting roles

To document a role, you have to add a role argument spec by creating a file meta/argument_specs.yml in your role. See role_argument_spec for details. As an example, you can look at the argument specs file of the telekom_mms.icinga_director.ansible_icinga role <telekom_mms.icinga_director.ansible_icinga#role> on GitHub.

Verifying your collection documentation

You can use antsibull-docs to lint your collection documentation. See Linting collection documentation. for details.

Build a docsite with antsibull-docs

You can use antsibull-docs to build a Sphinx-based docsite for your collection:

  1. Create your collection and make sure you can use it with ansible-core by adding it to your COLLECTIONS_PATHS.
  2. Create a directory dest and run antsibull-docs sphinx-init --use-current --dest-dir dest namespace.name, where namespace.name is the name of your collection.
  3. Go into dest and run pip install -r requirements.txt. You might want to create a venv and activate it first to avoid installing this globally.
  4. Then run ./build.sh.
  5. Open build/html/index.html in a browser of your choice.

See antsibull-docs documentation for complete details.

If you want to add additional documentation to your collection next to the plugin, module, and role documentation, see collections_doc_dir.

Migrating Ansible content to a different collection

You might decide to move content from one collection to another; for example, to extract a set of related modules out of community.general or community.network to create a more focused collection. When you migrate content between collections, you must take certain steps to ensure users can follow the transition.

Migrating content

If the collection from which you are going to migrate content is included in the Ansible community package, ensure the target collection satisfies the collections_requirements. After you satisfy the requirements, you can migrate the content as follows:

  1. Copy content from the source (old) collection to the target (new) collection.
  2. Change M(), examples, seealso, extended_documentation_fragments to use actual FQCNs in moved content, old collection, and in other collections that have references to the content.
  3. Move all related issues, pull requests, and wiki pages.
  4. Look through the docs/docsite directory of the ansible-documentation GitHub repository (for example, using the grep command-line utility) to check if there are examples using the moved modules and plugins so that you can update those FQCNs.
  5. Deprecate the module/plugin with removal_version scheduled for the next major version in meta/runtime.yml of the old collection. The deprecation must be released after the copied content has been included in a release of the new collection.
  6. When the next major release of the old collection is prepared:

To implement these changes, you need to create at least three PRs:

  1. Create a PR against the new collection to copy the content.
  2. Deprecate the module/plugin in the old collection.
  3. Later create a PR against the old collection to remove the content according to the schedule.

Adding the content to the new collection

Create a PR in the new collection to:

Removing the content from the old collection

Create a PR against the source collection repository to remove the modules, module_utils, plugins, and docs_fragments related to this migration:

  1. If you are removing an action plugin, remove the corresponding module that contains the documentation.
  2. If you are removing a module, remove any corresponding action plugin that should stay with it.
  3. Remove any entries about removed plugins from meta/runtime.yml. Ensure they are added into the new repo.
  4. Remove sanity ignore lines from tests/sanity/ignore\*.txt
  5. Remove associated integration tests from tests/integrations/targets/ and unit tests from tests/unit/plugins/.
  6. if you are removing from content from community.general or community.network, remove entries from .github/BOTMETA.yml.
  7. Carefully review meta/runtime.yml for any entries you may need to remove or update, in particular deprecated entries.
  8. Update meta/runtime.yml to contain redirects for EVERY PLUGIN, pointing to the new collection name.

Maintainers for the old collection have to make sure that the PR is merged in a way that it does not break user experience and semantic versioning:

  1. A new version containing the merged PR must not be released before the collection the content has been moved to has been released again, with that content contained in it. Otherwise the redirects cannot work and users relying on that content will experience breakage.
  2. Once 1.0.0 of the collection from which the content has been removed has been released, such PRs can only be merged for a new major version (in other words, 2.0.0, 3.0.0, and so on).

Updating BOTMETA.yml

The BOTMETA.yml, for example in community.general collection repository, is the source of truth for:

If the old and/or new collection has ansibullbot, its BOTMETA.yml must be updated correspondingly.

Ansibulbot will know how to redirect existing issues and PRs to the new repo. The build process for docs.ansible.com will know where to find the module docs.

$modules/monitoring/grafana/grafana_plugin.py:
    migrated_to: community.grafana
$modules/monitoring/grafana/grafana_dashboard.py:
    migrated_to: community.grafana
$modules/monitoring/grafana/grafana_datasource.py:
    migrated_to: community.grafana
$plugins/callback/grafana_annotations.py:
    maintainers: $team_grafana
    labels: monitoring grafana
    migrated_to: community.grafana
$plugins/doc_fragments/grafana.py:
    maintainers: $team_grafana
    labels: monitoring grafana
    migrated_to: community.grafana

Example PR

collections
Learn how to install and use collections.

contributing_maintained_collections
Guidelines for contributing to selected collections

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Ansible collection creator path

If you are unfamiliar with Ansible collections, first take a look at the Using Ansible collections guide<collections>.

Ansible collections are a distribution format for Ansible content that can include playbooks, roles, modules, and plugins. A typical collection addresses a set of related use cases. For example, the community.dns collection includes modules and plugins to work with DNS.

You can install collections made by others or share yours with the community through a distribution server such as Ansible Galaxy. Certified collections can be published to the Red Hat Automation Hub, a part of the Red Hat Ansible Automation Platform.

Creating and sharing collections is a great way of contributing to the Ansible project.

The Ansible community package consists of ansible-core, which, among other core components, includes the ansible.builtin collection maintained by the Core team, and a set of collections maintained by the community.

The purpose of this guide is to give you as a (potential) content creator a consistent overview of the Ansible collection creator journey from an idea for the first module/role to having your collection included in the Ansible community package. The Collection development guidelines section<developing_collections> provides references to more detailed aspects of this journey. The overall journey consists of the following milestones:

Examine currently available solutions

If you have an idea for a new role or module/plugin, there is no need to reinvent the wheel if there is already a sufficient solution that solves your automation issue.

Therefore, first examine the currently available content including:

In case the solutions you found are not fully sufficient or have flaws, consider improving them rather than creating your own. Each collection includes information on where to create issues for that collection to propose your enhancement ideas.

If you already have your content written and used in your workflows, you can still consider integrating it to the existing solutions. However, if these options do not apply to your collection ideas, we encourage you to create and share your own.

Create your content

You tried <examine_existing_content> but have not found any sufficient solution for your automation issue.

Use one of the following guides:

Put your content in a collection

You created <create_content> new content.

Now it is time to create a reusable and sharable collection. Use the Developing collections guide<developing_collections> to learn how.

We recommend you to use the collection_template repository as a basis for your collection.

Write good user collection documentation

Your collection README.md file should contain a quick-start installation and usage guides. You can use the community.general collection README file as an example.

If your collection contains modules or plugins, make sure their documentation is comprehensive. Use the Module format and documentation guide<developing_modules_documenting> and Ansible documentation style guide<style_guide> to learn more.

Publish your collection source code

Publish your collection on a platform for software development and version control such as GitHub.

It can be your personal repository or your organization's one. You can also request<request_coll_repo> a repository under the ansible-collections organization.

Make sure your collection contains exhaustive license information. Ansible is an open source project, so we encourage you to license it under one of open source licenses. If you plan to submit your collection for inclusion in the Ansible community package, your collection must satisfy the licensing requirements<coll_licensing_req>.

If you have used the collection_template repository we recommended earlier as a skeleton for your collection, it already contains the GNU GPL v3 license.

Follow a versioning convention

When releasing new versions of your collections, take the following recommended practices into consideration:

Understand and implement testing and CI

This section is applicable to collections containing modules and plugins.

For role testing, see the Ansible Molecule project.

Add tests

Testing your collection ensures that your code works well and integrates with other components such as ansible-core.

Take a look at the following documents:

Implement continuous integration

Now make sure when pull requests are created in your collection repository they are automatically tested using a CI tool such as GitHub Actions or Azure Pipelines.

The collection_template repository contains GitHub Actions templates you can adjust and use to enable the workflows in your repository.

Provide good contributor & maintainer documentation

See the collection_template/README.md as an example.

Publish your collection on distribution servers

To distribute your collection and allow others to conveniently use it, publish your collection on one or more distribution servers. See the Distributing collections guide<distributing_collections> to learn how.

Make your collection a part of Ansible community package

Make your collection satisfy the Ansible community package collections requirements<collections_requirements>. The Steering Committee <steering_responsibilities> selects collections for inclusion. See the collection inclusion criteria and process<steering_inclusion> for details.

Maintain

Maintain your collection. See the Ansible collection maintainer guidelines<maintainers> for details.

Communicate

Engage with the community. Take a look at the Ansible communication guide<communication> to see available communication options.

developing_collections
A set of guidelines about collection development aspects

Resource reporting

Ansible modules often interact with external systems such as clouds, hypervisors, and network controllers to manage resources that are not declared in your inventory. Resource reporting gives you structured visibility into what your automation actually touches, even for resources that never appear in the inventory.

You can add resource reporting to your Ansible collection by creating a lightweight YAML query file that maps module return values to a standardized resource taxonomy. Each entry uses a jq expression to extract resource metadata from module output.

How resource reporting works

Resource reporting uses a query file inside your collection to describe the resources that your modules manage. When a module runs, the query file tells any consuming tool how to extract three pieces of information from the module return values:

The result is machine-readable documentation about what your modules manage. For collection developers, resource reporting is like having excellent return value documentation that tools can consume automatically. For users, it creates a consistent, unified picture of automation activity across different vendors and platforms.

The query file format, resource taxonomy, and jq expression pattern are an open schema. Any automation platform, reporting tool, or custom script can consume the query file to extract structured resource data from module output.

The normalized resource taxonomy

The normalized resource taxonomy maps vendor-specific resource types to standard names. For example, a VMware VM and an AWS EC2 instance are both type virtual_machine. An Azure load balancer and an F5 BIG-IP VIP are both type load_balancer.

The taxonomy organizes resources into categories and device types. When you write a query, set the facts.device_type field to the snake_case value from the tables below.

Compute

Resource device_type value
Virtual machines virtual_machine
Containers (managed) container
Hypervisors hypervisor
Bare metal bare_metal
Serverless functions serverless_function
Auto scaling groups auto_scaling_group

Networking

Resource device_type value
Switches switch
Routers router
Firewalls firewall
Load balancers load_balancer
Virtual private clouds vpc
Subnets subnet
VPNs vpn
Gateways gateway
DNS services dns_service
Wireless access points wireless_access_point
SD-WAN sd_wan

Storage

Resource device_type value
Object storage object_storage
Block storage block_storage
File storage file_storage
Archive storage archive_storage

Database

Resource device_type value
Relational (SQL) database_relational
NoSQL database_nosql
Data warehouse data_warehouse
In-memory or cache database_cache

DevOps and app integration

Resource device_type value
CI/CD platforms ci_cd_platform
Container registries container_registry
Message queues message_queue
API endpoints api_endpoint

If your resource does not fit one of these standard types, open a topic on the Ansible forum to propose a new device type.

Adding resource reporting to a collection

To add resource reporting to your collection, create a query file and write jq expressions that extract resource metadata from your module return values.

Creating the query file

Create the file extensions/audit/event_query.yml in your collection root directory. This is the standard location for embedded query files that ship with your collection.

my_namespace/my_collection/
├── extensions/
│   └── audit/
│       └── event_query.yml
├── plugins/
├── meta/
└── galaxy.yml

Writing the query

The query file maps each module to a jq expression using the Fully Qualified Collection Name (FQCN) as the key. The jq expression runs against the module return values and must output a JSON object with the following fields:

Field Requirement Description
name Required A human-readable display name for the resource, for example a VM name or switch hostname.
canonical_facts Required A dictionary of facts that uniquely identify the resource and deduplicate it across job runs.
facts Optional Metadata for categorization. Must include device_type when present.

The following example shows a complete query file entry:

yaml/event_query_vmware.yaml

Choosing canonical facts

Select fields that are globally unique and stable for the life of the resource.

json/canonical_facts.json

Setting the device type

Map your resource to a standard device_type value from the normalized resource taxonomy <resource_reporting_taxonomy>. You can optionally include a platform field to indicate the underlying platform, for example aws, vmware, or azure.

json/facts_device_type.json

Platform-specific examples

Different platforms require different querying strategies based on how their APIs return data.

VMware (flat structure)

VMware modules return a flat structure where the top-level key defines the node type. Map directly to the returned keys.

yaml/event_query_vmware.yaml

Azure (hierarchical structure)

Azure resources are hierarchical and use verbose resource IDs. Because the full Azure resource ID contains the resource type, you can use a jq regex to dynamically capture and categorize it.

yaml/event_query_azure.yaml

In this example, Microsoft.Compute in the resource ID is captured and lowercased to "compute".

AWS (implied types)

AWS info modules often return a list of resources, so your jq expression must iterate over them. The resource type is implied by the module you query rather than included in the resource ID.

yaml/event_query_aws.yaml

To manage the mapping from AWS module to resource type, define a mapping dictionary in your expression:

json/aws_type_mapping.json

Module matching rules

The query file uses module FQCNs as keys. You can target modules with exact matches or wildcard patterns.

Exact match

Target a specific module.

yaml/event_query_exact_match.yaml

Wildcard match

Target all modules in a collection. Use this carefully because different modules return different data structures.

yaml/event_query_wildcard_match.yaml

Testing your query

Before publishing your collection, verify your jq expression against real module output.

  1. Run a playbook with your module and capture the JSON output. Register the return value or run the module with -vvv to see the full output.

    yaml/test_playbook_resource_reporting.yaml

  2. Save the module output to a file and test your jq expression from the command line.

    cat module_output.json | jq '{
  name: .instance.hw_name,
  canonical_facts: {
    uuid: .instance.hw_uuid
  },
  facts: {
    device_type: "virtual_machine"
  }
}'

  3. Validate that the output JSON contains a valid name string and non-empty canonical_facts.

developing_collections
Learn how to develop Ansible collections

collection_structure
Directories and files included in a collection

Ansible forum
Got questions? Need help? Want to share your ideas? Visit the Ansible forum

Using shared resources in collections

Although developing Ansible modules contained in collections is similar to developing standalone Ansible modules, you use shared resources like documentation fragments and module utilities differently in collections. You can use documentation fragments within and across collections. You can use optional module utilities to support multiple versions of ansible-core in your collection. Collections can also depend on other collections.

Using documentation fragments in collections

To include documentation fragments in your collection:

  1. Create the documentation fragment: plugins/doc_fragments/fragment_name.
  2. Refer to the documentation fragment with its FQCN.
extends_documentation_fragment:
  - kubernetes.core.k8s_name_options
  - kubernetes.core.k8s_auth_options
  - kubernetes.core.k8s_resource_options
  - kubernetes.core.k8s_scale_options

module_docs_fragments covers the basics for documentation fragments. The kubernetes.core collection includes a complete example.

If you use FQCN, you can use documentation fragments from one collection in another collection.

Leveraging optional module utilities in collections

Optional module utilities let you adopt the latest features from the most recent ansible-core release in your collection-based modules without breaking your collection on older Ansible versions. With optional module utilities, you can use the latest features when running against the latest versions, while still providing fallback behaviors when running against older versions.

This implementation, widely used in Python programming, wraps optional imports in conditionals or defensive try/except blocks, and implements fallback behaviors for missing imports. Ansible's module payload builder supports these patterns by treating any module_utils import nested in a block (e.g., if, try) as optional. If the requested import cannot be found during the payload build, it is simply omitted from the target payload and assumed that the importing code will handle its absence at runtime. Missing top-level imports of module_utils packages (imports that are not wrapped in a block statement of any kind) will fail the module payload build, and will not execute on the target.

For example, the ansible.module_utils.common.respawn package is only available in Ansible 2.11 and higher. The following module code would fail during the payload build on Ansible 2.10 or earlier (as the requested Python module does not exist, and is not wrapped in a block to signal to the payload builder that it can be omitted from the module payload):

from ansible.module_utils.common.respawn import respawn_module

By wrapping the import statement in a try block, the payload builder will omit the Python module if it cannot be located, and assume that the Ansible module will handle it at runtime:

try:
    from ansible.module_utils.common.respawn import respawn_module
except ImportError:
    respawn_module = None
...
if needs_respawn:
    if respawn_module:
        respawn_module(target)
    else:
        module.fail_json('respawn is not available in Ansible < 2.11, ensure that foopkg is installed')

The optional import behavior also applies to module_utils imported from collections.

Listing collection dependencies

We recommend that collections work as standalone, independent units, depending only on ansible-core. However, if your collection must depend on features and functionality from another collection, list the other collection or collections under dependencies in your collection's galaxy.yml file. Use the meta/runtime.yml file to set the ansible-core version that your collection depends on. For more information on the galaxy.yml file, see collections_galaxy_meta.

You can use Git repositories for collection dependencies during local development and testing. For example:

dependencies: {'git@github.com:organization/repo_name.git': 'devel'}

Do not use Git repositories as dependencies for published collections. Dependencies for published collections must be other published collections.

collections
Learn how to install and use collections.

contributing_maintained_collections
Guidelines for contributing to selected collections

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Collection structure

A collection is a simple data structure. None of the directories are required unless you have specific content that belongs in one of them. A collection does require a galaxy.yml file at the root level of the collection. This file contains all of the metadata that Galaxy and other tools need in order to package, build and publish the collection.

Collection directories and files

A collection can contain these directories and files:

collection/
├── docs/
├── extensions/
│   └── audit/
│       └── event_query.yml
├── galaxy.yml
├── meta/
│   └── runtime.yml
├── plugins/
│   ├── modules/
│   │   └── module1.py
│   ├── inventory/
│   └── .../
├── README.md
├── roles/
│   ├── role1/
│   ├── role2/
│   └── .../
├── playbooks/
│   ├── files/
│   ├── vars/
│   ├── templates/
│   └── tasks/
└── tests/

* Ansible only accepts .md extensions for the README file and any files in the /docs folder. * See the ansible-collections GitHub Org for examples of collection structure. * Not all directories are currently in use. Those are placeholders for future features.

galaxy.yml

A collection must have a galaxy.yml file that contains the necessary information to build a collection artifact. See collections_galaxy_meta for details.

extensions directory

Use the extensions directory to add content and metadata that extends the functionality of your collection beyond the standard plugins and roles. Each subdirectory groups related files for a specific capability. For example, the extensions/audit/event_query.yml file defines resource reporting queries that map module return values to a standardized resource taxonomy. See resource_reporting for details on adding resource reporting to your collection.

docs directory

Use the docs folder to describe how to use the roles and plugins the collection provides, role requirements, and so on.

For certified collections, Automation Hub displays documents written in markdown in the main docs directory with no subdirectories. This documentation is not published to docs.ansible.com.

For community collections included in the Ansible PyPI package, docs.ansible.com displays documents written in reStructuredText (.rst) in a docsite/rst/ subdirectory. Define the structure of your extra documentation in docs/docsite/extra-docs.yml:

---
sections:
- title: Scenario Guide
  toctree:
    - scenario_guide

The index page of the documentation for your collection displays the title you define in docs/docsite/extra-docs.yml with a link to your extra documentation. For an example, see the community.docker collection repo and the community.docker collection documentation.

You can add extra links to your collection index page and plugin pages with the docs/docsite/links.yml file. This populates the links under Description and Communications headings as well as links at the end of the individual plugin pages. See the collection_template links.yml file for a complete description of the structure and use of this file to create links.

Plugin and module documentation

Keep the specific documentation for plugins and modules embedded as Python docstrings. Use ansible-doc to view documentation for plugins inside a collection:

ansible-doc -t lookup my_namespace.my_collection.lookup1

The ansible-doc command requires the fully qualified collection name (FQCN) to display specific plugin documentation. In this example, my_namespace is the Galaxy namespace and my_collection is the collection name within that namespace.

The Galaxy namespace of an Ansible collection is defined in the galaxy.yml file. It can be different from the GitHub organization or repository name.

plugins directory

Add a 'per plugin type' specific subdirectory here, including module_utils which is usable not only by modules, but by most plugins by using their FQCN. This is a way to distribute modules, lookups, filters, and so on without having to import a role in every play.

Vars plugins in collections are not loaded automatically, and always require being explicitly enabled by using their fully qualified collection name. See enable_vars for details.

Cache plugins in collections may be used for fact caching, but are not supported for inventory plugins.

module_utils

When coding with module_utils in a collection, the Python import statement needs to take into account the FQCN along with the ansible_collections convention. The resulting Python import will look like from ansible_collections.{namespace}.{collection}.plugins.module_utils.{util} import {something}

The following example snippets show a Python and PowerShell module using both default Ansible module_utils and those provided by a collection. In this example the namespace is community, the collection is test_collection. In the Python example the module_util in question is called qradar such that the FQCN is community.test_collection.plugins.module_utils.qradar:

from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.common.text.converters import to_text

from ansible.module_utils.six.moves.urllib.parse import urlencode, quote_plus
from ansible.module_utils.six.moves.urllib.error import HTTPError
from ansible_collections.community.test_collection.plugins.module_utils.qradar import QRadarRequest

argspec = dict(
    name=dict(required=True, type='str'),
    state=dict(choices=['present', 'absent'], required=True),
)

module = AnsibleModule(
    argument_spec=argspec,
    supports_check_mode=True
)

qradar_request = QRadarRequest(
    module,
    headers={"Content-Type": "application/json"},
    not_rest_data_keys=['state']
)

Note that importing something from an __init__.py file requires using the file name:

from ansible_collections.namespace.collection_name.plugins.callback.__init__ import CustomBaseClass

In the PowerShell example the module_util in question is called hyperv such that the FQCN is community.test_collection.plugins.module_utils.hyperv:

#!powershell
#AnsibleRequires -CSharpUtil Ansible.Basic
#AnsibleRequires -PowerShell ansible_collections.community.test_collection.plugins.module_utils.hyperv

$spec = @{
    name = @{ required = $true; type = "str" }
    state = @{ required = $true; choices = @("present", "absent") }
}
$module = [Ansible.Basic.AnsibleModule]::Create($args, $spec)

Invoke-HyperVFunction -Name $module.Params.name

$module.ExitJson()

roles directory

Collection roles are mostly the same as existing roles, but with a couple of limitations:

The directory name of the role is used as the role name. Therefore, the directory name must comply with the above role name rules. The collection import into Galaxy will fail if a role name does not comply with these rules.

You can migrate 'traditional roles' into a collection but they must follow the rules above. You may need to rename roles if they don't conform. You will have to move or link any role-based plugins to the collection specific directories.

For roles imported into Galaxy directly from a GitHub repository, setting the role_name value in the role's metadata overrides the role name used by Galaxy. For collections, that value is ignored. When importing a collection, Galaxy uses the role directory as the name of the role and ignores the role_name metadata value.

playbooks directory

In prior releases, you could reference playbooks in this directory using the full path to the playbook file from the command line. In ansible-core 2.11 and later, you can use the FQCN, namespace.collection.playbook (with or without extension), to reference the playbooks from the command line or from import_playbook. This will keep the playbook in 'collection context', as if you had added collections: [ namespace.collection ] to it.

You can have most of the subdirectories you would expect, such files/, vars/ or templates/ but not roles/ since those are handled already in the collection.

Also, playbooks within a collection follow the same guidelines as any playbooks except for these few adjustments:

To run the plays, users can now use such commands as ansible-playbook --e 'targets=webservers' or ansible-playbook --limit webservers. Either way, the collection owner should document their playbooks and how to use them in the docs/ folder or README file.

tests directory

Ansible Collections are tested much like Ansible itself, by using the ansible-test utility which is released as part of Ansible, version 2.9.0 and newer. Because Ansible Collections are tested using the same tooling as Ansible itself, by using the ansible-test, all Ansible developer documentation for testing is applicable for authoring Collections Tests with one key concept to keep in mind.

See testing_collections for specific information on how to test collections with ansible-test.

When reading the developing_testing documentation, there will be content that applies to running Ansible from source code through a Git clone, which is typical of an Ansible developer. However, it is not always typical for an Ansible Collection author to be running Ansible from source but instead from a stable release, and to create Collections it is not necessary to run Ansible from source. Therefore, when references of dealing with ansible-test binary paths, command completion, or environment variables are presented throughout the developing_testing documentation; keep in mind that it is not needed for Ansible Collection Testing because the act of installing the stable release of Ansible containing ansible-test is expected to setup those things for you.

meta directory

runtime.yml

A collection can store some additional metadata in a runtime.yml file in the collection's meta directory. The runtime.yml file supports the top level keys:

execution-environment.yml

If your collection has requirements, you can specify them in the execution-environment.yml file in the meta directory. This ensures users do not need to add these requirements manually when building Execution Environments<getting_started_ee_index> containing your collection. See the collection-level metadata guide for details.

distributing_collections
Learn how to package and publish your collection

contributing_maintained_collections
Guidelines for contributing to selected collections

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Testing collections

Testing your collection ensures that your code works well and integrates well with the rest of the Ansible ecosystem. Your collection should pass the sanity tests for Ansible code. You should also add unit tests to cover the code in your collection and integration tests to cover the interactions between your collection and ansible-core.

Testing tools

The main tool for testing collections is ansible-test, Ansible's testing tool described in developing_testing and provided by both the ansible and ansible-core packages.

Use ansible-test tool from your collection directory, which must include ansible_collections in the path, for example collections/ansible_collections/community/general for the community.general collection. See testing_collections_guide and testing_running_locally for testing guidelines.

You can run several sanity tests, as well as run unit and integration tests for plugins using ansible-test. When you test collections, test against the ansible-core version(s) you are targeting.

You must always execute ansible-test from the root directory of a collection. You can run ansible-test in Docker containers without installing any special requirements. The Ansible team uses this approach in Azure Pipelines both in the ansible/ansible GitHub repository and in the large community collections such as community.general and community.network to automatically run the tests when pull requests are submitted.

Many collections which do not require running tests on different OS distributions use GitHub Actions as their continuous integration (CI) platform. The collection_template repository contains GitHub Actions workflow templates that collection developers are free to use to easily set up CI in their collection repositories.

The examples below demonstrate running tests in Docker containers.

Sanity tests

To run all sanity tests:

ansible-test sanity --docker default -v

See testing_sanity for more information. See the full list of sanity tests <all_sanity_tests> for details on the sanity tests and how to fix identified issues.

Adding unit tests

You must place unit tests in the appropriate tests/unit/plugins/ directory. For example, you would place tests for plugins/module_utils/foo/bar.py in tests/unit/plugins/module_utils/foo/test_bar.py or tests/unit/plugins/module_utils/foo/bar/test_bar.py. For examples, see the unit tests in community.general.

To run all unit tests for all supported Python versions:

ansible-test units --docker default -v

To run all unit tests only for a specific Python version:

ansible-test units --docker default -v --python 3.13

To run only a specific unit test:

ansible-test units --docker default -v --python 3.13 tests/unit/plugins/module_utils/foo/test_bar.py

You can specify Python requirements in the tests/unit/requirements.txt file. See testing_units for more information, especially on fixture files.

Adding integration tests

You must place integration tests in the appropriate tests/integration/targets/ directory. For module integration tests, you can use the module name alone. For example, you would place integration tests for plugins/modules/foo.py in a directory called tests/integration/targets/foo/. For non-module plugin integration tests, you must add the plugin type to the directory name. For example, you would place integration tests for plugins/connections/bar.py in a directory called tests/integration/targets/connection_bar/. For lookup plugins, the directory must be called lookup_foo, for inventory plugins, inventory_foo, and so on.

You can write two different kinds of integration tests:

For examples, see the integration tests in community.general. See also testing_integration for more details.

Since integration tests can install requirements, and set-up, start and stop services, we recommended running them in docker containers or otherwise restricted environments whenever possible. By default, ansible-test supports Docker images for several operating systems. See the list of supported docker images for all options. Use the default image mainly for platform-independent integration tests, such as those for cloud modules. The following examples use the fedora35 image.

To execute all integration tests for a collection:

ansible-test integration --docker fedora -v

If you want more detailed output, run the command with -vvv instead of -v. Alternatively, specify --retry-on-error to automatically re-run failed tests with higher verbosity levels.

To execute only the integration tests in a specific directory:

ansible-test integration --docker fedora -v connection_bar

You can specify multiple target names. Each target name is the name of a directory in tests/integration/targets/.

Collection testing with ansible-test

ansible-test has been the de facto collection testing tool since ansible 2.9. However, ansible-test is no longer receiving feature enhancements for new collection testing requirements. We ask collection maintainers and community members to join the discussion to improve collection testing tools on the Ansible Forum. Find related discussion topics via the collection-test-strat tag.

developing_testing
More resources on testing Ansible

contributing_maintained_collections
Guidelines for contributing to selected collections

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Developing `ansible-core`

Although ansible-core (the code hosted in the ansible/ansible repository on GitHub) includes a few plugins that can be swapped out by the playbook directives or configuration, much of the code there is not modular. The documents here give insight into how the parts of ansible-core work together.

core_branches_and_tags developing_program_flow_modules

developing_api
Learn about the Python API for task execution

developing_plugins
Learn about developing plugins

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Developing dynamic inventory

Ansible can pull inventory information from dynamic sources, including cloud sources, by using the supplied inventory plugins <inventory_plugins>. For details about how to pull inventory information, see dynamic_inventory. If the source you want is not currently covered by existing plugins, you can create your own inventory plugin as with any other plugin type.

In previous versions, you had to create a script or program that could output JSON in the correct format when invoked with the proper arguments. You can still use and write inventory scripts, as we ensured backwards compatibility through the script inventory plugin <script_inventory> and there is no restriction on the programming language used. If you choose to write a script, however, you will need to implement some features yourself such as caching, configuration management, dynamic variable and group composition, and so on. If you use inventory plugins <inventory_plugins> instead, you can use the Ansible codebase and add these common features automatically.

Topics

Inventory sources

Inventory sources are the input strings that inventory plugins work with. An inventory source can be a path to a file or to a script, or it can be raw data that the plugin can interpret.

The table below shows some examples of inventory plugins and the source types that you can pass to them with -i on the command line.

Plugin

Source

host list <host_list_inventory>

A comma-separated list of hosts

yaml <yaml_inventory>

Path to a YAML format data file

constructed <constructed_inventory>

Path to a YAML configuration file

ini <ini_inventory>

Path to an INI formatted data file

virtualbox <virtualbox_inventory>

Path to a YAML configuration file

script plugin <script_inventory>

Path to an executable that outputs JSON

Inventory plugins

Like most plugin types (except modules), inventory plugins must be developed in Python. They execute on the control node and should therefore adhere to the control_node_requirements.

Most of the documentation in developing_plugins also applies here. You should read that document first for a general understanding and then come back to this document for specifics on inventory plugins.

Normally, inventory plugins are executed at the start of a run, and before the playbooks, plays, or roles are loaded. However, you can use the meta: refresh_inventory task to clear the current inventory and execute the inventory plugins again, and this task will generate a new inventory.

If you use the persistent cache, inventory plugins can also use the configured cache plugin to store and retrieve data. Caching inventory avoids making repeated and costly external calls.

Developing an inventory plugin

The first thing you want to do is use the base class:

from ansible.plugins.inventory import BaseInventoryPlugin

class InventoryModule(BaseInventoryPlugin):

    NAME = 'myplugin'  # used internally by Ansible, it should match the file name but not required

If the inventory plugin is in a collection, the NAME should be in the 'namespace.collection_name.myplugin' format. The base class has a couple of methods that each plugin should implement and a few helpers for parsing the inventory source and updating the inventory.

After you have the basic plugin working, you can incorporate other features by adding more base classes:

from ansible.plugins.inventory import BaseInventoryPlugin, Constructable, Cacheable

class InventoryModule(BaseInventoryPlugin, Constructable, Cacheable):

    NAME = 'myplugin'

For the bulk of the work in a plugin, we mostly want to deal with 2 methods verify_file and parse.

verify_file method

Ansible uses this method to quickly determine if the inventory source is usable by the plugin. The determination does not need to be 100% accurate, as there might be an overlap in what plugins can handle and by default Ansible will try the enabled plugins as per their sequence.

def verify_file(self, path):
    ''' return true/false if this is possibly a valid file for this plugin to consume '''
    valid = False
    if super(InventoryModule, self).verify_file(path):
        # base class verifies that file exists and is readable by current user
        if path.endswith(('virtualbox.yaml', 'virtualbox.yml', 'vbox.yaml', 'vbox.yml')):
            valid = True
    return valid

In the above example, from the virtualbox inventory plugin <virtualbox_inventory>, we screen for specific file name patterns to avoid attempting to consume any valid YAML file. You can add any type of condition here, but the most common one is 'extension matching'. If you implement extension matching for YAML configuration files, the path suffix <plugin_name>.<yml|yaml> should be accepted. All valid extensions should be documented in the plugin description.

The following is another example that does not use a 'file' but the inventory source string itself, from the host list <host_list_inventory> plugin:

def verify_file(self, path):
    ''' don't call base class as we don't expect a path, but a host list '''
    host_list = path
    valid = False
    b_path = to_bytes(host_list, errors='surrogate_or_strict')
    if not os.path.exists(b_path) and ',' in host_list:
        # the path does NOT exist and there is a comma to indicate this is a 'host list'
        valid = True
    return valid

This method is just to expedite the inventory process and avoid unnecessary parsing of sources that are easy to filter out before causing a parse error.

parse method

This method does the bulk of the work in the plugin. It takes the following parameters:

The base class does some minimal assignment for reuse in other methods.

def parse(self, inventory, loader, path, cache=True):

     self.loader = loader
     self.inventory = inventory
     self.templar = Templar(loader=loader)

It is up to the plugin now to parse the provided inventory source and translate it into Ansible inventory. To facilitate this, the example below uses a few helper functions:

NAME = 'myplugin'

def parse(self, inventory, loader, path, cache=True):

     # call base method to ensure properties are available for use with other helper methods
     super(InventoryModule, self).parse(inventory, loader, path, cache)

     # this method will parse 'common format' inventory sources and
     # update any options declared in DOCUMENTATION as needed
     config = self._read_config_data(path)

     # if NOT using _read_config_data you should call set_options directly,
     # to process any defined configuration for this plugin,
     # if you don't define any options you can skip
     #self.set_options()

     # example consuming options from inventory source
     mysession = apilib.session(user=self.get_option('api_user'),
                                password=self.get_option('api_pass'),
                                server=self.get_option('api_server')
     )


     # make requests to get data to feed into inventory
     mydata = mysession.getitall()

     #parse data and create inventory objects:
     for colo in mydata:
         for server in mydata[colo]['servers']:
             self.inventory.add_host(server['name'])
             self.inventory.set_variable(server['name'], 'ansible_host', server['external_ip'])

The specifics will vary depending on API and structure returned. Remember that if you get an inventory source error or any other issue, you should raise AnsibleParserError to let Ansible know that the source was invalid or the process failed.

For examples on how to implement an inventory plugin, see the source code here: lib/ansible/plugins/inventory.

inventory object

The inventory object passed to parse has helpful methods for populating inventory.

add_group adds a group to inventory if it doesn't already exist. It takes the group name as the only positional argument.

add_child adds a group or host that exists in inventory to a parent group in inventory. It takes two positional arguments, the name of the parent group and the name of the child group or host.

add_host adds a host to inventory if it doesn't already exist, optionally to a specific group. It takes the host name as the first argument and accepts two optional keyword arguments, group and port. group is the name of a group in inventory, and port is an integer.

set_variable adds a variable to a group or host in inventory. It takes three positional arguments: the name of the group or host, the name of the variable, and the value of the variable.

To create groups and variables using Jinja2 expressions, see the section on implementing constructed features below.

To see other inventory object methods, see the source code here: lib/ansible/inventory/data.py.

inventory cache

To cache the inventory, extend the inventory plugin documentation with the inventory_cache documentation fragment and use the Cacheable base class.

extends_documentation_fragment:
  - inventory_cache

class InventoryModule(BaseInventoryPlugin, Constructable, Cacheable):

    NAME = 'myplugin'

Next, load the cache plugin specified by the user to read from and update the cache. If your inventory plugin uses YAML-based configuration files and the _read_config_data method, the cache plugin is loaded within that method. If your inventory plugin does not use _read_config_data, you must load the cache explicitly with load_cache_plugin.

NAME = 'myplugin'

def parse(self, inventory, loader, path, cache=True):
    super(InventoryModule, self).parse(inventory, loader, path)

    self.load_cache_plugin()

Before using the cache plugin, you must retrieve a unique cache key by using the get_cache_key method. This task needs to be done by all inventory modules using the cache, so that you don't use/overwrite other parts of the cache.

def parse(self, inventory, loader, path, cache=True):
    super(InventoryModule, self).parse(inventory, loader, path)

    self.load_cache_plugin()
    cache_key = self.get_cache_key(path)

The get_cache_key is a helper method that retrieves a unique key for the plugin name and the path parameter. Inventory plugins that generate multiple unique cache entries should not use this helper method because it is designed to get one key per inventory source. In general we recommend that plugin developers refactor to avoid multiple cache entries for inventory sources.

Now that you've enabled caching, loaded the correct plugin, and retrieved a unique cache key, you can set up the flow of data between the cache and your inventory using the cache parameter of the parse method. This value comes from the inventory manager and indicates whether the inventory is being refreshed (such as by the --flush-cache or the meta task refresh_inventory). Although the cache shouldn't be used to populate the inventory when being refreshed, the cache should be updated with the new inventory if the user has enabled caching. You can use self._cache like a dictionary. The following pattern allows refreshing the inventory to work in conjunction with caching.

def parse(self, inventory, loader, path, cache=True):
    super(InventoryModule, self).parse(inventory, loader, path)

    self.load_cache_plugin()
    cache_key = self.get_cache_key(path)

    # cache may be True or False at this point to indicate if the inventory is being refreshed
    # get the user's cache option too to see if we should save the cache if it is changing
    user_cache_setting = self.get_option('cache')

    # read if the user has caching enabled and the cache isn't being refreshed
    attempt_to_read_cache = user_cache_setting and cache
    # update if the user has caching enabled and the cache is being refreshed; update this value to True if the cache has expired below
    cache_needs_update = user_cache_setting and not cache

    # attempt to read the cache if inventory isn't being refreshed and the user has caching enabled
    if attempt_to_read_cache:
        try:
            results = self._cache[cache_key]
        except KeyError:
            # This occurs if the cache_key is not in the cache or if the cache_key expired, so the cache needs to be updated
            cache_needs_update = True
    if not attempt_to_read_cache or cache_needs_update:
        # parse the provided inventory source
        results = self.get_inventory()
    if cache_needs_update:
        self._cache[cache_key] = results

    # submit the parsed data to the inventory object (add_host, set_variable, etc)
    self.populate(results)

After the parse method is complete, the contents of self._cache is used to set the cache plugin if the contents of the cache have changed.

You have three other cache methods available:

constructed features

Inventory plugins can create host variables and groups from Jinja2 expressions and variables by using features from the constructed inventory plugin. To do this, use the Constructable base class and extend the inventory plugin's documentation with the constructed documentation fragment.

extends_documentation_fragment:
  - constructed

class InventoryModule(BaseInventoryPlugin, Constructable):

    NAME = 'ns.coll.myplugin'

There are three main options in the constructed documentation fragment:

compose creates variables using Jinja2 expressions. This is implemented by calling the _set_composite_vars method. keyed_groups creates groups of hosts based on variable values. This is implemented by calling the _add_host_to_keyed_groups method. groups creates groups based on Jinja2 conditionals. This is implemented by calling the _add_host_to_composed_groups method.

Each method should be called for every host added to inventory. Three positional arguments are required: the constructed option, a dictionary of variables, and a host name. Calling the method _set_composite_vars first will allow keyed_groups and groups to use the composed variables.

By default, undefined variables are ignored. This is permitted by default for compose so you can make the variable definitions depend on variables that will be populated later in a play from other sources. For groups, it allows using variables that are not always present without having to use the default filter. To support configuring undefined variables to be an error, pass the constructed option strict to each of the methods as a keyword argument.

keyed_groups and groups use any variables already associated with the host (for example, from an earlier inventory source). _add_host_to_keyed_groups and add_host_to_composed_groups can turn this off by passing the keyword argument fetch_hostvars.

Here is an example using all three methods:

def add_host(self, hostname, host_vars):
    self.inventory.add_host(hostname, group='all')

    for var_name, var_value in host_vars.items():
        self.inventory.set_variable(hostname, var_name, var_value)

    strict = self.get_option('strict')

    # Add variables created by the user's Jinja2 expressions to the host
    self._set_composite_vars(self.get_option('compose'), host_vars, hostname, strict=True)

    # Create user-defined groups using variables and Jinja2 conditionals
    self._add_host_to_composed_groups(self.get_option('groups'), host_vars, hostname, strict=strict)
    self._add_host_to_keyed_groups(self.get_option('keyed_groups'), host_vars, hostname, strict=strict)

By default, group names created with _add_host_to_composed_groups() and _add_host_to_keyed_groups() are valid Python identifiers. Invalid characters are replaced with an underscore _. A plugin can change the sanitization used for the constructed features by setting self._sanitize_group_name to a new function. The core engine also does sanitization, so if the custom function is less strict it should be used in conjunction with the configuration setting TRANSFORM_INVALID_GROUP_CHARS.

from ansible.inventory.group import to_safe_group_name

class InventoryModule(BaseInventoryPlugin, Constructable):

    NAME = 'ns.coll.myplugin'

    @staticmethod
    def custom_sanitizer(name):
        return to_safe_group_name(name, replacer='')

    def parse(self, inventory, loader, path, cache=True):
        super(InventoryModule, self).parse(inventory, loader, path)

        self._sanitize_group_name = custom_sanitizer

Common format for inventory sources

To simplify development, most plugins use a standard YAML-based configuration file as the inventory source. The file has only one required field plugin, which should contain the name of the plugin that is expected to consume the file. Depending on other common features used, you might need other fields, and you can add custom options in each plugin as required. For example, if you use the integrated caching, cache_plugin, cache_timeout and other cache-related fields could be present.

The 'auto' plugin

From Ansible 2.5 onwards, we include the auto inventory plugin <auto_inventory> and enable it by default. If the plugin field in your standard configuration file matches the name of your inventory plugin, the auto inventory plugin will load your plugin. The 'auto' plugin makes it easier to use your plugin without having to update configurations.

Inventory scripts

Even though we now have inventory plugins, we still support inventory scripts, not only for backwards compatibility but also to allow users to use other programming languages.

Inventory script conventions

Inventory scripts must accept the --list and --host <hostname> arguments. Although other arguments are allowed, Ansible will not use them. Such arguments might still be useful for executing the scripts directly.

When the script is called with the single argument --list, the script must output to stdout a JSON object that contains all the groups to be managed. Each group's value should be either an object containing a list of each host, any child groups, and potential group variables, or simply a list of hosts:

{
    "group001": {
        "hosts": ["host001", "host002"],
        "vars": {
            "var1": true
        },
        "children": ["group002"]
    },
    "group002": {
        "hosts": ["host003","host004"],
        "vars": {
            "var2": 500
        },
        "children":[]
    }

}

If any of the elements of a group are empty, they may be omitted from the output.

When called with the argument --host <hostname> (where <hostname> is a host from above), the script must print a JSON object, either empty or containing variables to make them available to templates and playbooks. For example:

{
    "VAR001": "VALUE",
    "VAR002": "VALUE"
}

Printing variables is optional. If the script does not print variables, it should print an empty JSON object.

Tuning the external inventory script

1.3

The stock inventory script system mentioned above works for all versions of Ansible, but calling --host for every host can be rather inefficient, especially if it involves API calls to a remote subsystem.

To avoid this inefficiency, if the inventory script returns a top-level element called "_meta", it is possible to return all the host variables in a single script execution. When this meta element contains a value for "hostvars", the inventory script will not be invoked with --host for each host. This behavior results in a significant performance increase for large numbers of hosts.

The data to be added to the top-level JSON object looks like this:

{

    # results of inventory script as above go here
    # ...

    "_meta": {
        "hostvars": {
            "host001": {
                "var001" : "value"
            },
            "host002": {
                "var002": "value"
            }
        }
    }
}

To satisfy the requirements of using _meta, to prevent ansible from calling your inventory with --host you must at least populate _meta with an empty hostvars object. For example:

{

    # results of inventory script as above go here
    # ...

    "_meta": {
        "hostvars": {}
    }
}

If you intend to replace an existing static inventory file with an inventory script, it must return a JSON object which contains an 'all' group that includes every host in the inventory as a member and every group in the inventory as a child. It should also include an 'ungrouped' group which contains all hosts which are not members of any other group. A skeleton example of this JSON object is:

{
    "_meta": {
      "hostvars": {}
    },
    "all": {
      "children": [
        "ungrouped"
      ]
    },
    "ungrouped": {
      "children": [
      ]
    }
}

An easy way to see how this should look is using ansible-inventory, which also supports --list and --host parameters like an inventory script would.

developing_api
Python API to Playbooks and Ad Hoc Task Execution

developing_modules_general
Get started with developing a module

developing_plugins
How to develop plugins

AWX
REST API endpoint and GUI for Ansible, syncs with dynamic inventory

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Adding modules and plugins locally<span id="using_local_modules_and_plugins"></span>

You can extend Ansible by adding custom modules or plugins. You can create them from scratch or copy existing ones for local use. You can store a local module or plugin on your Ansible control node and share it with your team or organization. You can also share plugins and modules by including them in a collection, then publishing the collection on Ansible Galaxy.

If you are using a local module or plugin but Ansible cannot find it, this page is all you need.

If you want to create a plugin or a module, see developing_plugins, developing_modules_general and developing_collections.

Extending Ansible with local modules and plugins offers shortcuts such as:

Modules and plugins: what is the difference?

If you are looking to add functionality to Ansible, you might wonder whether you need a module or a plugin. Here is a quick overview to help you understand what you need:

Adding modules and plugins in collections

You can add modules and plugins by creating a collection <developing_collections>. With a collection, you can use custom modules and plugins in any playbook or role. You can share your collection easily at any time through Ansible Galaxy.

The rest of this page describes other methods of using local, standalone modules or plugins.

Adding a module or plugin outside of a collection

You can configure Ansible to load standalone local modules or plugins in specific locations and make them available to all playbooks and roles (using configured paths). Alternatively, you can make a non-collection local module or plugin available only to certain playbooks or roles (with adjacent paths).

Adding standalone local modules for all playbooks and roles

To load standalone local modules automatically and make them available to all playbooks and roles, use the DEFAULT_MODULE_PATH configuration setting or the ANSIBLE_LIBRARY environment variable. The configuration setting and environment variable take a colon-separated list, similar to $PATH. You have two options:

To view your current configuration settings for modules:

ansible-config dump |grep DEFAULT_MODULE_PATH

After you save your module file in one of these locations, Ansible loads it and you can use it in any local task, playbook, or role.

To confirm that my_local_module is available:

This applies to all plugin types but requires specific configuration and/or adjacent directories for each plugin type, see below.

The ansible-doc command can parse module documentation from modules written in Python or an adjacent YAML file. If you have a module written in a programming language other than Python, you should write the documentation in a Python or YAML file adjacent to the module file. adjacent_yaml_doc

Adding standalone local modules for selected playbooks or a single role

Ansible automatically loads all executable files from certain directories adjacent to your playbook or role as modules. Standalone modules in these locations are available only to the specific playbook, playbooks, or role in the parent directory.

This applies to all plugin types but requires specific configuration and/or adjacent directories for each plugin type, see below.

Roles contained in collections cannot contain any modules or other plugins. All plugins in a collection must live in the collection plugins directory tree. All plugins in that tree are accessible to all roles in the collection. If you are developing new modules, we recommend distributing them in collections <developing_collections>, not in roles.

Adding a non-module plugin locally outside of a collection

You can configure Ansible to load standalone local plugins in a specified location or locations and make them available to all playbooks and roles. Alternatively, you can make a standalone local plugin available only to specific playbooks or roles.

Although modules are plugins, the naming patterns for directory names and environment variables that apply to other plugin types do not apply to modules. See local_modules.

Adding local non-module plugins for all playbooks and roles

To load standalone local plugins automatically and make them available to all playbooks and roles, use the configuration setting or environment variable for the type of plugin you are adding. These configuration settings and environment variables take a colon-separated list, similar to $PATH. You have two options:

To view your current configuration settings for non-module plugins:

ansible-config dump |grep plugin_type_PATH

After your plugin file is added to one of these locations, Ansible loads it and you can use it in any local module, task, playbook, or role. For more information on environment variables and configuration settings, see ansible_configuration_settings.

To confirm that plugins/plugin_type/my_local_plugin is available:

The ansible-doc command works for most plugin types, but not for action, filter, or test plugins. See ansible-doc for more details.

Adding standalone local plugins for selected playbooks or a single role

Ansible automatically loads all plugins from certain directories adjacent to your playbook or role, loading each type of plugin separately from a directory named for the type of plugin. Standalone plugins in these locations are available only to the specific playbook, playbooks, or role in the parent directory.

Roles contained in collections cannot contain any plugins. All plugins in a collection must live in the collection plugins directory tree. All plugins in that tree are accessible to all roles in the collection. If you are developing new plugins, we recommend distributing them in collections <developing_collections>, not in roles.

Some plugin types are needed early during Ansible execution, such as callbacks, inventory, and cache. These plugin types cannot be loaded dynamically and must exist in configured paths or be referenced by FQCN in configuration.

Using ansible.legacy to access custom versions of an ansible.builtin module

If you need to override one of the ansible.builtin modules and are using FQCN, you need to use ansible.legacy as part of the fully-qualified collection name (FQCN). For example, if you had your own copy module, you would access it as ansible.legacy.copy. See using_ansible_legacy for details on how to use custom modules with roles within a collection.

Using and developing module utilities

Ansible provides a number of module utilities, or snippets of shared code, that provide helper functions you can use when developing your own modules. The basic.py module utility provides the main entry point for accessing the Ansible library, and all Python Ansible modules must import something from ansible.module_utils. A common option is to import AnsibleModule:

from ansible.module_utils.basic import AnsibleModule

The ansible.module_utils namespace is not a plain Python package: it is constructed dynamically for each task invocation, by extracting imports and resolving those matching the namespace against a search path <ansible_search_path> derived from the active configuration.

Modules may only import from ansible.module_utils.*. Importing from other parts of the ansible namespace (for example, ansible.errors or ansible.parsing.dataloader) is not supported. The ansible package is not guaranteed to be present on the remote target where modules execute, so such imports may fail silently or unpredictably.

To reduce the maintenance burden in a collection or in local modules, you can extract duplicated code into one or more module utilities and import them into your modules. For example, if you have your own custom modules that import a my_shared_code library, you can place that into a ./module_utils/my_shared_code.py file like this:

from ansible.module_utils.my_shared_code import MySharedCodeClient

When you run ansible-playbook, Ansible will merge any files in your local module_utils directories into the ansible.module_utils namespace in the order defined by the Ansible search path <ansible_search_path>.

Naming and finding module utilities

You can generally tell what a module utility does from its name and/or its location. Generic utilities (shared code used by many different kinds of modules) live in the main ansible/ansible codebase, in the common subdirectory or in the root directory of lib/ansible/module_utils. Utilities used by a particular set of modules generally live in the same collection as those modules. For example:

Following this pattern with your own module utilities makes everything easy to find and use.

Standard module utilities

Ansible ships with an extensive library of module_utils files. You can find the module utility source code in the lib/ansible/module_utils directory under your main Ansible path. We describe the most widely used utilities below. For more details on any specific module utility, please see the source code for module_utils.

LICENSING REQUIREMENTS Ansible enforces the following licensing requirements:

Several commonly-used utilities migrated to collections in Ansible 2.10, including:

For a list of migrated content with destination collections, see the runtime.yml file.

Should you develop a module?<span id="developing_modules"></span>

Developing Ansible modules is easy, but often it is not necessary. Before you start writing a new module, ask:

  1. Does a similar module already exist?

An existing module may cover the functionality you want. Ansible collections include thousands of modules. Search our list of included collections <list_of_collections> or Ansible Galaxy to see if an existing module does what you need.

  1. Should you use or develop an action plugin instead of a module?

An action plugin may be the best way to get the functionality you want. Action plugins run on the control node instead of on the managed node, and their functionality is available to all modules. For more information about developing plugins, read the developing plugins page <developing_plugins>.

  1. Should you use a role instead of a module?

A combination of existing modules may cover the functionality you want. You can write a role for this type of use case. Check out the roles documentation<playbooks_reuse_roles>.

  1. Should you create a collection instead of a single module?

The functionality you want may be too large for a single module. If you want to connect Ansible to a new cloud provider, database, or network platform, you may need to develop a new collection<developing_modules_in_groups>.

If your use case isn't covered by an existing module, an action plugin, or a role, and you don't need to create multiple modules, then you're ready to start developing a new module. Choose from the topics below for next steps:

list_of_collections
Browse existing collections, modules, and plugins

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Conventions, tips, and pitfalls<span id="developing_modules_best_practices"></span>

Topics

As you design and develop modules, follow these basic conventions and tips for clean, usable code:

Scoping your module(s)

Especially if you want to contribute your module(s) to an existing Ansible Collection, make sure each module includes enough logic and functionality, but not too much. If these guidelines seem confusing, consider whether you really need to write a module <module_dev_should_you> at all.

Designing module interfaces

General guidelines & tips

Functions and Methods

Python tips

if __name__ == '__main__':
    main()

Importing and using shared code

import traceback

from ansible.module_utils.basic import missing_required_lib

LIB_IMP_ERR = None
try:
    import foo
    HAS_LIB = True
except:
    HAS_LIB = False
    LIB_IMP_ERR = traceback.format_exc()

Then in main(), just after the argspec, do

if not HAS_LIB:
    module.fail_json(msg=missing_required_lib("foo"),
                     exception=LIB_IMP_ERR)

And document the dependency in the requirements section of your module's documentation_block.

Handling module failures

When your module fails, help users understand what went wrong. If you are using the AnsibleModule common Python code, the failed element will be included for you automatically when you call fail_json. For polite module failure behavior:

Handling exceptions (bugs) gracefully

Creating correct and informative module output

Modules must output valid JSON only. Follow these guidelines for creating correct, useful module output:

If a module returns stderr or otherwise fails to produce valid JSON, the actual output will still be shown in Ansible, but the command will not succeed.

Following Ansible conventions

Ansible conventions offer a predictable user interface across all modules, playbooks, and roles. To follow Ansible conventions in your module development:

Module Security

Contributing your module to an existing Ansible collection<span id="developing_modules_checklist"></span>

If you want to contribute a module to an existing collection, you must meet the community's objective and subjective requirements. Please read the details below, and also review our tips for module development <developing_modules_best_practices>.

Modules accepted into certain collections are included in every Ansible release on PyPI. However, contributing to one of these collections is not the only way to distribute a module - you can create your own collection <developing_collections>, embed modules in roles on Galaxy or simply share copies of your module code for local use <developing_locally>.

Contributing modules: objective requirements

To contribute a module to most Ansible collections, you must:

Additional requirements may apply for certain collections. Review the individual collection repositories for more information.

Please make sure your module meets these requirements before you submit your PR/proposal. If you have questions, visit the Ansible communication guide<communication> for information on how to reach out to the community.

Contributing to Ansible: subjective requirements

If your module meets these objective requirements, collection maintainers will review your code to see if they think it is clear, concise, secure, and maintainable. They will consider whether your module provides a good user experience, helpful error messages, reasonable defaults, and more. This process is subjective, with no exact standards for acceptance. For the best chance of getting your module accepted, follow our tips for module development <developing_modules_best_practices>.

Other checklists

Module format and documentation<span id="developing_modules_documenting"></span>

In most cases if you want to contribute your module to an Ansible collection, you should write your module in Python and follow the standard format described below. If you are writing a Windows module, you should follow the Windows guidelines <developing_modules_general_windows>.

Before you open a pull request, in addition to following these guidelines, please also review and adhere to the practices outlined in the following sections:

Every Ansible module written in Python must begin with seven standard sections in a particular order, followed by the code. The sections in order are:

If you are curious why imports are not located at the top of the file, see the python_imports section.

If you see any discrepancies in older Ansible modules, please open a pull request with modifications that satisfy these guidelines.

Non-Python modules documentation

For modules written in languages other than Python, there are two approaches to handling documentation:

Python shebang

  1. Begin your Ansible module with the #!/usr/bin/python shebang so that ansible_python_interpreter works.
  1. Do not add an encoding comment such as # -*- coding: utf-8 -*-. It was required for Python 2 but is unnecessary in Python 3, which treats source files as UTF-8 by default. Since ansible-core 2.17 dropped Python 2 support, existing encoding comments can be removed.
#!/usr/bin/python

# Copyright: Contributors to the Ansible project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)

# Copyright: Contributors to the Ansible project

DOCUMENTATION block

Before committing your module documentation, please test it at the command line and as HTML <dev_testing_module_documentation>.

After the shebang, the copyright line, and the license section comes the DOCUMENTATION block. Ansible's online module documentation is generated from the DOCUMENTATION blocks in the source code of each module.

The DOCUMENTATION block must be valid YAML. To make it easier:

When writing module documentation, take the following statements into consideration:

Each documentation field is described below.

Documentation fields

module

short_description

description

version_added

author

deprecated

options

option-name

  • Name it as a declarative operation (not CRUD) that focuses on the final state, for example online:, rather than is_online:.
  • Make the name consistent with the rest of the module, as well as other modules in the same category.
  • When in doubt, look for other modules to find option names that are used for the same purpose, we like to offer consistency to our users.
  • There is no explicit field option-name. This entry is about the key of the option in the options dictionary.

description

  • Detailed explanation of what this option does. Write it in full sentences that start with a capital letter and end with a period.
  • The first entry is a description of the option itself; subsequent entries detail its use, dependencies, or format of possible values.
  • Do NOT list the possible values (that's what the choices: field is for, though it should explain what the values do if they are not obvious).
  • If an option is only sometimes required, describe the conditions. For example, "Required when O(state=present)."
  • Mutually exclusive options MUST be documented as the final sentence on each of the options.
  • You can use Ansible markup <ansible_markup> in this field.

required

  • Only needed if true.
  • If missing, we assume the option is not required.

default

  • If required is either false or missing, default may be specified (assumed null if missing).
  • Ensure that the default value in the docs matches the default value in the code.
  • The default field MUST NOT be listed as part of the description, unless it requires additional information or conditions.
  • If the option is a boolean value, you can use any of the boolean values recognized by Ansible (such as true/false or yes/no). Document booleans as true/false for consistency and compatibility with ansible-lint.

choices

  • List of option values.
  • Do NOT use it if empty.

type

  • Specifies the data type that option accepts, MUST match the argument_spec dictionary.
  • If an argument is type='bool', set it to type: bool and do NOT specify choices.
  • If an argument is type='list', specify elements.

elements

  • Specifies the data type for list elements in case type='list'.

aliases

version_added

  • Only needed if this option was added after initial module release; in other words, this is greater than the top (module) level version_added field.
  • This is a string, not a float, for example, for a module in ansible-core this could be version_added: '2.18'.
  • In collections, this MUST be the collection version the option was added to, not the Ansible version. For example, version_added: '1.0.0'.

suboptions

  • If this option takes a dict or list of dicts, you can define the structure here.
  • See azure.azcollection.azure_rm_securitygroup#module, azure.azcollection.azure_rm_azurefirewall#module, and openstack.cloud.baremetal_node_action#module for examples.

requirements

seealso

attributes

description

  • Required.
  • A string or a list of strings. Each string is one paragraph.
  • Explanation of what this attribute does. It should be written in full sentences.
  • You can use Ansible markup <ansible_markup> in this field.

details

  • Generally optional, but must be provided if support is partial.
  • A string or a list of strings. Each string is one paragraph.
  • Describes how support might not work as expected by the user.
  • You can use Ansible markup <ansible_markup> in this field.

support

  • Required.
  • Must be one of full, none, partial, or N/A.
  • Indicates whether this attribute is supported by this module or plugin.

membership

  • MUST ONLY be provided for the attribute action_group.
  • Lists the action groups this module or action is part of.
  • A string or a list of strings.

platforms

  • MUST ONLY be used for the attribute platform.
  • Lists the platforms the module or action supports.
  • A string or a list of strings.

version_added

  • Only needed if this attribute's support was extended after the module/plugin was created, in other words, this is greater than the top (module) level version_added field.
  • This is a string, and not a float, for example, version_added: '2.3'.
  • In collections, this must be the collection version the attribute's support was added to, not the Ansible version. For example, version_added: '1.0.0'.

notes

Documentation fragments

If you are writing multiple related modules, they may share common documentation, such as options, authentication details, file mode settings, notes: or seealso: entries. Rather than duplicate that information in each module's DOCUMENTATION block, you can save it once as a doc_fragment plugin and then include it in each module's documentation.

In Ansible, shared documentation fragments are contained in a ModuleDocFragment class in lib/ansible/plugins/doc_fragments/ or in the plugins/doc_fragments directory in a collection. To include a documentation fragment, add extends_documentation_fragment: FRAGMENT_NAME in your module documentation. Use the fully qualified collection name for the FRAGMENT_NAME (for example, kubernetes.core.k8s_auth_options).

Modules should only use items from a doc fragment if the module will implement all of the interface documented there in a manner that behaves the same as the existing modules which import that fragment. The goal is that items imported from the doc fragment will behave identically when used in another module that imports the doc fragment.

By default, only the DOCUMENTATION property from a doc fragment is inserted into the module documentation. It is possible to define additional properties in the doc fragment in order to import only certain parts of a doc fragment or mix and match as appropriate. If a property is defined in both the doc fragment and the module, the module value overrides the doc fragment.

Here is an example doc fragment named example_fragment.py:

class ModuleDocFragment(object):
    # Standard documentation
    DOCUMENTATION = r'''
    options:
      # options here
    '''

    # Additional section
    OTHER = r'''
    options:
      # other options here
    '''

To insert the contents of OTHER in a module:

extends_documentation_fragment: example_fragment.other

Or use both :

extends_documentation_fragment:
  - example_fragment
  - example_fragment.other

2.8

Since Ansible 2.8, you can have user-supplied doc_fragments by using a doc_fragments directory adjacent to play or role, just like any other plugin.

For example, all AWS modules should include:

extends_documentation_fragment:
- aws
- ec2

docfragments_collections describes how to incorporate documentation fragments in a collection.

EXAMPLES block

Immediately after the DOCUMENTATION block comes the EXAMPLES block. Here you show users how your module works with real-world examples in multi-line plain-text YAML format. The best examples are ready for the user to copy and paste into a playbook. Review and update your examples with every change to your module.

If the module has integration tests, add the example you want to add to the integration tests to make sure it works.

Best practices are:

EXAMPLES = r'''
- name: Ensure foo is installed
  namespace.collection.modulename:
    name: foo
    state: present
'''

RETURN block

Right after the EXAMPLES block comes the RETURN block. This section documents the information the module returns for use by other modules.

If your module does not return anything (apart from the standard returns made by ansible-core), specify it as RETURN = r''' # ''' Otherwise, for each value returned, provide the following fields. All the fields are required unless specified otherwise:

return name
Name of the returned field.

description
Detailed description of what this value represents. Capitalized and with a trailing dot. You can use Ansible markup <ansible_markup> in this field.

returned
When this value is returned, such as always, changed or success. This is a string and can contain any human-readable content.

type
Data type.

elements
If type='list', specifies the data type of the list's elements.

sample
One or more examples.

version_added
Only needed if this return was extended after initial module release, in other words, this is greater than the top (module) level version_added field. This is a string, and not a float, for example, version_added: '2.3'.

contains
Optional. To describe nested return values, set type: dict, or type: list/elements: dict, or if you really have to, type: complex, and repeat the elements above for each sub-field.

Here are two example RETURN sections, one with three simple fields and one with a complex nested field:

RETURN = r'''
dest:
    description: Destination file/path.
    returned: success
    type: str
    sample: /path/to/file.txt
src:
    description: Source file used for the copy on the target machine.
    returned: changed
    type: str
    sample: /home/httpd/.ansible/tmp/ansible-tmp-1423796390.97-147729857856000/source
md5sum:
    description: MD5 checksum of the file after running copy.
    returned: when supported
    type: str
    sample: 2a5aeecc61dc98c4d780b14b330e3282
'''

RETURN = r'''
packages:
    description: Information about package requirements.
    returned: success
    type: dict
    contains:
        missing:
            description: Packages that are missing from the system.
            returned: success
            type: list
            elements: str
            sample:
                - libmysqlclient-dev
                - libxml2-dev
        badversion:
            description: Packages that are installed but at bad versions.
            returned: success
            type: list
            elements: dict
            sample:
                - package: libxml2-dev
                  version: 2.9.4+dfsg1-2
                  constraint: ">= 3.0"
'''

Python imports

Immediately after the RETURN block, add the Python imports. All modules must use Python imports in the form:

from module_utils.basic import AnsibleModule

The use of "wildcard" imports such as from module_utils.basic import * is no longer allowed.

Why don't the imports go first?

Since the DOCUMENTATION, EXAMPLES, and RETURN blocks are essentially extra docstrings for the file and are not used by the module code itself, the import statements are placed after these special variables. Positioning the imports closer to the functional code helps consolidate related elements, improving readability, debugging, and overall comprehension.

Testing module documentation

Developing modules<span id="developing_modules_general"></span>

A module is a reusable, standalone script that Ansible runs on your behalf, either locally or remotely. Modules interact with your local machine, an API, or a remote system to perform specific tasks like changing a database password or spinning up a cloud instance. Each module can be used by the Ansible API, or by the ansible or ansible-playbook programs. A module provides a defined interface, accepts arguments, and returns information to Ansible by printing a JSON string to stdout before exiting.

If you need functionality that is not available in any of the thousands of Ansible modules found in collections, you can easily write your own custom module. When you write a module for local use, you can choose any programming language and follow your own rules. Use this topic to learn how to create an Ansible module in Python. After you create a module, you must add it locally to the appropriate directory so that Ansible can find and execute it. For details about adding a module locally, see developing_locally.

If you are developing a module in a collection <developing_collections>, see those documents instead.

Preparing an environment for developing Ansible modules

You just need ansible-core installed to test the module. Modules can be written in any language, but most of the following guide is assuming you are using Python. Modules for inclusion in Ansible itself must be Python or Powershell.

One advantage of using Python or Powershell for your custom modules is being able to use the module_utils common code that does a lot of the heavy lifting for argument processing, logging and response writing, among other things.

Creating a standalone module

It is highly recommended that you use a venv or virtualenv for Python development.

To create a standalone module:

  1. Create a library directory in your workspace. Your test play should live in the same directory.
  2. Create your new module file: $ touch library/my_test.py. Or just open/create it with your editor of choice.
  3. Paste the content below into your new module file. It includes the required Ansible format and documentation <developing_modules_documenting>, a simple argument spec for declaring the module options <argument_spec>, and some example code.
  4. Modify and extend the code to do what you want your new module to do. See the programming tips <developing_modules_best_practices> and Python 3 compatibility <developing_python_3> pages for pointers on writing clean and concise module code.

Creating a module in a collection

To create a new module in an existing collection called my_namespace.my_collection:

  1. Create your new module file: $ touch <PATH_TO_COLLECTION>/ansible_collections/my_namespace/my_collection/plugins/modules/my_test.py. Or just create it with your editor of choice.
  2. Paste the content below into your new module file. It includes the required Ansible format and documentation <developing_modules_documenting>, a simple argument spec for declaring the module options <argument_spec>, and some example code.
  3. Modify and extend the code to do what you want your new module to do. See the programming tips <developing_modules_best_practices> and Python 3 compatibility <developing_python_3> pages for pointers on writing clean and concise module code.

../../../../examples/scripts/my_test.py

Creating an info or a facts module

Ansible gathers information about the target machines using facts modules, and gathers information on other objects or files using info modules. If you find yourself trying to add state: info or state: list to an existing module, that is often a sign that a new dedicated _facts or _info module is needed.

In Ansible 2.8 and onwards, we have two type of information modules, they are *_info and *_facts.

If a module is named <something>_facts, it should be because its main purpose is returning ansible_facts. Do not name modules that do not do this with _facts. Only use ansible_facts for information that is specific to the host machine, for example network interfaces and their configuration, which operating system and which programs are installed.

Modules that query/return general information (and not ansible_facts) should be named _info. General information is non-host specific information, for example information on online/cloud services (you can access different accounts for the same online service from the same host), or information on VMs and containers accessible from the machine, or information on individual files or programs.

Info and facts modules, are just like any other Ansible Module, with a few minor requirements:

  1. They MUST be named <something>_info or <something>_facts, where <something> is singular.
  2. Info *_info modules MUST return in the form of the result dictionary<common_return_values> so other modules can access them.
  3. Fact *_facts modules MUST return in the ansible_facts field of the result dictionary<common_return_values> so other modules can access them.
  4. They MUST support check_mode <check_mode_dry>.
  5. They MUST NOT make any changes to the system.
  6. They MUST document the return fields<return_block> and examples<examples_block>.

You can add your facts into ansible_facts field of the result as follows:

module.exit_json(changed=False, ansible_facts=dict(my_new_fact=value_of_fact))

The rest is just like creating a normal module.

Verifying your module code

After you modify the sample code above to do what you want, you can try out your module. Our debugging tips <debugging_modules> will help if you run into bugs as you verify your module code.

Verifying your module code locally

The simplest way is to use ansible adhoc command:

ANSIBLE_LIBRARY=./library ansible -m my_test -a 'name=hello new=true' remotehost

If your module does not need to target a remote host, you can quickly and easily exercise your code locally like this:

ANSIBLE_LIBRARY=./library ansible -m my_test -a 'name=hello new=true' localhost

For a module developed in an existing collection called my_namespace.my_collection (as mentioned above):

$ ansible localhost -m my_namespace.my_collection.my_test -a 'name=hello new=true' --playbook-dir=$PWD

{
    "ANSIBLE_MODULE_ARGS": {
        "name": "hello",
        "new": true
    }
}

$ python library/my_test.py /tmp/args.json

You might also need to add your collection's path to the Python path. This tells Python to look for additional module_utils code in the given path. You can run the module code, as in the following example:

$ export PYTHONPATH=PATH_TO_COLLECTIONS:$PYTHONPATH
$ python -m ansible_collections.my_namespace.my_collection.plugins.modules.my_test /tmp/args.json

It should return output like this:

{"changed": true, "state": {"original_message": "hello", "new_message": "goodbye"}, "invocation": {"module_args": {"name": "hello", "new": true}}}

Verifying your module code in a playbook

You can easily run a full test by including it in a playbook, as long as the library directory is in the same directory as the play:

- name: test my new module
  hosts: localhost
  tasks:
  - name: run the new module
    my_test:
      name: 'hello'
      new: true
    register: testout
  - name: dump test output
    debug:
      msg: '{{ testout }}'

Testing your newly-created module

Review our testing <developing_testing> section for more detailed information, including instructions for testing module documentation <testing_module_documentation>, adding integration tests <testing_integration>, and more.

If contributing to Ansible, every new module and plugin should have integration tests, even if the tests cannot be run on Ansible CI infrastructure. In this case, the tests should be marked with the unsupported alias in aliases file <integration_aliases>.

Contributing back to Ansible

If you would like to contribute to ansible-core by adding a new feature or fixing a bug, create a fork of the ansible/ansible repository and develop against a new feature branch using the devel branch as a starting point. When you have a good working code change, you can submit a pull request to the Ansible repository by selecting your feature branch as a source and the Ansible devel branch as a target.

If you want to contribute a module to an Ansible collection <contributing_maintained_collections>, review our submission checklist <developing_modules_checklist>, programming tips <developing_modules_best_practices>, and strategy for maintaining Python 2 and Python 3 compatibility <developing_python_3>, as well as information about testing <developing_testing> before you open a pull request.

The Community Guide <ansible_community_guide> covers how to open a pull request and what happens next.

Communication and development support

Visit the Ansible communication guide<communication> for information on how to join the conversation.

Credit

Thank you to Thomas Stringer (@trstringer) for contributing source material for this topic.

Windows module development walkthrough

In this section, we will walk through developing, testing, and debugging an Ansible Windows module.

Because Windows modules are written in Powershell and need to be run on a Windows host, this guide differs from the usual development walkthrough guide.

What's covered in this section:

Windows environment setup

Unlike Python module development which can be run on the host that runs Ansible, Windows modules need to be written and tested for Windows hosts. While evaluation editions of Windows can be downloaded from Microsoft, these images are usually not ready to be used by Ansible without further modification. The easiest way to set up a Windows host so that it is ready to by used by Ansible is to set up a virtual machine using Vagrant. Vagrant can be used to download existing OS images called boxes that are then deployed to a hypervisor like VirtualBox. These boxes can either be created and stored offline or they can be downloaded from a central repository called Vagrant Cloud.

This guide will use the Vagrant boxes created by the packer-windoze repository which have also been uploaded to Vagrant Cloud. To find out more info on how these images are created, please go to the GitHub repo and look at the README file.

Before you can get started, the following programs must be installed (please consult the Vagrant and VirtualBox documentation for installation instructions):

Create a Windows server in a VM

To create a single Windows Server 2016 instance, run the following:

vagrant init jborean93/WindowsServer2016
vagrant up

This will download the Vagrant box from Vagrant Cloud and add it to the local boxes on your host and then start up that instance in VirtualBox. When starting for the first time, the Windows VM will run through the sysprep process and then create a HTTP and HTTPS WinRM listener automatically. Vagrant will finish its process once the listeners are online, after which the VM can be used by Ansible.

Create an Ansible inventory

The following Ansible inventory file can be used to connect to the newly created Windows VM:

[windows]
WindowsServer  ansible_host=127.0.0.1

[windows:vars]
ansible_user=vagrant
ansible_password=vagrant
ansible_port=55986
ansible_connection=winrm
ansible_winrm_transport=ntlm
ansible_winrm_server_cert_validation=ignore

The port 55986 is automatically forwarded by Vagrant to the Windows host that was created, if this conflicts with an existing local port then Vagrant will automatically use another one at random and display show that in the output.

The OS that is created is based on the image set. The following images can be used:

When the host is online, you can access it by RDP on 127.0.0.1:3389 but the port may differ depending on whether there was a conflict. To get rid of the host, run vagrant destroy --force and Vagrant will automatically remove the VM and any other files associated with it.

While this is useful when testing modules on a single Windows instance, these host won't work without modification with domain based modules. The Vagrantfile at ansible-windows can be used to create a test domain environment to be used in Ansible. This repo contains three files which are used by both Ansible and Vagrant to create multiple Windows hosts in a domain environment. These files are:

By default, these files will create the following environment:

The domain name and accounts can be modified by changing the variables domain_* in the inventory.yml file if it is required. The inventory file can also be modified to provision more or less servers by changing the hosts that are defined under the domain_children key. The host variable ansible_host is the private IP that will be assigned to the VirtualBox host only network adapter while vagrant_box is the box that will be used to create the VM.

Provisioning the environment

To provision the environment as is, run the following:

git clone https://github.com/jborean93/ansible-windows.git
cd vagrant
vagrant up

Vagrant provisions each host sequentially so this can take some time to complete. If any errors occur during the Ansible phase of setting up the domain, run vagrant provision to rerun just that step.

Unlike setting up a single Windows instance with Vagrant, these hosts can also be accessed using the IP address directly as well as through the forwarded ports. It is easier to access it over the host only network adapter as the normal protocol ports are used, for example RDP is still over 3389. In cases where the host cannot be resolved using the host only network IP, the following protocols can be access over 127.0.0.1 using these forwarded ports:

Replace xx with the entry number in the inventory file where the domain controller started with 00 and is incremented from there. For example, in the default inventory.yml file, WinRM over HTTPS for SERVER2012R2 is forwarded over port 29804 as it is the fourth entry in domain_children.

Windows new module development

When creating a new module there are a few things to keep in mind:

A very basic Powershell module win_environment incorporates best practices for Powershell modules. It demonstrates how to implement check-mode and diff-support, and also shows a warning to the user when a specific condition is met.

A slightly more advanced module is win_uri which additionally shows how to use different parameter types (bool, str, int, list, dict, path) and a selection of choices for parameters, how to fail a module and how to handle exceptions.

As part of the new AnsibleModule wrapper, the input parameters are defined and validated based on an argument spec. The following options can be set at the root level of the argument spec:

The actual input options for a module are set within the options value as a dictionary. The keys of this dictionary are the module option names while the values are the spec of that module option. Each spec can have the following options set:

When type=dict, or type=list and elements=dict, the following keys can also be set for that module option:

A module type can also be a delegate function that converts the value to whatever is required by the module option. For example the following snippet shows how to create a custom type that creates a UInt64 value:

$spec = @{
    uint64_type = @{ type = [Func[[Object], [UInt64]]]{ [System.UInt64]::Parse($args[0]) } }
}
$uint64_type = $module.Params.uint64_type

When in doubt, look at some of the other core modules and see how things have been implemented there.

Sometimes there are multiple ways that Windows offers to complete a task; this is the order to favor when writing modules:

PowerShell modules support a small subset of the #Requires options built into PowerShell as well as some Ansible-specific requirements specified by #AnsibleRequires. These statements can be placed at any point in the script, but are most commonly near the top. They are used to make it easier to state the requirements of the module without writing any of the checks. Each requires statement must be on its own line, but there can be multiple requires statements in one script.

These are the checks that can be used within Ansible modules:

The #AnsibleRequires -PowerShell and #AnsibleRequires -CSharpUtil support further features such as:

See the below examples for more details:

# Imports the PowerShell Ansible.ModuleUtils.Legacy provided by Ansible itself
#AnsibleRequires -PowerShell Ansible.ModuleUtils.Legacy

# Imports the PowerShell my_util in the my_namesapce.my_name collection
#AnsibleRequires -PowerShell ansible_collections.my_namespace.my_name.plugins.module_utils.my_util

# Imports the PowerShell my_util that exists in the same collection as the current module
#AnsibleRequires -PowerShell ..module_utils.my_util

# Imports the PowerShell Ansible.ModuleUtils.Optional provided by Ansible if it exists.
# If it does not exist then it will do nothing.
#AnsibleRequires -PowerShell Ansible.ModuleUtils.Optional -Optional

# Imports the C# Ansible.Process provided by Ansible itself
#AnsibleRequires -CSharpUtil Ansible.Process

# Imports the C# my_util in the my_namespace.my_name collection
#AnsibleRequires -CSharpUtil ansible_collections.my_namespace.my_name.plugins.module_utils.my_util

# Imports the C# my_util that exists in the same collection as the current module
#AnsibleRequires -CSharpUtil ..module_utils.my_util

# Imports the C# Ansible.Optional provided by Ansible if it exists.
# If it does not exist then it will do nothing.
#AnsibleRequires -CSharpUtil Ansible.Optional -Optional

For optional require statements, it is up to the module code to then verify whether the util has been imported before trying to use it. This can be done by checking if a function or type provided by the util exists or not.

While both #Requires -Module and #AnsibleRequires -PowerShell can be used to load a PowerShell module it is recommended to use #AnsibleRequires. This is because #AnsibleRequires supports collection module utils, imports by relative util names, and optional util imports.

C# module utils can reference other C# utils by adding the line using Ansible.<module_util>; to the top of the script with all the other using statements.

Windows module utilities

Like Python modules, PowerShell modules also provide a number of module utilities that provide helper functions within PowerShell. These module_utils can be imported by adding the following line to a PowerShell module:

#Requires -Module Ansible.ModuleUtils.Legacy

This will import the module_util at ./lib/ansible/module_utils/powershell/Ansible.ModuleUtils.Legacy.psm1 and enable calling all of its functions. As of Ansible 2.8, Windows module utils can also be written in C# and stored at lib/ansible/module_utils/csharp. These module_utils can be imported by adding the following line to a PowerShell module:

#AnsibleRequires -CSharpUtil Ansible.Basic

This will import the module_util at ./lib/ansible/module_utils/csharp/Ansible.Basic.cs and automatically load the types in the executing process. The Ansible.Secrets C# module util provides the [Ansible.Secrets.SecretMasker] class for registering secrets and masking strings, see developing_secret_masking. C# module utils can reference each other and be loaded together by adding the following line to the using statements at the top of the util:

using Ansible.Become;

There are special comments that can be set in a C# file for controlling the compilation parameters. The following comments can be added to the script;

As well as this, the following pre-processor symbols are defined;

A combination of these flags help to make a module util interoperable on both .NET Framework and .NET Core, here is an example of them in action:

#if CORECLR
using Newtonsoft.Json;
#else
using System.Web.Script.Serialization;
#endif

//AssemblyReference -Name Newtonsoft.Json.dll -CLR Core
//AssemblyReference -Name System.Web.Extensions.dll -CLR Framework

// Ignore error CS1702 for all .NET types
//NoWarn -Name CS1702

// Ignore error CS1956 only for .NET Framework
//NoWarn -Name CS1956 -CLR Framework

The following is a list of module_utils that are packaged with Ansible and a general description of what they do:

For more details on any specific module utility and their requirements, please see the Ansible module utilities source code.

PowerShell module utilities can be stored outside of the standard Ansible distribution for use with custom modules. Custom module_utils are placed in a folder called module_utils located in the root folder of the playbook or role directory.

C# module utilities can also be stored outside of the standard Ansible distribution for use with custom modules. Like PowerShell utils, these are stored in a folder called module_utils and the file name must end in the extension .cs, start with Ansible. and be named after the namespace defined in the util.

The below example is a role structure that contains two PowerShell custom module_utils called Ansible.ModuleUtils.ModuleUtil1, Ansible.ModuleUtils.ModuleUtil2, and a C# util containing the namespace Ansible.CustomUtil:

meta/
  main.yml
defaults/
  main.yml
module_utils/
  Ansible.ModuleUtils.ModuleUtil1.psm1
  Ansible.ModuleUtils.ModuleUtil2.psm1
  Ansible.CustomUtil.cs
tasks/
  main.yml

Each PowerShell module_util must contain at least one function that has been exported with Export-ModuleMember at the end of the file. For example

Export-ModuleMember -Function Invoke-CustomUtil, Get-CustomInfo

Exposing shared module options

PowerShell module utils can easily expose common module options that a module can use when building its argument spec. This allows common features to be stored and maintained in one location and have those features used by multiple modules with minimal effort. Any new features or bugfixes added to one of these utils are then automatically used by the various modules that call that util.

An example of this would be to have a module util that handles authentication and communication against an API. This util can be used by multiple modules to expose a common set of module options like the API endpoint, username, password, timeout, cert validation, and so on without having to add those options to each module spec.

The standard convention for a module util that has a shared argument spec would have

Because these options can be shared across various module it is highly recommended to keep the module option names and aliases in the shared spec as specific as they can be. For example do not have a util option called password, rather you should prefix it with a unique name like acme_password.

Failure to have a unique option name or alias can prevent the util being used by module that also use those names or aliases for its own options.

The following is an example module util called ServiceAuth.psm1 in a collection that implements a common way for modules to authentication with a service.

Invoke-MyServiceResource {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory=$true)]
        [ValidateScript({ $_.GetType().FullName -eq 'Ansible.Basic.AnsibleModule' })]
        $Module,

        [Parameter(Mandatory=$true)]
        [String]
        $ResourceId,

        [String]
        $State = 'present'
    )

    # Process the common module options known to the util
    $params = @{
        ServerUri = $Module.Params.my_service_url
    }
    if ($Module.Params.my_service_username) {
        $params.Credential = Get-MyServiceCredential
    }

    if ($State -eq 'absent') {
        Remove-MyService @params -ResourceId $ResourceId
    } else {
        New-MyService @params -ResourceId $ResourceId
    }
}

Get-MyNamespaceMyCollectionServiceAuthSpec {
    # Output the util spec
    @{
        options = @{
            my_service_url = @{ type = 'str'; required = $true }
            my_service_username = @{ type = 'str' }
            my_service_password = @{ type = 'str'; no_log = $true }
        }

        required_together = @(
            ,@('my_service_username', 'my_service_password')
        )
    }
}

$exportMembers = @{
    Function = 'Get-MyNamespaceMyCollectionServiceAuthSpec', 'Invoke-MyServiceResource'
}
Export-ModuleMember @exportMembers

For a module to take advantage of this common argument spec it can be set out like

#!powershell

# Include the module util ServiceAuth.psm1 from the my_namespace.my_collection collection
#AnsibleRequires -PowerShell ansible_collections.my_namespace.my_collection.plugins.module_utils.ServiceAuth

# Create the module spec like normal
$spec = @{
    options = @{
        resource_id = @{ type = 'str'; required = $true }
        state = @{ type = 'str'; choices = 'absent', 'present' }
    }
}

# Create the module from the module spec but also include the util spec to merge into our own.
$module = [Ansible.Basic.AnsibleModule]::Create($args, $spec, @(Get-MyNamespaceMyCollectionServiceAuthSpec))

# Call the ServiceAuth module util and pass in the module object so it can access the module options.
Invoke-MyServiceResource -Module $module -ResourceId $module.Params.resource_id -State $module.params.state

$module.ExitJson()

Options defined in the module spec will always have precedence over a util spec. Any list values under the same key in a util spec will be appended to the module spec for that same key. Dictionary values will add any keys that are missing from the module spec and merge any values that are lists or dictionaries. This is similar to how the doc fragment plugins work when extending module documentation.

To document these shared util options for a module, create a doc fragment plugin that documents the options implemented by the module util and extend the module docs for every module that implements the util to include that fragment in its docs.

Windows playbook module testing

You can test a module with an Ansible playbook. For example:

---
- name: test out windows module
  hosts: windows
  tasks:
  - name: test out module
    win_module:
      name: test name

This can be useful for seeing how Ansible runs with the new module end to end. Other possible ways to test the module are shown below.

Windows debugging

Debugging a module currently can only be done on a Windows host. This can be useful when developing a new module or implementing bug fixes. These are some steps that need to be followed to set this up:

# Set $ErrorActionPreference to what's set during Ansible execution
$ErrorActionPreference = "Stop"

# Set the first argument as the path to a JSON file that contains the module args
$args = @("$($pwd.Path)\args.json")

# Or instead of an args file, set $complex_args to the pre-processed module args
$complex_args = @{
    _ansible_check_mode = $false
    _ansible_diff = $false
    path = "C:\temp"
    state = "present"
}

# Import any C# utils referenced with '#AnsibleRequires -CSharpUtil' or 'using Ansible.;
# The $_csharp_utils entries should be the context of the C# util files and not the path
Import-Module -Name "$($pwd.Path)\powershell\Ansible.ModuleUtils.AddType.psm1"
$_csharp_utils = @(
    [System.IO.File]::ReadAllText("$($pwd.Path)\csharp\Ansible.Basic.cs")
)
Add-CSharpType -References $_csharp_utils -IncludeDebugInfo

# Import any PowerShell modules referenced with '#Requires -Module`
Import-Module -Name "$($pwd.Path)\powershell\Ansible.ModuleUtils.Legacy.psm1"

# End of the setup code and start of the module code
#!powershell

You can add more args to $complex_args as required by the module or define the module options through a JSON file with the structure:

{
    "ANSIBLE_MODULE_ARGS": {
        "_ansible_check_mode": false,
        "_ansible_diff": false,
        "path": "C:\\temp",
        "state": "present"
    }
}

There are multiple IDEs that can be used to debug a Powershell script, two of the most popular ones are

To be able to view the arguments as passed by Ansible to the module follow these steps.

Windows unit testing

Currently there is no mechanism to run unit tests for Powershell modules under Ansible CI.

Windows integration testing

Integration tests for Ansible modules are typically written as Ansible roles. These test roles are located in ./test/integration/targets. You must first set up your testing environment, and configure a test inventory for Ansible to connect to.

In this example we will set up a test inventory to connect to two hosts and run the integration tests for win_stat:

This will execute all the tests currently defined for that role. You can set the verbosity level using the -v argument just as you would with ansible-playbook.

When developing tests for a new module, it is recommended to test a scenario once in check mode and twice not in check mode. This ensures that check mode does not make any changes but reports a change, as well as that the second run is idempotent and does not report changes. For example:

- name: remove a file (check mode)
  win_file:
    path: C:\temp
    state: absent
  register: remove_file_check
  check_mode: true

- name: get result of remove a file (check mode)
  win_command: powershell.exe "if (Test-Path -Path 'C:\temp') { 'true' } else { 'false' }"
  register: remove_file_actual_check

- name: assert remove a file (check mode)
  assert:
    that:
    - remove_file_check is changed
    - remove_file_actual_check.stdout == 'true\r\n'

- name: remove a file
  win_file:
    path: C:\temp
    state: absent
  register: remove_file

- name: get result of remove a file
  win_command: powershell.exe "if (Test-Path -Path 'C:\temp') { 'true' } else { 'false' }"
  register: remove_file_actual

- name: assert remove a file
  assert:
    that:
    - remove_file is changed
    - remove_file_actual.stdout == 'false\r\n'

- name: remove a file (idempotent)
  win_file:
    path: C:\temp
    state: absent
  register: remove_file_again

- name: assert remove a file (idempotent)
  assert:
    that:
    - not remove_file_again is changed

Windows communication and development support

Join the Ansible Forum<ansible_forum> and use the windows tag for discussions about Ansible development for Windows.

Creating a new collection

Starting with Ansible 2.10, related modules should be developed in a collection. The Ansible Core team and community compiled these module development tips and tricks to help companies developing Ansible modules for their products and users developing Ansible modules for third-party products. See developing_collections for a more detailed description of the collections format and additional development guidelines.

LICENSING REQUIREMENTS Ansible enforces the following licensing requirements:

Before you start coding

This list of prerequisites is designed to help ensure that you develop high-quality modules that work well with ansible-core and provide a seamless user experience.

Naming conventions

Fully Qualified Collection Names (FQCNs) for plugins and modules include three elements:

For example, community.mongodb.mongodb_linux or cisco.meraki.meraki_device.

It is convenient if the organization and repository names on GitHub (or elsewhere) match your namespace and collection names on Ansible Galaxy, but it is not required. The plugin names you select, however, are always the same in your code repository and in your collection artifact on Galaxy.

Speak to us

Circulating your ideas before coding helps you adopt good practices and avoid common mistakes. After reading the "Before you start coding" section you should have a reasonable idea of the structure of your modules. Write a list of your proposed plugin and/or module names, with a short description of what each one does. Circulate that list on the Ansible Forum<ansible_forum> so the Ansible community can review your ideas for consistency and familiarity. Names and functionality that are consistent, predictable, and familiar make your collection easier to use.

Where to get support

Ansible has a thriving and knowledgeable community of module developers that is a great resource for getting your questions answered. Visit the Ansible communication guide<communication> for details.

Required files

Your collection should include the following files to be usable:

When you have these files ready, review the developing_modules_checklist again. If you are creating a new collection, you are responsible for all procedures related to your repository, including setting rules for contributions, finding reviewers, and testing and maintaining the code in your collection.

New to Git or GitHub

We realize this may be your first use of Git or GitHub. The following guides may be of use:

Developing plugins<span id="developing_plugins"></span>

Plugins augment Ansible's core functionality with logic and features that are accessible to all modules. Ansible collections include a number of handy plugins, and you can easily write your own. All plugins must:

Once you've reviewed these general guidelines, you can skip to the particular type of plugin you want to develop.

Writing plugins in Python

You must write your plugin in Python so it can be loaded by the PluginLoader and returned as a Python object that any module can use. Since your plugin will execute on the control node, you must write it in a compatible version of Python <control_node_requirements>.

Raising errors

You should return errors encountered during plugin execution by raising AnsibleError() or a similar class with a message describing the error. When wrapping other exceptions into error messages, you should always use the to_native Ansible function to ensure proper string compatibility across Python versions:

from ansible.module_utils.common.text.converters import to_native

try:
    cause_an_exception()
except Exception as e:
    raise AnsibleError('Something happened, this was original exception: %s' % to_native(e))

Since Ansible evaluates variables only when they are needed, filter and test plugins should propagate the exceptions jinja2.exceptions.UndefinedError and AnsibleUndefinedVariable to ensure undefined variables are only fatal when necessary.

Check the different AnsibleError objects and see which one applies best to your situation. Check the section on the specific plugin type you're developing for type-specific error handling details.

String encoding

You must convert any strings returned by your plugin into Python's unicode type. Converting to unicode ensures that these strings can run through Jinja2. To convert strings:

from ansible.module_utils.common.text.converters import to_text
result_string = to_text(result_string)

Plugin configuration & documentation standards

To define configurable options for your plugin, describe them in the DOCUMENTATION section of the python file. Callback and connection plugins have declared configuration requirements this way since Ansible version 2.4; most plugin types now do the same. This approach ensures that the documentation of your plugin's options will always be correct and up-to-date. To add a configurable option to your plugin, define it in this format:

options:
  option_name:
    description: describe this config option
    default: default value for this config option
    env:
      - name: MYCOLLECTION_NAME_ENV_VAR_NAME
    ini:
      - section: mycollection_section_of_ansible.cfg_where_this_config_option_is_defined
        key: key_used_in_ansible.cfg
    vars:
      - name: mycollection_name_of_ansible_var
      - name: mycollection_name_of_second_var
        version_added: X.x
    required: True/False
    secret: True/False
    type: boolean/float/integer/list/none/path/pathlist/pathspec/string/tmppath
    version_added: X.x

The supported configuration fields are:

env
List of environment variables that can be used to set this option. Each entry includes a name field specifying the environment variable name. The name should be in uppercase and should be prefixed with the collection name. Multiple environment variables can be listed for the same option. The last set environment variable in the list takes precedence if multiple are set. This is commonly used for plugins (especially inventory plugins) to allow configuration through environment variables. Examples: VMWARE_PORT, GRAFANA_PASSWORD

ini
List of configuration file settings that can be used to set this option. Each entry includes a section field for the configuration file section and a key field for the configuration key. Both should be in lowercase and should be prefixed with the collection name. Multiple configuration settings can be listed for the same option. The last set configuration setting in the list takes precedence if multiple are set. This allows plugins to be configured with ansible.cfg. Example: grafana_password

vars
List of Ansible variables that can be used to set this option. Each entry includes a name field specifying the variable name. The name should be in lowercase and should be prefixed with the collection name. Multiple variables can be listed for the same option. The last set variable in the list takes precedence if multiple are set. Variables follow Ansible's variable precedence rules. This allows plugins to be configured with Ansible variables. Example: ansible_vmware_port

secret
Boolean that marks the option as sensitive. When set to true, the resolved value is registered as a secret and is masked in Ansible output regardless of the source that set it. Only supported for options of type str, string, and list (string elements), and not supported on suboptions. Use it for passwords, tokens, private keys, and similar values. See plugin_config_secret for details.

2.22

General precedence rules

The precedence rules for configuration sources are listed below, starting with the highest precedence values:

Accessing configuration settings

To access the configuration settings in your plugin, use self.get_option(<option_name>). Some plugin types handle this differently:

If you need to populate settings explicitly, use a self.set_options() call.

Configuration sources follow the precedence rules for values in Ansible. When there are multiple values from the same category, the value defined last takes precedence. For example, in the above configuration block, if both name_of_ansible_var and name_of_second_var are defined, the value of the option_name option will be the value of name_of_second_var. Refer to general_precedence_rules for further information.

Plugins that support embedded documentation (see ansible-doc for the list) should include well-formed doc strings. If you inherit from a plugin, you must document the options it takes, either through a documentation fragment or as a copy. See module_documenting for more information on correct documentation. Thorough documentation is a good idea even if you're developing a plugin for local use.

In ansible-core 2.14 we added support for documenting filter and test plugins. You have two options for providing documentation:

Developing particular plugin types

Action plugins

Action plugins let you integrate local processing and local data with module functionality.

To create an action plugin, create a new class with the Base(ActionBase) class as the parent:

from ansible.plugins.action import ActionBase

class ActionModule(ActionBase):
    pass

From there, execute the module using the _execute_module method to call the original module. After successful execution of the module, you can modify the module return data.

module_return = self._execute_module(module_name='<NAME_OF_MODULE>',
                                     module_args=module_args,
                                     task_vars=task_vars, tmp=tmp)

For example, if you wanted to check the time difference between your Ansible control node and your target machine(s), you could write an action plugin to check the local time and compare it to the return data from Ansible's setup module:

#!/usr/bin/python
# Make coding more python3-ish, this is required for contributions to Ansible
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

from ansible.plugins.action import ActionBase
from datetime import datetime


class ActionModule(ActionBase):
    def run(self, tmp=None, task_vars=None):
        super(ActionModule, self).run(tmp, task_vars)
        module_args = self._task.args.copy()
        module_return = self._execute_module(module_name='setup',
                                             module_args=module_args,
                                             task_vars=task_vars, tmp=tmp)
        ret = dict()
        remote_date = None
        if not module_return.get('failed'):
            for key, value in module_return['ansible_facts'].items():
                if key == 'ansible_date_time':
                    remote_date = value['iso8601']

        if remote_date:
            remote_date_obj = datetime.strptime(remote_date, '%Y-%m-%dT%H:%M:%SZ')
            time_delta = datetime.utcnow() - remote_date_obj
            ret['delta_seconds'] = time_delta.seconds
            ret['delta_days'] = time_delta.days
            ret['delta_microseconds'] = time_delta.microseconds

        return dict(ansible_facts=dict(ret))

This code checks the time on the control node, captures the date and time for the remote machine using the setup module, and calculates the difference between the captured time and the local time, returning the time delta in days, seconds and microseconds.

For practical examples of action plugins, see the source code for the action plugins included with Ansible Core

Cache plugins

Cache plugins store gathered facts and data retrieved by inventory plugins.

Import cache plugins using the cache_loader so you can use self.set_options() and self.get_option(<option_name>). If you import a cache plugin directly in the code base, you can only access options by the ansible.constants, and you break the cache plugin's ability to be used by an inventory plugin.

from ansible.plugins.loader import cache_loader
[...]
plugin = cache_loader.get('custom_cache', **cache_kwargs)

There are two base classes for cache plugins, BaseCacheModule for database-backed caches, and BaseCacheFileModule for file-backed caches.

To create a cache plugin, start by creating a new CacheModule class with the appropriate base class. If you're creating a plugin using an __init__ method you should initialize the base class with any provided args and kwargs to be compatible with inventory plugin cache options. The base class calls self.set_options(direct=kwargs). After the base class __init__ method is called self.get_option(<option_name>) should be used to access cache options.

New cache plugins should take the options _uri, _prefix, and _timeout to be consistent with existing cache plugins.

from ansible.plugins.cache import BaseCacheModule

class CacheModule(BaseCacheModule):
    def __init__(self, *args, **kwargs):
        super(CacheModule, self).__init__(*args, **kwargs)
        self._connection = self.get_option('_uri')
        self._prefix = self.get_option('_prefix')
        self._timeout = self.get_option('_timeout')

If you use the BaseCacheModule, you must implement the methods get, contains, keys, set, delete, flush, and copy. The contains method should return a boolean that indicates if the key exists and has not expired. Unlike file-based caches, the get method does not raise a KeyError if the cache has expired.

If you use the BaseFileCacheModule, you must implement _load and _dump methods that will be called from the base class methods get and set.

If your cache plugin stores JSON, use AnsibleJSONEncoder in the _dump or set method and AnsibleJSONDecoder in the _load or get method.

For example cache plugins, see the source code for the cache plugins included with Ansible Core.

Note that cache plugin implementation is an internal detail, and should not be relied upon by external uses such as interrogation or consumption in a playbook.

It is assumed that a cache produced at any point in time, is usable at any future point in time, as the underlying implementation, or information provided within may change.

If the planned use case of a cache is external interrogation or consumption, we recommend to be explicit about the fetching and storing of that data, such as creating a playbook that gathers facts and stores them in the format you need them in, and then stores that data explicitly outside of the concept of a cache, where it can be relied upon.

Callback plugins

Callback plugins add new behaviors to Ansible when responding to events. By default, callback plugins control most of the output you see when running the command line programs.

To create a callback plugin, create a new class with the Base(Callbacks) class as the parent:

from ansible.plugins.callback import CallbackBase

class CallbackModule(CallbackBase):
    pass

From there, override the specific methods from the CallbackBase that you want to provide a callback for. For plugins intended for use with Ansible version 2.0 and later, you should only override methods that start with v2. For a complete list of methods that you can override, please see __init__.py in the lib/ansible/plugins/callback directory.

The following is a modified example of how Ansible's timer plugin is implemented, but with an extra option so you can see how configuration works in Ansible version 2.4 and later:

# Make coding more python3-ish, this is required for contributions to Ansible
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

# not only visible to ansible-doc, it also 'declares' the options the plugin requires and how to configure them.
DOCUMENTATION = '''
name: timer
callback_type: aggregate
requirements:
    - enable in configuration
short_description: Adds time to play stats
version_added: "2.0"  # for collections, use the collection version, not the Ansible version
description:
    - This callback just adds total play duration to the play stats.
options:
  format_string:
    description: format of the string shown to user at play end
    ini:
      - section: callback_timer
        key: format_string
    env:
      - name: ANSIBLE_CALLBACK_TIMER_FORMAT
    default: "Playbook run took %s days, %s hours, %s minutes, %s seconds"
'''
from datetime import datetime

from ansible.plugins.callback import CallbackBase


class CallbackModule(CallbackBase):
    """
    This callback module tells you how long your plays ran for.
    """
    CALLBACK_VERSION = 2.0
    CALLBACK_TYPE = 'aggregate'
    CALLBACK_NAME = 'namespace.collection_name.timer'

    # only needed if you ship it and don't want to enable by default
    CALLBACK_NEEDS_ENABLED = True

    def __init__(self):

      # make sure the expected objects are present, calling the base's __init__
      super(CallbackModule, self).__init__()

      # start the timer when the plugin is loaded, the first play should start a few milliseconds after.
      self.start_time = datetime.now()

    def _days_hours_minutes_seconds(self, runtime):
      ''' internal helper method for this callback '''
      minutes = (runtime.seconds // 60) % 60
      r_seconds = runtime.seconds - (minutes * 60)
      return runtime.days, runtime.seconds // 3600, minutes, r_seconds

    # this is only event we care about for display, when the play shows its summary stats; the rest are ignored by the base class
    def v2_playbook_on_stats(self, stats):
      end_time = datetime.now()
      runtime = end_time - self.start_time

      # Shows the usage of a config option declared in the DOCUMENTATION variable. Ansible will have set it when it loads the plugin.
      # Also note the use of the display object to print to screen. This is available to all callbacks, and you should use this over printing yourself
      self._display.display(self._plugin_options['format_string'] % (self._days_hours_minutes_seconds(runtime)))

Note that the CALLBACK_VERSION and CALLBACK_NAME definitions are required for properly functioning plugins for Ansible version 2.0 and later. CALLBACK_TYPE is mostly needed to distinguish 'stdout' plugins from the rest, since you can only load one plugin that writes to stdout.

New in ansible-core 2.22, the task result a callback plugin receives has registered secrets already masked, and anything a callback writes through Display() is masked as well. A callback that writes other data, such as task names, to a file or an external service should pass it through mask_secrets() first. See developing_callbacks_masking for the details and an example.

For example callback plugins, see the source code for the callback plugins included with Ansible Core

New in ansible-core 2.11, callback plugins are notified (by the v2_playbook_on_task_start) of meta<meta_module> tasks. By default, only explicit meta tasks that users list in their plays are sent to callbacks.

There are also some tasks which are generated internally and implicitly at various points in execution. Callback plugins can opt-in to receiving these implicit tasks as well, by setting self.wants_implicit_tasks = True. Any Task object received by a callback hook will have an .implicit attribute, which can be consulted to determine whether the Task originated from within Ansible, or explicitly by the user.

Connection plugins

Connection plugins allow Ansible to connect to target hosts so it can execute tasks on them. Ansible ships with many connection plugins, but only one can be used per host at a time. The most commonly used connection plugins are native ssh, paramiko, and local. All of these can be used with ad-hoc tasks and in playbooks.

To create a new connection plugin (for example, to support SNMP, Message bus, or other transports), copy the format of one of the existing connection plugins and drop it into connection directory on your local plugin path <local_plugins>.

Connection plugins can support common options (such as the --timeout flag) by defining an entry in the documentation for the attribute name (in this case timeout). If the common option has a non-null default, the plugin should define the same default since a different default would be ignored.

For example connection plugins, see the source code for the connection plugins included with Ansible Core.

Filter plugins

Filter plugins manipulate data. They are a feature of Jinja2 and are also available in Jinja2 templates used by the template module. As with all plugins, they can be easily extended, but instead of having a file for each one you can have several per file. Most of the filter plugins shipped with Ansible reside in a core.py.

Filter plugins do not use the standard configuration system described above, but since ansible-core 2.14 can use it as plain documentation.

Since Ansible evaluates variables only when they are needed, filter plugins should propagate the exceptions jinja2.exceptions.UndefinedError and AnsibleUndefinedVariable to ensure undefined variables are only fatal when necessary.

try:
    cause_an_exception(with_undefined_variable)
except jinja2.exceptions.UndefinedError as e:
    raise AnsibleUndefinedVariable("Something happened, this was the original exception: %s" % to_native(e))
except Exception as e:
    raise AnsibleFilterError("Something happened, this was the original exception: %s" % to_native(e))

For example filter plugins, see the source code for the filter plugins included with Ansible Core.

Inventory plugins

Inventory plugins parse inventory sources and form an in-memory representation of the inventory. Inventory plugins were added in Ansible version 2.4.

You can see the details for inventory plugins in the developing_inventory page.

Lookup plugins

Lookup plugins pull in data from external data stores. Lookup plugins can be used within playbooks both for looping --- playbook language constructs like with_fileglob and with_items are implemented through lookup plugins --- and to return values into a variable or parameter.

Lookup plugins are expected to return lists, even if just a single element.

Ansible includes many filters <playbooks_filters> which can be used to manipulate the data returned by a lookup plugin. Sometimes it makes sense to do the filtering inside the lookup plugin, other times it is better to return results that can be filtered in the playbook. Keep in mind how the data will be referenced when determining the appropriate level of filtering to be done inside the lookup plugin.

Here's a simple lookup plugin implementation --- this lookup returns the contents of a text file as a variable:

# python 3 headers, required if submitting to Ansible
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

DOCUMENTATION = r"""
  name: file
  author: Daniel Hokka Zakrisson (@dhozac) <daniel@hozac.com>
  version_added: "0.9"  # for collections, use the collection version, not the Ansible version
  short_description: read file contents
  description:
      - This lookup returns the contents from a file on the Ansible control node's file system.
  options:
    _terms:
      description: path(s) of files to read
      required: True
    option1:
      description:
            - Sample option that could modify plugin behavior.
            - This one can be set directly ``option1='x'`` or in ansible.cfg, but can also use vars or environment.
      type: string
      ini:
        - section: file_lookup
          key: option1
  notes:
    - if read in variable context, the file can be interpreted as YAML if the content is valid to the parser.
    - this lookup does not understand globbing --- use the fileglob lookup instead.
"""
from ansible.errors import AnsibleError, AnsibleParserError
from ansible.plugins.lookup import LookupBase
from ansible.utils.display import Display

display = Display()

class LookupModule(LookupBase):

    def run(self, terms, variables=None, **kwargs):

      # First of all populate options,
      # this will already take into account env vars and ini config
      self.set_options(var_options=variables, direct=kwargs)

      # lookups in general are expected to both take a list as input and output a list
      # this is done so they work with the looping construct 'with_'.
      ret = []
      for term in terms:
          display.debug("File lookup term: %s" % term)

          # Find the file in the expected search path, using a class method
          # that implements the 'expected' search path for Ansible plugins.
          lookupfile = self.find_file_in_search_path(variables, 'files', term)

          # Don't use print or your own logging, the display class
          # takes care of it in a unified way.
          display.vvvv(u"File lookup using %s as file" % lookupfile)
          try:
              if lookupfile:
                  contents, show_data = self._loader._get_file_contents(lookupfile)
                  ret.append(contents.rstrip())
              else:
                  # Always use ansible error classes to throw 'final' exceptions,
                  # so the Ansible engine will know how to deal with them.
                  # The Parser error indicates invalid options passed
                  raise AnsibleParserError()
          except AnsibleParserError:
              raise AnsibleError("could not locate file in lookup: %s" % term)

          # consume an option: if this did something useful, you can retrieve the option value here
          if self.get_option('option1') == 'do something':
            pass

      return ret

The following is an example of how this lookup is called:

---
- hosts: all
  vars:
     contents: "{{ lookup('namespace.collection_name.file', '/etc/foo.txt') }}"
     contents_with_option: "{{ lookup('namespace.collection_name.file', '/etc/foo.txt', option1='donothing') }}"
  tasks:

     - debug:
         msg: the value of foo.txt is {{ contents }} as seen today {{ lookup('pipe', 'date +"%Y-%m-%d"') }}

For example lookup plugins, see the source code for the lookup plugins included with Ansible Core.

For more usage examples of lookup plugins, see Using Lookups<playbooks_lookups>.

Test plugins

Test plugins verify data. They are a feature of Jinja2 and are also available in Jinja2 templates used by the template module. As with all plugins, they can be easily extended, but instead of having a file for each one you can have several per file. Most of the test plugins shipped with Ansible reside in a core.py. These are specially useful in conjunction with some filter plugins like map and select; they are also available for conditional directives like when:.

Test plugins do not use the standard configuration system described above. Since ansible-core 2.14 test plugins can use plain documentation.

Since Ansible evaluates variables only when they are needed, test plugins should propagate the exceptions jinja2.exceptions.UndefinedError and AnsibleUndefinedVariable to ensure undefined variables are only fatal when necessary.

try:
    cause_an_exception(with_undefined_variable)
except jinja2.exceptions.UndefinedError as e:
    raise AnsibleUndefinedVariable("Something happened, this was the original exception: %s" % to_native(e))
except Exception as e:
    raise AnsibleFilterError("Something happened, this was the original exception: %s" % to_native(e))

For example test plugins, see the source code for the test plugins included with Ansible Core.

Vars plugins

Vars plugins inject additional variable data into Ansible runs that did not come from an inventory source, playbook, or command line. Playbook constructs like 'host_vars' and 'group_vars' work using vars plugins.

Vars plugins were partially implemented in Ansible 2.0 and rewritten to be fully implemented starting with Ansible 2.4. Vars plugins are supported by collections starting with Ansible 2.10.

Older plugins used a run method as their main body/work:

def run(self, name, vault_password=None):
    pass # your code goes here

Ansible 2.0 did not pass passwords to older plugins, so vaults were unavailable. Most of the work now happens in the get_vars method which is called from the VariableManager when needed.

def get_vars(self, loader, path, entities):
    pass # your code goes here

The parameters are:

This get_vars method just needs to return a dictionary structure with the variables.

Since Ansible version 2.4, vars plugins only execute as needed when preparing to execute a task. This avoids the costly 'always execute' behavior that occurred during inventory construction in older versions of Ansible. Since Ansible version 2.10, vars plugin execution can be toggled by the user to run when preparing to execute a task or after importing an inventory source.

The user must explicitly enable vars plugins that reside in a collection. See enable_vars for details.

Legacy vars plugins are always loaded and run by default. You can prevent them from automatically running by setting REQUIRES_ENABLED to True.

class VarsModule(BaseVarsPlugin):
    REQUIRES_ENABLED = True

Include the vars_plugin_staging documentation fragment to allow users to determine when vars plugins run.

DOCUMENTATION = '''
    name: custom_hostvars
    version_added: "2.10"  # for collections, use the collection version, not the Ansible version
    short_description: Load custom host vars
    description: Load custom host vars
    options:
      stage:
        ini:
          - key: stage
            section: vars_custom_hostvars
        env:
          - name: ANSIBLE_VARS_PLUGIN_STAGE
    extends_documentation_fragment:
      - vars_plugin_staging
'''

At times a value provided by a vars plugin will contain unsafe values. The utility function wrap_var provided by ansible.utils.unsafe_proxy should be used to ensure that Ansible handles the variable and value correctly. The use cases for unsafe data is covered in unsafe_strings.

from ansible.plugins.vars import BaseVarsPlugin
from ansible.utils.unsafe_proxy import wrap_var

class VarsPlugin(BaseVarsPlugin):
    def get_vars(self, loader, path, entities):
        return dict(
            something_unsafe=wrap_var("{{ SOMETHING_UNSAFE }}")
        )

For example vars plugins, see the source code for the vars plugins included with Ansible Core.

list_of_collections
Browse existing collections, modules, and plugins

developing_api
Learn about the Python API for task execution

developing_inventory
Learn about how to develop dynamic inventory sources

developing_modules_general
Learn about how to write Ansible modules

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

adjacent_yaml_doc
Alternate YAML files as documentation

Ansible module architecture<span id="flow_modules"></span>

If you are working on the ansible-core code, writing an Ansible module, or developing an action plugin, you may need to understand how Ansible's program flow executes. If you are just using Ansible Modules in playbooks, you can skip this section.

Types of modules

Ansible supports several different types of modules in its code base. Some of these are for backwards compatibility and others are to enable flexibility.

Action plugins

Action plugins look like modules to anyone writing a playbook. Usage documentation for most action plugins lives inside a module of the same name. Some action plugins do all the work, with the module providing only documentation. Some action plugins execute modules. The normal action plugin executes modules that don't have special action plugins. Action plugins always execute on the control node.

Some action plugins do all their work on the control node. For example, the debug <debug_module> action plugin (which prints text for the user to see) and the assert <assert_module> action plugin (which tests whether values in a playbook satisfy certain criteria) execute entirely on the control node.

Most action plugins set up some values on the control node, then invoke an actual module on the managed node that does something with these values. For example, the template <template_module> action plugin takes values from the user to construct a file in a temporary location on the control node using variables from the playbook environment. It then transfers the temporary file to a temporary file on the remote system. After that, it invokes the copy module <copy_module> which operates on the remote system to move the file into its final location, sets file permissions, and so on.

New-style modules

All of the modules that ship with Ansible fall into this category. While you can write modules in any language, all official modules (shipped with Ansible) use either Python or PowerShell.

New-style modules have the arguments to the module embedded inside of them in some manner. Old-style modules must copy a separate file over to the managed node, which is less efficient as it requires two over-the-wire connections instead of only one.

Python

New-style Python modules use the Ansiballz framework for constructing modules. These modules use imports from ansible.module_utils to pull in boilerplate module code, such as argument parsing, formatting of return values as JSON, and various file operations.

In Ansible, up to version 2.0.x, the official Python modules used the module_replacer framework. For module authors, Ansiballz is largely a superset of module_replacer functionality, so you usually do not need to understand the differences between them.

PowerShell

New-style PowerShell modules use the module_replacer framework for constructing modules. These modules get a library of PowerShell code embedded in them before being sent to the managed node.

JSONARGS modules

These modules are scripts that include the string <<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>> in their body. This string is replaced with the JSON-formatted argument string. These modules typically set a variable to that value like this:

json_arguments = """<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>"""

Which is expanded as:

json_arguments = """{"param1": "test's quotes", "param2": "\"To be or not to be\" - Hamlet"}"""

Ansible outputs a JSON string with bare quotes. Double quotes are used to quote string values, double quotes inside of string values are backslash escaped, and single quotes may appear unescaped inside of a string value. To use JSONARGS, your scripting language must have a way to handle this type of string. The example uses Python's triple quoted strings to do this. Other scripting languages may have a similar quote character that won't be confused by any quotes in the JSON or it may allow you to define your own start-of-quote and end-of-quote characters. If the language doesn't give you any of these then you'll need to write a non-native JSON module <flow_want_json_modules> or Old-style module <flow_old_style_modules> instead.

These modules typically parse the contents of json_arguments using a JSON library and then use them as native variables throughout the code.

Non-native want JSON modules

If a module has the string WANT_JSON in it anywhere, Ansible treats it as a non-native module that accepts a file name as its only command-line parameter. The file name is for a temporary file containing a JSON string containing the module's parameters. The module needs to open the file, read and parse the parameters, operate on the data, and print its return data as a JSON encoded dictionary to stdout before exiting.

These types of modules are self-contained entities. As of Ansible 2.1, Ansible only modifies them to change a shebang line if present.

Examples of Non-native modules written in ruby are in the Ansible for Rubyists repository.

Binary modules

From Ansible 2.2 onwards, modules may also be small binary programs. Ansible doesn't perform any magic to make these portable to different systems so they may be specific to the system on which they were compiled or require other binary runtime dependencies. Despite these drawbacks, you may have to compile a custom module against a specific binary library if that's the only way to get access to certain resources.

Binary modules take their arguments and return data to Ansible in the same way as want JSON modules <flow_want_json_modules>.

One example of a binary module written in go.

Old-style modules

Old-style modules are similar to want JSON modules <flow_want_json_modules>, except that the file that they take contains key=value pairs for their parameters instead of JSON. Ansible decides that a module is old-style when it doesn't have any of the markers that would show that it is one of the other types.

How modules are executed

When a user uses ansible or ansible-playbook, they specify a task to execute. The task is usually the name of a module along with several parameters to be passed to the module. Ansible takes these values and processes them in various ways before they are finally executed on the remote machine.

Executor/task_executor

The TaskExecutor receives the module name and parameters that were parsed from the playbook <playbooks> (or from the command-line in the case of /usr/bin/ansible). It uses the name to decide whether it is looking at a module or an Action Plugin <flow_action_plugins>. If it is a module, it loads the Normal Action Plugin <flow_normal_action_plugin> and passes the name, variables, and other information about the task and play to that Action Plugin for further processing.

The normal action plugin

The normal action plugin executes the module on the remote host. It is the primary coordinator of much of the work to actually execute the module on the managed machine.

Much of this functionality comes from the BaseAction class, which lives in plugins/action/__init__.py. It uses the Connection and Shell objects to do its work.

When tasks <tasks> are run with the async: parameter, Ansible uses the async Action Plugin instead of the normal Action Plugin to invoke it. That program flow is currently not documented. Read the source for information on how that works.

Executor/module_common.py

Code in executor/module_common.py assembles the module to be shipped to the managed node. The module is first read in, then examined to determine its type:

After the assembling step, one final modification is made to all modules that have a shebang line. Ansible checks whether the interpreter in the shebang line has a specific path configured with an ansible_$X_interpreter inventory variable. If it does, Ansible substitutes that path for the interpreter path given in the module. After this, Ansible returns the complete module data and the module type to the Normal Action <flow_normal_action_plugin> which continues execution of the module.

Assembler frameworks

Ansible supports two assembler frameworks: Ansiballz and the older Module Replacer.

Module Replacer framework

The Module Replacer framework is the original framework implementing new-style modules, and is still used for PowerShell modules. It is essentially a preprocessor (like the C Preprocessor for those familiar with that programming language). It does straight substitutions of specific substring patterns in the module file. There are two types of substitutions:

Ansiballz framework

The Ansiballz framework was adopted in Ansible 2.1 and is used for all new-style Python modules. Unlike the Module Replacer, Ansiballz uses real Python imports of things in ansible/module_utils instead of merely preprocessing the module. It does this by constructing a zipfile -- which includes the module file, files in ansible/module_utils that are imported by the module, and some boilerplate to pass in the module's parameters. The zipfile is then Base64 encoded and wrapped in a small Python script which decodes the Base64 encoding and places the zipfile into a temp directory on the managed node. It then extracts just the Ansible module script from the zip file and places that in the temporary directory as well. Then it sets the PYTHONPATH to find Python modules inside of the zip file and imports the Ansible module as the special name, __main__. Importing it as __main__ causes Python to think that it is executing a script rather than simply importing a module. This lets Ansible run both the wrapper script and the module code in a single copy of Python on the remote machine.

* Ansible wraps the zipfile in the Python script for two reasons:

* Prior to Ansible 2.7, the module was executed by a second Python interpreter instead of being executed inside of the same process. This change was made once Python-2.4 support was dropped to speed up module execution.

In Ansiballz, any imports of Python modules from the ansible.module_utils package trigger inclusion of that Python file into the zipfile. Instances of #<<INCLUDE_ANSIBLE_MODULE_COMMON>> in the module are turned into from ansible.module_utils.basic import * and ansible/module-utils/basic.py is then included in the zipfile. Files that are included from module_utils are themselves scanned for imports of other Python modules from module_utils to be included in the zipfile as well.

Passing args

Arguments are passed differently by the two frameworks:

If you are writing modules, remember that the way we pass arguments is an internal implementation detail: it has changed in the past and will change again as soon as changes to the common module_utils code allow Ansible modules to forgo using ansible.module_utils.basic.AnsibleModule. Do not rely on the internal global _ANSIBLE_ARGS variable.

Very dynamic custom modules which need to parse arguments before they instantiate an AnsibleModule may use _load_params to retrieve those parameters. Although _load_params may change in breaking ways if necessary to support changes in the code, it is likely to be more stable than either the way we pass parameters or the internal global variable.

Prior to Ansible 2.7, the Ansible module was invoked in a second Python interpreter and the arguments were then passed to the script over the script's stdin.

Internal arguments

Both module_replacer and Ansiballz send additional arguments to the Ansible module beyond those which the user specified in the playbook. These additional arguments are internal parameters that help implement global Ansible features. Modules often do not need to know about these explicitly because the features are implemented in ansible.module_utils.basic. However, certain features need support from modules and some knowledge of the internal arguments is useful.

The internal arguments in this section are global. If you need to add a local internal argument to a custom module, create an action plugin for that specific module. See _original_basename in the copy action plugin for an example.

ansible_no_log

Type: bool

Set to True whenever an argument in a task or play specifies no_log. Any module that calls the AnsibleModule.log function handles this action automatically. If you have a module that implements its own logging then you need to check the value of _ansible_no_log. To access _ansible_no_log in a module, instantiate the AnsibleModule utility and then check the value of AnsibleModule.no_log.

no_log specified in a module's argument_spec is handled by a different mechanism.

ansible_debug

Type: bool

Operates verbose logging and logging of external commands that a module executes. If the module uses the AnsibleModule.debug function rather than the AnsibleModule.log function then the messages are only logged if you set the _ansible_debug argument to True. To access _ansible_debug in a module, instantiate the AnsibleModule utility and access AnsibleModule._debug. For more details, see DEFAULT_DEBUG.

ansible_diff

Type: bool

With this parameter you can configure your module to show a unified diff of changes that will be applied to the templated files. To access _ansible_diff in a module, instantiate the AnsibleModule utility and access AnsibleModule._diff. You can also access this parameter using the diff keyword in your playbook, or the relevant environment variable. For more details, see playbook_keywords and the DIFF_ALWAYS configuration option.

ansible_verbosity

Type: int

You can use this argument to control the level (0 for none) of verbosity in logging.

ansible_selinux_special_fs

Type: list Elements: strings

This argument provides modules with the names of file systems which should have a special SELinux context. They are used by the AnsibleModule methods which operate on files (changing attributes, moving, and copying).

Most modules can use the built-in AnsibleModule methods to manipulate files. To access in a module that needs to know about these special context file systems, instantiate AnsibleModule and examine the list in AnsibleModule._selinux_special_fs.

This argument replaces ansible.module_utils.basic.SELINUX_SPECIAL_FS from module_replacer. In the module replacer framework the argument was formatted as a comma-separated string of file system names. Under the Ansiballz framework it is a list. You can access _ansible_selinux_special_fs using the corresponding environment variable. For more details, see the DEFAULT_SELINUX_SPECIAL_FS configuration option.

2.1

ansible_syslog_facility

This argument controls which syslog facility the module logs to. Most modules should just use the AnsibleModule.log function which will then make use of this. If a module has to use this on its own, it should instantiate the AnsibleModule method and then retrieve the name of the syslog facility from AnsibleModule._syslog_facility. The Ansiballz code is less elegant than the module_replacer code:

# Old module_replacer way
import syslog
syslog.openlog(NAME, 0, syslog.LOG_USER)

# New Ansiballz way
import syslog
facility_name = module._syslog_facility
facility = getattr(syslog, facility_name, syslog.LOG_USER)
syslog.openlog(NAME, 0, facility)

For more details, see the DEFAULT_SYSLOG_FACILITY configuration option.

2.1

ansible_version

This argument passes the version of Ansible to the module. To access it, a module should instantiate the AnsibleModule method and then retrieve the version from AnsibleModule.ansible_version. This replaces ansible.module_utils.basic.ANSIBLE_VERSION from module_replacer.

2.1

ansible_module_name

Type: str

This argument passes the information to modules about their name. For more details see, the configuration option DEFAULT_MODULE_NAME.

ansible_keep_remote_files

Type: bool

This argument provides instructions that modules must be ready if they need to keep the remote files. For more details, see the DEFAULT_KEEP_REMOTE_FILES configuration option.

ansible_socket

This argument provides modules with a socket for persistent connections. The argument is created using the PERSISTENT_CONTROL_PATH_DIR configuration option.

ansible_shell_executable

Type: bool

This argument ensures that modules use the designated shell executable. For more details, see the ansible_shell_executable remote host environment parameter.

ansible_tmpdir

Type: str

This argument provides instructions to modules that all commands must use the designated temporary directory, if created. The action plugin designs this temporary directory.

Modules can access this parameter by using the public tmpdir property. The tmpdir property will create a temporary directory if the action plugin did not set the parameter.

The directory name is generated randomly, and the root of the directory is determined by one of these:

As a result, using the ansible.cfg configuration file to activate or customize this setting will not guarantee that you control the full value.

ansible_remote_tmp

The module's tmpdir property creates a randomized directory name in this directory if the action plugin did not set _ansible_tmpdir. For more details, see the remote_tmp <ansible.builtin.sh#shell:remote_tmp> parameter of the shell plugin.

Module return values & Unsafe strings

At the end of a module's execution, it formats the data that it wants to return as a JSON string and prints the string to its stdout. The normal action plugin receives the JSON string, parses it into a Python dictionary, and returns it to the executor.

If Ansible templated every string return value, it would be vulnerable to an attack from users with access to managed nodes. If an unscrupulous user disguised malicious code as Ansible return value strings, and if those strings were then templated on the control node, Ansible could execute arbitrary code. To prevent this scenario, Ansible marks all strings inside returned data as Unsafe, emitting any Jinja2 templates in the strings verbatim, not expanded by Jinja2.

Strings returned by invoking a module through ActionPlugin._execute_module() are automatically marked as Unsafe by the normal action plugin. If another action plugin retrieves information from a module through some other means, it must mark its return data as Unsafe on its own.

In case a poorly-coded action plugin fails to mark its results as "Unsafe," Ansible audits the results again when they are returned to the executor, marking all strings as Unsafe. The normal action plugin protects itself and any other code that it calls with the result data as a parameter. The check inside the executor protects the output of all other action plugins, ensuring that subsequent tasks run by Ansible will not template anything from those results either.

Special considerations

Pipelining

Ansible can transfer a module to a remote machine in one of two ways:

Pipelining only works with modules written in Python at this time because Ansible only knows that Python supports this mode of operation. Supporting pipelining means that whatever format the module payload takes before being sent over the wire must be executable by Python through stdin.

Why pass args over stdin?

Passing arguments through stdin was chosen for the following reasons:

AnsibleModule

Argument spec

The argument_spec provided to AnsibleModule defines the supported arguments for a module, as well as their type, defaults and more.

Example argument_spec:

module = AnsibleModule(argument_spec=dict(
    top_level=dict(
        type='dict',
        options=dict(
            second_level=dict(
                default=True,
                type='bool',
            )
        )
    )
))

This section will discuss the behavioral attributes for arguments:

type

type allows you to define the type of the value accepted for the argument. The default value for type is str. Possible values are:

The raw type, performs no type validation or type casting, and maintains the type of the passed value.

elements

elements works in combination with type when type='list'. elements can then be defined as elements='int' or any other type, indicating that each element of the specified list should be of that type.

default

The default option allows sets a default value for the argument for the scenario when the argument is not provided to the module. When not specified, the default value is None.

fallback

fallback accepts a tuple where the first argument is a callable (function) that will be used to perform the lookup, based on the second argument. The second argument is a list of values to be accepted by the callable.

The most common callable used is env_fallback which will allow an argument to optionally use an environment variable when the argument is not supplied.

Example:

username=dict(fallback=(env_fallback, ['ANSIBLE_NET_USERNAME']))

choices

choices accepts a list of choices that the argument will accept. The types of choices should match the type.

required

required accepts a boolean, either True or False that indicates that the argument is required. When not specified, required defaults to False. This should not be used in combination with default.

no_log

no_log accepts a boolean, either True or False, that indicates explicitly whether or not the argument value should be masked in logs and output.

How the value is hidden depends on the ansible-core version. On ansible-core 2.22 and later, the value is registered as a secret and masked wherever Ansible writes output, while the module result keeps the real value so later tasks can use it. On earlier versions, the value is removed from the module result and replaced with VALUE_SPECIFIED_IN_NO_LOG_PARAMETER, and is stripped from the module logs. See developing_secret_masking for details of the masking behavior.

[!NOTE] In the absence of no_log, if the parameter name appears to indicate that the argument value is a password or passphrase (such as "admin_password"), a warning will be shown and the value will be masked in logs but not output. To disable the warning and masking for parameters that do not contain sensitive information, set no_log to False.

aliases

aliases accepts a list of alternative argument names for the argument, such as the case where the argument is name but the module accepts aliases=['pkg'] to allow pkg to be interchangeably with name. Use of aliases can make module interfaces confusing, so we recommend adding them only when necessary. If you are updating argument names to fix a typo or improve the interface, consider moving the old names to deprecated_aliases rather than keeping them around indefinitely.

options

options implements the ability to create a sub-argument_spec, where the sub options of the top level argument are also validated using the attributes discussed in this section. The example at the top of this section demonstrates use of options. type or elements should be dict is this case.

apply_defaults

apply_defaults works alongside options and allows the default of the sub-options to be applied even when the top-level argument is not supplied.

In the example of the argument_spec at the top of this section, it would allow module.params['top_level']['second_level'] to be defined, even if the user does not provide top_level when calling the module.

removed_in_version

removed_in_version indicates which version of ansible-core or a collection a deprecated argument will be removed in. Mutually exclusive with removed_at_date, and must be used with removed_from_collection.

Example:

option = {
  'type': 'str',
  'removed_in_version': '2.0.0',
  'removed_from_collection': 'testns.testcol',
},

removed_at_date

removed_at_date indicates that a deprecated argument will be removed in a minor ansible-core release or major collection release after this date. Mutually exclusive with removed_in_version, and must be used with removed_from_collection.

Example:

option = {
  'type': 'str',
  'removed_at_date': '2020-12-31',
  'removed_from_collection': 'testns.testcol',
},

removed_from_collection

Specifies which collection (or ansible-core) deprecates this deprecated argument. Specify ansible.builtin for ansible-core, or the collection's name (format foo.bar). Must be used with removed_in_version or removed_at_date.

deprecated_aliases

Deprecates aliases of this argument. Must contain a list or tuple of dictionaries having some the following keys:

name

The name of the alias to deprecate. (Required.)

version

The version of ansible-core or the collection this alias will be removed in. Either version or date must be specified.

date

The a date after which a minor release of ansible-core or a major collection release will no longer contain this alias.. Either version or date must be specified.

collection_name

Specifies which collection (or ansible-core) deprecates this deprecated alias. Specify ansible.builtin for ansible-core, or the collection's name (format foo.bar). Must be used with version or date.

Examples:

option = {
  'type': 'str',
  'aliases': ['foo', 'bar'],
  'deprecated_aliases': [
    {
      'name': 'foo',
      'version': '2.0.0',
      'collection_name': 'testns.testcol',
    },
    {
      'name': 'foo',
      'date': '2020-12-31',
      'collection_name': 'testns.testcol',
    },
  ],
},

mutually_exclusive

If options is specified, mutually_exclusive refers to the sub-options described in options and behaves as in argument_spec_dependencies.

required_together

If options is specified, required_together refers to the sub-options described in options and behaves as in argument_spec_dependencies.

required_one_of

If options is specified, required_one_of refers to the sub-options described in options and behaves as in argument_spec_dependencies.

required_if

If options is specified, required_if refers to the sub-options described in options and behaves as in argument_spec_dependencies.

required_by

If options is specified, required_by refers to the sub-options described in options and behaves as in argument_spec_dependencies.

context

2.17

You can set the value of the context key to a dict of custom content. This allows you to provide additional context in the argument spec. The content provided is not validated or utilized by the core engine.

Example:

option = {
    'type': 'str',
    'context': {
        'disposition': '/properties/apiType',
    },
    'choices': ['http', 'soap'],
}

Dependencies between module options

The following are optional arguments for AnsibleModule():

module = AnsibleModule(
  argument_spec,
  mutually_exclusive=[
    ('path', 'content'),
  ],
  required_one_of=[
    ('path', 'content'),
  ],
)

mutually_exclusive

Must be a sequence (list or tuple) of sequences of strings. Every sequence of strings is a list of option names which are mutually exclusive. If more than one options of a list are specified together, Ansible will fail the module with an error.

Example:

mutually_exclusive=[
  ('path', 'content'),
  ('repository_url', 'repository_filename'),
],

In this example, the options path and content must not specified at the same time. Also the options repository_url and repository_filename must not be specified at the same time. But specifying path and repository_url is accepted.

To ensure that precisely one of two (or more) options is specified, combine mutually_exclusive with required_one_of.

required_together

Must be a sequence (list or tuple) of sequences of strings. Every sequence of strings is a list of option names which are must be specified together. If at least one of these options are specified, the other ones from the same sequence must all be present.

Example:

required_together=[
  ('file_path', 'file_hash'),
],

In this example, if one of the options file_path or file_hash is specified, Ansible will fail the module with an error if the other one is not specified.

required_one_of

Must be a sequence (list or tuple) of sequences of strings. Every sequence of strings is a list of option names from which at least one must be specified. If none one of these options are specified, Ansible will fail module execution.

Example:

required_one_of=[
  ('path', 'content'),
],

In this example, at least one of path and content must be specified. If none are specified, execution will fail. Specifying both is explicitly allowed; to prevent this, combine required_one_of with mutually_exclusive.

required_if

Must be a sequence of sequences. Every inner sequence describes one conditional dependency. Every sequence must have three or four values. The first two values are the option's name and the option's value which describes the condition. The further elements of the sequence are only needed if the option of that name has precisely this value.

If you want that all options in a list of option names are specified if the condition is met, use one of the following forms:

('option_name', option_value, ('option_a', 'option_b', ...)),
('option_name', option_value, ('option_a', 'option_b', ...), False),

If you want that at least one option of a list of option names is specified if the condition is met, use the following form:

('option_name', option_value, ('option_a', 'option_b', ...), True),

Example:

required_if=[
  ('state', 'present', ('path', 'content'), True),
  ('force', True, ('force_reason', 'force_code')),
],

In this example, if the user specifies state=present, at least one of the options path and content must be supplied (or both). To make sure that precisely one can be specified, combine required_if with mutually_exclusive.

On the other hand, if force (a boolean parameter) is set to true, yes and so on, both force_reason and force_code must be specified.

required_by

Must be a dictionary mapping option names to sequences of option names. If the option name in a dictionary key is specified, the option names it maps to must all also be specified. Note that instead of a sequence of option names, you can also specify one single option name.

Example:

required_by={
  'force': 'force_reason',
  'path': ('mode', 'owner', 'group'),
},

In the example, if force is specified, force_reason must also be specified. Also, if path is specified, then three three options mode, owner and group also must be specified.

Declaring check mode support

To declare that a module supports check mode, supply supports_check_mode=True to the AnsibleModule() call:

module = AnsibleModule(argument_spec, supports_check_mode=True)

The module can determine whether it is called in check mode by checking the boolean value module.check_mode. If it evaluates to True, the module must take care not to do any modification.

If supports_check_mode=False is specified, which is the default value, the module will exit in check mode with skipped=True and message remote module (<insert module name here>) does not support check mode.

Adding file options

To declare that a module should add support for all common file options, supply add_file_common_args=True to the AnsibleModule() call:

module = AnsibleModule(argument_spec, add_file_common_args=True)

You can find a list of all file options here. It is recommended that you make your DOCUMENTATION extend the doc fragment ansible.builtin.files (see module_docs_fragments) in this case, to make sure that all these fields are correctly documented.

The helper functions module.load_file_common_arguments() and module.set_fs_attributes_if_different() can be used to handle these arguments for you:

argument_spec = {
  'path': {
    'type': 'str',
    'required': True,
  },
}

module = AnsibleModule(argument_spec, add_file_common_args=True)
changed = False

# TODO do something with module.params['path'], like update its contents

# Ensure that module.params['path'] satisfies the file options supplied by the user
file_args = module.load_file_common_arguments(module.params)
changed = module.set_fs_attributes_if_different(file_args, changed)

module.exit_json(changed=changed)

Ansible and Python 3

The ansible-core code runs Python 3 (for specific versions check Control Node Requirements <control_node_requirements>). Contributors to ansible-core and to Ansible Collections should be aware of the tips in this document so that they can write code that will run on the same versions of Python as the rest of Ansible.

We do have some considerations depending on the types of Ansible code:

  1. code on the control node - code that runs on the machine where you invoke /usr/bin/ansible, only needs to support the control node's Python versions.
  2. modules - the code which Ansible transmits to and invokes on the managed machine. Modules need to support the 'managed node' Python versions, with some exceptions.
  3. shared module_utils code - the common code that is used by modules to perform tasks and sometimes used by code on the control node. Shared module_utils code needs to support the same range of Python as the modules.

However, the three types of code do not use the same string strategy. If you're developing a module or some module_utils code, be sure to read the section on string strategy carefully.

Minimum version of Python 3.x and Python 2.x

See Control Node Requirements <control_node_requirements> and Managed Node Requirements <managed_node_requirements> for the specific versions supported.

Your custom modules can support any version of Python (or other languages) you want, but the above are the requirements for the code contributed to the Ansible project.

Developing Ansible code that supports Python 2 and Python 3

The best place to start learning about writing code that supports both Python 2 and Python 3 is Lennart Regebro's book: Porting to Python 3. The book describes several strategies for porting to Python 3. The one we're using is to support Python 2 and Python 3 from a single code base

Understanding strings in Python 2 and Python 3

Python 2 and Python 3 handle strings differently, so when you write code that supports Python 3 you must decide what string model to use. Strings can be an array of bytes (like in C) or they can be an array of text. Text is what we think of as letters, digits, numbers, other printable symbols, and a small number of unprintable "symbols" (control codes).

In Python 2, the two types for these (str <python:str> for bytes and unicode <python:unicode> for text) are often used interchangeably. When dealing only with ASCII characters, the strings can be combined, compared, and converted from one type to another automatically. When non-ASCII characters are introduced, Python 2 starts throwing exceptions due to not knowing what encoding the non-ASCII characters should be in.

Python 3 changes this behavior by making the separation between bytes (bytes <python3:bytes>) and text (str <python3:str>) more strict. Python 3 will throw an exception when trying to combine and compare the two types. The programmer has to explicitly convert from one type to the other to mix values from each.

In Python 3 it is immediately apparent to the programmer when code is mixing the byte and text types inappropriately, whereas in Python 2, code that mixes those types may work until a user causes an exception by entering non-ASCII input. Python 3 forces programmers to proactively define a strategy for working with strings in their program so that they don't mix text and byte strings unintentionally.

Ansible uses different strategies for working with strings in the code on the control node, in modules <module_string_strategy>, and in module_utils <module_utils_string_strategy> code.

Control node string strategy: the Unicode Sandwich

Until recently ansible-core supported Python 2.x and followed this strategy, known as the Unicode Sandwich (named after Python 2's unicode <python:unicode> text type). For Unicode Sandwich we know that at the border of our code and the outside world (for example, file and network IO, environment variables, and some library calls) we are going to receive bytes. We need to transform these bytes into text and use that throughout the internal portions of our code. When we have to send those strings back out to the outside world we first convert the text back into bytes. To visualize this, imagine a 'sandwich' consisting of a top and bottom layer of bytes, a layer of conversion between, and all text type in the center.

For compatibility reasons you will see a bunch of custom functions we developed (to_text/to_bytes/to_native) and while Python 2 is not a concern anymore we will continue to use them as they apply for other cases that make dealing with unicode problematic.

While we will not be using most of it anymore, the documentation below is still useful for those developing modules that still need to support both Python 2 and 3 simultaneously.

Unicode Sandwich common borders: places to convert bytes to text in control node code

This is a partial list of places where we have to convert to and from bytes when using the Unicode Sandwich string strategy. It is not exhaustive but it gives you an idea of where to watch for problems.

Reading and writing to files

In Python 2, reading from files yields bytes. In Python 3, it can yield text. To make code that's portable to both we don't make use of Python 3's ability to yield text but instead do the conversion explicitly ourselves. For example:

from ansible.module_utils.common.text.converters import to_text

with open('filename-with-utf8-data.txt', 'rb') as my_file:
    b_data = my_file.read()
    try:
        data = to_text(b_data, errors='surrogate_or_strict')
    except UnicodeError:
        # Handle the exception gracefully -- usually by displaying a good
        # user-centric error message that can be traced back to this piece
        # of code.
        pass

Much of Ansible assumes that all encoded text is UTF-8. At some point, if there is demand for other encodings we may change that, but for now it is safe to assume that bytes are UTF-8.

Writing to files is the opposite process:

from ansible.module_utils.common.text.converters import to_bytes

with open('filename.txt', 'wb') as my_file:
    my_file.write(to_bytes(some_text_string))

Note that we don't have to catch UnicodeError here because we're transforming to UTF-8 and all text strings in Python can be transformed back to UTF-8.

Filesystem interaction

Dealing with file names often involves dropping back to bytes because on UNIX-like systems file names are bytes. On Python 2, if we pass a text string to these functions, the text string will be converted to a byte string inside of the function and a traceback will occur if non-ASCII characters are present. In Python 3, a traceback will only occur if the text string can't be decoded in the current locale, but it is still good to be explicit and have code which works on both versions:

import os.path

from ansible.module_utils.common.text.converters import to_bytes

filename = u'/var/tmp/くらとみ.txt'
f = open(to_bytes(filename), 'wb')
mtime = os.path.getmtime(to_bytes(filename))
b_filename = os.path.expandvars(to_bytes(filename))
if os.path.exists(to_bytes(filename)):
    pass

When you are only manipulating a filename as a string without talking to the filesystem (or a C library which talks to the filesystem) you can often get away without converting to bytes:

import os.path

os.path.join(u'/var/tmp/café', u'くらとみ')
os.path.split(u'/var/tmp/café/くらとみ')

On the other hand, if the code needs to manipulate the file name and also talk to the filesystem, it can be more convenient to transform to bytes right away and manipulate in bytes.

Make sure all variables passed to a function are the same type. If you're working with something like python3:os.path.join which takes multiple strings and uses them in combination, you need to make sure that all the types are the same (either all bytes or all text). Mixing bytes and text will cause tracebacks.

Interacting with other programs

Interacting with other programs goes through the operating system and C libraries and operates on things that the UNIX kernel defines. These interfaces are all byte-oriented so the Python interface is byte oriented as well. On both Python 2 and Python 3, byte strings should be given to Python's subprocess library and byte strings should be expected back from it.

One of the main places in Ansible's control node code that we interact with other programs is the connection plugins' exec_command methods. These methods transform any text strings they receive in the command (and arguments to the command) to execute into bytes and return stdout and stderr as byte strings. Higher level functions (like action plugins' _low_level_execute_command) transform the output into text strings.

Module string strategy: Native String

In modules we use a strategy known as Native Strings. This makes things easier on the community members who maintain so many of Ansible's modules, by not breaking backwards compatibility by mandating that all strings inside of modules are text and converting between text and bytes at the borders.

Native strings refer to the type that Python uses when you specify a bare string literal:

"This is a native string"

In Python 2, these are byte strings. In Python 3 these are text strings. Modules should be coded to expect bytes on Python 2 and text on Python 3.

Module_utils string strategy: hybrid

In module_utils code we use a hybrid string strategy. Although Ansible's module_utils code is largely like module code, some pieces of it are used by the control node as well. So it needs to be compatible with modules and with the control node's assumptions, particularly the string strategy. The module_utils code attempts to accept native strings as input to its functions and emit native strings as their output.

In module_utils code:

Module-utils functions are therefore often very defensive in nature. They convert their string parameters into text (using ansible.module_utils.common.text.converters.to_text) at the beginning of the function, do their work, and then convert the return values into the native string type (using ansible.module_utils.common.text.converters.to_native) or back to the string type that their parameters received.

Tips, tricks, and idioms for Python 2/Python 3 compatibility

Use forward-compatibility boilerplate

Use the following boilerplate code at the top of all python files to make certain constructs act the same way on Python 2 and Python 3:

# Make coding more python3-ish
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

__metaclass__ = type makes all classes defined in the file into new-style classes without explicitly inheriting from object <python3:object>.

The __future__ imports do the following:

absolute_import
Makes imports look in sys.path <python3:sys.path> for the modules being imported, skipping the directory in which the module doing the importing lives. If the code wants to use the directory in which the module doing the importing, there's a new dot notation to do so.

division
Makes division of integers always return a float. If you need to find the quotient use x // y instead of x / y.

print_function
Changes print <python3:print> from a keyword into a function.

* PEP 0328: Absolute Imports * PEP 0238: Division * PEP 3105: Print function

Prefix byte strings with b_

Since mixing text and bytes types leads to tracebacks we want to be clear about what variables hold text and what variables hold bytes. We do this by prefixing any variable holding bytes with b_. For example:

filename = u'/var/tmp/café.txt'
b_filename = to_bytes(filename)
with open(b_filename) as f:
    data = f.read()

We do not prefix the text strings instead because we only operate on byte strings at the borders, so there are fewer variables that need bytes than text.

Import Ansible's bundled Python six library

The third-party Python six library exists to help projects create code that runs on both Python 2 and Python 3. Ansible includes a version of the library in module_utils so that other modules can use it without requiring that it is installed on the remote system. To make use of it, import it like this:

from ansible.module_utils import six

Ansible can also use a system copy of six

Ansible will use a system copy of six if the system copy is a later version than the one Ansible bundles.

Handle exceptions with as

In order for code to function on Python 2.6+ and Python 3, use the new exception-catching syntax which uses the as keyword:

try:
    a = 2/0
except ValueError as e:
    module.fail_json(msg="Tried to divide by zero: %s" % e)

Do not use the following syntax as it will fail on every version of Python 3:

try:
    a = 2/0
except ValueError, e:
    module.fail_json(msg="Tried to divide by zero: %s" % e)

Update octal numbers

In Python 2.x, octal literals could be specified as 0755. In Python 3, octals must be specified as 0o755.

String formatting for control node code

Use str.format() for Python 2.6 compatibility

Starting in Python 2.6, strings gained a method called format() to put strings together. However, one commonly used feature of format() wasn't added until Python 2.7, so you need to remember not to use it in Ansible code:

# Does not work in Python 2.6!
new_string = "Dear {}, Welcome to {}".format(username, location)

# Use this instead
new_string = "Dear {0}, Welcome to {1}".format(username, location)

Both of the format strings above map positional arguments of the format() method into the string. However, the first version doesn't work in Python 2.6. Always remember to put numbers into the placeholders so the code is compatible with Python 2.6.

Python documentation on format strings:

Use percent format with byte strings

In Python 3.5 and later, byte strings do not have a format() method. However, it does have support for the older, percent-formatting.

b_command_line = b'ansible-playbook --become-user %s -K %s' % (user, playbook_file)

Rebasing a pull request

You may find that your pull request (PR) is out-of-date and needs to be rebased. This can happen for several reasons:

Rebasing the branch used to create your PR will resolve both of these issues.

Configuring your remotes

Before you can rebase your PR, you need to make sure you have the proper remotes configured. These instructions apply to any repository on GitHub, including collections repositories. On other platforms (bitbucket, gitlab), the same principles and commands apply but the syntax may be different. We use the ansible/ansible repository here as an example. In other repositories, the branch names may be different. Assuming you cloned your fork in the usual fashion, the origin remote will point to your fork:

$ git remote -v
origin  git@github.com:YOUR_GITHUB_USERNAME/ansible.git (fetch)
origin  git@github.com:YOUR_GITHUB_USERNAME/ansible.git (push)

However, you also need to add a remote which points to the upstream repository:

$ git remote add upstream https://github.com/ansible/ansible.git

Which should leave you with the following remotes:

$ git remote -v
origin  git@github.com:YOUR_GITHUB_USERNAME/ansible.git (fetch)
origin  git@github.com:YOUR_GITHUB_USERNAME/ansible.git (push)
upstream        https://github.com/ansible/ansible.git (fetch)
upstream        https://github.com/ansible/ansible.git (push)

Checking the status of your branch should show your fork is up-to-date with the origin remote:

$ git status
On branch YOUR_BRANCH
Your branch is up-to-date with 'origin/YOUR_BRANCH'.
nothing to commit, working tree clean

Rebasing your branch

Once you have an upstream remote configured, you can rebase the branch for your PR:

$ git pull --rebase upstream devel

This will replay the changes in your branch on top of the changes made in the upstream devel branch. If there are merge conflicts, you will be prompted to resolve those before you can continue.

After you rebase, the status of your branch changes:

$ git status
On branch YOUR_BRANCH
Your branch and 'origin/YOUR_BRANCH' have diverged,
and have 4 and 1 different commits each, respectively.
  (use "git pull" to merge the remote branch into yours)
nothing to commit, working tree clean

Don't worry, this is normal after a rebase. You should ignore the git status instructions to use git pull. We'll cover what to do next in the following section.

Updating your pull request

Now that you've rebased your branch, you need to push your changes to GitHub to update your PR.

Since rebasing re-writes Git history, you will need to use a force push:

$ git push --force-with-lease

Your PR on GitHub has now been updated. This will automatically trigger testing of your changes. You should check in on the status of your PR after tests have completed to see if further changes are required.

Getting help rebasing

If you need any help with rebasing your PR, or you have other development related questions, visit the Ansible communication guide<communication> for information on how to reach out to the community.

community_development_process
Information on roadmaps, opening PRs, Ansibullbot, and more

Handling secrets in modules and plugins

2.22

ansible-core 2.22 adds a secret masking API that modules and plugins can use to register sensitive values. Any registered value is replaced with $REDACTED$ wherever Ansible writes output, without changing the value itself. This page describes the API, how registered secrets move between the controller, its worker processes, and the managed node, and what you must do in callback plugins and plugin configuration to take part in masking.

For the user-facing behavior, including which values Ansible registers automatically and how registered secrets are matched, see playbooks_secret_masking.

How masking works

Ansible keeps a process-wide registry of secret strings. Every place that Ansible writes output, referred to as an egress boundary, runs the text through the registry and replaces any registered secret with the placeholder. The boundaries are:

Registration is append-only and non-destructive. Registering a value does not change it, and code that already holds the value keeps working with it. Secrets are never removed from the registry for the life of the process.

Secret masking is best effort and does not guarantee that a secret never reaches output. Modules and plugins must still avoid writing sensitive data where they can, and use no_log where a whole result is sensitive. The public API on this page is stable, but the matching algorithm and the other behaviors described here are implementation details that may change in future releases.

The log_path log file is a handler on the Python root logger. Only messages that Ansible writes through Display are masked before they reach it. Messages that a plugin, or a library it imports, emits with the standard logging module are written to the file unmasked. Pass such messages through mask_secrets() before logging them, or avoid logging sensitive data with logging at all.

Secrets registered in a worker process, such as an action, connection, lookup, or filter plugin running inside a forked worker, are sent back to the controller together with the worker's results and registered there. Secrets registered inside a module on the managed node are returned to the controller as part of the module result and registered there as well. In both cases the secret is masked in every later task, not only in the task that registered it.

In the other direction, the controller passes any registered secrets that appear in a module's arguments to the module, so that the module-side masking in AnsibleModule.log() and Ansible.Basic also covers them. Secrets that are not present in the module arguments are not sent to the managed node. A module that obtains a sensitive value another way, such as reading it from a file or receiving it from an API, must register the value itself for module-side masking to cover it, even if the controller already knows that value.

Secrets that a module registers during its run are returned inside the module's raw JSON result and are only registered on the controller when that result is processed. Connection plugin authors must audit any code that displays the raw standard output or standard error of a module, since that text contains the new secrets in plain text before the controller knows about them. The connection plugins shipped with ansible-core only display raw module output when ANSIBLE_DEBUG is enabled. Gate such output the same way, or do not display it at all.

Propagation from a worker to the controller happens when the worker sends its results, and only workers forked after that point inherit the new secret. Workers that are already running, such as those executing the same task for other hosts, do not see it. If a plugin needs a value masked across every host in the current task, register it on the controller before the task runs, for example in a vars plugin or through the register_secret filter in an earlier task.

Masking works by searching output for the exact registered string. A registered secret is masked wherever it appears, including inside a longer word or value, so avoid registering very short or common values. See secret_masking_length_rules for the rules that apply to short, overlapping, and adjacent secrets. If you know a secret can also appear in output in another shape or format, register that form as well so it is redacted too. A base64 encoded or URL encoded copy of a registered secret is a different string and is not masked on its own. The JSON-escaped form of a secret is the one exception. Because most Ansible output and module data is JSON, the registry derives the escaped forms of each secret itself, with and without \uXXXX escapes for non-ASCII characters, and masks those as well.

The ansible.module_utils.secrets API

The ansible.module_utils.secrets module is the public API for working with secrets. It is available to modules, module utilities, and all controller-side plugins.

from ansible.module_utils.secrets import register_secret, register_secrets, mask_secrets

register_secret(value)
Register a single string as a secret and return it unchanged. The return value makes it convenient to wrap an expression in place, for example token = register_secret(response['token']).

register_secrets(values)
Register every string in an iterable. Use this when you have several values to register at once.

mask_secrets(value, *, mask_placeholder='$REDACTED$')
Return a copy of value with every registered secret replaced by mask_placeholder. Use this when you write text to a destination that is not an Ansible egress boundary, such as a file, an external logging system, or a message queue.

All three functions expect Python str values. Convert bytes with to_text() before registering them. Leading and trailing spaces, tabs, carriage returns, and newlines are stripped from a value before it is registered, so there is no need to strip a value read from a file or a subprocess yourself. There is no minimum or maximum secret length but values that are empty after stripping are silently ignored by register_secret() and register_secrets(). In both cases register_secret() still returns the value exactly as it was given.

Registering secrets in a module

Values for options declared with no_log: True in the argument_spec are registered automatically when AnsibleModule validates the arguments. Register any other sensitive value that your module discovers at run time, such as a token returned by an API or a password read from a file on the managed node, as soon as you have it:

from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.secrets import register_secret


def main():
    module = AnsibleModule(
        argument_spec=dict(
            username=dict(type='str', required=True),
            password=dict(type='str', required=True, no_log=True),  # registered automatically
        ),
    )

    session = login(module.params['username'], module.params['password'])

    # The token is a new secret discovered by the module. Register it so it is masked
    # in the module log, the callback output, and any later task on the controller.
    token = register_secret(session['token'])

    # The real value is still returned so it can be used by later tasks. It is only
    # masked when written to output.
    module.exit_json(changed=False, token=token)


if __name__ == '__main__':
    main()

There is no need to strip the value from the result. The result keeps the real value, and the controller masks it when a callback or Display writes it out.

On ansible-core versions before 2.22, no_log values are removed from the module result and replaced with VALUE_SPECIFIED_IN_NO_LOG_PARAMETER, and are stripped from module logs. The secrets API is not available on those versions.

The AnsibleModule.log() method masks its message and the log_args values before writing to syslog. The module invocation log entry always replaces no_log options with $REDACTED$, regardless of the value's length, and masks any other registered secret it contains.

Modules that respawn under a different Python interpreter keep the secrets that were passed in from the controller, and any secret registered in the respawned process is returned to the controller as normal.

Registering secrets in a plugin

The same functions work in every controller-side plugin type. This example is a lookup plugin that reads a credential from an external service and registers it before returning it:

from ansible.module_utils.secrets import register_secret
from ansible.plugins.lookup import LookupBase
from ansible.utils.display import Display

display = Display()


class LookupModule(LookupBase):

    def run(self, terms, variables=None, **kwargs):
        self.set_options(var_options=variables, direct=kwargs)

        results = []
        for term in terms:
            value = fetch_credential(term)
            results.append(register_secret(value))

            # Display output is a masked boundary, so this prints the placeholder.
            display.vvv(f"fetched credential for {term}: {value}")

        return results

For plugin options that are always sensitive, prefer marking the option with secret: true in the plugin documentation instead of registering the value by hand. See plugin_config_secret.

Masking output you write yourself

Anything you write outside of an egress boundary is not masked. Call mask_secrets() before writing:

from ansible.module_utils.secrets import mask_secrets

with open('/var/log/my_plugin.log', 'a') as fd:
    fd.write(mask_secrets(f"request completed: {response_text}\n"))

Deprecated helpers

The following helpers are replaced by automatic masking and are deprecated:

Modules that called remove_values() or sanitize_keys() on their results, or that strip no_log values from a result before returning it, no longer need to. Remove those calls and return the real values.

The live argument of run_cmd() writes subprocess output directly to standard output, which bypasses masking. Run the subprocess yourself if you need to stream its output, and mask it with mask_secrets().

The heuristics that these helpers applied are gone as well. AnsibleModule.log() no longer rewrites password-like text such as user:password@host in URLs, and run_command() no longer replaces password-like arguments such as --password=... with ******** in the cmd value of a failure result. Only registered secrets are masked. If your module passes a secret on the command line or embeds one in a URL and that secret is not a no_log option, register it with register_secret() before use.

PowerShell and C# modules

The Ansible.Secrets C# module utility provides the same API for Windows modules. Import it with #AnsibleRequires and use the static Ansible.Secrets.SecretMasker class:

#!powershell

#AnsibleRequires -CSharpUtil Ansible.Basic
#AnsibleRequires -CSharpUtil Ansible.Secrets

$spec = @{
    options = @{
        username = @{ type = 'str'; required = $true }
        password = @{ type = 'str'; required = $true; no_log = $true }  # registered automatically
    }
}
$module = [Ansible.Basic.AnsibleModule]::Create($args, $spec)

$session = Connect-MyService -Username $module.Params.username -Password $module.Params.password

# Register the discovered token so it is masked in the Event Log and on the controller.
[Ansible.Secrets.SecretMasker]::RegisterSecret($session.Token)

# Mask a string yourself before writing it somewhere that is not an Ansible boundary.
$summary = [Ansible.Secrets.SecretMasker]::MaskString("Connected with token $($session.Token)")
Set-Content -Path C:\logs\my_module.log -Value $summary

$module.Result.token = $session.Token
$module.ExitJson()

The SecretMasker class exposes:

RegisterSecret(string secret) and RegisterSecret(SecureString secret)
Register a value. Prefer the SecureString overload when you already hold the value as a SecureString, so the plain text is not exposed to PowerShell AMSI notifications.

MaskString(string value) and MaskString(string value, string maskPlaceholder)
Return a copy of value with registered secrets replaced by $REDACTED$ or the placeholder you supply.

The matching rules match the Python implementation. Ansible.Basic registers no_log option values automatically, masks messages written through its event logging, and returns any newly registered secrets to the controller when the module exits.

Marking plugin configuration options as secret

Plugin configuration options can be marked with secret: true. When a marked option is resolved, its value is registered as a secret regardless of the source that set it: an environment variable, an ansible.cfg entry, an Ansible variable, a command line option, or a direct argument to the plugin.

options:
  api_token:
    description: Token used to authenticate to the service.
    type: str
    secret: true
    env:
      - name: MYCOLLECTION_API_TOKEN
    vars:
      - name: mycollection_api_token

secret is only supported for options of type str, string, and list. For a list option, each string element is registered and non-string elements are ignored. Declaring secret: true on an option of any other type is an error when the plugin's configuration is loaded, so the plugin fails to load rather than silently leaving the value unmasked.

secret is not supported on suboptions. Sub-option definitions are documentation only and are not processed by the configuration manager, so there is no point at which their values could be registered. If a sub-option holds a sensitive value, register it in the plugin with register_secret() after reading the option.

The ansible-core connection and become plugins use this keyword for their password, private key, and passphrase options. When you write a plugin that accepts a password or token, mark the option as secret instead of relying on callers to add no_log or use register_secret.

Callback plugins

The result mapping of every task result handed to a callback plugin is masked before the plugin sees it. Ansible walks CallbackTaskResult.result and replaces registered secrets at any depth, including inside loop results, so a callback never receives an unmasked secret value in the result. The walk covers:

Booleans, bytes, and None are returned unchanged. Only result.result is masked. Anything else the callback can reach is not:

Output sent through Display() is masked automatically wherever the data came from. Output written anywhere else, such as a file, a socket, an HTTP request, or a database, should be passed through mask_secrets() first so that data from outside result.result is masked as well. A callback that only writes through Display() needs no changes. The junit and tree callbacks shipped with ansible-core write to files and call mask_secrets() on everything they write, which also covers the task and play names in their output:

import json

from ansible.module_utils.secrets import mask_secrets
from ansible.plugins.callback import CallbackBase


class CallbackModule(CallbackBase):
    CALLBACK_VERSION = 2.0
    CALLBACK_TYPE = 'notification'
    CALLBACK_NAME = 'namespace.collection_name.json_file'
    CALLBACK_NEEDS_ENABLED = True

    def v2_runner_on_ok(self, result):
        # result.result is already masked, but the task name is not, so mask the
        # serialized form before writing it.
        entry = {'task': result.task_name, 'host': result.host.name, 'result': result.result}
        entry_json = mask_secrets(json.dumps(entry, default=str))

        with open('/var/log/ansible-results.jsonl', 'a') as fd:
            fd.write(entry_json + '\n')

The mask_secrets() function masks a secret in its literal form and in its JSON-escaped form, so a callback can serialize data with json.dumps() first and mask the resulting text, as the example above does. A secret that contains a double quote, a backslash, a control character, or any other character that is written differently when encoded as JSON is still recognized as a secret:

import json

from ansible.module_utils.secrets import mask_secrets, register_secret

register_secret('pa"ss\\wörd!')

# The literal value is masked.
mask_secrets('pa"ss\\wörd!')      # $REDACTED$

# The JSON-escaped form is masked as well.
result = {'password': 'pa"ss\\wörd!'}
result_json = json.dumps(result)  # {"password": "pa\"ss\\w\u00f6rd!"}
mask_secrets(result_json)         # {"password": "$REDACTED$"}

Masking the serialized text is not always enough, or not what you want, for data that does not come from result.result. Mask each value before serializing it instead when:

All of the limitations of secret masking <secret_masking_limitations> described in the user guide apply to the masked result and to any output a callback masks itself, so read them before designing how your callback handles results.

playbooks_secret_masking
User guide for secret masking, including the matching rules and limitations

developing_plugins
Developing plugins

developing_modules_general
Developing modules

developing_modules_general_windows
Developing Windows modules

Migrating Roles to Roles in Collections on Galaxy

You can migrate any existing standalone role into a collection and host the collection on Galaxy. With Ansible collections, you can distribute many roles in a single cohesive unit of reusable automation. Inside a collection, you can share custom plugins across all roles in the collection instead of duplicating them in each role's library/` directory.

You must migrate roles to collections if you want to distribute them as certified Ansible content.

If you want to import your collection to Galaxy, you need a Galaxy namespace.

See developing_collections for details on collections.

Comparing standalone roles to collection roles

Standalone roles <playbooks_reuse_roles> have the following directory structure:

role/
├── defaults
├── files
├── handlers
├── library
├── meta
├── module_utils
├── [*_plugins]
├── tasks
├── templates
├── tests
└── vars

The highlighted directories above will change when you migrate to a collection-based role. The collection directory structure includes a roles/ directory:

mynamespace/
└── mycollection/
  ├── docs/
  ├── galaxy.yml
  ├── plugins/
  │   ├── modules/
  │   │   └── module1.py
  │   ├── inventory/
  │   └── .../
  ├── README.md
  ├── roles/
  │   ├── role1/
  │   ├── role2/
  │   └── .../
  ├── playbooks/
  │   ├── files/
  │   ├── vars/
  │   ├── templates/
  │   └── tasks/
  └── tests/

You will need to use the Fully Qualified Collection Name (FQCN) to use the roles and plugins when you migrate your role into a collection. The FQCN is the combination of the collection namespace, collection name, and the content item you are referring to.

So for example, in the above collection, the FQCN to access role1 would be:

mynamespace.mycollection.role1

A collection can contain one or more roles in the roles/ directory and these are almost identical to standalone roles, except you need to move plugins out of the individual roles, and use the FQCN (Fully Qualified Collection Name) in some places, as detailed in the next section.

In standalone roles, some of the plugin directories referenced their plugin types in the plural sense; this is not the case in collections.

Migrating a role to a collection

To migrate from a standalone role that contains no plugins to a collection role:

  1. Create a local ansible_collections directory and cd to this new directory.
  2. Create a collection. If you want to import this collection to Ansible Galaxy, you need a Galaxy namespace.
$ ansible-galaxy collection init mynamespace.mycollection

This creates the collection directory structure.

  1. Copy the standalone role directory into the roles/ subdirectory of the collection. Roles in collections cannot have hyphens in the role name. Rename any such roles to use underscores instead.
$ mkdir mynamespace/mycollection/roles/my_role/
$ cp -r /path/to/standalone/role/mynamespace/my_role/\* mynamespace/mycollection/roles/my_role/
  1. Update galaxy.yml to include any role dependencies.
  2. Update the collection README.md file to add links to any role README.md files.

Migrating a role that contains plugins to a collection

To migrate from a standalone role that has plugins to a collection role:

  1. Create a local ansible_collections directory and cd to this new directory.
  2. Create a collection. If you want to import this collection to Ansible Galaxy, you need a Galaxy namespace.
$ ansible-galaxy collection init mynamespace.mycollection

This creates the collection directory structure.

  1. Copy the standalone role directory into the roles/ subdirectory of the collection. Roles in collections cannot have hyphens in the role name. Rename any such roles to use underscores instead.
$ mkdir mynamespace/mycollection/roles/my_role/
$ cp -r /path/to/standalone/role/mynamespace/my_role/\* mynamespace/mycollection/roles/my_role/
  1. Move any modules to the plugins/modules/ directory.
$ mv -r mynamespace/mycollection/roles/my_role/library/\* mynamespace/mycollection/plugins/modules/
  1. Move any other plugins to the appropriate plugins/PLUGINTYPE/ directory. See migrating_plugins_collection for additional steps that may be required.
  2. Update galaxy.yml to include any role dependencies.
  3. Update the collection README.md file to add links to any role README.md files.
  4. Change any references to the role to use the FQCN (Fully Qualified Collection Name).
---
- name: example role by FQCN
  hosts: some_host_pattern
  tasks:
    - name: import FQCN role from a collection
      import_role:
        name: mynamespace.mycollection.my_role

You can alternately use the collections keyword to simplify this:

---
- name: example role by FQCN
  hosts: some_host_pattern
  collections:
    - mynamespace.mycollection
  tasks:
    - name: import role from a collection
      import_role:
        name: my_role

Migrating other role plugins to a collection

To migrate other role plugins to a collection:

  1. Move each nonmodule plugins to the appropriate plugins/PLUGINTYPE/ directory. The mynamespace/mycollection/plugins/README.md file explains the types of plugins that the collection can contain within optionally created subdirectories.
$ mv -r mynamespace/mycollection/roles/my_role/filter_plugins/\* mynamespace/mycollection/plugins/filter/
  1. Update documentation to use the FQCN. Plugins that use doc_fragments need to use FQCN (for example, mydocfrag becomes mynamespace.mycollection.mydocfrag).
  2. Update relative imports work in collections to start with a period. For example, ./filename and ../asdfu/filestuff works but filename in same directory must be updated to ./filename.

If you have a custom module_utils or import from __init__.py, you must also:

  1. Change the Python namespace for custom module_utils to use the FQCN (Fully Qualified Collection Name) along with the ansible_collections convention. See update_module_utils_role.
  2. Change how you import from __init__.py. See update_init_role.

Updating module_utils

If any of your custom modules use a custom module utility, once you migrate to a collection you cannot address the module utility in the top level ansible.module_utils Python namespace. Ansible does not merge content from collections into the Ansible internal Python namespace. Update any Python import statements that refer to custom module utilities when you migrate your custom content to collections. See module_utils in collections <collection_module_utils> for more details.

When coding with module_utils in a collection, the Python import statement needs to take into account the FQCN (Fully Qualified Collection Name) along with the ansible_collections convention. The resulting Python import looks similar to the following example:

from ansible_collections.{namespace}.{collectionname}.plugins.module_utils.{util} import {something}

You need to follow the same rules in changing paths and using namespaced names for subclassed plugins.

The following example code snippets show a Python and a PowerShell module using both default Ansible module_utils and those provided by a collection. In this example the namespace is ansible_example and the collection is community.

In the Python example the module_utils is helper and the FQCN (Fully Qualified Collection Name) is ansible_example.community.plugins.module_utils.helper:

from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.common.text.converters import to_text
from ansible.module_utils.six.moves.urllib.parse import urlencode
from ansible.module_utils.six.moves.urllib.error import HTTPError
from ansible_collections.ansible_example.community.plugins.module_utils.helper import HelperRequest

argspec = dict(
    name=dict(required=True, type='str'),
    state=dict(choices=['present', 'absent'], required=True),
)

module = AnsibleModule(
    argument_spec=argspec,
    supports_check_mode=True
)

_request = HelperRequest(
  module,
    headers={"Content-Type": "application/json"},
     data=data
)

In the PowerShell example the module_utils is hyperv and the FQCN (Fully Qualified Collection Name) is ansible_example.community.plugins.module_utils.hyperv:

#!powershell
#AnsibleRequires -CSharpUtil Ansible.Basic
#AnsibleRequires -PowerShell ansible_collections.ansible_example.community.plugins.module_utils.hyperv

$spec = @{
    name = @{ required = $true; type = "str" }
  state = @{ required = $true; choices = @("present", "absent") }
}
$module = [Ansible.Basic.AnsibleModule]::Create($args, $spec)

Invoke-HyperVFunction -Name $module.Params.name

$module.ExitJson()

Importing from __init__.py

Because of the way that the CPython interpreter does imports, combined with the way the Ansible plugin loader works, if your custom embedded module or plugin requires importing something from an __init__.py file, that also becomes part of your collection. You can either originate the content inside a standalone role or use the file name in the Python import statement. The following example is an __init__.py file that is part of a callback plugin found inside a collection named ansible_example.community.

from ansible_collections.ansible_example.community.plugins.callback.__init__ import CustomBaseClass

Example: Migrating a standalone role with plugins to a collection

In this example we have a standalone role called my-standalone-role.webapp to emulate a standalone role that contains dashes in the name (which is not valid in collections). This standalone role contains a custom module in the library/ directory called manage_webserver.

my-standalone-role.webapp
├── defaults
├── files
├── handlers
├── library
├── meta
├── tasks
├── templates
├── tests
└── vars
  1. Create a new collection, for example, acme.webserver:
$ ansible-galaxy collection init acme.webserver
- Collection acme.webserver was created successfully
$ tree acme -d 1
acme
└── webserver
   ├── docs
   ├── plugins
   └── roles
  1. Create the webapp role inside the collection and copy all contents from the standalone role:
$ mkdir acme/webserver/roles/webapp
$ cp my-standalone-role.webapp/* acme/webserver/roles/webapp/
  1. Move the manage_webserver module to its new home in acme/webserver/plugins/modules/:
$ cp my-standalone-role.webapp/library/manage_webserver.py acme/webserver/plugins/modules/manage.py

This example changed the original source file manage_webserver.py to the destination file manage.py. This is optional but the FQCN (Fully Qualified Collection Name) provides the webserver context as acme.webserver.manage.

  1. Change manage_webserver to acme.webserver.manage in tasks/ files in the role ( for example, my-standalone-role.webapp/tasks/main.yml) and any use of the original module name.

This name change is only required if you changed the original module name, but illustrates content referenced by FQCN (Fully Qualified Collection Name) can offer context and in turn can make module and plugin names shorter. If you anticipate using these modules independent of the role, keep the original naming conventions. Users can add the collections keyword <collections_using_playbook> in their playbooks. Typically roles are an abstraction layer and users won't use components of the role independently.

Example: Supporting standalone roles and migrated collection roles in a downstream RPM

A standalone role can co-exist with its collection role counterpart (for example, as part of a support lifecycle of a product). This should only be done for a transition period, but these two can exist in downstream in packages such as RPMs. For example, the RHEL system roles could coexist with an example of a RHEL system roles collection and provide existing backwards compatibility with the downstream RPM.

This section walks through an example creating this coexistence in a downstream RPM and requires Ansible 2.9.0 or later.

To deliver a role as both a standalone role and a collection role:

  1. Place the collection in /usr/share/ansible/collections/ansible_collections/.
  2. Copy the contents of the role inside the collection into a directory named after the standalone role and place the standalone role in /usr/share/ansible/roles/.

All previously bundled modules and plugins used in the standalone role are now referenced by FQCN (Fully Qualified Collection Name) so even though they are no longer embedded, they can be found from the collection contents. This is an example of how the content inside the collection is a unique entity and does not have to be bound to a role or otherwise. You could alternately create two separate collections: one for the modules and plugins and another for the standalone role to migrate to. The role must use the modules and plugins as FQCN (Fully Qualified Collection Name).

The following is an example RPM spec file that accomplishes this using this example content:

Name: acme-ansible-content
Summary: Ansible Collection for deploying and configuring ACME webapp
Version: 1.0.0
Release: 1%{?dist}
License: GPLv3+
Source0: acme-webserver-1.0.0.tar.gz

Url: https://github.com/acme/webserver-ansible-collection
BuildArch: noarch

%global roleprefix my-standalone-role.
%global collection_namespace acme
%global collection_name webserver

%global collection_dir %{_datadir}/ansible/collections/ansible_collections/%{collection_namespace}/%{collection_name}

%description
Ansible Collection and standalone role (for backward compatibility and migration) to deploy, configure, and manage the ACME webapp software.

%prep
%setup -qc

%build

%install

mkdir -p %{buildroot}/%{collection_dir}
cp -r ./* %{buildroot}/%{collection_dir}/

mkdir -p %{buildroot}/%{_datadir}/ansible/roles
for role in %{buildroot}/%{collection_dir}/roles/*
  do
     cp -pR ${role} %{buildroot}/%{_datadir}/ansible/roles/%{roleprefix}$(basename ${role})

     mkdir -p %{buildroot}/%{_pkgdocdir}/$(basename ${role})
     for docfile in README.md COPYING LICENSE
      do
      if [ -f ${role}/${docfile} ]
          then
          cp -p ${role}/${docfile} %{buildroot}/%{_pkgdocdir}/$(basename ${role})/${docfile}
      fi
     done
done


%files
%dir %{_datadir}/ansible
%dir %{_datadir}/ansible/roles
%dir %{_datadir}/ansible/collections
%dir %{_datadir}/ansible/collections/ansible_collections
%{_datadir}/ansible/roles/
%doc %{_pkgdocdir}/*/README.md
%doc %{_datadir}/ansible/roles/%{roleprefix}*/README.md
%{collection_dir}
%doc %{collection_dir}/roles/*/README.md
%license %{_pkgdocdir}/*/COPYING
%license %{_pkgdocdir}/*/LICENSE

Using ansible.legacy to access local custom modules from collections-based roles

Some roles within a collection use local custom modules <developing_locally> that are not part of the collection itself. If there is a conflict between the custom module short name and the collection module name, you need to specify which module your tasks call. You can update the tasks to change local_module_name to ansible.legacy.local_module_name to ensure you are using the custom module.

The lifecycle of an Ansible module or plugin

Modules and plugins in the main Ansible repository have a defined life cycle, from the first introduction to final removal. The module and plugin lifecycle is tied to the Ansible release cycle <release_cycle>. A module or plugin may move through these four stages:

  1. When a module or plugin is first accepted into Ansible, we consider it in tech preview and will mark it as such in the documentation.
  2. If a module or plugin matures, the 'preview' mark in the documentation is removed. Backward compatibility for these modules and plugins is maintained but not guaranteed, which means their parameters should be maintained with stable meanings.
  3. If a module's or plugin's target API changes radically, or if someone creates a better implementation of its functionality, we may mark it deprecated. Modules and plugins that are deprecated are still available but they are reaching the end of their life cycle. We retain deprecated modules and plugins for 4 release cycles with deprecation warnings to help users update playbooks and roles that use them.
  4. When a module or plugin has been deprecated for four release cycles, it is removed and replaced with a tombstone entry in the routing configuration. Modules and plugins that are removed are no longer shipped with Ansible. The tombstone entry helps users find alternative modules and plugins.

For modules and plugins in collections, the lifecycle is similar. Since ansible-base 2.10, it is no longer possible to mark modules as 'preview' or 'stable'.

Deprecating modules and plugins in the Ansible main repository

To deprecate a module in ansible-core, you must:

  1. Rename the file so it starts with an _, for example, rename old_cloud.py to _old_cloud.py. This keeps the module available and marks it as deprecated on the module index pages.
  2. Mention the deprecation in the relevant changelog (by creating a changelog fragment with a section deprecated_features).
  3. Reference the deprecation in the relevant porting_guide_core_x.y.rst.
  4. Add deprecated: to the documentation with the following sub-values:

removed_in
A string, such as "2.10"; the version of Ansible where the module will be replaced with a docs-only module stub. Usually current release +4. Mutually exclusive with removed_at_date:.

removed_at_date
(Added in ansible-base 2.10). An ISO 8601 formatted date when the module will be removed. Usually 2 years from the date the module is deprecated. Mutually exclusive with removed_in:.

why
Optional string that used to detail why this has been removed.

alternatives
Inform users they should do instead, for example, Use M(whatmoduletouseinstead) instead..

Deprecating modules and plugins in a collection

To deprecate a module in a collection, you must:

  1. Add a deprecation entry to plugin_routing in meta/runtime.yml. For example, to deprecate the module old_cloud, add:

    plugin_routing:
    modules:
        old_cloud:
            deprecation:
                removal_version: 2.0.0
                warning_text: Use foo.bar.new_cloud instead.

    For other plugin types, you have to replace modules: with <plugin_type>:, for example lookup: for lookup plugins. When deprecating action plugins, you need to add two entries: one for the action plugin and one for the module file that contains the documentation.

    Instead of removal_version, you can also use removal_date with an ISO 8601 formatted date after which the module will be removed in a new major version of the collection.

  2. Mention the deprecation in the relevant changelog. If the collection uses antsibull-changelog, create a changelog fragment with a section deprecated_features.

  3. Add deprecated: to the documentation of the module or plugin with the following sub-values:

removed_in
A string, such as "2.10"; the version of Ansible where the module will be replaced with a docs-only module stub. Usually current release +4. Mutually exclusive with removed_at_date:.

removed_at_date
(Added in ansible-base 2.10). An ISO 8601 formatted date when the module will be removed. Usually 2 years from the date the module is deprecated. Mutually exclusive with removed_in:.

why
String that used to detail why this has been removed.

alternative
Inform users they should do instead, for example, Use M(whatmoduletouseinstead) instead.. See module_documents_linking for ways to reference entities other than modules.

Changing a module or plugin name in the Ansible main repository

You can also rename a module and keep a deprecated alias to the old name by using a symlink that starts with _. This example allows the stat module to be called with fileinfo, making the following examples equivalent:

ln -s stat.py _fileinfo.py
ansible -m stat -a "path=/tmp" localhost
ansible -m fileinfo -a "path=/tmp" localhost

Renaming a module or plugin in a collection, or redirecting a module or plugin to another collection

To rename a module or plugin in a collection, or to redirect a module or plugin to another collection, you need to add a redirect entry to plugin_routing in meta/runtime.yml. For example, to redirect the module old_cloud to foo.bar.new_cloud, add:

plugin_routing:
    modules:
        old_cloud:
            redirect: foo.bar.new_cloud

If you want to deprecate the old name, add a deprecation: entry (see above):

plugin_routing:
    modules:
        old_cloud:
            redirect: foo.bar.new_cloud
            deprecation:
                removal_version: 2.0.0
                warning_text: Use foo.bar.new_cloud instead.

You need to use the Fully Qualified Collection Name (FQCN) of the new module/plugin name, even if it is located in the same collection as the redirect. By using a FQCN from another collection, you redirect the module/plugin to that collection.

If you need to support Ansible 2.9, please note that Ansible 2.9 does not know about meta/runtime.yml. With Ansible 2.9 you can still rename plugins and modules inside one collection by using symbolic links. Note that ansible-base 2.10, ansible-core 2.11, and newer will prefer meta/runtime.yml entries over symbolic links.

Tombstoning a module or plugin in a collection

To remove a deprecated module or plugin from a collection, you need to tombstone it:

  1. Remove the module or plugin file with related files like tests, documentation references, and documentation.

  2. Add a tombstone entry in meta/runtime.yml. For example, to tombstone the module old_cloud, add:

    plugin_routing:
    modules:
        old_cloud:
            tombstone:
                removal_version: 2.0.0
                warning_text: Use foo.bar.new_cloud instead.

    Instead of removal_version, you can also use removal_date with an ISO 8601 formatted date. The date should be the date of the next major release.

Ansible architecture

Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs.

Being designed for multi-tier deployments since day one, Ansible models your IT infrastructure by describing how all of your systems inter-relate, rather than just managing one system at a time.

It uses no agents and no additional custom security infrastructure, so it is easy to deploy - and most importantly, it uses a very simple language (YAML, in the form of Ansible Playbooks) that allows you to describe your automation jobs in a way that approaches plain English.

In this section, we'll give you a really quick overview of how Ansible works so you can see how the pieces fit together.

Modules

Ansible works by connecting to your nodes and pushing out scripts called "Ansible modules" to them. Most modules accept parameters that describe the desired state of the system. Ansible then executes these modules (over SSH by default), and removes them when finished. Your library of modules can reside on any machine, and there are no servers, daemons, or databases required.

You can write your own modules <developing_modules_general>, though you should first consider whether you should <developing_modules>. Typically you'll work with your favorite terminal program, a text editor, and probably a version control system to keep track of changes to your content. You may write specialized modules in any language that can return JSON (Ruby, Python, bash, and so on).

Module utilities

When multiple modules use the same code, Ansible stores those functions as module utilities to minimize duplication and maintenance. For example, the code that parses URLs is lib/ansible/module_utils/url.py. You can write your own module utilities <developing_module_utilities> as well. Module utilities may only be written in Python or in PowerShell.

Plugins

Plugins <plugins_lookup> augment Ansible's core functionality. While modules execute on the target system in separate processes (usually that means on a remote system), plugins execute on the control node within the /usr/bin/ansible process. Plugins offer options and extensions for the core features of Ansible - transforming data, logging output, connecting to inventory, and more. Ansible ships with a number of handy plugins, and you can easily write your own <developing_plugins>. For example, you can write an inventory plugin <developing_inventory> to connect to any datasource that returns JSON. Plugins must be written in Python.

Inventory

By default, Ansible represents the machines it manages in a file (INI, YAML, and so on) that puts all of your managed machines in groups of your own choosing.

To add new machines, there is no additional SSL signing server involved, so there's never any hassle deciding why a particular machine didn't get linked up due to obscure NTP or DNS issues.

If there's another source of truth in your infrastructure, Ansible can also connect to that. Ansible can draw inventory, group, and variable information from sources like EC2, Rackspace, OpenStack, and more.

Here's what a plain text inventory file looks like:

[webservers]
www1.example.com
www2.example.com

[dbservers]
db0.example.com
db1.example.com

Once inventory hosts are listed, variables can be assigned to them in simple text files (in a subdirectory called 'group_vars/' or 'host_vars/') or directly in the inventory file.

Or, as already mentioned, use a dynamic inventory to pull your inventory from data sources like EC2, Rackspace, or OpenStack.

Playbooks

Playbooks can finely orchestrate multiple slices of your infrastructure topology, with very detailed control over how many machines to tackle at a time. This is where Ansible starts to get most interesting.

Ansible's approach to orchestration is one of finely-tuned simplicity, as we believe your automation code should make perfect sense to you years down the road and there should be very little to remember about special syntax or features.

Here's what a simple playbook looks like:

---
- hosts: webservers
  serial: 5 # update 5 machines at a time
  roles:
  - common
  - webapp

- hosts: content_servers
  roles:
  - common
  - content

The Ansible search path

Modules, module utilities, plugins, playbooks, and roles can live in multiple locations. If you write your own code to extend Ansible's core features, you may have multiple files with similar or the same names in different locations on your Ansible control node. The search path determines which of these files Ansible will discover and use on any given playbook run.

Ansible's search path grows incrementally over a run. As Ansible finds each playbook and role included in a given run, it appends any directories related to that playbook or role to the search path. Those directories remain in scope for the duration of the run, even after the playbook or role has finished executing. Ansible loads modules, module utilities, and plugins in this order:

  1. Directories adjacent to a playbook specified on the command line. If you run Ansible with ansible-playbook /path/to/play.yml, Ansible appends these directories if they exist:

    /path/to/modules
/path/to/module_utils
/path/to/plugins

  2. Directories adjacent to a playbook that is statically imported by a playbook specified on the command line. If play.yml includes - import_playbook: /path/to/subdir/play1.yml, Ansible appends these directories if they exist:

    /path/to/subdir/modules
/path/to/subdir/module_utils
/path/to/subdir/plugins

  3. Subdirectories of a role directory referenced by a playbook. If play.yml runs myrole, Ansible appends these directories if they exist:

    /path/to/roles/myrole/modules
/path/to/roles/myrole/module_utils
/path/to/roles/myrole/plugins

  4. Directories specified as default paths in ansible.cfg or by the related environment variables, including the paths for the various plugin types. See ansible_configuration_settings for more information. Sample ansible.cfg fields:

    DEFAULT_MODULE_PATH
DEFAULT_MODULE_UTILS_PATH
DEFAULT_CACHE_PLUGIN_PATH
DEFAULT_FILTER_PLUGIN_PATH

    Sample environment variables:

    ANSIBLE_LIBRARY
ANSIBLE_MODULE_UTILS
ANSIBLE_CACHE_PLUGINS
ANSIBLE_FILTER_PLUGINS

  5. The standard directories that ship as part of the Ansible distribution.

Modules, module utilities, and plugins in user-specified directories will override the standard versions. This includes some files with generic names. For example, if you have a file named basic.py in a user-specified directory, it will override the standard ansible.module_utils.basic.

If you have more than one module, module utility, or plugin with the same name in different user-specified directories, the order of commands at the command line and the order of includes and roles in each play will affect which one is found and used on that particular play.

Adjacent YAML documentation files

YAML documentation for plugins

For most Ansible plugins, the documentation is in the same file as the code. This approach does not work for cases when:

These cases require plugins to provide documentation in an adjacent .py file. As of ansible-core 2.14, you can provide documentation as adjacent YAML files instead. The format of a YAML documentation file is nearly identical to its Python equivalent, except it is pure YAML.

YAML format

In Python, each section is a variable DOCUMENTATION = r""" ... """ while in YAML, it is a mapping key DOCUMENTATION: ....

Here is a longer example that shows documentation as embedded in a Python file:

DOCUMENTATION = r'''
  description: something
  options:
    option_name:
      description: describe this config option
      default: default value for this config option
      env:
        - name: NAME_OF_ENV_VAR
      ini:
        - section: section_of_ansible.cfg_where_this_config_option_is_defined
          key: key_used_in_ansible.cfg
      vars:
        - name: name_of_ansible_var
        - name: name_of_second_var
          version_added: X.x
      required: True/False
      type: boolean/float/integer/list/none/path/pathlist/pathspec/string/tmppath
      version_added: X.x
'''

EXAMPLES = r'''
  # TODO: write examples
'''

This example shows the same documentation in YAML format:

DOCUMENTATION:
  description: something
  options:
    option_name:
      description: describe this config option
      default: default value for this config option
      env:
        - name: NAME_OF_ENV_VAR
      ini:
        - section: section_of_ansible.cfg_where_this_config_option_is_defined
          key: key_used_in_ansible.cfg
      vars:
        - name: name_of_ansible_var
        - name: name_of_second_var
          version_added: X.x
      required: True/False
      type: boolean/float/integer/list/none/path/pathlist/pathspec/string/tmppath
      version_added: X.x

EXAMPLES: # TODO: write examples

As the examples above show, Python variables already contain YAML. The main change to use YAML documentation is to simply move the YAML out of such variables.

Any adjacent YAML documentation files must be in the same directory as the plugin or module that they document. This means the documentation is available in any directory that contains the plugins or modules.

Supported plugin types

YAML documentation is mainly intended for filters, tests, and modules. While it is possible to use it with other plugin types, Ansible always recommends having documentation in the same file as the code for most cases.

list_of_collections
Browse existing collections, modules, and plugins

developing_api
Learn about the Python API for task execution

developing_inventory
Learn about how to develop dynamic inventory sources

developing_modules_general
Learn about how to write Ansible modules

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Basic rules

Use standard American English

Ansible uses Standard American English. Watch for common words that are spelled differently in American English (color vs colour, organize vs organise, and so on).

Write for a global audience

Everything you say should be understandable by people of different backgrounds and cultures. Avoid idioms and regionalism and maintain a neutral tone that cannot be misinterpreted. Avoid figurative language, metaphors, and humor.

Follow naming conventions

Always follow naming conventions and trademarks.

Use clear sentence structure

Clear sentence structure means:

Some examples of improving sentences:

Bad:
The unwise walking about upon the area near the cliff edge may result in a dangerous fall and therefore it is recommended that one remains a safe distance to maintain personal safety.

Better:
Danger! Stay away from the cliff.

Bad:
Furthermore, large volumes of water are also required for the process of extraction.

Better:
Extraction also requires large volumes of water.

Use one sentence per line

Consider using a technique called one sentence per line when composing paragraphs, or use at least one line per sentence (following https://sembr.org/). This technique allows you to format RST and Markdown source in a natural and semantic way that is easy to review and edit.

Lines of text that occur next to each other are rendered as a paragraph so readers do not notice the line breaks between sentences. To create a new paragraph, add an empty line between sentences. You can still soft wrap lines of text when using one sentence per line. The important thing is that each line starts at the left margin.

Using one sentence per line reduces cognitive load and makes life much easier for people who review changes. Diffs become much more straightforward and easier to parse. For example, when multiple sentences are wrapped at a fixed column width, a change at the start of a paragraph causes the remaining lines in the paragraph to reposition.

In addition to the advantages for version control, one sentence per line makes it easier to do the following:

For an example of the one sentence per line in the Ansible documentation, take a look at the RST source under docs/docsite/rst/getting_started/.

Avoid verbosity

Write short, succinct sentences. Avoid terms like:

Avoid negative constructions

Where possible, state what something is or does rather than what it is not. Positive statements are clearer and easier to translate.

Bad:
Do not close the editor without saving.

Better:
Save the file before closing the editor.

Preserve existing headings and anchors

Do not change existing headings or RST anchors when editing existing documentation.

Highlight menu items and commands

When documenting menus or commands, it helps to bold what is important.

For menu procedures, bold the menu names, button names, and so on to help the user find them on the GUI:

  1. On the File menu, click Open.
  2. Type a name in the username field.
  3. In the Open dialog box, click Save.
  4. On the toolbar, click the Open File icon.

For code or command snippets, use the RST code-block directive:

.. code-block:: bash

  ssh my_vyos_user@vyos.example.net
  show config

Grammar and Punctuation

Common Styles and Usage, and Common Mistakes

Ansible

Capitalization

If it is not a real product, service, or department at Ansible, don't capitalize it. Not even if it seems important. Capitalize only the first letter of the first word in headlines.

Do not use uppercase letters for emphasis.

Colon

A colon is generally used before a list or series: - The Triangle Area consists of three cities: Raleigh, Durham, and Chapel Hill.

But not if the list is a complement or object of an element in the sentence: - Before going on vacation, be sure to (1) set the alarm, (2) cancel the newspaper, and (3) ask a neighbor to collect your mail.

Use a colon after "as follows" and "the following" if the related list comes immediately after: wedge The steps for changing directories are as follows:

  1. Open a terminal.
  2. Type cd...

Use a colon to introduce a bullet list (or dash, or icon/symbol of your choice):

In the Properties dialog box, you'll find the following entries:

Commas

Use serial commas, the comma before the "and" in a series of three or more items:

It is easier to read that way and helps avoid confusion. The primary exception to this you will see is in PR, where it is traditional not to use serial commas because it is often the style of journalists.

Commas are always important, considering the vast difference in meanings of the following two statements.

Correct punctuation could save Grandma's life.

If that does not convince you, maybe this will:

image

Contractions

Do not use contractions in Ansible documents.

Em dashes

When possible, use em-dashes with no space on either side. When full em-dashes aren't available, use double-dashes with no spaces on either side--like this.

A pair of em dashes can be used in place of commas to enhance readability. Note, however, that dashes are always more emphatic than commas.

A pair of em dashes can replace a pair of parentheses. Dashes are considered less formal than parentheses; they are also more intrusive. If you want to draw attention to the parenthetical content, use dashes. If you want to include the parenthetical content more subtly, use parentheses.

When dashes are used in place of parentheses, surrounding punctuation should be omitted. Compare the following examples.

Upon discovering the errors (all 124 of them), the publisher immediately recalled the books.

Upon discovering the errors—all 124 of them—the publisher immediately recalled the books.

When used in place of parentheses at the end of a sentence, only a single dash is used.

After three weeks on set, the cast was fed up with his direction (or, rather, lack of direction).

After three weeks on set, the cast was fed up with his direction—or, rather, lack of direction.

Exclamation points (!)

Do not use them at the end of sentences. An exclamation point can be used when referring to a command, such as the bang (!) command.

Gender References

Do not use gender-specific pronouns in documentation. It is far less awkward to read a sentence that uses "they" and "their" rather than "he/she" and "his/hers."

It is fine to use "you" when giving instructions and "the user," "new users," and so on. in more general explanations.

Never use "one" in place of "you" when writing technical documentation. Using "one" is far too formal.

Never use "we" when writing. "We" aren't doing anything on the user side. Ansible's products are doing the work as requested by the user.

Hyphen

The hyphen's primary function is the formation of certain compound terms. Do not use a hyphen unless it serves a purpose. If a compound adjective cannot be misread or, as with many psychological terms, its meaning is established, a hyphen is not necessary.

Use hyphens to avoid ambiguity or confusion:

a little-used car
a little used-car

cross complaint
cross-complaint

high-school girl
high schoolgirl

fine-tooth comb (most people do not comb their teeth)

third-world war
third world war

image

In professionally printed material (particularly books, magazines, and newspapers), the hyphen is used to divide words between the end of one line and the beginning of the next. This allows for an evenly aligned right margin without highly variable (and distracting) word spacing.

Lists

Introduce every list with a complete sentence. Use bulleted lists for unordered items and numbered lists for sequential steps.

Keep the structure of bulleted lists equivalent and consistent. If one bullet is a verb phrase, they should all be verb phrases. If one is a complete sentence, they should all be complete sentences, and so on.

Capitalize the first word of each bullet. Unless it is obvious that it is just a list of items, such as a list of items like: * computer * monitor * keyboard * mouse

When the bulleted list appears within the context of other copy, (unless it is a straight list like the previous example) add periods, even if the bullets are sentence fragments. Part of the reason behind this is that each bullet is said to complete the original sentence.

In some cases where the bullets are appearing independently, such as in a poster or a homepage promotion, they do not need periods.

When giving instructional steps, use numbered lists instead of bulleted lists.

Months and States

Abbreviate months and states according to AP. Months are only abbreviated if they are used in conjunction with a day. Example: "The President visited in January 1999." or "The President visited Jan. 12."

Months: Jan., Feb., March, April, May, June, July, Aug., Sept., Nov., Dec.

States: Ala., Ariz., Ark., Calif., Colo., Conn., Del., Fla., Ga., Ill., Ind., Kan., Ky., La., Md., Mass., Mich., Minn., Miss., Mo., Mont., Neb., Nev., NH, NJ, NM, NY, NC, ND, Okla., Ore., Pa., RI, SC, SD, Tenn., Vt., Va., Wash., W.Va., Wis., Wyo.

Numbers

Numbers between one and nine are written out. 10 and above are numerals. The exception to this is writing "4 million" or "4 GB." It is also acceptable to use numerals in tables and charts.

Only

Place the word "only" immediately before the word or phrase that it modifies.

Phone Numbers

Phone number style: 1 (919) 555-0123 x002 and 1 888-GOTTEXT

Procedures

Include one action per procedure step. Start each step with an imperative verb. For procedures with many steps, include a summary at the start that explains the overall goal or the groups of steps required. Do not include system responses or expected output as separate numbered steps. Describe expected results within the step or as unnumbered text after the step. For example:

  1. Run the ansible-playbook command. The output displays a list of completed tasks.
  2. Verify the results.

Pronouns

Ensure every pronoun refers to a single, clear noun. If the reference is ambiguous, repeat the noun instead. Place pronouns as close as possible to the noun they reference.

Quotations (Using Quotation Marks and Writing Quotes)

"Place the punctuation inside the quotes," the editor said.

Except in rare instances, use only "said" or "says" because anything else just gets in the way of the quote itself, and also tends to editorialize.

Place the name first right after the quote:
"I like to write first-person because I like to become the character I'm writing," Wally Lamb said.

Not:
"I like to write first-person because I like to become the character I'm writing," said Wally Lamb.

Semicolon

Use a semicolon to separate items in a series if the items contain commas:

Use a semicolon before a conjunctive adverb (however, therefore, otherwise, namely, for example, and so on): - I think; therefore, I am.

Spacing after sentences

Use only a single space after a sentence.

One sentence per line is preferred to wrapping sentences over multiple lines. See one_sentence_per_line.

That

Include the word "that" in subordinate clauses for clarity, especially for global audiences and translation. For example, write "Verify that the file exists" rather than "Verify the file exists."

Time

Ansible documentation style guide

Welcome to the Ansible style guide! To create clear, concise, consistent, useful materials on docs.ansible.com, follow these guidelines:

Linguistic guidelines

We want the Ansible documentation to be:

We want reading the docs to feel like having an experienced, friendly colleague explain how Ansible works.

Stylistic cheat-sheet

This cheat-sheet illustrates a few rules that help achieve the "Ansible tone":

Rule Good example Bad example
Use active voice You can run a task by A task can be run by
Use the present tense This command creates a This command will create a
Address the reader As you expand your inventory When the number of managed nodes grows
Use standard English Return to this page Hop back to this page
Use American English The color of the output The colour of the output

Title and heading case

Titles and headings should be written in sentence case. For example, this section's title is Title and heading case, not Title and Heading Case or TITLE AND HEADING CASE.

Use gerunds (-ing words) for high-level task headings and imperative verbs for subtask headings. For example, use Installing Ansible for a chapter heading and Install the package for a step-level heading within it.

Avoid using Latin phrases

Latin words and phrases like e.g. or etc. are easily understood by English speakers. They may be harder to understand for others and are also tricky for automated translation.

Use the following English terms in place of Latin terms or abbreviations:

Latin English
i.e in other words
e.g. for example
etc and so on
via by/ through
vs./versus rather than/against

reStructuredText guidelines

The Ansible documentation is written in reStructuredText and processed by Sphinx. We follow these technical or mechanical guidelines on all rST pages:

Heading notation

Section headings in reStructuredText can use a variety of notations. Sphinx will 'learn on the fly' when creating a hierarchy of headings. To make our documents easy to read and to edit, we follow a standard set of heading notations. We use:

###############
Developer guide
###############

*******************
Ansible style guide
*******************

Mechanical guidelines
=====================

Internal navigation
-------------------

Adding anchors
^^^^^^^^^^^^^^

Paragraph that needs a title
""""""""""""""""""""""""""""

Syntax highlighting - Pygments

The Ansible documentation supports a range of Pygments lexers for syntax highlighting to make our code examples look good. Each code-block must be correctly indented and surrounded by blank lines.

The Ansible documentation allows the following values:

For example, you can highlight Python code using following syntax:

.. code-block:: python

   def my_beautiful_python_code():
      pass

Internal navigation

Anchors (also called labels) and links work together to help users find related content. Local tables of contents also help users navigate quickly to the information they need. All internal links should use the :ref: syntax. Every page should have at least one anchor to support internal :ref: links. Long pages, or pages with multiple levels of headings, can also include a local TOC.

Avoid raw URLs. RST and sphinx allow https://my.example.com, but this is unhelpful for those using screen readers. :ref: links automatically pick up the heading from the anchor, but for external links, always use the \`link title <link-url>\`_ format.

Adding anchors

.. _unique_page::

:ref:`unique_page`
:ref:`this page <unique_page>`

The second example adds custom text for the link.

Use the :ansplugin: RST role to link to modules and plugins using their Fully Qualified Collection Name (FQCN):

The ansible.builtin.copy module can be linked with
:ansplugin:`ansible.builtin.copy#module`

If you want to specify an explicit type, use:
:ansplugin:`the copy module <ansible.builtin.copy#module>`

This displays as "The ansible.builtin.copy module can be linked with ansible.builtin.copy#module" and "If you want to specify an explicit type, use: the copy module <ansible.builtin.copy#module>".

Instead of #module, you can also specify #<plugin_type> to reference to a plugin of type <plugin_type>:

:ansplugin:`arista.eos.eos_config <arista.eos.eos_config#module>`
:ansplugin:`kubernetes.core.kubectl connection plugin <kubernetes.core.kubectl#connection>`
:ansplugin:`ansible.builtin.file lookup plugin <ansible.builtin.file#lookup>`

ansible.builtin is the FQCN for modules included in ansible-core.

Use the :ansopt: and :ansretval: roles to reference options and return values of modules and plugins while showing the option's resp. return value's name and optionally a value. Use the :ansoptref: and :ansretvalref: roles to reference options and return values of modules and plugins while displaying a provided title.

The following example shows their usage:

:ansopt:`ansible.builtin.file#module:path` references the ``path`` parameter of the
``ansible.builtin.file`` module; :ansopt:`ansible.builtin.file#module:path=/root/.ssh/known_hosts`
shows the assignment ``path=/root/.ssh/known_hosts`` as a clickable link.

You can :ansoptref:`provide a path <ansible.builtin.file#module:path>`
to the :ansplugin:`ansible.builtin.file#module`; its value is
:ansretvalref:`returned as a return value <ansible.builtin.file#module:path>`.

:ansretval:`ansible.builtin.stat#module:stat.exists` references the ``stat.exists`` return value
of the ``ansible.builtin.stat`` module. You can also use ``=`` as for option values:
:ansretval:`ansible.builtin.stat#module:stat.exists=true` shows ``stat.exists=true``.

:ansopt:`foo` and :ansopt:`foo=bar` use the same markup for an option and an option
assignment without a link; the same is true for return values: :ansretval:`foo` and
:ansretval:`foo=bar`.

This displays as:

ansible.builtin.file#module:path references the path parameter of the ansible.builtin.file module; ansible.builtin.file#module:path=/root/.ssh/known_hosts shows the assignment path=/root/.ssh/known_hosts as a clickable link.

You can provide a path <ansible.builtin.file#module:path> to the ansible.builtin.file#module; its value is returned as a return value <ansible.builtin.file#module:path>.

ansible.builtin.stat#module:stat.exists references the stat.exists return value of the ansible.builtin.stat module. You can also use = as for option values: ansible.builtin.stat#module:stat.exists=true shows stat.exists=true.

foo and foo=bar use the same markup for an option and an option assignment without a link; the same is true for return values: foo and foo=bar.

For both option and return values, . is used to reference suboptions and contained return values. Array stubs ([...]) can be used to indicate that something is a list, for example the :ansretval: argument ansible.builtin.service_facts#module:ansible_facts.services['systemd'].state references the ansible_facts.services.state return value of the ansible.builtin.service_facts module (ansible.builtin.service_facts#module:ansible_facts.services['systemd'].state).

Adding local TOCs

The page you're reading includes a local TOC. If you include a local TOC:

The syntax is:

.. contents::
   :local:

Markdown guidelines

Some Ansible ecosystem documentation is written in markdown and processed by mkdocs. We follow these technical or mechanical guidelines on all .md pages:

Heading notation

Section headings in markdown can use a variety of notations. To make our documents easy to read and to edit, we follow a standard set of heading notations. We use:

# Installation

## Installing on Linux

Subsections add an additional # for each subsection. We recommend not going beyond #### as that suggests a deeply nested document that could present better as multiple pages.

Linking in Markdown

Using Mkdocs, you can format internal links` using the file name of the local file instead of an external URL.

[configuration](/configuration)

[dependency](/configuration/#dependency)

[Ansible Documentation](https://docs.ansible.com)

Code blocks

Markdown supports code blocks in the following format.

```text
docs/
    index.md
    user-guide/getting-started.md
    user-guide/configuration-options.md
    license.md
```

Accessibility guidelines

Ansible documentation has a goal to be more accessible. Use the following guidelines to help us reach this goal.

Images and alternative text

Ensure all icons, images, diagrams, and non text elements have a meaningful alternative-text description. Do not include screen captures of CLI output. Use a code block instead.

To add alt text in rst:

.. image:: path/networkdiag.png
   :width: 400
   :alt: SpiffyCorp network diagram

To add alt text in md:

![SpiffyCorp network diagram](path/networkdiag.png)

URLs and cross-reference links have descriptive text that conveys information about the content of the linked target. See style_links for how to format links in RST and see style_links_md for Markdown.

Tables

Tables have a simple, logical reading order from left to right, and top to bottom. Tables include a heading row and avoid empty or blank table cells. Label tables with a descriptive title.

For RST:

.. table:: File descriptions

  +----------+----------------------------+
  |File      |Purpose                     |
  +==========+============================+
  |foo.txt   |foo configuration settings  |
  +----------+----------------------------+
  |bar.txt   |bar configuration settings  |
  +----------+----------------------------+

For Markdown:

#### File descriptions

  |File       |Purpose                     |
  |---------- | -------------------------- |
  |foo.txt    | foo configuration settings |
  |bar.txt    | bar configuration settings |

Colors and other visual information

Accessibility resources

Use the following resources to help test your documentation changes:

More resources

These pages offer more help with grammatical, stylistic, and technical rules for documentation.

basic_rules voice_style trademarks grammar_punctuation spelling_word_choice preferred_terms search_hints resources

community_documentation_contributions
How to contribute to the Ansible documentation

testing_documentation_locally
How to build the Ansible documentation

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

Preferred terminology

The following is the list of preferred terms for Ansible ecosystem projects.

Ansible
Use on all occurrences or ansible if referring to the CLI command. Preference is to use this to refer to the Ansible community package.

Ansible Builder
Use on first occurrence or ansible-builder if referring to the CLI command. Subsequent occurrences can be Builder.

Ansible collection
Use on first occurrence. Subsequent occurrences can be collection.

Ansible Content Development Kit (CDK)
Use on first occurrence. Subsequent occurrences can be Ansible CDK.

Ansible Core
Use on all occurrences, though preference is ansible-core after first occurrence to distinguish from the Ansible community package.

Ansible Creator
Use on first occurrence or ansible-creator if referring to the CLI command.

Ansible Execution Environment (EE)
Use on first occurrence. Subsequent occurrences can be Execution Environment or EE.

Ansible Galaxy
Use on first occurrence. Subsequent occurrences can be Galaxy. Do not use Galaxy NG.

Ansible Lint
Use on all occurrences or ansible-lint if referring to the CLI command.

Ansible Molecule
Use on first occurrence. Subsequent occurrences can be Molecule or molecule if referring to the CLI command.

Ansible Navigator
Use first occurrence. Subsequent occurrences can be Navigator or ansible-navigator if referring to the CLI.

Ansible playbook
Use on first occurrence. Subsequent occurrences can be playbook.

ansible-pylibssh
Use on all occurrences.

Ansible role
Use on first occurrence. Subsequent occurrences can be role.

Ansible rule
Use on first occurrence. Subsequent occurrences can be rule.

Ansible rulebook
Use on first occurrence. Subsequent occurrences can be rulebook.

Ansible Runner
Use on first occurrence or ansible-runner if referring to the CLI command. Subsequent occurrences can be Runner.

Ansible SDK
Use on all occurrences.

Ansible Sign
Use on all occurrences or ansible-sign if referring to the CLI command.

ansible-test
Use on all occurrences as the CLI command.

Ansible VSCode Extension by Red Hat
Use on first occurrence. Subsequent occurrences can be Ansible VSCode Extension.

Pinakes
Use on all occurrences.

Receptor
Use on all occurrences or receptor if referring to the CLI command.

Resources

Writing documentation so search can find it

One of the keys to writing good documentation is to make it findable. Readers use a combination of internal site search and external search engines such as Google or DuckDuckGo.

To ensure Ansible documentation is findable, you should:

  1. Use headings that clearly reflect what you are documenting.
  2. Use numbered lists for procedures or high-level steps where possible.
  3. Avoid linking to GitHub blobs where possible.

Using clear headings in documentation

We all use simple English when we want to find something. For example, the title of this page could have been any one of the following:

What we are really trying to describe is - how do I write documentation so search engines can find my content? That simple phrase is what drove the title of this section. When you are creating your headings for documentation, spend some time to think about what you would type in a search box to find it, or more importantly, how someone less familiar with Ansible would try to find that information. Your heading should be the answer to that question.

One word of caution - you do want to limit the size of your headings. A full heading such as How do I write documentation so search engines can find my content? is too long. Search engines would truncate anything over 50 - 60 characters. Long headings would also wrap on smaller devices such as a smart phone.

Using numbered lists for zero position snippets

Google can optimize the search results by adding a feature snippet at the top of the search results. This snippet provides a small window into the documentation on that first search result that adds more detail than the rest of the search results, and can occasionally answer the reader's questions right there, or at least verify that the linked page is what the reader is looking for.

Google returns the feature snippet in the form of numbered steps. Where possible, you should add a numbered list near the top of your documentation page, where appropriate. The steps can be the exact procedure a reader would follow, or could be a high level introduction to the documentation topic, such as the numbered list at the top of this page.

Problems with GitHub blobs on search results

Search engines do not typically return GitHub blobs in search results, at least not in higher ranked positions. While it is possible and sometimes necessary to link to GitHub blobs from documentation, the better approach would be to copy that information into an .rst page in Ansible documentation.

Other search hints

While it may not be possible to adapt your documentation to all search optimizations, keep the following in mind as you write your documentation:

Spelling - Word Usage - Common Words and Phrases to Use and Avoid

Acronyms

Always uppercase. An acronym is a word formed from the initial letters of a name, such as ROM for Read-only memory, SaaS for Software as a Service, or by combining initial letters or part of a series of words, such as LILO for LInux LOader.

Spell out the acronym before using it in alone text, such as "The Embedded DevKit (EDK)..."

Do not define abbreviations that appear only a few times. Spell out the full term each time instead.

Allowlist/Denylist

Use "allowlist" and "denylist." Do not use "whitelist" or "blacklist."

Applications

When used as a proper name, use the capitalization of the product, such as GNUPro or Source-Navigator. When used as a command, use lowercase as appropriate, such as "To start GCC, type gcc."

"vi" is always lowercase.

As

This is often used to mean "because", but has other connotations, for example, parallel or simultaneous actions. If you mean "because", say "because".

Asks for

Use "requests" instead.

Assure/Ensure/Insure

Assure implies a sort of mental comfort. As in "I assured my husband that I would eventually bring home beer."

Ensure means "to make sure."

Insure relates to monetary insurance.

Back up

This is a verb. You "back up" files; you do not "backup" files.

Backup

This is a noun. You create "backup" files; you do not create "back up" files.

Backward

Correct. Avoid using backwards unless you are stating that something has "backwards compatibility."

Backwards compatibility

Correct as is.

By way of

Use "using" instead.

Can/May

Use "can" to describe actions or conditions that are possible. Use "may" only to describe situations where permission is being given. If either "can," "could," or "may" apply, use "can" because it is less tentative.

CD or cd

When referring to a compact disk, use CD, such as "Insert the CD into the CD-ROM drive." When referring to the change directory command, use cd.

CD-ROM

Correct. Do not use "cdrom," "CD-Rom," "CDROM," "cd-rom" or any other variation. When referring to the drive, use CD-ROM drive, such as "Insert the CD into the CD-ROM drive." The plural is "CD-ROMs."

Command line

Correct. Do not use "command-line" or "commandline" as a noun. If used as an adjective, "command-line" is appropriate, for example "command-line arguments".

Use "command line" to describes where to place options for a command, but not where to type the command. Use "shell prompt" instead to describe where to type commands. The line on the display screen where a command is expected. Generally, the command line is the line that contains the most recently displayed command prompt.

Daylight saving time (DST)

Correct. Do not use daylight savings time. Daylight Saving Time (DST) is often misspelled "Daylight Savings", with an "s" at the end. Other common variations are "Summer Time"and "Daylight-Saving Time". (https://www.timeanddate.com/time/dst/daylight-savings-time.html)

Download

Correct. Do not use "down load" or "down-load."

e.g.

Spell it out: "For example."

Failover

When used as a noun, a failover is a backup operation that automatically switches to a standby database, server or network if the primary system fails or is temporarily shut down for servicing. Failover is an important fault tolerance function of mission-critical systems that rely on constant accessibility. Failover automatically and transparently to the user redirects requests from the failed or down system to the backup system that mimics the operations of the primary system.

Fail over

When used as a verb, fail over is two words since there can be different tenses such as failed over.

Fewer

Fewer is used with plural nouns. Think things you could count. Time, money, distance, and weight are often listed as exceptions to the traditional "can you count it" rule, often thought of a singular amounts (the work will take less than 5 hours, for example).

File name

Correct. Do not use "filename."

File extensions

Include a period before file name extensions (for example, .yml).

File system

Correct. Do not use "filesystem." The system that an operating system or program uses to organize and keep track of files. For example, a hierarchical file system is one that uses directories to organize files into a tree structure. Although the operating system provides its own file management system, you can buy separate file management systems. These systems interact smoothly with the operating system but provide more features, such as improved backup procedures and stricter file protection.

For instance

For example," instead.

For further/additional/whatever information

Use "For more information"

For this reason

Use "therefore".

Forward

Correct. Avoid using "forwards."

Gigabyte (GB)

2 to the 30th power (1,073,741,824) bytes. One gigabyte is equal to 1,024 megabytes. Gigabyte is often abbreviated as G or GB.

Got

Avoid. Use "must" instead.

High-availability

Correct. Do not use "high availability."

Highly available

Correct. Do not use highly-available."

Hostname

Correct. Do not use host name.

ID

Use ID (all uppercase) when referring to "Identification" or "Identifier."

i.e.

Spell it out: "That is."

Installer

Avoid. Use "installation program" instead.

It's and its

"It's" is a contraction for "it is;" use "it is" instead of "it's." Use "its" as a possessive pronoun (for example, "the store is known for its low prices").

Less

Less is used with singular nouns. For example "View less details" wouldn't be correct but "View less detail" works. Use fewer when you have plural nouns (things you can count).

Linux

Correct. Do not use "LINUX" or "linux" unless referring to a command, such as "To start Linux, type linux." Linux is a registered trademark of Linus Torvalds.

Login

A noun used to refer to the login prompt, such as "At the login prompt, enter your username."

Log in

A verb used to refer to the act of logging in. Do not use "login," "loggin," "logon," and other variants. For example, "When starting your computer, you are requested to log in..."

Log on

To make a computer system or network recognize you so that you can begin a computer session. Most personal computers have no log-on procedure -- you just turn the machine on and begin working. For larger systems and networks, however, you usually need to enter a username and password before the computer system will allow you to execute programs.

Lots of

Use "Several" or something equivalent instead.

Make sure

This means "be careful to remember, attend to, or find out something." For example, "...make sure that the rhedk group is listed in the output." Try to use verify or ensure instead.

Manual/man page

Correct. Two words. Do not use "manpage"

MB

  1. When spelled MB, short for megabyte (1,000,000 or 1,048,576 bytes, depending on the context).
  2. When spelled Mb, short for megabit.

MBps

Short for megabytes per second, a measure of data transfer speed. Mass storage devices are generally measured in MBps.

MySQL

Common open source database server and client package. Do not use "MYSQL" or "mySQL."

Need to

Avoid. Use "must" instead.

Once

Use "once" to mean "one time." Do not use "once" as a conjunction meaning "after" or "when."

Primary/Replica

Use "primary" and "replica" or similar alternatives instead of "master" and "slave."

Read-only

Correct. Use when referring to the access permissions of files or directories.

Real time/real-time

Depends. If used as a noun, it is the actual time during which something takes place. For example, "The computer may partly analyze the data in real time (as it comes in) -- R. H. March." If used as an adjective, "real-time" is appropriate. For example, "XEmacs is a self-documenting, customizable, extensible, real-time display editor."

See vs Refer to

Use "See" instead of "Refer to" to indicate a reference (within a manual or website) or a cross-reference (to another manual or documentation source).

Since

This is often used to mean "because", but "since" has connotations of time, so be careful. If you mean "because", say "because".

Tells

Use "Instructs" instead.

That/which

"That" introduces a restrictive clause-a clause that must be there for the sentence to make sense. A restrictive clause often defines the noun or phrase preceding it. "Which" introduces a non-restrictive, parenthetical clause-a clause that could be omitted without affecting the meaning of the sentence. For example: The car was travelling at a speed that would endanger lives. The car, which was traveling at a speed that would endanger lives, swerved onto the sidewalk. Use "who" or "whom," rather than "that" or "which," when referring to a person.

Then/than

"Then" refers to a time in the past or the next step in a sequence. "Than" is used for comparisons.

image

Third-party

Correct. Do not use "third party".

Troubleshoot

Correct. Do not use "trouble shoot" or "trouble-shoot." To isolate the source of a problem and fix it. In the case of computer systems, the term troubleshoot is usually used when the problem is suspected to be hardware -related. If the problem is known to be in software, the term debug is more commonly used.

UK

Correct as is, no periods.

UNIX®

Correct. Do not use "Unix" or "unix." UNIX® is a registered trademark of The Open Group.

Unset

Don't use. Use Clear.

US

Correct as is, no periods.

User

When referring to the reader, use "you" instead of "user." For example, "The user must..." is incorrect. Use "You must..." instead. If referring to more than one user, calling the collection "users" is acceptable, such as "Other users may wish to access your database."

Username

Correct. Do not use "user name."

View

When using as a reference ("View the documentation available online."), do not use View. Use "Refer to" instead.

Within

Don't use to refer to a file that exists in a directory. Use "In".

World Wide Web

Correct. Capitalize each word. Abbreviate as "WWW" or "Web."

Webpage

Correct. Do not use "web page" or "Web page."

Web server

Correct. Do not use "webserver". For example, "The Apache HTTP Server is the default Web server..."

Website

Correct. Do not use "web site" or "Web site." For example, "The Ansible website contains ..."

Who/whom

Use the pronoun "who" as a subject. Use the pronoun "whom" as a direct object, an indirect object, or the object of a preposition. For example: Who owns this? To whom does this belong?

While

Use "while" only to refer to a period of time. Do not use "while" as a synonym for "although."

Will

Do not use future tense unless it is absolutely necessary. For example, do not use the sentence, "The next section will describe the process in more detail." Instead, use the sentence, "The next section describes the process in more detail."

Wish

Use "need" instead of "desire" and "wish." Use "want" when the reader's actions are optional (that is, they may not "need" something but may still "want" something).

x86

Correct. Do not capitalize the "x."

x86_64

Do not use. Do not use "Hammer". Always use "AMD64 and Intel® EM64T" when referring to this architecture.

You

Correct. Do not use "I," "he," or "she."

You may

Try to avoid using this. For example, "you may" can be eliminated from this sentence "You may double-click on the desktop..."

Trademark Usage

Why is it important to use the TM, SM, and ® for our registered marks?

Before a trademark is registered with the United States Patent and Trademark Office it is appropriate to use the TM or SM symbol depending whether the product is for goods or services. It is important to use the TM or SM as it is notification to the public that Ansible claims rights to the mark even though it has not yet been registered.

Once the trademark is registered, it is appropriate to use the symbol in place of the TM or SM. The symbol designation must be used in conjunction with the trademark if Ansible is to fully protect its rights. If we don't protect these marks, we run the risk of losing them in the way of Aspirin or Trampoline or Escalator.

General Rules:

Trademarks should be used on 1st references on a page or within a section.

Use Red Hat® Ansible® Automation Platform or Ansible®, on first reference when referring to products.

Use "Ansible" alone as the company name, as in "Ansible announced quarterly results," which is not marked.

Also add the trademark disclaimer. * When using Ansible trademarks in the body of written text, you should use the following credit line in a prominent place, usually a footnote.

For Registered Trademarks: - [Name of Trademark] is a registered trademark of Red Hat, Inc. in the United States and other countries.

For Unregistered Trademarks (TMs/SMs): - [Name of Trademark] is a trademark of Red Hat, Inc. in the United States and other countries.

For registered and unregistered trademarks: - [Name of Trademark] is a registered trademark and [Name of Trademark] is a trademark of Red Hat, Inc. in the United States and other countries.

Guidelines for the proper use of trademarks:

Always distinguish trademarks from surround text with at least initial capital letters or in all capital letters.

Always use proper trademark form and spelling.

Never use a trademark as a noun. Always use a trademark as an adjective modifying the noun.

Correct: Red Hat® Ansible® Automation Platform system performance is incredible.

Incorrect: Ansible's performance is incredible.

Never use a trademark as a verb. Trademarks are products or services, never actions.

Correct: "Orchestrate your entire network using Red Hat® Ansible® Automation Platform."

Incorrect: "Ansible your entire network."

Never modify a trademark to a plural form. Instead, change the generic word from the singular to the plural.

Correct: "Corporate demand for Red Hat® Ansible® Automation Platform software is surging."

Incorrect: "Corporate demand for Ansible is surging."

Never modify a trademark from its possessive form, or make a trademark possessive. Always use it in the form it has been registered.

Never translate a trademark into another language.

Never use trademarks to coin new words or names.

Never use trademarks to create a play on words.

Never alter a trademark in any way including through unapproved fonts or visual identifiers.

Never abbreviate or use any Ansible trademarks as an acronym.

The importance of Ansible trademarks

The Ansible trademark and the "A" logo in a shaded circle are our most valuable assets. The value of these trademarks encompass the Ansible Brand. Effective trademark use is more than just a name, it defines the level of quality the customer will receive and it ties a product or service to a corporate image. A trademark may serve as the basis for many of our everyday decisions and choices. The Ansible Brand is about how we treat customers and each other. In order to continue to build a stronger more valuable Brand we must use it in a clear and consistent manner.

The mark consists of the letter "A" in a shaded circle. As of 5/11/15, this was a pending trademark (registration in process).

Common Ansible Trademarks

Other Common Trademarks and Resource Sites:

Voice Style

The essence of the Ansible writing style is short sentences that flow naturally together. Mix up sentence structures. Vary sentence subjects. Address the reader directly. Ask a question. And when the reader adjusts to the pace of shorter sentences, write a longer one.

Active Voice

Use the active voice ("Start Linuxconf by typing...") rather than passive ("Linuxconf can be started by typing...") whenever possible. Active voice makes for more lively, interesting reading. Also avoid future tense (or using the term "will") whenever possible For example, future tense ("The screen will display...") does not read as well as an active voice ("The screen displays"). Remember, the users you are writing for most often refer to the documentation while they are using the system, not after or in advance of using the system.

Sentence length

Aim for 32 words or fewer per sentence. Varying sentence length improves readability, but most sentences should stay concise.

Precise verbs

Prefer precise, specific verbs over weak verbs such as "is", "are", "occur", or "happen." Where possible, start sentences with a real subject instead of expletive constructions such as "There is", "There are", or "It is."

Prefer single-word verbs

Where a single-word verb exists, prefer it over a phrasal verb. For example, write "click the button" rather than "click on the button", and "omit the parameter" rather than "leave out the parameter."

Avoid terms of politeness

Do not use "please", "thank you", "kindly", or similar terms of politeness in technical content. Use direct imperative instructions instead. For example, write "Configure the inventory file before running the playbook" rather than "Please configure the inventory file before running the playbook."

Avoid anthropomorphism

Do not assign human qualities to software or components. Describe what the user does or what happens, not what the software "thinks", "knows", "wants", or "decides." For example, write "The module returns a list of packages to install" rather than "The module knows which packages to install."

Testing Ansible

testing_sanity testing_units testing_units_modules testing_integration testing_httptester testing_running_locally testing_documentation

Why test your Ansible contributions?

If you're a developer, one of the most valuable things you can do is to look at GitHub issues and help fix bugs, since bug-fixing is almost always prioritized over feature development. Even for non-developers, helping to test pull requests for bug fixes and features is still immensely valuable.

Ansible users who understand how to write playbooks and roles should be able to test their work. GitHub pull requests will automatically run a variety of tests (for example, Azure Pipelines) that show bugs in action. However, contributors must also test their work outside of the automated GitHub checks and show evidence of these tests in the pull request to ensure that their work will be more likely to be reviewed and merged.

Read on to learn how Ansible is tested, how to test your contributions locally, and how to extend testing capabilities.

If you want to learn about testing collections, read testing_collections

Types of tests

At a high level, we have the following classifications of tests:

sanity

integration

units

Testing within GitHub & Azure Pipelines

Organization

When Pull Requests (PRs) are created they are tested using Azure Pipelines, a Continuous Integration (CI) tool. Results are shown at the end of every PR.

When Azure Pipelines detects an error and it can be linked back to a file that has been modified in the PR then the relevant lines will be added as a GitHub comment. For example:

The test `ansible-test sanity --test pep8` failed with the following errors:

lib/ansible/modules/network/foo/bar.py:509:17: E265 block comment should start with '# '

The test `ansible-test sanity --test validate-modules` failed with the following error:
lib/ansible/modules/network/foo/bar.py:0:0: E307 version_added should be 2.4. Currently 2.3

From the above example we can see that --test pep8 and --test validate-modules have identified an issue. The commands given allow you to run the same tests locally to ensure you've fixed all issues without having to push your changes to GitHub and wait for Azure Pipelines, for example:

If you haven't already got Ansible available, use the local checkout by running:

source hacking/env-setup

Then run the tests detailed in the GitHub comment:

ansible-test sanity --test pep8
ansible-test sanity --test validate-modules

If there isn't a GitHub comment stating what's failed you can inspect the results by clicking on the "Details" button under the "checks have failed" message at the end of the PR.

Rerunning a failing CI job

Occasionally you may find your PR fails due to a reason unrelated to your change. This could happen for several reasons, including:

If either issue appears to be the case, you can rerun the Azure Pipelines test by:

If the issue persists, please contact the community. Visit the Ansible communication guide<communication> for details.

How to test a PR

Ideally, the code should add tests that prove that the code works. That's not always possible and tests are not always comprehensive, especially when a user doesn't have access to a wide variety of platforms, or is using an API or web service. In these cases, live testing against real equipment can be more valuable than automation that runs against simulated interfaces. In any case, things should always be tested manually the first time as well.

Thankfully, helping to test Ansible is pretty straightforward, assuming you are familiar with how Ansible works.

Setup: Installing Pytest and required Pytest libraries

Ansible's unit testing framework leverages the pytest library. Before diving into testing, ensure you have pytest installed alongside any additional pytest libraries such as pytest-mock and pytest-xdist.

Refer to the documentation for more information: testing_units.

Setup: Checking out a Pull Request

You can do this by:

Here's how:

Testing source code from GitHub pull requests sent to us does have some inherent risk, as the source code sent may have mistakes or malicious code that could have a negative impact on your system. We recommend doing all testing on a virtual machine, whether a cloud instance, or locally. Some users like Vagrant or Docker for this, but they are optional. It is also useful to have virtual machines of different Linux or other flavors, since some features (for example, package managers such as apt or yum) are specific to those OS versions.

Create a fresh area to work:

git clone https://github.com/ansible/ansible.git ansible-pr-testing
cd ansible-pr-testing

Next, find the pull request you'd like to test and make a note of its number. It will look something like this:

Use os.path.sep instead of hardcoding / #65381

Only test ansible:devel

It is important that the PR request target be ansible:devel, as we do not accept pull requests into any other branch. Dot releases are cherry-picked manually by Ansible staff.

Use the pull request number when you fetch the proposed changes and create your branch for testing:

git fetch origin refs/pull/XXXX/head:testing_PRXXXX
git checkout testing_PRXXXX

The first command fetches the proposed changes from the pull request and creates a new branch named testing_PRXXXX, where the XXXX is the actual number associated with the pull request (for example, 65381). The second command checks out the newly created branch.

If the GitHub user interface shows that the pull request will not merge cleanly, we do not recommend proceeding if you are not somewhat familiar with Git and coding, as you will have to resolve a merge conflict. This is the responsibility of the original pull request contributor.

Some users do not create feature branches, which can cause problems when they have multiple, unrelated commits in their version of devel. If the source looks like someuser:devel, make sure there is only one commit listed on the pull request.

The Ansible source includes a script that allows you to use Ansible directly from source without requiring a full installation that is frequently used by developers on Ansible.

Simply source it (to use the Linux/Unix terminology) to begin using it immediately:

source ./hacking/env-setup

This script modifies the PYTHONPATH environment variables (along with a few other things), which will be temporarily set as long as your shell session is open.

Testing the Pull Request

At this point, you should be ready to begin testing!

Some ideas of what to test are:

Run sanity tests

ansible-test sanity

More information: testing_sanity

Run unit tests

ansible-test units

More information: testing_units

Run integration tests

ansible-test integration -v ping

More information: testing_integration

Any potential issues should be added as comments on the pull request (and it is acceptable to comment if the feature works as well), remembering to include the output of ansible --version

Example:

Works for me! Tested on `Ansible 2.3.0`.  I verified this on CentOS 6.5 and also Ubuntu 14.04.

If the PR does not resolve the issue, or if you see any failures from the unit/integration tests, just include that output instead:

This change causes errors for me.

When I ran this Ubuntu 16.04 it failed with the following:

  ```
  some output
  StackTrace
  some other output
  ```

Code Coverage Online

The online code coverage reports is a good way to identify areas for testing improvement in Ansible. By following red colors you can drill down through the reports to find files that have no tests at all. Adding both integration and unit tests that show clearly how code should work, verify important Ansible functions and increase testing coverage in areas where there is none is a valuable way to help improve Ansible.

The code coverage reports only cover the devel branch of Ansible where new feature development takes place. Pull requests and new code will be missing from the codecov.io coverage reports so local reporting is needed. Most ansible-test commands allow you to collect code coverage, this is particularly useful to indicate where to extend testing. See testing_running_locally for more information.

Want to know more about testing?

If you'd like to know more about the plans for improving testing Ansible then why not join the Ansible community forum.

action-plugin-docs

Each action plugin should have a matching module of the same name to provide documentation.

ansible-doc

Verifies that ansible-doc can parse module documentation on all supported Python versions.

ansible-requirements

test/lib/ansible_test/_data/requirements/sanity.import-plugins.txt must be an identical copy of requirements.txt found in the project's root.

bin-symlinks

The bin/ directory in Ansible must contain only symbolic links to executable files. These files must reside in the lib/ansible/ or test/lib/ansible_test/ directories.

This is required to allow ansible-test to work with containers and remote hosts when running from an installed version of Ansible.

boilerplate

Python files should include the following boilerplate:

from __future__ import annotations

changelog

Basic linting of changelog fragments with antsibull-changelog lint.

One or more of the following sections are required:

New modules and plugins must not be included in changelog fragments.

See collection_changelogs for details.

compile

All Python source files must successfully compile using all supported Python versions.

The list of supported Python versions is dependent on the version of ansible-core that you are using. Make sure you consult the version of the documentation which matches your ansible-core version. You can find an overview for this and previous versions in ansible_core_support_matrix.

Control node code, including plugins in Ansible Collections, must support the following Python versions:

Code which runs on targets (modules and module_utils) must support all control node supported Python versions, as well as the additional Python versions supported only on targets:

Ansible Collections can be configured to support a subset of the target-only Python versions.

deprecated-config

DOCUMENTATION config is scheduled for removal

empty-init

The __init__.py files under the following directories must be empty. For some of these (modules and tests), __init__.py files with code won't be used. For others (module_utils), we want the possibility of using Python namespaces which an empty __init__.py will allow for.

future-import-boilerplate

Most Python files should include the following boilerplate at the top of the file, right after the comment header:

from __future__ import (absolute_import, division, print_function)

This uses Python 3 semantics for absolute versus relative imports, division, and print. By doing this, we can write code which is portable between Python 2 and Python 3 by following the Python 3 semantics.

absolute_import

When Python 2 encounters an import of a name in a file like import copy it attempts to load copy.py from the same directory as the file is in. This can cause problems if there is a python file of that name in the directory and also a python module in sys.path with that same name. In that case, Python 2 would load the one in the same directory and there would be no way to load the one on sys.path. Python 3 fixes this by making imports absolute by default. import copy will find copy.py from sys.path. If you want to import copy.py from the same directory, the code needs to be changed to perform a relative import: from . import copy.

division

In Python 2, the division operator (/) returns integer values when used with integers. If there was a remainder, this part would be left off (aka, floor division). In Python 3, the division operator (/) always returns a floating point number. Code that needs to calculate the integer portion of the quotient needs to switch to using the floor division operator (//) instead.

print_function

In Python 2, python:print is a keyword. In Python 3, python3:print is a function with different parameters. Using this __future__ allows using the Python 3 print semantics everywhere.

ignores

Sanity tests for individual files can be skipped, and specific errors can be ignored.

When to Ignore Errors

Sanity tests are designed to improve code quality and identify common issues with content. When issues are identified during development, those issues should be corrected.

As development of Ansible continues, sanity tests are expanded to detect issues that previous releases could not. To allow time for existing content to be updated to pass newer tests, ignore entries can be added. New content should not use ignores for existing sanity tests.

When code is fixed to resolve sanity test errors, any relevant ignores must also be removed. If the ignores are not removed, this will be reported as an unnecessary ignore error. This is intended to prevent future regressions due to the same error recurring after being fixed.

When to Skip Tests

Although rare, there are reasons for skipping a sanity test instead of ignoring the errors it reports.

If a sanity test results in a traceback when processing content, that error cannot be ignored. If this occurs, open a new bug report for the issue so it can be fixed. If the traceback occurs due to an issue with the content, that issue should be fixed. If the content is correct, the test will need to be skipped until the bug in the sanity test is fixed.

Caution should be used when skipping sanity tests instead of ignoring them. Since the test is skipped entirely, resolution of the issue will not be automatically detected. This will prevent prevent regression detection from working once the issue has been resolved. For this reason it is a good idea to periodically review skipped entries manually to verify they are required.

Ignore File Location

The location of the ignore file depends on the type of content being tested.

Ansible Collections

Since sanity tests change between Ansible releases, a separate ignore file is needed for each Ansible major release.

The file name is tests/sanity/ignore-X.Y.txt where X.Y is the Ansible release being used to test the collection.

Maintaining a separate file for each Ansible release allows a collection to pass tests for multiple versions of Ansible.

Ansible

When testing Ansible, all ignores are placed in the test/sanity/ignore.txt file.

Only a single file is needed because ansible-test is developed and released as a part of Ansible itself.

Ignore File Format

The ignore file contains one entry per line. Each line consists of two columns, separated by a single space. Comments may be added at the end of an entry, started with a hash (#) character, which can be proceeded by zero or more spaces. Blank and comment only lines are not allowed.

The first column specifies the file path that the entry applies to. File paths must be relative to the root of the content being tested. This is either the Ansible source or an Ansible collection. File paths cannot contain a space or the hash (#) character.

The second column specifies the sanity test that the entry applies to. This will be the name of the sanity test. If the sanity test is specific to a version of Python, the name will include a dash (-) and the relevant Python version. If the named test uses error codes then the error code to ignore must be appended to the name of the test, separated by a colon (:).

Below are some example ignore entries for an Ansible collection:

roles/my_role/files/my_script.sh shellcheck:SC2154 # ignore undefined variable
plugins/modules/my_module.py validate-modules:missing-gplv3-license # ignore license check
plugins/modules/my_module.py import-3.8 # needs update to support collections.abc on Python 3.8+

It is also possible to skip a sanity test for a specific file. This is done by adding !skip after the sanity test name in the second column. When this is done, no error code is included, even if the sanity test uses error codes.

Below are some example skip entries for an Ansible collection:

plugins/module_utils/my_util.py validate-modules!skip # waiting for bug fix in module validator
plugins/lookup/my_plugin.py compile-2.6!skip # Python 2.6 is not supported on the control node

See the full list of sanity tests <all_sanity_tests>, which details the various tests and details how to fix identified issues.

Ignore File Errors

There are various errors that can be reported for the ignore file itself:

import

Ansible allows unchecked imports<allowed_unchecked_imports> of some libraries from specific directories. Importing any other Python library requires handling import errors<handling_import_errors>. This enables support for sanity tests such as testing_validate-modules and provides better error messages to the user.

Please see the frequently asked questions<frequently_asked_questions> for answers to common questions about resolving errors reported by this test.

Handling import errors

Ansible executes across many hosts and can use multiple Python interpreters at the same time, which may even have different versions. To ensure users get an actionable and easy to understand error we try to ensure any non-core imports in modules/plugins are guarded to avoid a traceback, which most users won't be able to understand, much less use to solve the issue.

Another reason Ansible does this is to import the code for inspection. This allows Ansible to easily test, document, configure, etc based on the code without having to install any and all requirements everywhere, especially when that is not the context in which you execute the code.

The code below shows examples of how to avoid errors on import and then use the provided missing_required_lib to ensure the user knows which LIBRARY is missing, on which HOST it is missing and the specific INTERPRETER that requires it.

In modules

Instead of using import another_library:

import traceback

from ansible.module_utils.basic import missing_required_lib

try:
    import another_library
except ImportError:
    HAS_ANOTHER_LIBRARY = False
    ANOTHER_LIBRARY_IMPORT_ERROR = traceback.format_exc()
else:
    HAS_ANOTHER_LIBRARY = True
    ANOTHER_LIBRARY_IMPORT_ERROR = None

The missing_required_lib import above will be used below.

Then in the module code, normally inside the main method:

module = AnsibleModule(...)

if not HAS_ANOTHER_LIBRARY:
    module.fail_json(
        msg=missing_required_lib('another_library'),
        exception=ANOTHER_LIBRARY_IMPORT_ERROR)

In plugins

Instead of using import another_library:

try:
    import another_library
except ImportError as imp_exc:
    ANOTHER_LIBRARY_IMPORT_ERROR = imp_exc
else:
    ANOTHER_LIBRARY_IMPORT_ERROR = None

Then in the plugin code, for example in the run method of the plugin (some plugins don't have a run method and will require it in the __init__ method instead):

if ANOTHER_LIBRARY_IMPORT_ERROR:
    raise AnsibleError('another_library must be installed to use this plugin') from ANOTHER_LIBRARY_IMPORT_ERROR

When used as base classes

This solution builds on the previous two examples. Make sure to pick the appropriate one before continuing with this solution.

Sometimes an import is used in a base class, for example:

from another_library import UsefulThing

class CustomThing(UsefulThing):
    pass

One option is make the entire class definition conditional:

if not ANOTHER_LIBRARY_IMPORT_ERROR:
    class CustomThing(UsefulThing):
        pass

Another option is to define a substitute base class by modifying the exception handler:

try:
    from another_library import UsefulThing
except ImportError:
    class UsefulThing:
        pass
    ...

Allowed unchecked imports

Ansible allows the following unchecked imports from these specific directories:

Public dependencies of ansible-core are:

Frequently asked questions (FAQ)

Why do I get an ImportError when my module or plugin works?

The import sanity test is very different from other tests. By design, it can only see modules that are in the Python standard library. This means your module or plugin can work in your playbooks, integration tests and unit tests, while failing the import sanity test.

All occurrences of ImportError must be properly handled<handling_import_errors> in the module or plugin where it occurs.

Why isn't the test using the Python interpreter I specify?

The sanity test will use the Python interpreter you specify. However, it will create its own virtual environment using that Python interpreter.

All occurrences of ImportError must be properly handled<handling_import_errors> in the module or plugin where it occurs.

How do I use a custom virtual environment?

This is not possible, since doing so would defeat the purpose of the test.

All occurrences of ImportError must be properly handled<handling_import_errors> in the module or plugin where it occurs.

How do I specify where to find my imports?

This is not possible, since doing so would defeat the purpose of the test.

All occurrences of ImportError must be properly handled<handling_import_errors> in the module or plugin where it occurs.

How do I specify which requirements file to use?

This is not possible, since doing so would defeat the purpose of the test.

All occurrences of ImportError must be properly handled<handling_import_errors> in the module or plugin where it occurs.

How do I fix the ImportError without changing my code?

There is no way to fix an ImportError in your plugins or modules without making changes to them.

All occurrences of ImportError must be properly handled<handling_import_errors> in the module or plugin where it occurs.

Sanity Tests

The following sanity tests are available as --test options for ansible-test sanity when testing Ansible Collections. This list is also available using ansible-test sanity --list-tests --allow-disabled.

For information on how to run these tests, see sanity testing guide <testing_sanity>.

action-plugin-docs ansible-doc changelog compile deprecated-config future-import-boilerplate empty-init ignores import line-endings no-assert no-basestring no-dict-iteritems no-dict-iterkeys no-dict-itervalues no-get-exception no-illegal-filenames no-main-display no-smart-quotes no-unicode-literals pep8 pslint pylint replace-urlopen runtime-metadata shebang shellcheck symlinks update-bundled use-argspec-type-path use-compat-six validate-modules yamllint

Additional tests are available when testing Ansible Core:

ansible-requirements bin-symlinks boilerplate integration-aliases mypy no-unwanted-files obsolete-files package-data pymarkdown release-names required-and-default-attributes test-constraints

integration-aliases

Integration tests are executed by ansible-test and reside in directories under test/integration/targets/. Each test MUST have an aliases file to control test execution.

Aliases are explained in the following sections. Each alias must be on a separate line in an aliases file.

Groups

Tests must be configured to run in exactly one group. This is done by adding the appropriate group to the aliases file.

The following are examples of some of the available groups:

Groups are used to balance tests across multiple CI jobs to minimize test run time. They also improve efficiency by keeping tests with similar requirements running together.

When selecting a group for a new test, use the same group as existing tests similar to the one being added. If more than one group is available, select one randomly.

Setup

Aliases can be used to execute setup targets before running tests:

Requirements

Aliases can be used to express some test requirements:

Dependencies

Some test dependencies are automatically discovered:

Aliases can be used to declare dependencies that are not handled automatically:

Skipping

Aliases can be used to skip platforms using one of the following:

Platform versions, as specified using the --remote option with / removed, can also be skipped:

Windows versions, as specified using the --windows option can also be skipped:

Aliases can be used to skip Python major versions using one of the following:

For more fine grained skipping, use conditionals in integration test playbooks, such as:

when: ansible_distribution in ('Ubuntu')

Miscellaneous

There are several other aliases available as well:

Unstable

Tests which fail sometimes should be marked with the unstable alias until the instability has been fixed. These tests will continue to run for pull requests which modify the test or the module under test.

This avoids unnecessary test failures for other pull requests, as well as tests on merge runs and nightly CI jobs.

There are two ways to run unstable tests manually:

Tests will be marked as unstable by a member of the Ansible Core Team. GitHub issues will be created to track each unstable test.

Disabled

Tests which always fail should be marked with the disabled alias until they can be fixed.

Disabled tests are automatically skipped.

There are two ways to run disabled tests manually:

Tests will be marked as disabled by a member of the Ansible Core Team. GitHub issues will be created to track each disabled test.

Unsupported

Tests which cannot be run in CI should be marked with the unsupported alias. Most tests can be supported through the use of simulators and/or cloud plugins.

However, if that is not possible then marking a test as unsupported will prevent it from running in CI.

There are two ways to run unsupported tests manually:

Tests will be marked as unsupported by the contributor of the test.

Cloud

Tests for cloud services and other modules that require access to external APIs usually require special support for testing in CI.

These require an additional alias to indicate the required test plugin.

Some of the available aliases are:

Untested

Every module and plugin should have integration tests, even if the tests cannot be run in CI.

Issues

Tests that are marked as unstable or disabled will have an issue created to track the status of the test. Each issue will be assigned to one of the following projects:

Questions

For questions about integration tests reach out to the community on the Ansible Forum<ansible_forum> by creating a topic tagged with ansible-test, testing and other appropriate tags.

line-endings

All files must use \n for line endings instead of \r\n.

mypy

The mypy static type checker is used to check the following code against each Python version supported by the control node:

Additionally, the following code is checked against Python versions supported only on managed nodes:

See the mypy documentation

no-assert

Do not use assert in production Ansible python code. When running Python with optimizations, Python will remove assert statements, potentially allowing for unexpected behavior throughout the Ansible code base.

Instead of using assert you should utilize simple if statements, that result in raising an exception. There is a new exception called AnsibleAssertionError that inherits from AnsibleError and AssertionError. When possible, utilize a more specific exception than AnsibleAssertionError.

Modules will not have access to AnsibleAssertionError and should instead raise AssertionError, a more specific exception, or just use module.fail_json at the failure point.

no-basestring

Do not use isinstance(s, basestring) as basestring has been removed in Python3. You can import string_types, binary_type, or text_type from ansible.module_utils.six and then use isinstance(s, string_types) or isinstance(s, (binary_type, text_type)) instead.

If this is part of code to convert a string to a particular type, ansible.module_utils.common.text.converters contains several functions that may be even better for you: to_text, to_bytes, and to_native.

no-dict-iteritems

The dict.iteritems method has been removed in Python 3. There are two recommended alternatives:

for KEY, VALUE in DICT.items():
   pass

from ansible.module_utils.six import iteritems

for KEY, VALUE in iteritems(DICT):
    pass

no-dict-iterkeys

The dict.iterkeys method has been removed in Python 3. Use the following instead:

for KEY in DICT:
    pass

no-dict-itervalues

The dict.itervalues method has been removed in Python 3. There are two recommended alternatives:

for VALUE in DICT.values():
   pass

from ansible.module_utils.six import itervalues

for VALUE in itervalues(DICT):
    pass

no-get-exception

We created a function, ansible.module_utils.pycompat24.get_exception to help retrieve exceptions in a manner compatible with Python 2.4 through Python 3.6. We no longer support Python 2.4 and Python 2.5 so this is extraneous and we want to deprecate the function. Porting code should look something like this:

# Unfixed code:
try:
    raise IOError('test')
except IOError:
    e = get_exception()
    do_something(e)
except:
    e = get_exception()
    do_something_else(e)

# After fixing:
try:
    raise IOError('test')
except IOErrors as e:
    do_something(e)
except Exception as e:
    do_something_else(e)

no-illegal-filenames

Files and directories should not contain illegal characters or names so that Ansible can be checked out on any Operating System.

Illegal Characters

The following characters are not allowed to be used in any part of the file or directory name;

The following characters are not allowed to be used as the last character of a file or directory;

Illegal Names

The following names are not allowed to be used as the name of a file or directory excluding the extension;

For example, the file folder/COM1, folder/COM1.txt are illegal but folder/COM1-file or folder/COM1-file.txt is allowed.

no-main-display

As of Ansible 2.8, Display should no longer be imported from __main__.

Display is now a singleton and should be utilized like the following:

from ansible.utils.display import Display
display = Display()

There is no longer a need to attempt from __main__ import display inside a try/except block.

no-smart-quotes

Smart quotes (”“‘’) should not be used. Use plain ascii quotes ("') instead.

no-tests-as-filters

orphan

no-tests-as-filters

Using Ansible provided Jinja2 tests as filters will be removed in Ansible 2.9.

Prior to Ansible 2.5, Jinja2 tests included within Ansible were most often used as filters. The large difference in use is that filters are referenced as variable | filter_name while Jinja2 tests are referenced as variable is test_name.

Jinja2 tests are used for comparisons, whereas filters are used for data manipulation, and have different applications in Jinja2. This change is to help differentiate the concepts for a better understanding of Jinja2, and where each can be appropriately used.

As of Ansible 2.5 using an Ansible provided Jinja2 test with filter syntax will display a deprecation error.

no-unicode-literals

The use of from __future__ import unicode_literals has been deemed an anti-pattern. The problems with it are:

no-unwanted-files

Specific file types are allowed in certain directories:

obsolete-files

Directories in the Ansible source tree are sometimes made obsolete. Files should not exist in these directories. The new location (if any) is dependent on which directory has been made obsolete.

Below are some of the obsolete directories and their new locations:

This error occurs most frequently for open pull requests which add or modify files in directories which are now obsolete. Make sure the branch you are working from is current so that changes can be made in the correct location.

package-data

Verifies the contents of the built ansible-core sdist and wheel.

pep8

Python static analysis for PEP 8 style guideline compliance.

PEP 8 style guidelines are enforced by pycodestyle on all python files in the repository by default.

Running locally

The PEP 8 check can be run locally as follows:

ansible-test sanity --test pep8 [file-or-directory-path-to-check] ...

pslint

PowerShell static analysis for common programming errors using PSScriptAnalyzer.

pylint

Python static analysis for common programming errors.

pymarkdown

Check Markdown files for syntax and formatting issues.

release-names

Verifies that the most recent release name has been added to ./github/RELEASE_NAMES.yml

replace-urlopen

Use open_url from module_utils instead of urlopen.

required-and-default-attributes

Use only one of default or required with FieldAttribute.

runtime-metadata

Validates the schema for:

shebang

Most executable files should only use one of the following shebangs:

This does not apply to Ansible modules, which should not be executable and must always use #!/usr/bin/python.

Some exceptions are permitted. Ask if you have questions.

shellcheck

Static code analysis for shell scripts using the excellent shellcheck tool.

symlinks

If other types of symlinks are needed for tests they must be created as part of the test.

test-constraints

Constraints for test requirements should be in test/lib/ansible_test/_data/requirements/constraints.txt.

update-bundled

Check whether any of our known bundled code needs to be updated for a new upstream release.

This test can error in the following ways:

BUNDLED_METADATA has the following fields:

pypi_name
Name of the bundled package on pypi

version
Version of the package that we are including here

version_constraints
Optional PEP440 specifier for the version range that we are bundling. Currently, the only valid use of this is to follow a version that is compatible with the Python stdlib when newer versions of the pypi package implement a new API.

use-argspec-type-path

The AnsibleModule argument_spec knows of several types beyond the standard python types. One of these is path. When used, type path ensures that an argument is a string and expands any shell variables and tilde characters.

This test looks for use of os.path.expanduser <python:os.path.expanduser> in modules. When found, it tells the user to replace it with type='path' in the module's argument_spec or list it as a false positive in the test.

use-compat-six

Use six from module_utils instead of six.

validate-modules

Analyze modules for common issues in code and documentation.

Usage

cd /path/to/ansible/source
source hacking/env-setup
ansible-test sanity --test validate-modules

Help

Type ansible-test sanity --test validate-modules -h to display help for using this sanity test.

Extending validate-modules

The validate-modules tool has a schema.py that is used to validate the YAML blocks, such as DOCUMENTATION and RETURNS.

Codes
Error Code Type Level Sample Message
ansible-deprecated-module Documentation Error A module is deprecated and supposed to be removed in the current or an earlier Ansible version
collection-deprecated-module Documentation Error A module is deprecated and supposed to be removed in the current or an earlier collection version
ansible-deprecated-version Documentation Error A feature is deprecated and supposed to be removed in the current or an earlier Ansible version
ansible-module-not-initialized Syntax Error Execution of the module did not result in initialization of AnsibleModule
collection-deprecated-version Documentation Error A feature is deprecated and supposed to be removed in the current or an earlier collection version
deprecated-date Documentation Error A date before today appears as removed_at_date or in deprecated_aliases
deprecation-mismatch Documentation Error Module marked as deprecated or removed in at least one of the file name, its metadata, or in DOCUMENTATION (setting DOCUMENTATION.deprecated for deprecation or removing all Documentation for removed) but not in all three places.
doc-choices-do-not-match-spec Documentation Error Value for "choices" from the argument_spec does not match the documentation
doc-choices-incompatible-type Documentation Error Choices value from the documentation is not compatible with type defined in the argument_spec
doc-default-does-not-match-spec Documentation Error Value for "default" from the argument_spec does not match the documentation
doc-default-incompatible-type Documentation Error Default value from the documentation is not compatible with type defined in the argument_spec
doc-elements-invalid Documentation Error Documentation specifies elements for argument, when "type" is not list.
doc-elements-mismatch Documentation Error Argument_spec defines elements different than documentation does
doc-missing-type Documentation Error Documentation doesn't specify a type but argument in argument_spec use default type (str)
doc-required-mismatch Documentation Error argument in argument_spec is required but documentation says it is not, or vice versa
doc-type-does-not-match-spec Documentation Error Argument_spec defines type different than documentation does
documentation-error Documentation Error Unknown DOCUMENTATION error
documentation-syntax-error Documentation Error Invalid DOCUMENTATION schema
import-before-documentation Imports Error Import found before documentation variables. All imports must appear below DOCUMENTATION/EXAMPLES/RETURN
import-error Documentation Error Exception attempting to import module for argument_spec introspection
attributes-check-mode Documentation Error If a module documents the check_mode attribute, its support value must be compatible with the supports_check_mode parameter of AnsibleModule
attributes-check-mode-details Documentation Error If a module documents the check_mode attribute with support values partial or N/A, it must provide details
import-placement Locations Warning Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN
imports-improper-location Imports Error Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN
incompatible-choices Documentation Error Choices value from the argument_spec is not compatible with type defined in the argument_spec
incompatible-default-type Documentation Error Default value from the argument_spec is not compatible with type defined in the argument_spec
invalid-argument-name Documentation Error Argument in argument_spec must not be one of 'message', 'syslog_facility' as it is used internally by Ansible Core Engine
invalid-argument-spec Documentation Error Argument in argument_spec must be a dictionary/hash when used
invalid-argument-spec-options Documentation Error Suboptions in argument_spec are invalid
invalid-documentation Documentation Error DOCUMENTATION is not valid YAML
invalid-documentation-markup Documentation Error DOCUMENTATION or RETURN contains invalid markup
invalid-documentation-options Documentation Error DOCUMENTATION.options must be a dictionary/hash when used
invalid-examples Documentation Error EXAMPLES is not valid YAML
invalid-extension Naming Error Official Ansible modules must have a .py extension for python modules or a .ps1 for powershell modules
invalid-module-schema Documentation Error AnsibleModule schema validation error
invalid-removal-version Documentation Error The version at which a feature is supposed to be removed cannot be parsed (for collections, it must be a semantic version)
invalid-requires-extension Naming Error Module #AnsibleRequires -CSharpUtil should not end in .cs, Module #Requires should not end in .psm1
missing-doc-fragment Documentation Error DOCUMENTATION fragment missing
missing-existing-doc-fragment Documentation Warning Pre-existing DOCUMENTATION fragment missing
missing-documentation Documentation Error No DOCUMENTATION provided
missing-examples Documentation Error No EXAMPLES provided
missing-gplv3-license Documentation Error GPLv3 license header not found
missing-module-utils-basic-import Imports Warning Did not find ansible.module_utils.basic import
missing-module-utils-import-csharp-requirements Imports Error No Ansible.ModuleUtils or C# Ansible util requirements/imports found
missing-powershell-interpreter Syntax Error Interpreter line is not #!powershell
missing-python-interpreter Syntax Error Interpreter line is not #!/usr/bin/python
missing-return Documentation Error No RETURN documentation provided
missing-return-legacy Documentation Warning No RETURN documentation provided for legacy module
missing-suboption-docs Documentation Error Argument in argument_spec has sub-options but documentation does not define sub-options
module-incorrect-version-added Documentation Error Module level version_added is incorrect
module-invalid-version-added Documentation Error Module level version_added is not a valid version number
module-utils-specific-import Imports Error module_utils imports should import specific components, not *
multiple-utils-per-requires Imports Error Ansible.ModuleUtils requirements do not support multiple modules per statement
multiple-csharp-utils-per-requires Imports Error Ansible C# util requirements do not support multiple utils per statement
no-default-for-required-parameter Documentation Error Option is marked as required but specifies a default. Arguments with a default should not be marked as required
no-log-needed Parameters Error Option name suggests that the option contains a secret value, while no_log is not specified for this option in the argument spec. If this is a false positive, explicitly set no_log=False
nonexistent-parameter-documented Documentation Error Argument is listed in DOCUMENTATION.options, but not accepted by the module
option-incorrect-version-added Documentation Error version_added for new option is incorrect
option-invalid-version-added Documentation Error version_added for option is not a valid version number
parameter-invalid Documentation Error Argument in argument_spec is not a valid python identifier
parameter-invalid-elements Documentation Error Value for "elements" is valid only when value of "type" is list
implied-parameter-type-mismatch Documentation Error Argument_spec implies type="str" but documentation defines it as different data type
parameter-type-not-in-doc Documentation Error Type value is defined in argument_spec but documentation doesn't specify a type
parameter-alias-repeated Parameters Error argument in argument_spec has at least one alias specified multiple times in aliases
parameter-alias-self Parameters Error argument in argument_spec is specified as its own alias
parameter-documented-multiple-times Documentation Error argument in argument_spec with aliases is documented multiple times
parameter-list-no-elements Parameters Error argument in argument_spec "type" is specified as list without defining "elements"
parameter-state-invalid-choice Parameters Error Argument state includes get, list or info as a choice. Functionality should be in an _info or (if further conditions apply) _facts module.
python-syntax-error Syntax Error Python SyntaxError while parsing module
removal-version-must-be-major Documentation Error According to the semantic versioning specification (https://semver.org/), the only versions in which features are allowed to be removed are major versions (x.0.0)
return-syntax-error Documentation Error RETURN is not valid YAML, RETURN fragments missing or invalid
return-invalid-version-added Documentation Error version_added for return value is not a valid version number
subdirectory-missing-init Naming Error Ansible module subdirectories must contain an __init__.py
try-except-missing-has Imports Warning Try/Except HAS_ expression missing
undocumented-parameter Documentation Error Argument is listed in the argument_spec, but not documented in the module
unidiomatic-typecheck Syntax Error Type comparison using type() found. Use isinstance() instead
unknown-doc-fragment Documentation Warning Unknown pre-existing DOCUMENTATION error
use-boto3 Imports Error boto import found, new modules should use boto3
use-fail-json-not-sys-exit Imports Error sys.exit() call found. Should be exit_json/fail_json
use-module-utils-urls Imports Error requests import found, should use ansible.module_utils.urls instead
use-run-command-not-os-call Imports Error os.call used instead of module.run_command
use-run-command-not-popen Imports Error subprocess.Popen used instead of module.run_command
use-short-gplv3-license Documentation Error GPLv3 license header should be the short form <copyright> for new modules
mutually_exclusive-type Documentation Error mutually_exclusive entry contains non-string value
mutually_exclusive-collision Documentation Error mutually_exclusive entry has repeated terms
mutually_exclusive-unknown Documentation Error mutually_exclusive entry contains option which does not appear in argument_spec (potentially an alias of an option?)
required_one_of-type Documentation Error required_one_of entry contains non-string value
required_one_of-collision Documentation Error required_one_of entry has repeated terms
required_one_of-unknown Documentation Error required_one_of entry contains option which does not appear in argument_spec (potentially an alias of an option?)
required_together-type Documentation Error required_together entry contains non-string value
required_together-collision Documentation Error required_together entry has repeated terms
required_together-unknown Documentation Error required_together entry contains option which does not appear in argument_spec (potentially an alias of an option?)
required_if-is_one_of-type Documentation Error required_if entry has a fourth value which is not a bool
required_if-requirements-type Documentation Error required_if entry has a third value (requirements) which is not a list or tuple
required_if-requirements-collision Documentation Error required_if entry has repeated terms in requirements
required_if-requirements-unknown Documentation Error required_if entry's requirements contains option which does not appear in argument_spec (potentially an alias of an option?)
required_if-unknown-key Documentation Error required_if entry's key does not appear in argument_spec (potentially an alias of an option?)
required_if-key-in-requirements Documentation Error required_if entry contains its key in requirements list/tuple
required_if-value-type Documentation Error required_if entry's value is not of the type specified for its key
required_by-collision Documentation Error required_by entry has repeated terms
required_by-unknown Documentation Error required_by entry contains option which does not appear in argument_spec (potentially an alias of an option?)
version-added-must-be-major-or-minor Documentation Error According to the semantic versioning specification (https://semver.org/), the only versions in which features are allowed to be added are major and minor versions (x.y.0)

yamllint

Check YAML files for syntax and formatting issues.

Testing plugin documentation<span id="testing_module_documentation"></span>

A quick test while developing is to use ansible-doc -t <plugin_type> <name> to see if it renders, you might need to add -M /path/to/module if the module is not somewhere Ansible expects to find it.

Before you submit a plugin for inclusion in ansible-core, you must run tests to ensure that the argspec matches the documentation in your Python file, and that the argspec's and documentation's structure is correct.

The community pages offer more information on testing reStructuredText documentation <testing_documentation_locally> if you extend the Ansible documentation with additional RST files.

To ensure that your module documentation matches your argument_spec:

  1. Install required Python packages (drop '--user' in venv/virtualenv):

    pip install --user -r test/lib/ansible_test/_data/requirements/sanity.txt

  2. run the validate-modules test:

    ansible-test sanity --test validate-modules mymodule

If you have Docker or Podman installed, you can also use them with the --docker option, which uses an image that already has all requirements installed:

ansible-test sanity --docker --test validate-modules mymodule

For other plugin types the steps are similar, just adjusting names and paths to the specific type.

httptester

Topics

Overview

httptester is a docker container used to host certain resources required by testing_integration. This is to avoid CI tests requiring external resources (such as Git or package repos) which, if temporarily unavailable, would cause tests to fail.

HTTP Testing endpoint which provides the following capabilities:

Source files can be found in the http-test-container repository.

Extending httptester

If you have sometime to improve httptester, open an issue in the http-test-container repository.

Integration tests

Topics

The Ansible integration Test system.

Tests for playbooks, by playbooks.

Some tests may require credentials. Credentials may be specified with credentials.yml.

Some tests may require root.

Every new module and plugin should have integration tests, even if the tests cannot be run on Ansible CI infrastructure. In this case, the tests should be marked with the unsupported alias in aliases file <integration_aliases>.

Quick Start

It is highly recommended that you install and activate the argcomplete Python package. It provides tab completion in bash for the ansible-test test runner.

Configuration

ansible-test command

The example below assumes bin/ is in your $PATH. An easy way to achieve that is to initialize your environment with the env-setup command:

source hacking/env-setup
ansible-test --help

You can also call ansible-test with the full path:

bin/ansible-test --help

integration_config.yml

Making your own version of integration_config.yml can allow for setting some tunable parameters to help run the tests better in your environment. Some tests (for example, cloud tests) will only run when access credentials are provided. For more information about supported credentials, refer to the various cloud-config-*.template files in the tests/integration/ directory.

Prerequisites

Some tests assume things like hg, svn, and git are installed, and in path. Some tests (such as those for Amazon Web Services) need separate definitions, which will be covered later in this document.

(Complete list pending)

Non-destructive Tests

These tests will modify files in subdirectories, but will not do things that install or remove packages or things outside of those test subdirectories. They will also not reconfigure or bounce system services.

Running integration tests within containers

To protect your system from any potential changes caused by integration tests, and to ensure a sensible set of dependencies are available we recommend that you always run integration tests with the --docker option, for example --docker ubuntu. Get the list of supported container images by running ansible-test integration --help. You can find them in the target docker images section of the output. The default image is used for sanity and unit tests, as well as for platform independent integration tests such as those for cloud modules.

Run as follows for all POSIX platform tests executed by our CI system in a Fedora 34 container:

ansible-test integration shippable/ --docker fedora

You can exclude a specific test as well, such as for individual modules:

ansible-test integration --exclude git

You can target a specific test as well, such as for individual modules:

ansible-test integration ping

You can use the -v option to make the output more verbose:

ansible-test integration lineinfile -vvv

Use the following command to list all the available targets:

ansible-test integration --list-targets

Bash users

If you use bash with argcomplete, obtain a full list by doing: ansible-test integration <tab><tab>

Destructive Tests

These tests are allowed to install and remove some trivial packages. You will likely want to devote these to a virtual environment, such as Docker. They won't reformat your filesystem:

ansible-test integration destructive/ --docker fedora

Windows Tests

These tests exercise the winrm connection plugin and Windows modules. You'll need to define an inventory with a remote Windows Server to use for testing, and enable PowerShell Remoting to continue.

Running these tests may result in changes to your Windows host, so don't run them against a production/critical Windows environment.

Enable PowerShell Remoting (run on the Windows host by a Remote Desktop):

Enable-PSRemoting -Force

Define Windows inventory:

cp inventory.winrm.template inventory.winrm
${EDITOR:-vi} inventory.winrm

Run the Windows tests executed by our CI system:

ansible-test windows-integration -v shippable/

Tests in containers

If you have a Linux system with Docker or Podman installed, running integration tests using the same containers used by the Ansible continuous integration (CI) system is recommended.

Podman

By default, Podman will only be used if the Docker CLI is not installed. If you have Docker installed but want to use Podman, you can change this behavior by setting the environment variable ANSIBLE_TEST_PREFER_PODMAN.

Docker on non-Linux

Using Docker Engine to run Docker on a non-Linux host (such as macOS) is not recommended. Some tests may fail, depending on the image used for testing. Using the --docker-privileged option when running integration (not network-integration or windows-integration) may resolve the issue.

Running Integration Tests

To run all CI integration test targets for POSIX platforms in a Ubuntu container:

ansible-test integration shippable/ --docker ubuntu

You can also run specific tests or select a different Linux distribution. For example, to run tests for the ping module on a Ubuntu container:

ansible-test integration ping --docker ubuntu

Container Images

Container images are updated regularly. To see the current list of container images:

ansible-test integration --help

The list is under the target docker images and supported python version heading.

Other configuration for Cloud Tests

To run some tests, you must provide access credentials in a file named cloud-config-aws.yml or cloud-config-cs.ini in the tests/integration directory. Corresponding .template files are available for syntax help. The newer AWS tests now use the file tests/integration/cloud-config-aws.yml.

IAM policies for AWS

Ansible needs fairly wide ranging powers to run the tests in an AWS account. These rights can be provided to a dedicated user. These need to be configured before running the test.

testing-policies

The GitHub repository mattclay/aws-terminator contains two sets of policies used for all existing AWS module integration tests. The hacking/aws_config/setup_iam.yml playbook can be used to setup two groups:

Once the groups have been created, you'll need to create a user and make the user a member of these groups. The policies are designed to minimize the rights of that user. Please note that while this policy does limit the user to one region, this does not fully restrict the user (primarily due to the limitations of the Amazon ARN notation). The user will still have wide privileges for viewing account definitions, and will also be able to manage some resources that are not related to testing (for example, AWS lambdas with different names). Tests should not be run in a primary production account in any case.

Other Definitions required

Apart from installing the policy and giving it to the user identity running the tests, a lambda role ansible_integration_tests has to be created which has lambda basic execution privileges.

Network Tests

For guidance on writing network test see testing_resource_modules.

Where to find out more

If you'd like to know more about the plans for improving testing Ansible, join the Ansible community forum

Testing Ansible and Collections

This document describes how to run tests using ansible-test.

Setup

Before running ansible-test, set up your environment for testing_an_ansible_collection or testing_ansible_core, depending on which scenario applies to you.

If you use git for version control, make sure the files you are working with are not ignored by git. If they are, ansible-test will ignore them as well.

Testing an Ansible Collection

If you are testing an Ansible Collection, you need a copy of the collection, preferably a Git clone. For example, to work with the community.windows collection, follow these steps:

  1. Clone the collection you want to test into a valid collection root:

    git clone https://github.com/ansible-collections/community.windows ~/dev/ansible_collections/community/windows

    The path must end with /ansible_collections/{collection_namespace}/{collection_name} where {collection_namespace} is the namespace of the collection and {collection_name} is the collection name.

  2. Clone any collections on which the collection depends:

    git clone https://github.com/ansible-collections/ansible.windows ~/dev/ansible_collections/ansible/windows

    If your collection has any dependencies on other collections, they must be in the same collection root, since ansible-test will not use your configured collection roots (or other Ansible configuration).

    See the collection's galaxy.yml for a list of possible dependencies.

  3. Switch to the directory where the collection to test resides:

    cd ~/dev/ansible_collections/community/windows

Testing ansible-core

If you are testing ansible-core itself, you need a copy of the ansible-core source code, preferably a Git clone. Having an installed copy of ansible-core is not sufficient or required. For example, to work with the ansible-core source cloned from GitHub, follow these steps:

  1. Clone the ansible-core repository:

    git clone https://github.com/ansible/ansible ~/dev/ansible

  2. Switch to the directory where the ansible-core source resides:

    cd ~/dev/ansible

  3. Add ansible-core programs to your PATH:

    source hacking/env-setup

    You can skip this step if you only need to run ansible-test, and not other ansible-core programs. In that case, simply run bin/ansible-test from the root of the ansible-core source.

    If you have an installed version of ansible-core and are trying to run ansible-test from your PATH, make sure the program found by your shell is the one from the ansible-core source:

    which ansible-test

Commands

The most commonly used test commands are:

Run ansible-test --help to see a complete list of available commands.

For detailed help on a specific command, add the --help option after the command.

Environments

Most ansible-test commands support running in one or more isolated test environments to simplify testing.

Containers

Containers are recommended for running sanity, unit and integration tests, since they provide consistent environments. Unit tests will be run with network isolation, which avoids unintentional dependencies on network resources.

The --docker option runs tests in a container using either Docker or Podman.

If both Docker and Podman are installed, Docker will be used. To override this, set the environment variable ANSIBLE_TEST_PREFER_PODMAN to any non-empty value.

Choosing a container

Without an additional argument, the --docker option uses the default container. To use another container, specify it immediately after the --docker option.

The default container is recommended for all sanity and unit tests.

To see the list of supported containers, use the --help option with the ansible-test command you want to use.

The list of available containers is dependent on the ansible-test command you are using.

You can also specify your own container. When doing so, you will need to indicate the Python version in the container with the --python option.

Custom containers

When building custom containers, keep in mind the following requirements:

Docker and SELinux

Using Docker on a host with SELinux may require setting the system in permissive mode. Consider using Podman instead.

Docker Desktop with WSL2

These instructions explain how to use ansible-test with WSL2 and Docker Desktop without systemd support.

If your WSL2 environment includes systemd support, these steps are not required.

Configuration requirements
  1. Open Docker Desktop and go to the Settings screen.
  2. On the General tab:
    1. Uncheck the Start Docker Desktop when you log in checkbox.
    2. Check the Use the WSL 2 based engine checkbox.
  3. On the Resources tab under the WSL Integration section:
    1. Enable distros you want to use under the Enable integration with additional distros section.
  4. Click Apply and restart if changes were made.
Setup instructions

If all WSL instances have been stopped, these changes will need to be re-applied.

  1. Verify Docker Desktop is properly configured (see configuration_requirements).

  2. Quit Docker Desktop if it is running:

    1. Right click the Docker Desktop taskbar icon.
    2. Click the Quit Docker Desktop option.

  3. Stop any running WSL instances with the command:

    wsl --shutdown

  4. Verify all WSL instances have stopped with the command:

    wsl -l -v

  5. Start a WSL instance and perform the following steps as root:

    1. Verify the systemd subsystem is not registered:

      1. Check for the systemd cgroup hierarchy with the following command:

        grep systemd /proc/self/cgroup

      2. If any matches are found, re-check the configuration_requirements and follow the setup_instructions again.

    2. Mount the systemd cgroup hierarchy with the following commands:
    mkdir /sys/fs/cgroup/systemd
mount cgroup -t cgroup /sys/fs/cgroup/systemd -o none,name=systemd,xattr

  6. Start Docker Desktop.

You should now be able to use ansible-test with the --docker option.

Linux cgroup configuration

These changes will need to be re-applied each time the container host is booted.

For certain container hosts and container combinations, additional setup on the container host may be required. In these situations ansible-test will report an error and provide additional instructions to run as root:

mkdir /sys/fs/cgroup/systemd
mount cgroup -t cgroup /sys/fs/cgroup/systemd -o none,name=systemd,xattr

If you are using rootless Podman, an additional command must be run, also as root. Make sure to substitute your user and group for {user} and {group} respectively:

chown -R {user}:{group} /sys/fs/cgroup/systemd
Podman

When using Podman, you may need to stop existing Podman processes after following the linux_cgroup_configuration instructions. Otherwise Podman may be unable to see the new mount point.

You can check to see if Podman is running by looking for podman and catatonit processes.

Remote virtual machines

Remote virtual machines are recommended for running integration tests not suitable for execution in containers.

The --remote option runs tests in a cloud hosted ephemeral virtual machine.

An API key is required to use this feature, unless running under an approved Azure Pipelines organization.

To see the list of supported systems, use the --help option with the ansible-test command you want to use.

The list of available systems is dependent on the ansible-test command you are using.

Python virtual environments

Python virtual environments provide a simple way to achieve isolation from the system and user Python environments. They are recommended for unit and integration tests when the --docker and --remote options cannot be used.

The --venv option runs tests in a virtual environment managed by ansible-test. Requirements are automatically installed before tests are run.

Composite environment arguments

The environment arguments covered in this document are sufficient for most use cases. However, some scenarios may require the additional flexibility offered by composite environment arguments.

The --controller and --target options are alternatives to the --docker, --remote and --venv options.

When using the shell command, the --target option is replaced by three platform specific options.

Add the --help option to your ansible-test command to learn more about the composite environment arguments.

Additional Requirements

Some ansible-test commands have additional requirements. You can use the --requirements option to automatically install them.

When using a test environment managed by ansible-test the --requirements option is usually unnecessary.

Environment variables

When using environment variables to manipulate tests there some limitations to keep in mind. Environment variables are:

Interactive shell

Use the ansible-test shell command to get an interactive shell in the same environment used to run tests. Examples:

Code coverage

Code coverage reports make it easy to identify untested code for which more tests should be written. Online reports are available but only cover the devel branch (see developing_testing). For new code local reports are needed.

Add the --coverage option to any test command to collect code coverage data. If you aren't using the --venv or --docker options which create an isolated python environment then you may have to use the --requirements option to ensure that the correct version of the coverage module is installed:

ansible-test coverage erase
ansible-test units --coverage apt
ansible-test integration --coverage aws_lambda
ansible-test coverage html

Reports can be generated in several different formats:

To clear data between test runs, use the ansible-test coverage erase command.

Sanity Tests

Topics

Sanity tests are made up of scripts and tools used to perform static code analysis. The primary purpose of these tests is to enforce Ansible coding standards and requirements.

Tests are run with ansible-test sanity. All available tests are run unless the --test option is used.

Set up your environment

  1. Install ansible-core<installation_guide> that provides the ansible-test tool.
  1. Install podman or docker to avoid installing all the dependencies on your system.
  2. If you test files in a collection:

How to run

To run sanity tests using podman or docker, always use the default docker image by passing the --docker argument without specifying the image name.

  1. When testing files in a collection, change your location to your collection directory you created while setting up your environment<set_up_env_to_run_tests>:
cd ~/ansible_collections/<NAMESPACE>/<COLLECTION_NAME>
  1. To run all sanity tests in a container:
ansible-test sanity --docker

Available Tests

See the full list of sanity tests <all_sanity_tests>, which also details how to fix identified issues.

Unit Tests

Unit tests are small isolated tests that target a specific library or module. Unit tests in Ansible are currently the only way of driving tests from python within Ansible's continuous integration process. This means that in some circumstances the tests may be a bit wider than just units.

Topics

Available Tests

Unit tests can be found in test/units. Notice that the directory structure of the tests matches that of lib/ansible/.

Running Tests

To run unit tests using docker, always use the default docker image by passing the --docker or --docker default argument.

The Ansible unit tests can be run across the whole code base by doing:

cd /path/to/ansible/source
source hacking/env-setup
ansible-test units --docker -v

Against a single file by doing:

ansible-test units --docker -v apt

Or against a specific Python version by doing:

ansible-test units --docker -v --python 3.13 apt

If you are running unit tests against things other than modules, such as module utilities, specify the whole file path:

ansible-test units --docker -v test/units/module_utils/basic/test_imports.py

For advanced usage see the online help:

ansible-test units --help

You can also run tests in Ansible's continuous integration system by opening a pull request. This will automatically determine which tests to run based on the changes made in your pull request.

Installing dependencies

If you are running ansible-test with the --docker or --venv option you do not need to install dependencies manually.

Otherwise you can install dependencies using the --requirements option, which will install all the required dependencies needed for unit tests. For example:

ansible-test units --python 3.12 --requirements apache2_module

The list of unit test requirements can be found at test/units/requirements.txt.

This does not include the list of unit test requirements for ansible-test itself, which can be found at test/lib/ansible_test/_data/requirements/units.txt.

See also the constraints applicable to all test commands.

Extending unit tests

What a unit test isn't

If you start writing a test that requires external services then you may be writing an integration test, rather than a unit test.

Structuring Unit Tests

Ansible drives unit tests through pytest. This means that tests can either be written a simple functions which are included in any file name like test_<something>.py or as classes.

Here is an example of a function:

#this function will be called simply because it is called test_*()

def test_add():
    a = 10
    b = 23
    c = 33
    assert a + b == c

Here is an example of a class:

import unittest

class AddTester(unittest.TestCase):

    def SetUp():
        self.a = 10
        self.b = 23

    # this function will
    def test_add():
      c = 33
      assert self.a + self.b == c

   # this function will
    def test_subtract():
      c = -13
      assert self.a - self.b == c

Both methods work fine in most circumstances; the function-based interface is simpler and quicker and so that's probably where you should start when you are just trying to add a few basic tests for a module. The class-based test allows more tidy set up and tear down of pre-requisites, so if you have many test cases for your module you may want to refactor to use that.

Assertions using the simple assert function inside the tests will give full information on the cause of the failure with a trace-back of functions called during the assertion. This means that plain asserts are recommended over other external assertion libraries.

A number of the unit test suites include functions that are shared between several modules, especially in the networking arena. In these cases a file is created in the same directory, which is then included directly.

Module test case common code

Keep common code as specific as possible within the test/units/ directory structure. Don't import common unit test code from directories outside the current or parent directories.

Don't import other unit tests from a unit test. Any common code should be in dedicated files that aren't themselves tests.

Fixtures files

To mock out fetching results from devices, or provide other complex data structures that come from external libraries, you can use fixtures to read in pre-generated data.

You can check how fixtures are used in CPU_INFO_TEST_SCENARIOS

If you are simulating APIs you may find that Python placebo is useful. See testing_units_modules for more information.

Code Coverage For New or Updated Unit Tests

New code will be missing from the codecov.io coverage reports (see developing_testing), so local reporting is needed. Most ansible-test commands allow you to collect code coverage; this is particularly useful when to indicate where to extend testing.

To collect coverage data add the --coverage argument to your ansible-test command line:

ansible-test units --coverage apt
ansible-test coverage html

Results will be written to test/results/reports/coverage/index.html

Reports can be generated in several different formats:

To clear data between test runs, use the ansible-test coverage erase command. See testing_running_locally for more information about generating coverage reports.

testing_units_modules
Special considerations for unit testing modules

testing_running_locally
Running tests locally including gathering and reporting coverage data

Python 3 documentation - 26.4. unittest — Unit testing framework
The documentation of the unittest framework in python 3

Python 2 documentation - 25.3. unittest — Unit testing framework
The documentation of the earliest supported unittest framework - from Python 2.6

pytest: helps you write better programs
The documentation of pytest - the framework actually used to run Ansible unit tests

Unit Testing Ansible Modules

Topics

Introduction

This document explains why, how and when you should use unit tests for Ansible modules. The document doesn't apply to other parts of Ansible for which the recommendations are normally closer to the Python standard. There is basic documentation for Ansible unit tests in the developer guide testing_units. This document should be readable for a new Ansible module author. If you find it incomplete or confusing, please open a bug or ask for help on the Ansible Forum<ansible_forum>.

What Are Unit Tests?

Ansible includes a set of unit tests in the test/units directory. These tests primarily cover the internals but can also cover Ansible modules. The structure of the unit tests matches the structure of the code base, so the tests that reside in the test/units/modules/ directory are organized by module groups.

Integration tests can be used for most modules, but there are situations where cases cannot be verified using integration tests. This means that Ansible unit test cases may extend beyond testing only minimal units and in some cases will include some level of functional testing.

Why Use Unit Tests?

Ansible unit tests have advantages and disadvantages. It is important to understand these. Advantages include:

There are also some potential disadvantages of unit tests. Unit tests don't normally directly test actual useful valuable features of software, instead just internal implementation

Normally the Ansible integration tests (which are written in Ansible YAML) provide better testing for most module functionality. If those tests already test a feature and perform well there may be little point in providing a unit test covering the same area as well.

When To Use Unit Tests

There are a number of situations where unit tests are a better choice than integration tests. For example, testing things which are impossible, slow or very difficult to test with integration tests, such as:

Providing quick feedback

Example:
A single step of the rds_instance test cases can take up to 20 minutes (the time to create an RDS instance in Amazon). The entire test run can last for well over an hour. All 16 of the unit tests complete execution in less than 2 seconds.

The time saving provided by being able to run the code in a unit test makes it worth creating a unit test when bug fixing a module, even if those tests do not often identify problems later. As a basic goal, every module should have at least one unit test which will give quick feedback in easy cases without having to wait for the integration tests to complete.

Ensuring correct use of external interfaces

Unit tests can check the way in which external services are run to ensure that they match specifications or are as efficient as possible even when the final output will not be changed.

Example:
Package managers are often far more efficient when installing multiple packages at once rather than each package separately. The final result is the same: the packages are all installed, so the efficiency is difficult to verify through integration tests. By providing a mock package manager and verifying that it is called once, we can build a valuable test for module efficiency.

Another related use is in the situation where an API has versions which behave differently. A programmer working on a new version may change the module to work with the new API version and unintentionally break the old version. A test case which checks that the call happens properly for the old version can help avoid the problem. In this situation it is very important to include version numbering in the test case name (see Naming unit tests below).

Providing specific design tests

By building a requirement for a particular part of the code and then coding to that requirement, unit tests can_ sometimes improve the code and help future developers understand that code.

Unit tests that test internal implementation details of code, on the other hand, almost always do more harm than good. Testing that your packages to install are stored in a list would slow down and confuse a future developer who might need to change that list into a dictionary for efficiency. This problem can be reduced somewhat with clear test naming so that the future developer immediately knows to delete the test case, but it is often better to simply leave out the test case altogether and test for a real valuable feature of the code, such as installing all of the packages supplied as arguments to the module.

How to unit test Ansible modules

There are a number of techniques for unit testing modules. Beware that most modules without unit tests are structured in a way that makes testing quite difficult and can lead to very complicated tests which need more work than the code. Effectively using unit tests may lead you to restructure your code. This is often a good thing and leads to better code overall. Good restructuring can make your code clearer and easier to understand.

Naming unit tests

Unit tests should have logical names. If a developer working on the module being tested breaks the test case, it should be easy to figure what the unit test covers from the name. If a unit test is designed to verify compatibility with a specific software or API version then include the version in the name of the unit test.

As an example, test_v2_state_present_should_call_create_server_with_name() would be a good name, test_create_server() would not be.

Use of Mocks

Mock objects (from https://docs.python.org/3/library/unittest.mock.html) can be very useful in building unit tests for special / difficult cases, but they can also lead to complex and confusing coding situations. One good use for mocks would be in simulating an API. As for 'six', the 'mock' python package is bundled with Ansible (use import units.compat.mock).

Ensuring failure cases are visible with mock objects

Functions like module.fail_json are normally expected to terminate execution. When you run with a mock module object this doesn't happen since the mock always returns another mock from a function call. You can set up the mock to raise an exception as shown above, or you can assert that these functions have not been called in each test. For example:

module = MagicMock()
function_to_test(module, argument)
module.fail_json.assert_not_called()

This applies not only to calling the main module but almost any other function in a module which gets the module object.

Mocking of the actual module

The setup of an actual module is quite complex (see Passing Arguments below) and often isn't needed for most functions which use a module. Instead you can use a mock object as the module and create any module attributes needed by the function you are testing. If you do this, beware that the module exit functions need special handling as mentioned above, either by throwing an exception or ensuring that they haven't been called. For example:

class AnsibleExitJson(Exception):
    """Exception class to be raised by module.exit_json and caught by the test case"""
    pass

# you may also do the same to fail json
module = MagicMock()
module.exit_json.side_effect = AnsibleExitJson(Exception)
with self.assertRaises(AnsibleExitJson) as result:
    results = my_module.test_this_function(module, argument)
module.fail_json.assert_not_called()
assert results["changed"] == True

API definition with unit test cases

API interaction is usually best tested with the function tests defined in Ansible's integration testing section, which run against the actual API. There are several cases where the unit tests are likely to work better.

Defining a module against an API specification

This case is especially important for modules interacting with web services, which provide an API that Ansible uses but which are beyond the control of the user.

By writing a custom emulation of the calls that return data from the API, we can ensure that only the features which are clearly defined in the specification of the API are present in the message. This means that we can check that we use the correct parameters and nothing else.

Example: in rds_instance unit tests a simple instance state is defined:

def simple_instance_list(status, pending):
    return {u'DBInstances': [{u'DBInstanceArn': 'arn:aws:rds:us-east-1:1234567890:db:fakedb',
                              u'DBInstanceStatus': status,
                              u'PendingModifiedValues': pending,
                              u'DBInstanceIdentifier': 'fakedb'}]}

This is then used to create a list of states:

rds_client_double = MagicMock()
rds_client_double.describe_db_instances.side_effect = [
    simple_instance_list('rebooting', {"a": "b", "c": "d"}),
    simple_instance_list('available', {"c": "d", "e": "f"}),
    simple_instance_list('rebooting', {"a": "b"}),
    simple_instance_list('rebooting', {"e": "f", "g": "h"}),
    simple_instance_list('rebooting', {}),
    simple_instance_list('available', {"g": "h", "i": "j"}),
    simple_instance_list('rebooting', {"i": "j", "k": "l"}),
    simple_instance_list('available', {}),
    simple_instance_list('available', {}),
]

These states are then used as returns from a mock object to ensure that the await function waits through all of the states that would mean the RDS instance has not yet completed configuration:

rds_i.await_resource(rds_client_double, "some-instance", "available", mod_mock,
                     await_pending=1)
assert(len(sleeper_double.mock_calls) > 5), "await_pending didn't wait enough"

By doing this we check that the await function will keep waiting through potentially unusual that it would be impossible to reliably trigger through the integration tests but which happen unpredictably in reality.

Defining a module to work against multiple API versions

This case is especially important for modules interacting with many different versions of software; for example, package installation modules that might be expected to work with many different operating system versions.

By using previously stored data from various versions of an API we can ensure that the code is tested against the actual data which will be sent from that version of the system even when the version is very obscure and unlikely to be available during testing.

Ansible special cases for unit testing

There are a number of special cases for unit testing the environment of an Ansible module. The most common are documented below, and suggestions for others can be found by looking at the source code of the existing unit tests or asking the community<communication>.

Module argument processing

There are two problems with running the main function of a module:

Passing Arguments

To pass arguments to a module correctly, use the set_module_args method which accepts a dictionary as its parameter. Module creation and argument processing is handled through the AnsibleModule object in the basic section of the utilities. Normally this accepts input on STDIN, which is not convenient for unit testing. When the special variable is set it will be treated as if the input came on STDIN to the module. Simply call that function before setting up your module:

import json
from units.modules.utils import set_module_args
from ansible.module_utils.common.text.converters import to_bytes

def test_already_registered(self):
    set_module_args({
        'activationkey': 'key',
        'username': 'user',
        'password': 'pass',
    })

Handling exit correctly

The module.exit_json function won't work properly in a testing environment since it writes error information to STDOUT upon exit, where it is difficult to examine. This can be mitigated by replacing it (and module.fail_json) with a function that raises an exception:

def exit_json(*args, **kwargs):
    if 'changed' not in kwargs:
        kwargs['changed'] = False
    raise AnsibleExitJson(kwargs)

Now you can ensure that the first function called is the one you expected simply by testing for the correct exception:

def test_returned_value(self):
    set_module_args({
        'activationkey': 'key',
        'username': 'user',
        'password': 'pass',
    })

    with self.assertRaises(AnsibleExitJson) as result:
        my_module.main()

The same technique can be used to replace module.fail_json (which is used for failure returns from modules) and for the aws_module.fail_json_aws() (used in modules for Amazon Web Services).

Running the main function

If you do want to run the actual main function of a module you must import the module, set the arguments as above, set up the appropriate exit exception and then run the module:

# This test is based around pytest's features for individual test functions
import pytest
import ansible.modules.module.group.my_module as my_module

def test_main_function(monkeypatch):
    monkeypatch.setattr(my_module.AnsibleModule, "exit_json", fake_exit_json)
    set_module_args({
        'activationkey': 'key',
        'username': 'user',
        'password': 'pass',
    })
    my_module.main()

Handling calls to external executables

Module must use AnsibleModule.run_command in order to execute an external command. This method needs to be mocked:

Here is a simple mock of AnsibleModule.run_command (taken from test/units/modules/packaging/os/test_rhn_register.py):

with patch.object(basic.AnsibleModule, 'run_command') as run_command:
    run_command.return_value = 0, '', ''  # successful execution, no output
    with self.assertRaises(AnsibleExitJson) as result:
        my_module.main()
        self.assertFalse(result.exception.args[0]['changed'])
# Check that run_command has been called
run_command.assert_called_once_with('/usr/bin/command args')
self.assertEqual(run_command.call_count, 1)
self.assertFalse(run_command.called)

A Complete Example

The following example is a complete skeleton that reuses the mocks explained above and adds a new mock for Ansible.get_bin_path:

import json

from units.compat import unittest
from units.compat.mock import patch
from ansible.module_utils import basic
from ansible.module_utils.common.text.converters import to_bytes
from ansible.modules.namespace import my_module


def set_module_args(args):
    """prepare arguments so that they will be picked up during module creation"""
    args = json.dumps({'ANSIBLE_MODULE_ARGS': args})
    basic._ANSIBLE_ARGS = to_bytes(args)


class AnsibleExitJson(Exception):
    """Exception class to be raised by module.exit_json and caught by the test case"""
    pass


class AnsibleFailJson(Exception):
    """Exception class to be raised by module.fail_json and caught by the test case"""
    pass


def exit_json(*args, **kwargs):
    """function to patch over exit_json; package return data into an exception"""
    if 'changed' not in kwargs:
        kwargs['changed'] = False
    raise AnsibleExitJson(kwargs)


def fail_json(*args, **kwargs):
    """function to patch over fail_json; package return data into an exception"""
    kwargs['failed'] = True
    raise AnsibleFailJson(kwargs)


def get_bin_path(self, arg, required=False):
    """Mock AnsibleModule.get_bin_path"""
    if arg.endswith('my_command'):
        return '/usr/bin/my_command'
    else:
        if required:
            fail_json(msg='%r not found !' % arg)


class TestMyModule(unittest.TestCase):

    def setUp(self):
        self.mock_module_helper = patch.multiple(basic.AnsibleModule,
                                                 exit_json=exit_json,
                                                 fail_json=fail_json,
                                                 get_bin_path=get_bin_path)
        self.mock_module_helper.start()
        self.addCleanup(self.mock_module_helper.stop)

    def test_module_fail_when_required_args_missing(self):
        with self.assertRaises(AnsibleFailJson):
            set_module_args({})
            my_module.main()


    def test_ensure_command_called(self):
        set_module_args({
            'param1': 10,
            'param2': 'test',
        })

        with patch.object(basic.AnsibleModule, 'run_command') as mock_run_command:
            stdout = 'configuration updated'
            stderr = ''
            rc = 0
            mock_run_command.return_value = rc, stdout, stderr  # successful execution

            with self.assertRaises(AnsibleExitJson) as result:
                my_module.main()
            self.assertFalse(result.exception.args[0]['changed']) # ensure result is changed

        mock_run_command.assert_called_once_with('/usr/bin/my_command --value 10 --name test')

Restructuring modules to enable testing module set up and other processes

Often modules have a main() function which sets up the module and then performs other actions. This can make it difficult to check argument processing. This can be made easier by moving module configuration and initialization into a separate function. For example:

argument_spec = dict(
    # module function variables
    state=dict(choices=['absent', 'present', 'rebooted', 'restarted'], default='present'),
    apply_immediately=dict(type='bool', default=False),
    wait=dict(type='bool', default=False),
    wait_timeout=dict(type='int', default=600),
    allocated_storage=dict(type='int', aliases=['size']),
    db_instance_identifier=dict(aliases=["id"], required=True),
)

def setup_module_object():
    module = AnsibleAWSModule(
        argument_spec=argument_spec,
        required_if=required_if,
        mutually_exclusive=[['old_instance_id', 'source_db_instance_identifier',
                             'db_snapshot_identifier']],
    )
    return module

def main():
    module = setup_module_object()
    validate_parameters(module)
    conn = setup_client(module)
    return_dict = run_task(module, conn)
    module.exit_json(**return_dict)

This now makes it possible to run tests against the module initiation function:

def test_rds_module_setup_fails_if_db_instance_identifier_parameter_missing():
    # db_instance_identifier parameter is missing
    set_module_args({
        'state': 'absent',
        'apply_immediately': 'True',
     })

    with self.assertRaises(AnsibleFailJson) as result:
        my_module.setup_json

See also test/units/module_utils/aws/test_rds.py

Note that the argument_spec dictionary is visible in a module variable. This has advantages, both in allowing explicit testing of the arguments and in allowing the easy creation of module objects for testing.

The same restructuring technique can be valuable for testing other functionality, such as the part of the module which queries the object that the module configures.

Traps for maintaining Python 2 compatibility

If you use the mock library from the Python 2.6 standard library, a number of the assert functions are missing but will return as if successful. This means that test cases should take great care not use functions marked as new_ in the Python 3 documentation, since the tests will likely always succeed even if the code is broken when run on older versions of Python.

A helpful development approach to this should be to ensure that all of the tests have been run under Python 2.6 and that each assertion in the test cases has been checked to work by breaking the code in Ansible to trigger that failure.

Maintain Python 2.6 compatibility

Please remember that modules need to maintain compatibility with Python 2.6 so the unittests for modules should also be compatible with Python 2.6.

Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide

testing_units
Ansible unit tests documentation

testing_running_locally
Running tests locally including gathering and reporting coverage data

developing_modules_general
Get started developing a module

Python 3 documentation - 26.4. unittest — Unit testing framework
The documentation of the unittest framework in python 3

Python 2 documentation - 25.3. unittest — Unit testing framework
The documentation of the earliest supported unittest framework - from Python 2.6

pytest: helps you write better programs
The documentation of pytest - the framework actually used to run Ansible unit tests

Testing Your Code (from The Hitchhiker's Guide to Python!)
General advice on testing Python code

Uncle Bob's many videos on YouTube
Unit testing is a part of the of various philosophies of software development, including Extreme Programming (XP), Clean Coding. Uncle Bob talks through how to benefit from this

"Why Most Unit Testing is Waste"
An article warning against the costs of unit testing

'A Response to "Why Most Unit Testing is Waste"'
An response pointing to how to maintain the value of unit tests