Intro to Ansible Collections

The intention of this blog post is to provide an overview of Ansible Collections, including the problem they are solving, how they work, and how to use them.

The Problem

Ansible has had tremendous growth within the last several years and with growth comes growing pains. Ansible has accumulated several hundred modules in the core repository that are gated by the Ansible development team to ensure quality. This is causing a huge burden to support and becomes prohibitive in keeping up with PRs for new modules or bug fixes for existing modules. Just looking at the open issues and PRs on the Ansible Github indicates the tremendous burden. Currently there are over 4000 open issues and over 2000 open PRs.

Solving the Problem

Ansible Collections is Ansible’s attempt at alleviating the bottleneck created by having the Ansible development team manage PRs, allowing maintainers of modules to develop their modules outside of Ansible’s typical release cycle and provide quicker feature releases or bug fixes. Along with the previous reason there are a few other problems Ansible Collections is attempting to solve such as; plugin/role name collisions and difficult code sharing for most plugins. Both problems are solved by Ansible Collections use of namespaces. This helps to prevent collisions of similary named plugins, modules, etc. as well as having each collection available by it’s Fully Qualified Collection Name (FQCN). These manifest themselves in how you can install a specific collection as well as when developing modules since you can import plugins/module_utils from other collections by using their namespace. We won’t be diving into any development topics in this blog.

Click the following link to learn more about Ansible Galaxy Namespaces.

When Ansible first announced Ansible Collections it was very exciting news, but received mixed reviews from community members. There are some concerns over the quality of modules that will be moved outside of Ansible core and into Ansible Collections and how it will affect companies that have more stringent requirements when it comes to obtaining Open Source Software (OSS) from the Internet. However, there are some ways around obtaining Ansible Collections from the Internet, such as hosting your own galaxy server internally. If an organization already has an internal Ansible Galaxy server, this should be as simple as upgrading it to support collections.

Ansible is addressing some of the concerns with the quality of modules by introuding a partner program that entails a certification process to validate the collections retain the quality of modules that exist within core. The certified modules will be available in both their new Automation Hub and Galaxy.

Some of the certified partners can also be found here

Keep in mind that Ansible Collections was released with Ansible 2.8 and is still in it’s infancy. There will most likely be a lot more changes to them as feedback is provided by the community. Eventually, the majority of modules will be moved out of Ansible core and into Ansible Collections.

Ansible Collections Structure

Ansible Collections are able to include any kind of plugin available within Ansible Core along with roles, playbooks, etc. Below is an example structure of a collection:

collection/
├── docs/
├── galaxy.yml
├── plugins/
│   ├── modules/
│   │   └── module1.py
│   ├── inventory/
│   └── .../
├── README.md
├── roles/
│   ├── role1/
│   ├── role2/
│   └── .../
├── playbooks/
│   ├── files/
│   ├── vars/
│   ├── templates/
│   └── tasks/
└── tests/

As you can see, playbooks and roles can be included within the collection as well as other plugins. These are all available via the namespacing method discussed above.

Here are some links to Ansible documentation in regards to Ansible Collections: Using Collections Developing Collections

Install Ansible Collections

Let’s take a look at several methods to install Ansible Collections:

Install with default values: ~/.ansible/collections/ansible_collections

ansible-galaxy collection install fragmentedpacket.netbox_modules

Install a specific version

ansible-galaxy collection install fragmentedpacket.netbox_modules:0.0.8

Install version within specific version range

ansible-galaxy collection install "fragmentedpacket.netbox_modules:>0.0.7,<0.0.9"

Install via a requirements.yml file

---
collections:
# With just the collection name. This downloads the latest version of the collection
- fragmentedpacket.netbox_modules

# With the collection name, version, and source options
- name: fragmentedpacket.netbox_modules
  version: '0.8.0'
  source: 'The Galaxy URL to pull the collection from (default: ``--api-server`` from cmdline)'

Run the following command to install using the requirements file: ansible-galaxy collection install -r requirements.yml

Install with non-default path

NOTE: this path must be specified within ansible.cfg (collections_paths under [defaults] or the COLLECTIONS_PATHS environment variable) ansible-galaxy collection install fragmentedpacket.netbox_modules -p /other/path/to/collections

Install Ansible Collection from source git repo

(ansible) [root@54924523a23c src]# git clone git@github.com:fragmentedpacket/netbox_modules.git
(ansible) [root@54924523a23c src]# cd ansible_collection/git_repo
(ansible) [root@54924523a23c src]# ansible-galaxy collection build .
Created collection for fragmentedpacket.netbox_modules at /src/cloned-repos/ansible_collections/fragmentedpacket/netbox_modules
fragmentedpacket-netbox_modules-0.1.0.tar.gz
(ansible) [root@54924523a23c src]# ansible-galaxy collection install fragmentedpacket-netbox_modules-0.1.0.tar.gz
Process install dependency map
Starting collection install process
Installing 'fragmentedpacket.netbox_modules:0.1.0' to '/root/.ansible/collections/ansible_collections/fragmentedpacket/netbox_modules'

Now that we have the Ansible Collection installed, it is now time to use it. We’ll explain how in the next section.

Using Ansible Collections

This method uses the keyword collections at the play level:

---
- hosts: localhost
  connection: local
  gather_facts: no
  collections:
    - fragmentedpacket.netbox_modules

  tasks:
    - name: "Test collections"
      netbox_device:
        netbox_url: "http://localhost"
        netbox_token: "1234"
        data:
          name: "test"

    - import_role:
        name: netbox_modules_role

    - debug:
        msg: "{{ lookup('netbox', 'param1') | collection_filter_plugin('test') }}"

The other method that can be used is using the Fully Qualified Collection Name (FQCN):

---
- hosts: localhost
  connection: local
  gather_facts: no

  tasks:
    - name: "Test collections"
      fragmentedpacket.netbox_modules.netbox_device:
        netbox_url: "http://localhost"
        netbox_token: "1234"
        data:
          name: "test"

    - import_role:
        name: fragmentedpacket.netbox_modules.collection_role

    - debug:
        msg: "{{ lookup('fragmentedpacket.netbox_modules.netbox', 'param1') | fragmentedpacket.netbox_modules.collection_filter('test') }}"

If attempting to use a collection within a role, it currently does not appear to to inherit the play level directive for collections and will have to be specified within the task level of the roles:

- name: "Test collections"
  fragmentedpacket.netbox_modules.netbox_device:
    netbox_url: "http://localhost"
    netbox_token: "1234"
    data:
      name: "test"

OR

- name: "Test collections"
  collections:
    - fragmentedpacket.netbox_modules
  netbox_device:
    netbox_url: "http://localhost"
    netbox_token: "1234"
    data:
      name: "test"

We can validate it is using the Ansible Collection module by running the playbook with -vvv and looking for Using module file within the output.

Using module file /src/cloned-repos/ansible_collections/fragmentedpacket/netbox_device/plugins/modules/netbox_device.py

-Mikhail