Ansible Vault: A Primer

Ansible Vault is an Ansible feature that allows the secure storing and use of sensitive data. Once encrypted secrets can be safely distributed or tracked in source control without concern for exposure.

Setup Vault IDs

Ansible 2.4 introduced a new strategy for storing and using passwords for vaults. Passwords are now referenced with Vault IDs which are aliases to files containing the password. This allows for safer handling as paths are no longer exposed and are less likely to leak into source control repositories.

First, the password will need to be saved into a plain text file of a path of the maintainers choosing. This could be on a network share or a home folder.

echo "myExampleStagePassword" > vault-ids/.stage.vaultpass

If this is the first time generating the vault password, one can be generated on the fly using several different strategies, for example with OpenSSL.

echo "$(openssl rand -hex 32)" > vault-ids/.prod.vaultpass

There is no requirement for the file name or extension.

Using Vault IDs instead of referencing file paths can help eliminate headaches when executing playbooks and referencing passwords with relative paths. This will also allow for vault passwords to be centrally stored and not littered across a file system, tracked separately, and possibly not securely accessed. This also brings Tower/AWXs central credential storage strategy to local playbook executions.

Adding IDs to Config

After creating the password files, Ansible will need to be told where these files are located and the desired ID to be associated with it. If Ansible is being run locally, this is typically found in either /etc/ansible/ansible.cfg or ~/.ansible.cfg. The configuration key is vault_identity_list and the format is as follows:

vault_identity_list = vault-id@/path/to/key.file[,other-vault-id@/path/to/other.key]

Enter this configuration under the [defaults] section in the ansible config file.

So for the examples above:

[defaults]
vault_identity_list = example-stage@~/.vault-ids/.stage.vaultpass,example-prod@~/.vault-ids/.prod.vaultpass

This would provide the vault IDs example-stage and example-prod. When using a vault identity list, and unless otherwise specified, Ansible operations which require decrypting will try all identities to decrypt before failing. This helps eliminate specifying IDs when running playbooks.

Adding Passwords to Tower/AWX

If playbooks are being run from a tower instance, the passwords will need to be added to the credentials store.

1. Navigate to “Credentials” and click “+” to create a new credential.
2. Enter a human-interpretable name for future reference
3. Select “Vault” for the credential type
4. Paste/enter the vault password used above
5. Enter a playbook-usable identifier for the “vault identifier”. Note: if using a playbook both locally and in tower, this should be the same as the ID used above, although not required.
6. Click save. Add this credential to any job template created using the playbooks in the future.

Use Directly

Sometimes it may be useful to directly specify the password file to use, particularly if running somewhere that modifying the Ansible configuration file is prohibitive. Set the --vault-password-file parameter to the path where the file can be found.

$ ansible-vault ... --vault-password-file /path/to/password.file ...

Encrypt Secrets

After setting up vault ID(s) either individual variables or entire YAML files can be encrypted. The decision of which strategy to choose will largely come down to maintainability and playbook structure.

Encrypting with either strategy will provide files that can safely be tracked in version control systems as without their passwords they are largely useless and random. Very strong vault passwords should be used when storing encrypted variables.

Ansible recommends storing secrets in a file titled vault inside of group_vars, with the stored secret being referenced in the vars file.

Single Variable

Encrypting single variables is useful when updating existing playbooks to be more secure without having to change too much process to support it. As variables that may not be sensitive are still easily editable in plain text only operations such as password rotations or key updates need the additional step of leveraging ansible-vault commands. Single variable encryption also allows for granular diffing between commits which will be discussed further in entire file encryption.

Additionally, some organizations may separate security roles from network engineers and automation. These organizations may be able to prevent credential leakage since network engineers can make variable changes without the vault ID, leaving a tower to access encrypted variables for testing and execution.

To encrypt a variable, use the encrypt_string function of ansible-vault:

$ ansible-vault encrypt_string --encrypt-vault-id vault-id 'value' --name 'yaml_variable_key'

To continue the example from the above vault IDs:

$ ansible-vault encrypt_string --encrypt-vault-id example-stage 'Tacos are better.' --name 'top_secret_message'

top_secret_message: !vault |
  $ANSIBLE_VAULT;1.2;AES256;example-stage
  30376563336232366339396537383465636163393735336363356531346339303837393930376537
  3839343538346566363530376265666264353230373434640a323438353364623965663537653331
  34666432393631376434323539663234653632386163306362623039303732636532383561383261
  3632343266386262390a393736656530306637663036616464663334626263333834613533323564
  31366630383933396165363632383066356133626161313433663461616561636463
Encryption successful

Outputted is the encrypted variable value complete with the YAML key. This can be pasted directly in an existing YAML file and Ansible will handle it accordingly.

# vars/staging.yml
---
not_secret_message: "Nachos are good."
top_secret_message: !vault |
  $ANSIBLE_VAULT;1.2;AES256;example-stage
  30376563336232366339396537383465636163393735336363356531346339303837393930376537
  3839343538346566363530376265666264353230373434640a323438353364623965663537653331
  34666432393631376434323539663234653632386163306362623039303732636532383561383261
  3632343266386262390a393736656530306637663036616464663334626263333834613533323564
  31366630383933396165363632383066356133626161313433663461616561636463

A strict YAML linter in an IDE or text editor may give a warning of !vault not following syntax however Ansible will handle this correctly.

Entire File

If variables are not modified frequently or playbooks are structured to split secrets and non-secrets, it can be just as easy to encrypt an entire file. Additionally if large text blobs or numerics are intending to be encrypted, entire file encryption can help minimize the incorrect interpretation of variables.

Encrypting entire files will come at the cost of granular diffing between commits in source control. File encryption is not deterministic which means running the same operation on the same file multiple times will generate different files.

To encrypt an existing file, use the encrypt function of ansible-vault:

$ ansible-vault encrypt --encrypt-vault-id vault-id /path/to/variables.yml [--output /path/to/encrypted.variables.yml]

Specifying an output file path is optional, however note that without it the source file is encrypted in place and the unencrypted contents are removed. Using the example above:

$ ansible-vault encrypt --encrypt-vault-id example-prod vars/production.src.yml --output vars/production.yml

Encryption successful

The resulting file:

$ cat vars/production.yml

$ANSIBLE_VAULT;1.2;AES256;example-prod
61323163343737636631333439366364313338653834636662313263623063313736663332303337
3936653931323963353762366437393431313738336534310a333834636335303739366663623834
66356537363163333963386638393338633361393938336137613830386431326138663366353263
6430633233323439350a663239306332613565616434373237383766333835343036623835303532
32653534386133313462656662396665386136323731336334656530616466666330343831646335
33663431666536316436663030643930376630643636633134313335323336333037643136353032
39623062393765393232396666373131326633633732653133386234353262623765336530663431
35623831653139356439656238333966643739383031383434383836616366646165376262343530
3138

If no original source file exists, one can be created using the create function of ansible-vault:

$ ansible-vault create --encrypt-vault-id vault-id /path/to/encrypted.variables.yml

Using Secrets

CLI

Only entire files that have been encrypted with vault can be read through ansible-vault. Mixed-encrypted files using the single variable encryption strategy above require accessing through playbooks. This can however be done from a CLI through ansible leveraging the debug module.

$ ansible localhost -m debug -a var='variable_name' -e "@/path/to/vars.yml"

To view or edit existing encrypted files, use the view or edit commands of ansible-vault respectively:

$ ansible-vault [view|edit] --vault-id the-vault-id /path/to/encrypted.variables.yml

By default this will open the vi editor, but this can be overridden via EDITOR environment variable.

Playbooks

All that is needed to leverage encrypted variables in playbooks is to define the file paths in vars_files or through group_vars and host_vars.

Note: You won’t want to add encrypted secrets into an inventory file. Although this can be done, when importing playbooks into Tower/AWX basic inventory operations will require decrypting vault secrets every time. As well, only recent versions support this ability. Make sure the variables you are trying to store encrypted aren’t best suited in other places.

Example playbook:

# pb_example.yml
---
- name: "Debug"
  hosts: localhost
  vars_files:
    - "vars/staging.yml"
  tasks:
    - debug:
        var: "not_secret_message"
    - debug:
        var: "top_secret_message"

Running ansible-playbook pb_example.yml results in:

PLAY [Debug] **************************************************************

TASK [Gathering Facts] ****************************************************
ok: [localhost]

TASK [debug] **************************************************************
ok: [localhost] => {
    "not_secret_message": "Nachos are good."
}

TASK [debug] **************************************************************
ok: [localhost] => {
    "top_secret_message": "Tacos are better."
}

PLAY RECAP ****************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0   

As the vault IDs were set in ansible.cfg they do not need to be referenced with executing the playbook.

Wrap-up

Ansible Vault is a great tool to use to allow the secure sharing of credentials and data. Complexity is the enemy of security and making it easier for more team members to contribute to automation without having to worry about how the playbooks get executed can quickly speed up automation adoption.

For more information, please check out the Ansible Docs.