Updating Structured Data in Ansible & Jinja

Ken Celenza

August 13, 2019

HomeArrow BlogArrow Updating Structured Data in Ansible & Jinja

Updating a list or dictionary within python is a trivial manner, however within Ansible and Jinja, it is not as easy or well defined. The user must understand the scoping limitations, and how to navigate through them, in addition to the syntax, which is not always obvious.

This data was captured from an ntc-template, and the current structure does not fit in with X internal integration point, or preferred output.

show_mac_address_table:
  - destination_address: "aabb.cc00.6000"
    destination_port: "Fa1/0/36"
    type: "DYNAMIC"
    vlan: "153"
  - destination_address: "ca03.3efb.0000"
    destination_port: "Fa1/0/36"
    type: "DYNAMIC"
    vlan: "100"
  - destination_address: "f44d.3063.34f3"
    destination_port: "Fa1/0/35"
    type: "DYNAMIC"
    vlan: "22"
  - destination_address: "2c6b.f553.9d80"
    destination_port: "Fa1/0/48"
    type: "DYNAMIC"
    vlan: "254"

Updating and Creating an Ansible List

Here we will demonstrate several ways to iterate over the list for your preferred output.

Creating a list of MAC addresses

There is no scoping issues here, but it’s important to keep in mind two things.

  • The variable has to be initialized as a list already.
  • You do not want to over write the data each time, and adding + one list to another is the same as extend in python.
- name: "100 - SET FACT TO NORMALIZE DATA"
  set_fact:
    mac_table_normalized: "{{ mac_table_normalized | default([]) }} + [ '{{ item.destination_address }}' ]"
  loop: "{{ show_mac_address_table }}"

Breakdown:

  • mac_table_normalized | default([]) – If mac_table_normalized is not set, default it to an empty list.
  • ` + [ ` – Add current variable (ensured to be a list), to this new list of one item.
  • '{{ item["destination_address"] }}' – Working with a single list item entry, which is a dictionary in this case, obtain the value from destination_address key, and add it as the item into mac_table_normalized list.
  • loop: "{{ show_mac_address_table }}" Iterate over each element in the list, and assign it the local variable of item.

This produces a concise list of mac addresses now accessible within the playbook. Similarly, within Jinja, you could do the following.

{% set mac_table_normalized = [] %}
{% for mac in show_mac_address_table %}
{% set _ = mac_table_normalized.append(mac["destination_address"]) %}
{% endfor %}
{{ mac_table_normalized }}

This can be called via:

- name: "101 - SET FACT TO NORMALIZE DATA IN JINJA"
  set_fact:
    mac_table_normalized:  "{{ lookup('template', 'template1.j2') }}"

There are two primary things to consider here. First to initialize the list with {% set mac_table_normalized = []. Second, when you append to the list you can safely ignore the value assigned, but because of Jinja’s syntax, you still must use a set method, as it will create empty lines using standard {{ }} syntax. Assigning it to underscore, indicates to the reader of the code, that it is no longer needed after the current scope.

Another syntax that ends in the same result, often seen is.

{% set mac_table_normalized = [] %}
{% for mac in show_mac_address_table %}
{% if mac_table_normalized.append(mac["destination_address"]) %}{% endif %}
{% endfor %}

This can be called via:

- name: "102 - SET FACT TO NORMALIZE DATA IN JINJA"
  set_fact:
    mac_table_normalized: "{{ lookup('template', 'template2.j2') }}"

Transposing a list of dictionaries, to a preferred list of dictionaries

Still using the same data, you now want to format it, in a way that is preferable to an API you will call in a later step. So the API is expecting 4 key/value pairs: [device, vlan, mac, interface].

- name: "104 - SET FACT TO NORMALIZE DATA"
  set_fact:
    mac_table_normalized: "{{ mac_table_normalized | default([]) + [ { 'device': inventory_hostname, 'mac': item.destination_address, 'vlan': item.vlan, 'interface': item.destination_port } ] }}"
  loop: "{{ show_mac_address_table }}"

Most concepts are the same as previous, however, we create a dictionary on the fly, and appending that, instead of a single value.

In Jinja, you could accomplish the same with:

{% set mac_table_normalized = [] %}
{% for mac in show_mac_address_table %}
{% set key_changer = { "device": inventory_hostname, "mac": mac["destination_address"], "vlan": mac["vlan"], "interface": mac["destination_port"] } %}
{% set _ = mac_table_normalized.append(key_changer) %}
{% endfor %}
{{ mac_table_normalized }}

This can be called via:

- name: "105 - SET FACT TO NORMALIZE DATA IN JINJA"
  set_fact:
    mac_table_normalized:  "{{ lookup('template', 'template3.j2') }}"

Updating and creating a dictionary

You can also create a dictionary on the fly, which is helpful to access the data via dictionary in subsequent steps.

Transposing a list of dictionaries to a dictionary

Following a similar pattern, you can do:

- name: "107 - SET FACT TO NORMALIZE DATA"
  set_fact:
    mac_table_normalized: "{{ mac_table_normalized | default({}) | combine( {item.destination_address: { 'device': inventory_hostname, 'vlan': item.vlan, 'interface': item.destination_port }} )}}"
  loop: "{{ show_mac_address_table }}"

In this scenario, there is actually an filter that comes with Ansible called rekey_on_member.

- name: "109 - SET FACT TO NORMALIZE DATA"
  set_fact:
    mac_table_normalized: "{{ show_mac_address_table | rekey_on_member('destination_address') }}"
  loop: "{{ show_mac_address_table }}"

In Jinja, you could accomplish the same with:

{% set mac_table_normalized = {} %}
{% for mac in show_mac_address_table %}
{% set key_changer = { "device": inventory_hostname, "vlan": mac["vlan"], "interface": mac["destination_port"] } %}
{% set _ = mac_table_normalized.update({mac["destination_address"]: key_changer} ) %}
{% endfor %}
{{ mac_table_normalized }}

- name: "111 - SET FACT TO NORMALIZE DATA"
  set_fact:
    mac_table_normalized:  "{{ lookup('template', 'template4.j2') }}"
  loop: "{{ show_mac_address_table }}"
Conclusion

The syntax is not obvious, and there are actually several different ways to accomplish the same thing. If you end up with something that is simpler, would love to hear about it in in the comments.

We have only covered a few of the most common data translations, and you can see how complex they can become. At some point, it will likely make sense to break out into python, with a custom filter or module. However, in a pinch, it’s good to know what is available to use.

You can follow along with all of the examples by reviewing this gist.

-Ken

Tags :
ansible automation jinja netdevops structured-data tutorial
Contents
    Recent Posts
    December 5, 2024
    Introducing the Bootstrap Integration for the Nautobot SSoT App
    November 25, 2024
    Scalable Network Automation with Python Generators
    November 15, 2024
    Last Month in Nautobot – October 2024
    October 4, 2024
    Last Month in Nautobot – September 2024
    September 5, 2024
    Last Month in Nautobot – August 2024
    ntc img
    ntc img

    Contact Us to Learn More

    Share details about yourself & someone from our team will reach out to you ASAP!

    Author