Automation Principles – DRY

This is part of a series of posts focused on Network Automation Principles.

DRY in Computer Science

The phrase Do Not Repeat Yourself or DRY is a rather simple concept. If you are writing a piece of code multiple times, modularize the code in a way to ensure you are not copying code multiple times. The definition of DRY states:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

Once there are multiple copies, inevitably, one of the pieces of code will not function the same as others, based on some missed step. Additionally if you want to change functionality in one place, you would have to remember to change it in all places.

To put in context, Bart Simpson writing the same phrase on the chalkboard over and over is certainly not DRY.

Example

In this first contrived example you will see the duplication of code in how the VLAN data is being validated.

>>> def add_vlan(vlan_id):
...     if not isinstance(vlan_id, int):
...         raise TypeError("VLAN ID is not an integer")
...     elif not (vlan_id >=1 and vlan_id <= 4096):
...         raise ValueError("Invalid VLAN ID, which must be between 1-4096")
...     return "vlan {vlan_id}".format(vlan_id=vlan_id)
...
>>> def configure_port_vlan(interface, vlan_id):
...     if not isinstance(vlan_id, int):
...         raise TypeError("VLAN ID is not an integer")
...     elif not (vlan_id >=1 and vlan_id <= 4096):
...         raise ValueError("Invalid VLAN ID, which must be between 1-4096")
...     return "interface {interface}\n switchport access vlan {vlan_id}".format(interface=interface, vlan_id=vlan_id)
...
>>>

A common method to to modularize code is building functions. A function can be thought of as a piece of code that can be called to run that single action, and functions can call other functions. Here you can see how the validation functionality is broken out.

>>> def validate_vlan(vlan_id):
...     if not isinstance(vlan_id, int):
...         raise TypeError("VLAN ID is not an integer")
...     elif not (vlan_id >=1 and vlan_id <= 4096):
...         raise ValueError("Invalid VLAN ID, which must be between 1-4096")
...
>>>

The duplicated functionality is removed from the initial functions, and you can see it was tested to work the same.

>>> def add_vlan(vlan_id):
...     validate_vlan(vlan_id)
...     return "vlan {vlan_id}".format(vlan_id=vlan_id)
...
>>> def configure_port_vlan(interface, vlan_id):
...     validate_vlan(vlan_id)
...     return "interface {interface}\n switchport access vlan {vlan_id}".format(interface=interface, vlan_id=vlan_id)
...
>>> print(add_vlan(500))
vlan 500
>>> print(add_vlan(5000))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in add_vlan
  File "<stdin>", line 5, in validate_vlan
ValueError: Invalid VLAN ID, which must be between 1-4096
>>> print(configure_port_vlan("GigabitEthernet1/0/1", 50))
interface GigabitEthernet1/0/1
 switchport access vlan 50
>>> print(add_vlan("500"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in add_vlan
  File "<stdin>", line 3, in validate_vlan
TypeError: VLAN ID is not an integer
>>>

Each time you need to validate a VLAN ID, you simply can call the same function. If at a later date you decided that you wanted to validate the VLAN name for length as well, you can simply add that testing to the one function, instead of having to remember all of the multiple places the code can be.

DRY Data

You will note in the official definition, there is reference to “knowledge”, which as quoted from Pragmatic Programmer

A system’s knowledge is far broader than just its code. It refers to database schemas, test plans, the build system, even documentation

Simply put, code is not the only thing DRY applies to. One example understood by network engineers is managing data for a subnet mask. I often run into solutions of data and associated Jinja files that similar to:

subnets:
  - network: 10.1.1.0
    cidr: 24
    subnet_mask: 255.255.255.0
    wildcard_mask: 0.0.0.255
  - network: 10.1.2.0
    cidr: 24
    subnet_mask: 255.255.255.0
    wildcard_mask: 0.0.0.255

{% if ansible_network_os = "ios" %}
  ip address {{ subnet['network'] | ipaddr('add', 1 ) }} {{ subnet['subnet_mask'] }}{# example result: 10.1.1.1 255.255.255.0 #}
{% elif ansible_network_os = "nxos" %}
  ip address {{ subnet['network'] | ipaddr('add', 1 ) }}/{{ subnet['cidr'] }}{# example result: 10.1.1.1/24 #}
{% endif %}

This solution certainly has benefits, however, given the sheer amount of subnets being managed, it is likely at some point there will be a user that creates a cidr of 25 and subnet_mask as 255.255.255.0. However, the subnet and wildcard mask can be ascertained from the cidr, and maintaining the data multiple times is superfluous and error prone. The one-time cost of building a translation ensures that you are not repeating yourself and not falling into the issues being described here.

Instead you can manage your data a bit more DRY, such as:

subnets:
  - network: 10.1.1.0/24
  - network: 10.1.2.0/24

{% if ansible_network_os = "ios" %}
  ip address {{ subnet['network'] | ipaddr('add', 1 ) }} {{ subnet['network'] | ipaddr('netmask') }}{# example result: 10.1.1.1 255.255.255.0 #}
{% elif ansible_network_os = "nxos" %}
  ip address {{ subnet['network'] | ipaddr('add', 1 ) }}/{{ subnet['network'] | ipaddr('prefix') }}{# example result: 10.1.1.1/24 #}
{% endif %}

While this is a trivial example, the potential issue with data duplication is compounded for every feature that needs to be configured on a given device.

Getting WET in a DRY World

If DRY is not repeating any piece of knowledge, the reverse would be “Write Every Time” (WET). Now why would you want to repeat yourself? Well, as always, it depends, and there are always design considerations that should be taken into account.

As an example, a common pattern we at NTC end up seeing is customers that build a single Jinja template for IOS, NXOS, and EOS. Since these three OS’s have a lot of commonalities, from a DRY perspective you may be inclined to write your templates in consolidated templates.

.
├── bgp.j2
├── ntp.j2
└── vlan.j2

0 directories, 3 files

With an example Jinja file such as

{% for vlan in vlans %}
vlan {{ vlan['id'] }}
 name {{ vlan['name'] }}
{% endfor %}
!

This works out well for this case, however, as you move on to more complicated use cases, such as BGP, you will end with greater differences between the various OS’s.

{% if ansible_network_os == 'ios' %}
router bgp {{ bgp['asn'] }}
 bgp router-id {{ bgp['id'] }}
 bgp log-neighbor-changes
{% for neighbor in bgp['neighbors'] %}
 neighbor {{ neighbor['ip'] }} remote-as {{ neighbor['asn'] }}
 neighbor {{ neighbor['ip'] }} description {{ neighbor['description'] }}
{% endfor %}
 address-family ipv4
{% for net in bgp['networks'] %}
  network {{ net | ipaddr('network') }} mask {{ net | ipaddr('netmask') }}
{% endfor %}
{% for neighbor in bgp['neighbors'] %}
  neighbor {{ neighbor['ip'] }} activate
{% endfor %}
 exit-address-family
{# 
--------------
SWITCH TO NXOS
--------------
#}
{% elif ansible_network_os == 'nxos' %}
router bgp {{ bgp['asn'] }}
  router-id {{ bgp['id'] }}
  address-family ipv4 unicast
{% for net in bgp['networks'] %}
    network {{ net }}
{% endfor %}
{% for neighbor in bgp['neighbors'] %}
  neighbor {{ neighbor['ip'] }} remote-as {{ neighbor['asn'] }}
    description {{ neighbor['description'] }}
    address-family ipv4 unicast
{% endfor %}
{% endif %}

As you can see, it very quickly get’s complicated. A few basic options, each with pros and cons.

• Keep a flat structure, as originally shown, and leverage the similarities of the multiple OS’s to alleviate re-writing them, but add complexity to the use cases where there are differences.
• Keep a nested structure, one for each OS, and have the code copied over from one OS to the other, where it is the same configuration.
• Attempt to merge the two, and only leverage OS folder when complicated, and a default folder when not.

In this use case, my personal preference is to have a well defined OS structure as the next developer can review the tree structure (as depicted in the second option), and likely figure out where the configuration templates should go. Essentially, make it easier for the next developer to understand the layout, rather than worry about the duplication of code.

.
├── eos
│   ├── bgp.j2
│   ├── ntp.j2
│   └── vlan.j2
├── ios
│   ├── bgp.j2
│   ├── ntp.j2
│   └── vlan.j2
└── nxos
    ├── bgp.j2
    ├── ntp.j2
    └── vlan.j2

3 directories, 9 files

Situations like this will come up all the time, and you will simply have to use experience/intuition to understand the impact and make the best decision from there.