Introducing Netutils

If you have been in the Network Automation space for any period of time, you will quickly find yourself recreating the same small snippets of code over and over for common tasks such as IP or MAC address manipulation. I have personally found myself consistently copying code from one project to the next. Netutils intends to be a lightweight Python library that provides quick and easy access to these functions.

For years, this concept had been on my mind, but I was finally pushed into action by a short series of tweets, even being advised against pursuing 🙂

Tenets

In creating this library it was important to keep it well-scoped, which is the genesis for the following tenets.

• Must not be any dependencies required to run the library.
  • May be some dependencies, which will be managed by users in an opt-in fashion.
• Shall prefer functional programming over object-oriented programming.
• Shall retain a folder and file structure that is flat.
• Shall leverage docstrings as the primary documentation mechanism.
  • Must provide examples in the docstring of every public function.
• Shall retain a high test coverage.

As you can see, the tenets are there to protect the project from bloat, and in theory, become more likely to be used as the library will not come with a host of dependencies that do not apply to your specific use case.

Capabilities

At launch, the library has a series of capabilities provided.

• BGP ASN – Provides the ability to convert BGP ASN from integer to dot notation.
• Configuration
  • Cleaning – Provides the ability to remove or replace lines based on RegEx matches.
  • Compliance – Provides the ability to compare two configurations to sanely understand the differences.
  • Parsing – Provides the ability to parse configuration for the minor differences that are there.
• Interface – Provides the ability to work with interface names, expanding, abbreviating, and spliting the names.
• IP Address – Provides the ability to work with IP addresses, primarily exposing Python ipaddress functionality.
• Library Mapper – Provides mappings in expected vendor names between Netmiko, NAPALM, pyntc, ntc-templates, pyats, and scrapli.
• MAC Address – Provides the ability to work with MAC addresses such as validating or converting to integer.
• Password – Provides the ability to compare and encrypt common password schemas such as type5 and type7 Cisco passwords.
• Protocol Mapper – Provides a mapping for protocol names to numbers, and vice versa.
• Route – Provides the ability to pass in a list of routes and an IP Address and return the longest prefix-matched route.
• Vlans – Provide the ability to convert configuration into lists or lists into configuration.

Exploration

Let’s explore a few of the more interesting use functions provided by the library.

Configuration Compliance

This is perhaps the most interesting of use cases, and all credit goes to Jacob McGill for developing the library. The netutils.config.compliance.compliance function allows you to send two sets of configuration a compare them. There is a concept of features, which defines the scope or sections of the configuration. This function is the basis of what is used in the Nautobot Golden Config Plugin and can also be used to leverage idempotency. In fact, it was based off Ansible’s *_config module, which we at Network to Code used until we ran into performance limitations.

>>> from netutils.config.compliance import compliance
>>>
>>> features = [
...     {
...         "name": "hostname",
...         "ordered": True,
...         "section": [
...             "hostname"
...         ]
...     },
...     {
...         "name": "ntp",
...         "ordered": True,
...         "section": [
...             "ntp"
...         ]
...     }
... ]
>>>
>>> backup = "ntp server 192.168.1.1\nntp server 192.168.1.2 prefer"
>>>
>>> intended = "ntp server 192.168.1.1\nntp server 192.168.1.5 prefer"
>>>
>>> network_os = "cisco_ios"
>>>
>>> compliance(features, backup, intended, network_os, "string")
{
  "hostname": {
    "compliant": true,
    "missing": "",
    "extra": "",
    "cannot_parse": true,
    "unordered_compliant": true,
    "ordered_compliant": true,
    "actual": "",
    "intended": ""
  },
  "ntp": {
    "compliant": false,
    "missing": "ntp server 192.168.1.5 prefer",
    "extra": "ntp server 192.168.1.2 prefer",
    "cannot_parse": true,
    "unordered_compliant": false,
    "ordered_compliant": false,
    "actual": "ntp server 192.168.1.1\nntp server 192.168.1.2 prefer",
    "intended": "ntp server 192.168.1.1\nntp server 192.168.1.5 prefer"
  }
}
>>>

Route Lookups

Another interesting use case, introduced by Jeff Kala, was a route lookup. Given a list of routes, find the most specific one.

>>> from netutils.route import longest_prefix_match
>>>
>>> lookup = "10.1.1.245"
>>>
>>> routes = [{"network": "192.168.1.1", "mask": "255.255.255.255"},{"network": "10.1.1.0", "mask": "24"}, {"network": "10.1.0.0", "mask": "16"}]
>>> 
>>> longest_prefix_match(lookup, routes)
'10.1.1.0/24'
>>>

Mappers

Given the array of networking libraries out there including Netmiko, NAPALM, pyntc, ntc-templates, pyats, and scrapli, it is unlikely you’re using only one. Maintaining an inventory that maps between them can be a tedious, error-prone task. Netutils library provides dictionaries that allow you to map bi-directionally from one library to another.

As an example, you can see a simple mapper.

PYNTC_LIB_MAPPER_REVERSE = {
    "cisco_asa": "cisco_asa_ssh",
    "arista_eos": "arista_eos_eapi",
    "f5_tmsh": "f5_tmos_icontrol",
    "cisco_ios": "cisco_ios_ssh",
    "juniper_junos": "juniper_junos_netconf",
    "cisco_nxos": "cisco_nxos_nxapi",
    "cisco_wlc": "cisco_aireos_ssh",
}

From here you can see how we can take a known NAPALM OS from our inventory, and cast it to a variety of equivalents.

>>> from netutils import lib_mapper
>>>
>>> known_napalm_os = "junos"
>>>
>>> normalized_os = lib_mapper.NAPALM_LIB_MAPPER[known_napalm_os]
>>>
>>> normalized_os
'juniper_junos'
>>>
>>> juniper_junos_netconf
>>>
>>> print(lib_mapper.ANSIBLE_LIB_MAPPER_REVERSE[normalized_os])
junipernetworks.junos.junos
>>>
>>> print(lib_mapper.PYATS_LIB_MAPPER_REVERSE[normalized_os])
junos
>>>

Similarly, you can see how to map from protocol integer to protocol name, and back.

>>> from netutils.protocol_mapper import PROTO_NUM_TO_NAME, PROTO_NAME_TO_NUM
>>>
>>> PROTO_NUM_TO_NAME[1]
'ICMP'
>>>
>>> PROTO_NAME_TO_NUM['ICMP']
1
>>>

Quick Hits

Just to show a few more, take notice of these quick hits.

The following function will help in deploying a list of VLANs and match the configuration style in a standard IOS-like configuration.

>>> from netutils.vlan import vlanlist_to_config
>>>
>>> vlan_cfg = vlanlist_to_config([1, 2, 3, 5, 6, 1000, 1002, 1004, 1006, 1008, 1010, 1012, 1014, 1016, 1018])
>>>
>>> vlan_cfg
['1-3,5,6,1000,1002,1004,1006,1008,1010,1012,1014', '1016,1018']
>>> for index, line in enumerate(vlan_cfg):
...     if index == 0:
...         print(f"  switchport trunk allowed vlan {line}")
...     else:
...         print(f"  switchport trunk allowed vlan add {line}")
... 
  switchport trunk allowed vlan 1-3,5,6,1000,1002,1004,1006,1008,1010,1012,1014
  switchport trunk allowed vlan add 1016,1018
>>> 

You may want to compare a known password with a given encrypted password. This can help by verifying if the passwords are as expected for compliance reasons.

>>> from netutils.password import compare_type5
>>>
>>> compare_type5("cisco","$1$nTc1$Z28sUTcWfXlvVe2x.3XAa.")
True
>>> compare_type5("not_cisco","$1$nTc1$Z28sUTcWfXlvVe2x.3XAa.")
False
>>>

Oftentimes interfaces will come in various different shortened names, and it is helpful to normalize them.

>>> from netutils.interface import canonical_interface_name
>>>
>>> canonical_interface_name("Gi1/0/1")
'GigabitEthernet1/0/1'
>>>
>>> canonical_interface_name("Eth1")
'Ethernet1'
>>>

Testing

Testing is driven largely by doctest (special thanks to Adam Byczkowski for the suggestion) and pytest. Doctest will actually compile the examples of code created and indicate whether or not the result is as intended. This is helpful to keep documentation efforts to a minimum by creating self-documenting code. For configuration-based testing, it largely revolves around creating mocked data tests.

Overall, the intention is to maintain a high percentage of testing, and based on the structure and nature of the project, that should be easily doable.

Origins & Attribution

The code has its roots in common network automation tasks. Therefore, examples can be found throughout the open source world. With few exceptions (that are explicitly noted in the repo) the actual code is either unattributable, or was created by someone at Network to Code, often as a contribution to other open source libraries. In regard to what is “unattributable” code, the best example would be Cisco type7 encryption/decryption. This code was clearly not first introduced in this repo; however, it is unclear who the original author is in the sea of prior art. For further attribution, see the project README.