Welcome to the first post in this series about parsing unstructured data into structured data. When beginning your automation journey, you may start with quick wins that may not need to act upon operational data from show commands, but as you progress quickly through your journey, you will find the need to be able to parse the unstructured data obtained from your devices into structured data.
Unfortunately at this time, not all of us have been able to replace our “legacy” network equipment with all the newer networking products that come with APIs, streaming telemetry, etc. that help us programmatically interact with our network.
There are several parsing strategies that we will cover in greater detail along with methods to consume them:
We’ve covered parsing lightly in previous posts that use the parsing of unstructured data such as this post, to transform the data into something useable by other systems. This series will take us deeper into the “how” of parsing unstructured data.
Before we start diving too deep into the implementations, let’s discuss why parsing unstructured data into structured data is beneficial.
Parsing is the act of translating a language (unstructured data that humans can easily read) to another language (structured data that a computer can easily read). Below is an example of how we’d do some form of validation with unstructured data:
>>> unstructured_data = """
... Capability codes:
... (R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
... (W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other
...
... Device ID Local Intf Hold-time Capability Port ID
... S2 Fa0/13 120 B Gi0/13
... Cisco-switch-1 Gi1/0/7 120 Gi0/1
... Juniper-switch1 Gi2/0/1 120 B,R 666
... Juniper-switch1 Gi1/0/1 120 B,R 531
...
... Total entries displayed: 4
"""
>>> neighbors = [
... "S2",
... "Cisco-switch-1",
... "Juniper-switch1",
]
>>> for neighbor in neighbors:
... if neighbor in unstructured_data:
... print(f"{neighbor} on router")
S2 on router
Cisco-switch-1 on router
Juniper-switch1 on router
>>> neighbors = [
... {"name": "S2", "intf": "Fa0/13"},
... {"name": "Cisco-switch-1", "intf": "Gi1/0/7"},
... {"name": "Juniper-switch1", "intf": "Gi2/0/1"},
... {"name": "Juniper-switch1", "intf": "Gi1/0/1"},
... ]
>>> for neighbor in neighbors:
... for cfg_line in unstructured_data.splitlines():
... if neighbor["name"] in cfg_line and neighbor["intf"] in cfg_line:
... print(f"Neighbor {neighbor["name"]} is seen on {neighbor["intf"]}")
Neighbor S2 is seen on Fa0/13
Neighbor Cisco-switch-1 is seen on Gi1/0/7
Neighbor Juniper-switch1 is seen on Gi2/0/1
Neighbor Juniper-switch1 is seen on Gi1/0/1
Luckily, we can parse this data and perform meaningful comparisons on the data once we have transformed it into structured data. This gives us the ability to assert, with confidence, that the neighbors that are seen match the expected interfaces. This check can be critical in making sure that the correct configuration exists on the correct interfaces for each device.
Here is a short list that provides a few use cases as to why you may want to turn your unstructured data into structured data.
Each of the following posts will work with the unstructured LLDP data obtained from csr1000v routers and used to assert that the neighbors that the device sees are valid neighbors per a variable we will define within the next post. This will help to determine which neighbors we’re expecting to see connected to each router. We will want to do two different checks; that each neighbor is what we are expecting to see, and that there aren’t any extra neighbors that we’re not expecting to see.
After reading these posts, you should be able to parse any unstructured data obtained from devices into structured data that is meaningful to you along your network automation journey!
The next post in this series will go over the topology we’ll be using throughout this series and take a dive into NTC Templates with Ansible.
-Mikhail
Share details about yourself & someone from our team will reach out to you ASAP!