Network to Code has a repository created for maintaining templates that help to convert unstructured output from network devices into structured data. Google created TextFSM that provides a custom DSL to take command line output and return as structured data. Examples range from getting an output from a show lldp neighbor command and returning the list of neighbors to getting a list of addresses in an ARP table. This post is going to dive deeper into how to use these to get the data with Ansible.
You can find more about the ntc-templates on our Github page. These are templates that have been created by both Network to Code and by the community. It is an open source project that anyone is able to contribute a template to if there is not a template already created. Take a look at the readme for how you can help contribute to NTC Templates!
Ansible is not the only method for using TextFSM to get structured data returned from network device output. Several Python libraries such as Netmiko and Nornir are able to use TextFSM to return structured data by setting a flag to use TextFSM as well.
There are two primary methods for sending data through a TextFSM parser in Ansible. You will get to see examples for both of these. The first method available is to use the TextFSM Filter. The second is in conjunction with the Ansible Galaxy role Network Engine.
The first step towards leveraging these templates is getting the templates on to a compute device that is running Ansible. For this demo, we’ll be using a local machine and Git to clone the repository to a local directory.
git clone git@github.com:networktocode/ntc-templates.git
Next you will need to install TextFSM to the same Python interpreter that your Ansible installation is installed in. This demo will leverage Python3. For the installation you will also use the --user flag to install as the local user and --upgrade to ensure that the latest version of TextFSM is installed.
pip3 install textfsm --user --upgrade
For the second method of using Ansible Galaxy’s Network Engine you will also need to install the role as follows:
ansible-galaxy install ansible-network.network-engine
Now that the environment is setup it’s time to take a look at the Playbook and the execution.
This lab is setup as a 3 router traingle. RTR-1 will connect to one interface on RTR-2 and RTR-3. RTR-2 will have one connection to RTR-1 and RTR-3. RTR-3 will have the corresponding connections to RTR-1 and RTR-2.
---
- name: "PLAY 1: DEMO OF TEXTFSM"
hosts: routers
connection: network_cli
gather_facts: no
tasks:
- name: "TASK 1: GET COMMAND OUTPUT"
ios_command:
commands:
- show lldp neighbors
register: lldp_output
- name: "TASK 2: REGISTER OUTPUT TO DEVICE_NEIGHBORS VARIABLE"
set_fact:
device_neighbors: "{{ lldp_output.stdout[0] | parse_cli_textfsm('~/ntc-templates/templates/cisco_ios_show_lldp_neighbors.textfsm') }}"
- name: "TASK 3: PRINT OUTPUT"
debug:
msg: "{{ device_neighbors }}"
- name: "TASK 4: PRINT NEIGHBORS"
debug:
msg: "{{ item['LOCAL_INTERFACE'] }}: {{ item['NEIGHBOR'] }}"
loop: "{{ device_neighbors }}"
loop_control:
label: "{{ item['LOCAL_INTERFACE'] }}"
Walking through this first play, in Task 1 you are connecting to the device and running the command show lldp neighbors. This is saved to a variable named lldp_output that will be used later.
In Task 2 the output is being sent through the parse_cli_textfsm filter. The filter is being provided the library file to be used in the parsing. This is explicitely called out. The output from going through the TextFSM parser is then getting assigned to the variable device_neighbors, with each host in the execution having their own local instance of this variable.
In Task 3 you are getting to see the output of the variable. This shows the structured data. With this particular template you get the keys of CAPABILITIES, LOCAL_INTERFACES, NEIGHBOR, and NEIGHBOR_INTERFACE back. These are all defined within the TextFSM template.
In Task 4 you get to see a practical output to a screen if you want to audit and understand the neighbor relationships seen by LLDP on the network devices.
In this example you need to set a fact to be able to access the data later or continue to send the output through the parse_cli_textfsm filter every time you want to get at the structured data, such as a line number 15 in TASK 2.
Here is the output that corresponds with the play above.
ansible-playbook demo_ansible_textfsm.yml
PLAY [PLAY 1: DEMO OF TEXTFSM WITH CLI PARSER] *************************************************************************
TASK [TASK 1: GET COMMAND OUTPUT] **************************************************************************************
ok: [rtr-2]
ok: [rtr-3]
ok: [rtr-1]
TASK [TASK 2: REGISTER OUTPUT TO DEVICE_NEIGHBORS VARIABLE] ************************************************************
ok: [rtr-1]
ok: [rtr-2]
ok: [rtr-3]
TASK [TASK 3: PRINT OUTPUT] ********************************************************************************************
ok: [rtr-2] => {
"msg": [
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/0",
"NEIGHBOR": "rtr-3",
"NEIGHBOR_INTERFACE": "Gi0/0"
},
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/1",
"NEIGHBOR": "rtr-1",
"NEIGHBOR_INTERFACE": "Gi0/1"
}
]
}
ok: [rtr-1] => {
"msg": [
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/2",
"NEIGHBOR": "rtr-3",
"NEIGHBOR_INTERFACE": "Gi0/1"
},
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/1",
"NEIGHBOR": "rtr-2",
"NEIGHBOR_INTERFACE": "Gi0/1"
}
]
}
ok: [rtr-3] => {
"msg": [
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/1",
"NEIGHBOR": "rtr-1",
"NEIGHBOR_INTERFACE": "Gi0/2"
},
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/0",
"NEIGHBOR": "rtr-2",
"NEIGHBOR_INTERFACE": "Gi0/0"
}
]
}
TASK [TASK 4: PRINT NEIGHBORS FROM CLI PARSER] *************************************************************************
ok: [rtr-1] => (item=Gi0/2) => {
"msg": "Gi0/2: rtr-3"
}
ok: [rtr-1] => (item=Gi0/1) => {
"msg": "Gi0/1: rtr-2"
}
ok: [rtr-2] => (item=Gi0/0) => {
"msg": "Gi0/0: rtr-3"
}
ok: [rtr-2] => (item=Gi0/1) => {
"msg": "Gi0/1: rtr-1"
}
ok: [rtr-3] => (item=Gi0/1) => {
"msg": "Gi0/1: rtr-1"
}
ok: [rtr-3] => (item=Gi0/0) => {
"msg": "Gi0/0: rtr-2"
}
PLAY RECAP *************************************************************************************************************
rtr-1 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
rtr-2 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
rtr-3 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
The Ansible Role – Network Engine is a role that extracts information about network devices into Ansible Facts. You can use either TextFSM syntax or YAML (command_parser option) to extract information about a network device from the command output. You can read more about using the role at the network-engine Github page.
Here you will see a second method to get the same output with Network Engine instead of the CLI Parser. The biggest difference is that in this method, the Network Engine role will register the returned data to the ansible_facts instead of requiring you to set it to your own variable.
---
- name: "PLAY 2: DEMO OF TEXTFSM WITH NETWORK ENGINE"
hosts: routers
connection: network_cli
gather_facts: no
roles:
- ansible-network.network-engine
tasks:
- name: "TASK 1: GET COMMAND OUTPUT"
ios_command:
commands:
- show lldp neighbors
register: lldp_output
- name: "TASK 2: RUN THROUGH THE PARSER"
textfsm_parser:
file: "/Users/ntcblog/ntc-templates/templates/cisco_ios_show_lldp_neighbors.textfsm"
content: "{{ lldp_output.stdout[0] }}"
name: lldp_output
- name: "TASK 3: SHOW ANSIBLE FACTS OUTPUT"
debug:
msg: "{{ ansible_facts }}"
- name: "TASK 4: PRINT NEIGHBORS FROM ANSIBLE NETWORK ENGINE"
debug:
msg: "{{ item['LOCAL_INTERFACE'] }}: {{ item['NEIGHBOR'] }}"
loop: "{{ ansible_facts['lldp_output'] }}"
loop_control:
label: "{{ item['LOCAL_INTERFACE'] }}"
There is a new key that we needed to add to import the role that was installed with the ansible-galaxy command earlier. This is the roles: key that you see under gather_facts and contains a list of roles to import into the play. Here you see the import of the ansible-network.network-engine role.
In Task 1 you once again have the same command to gather the LLDP neighbors from the device.
In Task 2 instead of registering to a fact and sending through a filter, you now use the module textfsm_parser that takes file as a parameter that is the file of the TextFSM template. This is the same template referenced in the filter on the first play. You also pass content of what output you want to send through the parser. The last parameter is the name of the fact that you are registering to ansible_facts.
In Task 3, you once again get to see the output for ansible_facts. This will show that there are more details available about the device.
In Task 4 you get to see the same output as Play 1 where you get the neighbors printed out.
ansible-playbook demo_ansible_textfsm.yml
PLAY [PLAY 2: DEMO OF TEXTFSM WITH NETWORK ENGINE] *********************************************************************
TASK [TASK 1: GET COMMAND OUTPUT] **************************************************************************************
ok: [rtr-2]
ok: [rtr-3]
ok: [rtr-1]
TASK [TASK 2: RUN THROUGH THE PARSER] **********************************************************************************
ok: [rtr-1]
ok: [rtr-2]
ok: [rtr-3]
TASK [TASK 3: SHOW ANSIBLE FACTS OUTPUT] *******************************************************************************
ok: [rtr-1] => {
"msg": {
"discovered_interpreter_python": "/usr/bin/python",
"lldp_output": [
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/2",
"NEIGHBOR": "rtr-3",
"NEIGHBOR_INTERFACE": "Gi0/1"
},
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/1",
"NEIGHBOR": "rtr-2",
"NEIGHBOR_INTERFACE": "Gi0/1"
}
]
}
}
ok: [rtr-2] => {
"msg": {
"discovered_interpreter_python": "/usr/bin/python",
"lldp_output": [
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/0",
"NEIGHBOR": "rtr-3",
"NEIGHBOR_INTERFACE": "Gi0/0"
},
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/1",
"NEIGHBOR": "rtr-1",
"NEIGHBOR_INTERFACE": "Gi0/1"
}
]
}
}
ok: [rtr-3] => {
"msg": {
"discovered_interpreter_python": "/usr/bin/python",
"lldp_output": [
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/1",
"NEIGHBOR": "rtr-1",
"NEIGHBOR_INTERFACE": "Gi0/2"
},
{
"CAPABILITIES": "R",
"LOCAL_INTERFACE": "Gi0/0",
"NEIGHBOR": "rtr-2",
"NEIGHBOR_INTERFACE": "Gi0/0"
}
]
}
}
TASK [TASK 4: PRINT NEIGHBORS FROM ANSIBLE NETWORK ENGINE] *************************************************************
ok: [rtr-1] => (item=Gi0/2) => {
"msg": "Gi0/2: rtr-3"
}
ok: [rtr-1] => (item=Gi0/1) => {
"msg": "Gi0/1: rtr-2"
}
ok: [rtr-2] => (item=Gi0/0) => {
"msg": "Gi0/0: rtr-3"
}
ok: [rtr-3] => (item=Gi0/1) => {
"msg": "Gi0/1: rtr-1"
}
ok: [rtr-2] => (item=Gi0/1) => {
"msg": "Gi0/1: rtr-1"
}
ok: [rtr-3] => (item=Gi0/0) => {
"msg": "Gi0/0: rtr-2"
}
PLAY RECAP *************************************************************************************************************
rtr-1 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
rtr-2 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
rtr-3 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
This shows that there are multiple ways of pairing TextFSM templates with Ansible. Getting structured data out of unstructured output is extremely valuable when it comes to automating a network environment. There are over 300 different templates currently in the NTC-Templates repository to help get structured data out of your unstructured data.
-Josh
Does this all sound amazing? Want to know more about how Network to Code can help you do this, reach out to our sales team. If you want to help make this a reality for our clients, check out our careers page.
Share details about yourself & someone from our team will reach out to you ASAP!
