Extracting Operational Data from Juniper vMX Routers using PyEZ

  • September 19, 2016

Objective

This tutorial describes how to extract operational data from Juniper vMX devices. In order to do this, we'll be using the Juniper PyEZ Python library. The Juniper PyEZ library is a micro-framework for Python that enables you to remotely manage and automate devices running Junos OS, including the ability to extract critical configuration and operational data.

The Juniper PyEZ framework does this through the concept of PyEZ Tables and Views.

Juniper supports two types of tables, Op and Config, and in this tutorial, we are focused on operational. Operational (op) tables select items from the RPC reply of an operational command and configuration tables select data from specific hierarchies in the selected configuration database. We'll show exactly what this means from an operational perspective.

Topology

This tutorial was written using the Juniper vMX 5-node topology.

Gather Operational Data

As a network engineer, the most common way we gather operational data is through show commands. Here we see the output of a show lldp neighbors command.

ntc@vmx1> show lldp neighbors
Local Interface    Parent Interface    Chassis Id          Port info          System Name
fxp0               -                   00:05:86:71:53:c0   fxp0               vmx2
ge-0/0/2           -                   00:05:86:71:53:c0   ge-0/0/2           vmx2
fxp0               -                   00:05:86:71:96:c0   fxp0               vmx3
ge-0/0/0           -                   00:05:86:71:96:c0   ge-0/0/0           vmx3

However, Junos CLI is actually an API client to the underlying XML API. You can pipe any operational command to xml, thus viewing the same output, but now as an XML object. Here we see the same output from the show lldp neighbors command, but now as XML.

ntc@vmx1> show lldp neighbors | display xml
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/15.1F4/junos">
    <lldp-neighbors-information junos:style="brief">
        <lldp-neighbor-information>
            <lldp-local-port-id>fxp0</lldp-local-port-id>
            <lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
            <lldp-remote-chassis-id-subtype>Mac address</lldp-remote-chassis-id-subtype>
            <lldp-remote-chassis-id>00:05:86:71:53:c0</lldp-remote-chassis-id>
            <lldp-remote-port-id-subtype>Interface name</lldp-remote-port-id-subtype>
            <lldp-remote-port-id>fxp0</lldp-remote-port-id>
            <lldp-remote-system-name>vmx2</lldp-remote-system-name>
        </lldp-neighbor-information>
        <lldp-neighbor-information>
            <lldp-local-port-id>ge-0/0/2</lldp-local-port-id>
            <lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
            <lldp-remote-chassis-id-subtype>Mac address</lldp-remote-chassis-id-subtype>
            <lldp-remote-chassis-id>00:05:86:71:53:c0</lldp-remote-chassis-id>
            <lldp-remote-port-id-subtype>Interface name</lldp-remote-port-id-subtype>
            <lldp-remote-port-id>ge-0/0/2</lldp-remote-port-id>
            <lldp-remote-system-name>vmx2</lldp-remote-system-name>
        </lldp-neighbor-information>
        <lldp-neighbor-information>
            <lldp-local-port-id>fxp0</lldp-local-port-id>
            <lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
            <lldp-remote-chassis-id-subtype>Mac address</lldp-remote-chassis-id-subtype>
            <lldp-remote-chassis-id>00:05:86:71:96:c0</lldp-remote-chassis-id>
            <lldp-remote-port-id-subtype>Interface name</lldp-remote-port-id-subtype>
            <lldp-remote-port-id>fxp0</lldp-remote-port-id>
            <lldp-remote-system-name>vmx3</lldp-remote-system-name>
        </lldp-neighbor-information>
        <lldp-neighbor-information>
            <lldp-local-port-id>ge-0/0/0</lldp-local-port-id>
            <lldp-local-parent-interface-name>-</lldp-local-parent-interface-name>
            <lldp-remote-chassis-id-subtype>Mac address</lldp-remote-chassis-id-subtype>
            <lldp-remote-chassis-id>00:05:86:71:96:c0</lldp-remote-chassis-id>
            <lldp-remote-port-id-subtype>Interface name</lldp-remote-port-id-subtype>
            <lldp-remote-port-id>ge-0/0/0</lldp-remote-port-id>
            <lldp-remote-system-name>vmx3</lldp-remote-system-name>
        </lldp-neighbor-information>
    </lldp-neighbors-information>
    <cli>
        <banner></banner>
    </cli>
</rpc-reply>

ntc@vmx1>

So how does this map back to Tables and Views? Well, the Junos OS is very structured being that it's XML based. And since it's structured, Juniper created a more abstract way to extract data from Junos devices without having to do much native programming. In fact, it's just a few lines of code when using Tables and Views, to extract any operational you need from Juons devices. We'll see what this means soon.

Juniper Tables and Views

You can build your own Tables and Views, however PyEZ includes a number of them already, many of which were open source contributions from other network engineers. They can be found in the lib/jnpr/junos/op directory if you already installed in the PyEZ library or viewed here.

If you view the list of Tables/Views, you can see each has two files. There is a Python file (that is exactly same between all of them) and a YAML file. The Python file is how the YAML object gets dynamically loaded as Python objects. An example of a Table and View is shown below. The table instructs the Juniper vMX how to get the data we want and the view is how we want to view the data - in simplest terms, they fields in the view becomes key-value pairs for each object extracted.

This tutorial is going to show how to use Tables and Views, not create your own. That is a topic that deserves a few tutorials on its own!

---
LLDPNeighborTable:
  rpc: get-lldp-neighbors-information
  item: lldp-neighbor-information
  key: lldp-local-interface | lldp-local-port-id
  view: LLDPNeighborView

LLDPNeighborView:
  fields:
    local_int: lldp-local-interface | lldp-local-port-id
    local_parent: lldp-local-parent-interface-name
    remote_type: lldp-remote-chassis-id-subtype
    remote_chassis_id: lldp-remote-chassis-id
    remote_port_desc: lldp-remote-port-description
    remote_sysname: lldp-remote-system-name

From this example you can get a feel for what data will be returned. For instance, you can expect to see either lldp-local-interface or lldp-local-port-id as local_int. In addition, all the keys are in a condensed more user friendly manner. More importantly, you can easily see how these values map back to the command output earlier when you issues the show lldp neighbor | display xml command.

Accessing Data From Tables/Views

There are a few things that you need to when you want to use Tables/Views, i.e. use Python to extract operational data.

First, go through the Tables and Views on GitHub, and see which make sense based on the data you want to extract. This next example we show how to get LLDP neighbors from the device.

Second, import the proper objects in Python. Take special note of the second import statement. You can start to see how PyEZ dynamically loads objects into Python based on how they are defined in YAML. As an example, they will always be from.junos.op.name_of_yaml_and_python_file import TABLE_NAME_FROM_YAML_FILE. This is an advanced programming concept, but extremely valuable as it allows non-programmers to extract key information from Junos devices.

Example:

>>> from jnpr.junos import Device
>>> from jnpr.junos.op.lldp import LLDPNeighborTable
>>>
>>> device = Device(host='vmx1', user='ntc', password='ntc123')
>>> device.open()
Device(vmx1)
>>>
>>> neighbors = LLDPNeighborTable(device)
>>> neighbors.get()
LLDPNeighborTable:vmx1: 4 items
>>>

As soon as you import the proper objects, you simply need to pass the Junos Device object as a parameter to the Table object and call the get method. When you do this, you're making a query to the device and getting the data defined in the Table and View.

At this point you have connected to the device and gathered the data. Now we need to make use of the data. You can optionally explore the help and dir functions while in the Python shell to understand what methods are available to use.

>>>
>>> help(neighbors)

>>> dir(neighbors)
['D', 'GET_ARGS', 'GET_RPC', 'ITEM_NAME_XPATH', 'ITEM_XPATH', 'RPC', 'VIEW', <-- removed for brevity -->,  'get', 'hostname', 'is_container', 'items', 'key_list', 'keys', 'savexml', 'to_json', 'values', 'view', 'xml']
>>>

We can use the to_json method to quickly view the data as JSON object.

>>> import json
>>>
>>> output_json = json.loads(neighbors.to_json())
>>> print json.dumps(output_json, indent=4)
{
    "ge-0/0/2": {
        "remote_port_desc": null,
        "local_int": "ge-0/0/2",
        "remote_sysname": "vmx2",
        "local_parent": "-",
        "remote_chassis_id": "00:05:86:71:53:c0",
        "remote_type": "Mac address"
    },
    "fxp0": {
        "remote_port_desc": null,
        "local_int": "fxp0",
        "remote_sysname": "vmx3",
        "local_parent": "-",
        "remote_chassis_id": "00:05:86:71:96:c0",
        "remote_type": "Mac address"
    },
    "ge-0/0/0": {
        "remote_port_desc": null,
        "local_int": "ge-0/0/0",
        "remote_sysname": "vmx3",
        "local_parent": "-",
        "remote_chassis_id": "00:05:86:71:96:c0",
        "remote_type": "Mac address"
    }
}
>>>

Looping through Table Items

Viewing the data in real time is helpful, but ultimately you want to be able to work with the data. Here is a small example of what could be done. Whenever you extract information using Tables/Views, you can access each field that is in the View as a property of the individual item.

>>> for item in neighbors:
...   print "interface " + item.local_int + " is connected to: " + item.remote_sysname
...
interface fxp0 is connected to: vmx2
interface ge-0/0/2 is connected to: vmx2
interface fxp0 is connected to: vmx3
interface ge-0/0/0 is connected to: vmx3

Not shown in the example above, each item also has a property called name that is equal to the key you defined in the Table.

Other Examples

Here we can see the same process for viewing the arp table, that you also automatically get when you install the PyEZ library.

>>> from jnpr.junos import Device
>>> from jnpr.junos.op.arp import ArpTable
>>>
>>> device = Device(host='vmx1', user='ntc', password='ntc123')
>>> device.open()
Device(vmx1)
>>>
>>> arp = ArpTable(device)
>>> arp.get()
ArpTable:vmx1: 5 items
>>>
>>> import json
>>>
>>> output_json = json.loads(arp.to_json())
>>> print json.dumps(output_json, indent=4)
{
    "2c:c2:60:5f:41:f0": {
        "interface_name": "ge-0/0/2.0",
        "ip_address": "10.10.10.2",
        "mac_address": "2c:c2:60:5f:41:f0"
    },
    "2c:c2:60:61:6f:b6": {
        "interface_name": "em1.0",
        "ip_address": "128.0.0.16",
        "mac_address": "2c:c2:60:61:6f:b6"
    },
    "2c:c2:60:1b:df:85": {
        "interface_name": "ge-0/0/0.0",
        "ip_address": "10.10.30.3",
        "mac_address": "2c:c2:60:1b:df:85"
    },
    "2c:c2:60:ff:00:27": {
        "interface_name": "fxp0.0",
        "ip_address": "10.0.0.2",
        "mac_address": "2c:c2:60:ff:00:27"
    },
    "2c:c2:60:0e:ed:1e": {
        "interface_name": "fxp0.0",
        "ip_address": "10.0.0.5",
        "mac_address": "2c:c2:60:0e:ed:1e"
    }
}
>>>

Now that you have the data in a structured data language you can programmtically build off that. You can combine arp tables from multiple devices, compare mac address tables against arp tables, and the list goes on.

Sample Script

Here is a working script that you can also try using the Network to Code On Demand Labs.

#! /usr/bin/env python

from jnpr.junos.op.lldp import LLDPNeighborTable
from jnpr.junos import Device
from jnpr.junos.op.arp import ArpTable

def get_details(host):
    user = 'ntc'
    password = 'ntc123'
    device = Device(host=host, user='ntc', password='ntc123')
    device.open()
    arp = ArpTable(device)
    arp.get()
    neighbors = LLDPNeighborTable(device)
    neighbors.get()
    print "Device: " + host
    print "LLDP Information"
    for item in neighbors:
        print "interface " + item.local_int + " is connected to: " + item.remote_sysname
    print "ARP Information"
    for item in arp:
        print "interface " + item.interface_name + " has IP->MAC " + item.ip_address  + "->" + item.mac_address

def main():
    device_list = ['vmx1', 'vmx2', 'vmx3' ]
    for current_host in device_list:
        get_details(current_host)

if __name__ == "__main__":
    main()