My first REST API work with Python

Blog Detail

I come from a network engineering background, so when I started my network automation journey, it felt like I was banging my head against new technology with very little help. When I first started using APIs, our network team had just started with this brand new ACI “thing,” and I couldn’t find a well written “What is an API” blog with code I would call “appropriate.” In this blog, I will attempt to provide that for those of you that are just getting started with APIs. If you have been using APIs for a while, this may not be as useful.

One of the hardest parts of writing this was choosing an API to discuss. Some of the things I really wanted from an API were:

  • A Swagger page: I always thought of this as a webpage that has the API laid out nicely and gives the opportunity to run the API calls directly on it. This is more technically known as an Open API spec.
  • A platform that was common in the networking world that people would be likely to run across.
  • A platform that was easily available and didn’t require you to build a new system.

Sadly, this combo doesn’t seem to exist. In the end, I decided to go over 2 APIs:

  • NetBox
    • “An open source web application designed to help manage and document computer networks.”
    • Documentation that is easy for a beginner with its Swagger page.
    • https://netbox.readthedocs.io/en/stable/
  • ACI
    • A Network policy-based automation model for data centers.
    • Cisco DevNet has an always on sandbox where you can access a working ACI environment at any time.
    • When I was working on this API about a year ago, I found the documentation hard to navigate. I can see how it would work well for someone with more experience, but it didn’t work well for me at the time, and I haven’t kept up to date to know if their documentation has gotten more friendly.
    • https://learningnetwork.cisco.com/docs/DOC-32331

REST API Core Concepts

While I’m not going to go to in depth, I want to walk through a few examples to give you the general idea of how a REST API works.

There are two main methods of authentication. One where the API is provided a username and password and the system gives back a temporary token. This process is often used in systems where you are running a system such as ACI. In the other method, a token you generate from a UI serves as your authentication. These are often used for web applications such as Slack, Twitter, Facebook, and fortunately NetBox, so we can see how to do that.

The general idea behind the REST API is you are going to an URL rather than typing in commands. Let’s look at this like a Cisco CLI. For example, your command might be show run interface gi1. The equivalent URL might be something like device.com/api/conf/interface/gi1 so to work with gi2 the URL would just change device.com/api/conf/interface/gi2.

The main types of calls are below.

  • GET
    • Pulling data.
    • You can’t really cause an issue doing this as long as you don’t pull too much at one time, e.g. pulling 1,000,000 records may bog down the system.
    • Equivalent of a show command.
  • PUT/POST/PATCH
    • Changing or creating data.
    • Equivalent of configuration changes.
  • DELETE
    • Delete data.

With the same examples from above, we will do a GET on device.com/api/conf/interface/gi1. It returns the dictionary below:

{
    description: "Disconnected"
    status: "shut"
    mode: "access"
    vlans: 110
}

We now know the description on the interface, the VLAN, and that it is shut down. Let’s say we wanted to make a change, just update it as below and PUT it to the same URL.

{
    description: "VM Server Bob"
    status: "no shut"
    mode: "trunk"
    vlans: "10-100"
}

The port is now up, with a new description, as a trunk allowing VLANs 10-100 though. While not quite as easy to understand the url above this URL https://device.com/restconf/data/Cisco-IOS-XE-native:native/interface=GigabitEthernet/1 would work for the RESTCONF API for IOS-XE devices and has an easy to read URL.

NetBox

I am running an instance of NetBox in my lab, the IP is 192.168.0.159, so that’s what all my examples will be of. It’s not hardened or using https, so all of my examples are using http on port 8000. Your box may vary, adjust accordingly.

First, let’s build and grab a token from http://192.168.0.159:8000/user/api-tokens/.

Netbox_get_token

Next, we’ll look at the swagger documentation page at http://192.168.0.159:8000/api/docs/. Login to the session

authorize_swagger
paste_in_token

and find an action to do, (e.g. get a device from DCIM,) select that, then select “Try it out.”

try_it_out

We can pull all devices if we were to look at another option, but the API call gives us lots of options. A screenshot would be overly busy, so I am just going to pull one device in this example. Please note the required tag.

execute

The API call gives us all the data for that we asked for. Please pay special note to the cURL script. That is the call it made to the NetBox API. It sent a GET request to http://192.168.0.159:8000/api/dcim/devices/5/ please note the initial line we chose was /dcim/devices/{id}/

gives_all_data

The cURL script is running an HTTP request to the URL listed, and you could run the same request on nearly all systems. For example, right now you can run “curl http://www.google.com” and make the HTTP call to Google. One of the really nice things about that cURL script is we can go to https://curl.trillworks.com, paste in the cURL script, and it will give us the Python equivalent. NOTE that if the request has the token in it you should be careful. Swap out the token before posting it to the website unless you like giving admin credentials to strangers.

gives_python_code

Honestly there’s not much to be scared of here. Just working though Swagger and putting a few functions together from what you get from the website. Super easy… Now let’s look at ACI.

ACI

In this example, the Cisco DevNet ACI emulator will be used. Set the username, password, and the base URL. This URL will be in every call made. To compare to the NetBox example, the base_url would have been http://192.168.0.159:8000/api.

username = "admin"
password = "ciscopsdt"
base_url = "https://sandboxapicdc.cisco.com/api/"

The login script is honestly one of the most difficult parts of the whole process, as documentation is often sparse and there is no industry standard for field names across REST APIs. The process involves simply logging in, getting a token, and returning it. This is how many APIs work, and there are generally small differences for every API. Most of this was from sample code obtained from DevNet a few years ago.

def get_token(username, password, base_url):
    # Disable warning messages that come with using self signed certs
    requests.packages.urllib3.disable_warnings()
    # create credentials structure
    name_pwd = {"aaaUser": {"attributes": {"name": username, "pwd": password}}}
    json_credentials = json.dumps(name_pwd)
    # Create the full URL we need in order to login
    # You'll find we do this a lot
    login_url = base_url + "aaaLogin.json"
    # log in to API
    # requests is a library that can do HTTP transactions, in this case we POSTing up (Telling it) our
    # credentials to get the token back
    # Verify=False ignores self signed certs, but would kick off errors if we hadn't disabled them up above
    post_response = requests.post(login_url, data=json_credentials, verify=False)
    # getting the token from login response structure
    auth = json.loads(post_response.text)
    login_attributes = auth["imdata"][0]["aaaLogin"]["attributes"]
    auth_token = login_attributes["token"]
    # create token dictionary that we will use for authentication from here on out for auth cookie
    cookies = {}
    cookies["APIC-Cookie"] = auth_token
    return cookies

Let’s review the documentation for the ACI API. Once again, this is how it worked back in late 2018, the documentation may have changed since. Access the API inspector,

get_api_inspector

then proceed to the interesting area in the GUI, paying close attention to your last click. The example will return with the switch information, so I went to Fabric (top tap) -> Inventory -> POD1 (expanded). Sadly, the URL alone doesn't always give you exactly what you need. In this example I had to cut off the &subscription=yes` section. Often, determinng the URL can be the hardest part.

find_call

You can also see that the URL is asking for JSON in the pod-1.json? section. For ACI, it will return either XML or JSON. Personally, I find JSON easier to deal with in Python. Using the JSON library it’s not hard to convert JSON to a dictionary; https://www.w3schools.com/python/python_json.asp does a better job at explaining the details. Let’s take a look at this process.

import json
import requests
from pprint import pprint

def pull_pod1(cookies, base_url):
    sensor_url = (
        base_url + '/node/mo/topology/pod-1.json?query-target=children&target-subtree-class=fabricNode&query-target-filter=and(not(wcard(fabricNode.dn,%22__ui_%22)),and(ne(fabricNode.role,"controller")))'
    )
    get_response = requests.get(sensor_url, cookies=cookies, verify=False)
    return get_response.json()


username = "admin"
password = "ciscopsdt"
base_url = "https://sandboxapicdc.cisco.com/api/"
## Calling the authentication function above to get the auth cookie
cookies = get_token(username, password, base_url)
pprint(pull_pod1(cookies, base_url))

This gives us all the devices in POD1. I’ll just be putting in one device for the sake of space

{
  "imdata": [
    {
      "fabricNode": {
        "attributes": {
          "adSt": "on",
          "address": "10.0.112.64",
          "annotation": "",
          "apicType": "apic",
          "childAction": "",
          "delayedHeartbeat": "no",
          "dn": "topology/pod-1/node-101",
          "extMngdBy": "",
          "fabricSt": "active",
          "id": "101",
          "lastStateModTs": "2019-12-12T14:22:14.238+00:00",
          "lcOwn": "local",
          "modTs": "2019-12-12T14:23:02.315+00:00",
          "model": "N9K-C9396PX",
          "monPolDn": "uni/fabric/monfab-default",
          "name": "leaf-1",
          "nameAlias": "",
          "nodeType": "unspecified",
          "role": "leaf",
          "serial": "TEP-1-101",
          "status": "",
          "uid": "0",
          "vendor": "Cisco Systems, Inc",
          "version": ""
        }
      }
    }
  ]
}

Note: Output omitted and modified for the sake of brevity and clarity.

You can observe several key pieces of information about this switch. The DN info, ID, model number, serial number, etc. With ACI, I often had to pull one piece of data so that I could understand how to pull another piece of data. Finding the DN was almost always an important step in my programs.

There is an easier, albeit less stable way (based on my observations) to find the URLs we want. Right-click on an interesting object and select “Open in Object Store Browser.” That will direct to a login screen. After login, the UI brings up a page where it should display the query and give the output.

open_in_object_browser

To see the URL it used we click on “Show URL and response of last query.”

See_url_from_browser

You can navigate the API structure and find what you are looking for. It may take some practice to develop a full understanding of this method.

I hope this has been helpful. I know it can be overwhelming, but the benefits of working with an API vs clicking though a web-page are wonderful. For example, before I left my last job I wrote a Python program that could be run on demand. It pulled what VLANS went to what VM Servers and looked for errors on interfaces. We got a call that there were lots of outages going on. The other network engineers were manually logging into everything looking for issues, but I just ran my program and was able to find a missing VLAN on a trunk to a VM server. The VM failed because it could no longer communicate out-side of the VM server. It took approximately five minutes to find and fix the issue. With dozens of VM servers and very little info on what the actual issue was, I believe that outage would have been much longer if people were looking for the issue by hand.

-Daniel



ntc img
ntc img

Contact Us to Learn More

Share details about yourself & someone from our team will reach out to you ASAP!

Basic Slack use with Python

Blog Detail

At my last job many fellow enigneers would made the claim that “I like the idea, and I like the info you can provide, but I am not going to build an entire application, with the proper code and prerequistes, for this use case.” I ended up building a webpage, but with all the security concerns, and limited resources it wasn’t a good fit for production use. However, it was adopted in an unofficial manner and that in turn expanded who could use my programs significantly. Not just other network engineers, but it also made life easier for the system admins. For example: when a system admin wanted to know what VLANs were configured to which VM servers they could just go to a webpage, view the simplified layout, and it would provide them the information in a format they were comfortable with. External users didn’t have to put in tickets and wait a week, network engineering didn’t have to stop what we were doing and look at those tickets. So, I love the idea, but my implementation was not ready for production with a proper support model. I have seen several other developers doing something along these lines, but they often seem to hit the same issues I did. I had see a few of the bigger brain developers I know build some Slack integrations, and I thought “Hey that’s cool, but I bet it’s REALLY complex”…. I was wrong: it isn’t hard at all.

Prerequisites

Let’s take a look at what we do to build a Slack integrated program. After you get your Slack account and channels setup you need to get your token which should look something like this “xoxp-XXXXXXXX-XXXXXXXX-XXXXX”

Figuring out where the channel’s ID is can be a bit difficult. All you need to do though is right click the channel you want to interact with, and select copy link paste that into your text editor, and pull out the ID.

copy slack
link

To get your token just go to https://api.slack.com/custom-integrations/legacy-tokens Scroll down to the “Legacy information” section and click Issue token/Re-issue token

get_slack_token

Post a message:

This is all the Python code needed to build a function send a message in Slack.

def post_to_slack(message, channel, token):
	slack_token = token
	client = slack.WebClient(token=slack_token)
	client.chat_postMessage(
	channel = channel,
	text=message)

Pull the last message:

Similar amount of code for building a function to pull a message in Slack.

def get_last_message(token, channel):
	url = 'https://slack.com/api/conversations.history?token={}&channel={}&limit=1&pretty=1'.format(token, channel)
	r = requests.get(url)
	all_response = r.json()
	last_message = all_response['messages'][0]['text']
	return last_message

You can test this mini Slack application without worry about Python here: This particular API tests obtaining Slack conversations. Please note when it asks for a channel it is looking for a unique identifier like CP02W0ABQ, not #general. https://api.slack.com/methods/conversations.history/test

So we post in some text, and that causes our program to kick off it’s work.

slack img

Solution

Lets take a quick look at the final code used.

from pprint import pprint
import requests
import os
import slack
import json
import time
import netmiko

token = "xoxp-fake-token"      
#  The account that will be used to SSH into the devices
username = 'username'
password = 'password'

#Fake channel
channel ="BAR2FUFM"

#SSH or Telnet to the Cisco device
#If it fails to SSH and Telnet add an entry to Issues.csv
def make_connection(ip, username, password):
	try:
		net_connect = netmiko.ConnectHandler(device_type = 'cisco_ios', ip = ip, username = username, password = password)
		return net_connect
	except:
		try:
			return netmiko.ConnectHandler(device_type = 'cisco_ios_telnet', ip = ip, username = username, password = password)
		except:
			issue = ip + ", can't be ssh/telneted to"
			to_doc_a("Issues.csv", issue)
			to_doc_a("Issues.csv", '\n')
			return None

#Send a command to the device that has been SSHed to
def send_command(net_connect,command):
	return net_connect.send_command_expect(command)


#In this case you pass in the output from show interfaces but split into a list based on line return
#This splits the interfaces up into their own list
#So all[0]  will give you a list of lines of the first interface
#all[0][0] will give you the first line of the first interface
def find_child_text (file, text):
	all = []
	parse = CiscoConfParse(file)
	for obj in parse.find_objects(text):
		each_obj = []
		each_obj.append(obj.text)
		for each in obj.all_children:
			each_obj.append(each.text)
		all.append(each_obj)
	return all


#Post new message to slack channel in question
def post_to_slack(message, channel, token):
	slack_token = token
	client = slack.WebClient(token=slack_token)
	client.chat_postMessage(
	channel=channel,
	text=message)

#Pull last 1 message from slack channel in question
def get_last_message(token, channel):
	url = 'https://slack.com/api/conversations.history?token={}&channel={}&limit=1&pretty=1'.format(token, channel)
	r = requests.get(url)
	all_response = r.json()
	last_message = all_response['messages'][0]['text']
	return last_message

#All the devices in the lab, and they should all be eigrp neighbors		  
def check_lab():
	lab_devices=[
	'192.168.0.12',
	'192.168.0.13',
	'192.168.0.14',
	'192.168.0.104',
	#An IP I know won't be there so there are issues
	'1.1.1.1'
	]
	
	#Hit each device in the lab
	for ip in lab_devices:
		info_output = "*checking {}*".format(ip)
		#Post updates to slack so they know what's happening
		post_to_slack(info_output, channel, token)
		#SSH to device
		net_connect = make_connection(ip, username, password)
		#If it fails to connect post the error to slack and move to the next one
		if net_connect == None:
			issue = "      can't connect to {}".format(ip)
			post_to_slack(issue, channel, token)
			continue
		command = 'show ip eigrp nei'
		#Get the output from the command sent
		output = send_command(net_connect,command)
		#Chek and make sure each device is listed as a neighbor, or is the one we SSHed to
		for neighbor in lab_devices:
			if ip == neighbor:
				continue
			if neighbor not in output:
				issue = '      {} is not found in the active neighbors'.format(neighbor)
				#If it's not there post the issue to slack
				post_to_slack(issue, channel, token)
		#Pull the interface data, and look for collisions		
		command = 'show interfaces'
		output = send_command(net_connect,command)
		output = output.split("\n")
		interfaces = find_child_text (output, 'GigabitEthernet')
		for interface in interfaces:
			for line in interface:
				if "collisions" in line:
					if '0 collisions' not in line:
						interface_name = interface[0].split(' ')[0]
						issue = '      {} has collisions on {}'.format (interface_name, ip)
						post_to_slack(issue, channel, token)
	#Let the user know it's done
	post_to_slack("done", channel, token)
		
		


#Name of things people will type in to kick off an action
functions=['say hi', 'check lab']

def say_hi():
	post_to_slack("hi", channel, token)
	post_to_slack("done", channel, token)

#Run forever, check back every second to see if it wants you to do anything.	
while 1 == 1:
	last_message = get_last_message(token, channel)
	if last_message in  functions:
		#Start it off by posting something.  That way if something bad happens you don't have
		#An infinate loop of it trying to do the same thing because the last message posted was
		#A kick off prompt
		post_to_slack("processing", channel, token)
	if last_message == 'say hi':
		say_hi()
	if last_message == 'check lab':
		check_lab()
	time.sleep(1)

Conclusion

Using Slack’s native integrations, and a Python library, there wasn’t actually that much code or logic required. You can see everything bundled up to a single file with around 100 lines of code. It’s also important to realize the user experience for people who are not developers or engineers, and an integration such as this can provide a reasonable middle ground.



ntc img
ntc img

Contact Us to Learn More

Share details about yourself & someone from our team will reach out to you ASAP!