Developing Nautobot Plugins – Part 2

Blog Detail

This post is the second installment in a series on how to write plugins for Nautobot. Check out the Part 1 here. In this blog series we are developing a plugin for modeling and managing DNS zone data within Nautobot. In the first post, we covered setting up our development environment. In this post, we’ll create models, migrations, views, urls, and navigation. In the next post, we’ll cover tables and forms. Later posts will cover more advanced topics, such as filters, filter forms, APIs, query filters, and GraphQL integration.

We’ll cover models and something called migrations for our DNS management plugin. Models are central to how most plugins work and to how all applications written in Django work. We’ll also cover Views, URLs, and Navigation. We say ‘how most plugins work’ because some use cases may omit their own models or views and only augment existing models or views. But that is a more advanced topic.

DNS Management Plugin

Starting off in Part 2, we have little more than a development environment. However, we have everything we need to develop our plugin! Before we get into writing the first parts of the plugin, we should discuss what our plugin is and isn’t. Our plugin adds DNS models, currently not represented in Nautobot. While this is a real use case, it is just an example; there may be implications for its use that we have not thought about.

The requirements for our plugin are fairly simple: define objects in Nautobot that represent common DNS configurations. We want to have enough information stored in Nautobot to configure a DNS server and connect those DNS configurations to the data that we already store in Nautobot. Before we begin, having a general understanding of DNS record types is a good idea, and Cloudflare has a good reference.

There are many reasons to start creating a plugin, some of which may include adding in new data like ours, while other plugins can simply add new, or modify existing, views or information to Nautobot. Our expectation is that you should be comfortable adding new data with models and have a good foundation from which to modify Nautobot views or data.

Modeling to Models

To start developing, we have to represent our DNS configuration in data. We call this data modeling. One of the top-level configurations of a typical DNS server are the zones, and (most) DNS records are configured under zones. The DNS zone is a good place to start modeling. We’re likely to have multiple DNS zones which will help us organize and configure our other records types. The DNS zone must have an SOA record associated with it so that we can make the zone model contain the SOA record fields as well.

For our purposes, we also require A records to map DNS names to IPs of Devices, Servers, and Services; and CNAME records to redirect existing DNS names to other DNS names. With the information about DNS records, we can come up with the model shown below. Notice the relationship between the DNS Zone Model and the A and CNAME record models.

By convention, models are placed in the base of the plugin directory in a file called models.py. If there are many models, they can also be instantiated inside individual files under a models folder in the plugin directory.

We’ll call our initial Model the DnsZoneModel, and we’ll inherit from the Nautobot PrimaryModel. The PrimaryModel is a Nautobot-specific class that provides a baseline of functions and inheritance to make taking advantage of Nautobot features easier.

"""models.py"""

from nautobot.core.models.generics import PrimaryModel

class DnsZoneModel(PrimaryModel):
    """Model representing DNS zone and SOA record.
    """

Each model will have properties. In Django, these are represented by fields that indicate the type of data that should go into the field. There are many different field types, which you can read about in the model field type reference. Django field types tell Django how to store this data in the database and provides different capabilities for verifying that the data is proper before storing it. For instance, when we get to forms, if a user tried to put letters into a field that expected numbers, Django will return an error. Another example of this verification is that if you tried to create the object directly in an interactive Python session with incorrect data, a similar error should be returned.

Our DNS Zone Model should record all of the properties of the SOA record that are configurable, so we’ll create a Django field for each SOA option. The only option that seems to be programmatically controlled is the SERIAL field, so we’ll leave that out of our model for now. If we need it in the future, we can always add it in. Cross-referencing the description of the fields to the Django field reference, we can determine the field type and get the following fields and field types:

  • name: CharField
  • mname: CharField
  • rname: EmailField
  • refresh: IntegerField
  • retry: IntegerField
  • expire: IntegerField
  • ttl: IntegerField

It makes sense to use the CharField type for name and mname, because these fields can hold a max of 255 characters and domain names have a max length of 253 characters.

For rname, the resulting format of this field is slightly different than an email; however, the field should be an email address. We will use the EmailField to take advantage of the field verification that Django provides.

For refreshretryexpire, and ttl fields, we will use the IntegerField. The DNS RFC says that these are 32-bit numbers, while some of them are signed (meaning they can be negative), they all represent times which should mostly be positive. The Django IntegerField is a 32-bit integer data type and can hold values between -2147483648 to 2147483647. Because we don’t want negative numbers, we’ll need a way to validate our own limit. Specifically, we want a lower limit of 300 seconds, so we will use Django validators to provide guide rails on the inputs.

There are a variety of ways to validate fields within models; you can even validate the fields against each other. Read more about model validation here.

Related to the value, we also want to include a default value for our field. In this case, we can provide one by specifying the default argument. In our case, we will provide the recommended values for each of our IntegerFields as the default.

In addition to specifying the parameters of our model properties, such as the length or valid values, we can also add help text to each model. The help text cascades down to forms and other places where the model is referenced to provide a description of what the field is.

Lastly, in Nautobot, there is a specific field called a slug that can be used as a shorthand API-friendly name to reference a specific instance of the model instead of a longer UUID. It is a Nautobot best practice to add a slug to each model, and you can read more about this here. So we’ll also include a slug property for each model. Nautobot provides an AutoSlugField; and since the slug is a shorthand name, we will populate it from the name of the DnsZoneModel.

After implementing the rest of the properties, our DnsZoneModel should look like this:

"""models.py"""

from django.core.validators import MaxValueValidator, MinValueValidator 
from django.db import models
from nautobot.core.fields import AutoSlugField
from nautobot.core.models.generics import PrimaryModel

class DnsZoneModel(PrimaryModel):
    """Model representing DNS Zone and SOA record.
    """
    name = models.CharField(max_length=200, help_text="FQDN of the Zone, w/ TLD.")
    slug = AutoSlugField(populate_from="name")
    mname = models.CharField(max_length=200, help_text="FQDN of the Authoritative Name Server for Zone.")
    rname = models.EmailField(help_text="Admin Email for the Zone in the form user@zone.tld.")
    refresh = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default="86400",help_text="Number of seconds after which secondary name servers should query the master for the SOA record, to detect zone changes.")
    retry = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=7200, help_text="Number of seconds after which secondary name servers should retry to request the serial number from the master if the master does not respond.")
    expire = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=3600000, help_text="Number of seconds after which secondary name servers should stop answering request for this zone if the master does not respond. This value must be bigger than the sum of Refresh and Retry.")
    ttl = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=3600, help_text="Time To Live.")

With the major DnsZoneModel out of the way, we can create the other models as well.

For the ARecordModel, most of the fields should look similar to what we have discussed so far. However, I’d like to introduce another field type called the ForeignKey field. This allows us to create a specific relationship with objects from another model. A ForeignKey relationship says this object has some relationship to the other one. The ForeignKey field can be another model within Nautobot or a model within our own or another plugin. In this case, we’ll have two Foreign relationships:

  • to the DnsZoneModel to capture that the A record is part of a specific DNS zone
  • to the Nautobot IP Address model to capture the IP the A record is for

The ForeignKey field must specify another model to create a relationship to: the default first positional parameter (or the ‘to’ argument) of the ForeignKey field can be a bare model reference. Since the zone parameter is referencing the model we just created, we can reference it here. Additionally, we directly reference the zone model because it is imported at the same time as the rest of our plugin’s models.

The other way to reference a foreign model is the dotted path of the Django application model as a string. When we reference a model like this, it allows Django to lazily (on demand) load that model in if it’s not already loaded. In this case, we’re referencing the IPAddress model from the Nautobot ipam application. To find the application label, you can reference this list.

Another way is to use the handy Nautobot shell_plus feature! In your development environment, while Nautobot is running (with invoke debug or invoke start), execute invoke shell-plus. This will start an interactive Python shell with some enhancements, specifically, all the currently available models (including from our plugin!) are loaded. Since we have an idea of which model we’d like, we can type IPAddress (remember, Python is case sensitive) to verify that we have the model name correct. From there we can type IPAddress._meta.label, which will output the required information.

<span role="button" tabindex="0" data-code="inv shell-plus Running docker-compose command "ps –services –filter status=running" Running docker-compose command "exec nautobot nautobot-server shell_plus" # Shell Plus Model Imports from constance.backends.database.models import Constance from django.contrib.admin.models import LogEntry from django.contrib.auth.models import Group, Permission …[snip]… from nautobot.ipam.models import Aggregate, IPAddress, Prefix, RIR, Role, RouteTarget, Service, VLAN, VLANGroup, VRF …[snip]… from nautobot_example_dns_manager.models import ARecordModel, CNameRecordModel, DnsZoneModel …[snip]… In [1]: IPAddress Out[1]: nautobot.ipam.models.IPAddress In [2]: IPAddress._meta Out[2]:
inv shell-plus
Running docker-compose command "ps --services --filter status=running"
Running docker-compose command "exec nautobot nautobot-server shell_plus"
# Shell Plus Model Imports
from constance.backends.database.models import Constance
from django.contrib.admin.models import LogEntry
from django.contrib.auth.models import Group, Permission
...[snip]...
from nautobot.ipam.models import Aggregate, IPAddress, Prefix, RIR, Role, RouteTarget, Service, VLAN, VLANGroup, VRF
...[snip]...
from nautobot_example_dns_manager.models import ARecordModel, CNameRecordModel, DnsZoneModel
...[snip]...
In [1]: IPAddress
Out[1]: nautobot.ipam.models.IPAddress
In [2]: IPAddress._meta
Out[2]: <Options for IPAddress>
In [4]: IPAddress._meta.label
Out[4]: 'ipam.IPAddress'

You can also use the Python shell function dir() to see what other fields and functions are available in IPAddress._meta, as so: dir(IPAddress._meta). This can be useful for any other Django model as well!

In the ForeignKey field, we also find a second new argument. The on_delete argument. This required argument defines what to do with our object (the ARecordModel object) if the object we’re pointing to is deleted (or is attempted to be deleted). There are several options for the on_delete argument, some of which require other arguments to be set. Some common settings for on_delete are

  • CASCADE: Also delete our model instance
  • PROTECT: Don’t allow the deletion of the related object (if our object exists)
  • DO_NOTHING: Don’t do anything special (this could result in IntegrityErrors)

We’ll choose PROTECT because we want the user to have to delete our DNS object before removing the DnsZoneModel object.

For the ForeignKey field for IPAddress, we’ll choose CASCADE because we want to remove our ARecord instance if the associated IPAddress is deleted. In practice, PROTECT may also be a good option here (but it dependes on the use case).

"""models.py"""
class ARecordModel(PrimaryModel):
    """Model for representing A records.
    """
    zone = models.ForeignKey(DnsZoneModel, on_delete=models.PROTECT)
    slug = AutoSlugField(populate_from="name")
    name = models.CharField(max_length=200, help_text="Name of the Record. Will inherit the domain from the zone of which it is a part.")
    address = models.ForeignKey(to="ipam.IPAddress", on_delete=models.CASCADE, help_text="IP address for the record.")
    ttl = models.PositiveIntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=14400, help_text="Time To Live.")

And similarly for the CNameRecordModel:

"""models.py"""
class CNameRecordModel(PrimaryModel):
    """Model representing CName records.
    """
    zone = models.ForeignKey(DnsZoneModel, on_delete=models.PROTECT, help_text="Zone that the CName record belongs to.")
    name = models.CharField(help_text="DNS name of the CName record.", max_length=200)
    slug = AutoSlugField(populate_from="name")
    value = models.CharField(help_text="FQDN of where the CName record redirects to.", max_length=253)
    ttl = models.PositiveIntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=14400, help_text="Time To Live.")

Making the Models

Before we’re done with our models, we’ll have to tell Django to look at our new models and initialize them as database tables and fields. We do this with a Django/Nautobot server command called makemigrations. In your development environment, there is a handy invoke command to use: invoke makemigrations. Be sure to have your Nautobot server running before executing makemigrations.

Once makemigrations has been run, you should see a new migrations folder under your plugin folder. The files created here are made by Django and are normally good enough for everyday use. When doing ongoing development of a plugin that is released to the wild, you will need to pay more attention to keeping track of these between releases so that the models do not get out of sync with the database. For now, go wild. You’ll notice that the migration files basically create the database tables that we’ve specified in our models folder.

Once the migrations are created, we’ll need to run them by executing migrate in the same way: invoke migrate. This applies the migrations that we just created and actually creates the tables and other things that we saw in the previous migration files.

You should run makemigrations and migrate any time you change the models.py file (inv debug or inv start will also run migrate). This will make sure that any of the changes made to models get reflected in the database. If you run into issues with IntegrityErrors or other issues with objects in the DB not being consistent, you can always delete all the migrations and run makemigrations and migrations from the beginning.

However, there are two related instances where you do not want to delete your migrations:

  1. You already have a good deal of data in your plugin that you don’t want to lose.
  2. You have released the plugin and there are other users.

When you delete your migrations you are, essentially, erasing the history of how the database tables and columns were created and populated with data. Nautobot will not know what to do with the data that already exists in them, if any data exists.

The second reason is when you have already released a plugin, if you are changing or adding models, you’ll want to delete only until the last migration from the released plugin, for the same reason as above: users already have data in their database and likely won’t have the expertise to transform what they have into the required fields your migration would require. (While these are not usually totally unrecoverable, it takes advanced manipulation of the database to get the database back to a correct state if you mess up migrations).

Viewing Our Models

The next step in our process is to tell Django how to fetch and display the models that we’ve created. We use something called a View. (Read more about Django Views.) Views instruct Django about how to return an html response for our Models. Views can show tables, fetch related information to enrich what is displayed, or do any manner of other things to display information. Views are also specifically required to Add, Edit, Delete, and do Bulk operations for our objects. For our purposes, and to make our plugin usable, we’ll require a View for each of these operations.

In Nautobot, to help make editing and creating models easier, there are some nice generic and base views that we can use. (Read more about them here.)

Because the specific operations we are interested in are common to most Nautobot models, in Nautobot version 1.4 and above, it is simpler to instantiate all these Views by using something called a ViewSet. (Some advanced users may be familiar with these from the Django REST framework.) These help by simplifying the number of View classes that must be created.

Before NautobotUIViewSet was created, we’d have to create one Django View for each operation. Each of these Views would take very similar properties, leading to a lot of necessary but duplicate code. With NautobotUIViewSet (and related helpers/subclasses), you can create one ViewSet class and the individual views will be created automatically from the single ViewSet definition. Instead of creating four views for each of our three models (twelve view classes), we can create one ViewSet for each of our models. Read more about the NautobotUIViewSet here.

Views by convention are stored in the (you guessed it!) views.py file. From the NautobotUIViewSet documentation, we’re interested in a subset of all of the views available to us, so we’ll use some of the view.mixins instead of the NautobotUIViewSet class directly. We’ll need to create one ViewSet for each model.

By using view.mixins, we don’t need to provide quite as many dependencies as with using the full NautobotUIViewSet. You may however require all the views provided by the NautobotUIViewSet, but starting off with the mixins can make development faster and simpler.

We’ll have to import our models from models.py and the view_mixins.

"""views.py"""

from nautobot.core.views import mixins as view_mixins

from .models import DnsZoneModel, ARecordModel, CNameRecordModel

class DnsZoneModelUIViewSet(view_mixins.ObjectListViewMixin,
    view_mixins.ObjectDetailViewMixin,
    view_mixins.ObjectEditViewMixin,
    view_mixins.ObjectDestroyViewMixin,
    view_mixins.ObjectBulkDestroyViewMixin,
):
    queryset = DnsZoneModel.objects.all()


class ARecordModelUIViewSet(view_mixins.ObjectListViewMixin,
    view_mixins.ObjectDetailViewMixin,
    view_mixins.ObjectEditViewMixin,
    view_mixins.ObjectDestroyViewMixin,
    view_mixins.ObjectBulkDestroyViewMixin,
):
    queryset = ARecordModel.objects.all()


class CNameRecordModelUIViewSet(view_mixins.ObjectListViewMixin,
    view_mixins.ObjectDetailViewMixin,
    view_mixins.ObjectEditViewMixin,
    view_mixins.ObjectDestroyViewMixin,
    view_mixins.ObjectBulkDestroyViewMixin,
):
    queryset = CNameRecordModel.objects.all()

The last thing we have to do for now with our views is to tell each ViewSet which objects they need to fetch. We provide the queryset property to each ViewSet class. If you were getting fancy and wanted to show a subset of objects, you could provide a filter to the queryset with Model.objects.filter(). For now, we’re interested in all of the objects of each Model. There are other properties that we will have to create to get all of the views working, but we’ll return to those later in this article and in the next articles.

If any objects of each model exist in the database, you could use shell-plus to run the query and see all of the objects that get returned.

Creating URLs for Our Views

Once we have our views set up, we have to tell Django how to access them. By convention, this is done in the urls.py file, which you will create next. Our usage of the NautobotUIViewSetmixins also helps. We would have created a URL for each of our Views, twelve altogether. However, with the ViewSet, we’ll create just three with something called a ViewSetRouter.

"""urls.py"""

from nautobot.core.views.routers import NautobotUIViewSetRouter
from . import views

router = NautobotUIViewSetRouter()
router.register("dnszonemodel", views.DnsZoneModelUIViewSet)
router.register("arecordmodel", views.ARecordModelUIViewSet)
router.register("cnamerecordmodel", views.CNameRecordModelUIViewSet)

urlpatterns = []

urlpatterns += router.urls

In the router, we register each ViewSet with a name. This name is important; we’ll use it for links and other things as a friendly name, rather than hard-code a static URL. This way, if we ever update URLs, we can just change the name of our links rather than the whole static URL.

The NautobotUIViewSetRouter does a lot of things for us automatically; so if you are just learning Django, you may want to take a look at other plugins or Django apps to get a better idea of how URLs work. One of the things NautobotUIViewSetRouter creates are the URL paths. It also will check for specifically named templates. Have a look at the documentation to get the expected names for each type of mixin.

I’ve included urlpatterns in our View in case you need to add or reference a regular View or something else that doesn’t fit the NautobotUIViewSet pattern.

Getting There with Navigation

At this point in the application, you should be able to directly navigate to the URLs. You can check on the URLs we just created by navigating to Plugins > Installed Plugins > Nautobot Example DNS Manager and viewing the Views/URLs in the table to the right:

We’ll create another file to add in our navigation: navigation.py. We’ll import some helper classes to add the buttons in a uniform way.

"""Nautobot Example DNS Manager Navigation."""

from nautobot.core.apps import NavMenuAddButton, NavMenuGroup, NavMenuItem, NavMenuImportButton, NavMenuTab

Now, we can add these URLs to buttons in the navigation header of Nautobot. Because DNS is an IPAM operation, we’ll add our buttons to the IPAM drop-down menu. We’ll start with the menu_items Nautobot magic variable. Nautobot will look for the menu_items in navigation.py, which, if everything is correct, add the specified buttons.

We select the IPAM menu tab by using the NavMenuTab and providing the existing tab name: IPAM.

Because we want to add our DNS options, we should create a new grouping under IPAM (since it doesn’t really fit under the existing groups). We also want our group to appear under IP Addressing, because it is closely related.

We’ll add a NavMenuGroup to specify our DNS group. We’ll weight our group to be after IP Addressing and before the rest of the groups.

menu_items = (
    NavMenuTab(
        name="IPAM",
        groups=(
            NavMenuGroup(
                name="DNS",
                weight=150,

Inside the NavMenuGroup we’ll add our items. Each NavMenuItem specifies a link. In general, the larger menu item will bring you to the list view for that model. For each menu item, we can specify add and import buttons. For now, we’ll specify the NavMenuAddButton for each model and specify the URL path for the add View. We’ll revisit permissions in a future article; for now, we need to pass in the blank list to prevent errors.

menu_items = (
    NavMenuTab(
        name="IPAM",
        groups=(
            NavMenuGroup(
                name="DNS",
                weight=150,
                items=(
                    NavMenuItem(
                        link="plugins:nautobot_example_dns_manager:dnszonemodel_list",
                        name="DNS Zones",
                        permissions=[],
                        buttons=(
                            NavMenuAddButton(
                                link="plugins:nautobot_example_dns_manager:dnszonemodel_add",
                                permissions=[],
                            ),
                        ), 
                    ),
                    NavMenuItem(
                        link="plugins:nautobot_example_dns_manager:arecordmodel_list",
                        name="A Records",
                        permissions=[],
                        buttons=(
                            NavMenuAddButton(
                                link="plugins:nautobot_example_dns_manager:arecordmodel_add",
                                permissions=[],
                            ),
                        ),
                    ),
                    NavMenuItem(
                        link="plugins:nautobot_example_dns_manager:cnamerecordmodel_list",
                        name="CNAME Records",
                        permissions=[],
                        buttons=(
                            NavMenuAddButton(
                                link="plugins:nautobot_example_dns_manager:cnamerecordmodel_add",
                                permissions=[],
                            ),
                        ),
                    ),
                ),
            ),
        ),
    ),
)

At this point in our plugin development, we should have Models, Views, URLs, and Navigation. You should be able to go to your local Nautobot homepage and see the installed plugins. The DNS group should also appear in the Navigation bar under IPAM.

Minimally Viable Plugin

You’ll notice that when you click on the menu items or buttons, you’ll be taken to an error screen. We still have a few things to add before we’re able to see any of the specific pages, such as the template mentioned earlier.

To make sure that our Views are working before moving on to the next steps, we need to add a couple of things in a few different places.

In models.py for our models, we’ll define a function called get_absolute_url. Read more about this function here.

We’ll add another import to get that working:

from django.urls import reverse

And to our models, the function:

 def get_absolute_url(self):
    return reverse("plugins:nautobot_example_dns_manager:dnszonemodel", args=[self.slug])

This returns the exact URL for an instance of our object. Essentially, reverse looks up the path with the arguments provided. So if we had a DnsZoneModel object with a name such as nautobot.example.comreverse would return something like: localhost:8080/plugins/example-dns-manager/dnszonemodel/nautobot-example-comnautobot-example-com being the slug created from the name.

We’ll add one more thing for convenience: a __str__ representation of our objects, in case we need to view them in shell_plus. Nautobot also uses the __str__ representation in place of a UUID in generic views, which makes using the generic views and templates more user-friendly.

Our DnsZoneModel should look like this:

class DnsZoneModel(PrimaryModel):
    """Model representing DNS Zone and SOA record.
    """
    name = models.CharField(max_length=200, help_text="FQDN of the Zone, w/ TLD.")
    slug = AutoSlugField(populate_from="name")
    mname = models.CharField(max_length=200, help_text="FQDN of the Authoritative Name Server for Zone.")
    rname = models.EmailField(help_text="Admin Email for the Zone in the form user@zone.tld.")
    refresh = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default="86400",help_text="Number of seconds after which secondary name servers should query the master for the SOA record, to detect zone changes.")
    retry = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=7200, help_text="Number of seconds after which secondary name servers should retry to request the serial number from the master if the master does not respond.")
    expire = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=3600000, help_text="Number of seconds after which secondary name servers should stop answering request for this zone if the master does not respond. This value must be bigger than the sum of Refresh and Retry.")
    ttl = models.PositiveIntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=3600, help_text="Time To Live.")

    def get_absolute_url(self):
        return reverse("plugins:nautobot_example_dns_manager:dnszonemodel", args=[self.slug])

    def __str__(self):
        return self.name

The next thing we’ll add is a very basic table definition in a new file tables.py, just so that we can see something. This will use a generic template that Nautobot provides for list views to display the page.

"""table.py"""

from nautobot.utilities.tables import BaseTable

from .models import DnsZoneModel

class DnsZoneModelTable(BaseTable):

    class Meta(BaseTable.Meta):
        model = DnsZoneModel

Import the DnsZoneModelTable then add it under the View as a table_class.

"""views.py"""

from nautobot.core.views import mixins as view_mixins

from .models import DnsZoneModel, ARecordModel, CNameRecordModel
from .tables import DnsZoneModelTable

class DnsZoneModelUIViewSet(view_mixins.ObjectListViewMixin,
    view_mixins.ObjectDetailViewMixin,
    view_mixins.ObjectEditViewMixin,
    view_mixins.ObjectDestroyViewMixin,
    view_mixins.ObjectBulkDestroyViewMixin,
):
    queryset = DnsZoneModel.objects.all()
    table_class = DnsZoneModelTable

Once the table is in place, you should see a basic table view when you click on the DNS Zones in the Navigation menu. The table won’t have anything in it yet since we have not created any objects just yet. In the next few articles we’ll cover the table and templates more.

References


Conclusion

We’ve built the major substance of our plugin by defining Models, showed Django how to display the models with ViewSets, defined paths for our views with URLs, and placed links into the toolbar via navigation.py. Next up, we’ll actually fill in our Views with tables, be able to view individual objects, define forms for inputs, and add filters so we can search our objects.

-Stephen



ntc img
ntc img

Contact Us to Learn More

Share details about yourself & someone from our team will reach out to you ASAP!

Data Modeling for Network Engineers

Blog Detail

Structured Data, Schemas, Network as Code, and Sources of Truth—what do these all have in common? Data is central to all of them in concept and in practice. How do we work with all of that data? By modeling it of course! We’ll discuss how to start data modeling. Data modeling is not typically the focus of the conversation but is usually what you are working with, and it’s important to understand some ways to approach data modeling.

Data models are abstractions—they detail the information and relationships we need to take into account that, when combined with assumptions, come up with the entire description of the subject. Data modeling is more of an art than a science. Coming up with a data model will almost always be an iterative approach: having too much and paring back or having too little and adding.

Generally, there are two approaches to modeling:

  • Top-down: Starting with what you want and expanding/refining
  • Bottom-up: Starting with what you have and refining

In top-down modeling, you typically try to come up with the data that you think you’ll need; likely it won’t be all or the same data that you end up with. In this case, the data modeling exercise is much like developing a new design.

In bottom-up modeling, you will have many more points of data than your model will end up describing. The beginning of a bottom-up process would likely be a configuration itself. As you see as we begin to cover in our example, we start to peel the layers back as we make assumptions and take into account certain facts or axioms about our data, design, and configurations.

Diving into Topologies

We’re going to cover more of the bottom-up process today since that’s where a lot of NetDevOps are starting: with existing networks, data, and configurations that need to be reasoned over and modeled.

We’ll go over an example focused on modeling Devices’ Connections and what we need to model in order to derive their configurations. Consider the diagram below. It is a typical campus design where a location aggregation or distribution router aggregates all of the connections from the building and then connects to the core. This is a very common network design pattern. We’re interested in modeling the Layer 3 interfaces of Dist A in the diagram below. Generally, this modeling exercise should be applicable to most other Layer 3 interfaces in the network, but especially other Layer 3 interfaces connected to core devices on distribution switches.

Topology with core routers core-1 and core-2 connected to distribution router dist-a.

Configurations

Because our design has been ruthlessly standardized, the topology above is a good example for all of our campus buildings. From the diagram, we can immediately determine that our two distribution switches should have some standard ports dedicated to certain functions: 1 uplink to the campus core, 2 cross-connects, and 2 downlinks to access switches for each distribution switch. Here’s a snippet of the configuration for dist-a.

interface 1/1/55 no shutdown mtu 9198 qos trust none description Connection to C.Core 1 ip mtu 9198 ip address 10.10.1.2/30 arp timeout 600

interface 1/1/56 no shutdown mtu 9198 qos trust none description Connection to C.Core 2 ip mtu 9198 ip address 10.10.1.5/30 arp timeout 600

Keep in mind that we’re trying to get the minimum amount of data that will be able to regenerate the configuration above.

From this configuration, we could start with a model like this:


[{
    "name": "1/1/55",
    "shutdown": False,
    "lag": None,
    "mtu": 9198,
    "qos-trust": "none",
    "description": "Connection to C.Core 1",
    "routing": True,
    "trunking-mode": None,
    "allowed-vlan": "all",
    "native-vlan": None,
    "ip-mtu": 9198,
    "ip-address": "10.10.1.2/30",
    "arp-timeout": 600
},
{
    "name": "1/1/56",
    "shutdown": False,
    "lag": None,
    "mtu": 9198,
    "qos-trust": "none",
    "description": "Connection to C.Core 2",
    "routing": True,
    "trunking-mode": None,
    "allowed-vlan": "all",
    "native-vlan": None,
    "ip-mtu": 9198,
    "ip-address": "10.10.1.5/30",
    "arp-timeout": 600
}]

For data above, it is very verbose, touching on each possible configuration across most of the interfaces. This would quickly get out of hand if we need to create and manage this for every interface across all the switches across a campus. Let’s pare the model back some for our specific interface above.

We can start with taking away properties we can assume at the point of use (in the template or when building out the data) or assumed defaults:

If there is an IP, routing will be enabled and trunking will not. 
All of our configured interfaces will be enabled.
The MTU will always be 9198. 
If there is an IP, we need an IP-MTU statement.
ARP Timeout will always be 600.

With these assumptions, our model would look something like this

[{
    "name": "1/1/55",
    "description": "Connection to C.Core 1",
    "ip-address": "10.10.1.2/30",
},
{
    "name": "1/1/56",
    "description": "Connection to C.Core 2",
    "ip-address": "10.10.1.5/30",
}]

Our resulting data model above is much less verbose, making it easier to read and manage. Keep in mind the data that we will put into the model for rendering must be stored somewhere, so the fewer points of data we need for each interface the better.

Usage

After developing the data model, we can easily input the data and use it in a template such as the interface template below. While this specific template may match a certain switch model, the data input should or could match across different models of switches.

interface {{ interface["name"] }}
   mtu 9198
{% if interface["description"] | length > 1 %}
   description {{ interface["description"] }}
{% endif %}
{% if interface["ip_addresses"] | length > 0 %}
   no switchport
{% endif %}
{% if interface["ip_addresses"] | length > 0 %}
{% for addr in interface["ip_addresses"] %}
{% if addr["address"] is defined %}
   ip address {{ addr["address"] }}
{% endif %}
{% endfor %}
   ip mtu 9198
   ip arp timeout 600
{% endif %}
   no shutdown

Process

While developing the model, it may be helpful to keep track of the model in a spreadsheet or table. Every property of the model can be kept in a spreadsheet, to make it easier to view the model all at once. Keeping the expected variable types, whether a property is required or is optional, the source of the variable in an instance of the model, and finally an example of the property value are all helpful to explain and work with the data model.

Here is an example of values to keep track of for the data model above.

propertyattribute typerequiredexamplesystem of recorddescription / notes
namestringrequired1/1/55nautobot 
descriptionstringoptionalConnection to access sw 1nautobotdetermined by cable connection in nautobot
ip_addressesstringoptional10.10.1.2/30nautobotfrom nautobot device interface

Conclusion

We went over what data modeling is, how to get started with the data modeling process for network interfaces, how the data model could be used, and how to keep track of the model. In future blogs, we’ll go over the process for modeling other aspects of configuration and how the data model could be represented and/or derived from a Source of Truth. Thanks for reading!

Stephen Corry



ntc img
ntc img

Contact Us to Learn More

Share details about yourself & someone from our team will reach out to you ASAP!

Manipulating Data with Jinja Map and Selectattr

Blog Detail

Manipulating data with the Jinja filters, Map, Selectattr, and Select provides quick and efficient ways to create new data structures. This can simplify creating lookup dictionaries and avoids complex loops or Jinja templating options.

In many instances in ansible, I’ll use an ordered list of dictionaries as my data model; this specifically keeps the preferred ordering for the device and type of configuration as well as makes it easy to do operations on all the members of that list. For some operations that rely on specific dictionaries, it can be convoluted to dig that information out of the list, or work on a subset of the list, without going through loops and conditionals, without these tools.

Like all good overviews, it’s best explained with an example. We’ll start with this example data structure for some selected interfaces, a commonly used pattern.

---
interfaces:
  - name: "mgmt"
    enable: true
    dhcp: true
    ip_address: no
    description: "management"
  - name: "1/1/1"
    enable: true
    ip_address: 10.1.1.2/24
    description: "uplink to core-1"
  - name: "1/1/2"
    enable: true
    ip_address: 10.1.2.2/24
    description: "uplink to core-2"
  - name: "1/1/3"
    enable: true
    ip_address: 10.1.3.1/24
    description: "peerlink to agg-2"
  - name: "vlan42"
    ip_address: 10.42.0.2/24
    ip_helper: false
    description: "inband management"
  - name: "vlan44"
    ip_address: 10.44.0.3/24
    ip_helper: true
    fhrp: true
    description: "wifi aps"

Selecattr

Selectattr takes in a list of dictionaries and applies a test to each dictionary.

One of the first use cases might be to get all of the interfaces that have an IP address defined:

{{ interfaces | selectattr('ip_address') }}

With the selectattr filter, a Jinja test can be used as the second argument; this gives us a lot of flexibility when selecting our data. Additional arguments can be specified as inputs to the test specified in the second arg.

One thing to be careful of with the selectattr filter with this example: The attribute you use must be defined, but not all of our interfaces have ip_address defined. When the default test is applied to the value of the attribute (bool), the var doesn’t exist, which will result in an error. We’ll hit that error wherever ip_address is not defined in our interfaces data structure.

To get around this in our data, we specify the ‘defined’ parameter to test whether the attribute is there or not:

{{ interfaces | selectattr('ip_address', 'defined') }}

This returns a list of dictionaries where the ip_address attribute is present. Easy enough.

[
    {
        "enable": true,
        "ip_address": "10.1.1.2/24",
        "name": "1/1/1"
    },
    {...},
]

To be even more granular, we can chain the output to another selectattr filter and further test the ip_address or another attribute. For example, we can pass the list of interfaces with IP addresses and test whether the IP address on the interface is in another list of IP addresses:

{{ interfaces | selectattr('ip_address', 'defined') | selectattr('ip_address', 'in', '[10.1.3.1/24]') }}

Which gives us this output:

    [
        {
            "description": "peerlink to agg-2",
            "enable": true,
            "ip_address": "10.1.3.1/24",
            "name": "1/1/3"
        }
    ]

Map

Map allows us to turn a list of dictionaries into a simpler list or flatten the list of dictionaries. We specify the values of the list by giving Map the attribute we want to take from the dictionaries.

Now, let’s get a list of the IP addresses. We could easily use this in a route-map, prefix-list, or ACL.

{{ interfaces | map(attribute='ip_address', default='n/a') }}

Gives the output:

    [
        "n/a",
        "10.1.1.1/24",
        "10.1.2.1/24",
        "10.1.3.1/24",
        "10.42.0.1/24",
        "10.44.0.1/24"
    ]

Similar to the selectattr, Map expects the attribute to be defined. In this case we have to specify a default value in case the dictionary doesn’t contain the specified attribute, ip_address in this case.

It might be useful to ONLY have IP addresses in our list of IPs. Here we can combine the selectattr by filtering down the dictionaries where the ip_address attribute is defined and then using Map to filter the dictionaries down to the values of the ip_address attribute.

{{ interfaces | selectattr('ip_address', 'defined') | map(attribute='ip_address') }}

Output:

[
    "10.1.1.1/24",
    "10.1.2.1/24",
    "10.1.3.1/24",
    "10.42.0.1/24",
    "10.44.0.1/24"
]

The Map filter also gives us the ability to input another filter to use on the data in a list; this allows us to use a filter that doesn’t support working on lists to work on each element of the list without using a loop. Most of the Jinja built-in filters will work without the Map filter, however.

Other Useful Combinations

If we need to look up IP addresses by interface name, we can instead use the items2dict filter. This gives us the ability to map attributes of the dictionary to new keys and values. This is useful when specific addresses are used in specific ways, such as network-specific ACLs, prefix-lists, bgp neighbors, and others that need to reference interfaces in specific ways or different orders.

We first use selectattr to filter down to which dictionaries have an ip_address, then use items2dict to output a new dictionary where the interface name is the key and the ip_address is the value.

{{ interfaces | selectattr('ip_address', 'defined') | items2dict(key_name='name', value_name='ip_address') }}

This gives us a nice tidy lookup dictionary without any loops and a relatively readable call:

{
    "1/1/1": "10.1.1.1/24",
    "1/1/2": "10.1.2.1/24",
    "1/1/3": "10.1.3.1/24",
    "vlan42": "10.42.0.1/24",
    "vlan44": "10.44.0.1/24"
}

Conclusion

All in all, these are very handy tools to use to get new views of the same data without having to resort to too much looping or other odd manipulations. I hope you find them useful!

-Stephen Corry



ntc img
ntc img

Contact Us to Learn More

Share details about yourself & someone from our team will reach out to you ASAP!