Developing Nautobot Plugins – Part 2

This post is the second installment in a series on how to write plugins for Nautobot. Check out the Part 1 here. In this blog series we are developing a plugin for modeling and managing DNS zone data within Nautobot. In the first post, we covered setting up our development environment. In this post, we’ll create models, migrations, views, urls, and navigation. In the next post, we’ll cover tables and forms. Later posts will cover more advanced topics, such as filters, filter forms, APIs, query filters, and GraphQL integration.

We’ll cover models and something called migrations for our DNS management plugin. Models are central to how most plugins work and to how all applications written in Django work. We’ll also cover Views, URLs, and Navigation. We say ‘how most plugins work’ because some use cases may omit their own models or views and only augment existing models or views. But that is a more advanced topic.

DNS Management Plugin

Starting off in Part 2, we have little more than a development environment. However, we have everything we need to develop our plugin! Before we get into writing the first parts of the plugin, we should discuss what our plugin is and isn’t. Our plugin adds DNS models, currently not represented in Nautobot. While this is a real use case, it is just an example; there may be implications for its use that we have not thought about.

The requirements for our plugin are fairly simple: define objects in Nautobot that represent common DNS configurations. We want to have enough information stored in Nautobot to configure a DNS server and connect those DNS configurations to the data that we already store in Nautobot. Before we begin, having a general understanding of DNS record types is a good idea, and Cloudflare has a good reference.

There are many reasons to start creating a plugin, some of which may include adding in new data like ours, while other plugins can simply add new, or modify existing, views or information to Nautobot. Our expectation is that you should be comfortable adding new data with models and have a good foundation from which to modify Nautobot views or data.

Modeling to Models

To start developing, we have to represent our DNS configuration in data. We call this data modeling. One of the top-level configurations of a typical DNS server are the zones, and (most) DNS records are configured under zones. The DNS zone is a good place to start modeling. We’re likely to have multiple DNS zones which will help us organize and configure our other records types. The DNS zone must have an SOA record associated with it so that we can make the zone model contain the SOA record fields as well.

For our purposes, we also require A records to map DNS names to IPs of Devices, Servers, and Services; and CNAME records to redirect existing DNS names to other DNS names. With the information about DNS records, we can come up with the model shown below. Notice the relationship between the DNS Zone Model and the A and CNAME record models.

By convention, models are placed in the base of the plugin directory in a file called models.py. If there are many models, they can also be instantiated inside individual files under a models folder in the plugin directory.

We’ll call our initial Model the DnsZoneModel, and we’ll inherit from the Nautobot PrimaryModel. The PrimaryModel is a Nautobot-specific class that provides a baseline of functions and inheritance to make taking advantage of Nautobot features easier.

"""models.py"""

from nautobot.core.models.generics import PrimaryModel

class DnsZoneModel(PrimaryModel):
    """Model representing DNS zone and SOA record.
    """

Each model will have properties. In Django, these are represented by fields that indicate the type of data that should go into the field. There are many different field types, which you can read about in the model field type reference. Django field types tell Django how to store this data in the database and provides different capabilities for verifying that the data is proper before storing it. For instance, when we get to forms, if a user tried to put letters into a field that expected numbers, Django will return an error. Another example of this verification is that if you tried to create the object directly in an interactive Python session with incorrect data, a similar error should be returned.

Our DNS Zone Model should record all of the properties of the SOA record that are configurable, so we’ll create a Django field for each SOA option. The only option that seems to be programmatically controlled is the SERIAL field, so we’ll leave that out of our model for now. If we need it in the future, we can always add it in. Cross-referencing the description of the fields to the Django field reference, we can determine the field type and get the following fields and field types:

• name: CharField
• mname: CharField
• rname: EmailField
• refresh: IntegerField
• retry: IntegerField
• expire: IntegerField
• ttl: IntegerField

It makes sense to use the CharField type for name and mname, because these fields can hold a max of 255 characters and domain names have a max length of 253 characters.

For rname, the resulting format of this field is slightly different than an email; however, the field should be an email address. We will use the EmailField to take advantage of the field verification that Django provides.

For refresh, retry, expire, and ttl fields, we will use the IntegerField. The DNS RFC says that these are 32-bit numbers, while some of them are signed (meaning they can be negative), they all represent times which should mostly be positive. The Django IntegerField is a 32-bit integer data type and can hold values between -2147483648 to 2147483647. Because we don’t want negative numbers, we’ll need a way to validate our own limit. Specifically, we want a lower limit of 300 seconds, so we will use Django validators to provide guide rails on the inputs.

There are a variety of ways to validate fields within models; you can even validate the fields against each other. Read more about model validation here.

Related to the value, we also want to include a default value for our field. In this case, we can provide one by specifying the default argument. In our case, we will provide the recommended values for each of our IntegerFields as the default.

In addition to specifying the parameters of our model properties, such as the length or valid values, we can also add help text to each model. The help text cascades down to forms and other places where the model is referenced to provide a description of what the field is.

Lastly, in Nautobot, there is a specific field called a slug that can be used as a shorthand API-friendly name to reference a specific instance of the model instead of a longer UUID. It is a Nautobot best practice to add a slug to each model, and you can read more about this here. So we’ll also include a slug property for each model. Nautobot provides an AutoSlugField; and since the slug is a shorthand name, we will populate it from the name of the DnsZoneModel.

After implementing the rest of the properties, our DnsZoneModel should look like this:

"""models.py"""

from django.core.validators import MaxValueValidator, MinValueValidator 
from django.db import models
from nautobot.core.fields import AutoSlugField
from nautobot.core.models.generics import PrimaryModel

class DnsZoneModel(PrimaryModel):
    """Model representing DNS Zone and SOA record.
    """
    name = models.CharField(max_length=200, help_text="FQDN of the Zone, w/ TLD.")
    slug = AutoSlugField(populate_from="name")
    mname = models.CharField(max_length=200, help_text="FQDN of the Authoritative Name Server for Zone.")
    rname = models.EmailField(help_text="Admin Email for the Zone in the form user@zone.tld.")
    refresh = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default="86400",help_text="Number of seconds after which secondary name servers should query the master for the SOA record, to detect zone changes.")
    retry = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=7200, help_text="Number of seconds after which secondary name servers should retry to request the serial number from the master if the master does not respond.")
    expire = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=3600000, help_text="Number of seconds after which secondary name servers should stop answering request for this zone if the master does not respond. This value must be bigger than the sum of Refresh and Retry.")
    ttl = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=3600, help_text="Time To Live.")

With the major DnsZoneModel out of the way, we can create the other models as well.

For the ARecordModel, most of the fields should look similar to what we have discussed so far. However, I’d like to introduce another field type called the ForeignKey field. This allows us to create a specific relationship with objects from another model. A ForeignKey relationship says this object has some relationship to the other one. The ForeignKey field can be another model within Nautobot or a model within our own or another plugin. In this case, we’ll have two Foreign relationships:

• to the DnsZoneModel to capture that the A record is part of a specific DNS zone
• to the Nautobot IP Address model to capture the IP the A record is for

The ForeignKey field must specify another model to create a relationship to: the default first positional parameter (or the ‘to’ argument) of the ForeignKey field can be a bare model reference. Since the zone parameter is referencing the model we just created, we can reference it here. Additionally, we directly reference the zone model because it is imported at the same time as the rest of our plugin’s models.

The other way to reference a foreign model is the dotted path of the Django application model as a string. When we reference a model like this, it allows Django to lazily (on demand) load that model in if it’s not already loaded. In this case, we’re referencing the IPAddress model from the Nautobot ipam application. To find the application label, you can reference this list.

Another way is to use the handy Nautobot shell_plus feature! In your development environment, while Nautobot is running (with invoke debug or invoke start), execute invoke shell-plus. This will start an interactive Python shell with some enhancements, specifically, all the currently available models (including from our plugin!) are loaded. Since we have an idea of which model we’d like, we can type IPAddress (remember, Python is case sensitive) to verify that we have the model name correct. From there we can type IPAddress._meta.label, which will output the required information.

inv shell-plus
Running docker-compose command "ps --services --filter status=running"
Running docker-compose command "exec nautobot nautobot-server shell_plus"
# Shell Plus Model Imports
from constance.backends.database.models import Constance
from django.contrib.admin.models import LogEntry
from django.contrib.auth.models import Group, Permission
...[snip]...
from nautobot.ipam.models import Aggregate, IPAddress, Prefix, RIR, Role, RouteTarget, Service, VLAN, VLANGroup, VRF
...[snip]...
from nautobot_example_dns_manager.models import ARecordModel, CNameRecordModel, DnsZoneModel
...[snip]...
In [1]: IPAddress
Out[1]: nautobot.ipam.models.IPAddress
In [2]: IPAddress._meta
Out[2]: <Options for IPAddress>
In [4]: IPAddress._meta.label
Out[4]: 'ipam.IPAddress'

You can also use the Python shell function dir() to see what other fields and functions are available in IPAddress._meta, as so: dir(IPAddress._meta). This can be useful for any other Django model as well!

In the ForeignKey field, we also find a second new argument. The on_delete argument. This required argument defines what to do with our object (the ARecordModel object) if the object we’re pointing to is deleted (or is attempted to be deleted). There are several options for the on_delete argument, some of which require other arguments to be set. Some common settings for on_delete are

• CASCADE: Also delete our model instance
• PROTECT: Don’t allow the deletion of the related object (if our object exists)
• DO_NOTHING: Don’t do anything special (this could result in IntegrityErrors)

We’ll choose PROTECT because we want the user to have to delete our DNS object before removing the DnsZoneModel object.

For the ForeignKey field for IPAddress, we’ll choose CASCADE because we want to remove our ARecord instance if the associated IPAddress is deleted. In practice, PROTECT may also be a good option here (but it dependes on the use case).

"""models.py"""
class ARecordModel(PrimaryModel):
    """Model for representing A records.
    """
    zone = models.ForeignKey(DnsZoneModel, on_delete=models.PROTECT)
    slug = AutoSlugField(populate_from="name")
    name = models.CharField(max_length=200, help_text="Name of the Record. Will inherit the domain from the zone of which it is a part.")
    address = models.ForeignKey(to="ipam.IPAddress", on_delete=models.CASCADE, help_text="IP address for the record.")
    ttl = models.PositiveIntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=14400, help_text="Time To Live.")

And similarly for the CNameRecordModel:

"""models.py"""
class CNameRecordModel(PrimaryModel):
    """Model representing CName records.
    """
    zone = models.ForeignKey(DnsZoneModel, on_delete=models.PROTECT, help_text="Zone that the CName record belongs to.")
    name = models.CharField(help_text="DNS name of the CName record.", max_length=200)
    slug = AutoSlugField(populate_from="name")
    value = models.CharField(help_text="FQDN of where the CName record redirects to.", max_length=253)
    ttl = models.PositiveIntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=14400, help_text="Time To Live.")

Making the Models

Before we’re done with our models, we’ll have to tell Django to look at our new models and initialize them as database tables and fields. We do this with a Django/Nautobot server command called makemigrations. In your development environment, there is a handy invoke command to use: invoke makemigrations. Be sure to have your Nautobot server running before executing makemigrations.

Once makemigrations has been run, you should see a new migrations folder under your plugin folder. The files created here are made by Django and are normally good enough for everyday use. When doing ongoing development of a plugin that is released to the wild, you will need to pay more attention to keeping track of these between releases so that the models do not get out of sync with the database. For now, go wild. You’ll notice that the migration files basically create the database tables that we’ve specified in our models folder.

Once the migrations are created, we’ll need to run them by executing migrate in the same way: invoke migrate. This applies the migrations that we just created and actually creates the tables and other things that we saw in the previous migration files.

You should run makemigrations and migrate any time you change the models.py file (inv debug or inv start will also run migrate). This will make sure that any of the changes made to models get reflected in the database. If you run into issues with IntegrityErrors or other issues with objects in the DB not being consistent, you can always delete all the migrations and run makemigrations and migrations from the beginning.

However, there are two related instances where you do not want to delete your migrations:

1. You already have a good deal of data in your plugin that you don’t want to lose.
2. You have released the plugin and there are other users.

When you delete your migrations you are, essentially, erasing the history of how the database tables and columns were created and populated with data. Nautobot will not know what to do with the data that already exists in them, if any data exists.

The second reason is when you have already released a plugin, if you are changing or adding models, you’ll want to delete only until the last migration from the released plugin, for the same reason as above: users already have data in their database and likely won’t have the expertise to transform what they have into the required fields your migration would require. (While these are not usually totally unrecoverable, it takes advanced manipulation of the database to get the database back to a correct state if you mess up migrations).

Viewing Our Models

The next step in our process is to tell Django how to fetch and display the models that we’ve created. We use something called a View. (Read more about Django Views.) Views instruct Django about how to return an html response for our Models. Views can show tables, fetch related information to enrich what is displayed, or do any manner of other things to display information. Views are also specifically required to Add, Edit, Delete, and do Bulk operations for our objects. For our purposes, and to make our plugin usable, we’ll require a View for each of these operations.

In Nautobot, to help make editing and creating models easier, there are some nice generic and base views that we can use. (Read more about them here.)

Because the specific operations we are interested in are common to most Nautobot models, in Nautobot version 1.4 and above, it is simpler to instantiate all these Views by using something called a ViewSet. (Some advanced users may be familiar with these from the Django REST framework.) These help by simplifying the number of View classes that must be created.

Before NautobotUIViewSet was created, we’d have to create one Django View for each operation. Each of these Views would take very similar properties, leading to a lot of necessary but duplicate code. With NautobotUIViewSet (and related helpers/subclasses), you can create one ViewSet class and the individual views will be created automatically from the single ViewSet definition. Instead of creating four views for each of our three models (twelve view classes), we can create one ViewSet for each of our models. Read more about the NautobotUIViewSet here.

Views by convention are stored in the (you guessed it!) views.py file. From the NautobotUIViewSet documentation, we’re interested in a subset of all of the views available to us, so we’ll use some of the view.mixins instead of the NautobotUIViewSet class directly. We’ll need to create one ViewSet for each model.

By using view.mixins, we don’t need to provide quite as many dependencies as with using the full NautobotUIViewSet. You may however require all the views provided by the NautobotUIViewSet, but starting off with the mixins can make development faster and simpler.

We’ll have to import our models from models.py and the view_mixins.

"""views.py"""

from nautobot.core.views import mixins as view_mixins

from .models import DnsZoneModel, ARecordModel, CNameRecordModel

class DnsZoneModelUIViewSet(view_mixins.ObjectListViewMixin,
    view_mixins.ObjectDetailViewMixin,
    view_mixins.ObjectEditViewMixin,
    view_mixins.ObjectDestroyViewMixin,
    view_mixins.ObjectBulkDestroyViewMixin,
):
    queryset = DnsZoneModel.objects.all()


class ARecordModelUIViewSet(view_mixins.ObjectListViewMixin,
    view_mixins.ObjectDetailViewMixin,
    view_mixins.ObjectEditViewMixin,
    view_mixins.ObjectDestroyViewMixin,
    view_mixins.ObjectBulkDestroyViewMixin,
):
    queryset = ARecordModel.objects.all()


class CNameRecordModelUIViewSet(view_mixins.ObjectListViewMixin,
    view_mixins.ObjectDetailViewMixin,
    view_mixins.ObjectEditViewMixin,
    view_mixins.ObjectDestroyViewMixin,
    view_mixins.ObjectBulkDestroyViewMixin,
):
    queryset = CNameRecordModel.objects.all()

The last thing we have to do for now with our views is to tell each ViewSet which objects they need to fetch. We provide the queryset property to each ViewSet class. If you were getting fancy and wanted to show a subset of objects, you could provide a filter to the queryset with Model.objects.filter(). For now, we’re interested in all of the objects of each Model. There are other properties that we will have to create to get all of the views working, but we’ll return to those later in this article and in the next articles.

If any objects of each model exist in the database, you could use shell-plus to run the query and see all of the objects that get returned.

Creating URLs for Our Views

Once we have our views set up, we have to tell Django how to access them. By convention, this is done in the urls.py file, which you will create next. Our usage of the NautobotUIViewSetmixins also helps. We would have created a URL for each of our Views, twelve altogether. However, with the ViewSet, we’ll create just three with something called a ViewSetRouter.

"""urls.py"""

from nautobot.core.views.routers import NautobotUIViewSetRouter
from . import views

router = NautobotUIViewSetRouter()
router.register("dnszonemodel", views.DnsZoneModelUIViewSet)
router.register("arecordmodel", views.ARecordModelUIViewSet)
router.register("cnamerecordmodel", views.CNameRecordModelUIViewSet)

urlpatterns = []

urlpatterns += router.urls

In the router, we register each ViewSet with a name. This name is important; we’ll use it for links and other things as a friendly name, rather than hard-code a static URL. This way, if we ever update URLs, we can just change the name of our links rather than the whole static URL.

The NautobotUIViewSetRouter does a lot of things for us automatically; so if you are just learning Django, you may want to take a look at other plugins or Django apps to get a better idea of how URLs work. One of the things NautobotUIViewSetRouter creates are the URL paths. It also will check for specifically named templates. Have a look at the documentation to get the expected names for each type of mixin.

I’ve included urlpatterns in our View in case you need to add or reference a regular View or something else that doesn’t fit the NautobotUIViewSet pattern.

Getting There with Navigation

At this point in the application, you should be able to directly navigate to the URLs. You can check on the URLs we just created by navigating to Plugins > Installed Plugins > Nautobot Example DNS Manager and viewing the Views/URLs in the table to the right:

We’ll create another file to add in our navigation: navigation.py. We’ll import some helper classes to add the buttons in a uniform way.

"""Nautobot Example DNS Manager Navigation."""

from nautobot.core.apps import NavMenuAddButton, NavMenuGroup, NavMenuItem, NavMenuImportButton, NavMenuTab

Now, we can add these URLs to buttons in the navigation header of Nautobot. Because DNS is an IPAM operation, we’ll add our buttons to the IPAM drop-down menu. We’ll start with the menu_items Nautobot magic variable. Nautobot will look for the menu_items in navigation.py, which, if everything is correct, add the specified buttons.

We select the IPAM menu tab by using the NavMenuTab and providing the existing tab name: IPAM.

Because we want to add our DNS options, we should create a new grouping under IPAM (since it doesn’t really fit under the existing groups). We also want our group to appear under IP Addressing, because it is closely related.

We’ll add a NavMenuGroup to specify our DNS group. We’ll weight our group to be after IP Addressing and before the rest of the groups.

menu_items = (
    NavMenuTab(
        name="IPAM",
        groups=(
            NavMenuGroup(
                name="DNS",
                weight=150,

Inside the NavMenuGroup we’ll add our items. Each NavMenuItem specifies a link. In general, the larger menu item will bring you to the list view for that model. For each menu item, we can specify add and import buttons. For now, we’ll specify the NavMenuAddButton for each model and specify the URL path for the add View. We’ll revisit permissions in a future article; for now, we need to pass in the blank list to prevent errors.

menu_items = (
    NavMenuTab(
        name="IPAM",
        groups=(
            NavMenuGroup(
                name="DNS",
                weight=150,
                items=(
                    NavMenuItem(
                        link="plugins:nautobot_example_dns_manager:dnszonemodel_list",
                        name="DNS Zones",
                        permissions=[],
                        buttons=(
                            NavMenuAddButton(
                                link="plugins:nautobot_example_dns_manager:dnszonemodel_add",
                                permissions=[],
                            ),
                        ), 
                    ),
                    NavMenuItem(
                        link="plugins:nautobot_example_dns_manager:arecordmodel_list",
                        name="A Records",
                        permissions=[],
                        buttons=(
                            NavMenuAddButton(
                                link="plugins:nautobot_example_dns_manager:arecordmodel_add",
                                permissions=[],
                            ),
                        ),
                    ),
                    NavMenuItem(
                        link="plugins:nautobot_example_dns_manager:cnamerecordmodel_list",
                        name="CNAME Records",
                        permissions=[],
                        buttons=(
                            NavMenuAddButton(
                                link="plugins:nautobot_example_dns_manager:cnamerecordmodel_add",
                                permissions=[],
                            ),
                        ),
                    ),
                ),
            ),
        ),
    ),
)

At this point in our plugin development, we should have Models, Views, URLs, and Navigation. You should be able to go to your local Nautobot homepage and see the installed plugins. The DNS group should also appear in the Navigation bar under IPAM.

Minimally Viable Plugin

You’ll notice that when you click on the menu items or buttons, you’ll be taken to an error screen. We still have a few things to add before we’re able to see any of the specific pages, such as the template mentioned earlier.

To make sure that our Views are working before moving on to the next steps, we need to add a couple of things in a few different places.

In models.py for our models, we’ll define a function called get_absolute_url. Read more about this function here.

We’ll add another import to get that working:

from django.urls import reverse

And to our models, the function:

 def get_absolute_url(self):
    return reverse("plugins:nautobot_example_dns_manager:dnszonemodel", args=[self.slug])

This returns the exact URL for an instance of our object. Essentially, reverse looks up the path with the arguments provided. So if we had a DnsZoneModel object with a name such as nautobot.example.com, reverse would return something like: localhost:8080/plugins/example-dns-manager/dnszonemodel/nautobot-example-com, nautobot-example-com being the slug created from the name.

We’ll add one more thing for convenience: a __str__ representation of our objects, in case we need to view them in shell_plus. Nautobot also uses the __str__ representation in place of a UUID in generic views, which makes using the generic views and templates more user-friendly.

Our DnsZoneModel should look like this:

class DnsZoneModel(PrimaryModel):
    """Model representing DNS Zone and SOA record.
    """
    name = models.CharField(max_length=200, help_text="FQDN of the Zone, w/ TLD.")
    slug = AutoSlugField(populate_from="name")
    mname = models.CharField(max_length=200, help_text="FQDN of the Authoritative Name Server for Zone.")
    rname = models.EmailField(help_text="Admin Email for the Zone in the form user@zone.tld.")
    refresh = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default="86400",help_text="Number of seconds after which secondary name servers should query the master for the SOA record, to detect zone changes.")
    retry = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=7200, help_text="Number of seconds after which secondary name servers should retry to request the serial number from the master if the master does not respond.")
    expire = models.IntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=3600000, help_text="Number of seconds after which secondary name servers should stop answering request for this zone if the master does not respond. This value must be bigger than the sum of Refresh and Retry.")
    ttl = models.PositiveIntegerField(validators=[MinValueValidator(300), MaxValueValidator(2147483647)], default=3600, help_text="Time To Live.")

    def get_absolute_url(self):
        return reverse("plugins:nautobot_example_dns_manager:dnszonemodel", args=[self.slug])

    def __str__(self):
        return self.name

The next thing we’ll add is a very basic table definition in a new file tables.py, just so that we can see something. This will use a generic template that Nautobot provides for list views to display the page.

"""table.py"""

from nautobot.utilities.tables import BaseTable

from .models import DnsZoneModel

class DnsZoneModelTable(BaseTable):

    class Meta(BaseTable.Meta):
        model = DnsZoneModel

Import the DnsZoneModelTable then add it under the View as a table_class.

"""views.py"""

from nautobot.core.views import mixins as view_mixins

from .models import DnsZoneModel, ARecordModel, CNameRecordModel
from .tables import DnsZoneModelTable

class DnsZoneModelUIViewSet(view_mixins.ObjectListViewMixin,
    view_mixins.ObjectDetailViewMixin,
    view_mixins.ObjectEditViewMixin,
    view_mixins.ObjectDestroyViewMixin,
    view_mixins.ObjectBulkDestroyViewMixin,
):
    queryset = DnsZoneModel.objects.all()
    table_class = DnsZoneModelTable

Once the table is in place, you should see a basic table view when you click on the DNS Zones in the Navigation menu. The table won’t have anything in it yet since we have not created any objects just yet. In the next few articles we’ll cover the table and templates more.

References

• DNS RFC.
• CloudFlare SOA options refrence.
• CloudFlare DNS reference.
• Django model field type reference.
• ForeignKey on_delete methods.
• Nautobot Applications.
• Nautobot model validation.
• Adding Nautobot Slug field to your models.