Nautobot Apps and Data Model Relationships

Blog Detail

When developing a Nautobot App, there are multiple ways to integrate any new data models belonging to that App with the core data models provided by Nautobot itself. I’m writing to share a few quick tips about which approaches to choose.

Classes of Data Relationships

There are four basic classes of data relationships you might wish to implement in your App:

  1. One to One: Each record of type A relates to at most one record of type B and vice versa. For example, a VirtualChassis has at most one Device serving as the primary for that chassis, and a Device is the primary for at most one VirtualChassis.
  2. One to Many: Each record of type A relates to any number of records of type B, but each record of type B relates to at most one record of type A. For example, a Location may have many Racks, but each Rack has only one Location.
  3. Many to One: The reverse of the previous class. I’m calling it out as a separate item, because in some cases it needs to be handled differently when developing an App.
  4. Many to Many: Any number of records of type A relate to any number of records of type B. For example, a VRF might have many associated RouteTarget records as its import and export targets, and a RouteTarget might be reused across many VRF records.

Options for Implementing Data Relationships in Nautobot

The first, and seemingly easiest, approach to implement would be something like a CharField on your App’s model (or a String-type CustomField added to a core model) that identifies a related record(s) by its nameslug, or similar natural key. I’m including this only for completeness, as really you should never do this. It has many drawbacks, notably in terms of data validation and consistency. For example, there’s no inherent guarantee that the related record exists in the first place, or that it will continue to exist so long as you have a reference to it. Nautobot is built atop a relational database and as such has first-class support for representing and tracking object relationships at the database level. You should take advantage of these features instead!

The next, and most traditional, approach is to represent data relationships using native database features such as foreign keys. This has a lot of advantages, including database validation, data consistency, and optimal performance. In most cases, this will be your preferred approach when developing new data models in your App, but there are a few cases where it isn’t possible.

The final approach, which is specific to Nautobot, is to make use of Nautobot’s Relationship feature, which allows a user or developer to define arbitrary data relationships between any two models. This is an extremely powerful and flexible feature, and is especially useful to a Nautobot user who wishes to associate existing models in a new way, but from an App developer standpoint, it should often be your fallback choice rather than your first choice, because it lacks some of the performance advantages of native database constructs.

Implementing One-to-One Data Relationships

A one-to-one relationship between App data models, or between an App model and a core Nautobot model, should generally be implemented as a Django OneToOneField on the appropriate App data model. This is a special case of a ForeignKey and provides all of the same inherent performance and data consistency benefits. You can use Django features such as on_delete=models.PROTECT or on_delete=models.CASCADE to control how your data model will automatically respond when the other related model is deleted.

An example from the nautobot-firewall-models App:

class CapircaPolicy(PrimaryModel):
    """CapircaPolicy model."""

    device = models.OneToOneField(
        to="dcim.Device",
        blank=True,
        null=True,
        on_delete=models.CASCADE,
        related_name="capirca_policy",
    )

In this example, each CapircaPolicy maps to at most one Device, and vice versa. Deleting a Device will result in its associated CapircaPolicy being automatically deleted as well.

If, and only if, your App needs to define a new relationship between two core Nautobot models, you cannot use a OneToOneField because an App cannot directly modify a core model. In this case, your fallback option would be to create a one-to-one Relationship record as the way of adding this data relationship. This is a pretty rare case, so I don’t have a real-world example to point to, but it would conceptually be implemented using the nautobot_database_ready signal:

def handle_nautobot_database_ready(sender, *, apps, **kwargs):
    Relationship.objects.get_or_create(
        slug="originating_device_to_vrf",
        defaults={
            "name": "Originating Device to VRF",
            "type": RelationshipTypeChoices.TYPE_ONE_TO_ONE,
            "source_type": ContentType.objects.get_for_model(Device),
            "destination_type": ContentType.objects.get_for_model(VRF),
        },
    )

Implementing One-to-Many and Many-to-One Data Relationships

A one-to-many or many-to-one data relationship between two App models should be implemented as a standard Django ForeignKey field from the “many” model to the “one” model. The same approach works for a many-to-one relationship from an App model to a core Nautobot model.

An example from the nautobot-device-lifecycle-mgmt App:

class SoftwareLCM(PrimaryModel):
    """Software Life-Cycle Management model."""

    device_platform = models.ForeignKey(
        to="dcim.Platform",
        on_delete=models.CASCADE,
        verbose_name="Device Platform"
    )

In this example, many SoftwareLCM may all map to a single Platform, and deleting a Platform will automatically delete all such SoftwareLCM records.

Because, again, an App cannot directly modify a core model, this approach cannot be used for a one-to-many relation from an App model to a core model, or between two core models, because it would require adding a ForeignKey on the core model itself. In this case, you’ll need to create a Relationship, as in this example from the nautobot-ssot App’s Infoblox integration:

def nautobot_database_ready_callback(sender, *, apps, **kwargs):
    # ...

    # add Prefix -> VLAN Relationship
    relationship_dict = {
        "name": "Prefix -> VLAN",
        "slug": "prefix_to_vlan",
        "type": RelationshipTypeChoices.TYPE_ONE_TO_MANY,
        "source_type": ContentType.objects.get_for_model(Prefix),
        "source_label": "Prefix",
        "destination_type": ContentType.objects.get_for_model(VLAN),
        "destination_label": "VLAN",
    }
    Relationship.objects.get_or_create(name=relationship_dict["name"], defaults=relationship_dict)

Implementing Many-to-Many Data Relationships

A many-to-many data relationship involving App models should be implemented via a Django ManyToManyField. An example from the nautobot-circuit-maintenance App:

class NotificationSource(OrganizationalModel):
    # ...

    providers = models.ManyToManyField(
        Provider,
        help_text="The Provider(s) that this Notification Source applies to.",
        blank=True,
    )

One NotificationSource can provide notifications for many different Providers, and any given Provider may have multiple distinct NotificationSources.

Once again, the only exception is when a relationship between two core Nautobot models is desired, in which case use of a Relationship would be required. This is another fairly rare case and so I don’t have a real-world example to point to here, but it would follow the similar pattern to the other Relationship examples above.

Conclusion and Summary

Here’s a handy table summarizing which approach to take for various data relationships:

Model AModel BCardinalityRecommended Approach
App modelApp modelOne-to-OneOneToOneField on either model
App modelApp modelOne-to-ManyForeignKey on model B
App modelApp modelMany-to-OneForeignKey on model A
App modelApp modelMany-to-ManyManyToManyField on either model
App modelCore modelOne-to-OneOneToOneField on model A
App modelCore modelOne-to-ManyRelationship definition
App modelCore modelMany-to-OneForeignKey on model A
App modelCore modelMany-to-ManyManyToManyField on model A
Core modelCore modelOne-to-OneRelationship definition
Core modelCore modelOne-to-ManyRelationship definition
Core modelCore modelMany-to-OneRelationship definition
Core modelCore modelMany-to-ManyRelationship definition

Conclusion

I hope you’ve found this post useful. Go forth and model some data!

-Glenn



ntc img
ntc img

Contact Us to Learn More

Share details about yourself & someone from our team will reach out to you ASAP!

Nautobot Application: BGP Models

Blog Detail

We are happy to announce the release of a new application for Nautobot. With this application, it’s now possible to model your ASNs and BGP Peerings (internal and external) within Nautobot!

This is the first application of the Network Data Models family which gave us a great opportunity to test some new capabilities of the application framework introduced by Nautobot. Data modeling is an interesting exercise, and with BGP being a complex ecosystem, this has been an interesting project. This blog will present the application and some design principles that we had in mind when it was developed.

Nautobot

The development of this application was initially sponsored by the Riot Direct team at Riot Games. Thanks to them for contributing it back to the community.

Overview

This application adds the following new data models into Nautobot:

  • BGP Routing Instance : device-specific BGP process
  • Autonomous System : network-wide description of a BGP autonomous system (AS)
  • Peer Group Template : network-wide template for Peer Group objects
  • Peer Group : device-specific configuration for a group of functionally related BGP peers
  • Address Family : device-specific configuration of a BGP address family (AFI-SAFI)
  • Peering and Peer Endpoints : A BGP Peering is represented by a Peering object and two endpoints, each representing the configuration of one side of the BGP peering. A Peer Endpoint must be associated with a BGP Routing Instance.
  • Peering Role : describes the valid options for PeerGroupPeerGroupTemplate, and/or Peering roles

With these new models, it’s now possible to populate the Source of Truth (SoT) with any BGP peerings, internal or external, regardless of whether both endpoints are fully defined in the Source of Truth.

The minimum requirement to define a BGP peering is two IP addresses and one or two autonomous systems (one ASN for iBGP, two ASNs for eBGP).

Peering

Peering

Autonomous Systems

Autonomous Systems

Peer Endpoint

Peer Endpoint

Peer Group

Peer Group

Peering Roles

Peering Roles

Installing the Application

The application is available as a Python package in PyPI and can be installed atop an existing Nautobot installation using pip:

$ pip3 install nautobot-bgp-models

This application is compatible with Nautobot 1.3.0 and higher.

Once installed, the application needs to be enabled in the nautobot_config.py file:

# nautobot_config.py
PLUGINS = [
    # ...,
    "nautobot_bgp_models",
]

Design Principles

BGP is a protocol with a long and rich history of implementations. As we understand existing limitations of data modeling relevant to this protocol, we had to find right solutions both for innovations and improvements. In this section we explain our approach to the BGP data models.

Network View and Relationship First

One of the advantages of a Source of Truth is that it captures how all objects are related to each other and then exposes those relationships via the UI and API, making it easy for users to consume that information.

Instead of modeling a BGP session from a device point of view with a local IP address and a remote IP address, the decision to model a BGP peering as a relationship between two endpoints was chosen. This way, each endpoint has a complete understanding of what is connected on the other side, and information won’t be duplicated when a session between two devices exists in the SoT.

This design also accounts for external peering sessions where the remote device is not present in Nautobot, as is often the case when you are peering with a transit provider.

Start Simple

For the first version we decided to focus on the main building blocks that compose a BGP peering. Over time the BGP application will evolve to support more information: routing policy, community, etc. Before increasing the complexity we’d love to see how our customers and the community leverage the application.

Inheritance

Many of the Border Gateway Protocol implementations are based on the concept of inheritance. It’s possible to centralize almost all information into a Peer Group Template model, and all BGP endpoints associated with this Peer Group Template will inherit all its attributes.

The concept is very applicable to automation, and we wanted to have a similar concept in the SoT. As such, we implemented an inheritance system between some models:

  • PeerGroup inherits from PeerGroupTemplate.
  • PeerEndpoint inherits from PeerGroupPeerGroupTemplateBGPRoutingInstance.

As an example, a PeerEndpoint associated with a PeerGroup will automatically inherit attributes of the PeerGroup that haven’t been defined at the PeerEndpoint level. If an attribute is defined on both, the value defined on the PeerEndpoint will be used.

(*) Refer to the application documentation for all details about the implemented inheritance pattern.

The inherited values will be automatically displayed in the UI and can be retrieved from the REST API with the additional ?include_inherited=true parameter.

Inheritance

Extra Attributes

Extra attributes allow to describe models provided by the application with additional information. We made a design decision to allow application users to abstract their configuration parameters and store contextual information in this special field. What makes it very special is the support for inheritance. Extra attributes are not only inherited, but also intelligently deep-merged, thus allowing for inheriting and overriding attributes from related objects.

Integration with the Core Data Model

With Nautobot, one of our goals is to make it easy to extend the data model of the Source of Truth, not only by making it easy to introduce new models but also by allowing applications to extend the core data model. In multiple places, the BGP application is leveraging existing Core Data models.

Extensibility

We designed the BGP models to provide a sane baseline that will fit most of the use cases, and we encourage everyone to leverage all the extensibility features provided by Nautobot to store and organize the additional information that you need under each model or capture any relationship that is important for your organization.

All models introduced by this application support the same extensibility features supported by Nautobot, which include:

  • Custom fields
  • Custom links
  • Relationships
  • Change logging
  • Custom data validation logic
  • Webhooks in addition to the REST API and GraphQL.

An example can be seen in the Nautobot Sandbox where a relationship between a circuit and a BGP session was added to track the association between a BGP session and a given circuit.


Conclusion

More information on this application can be found at Nautobot BGP Plugin. You can also get a hands-on feel by visiting the public Nautobot Sandbox.

As usual, we would like to hear your feedback. Feel free to reach out to us on Network to Code’s Slack Channel!

-Damien & Marek



ntc img
ntc img

Contact Us to Learn More

Share details about yourself & someone from our team will reach out to you ASAP!

Introducing Nautobot Firewall Models

Blog Detail

Introducing the Nautobot Firewall Models application! A data-model driven solution for modeling layer 4 firewall policies and access lists. This will allow anyone to model policies to drive their network and security automation. Even better, the firewall models are built to take a vendor-agnostic approach; the models are robust to provide maximum flexibility.

The application is currently available for install directly via PyPI as a beta release with the intent to gather feedback from the community and implement a few enhancements to help with the user experience prior to a v1.0.0 release. The app requires Nautobotv1.3.0 or later, as it employs the use of DynamicGroups as an option in Policy assignment.

Setting Up a Development Environment

The application follows the same type of development environment as most apps found in the Nautobot namespace of GitHub from the stance of being built on PoetryDocker, and PyInvoke. This app does introduce a management command to Nautobot that is very helpful to use in a local development environment OR to just populate a small set of data for use in testing out the app. Management commands are a set of actions that the nautobot-server command has access to. A few common ones you see are createsuperuser or runserver. The management command published by the app is create_test_firewall_data. This command uses the same set of data used in running unittests to populate a set of test data. To follow a similar pattern for the other common management command, in the development environment you can run invoke testdata to wrap running the nautobot-server command. Why does this matter? If you’re using the local development environment, the invoke command is easiest. But if you are not using local development for the deployment, nautobot-server create_test_firewall_data is available.

Example Local Setup Commands

➜  plugins git clone git@github.com:nautobot/nautobot-plugin-firewall-models.git
➜  plugins cd nautobot-plugin-firewall-models 
➜  nautobot-plugin-firewall-models git:(develop) poetry install
➜  nautobot-plugin-firewall-models git:(develop) poetry shell
➜  nautobot-plugin-firewall-models git:(develop) cp development/creds.example.env development/creds.env
➜  nautobot-plugin-firewall-models git:(develop) invoke build
## I like running migrate, but it can be skipped as called by start.
➜  nautobot-plugin-firewall-models git:(develop) invoke migrate
➜  nautobot-plugin-firewall-models git:(develop) invoke start
## To do a one-time load of data, migrations must have finished.
➜  nautobot-plugin-firewall-models git:(develop) invoke testdata
Running docker-compose command "ps --services --filter status=running"
Running docker-compose command "exec nautobot nautobot-server create_test_firewall_data"
Attempting to populate dummy data.
Successfully populated dummy data!

Navigating the app is done via the Firewall nav bar dropdown.

 Navigation

Reviewing a Policy with Rules

After selecting Policy from navmenu, select a Policy from the list. To get more detail on the rule assigned to the Policy, click on Policy Rules Expanded.

 Policy Rules Expanded

Assigning the Policy to a Device OR DynamicGroup can be done via the Edit menu.

 Policy Rules Expanded

When initially assigning a PolicyRule to a Policy, the index of the relationship is not set. To set the index, click Edit Policy Rule Indexes and you will see No Index Set for the newly added PolicyRule. You can adjust multiple indexes at once and hit Submit.

 Policy Rules Index

The weight for assigned Devices or DynamicGroups is edited similarly to editing a PolicyRule index, however the weight does default to 100 on the initial assignment. 

Policy Rules Assigned Weight

Building a Simple Policy

Use the following steps to create a small Policy that has one PolicyRule: to allow HTTP/HTTPS access from an IPRange to an FQDN then an implicit deny.

  1. Create an IPRange with a starting address of 10.0.0.1 and ending address of 10.0.0.100.
  2. Create an FQDN named demo.nautobot.com.
  3. Create an AddressObject named User Workstations with the 10.0.0.1-10.0.0.100 IPRange assigned.
  4. Create a second AddressObject named Nautobot Demo FQDN with the demo.nautobot.com FQDN assigned.
  5. Since we have two ServiceObjects to target AND they are both created by the initial migrations, create a ServiceObjectGroup named Web Services with both the HTTP (TCP/80) and HTTPS (TCP/443) ServiceObjects assigned.
  6. Create a PolicyRule with the following attributes assigned:
    • name = Allow Users to Nautobot Demo
    • Source Address Objects = User Workstations
    • Destination Address Objects = Nautobot Demo FQDN
    • Service Object Groups = Web Services
    • Action = allow
    • Status = Active
    • Leave the remaining attributes blank. Policy Rule Creation
  7. Create a second PolicyRule with only the name set to DENY ALL and action set to deny.
  8. Create a Policy named Public demo access and assign both newly created PolicyRulesPolicy Creation
  9. Last, click Edit Policy Rule Indexes and assign the allowed rule an index of 10 and the deny rule an index of 100Policy Creation

Although this process does require navigating through a few views, all objects are backed by a full REST API and can be fully created and maintained via the API.


Conclusion
  1. To help with ingesting existing rules, one of the first items on the roadmap is to create an ingestion job that leverages ntc-templates to parse an existing CLI output and create the underlying objects to model the Policy. This will be part of several UI/UX enhancements that will be prioritized based on community feedback.
  2. For translating a Policy object to a network device configuration, Ken Celenza is currently prototyping the use of capirca for generating the configuration.
  3. Ability to create a single PolicyRule on one page without navigating to additional pages to create the nested objects.

For questions along the way, feel free to check out the #nautobot channel in NTC Slack and create issues or discussion on the GitHub Repo.

-Jeremy White



ntc img
ntc img

Contact Us to Learn More

Share details about yourself & someone from our team will reach out to you ASAP!