Blog Detail
As we’ve written previously, Poetry is the preferred method of managing Python projects using the PEP 621 method of storing project metadata in the pyproject.toml file. The intention behind using the PEP 621 format is to keep project metadata and related dependency management concise and contained in a single file. As Nautobot and all Apps are Django based applications that use Python, it makes Poetry the perfect solution for managing your Nautobot environment. This article will explain the various options Poetry provides for making managing and developing with Nautobot easier.
Managing Dependencies
The structure of your pyproject.toml file has been described in many other articles, so I won’t get into too much detail here. Below, I’ve provided an example of a new pyproject.toml for a Nautobot home lab that I’d like to manage using Poetry.
[tool.poetry]
name = "nautobot_homelab"
version = "0.1.0"
description = "Nautobot Home Lab Environment"
authors = ["Network to Code, LLC <info@networktocode.com>"]
[tool.poetry.dependencies]
python = "3.10"
nautobot = "1.5.5"
nautobot-capacity-metrics = "^2.0.0"
nautobot-homelab-plugin = {path = "plugins/homelab_plugin", develop = true}
nautobot-ssot = {git = "https://github.com/nautobot/nautobot-plugin-ssot.git", branch = "develop"}
[tool.poetry.dev-dependencies]
bandit = "*"
black = "*"
django-debug-toolbar = "*"
django-extensions = "*"
invoke = "*"
ipython = "*"
pydocstyle = "*"
pylint = "*"
pylint-django = "*"
pytest = "*"
requests_mock = "*"
yamllint = "*"
toml = "*"
As you can see, I’ve defined the versions to be used as Python 3.10 and Nautobot 1.5.5 for the environment. I’ve also included the Capacity Metrics App to enable use with my lab telemetry stack, an App called nautobot-homelab-plugin being locally developed, and finally the Single Source of Truth framework for use with the locally developed App. You’ll notice that those last two are defined using more than just the desired version. They were added into the Poetry environment using the local directory of the plugin being worked on or referencing a Git repository and branch where the code resides. The final group of dependencies are noted as dev-dependencies as they should only be used for development environments. This is where you’d put any packages that you wish to use while developing your Apps, such as code linters.
Local Path
The plugin added to the project via a local path was added by issuing the command poetry add --editable ./plugins/homelab_plugin at the command line. This works as long as Poetry finds another pyproject.toml file for that project in the specified folder. If found, it will include all documented dependencies when generating the project lockfile. This is extremely helpful when you are working with a local development environment and need to view your changes quickly. Adding the --editable tag will denote the path should be loaded in develop mode so changes will be dynamically loaded. This means that as you make changes to your App while developing it, you don’t have to rebuild the entire Python package for it to function. This makes it much easier and quicker to iterate on your App, as changes should be immediately reflected in your environment.
Git Repository
If the code for your App resides in a Git repository, it’s typically best to just reference the repository and branch where it’s found as opposed to cloning it locally. This is done by issuing the command poetry add git+https://github.com/nautobot/nautobot-plugin-ssot.git#develop at the command line. Using this method allows for you to retain the version control inherent to Git while still developing your App and testing it in your environment. This is especially handy when you’re working on a patch for some open-source project like the Infoblox SSoT App. As you don’t have direct access to the code, you would need to fork the repository and point to that for your environment. This enables you to test your fixes directly with your data and Nautobot before submitting a Pull Request back to the original repository for the fixes.
Local Development
Once you’ve determined all of the appropriate dependencies for your Nautobot environment, you should execute a poetry lock to generate the project lockfile. If you wish to use a local development environment, your next step would then be to issue a poetry install to install of those dependencies into the project virtual environment. This should include Nautobot and all of the dependencies you’ve defined in the pyproject.toml file. You will still be required to stand up either a Postgres or MySQL database and Redis server for full functionality. This can be quickly and easily accomplished by using a Docker container. Putting your secrets in a creds.env and all other environment variables in your development.env while using the following Docker Compose file will enable local development with your Poetry environment:
---
version: "3.8"
services:
postgres:
image: "postgres:13-alpine"
env_file:
- "development.env"
- "creds.env"
ports:
- "5432:5432"
volumes:
# - "./nautobot.sql:/tmp/nautobot.sql"
- "postgres_data:/var/lib/postgresql/data"
redis:
image: "redis:6-alpine"
command:
- "sh"
- "-c" # this is to evaluate the $NAUTOBOT_REDIS_PASSWORD from the env
- "redis-server --appendonly yes --requirepass $$NAUTOBOT_REDIS_PASSWORD"
env_file:
- "development.env"
- "creds.env"
ports:
- "6379:6379"
volumes:
postgres_data: {}
Docker Development
If you are working with Nautobot in a container-based environment, such as part of a Kubernetes or Nomad cluster, it might make sense to have your entire environment inside Docker containers instead of having Nautobot in your Poetry environment. However, you can still utilize the Poetry lockfile generated from the earlier step to create your Nautobot containers. By passing the desired Python and Nautobot versions to the Dockerfile below, you can generate a development container with Nautobot and your App installed.
ARG NAUTOBOT_VERSION
ARG PYTHON_VER
FROM ghcr.io/nautobot/nautobot:${NAUTOBOT_VERSION}-py${PYTHON_VER} as nautobot-base
USER 0
RUN apt-get update && \
apt-get upgrade -y && \
apt-get autoremove -y && \
apt-get clean all && \
rm -rf /var/lib/apt/lists/* && \
pip --no-cache-dir install --upgrade pip wheel
FROM ghcr.io/nautobot/nautobot-dev:${NAUTOBOT_VERSION}-py${PYTHON_VER} as nautobot-dev
CMD ["nautobot-server", "runserver", "0.0.0.0:8080", "--insecure"]
RUN apt-get update && \
apt-get upgrade -y && \
apt-get autoremove -y && \
apt-get clean all && \
rm -rf /var/lib/apt/lists/*
COPY ./pyproject.toml ./poetry.lock /source/
COPY ./plugins /source/plugins
# Install the Nautobot project to include Nautobot
RUN cd /source && \
poetry install --no-interaction --no-ansi && \
mkdir /tmp/dist && \
poetry export --without-hashes -o /tmp/dist/requirements.txt
# -------------------------------------------------------------------------------------
# Install all included plugins
# -------------------------------------------------------------------------------------
RUN for plugin in /source/plugins/*; do \
cd $plugin && \
poetry build && \
cp dist/*.whl /tmp/dist; \
done
COPY ./jobs /opt/nautobot/jobs
COPY nautobot_config.py /opt/nautobot/nautobot_config.py
WORKDIR /source
Having a self-contained environment for developing your App can be extremely helpful in ensuring that all variables are accounted for, can be easily reproduced by others, and will not impact the system you’re developing on. It also makes it easy to create a production environment by copying the appropriate files from your development container into the production container. This can also be utilized in a CI/CD pipeline for automated testing of your application.
Conclusion
Today we’ve gone over how you can use Poetry to manage your Nautobot environment. We’ve shown how you can have the Apps included in your environment by referencing a Git repository or simply referencing a local directory where the App resides. We’ve also seen how the environment that Poetry creates can be used for developing Nautobot Apps.
-Justin
Does this all sound amazing? Want to know more about how Network to Code can help you do this, reach out to our sales team. If you want to help make this a reality for our clients, check out our careers page.
Share :
Contact Us to Learn More
Share details about yourself & someone from our team will reach out to you ASAP!
