Introduction to Python Classes – Part 2

Last week we started a series on Python classes, and this week we’re continuing to talk about class design. We’re going to look at what it means when underscores (_s) appear at the beginning and/or end of a name in Python, and when you might want to use them or avoid using them in your own classes. We’ll be building on the example class introduced last week (below). First, a quick note on terminology: In Python parlance, “dunder” is short for “double underscore”, and it can refer to something with two underscores in front, like __method, or two underscores in front and behind, like __init__ (so one would usually pronounce __init__ “dunder init”).

# ip_man.py
import os

import requests


class IpManager:
    """Class to assign IP prefixes in Nautobot via REST API"""

    def __init__(self):
        self.base_url = "https://demo.nautobot.com/api"
        _token = self._get_token()
        self.headers = {
            "Accept": "application/json",
            "Authorization": f"Token {_token}",
        }

    @staticmethod
    def _get_token():
        """Method to retrieve Nautobot authentication token"""
        return os.environ["NAUTOBOT_TOKEN"]

    def get_prefix(self, prefix_filter):
        """Method to retrieve a prefix from Nautobot

        Args:
            prefix_filter (dict): Dictionary supporting a Nautobot filter

        Returns:
            obj: Requests object containing Nautobot prefix

        """
        url = f"{self.base_url}/ipam/prefixes/"
        response = requests.get(url=url, headers=self.headers, params=prefix_filter)
        return response

    def new_prefix(self, parent_prefix_id, new_prefix):
        """Method to add a new prefix within a parent prefix to Nautobot

        Args:
            parent_prefix_id (str): UUID identifying a parent prefix
            new_prefix (dict): Dictionary defining new prefix

        Returns:
            obj: Requests object containing new Nautobot prefix

            >>>
        """
        url = f"{self.base_url}/ipam/prefixes/{parent_prefix_id}/available-prefixes/"
        body = new_prefix
        response = requests.post(url=url, headers=self.headers, json=body)
        return response

    def get_available_ips(self, prefix_id):
        """Method to retrieve unused available IP addresses within a prefix

        Args:
            prefix_id (str): UUID identifying a prefix

        Returns:
            obj: Request object containing list of available IP addresses within Nautobot prefix

        """
        url = f"{self.base_url}/ipam/prefixes/{prefix_id}/available-ips/"
        response = requests.get(url=url, headers=self.headers)
        return response

    def delete_prefix(self, prefix_id):
        """Method to delete a Nautobot prefix
        Args:
            prefix_id (str): UUID identifying a prefix

        Returns:
            None

        """
        url = f"{self.base_url}/ipam/prefixes/{prefix_id}/"
        response = requests.delete(url=url, headers=self.headers)
        return response

One Trailing Underscore

One of the most notoriously difficult parts of programming is naming things. I often find myself wanting to name a variable id, hash, pass, or any number of words that are sadly already taken by Python builtins and the standard library. Python gives you the tools of your own demise here: it is possible to shadow (take the name of) builtins. Consider the following code:

>>> s = "Hello World"
>>> print(s)
Hello World
>>> print(id(s))
4421427760
>>> id = 5
>>> print(id(s))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable

While id is a built-in part of Python, there’s nothing stopping me from using id as a variable name. If I do this, and then later in my code I use “id”, I might now be referring to the wrong “id” (depending on the context), and I’ve made my code at least difficult to understand, if not actually buggy. Adding an underscore to the ends of variable names is a common convention to solve this problem, so id would become id_, for example.

One Leading Underscore

A leading underscore is a design choice that says to users, “Don’t touch this, please.” Methods, variables, etc. with one leading underscore are not considered part of a class or module’s public API. This is a PEP-8 naming convention. In our example class, we named _get_token with a leading underscore. By choosing to start the name with an underscore, we signaled to consumers of our class that they should probably not call this method directly. And that’s appropriate in the case of IpManager, because if someone is importing this class and using it in their code, they’re probably not interested in the functionality that this method provides, but in the other parts of IpManager that provide useful features. Note that this is just a convention; nothing in Python forces methods or variables with one leading underscore to actually be “private,” but users who access these internals directly should be aware that the results may be unpredictable.

Two Leading Underscores

Two leading underscores is a design choice that says to users, “Don’t touch this, please. I really mean it.” It invokes name mangling. These methods and variables can still be accessed by outside code, but the outside code has to work slightly harder. For example:

>>> class A:
        def __method(self):
            """A "private" method that we don't want others to call directly."""
            print("I am a method in A")
        def method(self):
            """A "public" method that relies on a private method."""
            self.__method()
>>> obj = A()
>>> obj.method()
I am a method in A
>>> obj.__method()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute '__method'
>>> obj._A__method()
I am a method in A

Notice that while code in the A class itself can call self.__method(), code outside of the class has to use the mangled name obj._A__method in order to access that method directly. Continuing with this example class, if we wrote

class B(A):
    pass
>>> b = B()

then there would be no b.__method, even though b is an instance of B which inherits from A, and therefore has all of A’s methods by default. But b does have _A__method, which is what name mangling buys you. So as a design pattern, this can be used to hide methods, even from inheritor classes.

Two Leading and Two Trailing Underscores

And finally, two underscores in front and two behind indicates a “magic” method or variable. These are usually things that Python objects get for free, but you can reference them (or override them) directly if you need to. Examples include comparison operators like __eq__, and even certain globals like __name__ (of if __name__ == "__main__": fame).

Consider the contents of the IpManager class (above). We defined six methods, including __init__, but if we look at the contents of the class once it’s loaded into the interpreter, there’s much more there:

>>> from ip_man import IpManager
>>> dir(IpManager)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_get_token', 'delete_prefix', 'get_available_ips', 'get_prefix', 'new_prefix']

These magic methods were all applied to our class automatically by Python. The reason for this is that a lot of basic functionality under the hood in Python depends on objects having these methods. For example, the __str__ magic method is what the interpreter checks when it wants to convert an object to a string:

>>> ip_mgr = IpManager()
>>> str(ip_mgr)
'<ip_man.IpManager object at 0x10de09cf0>'
>>> ip_mgr.__str__()
'<ip_man.IpManager object at 0x10de09cf0>'

And other common operations like equality are implemented the same way:

>>> ip_mgr == 1
False
>>> ip_mgr == ip_mgr
True
>>> ip_mgr.__eq__(ip_mgr)
True

Designing a Class

So let’s take what we’ve learned and build a new class! Say we need to write a program to do some logic involving network prefixes. We might decide to write an object-oriented class to represent the concept of a prefix, so that we can then work with a bunch of these objects to figure out things like: Where are they being used in our infrastructure? Do they overlap? Are we aware of prefixes that still need to be imported to Nautobot? etc. The prefix objects we retrieve from Nautobot using IpManager each have an ID and a representation of the prefix in CIDR notation, as well as other properties. For now, our prefix class will keep track of just those two attributes, but we can expand it in the future to cover more attributes as our needs evolve:

class Prefix:
    """Class to represent network prefixes"""

    def __init__(self, id, prefix, **kwargs):
        """Accepts Nautobot prefix dictionaries as input"""
        self.id = id
        self.prefix = prefix

    def __str__(self):
        """Represents this object as a string for pretty-printing"""
        return f"<{self.__class__.__name__} {self.prefix}>"

    def __repr__(self):
        """Represents this object as a string for use in the REPL"""
        return f"<{self.__class__.__name__} (id={self.id}, prefix={self.prefix})>"

    def __eq__(self, other):
        """Used to determine if two objects of this class are equal to each other"""
        return bool(other.id == self.id)

    def __hash__(self):
        """Used to calculate a unique hash for this object, in case we ever want to use it in a dictionary or a set"""
        return hash(self.id)

>>> ip_mgr = IpManager()
>>> r = ip_mgr.get_prefix({"prefix":"10.0.0.0/8"})
>>> nautobot_prefix_data = r.json()["results"]
>>> prefix_objects = [Prefix(**item) for item in nautobot_prefix_data]
>>> prefix_objects
[<Prefix (id=08dabdef-26f1-4389-a9d7-4126da74f4ec, prefix=10.0.0.0/8)>, <Prefix (id=b4e452a2-25b2-4f26-beb5-dbe2bc2af868, prefix=10.0.0.0/8)>]
>>> print(prefix_objects[0])
<Prefix 10.0.0.0/8>

Here we’ve implemented several custom magic methods. There are many more, but there’s no need to override them all if we don’t need them yet. Use of magic methods where appropriate can be a fantastic design pattern, because it lets us implement features in a way that users of our code will be able to take advantage of with little to no special knowledge of our code. For example, if we provided a string representation of a Prefix via a method called Prefix.to_string, that would be great, but by using Prefix.__str__ we allow people to simply call str(prefix) or print(prefix) and have it work as expected.

Note also that even though id in the __init__ method shadows the builtin id, I chose to leave it there instead of renaming it to id_ because in this case, we want to accept prefix config from Nautobot, and it uses id, not id_, so I made a design choice to keep the class easy to use. If I were going to send this code through a linter, I might have to add a comment explaining that this isn’t a problem in this case. But if at some point in the future I wrote some code further down in the same module that needed to use the builtin id, I’d be setting myself up for failure.