This blog will build on top of what was discussed in Part 1 of this series. If you have not, I highly recommend checking it out here. In it, I discuss Protocol Buffers, which are an integral part of gRPC. In this blog post, we’ll build a simple gRPC client/server setup so we can actually see the definition files in action. In order to get to a fully working gRPC client/server, we need to take the following steps:

• Create a service in our Protocol Buffer
• Create a request method within our service
• Create a response in our Protocol Buffer

Extending Our Protocol Buffer Definition File

Let’s go over the three additions we need to make to our Protocol Buffer definition file (the service, request, and response portions).

Adding the Service

Currently, our Protocol Buffer definition should look like this:

syntax = "proto2";

package tutorial;

message Interface {
  required string name = 1
  optional string description = 2
  optional int32 speed = 3
}

Let’s add the service block to our definition file.

service InterfaceService {
}

This line defines a service, named InterfaceService, which our gRPC server will be offering. Within this service block, we can add the methods that the gRPC client can call.

Adding the Request and Response Methods

Before we add the request and response methods, I need to discuss the different types of messages gRPC services can handle. Four basic implementations of gRPC request and response methods can be used:

1. Unary – This is similar to REST. The client sends a request, and the server sends a single response message.
2. Server Streaming – The client sends a message, and the server responds with a stream of messages.
3. Client Streaming – The client sends a stream of messages, and the server responds with a single message.
4. Bidirectional Streaming – Both the client and server send streams of messages.

Each method has its pros and cons. For the sake of keeping this blog short, I’ll be implementing a unary request/response gRPC service.

You can read official documentation on gRPC message types here.

Let’s add a unary request/response type to our InterfaceService in our definition file.

service InterfaceService {
  rpc StandardizeInterfaceDescription(Interface) returns (Interface) {}
}

We are defining an RPC remote method named StandardizeInterfaceDescription that takes in data that adheres to our Interface message type we defined in the first blog post. We also define that the method will return data that adheres to our Interface message type.

You can have a gRPC function take in and return different message types.

Now, our Protocol Buffer definition file titled interface.proto should look like this.

syntax = "proto2";


service Interface {
  rpc StandardizeInterfaceDescription(Interface) returns (Interface) {}
}

package tutorial;

message Interface {
  required string name = 1;
  optional string description = 2;
  optional int32 speed = 3;
}

Re-creating Our Python gRPC Files

Now that we have our updated interface.proto definition file, we need to recompile to update our auto-generated gRPC Python code. Here we will be using the grpcio-tools library rather than the Protocol Buffer compiler we used in the first blog post. The grpcio-tools library is the more all encompassing tool compared to the Protcol Buffer compiler. To install the grpcio-tools library, run the command pip install grpcio-tools. Then, making sure you are in the same directory as the interface.proto file, run the command python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. interface.proto.

The -I designates the proto_path or, in other words, the path to the destination for your proto definition file. The last argument is the name of your proto file.

After running this command, you should have two new files named interface_pb2.py and interface_pb2_grpc.py. The first, is the protobuf class code and the second is the gRPC code related to our service. Here is a snapshot of the current directory structure:

├── interface_pb2_grpc.py
├── interface_pb2.py
├── interface.proto.py

Creating the gRPC Server

Now let’s create the code needed for our gRPC server. Create a new file at the root of the directory we are working in and name it grpc_server.py. Copy the below code snippet into that file:

from concurrent import futures

import grpc
import interface_pb2
import interface_pb2_grpc


class InterfaceGrpc(interface_pb2_grpc.InterfaceServiceServicer):
    def StandardizeInterfaceDescription(self, request):
        standard_description = request.description.upper()
        return interface_pb2.Interface(
            name=request.name, description=standard_description, speed=request.speed
        )


def server():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=2))
    interface_pb2_grpc.add_InterfaceServiceServicer_to_server(InterfaceGrpc(), server)
    server.add_insecure_port("[::]:5001")
    print("Starting gRPC Server")
    server.start()
    server.wait_for_termination()


if __name__ == "__main__":
    server()

Let’s take a quick look at this file. At the top we are importing a number of things. First, from the concurrent library, we are importing futures. This just allows us to asynchrously execute callables. We also import the grpc library. Lastly, we import the two files we created earlier.

At the bottom of the file, our entry point into this file is the server() function, which does four main things:

1. Creates a gRPC server from the grpc library
2. Registers our Interface Service to the newly created gRPC server
3. Adds a port that gRPC server will listen on
4. Starts the server

Near the top of the file, we are extending the InterfaceServiceServicer from our interface_pb2_grpc file. Within this class is where we define the logic for the functions we created stubs for in our .proto file. Functions created in this class take a single required argument other than self listed below:

• request – This is the interface message type sent into our gRPC function.

The next few lines take the data passed in by the request argument, capitalizes the description, creates an interface message type and sends it back to the client. That’s the gRPC server code. Let’s quickly put together a new file for our gRPC client.

Creating the gRPC Client

Creating the gRPC client is pretty straightforward. Copy and paste the below code snippet into a file called grpc_client.py.

import grpc
import interface_pb2
import interface_pb2_grpc


def client():
    channel = grpc.insecure_channel("localhost:5001")
    grpc_stub = interface_pb2_grpc.InterfaceServiceStub(channel)
    interface = interface_pb2.Interface(
        name="GigabitEthernet0/1", description="Port to DMZ firewall", speed=20
    )
    retrieved_interface = stub.StandardizeInterfaceDescription(interface)
    print(retrieved_interface)


if __name__ == "__main__":
    client()

The client function does the following:

1. Creates an insecure channel with a gRPC server running on your localhost on port 5001.
2. Using the channel created in step 1, subscribes to the InterfaceService service we defined in our .proto file.
3. Creates an Interface message object to pass into the gRPC call.
4. Calls the StandardizeInterfaceDescription function call and passes in the interface object from step 3.
5. Lastly, prints out what was received from the gRPC call.

Running the Client and Server Together

Now, let’s run the client and server together so we can see this code in action! First, open a terminal in the directory where the grpc_server.py lives and run python3 grpc_server.py. You should have no prompt and get the line Starting gRPC Server.

Open another terminal in the same location and run python3 grpc_client.py.

~/repo/Sandbox/blog_grpc ❯ python3 grpc_client.py

name: "GigabitEthernet0/1"
description: "PORT TO DMZ FIREWALL"
speed: 20

You can run the client side over and over again. The server will stay up and respond to however many requests it receives.

If everything was successful, you will get the response shown above. As you can see, it capitalized our entire description just like we wanted. I would definitely suggest playing around with this simple implementation of a gRPC server and client. You can even start to include non-unary functions in your gRPC server to explore more in-depth.

Gathering information programmatically is a core component of network automation. We constantly have the need to get data from devices, metric appliances, automation orchestration servers, etc. We, more often than not, use REST (representational state transfer) APIs (application programming interfaces) to gather the information we want. In the first blog post of this series, I want to introduce gRPC. I will discuss what gRPC is at a high level and dive deeper into one of its main components, Protocol Buffers.

What Is gRPC?

Let’s break down gRPC into some of its main components so we can see what gRPC is doing under the hood.

Remote Procedure Call (RPC)

To begin to understand gRPC, we need to ignore the ‘g’ for a moment and quickly go over what RPC is. RPC allows a program hosted on one machine to call a subroutine (i.e., a function) on a remote machine without it knowing the function being called is remote. RPC uses a client-server model where the server hosts and executes a particular subroutine and the client calls it. Its main uses are in distributed systems where it may make sense to host a particular subroutine that is used by many on an RPC server so it can be called by RPC clients.

gRPC

So, what is gRPC? gRPC is an open-source high-performance Remote Procedure Call (RPC) framework. gRPC was released by Google in 2016. (The g surprisingly does not stand for Google.) The meaning of g changes every release) that brings RPC into the modern world. Some key features of gRPC include:

• Implementation of Protocol Buffers as the IDL (Interface Definition Language), which allows gRPC to be language agnostic
• Use of HTTP/2
• Bidirectional streaming and flow control
• Authentication

Procotol Buffers

Now that we have an understanding of what gRPC is, let’s dive into Protocol Buffers. Protocol Buffers are an open-source cross-platform data format used to serialize structured data. Protocol Buffers were developed by Google and, as mentioned above, are tightly coupled with gRPC. How exactly does gRPC use Protocol Buffers? Protocol Buffers are used as the Interface Definition Language (IDL) for gRPC. The IDL defines both the services a gRPC server provides as well as structure of payload messages. For the remainder of this blog post, I want to go through creating and using a Protocol Buffers file so that in the next blog, when we use it with gRPC, there is a better understanding of what is taking place.

Defining a “.proto” File

Protocol Buffers (or protobufs for short) use “.proto” files to describe the data structure you wish to serialize. Once you have your .proto file created, you can use the Protocol Buffers compiler to easily create code in a number of languages. Let’s pretend we want to be able to serialize a data structure that modeled a switch interface. Our data model will require the interface name but optionally take the interface description and speed. Create an interface.proto file in a directory of your choosing with the file contents shown below.

Protocol Buffer syntax is very similar to C++. However, it is its own syntax. Information can be found here.

syntax = "proto2";

package tutorial;

message Interface {
  required string name = 1
  optional string description = 2
  optional int32 speed = 3
}

Let’s dive into this short .proto file. The first line says which version of protobuf we are using. In this case, it’s version 2. The next line defines a package namespace. You can disregard this line when using Protocol Buffers with Python. If you are using C++, Java, or Go, you can read how the package specifier interacts with those languages here.

The next line is the start of our message. A “message” in protobuf is nothing more than an aggregate containing typed fields. Within our Interface message you can see three data types are defined. Two strings, one for the interface name and another for the interface description, and one integer for the speed. The “ = 1”, “ = 2”, and “ = 3” at the end of each line identify a unique “tag” that field will use in the binary encoding. The details of that process don’t need to be understood to utilize protobufs, but in-depth documentation can be found here.

Compiling the “.proto” File

Now that we have our .proto file, we can use the Protocol Buffer compiler to create Python source code. First, make sure you have the Protocol Buffer compiler installed. Next, in the same location as your interface.proto file, run the command protoc --python_out=. interface.proto. This will generate an interface_pb2.py. For the sake of brevity, I will not post the contents of that file here, but definitely take a brief look at it.

Using the Python Source Code

Now that we have a .py file, let’s look at what we can do with it. Start a Python shell and run through the commands below.

>>> import interface_pb2
>>> sw_interface = interface_pb2.Interface()
>>> sw_interface.name = "GigabitEthernet0/1"
>>> sw_interface.description = "SDWAN Interface"
>>> sw_interface.name
GigabitEthernet0/1
>>> sw_interface.description
SDWAN Interface

So far, it doesn’t seem like anything too special. However, our Interface class isn’t a generic Python object. Because it was generated from a .proto file, it came with some extras! First off, you can’t add data to a field not defined in the .proto file.

>>> sw_interface.duplex = "full"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Interface' object has no attribute 'duplex'

You also can’t assign an incorrect type.

>>> sw_interface.speed = "1000"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '1000' has type str, but expected one of: int, long

Notice how I’m using a string here rather than an integer

If we run a dir() on our object, we can see some other helper methods.

>>> dir(sw_interface)
['ByteSize', 'Clear', 'ClearExtension', 'ClearField', 'CopyFrom', 'DESCRIPTOR', 'DiscardUnknownFields', 'Extensions', 'FindInitializationErrors', 'FromString', 'HasExtension', 'HasField', 'IsInitialized', 'ListFields', 'MergeFrom', 'MergeFromString', 'ParseFromString', 'RegisterExtension', 'SerializePartialToString', 'SerializeToString', 'SetInParent', 'UnknownFields', 'WhichOneof', '_CheckCalledFromGeneratedFile', '_SetListener', '__class__', '__deepcopy__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '__unicode__', '_extensions_by_name', '_extensions_by_number', 'description', 'name', 'speed']

You can see we get quite a few helper methods that we didn’t code ourselves. The main one I’d like to draw attention to is the SerializeToString. Let’s run that on our object and see what we get.

>>> sw_interface.SerializeToString()
b'\n\x12GigabitEthernet0/1\x12\x0fSDWAN Interface'

This function is important. The byte string produced from the SerializeToString method is what is passed over the network when we start looking at how Protocol Buffers are used in gRPC.