What Is gRPC? – Part 1

Gathering information programmatically is a core component of network automation. We constantly have the need to get data from devices, metric appliances, automation orchestration servers, etc. We, more often than not, use REST (representational state transfer) APIs (application programming interfaces) to gather the information we want. In the first blog post of this series, I want to introduce gRPC. I will discuss what gRPC is at a high level and dive deeper into one of its main components, Protocol Buffers.

What Is gRPC?

Let’s break down gRPC into some of its main components so we can see what gRPC is doing under the hood.

Remote Procedure Call (RPC)

To begin to understand gRPC, we need to ignore the ‘g’ for a moment and quickly go over what RPC is. RPC allows a program hosted on one machine to call a subroutine (i.e., a function) on a remote machine without it knowing the function being called is remote. RPC uses a client-server model where the server hosts and executes a particular subroutine and the client calls it. Its main uses are in distributed systems where it may make sense to host a particular subroutine that is used by many on an RPC server so it can be called by RPC clients.

gRPC

So, what is gRPC? gRPC is an open-source high-performance Remote Procedure Call (RPC) framework. gRPC was released by Google in 2016. (The g surprisingly does not stand for Google.) The meaning of g changes every release) that brings RPC into the modern world. Some key features of gRPC include:

• Implementation of Protocol Buffers as the IDL (Interface Definition Language), which allows gRPC to be language agnostic
• Use of HTTP/2
• Bidirectional streaming and flow control
• Authentication

Procotol Buffers

Now that we have an understanding of what gRPC is, let’s dive into Protocol Buffers. Protocol Buffers are an open-source cross-platform data format used to serialize structured data. Protocol Buffers were developed by Google and, as mentioned above, are tightly coupled with gRPC. How exactly does gRPC use Protocol Buffers? Protocol Buffers are used as the Interface Definition Language (IDL) for gRPC. The IDL defines both the services a gRPC server provides as well as structure of payload messages. For the remainder of this blog post, I want to go through creating and using a Protocol Buffers file so that in the next blog, when we use it with gRPC, there is a better understanding of what is taking place.

Defining a “.proto” File

Protocol Buffers (or protobufs for short) use “.proto” files to describe the data structure you wish to serialize. Once you have your .proto file created, you can use the Protocol Buffers compiler to easily create code in a number of languages. Let’s pretend we want to be able to serialize a data structure that modeled a switch interface. Our data model will require the interface name but optionally take the interface description and speed. Create an interface.proto file in a directory of your choosing with the file contents shown below.

Protocol Buffer syntax is very similar to C++. However, it is its own syntax. Information can be found here.

syntax = "proto2";

package tutorial;

message Interface {
  required string name = 1
  optional string description = 2
  optional int32 speed = 3
}

Let’s dive into this short .proto file. The first line says which version of protobuf we are using. In this case, it’s version 2. The next line defines a package namespace. You can disregard this line when using Protocol Buffers with Python. If you are using C++, Java, or Go, you can read how the package specifier interacts with those languages here.

The next line is the start of our message. A “message” in protobuf is nothing more than an aggregate containing typed fields. Within our Interface message you can see three data types are defined. Two strings, one for the interface name and another for the interface description, and one integer for the speed. The “ = 1”, “ = 2”, and “ = 3” at the end of each line identify a unique “tag” that field will use in the binary encoding. The details of that process don’t need to be understood to utilize protobufs, but in-depth documentation can be found here.

Compiling the “.proto” File

Now that we have our .proto file, we can use the Protocol Buffer compiler to create Python source code. First, make sure you have the Protocol Buffer compiler installed. Next, in the same location as your interface.proto file, run the command protoc --python_out=. interface.proto. This will generate an interface_pb2.py. For the sake of brevity, I will not post the contents of that file here, but definitely take a brief look at it.

Using the Python Source Code

Now that we have a .py file, let’s look at what we can do with it. Start a Python shell and run through the commands below.

>>> import interface_pb2
>>> sw_interface = interface_pb2.Interface()
>>> sw_interface.name = "GigabitEthernet0/1"
>>> sw_interface.description = "SDWAN Interface"
>>> sw_interface.name
GigabitEthernet0/1
>>> sw_interface.description
SDWAN Interface

So far, it doesn’t seem like anything too special. However, our Interface class isn’t a generic Python object. Because it was generated from a .proto file, it came with some extras! First off, you can’t add data to a field not defined in the .proto file.

>>> sw_interface.duplex = "full"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Interface' object has no attribute 'duplex'

You also can’t assign an incorrect type.

>>> sw_interface.speed = "1000"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '1000' has type str, but expected one of: int, long

Notice how I’m using a string here rather than an integer

If we run a dir() on our object, we can see some other helper methods.

>>> dir(sw_interface)
['ByteSize', 'Clear', 'ClearExtension', 'ClearField', 'CopyFrom', 'DESCRIPTOR', 'DiscardUnknownFields', 'Extensions', 'FindInitializationErrors', 'FromString', 'HasExtension', 'HasField', 'IsInitialized', 'ListFields', 'MergeFrom', 'MergeFromString', 'ParseFromString', 'RegisterExtension', 'SerializePartialToString', 'SerializeToString', 'SetInParent', 'UnknownFields', 'WhichOneof', '_CheckCalledFromGeneratedFile', '_SetListener', '__class__', '__deepcopy__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '__unicode__', '_extensions_by_name', '_extensions_by_number', 'description', 'name', 'speed']

You can see we get quite a few helper methods that we didn’t code ourselves. The main one I’d like to draw attention to is the SerializeToString. Let’s run that on our object and see what we get.

>>> sw_interface.SerializeToString()
b'\n\x12GigabitEthernet0/1\x12\x0fSDWAN Interface'

This function is important. The byte string produced from the SerializeToString method is what is passed over the network when we start looking at how Protocol Buffers are used in gRPC.