A brief introduction to Google protobuf

Time:2020-7-30

What is protobuf

Protobuf is actually a data transmission format and specification similar to JSON or XML, which is used for communication between different applications or processes. The information transmitted during communication is packaged by message data structure defined by protobuf, and then compiled into binary code stream for transmission or storage.

Benefits of protobuf

In comparison, protobuf has the following advantages:

  • Simple enough
  • Small size after serialization: message size only needs 1 / 10 ~ 1 / 3 of XML
  • Fast parsing speed: parsing speed is 20 ~ 100 times faster than XML
  • Multilingual support
  • Better compatibility, a principle of protobuf design is to be able to well support downward or upward compatibility

How to install and use protobuf

install

Using python, the easy installation method is as follows

PIP install protobuf
Sudo apt get install protobuf compiler

If you compile and install yourself, you can refer to the installation instructions on git, although it is not clear:)

use

There are several steps to use protobuf:

  • Define message
  • Initialization message and store transfer message
  • Read the message and parse it

Here is a practical example to illustrate how to use protobuf. First, the actual directory structure of the project is shown


.
├── my
│  ├── helloworld_pb2.py
│  ├── helloworld_pb2.pyc
│  └── __init__.py
├── mybuffer.io
├── my.helloworld.proto
├── reader.py
└── writer.py

Define message

The message structure of protocol buffer is defined and described by a language called protocol buffer language. In fact, protocol buffer language is divided into two versions, version 2 and version 3. By default, version 2 is used without declaration. Let’s take version 2 as an example. Suppose we define the file name as my.helloworld.proto As follows:


package my;
message helloworld
{
  required int32 id = 1;
  required string str = 2;
  optional int32 wow = 3;
}

Then we need to compile using protoc


protoc -I=./ --python_out=./ ./my.helloworld.proto
  • -1: Is to set the source path
  • –python_ Out: used to set the output result after compilation. If using other languages, please use the option of the corresponding language
  • The last parameter is the proto file you want to compile

Now that you have defined the data structure for the message, let’s take a look at how to use it

Message initialization and storage transmission

Let’s go through writer.py To initialize the message and store it as a file. The code is as follows:


from my.helloworld_pb2 import helloworld

def main():
  hw = helloworld()
  hw.id = 123
  hw.str = "eric"
  print hw

  with open("mybuffer.io", "wb") as f:
    f.write(hw.SerializeToString())

if __name__ == "__main__":
  main()

implement writer.py The result of serialization is then stored in a file mybuffer.io And then see how to read it

Message reading and parsing

We passed reader.py To read and parse the message, the code is as follows:


from my.helloworld_pb2 import helloworld

def main():
  hw = helloworld()
  with open("mybuffer.io", "rb") as f:
    hw.ParseFromString(f.read())
    print hw.id
    print hw.str

if __name__ == "__main__":
  main()

Reference:

Developer Guide
Github

This article on the detailed explanation of Google protobuf concise tutorial article introduced here, more relevant Google protobuf content, please search the previous articles of developeppaer or continue to browse the relevant articles below, I hope you will support developeppaer more in the future!