Use of protobuf

The difference between protobuf and JSON / XML

Like XML and JSON, protocol buffer is a tool for structural data serialization, but their data formats are quite different:
First, the data obtained after protocol buffer serialization is not a readable string, but a binary stream
Secondly, the data information in XML and JSON format is contained in the serialized data, and the serialized data can be restored without any other information; However, to use protocol buffer, you need to define the data format (. Proto protocol file) in advance. To restore a serialized data, you need to use the defined data format
Finally, in the demand scenario of large amount of data transmission, protocol buffer is smaller (3-10 times), faster (20-100 times) and easier to use & maintain than XML and JSON; Moreover, protocol buffer can be used across platforms and voice.

Protobuf usage process

1. We wrote the protocol buffer syntax to describe the data structure to be stored Proto file

2. Compiled by protocol buffer compiler Proto file. Will The proto file is converted into the code file of the corresponding platform (Python, C + +, Java)

Enter the following command at the terminal to compile

protoc -I=$SRC_DIR --xxx_out=$DST_DIR   $SRC_DIR/addressbook.proto
​
#Parameter description
# 1. $ SRC_ Dir: Specifies the to compile Proto file directory (if not provided, use the current directory)
# 2. -- xxx_ Out: XXX set according to the type of code to be generated
  """
  For Java, XXX = Java, i.e. -- Java_ out
  For C + +, XXX = CPP, i.e. -- CPP_ out
  """
# 3. $ DST_ Dir: Directory generated by compiled code (usually set the same as $src_dir)
# 4.  The last path parameter: needs to be compiled Specific path of proto file
#After compilation, Protoco buffer will generate corresponding code files according to different platforms

Go use protobuf

1. Download the protobuf compiler, protoc,github.com/protocolbuffers/protobu…, install the corresponding version according to different systems, write the bin directory to the environment variable, and execute protocol – version successfully, indicating that the installation is successful

>libprotoc 3.17.1

2. Get protobuf’s compiler plug-in protoc Gen go, and use other plug-ins for other languages
The warehouse is officially going to be abandoned and used as a new one

go get -u github.com/golang/protobuf/protoc-gen-go

3. Write proto files, such as hello proto

syntax = "proto3";
package proto;
option go_package ="./proto";
​
service Greeter {
rpc Hello(HelloRequest) returns (HelloResponse) {}
}
message HelloRequest {
string name = 1;
}
message HelloResponse {
string greeting = 1;
}

4. Compilation Proto file, found to generate greeter pb. Go file

 protoc --go_out=. proto/greeter.proto

Principle of protocol compiler

The protocol compiler supports different languages through the plug-in mechanism, such as the appearance of the protocol command--xxx_outIf there is no built-in XXX plug-in, protoc will continue to query whether there is an executable program named protoc Gen XXX in the current system, and finally generate code through the queried plug-in. So we use – go_ Out means to use the protocol Gen go executable program.

If we write a new protocol Gen go my command and register the plug-in netrpc, we can use the command – go my_ out=plugins=netrpc。

protoc-gen-go

The executable file also implements a layer of static plug-in system. For example, protocol Gen go has a built-in grpc plug-in, which can be used by users--go_out=plugins=grpcParameter to generate grpc related code, otherwise only related code will be generated for message.

Custom plug-ins

We can imitate protocol Gen go and customize a plug-in.

Let’s take a look at the main function (old version) of the protocol Gen go source code, so just register a custom plug-in object with the generator without modifying the main function

package main
​
import (
  "io/ioutil"
  "os"
​
  "github.com/golang/protobuf/proto"
  "GitHub. COM / golang / protobuf / protoc Gen go / generator" // our newly written plugin will be registered here
)
​
func main() {
  g := generator.New()
  data, err := ioutil.ReadAll(os.Stdin)
  if err != nil {
  g.Error(err, "reading input")
  }
  if err := proto.Unmarshal(data, g.Request); err != nil {
  g.Error(err, "parsing input proto")
  }
  ...
}

Custom plugin object, plugins / netrpc go

The generator will process the proto file to generate the generator The filedescriptor object holds the information inside.

package plugins
​
import (
  "github.com/golang/protobuf/protoc-gen-go/generator"
  "google.golang.org/protobuf/types/descriptorpb"
)
​
//Customize the protocol Gen go plug-in through -- go_ Out = plugins = netrpc to generate go code
​
type NetrpcPlugin struct {
  *generator.Generator
}
​
func init() {
  //Register with generator
  generator.RegisterPlugin(new(NetrpcPlugin))
}
​
func (p *NetrpcPlugin) Name() string {
  return "netrpc"
}
func (p *NetrpcPlugin) Init(g *generator.Generator) {
  p.Generator = g
}
​
func (p *NetrpcPlugin) GenerateImports(file *generator.FileDescriptor) {
  if len(file.Service) > 0 {
  p.genImportCode(file)
  }
}
​
func (p *NetrpcPlugin) Generate(file *generator.FileDescriptor) {
  for _,  svc := range file. Service {// process the service defined in the proto file
  p.genServiceCode(svc)
  }
}
​
func (p *NetrpcPlugin) genImportCode(file *generator.FileDescriptor) {
  p.P("// TODO: import code")
}
​
func (p *NetrpcPlugin) genServiceCode(svc *descriptorpb.ServiceDescriptorProto) {
  p.P("// TODO: service code, Name = " + svc.GetName())
}
​

Compile the custom plug-in to generate the executable file protocol Gen go netrpc

go build -o protoc-gen-go-netrpc
CP protocol Gen go netrpc / home / CSX / go / bin # copy the generated executable file to the gopath / bin directory

Use a custom executable to handle hello proto

protoc --go-netrpc_out=plugins=netrpc:. hello.proto

Compiling protobuf using the go micro plug-in

This command will generate a protocol Gen micro (. Exe) in the bin directory. The protocol compiler uses the protocol Gen micro plug-in to Convert proto file into micro code style file

go get github.com/micro/protoc-gen-micro/v2

2. Compilation Proto file will generate greeter pb. go, greeter. pb. mico. go

protoc --proto_path=. --micro_out=. --go_out=. proto/greeter.proto 

Compiling protobuf with grpc gateway framework

The two commands will generate protocol Gen go and protocol Gen grpc gateway in the $gopath / bin directory

$gopath defaults to Cd ~ / go

go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway

Protocol Gen go uses the new warehouse

github. COM / golang / protobuf is ready for cancellation

Use new library

google.golang.org/protobuf

Install go plug-in

The compiler plugin protoc-gen-go will be installed in $GOBIN, defaulting to $GOPATH/bin

go install google.golang.org/protobuf/cmd/[email protected]
protoc --go_out=. hello.proto

Grpc usage

The new protocol Gen go command no longer supports the plugin parameter. You need to download the protocol Gen go grpc command again

go install google.golang.org/grpc/cmd/[email protected]
protoc --go_ out=.  hello. Proto # compiles message to get variables
protoc --go-grpc_ out=.  hello. Proto # compiles other data and relies on the message variable

Two files need to be generated, hello pb. go, hello_ grpc. pb. go. Merge into one statement

protoc –go_out=. –go-grpc_out=. hello.proto

Protobuf3 syntax

www.cnblogs.com/tohxyblog/p/897476…

This work adoptsCC agreement, reprint must indicate the author and the link to this article