[go] grpc + protobuf learning notes

Time:2022-4-16

RPC

What is RPC

RPCRefers to remote procedure call. Allows a program running on one computer to call a subroutine of another computer, and the call is as simple as calling a local function.

For example, if application C running on server a wants to call application d running on server B, it can be easily called through RPC, which is as simple as calling a local function.

Principle of RPC

The picture comes from a worm in the up main bend of station BTo you who love grpc! Where is the magic code? Go actual combat #9: grpc
Very good teaching video, I recommend you to watch!
[go] grpc + protobuf learning notes

The calling process of RPC service is as follows:

1. Clientclientcallclient stub, this is a local call

2、client stubNext, the clientclientThe incoming message is encoded into a unified format

3. Then byrpc clientSend message torpc server

4、rpc serverReceived byrpc clientNews from

5. AfterserverThe received message will be decoded

6. Then find the corresponding interface through routing for processing

7. Finally, the result is returned to the clientclient

stubThe Chinese name is pile code, which is responsible for processing the upper applicationrpcThe data encoding and decoding, sending and receiving requests and other dirty work in the call, while the client calls the RPC service as if it were called locallystubLike a sub function of, you don’t need to pay too much attention to other processes.

IDL

IDL(interface definition language).

IDL describes the interface in a neutral way, so that objects running on different platforms and programs written in different languages can communicate with each other. For example, one component is written in C + + and the other component is written in go. IDL can provide common message intermediates for these two components.

In RPC,ProtobufIs a common IDL.

Common RPC frameworks

  • Hessian
  • Montan
  • rpcx
  • gRPC
  • Thrift
  • Dubbo
  • Dubbox
  • Spring Cloud

Protobuf

[go] grpc + protobuf learning notes

introduce

Protocol BuffersIt is a language independent, platform independent and extensible method of serializing structured data. It is commonly used in communication protocol, data storage and so on.

Compared with JSON and XML, it is smaller and faster, so it is more favored by developers.

Pb is short for protobuf.

install

Protobuf compiler

1. Download protobuf and compile and install

wget https://github.com/protocolbuffers/protobuf/releases/download/v3.20.0-rc1/protobuf-all-3.20.0-rc-1.zip

unzip protobuf-all-3.20.0-rc-1.zip

cd protobuf-3.20.0-rc-1/

./configure

make

make install

2. Passgo getinstall

go get google.golang.org/protobuf

Verify that the installation was successful

protoc --version
Libprotocol 3.20.0-rc1 # output version number to prove successful installation

Protobuf compiler plug-in: Protocol Gen go

protoc-gen-goEnhanced the protocol compiler to know how to.protoFile generates go code.

protoc-gen-goIncluded ingoogle.golang.org/protobufPackage, can be installed directlygoogle.golang.org/protobufThe package is in place in one step without separate installation.

Separate installationprotoc-gen-goCan passgo getTo install, go is requiredv1.16Or later

go get -u github.com/golang/protobuf/protoc-gen-go

Protobuf runtime library

ShouldprotobufThe module contains a set of go packages that provide APIs for the runtime of protobufs in go. This provides a set ofDefine what the message isAnd in various formats (for example,wireJSONandtext)The ability to serialize messages.

The runtime library is included ingoogle.golang.org/protobufIn,google.golang.org/protobufIt consists of two parts:protocol-gen-goAnd this runtime library.

adoptgo getInstallation:

go get google.golang.org/protobuf

Protobuf file syntax

Simple example of protobuf data structure definition:

syntax = "proto3";

option go_package = "../stub";

service HelloService {
  rpc SayHello (HelloRequest) returns (HelloResponse);
}

message HelloRequest {
  string name = 1;
}

message HelloResponse {
  string message = 1;
}

explain

1. Protobuf Syntax version declaration

syntax = "proto3";

2. Declare the package name as go when generating source code

option go_package = "../stub";

Bao Mingzhong.In the generated go package name, it will be converted to_

If there is no such declaration, an error will be reported during compilation:

Missing 'go_package' option in "test.proto"

3. RPC service definition

service HelloService {
  rpc SayHello (HelloRequest) returns (HelloResponse);
}

4. Message definition

Each field in the message contains three attributes: type, field name and field number.

message HelloRequest {
  string name = 1;
}

message HelloResponse {
  string message = 1;
}

The above is a brief introduction to protobuf syntax. For more detailed syntax, please refer to the official document:Protocel Buffers

In addition, the official also explained some specifications of protobuf:Specification Guide

data type

.proto Type C++ Type Java Type Go Type PHP Type
double double double float64 float
float float float float32 float
int32 int32 int int32 integer
int64 int64 long int64 integer/string
uint32 uint32 int uint32 integer
uint64 uint64 long uint64 integer/string
sint32 int32 int int32 integer
sint64 int64 long int64 integer/string
fixed32 uint32 int uint32 integer
fixed64 uint64 long uint64 integer/string
sfixed32 int32 int int32 integer
sfixed64 int64 long int64 integer/string
bool bool boolean bool boolean
string string String string string
bytes string ByteString []byte string

Protocol command line

View the help of the protocol command line

protoc --help

Common parameters

-I[PATH], --peoro_path=[PATH]
#Specify the directory of import search, and multiple directories can be specified; The directories are searched in order. If not, the current working directory is used. If not found in these directories, the -- descriptor in the descriptor will be checked_ set_ To find the required prototype file.

--go_out=[PATH]
#Generate the code of go language and put it into the path directory

--go-opt
#Specify parameters, such as -- go_ opt=paths=source_ Relative indicates that the generated file output uses a relative path.

More detailedprotocSyntax can be found in official documents:https://developers.google.com/protocol-buffers/docs/reference/go-generated

gRPC

[go] grpc + protobuf learning notes

introduce

Grpc is aHigh performanceOpen SourceandGeneric RPC framework, designed for mobile and HTTP 2.0

Official documents:https://grpc.io/docs/

Grpc Architecture Overview

[go] grpc + protobuf learning notes

Multilingual support

  • C++
  • C#
  • Dart
  • Go
  • Java
  • Node.js
  • Objective-C
  • PHP
  • Python
  • Ruby

characteristic

1. Based on HTTP 2.0

2. Serialization with protobuf

3. Client and server are based on the same IDL

4. Good support of mobile network

5. Multilingual support

install

adoptgo getInstallation:

go get google.golang.org/grpc

Four communication modes of grpc

Unary unary mode

To put it bluntly, it is the request response mode, one request and one response.

Client side streaming

The client constantly requests the server in the form of a stream, and the server only returns a response after the request is completed.

Server side streaming

The client only requests the server once, and then the server continuously transmits the response to the server in the form of similar flow.

Bidirectional streaming

Both client and server request and respond to each other in the form of flow.

Preparation of protobuf file in grpc

In grpc, the general protobuf file mainly writes two parts of information, one is message and the other is service.

Code pile generation command

use--go-grpc_out

--go-grpc_out=[PATH]
#Generate the pile code of grpc and put it into the path directory

You can do the same,protoThe file specifiesRPCServices,protoc-gen-goCan generate andgrpcCompatible code, we just need toplugins=grpcParameters passed to--go_out, you can achieve this goal. However, this method has been deprecated in the new version.

--go_out=plugins=grpc:[PATH]
#Plugins = plugin1 + plugin2: Specifies the list of child plug-ins to load

example:

protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative route.proto
# --go_ Out = [path]: generate code files compatible with golang