Jsonrpc of go daily library

Time:2020-5-31

brief introduction

In the last article we introduced the go standard librarynet/rpcUsage of. By default,rpcLibrary internal usegobFormat to transmit data. We copygobThe codec of implements ajsonFormatted. Actually standard librarynet/rpc/jsonrcpAlready implemented in. This article is a supplement to the previous article.

Quick use

The standard library does not need to be installed.

First, the server usesnet/rpc/jsonrpcAfter that, we don’t have to write it ourselvesjsonCodec for:

package main

import (
  "log"
  "net"
  "net/rpc"
  "net/rpc/jsonrpc"
)

type Args struct {
  A, B int
}

type Arith int

func (t *Arith) Multiply(args *Args, reply *int) error {
  *reply = args.A * args.B
  return nil
}

func main() {
  l, err := net.Listen("tcp", ":1234")
  if err != nil {
    log.Fatal("listen error:", err)
  }

  arith := new(Arith)
  rpc.Register(arith)

  for {
    conn, err := l.Accept()
    if err != nil {
      log.Fatal("accept error:", err)
    }

    //Pay attention to this line
    go rpc.ServeCodec(jsonrpc.NewServerCodec(conn))
  }
}

Direct calljsonrpc.NewServerCodec(conn)Create a server-sidecodecClients are similar:

func main() {
  conn, err := net.Dial("tcp", ":1234")
  if err != nil {
    log.Fatal("dial error:", err)
  }

  //Here, here
  client := rpc.NewClientWithCodec(jsonrpc.NewClientCodec(conn))

  args := &Args{7, 8}
  var reply int
  err = client.Call("Arith.Multiply", args, &reply)
  if err != nil {
    log.Fatal("Multiply error:", err)
  }
  fmt.Printf("Multiply: %d*%d=%d\n", args.A, args.B, reply)
}

Run the server program first:

$ go run main.go

Then run the client program in a new console:

$ go run client.go
Multiply: 7*8=56

The following code basically usesjsonrpcAll the programs of should be written:

conn, err := net.Dial("tcp", ":1234")
if err != nil {
  log.Fatal("dial error:", err)
}

client := rpc.NewClientWithCodec(jsonrpc.NewClientCodec(conn))

thereforejsonrpcFor convenience, aDialmethod. useDialSimplify the client program above:

func main() {
  client, err := jsonrpc.Dial("tcp", ":1234")
  if err != nil {
    log.Fatal("dial error:", err)
  }

  args := &Args{7, 8}
  var reply int
  err = client.Call("Arith.Multiply", args, &reply)
  if err != nil {
    log.Fatal("Multiply error:", err)
  }
  fmt.Printf("Multiply: %d*%d=%d\n", args.A, args.B, reply)
}

The effect is the same.

Json-rpc standard

Json-rpc 1.0 standard was released in 2005, and after several years of evolution, version 2.0 was released in 2010. The content of json-rpc standard can be found in https://www.jsonrpc.org/specification see. Go standard librarynet/rpc/jsonrpcImplemented version 1.0. The implementation of version 2.0 can be found inpkg.go.devSearch upjson-rpc+2.0。 This article is based on version 1.0.

Json-rpc transfers a single object, serialized to JSON format. The request object contains the following three properties:

  • method: the method called by the request;
  • params: an array represents the parameters passed to the method;
  • id: request ID. ID can be of any type. When receiving a response, judge which request it corresponds to according to this property.

The response object contains the following three properties:

  • result: object returned by method, iferrorWhen not empty, the property must benull
  • error: indicates whether the call is in error;
  • id: the ID of the corresponding request.

The standard also defines a notification type, in addition toidProperty isnullIn addition, the properties of the notification object are exactly the same as the request object.

callclient.Call("echo", "Hello JSON-RPC", &reply)When:

Request: {"method": "echo", "params": ["Hello json-rpc"], "Id": 1}
Response: {"result": "Hello json-rpc", "error": null, "Id": 1}

Simple load balancing using zookeeper

Let’s usezookeeperImplement a simpleclientLoad balancing on the side.zookeeperAll servers that can provide services are recorded in, and each time the client requests, one is randomly selected.In our example, the request must be stateless。 First of all, we need to modify the server program to extract the listening address. Through theflagappoint:

package main

import (
  "flag"
  "log"
  "net"
  "net/rpc"
  "net/rpc/jsonrpc"
)

var (
  addr *string
)

type Args struct {
  A, B int
}

type Arith int

func (t *Arith) Multiply(args *Args, reply *int) error {
  *reply = args.A * args.B
  return nil
}

func init() {
  addr = flag.String("addr", ":1111", "addr to listen")
}

func main() {
  flag.Parse()

  l, err := net.Listen("tcp", *addr)
  if err != nil {
    log.Fatal("listen error:", err)
  }

  arith := new(Arith)
  rpc.Register(arith)

  for {
    conn, err := l.Accept()
    if err != nil {
      log.Fatal("accept error:", err)
    }

    go rpc.ServeCodec(jsonrpc.NewServerCodec(conn))
  }
}

About which servers are available, we store them inzookeeperMedium.

First, start azookeeperThe procedure of. You can download directly running Windows programs on the Apache zookeeper website. After downloading, unzip theconfTemplate configuration in folderzoo_sample.cfgMake a copy and change the file name tozoo.cfg。 Open in editorzoo.cfg, willdataDirChange to an existing directory, or create a new directory. I am herebinCreated adataDirectory, and then setdataDir=../data。 Switch tobinExecute under directoryzkServer.batzookeeperThe program runs. usezkClient.batConnect thiszookeeper, add a node, and set the data:

$ create /rpcserver
$ set /rpcserver 127.0.0.1:1111,127.0.0.1:1112,127.0.0.1:1113

We use,Separate multiple server addresses.

When you’re ready, you’re ready to write the client code. We implement a proxy class to monitorzookeeperData changes based onzookeeperThe new address creates a connection to the server, deletes the old connection, and randomly forwards the call request to a server for processing:

type Proxy struct {
  zookeeper     string
  clients       map[string]*rpc.Client
  events        <-chan zk.Event
  zookeeperConn *zk.Conn
  mutex         sync.Mutex
}

func NewProxy(addr string) *Proxy {
  return &Proxy{
    zookeeper: addr,
    clients:   make(map[string]*rpc.Client),
  }
}

Here we usego-zookeeperThis library requires additional installation:

$ go get github.com/samuel/go-zookeeper/zk

When the program starts, the proxy objectzookeeperTo get the server address and create a connection:

func (p *Proxy) Connect() {
  c, _, err := zk.Connect([]string{p.zookeeper}, time.Second) //*10)
  if err != nil {
    panic(err)
  }

  data, _, event, err := c.GetW("/rpcserver")
  if err != nil {
    panic(err)
  }

  p.events = event
  p.zookeeperConn = c

  p.CreateClients(string(data))
}

func (p *Proxy) CreateClients(server string) {
  p.mutex.Lock()
  defer p.mutex.Unlock()

  addrs := strings.Split(server, ",")
  allAddr := make(map[string]struct{})
  for _, addr := range addrs {
    allAddr[addr] = struct{}{}
    if _, exist := p.clients[addr]; exist {
      continue
    }

    client, err := jsonrpc.Dial("tcp", addr)
    if err != nil {
      log.Println("jsonrpc Dial error:", err)
      continue
    }

    p.clients[addr] = client
    log.Println("new addr:", addr)
  }

  for addr := range p.clients {
    if _, exist := allAddr[addr]; !exist {
      //Address not in zookeeper, delete corresponding connection
      oldClient.Close()
      delete(p.clients, addr)

      log.Println("delete addr", addr)
    }
  }
}

At the same time, we need to monitorzookeeperWhen adding or deleting a server address,ProxyTo update the connection in a timely manner:

func (p *Proxy) Run() {
  for {
    select {
    case event := <-p.events:
      if event.Type == zk.EventNodeDataChanged {
        data, _, err := p.zookeeperConn.Get("/rpcserver")
        if err != nil {
          log.Println("get zookeeper data failed:", err)
          continue
        }

        p.CreateClients(string(data))
      }
    }
  }
}

Client principal program useProxyThe structure is very convenient:

package main

import (
  "flag"
  "fmt"
  "math/rand"
)

var (
  zookeeperAddr *string
)

func init() {
  zookeeperAddr = flag.String("addr", ":2181", "zookeeper address")
}

type Args struct {
  A, B int
}

func main() {
  flag.Parse()

  fmt.Println(*zookeeperAddr)
  p := NewProxy(*zookeeperAddr)
  p.Connect()

  go p.Run()

  for i := 0; i < 10; i++ {
    var reply int
    args := &Args{rand.Intn(1000), rand.Intn(1000)}
    p.Call("Arith.Multiply", args, &reply)
    fmt.Printf("%d*%d=%d\n", args.A, args.B, reply)
  }

  //Data in zookeeper can be modified during sleep
  time.Sleep(1 * time.Minute)

  //Use new address for random
  for i := 0; i < 100; i++ {
    var reply int
    args := &Args{rand.Intn(1000), rand.Intn(1000)}
    p.Call("Arith.Multiply", args, &reply)
    fmt.Printf("%d*%d=%d\n", args.A, args.B, reply)
  }
}

Create a proxy object and listen in a new goroutinezookeeperevent. And then throughProxyOfCallCall the remote server’s method:

func (p *Proxy) Call(method string, args interface{}, reply interface{}) error {
  var client *rpc.Client
  var addr string
  idx := rand.Int31n(int32(len(p.clients)))
  var i int32
  p.mutex.Lock()
  for a, c := range p.clients {
    if i == idx {
      client = c
      addr = a
      break
    }
    i++
  }
  p.mutex.Unlock()

  fmt.Println("use", addr)
  return client.Call(method, args, reply)
}

First, we need to start three server programs and listen to ports 1111, 1112 and 1113 respectively. We need three consoles:

Console 1:

$ go run main.go -addr :1111

Console 2:

$ go run main.go -addr :1112

Console 3:

$ go run main.go -addr :1113

The client starts in a new console, specifyingzookeeperAddress:

$ go run . -addr=127.0.0.1:2181

In the output, we can see how the servers are randomly selected.

We can try to run the client program from a server addresszookeeperDelete from. I purposely added a one minute delay to the program. staysleepIn the process, throughzkClient.cmdtake127.0.0.1:1113This address is fromzookeeperDelete from:

$ set /rpcserver 127.0.0.1:1111,127.0.0.1:1112

Console output:

$ 2020/05/10 23:47:47 delete addr 127.0.0.1:1113

And subsequent requests will not be sent to127.0.0.1:1113This is the server.

In fact, in the actual project,ProxyIt is usually a stand-alone server, rather than on the client side. The above example is just for convenience.

summary

RPC bottom layer can use a variety of protocols to transfer data, JSON / XML / protobuf can. Suggestions interested in RPCrpcxThis library, https://github.com/smallnest/rpcx 。 Very powerful!

If you find a fun and easy-to-use go language library, you are welcome to submit the issue to GitHub, the daily go library

reference resources

  1. jsonrpc GitHub:https://golang.org/pkg/net/rpc/jsonrpc/
  2. Go daily GitHub: https://github.com/darjun/go-daily-lib

I

My blog: https://darjun.github.io

Welcome to my WeChat official account, GoUpUp, learn together and make progress together.

Jsonrpc of go daily library

Recommended Today

The way of nonlinear optimization

Mathematical knowledge 1、 Nonlinear functionLinear function is another name of a function of first degree, then nonlinear function means that the function image is not a function of a straight line.Nonlinear functions include exponential function, power function, logarithmic function, polynomial function and so on. 2、 Taylor expansion1. Taylor formula:Taylor’s formula is to add a_ The […]