Implementation example of go operation etcd

Time:2021-9-26

etcdIt is an open-source, distributed key value pair data storage system, which provides shared configuration, service registration and discovery. This paper mainly introduces the installation and use of etcd.

Etcdetcd introduction

etcdIt is an open source and highly available distributed key value storage system developed with go language, which can be used to configure sharing and service registration and discovery.

Similar projects include zookeeper and consumer.

Etcd has the following characteristics:

  • Full replication: each node in the cluster can use a full archive
  • High availability: etcd can be used to avoid single point of failure of hardware or network problems
  • Consistency: each read returns the latest write across multiple hosts
  • Simple: includes a well-defined, user oriented API (grpc)
  • Security: automated TLS with optional client certificate authentication is implemented
  • Fast: base speed of 10000 writes per second
  • Reliable: the raft algorithm is used to achieve a highly consistent and highly available service storage directory

Etcd application scenario

Service discovery

Service discovery is also one of the most common problems in distributed systems, that is, how to find each other and establish connections for processes or services in the same distributed cluster. In essence, service discovery is to find out whether any process in the cluster is listening to UDP or TCP ports, and can find and connect by name.

Implementation example of go operation etcd

Configuration center

Put some configuration information on etcd for centralized management.

The usage of such scenarios is usually as follows: when the application starts, it actively obtains the configuration information from etcd. At the same time, it registers a watcher on the etcd node and waits. In the future, etcd will notify the subscriber in real time every time the configuration is updated, so as to obtain the latest configuration information.

Distributed lock

Because etcd uses the raft algorithm to maintain strong data consistency, the values stored in the cluster in a certain operation must be globally consistent, so it is easy to implement distributed locking. Lock service can be used in two ways: one is to keep exclusive, and the other is to control timing.

Keep exclusive, that is, only one user who obtains the lock can get it in the end. Etcd provides a CAS for distributed atomic locking(compareandswap)API. By settingprevexistValue, which can ensure that when multiple nodes create a directory at the same time, only one is successful. The successful user can be regarded as having obtained the lock.

Control timing, that is, all users who want to obtain locks will be scheduled to execute, but the order of obtaining locks is globally unique and determines the execution order. Etcd also provides a set of APIs (automatically creating ordered keys) for this purpose. When creating values for a directory, it is specified aspostAction, so etcd will automatically generate a current maximum value in the directory as the key and store the new value (client number). You can also use the API to list all the key values in the current directory in order. At this time, the value of these keys is the timing of the client, and the value stored in these keys can be the number representing the client.

Implementation example of go operation etcd

Why use etcd instead of zookeeper?

Zookeeper can realize these functions realized by etcd. So why use etcd instead of zookeeper?

Why not choose zookeeper?

  • Deployment and maintenance are complex, and its use is complexpaxosThe strong consistency algorithm is complex and difficult to understand. The official only providedjavaandcTwo language interface.
  • usejavaWriting introduces a large number of dependencies. The operation and maintenance personnel are troublesome.
  • In recent years, the development is slow, not as good asetcdandconsulWaiting for a rising star.

Why etcd?

  • Simple. Using go language to write, the deployment is simple; Support HTTP / JSON API, easy to use; The raft algorithm is used to ensure strong consistency and make it easy for users to understand.
  • Etcd default data is persisted as soon as it is updated.
  • Etcd supports SSL client security authentication.

Finally, as a young project, etcd is in high-speed iteration and development, which is both an advantage and a disadvantage. The advantage is that it has unlimited possibilities in the future, but the disadvantage is that it can not be tested by large projects for a long time. However, at presentcoreoskubernetesandcloudfoundryAnd other well-known projects have been used in the production environmentetcdSo in general, etcd is worth trying.

Etcd cluster

As a highly available key value storage system, etcd is designed for clustering. Because the raft algorithm requires the majority of nodes to vote when making decisions, etcd generally recommends an odd number of nodes in the deployment cluster, and the recommended number is 3, 5 or 7 nodes to form a cluster.

An example of building a 3-node cluster:

Specify cluster members in each etcd node. In order to distinguish different clusters, it is best to configure a unique token at the same time.

The following is the cluster information defined in advance, wheren1n2andn3Represents three different etcd nodes.

?
1
2
3
token=token-01
cluster_state=new
cluster=n1=http://10.240.0.17:2380,n2=http://10.240.0.18:2380,n3=http://10.240.0.19:2380

stayn1Execute the following command on this machine to start etcd:

?
1
2
3
4
5
etcd --data-dir=data.etcd --name n1 \
    --initial-advertise-peer-urls http://10.240.0.17:2380 --listen-peer-urls http://10.240.0.17:2380 \
    --advertise-client-urls http://10.240.0.17:2379 --listen-client-urls http://10.240.0.17:2379 \
    --initial-cluster ${cluster} \
    --initial-cluster-state ${cluster_state} --initial-cluster-token ${token}

stayn2Start etcd on this machine by executing the following command:

?
1
2
3
4
5
etcd --data-dir=data.etcd --name n2 \
    --initial-advertise-peer-urls http://10.240.0.18:2380 --listen-peer-urls http://10.240.0.18:2380 \
    --advertise-client-urls http://10.240.0.18:2379 --listen-client-urls http://10.240.0.18:2379 \
    --initial-cluster ${cluster} \
    --initial-cluster-state ${cluster_state} --initial-cluster-token ${token}

stayn3Start etcd on this machine by executing the following command:

?
1
2
3
4
5
etcd --data-dir=data.etcd --name n3 \
    --initial-advertise-peer-urls http://10.240.0.19:2380 --listen-peer-urls http://10.240.0.19:2380 \
    --advertise-client-urls http://10.240.0.19:2379 --listen-client-urls http://10.240.0.19:2379 \
    --initial-cluster ${cluster} \
    --initial-cluster-state ${cluster_state} --initial-cluster-token ${token}

The etcd official website provides an etcd storage address that can be accessed from the public network. You can get the directory of etcd service through the following command and use it as-discoveryParameter usage.

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
curl https://discovery.etcd.io/new?size=3
https://discovery.etcd.io/a81b5818e67a6ea83e9d4daea5ecbc92
 
# grab this token
token=token-01
cluster_state=new
discovery=https://discovery.etcd.io/a81b5818e67a6ea83e9d4daea5ecbc92
 
 
etcd --data-dir=data.etcd --name n1 \
    --initial-advertise-peer-urls http://10.240.0.17:2380 --listen-peer-urls http://10.240.0.17:2380 \
    --advertise-client-urls http://10.240.0.17:2379 --listen-client-urls http://10.240.0.17:2379 \
    --discovery ${discovery} \
    --initial-cluster-state ${cluster_state} --initial-cluster-token ${token}
 
 
etcd --data-dir=data.etcd --name n2 \
    --initial-advertise-peer-urls http://10.240.0.18:2380 --listen-peer-urls http://10.240.0.18:2380 \
    --advertise-client-urls http://10.240.0.18:2379 --listen-client-urls http://10.240.0.18:2379 \
    --discovery ${discovery} \
    --initial-cluster-state ${cluster_state} --initial-cluster-token ${token}
 
 
etcd --data-dir=data.etcd --name n3 \
    --initial-advertise-peer-urls http://10.240.0.19:2380 --listen-peer-urls http://10.240.0.19:2380 \
    --advertise-client-urls http://10.240.0.19:2379 --listen-client-urls http:/10.240.0.19:2379 \
    --discovery ${discovery} \
    --initial-cluster-state ${cluster_state} --initial-cluster-token ${token}

At this point, the etcd cluster is set up and can be usedetcdctlTo connect etcd.

?
1
2
3
4
5
6
7
export etcdctl_api=3
host_1=10.240.0.17
host_2=10.240.0.18
host_3=10.240.0.19
endpoints=$host_1:2379,$host_2:2379,$host_3:2379
 
etcdctl --endpoints=$endpoints member lis

Go language operation etcd

Here, the official package is used to connect etcd and perform related operations.

install

?
1
go get go.etcd.io/etcd/clientv3

Put and get operations

putCommand is used to set key value pair data,getThe command is used to get the value according to the key.

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
package main
 
import (
    "context"
    "fmt"
    "time"
 
    "go.etcd.io/etcd/clientv3"
)
 
// etcd client put/get demo
// use etcd/clientv3
 
func main() {
    cli, err := clientv3.new(clientv3.config{
        endpoints:   []string{"127.0.0.1:2379"},
        dialtimeout: 5 * time.second,
    })
    if err != nil {
        // handle error!
        fmt.printf("connect to etcd failed, err:%v\n", err)
        return
    }
    fmt.println("connect to etcd success")
    defer cli.close()
    // put
    ctx, cancel := context.withtimeout(context.background(), time.second)
    _, err = cli.put(ctx, "q1mi", "dsb")
    cancel()
    if err != nil {
        fmt.printf("put to etcd failed, err:%v\n", err)
        return
    }
    // get
    ctx, cancel = context.withtimeout(context.background(), time.second)
    resp, err := cli.get(ctx, "q1mi")
    cancel()
    if err != nil {
        fmt.printf("get from etcd failed, err:%v\n", err)
        return
    }
    for _, ev := range resp.kvs {
        fmt.printf("%s:%s\n", ev.key, ev.value)
    }
}

Watch operation

watchUsed to get notifications of future changes.

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
package main
 
import (
    "context"
    "fmt"
    "time"
 
    "go.etcd.io/etcd/clientv3"
)
 
// watch demo
 
func main() {
    cli, err := clientv3.new(clientv3.config{
        endpoints:   []string{"127.0.0.1:2379"},
        dialtimeout: 5 * time.second,
    })
    if err != nil {
        fmt.printf("connect to etcd failed, err:%v\n", err)
        return
    }
    fmt.println("connect to etcd success")
    defer cli.close()
    // watch key:q1mi change
    rch := cli.watch(context.background(), "q1mi") // <-chan watchresponse
    for wresp := range rch {
        for _, ev := range wresp.events {
            fmt.printf("type: %s key:%s value:%s\n", ev.type, ev.kv.key, ev.kv.value)
        }
    }
}

Save the above code for compilation and execution, and the program will wait in etcdq1miChanges in this key.

For example, we open the terminal and execute the following commands to modify, delete and setq1miThis key.

?
1
2
3
4
5
6
7
8
etcd> etcdctl.exe --endpoints=http://127.0.0.1:2379 put q1mi "dsb2"
ok
 
etcd> etcdctl.exe --endpoints=http://127.0.0.1:2379 del q1mi
1
 
etcd> etcdctl.exe --endpoints=http://127.0.0.1:2379 put q1mi "dsb3"
ok

The above procedures can receive the following notification.

?
1
2
3
4
5
watch>watch.exe
connect to etcd success
type: put key:q1mi value:dsb2
type: delete key:q1mi value:
type: put key:q1mi value:dsb3

Lease lease

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
package main
 
import (
    "fmt"
    "time"
)
 
// etcd lease
 
import (
    "context"
    "log"
 
    "go.etcd.io/etcd/clientv3"
)
 
func main() {
    cli, err := clientv3.new(clientv3.config{
        endpoints:   []string{"127.0.0.1:2379"},
        dialtimeout: time.second * 5,
    })
    if err != nil {
        log.fatal(err)
    }
    fmt.println("connect to etcd success.")
    defer cli.close()
 
    // Create a5Second lease
    resp, err := cli.grant(context.todo(), 5)
    if err != nil {
        log.fatal(err)
    }
 
    // 5Seconds later,/nazha/ The key will be removed
    _, err = cli.put(context.todo(), "/nazha/", "dsb", clientv3.withlease(resp.id))
    if err != nil {
        log.fatal(err)
    }
}

keepalive

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
package main
 
import (
    "context"
    "fmt"
    "log"
    "time"
 
    "go.etcd.io/etcd/clientv3"
)
 
// etcd keepalive
 
func main() {
    cli, err := clientv3.new(clientv3.config{
        endpoints:   []string{"127.0.0.1:2379"},
        dialtimeout: time.second * 5,
    })
    if err != nil {
        log.fatal(err)
    }
    fmt.println("connect to etcd success.")
    defer cli.close()
 
    resp, err := cli.grant(context.todo(), 5)
    if err != nil {
        log.fatal(err)
    }
 
    _, err = cli.put(context.todo(), "/nazha/", "dsb", clientv3.withlease(resp.id))
    if err != nil {
        log.fatal(err)
    }
 
    // the key 'foo' will be kept forever
    ch, kaerr := cli.keepalive(context.todo(), resp.id)
    if kaerr != nil {
        log.fatal(kaerr)
    }
    for {
        ka := <-ch
        fmt.println("ttl:", ka.ttl)
    }
}

Implementation of distributed lock based on etcd

go.etcd.io/etcd/clientv3/concurrencyImplement concurrent operations on etcd, such as distributed locks, barriers and elections.

Import the package:

?
1
import "go.etcd.io/etcd/clientv3/concurrency"

Example of distributed lock based on etcd implementation:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
cli, err := clientv3.new(clientv3.config{endpoints: endpoints})
if err != nil {
    log.fatal(err)
}
defer cli.close()
 
// Create two separate sessions to demonstrate lock contention
s1, err := concurrency.newsession(cli)
if err != nil {
    log.fatal(err)
}
defer s1.close()
m1 := concurrency.newmutex(s1, "/my-lock/")
 
s2, err := concurrency.newsession(cli)
if err != nil {
    log.fatal(err)
}
defer s2.close()
m2 := concurrency.newmutex(s2, "/my-lock/")
 
// Session S1 acquire lock
if err := m1.lock(context.todo()); err != nil {
    log.fatal(err)
}
fmt.println("acquired lock for s1")
 
m2locked := make(chan struct{})
go func() {
    defer close(m2locked)
    // Wait until session S1 is released/my-lock/Lock of
    if err := m2.lock(context.todo()); err != nil {
        log.fatal(err)
    }
}()
 
if err := m1.unlock(context.todo()); err != nil {
    log.fatal(err)
}
fmt.println("released lock for s1")
 
<-m2locked
fmt.println("acquired lock for s2")

Output:

acquired lock for s1
released lock for s1
acquired lock for s2

View documentation to learn more

Other operations

Please check for other operationsEtcd / clientv3 official document

Reference link:

https://etcd.io/docs/v3.3.12/demo/

https://www.infoq.cn/article/etcd-interpretation-application-scenario-implement-principle/  Code changes the world, down-to-earth, pythonGolang。

This is the end of this article about the implementation example of go operation etcd. For more information about go operation etcd, please search the previous articles of developpaper or continue to browse the relevant articles below. I hope you will support developpaper in the future!

Recommended Today

Seven Python code review tools recommended

althoughPythonLanguage is one of the most flexible development languages at present, but developers often abuse its flexibility and even violate relevant standards. So PythoncodeThe following common quality problems often occur: Some unused modules have been imported Function is missing arguments in various calls The appropriate format indentation is missing Missing appropriate spaces before and after […]