Sentinel go integrates Nacos to realize external dynamic data source

Time:2021-5-12

Sentinel go integrates Nacos to realize external dynamic data source

Introduction:In 2020, sentinel will launch the native version of goSentinel-GolangAnd continue to make breakthroughs in the cloud native field. This paper will explain how to integrate in sentinel golang with a caseNacosAs an external dynamic data source, the flow control rules are stored in Nacos, and the rules are dynamically updated in real time.

This paper is divided into two parts

  1. The sentinel flow control rules are defined in the code to achieve the current limiting effect.
  2. The sentinel flow control rules are defined in the configuration center of Nacos to achieve the effect of current limiting and dynamically update the rules in Nacos to achieve dynamic flow control.

The related background knowledge will be introduced in detail below.

1. Sentinel

With the popularity of micro services, the stability between services becomes more and more important. Sentinel takes the traffic as the breakthrough point, and protects the stability of the service from multiple dimensions such as traffic control, fuse degradation and system load protection.

Sentinel has the following characteristics:

  • Rich application scenariosSentinel has undertaken the core scenarios of Alibaba’s “double 11” traffic promotion in recent 10 years, such as seckill (i.e. burst traffic control within the range of system capacity), message peak shaving and valley filling, cluster traffic control, real-time fuse downstream unavailable applications, etc.
  • Complete real time monitoringSentinel also provides real-time monitoring function. You can see the second level data of a single machine accessing the application in the console, and even the summary operation of clusters with a scale of less than 500.
  • Extensive open source ecologySentinel provides out of the box integration modules with other open source frameworks / libraries, such as spring cloud, Dubbo and grpc. You only need to introduce the corresponding dependency and make simple configuration to quickly access sentinel.
  • Perfect SPI extension pointSentinel provides simple and perfect SPI extension interface. You can quickly customize the logic by implementing extended interfaces. For example, customizing rule management, adapting dynamic data source and so on.

1.1 Sentinel’s history

  • Sentinel was born in 2012, and its main function is to control the inlet flow.
  • From 2013 to 2017, sentinel developed rapidly within Alibaba group, becoming a basic technology module, covering all core scenarios. Sentinel has accumulated a lot of traffic consolidation scenarios and production practices.
  • In 2018, sentinel will open source and continue to evolve.
  • In 2019, sentinel will gradually explore the direction of multilingual expansion, and successively launch C + + native version and envoy cluster traffic control support.
  • In 2020, sentinel will launch the go native version, expecting to continue to make breakthroughs in the cloud native field.https://github.com/alibaba/sentinel-golang

2. Nacos

NacosIt is a platform for dynamic service discovery, configuration management and service management that is easier to build cloud native applications. Nacos is derived from configserver and diamond in Alibaba and is their open source implementation. Having experienced the test of the peak traffic of double 11 and the super large capacity of Alibaba economy, it has accumulated ten years of experience of Alibaba soft load team in this field, which has a good guarantee in stability and functionality.

Sentinel go integrates Nacos to realize external dynamic data source
(sentinel go integrated Nacos dynamic data source architecture)

At present, the current limiting and fusing strategies in sentinel are implemented based on rules. The purpose of providing dynamic data source extension is to load and update rule data through some configuration center middleware (such asnacos, etcd, confil, etc.) to achieve dynamic update.

3. Sentinel go current limiting demo

When Nacos is not integrated, rules are defined inside the code and no external data source is used.

3.1 installation

go get github.com/alibaba/sentinel-golang

3.2 demo sample

Using sentinel can be divided into the following steps:

  1. Configure and initialize sentinel
  2. Buried point (defined resource)
  3. Configuration rules
package main

import (
    "fmt"
    "log"
    "math/rand"
    "time"

    sentinel "github.com/alibaba/sentinel-golang/api"
    "github.com/alibaba/sentinel-golang/core/base"
    "github.com/alibaba/sentinel-golang/core/flow"
    "github.com/alibaba/sentinel-golang/util"
)

func main() {
    // We should initialize Sentinel first.
    err := sentinel.InitDefault()
    if err != nil {
        log.Fatalf("Unexpected error: %+v", err)
    }

    _, err = flow.LoadRules([]*flow.FlowRule{
        {
            Resource:        "some-test",
            MetricType:      flow.QPS,
            Count:           10,
            ControlBehavior: flow.Reject,
        },
    })
    if err != nil {
        log.Fatalf("Unexpected error: %+v", err)
        return
    }

    ch := make(chan struct{})

    for i := 0; i < 10; i++ {
        go func() {
            for {
                e, b := sentinel.Entry("some-test", sentinel.WithTrafficType(base.Inbound))
                if b != nil {
                    // Blocked. We could get the block reason from the BlockError.
                    time.Sleep(time.Duration(rand.Uint64()%10) * time.Millisecond)
                } else {
                    // Passed, wrap the logic here.
                    fmt.Println(util.CurrentTimeMillis(), "passed")
                    time.Sleep(time.Duration(rand.Uint64()%10) * time.Millisecond)

                    // Be sure the entry is exited finally.
                    e.Exit()
                }

            }
        }()
    }
    <-ch
}

Official expale:https://github.com/alibaba/sentinel-golang/tree/master/example

4. Sentinel go integrated Nacos

Sentinel go integrates Nacos to realize the function of external dynamic data source

4.1 deployment of Nacos

4.1.1 version selection

You can visit therelease notesandBlogThe current recommended stable version is 1.3.1.

4.1.2 environmental preparation

Nacos dependenceJavaEnvironment. If you are building and running Nacos from code, you also need to configure it for thisMavenEnvironment, please make sure that it is installed and used in the following version environment:

  1. 64 bit OS, supports Linux / Unix / MAC / windows, and Linux / Unix / MAC is recommended.
  2. 64 bit JDK 1.8+;download & to configure
  3. Maven 3.2.x+;download & to configure

4.1.3 download source code or installation package

You can get Nacos through source code and distribution package.

Download source code from GitHub

git clone https://github.com/alibaba/nacos.git
cd nacos/
mvn -Prelease-nacos -Dmaven.test.skip=true clean install -U  
ls -al distribution/target/
// change the $version to your actual path
cd distribution/target/nacos-server-$version/nacos/bin

How to download compressed package after compilation

You can choose fromLatest stable versiondownloadnacos-server-$version.zipBag.

Unzip Nacos server - $version.zip or tar - xvf Nacos server - $version.tar.gz
  cd nacos/bin

4.1.4 starting the server

Linux/Unix/Mac
Start command (stand alone stands for stand-alone mode, non cluster mode)
sh startup.sh -m standalone
If you are using the Ubuntu system, or if you run the script with an error prompt [[symbol not found], you can try to run it as follows:
bash startup.sh -m standalone

Windows
Start command:
cmd startup.cmd
Or double-click startup.cmd to run the file.

Deployment successful accesshttp://127.0.0.1:8848/nacos  
User name / password: Nacos / Nacos

4.2 sentinel current limiting configuration to Nacos

  1. Log in to Nacos Web
  2. In configuration management, create a new configuration
  3. Enter dataid, group (dataid, group can be customized when creating, dataid = flow, group = sentinel go created in this article)
  4. Paste the sample data source into the configuration content.

4.2.1 example of Nacos external data source

This example is a demo configuration for flow control. When the number of concurrent traffic is more than 100, it will be rejected directly.

Please refer to the configuration descriptionhttps://github.com/alibaba/sentinel-golang/wiki/ flow control

[
    {
        "resource": "some-test",
        "metricType": 1,
        "count": 100.0,
        "controlBehavior":0
    }
]

After creation, you can see the corresponding current limiting configuration in the Nacos configuration list.

Sentinel go integrates Nacos to realize external dynamic data source

4.3 Nacos data source integration

4.3.1 create project

  1. edition

    1. Sentinel golang version uses 0.6.0, Nacos SDK go uses 1.0.0
  2. go.mod
module sentinel-go-nacos-example

go 1.13

require (
    github.com/alibaba/sentinel-golang v0.6.0
    github.com/nacos-group/nacos-sdk-go v1.0.0
)
  1. main.go
package main

import (
    "fmt"
    "math/rand"
    "sync/atomic"
    "time"

    sentinel "github.com/alibaba/sentinel-golang/api"
    "github.com/alibaba/sentinel-golang/core/base"
    "github.com/alibaba/sentinel-golang/ext/datasource/nacos"
    "github.com/alibaba/sentinel-golang/util"
    "github.com/nacos-group/nacos-sdk-go/clients"

    "github.com/alibaba/sentinel-golang/ext/datasource"
    "github.com/nacos-group/nacos-sdk-go/common/constant"
)

type Counter struct {
    pass  *int64
    block *int64
    total *int64
}

func main() {
    //Flow counter, for the sake of more intuitive flow control log printing, has nothing to do with integrated Nacos data source.
    counter := Counter{pass: new(int64), block: new(int64), total: new(int64)}

    //Nacos server address
    sc := []constant.ServerConfig{
        {
            ContextPath: "/nacos",
            Port:        8848,
            IpAddr:      "127.0.0.1",
        },
    }
    //Nacos client related parameter configuration, specific configuration can be referred to https://github.com/nacos-group/nacos-sdk-go
    cc := constant.ClientConfig{
        TimeoutMs: 5000,
    }
    //Generate Nacos config client
    client, err := clients.CreateConfigClient(map[string]interface{}{
        "serverConfigs": sc,
        "clientConfig":  cc,
    })
    if err != nil {
        fmt.Printf("Fail to create client, err: %+v", err)
        return
    }
    //Register flow control rule handler
    h := datasource.NewFlowRulesHandler(datasource.FlowRuleJsonArrayParser)
    //Create nacosdatasource data source
    //Sentinel go corresponds to the group that creates the configuration file in Nacos
    //Flow corresponds to the dataid of the configuration file created in Nacos
    nds, err := nacos.NewNacosDataSource(client, "sentinel-go", "flow", h)
    if err != nil {
        fmt.Printf("Fail to create nacos data source client, err: %+v", err)
        return
    }
    //Nacos data source initialization
    err = nds.Initialize()
    if err != nil {
        fmt.Printf("Fail to initialize nacos data source client, err: %+v", err)
        return
    }
    //Start statistics
    go timerTask(&counter)

    //Simulated flow
    ch := make(chan struct{})
    for i := 0; i < 10; i++ {
        go func() {
            for {
                atomic.AddInt64(counter.total, 1)
                //Some test corresponds to the resource in the Nacos flow control configuration file
                e, b := sentinel.Entry("some-test", sentinel.WithTrafficType(base.Inbound))
                if b != nil {
                    atomic.AddInt64(counter.block, 1)
                    // Blocked. We could get the block reason from the BlockError.
                    time.Sleep(time.Duration(rand.Uint64()%10) * time.Millisecond)
                } else {
                    atomic.AddInt64(counter.pass, 1)
                    time.Sleep(time.Duration(rand.Uint64()%10) * time.Millisecond)

                    // Be sure the entry is exited finally.
                    e.Exit()
                }

            }
        }()
    }
    <-ch
}

//statistic print
func timerTask(counter *Counter) {
    fmt.Println("begin to statistic!!!")
    var (
        oldTotal, oldPass, oldBlock int64
    )
    for {
        time.Sleep(1 * time.Second)
        globalTotal := atomic.LoadInt64(counter.total)
        oneSecondTotal := globalTotal - oldTotal
        oldTotal = globalTotal

        globalPass := atomic.LoadInt64(counter.pass)
        oneSecondPass := globalPass - oldPass
        oldPass = globalPass

        globalBlock := atomic.LoadInt64(counter.block)
        oneSecondBlock := globalBlock - oldBlock
        oldBlock = globalBlock
        fmt.Println(util.CurrentTimeMillis()/1000, "total:", oneSecondTotal, " pass:", oneSecondPass, " block:", oneSecondBlock)
    }
}

4.3.2 operation results

Sentinel go integrates Nacos to realize external dynamic data source

4.3.3 dynamic update of current limiting configuration

During the start-up of the project, the flow control configuration parameters are modified in Nacos. Count from 100 to 400

Sentinel go integrates Nacos to realize external dynamic data source

You can see that the log of reloading rule is printed, and the flow control changes dynamically from 100 to 400

Sentinel go integrates Nacos to realize external dynamic data source

summary

When using Nacos as external dynamic data source in sentinel go, you only need to change the part of declaring rule and loading rule to read from Nacos data source.

In this paper, we only introduce the integration of flow control, fusing, warmup and hotspot parameters are the same, as long as you modify the configuration content on demand

Configuration content reference address:https://github.com/alibaba/sentinel-golang/wiki

Key code:

    h := datasource.NewFlowRulesHandler(datasource.FlowRulesJsonConverter)
    nds, err := nacos.NewNacosDataSource(client, "sentinel-go", "flow", h)
    if err != nil {
        fmt.Printf("Fail to create nacos data source client, err: %+v", err)
        return
    }
    err = nds.Initialize()
    if err != nil {
        fmt.Printf("Fail to initialize nacos data source client, err: %+v", err)
        return
    }

Related links

About the author

Zhang Binbin, GitHub account: sanxun0325, Nacos committer, sentinel golang contributor, now works in openjaw micro service team. At present, he is mainly responsible for the development of related projects of Nacos and sentinel golang community, as well as the promotion and integration of Nacos in golang micro service ecology.

Alibaba cloud nativeFocus on micro service, Serverless, container, Service Mesh and other technology areas, focusing on cloud native technology trends, cloud native large-scale landing practice, do the best understanding of the official account of cloud developers.