Go actual combat project – Session of beego, use of log file and selection of redis

Time:2021-2-22

Go actual combat project – Session of beego, use of log file and selection of redis

Simple use of session

Go standard library doesn’t realize this function. It can only be realized by itself. Oh, no, it’s a third-party library. Fortunately, beego has its own session function, which has been mentioned before. We only use it simply, and we have to implement the high concurrency scenario by ourselves. It’s hard to rely on this framework alone. Let’s see how to use it
1. It needs to be turned on before calling
beego.BConfig.WebConfig . Session.SessionOn =True / / start session
At present, beego supports four storage engines of session: memory, file, redis and mysql
The default is memory, but it will be invalid after you restart it. In addition to writing demo, even the process of keeping alive is very painful. Based on the previous PHP framework’s method of saving files, my side is also storing files.
2. Set up storage engine
beego.BConfig.WebConfig . Session.SessionProvider =”File” / / specifies the file storage method
3. Set storage path
beego.BConfig.WebConfig . Session.SessionProviderConfig =”. /. TMP” / / specifies the file storage path address, or it can not be specified. There is a default address.
It is recommended to add “.” to the name of the stored folder to facilitate direct filtering when git submits. However, in general, you should not download it if you have nothing to do, or you can put it in a path other than the project. This is the permanent save, restart is still valid.

Use of local logs

Different from PHP, we need to look at the log records for debugging errors of resident memory code or finding online problems. After all, we can’t see the console at that time. It’s not realistic to find problems on the console. So it is necessary to add log, which is a bad habit for students who are used to PHP development. After all, script debugging is too simple and easy, and the modification takes effect immediately. Beego’s startup log is also very simple. Just set it directly. It supports multiple files and is divided according to the rules. By default, it is also divided according to the date.
logs.SetLogger(logs.AdapterFile, {"filename":"./logs/callout.log"})
According to this setting, there will be a separate log every day. The default name is callout.2020-10-13.001.log. We can also dynamically change the folder and do it according to the date. This can improve the efficiency of troubleshooting and save a lot of trouble. Multi file settings just need to logs.AdapterFile Change to logs.AdapterMultiFile , the latter is basically the same, you can add segmentation rules. There are simple descriptions in the official documents. If we don’t go deep into them, we can use them. Programmer’s classic words: it’s not that it can’t be used.

The use of redis

Redis, many libraries come with their own connection pool, so we don’t have to build the wheel ourselves. Of course, the improvement of building our own is quite obvious. When you start to use it, it must be the most mainstream third-party framework, redigo. Is it still in use? ” github.com/garyburd/redigo/redis ”Just import it directly. But because my company’s redis is deployed in a cluster way, so considering the use of this, I have no choice but to use Google’s parent-child library go redis. Use both. After all, they are both mainstream frameworks

The use of redigo

Direct connection

func ConnectRedis() redis.Conn {
    conn, _ := redis.Dial("tcp", "127.0.0.1:6379")
    return conn
}

Connection pool connection

func ConnectRedisPool() redis.Conn {
    connPool := &redis.Pool{
        Dial: func() (conn redis.Conn, err error) {
            conn, err = redis.Dial("tcp", "127.0.0.1:6379")
            if err != nil {
                return nil, err
            }
            return conn, nil
        },
        TestOnBorrow:    nil,
        MaxIdle:         1,
        MaxActive:       10,
        IdleTimeout:     180 * time.Second,
        Wait:            true,
        MaxConnLifetime: 0,
    }

    return connPool.Get()
}

When using, remember to call close() function with defer. Normal use is the way of do. For example, simple setting and getting chestnuts

//@router  /process/test [get]
Func (c * processcontrollers) setredis() {// redigo connection acquisition mode
    redisPool := redisClient.ConnectRedisPool()
    defer redisPool.Close()
    _, err := redisPool.Do("SET", "wjw_key", "wjw")
    if err == nil {
        c. Data ["JSON"] = return success ("request success", nil, 0)
    } else {
        c.Data["json"] = ReturnError(-4007, err.Error())
    }
    c.ServeJSON()
}

// @router  /process/getRedis [get]
func (c *ProcessControllers) GetRedis() {
    redisPool := redisClient.ConnectRedisPool()
    defer redisPool.Close()
    data, err := redis.String(redisPool.Do("GET", "wjw_key"))
    if err == nil {
        c. Data ["JSON"] = return success ("request success", data, 0)
    } else {
        c.Data["json"] = ReturnError(-4007, err.Error())
    }
    c.ServeJSON()
}

Redigo does not support the use of clusters. I don’t know why so many enterprises choose to use it. Does it mean that many enterprises do not have cluster or sentry mode? How to deal with disaster tolerance and fault tolerance? Does it just mean that many companies are practical and there is no need to make so many twists and turns.

The use of go redis

Produced by Google, support cluster and sentry mode connection. This is also more attractive than redigo.

Direct connection

func ConnectRedis() *redis.Client {
    client := redis.NewClient(&redis.Options{
        Addr:     "127.0.0.1:6379",
        Password: '// do not set the password
        DB: 0, // use default
    })
    return client
}

In fact, the efficiency of direct connection is very low. In addition to writing demo verification function, the actual development should not be reluctant to use it. Most of them are used in connection pool.

Connection pool connection

func ConnectRedisPool() *redis.Client {
    client := redis.NewClient(&redis.Options{
        Addr:         "127.0.0.1:6379",
        Password: '// do not set the password
        DB: 0, // use default
        Poolsize: 15, // the maximum number of socket connections in the connection pool, which is 5 times the number of CPUs by default, 5* runtime.NumCPU
        Minidleconns: 10, // create a specified number of idle connections in the startup phase, and the number of connections that maintain idle state for a long time is not less than the specified number
        DialTimeout:  5 *  time.Second , // the connection establishment timeout is 5 seconds by default.
        ReadTimeout:  3 *  time.Second , // read timeout, 3 seconds by default, - 1 means cancel read timeout
        WriteTimeout: 3 *  time.Second , // write timeout, which is equal to read timeout by default
        PoolTimeout:  4 *  time.Second , // when all connections are busy, the maximum waiting time for a client to wait for an available connection is + 1 second by default.

        //Idle connection check includes idletimeout, maxconnage
        IdleCheckFrequency: 60 *  time.Second , // the cycle of idle connection check, which is 1 minute by default, - 1 means that no periodic check is performed, and idle connections are only processed when the client obtains the connection.
        IdleTimeout:        5 *  time.Minute , // idle timeout, 5 minutes by default, - 1 means cancel idle timeout check
        MaxConnAge:         0 *  time.Second , // the connection survival time starts from creation. If the time exceeds the specified time, the connection will be closed. The default value is 0, that is, the connection with longer survival time will not be closed

        //Retrial policy in case of command execution failure
        Maxretries: 0, // the maximum number of retries when the command fails to execute. The default value is 0, that is, no retries
        MinRetryBackoff: 8 *  time.Millisecond , // the lower limit of the retrial interval is calculated each time. The default value is 8 ms, and - 1 is the cancel interval
        MaxRetryBackoff: 512 *  time.Millisecond , // the upper limit of the retrial interval is calculated each time. The default is 512 milliseconds, and - 1 is the cancel interval
        Dialer: func() (conn net.Conn, err error) {
            netDialer := &net.Dialer{
                Timeout:   5 * time.Second,
                KeepAlive: 5 * time.Minute,
            }
            return netDialer.Dial("tcp", "127.0.0.1:6379")
        },
        OnConnect: func(conn * redis.Conn )Error {// this hook function is called only when the client needs to obtain a connection from the connection pool when executing the command, and if the connection pool needs to create a new connection
            fmt.Printf("conn=%v\n", conn)
            return nil
        },
    })
    return client
}

These comments, which can be seen in the source code, all have a more detailed description in English, just with the help of tool translation. Focus on the following, is also used by the author.

Connection in cluster / sentry mode

func ConnectRedisClusterPool() *redis.ClusterClient {
    client :=  redis.NewClusterClient (& redis.ClusterOptions {// sentinel mode: new sentinel client
        //Addrs: [] string {"127.0.0.1:6379", "127.0.0.1:6380"}, // cluster node address. Theoretically, as long as you fill in one available node, the client can automatically obtain all the node information of the cluster. However, it's better to fill in more nodes to increase the disaster recovery capability, because if only one node is filled in, if there is an exception in this node, the go application can't get the cluster information during the startup process.
        Addrs:        []string{"127.0.0.1:6379"},
        Password: '// do not set the password
        Maxredirects: 8, // when encountering network error or moved / ask redirection command, the maximum number of retries is 8
        Poolsize: 15, // the maximum number of socket connections in the connection pool, which is 5 times the number of CPUs by default, 5* runtime.NumCPU
        Minidleconns: 10, // create a specified number of idle connections in the startup phase, and the number of connections that maintain idle state for a long time is not less than the specified number
        DialTimeout:  5 *  time.Second , // the connection establishment timeout is 5 seconds by default.
        ReadTimeout:  3 *  time.Second , // read timeout, 3 seconds by default, - 1 means cancel read timeout
        WriteTimeout: 3 *  time.Second , // write timeout, which is equal to read timeout by default
        PoolTimeout:  4 *  time.Second , // when all connections are busy, the maximum waiting time for a client to wait for an available connection is + 1 second by default.

        //Idle connection check includes idletimeout, maxconnage
        IdleCheckFrequency: 60 *  time.Second , // the cycle of idle connection check, which is 1 minute by default, - 1 means that no periodic check is performed, and idle connections are only processed when the client obtains the connection.
        IdleTimeout:        5 *  time.Minute , // idle timeout, 5 minutes by default, - 1 means cancel idle timeout check
        MaxConnAge:         0 *  time.Second , // the connection survival time starts from creation. If the time exceeds the specified time, the connection will be closed. The default value is 0, that is, the connection with longer survival time will not be closed

        //Retrial policy in case of command execution failure
        Maxretries: 0, // the maximum number of retries when the command fails to execute. The default value is 0, that is, no retries
        MinRetryBackoff: 8 *  time.Millisecond , // the lower limit of the retrial interval is calculated each time. The default value is 8 ms, and - 1 is the cancel interval
        MaxRetryBackoff: 512 *  time.Millisecond , // the upper limit of the retrial interval is calculated each time. The default is 512 milliseconds, and - 1 is the cancel interval
        //Node selection policy for commands that contain only read operations. The default is false, that is, it can only be executed on the master node.
        Readonly: false, // if set to true, read only commands can be executed on the slave node
        //The default is false. If it is set to true, readonly will be set to true automatically, which means that when processing read-only command, the node with the shortest response time of ping() can be selected from the master node corresponding to a slot and all slave nodes to read data
        RouteByLatency: false,
        //The default is false. If it is set to true, readonly will be set to true automatically, which means that when processing read-only commands, one node can be randomly selected from the master node and all slave nodes corresponding to a slot to read data
        RouteRandomly: false,

        //Users can customize the function to read node information, such as zookeeper in non cluster mode.
        //However, if it is oriented to redis cluster, the client will automatically obtain node information from the cluster through the cluster slots command without using this function.
        //ClusterSlots: func() ([]redis.ClusterSlot, error) {
        //    return nil,nil
        //},
        OnConnect: func(conn *redis.Conn) error {
            fmt.Printf("conn=%v\n", conn)
            return nil
        },
    })
    return client
}

The method of use is the same as that of connection pool, except that the type of call is changed to newclusterclient. Remember, if it is in sentry mode, just change to newsentinel client connection.
Call mode, also directly post two pieces of code:

// @router  /process/setRedis [get]
Func (c * processcontrollers) setredis() {// go redis connection acquisition method
    redisPool := redisClient.ConnectRedisClusterPool()
    defer redisPool.Close()
    _, err := redisPool.Ping().Result()
    if err == nil {
        err = redisPool.Set("wjw_key", "wjw hello redis", 0).Err()
        if err == nil {
            c. Data ["JSON"] = return success ("request success", nil, 0)
        } else {
            c.Data["json"] = ReturnError(-4007, err.Error())
        }
    } else {
        c.Data["json"] = ReturnError(-4007, err.Error())
    }
    c.ServeJSON()
}

// @router  /process/getRedis [get]
func (c *ProcessControllers) GetRedis() {
    redisPool := redisClient.ConnectRedisClusterPool()
    defer redisPool.Close()
    _, err := redisPool.Ping().Result()
    if err == nil {
        res, err := redisPool.Get("wjw_key").Result()
        if err == nil {
            c. Data ["JSON"] = return success ("request success", res, 0)
        } else {
            c.Data["json"] = ReturnError(-4008, err.Error())
        }
    } else {
        c.Data["json"] = ReturnError(-4009, err.Error())
    }
    c.ServeJSON()
}

It’s not a difficult technology, but the subsequent use is the time to test the skills.

This work adoptsCC agreementReprint must indicate the author and the link of this article