Several implementation cases of golang connection pool

Time:2021-6-11

Because of the three handshakes of TCP and other reasons, establishing a connection is a relatively high cost behavior. Therefore, in a program that needs to interact with a specific entity many times, it is necessary to maintain a connection pool, in which there are reusable connections for reuse.

To maintain a connection pool, the most basic requirement is to:Thread safeEspecially in languages where golang is a goroutine feature.

By Xiao Ling
Source: Nuggets
Link to the original text:https://juejin.im/post/5e58e3b7f265da57537…

Simple connection pooling

type Pool struct {
    M sync. Mutex // ensure the thread safety of closed when multiple goroutines are accessed
    Res Chan io. Closer // connect the stored Chan
    Factory func() (IO. Closer, error) // factory method of new connection
    Closed bool // connection pool closed flag
}

In this simple connection pool, we use Chan to store the connections in the pool. The method of building new structure is relatively simple

func New(fn func() (io.Closer, error), size uint) (*Pool, error) {
    if size <= 0 {
        Return nil, errors. New ("the value of size is too small.")
    }
    return &Pool{
        factory: fn,
        res:     make(chan io.Closer, size),
    }, nil
}

Just provide the corresponding factory function and connection pool size.

Get a connection

So how do we get resources from it? Because our internal storage connection structure is Chan, we only need a simpleselectTo ensure thread safety:

//Get a resource from the resource pool
func (p *Pool) Acquire() (io.Closer,error) {
    select {
    case r,ok := <-p.res:
        Log. Println ("acquire: shared resource")
        if !ok {
            return nil,ErrPoolClosed
        }
        return r,nil
    default:
        Log. Println ("acquire: new generated resource")
        return p.factory()
    }
}

First, we get it from the Chan of res in the connection pool. If not, we will use the factory function that we have already prepared to construct the connection. At the same time, we use theokFirst determine whether the connection pool has been closed. If it has been closed, we will return the connection closed error that we have already prepared.

Close connection pool

Now that we talk about closing the connection pool, how do we close the connection pool?

//Close the resource pool and release resources
func (p *Pool) Close() {
    p.m.Lock()
    defer p.m.Unlock()

    if p.closed {
        return
    }

    p.closed = true

    //Close the channel and stop writing
    close(p.res)

    //Close the resources in the channel
    for r:=range p.res {
        r.Close()
    }
}

Here we need to do it firstp.m.Lock()\Lock operation, because we need to lock the*closed*Read and write. You need to set this flag bit first, and then close the Chan res, so thatAcquireMethod can no longer get a new connection. Let’s talk about it againresThe connection in Chan is closed.

Release the connection

To release a connection, the first prerequisite is that the connection pool has not been closed. If the connection pool has been closed, go toresIf the connection is sent inside, it will trigger panic.

func (p *Pool) Release(r io.Closer){
    //Ensure that the operation and the operation of the close method are safe
    p.m.Lock()
    defer p.m.Unlock()

    //If all the resource pools are closed, the resource that has not been released will be saved. Just release it
    if p.closed {
        r.Close()
        return
    }

    select {
    case p.res <- r:
        Log. Println ("resources are released into the pool")
    default:
        Log. Println ("resource pool is full, free this resource")
        r.Close()
    }
}

The above is a simple and thread safe connection pool implementation. What we can see is that although the connection pool has been implemented, there are still several small disadvantages

  1. We have no limit on the maximum number of connections. If the thread pool is empty, we will directly create a new connection and return it by default. Once the concurrency is high, new connections will be created continuously, which is easy to cause (especially MySQL)too many connectionsThe error occurred in the report.
  2. Since we need to guarantee the maximum number of connections available, we don’t want to set the number too high. We hope that we can maintain a certain number of idle connections idlenum in idle time, but we also hope that we can limit the maximum number of available connections maxnum.
  3. The first is the case of too much concurrency. What if the amount of concurrency is too little? Now after we create a new connection and return it, we will not use it for a long time. The connection is likely to have been established hours or more ago. We have no way to guarantee the availability of a long idle connection. It is possible that the connection we get next time is a failed connection.

Then we can see from the mature MySQL connection pool library and redis connection pool library how they solve these problems.

SQL connection pool of golang standard library

Golang’s connection pool implements standard librarydatabase/sql/sql.goNext. When we run:

db, err := sql.Open("mysql", "xxxx")

A connection pool is opened when the connection pool is opened. We can take a look backdbThe structure of the system is as follows:

type DB struct {
    waitDuration int64 // Total time waited for new connections.
    mu           sync.Mutex // protects following fields
    freeConn     []*driverConn
    connRequests map[uint64]chan connRequest
    nextRequest  uint64 // Next key to use in connRequests.
    numOpen      int    // number of opened and pending open connections
    // Used to signal the need for new connections
    // a goroutine running connectionOpener() reads on this chan and
    // maybeOpenNewConnections sends on the chan (one send per needed connection)
    // It is closed during db.Close(). The close tells the connectionOpener
    // goroutine to exit.
    openerCh          chan struct{}
    closed            bool
    maxIdle           int                    // zero means defaultMaxIdleConns; negative means 0
    maxOpen           int                    // <= 0 means unlimited
    maxLifetime       time.Duration          // maximum amount of time a connection may be reused
    cleanerCh         chan struct{}
    waitCount         int64 // Total number of connections waited for.
    maxIdleClosed     int64 // Total number of connections closed due to idle.
    maxLifetimeClosed int64 // Total number of connections closed due to max free limit.
}

It omits some fields that don’t need attention for the time being. We can see the internal storage connection structure of DBfreeConn, not the Chan we used before, but[]*driverConn, a connection slice. At the same time, we can see that there aremaxIdleAnd other related variables to control the number of idle connections. It’s worth noting that,DBInitialization function forOpenFunction does not create a new database connection. Which function is the new connection in? We can do it hereQueryMethod, we can see this function:func (db *DB) conn(ctx context.Context, strategy connReuseStrategy) (*driverConn, error). The way we get connections from the connection pool starts here:

Get a connection

// conn returns a newly-opened or cached *driverConn.
func (db *DB) conn(ctx context.Context, strategy connReuseStrategy) (*driverConn, error) {
    //First judge whether the DB has been closed.
    db.mu.Lock()
    if db.closed {
        db.mu.Unlock()
        return nil, errDBClosed
    }
    //Pay attention to check whether the context has been timed out or not.
    select {
    default:
    case <-ctx.Done():
        db.mu.Unlock()
        return nil, ctx.Err()
    }
    lifetime := db.maxLifetime

    //Here, if there are free connections in the freeconn slice, the left pop will be listed. Note that this is a slicing operation, so you need to lock and unlock after obtaining. At the same time, judge whether the returned connection has expired.
    numFree := len(db.freeConn)
    if strategy == cachedOrNewConn && numFree > 0 {
        conn := db.freeConn[0]
        copy(db.freeConn, db.freeConn[1:])
        db.freeConn = db.freeConn[:numFree-1]
        conn.inUse = true
        db.mu.Unlock()
        if conn.expired(lifetime) {
            conn.Close()
            return nil, driver.ErrBadConn
        }
        // Lock around reading lastErr to ensure the session resetter finished.
        conn.Lock()
        err := conn.lastErr
        conn.Unlock()
        if err == driver.ErrBadConn {
            conn.Close()
            return nil, driver.ErrBadConn
        }
        return conn, nil
    }

    //This is the point of waiting for the connection. When the idle connection is empty, a new request will be created and waiting will start
    if db.maxOpen > 0 && db.numOpen >= db.maxOpen {
        //The following action is equivalent to inserting your own number plate into the connrequests map.
        //After the number plate is inserted, there is no need to block here and wait for the logic to continue to go down.
        req := make(chan connRequest, 1)
        reqKey := db.nextRequestKeyLocked()
        db.connRequests[reqKey] = req
        db.waitCount++
        db.mu.Unlock()

        waitStart := time.Now()

        // Timeout the connection request with the context.
        select {
        case <-ctx.Done():
            //When context cancels the operation, remember to take your own number plate from the map connrequests.
            db.mu.Lock()
            delete(db.connRequests, reqKey)
            db.mu.Unlock()

            atomic.AddInt64(&db.waitDuration, int64(time.Since(waitStart)))

            select {
            default:
            case ret, ok := <-req:
                //It's worth noting here, because it has been cancelled by context. But I just put my number in the queue. It means that you may have sent a connection, so you have to pay attention to return it!
                if ok && ret.conn != nil {
                    db.putConn(ret.conn, ret.err, false)
                }
            }
            return nil, ctx.Err()
        case ret, ok := <-req:
            //The following is the operation after the connection has been obtained. Check the status of the connection. Because it may have expired and so on.
            atomic.AddInt64(&db.waitDuration, int64(time.Since(waitStart)))

            if !ok {
                return nil, errDBClosed
            }
            if ret.err == nil && ret.conn.expired(lifetime) {
                ret.conn.Close()
                return nil, driver.ErrBadConn
            }
            if ret.conn == nil {
                return nil, ret.err
            }
            ret.conn.Lock()
            err := ret.conn.lastErr
            ret.conn.Unlock()
            if err == driver.ErrBadConn {
                ret.conn.Close()
                return nil, driver.ErrBadConn
            }
            return ret.conn, ret.err
        }
    }
    //Here is how to create a connection if the above restriction does not exist.
    db.numOpen++ // optimistically
    db.mu.Unlock()
    ci, err := db.connector.Connect(ctx)
    if err != nil {
        db.mu.Lock()
        db.numOpen-- // correct for earlier optimism
        db.maybeOpenNewConnections()
        db.mu.Unlock()
        return nil, err
    }
    db.mu.Lock()
    dc := &driverConn{
        db:        db,
        createdAt: nowFunc(),
        ci:        ci,
        inUse:     true,
    }
    db.addDepLocked(dc, dc)
    db.mu.Unlock()
    return dc, nil
}
Copy code

In short,DBIn addition to using slice to store connections, the structure also adds a mechanism similar to queuingconnRequestsTo solve the process of obtaining the waiting connection. At the same time, it has a good consideration in judging the health of the connection. So now that we have the queuing mechanism, how do we return the connection?

Release the connection

We can find it directlyfunc (db *DB) putConnDBLocked(dc *driverConn, err error) boolThis is the way. As the note says, the main purpose of this method is to:

Satisfy a connRequest or put the driverConn in the idle pool and return true or return false.

Let’s take a look at the key lines

...
    //If you have exceeded the maximum open number, you do not need to return to the pool
    if db.maxOpen > 0 && db.numOpen > db.maxOpen {
        return false
    }
    //Here's the point. Basically, you can randomly select a waiting request from the connrequest map. Take it out and send it to him. You don't have to return the pool.
    if c := len(db.connRequests); c > 0 {
        var req chan connRequest
        var reqKey uint64
        for reqKey, req = range db.connRequests {
            break
        }
        Delete (DB. Connrequests, reqkey) // delete the queued request.
        if err == nil {
            dc.inUse = true
        }
        //Connect to the queued connection.
        req <- connRequest{
            conn: dc,
            err:  err,
        }
        return true
    } else if err == nil && !db.closed {
        //Since no one is queuing up, let's see if there is a maximum number of connections. Return it to freeconn before it arrives.
        if db.maxIdleConnsLocked() > len(db.freeConn) {
            db.freeConn = append(db.freeConn, dc)
            db.startCleanerLocked()
            return true
        }
        db.maxIdleClosed++
    }
...

We can see that when angelica is still connected, if there is a request waiting in the queue, it will not be returned to the pool and sent directly to the person waiting.

Now we have basically solved the small problems mentioned above. There won’t be too many connections to control too many connections. It’s also good to keep the minimum number of connection pools. At the same time, we also do the related operation to check the health of the connection.

It is worth noting that, as a standard library code, the relevant comments and code are very perfect, really refreshing to see.

redisRedis client implemented by golang

How does the redis client implemented by golang implement connection pooling. The idea here is very wonderful. I can still learn a lot of good ideas. Of course, due to the lack of code comments, the first bite is still a bit confused. The relevant code address is athttps://github.com/go-redis/redis/blob/mas…You can see that.

Its connection pool structure is as follows

type ConnPool struct {
    ...
    queue chan struct{}

    connsMu      sync.Mutex
    conns        []*Conn
    idleConns    []*Conn
    poolSize     int
    idleConnsLen int

    stats Stats

    _closed  uint32 // atomic
    closedCh chan struct{}
}

We can see that the structure of the storage connection inside is still slice. But we can focus on itqueueconnsidleConnsThese variables will be mentioned later. But it’s worth noting that! As we can see, there are two[]*ConnStructure:connsidleConnsSo here’s the problem:

Where is the connection?

New connection pool connection

Let’s start with the new connection pool connection

func NewConnPool(opt *Options) *ConnPool {
    ....
    p.checkMinIdleConns()

    if opt.IdleTimeout > 0 && opt.IdleCheckFrequency > 0 {
        go p.reaper(opt.IdleCheckFrequency)
    }
    ....
}

The function that initializes the connection pool is different from the previous two.

  1. checkMinIdleConnsMethod to fill the connection pool with idle connections when the connection pool is initialized.
  2. go p.reaper(opt.IdleCheckFrequency)When the connection pool is initialized, a go process will be started to periodically eliminate the connections to be eliminated in the connection pool.

Get a connection

func (p *ConnPool) Get(ctx context.Context) (*Conn, error) {
    if p.closed() {
        return nil, ErrClosed
    }

    //This is different from the previous process of SQL getting the join function. SQL is to first see if there are idle connections in the connection pool, and if there are some, first get them and then queue them. Here is to directly queue to obtain the token, and the queuing function will be analyzed later.
    err := p.waitTurn(ctx)
    if err != nil {
        return nil, err
    }
    //If there is no error in front of us, we will be waiting in line. Next is the acquisition process.
    for {
        p.connsMu.Lock()
        //Get an idle connection from the idle connection first.
        cn := p.popIdle()
        p.connsMu.Unlock()

        if cn == nil {
            //Jump out of the loop when there is no idle connection.
            break
        }
        //Judge whether it is out of date. If it is, close it and continue to take it out.
        if p.isStaleConn(cn) {
            _ = p.CloseConn(cn)
            continue
        }

        atomic.AddUint32(&p.stats.Hits, 1)
        return cn, nil
    }

    atomic.AddUint32(&p.stats.Misses, 1)

    //If there is no idle connection, a new connection will be created here.
    newcn, err := p.newConn(ctx, true)
    if err != nil {
        //Return the token.
        p.freeTurn()
        return nil, err
    }

    return newcn, nil
}

We can try to answer the opening question: where is the connection? The answer is fromcn := p.popIdle()It can be seen from this sentence that the action of getting a connection comes fromidleConnsAnd the functions inside prove this. At the same time, my understanding is:

  1. The queuing of SQL means that after I apply for connection to the connection pool, I will tell the connection pool my number. As soon as you see that there is free time, call my number. I promised, and then the connection pool gave me a connection directly. If I don’t return it, the connection pool won’t call the next number.
  2. What redis means here is that I go to the connection pool to apply for a token instead of a connection. I’ve been waiting in line. After the connection pool gave me a token, I went to the warehouse to find a free connection or create a new one myself. When the connection is used up, you have to return the token as well as the connection. Of course, if my new connection fails, even if I can’t get the connection home, I have to give the token back to the connection pool. Otherwise, the number of tokens in the connection pool will be less and the maximum number of connections will be smaller.

And:

func (p *ConnPool) freeTurn() {
    <-p.queue
}
func (p *ConnPool) waitTurn(ctx context.Context) error {
...
    case p.queue <- struct{}{}:
        return nil
...
}

It depends on the Chan of queue to maintain the number of tokens.

thatconnsWhat is the role of the government? Let’s take a look at the new connection function

make new connection

func (p *ConnPool) newConn(ctx context.Context, pooled bool) (*Conn, error) {
    cn, err := p.dialConn(ctx, pooled)
    if err != nil {
        return nil, err
    }

    p.connsMu.Lock()
    p.conns = append(p.conns, cn)
    if pooled {
        //If the connection pool is full, it will be removed later.
        if p.poolSize >= p.opt.PoolSize {
            cn.pooled = false
        } else {
            p.poolSize++
        }
    }
    p.connsMu.Unlock()
    return cn, nil
}

The basic logic came out. If you create a new connection, I will not put it directly in theidleConnsInside, but firstconnsInside. At the same time, let’s see if the pool is full. If it is full, it will be marked when it is returned and deleted later. When will this be deleted? That’s the time to return the connection.

Return connection

func (p *ConnPool) Put(cn *Conn) {
    if cn.rd.Buffered() > 0 {
        internal.Logger.Printf("Conn has unread data")
        p.Remove(cn, BadConnError{})
        return
    }
    //That's what we just said at the end. Those marked not to enter the pool are deleted here. Of course, there will be freeturn operation inside.
    if !cn.pooled {
        p.Remove(cn, nil)
        return
    }

    p.connsMu.Lock()
    p.idleConns = append(p.idleConns, cn)
    p.idleConnsLen++
    p.connsMu.Unlock()
    //We can see the obvious action of returning the number plate.
    p.freeTurn()
}

In fact, the process of return is fromconnsTransfer toidleConnsIt’s a process of change. Of course, if you find that the connection has been createdOversoldAfter that, it will not be transferred when it is returned. It will be deleted directly.

Wait, there seems to be something wrong with the above logic? Let’s take a look at the connection process

  1. beforewaitTurnTo get the token. The number of tokens is based on the number of tokens in the poolqueueIt’s up to you.
  2. Got the token. Go to the warehouseidleConnsTake the free connection inside. If you don’t have it, you’ll be on your ownnewConnOne, and put him on recordconnsInside.
  3. When used up, callputReturn: that is, fromconnsTransfer toidleConns. Check it when you return itnewConnIt’s time to mark oversold. If it is, it will not be transferred toidleConns

I was puzzled for a long time. Since I always need to get a token to get a connection, the number of tokens is fixed. Why is it oversold? Looking at the source code, my answer is:

althoughGetMethod to get the connectionnewConnThis private method is subject to token control, so there will be no oversold. But this method accepts the parameterspooled bool. So I guess it’s because I’m worried that when other people call this method, no matter what, they will pass true, causing the poolsize to get bigger and bigger.

In general, the number of connections in the connection pool of redis is still under controlqueueThis Chan, which I call a token, operates.

summary

As you can see above, the most basic guarantee of connection pool is thread safety when obtaining connections. However, in the implementation of many additional features, they are implemented from different perspectives. It’s very interesting. But no matter the storage structure is Chan or slice, it can achieve this well. If you use slice to store connections like SQL or redis, you have to maintain a structure to represent the effect of queuing.

Several implementation cases of golang connection pool

This work adoptsCC agreementReprint must indicate the author and the link of this article

Official account: Network Management BI, Golang, Laravel, Docker, K8s and other learning experience sharing