Talk about Mongo driver’s connection pool

Time:2021-1-18

preface

For the server, before a single link reaches the maximum throughput, less links means less context switching and less memory consumption (TCP stack memory consumption, application layer buffer). Therefore, in addition to the conventional indexing, fragmentation, read-write separation, the design of connection pool also has a very important impact on the database performance
The language we use has no official driver, and the connection pool is implemented with a fixed size. In the case of few nodes, the problem is not big. When the number of nodes is increasing, the problem can not be ignored. Therefore, refer to the implementation of official driver

mongo offical driver

offical drivers

c driver

source

https://github.com/mongodb/mo…
commit c99f53b9bcdf5b3756f6c70d8148fbb86015b6f4

document

http://mongoc.org/libmongoc/current/connection-pooling.html
C driver has single mode / pooled mode. Obviously, calling thread and link will not be one-to-one relationship. Pooled mode is more recommended
Reference connection pool options
http://mongoc.org/libmongoc/current/mongoc_uri_t.html#connection-pool-options
C driver abandoned min_ pool_ Size, as you can see from mongoc-client-pool. C: 333 code

void
mongoc_client_pool_push (mongoc_client_pool_t *pool, mongoc_client_t *client)
{
   ENTRY;

   BSON_ASSERT (pool);
   BSON_ASSERT (client);

   bson_mutex_lock (&pool->mutex);
   _mongoc_queue_push_head (&pool->queue, client);

   if (pool->min_pool_size &&
       _mongoc_queue_get_length (&pool->queue) > pool->min_pool_size) {
      mongoc_client_t *old_client;
      old_client = (mongoc_client_t *) _mongoc_queue_pop_tail (&pool->queue);
      if (old_client) {
         mongoc_client_destroy (old_client);
         pool->size--;
      }
   }

   mongoc_cond_signal (&pool->cond);
   bson_mutex_unlock (&pool->mutex);

   EXIT;
}

min_ pool_ The actual meaning of size is, max_ pool_ Size determines the maximum number of links, min_ pool_ Size determines the minimum number of links to be maintained at the same time. Obviously, this will lead to frequent Link Creation and disconnection under heavy workload, which will degrade the performance. Therefore, it is not recommended

c++ driver

source

https://github.com/mongodb/mo…
commit dfe361bf672809beba0f6164fafad9b088d55fef

document

http://mongocxx.org/mongocxx-v3/connection-pools/
As you can see, the default value of minpoolsize is 0. Since the link creation time is short enough, it is reasonable to create it again

maxPoolSize The maximum number of clients created by a mongocxx::pool (both in the pool and checked out). The default value is 100. Once it is reached, mongocxx::pool::acquire blocks until another thread returns a client to the pool.
minPoolSize Sets a target size for the pool when idle. Once this many clients have been created, there will never be fewer than this many clients in the pool. If additional clients above minPoolSize are created, they will be destroyed when returned to the pool. The default value is “0”, which disables this feature. When disabled, clients are never destroyed.

Note that the C + + driver reuses the connection pool code of the C driver, only uses the smart pointer to do the raii_ pool_ For size, please refer to the documentation of C driver

java driver

source

https://github.com/mongodb/mo…
6d20b9128bd6966b31c23f7aab681c056aaefc72

document

https://mongodb.github.io/mongo-java-driver/3.4/javadoc/com/mongodb/MongoClientOptions.html
Java driver is very wordy. From the API point of view, the connection pool options are relatively rich

  1. No minpoolsize, only maxpoolsize
  2. Have idle time design
  3. Maxconnection, lifetime, socket timeout, heartbeat frequency, etc

elixir DBCollection

https://hexdocs.pm/db_connection/DBConnection.html
Elixir’s DB connection pool is fixed size. Not available

summary

  • A shared connection pool is recommended
  • By kicking out idle links, try to reduce the number of links. When not in use, the pool size can be 0