Three popular databases of NoSQL, memcached, redis and mongodb

Time:2019-10-21

Preface

NoSQL, generally refers to non relational database. With the continuous development of the Internet, the traditional relational database has been unable to cope with the new Internet model websites, especially the super large-scale and high concurrent SNS type pure dynamic websites, which has exposed many problems that are difficult to overcome, while the non relational database has been developed rapidly due to its own characteristics. The production of NoSQL database is to solve the challenges brought by large-scale data collection and multiple data types, especially the application problems of big data. Today, we mainly talk about three nosqls that are used more: memcached, redis and mongodb.

Memcached

Advantage
1. Memcached can take advantage of multi-core advantages, and the throughput of a single instance is very high, which can reach hundreds of thousands of QPS (depending on the byte size of key and value and the performance of server hardware, the peak of QPS in daily environment is about 4-6w). Suitable for maximum carrying capacity.

2. It supports direct configuration as session handle.

shortcoming
1. Only simple key / value data structure is supported, unlike redis, which can support rich data types.

2. It can’t be persisted, data can’t be backed up, can only be used for caching, and all data will be lost after restart.

3. Data synchronization cannot be performed and data in MC cannot be migrated to other MC instances.

4. For memcached memory allocation, the slab allocation mechanism is used to manage memory. When the value size distribution is quite different, the memory utilization will be reduced, and problems such as kicking will still occur when the utilization is low. Users need to pay attention to value design.

Redis

Advantage
1. Support a variety of data structures, such as string, list, dict, set, Zset, and hyperloglog.

2. It supports persistent operation. It can persist AOF and RDB data to disk for data backup or data recovery. It is a better means to prevent data loss.

3. It supports data replication through replication, real-time synchronous data replication through master slave mechanism, multi-level replication and incremental replication. Master slave mechanism is an important means for redis to conduct ha.

4. Single thread request, serial execution of all commands, and data consistency need not be considered in the case of concurrency.

5. Support pub / sub message subscription mechanism, which can be used for message subscription and notification.

6. Support simple transaction requirements, but there are few industry use scenarios and they are not mature.

shortcoming
1. Redis can only use single thread, and its performance is limited by CPU performance, so the maximum single instance CPU can reach 5-6wqps per second (depending on data structure, data size and server hardware performance, the peak of QPS in daily environment is about 1-2W).

2. It supports simple transaction requirements, but there are few use scenarios in the industry, which are not mature, both advantages and disadvantages.

3. Redis consumes more memory in string type. You can use dict (hash table) to compress storage to reduce memory consumption.

MongoDB

Advantage
1. Higher write load, mongodb has higher insertion speed.

2. Deal with large-scale single tables. When the data table is too large, it can easily split the tables.

3. High availability. Setting M-S is not only convenient but also fast. Mongodb can realize node (Data Center) failover quickly, safely and automatically.

4. Fast query. Mongodb supports two-dimensional spatial index, such as pipeline, so it can quickly and accurately obtain data from the specified location. Mongodb will load the data in the database into memory in the form of file mapping after startup. If the memory resource is very rich, it will greatly improve the query speed of the database.

5. With the explosive growth of unstructured data, adding columns may lock the entire database in some cases, or increase the load, resulting in performance degradation. Due to mongodb’s weak data structure mode, adding a new field will not have any impact on the old table, and the whole process will be unusually fast.

shortcoming
1. Transaction is not supported.

2. Mongodb takes up too much space.

3. Mongodb does not have mature maintenance tools.

Differences between memcached, redis and mongodb

1. performance
The performance of the three is relatively high. In general, memcached and redis are almost the same, higher than mongodb.

2. convenience
Memcached has a single data structure.

Redis is richer. In terms of data operation, redis is better, with less network IO times.

Mongodb supports rich data expression, index, most similar to relational database, and very rich query languages.

3. Storage space
Memcached can modify the maximum available memory and adopts LRU algorithm.

Redis added its own VM features after version 2.0, breaking through the limit of physical memory; you can set the expiration time for key value (similar to memcached).

Mongodb is suitable for large amount of data storage. It relies on the operating system VM for memory management. It also eats a lot of memory. Services should not be combined with other services.

4. usability
Memcached itself has no data redundancy mechanism, nor is it necessary; for fault prevention, a mature hash or ring algorithm is used to solve the jitter problem caused by a single point of failure.

Redis relies on the client to realize distributed read-write; when the master-slave node reconnects the master node, it depends on the entire snapshot, no incremental replication. Because of performance and efficiency problems, single point problems are more complex; automatic sharding is not supported, and the consistent hash mechanism needs to be set by the program. One alternative is to use active replication (multiple storage) or incremental replication (self Implementation) instead of redis’s own replication mechanism, to balance consistency and performance.

Mongodb supports master slave, replica (Paxos election algorithm is used internally, and automatic fault recovery is used), and auto sharding mechanism, which shields the client from failover and segmentation mechanism.

5. reliability
Memcached is not supported. It is usually used for caching to improve performance.

Redis support (snapshot, AOF): it relies on snapshot for persistence. AOF enhances reliability and has an impact on performance.

Mongodb started from version 1.8 to use binlog mode to support the reliability of persistence.

6. consistency
In the concurrent scenario, memcached uses CAS to ensure consistency.

Redis transaction support is weak, which can only guarantee the continuous execution of each operation in the transaction.

Mongodb does not support transactions.
7. Data analysis

Mongodb has built-in data analysis function (MapReduce), which is not supported by the other two.

8. Application scenarios
Memcached: used to reduce database load and improve performance in dynamic system; used to cache and improve performance (suitable for reading more and writing less, sharding can be used for large amount of data).

Redis: high performance operation and operation with small amount of data.

Mongodb: it mainly solves the access efficiency of massive data.

Summary and analysis

1. Memcached is better for simple access to key value (mainly read) data. If you want to support data persistence, the amount is not large, and the operation is very frequent. For advanced applications such as queues with multiple data types (such as sets and hashes), you can use redis, but mongodb when the data volume is large.

2. Many clients of memcached are more mature and stable. Redis protocol is more complex than memcached. Can redis be faster than memcached? But the test result is that redis has an absolute advantage.

3. The data in the apsaradb for memcached instance is stored in memory. In case of downtime, power failure in the computer room, or normal upgrade and maintenance of apsaradb for memcached instance, the data in memory will be lost. Therefore, apsaradb for memcached cannot be used as a persistent data storage service. Redis data is stored in memory. If no persistence is configured, all data will be lost after redis restarts. Therefore, it is necessary to turn on the persistence function of redis and save the data to disk. When redis restarts, data can be recovered from disk to achieve persistence.

4. For redis and mongodb, they are generally called redis cache and mongodb database. Redis mainly stores data in memory. The nature of its “cache” is far greater than that of its “data storage”, in which data addition, deletion, modification and query are just as simple as variable operation; mongodb is a “data storage” system, which can add many items, just like SQL database.

5. Mongodb and redis are both NoSQL and adopt structural data storage. There is a certain difference between the two in the use scenarios, which is mainly due to the different processing methods of the two in memory mapping. Mongodb recommends cluster deployment. Considering the cluster scheme more, redis focuses on process sequential writing. Although it supports clusters, it is only in the master-slave mode.

Related information

Brief introduction to NoSQL difference between redis, memchche and mongodb