Redis hybrid storage best practice guide

Time:2020-10-27

Redis hybrid storage instance is a cloud database product independently developed by Alibaba cloud, which is compatible with redis protocol and features. The hybrid storage instance breaks through the limitation that all redis data must be stored in memory, uses disk to store full data, and caches hot data to memory, so as to achieve a perfect balance between accessibility and storage cost.

Architecture and features

Redis hybrid storage best practice guide

Command compatibility

Hybrid storage is compatible with most redis commands. Compared with native redis, the following commands are not supported or restricted. The main reason for not supporting them is that performance is considered. If they are used in business, please submit a work order.

Keys List (linked list) Scripting (Lua script)
RENAME LINSERT Script does not support the load and debug subcommands
RENAMENX LREM
MOVE
SWAPDB
Sort does not support the store option

Selection Guide – scenarios

Redis hybrid storage best practice guide

Selection Guide – Specifications

When selecting a hybrid storage instance, you need to select the appropriate [memory configuration + disk configuration]; the disk determines the total amount of data that can be stored, and the memory determines the total amount of hot data that can be stored. During the instance production, the appropriate CPU resource configuration will be selected according to the storage specification configuration. Currently, the custom CPU core is not supported.

For example, the instance [64GB memory + 256gb disk] means that the instance can store up to 256gb of data (subject to the total physical file size of kV storage engine), and 64GB data can be cached in memory.

Case 1: user a uses redis cluster to store 100GB of data. The total access to QPS is less than 2W, and 80% of the data is rarely accessed. User a can use the mixed storage instance of [32GB memory + 128GB disk], saving nearly 70gb of memory storage and reducing the storage cost by 50% +.

Case 2:User B builds its own pika / SSDB instance in IDC to solve the problem of high storage cost of redis. It stores about 400gb of data, of which about 10% of the data are actively accessed. The cluster operation and maintenance burden is very heavy, and it wants to migrate to the cloud database. User B can use [64GB memory + 512gb disk] mixed storage instance to ensure that the operation and maintenance is free and the service quality is not degraded.

Note:Since redis data is stored in the kV storage engine, metadata information will be added to each key, and the storage space occupation will be enlarged to a certain extent. It is suggested that appropriate margin should be reserved in the selection of disk space, which is estimated as 1.2-1.5 times of the actual storage demand.

performance index

The performance of redis hybrid storage is highly related to the ratio of memory to disk, as well as business access. According to the specification configuration and business access mode, the performance of simple set / get can fluctuate from thousands to tens of thousands. In the best case, all accesses are memory hits, and the performance is basically the same as that of redis memory version; in the worst case, all accesses need to be read from the disk.

Test scenario: 2000W key, value size is 1KB, 25% of hot keys can be stored in memory, the test data of get request is as follows

Test set Memory version (100% data in memory) Mixed storage Version (25% data in memory)
Random access 123000 1.5
Gaussian distribution 80% of the probability of access to 20% of the key 12.0 5.4
99% probability of Gaussian distribution accessing 1% key 13.5 11.4

Application scenarios

Live video

There are a lot of hot data in live video business, and most of the requests come from popular live broadcasting rooms. Using redis hybrid storage instance, the data of popular live broadcasting room is reserved in memory, and the data of inactive live broadcasting room is automatically stored on disk, which can achieve the best utilization effect of limited memory.

E-commerce

E-commerce applications have a large number of commodity data, and the newly launched products will be frequently accessed, while the older ones are not popular; redis is used The hybrid storage instance can easily break through the memory capacity limit and store a large number of commodity data to disk. In normal business requests, active commodity data will be gradually cached in memory to meet business requirements at the lowest cost.

Online education

For online education scenarios, there are a large number of data such as courses, question banks, teacher-student exchange information, and so on. Generally, only popular courses and the latest question bank will be accessed frequently; redis is used The hybrid storage mode stores a large amount of course information to disk, and the active course and question bank information will be transferred into memory and resident in memory, so as to ensure the performance of high-frequency access data and achieve the balance between performance and storage cost.

Other scenarios

Other data access has obvious hot and cold characteristics, and redis hybrid storage can be used to reduce the storage cost for scenarios with low performance requirements.

Cloud database for redis

A stable, reliable, excellent performance, scalable database service. Based on APStar distributed system and full SSD disk high-performance storage, it supports two sets of high availability architectures: the primary and standby version and the cluster version.



Author: Zhang Youdong

Read the original

This article is the original content of yunqi community, which can not be reproduced without permission.