Practice and summary of redis cache performance

Time:2020-10-18

1、 Preface

In Internet applications, cache becomes the key component of high concurrency architecture. This blog mainly introduces the typical scenarios of cache usage, practical case analysis, redis usage specification and regular redis monitoring.

2、 Common cache comparison

Common cache schemes include local cache, including HashMap / concurrent HashMap, ehcache, Memcache, guava cache, etc., and cache middleware includes redis and TAIR.

Practice and summary of redis cache performance

3、 Redis usage scenarios

1. Counting

Redis realizes fast counting and caching functions.

For example, the number of online viewers of video or live broadcast will increase by 1 every time the user plays it.

2. Session centralized management

Session can be stored in the application service, which is a JVM. However, this solution will have consistency problems. In addition, high concurrency will cause memory overflow of the JVM. Redis centralizes the user’s session management. In this case, as long as the high availability and scalability of redis are ensured, each user’s update or query login will be directly obtained from the information in redis.

Speed limit

For example, the service requires that the user can only get five verification codes within one minute.

4. Ranking

The query speed of relational database is generally slow in ranking, so we can sort the hot data with the help of redis sortedset.

For example, in the project, if you need to count the anchor’s gold absorption ranking, you can take the anchor’s ID as a member, and the corresponding heat value of the activity gift given on the day as the score. You can get the anchor activity daily list through zrangebyscore.

5. Distributed lock

In the actual multi process concurrent scenario, distributed lock is used to limit the concurrent execution of programs. It is mainly used to prevent cache breakdown in high concurrency scenarios.

The distributed lock is actually “occupying the pit”. When another process is executing setnx, it is found that the ID bit is already 1, so it has to give up or wait.

4、 Case analysis

1. [case] expiration setting – set command will remove the expiration time

The expiration time can be set for all data structures of redis. If a string has an expiration time set and then reset it, the expiration time will disappear. Therefore, it is necessary to reasonably evaluate the redis capacity in the project, so as to avoid that there is no expiration policy due to frequent set, which will indirectly lead to full memory.

The following is a screenshot of redis source code:

Practice and summary of redis cache performance

2. [case] about setting bug of jedis 2.9.0 and below

It is found that jedis has a bug when calling the expiredat command. The final call is the pexpire command. This bug will cause the key to expire for a long time, resulting in redis memory overflow and other problems. It is recommended to upgrade to jedis 2.9.1 and above.

BinaryJedisCluster.java The source code is as follows:

@Override
  public Long pexpireAt(final byte[] key, final long millisecondsTimestamp) {
    return new JedisClusterCommand<Long>(connectionHandler, maxAttempts) {
      @Override
      public Long execute(Jedis connection) {
        return  connection.pexpire (key, milliseconddstimestamp); // here pexpire should be pexpireat
      }
    }.runBinary(key);
  }

Compared with pexpire and pexpiraat:

Practice and summary of redis cache performance

For example, the time we currently use is 17:00:00 on June 14, 2018, and its UNIX timestamp is 1528966800000 Ms. when we use the pexpireat command, the corresponding key will expire immediately because it is in the past.

When we misuse the pexpire command, the key will not expire immediately, but will not expire until 1528966800000 milliseconds. The expiration time of the key will be quite long, about a few weeks later, which may lead to redis memory overflow, server crash and other problems.

3. [case] cache is broken down

The cache key has an expiration policy. If there are a large number of concurrent requests for this key at this point in time, these requests will generally return the source data from the back-end dB and set it back to the cache when the cache is expired. At this time, large concurrent requests may temporarily hang the back-end DB.

The commonly used optimization schemes in the industry aretwo types

First:The distributed lock is used to ensure high concurrency, and only one thread can return to the source backend dB.

The second type:To ensure that the redis key of high concurrency requests is always valid, use non user requests back to the source back-end, and change to active return to the source. Generally, asynchronous tasks can be used for active cache refresh.

4. [case] redis standalone architecture forbids the use of non-0 libraries

Redis performs the switch between select 0 and select 1, resulting in performance loss.

Redistemplate gets the link first when executing the execute method.

Practice and summary of redis cache performance

Execute to RedisConnectionUtils.java There will be a method to get the link.

Practice and summary of redis cache performance

Je disConnectionFactory.java The jedisconnection constructor will be called. Note that dbindex here is the database number, such as: 1

Practice and summary of redis cache performance

Continue to follow up the jedisconnection code. When the selected library is greater than 1, there will be a select DB operation. If you always use the 0 library, you don’t need to execute additional library cutting command. Knowing the first place to cut select 1, where does select 0 come from?

Practice and summary of redis cache performance

In fact, the client will release the link when using redis, but the redistemplate has already released it for us automatically. Let’s go back to the place where redistemplate executed the execute (…) method in the beginning.

Practice and summary of redis cache performance

It’s still below RedisConnectionUtils.java To execute the code for the link closure.

Practice and summary of redis cache performance

According to the meaning of code comment, if the selected library number is not 0, the spring data redis framework will reset select 0 every time!

Practice and summary of redis cache performance

In vivo mall business, the performance of the product detail page interface has been improved more than three times after the above optimization.

Further verify that the database switch can affect the performance by at least 3 times (depending on the specific business).

Rediscluster cluster cluster database is 0 database by default. Other databases cannot be selected, which avoids this problem.

5. [case] watch out for time complexity O (n) redis command

Redis is single threaded, so it is thread safe.

Redis uses non blocking IO, and the time complexity of most commands is O (1).

It is very dangerous to use time-consuming commands, which will take up a large amount of processing time of a single thread, resulting in all requests being slowed down.

For example: get the smembers myset in all set sets, and return all the members in the specified hash with time complexity O (n).

The cache value set becomes larger. When the high-level parallel interface requests, the relevant data will be read from redis. The read time of each request becomes longer and the stack is constantly. As a result, the hot key situation occurs. A certain partition of redis is blocked, and the CPU utilization rate reaches 100%.

6. [case] cache hot key

In redis, the key with high access frequency is called the hot key. When a hot key request to the server host, due to the large amount of requests, the host resource is insufficient or even down, affecting the normal service.

The hot key problem is as followsThere are two reasons

  1. The data consumed by users is much larger than that of production, such as hot-selling goods or instant sale products, hot news, hot comments, etc. these typical scenarios of reading more and writing less will produce hot issues.
  2. If the request fragment set exceeds the performance limit of a single server, such as the fixed name key, and the hash falls into one server, when the access volume is very large, it will lead to the hot key problem.

So in real business,How to identify the hot key?

  1. Based on business experience, estimate which hot keys are;
  2. Client statistics collection, local statistics or reporting;
  3. If the server has an agent layer, it can collect and report in the agent layer;

When we recognize the hot key,How to solve the hot key problem

  1. Redis cluster expansion: increase partitioned copies to balance read traffic;
  2. Further hash the hot key, such as backing up a key as key1, key2 Keyn, n backups of the same data are distributed to different partitions. When accessing, one of the N backups can be accessed randomly to further share the read traffic.
  3. Use the secondary cache, which is the local cache.

When a hot key is found, the corresponding data of the hot key is first loaded into the local cache of the application server to reduce the read requests to redis.

5、 Redis specification

1. Prohibit use of non database 0

explain:

Redis standalone architecture. It is forbidden to use other databases in redis.

Reason:

  • For future business migration redis cluster to maintain compatibility
  • When multiple databases are switched with select, CPU resources are consumed more.
  • It is easier to automate operation and maintenance management. For example, the scan / dbsize command is only used when database.
  • Due to thread safety problems, some redis clients do not support single instance multiple databases.

2. Key design specification

Name the key prefix according to the business function to prevent key conflict. It is recommended to separate the prefix with colon. For example, business name: table nameFor example live:rank : user:weekly :1:202003。

The length of the key is less than 30 characters. The key name itself is a string object. The maximum length of redis hard coding is 512MB.

In the redis cache scenario, it is recommended that TTL values be set for all keys to ensure that unused keys can be cleaned up or eliminated in time.

Special characters such as spaces, line breaks, single and double quotation marks, and other escape characters are not allowed in key design.

3. Value design specification

The size of a single value must be controlled within 10KB. If the number of single instance keys is too large, it may lead to delayed recovery of expired keys.

For complex data types such as set, hash, list, etc., the number of elements in the data structure should be reduced as much as possible. It is recommended that the number of elements should not exceed 1000.

4. Focus on command time complexity

O (1) command, such as get scard, is recommended.

The O (n) command focuses on the number of N. the following command needs to control the n value at the business level.

  • hgetall
  • lrange
  • smembers
  • zrange

For example, if the time complexity of smember command is O (n), when n continues to increase, redis CPU will continue to soar, blocking the execution of other commands;

5. Pipeline use

explain:

Pipeline is a way of redis batch submission, that is to establish a connection for multiple command operations and send them to redis for execution. It has better performance than a single round commit.

The execution of a command by redis client consists of four processes: sending command, queuing command, executing command and returning result.

Practice and summary of redis cache performance

The commonly used mget and Mset commands can effectively save RTT (round trip time of command execution). However, hgetall does not have mhgetall and does not support batch operation. At this point, you need to use the pipeline command

Practice and summary of redis cache performance

For example, in the live broadcast media project, you need to query the anchor’s daily, weekly, and monthly rankings at the same time, and use pipeline to submit multiple commands at a time, and return three list data at the same time.

6. Online disable command

  • Disable monitor

It is forbidden to use the monitor command in the production environment. Under the condition of high concurrency, the monitor command may cause memory explosion and affect the performance of redis

  • Do not use keys

If the keys are too many, the traversal of other commands will be very slow. Therefore, the keys and keys pattern commands are prohibited.

It is recommended to use the scan command instead of the keys command online.

  • Flushall and flushdb are prohibited

Delete all records in all databases in redis, and the command is atomic and will not terminate execution. Once executed, it will not fail.

  • Do not use save

Blocking the current redis server until the persistence operation is completed. For instances with large memory, it will cause long-term blocking.

  • BGREWRITEAOF

Manual AOF, manual persistence will cause long-term blocking for large memory instances.

  • Config

Config is a client configuration mode, which is not conducive to redis operation and maintenance. It is recommended to set it in redis configuration file.

6、 Redis monitoring

1. Slow query

Method 1Slowlog get slow query log

127.0.0.1:{port}> slowlog get 5

1) 1) (integer) 47

   2) (integer) 1533810300

   3) (integer) 175833

   4) 1) “DEL”

      2) “spring:session:expirations:1533810300000”

2) 1) (integer) 46

   2) (integer) 1533810300

   3) (integer) 117400

   4) 1) “SMEMBERS”

Method 2More comprehensive slow queries can be monitored through the cachecloud tool.

Path: “Application List” – click the relevant application name – click the “slow query” tab page.

Click “slow query” to focus on the number of slow queries and related commands.

2. Monitor the CPU core usage rate of redis instance binding

Since redis is a single thread, the CPU core utilization rate of redis instance binding is mainly monitored.

Generally, the CPU resource utilization rate is about 10%. If the utilization rate is higher than 20%, consider whether RDB persistence is used.

3. Redis partition load balancing

In the current redis cluster architecture mode, a cluster composed of three masters and three slaves focuses on the traffic balance of requests in each partition of the redis cluster;

Get: redis cli – P {port} – H {host} — stat

In general, alarm is required when it exceeds 12W.

4. Focus on big key bigkey

Through the tools provided by redis, redis cli regularly scans the corresponding redis key for optimization.

The specific commands are as follows: redis cli – H 127.0.0.1 – P {port} — bigkeys or redis memory for key – s {IP} – P {port} XXX_ KEY

Generally, more than 10k is a big key, which needs to be focused on. It is recommended to optimize from the business level.

5. Monitor the memory size of redis

View the info memory command to avoid the performance problems caused by the depletion of the allocated maxmemory in high concurrency scenarios.

Focus on used_ memory_ When the increment of the value value corresponding to the human configuration item is too high, it is necessary to focus on the evaluation.

7、 Summary

Combining with the specific business characteristics, we can reasonably evaluate the memory capacity required by redis, select the data type, and set the single key size, so as to better serve the business and provide high-performance guarantee for the business.


Author: Jessica Chen