These pits of redis distributed lock, I suspect you are a fake developer

Time:2021-11-27

Absrtact: what problems have you encountered with locks?

1、 Vernacular distributed

What is distributed? In the simplest words, it is to distribute functions on different machines in order to reduce the pressure of a single server;

For example:

Originally, a programmer can complete a project: requirements – > Design – > coding – > testing

These pits of redis distributed lock, I suspect you are a fake developer

However, when there are many projects, one person can’t carry them, which requires different people to divide their work and cooperate

These pits of redis distributed lock, I suspect you are a fake developer

This is a simple distributed collaborative work;

2、 Distributed lock

First, let’s look at a problem. If a certain link is terminated or occupied, unknowable things will happen

These pits of redis distributed lock, I suspect you are a fake developer

This will happen. The designed or semi-finished products will be damaged, resulting in errors in the later links;

At this time, we need to introduceDistributed lockThe concept of;

What is distributed lock?

  • When there is only one copy of data (or limited) in the distributed model, the lock technology needs to be used to control the number of processes that modify the data at a certain time.
  • A status value is used to represent the lock, and the occupation and release of the lock are identified by the status value.

Conditions for distributed locks:

  • It can ensure that in the distributed application cluster, the same method can only be executed by one thread on one machine at the same time.
  • If this lock is a reentrant lock (avoid deadlock)
  • This lock should preferably be a blocking lock
  • This lock should be a fair lock
  • It has highly available lock acquisition and release functions
  • The performance of acquiring and releasing locks is better

Implementation of distributed lock:

There are many ways to implement distributed locks, such as file locks, databases, redis, and so on. In practice, redis has better performance than distributed locks;

3、 Redis implements distributed locks

First look at two commands:

setnx:Set the value of the key to value if and only if the key does not exist. If the given key already exists, setnx will not do anything. Setnx is short for set if not exists.

These pits of redis distributed lock, I suspect you are a fake developer

expire: EXPIRE key seconds

For givenkeySet the lifetime whenkeyWhen expired (lifetime is)0), it will be deleted automatically

These pits of redis distributed lock, I suspect you are a fake developer

Distributed lock based process:

These pits of redis distributed lock, I suspect you are a fake developer

This is a simple implementation process of distributed lock. The specific code implementation is also very simple, so I won’t repeat it;

4、 Redis implementation of distributed locks

If there is such a problem: ifsetnxIt was successful, butexpireIf the setting fails, if the lock release fails later, the lock will never be obtained and the business will be locked?

Solution: usesetCommand to set the lock and expiration time at the same time

setParameters:

These pits of redis distributed lock, I suspect you are a fake developer

Practice:

These pits of redis distributed lock, I suspect you are a fake developer

This perfectly solves the atomicity of distributed locks;

What problems have you encountered with locks? How to solve it?

Resource not closed

Because the current thread obtains the redis lock and fails to release the lock in time after processing the business, other threads will always try to obtain the lock and block. For example, the jedis client will report the following error message

1redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
There are no idle threads in the redis thread pool to process client commands. Remember to close using the native method!

The solution is also very simple. As long as we are more careful, the thread that gets the lock will release the lock in time after processing the business

B’s lock was released by A

We know that the principle of redis lock implementation lies in the setnx command. When the key does not exist, set the value of the key to value, and the return value is 1; If the given key already exists, setnx does not take any action and the return value is 0.

SETNX key value

Let’s imagine this scenario: threads a and B try to lock key mylock. Thread a gets the lock first (if the lock expires after 3 seconds), and thread B is waiting to try to get the lock. There is no problem at this point.

If the business logic is time-consuming and the execution time has exceeded the expiration time of the redis lock, the lock of thread a is automatically released (the key is deleted). Thread B detects that the key mylock does not exist and executes the setnx command to get the lock.

However, after thread a executes the business logic, it still releases the lock (delete the key), which causes thread B’s lock to be released by thread a.

In order to avoid the above situation, we generally need to bring our own unique value value to identify each thread when locking, and only release the key of the specified value, otherwise there will be a chaotic scenario of releasing the lock

Generally, we can set value as the business prefix_ The current thread ID or UUID can be released only if the current value is the same

The lock has expired and the business has not been completed

The scenario where the redis distributed lock expires but the business logic is not fully executed. However, let’s think about the problem in another way. Won’t it be solved by making the expiration time of redis lock longer?

There is still a problem. We can manually increase the expiration time of the redis lock when locking, but how long is it appropriate? The execution time of business logic is uncontrollable, and too long adjustment will affect the operation performance.

If only the expiration time of the redis lock could be automatically renewed.

In order to solve this problem, we use redis client redison. Redison solves some thorny problems of redis in distributed environment. Its purpose is to let users pay less attention to redis and spend more energy on processing business logic.

Redisson encapsulates distributed locks well. You only need to call API.

1  RLock lock = redissonClient.getLock("stockLock");

After successfully locking, reisson will register a scheduled task to monitor the lock, check the lock every 10 seconds, and renew the expiration time if it still holds the lock. The default expiration time is 30 seconds. This mechanism is also called “watchdog”

Redis master-slave replication

The most common scheme for redis high availability is master-slave replication, which also digs a hole for redis distributed locks.

In the redis cluster environment, if client a wants to lock, it will select a master node to write the key mylock according to the routing rules. After locking is successful, the master node will asynchronously copy the key to the corresponding slave node.

If the redis master node goes down and the slave node fails to replicate, in order to ensure the cluster availability, the active / standby switch will be performed, and the slave will become the redis master. Client B successfully locks the new master node, while client a thinks it has successfully locked itself. In addition, the delay of master-slave replication will also cause the problem of locking and unlocking delay.

This will cause multiple clients to lock a distributed lock at the same time, resulting in the generation of various dirty data.

After all, redis maintains AP rather than CP. if you want to pursue strong consistency, you can use zookeeper distributed lock

This article is shared from Huawei cloud community “redis distributed lock? Easy to step on the pit”, original author: Minjie.

Click focus to learn about Huawei cloud’s new technologies for the first time~