Architecture design | peak shaving of high concurrent traffic and locking mechanism of shared resources

Time:2021-11-27

Source code of this article:GitHub · click here || Gitee · point here

1、 Introduction to high concurrency

In the business architecture of the Internet, high concurrency is one of the most difficult businesses. Common usage scenarios: second kill, rush purchase and ticket booking system; There are many complex problems to be handled in highly concurrent processes, mainly involving the following aspects:

  • Flow management, undertake peak shaving level by level;
  • Gateway control, routing request, interface fusing;
  • Concurrency control mechanism, resource locking;
  • Distributed architecture, isolation of services and databases;

The core of high concurrency business is traffic control, which controls the sinking speed of traffic, or controls the size of the container to undertake traffic. Excess direct overflow is a relatively complex process. The second is to access shared resources under multithreading concurrency. The process needs a locking mechanism to avoid confusion in data writing.

2、 Second kill scene

1. Pre rush purchase business

Before the event is officially started, make an event appointment and collect and control part of the traffic. At the time point of the real spike, many data may have been preprocessed, which can greatly reduce the pressure of the system. With a certain reservation flow, you can also prepare the inventory system in advance to kill two birds with one stone.

Scene: event reservation, deposit reservation, high-speed rail ticket pre purchase.

2. Batch buying

The implementation mechanism of batch rush purchase and rush purchase scenarios is the same, but it relieves a lot of pressure in terms of traffic. The compression resistance of the second kill 10W inventory system and the second kill 100 inventory system is not the same level. If you spike 10W inventory, the system will bear at least several times more traffic impact than 10W. If you spike 100 inventory, the system may bear hundreds or thousands of traffic. The strategy mechanism here is explained in detail below.

Scenario: Rush buying in multiple sessions at different times, and high-speed rail tickets are released in batches.

3. Real time spike

The most difficult scenario is the on-time real-time second kill activity. If you grab 1W goods on time at 10:00 sharp, high concurrent traffic will flow around this time point, refresh the page, or request the interface for rush purchase. This scenario is the most complex to deal with.

  • First, the system should accept the influx of traffic;
  • The continuous refresh of the page should be loaded in real time;
  • Flow control and locking of high concurrent requests;
  • Service isolation and system protection of database design;

Scene: 618 on time rush purchase, double 11 on time second kill, e-commerce promotion second kill.

3、 Flow peak clipping

Architecture design | peak shaving of high concurrent traffic and locking mechanism of shared resources

1. Nginx agent

Nginx is a high-performance HTTP and reverse proxy web server. It is often used as a unified proxy layer and load balancing strategy in cluster services. It can also be used as a traffic control layer to provide two flow limiting methods: one is to control the rate and the other is to control the number of concurrent connections.

Based on leaky bucket algorithm, it provides the ability to limit request processing rate; Limit the access frequency of IP. When the traffic suddenly increases, the exceeding requests will be rejected; You can also limit the number of concurrent connections.

In the high concurrency spike scenario, the traffic can be controlled in a relatively stable state through various restriction strategies of the nginx layer.

2. CDN node

The proxy node of CDN static file and the service of second kill scenario have such an operation feature. Before the countdown starts, a large number of users will constantly refresh the page. At this time, the static page can be handed over to the CDN level agent to share the pressure of the data service interface.

The CDN level can also do a layer of flow restriction. A layer of strategy is built in the page. Assuming that 10W users click rush purchase, they can only release 1W traffic, and other direct prompts can be completed. This is also one of the commonly used means.

Implication: in the rush buying activities you usually participate in, your request may not reach the data interface level at all, so you can respond quickly. The goods have been robbed, so you can think of it yourself.

3. Gateway control

The gateway layer handles service interface routing. In addition to some verification, the most important thing is to integrate some policies into the gateway. For example, after the above layers of flow control, the request is close to the core data interface. At this time, some policy controls are built in the gateway layer: if the activity is to activate old users, the gateway layer can quickly judge the user attributes, Old users will release the request; If the purpose of the activity is to pull new users, more new users will be released.

After these levels of control, there is not much traffic left, and then the data operation of rush purchase is really started.

Implication: if 10W people participate in rush buying activities, the rush buying traffic that really sinks to the bottom layer may be 1W or even less, which will be processed in the decentralized cluster services.

4. Concurrent fusing

Among the interfaces of distributed services, there is also the most refined layer of control. For an interface, the number of requests processed in units is controlled. Considering the response time based on the interface, the faster the response, the higher the concurrency in unit time. The logic here is not difficult to understand.

Implication: the traffic is controlled layer by layer, and the pressure shared at the data interface level is not large. At this time, we are facing the locking problem in the second kill business.

4、 Distributed locking

1. Pessimistic lock

Mechanism description

All requesting threads must acquire locks before they can perform database operations. Based on the serialization mode, threads that do not acquire locks are in a waiting state, and a retry mechanism is set to try to acquire locks again after unit time, or return directly.

Process Mapping

Architecture design | peak shaving of high concurrent traffic and locking mechanism of shared resources

Redis basic command

Setnx: the idea of locking is to set the key to value if the key does not exist. If the key already exists, setnx will not take any action. And you can set the expiration time for the key. After expiration, other threads can continue to try the lock acquisition mechanism.

Use this command of redis to simulate lock acquisition.

code implementation

Here is the lock acquisition and release mechanism based on redis.

import org.springframework.stereotype.Component;
import redis.clients.jedis.Jedis;
import javax.annotation.Resource;
@Component
public class RedisLock {

    @Resource
    private Jedis jedis ;

    /**
     *Acquire lock
     */
    public boolean getLock (String key,String value,long expire){
        try {
            String result = jedis.set( key, value, "nx", "ex", expire);
            return result != null;
        } catch (Exception e){
            e.printStackTrace();
        }finally {
            if (jedis != null) jedis.close();
        }
        return false ;
    }

    /**
     *Release lock
     */
    public boolean unLock (String key){
        try {
            Long result = jedis.del(key);
            return result > 0 ;
        } catch (Exception e){
            e.printStackTrace();
        }finally {
            if (jedis != null) jedis.close();
        }
        return false ;
    }
}

This is based on the API implementation of jedis, and a configuration file is provided here.

@Configuration
public class RedisConfig {

    @Bean
    public JedisPoolConfig jedisPoolConfig (){
        JedisPoolConfig jedisPoolConfig = new JedisPoolConfig() ;
        jedisPoolConfig.setMaxIdle(8);
        jedisPoolConfig.setMaxTotal(20);
        return jedisPoolConfig ;
    }

    @Bean
    public JedisPool jedisPool (@Autowired JedisPoolConfig jedisPoolConfig){
        return new JedisPool(jedisPoolConfig,"127.0.0.1",6379) ;
    }

    @Bean
    public Jedis jedis (@Autowired JedisPool jedisPool){
        return jedisPool.getResource() ;
    }
}

Problem description

During the actual system operation, the following situations may occur: after thread 01 obtains the lock, the process is suspended, and the subsequent execution is not executed. After the lock fails, thread 02 obtains the lock again. After the database is updated, thread 01 recovers. At this time, the state after holding the lock will easily lead to data disorder after continued execution.

At this time, the concept of lock version needs to be introduced. Suppose that thread 01 obtains lock version 1. If it is not executed, thread 02 obtains lock version 2. After execution, through the comparison of lock versions, the lock version of thread 01 is too low, and the data update will fail.

CREATE TABLE `dl_data_lock` (
    `id` INT (11) NOT NULL AUTO_ Increment comment 'primary key ID',
    `Inventory ` int (11) default '0' comment 'inventory',
    `lock_ Value ` int (11) not null default '0' comment 'lock version',
    PRIMARY KEY (`id`)
)Engine = InnoDB default charset = utf8 comment = 'lock mechanism table';

Description: lock_ Value is the record lock version as a condition for controlling data update.

<update id="updateByLock">
    UPDATE dl_data_lock SET inventory=inventory-1,lock_value=#{lockVersion}
    WHERE id=#{id} AND lock_value &lt;#{lockVersion}
</update>

Note: the update operation here not only requires the thread to obtain the lock, but also determines that the thread lock version cannot be lower than the latest lock version in the current update record.

2. Optimistic lock

Mechanism description

Optimistic locks are mostly controlled based on data records. When updating the database, it is judged based on the pre query conditions. If the queried data has not been modified, the update operation is successful. If the pre query results are not valid as the update conditions, the data write fails.

Process Mapping

Architecture design | peak shaving of high concurrent traffic and locking mechanism of shared resources

code implementation

In the business process, first query the records to be updated, and then take the read columns as the update conditions.

@Override
public Boolean updateByInventory(Integer id) {
    DataLockEntity dataLockEntity = dataLockMapper.getById(id);
    if (dataLockEntity != null){
        return dataLockMapper.updateByInventory(id,dataLockEntity.getInventory())>0 ;
    }
    return false ;
}

For example, if you want to update the inventory, take the read inventory data as the update condition. If the read inventory is 100 and the inventory changes during the update, the update condition naturally cannot be established.

<update id="updateByInventory">
    UPDATE dl_data_lock SET inventory=inventory-1 WHERE id=#{id} AND inventory=#{inventory}
</update>

5、 Distributed services

1. Service protection

When dealing with high concurrency seckill scenarios, the service often hangs up. It is common for some app marketing pages to prompt the loss of active pages, but it does not affect the operation of the overall application. This is the service isolation and protection mechanism.

Based on the distributed service structure, the highly concurrent business services can be independent, and the overall service will not be affected due to the suspension of the second kill service, resulting in the scenario of service avalanche.

2. Database protection

Database protection and service protection complement each other. Under the distributed service architecture, the service and database correspond. In theory, the second kill service corresponds to the second kill database, which will not lead to the downtime of the whole database because the second kill database hangs up.

6、 Source code address

GitHub · address
https://github.com/cicadasmile/data-manage-parent
Gitee · address
https://gitee.com/cicadasmile/data-manage-parent

Architecture design | peak shaving of high concurrent traffic and locking mechanism of shared resources

Recommended reading: architecture design series, radish and green vegetables, each needs its own

Serial number title
00 Architecture design: single service, cluster, distributed, basic differences and connections
01 Architecture design: Global ID generation strategy in distributed business system
02 Architecture design: distributed system scheduling, zookeeper cluster management
03 Architecture design: interface idempotency principle, anti duplicate submission token management
04 Architecture design: cache management mode, monitoring and memory recycling strategy
05 Architecture design: asynchronous processing flow, detailed explanation of various implementation modes