High concurrency and multithreading


1、 What is high concurrency

High concurrency is a problem encountered in the process of system operation“A large number of operation requests are encountered in a short time”The situation mainly occurs inA large number of requests are received in web system(e.g. ticket snatching of 12306; Tmall double 11. The occurrence of this situation will cause the system to perform a large number of operations in this period of time, such as resource requests, database operations and so on.

2、 High concurrency processing index

Some common indicators related to high concurrency are: response time, throughput, query rate per second, QPS, number of concurrent users
1. Response time

Response time: the time when the system responds to a request. For example, the system needs 200ms to process an HTTP request, which is the response time of the system

2. Throughput

Throughput: the number of requests processed per unit time.

3. QPS (query per second)

QPS: response requests per second. In the field of Internet, the difference between this indicator and throughput is not so obvious.

4. Number of concurrent users

Number of concurrent users: the number of users carrying normal system functions at the same time. For example, in an instant messaging system, the number of simultaneous users represents the number of concurrent users to a certain extent.

3、 The relationship and difference between high concurrency and multithreading

“High concurrency and multithreading” are always mentioned together. It seems that they are equal, but in fact they are notHigh concurrency ≠ multithreading

3.1 multithreading

  • Multithreading is a feature of Java, because CPU is multi-core and multi-threaded, which can execute several tasks at the same time. In order to improve the execution efficiency of JVM, Java provides this mechanism of multithreading to enhance the efficiency of data processing.
  • Multithreading corresponds to CPU, and high concurrency corresponds to access request,You can use a single thread to process all access requests, or you can use multiple threads to process access requests at the same time.
  • In the past single CPU era, a single task can only execute a single program at a time point. Later, it developed to the stage of multi task, and the computer can execute multi task or multi process in parallel at the same time. Although it is not the real sense of “the same point in time”, it isMultiple tasks or processes share a CPU, and the operating system is responsible for switching the CPU between multiple tasks, so that each task has a chance to get a certain time slice to run
  • Later, it developed to multithreading technology, which makes it possible to have multiple threads to execute in parallel within a program. The execution of a thread can be considered as a CPU executing the program. When a program runs under multithreading, it is as if there are multiple CPUs executing the program at the same time.
  • In short, multithreading can be understood as follows:Multithreading is a programming method to deal with high concurrency, that is, concurrency needs to be realized by multithreading.

3.2 high concurrency

  • High concurrency is not the exclusive thing of Java. It is a language independent and generalized concept proposed to provide better Internet services. Typical scenes, such as 12306 grabbing train tickets, tmall double 11 second killing activities, etc.
  • The occurrence of this situation will cause the system to perform a large number of operations in this period of time, such as resource requests, database operations and so on. If the high concurrency processing is not good, it will not only reduce the user’s experience (request response time is too long), but also may lead to system downtime, serious even lead to oom exception, system stop working and so on

 * If we want the system to adapt to the high concurrency state, we need to optimize the system from all aspects,Hardware, network, system architecture, development language selection, data structure application, algorithm optimization, database optimizationMultithreading is just one of the solutions.

4、 Multithread concurrency Technology

High concurrency and multithreading

Java multithreading programming will involve the following technical points:
1. Three elements of concurrent programming (volatile)
AtomicityA particle that can no longer be divided. In Java, atomicity means that one or more operations either all succeed or all fail.
Orderliness: the sequence of program execution is in the sequence of code execution( Processor may reorder instructions)
visibility : when multiple threads access the same variable, if one thread modifies it, other threads can get the latest value immediately.
2、   Five states of thread
High concurrency and multithreading
Creation status: when creating a thread with the new operator
Ready state: to call the start method, the thread in the ready state may not immediately execute the run method, but also need to wait for the CPU to schedule
running state : the CPU starts scheduling threads and executing the run method
Blocking state: the thread is blocked for some reasons, such as calling the sleep method, trying to get a lock, etc
State of death: run method finished executing or encountered an exception during execution
Specific thread declaration cycle diagram:
High concurrency and multithreading
3. Pessimistic lock and optimistic lock
Pessimistic lock: every operation will lock, which will cause thread blocking(synchronized)。
Optimistic lock: each operation does not lock, but assumes that there is no conflict to complete an operation. If the operation fails due to the conflict, it will be retried until it succeeds, which will not cause thread blocking.
4. Cooperation between threads
The cooperation between threads includes: wait / notify / notifyAll / condition / lock, etc
5. Synchronized keyword
Synchronized is a key word in Java, which is a kind of synchronization lock. It modifies the following objects:
1)、Modify a code block: the modified code block is called synchronous statement block. Its scope of action is the code enclosed by braces {}, and its object is the object calling the code block
2)、Modify a method: the modified method is called synchronous method. Its scope of action is the whole method, and its object is the object calling this method
3)、Modify a static method: its scope of action is the entire static method, and the object of action is all objects of this class
4)、Modify a class: the scope of its function is the part enclosed in brackets after synchronized, and the main object is all objects of this class.
The full name of CAS isCompare And SwapIt is a technology to realize concurrent application. The operation contains three operands – memory location (V), expected original value (a), and new value (b). If the value of the memory location matches the expected original value, the processor automatically updates the location value to the new value. Otherwise, the processor does nothing.
There are three problems in CAS

  • ABA problem (atomicreference);
  • The cycle time is long and the cost is large;
  • Atomic integer and unsafe, which can only guarantee one shared variable;

7. Thread pool
If we use threads to create a thread, although it is simple, there are big problems. If there are a large number of concurrent threads, and each thread executes a short task, it will end. In this way, creating threads frequently will greatly reduce the efficiency of the system, because it takes time to create and destroy threads frequently. Thread pool reuse can greatly reduce the performance loss caused by frequent creation and destruction of threads.

5、 High concurrency technology solutions

High concurrency and multithreading

5.1. How to improve the high concurrency capability of the system?

1. Static resources combined with CDN (content delivery network) to solve the problem of image file access
2. Distributed cache: redis, memcached, etc.
3. Message queue middleware: rocketmq, ActiveMQ, Kafka, etc., which can handle a large number of messages asynchronously.
4. Application splitting: a project is split into multiple project deployment, and RPC (micro service) is used to solve the communication between multiple projects.
5. Database vertical split and horizontal split (sub database sub table), etc.
6. Database read and write separation, to solve the problem of big data query.
7. Using NoSQL, such as mongodb combined with MySQL.
8. Establish service degradation and current limiting mechanism in the case of big data access.