2 W words long article takes you in-depth understanding of thread pool

Time:2021-3-9

preface

Thread pool can be said to be a necessary knowledge point for Java advancement, and it is also a necessary test point for interview. Many people may be able to explain the working principle of thread pool after reading this article, but this is far from enough. If they meet more experienced interviewers and continue to ask questions, they may be hanged. Consider the following questions:

  1. What’s the difference between Tomcat’s thread pool implementation and JDK’s thread pool implementation? Is there a Tomcat like thread pool implementation in Dubbo?
  2. Our gateway Dubbo call thread pool once had such a problem: during pressure test, the interface can return normally, but the RT of the interface is very high. Suppose the core thread size is 500, the maximum thread size is 800, and the buffer queue is 5000. Can you find some problems in this setting and tune these parameters?
  3. Are there really core threads and non core threads in the thread pool?
  4. After the thread pool is shut down, can new threads be generated?
  5. Must the thread return immediately after it throws the task to the thread pool?
  6. Will new threads be added after thread exceptions in the thread pool, and how to catch the exceptions thrown by these threads?
  7. How to set the size of thread poolDynamic settingsParameters of thread pool
  8. How about the state machine of thread pool?
  9. Why does Ali java code specification not allow the use of executors to quickly create thread pools?
  10. What problems should be avoided when using thread pool? Can we simply talk about the best practice of thread pool?
  11. How to close thread pool gracefully
  12. How to monitor thread pool

I believe many people will be confused when they look at these problems

2 W words long article takes you in-depth understanding of thread pool

In fact, most of the answers to these questions are hidden in the source code of thread pool, so it is very important to have an in-depth understanding of the source code of thread pool. In this chapter, we will learn the source code of thread pool. I believe most of the above questions can be answered after reading it. We will also discuss some other questions with you in this article.

This article will introduce the principle of thread pool from the following aspects.

  1. Why use thread pool
  2. How does thread pool work
  3. Two ways for thread pool to submit tasks
  4. Source code analysis of ThreadPoolExecutor
  5. Answer the opening question
  6. Thread pool best practices
  7. summary

I believe you will have a better understanding of the thread pool after reading it. Liver text is not easy. Don’t finish the three links after reading it.

Why use thread pool

As mentioned above, there are three major costs for creating threads, as follows:

1. In fact, the thread model in Java is based on the native thread model of the operating system, that is, Java In fact, threads in are implemented based on kernel threads. System calls are required to create, destruct and synchronize threads. System calls need to switch back and forth between user mode and kernel mode, which is relatively expensive. The life cycle of threads includes “thread creation time”, “thread execution time”, “thread execution time”, and so on, “Thread destroy time”. System calls are required for creation and destruction. 2. Each thread needs to be supported by a kernel thread, which means that each thread needs to consume a certain amount of kernel resources (such as the stack space of the kernel thread). Therefore, the threads that can be created are limited. By default, the thread stack size of a thread is 1 m, and there is a graph and a truth

2 W words long article takes you in-depth understanding of thread pool

As shown in the figure, in Java 8, creating 19 threads (thread # 19) requires creating 19535 KB, that is, about 1 m. reserved means that if 19 threads are created, the operating system guarantees that it will allocate so much space (in fact, it is not necessarily allocated), while committed means the actual allocated space.

Voice over: note that this is the space occupied by threads in Java 8, but in Java 11, threads are greatly optimized. It only takes about 40 KB to create a thread, and the space consumption is greatly reduced

3. There are too many threads, which leads to the cost of context switching.

It can be seen that the creation of threads is expensive, so it is necessary to manage these threads in the form of thread pool. The thread size and management thread should be set reasonably in the thread pool to achieve the goal of high efficiencyReasonable creation thread size to maximize revenue and minimize riskFor developers, to complete tasks, they don’t need to care about how to create threads, how to destroy them, and how to cooperate. They only need to care about when the submitted tasks are completed. The thread pool can do all the trivial work, such as thread tuning and monitoring, so that developers can get a great relief!

The idea of thread pool is applied in many places, such as database connection pool and HTTP connection pool, which avoids the creation of expensive resources, improves performance and liberates developers.

ThreadPoolExecutor design architecture

First, let’s look at the design diagram of the executor framework

2 W words long article takes you in-depth understanding of thread pool

  • Executor: the top-level executor interface only provides an execute interface, which decouples the submitted task from the executed task. This method is the core and the focus of our source code analysis. This method is finally implemented by ThreadPoolExecutor,
  • Executorservice extends the interface of executor and implements the methods of terminating executors and submitting tasks individually or in batches
  • Abstractexecutorservice implements the executorservice interface and implements all methods except execute. Only the most important execute method is given to the ThreadPoolExecutor implementation.

Although there are many layers in this kind of layered design, each layer has its own function and clear logic, which is worth learning.

How does thread pool work

First, let’s look at how to create a thread pool

ThreadPoolExecutor threadPool = new ThreadPoolExecutor(10, 20, 600L,
                    TimeUnit.SECONDS, new LinkedBlockingQueue<>(4096),
                    new NamedThreadFactory("common-work-thread"));
//Set the rejection policy, which is "abortpolicy" by default
threadPool.setRejectedExecutionHandler(new ThreadPoolExecutor.AbortPolicy());

Let’s look at the construction method. The signature is as follows

public ThreadPoolExecutor(int corePoolSize,
                              int maximumPoolSize,
                              long keepAliveTime,
                              TimeUnit unit,
                              BlockingQueue<Runnable> workQueue,
                              ThreadFactory threadFactory,
                              RejectedExecutionHandler handler) {
            //Omitting some codes
}

To understand the meaning of these parameters, we must be clear about the process of thread pool submitting tasks and executing tasks, as follows

2 W words long article takes you in-depth understanding of thread pool

The picture is from meituan technical team

The steps are as follows

1. Corepoolsize: if the thread is still running after the task is submitted, when the number of threads is less than the corepoolsize value, no matter whether the threads in the thread pool are busy or not, the thread will be created and the task will be handed over to the newly created thread for processing. If the number of threads is less than or equal to corepoolsize, these threads will not be recycled unless allowcorethreadtimeout is set to True, but generally not, because frequent creation of destroy threads can greatly increase the overhead of system calls.

2. Workqueue: if the number of threads is greater than the number of cores (core pool size) and less than the maximum number of threads (maximum pool size), the task will be dropped to the blocking queue first, and then the thread will block the queue to pull the task execution.

3、maximumPoolSize: The maximum number of threads that can be created in the thread pool. If the queue is full when the task is submitted and the number of threads does not reach this set value, the thread will be created and the submitted task will be executed. If the queue is full when the task is submitted but the number of thread pools has reached this value, it means that the load capacity of the thread pool process has been exceeded, and the rejection policy will be executed. It’s understandable that we can’t let the thread pool run A steady stream of tasks come in and crush the thread pool. First of all, we need to ensure that the thread pool can work properly.

4. Rejectedexecutionhandler: there are four rejection policies

  • Abortpolicy: discards the task and throws an exception, which is also the default policy;
  • Callerrunspolicy: the caller’s thread is used to execute the task, so the initial question “must the thread return immediately after dropping the task to the thread pool?” we can answer that if we use the callerrunspolicy policy policy, the thread submitting the task (such as the main thread) can’t guarantee to return immediately after submitting the task. When the reject is triggered Strategy has to deal with the task itself.
  • Discardoldestpolicy: discards the top task in the blocking queue and executes the current task.
  • Discardpolicy: discards tasks directly without throwing any exceptions. This policy is only applicable to unimportant tasks.

5. Keepalivetime: the thread survival time. If the threads beyond the size of corepoolsize are in idle state within this time, they will be recycled

6. Threadfactory: you can use this parameter to set the name of the thread pool, specify the default uncaughtexceptionhandler (what’s the use, which will be explained later), and even set the thread as a guard thread.

Now the question is how to set these parameters reasonably.

Let’s start with the thread size settings

< Java Concurrent Programming Practice > > tells us that there should be two situations

  1. For CPU intensive tasks, when the size of the thread pool is NCU + 1, the system with NCU processors can usually achieve the optimal utilization rate. The reason for + 1 is that when the computing intensive thread pauses occasionally due to page failure or other reasons, the “extra” thread can also ensure that the CPU clock cycle will not be wasted, which is called CPU Dense means that threads are always busy. In this way, the size of thread pool is set to ncpu + 1 to avoid context switching of threads, make threads always busy, and maximize the utilization of CPU.
  2. For IO intensive tasks, it also gives the following formula

2 W words long article takes you in-depth understanding of thread pool

Just take a look at these formulas. They can’t be used in actual business scenarios. These formulas are too theoretical and can only be used as a theoretical reference. For example, when you say that the size of thread pool is n + 1 for CPU intensive tasks, but in fact there is often more than one thread pool in business. In this case, the formula is too confusing

2 W words long article takes you in-depth understanding of thread pool


Let’s look at the size setting of workqueue

As can be seen from the above, if the maximum number of threads is greater than the number of core threads, new threads will be added only when the core thread is full and the workqueue is full. That is to say, if the workqueue is an unbounded queue, new threads will never be added when the number of threads is increased to corepoolsize, that is, the maximum poolsize The settings of are invalid, and the rejectedexecutionhandler rejection policy cannot be triggered. Tasks will only be continuously filled into the workqueue until oom.

2 W words long article takes you in-depth understanding of thread pool


So the workqueue should be a bounded queue, at least to ensure that the thread pool can work normally when the task is overloaded. Then which are bounded queues and which are unbounded queues.

Bounded queue we often use the following two methods

  • Linkedblockingqueue: a bounded queue composed of linked lists, which arranges the elements in the order of first in first out (FIFO). However, when creating, you need to specify its size, otherwise its size is set to zero by default Integer.MAX_ Value, which is equivalent to an unbounded queue
  • Arrayblockingqueue: array implementation of the bounded queue, according to the first in first out (FIFO) order of the elements.

We often use priority blocking queue When a task is inserted into this priority queue, its weight can be specified to give priority to the execution of these tasks. However, this queue is rarely used. The reason is very simple. The execution order of tasks in the thread pool is generally equal. If there are some types of tasks that need to be executed first, it’s better to open another thread pool to separate different task types from different thread pools In recent years, it is also a practice of rational use of thread pool.

Speaking of this, I believe you should be able to answer the question at the beginning: “why does Ali java code specification not allow the use of executors to quickly create thread pools? In addition, the following two most common creation methods are used

2 W words long article takes you in-depth understanding of thread pool


The maximum number of threads for the newcachedthreadpool method is set to Integer.MAX_ When the newsinglethreadexecution method creates a workqueue, the linkedblockingqueue does not declare its size, which is equivalent to creating an unbounded queue. If you are not careful, it will lead to oom.

How to set threadfactory

In general business, there are multiple thread pools. If there is a problem with a thread pool, it is very important to locate which thread has a problem, so it is necessary to give each thread pool a name. Our company uses Dubbo’s namedthreadfactory to generate threadfactory, which is very simple to create

new NamedThreadFactory("demo-work")

Its implementation is very ingenious. You can see its source code for interest. Every time it is called, a counter at the bottom layer will add one, which will be named “demo-work-thread-1”, “demo-work-thread-2” and “demo-work-thread-3” successively.

In the actual business scenario, it is difficult to determine the core pool size, work queue and maximum pool size If there is a problem with the size of the thread pool, generally speaking, these parameters can only be reset and then released, which often takes some time. This article of meituan gives a bright solution. When a problem (thread pool monitoring alarm) is found, these parameters can be dynamically adjusted to make these parameters take effect in real time, and can be solved in time when a problem is found, so as to ensure the reliability It’s a good idea.

Two ways for thread pool to submit tasks

After the thread pool is created, there are two ways to submit tasks to it: call the execute and submit methods. Let’s take a look at the method signatures of these two methods

//Method 1: Execute method
public void execute(Runnable command) {
}
//Mode 2: three methods of submit in executorservice
<T> Future<T> submit(Callable<T> task);
<T> Future<T> submit(Runnable task, T result);
Future<?> submit(Runnable task);

The difference is that the call to execute has no return value, while the call to submit can return future. What can future do? Look at its interface

public interface Future<V> {
    /**
     *Cancels the task being executed. If the task has been executed or cancelled, or cannot be cancelled for some reason, it returns "false"
     *If the task is not started or the task has started but can be interrupted (mayinterruptif running is true), then
     *You can cancel / interrupt this task
     */
    boolean cancel(boolean mayInterruptIfRunning);
    /**
     *Has the task been cancelled before completion
     */
    boolean isCancelled();
    /**
     *If the process is executed normally, or an exception is thrown, or the task caused by cancellation is completed, it will return "true"
     */
    boolean isDone();
    /**
     *Block the result of waiting task
     */
    V get() throws InterruptedException, ExecutionException;
    /**
     *Block the result of waiting task,不过这里指定了时间,如果在 timeout 时间内任务还未执行完成,
     *The exception "timeoutexception" is thrown
     */
    V get(long timeout, TimeUnit unit)
        throws InterruptedException, ExecutionException, TimeoutException;
}

You can use future to cancel the task, judge whether the task has been cancelled / completed, or even block and wait for the result.

Why can submit not only submit a runnable task, but also return the execution result of a future task

2 W words long article takes you in-depth understanding of thread pool


It turns out that the task is encapsulated as runnablefuture by newtaskfor before the final execution, and the class futuretask is returned by newtaskfor. The structure is as follows

2 W words long article takes you in-depth understanding of thread pool


We can see that futuretask interface implements both runnable interface and future interface, so when submitting tasks, we can also use future interface to cancel tasks, get task status and wait for execution results.

In addition to the difference of whether the execution result can be returned, there is another important difference between execute and submit, that is, if an exception occurs, it cannot be caught. By default, the uncaughtexception method of ThreadGroup will be executed (the logic corresponding to number 2 in the figure below)

2 W words long article takes you in-depth understanding of thread pool

Therefore, if you want to monitor the exceptions when executing the execute method, you need to specify an uncaughtexceptionhandler through threadfactory. In this way, 1 in the above figure will be executed, and then the logic in uncaughtexceptionhandler will be executed, as shown below:

//1. Implement a thread pool factory
ThreadFactory factory = (Runnable r) -> {
    //Create a thread
    Thread t = new Thread(r);
    //Set the uncaughtexceptionhandler object to the created thread to implement the default logic of exception
    t.setDefaultUncaughtExceptionHandler((Thread thread1, Throwable e) -> {
        //Set statistical monitoring logic here
        System.out.println ("exceptionhandler set by thread factory" + e.getmessage());
    });
    return t;
};
//2. Create a self-defined thread pool and use the self-defined thread factory
ExecutorService service = new ThreadPoolExecutor(1, 1, 0, TimeUnit.MILLISECONDS,new LinkedBlockingQueue(10),factory);
//3. Submit task
service.execute(()->{
    int i=1/0;
});

Executing the above logic will eventually output “exceptionhandler / by zero” set by the thread factory. In this way, we can execute our monitoring logic through the default uncaughtexceptionhandler.

If you use submit, how can you catch an exception future.get You can capture it

Callable testCallable = xxx;
Future future = executor.submit(myCallable);
try {
    future1.get(3));
} catch (InterruptedException e) {
    e.printStackTrace();
} catch (ExecutionException e) {
    e.printStackTrace();
}

So why does future capture asynchrony only when it gets? Because when it throws an exception when it executes submit, the exception is saved, but it is thrown when it gets

2 W words long article takes you in-depth understanding of thread pool


Why God’s article on the execution process of execute and submit is very thorough, so I don’t want to pick up the wisdom of others. I suggest you taste it well, and the harvest will be great!

Source code analysis of ThreadPoolExecutor

After so much foreshadowing, we finally come to the core of the source code analysis.

For thread pool, we are most concerned about its “state” and “number of runnable threads”. Generally speaking, we can choose to use two variables to record. However, Doug lea only uses one variable (CTL) to achieve its goal. We know that the more variables there are, the worse the maintainability of the code and the easier it is to get bugs, So only one variable can achieve the effect of two variables, which greatly improves the maintainability of the code. So how did he design it

// ThreadPoolExecutor.java
public class ThreadPoolExecutor extends AbstractExecutorService {
    private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
    private static final int COUNT_BITS = Integer.SIZE - 3;
    private static final int CAPACITY   = (1 << COUNT_BITS) - 1;
    //Results: 111 million million
    private static final int RUNNING    = -1 << COUNT_BITS;
    //Results: 000 million million million
    private static final int SHUTDOWN   =  0 << COUNT_BITS;
    //Results: 001 million million million million
    private static final int STOP       =  1 << COUNT_BITS;
    //Results: 010 million million million
    private static final int TIDYING    =  2 << COUNT_BITS;
    //Results: 011 million million million
    private static final int TERMINATED =  3 << COUNT_BITS;
    //Gets the state of the thread pool
    private static int runStateOf(int c)     { return c & ~CAPACITY; }
    //Get the number of threads
    private static int workerCountOf(int c)  { return c & CAPACITY; }
}

As you can see, CTL is an integer variable of atomic class, with 32 bits. The lower 29 bits represent the number of threads, and the maximum 29 bits represent (2 ^ 29) – 1 (about 500 million), which is enough to record the thread size. If it is not enough in the future, CTL can be declared as atomiclong, the upper 3 bits represent the state of thread pool, and the 3 bits represent 8 Because there are only five states in the thread pool, three bits are enough. The five states of the thread pool are as follows

  • Running: receive new tasks and continue to process tasks in workqueue
  • Shutdown: no longer receive new tasks, but continue to process tasks in workqueue
  • Stop: no longer receive new tasks, no longer process tasks in workqueue, and will interrupt the thread that is processing tasks
  • Tidying: when all tasks are finished and the number of threads (workcount) is 0, it is in this state. After entering this state, the hook method terminated () will be called to enter the terminated state
  • Terminated: This is the state after calling the terminated() method

The state flow and trigger conditions of thread pool are as follows

2 W words long article takes you in-depth understanding of thread pool

With these foundations, let’s analyze the source code of execute

public void execute(Runnable command) {
    if (command == null)
        throw new NullPointerException();
    int c = ctl.get();
    //If the current number of threads is less than the number of core threads (corepoolsize), no matter whether the core thread is busy or not, the thread will be created until it reaches "corepoolsize"
    if (workerCountOf(c) < corePoolSize) {
        //Create a thread and give the task to "worker" (in this case, the task is "firsttask" in "worker")
        if (addWorker(command, true))
            return;
        c = ctl.get();
    }
    //If the thread pool is in "running" state and the number of threads is greater than "corepoolsize" or 
    //If the number of threads is less than "corepoolsize", but the creation of thread fails, the task will be thrown into "workqueue"
    if (isRunning(c) && workQueue.offer(command)) {
        int recheck = ctl.get();
        //Here, we need to check again whether the thread pool is in the "running" state, because the thread pool state may change after the task is queued (for example, the "shutdown" method is called). If the thread state changes, the task will be removed and the rejection policy will be executed
        if (! isRunning(recheck) && remove(command))
            reject(command);
        //If the thread pool is in the "running" state and the number of threads is less than 0, the new thread will speed up the processing of tasks in "workqueue"
        else if (workerCountOf(recheck) == 0)
            addWorker(null, false);
    }
    //This logic indicates that if the number of threads is greater than "corepoolsize" and the task fails to queue, the thread will be created with the maximum number of threads (maximumpoolsize) as the bound. If it fails, it means that the number of threads exceeds "maximumpoolsize", then the rejection policy will be executed
    else if (!addWorker(command, false))
        reject(command);
}

As you can see from this code, creating a thread is realized by calling addworker. Before analyzing addworker, it is necessary to briefly mention worker. The thread pool encapsulates every thread executing a task in the form of worker. It is very vivid to call worker. The essence of thread pool is the producer consumer model. The producer constantly throws tasks into workqueue, and workqueue is called worker Just like an assembly line, it continuously transports tasks, while the worker continuously takes tasks to execute

2 W words long article takes you in-depth understanding of thread pool


So the problem is, why encapsulate the thread in the worker? After the thread pool gets the task, it will be directly thrown to the thread for processing, or let the thread go to the workqueue to process it?

The main purpose of encapsulating thread as worker is to better manage thread interrupt

Let’s look at the definition of worker

//It can be seen here that "worker" is not only a "runnable" task, but also implements "AQS" (in fact, it implements an exclusive lock with "AQS", so that "worker" will be locked when it runs. When it executes "shutdown", "setcorepoolsize", "setmaximumpoolsize" and other methods, it will try to interrupt the thread (interruptidleworkers), and in this method, it will try to obtain "wor" first If the lock is not successful, it means that the worker is running. At this time, the worker will finish the task first, and then close the worker's thread to achieve the purpose of closing the thread gracefully.)
private final class Worker
    extends AbstractQueuedSynchronizer
    implements Runnable
    {
        private static final long serialVersionUID = 6138294804551838833L;
        //The thread that actually performs the task
        final Thread thread;
        //As mentioned above, if the current number of threads is less than the number of core threads, create a thread and hand over the submitted task to "worker" for processing, then "firsttask" is the submitted task. If "worker" gets the task from "workqueue", then "firsttask" is empty
        Runnable firstTask;
        //Count the number of tasks completed
        volatile long completedTasks;
        Worker(Runnable firstTask) {
            //Initialize to - 1, so interrupt is forbidden before thread running (call runworker). In the "interruptifstarted() method," getstate() > = 0 "will be judged
            setState(-1); 
            this.firstTask = firstTask;
            //Create a thread according to the thread pool's "threadfactory" and pass the "worker" itself to the thread (because the "worker" implements the "runnable" interface)
            this.thread = getThreadFactory().newThread(this);
        }
        public void run() {
            //This method is called when thread is started
            runWorker(this);
        }
       
        //1 ﹣ for locked, 0 ﹣ for unlocked
        protected boolean isHeldExclusively() {
            return getState() != 0;
        }
        //Attempt to acquire lock
        protected boolean tryAcquire(int unused) {
            //From here, we can see that it is an exclusive lock, because after obtaining the lock, CAS can not set "state". Here we can also understand the function of setting "state" to - 1. In this case, it is never possible to obtain the lock, and "worker" must obtain the lock before it is interrupted
            if (compareAndSetState(0, 1)) {
                setExclusiveOwnerThread(Thread.currentThread());
                return true;
            }
            return false;
        }
        //Attempt to release lock
        protected boolean tryRelease(int unused) {
            setExclusiveOwnerThread(null);
            setState(0);
            return true;
        }    
        public void lock()        { acquire(1); }
        public boolean tryLock()  { return tryAcquire(1); }
        public void unlock()      { release(1); }
        public boolean isLocked() { return isHeldExclusively(); }
            
        //To interrupt a thread, this method will be called by "shutdown now". It can be seen that "shutdown now" does not need to obtain a lock to interrupt a thread. In other words, if a thread is running, it will still be interrupted for you. Therefore, generally speaking, we don't need "shutdown now" to interrupt a thread. It's too rude. When the thread is interrupted, it is likely that it is executing a task, affecting the execution of the task
        void interruptIfStarted() {
            Thread t;
            //Interrupt is also conditional. It must be "state > = 0" and "t! = null" and the thread is not interrupted
            //If ﹣ state = = - 1, the interrupt is not executed, and the meaning of ﹣ setstate (- 1) is understood again
            if (getState() >= 0 && (t = thread) != null && !t.isInterrupted()) {
                try {
                    t.interrupt();
                } catch (SecurityException ignore) {
                }
            }
        }
    }

Through the above analysis of the worker class, I believe it is not difficult to understandThe main purpose of encapsulating thread as worker is to better manage thread interruptThat’s a sentence.

After understanding the meaning of worker, let’s look at the method of addworker

private boolean addWorker(Runnable firstTask, boolean core) {
    retry:
    for (;;) {
        int c = ctl.get();
        //Gets the state of the thread pool
        int rs = runStateOf(c);
        //If the state of thread pool > = shutdown, that is, "shutdown, stop, tidying, terminated", there is only one case where it is possible to create a thread, that is, "shutdown", When the queue is not empty, firsttask = = null means to create a thread that does not receive new tasks (this thread will get tasks from "workqueue" and execute them again). In this case, the thread is created to speed up the processing of tasks in "workqueue"
        if (rs >= SHUTDOWN &&
            ! (rs == SHUTDOWN &&
               firstTask == null &&
               ! workQueue.isEmpty()))
            return false;
        for (;;) {
            //Get the number of threads
            int wc = workerCountOf(c);
            //If it exceeds the maximum capacity of the thread pool (more than 500 million, basically impossible)
            //Or ⁃ exceeds ⁃ corepoolsize (core ⁃ is ⁃ true) or ⁃ maximumpoolsize (core ⁃ is ⁃ false)
            //Return false
            if (wc >= CAPACITY ||
                wc >= (core ? corePoolSize : maximumPoolSize))
                return false;
            //Otherwise, "CAS" will increase the number of threads, and if you successfully jump out of the double loop
            if (compareAndIncrementWorkerCount(c))
                break retry;
            c = ctl.get();  // Re-read ctl
            //If the thread running state changes, jump to the outer loop to continue execution
            if (runStateOf(c) != rs)
                continue retry;
            //The reason is that "CAS" fails to increase the number of threads, and continues to execute the inner loop of "retry"
        }
    }
    boolean workerStarted = false;
    boolean workerAdded = false;
    Worker w = null;
    try {
        //If it can be executed here, it means that the conditions for adding "worker" are met, so "worker" is created and ready to be added to the thread pool to execute tasks
        w = new Worker(firstTask);
        final Thread t = w.thread;
        if (t != null) {
            //Locking is because "W" will be added to "workers" below. Workers "is" HashSet "and is not thread safe, so it needs to be guaranteed by locking
            final ReentrantLock mainLock = this.mainLock;
            mainLock.lock();
            try {
                //Check the state of thread pool again to prevent interruption when executing this step
                int rs = runStateOf(ctl.get());
                //If the thread pool state is less than "shutdown" (i.e., "running),",
                //Or if the status is "shutdown" but "firsttask = = null (it means that the task is not received, but only the task in" workqueue "is created), the condition of adding" worker "is satisfied
                if (rs < SHUTDOWN ||
                    (rs == SHUTDOWN && firstTask == null)) {
                                        //If the thread has been started, there is obviously a problem (because the thread has not been started since the "worker" was created), an exception is thrown
                    if (t.isAlive()) 
                        throw new IllegalThreadStateException();
                    workers.add(w);
                    int s = workers.size();
                    //Record the maximum thread pool size for monitoring
                    if (s > largestPoolSize)
                        largestPoolSize = s;
                    workerAdded = true;
                }
            } finally {
                mainLock.unlock();
            }
            //This indicates that adding "worker" to "workers" is successful. At this time, start the thread
            if (workerAdded) {
                t.start();
                workerStarted = true;
            }
        }
    } finally {
        //Failed to add thread, execute the "addworkerfailed" method, mainly remove "worker" from "workers", reduce the number of threads, and try to close the thread pool
        if (! workerStarted)
            addWorkerFailed(w);
    }
    return workerStarted;
}

From this code, we can see the unpredictability of multithreading. We find that when the conditions are met, we check the thread state again to prevent interruption and other thread pool state changes. This also gives us inspiration: all kinds of critical conditions in multithreading environment must be considered in place.

After the addworker is successfully created, the thread starts to execute (t.start()). Since the worker itself is passed to the thread when the worker is created, the run method of the worker will be called after the thread is started

public void run() {
    runWorker(this);
}

You can see that the runworker method will eventually be called. Next, let’s analyze the runworker method

final void runWorker(Worker w) {
    Thread wt = Thread.currentThread();
    Runnable task = w.firstTask;
    w.firstTask = null;
    //Unlock will call the "tryrelease" method to set the "state" to "0", which means that interrupts are allowed. We have mentioned above in "interruptifstarted(), that is," state > = 0 "
    w.unlock();
    boolean completedAbruptly = true;
    try {
        //If a thread is created when a task is submitted and the task is thrown to it, the task is executed first
        //Otherwise, get "task" from the task queue to execute (that is, "gettask() method)
        while (task != null || (task = getTask()) != null) {
            w.lock();
            
            //If the thread pool state is > = stop (that is, < stop, tidying, terminated), the thread should be interrupted
            //If the thread pool state is < stop, the thread should not be interrupted( Thread.interrupted () returns ﹣ true, and clears the flag bit), and judges the thread pool state again (to prevent the execution of ﹣ shutdown now() when clearing the flag bit). If the thread pool is ﹣ stop at this time, the thread will be interrupted
            if ((runStateAtLeast(ctl.get(), STOP) ||
                 (Thread.interrupted() &&
                  runStateAtLeast(ctl.get(), STOP))) &&
                !wt.isInterrupted())
                wt.interrupt();
            try {
                //Before performing tasks, subclasses can implement this hook method for statistical purposes
                beforeExecute(wt, task);
                Throwable thrown = null;
                try {
                    task.run();
                } catch (RuntimeException x) {
                    thrown = x; throw x;
                } catch (Error x) {
                    thrown = x; throw x;
                } catch (Throwable x) {
                    thrown = x; throw new Error(x);
                } finally {
                    //After executing the task, the subclass can implement this hook method for statistics
                    afterExecute(task, thrown);
                }
            } finally {
                task = null;
                w.completedTasks++;
                w.unlock();
            }
        }
        completedAbruptly = false;
    } finally {
        //If there are only two possibilities, one is that the execution process is interrupted abnormally, and the other is that there are no tasks in the queue. From this, we can see that there is no difference between core threads and non core threads. This method will be executed when any task is abnormal or exits normally. This method will reduce the number of threads to - 1 according to the situation
        processWorkerExit(w, completedAbruptly);
    }
}

Let’s see what the processworkerexit method looks like

private void processWorkerExit(Worker w, boolean completedAbruptly) {
        //If it exits abnormally, CAS ﹣ executes the operation of ﹣ 1 ﹣ in the line pool
    if (completedAbruptly) 
        decrementWorkerCount();
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        completedTaskCount += w.completedTasks;
        //Locking ensures that the thread safely removes the worker 
        workers.remove(w);
    } finally {
        mainLock.unlock();
    }
    //Since woker exits abnormally, the thread pool state may have changed (such as "shutdown", etc.), so try to close the thread pool
    tryTerminate();
    int c = ctl.get();
    //If the thread pool is in "stop" state, if "woker" exits abnormally, a new "woker" will be added. If it exits normally, at least one thread will be running to execute the task in "wokerqueue" under the condition that "wokerqueue" is not empty

Next, we analyze the method of woker getting tasks from workqueue, gettask

private Runnable getTask() {
    boolean timedOut = false; // Did the last poll() time out?
    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);
        //If the thread pool state is at least "stop" or
        //Thread pool state = = shutdown and task queue is empty
        //Then reduce the number of threads and return "null". In this case, the "runworker" analyzed above will execute "processworkerexit", so that the "woker" who obtains this task will exit
        if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
            decrementWorkerCount();
            return null;
        }
        int wc = workerCountOf(c);
        //If "allowcorethreadtimeout" is "true", any thread in "idle" state during "keepalivetime" will be recycled. If the number of threads is greater than "corepoolsize", the thread in "idle" state during "keepalivetime" will be recycled
        boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
        //There are several conditions for worker to be recycled. This is relatively simple, so I'll skip it
        if ((wc > maximumPoolSize || (timed && timedOut))
            && (wc > 1 || workQueue.isEmpty())) {
            if (compareAndDecrementWorkerCount(c))
                return null;
            continue;
        }
        try {
           //If the task is not acquired within the time of "keepalivetime", it indicates that the task has timed out, and "timedout" is "true"
            Runnable r = timed ?
                workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
                workQueue.take();
            if (r != null)
                return r;
            timedOut = true;
        } catch (InterruptedException retry) {
            timedOut = false;
        }
    }
}

After the above source code analysis, I believe we have a very good understanding of the working principle of thread pool. Let’s go over some other useful methods. At the beginning, we mentioned the monitoring problem of thread pool. Let’s see what indicators can be monitored

  • Int getcorepoolsize(): gets the number of core threads.
  • Int getlargestpoolsize(): historical peak number of threads.
  • Int getmaximumpoolsize(): maximum number of threads (thread pool thread capacity).
  • Int getactivecount(): current number of active threads
  • Int getpoolsize(): the total number of threads in the current thread pool
  • Blockingqueuegetqueue() is the task queue of the current thread pool, according to which the total number of backlog tasks can be obtained, getQueue.size ()

The idea of monitoring is also very simple. Start a scheduled thread pool executor to collect these thread pool indicators on a regular basis. Generally, some open source tools such as grafana + Prometheus + micrometer are used.

How to preheat the core thread pool

Use the prestartallcorethreads() method. This method will create corepoolsize threads at one time. It doesn’t need to wait until the task is submitted. After the thread is submitted and created, these threads can process the task as soon as it is submitted.

How to dynamically adjust thread pool parameters

  • Setcorepoolsize (int corepoolsize) adjusts the core thread pool size
  • setMaximumPoolSize(int maximumPoolSize)
  • Setkeepalivetime() sets the thread lifetime

Answer the opening question

Other questions are basically answered in the section of source code analysis. Here are some other questions

1. What’s the difference between Tomcat’s thread pool and JDK’s thread pool implementation? Is there a Tomcat like thread pool implementation in Dubbo? There’s a thing called eagerthreadpool in Dubbo. You can see its instructions

2 W words long article takes you in-depth understanding of thread pool

It can be seen from the comments that if the core threads are in busy state and if new requests come in, eagerthreadpool will choose to create the thread first instead of putting it into the task queue, so that it can respond to these requests more quickly.

Tomcat implementation is similar to this, but slightly different. When Tomcat starts, minsparethreads threads will be created first. If these threads are busy after receiving requests for a period of time, they will be created in minsparethreads steps every time. In essence, it is also for faster response to processing requests. The specific source code can be seen in its ThreadPool implementation, which will not be expanded here.

2. Our gateway Dubbo call thread pool once had such a problem: during pressure test, the interface can return normally, but the RT of the interface is very high. Suppose the core thread size is 500, the maximum thread size is 800, and the buffer queue is 5000. Can you find some problems in this setting and tune these parameters? This parameter obviously shows the problem. First of all, the task queue is set too large. After the task reaches the core thread, if there are any more requests, it will enter the task queue first. After the queue is full, the thread will be created, which also requires a lot of overhead. So we later set the core thread to be the same as the largest thread, and call prestartallcorethreads () To warm up the core thread, you don’t have to create a thread when the request comes.

Some best practices of thread pool

1. The tasks executed by thread pool should be independent of each other. If they depend on each other, they may lead to deadlock, such as the following code

ExecutorService pool = Executors
  .newSingleThreadExecutor();
pool.submit(() -> {
  try {
    String qq=pool.submit(()->"QQ").get();
    System.out.println(qq);
  } catch (Exception e) {
  }
});

2. Core tasks and non core tasks are best separated by multiple thread pools

Once upon a time, there was such a fault in our business: suddenly, many users could not receive the message. After investigation, we found that the SMS was sent in a thread pool, and other timing scripts also used this thread pool to perform tasks. This script could generate hundreds of tasks a minute, which resulted in that the SMS method had no chance to be implemented in the thread pool We use two thread pools to separate sending text messages from executing scripts.

3. Add thread pool monitoring and set thread pool dynamically

As mentioned above, it is difficult to determine the parameters of the thread pool at one time. Since it is difficult to determine and to ensure that problems are solved in time, we need to increase monitoring for the thread pool, monitor the size of the queue and the number of threads. We can set 3 For example, if the queue task is always full within minutes, an alarm will be triggered, which can give an early warning. If the online thread pool parameter setting is unreasonable and the degradation and other operations are triggered, the number of core threads and the maximum number of threads can be modified in real time by dynamically setting the thread pool to fix the problem in time.

summary

This paper analyzes the working principle of thread pool in detail. I believe that we should have a deeper understanding of its working mechanism, and also have a clearer understanding of the first few problems. In essence, the purpose of setting thread pool is to maximize performance and minimize risk by using effective resources. At the same time, the use of thread pool is essentially to better serve users, which is not difficult Understand tomcat, Dubbo to set up their own thread pool.

Finally, I welcome you to my official account, Java, to discuss together and make progress together, and pull you into the readers group. Let’s warm up together.

After three things ❤️

========

If you think this content is quite helpful to you, I’d like to invite you to do me three small favors:

Like, forward, have your “praise and comment”, is my creation power.

Pay attention to the official account of “Java didi” and share original knowledge without any time.

At the same time, we can look forward to the following articles