Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

Time:2020-11-26

In every era, people who can learn will not be treated badly

Hello, I’m yes.

Recently, I saw the time wheel algorithm in Kafka. I remember I saw this thing in netty before. I didn’t pay much attention to it. Today, let’s see what the time wheel is.

Why do we use time round to implement the algorithm?

Isn’t timer provided by Java for delay operation?

Is there any delay queue with thread pool or scheduled ThreadPool?

Let’s take a brief look at the implementation of timer, delayqueue and scheduledthreadpool, and see how they implement delay tasks. There is no secret under the source code. Let’s take a look at why netty and Kafka deliberately implement time turns to handle delayed tasks.

If I read it on my mobile phone, I can describe it clearly without looking at the code. But it looks better on the computer.

Timer

Timer can implement delayed tasks or periodic tasks. Let’s take a look at timer core properties and constructors.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

The core is a priority queue and encapsulated thread to execute tasks. From this, we can also see that a timer has only one thread to execute tasks.

Let’s look at the periodicity and latency of the task. First of allIn a nutshellFirst of all, maintain a small top heap, that is, the fastest task to be executed is ranked first in the priority queue. According to the characteristics of the heap, we know that the time complexity of insert and delete is O (logn).

Then timerthread constantly compares the execution time of the first task in the queue with the current time. If the time is up, check whether the task is a periodic task. If so, modify the current task time to the next execution time. If not, remove the task from the priority queue. Finally, carry out the task. Call if the time is not upwait()Wait.

Take a look at the figure below and sort out the process.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

Once you know the process, look at the code, and this is about it. If you are not happy with the code, you can skip the code part, which has little effect.

Let’s take a look at the taskqueue first. It’s just a common heap insertion operation.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

Let’s look at the timerthreadrunOperation.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

To sum up

We can see that the timer actually maintains a priority queue according to the execution time of the task, and plays a thread to pull the task execution continuously.

What are the disadvantages?

firstThe time complexity of insertion and deletion of priority queue is O (logn)When the amount of data is large, the performance of frequent in and out of heap needs to be considered.

And it isSingle thread executionIf the execution time of one task is too long, the execution time of the next task will be affected (of course, the run of your task can be executed asynchronously).

And you can see from the codeThere is nothing to do with the exceptionWhen a task fails, the subsequent tasks cannot be executed.

ScheduledThreadPoolExecutor

Before we talk about the scheduled thread pool executor, let’s take a look at the timer’s comments. They are all dry goods. Don’t miss them. I made some changes to highlight the next point.

Java 5.0 introduced ScheduledThreadPoolExecutor, It is effectively a more versatile replacement for the Timer, it allows multiple service threads. Configuring with one thread makes it equivalent to Timer。

1.5 introduces the scheduledthreadpool executor, which is an alternative to the timer with more functions,Allow multiple service threads。 If you set up a service thread, there is no difference between it and timer.

From the comments, we can see that compared with timer, it may be the difference between single track running task and multi-threaded running task. Let’s take a look.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

It inherits ThreadPoolExecutor and implements scheduledexecutorservice. Qualitative operation is almost normal thread pool. The difference lies in two aspects: one is the scheduled future task and the other is the delayed work queue.

In fact, the delayed work queue is the priority queue, which is also a small top heap implemented by using arrays. The scheduledfuturetask inherits from futuretask and rewrites the run method to meet the requirements of periodic tasks.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

To sum up

The general process of the scheduledthreadpool executor is similar to that of timerMaintain a priority queueThen, the periodic task is implemented by rewriting the run method of task. The main difference is thatIt can run tasks in multiple threads without single thread blocking

Moreover, the java thread pool is set to eat the error if the task fails. thereforeA task error will not affect subsequent tasks

DelayQueue

There is also a delay queue in Java, and all elements added to the delay queue must implement the delayed interface. The internal delay queue is implemented by using PriorityQueue, so the priority queue is still used! The delayed interface inherits compatible, so priority queues are sorted by delay.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

Then let’s look at how the delayed queue gets the elements.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

To sum up

It is also implemented by using priority queue. The element returns the delay time by implementing the delayed interface. However, a delay queue is a container that requires other threads to get and execute tasks.

This is a summary of understanding timer, scheduledthreadpool and delayqueueThey are all through the priority queue to get the earliest task to be executedTherefore, the time complexity of inserting and deleting tasks is O (logn), and the periodic tasks of timer and scheduledthreadpool are completed by resetting the next execution time of tasks.

The problem lies in the time complexity. The insertion and deletion time complexity is O (logn). If the number of frequent inserts and deletions is m, the total time complexity is O (mlogn). This time complexity can not meet the performance requirements of Kafka middleware, while the insertion and deletion time complexity of time round algorithm is O (1). Let’s take a look at how the time wheel algorithm is implemented.

Time round algorithm

As the saying goes, art comes from life, and technology can also find inspiration from daily life. Let’s first look at a watch, um, a gold watch.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

See clearly, the time wheel is very similar to the existence of the watch clock. The time wheel is implemented by a circular array. Each element of the array can be called a slot, just like HashMap.

Inside the slot, bidirectional linked list is used to store the tasks to be executed. The operation time complexity of the added and deleted linked lists is O (1). The slot itself also refers to the time precision. For example, if one slot is scanned every second, the highest accuracy of this time wheel is 1 second.

In other words, tasks with a delay of 1.2 seconds and tasks of 1.5 seconds will be added to the same slot, and then traverse the linked list in this slot to execute tasks in 1 second.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

It can be seen from the figure that at this time, the pointer points to the first slot. There are eight slots 0 ~ 7. Assuming that the time unit of the slot is 1 second, now we want to add a task with a delay of 5 seconds. The calculation method is 5% 8 + 1 = 6, which is placed in the slot with slot position 6 and subscript 5. More specifically, it’s the end of the bidirectional list that goes to the slot.

Then the pointer moves clockwise one space per second, so that it sweeps to the next grid and traverses the bidirectional linked list in this cell to execute the task. Then recycling continues.

We can see that the time complexity of the insertion task from calculating slot to inserting linked list is O (1). What if you want to add a task that will be executed in 50 seconds? Doesn’t this slot seem to be enough? Is it necessary to add grooves? Expand as HashMap?

No, there are two common ways,One is through the concept of increasing rounds。 50% 8 + 1 = 3, that is, it should be placed in the position where the slot is 3 and the subscript is 2. Then (50 – 1) / 8 = 6, that is, the number of rounds is recorded as 6. That is to say, after 6 rounds of cycling, scanning to the slot of subscript 2 will trigger this task. This is what HashedWheelTimer in netty uses.

There is also a multi-level time wheelThis is more like our watch, like our second hand, minute hand, minute hand and hour hand.

This is how the multi-level time wheel is realized. If the above figure is the first floor, then the first floor goes around and the second floor goes one grid.

We can know that the second layer of a grid is 8 seconds, assuming that the second layer is also 8 slots, then if the second layer takes a circle and the third layer takes a grid, we can know that the third layer is 64 seconds.

Then a grid of three layers with eight slots per layer, a total of 24 slots, can handle tasks with a maximum delay of 512 seconds.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

If a task is delayed for 500 seconds, then it must be placed in the third layer at the beginning. When the time exceeds 436 seconds, it will take 64 seconds to trigger the execution of the task. At this time, it is relatively a delay of 64 seconds Second task, so it will be lowered in the second layer, the first layer can not put it.

In another 56 seconds, it will be a task with a delay of 8 seconds, so it will be demoted to the first layer and wait for execution.

The degradation is to ensure the consistency of time accuracy。 Kafka uses a multi-level time round algorithm.

Time wheel in netty

In netty, the implementation class of time wheel is HashedWheelTimer. The wheel in the code is the loop array of the picture above. The design of mask is the same as HashMap. By limiting the size of the array to the power of 2, the bit operation is used to replace the modular operation to improve the performance. Tick duration is the time of each grid, that is, the accuracy. You can see that a worker thread is equipped to handle the execution of the task.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

Let’s take a look at how tasks are added.

[the transfer of the external chain image failed. The source station may have anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-tvwwiazo-1596457022929)(https://upload-images.jianshu…]

We can see that the task is not directly added to the time wheel, but first enters an MPSC queue. I will briefly say that MPSC is a concurrent queue in jctools. It is used when multiple producers can access the queue at the same time, but only one consumer will access the queue. Limited space, interested friends to understand the implementation.

Then let’s look at how worker threads work.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

It’s very intuitive. Let’s take a look at how waitforexttick gets the next execution time.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

To put it simply, calculate the next time to check through tick duration and the number of times that have been ticked. Wait for sleep before the time is up.

Let’s look at how the task enters the slot.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

The annotation is very clear, and the implementation is consistent with the above analysis.

Finally, let’s look at how to implement it.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

That is, through the double judgment of the number of rounds and time, the removal task is completed.

To sum up

Generally speaking, the implementation of netty is the realization of the time round through the number of rounds mentioned above, which is completely consistent. It can be seen that the time precision is controlled by tickduration, and the worker thread does other operations besides processing the task when it is executed, so the task may not be executed accurately.

Moreover, if a new thread is not started or the task is thrown to the thread pool for execution, the time-consuming task will block the execution of the next task.

And there will be a lot of useless tick propulsion. For example, if the tick duration is 1 second, there will be a task with a delay of 350 seconds, that is, there will be 349 useless operations.

On the other hand, if tasks are executed quickly (of course, you can execute them asynchronously), and the number of tasks is large, the time complexity of adding or deleting tasks is O (1). The time round is more suitable than the delay task implemented by priority queue.

Time wheel in Kafka

As we mentioned above, the time wheel in Kafka is a multi-level implementation of time wheel. Generally speaking, the implementation is consistent with the idea mentioned above. But the details are different, and some optimization has been done.

Let’s look at how to add tasks. Set the absolute time of task execution when adding.

[the transfer of the external chain image failed. The source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-01xhbq65-1596457022933)(https://upload-images.jianshu…]

So how does the time wheel work? In netty, scanning at a fixed time interval is used to push the time wheel. We have analyzed the situation that there will be room for propulsion.

Kafka, on the other hand, uses the idea of exchanging space for time to save each slot through the delayqueue, and sorts the expiration time of each slot. In this way, the slot with the earliest task to be performed will have priority. If it’s not time, then delayQueue.poll It’s blocked, so there’s no time to push.

Let’s take a look at the method of propulsion.

Interviewer: do you know the time wheel algorithm? How is it applied in netty and Kafka? Why not use timers and delayed thread pools?

From the add method above, we know that every comparison is based onexpiration < currentTime + intervalFor comparison, andadvanceClockIt is used to promote the update of current time.

To sum up

Kafka uses a multi-level time wheel to achieve, and creates a time wheel on demand, uses the absolute time of the task to judge the delay, and maintains an expiration time for each slot (which is also the bi-directional linked list of tasks stored in the slot). The delayqueue is used to sort the expiration time of each slot to advance the time and prevent the existence of empty advancement.

Each push will update the current time stamp to the current time stamp. Of course, some fine tuning is made to make the current time an integral multiple of the tickems. And each push will re insert the demoted tasks into the demotion.

We can see that the element of delayqueue here is each slot, not a task, so the number is much less. This should be a trade-off between the time complexity of the delay queue for slot operation and the impact of empty advancement.

summary

Firstly, timer, delayqueue and scheduledthreadpool are introduced. They are all implemented based on priority queue. The time complexity of O (logn) is often queued and out of the queue when the number of tasks is large.Therefore, it is suitable for a small number of tasks

The timer is single threaded, and there is a risk of blocking. If the exception is not handled, the timer will hang if a task fails. Compared with timer, scheduledthreadpool can execute tasks by multithreading first, and the thread pool handles exceptions so that there is no impact between tasks.

Timers and scheduledthreadpool can perform tasks periodically. The delayqueue is a priority blocking queue.

In contrast, time wheel is more suitable for large number of tasks, and the time complexity of task insertion and deletion is O (1)。 There are two ways to deal with the delay exceeding the range of time round. One is to add a field – round number, which is how netty implements it. The second is multi-level time round, which is implemented in kakfa.

In contrast, the implementation of netty will have the problem of empty propulsion, while Kafka uses delayqueue as the unit of slot, and uses the idea of space for time to solve the problem of empty propulsion.

We can see that the implementation of delayed tasks is not very accurate, and more or less there will be blocking, even if you execute asynchronously, when there are not enough threads, it will still block.

Shoulders Of Giants

Understanding Kafka: core design and practice principles
https://www.cnblogs.com/luozh…


I’m yes, from a little bit to a billion. See you next time