What did we learn from RocketMQ: Name Server

Time:2019-5-15

Wechat Public Number:IT quarter hour
Large-scale realistic non-seriousness scene
Share with you in a quarter of an hour the high-quality technical architecture and experience, and be a programmer with plot.
Pay attention to more exciting content. For questions or suggestions, please leave a message with the public number.

What did we learn from RocketMQ: Name Server

order

Long, long ago, the way people communicated with each other was to talk face to face. You said, I heard, although simple and reliable, the disadvantages were also great.
For example, when you become a leader of an army, every subordinate will report to you as soon as possible. One is OK, but when there are tens or hundreds of subordinates, they are chattering and reporting to you every day, regardless of the occasion, at any time, then you may not hear anything, and your head will blow up. At this time, you stop, all stop for me, to report the situation, go to the door to queue, one by one, this is called traffic peak cutting, a group of people do not rush up, all obediently queue for me.
Then you listen to them one after another for 24 hours. You are really sleepy. You wonder if you can’t. If you go on like this, you may be jealous of the talent. So you say that people, pens and paper, all write the news they want to report on paper. After that, you tell Lu Xiucai. Then you listen to Lu Xiucai’s instructions and stack it up along the right wall roots of the room in accordance with the instructions. Qi, the person who reports can retire to do what to do, wait for me to rest, and then look at your report content, which is called asynchronous processing, you can finally control the progress of information acquisition by yourself, and go to bed.
When the report is written on paper and stacked, the person can back down and do what he should do instead of waiting for the report at the door. This is called decoupling.
Peak shaving, asynchronous, decoupling. This is the three most common scenarios for message queues.
The subordinates in the story are the role of message producer. The right wall of the house is the land of message persistence. Lu Xiucai is the message dispatching center, and you are the role of message consumer. Where should the information reported by the subordinates be stacked and where can the information be found? Only by Lu Xiucai’s amazing memory can the information be accurately put in and consumed.

Message Dispatching Center is today’s protagonist

In RocketMQ, there is a role called Name Server, which is the general control of the whole distributed message scheduling and the soul of RocketMQ. Without it, RocketMQ would collapse and fail to work.
So how does it work?
Let’s start with a physical architecture diagram of RocketMQ:

What did we learn from RocketMQ: Name Server

Like spider silk? Don’t be afraid. In other words, forget the picture first.

Let’s make an analogy of real life. If one person wants to send an express to another person, he or she needs to search the post offices on the Internet first, then select one of them, deliver the express to it, and then send it to the target person by the post office.

What did we learn from RocketMQ: Name Server

To complete this whole business process, first of all, we need to register the post office’s own information on the satellite network, the satellite is responsible for information scheduling, so that the sender knows which post office to choose, the recipient knows which post office the express arrives through the satellite network, and can contact the post office to communicate the appropriate distribution time, while the post office is responsible for receiving the distribution and storage express.
The RocketMQ sketch is as follows:

What did we learn from RocketMQ: Name Server

Producer: The message producer, who sends the message to the message server, is the sender in the graph.
Name Server: In the routing registration, the satellite is in the picture.
Broker: The message storage server is the post office in the picture.
Consumer: Message consumers, not today’s focus, not marked in the diagram, is the recipient.

Thus, as the coordinator of distributed message queue, NameServer has the function of information routing registration and discovery.

Routing registration

After the completion of the post office, it needs to connect with satellite network and integrate itself into the management of satellite network, which is equivalent to announcing to the outside world that my post office has started operation and can send and receive mail express.
After the post office is connected to the grid, how can the satellite continuously and timely perceive the adjustment of the post office online and its own information so that the satellite can coordinate the post office at any time? At this time, the post office needs to send a message to the satellite regularly:
“Beep – I’m post office C, No. SHC, address XXXXX, belong to Shanghai Cluster of China, online, at this moment, March 15, 2019, 13:21 seconds.”
When the satellite receives the message, it takes a small notebook to record it:
“Post Office B, BJB, Beijing, March 15, 2019, 13:10 seconds, alive…”
“Post Office A, BJA, Beijing, March 15, 2019, 13:15 seconds, alive…”
“Post Office C, SHC, Shanghai, March 15, 2019, 13:21 seconds, alive…”
……

What did we learn from RocketMQ: Name Server

The above story tells us the basic principle of NameServer routing registration.
NameServer is the equivalent of a satellite, which maintains a Broker table inside to store Broker information dynamically.
Broker is equivalent to the post office. At startup, it first traverses the Name Server list, then initiates registration requests, maintains long connections, and then sends heartbeat packets to Name Server every 30 seconds. The heartbeat package contains BrokerId, Broker address, Broker name, Broker cluster name and so on. After Name Server receives heartbeat packets, it updates the timestamp to record the Broker’s heartbeat. Latest survival time.
In order to prevent the insecurity caused by concurrent modification of the Broker table, the ReadWriteLock read-write lock is introduced in the routing registration operation. This design highlight allows multiple message producers to read concurrently, which ensures high concurrency when sending messages, but NameServer can only handle one Broker heartbeat packet at the same time. Multiple heartbeat packets are processed serially. This is also the classic use of read-write locks, that is, read more and write less.

Route elimination

What did we learn from RocketMQ: Name Server

Suddenly, one day, the computer room of Post Office C came into the mouse, bit off the power line, and the satellite did not know that the post office C business was out of order. The satellite still sent the post office table information with Post Office C to the sender (producer). The sender contacted Post Office C to send express mail, but the post office C machine room was out of order, business was suspended, and was paralyzed. Naturally, it was impossible to receive express mail.
On the other hand, because express mail can not be received by post office C, it can not be forwarded to the recipient. Customers can not wait for their express mail for a long time and complain one after another, for which the management of post office C is blamed.
So the technical department of the General Post Office began to study and discuss how to make the satellite perceive that the post office is “disconnected”, so that it can automatically eliminate the malfunction of the post office and deliver its responsible business to other normal post offices for processing, so that there will be no problems in a post office, which will lead to some of the business under the jurisdiction of the post office can not be handled.
There are many different opinions. Finally, a plan was finalized to let the satellite scan the post office information table every other time. If it is found that the difference between the reporting time and the scanning time of a post office exceeds a preset threshold, the post office will be judged to be “disconnected” and the post office information will be removed from the post office table. In this way, the post office information inquired by the sender is the normal post office information.
After the new function was put into operation online, the effect was good. We no longer need to worry about the business stagnation caused by the malfunction of a post office, and we live the life of making tea newspapers.

The story also appeared in RocketMQ.
Normally, if Broker is closed, the long connection is disconnected from NameServer. Netty’s channel closure listener monitors the disconnection event and then removes the Broker information.
In exceptional cases, NameServer has a timing task that scans the Broker table every 10 seconds. If the latest timestamp of a Broker’s heart packet exceeds 120 seconds from the current time, the Broker will also be judged to be invalid and removed.

Careful people will find a problem. Name Server does not actively notify the producer after clearing the inactivated Broker. The producer requests Name Server every 30 seconds and gets the latest routing table. That means that the message producer always has a 30-second delay and can’t sense the downtime of the Broker server in real time. So in 30 seconds, producers will still send messages to inactivated Broker, so how can the high availability of messages be guaranteed?
To solve this problem, we must first talk about Broker’s load strategy. The message sending queue adopts polling mechanism by default, and the message sending queue chooses abnormal retry mechanism by default to ensure the high availability of message sending. After the Broker downtime, although the message sender cannot perceive the Broker downtime at the first time, when the message producer sends the message to Broker and returns an exception, the message producer chooses a message queue on another Broker, which avoids the Broker that has a fault. Combining with retry mechanism, the message sender can achieve high availability skillfully, and because no Name Server notification is needed. Many uncertain producers also reduce the complexity of NameServer implementation.

In order to reduce the complexity of NameServer implementation, another design highlight is that NameServer is independent of each other, that is to say, the data between NameServer servers at a certain time will not be identical, but the exception retry mechanism makes this difference not cause any impact.

What did we learn from RocketMQ: Name Server

Route discovery

The satellites in the sky are limited and changeable, while the senders on the ground are numerous and changeable. So the sender wants to know which post offices are available. Obviously, the most appropriate way is to send a request to the satellite and pull the information from the post office table instead of waiting for the satellite to push it to everyone.
So in RocketMQ, NameServer does not actively push the meeting client, but pulls the latest routing information of the subject by the client.
What did we learn from RocketMQ: Name Server

CAP theory

Name Server, as the registry and discovery center, is the soul of the whole distributed message queue scheduling. When it comes to distribution, it can not escape the CAP theory. C is Consistency, A is Availability, P is Partiton Tolerance. For distributed architecture, network conditions are uncontrollable and network partition is inevitable. Therefore, partition fault tolerance is necessary. Name Server is also in AP. It’s the choice in the CP, because NameServer is independent of each other, obviously, it’s an AP design.

The reason for replacing Zookeeper

ZooKeeper provides coordination services for distributed applications. So why does RocketMQ build its own wheels and develop cluster management programs? Because ZooKeeper has powerful functions, including automatic Master Election, RocketMQ’s architecture design determines that it does not need Master Election, does not need these complex functions, only needs a lightweight metadata server.
Middleware requires high stability. RocketMQ’s Name Server has very little code and is easy to maintain, so it does not need to rely on another middleware to reduce the overall maintenance cost.

What have you learned?

1. Realization Principle of Long Connection Programming Model Jump
2. The classical mode of read-write lock in multithreaded programming
3. Pursue simple, effective and reliable realization

In the following words

To study the source code of NameServer, click on the link: https://github.com/MrChiu/Roc…
It is easy to read through the code with my annotations.

What did we learn from RocketMQ: Name Server