Rocketmq development history
Rocketmq is a message middleware open source by Alibaba. It was open source in 2012 and became the top project of Apache in 2017. Rocketmq is widely used in Alibaba. Thousands of applications in Alibaba run on rocketmq. Hundreds of millions of messages need to be processed during the double 11 Festival, and the TPS can reach hundreds of thousands. At present, it supports access to Java, C / C + +, Python and go.
Rocketmq now has two versions, one is the community open source version and the other is the commercial cloud service version (aliwaremq). Latest version: 4.8.0 (demo version of this article).
Its core design draws lessons from Kafka, so when we study rocketmq, we will find many of the same features as Kafka, but they are quite different from Kafka in some functions. It has the following features:
- Support cluster model, load balancing and horizontal expansion capability
- 100 million level message stacking capacity
- It adopts the principle of zero copy, sequential disk writing and random reading
- The underlying communication framework adopts netty NiO
- Nameserver replaces zookeeper to realize service addressing and service coordination
- Message failure retry mechanism, message queryable
- It emphasizes that the cluster has no single point, is scalable, is highly available at any point, and is horizontally scalable
- After many double eleven tests
High availability architecture
Rocketmq naturally supports clusters very well. It has the following modes:
- Advantages: there are no advantages except simple configuration
- Disadvantages: unreliable. Restarting or shutting down the machine will make the whole service unavailable
- Advantages: simple configuration and highest performance
- Disadvantages: a small number of messages may be lost (related to configuration). During the restart or downtime of a single machine, messages not consumed under the machine cannot be subscribed before the machine is restored, affecting the real-time performance of messages
Multi master multi slave
Each master is equipped with a slave. There are multiple pairs of master slave. The cluster adopts asynchronous replication. The active and standby have a short message delay of milliseconds
- Advantages: the performance is almost the same as that of multiple masters, with high real-time performance. The switching between active and standby is transparent to applications without manual intervention
- Disadvantages: a small amount of messages will be lost when the Master goes down or the disk is damaged
Multi master multi slave
Each master is equipped with a slave. There are multiple pairs of master slave. The cluster adopts synchronous double write mode. Both active and standby write successfully and return success to the application
- Advantages: service availability and data availability are very high
- Disadvantages: the performance is slightly lower than that of asynchronous clusters. The current version cannot automatically switch between primary and standby
Two master and two slave asynchronous cluster installation
This paper adopts the two master and two slave installation mode, that is, the fourth multi master and multi slave
First of all, I bought two ECS servers. I usually use them to learn how to install some middleware, which is more convenient. If you have conditions, you can also buy them. Of course, you can also operate by installing virtual machines locally.
Next, I will install two master and two slave servers through these two servers. First, I will plan their ports
First machine 188.8.131.52 #Port planning: 9876 NameServer1 10910 BrokerA-master 10921 BrokerB-slave Second machine 184.108.40.206 #Port planning: 9876 NameServer2 10920 BrokerB-master 10911 BrokerA-slave
The drawing looks clearer
Go to the official website rocketmq.apache.org/ to get the latest download address, which is marked in the red box
The address here can be downloaded
cd /jackxu wget https://mirror.bit.edu.cn/apache/rocketmq/4.8.0/rocketmq-all-4.8.0-bin-release.zip
After downloading, unzip it and change its name
unzip rocketmq-all-4.8.0-bin-release.zip mv unzip rocketmq-all-4.8.0-bin-release.zip rocketmq
Configure the first machine
Download and unzip on both machines. In the rocketmq / conf directory, there are three recommended configuration modes:
- 2m-2s-async (2 master and 2 slave asynchronous) — this paper adopts this method
- 2m-2s-sync (2 master and 2 slave synchronization)
- 2m noslave (2 main)
Now you need to modify the files in 2m-2s-async on both machines. Modify broker-a.properties on 220.127.116.11 machine
cd /jackxu/rocketmq/conf/2m-2s-async vim broker-a.properties
Modify cluster name
#Broker Listening port for external services listenPort=10910 #Allow Broker Automatically create topic. It is recommended to open offline and close online autoCreateTopicEnable=true #Allow Broker Automatically create subscription groups. It is recommended to open offline and close online autoCreateSubscriptionGroup=true #Nameserver address, semicolon split namesrvAddr=18.104.22.168:9876;22.214.171.124:9876 #Storage path storePathRootDir=/jackxu/rocketmq/store/broker-a #commitLog Storage path storePathCommitLog=/jackxu/rocketmq/store/broker-a/commitlog #Consumption queue storage path storage path storePathConsumeQueue=/jackxu/rocketmq/store/broker-a/consumequeue #Message index storage path storePathIndex=/jackxu/rocketmq/store/broker-a/index #checkpoint File storage path storeCheckpoint=/jackxu/rocketmq/store/checkpoint #abort File storage path abortFile=/jackxu/rocketmq/store/abort
Modify cluster name
#Broker Listening port for external services listenPort=10921 #Allow Broker Automatically create topic. It is recommended to open offline and close online autoCreateTopicEnable=true #Allow Broker Automatically create subscription groups. It is recommended to open offline and close online autoCreateSubscriptionGroup=true #Nameserver address, semicolon split namesrvAddr=126.96.36.199:9876;188.8.131.52:9876 #Storage path storePathRootDir=/jackxu/rocketmq/store/broker-b-s #commitLog Storage path storePathCommitLog=/jackxu/rocketmq/store/broker-b-s/commitlog #Consumption queue storage path storage path storePathConsumeQueue=/jackxu/rocketmq/store/broker-b-s/consumequeue #Message index storage path storePathIndex=/jackxu/rocketmq/store/broker-b-s/index #checkpoint File storage path storeCheckpoint=/jackxu/rocketmq/store/checkpoint #abort File storage path abortFile=/jackxu/rocketmq/store/abort
Configure second machine
Modify broker-b.properties on 184.108.40.206 machine
cd /jackxu/rocketmq/conf/2m-2s-async vim broker-b.properties
Modify cluster name
#Broker Listening port for external services listenPort=10920 #Allow Broker Automatically create topic. It is recommended to open offline and close online autoCreateTopicEnable=true #Allow Broker Automatically create subscription groups. It is recommended to open offline and close online autoCreateSubscriptionGroup=true #Nameserver address, semicolon split namesrvAddr=220.127.116.11:9876;18.104.22.168:9876 #Storage path storePathRootDir=/jackxu/rocketmq/store/broker-b #commitLog Storage path storePathCommitLog=/jackxu/rocketmq/store/broker-b/commitlog #Consumption queue storage path storage path storePathConsumeQueue=/jackxu/rocketmq/store/broker-b/consumequeue #Message index storage path storePathIndex=/jackxu/rocketmq/store/broker-b/index #checkpoint File storage path storeCheckpoint=/jackxu/rocketmq/store/checkpoint #abort File storage path abortFile=/jackxu/rocketmq/store/abort
Modify cluster name
#Broker Listening port for external services listenPort=10911 #Allow Broker Automatically create topic. It is recommended to open offline and close online autoCreateTopicEnable=true #Allow Broker Automatically create subscription groups. It is recommended to open offline and close online autoCreateSubscriptionGroup=true #Nameserver address, semicolon split namesrvAddr=22.214.171.124:9876;126.96.36.199:9876 #Storage path storePathRootDir=/jackxu/rocketmq/store/broker-a-s #commitLog Storage path storePathCommitLog=/jackxu/rocketmq/store/broker-a-s/commitlog #Consumption queue storage path storage path storePathConsumeQueue=/jackxu/rocketmq/store/broker-a-s/consumequeue #Message index storage path storePathIndex=/jackxu/rocketmq/store/broker-a-s/index #checkpoint File storage path storeCheckpoint=/jackxu/rocketmq/store/checkpoint #abort File storage path abortFile=/jackxu/rocketmq/store/abort
Create data directory
The first machine 188.8.131.52 is executed and only needs to be executed once
mkdir -p /jackxu/rocketmq/store/broker-a /jackxu/rocketmq/store/broker-a/consumequeue /jackxu/rocketmq/store/broker-a/commitlog /jackxu/rocketmq/store/broker-a/index /jackxu/rocketmq/logs /jackxu/rocketmq/store/broker-b-s /jackxu/rocketmq/store/broker-b-s/consumequeue /jackxu/rocketmq/store/broker-b-s/commitlog /jackxu/rocketmq/store/broker-b-s/index
The second machine is 184.108.40.206 and only needs to be executed once
mkdir -p /jackxu/rocketmq/store/broker-a-s /jackxu/rocketmq/store/broker-a-s/consumequeue /jackxu/rocketmq/store/broker-a-s/commitlog /jackxu/rocketmq/store/broker-a-s/index /jackxu/rocketmq/logs /jackxu/rocketmq/store/broker-b /jackxu/rocketmq/store/broker-b/consumequeue /jackxu/rocketmq/store/broker-b/commitlog /jackxu/rocketmq/store/broker-b/index
Start both nameservers
It is executed separately on two machines, & which means it is running in the background. By default, nameserver listens to port 9876
nohup sh /jackxu/rocketmq/bin/mqnamesrv >/jackxu/rocketmq/logs/mqnamesrv.log 2>&1 &
tail -f /jackxu/rocketmq/logs/mqnamesrv.log
Start up in the following order, – C is the configuration file of the specified broker
1. Start main a of 73
nohup sh /jackxu/rocketmq/bin/mqbroker -c /jackxu/rocketmq/conf/2m-2s-async/broker-a.properties > /jackxu/rocketmq/logs/broker-a.log 2>&1 &
2. Starting a from 86
nohup sh /jackxu/rocketmq/bin/mqbroker -c /jackxu/rocketmq/conf/2m-2s-async/broker-a-s.properties > /jackxu/rocketmq/logs/broker-a-s.log 2>&1 &
3. Start main B of 86
nohup sh /jackxu/rocketmq/bin/mqbroker -c /jackxu/rocketmq/conf/2m-2s-async/broker-b.properties > /jackxu/rocketmq/logs/broker-b.log 2>&1 &
4. Starting B from 73
nohup sh /jackxu/rocketmq/bin/mqbroker -c /jackxu/rocketmq/conf/2m-2s-async/broker-b-s.properties > /jackxu/rocketmq/logs/broker-b-s.log 2>&1 &
tail -f /jackxu/rocketmq/conf/2m-2s-async/broker-a.properties tail -f /jackxu/rocketmq/conf/2m-2s-async/broker-a-s.properties tail -f /jackxu/rocketmq/conf/2m-2s-async/broker-b.properties tail -f /jackxu/rocketmq/conf/2m-2s-async/broker-b-s.properties
Check whether the startup is successful
Enter the JPS command. There are four processes in total. No problem
However, it will not be so smooth in most cases. I have some pits in the installation:
- 1. The JDK is not installed because this is my new server and there is no JDK environment on it. As we all know, rocketmq is written in Java, so it needs a Java environment to run. The JPS command is to view the java process
- 2. There is not enough memory, because the default configuration file sets a lot of memory, but my machine has only 1 core 2G, which is certainly not enough, so it will report that there is not enough memory at startup
The solution is to modify the configuration file
vim /jackxu/rocketmq/bin/runserver.sh vim /jackxu/rocketmq/bin/runbroker.sh
Reduce the memory size of nameserver and broker, and then restart
When stopping the service, you should first stop the broker and then stop the nameserver.
cd /jackxu/rocketmq/bin sh mqshutdown namesrv sh mqshutdown broker
Web console installation
Unlike Kafka, rocket officially provides a visual console, which you can download here
This is an extension of rocketmq, which includes not only the console extension, but also the docking and extension of big data flume, HBase and other components.
Download the source code. Generally, the download speed will be slow. If you can’t wait, you can go to the online disk to download the link:
cd /jackxu wget https://github.com/apache/rocketmq-externals/archive/master.zip
cd /jackxu/rocketmq-externals-master/rocketmq-console/src/main/resources vim application.properties #This is a spring Boot project, we need to modify the parameters inside #Modify port number: server.port=7298 #Modify name Server address (multiple addresses are separated by English semicolons) rocketmq.config.namesrvAddr=220.127.116.11:9876;18.104.22.168:9876
To make a jar package, you need to install Maven first. If the compilation is slow, you can replace it with the image of Taobao
cd /jackxu/rocketmq-externals-master/rocketmq-console mvn clean package -Dmaven.test.skip=true
Start the jar package
cd target java -jar rocketmq-console-ng-2.0.0.jar
Visit http://22.214.171.124:7298/ ， You can see that it has been started successfully
There are two pit stepping points here, which will report an error when starting
- (1) 126.96.36.199:9876 when spring boot starts; 188.8.131.52:9876, my first reaction here is to close the firewall
systemctl disable firewalld.service
I still can’t start it again, and then my second reaction is that the port is not open, telnet
telnet 184.108.40.206 9876
Sure enough, telnet doesn’t work. The reason is that I install it on the ECS. The default port is not open. I need to open it on the console. This is also an experience.
- (2) Another error was reported when I started up again. I couldn’t connect to 172.26.182.88:9876. At that time, I was surprised that there was no such address in the address I wrote. How could I connect it, and the IP looked like an intranet address. Later, I opened the Alibaba cloud console and found that it was really the intranet address of the machine, and then I Baidu again, The probable reason is that rocketmq is an open source Alibaba machine, and that machine is an Alibaba cloud machine. Its source code should give priority to the intranet address of Alibaba cloud. The online method is to explicitly add the public network address to the configuration file.
#Add public IP brokerIP1=220.127.116.11
- Operation and maintenance: mainly setting nameserver and configuring vipchannel.
- Cockpit: dashboard on the console. You can view the number and trend of messages by broker and subject respectively.
- Cluster: the cluster situation of the entire rocketmq, including fragmentation, number, address, version, TPS for message production and message consumption, which can be used as a data indicator during performance test.
- Topic: that is, topic. You can add / update topic and view the information of topic, such as status, routing, consumer management and sending messages.
- Consumer: you can view / create a new consumer group in the current broker, including consumer information and consumption progress.
- Producer: you can view the producer group under the production group in the current broker, including producer information and producer status.
- Message: you can query specific messages according to topic, messageid and messagekey.
- User center: language switching is related to login (login requires opening the corresponding configuration in the console configuration, and login is not required by default).
The most commonly used are cluster, theme, consumer and message.
The following describes the properties in the installation rocketmq configuration file. First, the cluster name is the same. The cluster name of the above four machines is brokerclustername = jackxu cluster. Second, connect to the same nameserver, namesrvaddr = 18.104.22.168:9876; 22.214.171.124:9876。 In the configuration file, brokerid = 0 represents master and brokerid = 1 represents slave.
There are also these two properties in the configuration file
The broker role can be configured as sync in the master broker_ Master or Async_ Master is uniformly configured as slave in the slave broker
The recommended configuration is asynchronous disk brushing + synchronous replication.
Ha and failover
In previous versions, rocketmq only had a master / slave deployment mode. There was one master in a group of brokers, with 0 to more than one slave. This mode provides a certain degree of high availability.
When the master is suspended, the slave can still provide read services. By default, all reads and writes are on the master. If slavereadenable = true is enabled, the slave can also participate in the read load. However, the default switch to read from the server with brokerid 1 is only when the backlog of messages on the master exceeds 40% of the physical memory. The specific switch to which machine is controlled by the which brokerwhenconsumeslowly parameter.
org.apache.rocketmq.common.subscription.SubscriptionGroupConfig private long whichBrokerWhenConsumeSlowly=1;
Due to the existence of multiple masters, when one master hangs, it can be written to other masters.
In version 4.5.0 released in March 2019, rocketmq uses the dledger technology to solve the problem of automatic master selection. Dledger is a commitlog repository based on raft protocol. It is also the key to rocketmq’s new high availability multi replica architecture. Its advantage is that it does not need to introduce external components, the automatic master selection logic is integrated into the process of each node, and the master selection can be completed through communication between nodes.
In this case, the commitlog is managed by dledger and has the function of selecting the master. It is not enabled by default. If it needs to be enabled, the following configuration needs to be added to the configuration file:
# Enable dledger enableDLegerCommitLog=true # DLedger Raft Group name dLegerGroup=broker-a # DLedger The address and port of each node in the group. At least 3 nodes are required dLegerPeers=n0-192.168.44.163:10911;n1-192.168.44.164:10911;n2-192.168.44.165:10911 # Current node ID dLegerSelfId=n0
When learning a middleware, children must practice the installation. First, they should experience the atmosphere of the installation process and understand some parameter configurations. Although we may not touch it at ordinary times, we still need to learn, not necessarily fine, at least.
Source: Jack_ xu |https://juejin.cn/post/692950…